Roi filter within

[joshqsumner]

Describe your changes
Adding a 'within' method to pcv.roi.filter based on cutto but filtering for only objects completely within (do not touch the border of) the ROI.

Type of update
Is this a feature enhancement.

Associated issues
Closes #1651

Additional context
I don't know how common this use case really is so I might have been too specific in what I was building this towards. If there are other ideas about when/why someone would want to do this kind of filtering I'd be happy to have some discussion here about that and whether the implementation is adequate for it.

help wanted
Also there is a section here that could definitely use a close review and is marked with a note comment. Line 535 in _helpers.py I am grabbing colors from a contour placed on the original mask to see if something is masked or not. If I don't do that then I end up drawing things that were originally gaps between leaves as white sections since they are fully within the ROI. Maybe there is a better cv2 way to do that but it was not obvious to me from the docs.

For the reviewer
See this page for instructions on how to review the pull request.

• PR functionality reviewed in a Jupyter Notebook
• All tests pass
• Test coverage remains 100%
• Documentation tested
• New documentation pages added to plantcv/mkdocs.yml
• Changes to function input/output signatures added to updating.md
• Code reviewed
• PR approved

[deepsource-io]

Here's the code health analysis summary for commits d229f7a..ffd8888. View details on DeepSource ↗.

Analysis Summary

Analyzer	Status	Summary	Link
Python	✅ Success		View Check ↗
Test coverage	✅ Success		View Check ↗

Code Coverage Report

Metric	Aggregate	Python
Branch Coverage	100%	100%
Composite Coverage	100%	100%
Line Coverage	100%	100%
New Branch Coverage	100%	100%
New Composite Coverage	100%	100%
New Line Coverage	100%, ✅ Above Threshold	100%, ✅ Above Threshold

---

💡 If you’re a repository administrator, you can configure the quality gates from the settings.

[nfahlgren]

I tested this approach with a multi-plant image where the ROI fully encompassed one plant and partially another. In partial mode you get both plants as expected, but in within mode I didn't get the fully enclosed plant and instead just got some small background noise.

I think with the hierarchical contours it gets complicated, so what if within mode was added to pcv.roi.quick_filter instead?

If quick_filter was modified like this I think it does what I was expecting:

def quick_filter(mask, roi, roi_type="partial"):
    """Quickly filter a binary mask using a region of interest.

    Parameters
    ----------
    mask : numpy.ndarray
        Binary mask to filter.
    roi : plantcv.plantcv.classes.Objects
        PlantCV ROI object.

    Returns
    -------
    numpy.ndarray
        Filtered binary mask.
    """
    # Increment the device counter
    params.device += 1

    # Store debug
    debug = params.debug
    params.debug = None

    # Label objects in the image from 1 to n (labeled mask)
    labels, num = label(label_image=mask, return_num=True)

    # Convert the input ROI to a binary mask (only works on single ROIs)
    roi_mask = roi2mask(img=mask, roi=roi)

    # Convert the labeled mask and ROI mask to float data types
    roi_mask = roi_mask.astype(float)
    labels = labels.astype(float)

    # Set the ROI mask value to 0.5
    roi_mask[np.where(roi_mask == 255)] = 0.5

    # Add the labeled mask and ROI mask together
    summed = roi_mask + labels

    for i in range(1, num + 1):
        # For each label, if at least one pixel of the object overlaps the ROI
        # set all the label values to the label plus 0.5
        if roi_type.upper() == "PARTIAL" and i + 0.5 in summed:
            summed[np.where(summed == i)] = i + 0.5
        # If one pixel of the object falls outside the ROI
        # set all the label values to zero
        elif roi_type.upper() == "WITHIN" and i in summed:
            summed[np.where(labels == i)] = 0

    # Objects that do not overlap the ROI will round to an integer and have
    # the same value before and after rounding.
    # Objecs that overlap the ROI will round up/down and will not have the same value
    # Where the values are equal (not overlapping)
    summed[np.where(summed == summed.round())] = 0

    # The summed image now only contains objects that overlap the ROI
    # Subtract 0.5 to remove the ROI mask
    summed = summed - 0.5

    # Round and set the data type back to uint8
    summed = summed.round().astype("uint8")

    # Make sure the mask is binary
    summed[np.where(summed > 0)] = 255

    # Print/plot debug image
    params.debug = debug
    _debug(visual=summed, filename=os.path.join(params.debug_outdir, f"{params.device}_roi_filter.png"), cmap="gray")

    return summed

[joshqsumner]

Using some docs images I did find an example that didn't work how I was expecting:

This worked as expected:

from plantcv import plantcv as pcv
pcv.params.debug = None
img, path, filename = pcv.readimage(filename="/home/josh/plantcv/docs/img/documentation_images/multi/original_multi_image.jpg")
b = pcv.rgb2gray_lab(rgb_img=img, channel='b')
thresh = pcv.threshold.binary(gray_img=b, threshold=150, object_type='light')
mask = pcv.fill(bin_img=thresh, size=50)
mask = pcv.closing(mask)
pcv.params.debug = "plot"
rect = pcv.roi.rectangle(mask, x = 100, y = 100, h = 200, w = 100)
good = pcv.roi.filter(mask, rect, roi_type = "within")

but this did not

# ...
rect = pcv.roi.rectangle(mask, x = 100, y = 70, h = 200, w = 100)
bad = pcv.roi.filter(mask, rect, roi_type = "within")

Looking at the masks that were not kept correctly I think it is the hierarchy where there are "holes" in the object, which probably is causing a problem with the "if all points in contour are white then" logic.

I just committed a change that uses the parent hierarchy instead of checking for "all" of the parent contour. That makes my example above work as expected. If you have time @nfahlgren could you check if that works for your example too? If not then I can move it to roi.quick_filter like you laid out, that does seem simpler. I guess this probably doesn't work well for a "bullseye" shape but I'm not sure I've seen a plant like that.

[nfahlgren]

It does work on the example I had now!