Sam2 multi polygons

[probicheaux]

Description

BREAKING CHANGE: changes format of sam2response

Allow inference to output multiple polygons from cv2.findContours
Also adds not-a-point at [0, 0] to empty prompts because sam2 performs better:

The below shows unprompted sam2 with different settings.
Before this pr:

After adding multiple masks to return:

After adding multiple masks with dummy not-a-point

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

Staging

Any specific deployment considerations

n/a

Docs

• Docs updated? What were the changes:

[probicheaux]

@PawelPeczek-Roboflow can you take a look at this? in particular, wondering if what I did here is ok -- not totally allowed by the types but I am not sure if constructing the full workflow image type here is overkill when the function needs just the height/width of the image

[yeldarby]

Also adds not-a-point at [0, 0] to empty prompts because sam2 performs better:

What'd happen if we did a positive point at the centerpoint of the image instead? My guess is in a lot of images that'd produce a much better result given the most important object is often centered.

[probicheaux]

@yeldarby to be clear, not a point is a special padding label, not a negative poitn

I'd rather let the user decide to put a positive point in the middle of the image if they want it, otherwise let sam figure out what it should segment (even tho those results tend to be bad)

[yeldarby]

I understand. But defaults are powerful. The special point doesn’t seem like it’d ever be a better default than the center point would be (just like a null set doesn’t seem better than the special point).

Agree that almost always the user should be giving a prompt anyway, but the case we’re considering here is what we should do when they don’t.

[probicheaux]

. The special point doesn’t seem like it’d ever be a better default than the center point would be (just like a null set doesn’t seem better than the special point)

hmm, I would think it'd be better on images where the object in focus isn't in the center of the image

as for the null set, I mean -- theoretically it should produce the same output as adding in a random point and labeling it as not a point, but for some reason the prompt encoder seems to work better with this input.

[probicheaux]

@PawelPeczek-Roboflow tests are passing now

[yeldarby]

hmm, I would think it'd be better on images where the object in focus isn't in the center of the image

I think it'd be similar, right? (bad in both cases since the -1 point and the center point both wouldn't be signaling to highlight the salient object)

[PawelPeczek-Roboflow]

LGTM, please add those small adjustments

[probicheaux]

@PawelPeczek-Roboflow your comments have been addressed!