Image Masking in v 2023

[jgmenzer]

Hi All,

I am curious if anyone can provide some documentation/help using the ImageMasking node (or best practice outside of this node)? I combed through some old posts, but the seemed they were before the ImageMasking node was implemented.

My set up: using a turntable with reference sheet (sam porter's) inside a photo box. I can change out the colors, but would prefer black or white as background.

Questions: What is the input/output for this node (e.g. where in the pipeline should it go? If using black/white background should I be using HSV or greyscale? If using HSV, I see parameters for green, white, or black, but not sure which Hue value to use (or maybe it doesn't matter here)?

I appreciate any help, and apologize if this is redundant from somewhere else, Jeremy

[natowi]

You can get the setting by hovering over the parameter (but yes, the hue info is missing):
HSV
Green : default values
White : Hue=AnyValue, Tolerance=1, minSaturation=0, maxSaturation=0.1, minValue=0.8, maxValue=1
Black : Hue=AnyValue, Tolerance=1, minSaturation=0, maxSaturation=0.1, minValue=0, maxValue=0.2

The Input from the ImageMasking connects to CameraInit.
The output of the ImageMasking node can be connected to FeatureExtraction (use mask for in camera estimation) Masking Input and the PrepareDenseScene Masking Input (applies the undistort to the mask and combines it with the RGB to a RGBA image).

There are also some new features for marker detection that will be useful (I need to write up some text for it ;) )

[natowi]

Hi Jeremy, could you share a screenshot of your graph and the sfm transform settings?

[jgmenzer]

Hey natowi - I've used your template (no settings changed). Then added the SfMTransform.

For SfMTransform: Transform Method - from_markers, Landmark Describer types -cctag3 (I also tried cctag3 and dpsift). My Marker positions, all others default.

Not sure if this will help:

Thanks, Jeremy

[natowi]

Did you try with dspsift and cctag checked at the same time?

//Try duplicating the StructureFromMotion node and use sfmTransform after the FeatureMatching CCTAG node, then add SfM Alignment node and use the CCTAG input as reference and the StructureFromMotion as input, use the output to continue.
Something in this direction:

[jgmenzer]

Yes I tried dpsift and cctag checked at the same time, same error.

Trying what you suggest directly above is working (I actually tried this earlier, but had poses as reference not sfmData file.....). For alignment method, I used "from_cameras_filepath" all other methods fail to find similarity between the two scenes. Does this make sense to you/sound correct?

Thanks so much!!! This should work for me now.

Also FYI -
Not sure if I should open a different issue. I am running an augmentation after this process to add the bottom of a lithic artifact to the model (covered by clay to hold it up). So with the white masking background, the augmentation works well and I get a model of the complete object. I also tried with black background (it fails in the augmented SfM) and with a green background, so I wouldn't have to worry about masking out the cctags, that also fails in the augmented sfm. I thought that was odd as the other colors are masked out and I get similar numbers of features and landmarks detect. Thoughts here?

[natowi]

Good to hear that it works for you.

You can open a new issue for your other question.

//
Note: sfm transform may have some bugs: #1994