Instructions of Using COLMAP

[SherryXTChen]

Hi, thank you for sharing this amazing work!

I would like to try a personal video using your model and wonder if you can share more detailed instructions on how to use COLMAP to get the training data. For instance, do you use only sparse reconstruction to get the data? If so, what do you set for the data type and shared intrinsics options? Furthermore, how do you deal with merging iterations of sparse reconstruction?

Thanks

[zhengqili]

Hi, I only use COLMAP SfM sparse reconstruction to get input data such as camera poses and near/far depth plane. I set my initial vertical FOV to be 60 degrees for most real-world videos with first-order radial distortion. I also mask moving objects by using off-the-shelf instance segmentation algorithm such as MaskRCNN.

Some other project that includes instruction of running COLMAP on real-world video and might be useful: https://github.com/facebookresearch/consistent_depth

[SherryXTChen]

Hi, I only use COLMAP SfM sparse reconstruction to get input data such as camera poses and near/far depth plane. I set my initial vertical FOV to be 60 degrees for most real-world videos with first-order radial distortion. I also mask moving objects by using off-the-shelf instance segmentation algorithm such as MaskRCNN.

Some other project that includes instruction of running COLMAP on real-world video and might be useful: https://github.com/facebookresearch/consistent_depth

Thank you!

[zhangchi3]

Hi, I only use COLMAP SfM sparse reconstruction to get input data such as camera poses and near/far depth plane. I set my initial vertical FOV to be 60 degrees for most real-world videos with first-order radial distortion. I also mask moving objects by using off-the-shelf instance segmentation algorithm such as MaskRCNN.

Some other project that includes instruction of running COLMAP on real-world video and might be useful: https://github.com/facebookresearch/consistent_depth

@zhengqili Hi, I am following this great project. And I want to recreate the kid-running results from scratch with the data your provided. However, it seems that I can't get the correct COLMAP results. May I ask for the commands you used for COLMAP,particularly did you use sequential_matcher or exhaustive_matcher.

Here is my commands:
colmap feature_extractor --database_path $DATA_DIR/database.db
--image_path $DATA_DIR/images
--ImageReader.camera_model PINHOLE
--ImageReader.single_camera 1
--ImageReader.mask_path $DATA_DIR/colmap_masks

colmap sequential_matcher
--database_path $DATA_DIR/database.db

mkdir $DATA_DIR/sparse

colmap mapper
--database_path $DATA_DIR/database.db
--image_path $DATA_DIR/images
--output_path $DATA_DIR/sparse

mkdir $DATA_DIR/dense

colmap image_undistorter
--image_path $DATA_DIR/images
--input_path $DATA_DIR/sparse/0
--output_path $DATA_DIR/dense
--output_type COLMAP

Thanks!

[zhengqili]

Hi I remember I used exhaustive_matcher but I don't think if using sequential_matcher matters a lot. You need to create masks for moving kid before feeding images to the COLMAP (check here: https://colmap.github.io/faq.html#mask-image-regions)

[zhangchi3]

Thanks a lot! I use MaskRCNN to create masks for moving kid. One more question, you said that you set the initial vertical FOV to be 60 degree with first-order distortion. Are these seetings for COLMAP SfM? I don't find any colmap command that need to set FOV and the distortion type.

[kwea123]

Hi I remember I used exhaustive_matcher but I don't think if using sequential_matcher matters a lot. You need to create masks for moving kid before feeding images to the COLMAP (check here: https://colmap.github.io/faq.html#mask-image-regions)

@zhengqili Yes, this step is indispensable in order to get good pose estimates, however I think you missed this part in the readme, so now I start to think if those bad results from #15 #18 is because of bad pose reconstructions. People without experience with COLMAP might just feed the original images without masking.

[SherryXTChen]

Hi I remember I used exhaustive_matcher but I don't think if using sequential_matcher matters a lot. You need to create masks for moving kid before feeding images to the COLMAP (check here: https://colmap.github.io/faq.html#mask-image-regions)

@zhengqili Yes, this step is indispensable in order to get good pose estimates, however I think you missed this part in the readme, so now I start to think if those bad results from #15 #18 is because of bad pose reconstructions. People without experience with COLMAP might just feed the original images without masking.

@kwea123 Thank you. I am new to COLMAP and didn't realize that the mask needs to be created before pose reconstruction. If you don't mind me asking, is there any model most suitable for this task? I saw MaskRCNN being used, but isn't it an object detection/segmentation model? Should I use some motion detection/segmentation model instead?

[kwea123]

To my knowledge there is no true motion segmentation for the moment. I think using maskrcnn is good enough if you define the motion classes correctly according to your data. If your data contains uncommon objects or large moving shadows, I suggest that you manually label them.

[SherryXTChen]

To my knowledge there is no true motion segmentation for the moment. I think using maskrcnn is good enough if you define the motion classes correctly according to your data. If your data contains uncommon objects or large moving shadows, I suggest that you manually label them.

Thank you!