Skip to content

GSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion

License

Notifications You must be signed in to change notification settings

GS-Fusion/GSFusion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 

Repository files navigation

GSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion

Jiaxin Wei and Stefan Leutenegger

teaser

All the reported results are obtained from a single Nvidia RTX 3060 GPU.

Abstract: Traditional volumetric fusion algorithms preserve the spatial structure of 3D scenes, which is beneficial for many tasks in computer vision and robotics. However, they often lack realism in terms of visualization. Emerging 3D Gaussian splatting bridges this gap, but existing Gaussian-based reconstruction methods often suffer from artifacts and inconsistencies with the underlying 3D structure, and struggle with real-time optimization, unable to provide users with immediate feedback in high quality. One of the bottlenecks arises from the massive amount of Gaussian parameters that need to be updated during optimization. Instead of using 3D Gaussian as a standalone map representation, we incorporate it into a volumetric mapping system to take advantage of geometric information and propose to use a quadtree data structure on images to drastically reduce the number of splats initialized. In this way, we simultaneously generate a compact 3D Gaussian map with fewer artifacts and a volumetric map on the fly. Our method, GSFusion, significantly enhances computational efficiency without sacrificing rendering quality, as demonstrated on both synthetic and real datasets.

Note

This repository contains the code used in the paper "GSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion". The code will be released upon acceptance of the paper.

Evaluation

We develop an automatic evaluation system for GSFusion and provide several pre-trained models for assessment. You can download the necessary data here, and follow the instructions in GSFusion_eval to get started.

Citation

If you find our paper and code useful, please cite us:

@misc{wei2024gsfusiononlinergbdmapping,
      title={GSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion}, 
      author={Jiaxin Wei and Stefan Leutenegger},
      year={2024},
      eprint={2408.12677},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2408.12677}, 
}

About

GSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published