Shape of Motion: 4D Reconstruction from a Single Video

The article discusses a method called Shape of Motion that reconstructs a 4D scene from a single monocular video. The approach aims to address the challenges of monocular dynamic reconstruction by explicitly modeling 3D motion in dynamic scenes captured in monocular videos. The method utilizes a compact set of SE(3) motion bases to represent scene motion and incorporates data-driven priors like monocular depth maps and 2D tracks to create a globally consistent representation of the dynamic scene. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in long-range 3D/2D motion estimation and novel view synthesis for dynamic scenes. The article also includes comparisons with other methods in 3D tracking, novel view synthesis, and 2D tracking, highlighting the strengths and limitations of the Shape of Motion approach. Additionally, the article acknowledges contributors and funding sources for the project.