StructurefromMotion

Structure from motion (SfM) is a computational method for recovering the three-dimensional structure of a scene from a collection of two-dimensional images taken from different viewpoints. It also estimates the poses and intrinsic parameters of the cameras used to capture the images.

It relies on multi-view geometry and robust feature correspondences. By identifying matching image features across views and using the geometry of multiple cameras, SfM recovers camera positions and orientations as well as a sparse set of 3D points representing the scene. In monocular SfM, scale is initially indeterminate and must be fixed with additional information.

The typical pipeline begins with feature detection and matching (for example SIFT or ORB), then pairwise pose estimation and outlier rejection (RANSAC) to obtain relative camera motion. A view graph is built and refined to consistent global poses. Triangulation produces a sparse 3D point cloud. Bundle adjustment jointly optimizes all camera parameters and 3D points to minimize reprojection error.

Variations include incremental SfM, which adds cameras one by one, and global SfM, which estimates all poses simultaneously. After the sparse reconstruction, dense models can be produced with multi-view stereo or related techniques to generate depth maps and fuller surfaces.

Challenges include noise, occlusions, and changing lighting, as well as scale ambiguity in monocular setups and computational demands. Handling outliers and non-rigid scenes requires robust estimation and careful data management.

Outputs typically include a sparse 3D point cloud and a set of camera parameters; with additional processing, dense reconstructions and textured surfaces are possible. Applications span cultural heritage, architecture, surveying, robotics, autonomous navigation, and virtual reality. Widely used software includes COLMAP, OpenMVG/OpenMVS, and VisualSFM.