Structure from Motion from Multiple Views

Since R2026a
Open Live Script

Structure from Motion (SfM) is a computer vision technique for estimating the 3-D structure of a scene from a collection of 2-D images [1]. It plays a critical role in applications such as robot navigation, autonomous driving, augmented reality, and 3-D scene reconstruction.

This example series provides a comprehensive walkthrough of an incremental SfM pipeline, demonstrating how to estimate camera poses and reconstruct a 3-D scene from a set of calibrated images.

Incremental Structure from Motion Workflow Steps

The series is divided into four modular examples, each focusing on a key step in the incremental SfM pipeline. Each example includes detailed code and explanations to help you understand the underlying algorithms and implementation. For more information about the individual steps, visit these example links:

  1. Create View Graph Using Bag of Features — Extract features from images and build a view graph based on visual similarity

  2. Refine View Graph Using Geometric Verification — Improve the view graph by verifying geometric consistency between image pairs.

  3. Reconstruct 3-D Scene from Geometrically Refined Pair of Initial Views — Select a robust image pair and initialize the 3D reconstruction.

  4. Reconstruct Complete 3-D Scene Using Incremental Structure from Motion — Integrate remaining views incrementally to complete the 3-D reconstruction.

Ready-to-Run SfM: Run the Full Structure from Motion Pipeline

While the individual examples in this series provide an in-depth exploration of each stage in the incremental SfM pipeline, this section offers a streamlined way to run the entire SfM workflow end-to-end using encapsulated helper functions. These helper functions encapsulate the code from the four examples and allow you to process your own image data or use sample datasets to reconstruct a 3-D scene with minimal setup:

  • helperCreateViewGraph

  • helperRefineViewGraph

  • helperReconstructFromInitialPairView

  • helperIncrementalSfM

These helper functions allow you to process your own image data or use sample datasets to reconstruct a 3-D scene with minimal setup. To access these helper functions, click the Copy Command button next to the example in the documentation. Then, paste the copied command into the MATLAB Command Window.

Run SfM on the TUM RGB-D Benchmark Dataset

This section demonstrates how to run the full structure from motion pipeline using a subset of the TUM RGB-D Benchmark dataset [2].

Download the Dataset

url = "https://ssd.mathworks.com/supportfiles/3DReconstruction/tum_rgbd_data.zip";
downloadFolder = tempdir;

filename = fullfile(downloadFolder, "tum_rgbd_data.zip");
imageFolder = fullfile(downloadFolder, "sfmTrainingDataTUMRGBD", "images");

if ~exist(imageFolder, "dir")
    disp('Downloading TUM RGB-D Dataset (43 MB)...');
    websave(filename, url);
    unzip(filename, downloadFolder);   
end

Load Images and Camera Intrinsics

imds = imageDatastore(imageFolder);
intrinsicsFile = fullfile(downloadFolder, "sfmTrainingDataTUMRGBD", "cameraInfo.mat");
data = load(intrinsicsFile);
intrinsics = data.intrinsics;

Run the SfM Pipeline

viewGraph = helperCreateViewGraph(imds);
viewGraph = helperRefineViewGraph(viewGraph, intrinsics);
[viewGraph, wpSet] = helperReconstructFromInitialPairView(viewGraph, intrinsics);
[viewGraph, wpSet, point3dColors] = helperIncrementalSfM(viewGraph, wpSet, intrinsics, imds);

% Display the final reconstruction with color
pcshow(wpSet.WorldPoints, point3dColors, MarkerSize=30, VerticalAxis="y", VerticalAxisDir="down");

Figure contains an axes object. The axes object contains an object of type scatter.

With the estimated camera poses and the sparse 3-D point cloud, you can perform dense reconstruction of the scene. See following examples for more details on the dense reconstruction workflow.

References

[1] Schonberger, Johannes L., and Jan-Michael Frahm. "Structure-from-motion revisited." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104-4113. 2016.

[2] Sturm, Jürgen, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. "A benchmark for the evaluation of RGB-D SLAM systems". In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 573-580, 2012.

See Also

Topics