Adjust collection of 3-D points and camera poses

Syntax

``````[xyzRefinedPoints,refinedPoses] = bundleAdjustment(xyzPoints,pointTracks,cameraPoses,intrinsics)``````
``[wpSetRefined,vSetRefined,pointIndex] = bundleAdjustment(wpSet,vSet,viewIDs,intrinsics)``
``````[___,reprojectionErrors] = bundleAdjustment(___)``````
``[___] = bundleAdjustment(___,Name=Value)``

Description

example

``````[xyzRefinedPoints,refinedPoses] = bundleAdjustment(xyzPoints,pointTracks,cameraPoses,intrinsics)``` refines 3-D points and camera poses to minimize reprojection errors. The refinement procedure is a variant of the Levenberg-Marquardt algorithm. The function uses the same global reference coordinate system to return both the 3-D points and camera poses.```
````[wpSetRefined,vSetRefined,pointIndex] = bundleAdjustment(wpSet,vSet,viewIDs,intrinsics)` refines 3-D points from the world point set, `wpSet`, and refines camera poses from the image view set, `vSet`. `viewIDs` specify the camera poses in `vSet` to refine.```
``````[___,reprojectionErrors] = bundleAdjustment(___)``` returns the mean reprojection error for each 3-D world point, in addition to the arguments from the previous syntax.```
````[___] = bundleAdjustment(___,Name=Value)` specifies options using one or more name-value arguments in addition to any combination of arguments from previous syntaxes. For example, `MaxIterations=50` sets the number of iterations to `50`. Unspecified arguments have default values.```

Examples

collapse all

`data = load("globeBA.mat");`

Refine the camera poses and points.

```[xyzRefinedPoints,refinedPoses] = ... bundleAdjustment(data.xyzPoints,data.pointTracks,data.cameraPoses,data.intrinsics);```

Display the 3-D points and camera poses before and after the refinement.

```pcshowpair(pointCloud(data.xyzPoints), pointCloud(xyzRefinedPoints), ... AxesVisibility="on", VerticalAxis="y", VerticalAxisDir="down", MarkerSize=40); hold on plotCamera(data.cameraPoses, Size=0.1, Color="m"); plotCamera(refinedPoses, Size=0.1, Color="g"); legend("Before refinement", "After refinement", color="w");```

Input Arguments

collapse all

Unrefined 3-D points, specified as an M-by-3 matrix of [x y z] locations.

Matching points across multiple images, specified as an N-element array of `pointTrack` objects. Each element contains two or more matching points across multiple images.

Camera pose information, specified as a two-column table with columns `ViewId` and `AbsolutePose`. The view IDs relate to the IDs of the objects in the `pointTracks` argument. You can use the `poses` object function to obtain the `cameraPoses` table.

Camera intrinsics, specified as a `cameraIntrinsics` object or an N-element array of `cameraIntrinsics` objects. N is the number of camera poses or the number of IDs in `viewIDs`. Use a single `cameraIntrinsics` object when images are captured using the same camera. Use a vector `cameraIntrinsics` objects when images are captured by different cameras.

3-D world points, specified as a `worldpointset` object.

Camera poses, specified as an `imageviewset` object.

View identifiers, specified as an N-element array. The `viewIDs` represent which camera poses to refine specifying their related views in `imageviewset`.

Name-Value Arguments

Specify optional pairs of arguments as `Name1=Value1,...,NameN=ValueN`, where `Name` is the argument name and `Value` is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: `MaxIterations=50` sets the number of iterations to `50`.

Before R2021a, use commas to separate each name and value, and enclose `Name` in quotes.

Maximum number of iterations before the Levenberg-Marquardt algorithm stops, specified as a positive integer.

Absolute termination tolerance of the mean squared reprojection error in pixels, specified as positive scalar.

Relative termination tolerance of the reduction in reprojection error between iterations, specified as positive scalar.

Flag to indicate lens distortion, specified as `false` or `true`. When you set `PointsUndistorted` to `false`, the 2-D points in `pointTracks` or in `vSet`must be from images with lens distortion. To use undistorted points, first use the `undistortImage` function to remove distortions from the images, then set `PointsUndistorted`.

View IDs for fixed camera pose, specified as a vector of nonnegative integers. Each ID corresponds to the `ViewId` of a fixed camera pose in `cameraPoses`. An empty value for `FixedViewIDs` means that all camera poses are optimized.

Solver, specified as `"sparse-linear-algebra"` or `"preconditioned-conjugate-gradient"`. Use the `"sparse-linear-algebra"` solver for low sparsity images. Low sparsity indicates that many camera views observe some of the same world points. Use the `"preconditioned-conjugate-gradient"` (PCG) solver, from the general graphic optimization (g2o) library, for high sparsity images. High sparsity indicates that each camera view observes, only a small portion of the world points, specified by `xyzPoints`.

Display progress information, specified as `false` or `true`.

Output Arguments

collapse all

3-D locations of refined world points, returned as an M-by-3 matrix of [x y z] locations.

Data Types: `single` | `double`

Refined camera poses, returned as a two-column table. The table contains columns for `ViewId` and `AbsolutePose`.

Refined 3-D world points, returned as a `worldpointset` object.

Refined camera poses, specified as an `imageviewset` object.

Indices to `wpSetRefined` of refined 3-D world points, returned as an M-element array.

Reprojection errors, returned as an M-element vector. The function projects each world point back into each camera. Then, in each image, the function calculates the reprojection error as the distance between the detected and the reprojected point. The `reprojectionErrors` vector contains the average reprojection error for each world point.

References

[1] Lourakis, Manolis I. A., and Antonis A. Argyros. "SBA: A Software Package for Generic Sparse Bundle Adjustment." ACM Transactions on Mathematical Software 36, no. 1 (March 2009): 2:1–2:30.

[2] Hartley, Richard, and Andrew Zisserman. Multiple View Geometry in Computer Vision. 2nd ed. Cambridge, UK ; New York: Cambridge University Press, 2003.

[3] Triggs, Bill, Philip F. McLauchlan, Richard I. Hartley, and Andrew W. Fitzgibbon. "Bundle Adjustment — A Modern Synthesis." In Proceedings of the International Workshop on Vision Algorithms, 298–372. Springer-Verlag, 1999.

Version History

Introduced in R2016a

expand all