Track Objects and Estimate Motion
R2026bMotion estimation and tracking are key activities in many computer vision applications, including activity recognition, traffic monitoring, automotive safety, and surveillance.
Computer Vision Toolbox™ provides video tracking algorithms, such as continuously adaptive mean
shift (CAMShift) and Kanade-Lucas-Tomasi (KLT). You can use these algorithms for
tracking a single object, or as building blocks in a more complex tracking system.
The toolbox also provides a framework for multiple object tracking, which includes a
Kalman filter and the Hungarian algorithm for assigning object
detections to tracks. For prompt-driven video object segmentation and tracking, the
toolbox provides sam2VideoObjectSegmenter object that uses the Segment Anything Model 2
(SAM 2) to segment and track objects consistently across video frames.
Motion estimation is the process of determining the movement of blocks between adjacent video frames. This toolbox includes motion estimation algorithms, such as optical flow, block matching, and template matching. These algorithms create motion vectors, which can relate to the whole image, blocks, arbitrary patches, or individual pixels. For block and template matching, the evaluation metrics for finding the best match include mean square error (MSE), mean absolute deviation (MAD), maximum absolute difference (MaxAD), sum of absolute difference (SAD), and sum of squared difference (SSD).
Functions
Topics
- Multiple Object Tracking
Locate a moving object or multiple objects over time in a video stream.
- What Is Optical Flow?
Track moving objects in videos using optical flow analysis.
Featured Examples
Motion-Based Multiple Object Tracking
Perform automatic detection and motion-based tracking of moving objects in a video from a stationary camera.
Multi-Object Tracking with DeepSORT
Integrate appearance features from a re-Identification (Re-ID) deep neural network with a multi-object tracker.
Visual Tracking of Occluded and Unresolved Objects
Resolve challenging tracking scenarios when objects are occluded or when they are in close proximity to each other.
Implement Simple Online and Realtime Tracking
Implement the Simple Online and Realtime (SORT) object tracking algorithm using the Computer Vision Toolbox and the Sensor Fusion and Tracking Toolbox™.
Automate Labeling of Objects in Video Using RAFT Optical Flow
Use a pretrained RAFT optical flow estimation network to propagate a predefined object mask from one frame to the next in a video sequence.
Reidentify People Throughout a Video Sequence Using ReID Network
Track people throughout a video sequence using re-identification with a residual network.
Re-identify and Segment People Across Videos with SAM 2
Automatically transfer pixel-accurate person masks from one video into another.
Track Objects Across Multiple Cameras Using Ground-Plane Fusion
Track vehicles and pedestrians across calibrated cameras by fusing ground-plane projections into globally consistent track IDs.
Automate Ground Truth Labeling for Object Tracking and Re-Identification
Create an automation algorithm to automatically label data for object tracking and for object re-identification.
Convert Ground Truth Labeling Data for Object Re-Identification
Convert a groundTruth object to the re-identification training data format.
Convert Ground Truth Labeling Data for Object Tracking
Convert a groundTruth object to the trackCLEARmetrics truth format.
Tracking Pedestrians from a Moving Car
Track pedestrians using a camera mounted in a moving car.
Track a Face in Scene
Track a face using a minimum Eigen features detection function and a point tracker.
Face Detection and Tracking Using Live Video Acquisition
Automatically detect and track a face in a live video stream, using the KLT algorithm.
Face Detection and Tracking Using CAMShift
Automatically detect and track a face.
Face Detection and Tracking Using the KLT Algorithm
Automatically detect and track a face using feature points. The approach in this example keeps track of the face even when the person tilts his or her head, or moves toward or away from the camera.
Code Generation for Face Tracking with PackNGo
Generate code from Face Detection and Tracking Using the KLT Algorithm example with packNGo function. The packNGo (MATLAB Coder) function packages all relevant files in a compressed zip file so you can relocate, unpack, and rebuild your project in another development environment without MATLAB present. This example also shows how to create a makefile for the packNGo content, rebuild the source files and finally run the standalone executable outside MATLAB environment.
Use Kalman Filter for Object Tracking
Use the vision.KalmanFilter object and configureKalmanFilter function to track objects.
Import Camera-Based Datasets in MOT Challenge Format for Object Tracking
Read camera image sequences and convert the ground truth and detections to Sensor Fusion and Tracking Toolbox formats using a custom dataset.
Multi-Object Tracking and Human Pose Estimation
Detect multiple people, track them, and estimate their body poses in a video by using pretrained deep learning networks and a global nearest-neighbor (GNN) assignment tracking approach.
Stabilize Video Using Optical Flow
Use pretrained RAFT deep learning model to stabilize a video.
Teaching Resources
Computer Vision Basics
Learn the fundamentals of image segmentation in computer vision.
