Auto detecting multiple points and tracking their locations

I have a binary video with a few hundred moving white dots that I would like to track and get their respective x and y location individually. I tried following this example: https://www.mathworks.com/help/vision/ug/motion-based-multiple-object-tracking.html to auto track the points but it seems like the the white dots are too small or too dense to be picked up. Anyone have any advice?

Answer 1

[Alan]

Hi Pieter,

The example you mentioned in the given link can be fine-tuned to better suit your video. There are 2 distinct phases of object tracking:

1. Object detection and track creation (a track is a detected object and its position)
2. Track creation (involves updating the position of the tracked object and deleting the track if the object disappears for a certain threshold of frames)

I could help you out with the object detection part as I don’t have your video it would be difficult for me to create one. However, I could create a single frame containing dots:

I = zeros(100,150, 'uint8'); 
I2 = imnoise(I, "salt & pepper", 0.01); 
I3 = imdilate(I2, strel("square", 3)); 
imshow(I3); 

There is a function named detectObjects in the example link you have shared, which does a poor job of preprocessing this image with morphological operations. Here is just the pre-processing part:

% Function performing the morphological operations in the example 
function mask = morph_ops1(mask) 
 
    mask = imopen(mask, strel('rectangle', [3,3])); 
    mask = imclose(mask, strel('rectangle', [15, 15])); 
    mask = imfill(mask, 'holes'); 
end 
I4 = morph_ops1(I3); 
imshow(I4); 
title('Original Morphological Operations') 

The following is a template function for blob analysis you can use:

function plotBlobs(img, originalImage, border) 

img = imbinarize(img); 

% Step 3: Create the Blob Analysis Object 
blobAnalyzer = vision.BlobAnalysis('BoundingBoxOutputPort', true, ... 
    'AreaOutputPort', false, ... 
    'CentroidOutputPort', true, ... 
    'MinimumBlobArea', 1, ... 
    'MaximumCount', 1000); % maximum number of dots to be tracked 

% Step 4: Analyze the Blobs 
[centroids, boxes] = step(blobAnalyzer, img); 

% Step 5: Display the Results 
imshow(originalImage); % Display the original image 
hold on; 
for i = 1:length(boxes) 
    if border 
        rectangle('Position', boxes(i, :), 'EdgeColor', 'r'); % Draw bounding boxes 
    else 
        plot(centroids(i,1), centroids(i,2), 'b.'); % Mark centroids 
    end 
end 
hold off; 
end 

Let’s run blob analysis on the obtained image from the example’s preprocessing:

plotBlobs(I4, I4, true);  

As you can see most of the nearby dots have been merged, and hence the blob analysis did a bad job. To tackle this, you can modify the code to apply a series of 2 gentle erode operation as follows:

I5 = I3; 
I5 = imerode(I5, strel("square", 2)); 
I5 = imerode(I5, strel("square", 2)); 
imshow(I5)  

Let us know find blobs:

plotBlobs(I5, I5, true);  

We can also plot the blob centroids on top of the original image to see if it aligns with the dots:

plotBlobs(I5, I3, false);  

The centroids seem to roughly match the dots. If you wish to use the x-y coordinates, you can access them in the centroids variable in the plotBlobs function. To know more about the blob analysis, you can check out: https://www.mathworks.com/help/releases/R2021a/vision/ref/vision.blobanalysis-system-object.html

Here is a complete set of morphological operations using MATLAB to customize your preprocessing stage: https://www.mathworks.com/help/releases/R2021a/images/morphological-dilation-and-erosion.html

After solving the object detection part, you can configure the Kalman filter of each dot by specifying different arguments:

1. You could set the MotionModel argument based on the whether the dots move with a constant velocity or constant acceleration.
2. The InitialLocation argument can be set to the centroid of the blob.
3. Since the centroid locations doesn’t exactly match the dots’ location the InitialEstimateError argument can also be specified for the tracker to accommodate the error.

Details on other arguments of configureKalmanFilter can be seen here:

https://www.mathworks.com/help/releases/R2021a/vision/ref/configurekalmanfilter.html

Based on how often dots eclipse or go out of the frame, you can also adjust the invisibleForTooLong and ageThreshold variables given in the example.

Regards.