Single Object Tracking using SIFT features.

I am doing a project in MatLab that consists of the following:

I have got a folder containing 600 frames that, together, are a video of a motorcycle chasing, probably filmed from an helicopter. The thing is that I have to track this motorcycle along all frames, but I am having some issues.

The strategy I am following is taking the first frame and extracting the motorcycle from it. Then, I compare this extract with every frame, using the SIFT features (I'll attach the code below) but I am getting some problems:

1. The call to T = estimateGeometricTransform2D(m_kp_obj,m_kp_esc,"affine"); sometimes tells me there are not sufficeint match points. It basically works for some frames but others fail.
2. There are some frames from where nbox2 gives me a negative number, and I don't get why (there are others that work correctly).

I'll attach the code below. I really hope you can help me to solve these issues. Feel free to try the code and to apply any changes you feel convinient. Here is the link to the frames, the folder is called MotorcycleChase.zip ->https://drive.google.com/embeddedfolderview?id=1VE8QTjD8yWYvn5pYFj4DjVkIPEMmc4s_#list

Thank you all! You are saving my life!

% myTracker, codi inicial del short project

close all

% Llegim el fitxer d'anotacions groundtruth_rect.txt: frame, bounding boxes, is_lost

BB = importdata('./MotorcycleChase/groundtruth_rect.txt');

Idir = dir('./MotorcycleChase/img/*.jpg');

% figure

% hold on % mantenim sempre oberta la mateixa figura

filename = horzcat(Idir(1).folder,'/',Idir(1).name);

I = imread(filename);

imshow(I);

moto = imcrop(I,BB(1,2:5));

imshow(moto);

im_obj = rgb2gray(moto);

kp_obj = detectSIFTFeatures(im_obj);

kp_obj = selectStrongest(kp_obj,50);

[feat_obj,kp_obj] = extractFeatures(im_obj,kp_obj);

nf = size(Idir);

for i = 2:nf

filename = horzcat(Idir(i).folder,'/',Idir(i).name);

im_esc = rgb2gray(imread(filename));

kp_esc = detectSIFTFeatures(im_esc);

kp_esc = selectStrongest(kp_esc,50);

[feat_esc,kp_esc] = extractFeatures(im_esc,kp_esc);

pairs = matchFeatures(feat_obj,feat_esc,'MatchThreshold',5);

% Check if we have enough matched points

m_kp_obj = kp_obj(pairs(:,1),:);

m_kp_esc = kp_esc(pairs(:,2),:);

T = estimateGeometricTransform2D(m_kp_obj,m_kp_esc,"affine");

[f,c] = size(im_obj);

box = [1, 1; c, 1; c, f; 1, f; 1, 1];

nbox = transformPointsForward(T,box);

nbox2 = [nbox(1,1), nbox(1, 2), nbox(2, 1)-nbox(1,1), nbox(3,2)-nbox(1,2)];

overlapRatio = bboxOverlapRatio(nbox2,BB(i,2:5))

imshow(im_esc); % mostra el frame

rectangle('Position',nbox2,'EdgeColor','blue');

drawnow

end

Answer 1

[Diwakar Diwakar]

1. "Not sufficient match points" error: The error message you're receiving indicates that there are not enough matching SIFT features between the object (motorcycle) in the first frame and the subsequent frames. This can happen if the motorcycle's appearance significantly changes across frames or if the SIFT features are not robust enough to handle such variations.

To improve feature matching, you can try using a different feature descriptor, such as SURF (Speeded-Up Robust Features) or ORB (Oriented FAST and Rotated BRIEF). These descriptors may provide better performance in terms of matching under different conditions. You can replace the detectSIFTFeatures and extractFeatures functions with their SURF or ORB counterparts in your code.

kp_obj = detectSURFFeatures(im_obj);
kp_esc = detectSURFFeatures(im_esc);

[feat_obj, kp_obj] = extractFeatures(im_obj, kp_obj);
[feat_esc, kp_esc] = extractFeatures(im_esc, kp_esc);

2. Negative values for nbox2: The negative values in nbox2 indicate that the bounding box coordinates are not in the expected format. The format should be [x, y, width, height], where (x, y) represents the top-left corner of the bounding box, and width and height represent the dimensions.

To fix this issue, you can check if the width and height values calculated from nbox are negative and correct them accordingly. Here's an example of how to handle this:

nbox2 = [nbox(1, 1), nbox(1, 2), abs(nbox(2, 1) - nbox(1, 1)), abs(nbox(3, 2) - nbox(1, 2))];

Remember to adjust any other parts of your code that rely on the changes made. Good luck with your project!

[Pol]

Hi!

Thank you very much for your answer!

It works a little bit better, but it still crashes with some frames at the end. I would really appreciate if you could test it on your own and do the proper modifications, that would really help me a lot since I don't know how to proper modify the code or identify mistakes.

Please you would help me a lot.

Thanks again for your answer. <3