How do you track an object and plot its motion against time?

I'm trying to track the motion of a point on an oscillating pendulum and plot it's displacement versus time graph.

I know i have to : 1) upload the video

2) read each frame

3) track a point on each frame

4) plot the motion of that point w.r.t time

And so far, i've used this:

vidObj = VideoReader('115.2ANGLE.mp4');
vidObj.NumFrames
 
while hasFrame(vidObj)
  frame = readFrame(vidObj);
  imshow(frame)
  pause(1/vidObj.FrameRate);
end
 
a=VideoReader('115.2ANGLE.mp4');
for img = 1:a.NumFrames;
   filename=strcat('frame',num2str(img),'.jpg');
   b = read(a, img);
   imshow(b);
   imwrite(b,filename);
end

But all this does is tell me how many frames there are.

The video in question is: https://imgur.com/ZytKDKd and i need to track the black dot on the white, oscillating part.

I've already accomplished this using tracker software, and i'd like to get a graph similar to this one, (which i got for an initial displacement of 24 deg)

Any idea how i can do the same in matlab?

Answer 1

[Image Analyst]

You need to segment each frame to find where the object(s) is/are. Maybe you can do this by threholding or imbinarize(). I'd try these steps (untested)

% Convert to gray scale
grayImage = rgb2gray(frame);
% Crop frame to get rid of white window to the right.
grayImage = grayImage(:, someColumn : end); % You need to determine the column
%========================================================
% First find the white paddle.
% Threshold or call imbinarize()
paddleMask = imbinarize(grayImage);
% Fill holes
paddleMask = imfill(paddleMask, 'holes');
% Take largest blob only.  That's the white paddle
paddleMask = bwareafilt(paddleMask, 1);
% Find centroid of white paddle.
props = regionprops(paddleMask, 'Centroid')
% Get x and y
xPaddle(frameIndex) = props.Centroid(1)
yPaddle(frameIndex) = props.Centroid(2)
%========================================================
%  Now get the black dot.
blackBlobs = grayImage < someThresholdValue;
% Erase junk outside the paddle
blackBlobs = blackBlobs & paddleMask;
% Note, black dot might not be resolved enough to find it!
% Take largest blob only.  That's the biggest black dot.
blackBlobs = bwareafilt(blackBlobs, 1);
% Find centroid of black dot.
props = regionprops(blackBlobs, 'Centroid')
% Get x and y
xDot(frameIndex) = props.Centroid(1)
yDot(frameIndex) = props.Centroid(2)

frameIndex is what you misleadingly called "img". It's not an image, it's a loop interation counter/index.

[KLETECH MOTORSPORTS]

I got a bunch of errors when i used this. Also i don't understand what you've done at all. I am extremely new to image processing btw.

Is there a simpler way?

[Image Analyst]

Please attach the video here with the paper clip icon. In the meantime, try this and adapt as needed.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 14;

vidObj = VideoReader('rhinos.avi')
numberOfFrames = vidObj.NumFrames
xPaddle = zeros(1, numberOfFrames);
yPaddle = zeros(1, numberOfFrames);
xDot = zeros(1, numberOfFrames);
yDot = zeros(1, numberOfFrames);
for frameIndex = 1 : numberOfFrames
	frame = read(vidObj, frameIndex);
	subplot(2, 3, 1);
	imshow(frame)
	axis('on', 'image');
	title('Original Image', 'fontSize', fontSize);
	drawnow;
	% Convert to gray scale
	grayImage = rgb2gray(frame);
	% Crop frame to get rid of white window to the right.
	someColumn = 250; % Whatever...
	grayImage = grayImage(:, 1 : someColumn); % You need to determine the column
	subplot(2, 3, 2);
	imshow(grayImage)
	axis('on', 'image');
	impixelinfo;
	title('Gray Scale Image', 'fontSize', fontSize);
	drawnow;
	
	%========================================================
	% First find the white paddle.
	% Threshold or call imbinarize()
	paddleMask = imbinarize(grayImage);
	% Fill holes
	paddleMask = imfill(paddleMask, 'holes');
	% Take largest blob only.  That's the white paddle
	paddleMask = bwareafilt(paddleMask, 1);
	subplot(2, 3, 3);
	axis('on', 'image');
	imshow(paddleMask)
	title('Paddle Mask', 'fontSize', fontSize);
	drawnow;
	% Find centroid of white paddle.
	props = regionprops(paddleMask, 'Centroid');
	% Get x and y
	xPaddle(frameIndex) = props.Centroid(1);
	yPaddle(frameIndex) = props.Centroid(2);
	%========================================================
	%  Now get the black dot.
	thresholdValue = 200;
	blackBlobs = grayImage < thresholdValue;
	% Erase junk outside the paddle
	blackBlobs = blackBlobs & paddleMask;
	% Note, black dot might not be resolved enough to find it!
	% Take largest blob only.  That's the biggest black dot.
	blackBlobs = bwareafilt(blackBlobs, 1);
	subplot(2, 3, 4);
	imshow(blackBlobs)
	axis('on', 'image');
	title('Black Dot', 'fontSize', fontSize);
	drawnow;
	% Find centroid of black dot.
	props = regionprops(blackBlobs, 'Centroid');
	% Get x and y
	xDot(frameIndex) = props.Centroid(1);
	yDot(frameIndex) = props.Centroid(2);
	subplot(2, 3, 5:6);
	plot(xDot, 'b.-', 'LineWidth', 2, 'MarkerSize', 15);
	hold on;
	plot(yDot, 'r.-', 'LineWidth', 2, 'MarkerSize', 15);
	grid on;
	title('Black Dot.  X = Blue, y = red', 'fontSize', fontSize);
	xlabel('Frame Number', 'fontSize', fontSize);
	ylabel('Column or Row', 'fontSize', fontSize);
	if frameIndex == 1
		% Maximize the figure window.
		g = gcf;
		g.WindowState = 'maximized';
	end
	drawnow;
end

fprintf('Done running %s.m ...\n', mfilename);

[KLETECH MOTORSPORTS]

I can't attach the video because it's too big, but here's the link : https://imgur.com/ZytKDKd

[Image Analyst]

You can because it's 1.8 MB and the upload limit is 5 MB. So I've downloaded it, made some improvements to the code for you, and the results are below and attached.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 14;

vidObj = VideoReader('ZytKDKd - Imgur.mp4')
numberOfFrames = vidObj.NumFrames
% numberOfFrames = 10 % For making testing quicker
xPaddle = nan(1, numberOfFrames);
yPaddle = nan(1, numberOfFrames);
xDot = nan(1, numberOfFrames);
yDot = nan(1, numberOfFrames);
for frameIndex = 1 : numberOfFrames
	frame = read(vidObj, frameIndex);
	subplot(2, 3, 1);
	imshow(frame)
	axis('on', 'image');
	caption = sprintf('Original Image Frame %d of %d', frameIndex, numberOfFrames);
	title(caption, 'fontSize', fontSize);
	drawnow;
	% Convert to gray scale
	grayImage = rgb2gray(frame);
	% Crop frame to get rid of white window to the right.
	grayImage = grayImage(:, 1 : 500); % You need to determine the column
	subplot(2, 3, 2);
	imshow(grayImage)
	axis('on', 'image');
	impixelinfo;
	caption = sprintf('Gray Scale Image Frame %d of %d', frameIndex, numberOfFrames);
	title(caption, 'fontSize', fontSize);
	drawnow;
	
	%========================================================
	% First find the white paddle.
	% Threshold or call imbinarize()
	paddleMask = imbinarize(grayImage);
	% Fill holes
	paddleMask = imfill(paddleMask, 'holes');
	% Take largest blob only.  That's the white paddle
	paddleMask = bwareafilt(paddleMask, 1);
	% We need to shrink the paddle because the edges are less bright and they
	% get detected when we try to get the dot.  So let's explude edges.
	se = strel('disk', 15, 0);
	paddleMask = imerode(paddleMask, se);
	subplot(2, 3, 3);
	axis('on', 'image');
	imshow(paddleMask)
	impixelinfo;
	title('Paddle Mask', 'fontSize', fontSize);
	drawnow;
	% Find centroid of white paddle.
	props = regionprops(paddleMask, 'Centroid');
	% Skip it if the paddle can't be found, like it's too dark or something.
	if isempty(props)
		continue;
	end
	% Get x and y
	xPaddle(frameIndex) = props.Centroid(1);
	yPaddle(frameIndex) = props.Centroid(2);
	%========================================================
	%  Now get the black dot.
	thresholdValue = 220;
	blackBlobs = grayImage < thresholdValue;
	% Erase junk outside the paddle
	blackBlobs = blackBlobs & paddleMask;
	% Erase the axle.
	blackBlobs(104:157, 264:319) = false;
	% Note, black dot might not be resolved enough to find it!
	% Take blobs only if they're in the 2-50 pixel range.
	% That should get us the black dot.
	props = regionprops(blackBlobs, 'Centroid', 'Area');
	allAreas = [props.Area]
	blackBlobs = bwareafilt(blackBlobs, [2, 70]);
	% Sometimes there is noise, and we get 2 blobs, so just take the largest blob in that range.
	blackBlobs = bwareafilt(blackBlobs, 1);
	subplot(2, 3, 4);
	imshow(blackBlobs)
	impixelinfo;
	axis('on', 'image');
	title('Black Dot', 'fontSize', fontSize);
	drawnow;
	% Find centroid of black dot.
	props = regionprops(blackBlobs, 'Centroid', 'Area');
	allAreas = [props.Area]
	% Skip it if the dot can't be found, like it's too blurred or something.
	if isempty(props)
		fprintf('No dot found on frame #%d of %d.\n', frameIndex, numberOfFrames);
		continue;
	end
	% Get x and y
	xDot(frameIndex) = props.Centroid(1);
	yDot(frameIndex) = props.Centroid(2);
	subplot(2, 3, 5:6);
	plot(xDot, 'b.-', 'LineWidth', 1, 'MarkerSize', 15);
	hold on;
	plot(yDot, 'r.-', 'LineWidth', 1, 'MarkerSize', 15);
	grid on;
	title('Black Dot.  X = Blue, y = red', 'fontSize', fontSize);
	xlabel('Frame Number', 'fontSize', fontSize);
	ylabel('Column or Row', 'fontSize', fontSize);
	if frameIndex == 1
		% Maximize the figure window.
		g = gcf;
		g.WindowState = 'maximized';
	end
	drawnow;
end
% Interpolate any missing ones.
missingIndexes = isnan(xDot);
if any(missingIndexes)
	xFit = 1 : numberOfFrames;
	xDot = interp1(find(~missingIndexes), xDot(~missingIndexes), xFit, 'spline');
	yDot = interp1(find(~missingIndexes), yDot(~missingIndexes), xFit, 'spline');
	subplot(2, 3, 5:6);
	plot(xDot, 'b.-', 'LineWidth', 1, 'MarkerSize', 15);
	hold on;
	plot(yDot, 'r.-', 'LineWidth', 1, 'MarkerSize', 15);
	grid on;
	title('Black Dot.  X = Blue, y = red', 'fontSize', fontSize);
	xlabel('Frame Number', 'fontSize', fontSize);
	ylabel('Column or Row', 'fontSize', fontSize);
end
fprintf('Done running %s.m ...\n', mfilename);

Shown is the result for the first 20 frames. It could be improved even more but I've already spent way too much time on this than normal and I have to leave something left for you to do, right?

[KLETECH MOTORSPORTS]

Right. I'll check it out, thanks.

[Image Analyst]

If it works, could you then click the "Accept this answer" link? Thanks in advance. Otherwise let me know of any errors.

[Sara K Takiguchi]

Hi i am using this code for my project on 2021b Matlab and getting this error code:

Intermediate dot '.' indexing produced a comma-separated list with 12 values, but it must produce a

single value when followed by subsequent indexing operations.

Error in motiontrackingandplotting (line 60)

xPaddle(frameIndex) = props.Centroid(1);

Do you know what this means and how to fix it?

[Image Analyst]

Your segmentation probably didn't find any blobs so props is empty.

If you have any more questions, then, in a new thread (not here), attach your data and code to read it in with the paperclip icon after you read this:

TUTORIAL: How to ask a question (on Answers) and get a fast answer