Help me to understand function interp2

Can anyone help me understand the interp2 function well?

I need to reconstruct a video given optical flow vectors. I did it in the following way:

while hasFrame(video) 
    frameRGB = readFrame(video);
    frameGray = im2gray(frameRGB);  
    flow = estimateFlow(of,frameGray);
    [x, y] = meshgrid(1:size(frameGray,2), 1:size(frameGray,1));
Vx = flow.Vx; %optical flow vector
Vy = flow.Vy;
frame_double = im2double(frameGray);
o=o+1;
frame_predicted = interp2(frame_double, x-Vx, y-Vy, 'linear', 0);
if o>1
    frame_previous_real=frame_double;
elseif o==1
    frame_previous_real=frame_double;
end
end

Is it correct to consider the current frame to calculate the predicted frame?

And what will come out of inter2 is the next predicted frame and so on throughout the cycle?

Thank you very much to those who will answer me

Answer 1

[Jaynik]

Hi Nicole,

I understand that you are trying to understand how the “interp2” function works and want to understand if it is correct to consider current frame to estimate the next frame.

The “interp2” function is used to perform interpolation in two dimensions. Interpolation is a process of estimating values between known data points based on the available information. “interp2” takes a set of known data points, typically represented by a grid where each datapoint has some values associated with it as input, and it uses different interpolation methods like the ‘linear’ method that you have used to estimate the values at locations that are not part of the original grid. It calculates these estimates based on the values of the neighbouring data points. The output is the estimated values at the desired locations within the grid.

In the code, you need to keep transforming the current frame using “interp2” and generate the predicted frame. If you want to continue predicting subsequent frames, you will need to iteratively update the next frames and predict frames based on the optical flow vectors of the current frame.

Below is the code for the solution mentioned above:

frame_previous = [];
while hasFrame(video)
    frameRGB = readFrame(video);
    frameGray = im2gray(frameRGB);
    flow = estimateFlow(of, frameGray);
    Vx = flow.Vx;
    Vy = flow.Vy;
    if isempty(frame_previous)
        frame_predicted = frameGray;
    else
        [x, y] = meshgrid(1:size(frameGray, 2), 1:size(frameGray, 1));
        frame_previous_double = im2double(frame_previous);
        frame_predicted = interp2(frame_previous_double, x-Vx, y-Vy, 'linear', 0); 
    end
    frame_previous = frameGray; 
end

To save the reconstructed frames as a video, you will need to create a "VideoWriter" object and write the frames using the "writeVideo" function.

Please refer to the following documentation for more information:

• VideoWriter: https://www.mathworks.com/help/matlab/ref/videowriter.html
• writeVideo: https://www.mathworks.com/help/matlab/ref/videowriter.writevideo.html
• interp2: https://www.mathworks.com/help/matlab/ref/interp2.html

Hope this helps!

[NICOLE LONGHI]

Thank you so much! Your answer is very helpful