Extract frames from video every n-seconds

Hej,

I am new here and not sure weather my question is dumb or not. But I am getting really frustrated and need help :/

So I have several .dav files (each 1h real time long, will explain later). I merged these and converted to m4v (also other formats are possible, also tested avi) so they are over a timespan of 24h real time. The thing with the videos is, that they have a time stamp in the upper right corner, stating the time. So I know they are recorded over 1h (total 24h). When playing the video, it gets played with double speed, so the real video file is only circa 30mins (12h) long. Thats odd, but I can work with that (I thought).

I want to extract a frame from the video every 10 sec (real time). So when I know the fps of the video, I can do that easily. For example:

fps = 25

A frame every 10 seconds (video time) = 250 frames "distance"

Since double the speed: every 125th frame from the video gives me a frame every 10sec (in real time)

What I noticed: In the beginning the time stamps fit the 10sec distance, so 1st frame: 23:59:59; 2nd frame: 00:00:09; 3rd frame: 00:00:19 and so on. After around 12min (6min video time) this happens: 00:12:39; 00:12:48; 00:12:58; 00:13:08; 00:13:19; 00:13:28.

So the time stamp and framrate seem to be out of sync. But I really need the 10sec distance, since I have a corresponding dataset with an entry every 10sec. To match them I need the corresponding frames. Has someone an idea what could solve my problem?

On a side note: When I looked at the converted videos and extracted all frames, it seemed like the framerate was not constant. So I used a software to make it a constant framerate. But comparing the time stamp and the number of frames tells me that the framerate varys between 9 and 11 when using a framerate of 10. Is that normal or is it just the time stamp which is running unconncected from the video?

My code is down here:

vid=VideoReader(filename);
duration=vid.Duration;
fpsread=fps/2*10; %since double speed and every 10sec

for i = 1:fpsread:framestot
    frame = read(vid,i);
    time=int2str((i-1)/(fps/2));
    imwrite(frame,[savedirectory, time, '.jpg']);
end

I would be really happy if someone could help me. Thank you :)

[Bjorn Gustavsson]

This is (as far as I understand) a "mean-spirited" problem. By that I mean that you have 2 clocks, one that puts the time-stamp into your frames, and one that takes frames from the camera at supposedly a fixed frame-rate, and they don't agree. Which one to belive? I don't know, recently I've had a similar problem with videos of "unusual" framerates, I decided to go with the time-stamp and hope that is correcter. Your choise might be different, I sincerely whish you a safe journey.

What you could do to dig into this is to record a video of a "certified clock" of some sort, just to see what happens. If you then can trust the certified clock you will see which of your two original clocks misbehaves worse.

Answer 1

[Walter Roberson]

Non-constant framerate is increasingly common.

You can read specific times instead of regular frames. See https://www.mathworks.com/help/matlab/ref/videoreader.html#buv_tta-1

[Sven Boemwollen]

Okay. I could do that yes.

For example I start at 00:00 (=1st frame) and say okay next step at 00:10 (and give me that 2nd frame), the time between these two frames is 10sec video time. But according to the time stamp (over several frames), there are some frames missing and by that the time gets delayed. So time stamp and the 10sec distance do not dorrespond anymore.

I probably could use OCR to read the time stamp in the upper right corner (last digit would be enough) and when a 9 appears, I take that frame... what do you think about that?

[Walter Roberson]

The primary indexing information in movies is considered to be timestamps, not frame numbers. Whatever the 11th (say) frame is, it will have a timestamp recorded in its metadata, and it might not happen to be for 00:10. But the real 00:10 frame will have a 00:10 timestamp in its metadata, and when you set the time property in the video object, the video reader will find that frame no matter where it is.

(To be more accurate: it will find the earliest frame with time stamp at least as great as the one the time was set to. So if there is a frame at 00:09.93 and another at 00:10.24 then the 00:10.24 would be used. It does not take the nearest frame, it takes the first on-or-after the given time.)

If the timestamps in the metadata do not agree with the time you can read off from the image, then the movie is incorrectly recorded (or the clock was wrong.)

[Sven Boemwollen]

Thank you :)

So if I have a variable framerate, this wont effect the method you proposed, right? Or just slightly but saying matlab to etxract a frame at 00:00 and then every followin 10sec will defenetly give me a frame from every 10 sec after the "starting time" (independent from the framerate). That is great. But I think then I really have a porblem with my visible timestamp since it is "moving" slower than the actual video time then.

[Walter Roberson]

You could have a frame at 00:00 and another at 00:03 and 00:09 and 00:10 and as long as the internal metadata is right, when you ask for 00:10 it will find it, no matter that the frame times were completely irregular for the earlier frames.

It is something you can easily test, by setting the timestamp to 00:10, reading a single frame, viewing it, examining the clock time on it.