Accurately extract frames every N seconds using FFmpeg

Question

The Usage

I extract images from videos using ffmpeg.

I dump one down-scaled image every 10 seconds, inclusive, that I combine into montages with imagemagick. These montages again are used to show a preview from the video when hovering the scrubber in a web-based video player. (Calculating which image in the montage to show).

The Command

After playing around I ended up with the following command where the idea is speed over quality:

ffmpeg \
    -loglevel error \
    -hwaccel cuvid \
    -hwaccel_output_format cuda \
    -c:v h264_cuvid \
    -i "$video_file" \
    -r 0.1 \
    -filter:v "scale_cuda=w=-1:h=100,thumbnail_cuda=2,hwdownload,format=nv12" \
    -color_range 2 \
    f%09d.jpg

This seemed to work fine. The shots are off by about ± 0.5 - 1 sec here and there, but that is livable.

The Problem

Issue is that ffmpeg produces one extra image at start of videos. E.g. files are:

file             time
f000000001.jpg   00:00:00
f000000002.jpg   00:00:00
f000000003.jpg   00:00:10
f000000004.jpg   00:00:20
f000000005.jpg   00:00:30
...

Sometimes first and second is off by a few milliseconds.

As I know it (now) I can simply delete the first image and proceed with the rest, but not sure why this happens and if it is a bug or something else.

Put another way: I need to know if the "effect" of two first frames are reliable so that I can delete it in other versions of ffmpeg as well.

As I use the images to show 10 sec. snapshot from the video at a specified time it is off by 10 seconds if I do not delete first image generated. If for some reason it should then not create a dupe at start, other version or what ever, deleting the first image would create the same issue.

Montage

(If of interest the montages are created something like):

montage -tile 5x -geometry +0+0 -background none [file1  - file50 ]  montage01.jpg
montage -tile 5x -geometry +0+0 -background none [file51 - file100]  montage02.jpg
...

---

Command I use now based on answer (shell):

# Set on call or global:
file_in=sample.mp4
pix_fmt=yuvj420p
sec_snap_interval=10
nr_start=1
pfx_out=snap


ffmpeg \
    -loglevel warning \
    -hwaccel cuvid \
    -hwaccel_output_format cuda \
    -c:v h264_cuvid \
    -i "$file_in" \
    -pix_fmt "$pix_fmt" \
    -filter:v "
        scale_cuda=
            w = -1 :
            h = 100,
        thumbnail_cuda = 2,
        hwdownload,
        format = nv12,
        select = 'bitor(
            gte(t - prev_selected_t, $sec_snap_interval),
            isnan(prev_selected_t)
        )'
    " \
    -vsync passthrough \
    -color_range 2 \
    -start_number "$nr_start" \
    "$pfx_out%09d.jpg"

Answer 1

Using -r 0.1 sets the output framerate to 0.1Hz, but it's not guaranteed to get frame from the input video exactly every 10 seconds (I am not sure why).

One way for solving it is using select filter.

Example (without GPU acceleration):

ffmpeg -i input.mp4 -vf "select=bitor(gte(t-prev_selected_t\,10)\,isnan(prev_selected_t))" -vsync 0 f%09d.jpg

---

• bitor(gte(t-prev_selected_t\,10) is 1 when difference between "passed" timestamps is grater of equal 10 seconds.
  When expression is evaluated to 1, the frame is "selected" and passed to the output.
• bitor with isnan(prev_selected_t) passes the first frame, where prev_selected_t is NaN (has no value).
• -vsync 0 applies "passthrough" - Each frame is passed with its timestamp from the demuxer to the muxer.

---

Here is an example with scale_cuda and thumbnail_cuda:

ffmpeg \
    -loglevel error \
    -hwaccel cuvid \
    -hwaccel_output_format cuda \
    -c:v h264_cuvid \
    -i "$video_file" \
    -filter:v "scale_cuda=w=-1:h=100,thumbnail_cuda=2,hwdownload,format=nv12,select=bitor(gte(t-prev_selected_t\,10)\,isnan(prev_selected_t))" \
    -vsync 0 \
    -color_range 2 \
    f%09d.jpg   

• Due to the usage of thumbnail_cuda filter, we have to place the select filter at the end.

---

Testing:
Build synthetic video with frame counter at 10fps:

ffmpeg -y -f lavfi -r 10 -i testsrc=size=128x72:rate=1:duration=1000 -vf setpts=N/10/TB -vcodec libx264 -pix_fmt yuv420p input.mp4

---

Output frames after executing the above command:

As you can see, the selected frames are exactly every 10 seconds.