How do I quickly generate low quality animated GIFs with ffmpeg?

Question

We generate a lot of thumbnail GIFs whose quality does not matter nearly as much as the time it takes to generate them. Generating high quality GIFs with ffmpeg is very well covered, but I'm not having much luck figuring out how to generate low quality ones as quickly as possible.

The palette computation takes up most of the execution time with the following command (taken from the multi-chain filtergraph answer here: How to efficiently create a best-palette gif from a video portion straight from the web):

ffmpeg -y -threads 8 -r 24 -f image2 -start_number 1 -i "frames.%04d.jpg" -filter_complex "fps=24,scale=150:-1:flags=fast_bilinear,split=2 [a][b]; [a] palettegen [pal] fifo [b]; [b] [pal] paletteuse" output.gif

The execution time of that command with 1000 frames is about 72 seconds. About 67 seconds of that is the palette pass, and then it blazes through the actual GIF generation in about 5 seconds. I would like to get the whole execution time down as much as possible, and willing to sacrifice a lot of image quality for speed.

Answer 1

Your use of the palettegen/paletteuse filters is making the command run slower. The simple way to achieve a lower-quality GIF would be:

ffmpeg -f image2 -i "frames.%04d.jpg" output.gif

With additional scaling:

ffmpeg -f image2 -i "frames.%04d.jpg" -vf scale=150:-1 output.gif

You can also drop frames in the output GIF, i.e. sample the frames, so that not all of them are processed. E.g. to have only 1 FPS output, by using an fps filter:

ffmpeg -i "frames.%04d.jpg" -vf "fps=fps=1,scale=150:-1" output.gif

Answer 2

I was tasked with reducing the amount of time it took to generate an animated GIF as close as possible to 30 frames in length at 150 pixels width. Most of the sequences we generate are under 1000 frames. We had a 15,000 frame sequence and our render nodes were taking 17 minutes to produce this ~30-frame GIF, which is unacceptably slow.

We were using ffmpeg as a demuxer and piping to imagemagick. Through several hours of experimentation, I arrived at the following conclusions:

• The number of input frames you ask ffmpeg to process is BY FAR the most impactful input in terms of execution speed. If using the concat demuxer to skip input frames is an option, this will make the biggest performance difference. By taking every 5th frame, I was able to reduce total computation time to 1 minute 45 seconds with high-quality lanczos rescaling and per-frame palette computation. Generating our 30-frame preview thumbnail now takes under 1 second.

• The rescaling algorithm was the next largest performance impactor (but a far distant second). Using fast_bilinear instead of lanczos saved 150 seconds of compute time over all 15,000 frames.

• The least impactful variable was palette computation, and this varied with rescale algorithm. Over 15,000 frames using lanczos, we saved around 17 seconds of execution time if we eliminated palette computation. Using fast_bilinear, we saved around 75 seconds of execution time.

Because the rescaling algorithm and palette computation were negligible, we ended up keeping them at the highest quality. We have reduced our computation time from 17 minutes to under 1 second mostly by telling ffmpeg to skip reading input files.

KEY TAKEAWAY: SKIPPING INPUT FRAMES vs SKIPPING OUTPUT FRAMES

The reason our process was taking so long is that frame dropping does not help execution time when using the image2 demuxer. If you muck with the -r flag and the fps filter, you will affect the number of frames which appear in the final GIF, but ffmpeg appears to still do something with all 15,000 input frames.

The only way I could find to have ffmpeg skip input frames is by using the concat demuxer.

Here is how I now generate high-quality animated GIF thumbnails on my dev machine in under 1 second by skipping input frames:

# create text file which describes the ~30 input frames we want ffmpeg to process
seq -f "file 'left_frames.%04g.jpg'" 10000 500 25000 > tmp.txt

# generate the animated gif using ffmpeg only
ffmpeg -f concat -i tmp.txt -filter_complex "scale=150:-1:flags=lanczos,split=2 [a][b]; [a] palettegen [pal]; [b] fifo [b]; [b] [pal] paletteuse" output.gif