How does FFMpeg read from a remote s3 url

Question

I'm intrigued how does FFMpeg read a video file from a remote url (for example, an aws s3 presigned url).

• Does it download the file in chunks and then process it chunk by chunk?
• Or does it download some bytes or frames and then continues to process it?
• Or it internally streams the file content and starts processing it?

Because I've observed as soon as I trigger the command it starts processing and the progress can be seen from the very first second. It doesn't show any latency that would naturally come due to a download phase. I've even tried it for larger video files of Gigabytes.

Sample command:

ffmpeg -i "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4" -c:v libx264 -vf scale=1024:576 -c:a aac output.mp4

Just want to understand the working and behind-the-scenes. I'll appreciate if you can also point me to some resources and documentation where I can get to know more about it.

Answer 1

Every HTTP client works by streaming the files – that's how downloads work. Downloading a file via HTTP is just "streaming to a file on disk" in the sense that the HTTP client continuously receives new data (generally in IP-packet-sized chunks of around 1.5kB each) and has to write it out as it comes (with some buffering).

So in most cases, all that's happening is that there is no "save to disk" step, instead the decoder directly consumes data received by the HTTP client.

Sometimes the file format might not be fully compatible with stream decoding (e.g. the MP4 metadata might be stored at the end of the file instead of the beginning, if the author forgot to encode it with +faststart), in which case FFmpeg will need to make additional HTTP 'Range' requests to directly receive data from specific positions within the file.