FFmpeg concatenation dropping part of first video

Question

I am using ffmpeg.exe to concatenate two videos together. The first video is essentially a five-second intro rendered at 1080p at 25 frames per second, and it will always stay the same format. The second is a random video that can be any length and format, but will be an MP4 file (h264). This particular test case happens to be a video of resolution 400x240 at 30fps.

So from this, I have a 5 second intro video at 1080p25 being concatenated to a 240p30 video. Since they are both MP4 files, I first transcode each to an mpegts file using the following:

ffmpeg -y -i intro.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts intro.mp4.ts
ffmpeg -y -i video.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts video.mp4.ts

Now that the two files are converted to joinable formats, I run the following command to encode the two videos being joined:

ffmpeg -y -i "concat:intro.mp4.ts|video.mp4.ts" -c:v libx264 -preset ultrafast -crf 20 -vf fps=60 -b:v 8M -minrate 8M -maxrate 12M -bufsize -8M -c:a aac -b:a 256k output.mp4

This works almost as expected, but one thing keeps happening: When the encode gets to the 5 second mark (either the end of the intro or the beginning of the actual video), it displays this warning: DTS 126000 < 619200 out of order

The rest of the conversion runs fine. However, when I play back the output file, the intro plays, and then at the very end of the intro it jumps to the start of the next video. Some video players just skip the gap. Other players will play the rest of the intro without the sound in that particular area, leading me to believe that this is an audio issue.

Since the rest of the file plays just fine, I was experimenting with the settings and was surprised to find that when i change the x264 preset from ultrafast to medium or slow, it made the problem worse: The skip would happen earlier in the intro, but leave the actual video unaffected. Where ultrafast would cause the skip to omit the last half a second of video/audio, medium would cause a second to a second and a half, and slow would cut out over two seconds.

What is causing this, and how can I correct this problem? I have read suggestions of re-encoding both videos before concatenating, but I'd like to avoid that if possible, since the intro is going to stay the same every time. If that's the only solution, is there a way to have all of that performed with a single call to FFmpeg? The tool is called by a C# program I am writing.

Answer 1

You can use the suggestions from this answer to scale the videos if necessary:

ffmpeg -y -i intro.mp4 -i video.mp4 -filter_complex "\
[0:v:0]scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2[v0]; \
[1:v:0]scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2[v1]; \
[v0][0:a:0][v1][1:a:0]concat=n=2:v=1:a=1[v][a] \
" -map "[v]" -map "[a]" output.mp4

Note that I've applied this to all inputs, but you can of course selectively do that to the ones you know will be different from the output size. In case of different SAR you can use the setsar filter after pad, but I don't think it's strictly necessary:

[0:v:0]scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,setsar=sar=1[v0]