From the concat filter documentation:

All corresponding streams must have the same parameters in all segments; the filtering system will automatically select a common pixel format for video streams, and a common sample format, sample rate and channel layout for audio streams, but other settings, such as resolution, must be converted explicitly by the user.

You will have to scale and pad (or crop) to make 480x640 fit into 640x480:

ffmpeg -i 480.mp4 -i 640.mp4 -filter_complex \
"[0:v]scale=640:480:force_original_aspect_ratio=decrease,pad=640:480:(ow-iw)/2:(oh-ih)/2[v0]; \
 [v0][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" \
-map "[v]" -map "[a]" -c:v libx264 -c:a aac -movflags +faststart output.mp4

• Your ffmpeg is old: you should really consider updating to a build from the current git master branch. The easiest way to do that is to just download a binary.

• If you're scripting this then you should "conform" each input to common parameters using fps, format/aformat, and scale filters to avoid ffmpeg from choosing unexpected and undesired "common formats".

• Also see Resizing videos with ffmpeg to fit into static sized player.

From the concat filter documentation:

All corresponding streams must have the same parameters in all segments; the filtering system will automatically select a common pixel format for video streams, and a common sample format, sample rate and channel layout for audio streams, but other settings, such as resolution, must be converted explicitly by the user.

You will have to scale and pad (or crop) to make 480x640 fit into 640x480:

ffmpeg -i 480.mp4 -i 640.mp4 -filter_complex \
"[0:v]scale=640:480:force_original_aspect_ratio=decrease,pad=640:480:(ow-iw)/2:(oh-ih)/2[v0]; \
 [v0][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" \
-map "[v]" -map "[a]" -c:v libx264 -c:a aac -movflags +faststart output.mp4

• Your ffmpeg is old: you should really consider updating to a build from the current git master branch. The easiest way to do that is to just download a binary.

• If you're scripting this then you should "conform" each input to common parameters using fps, format/aformat, and scale filters to avoid ffmpeg from choosing unexpected and undesired "common formats".

• Also see Resizing videos with ffmpeg to fit into static sized player.

From the concat filter documentation:

All corresponding streams must have the same parameters in all segments; the filtering system will automatically select a common pixel format for video streams, and a common sample format, sample rate and channel layout for audio streams, but other settings, such as resolution, must be converted explicitly by the user.

You will have to scale and pad (or crop) to make 480x640 fit into 640x480:

ffmpeg -i 480.mp4 -i 640.mp4 -filter_complex \
"[0:v]scale=640:480:force_original_aspect_ratio=decrease,pad=640:480,setsar=1[v0]; \
 [1:a]aformat=channel_layouts=stereo[a1]; \
 [v0][0:a][1:v][a1]concat=n=2:v=1:a=1[v][a]" \
-map "[v]" -map "[a]" -c:v libx264 -c:a aac -movflags +faststart output.mp4

From the concat filter documentation:

All corresponding streams must have the same parameters in all segments; the filtering system will automatically select a common pixel format for video streams, and a common sample format, sample rate and channel layout for audio streams, but other settings, such as resolution, must be converted explicitly by the user.

You will have to scale and pad (or crop) to make 480x640 fit into 640x480:

ffmpeg -i 480.mp4 -i 640.mp4 -filter_complex \
"[0:v]scale=640:480:force_original_aspect_ratio=decrease,pad=640:480:(ow-iw)/2:(oh-ih)/2[v0]; \
 [v0][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" \
-map "[v]" -map "[a]" -c:v libx264 -c:a aac -movflags +faststart output.mp4

• Your ffmpeg is old: you should really consider updating to a build from the current git master branch. The easiest way to do that is to just download a binary.

• If you're scripting this then you should "conform" each input to common parameters using fps, format/aformat, and scale filters to avoid ffmpeg from choosing unexpected and undesired "common formats".

• Also see Resizing videos with ffmpeg to fit into static sized player.