1

I'm trying to transcode a video (with audio) to HEVC using Intel's SVT-HEVC encoder. The video's audio tracks are in sync before transcoding.

Both ffprobe and mediainfo agree that the input video is at 23.98 fps (24 / 1.001):

# ffprobe
Stream #0:0: Video: h264 (High), yuv420p(progressive), 1872x1080 [SAR 1:1 DAR 26:15], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)

# mediainfo
Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Width                                    : 1 872 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 1.733
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS

Now I've constructed the following ffmpeg + SvtHevcEncApp command-line on my Linux system to transcode this video into H.265:

inputFile="testmovie.mp4"
outputFile="testmovie.hevc"

: ${frameCount:=68808}
: ${videoWidth:=1872}
: ${videoHeight:=1080}
: ${videoBitRate:=2359296}
: ${videoFrameRateNum:=24000}
: ${videoFrameRateDenom:=1001}

ffmpeg \
        -hide_banner \
        -i "$inputFile" \
        -nostdin \
        -f rawvideo \
        -pix_fmt yuv420p \
        - | SvtHevcEncApp \
                -i stdin \
                -n $frameCount \
                -w $videoWidth \
                -h $videoHeight \
                -rc 1 \
                -tbr $videoBitRate \
                -fps-num $videoFrameRateNum \
                -fps-denom $videoFrameRateDenom \
                -profile main \
                -encMode 0 \
                -bit-depth 8 \
                -tier main \
                -b "$outputFile"

I can then observe how ffmpeg hands the frames over to SvtHevcEncApp and the latter encodes them into an .hevc stream.

My problem is that when I remux the .hevc stream with the audio tracks, they quickly run out of sync (1 minute into the clip, the audio track is more than 1 second off already). Despite both the original clip and the transcoded clip showing 23.98 fps in ffprobe and mediainfo.

A scene that happens in the original clip at 62.042 seconds happens in the HEVC clip at 59.542 seconds. Since it isn't playing at a faster framerate, it SvtHevcEncApp must be skipping/losing frames.

What could be going wrong here? Is it possible that ffmpeg alters the frame rate (i.e. duplicates some frames) as it outputs them to stdout?

Improve this question
2
  • 1
    If you extract the avc stream (with -f h264 -c:v copy) and then remux it like you did with the hevc one, does the problem occur?
    – Tom Yan
    Commented May 7, 2023 at 16:40
  • I tested this with ffmpeg -i testmovie.mp4 -c:v copy videoonly.mp4 and on a raw AVC stream via mkvextract -i testmovie.mkv tracks 0:videoonly.avc, then remuxed with mkvtoolnix. In both cases, video + audio remained perfectly in sync. Transcoding with the built-in libx265 also avoids the issue (but takes weeks rather than hours).
    – Cygon
    Commented May 7, 2023 at 18:36

1 Answer 1

Reset to default
0

I never found out why the audio runs out of sync, except some vague feeling that MKVtoolnix may not be figuring out the framerate of the .hevc file correctly.

My workaround so far has been to convert the .hevc file to .mkv with some very specific parameters:

mkvOutputFile=${outputFile%.*}.mkv

mkvmergeParameters=()
mkvmergeParameters+=(--default-duration 0:${videoFrameRateNum}/${videoFrameRateDenom}fps)
mkvmergeParameters+=(--fix-bitstream-timing-information 0)
mkvmerge \
    "${mkvmergeParameters[@]}" \
    "$outputFile" \
    -o "$mkvOutputFile"

This produces an .mkv file from the .hevc file that will continue to play at the correct frame rate when muxed with other streams (i.e. audio).

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .