1

I am trying to use hardware acceleration to horizontally flip videos but I can't seem to get it right. I don't want to change the codec but just keep the same (just hflipped). I am using Intel Arc Pro A30M Graphics on a Windows 11 Pro machine.

Without hardware acceleration I normally do this:

.\ffmpeg -i input.mp4 -vf hflip -c:a copy output.mp4

which works but it is slow (2-4x).

With hardware acceleration, I tried this but it fails:

.\ffmpeg.exe -hwaccel qsv -i input.mp4  -c:a copy -vf hflip -c:v h264_qsv output.mp4

I get the following error:

Impossible to convert between the formats supported by the filter
'graph 0 input from stream 0:0' and the filter 'auto_scale_0'

What am I missing here?

The full log is:

ffmpeg version 2024-01-07-git-90bef6390f-full_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 36.101 / 58. 36.101
  libavcodec     60. 37.100 / 60. 37.100
  libavformat    60. 20.100 / 60. 20.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 17.100 /  9. 17.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
[vist#0:0/h264 @ 000001cb0a934100] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.29.100
  Duration: 02:20:24.64, start: 0.000000, bitrate: 3808 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 3674 kb/s, 23.98 fps, 23.98 tbr, 24k tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
[AVHWDeviceContext @ 000001cb0a926d00] Using device 8086:56b0 (Intel(R) Arc(TM) Pro A30M Graphics).
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_qsv) -> h264 (h264_qsv))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scale_0'
[vf#0:0 @ 000001cb0a0dec00] Error reinitializing filters!
[vf#0:0 @ 000001cb0a0dec00] Task finished with error code: -40 (Function not implemented)
[vf#0:0 @ 000001cb0a0dec00] Terminating thread with return code -40 (Function not implemented)
[vost#0:0/h264_qsv @ 000001cb0a08d140] Could not open encoder before EOF
[vost#0:0/h264_qsv @ 000001cb0a08d140] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_qsv @ 000001cb0a08d140] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 000001cb0ac7a040] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A
Conversion failed!
Improve this question

1 Answer 1

Reset to default
1

Since hflip filter is executed on the CPU, and the h264_qsv decoder requires the data to be in the memory of the GPU, we have to add hwdownload and hwupload filters:

ffmpeg.exe -y -init_hw_device qsv=hw,child_device_type=dxva2 -hwaccel qsv -hwaccel_output_format qsv -c:v h264_qsv -i video_with_audio.mp4 -c:a copy -vf hwdownload,format=nv12,hflip,hwupload=extra_hw_frames=10 -c:v h264_qsv output.mp4

Even when the GPU is an internal GPU that uses the system memory for storing the decoded/encoded frames, we still have to use the sequence:
QSV Decode -> download -> flip -> upload -> QSV Encode.
The reason for that is that QSV hardware acceleration uses memory buffers that are allocated using the GPU device driver (and even if not, FFmpeg consider the buffers as "device memory" buffers).

After hwdownload, the pixel format is qsv, and we have to use format=nv12 before hflip filter (hflip doesn't support qsv pixel format).

hwupload=extra_hw_frames=10 uploads the frames to the "device" (to GPU memory), while allocating 10 extra buffers (the extra buffers may be needed by the encoder).

There is a better solution:
We may use GPU accelerated horizontal flipping using vpp_qsv filter:

ffmpeg.exe -y -init_hw_device qsv=hw,child_device_type=dxva2 -hwaccel qsv -filter_hw_device hw -hwaccel_output_format qsv -extra_hw_frames 10 -c:v h264_qsv -i input.mp4 -c:a copy -vf vpp_qsv=transpose=hflip -c:v h264_qsv output.mp4

The above command, executes the entire decode -> flip -> encode pipeline on the GPU - it supposed to be faster...

Note: The above command in working on my machine using ffmpeg version 5.1.2-full_build-www.gyan.dev.

It is working only when the monitor is connected to Intel HD Graphics iGPU (and not working when the monitor is connected to NVIDIA GPU).

There are multiple known issues using the vpp_qsv filter, and it might not work.

Improve this answer
8
  • Thanks for response. When I issue both commands, I get the following error: [AVHWDeviceContext @ 0000029e588f8740] Error creating a MFX session: -9. Device creation failed: -1313558101. Failed to set value 'qsv=hw,child_device_type=dxva2' for option 'init_hw_device': Unknown error occurred Error parsing global options: Unknown error occurred I am using an i9-12950HX CPU / Laptop.
    – Sha
    Commented Jan 10 at 8:59
  • Try without -init_hw_device qsv=hw,child_device_type=dxva2. Using FFmpeg 5.1, I am getting a warning message without it. I am using Windows 10, with much older CPU.
    – Rotem
    Commented Jan 10 at 10:02
  • Removing that, works: ffmpeg.exe -y -hwaccel qsv -hwaccel_output_format qsv -c:v h264_qsv -i input.mp4 -c:a copy -vf hwdownload,format=nv12,hflip,hwupload=extra_hw_frames=10 -c:v h264_qsv output.mp4 and runs at 13.2x. But now the bitrate is reduced to something like 1160 kbps with noticeable quality degradation. Original bitrate is around 3800 kbps. Is it possible to keep a similar or original bitrate? I noticed this from ffmpeg: Stream #0:0(und): Video: h264 (avc1 / 0x31637661), qsv(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 1000 kb/s, 23.98 fps, 24k tbn (default)
    – Sha
    Commented Jan 10 at 10:22
  • FFmpeg doesn't look at the bitrate of the input while re-encoding. You have to tune the encoding parameters for getting the desired qualify. For getting quality that resembles the input, you probably have to select higher bitrate for compensating for the re-encoding artifacts.
    – Rotem
    Commented Jan 10 at 11:28
  • Okay I was not aware. When you use the normal CPU command, e.g.: .\ffmpeg -i input.mp4 -vf hflip -c:a copy output.mp4 ffmpeg will use same bitrate as original file. But hardware conversion does this differently?
    – Sha
    Commented Jan 10 at 12:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .