You'll need to initialize your hardware accelerator correctly, as shown in the documentation below (perhaps we should create a wiki entry for this in time?):
Assume the following snippet:
ffmpeg -re -threads 4 -loglevel debug \
-init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device intel -filter_hw_device intel \
-i 'udp://$ingest_ip:$ingest_port?fifo_size=9000000' \
-vf 'format=nv12|vaapi,hwupload' \
-c:v h264_vaapi -b:v $video_bitrate$unit -maxrate:v $video_bitrate$unit -qp:v 21 -sei +identifier+timing+recovery_point -profile:v main -level 4 \
-c:a aac -b:a $audio_bitrate$unit -ar 48000 -ac 2 \
-flags -global_header -fflags +genpts -f mpegts 'udp://$feed_ip:$feed_port'
Where:
(a). VAAPI is available, and we will bind the DRM node /dev/dri/renderD128 to the encode session, and
(b). We are taking a udp input, where $ingest_ip:$port_ip corresponds to a known UDP input stream, matching the IP and port pairing respectively, with a defined fifo size (as indicated by the '?fifo_size=n' parameter).
(c). Encoding to an output udp stream packaged as an MPEG Transport stream (see the muxer in use, mpegts), with the necessary parameters matching the output IP and port pairing respectively.
(d). Defined video bitrates ($video_bitrate$unit, where $unit can be either K or M, as you see fit) and audio bitrates ($audio_bitrate$unit, where $unit should be in K, for AAC LC-based encodings) as shown above, with appropriate encoder settings passed to the vaapi encoders. For your reference, there are four available video encoders in FFmpeg as at the time of writing, namely:
i. h264_vaapi
ii. hevc_vaapi
iii. vp8_vaapi
iii. vp9_vaapi
With the omission of the mjpeg encoder (as its' not of interest in this context), and each of these encoders' documentation can be accessed via:
ffmpeg -hide_banner -h encoder=$encoder_name
Where $encoder_name matches the encoders on the list above.
For VAAPI, the following notes apply:
- VAAPI-based encoders can only take input as VAAPI surfaces, so it will typically need to be preceeded by a hwupload instance to convert a normal frame into a vaapi format frame. Note that the internal format of the surface will be derived from the format of the hwupload input, so additional format filters may be required to make everything work, as shown in the snippet above:
i. -init_hw_device vaapi=intel:/dev/dri/renderD128 initializes a hardware device named vaapi (that can be called up later via the -hwaccel_device and -filter_hw_device as demonstrated above) bound to the DRM render node /dev/dri/renderD128. The intel: prefix can be dropped, but its' often useful to identify what render node was used by a vendor name in an environment where more than one VAAPI-capable device exist, such as a rig with an Intel IGP and an AMD GPU.
ii. Take note of the format constraint defined by -hwaccel_output_format vaapi. This is needed to satisfy the condition in 1.
iii. We then pick up the named hardware acceleration implementation, vaapi, and call it up for both the hardware accelerator device (-hwaccel_device) and the device to which we will upload the hardware frames via the hwupload filter (-filter_hw_device). Omitting the latter will result in encoder initialization failure, as you observed.
iv. Now, inspect the video filter syntax closely:
-vf 'format=nv12|vaapi,hwupload'
This video filter chain converts any unsupported video frames to the VAAPI hardware format, applying a known constraint prior to uploading the frames to the device via hwupload. This is done for safery reasons; you cannot assume that the decoded format will be accepted by the encoder. Performance in this mode will vary, based on source, the decoder device and the VAAPI driver in use.
v. Now, for the video encoder (defined by -c:v $encoder_name), pass your arguments as needed. You can modify the example I provided in the snippet above, though its' wise to refer to the encoder documentation as explained earlier should you need further tuning.
Bonus: Dealing with the Intel-based QSV encoders:
I'm including this section for future reference, for these who use Intel's open source MSDK for FFmpeg's QSV enablement and the associated encoders. See the snippet below:
ffmpeg -re -threads 4 -loglevel debug \
-init_hw_device qsv=qsv:MFX_IMPL_hw_any -hwaccel qsv -filter_hw_device qsv \
-i 'udp://$ingest_ip:$ingest_port?fifo_size=9000000' \
-vf 'hwupload=extra_hw_frames=10,vpp_qsv:deinterlace=2,format=nv12' \
-c:v h264_qsv -b:v $video_bitrate$unit -rdo 1 -pic_timing_sei 1 -recovery_point_sei 1 -profile high -aud 1 \
-c:a aac -b:a $audio_bitrate$unit -ar 48000 -ac 2 \
-flags -global_header -fflags +genpts -f mpegts 'udp://$feed_ip:$feed_port'
You can see the similarities.
The QSV encoders use VAAPI-style mappings (as explained above), but with an extra constraint placed for the hwupload filter: The hwupload=extra_hw_frames=10 parameter must be used, or the encoder's initialization will fail.
One of the reasons I cannot recommend QSV's encoders, despite their supposedly better output quality, are their fragile mappings, that often exit with some of the most unhelpful errors often unrelated to how the encoder failed.
Where possible, stick to VAAPI. QSV's usefulness (where applicable) is for low power encoding, as is the case with Intel's Apollolake and anemic Cannonlake initial offerings.
Hope this documentation will be of use to you.
