QSV decoder fails to get configured and issues Error initializing the MFX video decoder: unsupported (-3)

[tipoman]

I have a RTSP server streaming h265(HEVC) video.

I can play the video with Intel HW acceleration like this:

ffplay -loglevel info  -vcodec hevc_qsv -rtsp_transport tcp -i rtsp://192.168.1.88

This uses two-way data RTP+RTCP exchange via additional UDP port.
I need to be able to play the video by only receiving the RTP stream.
I disabled incoming traffic on the server, made the server stream the video to the client IP on UDP port 5600.
By looking at the messages log I sniffed the SDP info and saved it to a file named 60.sdp
Added a=rtcp-mux option and was able to successfully see the video with this pipeline:

ffmpeg -loglevel trace -protocol_whitelist udp,file,hevc,rtp,crypto -i ~/Videos/60.sdp -vcodec rawvideo -f matroska - | ffplay -i -

60.sdp file

v=0
o=StreamingServer 38990265062388 38990265062388 IN IP4 0.0.0.0
s=RTSP Session
c=IN IP4 127.0.0.1
t=0 0
a=range:npt=0-
m=video 5600 RTP/AVP 97
a=control:video
a=rtpmap:97 H265/90000
a=fmtp:97 profile-level-id=016000; packetization-mode=1; 
a=framerate:60
a=recvonly
a=rtcp-mux

So far everything worked as expected.

Then I added the HW acceleration option like this:

ffmpeg -hide_banner -loglevel trace -vcodec hevc_qsv -protocol_whitelist udp,file,hevc,rtp,crypto -i ~/Videos/60.sdp -vcodec rawvideo -f matroska - | ffplay -hide_banner -i -

and got this error:

[AVHWDeviceContext @ 0x5555b8020ac0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.3.2 ().
[AVHWDeviceContext @ 0x5555b8020ac0] Driver not found in known nonstandard list, using standard behaviour.
[hevc_qsv @ 0x5555b7fe5500] Initialized an internal MFX session using hardware accelerated implementation
[hevc_qsv @ 0x5555b7fe5500] Error initializing the MFX video decoder: unsupported (-3)
Error while decoding stream #0:0: Function not implemented

Just to confirm, I'm able to play the same video stream on the local UDP port with hardware acceleration with GStreamer like this :

gst-launch-1.0 udpsrc port=5600 buffer-size=65536 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H265" ! rtph265depay ! queue max-size-buffers=1 ! vaapih265dec ! autovideosink sync=false

Some details on the system.

i3-1215u, Ubuntu 22.04.3
vainfo
Trying display: wayland
Trying display: x11
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.19 (libva 2.19.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 23.3.2 ()

ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers

Seems there is some problem of configuring the Intel QSV decoder from the ffmpeg pipeline.
I think SPS/PPS/VPS settings are not set, but I can not be sure.
Am I missing something or this is an issue in libavcodec/qsvdec.c ?

[wenbin,chen]

This may be a same issue with ​https://trac.ffmpeg.org/ticket/9766
And I comment ​https://trac.ffmpeg.org/ticket/9766#comment:15. Hope it helps you.
Also I suggest to use vaapi/d3d11va to decode stream without sps/pps as they use ffmpeg build-in parser to parse stream.

[wenbin,chen]

According to ​https://spec.oneapi.io/onevpl/latest/API_ref/VPL_func_vid_decode.html#_CPPv431MFXVideoDECODE_DecodeFrameAsync10mfxSessionP12mfxBitstreamP16mfxFrameSurface1PP16mfxFrameSurface1P12mfxSyncPoint

“Before decoding the first frame, a sequence header (sequence parameter set in H.264 or sequence header in MPEG-2 and VC-1) must be present. The function skips any bitstreams before it encounters the new sequence header”

[wenbin,chen]

After commit ae06111d7406e541ac232318bcf50f8510a36f38. bsf can be used after demuxing.

Add dump_extra bsf and pull stream like this:

 ffmpeg -v verbose -bsf:v dump_extra -hwaccel qsv -i rtsp://localhost:8554/mystream -f null -

solve the problem on my side.