Error return code from hardware decoder is ignored in h264 decoder

[wl2776]

The function field_end(), located in the file libavcodec/h264.c

​http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/h264.c;h=66174778df67f1f946b1f217c1bbcb2a64748691;hb=HEAD#l2503

Lines 2528 - 2531 contain call to the end_frame function, processing its return code and writing diagnostic message.

However, the function return code, stored in the variable err, doesn't depend on this error code.

Moreover, return code of the function field_end is entirely ignored.
This function is called from

static int decode_frame()
(file libavcodec/h264.c, line 4037
​http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/h264.c;h=66174778df67f1f946b1f217c1bbcb2a64748691;hb=HEAD#l4037 )

and

static int decode_slice_header()
(file libavcodec/h264.c, line 2621
​http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/h264.c;h=66174778df67f1f946b1f217c1bbcb2a64748691;hb=HEAD#l2621 )

So, when an application calls avcodec_decode_video2() and the hardware decoder cannot decode frame, the application anyway receives got_picture = 1, but the frame is bad.

It my case the frame is filled with zeros, resulting in green rectangles.

[Carl Eugen Hoyos]

Please add information how you trigger this problem (which hardware, software and drivers).

Did you try to write a patch?

[wl2776]

I'm developing a video player, playing video streams from IP cameras. It contains its own RTP parser and uses libav's decoder.
My player uses DXVA2, the code for its support was copied from the VLC sources.

Please add information how you trigger this problem (which hardware, software and drivers).

I tried two video adapters: Intel Q43/Q45 chipset (integrated) and ATI Radeon HD 4550.
ATI drivers are the latest ones (version 8.930.0.0), Intel's drivers are somewhat older (version 8.15.10.1855)
OS was Windows 7, 32bit, in both cases.

I triggered this problem either by running several instances of the player or by simulating low network bandwidth (high network load) with the Network Emulator Tookit ( ​http://blog.mrpol.nl/2010/01/14/network-emulator-toolkit/ )

Anyway, I don't think this is relevant to the problem.
The problem is in the other place - not hardware bugs/errors, but ignoring hardware's report about it.

Did you try to write a patch?

In process.
I'm digging the code and trying to find a way to signal this hardware decoder error to the higher-level libav functions and the application.

[wl2776]

Looks like my initial diagnostics was wrong.
I've added some debugging output, and got_picture was 0 after all hardware errors.

However, I was observing green frames accompanied with the following messages:

[h264 @ 0E86DC60]=== avcodec_decode_video2 start ===
[h264 @ 0E86DC60]Frame num gap 2 13
[h264 @ 0E86DC60]Frame num gap 2 14
[h264 @ 0E86DC60]Frame num gap 2 15
[h264 @ 0E86DC60]Frame num gap 2 0
[h264 @ 0E86DC60]no picture ooo
[h264 @ 0E86DC60]illegal short term buffer state detected
[h264 @ 0E86DC60]=== avcodec_decode_video2 end === bytes_used 59245 got 1 (360)

The first and last line were produced by my application and lines in the middle were generated by the decoder.

[Carl Eugen Hoyos]

Sorry if I miss something but don't you expect green frames if the hardware decoding fails?

[wl2776]

No, I don't.
I prefer got_picture = 0 in such cases. This green flickering annoys and distracts users.

Further testing shows that sometimes these green frames are used as reference pictures - this is inacceptable.

[Carl Eugen Hoyos]

Didn't you write that got_picture actually is 0 in such cases?

[wl2776]

Replying to cehoyos:

Didn't you write that got_picture actually is 0 in such cases?

Yes, I did.

I've added analysis of the field_end() return code and calls to av_log in case of error in decode_frame and decode_slice_header.

Till now I've seen that got_picture was 360 after such an error. This has happened, however, only once during 2 hours of testing.

And in the debug output above in my comment3, got_picture is 360 (not zero), and there are no error messages about errors in field_end() - so such green frames are returned even after, probably, successfull decoding

So, I think I should change the title of this bug report.
The proper problem description would be -

Hardware assisted decoding returns frames, filled entirely with zeros, causing sometimes bad picture flickering.

[Carl Eugen Hoyos]

Replying to wl2776:

So, I think I should change the title of this bug report.
The proper problem description would be -

Hardware assisted decoding returns frames, filled entirely with zeros, causing sometimes bad picture flickering.

But iiuc this happens because the hardware fails (does not actually return a buffer containing decoded picture data) or do you believe there is a bug in FFmpeg?

[wl2776]

Replying to cehoyos:

Replying to wl2776:

So, I think I should change the title of this bug report.
The proper problem description would be -

Hardware assisted decoding returns frames, filled entirely with zeros, causing sometimes bad picture flickering.

But iiuc this happens because the hardware fails (does not actually return a buffer containing decoded picture data) or do you believe there is a bug in FFmpeg?

Ok, well, for now, I should assume that hardware decoder silently fails, without any indications. I tried DXVA2 Status reporting facility, it reports that everything is OK.
I cannot surely say about any bugs in FFmpeg, I don't know if the HW decoder fails because of FFmpeg sends it incorrect data or because data is invalid per se.

So, thank you for very useful discussion, sorry for disturbance, I think, this ticket should be closed now.