Enhancement Request: Improve VAAPI Color Type Detection for HDR Content

[Peyton Lee]

Summary:
I am encountering an issue where HDR content is being incorrectly detected as BT.709 color type when using VAAPI filter (scale_vaapi). I believe that the color type detection for VAAPI needs to be enhanced to properly handle HDR content.

Description:
While trying to play an HDR video using FFmpeg with VAAPI, the color type is mistakenly identified as BT.709 instead of the correct HDR color type (e.g., BT.2020).

Specifically, in my test video:

• Color Primaries = 9 (BT.2020)
• Color Transfer = 16 (SMPTE2084)
• Color Space = 1 (BT.709)

In the function vaapi_vpp_fill_colour_standard, the best format is determined as BT.709. I believe the logic in this function should be enhanced to better handle such cases.

Suggested Enhancement:
Enhance the color type detection logic in the VAAPI module to correctly identify HDR color types. For example, additional checks of the transfer function (SMPTE2084 is often used in HDR content) to ensure accurate color type recognition for HDR content, especially in the vaapi_vpp_fill_colour_standard function.

Thank you for considering this request.

[MasterQuestionable]

͏    Would you help answering this question?
͏    ​https://github.com/mpv-player/mpv/issues/14377#issuecomment-2185350874

͏    In my not very through testing, I noted 8bpc (8 bit-per-component) content may already make:
͏    Color gradient of ~ 4 range hardly noticeable (to almost entirely unnoticeable).
͏    And ~ 2 side-by-side is basically entirely visually indistinguishable [1].
͏    .
͏    So I much wonder what HDR does really?

[ [1]
͏    For most colors, yes.
͏    However for certain grayish colors, e.g. #2b2932 vs. #2b2a32:
͏    Differed only by 1: but have quite distinct visual difference.
͏    .
͏    E.g. 2: #25232c vs. #25242c ]

[Peyton Lee]

Replying to MasterQuestionable:

I'm not entirely sure about the composition of HDR .PNG files, but when playing HDR videos, the video post processor needs to recognize that the video is in HDR format in order to perform HDR color transformation. Skipping this step would result in a darker color rendition on screen during playback.

͏    Would you help answering this question?
͏    ​https://github.com/mpv-player/mpv/issues/14377#issuecomment-2185350874

͏    In my not very through testing, I noted 8bpc (8 bit-per-component) content may already make:
͏    Color gradient of ~ 4 range hardly noticeable (to almost entirely unnoticeable).
͏    And ~ 2 side-by-side is basically entirely visually indistinguishable.
͏    .
͏    So I much wonder what HDR does really?

[MasterQuestionable]

͏    Much PNG irrelevant.
͏    I mean the overall necessity of HDR.

͏    Wouldn't it be preferable to not even bother HDR outright?
͏    (normalizing all contents to use a single definition)

͏    As workaround, try various colorspace conversions in:
͏    ​https://trac.ffmpeg.org/wiki/colorspace

[Peyton Lee]

Thank you for your response and suggestion.
I understand your pint now, but my intention is to leverage new hardware capabilities for this HDR conversion.

Specifically, I am working with hardware that is designed to handle color space conversion and scaling more efficiently, and I believe enhancing FFmpeg's VAAPI module to support this would be highly beneficial.

Our developing hardware (currently accessed through VAAPI) can handle HDR conversion tasks more efficiently, leading to lower power consumption and better color performance. This hardware is continuously being optimized, and leveraging it for these tasks would provide significant advantages.

If avfilter can indicate through the color type that the current video is in HDR (BT.2020) format, it will significantly increase the support for our hardware.

[MasterQuestionable]

͏    Per <colorspace>, there are various types of unnecessarily sophisticated havocs...
͏    More than just simple "BT.2020".

͏    If you deem certain assertion reasonable enough:
͏    You may pass the relevant color primaries directly to VAAPI as workaround.
͏    (I believe? not familiar with the API, but which should be)

͏    Improving the detection logic in "vaapi_vpp_fill_colour_standard"...
͏    Well, help appreciated.