hevc_amf with 10-bit video incorrectly forces output color space to BT2020

[Adrian]

Summary of the bug:
Encoding any 10-bit video using hevc_amf encoder forces it's output color space to BT2020, even if the original file is using BT709. This is caused by this code here: ​https://github.com/FFmpeg/FFmpeg/blob/08b1bffa49715a9615acc025dfbea252d8409e1f/libavcodec/amfenc_hevc.c#L257-L266

Example video file: ​https://drive.google.com/file/d/1OFyV08QcJnrK8fNYCeQt2jTvW-Olt5Dw/view?usp=sharing

How to reproduce:

% ffmpeg.exe -i DJI_20241020171413_0160_D.MP4 -c:v hevc_amf -t 2 out.mp4
ffmpeg 7.1

ffprobe then showsyuv420p10le(tv, bt709/bt2020/smpte2084, progressive) and colors are messed up (depends on the player, but for example in Windows' Media Player)

[Adrian]

Looks like same issue for av1_amf

[Balling]

smpte2084 typically assumes primaries to be Bt.2020, for example in HDMI that is part of the standard.

The matrix on the other hand while ideally has to be the same as primaries (as that allows to just take Y plane and get black and white PQ image) can be anything.

[Adrian]

I don't think it should assume anything, rather it should just set the values based on avctx->color_primaries and avctx->color_trc

[Balling]

Please open sample for everyone

[Balling]

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20241010000834.1909-1-aicommander@gmail.com/

The RGB->YUV color conversion hardware available on current AMD GPUs
only supports BT.709 and BT.2020+PQ, but YUV input can be accepted
without restrictions since no color conversion is taking place

[Adrian]

This patch looks good to me, should fix this issue

Sample access is updated