Opened 4 years ago
Last modified 15 months ago
#9188 open enhancement
Add support for Vulkan Video decode/encode extensions
Reported by: | Jacob Lifshay | Owned by: | Lynne |
---|---|---|---|
Priority: | wish | Component: | undetermined |
Version: | git-master | Keywords: | vulkan |
Cc: | programmerjake@gmail.com, Lynne | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Vulkan Video is aiming to be an industry-standard cross-platform interface to video decode/encode hardware (and probably also software).
I'm proposing adding support for using Vulkan Video as a new ffmpeg codec backend. (probably a better way to describe that...)
They (as of 2021-04-14) have provisional support for encode/decode for h.264 and decode for h.265, they are planning on shortly gaining support for encode for h.265, decode for vp9, and encode/decode for av1.
An Introduction to Vulkan Video - The Khronos Group Inc
Supporting Vulkan Video would allow hardware vendors (such as the Libre-SOC project, which I'm part of) to only need to implement each codec once, rather than N times for each of N different software libraries that need support.
Funding may be available from Libre-SOC, will have to check.
Change History (4)
comment:1 by , 4 years ago
Cc: | added |
---|
comment:2 by , 4 years ago
Keywords: | vulkan added; Vulkan Video VK_KHR_video_queue removed |
---|---|
Priority: | normal → wish |
Version: | unspecified → git-master |
comment:3 by , 17 months ago
Cc: | added |
---|---|
Owner: | set to |
Status: | new → open |
comment:4 by , 15 months ago
The Vulkan decoding in ffmpeg 6.0 is an excellent addition, albeit unfinished.
The lack of entries in the CLI's decoders list (e.g. ffmpeg -h decoder=hevc_vulkan
) makes learning and troubleshooting difficult.
There seems to be a disconnect between the pixel formats that can be decoded and the pixel formats that can be passed to Vulkan-based filters in the filter graph. The VK decoder passes 'yuvj420p' to vk_libplacebo (which is referred to as 'libplacebo' in the CLI) where the Vulkan device then complains that the format is unsupported. Explicitly converting the incoming frames to a compatible format will use swscale
when copying to system memory is undesirable.
Excerpts of verbose command line output: https://imgur.com/a/khx9nix
Decoding implemented and merged in git master.
I've implemented encoding, but the spec is still in beta, and there are large issues with it that have to be addressed before it can be usable and merged.