Introduction
This page shows the results from the student projects that were part of GSoC 2024.
Projects
Enhancement of FFmpeg's libvmaf Filter (Yigithan Yigit)
My project was adding a public API to libmvaf(vmaf) to allow per-frame metadata propagation. This explanation might be little abstract let me explain;
Libvmaf can make feature extractions and fuse those extracted features to calculate/predict model feature(s) (vmaf itself is a predicted). You can see that extracted features by setting output file in vmaf cli or FFmpeg, but problem is you can only see that file when process is finished, in other words you can’t see extracted features while doing feature extraction or prediction.
My project was propagate those features to a user defined callback function when features are ready. In short I made program able to show extracted features when they extracted instead of waiting to process finish.
Implementing the eSSIM metric (Andrea Mastroberti)
The goal was to implement options for the vf_ssim.c filter in libavfilter such that it could compute eSSIM (enhanced SSIM). To do this, I've implemented options for integral images, resizable window size/stride for local statistics and pooling methods to aggregate said statistics (in particular, Minkowski pooling).
https://gist.github.com/dre-droid/effd22ce0f7ed13914d446367b4bab64
x86 assembly code for the VVC decoder (Stone Chen)
The goal was to implement x86 assembly code for the VVC decoder, focusing on two major functions: the DMVR SAD and Deblocking. Stone contributed by upstreaming the SAD function, achieving a performance boost of approximately 16-25x for SAD itself. The Deblocking function is nearly complete, with only the Luma long filter remaining. He will continue working on this until it is fully upstreamed, sharing much of the code with HEVC.
Updating FFmpeg's SITI Video Filter to Support HDR and ITU-T P.910:2022 Compliance (Poorva Gaikar)
My project focused on enhancing FFmpeg's SITI (Spatial Information and Temporal Information) filter to ensure compliance with the latest ITU-T P.910 specification, specifically the 07/22 revision. This involved updating the calculation methods to support high dynamic range (HDR) content.
The SITI filter measures spatial and temporal distortions in video, and my work included refining the Spatial Information (SI) and Temporal Information (TI) calculations for greater accuracy. To achieve this, I integrated the VQEG SITI-tools' Python reference implementation into the C-based FFmpeg codebase, ensuring consistency between both implementations.
In addition to ensuring compliance with ITU-T P.910, I also addressed issues related to the scaling and normalization of luma values, which are crucial for processing HDR content.
Vulkan HWAccel for VC2 decoder (Petro Mozil)
The project's aim is to create a HWAccel for the VC2 decoder. Since VC2 is a subset of dirac, the hwaccel is in the dirac implementation. Decoding the VC2 stream consists of (roughly) three steps:
- Decoding the Golomb encoding
- Dequantization
- Inverse wavelet transform
The hwaccel focuses on the dequantization and wavelet transforms, since paralellizing Golomb is nearly not feasible. The HWAccel currently support all waveletss and yuv420 / yuv444 pixel formats. It currently does not support the yuv422 pixel format
https://gist.github.com/pmozil/a8ff5b07672321f884ad363d49c4c414
Vulkan VC2 encoder (George Stergianas)
In this project a Vulkan GPGPU encoder was implemented for the VC2 video format. It supports wavelet depths from 1 up to 4, yuv420p/yuv444p formats and various slice sizes. At the time of writing, only Haar and Legall(5_3) wavelets are implemented. The first step performed by the encoder, is to upload plane data to a coefficient buffer and then run a number of dispatches in succession to apply wavelet transform on input data. After that a compute shader that calculates optimal slice sizes is executed as well as the actual encoding process itself. The encoder makes exclusive use of buffer device address and compute shaders.
https://gist.github.com/raphaelthegreat/e166a3423401d6e31fbc4876dc641135
