Introduction
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter, and stream audio and video. It is used by a number of media players (including VLC and MPlayer), video websites (including YouTube and Vimeo), web browsers (including Google Chrome/Chromium and Firefox), social media (including Facebook and Twitter) and countless other software projects in the multimedia field and beyond.
This is our ideas page for Google Summer of Code 2023. See the GSoC Timeline for important dates.
At the end of the program you can find all the results on the results page.
Information for Students
Getting Started
- Get to know FFmpeg. If you are a student interested in contributing to FFmpeg, it is recommended to start by subscribing to the ffmpeg-devel mailing-list, visiting our !Libera.Chat IRC channels (#ffmpeg-devel and #ffmpeg although you will find most developers on the first one), and exploring both the codebase and the development workflow. Feel free to contact us if you have any questions. Also do not hesitate to answer questions from other students on our IRC channel if you know the answer to something.
- Find a project. Listed on this page are mentored and un-mentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Un-mentored projects are additional ideas you may want to consider, but you will have to contact us to find a mentor. You can also propose your own project, if you can think of one that better fits your interest and skill level. If a project description is unclear or you have any questions, please get in touch with its mentor and/or join our IRC channel at #ffmpeg-devel.
- Contact us. If you decide on a project, get in touch with the community and let us know. If you want to work on a qualification task, let the respective mentor know so we can avoid duplicated efforts.
- Apply. Students should apply definitely before deadline on April 4th. The "work" period begins on Mai 29th and ends in November. Take a look at GSoC timeline for additional information. Note, make sure you apply to Google before April 4th, even if you have not yet finished your qualification task. Please apply as soon as possible: Applications can be improved until the 4th of April but not afterwards!
Note: A friendly reminder that while the application to GSoC is important for you and GSoC, FFmpeg mentors will not base their decision solely on the GSoC application. We will judge applicants based on their qualification tasks to understand their abilities in coding, learning the tools, communication skills etc. So please do not worry about your application being perfect for us. Although it is very important to follow GSoC's application rules so they can pay you.
Qualification Tasks
In order to get accepted you have to complete a small qualification task which in all cases include sending a patch to the development mailing list. FFmpeg development can be quite challenging and the qualification task helps us figure out whether you are motivated enough and have the potential to deliver successfully.
The qualification tasks are usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the FFmpeg Bug Tracker for qualification task ideas. In general qualification tasks should include submitting a patch to the ffmpeg-devel mailing list which passes review and is accepted into the FFmpeg codebase. It will be common for such patches to need multiple iterations of submissions and reviews, so don't wait too long with the first submission! Note, please avoid picking a qualification task which another student is already working on, each student should work on a different qualification task.
Development
If you are selected for a particular project then you are not only expected to present a working implementation but you should also submit your work for inclusion for the ffmpeg codebase. This should be done at least 2-3 weeks before the end of the second work period by sending patches to the ffmpeg-devel mailing list where the ffmpeg community and your mentor will review your work. You will likely be asked to make some changes and resend improved versions. If you feel that no consensus is reached about how something should be done then follow the advice of your mentor.
In order to create good quality patches make sure to read the Developer Documentation.
Contacting FFmpeg
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:
- Mailing-list: ffmpeg-devel
- IRC: #ffmpeg-devel on Libera.Chat
- FFmpeg GSoC Admins: Michael Niedermayer (michaelni in #ffmpeg-devel on Libera.Chat IRC, michaelni@gmx.at), Reynaldo Verdejo (reynaldo in #ffmpeg-devel on Libera.Chat IRC, R Verdejo on g mail), Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Libera.Chat IRC, ceffmpeg at gmail dot com), Thilo Borgmann (thilo in #ffmpeg-devel on Libera.Chat, thilo.borgmann at mail dot de)
You may also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.
Mentored Project Ideas
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor one of these ideas rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.
VC-2 Vulkan encoder
Description: Hardware encoders for mainstream codecs are commonplace, but come with many limitations, such as speed issues at high bitrates, limited pixel format support, and quality issues. VC-2 is a hardware-optimized wavelet codec which is meant for mezzanine streams (high bitrate, intra-only). Although it was designed for FPGA, it's suitable for being implemented purely as a GPU shader. It's very fast, high-quality and CBR, making it ideal for low-latency point-to-point intermediate streams.
Expected results: Write a VC-2 encoder in Vulkan, supporting wavelet depths of at least 2.
Prerequisites: Good C, GLSL, and Vulkan knowledge.
Difficulty: Hard
Qualification Task: Write a Haar transform in GLSL, able to run multiple times on an image to increase the decomposition depth.
Mentor: Lynne (Lynne in #ffmpeg-devel on Libera.Chat IRC, or @lynne:pars.ee on Matrix)
Backup Mentor: Niklas Haas (haasn in #ffmpeg-devel on Libera.Chat IRC)
Duration: 350 hours
Assembly code for VVC decoder
Description: VVC is the successor to the well-known codec HEVC. The whole video industry has high anticipation for VVC. The goal of the ffvvc project is to provide a VVC decoder for FFmpeg. You will help add assembly code to speedup the decoder.
Expected results: Implemented one of ALF/SAO/Deblock/Predict in assembly code.
Prerequisites: Good C and Assembly code, basic familiarity with Git, and knowledge of codecs.
Difficulty: Hard
Qualification Task: Add any assembly code to ffvvc. (hint: you can start with SAO, it's the same thing as HEVC)
Mentor: Nuo Mi (nuomi2021 at gmail dot com)
Backup Mentor: J. Dekker (jdek on IRC, jdek@itanimul.li)
Duration: 350 hours
ARM Assembly code for VVC decoder
Description: VVC is the successor to the well-known codec HEVC. The whole video industry has high anticipation for VVC. The goal of the ffvvc project is to provide a VVC decoder for FFmpeg. You will help add assembly code to speedup the decoder.
Expected results: 500+ lines of ARM code accepted by ffvvc
Prerequisites: Good C and ARM Assembly code, basic familiarity with Git, and knowledge of codecs.
Difficulty: Hard
Qualification Task: Add any ARM assembly code to ffvvc.
Mentor: Nuo Mi (nuomi2021 at gmail dot com)
Backup Mentor: J. Dekker (jdek on IRC, jdek@itanimul.li)
Duration: 350 hours
Improve VVC Decoder
Description: VVC is the successor to the well-known codec HEVC. The whole video industry has high anticipation for VVC. The goal of the ffvvc project is to provide a VVC decoder for FFmpeg. You will help add new features or fix bugs for ffvvc.
Expected results: 500+ lines of code merged into the ffvvc project.
Prerequisites: Good C and Assembly code, basic familiarity with Git, and knowledge of codecs.
Difficulty: Hard
Qualification Task: Fix any issue on https://github.com/ffvvc/FFmpeg/issues or any patch merged by ffvvc.
Mentor: Nuo Mi (nuomi2021 at gmail dot com)
Backup Mentor: Anton Khirnov (elenril on IRC, anton@khirnov.net)
Duration: 350 hours
AArch64 ASM for HEVC decoder
Description: Modern codecs require handwritten SIMD assembly for good performance, much of this has been implemented for HEVC but there are still some parts missing. You will help add assembly code to speedup the decoder. Many functions are extremely similar (such as different sizes or bit-depths) of filters and in some cases the existing code can be referenced with minor modifications.
Expected results: 500+ lines of aarch64 assembly, merged into FFmpeg.
Prerequisites: Good C and aarch64 assembly knowledge, familiarity with Git.
Difficulty: Medium
Qualification Task: Implement one of the missing HEVC ASM functions.
Mentor: J. Dekker (jdek on IRC, jdek@itanimul.li)
Backup Mentor: Thilo Borgmann (thilo.borgmann at mail dot de)
Duration: 350 hours
FATE test enhancement
Description: Current FATE (FFmpeg Automated Testing Environment) missed some functions. Like vvc decoder, vvc cbs, and hardware codec tests. This task will focus on adding new tests to the FATE
Expected results: 300+ lines of test/bugfix code merged into the ffvvc project or ffmpeg upstream.
Prerequisites: Good C and Assembly code, basic familiarity with Git, and knowledge of codecs.
Difficulty: Medium
Qualification Task: run FATE test for hevc conformance, tell us how to generate https://github.com/FFmpeg/FFmpeg/blob/master/tests/ref/fate/hevc-conformance-PS_A_VIDYO_3
Mentor: Nuo Mi (nuomi2021 at gmail dot com)
Backup Mentor: Thilo Borgmann (thilo.borgmann at mail dot de)
Duration: 175 hours
Audio Overlay Filter
Description: Audio overlay filter, useful to mix some samples in stream with arbitrary other samples at arbitrary points in stream as specified by user, final result may be for example to censor some parts of input audio stream.
Expected results: Full implementation with filter documentation merge in FFmpeg upstream.
Prerequisites: Good C skills, basic familiarity with git.
Difficulty: Easy
Qualification Task: Draw visual representation how this filter will process input samples and user options and also write higher abstract level pseudo code that describes filter functionality.
Mentor: Paul B Mahol (durandal_1707 on #ffmpeg-devel on Libera.Chat IRC) (onemda at gmail dot com)
Backup Mentor: Thilo Borgmann (thilo.borgmann at mail dot de)
Duration: 175 hours
Audio CLAP Host Filter
Description: Audio CLAP host filter, for handling CLAP - CLever Audio Plug-in plugins in FFmpeg. Similar to already available lv2 and ladspa filters.
Expected results: Full implementation with filter documentation merge in FFmpeg upstream.
Prerequisites: Good C skills, basic familiarity with git.
Difficulty: Medium-to-Hard
Qualification Task: Write basic filter that will detect CLAP host and print current CLAP host version to userspace as FFmpeg audio filter.
Mentor: Paul B Mahol (durandal_1707 on #ffmpeg-devel on Libera.Chat IRC) (onemda at gmail dot com)
Backup Mentor: Thilo Borgmann (thilo.borgmann at mail dot de)
Duration: 350 hours
Unmentored Projects
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.
Template Project
Description: Template Description
Expected results: Template results
Prerequisites: Template prequisites
Difficulty:
Qualification Task: Template Qualification Task
Mentor: someone @ somehwere.net
Backup Mentor: someoneelse @ somewhere.net
Duration: 175 or 350 hours
Your Own Project Idea
A student can also propose a self-defined project! The amount of work defined by such a project should last for the majority of the GSoC work period duration and the task must be approved by the community. Also, a capable mentor from the community must be assigned. Ideas and inspiration for such project ideas could be found by browsing bugs and feature requests on our bug tracker but are of course not limited to such.
Students can discuss an idea with the community via the ffmpeg-devel mailing-list, the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins directly for more information. It is obviously of importance to start such a discussion well before the application deadline.
