Introduction
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter, and stream audio and video. It is used by a number of media players (including VLC and MPlayer), video websites (including YouTube and Vimeo), web browsers (including Google Chrome/Chromium and Firefox), social media (including Facebook and Twitter) and countless other software projects in the multimedia field and beyond.
This is our ideas page for Google Summer of Code 2022. See the GSoC Timeline for important dates.
At the end of the program you can find all the results on the results page.
Information for Students
Getting Started
- Get to know FFmpeg. If you are a student interested in contributing to FFmpeg, it is recommended to start by subscribing to the ffmpeg-devel mailing-list, visiting our !Libera.Chat IRC channels (#ffmpeg-devel and #ffmpeg although you will find most developers on the first one), and exploring both the codebase and the development workflow. Feel free to contact us if you have any questions. Also do not hesitate to answer questions from other students on our IRC channel if you know the answer to something.
- Find a project. Listed on this page are mentored and un-mentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Un-mentored projects are additional ideas you may want to consider, but you will have to contact us to find a mentor. You can also propose your own project, if you can think of one that better fits your interest and skill level. If a project description is unclear or you have any questions, please get in touch with its mentor and/or join our IRC channel at #ffmpeg-devel.
- Contact us. If you decide on a project, get in touch with the community and let us know. If you want to work on a qualification task, let the respective mentor know so we can avoid duplicated efforts.
- Apply. Students should apply definitely before deadline on April 19th. The "work" period begins on June 13th and ends in September. Take a look at GSoC timeline for additional information. Note, make sure you apply to Google before April 19th, even if you have not yet finished your qualification task. Please apply as soon as possible: Applications can be improved until the 19th of April but not afterwards!
Note: A friendly reminder that while the application to GSoC is important for you and GSoC, FFmpeg mentors will not base their decision solely on the GSoC application. We will judge applicants based on their qualification tasks to understand their abilities in coding, learning the tools, communication skills etc. So please do not worry about your application being perfect for us. Although it is very important to follow GSoC's application rules so they can pay you.
Qualification Tasks
In order to get accepted you have to complete a small qualification task which in all cases include sending a patch to the development mailing list. FFmpeg development can be quite challenging and the qualification task helps us figure out whether you are motivated enough and have the potential to deliver successfully.
The qualification tasks are usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the FFmpeg Bug Tracker for qualification task ideas. In general qualification tasks should include submitting a patch to the ffmpeg-devel mailing list which passes review and is accepted into the FFmpeg codebase. It will be common for such patches to need multiple iterations of submissions and reviews, so don't wait too long with the first submission! Note, please avoid picking a qualification task which another student is already working on, each student should work on a different qualification task.
Development
If you are selected for a particular project then you are not only expected to present a working implementation but you should also submit your work for inclusion for the ffmpeg codebase. This should be done at least 2-3 weeks before the end of the second work period by sending patches to the ffmpeg-devel mailing list where the ffmpeg community and your mentor will review your work. You will likely be asked to make some changes and resend improved versions. If you feel that no consensus is reached about how something should be done then follow the advice of your mentor.
In order to create good quality patches make sure to read the Developer Documentation.
Contacting FFmpeg
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:
- Mailing-list: ffmpeg-devel
- IRC: #ffmpeg-devel on Libera.Chat
- FFmpeg GSoC Admins: Michael Niedermayer (michaelni in #ffmpeg-devel on Libera.Chat IRC, michaelni@gmx.at), Reynaldo Verdejo (reynaldo in #ffmpeg-devel on Libera.Chat IRC, R Verdejo on g mail), Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Libera.Chat IRC, ceffmpeg at gmail dot com), Thilo Borgmann (thilo.borgmann at mail dot de)
You may also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.
Mentored Project Ideas
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor one of these ideas rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.
Fuzzing FATE tests
Description: FATE is FFmpegs self / regression test system. Fuzzing with oss-fuzz is used to find corner cases which trigger anomalies like undefined behavior. FATE tests are sub optimal to be part of fuzzing as they use hardcoded parameters as input for each test. For example libavcodec/tests/snowenc.c runs some random and fixed tests but no tests controlled by some data from a fuzzer. So even if a anomaly occurs during a random test, this would not be reproducible nor could a fuzzer direct execution towards more anomalous cases.
Expected results: Connect about 1/3 to half, more if time permits of */tests/*.c tests to oss-fuzz so that the fuzzer can fully affect the data in each test.
Prerequisites: Good C and Makefile coding skills, basic familiarity with Git, knowledge of oss-fuzz.
Difficulty: Medium
Qualification Task: Connect 2 of the FATE tests from 2 different parts of libavcodec / libavfilter / libavformat / libswscale / libswresample to work under oss-fuzz. Passing qualification will be judged by having working code pushed into FFmpeg git and oss-fuzz git before the deadline given to us for choosing projects.
Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Libera.Chat IRC, michaelni@gmx.at)
Backup Mentor: Thilo Borgmann (thilo.borgmann at mail dot de)
Duration: 350 hours
Extend vsrc_mandelbrot filter
Description: The vsrc mandelbrot filter generates a realtime zoom through the mandelbrot fractal with various ways of coloring it. This principle can be extended relatively easily towards other fractals.
Expected results: Support about 50 other fractals. The student has to choose and implement them. As well as appropriate coloring schemes for each. If it turns out that 50 is too much work it is understood that a smaller number will be adequate too.
Prerequisites: Good C coding skills, good knowledge of fractals and mathematics needed for turning them into pretty pictures, basic familiarity with Git.
Difficulty: Easy
Qualification Task: Add 2 other fractals to vsrc mandelbrot. To pass qualification the change must be accepted in FFmpeg git before the deadlines to request the slot number.
Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Libera.Chat IRC, michaelni@gmx.at)
Backup Mentor: Thilo Borgmann (thilo.borgmann at mail dot de)
Duration: 175 hours
Add support for Part 15 to the JPEG 2000 decoder
Description: FFmpeg contains a native JPEG 2000 decoder. It is missing support for JPEG 2000 Part 15 (ISO/IEC 15444-15 | Rec. ITU-T T.814), which (a) adds a new entropy coder which can be 10x faster and (b) introduces a JPH file format which is a minor extension to the JP2 file format. Several implementations of the entropy coder exist, e.g. https://github.com/aous72/OpenJPH, https://github.com/uclouvain/openjpeg, https://github.com/osamu620/OpenHTJ2K
Expected results: Primary objective: successfully decode a collection of test codestreams. Stretch goal: successfully decode a collection of JPH files.
Prerequisites: Good C and Makefile coding skills, basic familiarity with Git, knowledge of JPEG 2000 and/or entropy coding is a plus.
Difficulty: Hard
Qualification Task: Add support for detecting Part 15 code blocks to the native JPEG 2000 decoder so that it can fail gracefully.
Mentor: Pierre-Anthony Lemieux (pal@palemieux.com)
Backup Mentor: Chris Hafey (cehafey@amazon.com)
Duration: 350 hours
GPU (CUDA) accelerate common software video filters
Description: FFmpeg already has a couple CUDA based video filters, but for a lot of commonly used functionality you have to fall back to software processing. A lot of existing software filters can be ported to CUDA more or less directly. Or completely new ones can be implemented.
Expected results: Port or create from scratch one or multiple CUDA based video filters. The number of filters entirely depends on the difficulty and complexity of the filters chosen.
Prerequisites: Access to the necessary Hardware. Basic understanding of git and Makefiles. Intermediate to Good C skills and at least a basic understanding of C++. Understanding of filtering algorithms and how shaders/CUDA kernels operate on GPUs.
Difficulty: Medium
Qualification Task: Port a software filter of your choice to CUDA. Does not need to be fully featured(no extensive pixel format support or options).
Mentor: Timo Rothenpieler (BtbN in #ffmpeg-devel on Libera.Chat IRC, timo@rothenpieler.org)
Backup Mentor: Philip Langdale(philipl in #ffmpeg-devel on Libera.Chat IRC, ffmpeg.philipl@overt.org)
Duration: 175
FATE Tests for Hardware(CUDA/NVDEC)-Accelerated filters/decoders
Description: As of right now, no tests exists for any of the CUDA/NVDEC accelerated filters and decoders. This is quite unfortunate, since they can end up with unintended changes in behavior just like anything else.
Expected results: Write FATE tests that cover one or several of the CUDA/NVDEC based filters and/or decoders, depending on complexity and the necessary initial boilerplate. The focus on which decoders/filters to cover is free to decide, since creating the initial boilerplate to run those tests will be a large part of the project.
Prerequisites: Access to the necessary Hardware. Basic understanding of git, good familiarity with Makefiles and C.
Difficulty: Medium
Qualification Task: Outline how testing Hardware-Accelerated filters could be tested and implement initial boilerplate and at least one simple test.
Mentor: Timo Rothenpieler (BtbN in #ffmpeg-devel on Libera.Chat IRC, timo@rothenpieler.org)
Backup Mentor: Philip Langdale(philipl in #ffmpeg-devel on Libera.Chat IRC, ffmpeg.philipl@overt.org)
Duration: 175
Checkasm
Description: Assembly functions in FFmpeg are not unit tested like they are in x264 or dav1d.
Expected results: Add tests for most asm functions which are not tested. Less if this is 175 hour duration.
Prerequisites: Good C coding skills, basic familiarity with Git. Some x86 assembly knowledge can be advantageous.
Difficulty: Easy
Qualification Task: Add tests for 2 separate asm functions, and pass review on the ffmpeg-devel mailing list. To pass qualification the code must be accepted into git master.
Mentor: J. Dekker (Illya in #ffmpeg-devel on IRC, jdek [at] itanimul [dot] li)
Backup Mentor: Thilo Borgmann (thilo.borgmann at mail dot de)
Duration: 175 or 350 hours
Unmentored Projects
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.
ffcall
Description: Simple tool which shows how to use libavcodec & libavformat for creating a 2 way video conference tool.
Expected results: You will write from scratch a tool which allows 2 or more people to communicate with video and audio across the network. The tool can reuse parts from the codebase like ffplay or other input/output related code. The tool also has to use strong end to end encryption.
Prerequisites: Good C coding skills, basic familiarity with Git. Experience with writing software communicating over the network.
Difficulty: Hard
Qualification Task: Write a one way audio or video (not both needed) communication tool with libavcodec & libavformat. The tool must pass code review for the qualification to pass.
Mentor: tbc
Backup Mentor: tbc
Duration: 350 hours
Template Project
Description: Template Description
Expected results: Template results
Prerequisites: Template prequisites
Difficulty:
Qualification Task: Template Qualification Task
Mentor: someone @ somehwere.net
Backup Mentor: someoneelse @ somewhere.net
Duration: 175 or 350 hours
Your Own Project Idea
A student can also propose a self-defined project! The amount of work defined by such a project should last for the majority of the GSoC work period duration and the task must be approved by the community. Also, a capable mentor from the community must be assigned. Ideas and inspiration for such project ideas could be found by browsing bugs and feature requests on our bug tracker but are of course not limited to such.
Students can discuss an idea with the community via the ffmpeg-devel mailing-list, the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins directly for more information. It is obviously of importance to start such a discussion well before the application deadline.
