Introduction

FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter and stream audio and video. It includes libavcodec - the leading audio/video codec library.

​Google Summer of Code (GSoC) is a program that offers students stipends to write code for open source projects. Through the guidance of mentors, students gain valuable experience interacting with and coding for open source projects like FFmpeg. Additionally, the project and its users benefit from code created from students who often continue contributing as developers. FFmpeg participated in several past editions: ​2006, ​2007, ​2008, ​2009, ​2010, and ​2011. We look forward to being involved this year.

This is our ideas page for ​Google Summer of Code 2014. See the ​GSoC Timeline for important dates.

FFmpeg has not been accepted this year, but feel free to do any of the qualification tasks and projects for fun.

---

Information for Students

Getting Started

1. Get to know FFmpeg. If you are a student and interested in contributing to an FFmpeg GSoC project it is recommended to start by subscribing to the ​ffmpeg-devel mailing-list, visiting our IRC channels (#ffmpeg-devel and #ffmpeg), and exploring the codebase and the development workflow. Feel free to contact us if you have any questions.

2. Find a project. Listed on this page are mentored and unmentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Unmentored projects are additional ideas that you may consider, but you will have to contact us to find a mentor. You may also propose your own project that may be a better match for your interest and skill level. If a project description is unclear or you have any questions, do not hesitate to contact its mentor or admin.

3. Contact us. If you find a project that you are interested in then get in touch with the community and let us know. In case you want to work on a qualification task, you should ask the respective mentor(s) so that the task can be claimed.

4. Apply. Student proposal period begins 10 March 19:00 UTC and ends 21 March 19:00 UTC. See the See the ​GSoC timeline for additional information.

Qualification Tasks

In order to get accepted you will be requested to complete a small task in the area you want to contribute. FFmpeg GSoC projects can be challenging, and a qualification task will show us that you are motivated and have the potential to successfully finish a project.

The qualification task is usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the ​FFmpeg Bug Tracker for qualification task ideas.

Contacting FFmpeg

If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:

• Mailing-list: ​ffmpeg-devel
• IRC: #ffmpeg-devel on ​Freenode
• FFmpeg GSoC Admin: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, ​michaelni@gmx.at)
• FFmpeg GSoC Backup Admins: Stefano Sabatini (saste in #ffmpeg-devel on Freenode IRC, stefasab AT gmail DOT com. ), Reynaldo Verdejo (reynaldo in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail)

You may also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.

---

Mentored Projects

This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.

Animated Portable Network Graphics (APNG)

Description: Add support for Animated PNGs. The little bouncing ball animation shown to the right is such an APNG file.

Specification: ​https://wiki.mozilla.org/APNG_Specification

Expected results:

• APNG demuxer
  • implement robust probing:
    • PNG images are not misdetected as APNG animations
    • APNG animations are not misdetected as PNG images
  • splits stream into sensible packets (so they can be easily reused in APNG muxer)
  • survives fuzzing (zzuf)
  • add FATE coverage, coverage should be at least 70%
  • test code under valgrind so no invalid reads/writes happen

• APNG decoder
  • use existing PNG decoder code (write decoder in same file)
  • implement parsing of all APNG chunks (acTL, fcTL, fdAT)
  • error handling
  • survives fuzzing (zzuf)
  • add test for FATE, coverage should be at least 75%
  • CRC checksum validation
  • test code under valgrind so no invalid reads/writes happen

• APNG muxer && APNG encoder
  • use existing PNG encoder code (write encoder in same file)
  • write compliant files, make sure they play correctly in major web browsers that support APNG
  • add test for FATE

Prerequisites: C coding skills, basic familiarity with git.

Qualification Task: Implement format autodetection for imagepipe and image demuxer.

Mentor: Paul B Mahol (durandal_1707 in #ffmpeg-devel on Freenode IRC)

Backup mentor: Peter Ross (pross-au in #ffmpeg-devel on Freenode IRC)

FFv1 P frame support

Description: FFv1 is one of the most efficient intra-only lossless video codecs. Your work will be to add support for P frames with motion compensation and motion estimation support (the existing motion estimation code in libavcodec can be reused here). Then fine-tune it until the best compression rate is achieved. This will make FFv1 competitive with existing I+P frame lossless codecs like lossless H.264.

Expected results: State of the art P frame support in the FFv1 encoder and decoder implementation.

Prerequisites: C coding skills, basic familiarity with git, solid understanding of video coding especially with motion compensation.

Qualification Task: Implement support for simple P frames without motion compensation in FFv1. That is so that each frame stores the difference to the previous frame.

Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, ​michaelni@gmx.at)

Backup mentor: Derek Buitenhuis (Daemon404 in #ffmpeg-devel on Freenode IRC, derek.buitenhuis ut gmail)

Subtitles

Description: FFmpeg has been working on improving its subtitles support recently, notably by adding the support for various text subtitles and various hardsubbing (burning the subtitles onto the video) facilities. While the theme may sound relatively simple compared to audio/video signal processing, the project carries an historical burden not easy to deal with, and introduces various issues very specific to its sparse form.

Expected results:

• Add support for new subtitles formats. Example: a demuxer for .SUP files, just like VobSub but for Blu-Ray, or a VobSub muxer.
• Improve text subtitles decoders. Typically, this can be supporting advanced markup features in SAMI or WebVTT.
• Update the API to get rid of the clumsy internal text representation of styles
• Proper integration of subtitles into libavfilter. This is the ultimate goal, as it will notably allow a complete subtitles rendering for applications such as ffplay.
• BONUS: if everything goes well, the student will be allowed to add basic support for teletext

Prerequisites: C coding skills, basic familiarity with git. Some background in fansubbing area (notably ASS experience) would be a bonus but is not strictly required.

Qualification Task: write one subtitles demuxer and decoder (for example support for Spruce subtitles format). This is in order to make sure the subtitles chain is understood.

Mentor: Clément Bœsch (ubitux in #ffmpeg-devel on Freenode IRC)

Backup Mentor: TBA, possibly Nicolas George (Cigaes in #ffmpeg-devel on Freenode IRC)

Postproc optimizations

Description: FFmpeg contains libpostproc, which is used to postprocess 8x8 DCT-MC based video and images (jpeg, mpeg-1/2/4, H.263 among others). Postprocessing removes blocking (and other) artifacts from low bitrate / low quality images and videos. The code though has been written a long time ago and its SIMD optimizations need to be updated to what modern CPUs support (AVX2 and SSE2+).

Expected results:

• Convert all gcc inline asm in libpostproc to YASM.
• Restructure the code so that it works with block sizes compatible with modern SIMD.
• Add Integer SSE2 and AVX2 optimizations for each existing MMX/MMX2/3dnow optimization in libpostproc.

Prerequisites: C coding skills, good x86 assembly coding skills, basic familiarity with git.

Qualification Task: convert 1 or 2 MMX2 functions to SSE2 and AVX2.

Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, ​michaelni@gmx.at)

Backup Mentor: TBA

Bayer RGB colorspaces

Description: Several image and video format store pixels using Bayer-pattern colorspaces. Supporting these format would broaden FFmpeg's applicability to RAW still and video photography processing.

Expected results:

• Rebase existing patches
• Implement high quality bayer transformations in libswscale (plain C)
• Add bayer formats to the libavutil pixfmt enumeration routines
• SIMD optimizations of the libswscale transformations
• Complete PhotoCINE demuxer to support Bayer format; or another format of your choosing (e.g. Magic Lantern Video)

Optional goodies:

• Extend TIFF decoder to support DNG-Bayer format
• Support a popular proprietary camera format (many to choose from; see dcraw project)

Prerequisites: C coding skills, basic familiarity with git.

Qualification Task: Implement a simple and working Bayer->RGB transform in libswscale

Mentor: Peter Ross (pross-au in #ffmpeg-devel on Freenode IRC)

Backup Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, ​michaelni@gmx.at)

MPEG-4 Audio Lossless Coding (ALS) encoder

Description: A MPEG-4 ALS decoder was implemented several years ago but an encoder is still missing in the official codebase. A rudimentary encoder has already been written and is available on ​github. For this project, that encoder is first to be updated to fit into the current codebase of FFmpeg and to be tested for conformance using the ​reference codec and specifications. Second, the encoder is to be brought through the usual reviewing process to hit the codebase at the end of the project.

Expected results:

• Update the existing encoder to fit into the current codebase.
• Ensure conformance of the encoder by verifying using the reference codec and generate a test case for ​FATE.
• Ensure the FFmpeg decoder processes all generated files without warnings.
• Enhance the rudimentary feature set of the encoder.

Prerequisites: C coding skills, basic familiarity with git. A certain interest in audio coding and/or knowledge about the FFmpeg codebase could be beneficial.

Qualification Task: Add floating point support to MPEG-4 ALS decoder

Mentor: Paul B Mahol (durandal_1707 in #ffmpeg-devel on Freenode IRC)

Backup Mentor: Stefano Sabatini (saste in #ffmpeg-devel on Freenode IRC, stefasab AT gmail DOT com)

Hardware Acceleration API Software / Tracing Implementation

Description: Our support for hardware accelerated decoding basically remains untested. This is in part due to FFmpeg only implementing part of the required steps, and in part since it requires specific operating systems and hardware.

The idea would be to start with a simple stub implementation of an API like e.g. VDPAU that provides only the most core functions. These would then serialize out the function calls and data to allow easy comparison and regression testing. Improvements to this approach are adding basic input validation and replay capability to allow testing regression data against real hardware. This would be similar to what ​apitrace does for OpenGL.

A further step would be to actually add support for decoding in software, so that full testing including visual inspection is possible without the need for special hardware.

Prerequisites: C coding skills, basic familiarity with git

Qualification Task: Anything related to the hardware acceleration code, though producing first ideas and code pieces for this task would also be reasonable

Mentor: Reimar Döffinger (reimar in #ffmpeg-devel on Freenode IRC, but since I'm rarely there better email me first: Reimar.Doeffinger [at] gmx.de)

Backup Mentor: TBA

Missing AAC decoder features

Description: FFmpeg contains an AAC decoder which is missing some features. The task is to rebase existing implementations to the current FFmpeg code-base, test them and fix the remaining bugs.

• AAC BSAC decoder: This has already been started, but the existing decoder still fails on many samples
• AAC SSR decoder
• AAC 960/120 MDCT window

Qualification Task: See the ​FFmpeg bug tracker for AAC issues, fixing one of them or rebasing the existing incomplete BSAC decoder for current git head and fixing one or more existing bugs are possible qualification tasks.

Prerequisites: C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.

Mentor: Baptiste Coudurier (bcoudurier in #ffmpeg-devel on Freenode IRC),

Backup Mentor: TBA

AAC Encoder Improvements

Description: Currently, FFmpeg contains an experimental native AAC encoder (native meaning that it doesn't depend on external libraries), but it is lacking in terms of extension support, since it currently only implements AAC-LC (the low-complexity profile). Extending feature support of FFmpeg's native encoder is needed to make it competitive with other encoders.

• AAC native encoder missing extensions
  • Implement ​AAC PNS
  • Implement ​intensity stereo
  • Implement AAC LTP (long term prediction)
  • Implement ​AAC SBR (spectral band replication)

Prerequisites: C coding skills, basic familiarity with git, knowledge about transform based audio coding would be useful.

Mentor: Claudio Freire (klaussfreire in g mail)

Backup Mentor: TBA

DTS / DCA Improvements

Description: FFmpeg contains a DTS decoder, but its missing several features

• DTS-HD decoder improvements: A possible qualification task is to implement ticket ​#1920
  • Add support for X96 extension (96khz)
  • Add support for pure DTS-HD streams that do not contain a DTS core
  • Add support for multiple assets
  • Add support for LBR extension

Prerequisites: C coding skills, basic familiarity with git. Good understanding of DTS and related audio coding is a strict requirement.

Mentor: Benjamin Larsson (merbanan/merbzt in #ffmpeg-devel on Freenode IRC)

Backup Mentor: TBA

MXF Demuxer Improvements

Description: The MXF demuxer needs a proper, compact way to map EssenceContainer ULs to WrappingKind. See ​ticket #2776 in our bug tracker, and ​ticket #1916 contains additional relevant information.

Essence in MXF is typically stored in one of two ways: as an audio/video interleave or with each stream in one huge chunk (such as 1 GiB audio followed by 10 GiB video). Previous ways of telling these apart have been technically wrong, but has worked due to a lack of samples demonstrating the contrary.

Expected results: The sample in ticket ​ticket #2776 should demux correctly. Add a test case in ​FATE. The solution should grow libavformat by no more than 32 KiB.

Prerequisites: C coding skills, basic familiarity with git.

Qualification Task: Investigate if there may be a compact way of representing the UL -> WrappingKind mapping specified in the ​official RP224 Excel document. The tables takes up about half a megabyte verbatim which is unacceptable in a library as large as libavformat.

Mentor: Tomas Härdin (thardin in #ffmpeg-devel on Freenode IRC)

Backup Mentor: TBA

Symmetric-key block ciphers

Description: FFmpeg contains the libavutil library, which is a utility library to aid portable multimedia programming. It contains, among other things, standard cryptographic algorithms, like AES or Blowfish. The goal of this project is to add support for three other common block ciphers: CAST-128, Twofish and Camellia.

Specification:

• ​http://tools.ietf.org/search/rfc2144
• ​https://www.schneier.com/twofish.html
• ​http://tools.ietf.org/html/rfc3713

Expected Results:

• Implement CAST-128, Twofish and Camellia in C
• Support for ECB and CBC modes
• Ensure code works with all key sizes supported by an algorithm
• Ensure code is conformant to the standard and passes test vectors
• Ensure code does not attempt any invalid read or write using valgrind
• Simple C-level optimization to get comparable speed with other crypto libraries

Prerequisites: C coding skills, basic familiarity with git. Basic understanding of cryptography. Understanding of block ciphers. C-level code optimization.

Qualification Task: Anything related to cryptography, like proving you can implement a cryptographic hash function or a HMAC function in C. The student can also propose his own qualification task.

Mentor: Giorgio Vazzana (Holden in #ffmpeg-devel on Freenode IRC)

Backup Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, ​michaelni@gmx.at)

VDPAU filter

Description: VDPAU is not only about hardware-accelerated decoding but also allows some postprocessing – most notably deinterlacing. Other features of the API include scaling, noise removal, and a sharpening filter.

Expected results: Implement a filter that allows usage of the postprocessing features as defined by the VDPAU API. This should particularly include the deinterlacer, and ideally all features should be usable to allow comparing the quality and performance of different hardware and hardware vs. software.

Prerequisites: C coding skills, and you will need hardware that allows VDPAU post-processing.

Qualification Task: Show that you are familiar with the FFmpeg filter system, either by porting one or more of the remaining postprocessing filters from the MPlayer compatibility layer (files vf_*pp*.c in libavfilter/libmpcodecs, see for comparison commit ​a2c547ff) to a native filter, or by implementing a very simple inverse telecine filter under LGPL that allows reversal of the effect of the telecine filter.

Mentor: Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Freenode IRC, ce AT hoyos.ws)

Backup Mentor: Reimar Döffinger (reimar in #ffmpeg-devel on Freenode IRC, Reimar.Doeffinger [at] gmx.de)

Basic servers for network protocols

Description: libavformat contains clients for various network protocols used in multimedia streaming: HTTP, RTMP, MMS, RTSP. Your work will be to implement the server side for one or several of these protocols.

The libavformat framework is not designed to build general-purpose server applications with several clients, and nothing similar to the configuration features of real servers like Apache is expected, but libavformat should be able to stream a single predefined bytestream to/from a single client.

Note: server support is already implemented for the receiving side of RTSP.

Expected results: basic servers for network protocols capable of interoperating with third-party clients.

Prerequisites: C coding skills, basic familiarity with git, network programming.

Qualification Task: proof-of-concept server for one of the protocol, capable of interacting with a particular client in controlled circumstances; or anything network-related, e.g. fixing a ticket in our ​bug tracker.

Mentor: Nicolas George (Cigaes in #ffmpeg-devel on Freenode IRC)

Backup mentor: Reynaldo Verdejo (reynaldo in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail)

---

Unmentored Projects

This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.

Misc Libavfilter extension

Description: Libavfilter is the FFmpeg filtering library. It currently supports audio and video filtering and generation support. This work may focus on porting, fixing, extending, or writing new audio and video filters from scratch.

Candidate filters for porting may be the remaining MPlayer filters currently supported through the mp wrapper, libaf MPlayer filters, and filters from other frameworks (e.g. mjpegtools, transcode, avisynth, virtualdub, etc.). In case of mp ports, the student should verify that the new filter produces the same output and is not slower.

Some ideas for more filters:

• a frequency filtering domain filter relying on the FFT utils in libavcodec
• a controller filter which allows to send commands to other filters (e.g. to adjust volume, contrast, etc.), e.g. like the sendcmd filter but through an interactive GUI
• a lua scripting filter, which allows to implement filtering custom logic in lua

For more ideas check ​trac libavfilter tickets.

Expected results: Write or port audio and video filters and possibly fix/extend libavfilter API and design when required.

Prerequisites: C coding skills, basic familiarity with git. Some background on DSP and image/sound processing techniques would be a bonus but is not strictly required.

Qualification Task: write or port one or more filters

Mentor: TBA

Backup mentor: Clément Bœsch (ubitux in #ffmpeg-devel on Freenode IRC)

TrueHD encoder

Description: FFmpeg currently does not support encoding to TrueHD, one of the lossless audio formats used on Bluray discs. This task consists of implementing a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding.

Expected results: a TrueHD encoder that allows to losslessly encode audio to play it on hardware devices capable of TrueHD decoding with a competetive compression rate

Prerequisites: C coding skills, basic familiarity with git

Qualification Task: write a encoder that produces a valid bitstream that can be decoded without errors to silence.

Opus decoder

Description: Opus decoding is currently supported only through the external libopus library

Expected results: a native opus decoder in ffmpeg

Prerequisites: C coding skills, basic familiarity with git, some knowledge about opus would be benefical

Qualification Task: write part of the decoder, like write the bitstream decoding or header decoding and show that it is working.

VC-1 interlaced

Description: The FFmpeg VC-1 decoder has improved over the years, but many samples are still not decoded bit-exact and real-world interlaced streams typically show artefacts. This task is difficult.

Expected results:

• Implement missing parts of interlacing
• Make more reference samples bit-exact

Prerequisites: C coding skills, basic familiarity with git

Qualification Task: Find a bug in the current decoder implementation and fix it.

JPEG 2000

Description: FFmpeg contains an experimental native JPEG 2000 encoder and decoder. Both are missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult

Expected results: A jpeg 2000 decoder which can decode most lossy samples without artifacts and lossless samples exactly.

Qualification Task: Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.

Prerequisites: C coding skills, basic familiarity with git, good knowledge of Jpeg-2000

Hardware Accelerated Video Encoding with VA-API

Description: FFmpeg already supports hardware accelerated decoding for multiple codecs but still lacks support for hardware accelerated encoding. The aim of the project is to add support for encoding with VA-API specifically, while keeping a generic enough approach in mind so that other hardware accelerators (TI-DSP, CUDA?) could be supported as well. This means that new hwaccel hooks are needed and two operational modes are possible: either (i) driver or hardware pack headers themselves, or (ii) lattitude is left to perform this task at the FFmpeg library level.

Expected results: Allow MPEG-2 and H.264 encoding with VA-API, while supporting variable bitrate (VBR) by default, and allowing alternate methods like constant bitrate (CBR) or constant QP (CQP) where appropriate or requested.

• MPEG-2 encoding:
  • Add basic encoding with I/P frames (handle the -g option)
  • Add support for B frames (handle the -bf option)
  • Add support for constant bitrate (CBR, i.e. maxrate == bitrate and bufsize set)
  • (Optionally) add support for interlaced contents
• H.264 encoding:
  • Add basic encoding with I/P frames (handle the -g option)
  • Add support for B frames (handle the -bf option)
  • Add support for constant bitrate (CBR, i.e. maxrate == bitrate and bufsize set)
  • Add support for constant QP (CQP, i.e. handle the -cqp option)
  • Add support for more than one reference frame, while providing/using API to query the hardware capabilities
  • Work on HRD conformance. May require to write an independent tool to assess that
  • (Optionally) add configurability of the motion estimatation method to use. Define new types for HW accelerated encoding with at least two levels/hints for the accelerator.
• FFmpeg applications:
  • Define common hwaccel interface for encoding
  • Add initial support for hardware accelerated encoding to the ffmpeg application

Prerequisites: C coding skills, basic familiarity with git, hardware supporting VA-API for encoding.

Qualification Task: Anything related to the Hardware Acceleration (hwaccel) API, or to its related users. e.g. port VDPAU acceleration to use hwaccel, add JPEG decoding support with VA-API, etc.

Mentor: TBA

Backup Mentor: TBA, possibly Tushar Gohad

H.264 Multiview Video Coding (MVC)

Description: MVC samples exist and the codec is used on Blu-ray media, but FFmpeg is missing a decoder. Since this project also consists of some changes in the current architecture, it is especially important that this project is discussed on the ffmpeg-devel mailing list.

Expected results: Create MVC decoder and add a test for ​FATE.

Prerequisites: C coding skills, familiarity with git/source code control systems.

Qualification Task: Perform work that demonstrates understanding of MVC and that is a subpart of the whole MVC implementation.

Mentor: TBA

Backup mentor: TBA

Browsing content on the server

Description: libavformat supports network protocols like FTP, SFTP, HTTP.

The libavformat framework is able to read, seek and write to files located on server as they were local files. The libavformat is not able to list files nor directories located on remote servers. Your work will be to implement browsing files on remote servers.

Note: the libavformat will be prepared with suitable API and only protocol related code will need to be implemented.

Expected results: Application that uses libavformat will be provided with an API to list files and directories (with their attributes like size, creation date, etc) located on remote servers. Implementation for FTP, HTTP and SFTP is required. Optionally file protocol can be also implemented.

Prerequisites: C coding skills, basic familiarity with git, basic knowledge about FTP and HTTP protocols and libssh library.

Qualification Task: Implement browsing routine for one of mentioned protocols (implementation may cover file/directory names only, work with specific servers and under defined conditions).

Mentor: TBA

Backup mentor: TBA, possibly Lukasz Marek (lukaszmluki in #ffmpeg-devel on Freenode IRC, lukasz.m.luki on g mail)

Your Own Project Idea

A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our bug tracker. The work should last the majority of the GSoC duration, the task must be approved by the developers, and a mentor must be assigned.

Students can discuss an idea in the ffmpeg-devel mailing-list, the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.