Testing Coverage of FATE Test Suite

One can easily discover untested components of FFmpeg using coverage testing. A online coverage report is available at ​http://coverage.ffmpeg.org/, but you can generate one yourself with ​gcov and either ​lcov or ​gcovr.

Prerequisites

You have to use gcov for generating coverage information. If you want a good-looking HTML layout of the coverage information, you will need either lcov or gcovr (3.0 or later).

On Debian-derived systems, you can just do:

sudo apt-get install gcov lcov

If you want to use gcovr instead, you can do:

sudo apt-get install gcov python-setuptools
sudo easy_install gcovr

Warning: gcovr seems to have a bug that hangs when generating output for a large project like FFmpeg. See ​https://software.sandia.gov/trac/fast/ticket/3935 and ​https://github.com/gcovr/gcovr/issues/11. Although it says "fixed", the issue is still reproducible with gcovr 3.1.

Note: The gcovr package in Ubuntu Saucy or earlier is too old to generate HTML output.

./configure

First, configure the build with the special gcov toolchain:

./configure --toolchain=gcov --rest-of-options
# For example, I use:
# ./configure --toolchain=gcov --enable-gpl --enable-avresample --samples=`pwd`/fate-suite

Using lcov

If you are using lcov, you have to first reset its counter:

$!sh
make lcov-reset

Then, run the FATE suite:

# The -j parameter controls thread number for parallel building
make -j6 fate

Finally, generate the HTML output:

make lcov

Using gcovr

Run the FATE suite:

# The -j parameter controls thread number for parallel building
make -j6 fate

Then generate output:

gcovr -b -r $SOURCEDIR -e '/usr/.*' --html --object-directory $OBJDIR