ffmpeg segfault on solaris due to null passed to vsnprintf

[Alasdair Lumsden]

Hi,

When I accidently ran "ffmpeg /tmp/engineering.mpg -ab 128 -b 400 /tmp/al.mpg" by mistake, (missed out the -i) I got a segfault:

root ~ (ipstest01.alasdair): /ec/bin/ffmpeg /tmp/engineering.mpg -ab 128 -b 400 /tmp/al.mpg
ffmpeg version 0.8, Copyright (c) 2000-2011 the FFmpeg developers
  built on Jul 18 2011 14:36:32 with gcc 4.4.5
  configuration: --prefix=/ec --mandir=/ec/share/man --cc=/ec/bin/gcc --extra-cflags=-I/ec/include --disable-static --enable-shared --enable-pic --enable-nonfree --enable-version3 --enable-gpl --disable-ffplay --disable-ffserver --disable-mmx --enable-libfaac --enable-libmp3lame --enable-libx264 --enable-libopencore-amrnb --enable-librtmp --disable-libdirac --disable-libopenjpeg --disable-libschroedinger --disable-libxvid --disable-libgsm --disable-libspeex --disable-libvorbis --disable-libtheora --disable-avdevice --bindir=/ec/bin --libdir=/ec/lib --shlibdir=/ec/lib --cpu=i686 --extra-ldflags='-L/ec/lib -R/ec/lib'
  libavutil    51.  9. 1 / 51.  9. 1
  libavcodec   53.  7. 0 / 53.  7. 0
  libavformat  53.  4. 0 / 53.  4. 0
  libavfilter   2. 23. 0 /  2. 23. 0
  libswscale    2.  0. 0 /  2.  0. 0
  libpostproc  51.  2. 0 / 51.  2. 0
Segmentation Fault (core dumped)

This is on Solaris 10. On Linux, the same command line arguments don't produce a segfault:

alasdair ~ (linux01): ffmpeg/bin/ffmpeg /tmp/engineering.mpg -ab 128 -b 400 /tmp/al.mpg
ffmpeg version 0.8, Copyright (c) 2000-2011 the FFmpeg developers
  built on Aug  3 2011 18:31:10 with gcc 4.1.2 20080704 (Red Hat 4.1.2-48)
  configuration: --prefix=/home/alasdair/ffmpeg --extra-cflags='-I/home/alasdair/lame/include -I/home/alasdair/x264/include' --extra-ldflags='-L/home/alasdair/lame/lib -L/home/alasdair/x264/lib -R/home/alasdair/x264/lib' --enable-static --enable-shared --enable-pic --enable-nonfree --enable-version3 --enable-gpl --disable-ffplay --disable-ffserver --disable-mmx --enable-libx264 --disable-libdirac --disable-libopenjpeg --disable-libschroedinger --disable-libxvid --disable-libgsm --disable-libspeex --disable-libvorbis --disable-libtheora --disable-avdevice --enable-libmp3lame
  libavutil    51.  9. 1 / 51.  9. 1
  libavcodec   53.  7. 0 / 53.  7. 0
  libavformat  53.  4. 0 / 53.  4. 0
  libavfilter   2. 23. 0 /  2. 23. 0
  libswscale    2.  0. 0 /  2.  0. 0
  libpostproc  51.  2. 0 / 51.  2. 0
Incompatible sample format '(null)' for codec 'mp2', auto-selecting format 's16'
File '/tmp/engineering.mpg' already exists. Overwrite ? [y/N] n
Not overwriting - exiting

However I spotted the "(null)" in the print statement, and pstack against the core showed:

# pstack core
core 'core' of 3366:    /ec/bin/ffmpeg /tmp/engineering.mpg -ab 128 -b 400 /tmp/al.mpg
 feea5acc strlen   (806483a, 8047a4c, 80475a0, 0) + c
 fef03827 vsnprintf (80475e0, 400, 806481c, 8047a4c) + 73
 fee5a9fd av_log_default_callback () + 81

I rebuilt ffmpeg with debugging symbols, and running this through gdb shows:

# gdb /ec/bin/ffmpeg
GNU gdb (GDB) 7.2
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-pc-solaris2.10".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /ec/bin/ffmpeg...done.
(gdb) run /tmp/engineering.mpg -ab 128 -b 400 /tmp/al.mpg
Starting program: /ec/bin/ffmpeg /tmp/engineering.mpg -ab 128 -b 400 /tmp/al.mpg
[Thread debugging using libthread_db enabled]
[New Thread 1 (LWP 1)]
ffmpeg version 0.8, Copyright (c) 2000-2011 the FFmpeg developers
  built on Aug  4 2011 10:51:17 with gcc 4.4.5
  configuration: --prefix=/ec --mandir=/ec/share/man --cc=/ec/bin/gcc --extra-cflags=-I/ec/include --enable-static --enable-shared --enable-pic --enable-nonfree --enable-version3 --enable-gpl --disable-ffplay --disable-ffserver --disable-mmx --enable-libfaac --enable-libmp3lame --enable-libx264 --enable-libopencore-amrnb --enable-librtmp --disable-libdirac --disable-libopenjpeg --disable-libschroedinger --disable-libxvid --disable-libgsm --disable-libspeex --disable-libvorbis --disable-libtheora --disable-avdevice --disable-stripping --bindir=/ec/bin --libdir=/ec/lib --shlibdir=/ec/lib --extra-ldflags='-L/ec/lib -R/ec/lib'
  libavutil    51.  9. 1 / 51.  9. 1
  libavcodec   53.  7. 0 / 53.  7. 0
  libavformat  53.  4. 0 / 53.  4. 0
  libavfilter   2. 23. 0 /  2. 23. 0
  libswscale    2.  0. 0 /  2.  0. 0
  libpostproc  51.  2. 0 / 51.  2. 0

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0xfeea5acc in strlen () from /lib/libc.so.1
(gdb) bt
#0  0xfeea5acc in strlen () from /lib/libc.so.1
#1  0xfef006f2 in _ndoprnt () from /lib/libc.so.1
#2  0xfef03827 in vsnprintf () from /lib/libc.so.1
#3  0xfee5a9fd in av_log_default_callback (ptr=0x0, level=24, fmt=0x806481c "Incompatible sample format '%s' for codec '%s', auto-selecting format '%s'\n", vl=0x8047a8c "")
    at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/libavutil/log.c:112
#4  0xfee5a8be in av_vlog (avcl=0x0, level=24, fmt=0x806481c "Incompatible sample format '%s' for codec '%s', auto-selecting format '%s'\n", vl=0x8047a8c "")
    at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/libavutil/log.c:150
#5  0xfee5a96c in av_log (avcl=0x0, level=115, fmt=0x806481c "Incompatible sample format '%s' for codec '%s', auto-selecting format '%s'\n")
    at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/libavutil/log.c:144
#6  0x08054a87 in choose_sample_fmt (st=0x8084840, codec=<value optimized out>) at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/ffmpeg.c:623
#7  0x0805ed34 in new_audio_stream (oc=0x807f540, file_idx=<value optimized out>) at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/ffmpeg.c:3729
#8  0x0805fa66 in opt_output_file (opt=0x0, filename=0x0) at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/ffmpeg.c:3940
#9  0x0806285a in parse_options (argc=7, argv=0x8047d14, options=0x8077760, parse_arg_function=0x805f1dc <opt_output_file>)
    at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/cmdutils.c:292
#10 0x0805e0e5 in main (argc=7, argv=0x8047d14) at /export/home/s10-userland/s10-userland/components/ffmpeg/ffmpeg-0.8/ffmpeg.c:4556

From what I can gather, vsnprintf has been enhanced on Linux to print "(null)" for null values. On platforms without this enhancement, such as Solaris, instead you get a nice segfault.

I imagine this is quite an easy fix.

Cheers,

Alasdair

[Michael Niedermayer]

Should be fixed in git master