Transcoding results in dropped frames and lowering fps

[Barvinok]

Summary of the bug:

The input video file is recorded by SonyEricsson Xperia X2 camera, has an extension MP4 and contain "aac mp4a" and "mpeg4 (Simple Profile)" according to ffprobe (see below).

I wanted to convert it using libx264/libvorbis as encoders and matroska as container. The command I used was:

/opt/ffmpeg/bin/ffmpeg -i 00-00-00-00000001.mp4 \
-vcodec libx264 -crf 22 \
-acodec libvorbis -ab 64k test.mkv

The actual invocation is a bit longer, but this one is for reproducing the effect. At first glance, ffmpeg seemed to process it all right:

ffmpeg version N-43733-ga93c221 Copyright (c) 2000-2012 the FFmpeg developers
  built on Aug 19 2012 21:27:54 with gcc 4.5.3 (Gentoo 4.5.3-r2 p1.1, pie-0.4.7)
  configuration: --prefix=/opt/ffmpeg --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-march=native -mtune=native -O2 -pipe -mmmx -msse -msse2 -msse4a -m3dnow -fomit-frame-pointer -funroll-loops' --extra-cflags='-march=native -mtune=native -O2 -pipe -mmmx -msse -msse2 -msse4a -m3dnow -fomit-frame-pointer -funroll-loops' --extra-cxxflags='-march=native -mtune=native -O2 -pipe -mmmx -msse -msse2 -msse4a -m3dnow -fomit-frame-pointer -funroll-loops' --disable-static --enable-gpl --enable-version3 --enable-postproc --enable-avfilter --disable-stripping --disable-debug --disable-doc --disable-network --disable-vaapi --disable-vdpau --disable-ffplay --enable-gnutls --enable-libmp3lame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --disable-indev=v4l --disable-indev=v4l2 --disable-indev=alsa --disable-indev=oss
  libavutil      51. 70.100 / 51. 70.100
  libavcodec     54. 53.100 / 54. 53.100
  libavformat    54. 25.104 / 54. 25.104
  libavdevice    54.  2.100 / 54.  2.100
  libavfilter     3. 11.101 /  3. 11.101
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '00-00-00-00000001.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: mp41isom
    creation_time   : 1980-01-17 22:55:01
  Duration: 00:00:59.54, start: 0.000000, bitrate: 1168 kb/s
    Stream #0:0(eng): Audio: aac (mp4a / 0x6134706D), 16000 Hz, mono, s16, 23 kb/s
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : AudioHandler
    Stream #0:1(eng): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 1142 kb/s, 26.79 fps, 15 tbr, 1k tbn, 15 tbc
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : VideoHandler
[libx264 @ 0x1010bc0] using SAR=1/1
[libx264 @ 0x1010bc0] using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisalign LZCNT
[libx264 @ 0x1010bc0] profile High, level 2.2
[libx264 @ 0x1010bc0] 264 - core 120 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'test.mkv':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: mp41isom
    encoder         : Lavf54.25.104
    Stream #0:0(eng): Video: h264, yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 1k tbn, 15 tbc
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: vorbis, 16000 Hz, mono, flt, 64 kb/s
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : AudioHandler
Stream mapping:
  Stream #0:1 -> #0:0 (mpeg4 -> libx264)
  Stream #0:0 -> #0:1 (aac -> libvorbis)
Press [q] to stop, [?] for help
Que input is backward in timeze=       5kB time=00:00:00.00 bitrate=   0.0kbits/s dup=0 drop=28
frame=  892 fps=9.8 q=26.0 Lsize=    7511kB time=00:00:57.91 bitrate=1062.5kbits/s dup=0 drop=703
video:7258kB audio:229kB subtitle:0 global headers:4kB muxing overhead 0.265491%
[libx264 @ 0x1010bc0] frame I:4     Avg QP:18.68  size: 21682
[libx264 @ 0x1010bc0] frame P:817   Avg QP:21.72  size:  8581
[libx264 @ 0x1010bc0] frame B:71    Avg QP:24.38  size:  4714
[libx264 @ 0x1010bc0] consecutive B-frames: 85.0% 12.3%  2.7%  0.0%
[libx264 @ 0x1010bc0] mb I  I16..4: 14.3% 77.7%  8.0%
[libx264 @ 0x1010bc0] mb P  I16..4:  2.5%  8.0%  0.5%  P16..4: 55.0% 20.5%  9.5%  0.0%  0.0%    skip: 4.1%
[libx264 @ 0x1010bc0] mb B  I16..4:  0.7%  2.5%  0.1%  B16..8: 42.9%  9.4%  2.0%  direct:15.4%  skip:26.9%  L0:48.3% L1:31.7% BI:20.0%
[libx264 @ 0x1010bc0] 8x8 transform intra:73.4% inter:75.0%
[libx264 @ 0x1010bc0] coded y,uvDC,uvAC intra: 61.8% 90.1% 45.1% inter: 42.7% 68.5% 9.9%
[libx264 @ 0x1010bc0] i16 v,h,dc,p: 26% 33% 15% 25%
[libx264 @ 0x1010bc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 26% 29%  3%  4%  4%  6%  3%  5%
[libx264 @ 0x1010bc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 29% 15%  3%  7%  6%  7%  4%  4%
[libx264 @ 0x1010bc0] i8c dc,h,v,p: 46% 28% 20%  6%
[libx264 @ 0x1010bc0] Weighted P-Frames: Y:6.2% UV:0.9%
[libx264 @ 0x1010bc0] ref P L0: 73.7% 18.0%  6.1%  2.0%  0.1%
[libx264 @ 0x1010bc0] ref B L0: 91.5%  8.2%  0.3%
[libx264 @ 0x1010bc0] ref B L1: 98.6%  1.4%
[libx264 @ 0x1010bc0] kb/s:996.45

What worried me is the discrepancy between reported fps and tbr in the original stream, also of note was the large quantity of the dropped frames - nearly half of them.

And indeed, while the original material had FPS of roughly 26, the converted file displayed FPS of 15, hence the dropped frames, though I did not specify -r option. And the difference in FPS is clearly visible by naked eye.

Expected: I expected ffmpeg to preserve the FPS of the original material in the absence of -r option.

[Barvinok]

The original file in question (8.3 MB) is available from
​http://www.barvinok.net/xfer/00-00-00-00000001.mp4
and also mirrored (different server) on
​http://dichron.barvinok.net/tmp/00-00-00-00000001.mp4

[Carl Eugen Hoyos]

$ ffmpeg -i 00-00-00-00000001.mp4 -qscale 2 out.avi
ffmpeg version N-43737-g76c3fff Copyright (c) 2000-2012 the FFmpeg developers
  built on Aug 20 2012 11:13:22 with gcc 4.6.1 (GCC)
  configuration: --cc=/usr/local/gcc-4.6.1/bin/gcc --enable-libspeex
  libavutil      51. 70.100 / 51. 70.100
  libavcodec     54. 53.100 / 54. 53.100
  libavformat    54. 25.104 / 54. 25.104
  libavdevice    54.  2.100 / 54.  2.100
  libavfilter     3. 11.101 /  3. 11.101
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '00-00-00-00000001.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: mp41isom
    creation_time   : 1980-01-17 22:55:01
  Duration: 00:00:59.54, start: 0.000000, bitrate: 1168 kb/s
    Stream #0:0(eng): Audio: aac (mp4a / 0x6134706D), 16000 Hz, mono, s16, 23 kb/s
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : AudioHandler
    Stream #0:1(eng): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 1142 kb/s, 26.79 fps, 15 tbr, 1k tbn, 15 tbc
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : VideoHandler
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out.avi':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: mp41isom
    ISFT            : Lavf54.25.104
    Stream #0:0(eng): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 15 tbn, 15 tbc
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: ac3 ([0] [0][0] / 0x2000), 16000 Hz, mono, flt, 96 kb/s
    Metadata:
      creation_time   : 1980-01-17 22:55:01
      handler_name    : AudioHandler
Stream mapping:
  Stream #0:1 -> #0:0 (mpeg4 -> mpeg4)
  Stream #0:0 -> #0:1 (aac -> ac3)
Press [q] to stop, [?] for help
frame=  892 fps=249 q=2.0 Lsize=   14929kB time=00:00:57.88 bitrate=2112.6kbits/s dup=0 drop=703
video:14206kB audio:677kB subtitle:0 global headers:0kB muxing overhead 0.304437%