A/V desync with mpeg1video and mp2 in mpeg-ps

[burek]

I'll try to repost this thing here, because a guy, that reported this thing, cannot register properly on this trac, so I need to proxy the whole thing.. The original text can be found here: ​http://ffmpeg.gusari.org/viewtopic.php?f=11&t=599&p=647

--- CUT ---

/usr/local/bin/ffmpeg -y -i /home/tmp/movies/8DMValgTc9HXlmbZcLhvZMr9azogtoek.mpg -vcodec libx264 -preset:v medium -s 640x480 -b:v 400k -acodec libfaac -ab 48k -ac 2 /home/tmp/content/movies/7767_tmp.mp4


ffmpeg version N-40558-g2a793ff Copyright (c) 2000-2012 the FFmpeg developers
built on May 10 2012 21:32:49 with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
configuration: --enable-libfaac --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-gpl --enable-nonfree
libavutil 51. 50.100 / 51. 50.100
libavcodec 54. 21.101 / 54. 21.101
libavformat 54. 4.100 / 54. 4.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 72.105 / 2. 72.105
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 11.100 / 0. 11.100
libpostproc 52. 0.100 / 52. 0.100
[mpeg @ 0x944b3c0] max_analyze_duration 5000000 reached at 5015467
Input #0, mpeg, from '/home/tmp/movies/PlYt2l3NCHFzjyNoX16GolA9jmd5vqxS.mpg':
Duration: 26:47:27.73, start: 0.622000, bitrate: 16 kb/s
Stream #0:0[0x1e0]: Video: mpeg1video, yuv420p, 640x480 [SAR 1:1 DAR 4:3], 1452 kb/s, 25 fps, 25 tbr, 90k tbn, 25 tbc
Stream #0:1[0x1c0]: Audio: mp2, 44100 Hz, stereo, s16, 128 kb/s
[buffer @ 0x9453320] w:640 h:480 pixfmt:yuv420p tb:1/1000000 sar:1/1 sws_param:flags=2
[buffersink @ 0x9449340] No opaque field provided
[scale @ 0x94495a0] w:640 h:480 fmt:yuv420p sar:1/1 -> w:640 h:480 fmt:yuv420p sar:1/1 flags:0x4
[libx264 @ 0x9451fa0] using SAR=1/1
[libx264 @ 0x9451fa0] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
[libx264 @ 0x9451fa0] profile High, level 3.0
[libx264 @ 0x9451fa0] 264 - core 124 r2197 69a0443 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:
0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2
threads=12 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 wei
ghtp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=400 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 a
q=1:1.00
[libfaac @ 0x9448aa0] channel_layout not specified
Output #0, mp4, to '/home/tmp/content/movies/7762_tmp.mp4':
Metadata:
encoder : Lavf54.4.100
Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 400 kb/s, 25 tbn, 25 tbc
Stream #0:1: Audio: aac (@[0][0][0] / 0x0040), 44100 Hz, stereo, s16, 48 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg1video -> libx264)
Stream #0:1 -> #0:1 (mp2 -> libfaac)
Press [q] to stop, [?] for help
[libfaac @ 0x9448aa0] Que input is backward in time
Last message repeated 4 times
[mp4 @ 0x9451ac0] Audio timestamp 2147483392 < 2147483521 invalid, cliping
[libfaac @ 0x9448aa0] Que input is backward in time
[mp4 @ 0x9451ac0] Audio timestamp 2147483264 < 2147483522 invalid, cliping
[libfaac @ 0x9448aa0] Que input is backward in time
[mp4 @ 0x9451ac0] Audio timestamp 2147483008 < 2147483524 invalid, cliping

I get the above outputted by ffmpeg when encoding from a movie file to x264 mp4 aac movie file.

Sometimes it hangs and the output file keeps growing and growing up to 500Mb - 700Mb, while it should be just an end file of arround 40Mb-60Mb

And here it ends it hangs and the 7762_tmp.mp4 file is growing and growing till i kill the process.

And some times it does finish encoding but keeps outputting lines like below, but the file is useless, its some wrongly encoded file and does not work.
[mp4 @ 0x9451ac0] Audio timestamp 2147483392 < 2147483521 invalid, cliping

Any ideas?

--- END ---

[Carl Eugen Hoyos]

Is this only reproducible if audio and video are encoded or also with -vn?
Is an external library (libfaac) necessary to reproduce the problem or is it also reproducible with -acodec aac? Is -ab 48k necessary to reproduce?

Is there a sample file?

[burek]

I'll ask him to try with -vn and paste the results here. Same for the sample file.

For the second question:

/usr/local/bin/ffmpeg -y -i /home/tmp/movies/8DMValgTc9HXlmbZcLhvZMr9azogtoek.mpg -vcodec libx264 -preset:v medium -s 640x480 -b:v 400k -acodec aac -strict experimental -ab 48k -ac 2 -ar 44100 /home/tmp/content/movies/7767_tmp.mp4

ffmpeg version N-40558-g2a793ff Copyright (c) 2000-2012 the FFmpeg developers
built on May 10 2012 21:32:49 with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
configuration: --enable-libfaac --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-gpl --enable-nonfree
libavutil 51. 50.100 / 51. 50.100
libavcodec 54. 21.101 / 54. 21.101
libavformat 54. 4.100 / 54. 4.100
libavdevice 53. 4.100 / 53. 4.100
libavfilter 2. 72.105 / 2. 72.105
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 11.100 / 0. 11.100
libpostproc 52. 0.100 / 52. 0.100
[mpeg @ 0x9a4e3c0] max_analyze_duration 5000000 reached at 5015467
Input #0, mpeg, from '/home/tmp/movies/PlYt2l3NCHFzjyNoX16GolA9jmd5vqxS.mpg':
Duration: 26:47:27.73, start: 0.622000, bitrate: 16 kb/s
Stream #0:0[0x1e0]: Video: mpeg1video, yuv420p, 640x480 [SAR 1:1 DAR 4:3], 1452 kb/s, 25 fps, 25 tbr, 90k tbn, 25 tbc
Stream #0:1[0x1c0]: Audio: mp2, 44100 Hz, stereo, s16, 128 kb/s
[buffer @ 0x9a56540] w:640 h:480 pixfmt:yuv420p tb:1/1000000 sar:1/1 sws_param:flags=2
[buffersink @ 0x9a4c320] No opaque field provided
[scale @ 0x9a4c580] w:640 h:480 fmt:yuv420p sar:1/1 -> w:640 h:480 fmt:yuv420p sar:1/1 flags:0x4
Incompatible sample format 's16' for codec 'aac', auto-selecting format 'flt'
[libx264 @ 0x9a54fa0] using SAR=1/1
[libx264 @ 0x9a54fa0] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
[libx264 @ 0x9a54fa0] profile High, level 3.0
[libx264 @ 0x9a54fa0] 264 - core 124 r2197 69a0443 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=400 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/home/tmp/content/movies/7762_tmp.mp4':
Metadata:
encoder : Lavf54.4.100
Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 400 kb/s, 25 tbn, 25 tbc
Stream #0:1: Audio: aac (@[0][0][0] / 0x0040), 44100 Hz, stereo, flt, 48 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg1video -> libx264)
Stream #0:1 -> #0:1 (mp2 -> aac)
Press [q] to stop, [?] for help
[aac @ 0x9a55640] Que input is backward in time
Last message repeated 2 times
[mp4 @ 0x9a54ac0] Audio timestamp 2147483392 < 2147483521 invalid, cliping
[aac @ 0x9a55640] Que input is backward in time
[mp4 @ 0x9a54ac0] Audio timestamp 2147483264 < 2147483522 invalid, cliping
[aac @ 0x9a55640] Que input is backward in time
[mp4 @ 0x9a54ac0] Audio timestamp 2147483136 < 2147483523 invalid, cliping
[aac @ 0x9a55640] Que input is backward in time
[mp4 @ 0x9a54ac0] Audio timestamp 2147483008 < 2147483524 invalid, cliping
[mp4 @ 0x9a54ac0] Audio timestamp 2147482880 < 2147483525 invalid, cliping
[aac @ 0x9a55640] Que input is backward in time
[mp4 @ 0x9a54ac0] Audio timestamp 2147482752 < 2147483526 invalid, cliping

"...and here it hangs and file grows and grows, till i kill ffmpeg."

[Carl Eugen Hoyos]

Replying to burek:

For the second question:

This is now ticket #1385
Please do not report different problems in one ticket, it makes following them very difficult.

[burek]

To be honest, I haven't realized there are 2 problems in this issue, I've just pasted the forum posts to make things easier to analyze and track. Now, I'm not sure where to put these samples that guy has provided using ffmpeg ftp server, so I'll leave them here too. Files uploaded were named like:
invalid_cliping_on_screen_but_encodes_finishes.mpg
invalid_cliping_on_screen_hangs_ffpmpeg_endless_file_grow.mpg

[Carl Eugen Hoyos]

Sample uploaded to samples/ffmpeg-bugs/trac/ticket1329

Without -async 1, the resulting file is very heavily out-of-sync, with -async 1, small desync is still visible imo after 30 seconds, not a regression afaict.
Seeking does not work for the sample, works fine with MPlayer.

$ ffmpeg -async 1 -i invalid_cliping_on_screen_but_encodes_finishes.mpg -q:v 2 -strict experimental out.mp4
ffmpeg version N-41352-gc8a1101 Copyright (c) 2000-2012 the FFmpeg developers
  built on Jun  6 2012 07:55:08 with gcc 4.3.2
  configuration: --cc=/usr/local/gcc-4.3.2/bin/gcc --enable-gpl --enable-libopenjpeg --enable-libvorbis --enable-libspeex --enable-libmp3lame --enable-libtheora --extra-ldflags=-lm --enable-libvpx --enable-libxavs
  libavutil      51. 56.100 / 51. 56.100
  libavcodec     54. 25.100 / 54. 25.100
  libavformat    54.  6.101 / 54.  6.101
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 78.100 /  2. 78.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[mpeg @ 0x90143c0] max_analyze_duration 5000000 reached at 5015467
Input #0, mpeg, from 'invalid_cliping_on_screen_but_encodes_finishes.mpg':
  Duration: 26:31:34.47, start: 0.620000, bitrate: 0 kb/s
    Stream #0:0[0x1e0]: Video: mpeg1video, yuv420p, 640x480 [SAR 1:1 DAR 4:3], 1452 kb/s, 25 fps, 25 tbr, 90k tbn, 25 tbc
    Stream #0:1[0x1c0]: Audio: mp2, 44100 Hz, stereo, s16, 128 kb/s
[buffer @ 0x901c620] w:640 h:480 pixfmt:yuv420p tb:1/90000 sar:1/1 sws_param:flags=2
[ffmpeg_buffersink @ 0x901c580] No opaque field provided
-async is forwarded to lavfi similarly to -af aresample=min_comp=0.001:min_hard_comp=0.100000.
[aresample @ 0x9015760] chl:stereo fmt:s16 r:44100Hz -> chl:stereo fmt:flt r:44100Hz
Output #0, mp4, to 'out.mp4':
  Metadata:
    encoder         : Lavf54.6.101
    Stream #0:0: Video: mpeg4 ( [0][0][0] / 0x0020), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 tbn, 25 tbc
    Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, flt, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg1video -> mpeg4)
  Stream #0:1 -> #0:1 (mp2 -> aac)
Press [q] to stop, [?] for help
frame= 1279 fps= 65 q=2.0 Lsize=   23874kB time=00:00:51.16 bitrate=3822.8kbits/s dup=3 drop=0
video:23092kB audio:748kB global headers:0kB muxing overhead 0.141022%

[Vadim]

I have the same issue when using "asyncts" audio filter:

ffmpeg -i udp://@237.255.1.10:1234 -codec:v copy -af asyncts=compensate=1:min_delta=0.1 -acodec libfaac -f mpegts tcp://example.com:9992

ffmpeg version 0.11 Copyright (c) 2000-2012 the FFmpeg developers
  built on Jan 19 2012 07:34:53 with gcc 4.6.2
  configuration: --shlibdir=/usr/lib64 --prefix=/usr --mandir=/usr/share/man --libdir=/usr/lib64 --enable-shared --disable-static --enable-debug --disable-stripping --extra-cflags='-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fPIC -I/usr/include/gsm' --enable-gpl --enable-x11grab --enable-version3 --enable-pthreads --enable-libfaac --enable-nonfree --enable-avfilter --enable-libpulse --enable-libvpx --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libspeex --enable-libxvid --enable-libx264 --enable-libschroedinger --enable-libgsm --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-postproc --enable-libdc1394 --enable-librtmp --enable-libfreetype
  libavutil      51. 54.100 / 51. 54.100
  libavcodec     54. 23.100 / 54. 23.100
  libavformat    54.  6.100 / 54.  6.100
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 77.100 /  2. 77.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[mpegts @ 0x62d140] Unable to seek back to the start
[mpeg2video @ 0x657a80] mpeg_decode_postinit() failure
    Last message repeated 4 times
[mp3 @ 0x65de00] Header missing
[mpegts @ 0x62d140] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x62d140] Estimating duration from bitrate, this may be inaccurate
Input #0, mpegts, from 'udp://@237.255.1.10:1234':
  Duration: N/A, start: 69244.458456, bitrate: 15096 kb/s
  Program 1
    Metadata:
      service_name    : MyChannel
      service_provider: MyChannel
    Stream #0:0[0x134]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 15000 kb/s, 25.61 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x100](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, mono, s16, 96 kb/s
[AVAudioResampleContext @ 0x6d86a0] Using s16p as internal sample format
[mpegts @ 0x695e60] muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'tcp://truba.com:9992':
  Metadata:
    encoder         : Lavf54.6.100
    Stream #0:0: Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 16:15 DAR 4:3], q=2-31, 15000 kb/s, 25.61 fps, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: aac, 48000 Hz, mono, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (mp2 -> libfaac)
Press [q] to stop, [?] for help
[mp2 @ 0x65de00] Header missing
Error while decoding stream #0:1
Error in av_buffersink_get_buffer_ref(): Invalid argument
frame=  166 fps=103 q=-1.0 size=    4698kB time=00:00:06.44 bitrate=5970.8kbits/s

Everything seems to be OK until source runs out of data (EOF and connnection close are tested) or user decides to abort ffmpeg execution.
In both cases ffmpeg does not terminate and starts to produce huge amount of messages (thousands per second) to stderr:

Que input is backward in time
[libfaac @ 0x7fef00] Que input is backward in time
    Last message repeated 3 times
[mpegts @ 0x66c700] Audio timestamp 477587 < 477592 invalid, cliping
[libfaac @ 0x7fef00] Que input is backward in time
[mpegts @ 0x66c700] Audio timestamp 477583 < 477593 invalid, cliping
[mpegts @ 0x66c700] Audio timestamp 477579 < 477594 invalid, cliping
[libfaac @ 0x7fef00] Que input is backward in time
[mpegts @ 0x66c700] Audio timestamp 477576 < 477595 invalid, cliping
[libfaac @ 0x7fef00] Que input is backward in time
[mpegts @ 0x66c700] Audio timestamp 477572 < 477596 invalid, cliping
[libfaac @ 0x7fef00] Que input is backward in time
[mpegts @ 0x66c700] Audio timestamp 477568 < 477597 invalid, cliping
[libfaac @ 0x7fef00] Que input is backward in time

until kill or ctrl+c is issued by user.

Problem occurs only when asyncts filter is used.

[Carl Eugen Hoyos]

Replying to lvm:

I have the same issue when using "asyncts" audio filter:

How is your problem (about an audio filter misbehaving on input starvation iiuc) related to desync for a specific mpeg input stream?

Please open a new ticket, if the problem is reproducible without using external libraries, don't use them (libfaac), using external libraries makes reproducing and fixing tickets much harder, if the problem is reproducible with -vn, please use it, if the problem is reproducible with a file as input or output, please use a file instead of a network stream.

[Carl Eugen Hoyos]

Fixed in FFmpeg 1.1 and git head.