Regression with -vsync 1

[jwjwjw]

Summary of the bug:

It seems that the latest version of ffmpeg from git has a bug with -vsync 1. I re-encoded some footage from BBC television and the A/V sync slightly drifts over time which is noticeable with the lipsync being off, even though the original file has no sync issues. Using -vsync 1 has always fixed this in the past but it no longer works with the latest version of ffmpeg (N-43888-gfef9e84). It does, however, work with an earlier version, N-31890-gecaf514.

Included is a sample file (bbc1sample.mp4) as well as two ffmpeg output files, bbc1-ok.mov and bbc1-fail.mov, the 'ok' being encoded using N-31890-gecaf514 which is in perfect sync and the 'fail' being from the latest git master (N-43888-gfef9e84) which slowly drifts out of sync (I think it's out by around 0.1s in the sample media because I couldn't upload a larger clip, but as time goes by the sync gets worse and you can notice the lipsync is slightly off when the man is talking in vision). Both used identical ffmpeg options, just different ffmpeg versions.

However, playing the encoded file in QuickTime is fine. No sync issues. Using VLC or encoding to FLV and playing it in Flash does exhibit the sync issue. The original bbc1sample.mp4 plays fine in all players. Frame by frame visual comparisons between the 'ok' and 'fail' clips show a mismatch.

N-31890-gecaf514 results in a file with a duration of 00:00:49.64 and the latest version produces a file with a duration of 00:00:49.72. The sample outputs were encoded using libx264 and libfaac to reduce the file size but the problem occurs regardless of video or audio codec and the use or omission of -async has no effect either.

I have included log files which should hopefully shed more light on it.

How to reproduce:

% ffmpeg -i bbc1sample.mp4 -vb 300k -vsync 1 -async 1 -acodec libfaac -ab 192k -ac 2 -ar 44100 -vcodec mpeg4 bbc1-fail.mov
ffmpeg version N-43888-gfef9e84 Copyright (c) 2000-2012 the FFmpeg developers
  built on Aug 25 2012 01:37:52 with gcc 4.4.5 (Debian 4.4.5-8)
  configuration: --enable-libdc1394 --prefix=/usr --extra-cflags='-Wall -g ' --enable-shared --enable-libmp3lame --enable-gpl --enable-libvorbis --enable-pthreads --enable-libfaac --enable-libxvid --enable-postproc --enable-x11grab --enable-libgsm --enable-libtheora --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libx264 --enable-libspeex --enable-nonfree --disable-stripping --enable-avfilter --enable-libschroedinger --disable-encoder=libschroedinger --enable-version3 --enable-libopenjpeg --enable-libvpx --extra-libs=-lgcrypt --disable-altivec --disable-armv5te --disable-armv6 --disable-vis
  libavutil      51. 70.100 / 51. 70.100
  libavcodec     54. 54.100 / 54. 54.100
  libavformat    54. 25.104 / 54. 25.104
  libavdevice    54.  2.100 / 54.  2.100
  libavfilter     3. 13.101 /  3. 13.101
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[h264 @ 0xf81a20] AVC: nal size 21839902
[h264 @ 0xf81a20] no frame!
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xf7b240] max_analyze_duration 5000000 reached at 5022161
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbc1sample.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.25.104
  Duration: 00:00:49.58, start: 0.000000, bitrate: 900 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 798 kb/s, 28.27 fps, 25 tbr, 1k tbn, 50 tbc
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, s16, 93 kb/s
    Metadata:
      handler_name    : SoundHandler
-async is forwarded to lavfi similarly to -af aresample=min_comp=0.001:min_hard_comp=0.100000.
Output #0, mov, to 'bbc1-fail.mov':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.25.104
    Stream #0:0(und): Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=2-31, 300 kb/s, 25 tbn, 25 tbc
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, s16, 192 kb/s
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg4)
  Stream #0:1 -> #0:1 (aac -> libfaac)
Press [q] to stop, [?] for help
[h264 @ 0x12a07a0] AVC: nal size 21839902
    Last message repeated 1 times
[h264 @ 0x12a07a0] no frame!
Error while decoding stream #0:0: Operation not permitted
frame= 1243 fps=189 q=31.0 Lsize=    3069kB time=00:00:49.57 bitrate= 507.2kbits/s dup=1 drop=159    
video:2116kB audio:920kB subtitle:0 global headers:0kB muxing overhead 1.077261%

[jwjwjw]

Log of the old version of ffmpeg that generates an in sync file

[jwjwjw]

Log of the latest version of ffmpeg that generates an out of sync file

[jwjwjw]

The sample file and the outputs are too large to upload to trac.

​http://jwjwjw.com/ffmpeg/bbc1sample.mp4 is the sample media used for the encoding.

​http://jwjwjw.com/ffmpeg/bbc1-ok.mov is an example output from the old working version of ffmpeg.

​http://jwjwjw.com/ffmpeg/bbc1-fail.mov is an example output from the new version of ffmpeg.

[Carl Eugen Hoyos]

For me, all three videos play in-sync, so please upload a longer original, see ​http://ffmpeg.org/bugreports.html

Can you reproduce the problem with MPlayer or ffplay, or are you reporting an issue with vlc?

[jwjwjw]

I don't think it is an issue with vlc. I don't have MPlayer but I also see the problem using ffplay.

I am recording a longer clip now. But even with the previously provided clip, converting it to an image sequence shows the problem:

ffmpeg -i bbc1-fail.mov badjpg/video%d.jpg
ffmpeg -i bbc1-ok.mov goodjpg/video%d.jpg

Despite having identical frame rates and audio, the scene changes from the track to the man in vision at video402.jpg in the 'fail' clip and video400.jpg in the 'ok' clip. Do you see that too, and if so, isn't that confirmation of the bug as it's changing 2 frames later in the failed clip so the sync is out by 2 frames?

[jwjwjw]

Here is another sample file: ​http://jwjwjw.com/ffmpeg/bbcnews-original.mp4
Encoded using git-master: ​http://jwjwjw.com/ffmpeg/bbcnews-git-master.mp4
Encoded using N-31890-gecaf514: ​http://jwjwjw.com/ffmpeg/bbcnews-old-version.mp4

Again the sync is only slightly out in the git master version, but at least for me it is noticeable and definitely not present in the version encoded by N-31890-gecaf514.

Both versions were encoded using identical ffmpeg parameters:

ffmpeg -i bbcnews-original.mp4 -vb 500k -ac 2 -ar 44100 -vcodec libx264 -acodec libfaac -vsync 1 -async 1 -ab 128k bbcnews.mp4

Unfortunately at the moment I am not able to record a longer clip. The sync is out in vlc, ffplay and flash.

[jwjwjw]

Here is a frame by frame comparison between the two files. As you can see, git master's encoded output is 2 frames ahead of the old version, which leads to the video being 2 frames out of sync with the audio which is identical in both versions.

[jwjwjw]

Sorry, git master is 2 frames BEHIND the old version, not ahead.

[jwjwjw]

After running git bisect I don't think the problem is specifically related to -vsync actually, it's just a regression as a result of the fix for #137.

Regression since 8f5bb35

[Carl Eugen Hoyos]

Is this also reproducible without -vsync 1 -async 1 ?

[jwjwjw]

Yes, I carried out further testing of the ​http://jwjwjw.com/ffmpeg/bbcnews-original.mp4 source file using a basic command line to rule out any other options or external libraries:

ffmpeg -i bbcnews-original.mp4 -c:v mpeg4 -c:a aac -strict experimental output.mp4

It yields the following results in different versions of ffmpeg:

c5ea3a0 onwards up to and including git-master (file is out of sync):

  Duration: 00:01:37.84, start: 0.000000

8f5bb35 (file is out of sync):

  Duration: 00:01:37.80, start: 0.000000

pre-8f5bb35 (file is in sync perfectly):

  Duration: 00:01:37.76, start: 0.000000

The following command shows that there are additional frames at the start of the out of sync output that are not present in the in sync output, whilst the audio is the same in all 3 clips (so the sync is out by the number of additional frames):

ffmpeg -i output.mp4 output%d.jpg

So it seems to be a regression since 8f5bb35 but something also changed in c5ea3a0 too.

[jwjwjw]

ffmpeg log from 8f5bb35 (generates out of sync file with a duration of 00:01:37.80)

[jwjwjw]

ffmpeg log from fbbe092 (generates an in sync file with a duration of 00:01:37.76)

[jwjwjw]

ffmpeg log from c5ea3a0 (generates out of sync file with a duration of 00:01:37.84) - same as git-master

[Michael Niedermayer]

Please elaborate if you are reporting a problem with AV sync OR a problem with frame duplication OR a problem with file duration.

The AV sync issue is not reproduceable with the test samples provided, maybe a longer sample would help

The change in duration or frame duplication is not a proof of av desync and there is currently no solid
argument why either duration or frame duplication pattern would be more correct than the other.

[jwjwjw]

It is a pity that you can't see it in these samples because I can see it quite clearly in ffplay/vlc using the samples provided. It's just a fraction of a second but it is noticeable for me. I can't provide more samples at present.

The problem is AV sync which is only apparent in the output files based on the bbcnews-original.mp4 sample that have been generated with a duration that is not 00:01:37.76.

[jwjwjw]

Please check this video showing both the clips side by side (old version left, git master right). For reference, the audio is taken from the right hand video but it is actually the left hand video that is in sync, which proves that the audio is identical in both clips.

You can see the frames changing on the left hand video a fraction of a second before the right hand side, meaning the right hand side is slightly out of sync. I don't know what else I can do to demonstrate the problem. It's really clear and distracting for me and is definitely not present in the old version of ffmpeg.

​http://jwjwjw.com/ffmpeg/comparison.mp4

[Carl Eugen Hoyos]

Replying to jwjwjw:

You can see the frames changing on the left hand video a fraction of a second before the right hand side

Why do you think the left side is correct? (The reason I ask is that I believe both are not in sync.)
I believe it would be better if you could provide the original dvb stream (assuming it was recorded from dvb).

[jwjwjw]

It was taken directly from their web stream. The reason why I think the left is correct is because it matches the original mp4 file.

[Michael Niedermayer]

mp4 / mov AV sync issues should have been fixed with the recent commits. This does not mean that the video frames will match some imagined correct reference just the AV sync itself should be correct (and i can confirm that AV sync was not correct previously with test media where it could be tested clearer).
Iam thus closing this but doent hesitate to reopen if some AV sync issue remains or open a new ticket if some other issue remains