Audio Out of Sync after transcode

[ronag]

Summary of the bug:

After transcoding the audio is off by 2 frames.

How to reproduce:

ronagy$ ffmpeg -i original.mov -flags +ilme+ildct transcoded.mov
ffmpeg version git-2016-05-05-5b174dd Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.3.0 (clang-703.0.29)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-libfreetype --enable-libvorbis --enable-libvpx --enable-librtmp --enable-libass --enable-ffplay --enable-libfdk-aac --enable-libopus --enable-libx265 --enable-nonfree --enable-vda
  libavutil      55. 23.100 / 55. 23.100
  libavcodec     57. 38.100 / 57. 38.100
  libavformat    57. 36.100 / 57. 36.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 44.100 /  6. 44.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'original.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf57.25.100
  Duration: 00:00:21.28, start: 0.000000, bitrate: 130459 kb/s
    Stream #0:0(eng): Video: dnxhd (AVdn / 0x6E645641), yuv422p(tv, bt709/unknown/unknown), 1920x1080, 121241 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
    Metadata:
      handler_name    : DataHandler
      timecode        : 00:00:00:00
    Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 7.1, s32 (24 bit), 9216 kb/s (default)
    Metadata:
      handler_name    : DataHandler
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s
    Metadata:
      handler_name    : DataHandler
      timecode        : 00:00:00:00
No pixel format specified, yuv422p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x7ff6b3811e00] interlace + weightp is not implemented
[libx264 @ 0x7ff6b3811e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x7ff6b3811e00] profile High 4:2:2, level 4.0, 4:2:2 8-bit
[libx264 @ 0x7ff6b3811e00] 264 - core 148 r2601 a0cd7d3 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=tff bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[mov @ 0x7ff6b3810c00] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
    Last message repeated 1 times
Output #0, mov, to 'transcoded.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf57.36.100
    Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv422p, 1920x1080, q=2-31, 0.04 fps, 12800 tbn (default)
    Metadata:
      handler_name    : DataHandler
      timecode        : 00:00:00:00
      encoder         : Lavc57.38.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 7.1, fltp (24 bit), 469 kb/s (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc57.38.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (dnxhd (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (pcm_s24le (native) -> aac (native))
Press [q] to stop, [?] for help
frame=  532 fps=106 q=-1.0 Lsize=    1833kB time=00:00:21.29 bitrate= 705.2kbits/s speed=4.25x    
video:602kB audio:1213kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.012604%
[libx264 @ 0x7ff6b3811e00] frame I:3     Avg QP:17.24  size: 57340
[libx264 @ 0x7ff6b3811e00] frame P:143   Avg QP:24.25  size:  2370
[libx264 @ 0x7ff6b3811e00] frame B:386   Avg QP:31.72  size:   271
[libx264 @ 0x7ff6b3811e00] consecutive B-frames:  2.6%  1.5%  1.1% 94.7%
[libx264 @ 0x7ff6b3811e00] mb I  I16..4: 25.6% 66.1%  8.4%
[libx264 @ 0x7ff6b3811e00] mb P  I16..4:  0.3%  0.3%  0.1%  P16..4:  2.3%  0.3%  0.2%  0.0%  0.0%    skip:96.5%
[libx264 @ 0x7ff6b3811e00] mb B  I16..4:  0.0%  0.1%  0.0%  B16..8:  0.9%  0.1%  0.0%  direct: 0.0%  skip:98.9%  L0:28.6% L1:69.1% BI: 2.2%
[libx264 @ 0x7ff6b3811e00] field mbs: intra: 9.2% inter:6.5% skip:0.7%
[libx264 @ 0x7ff6b3811e00] 8x8 transform intra:63.2% inter:65.4%
[libx264 @ 0x7ff6b3811e00] coded y,uvDC,uvAC intra: 7.5% 12.5% 8.4% inter: 0.5% 0.1% 0.0%
[libx264 @ 0x7ff6b3811e00] i16 v,h,dc,p: 74% 24%  2%  0%
[libx264 @ 0x7ff6b3811e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 40%  2% 56%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x7ff6b3811e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 36% 17% 25%  4%  4%  4%  4%  4%  3%
[libx264 @ 0x7ff6b3811e00] i8c dc,h,v,p: 68% 11% 20%  1%
[libx264 @ 0x7ff6b3811e00] ref P L0: 90.7%  4.1%  4.7%  0.0%  0.3%  0.1%
[libx264 @ 0x7ff6b3811e00] ref B L0: 66.9% 27.7%  5.1%  0.1%  0.2%  0.1%
[libx264 @ 0x7ff6b3811e00] ref B L1: 96.1%  3.6%  0.3%  0.0%
[libx264 @ 0x7ff6b3811e00] kb/s:231.39
[aac @ 0x7ff6b3813600] Qavg: 573.770

Files are available here:

​https://www.dropbox.com/sh/9qi9813i3jm1mlz/AACkgGs5IP4AizOja0HfgtdUa?dl=0

[ronag]

original sync

[ronag]

transcoded sync

[Carl Eugen Hoyos]

I believe this is a duplicate of one or more other tickets, I unfortunately cannot comment on these tickets as I have no idea how to reproduce them.

[ronag]

@cehoyos: Why can't you reproduce this? I've provided the files and the command line. What are you missing?

[Carl Eugen Hoyos]

I am not saying I am missing something, I just don't know how to reproduce an off-by-two-frames. (I also believe the issue is not reproducible at all but that is not my point.)

Do you believe the issue is also reproducible with -vcodec mpeg4 -qscale 2?

[Kieran Kunhya]

Probably crap b-frames handling in mov. Turn off bframes.
Kieran

[ronag]

@Kieran: That did solve it.

[ronag]

Unfortunately it's all over the place. Especially when I involve a -ss infront of -i.

@cehoyos: What can I do to help finding a resolution to this issue?

[Carl Eugen Hoyos]

Replying to ronag:

What can I do to help finding a resolution to this issue?

Did you already test if you can reproduce the issue with -vcodec mpeg4 -qscale 2?
Is the issue more visiable (more than two frames off) if you use eight b-frames?

[ronag]

If I run:

ffmpeg -ss 10.032 -i original.mov -flags +ilme+ildct -c:v mpeg4 -qscale:v 2 seeked2.mov

It's off by 34 frames.

[Carl Eugen Hoyos]

Is -flags +ilme+ildct needed to reproduce the issue?

[ronag]

ffmpeg -ss 10.032 -i original.mov -flags +ilme+ildct seeked3.mov

Is off by 2 frames

ffmpeg -ss 10.032 -i original.mov -flags +ilme+ildct -bf:v 0 seeked4.mov

Is correct.

[ronag]

@cehoyos: No. It's the same without the flags.

[ronag]

With the following it's incorrect even when using kierans trick.

ffmpeg -ss 10.032 -i original.mxf -bf:v 0 seeked8.mov

Off by 1 frame.

[ronag]

With the following it's incorrect even when using kierans trick.

ffmpeg -ss 10.032 -i original.mxf -bf:v 0 seeked8.mxf

Off by 1 frame.

[Carl Eugen Hoyos]

I tested the following command line with current FFmpeg git head:

$ ffmpeg -ss 10.032 -i original.mov -qscale 2 -vcodec mpeg4 out.mov

I tested playback of the output file with the following two commands:

$ vlc --rate=0.25 out.mov

$ mplayer -speed 0.1 out.mov

Both do not show out-of-sync audio afaict.

[ronag]

@cehoyos: Yes, you are correct. With that command line it works. Not sure what I did wrong before.

[ronag]

@cehoyos:

However, using the file "3.mxf"

ffmpeg -ss 10.032 -i 3.mxf -qscale 2 -vcodec mpeg4 out10.mov

It's off by 1 frame.

[Carl Eugen Hoyos]

Can we keep one input file for testing please?

Is the effect accumulating? Does repeating the command line twenty times lead to twenty frames off?

Please understand that imo, there is no such thing as "off-by-one-frame" for real-world video, playback equipment (and speaker distance) is not identical for all users.

[ronag]

Can we keep one input file for testing please?

Sure.

Please understand that imo, there is no such thing as "off-by-one-frame" for real-world video, playback equipment (and speaker distance) is not identical for all users.

I need the video and audio to be in sync since it's to be used in After Effects and AVID edit suites. The files are not directly used for playback.

[enevills]

I see the same thing when making x264 files with AAC audio. After Effects and Avid see files at 2 frames off.

Command:

ffmpeg -loglevel debug -i 'file.mov' -pix_fmt yuv420p -vf scale=960:540 -c:v libx264 -preset veryslow -crf 23 -x264opts ref=4:qpmin=4 -color_primaries 1 -color_trc 1 -colorspace 1 -profile:v main -level 4.0 -f mp4 -ac 2 -c:a aac -b:a 192k -strict -2 -threads 3 -y 'export.mp4'

Sample File: ​https://goo.gl/yuV0st

[Carl Eugen Hoyos]

Replying to enevills:

I see the same thing when making x264 files with AAC audio.

Do I understand correctly that this is only reproducible with x264 and not with -vcodec mpeg4 -qscale 2?

[enevills]

I just gave -vcodec mpeg4 -qscale 2 and it is 1 frame early. Here is a screenshot from After Effects CC 2015. ​http://imgur.com/7w40W8M Notice in the bottom right, the playhead is a frame before the waveform beep.

My command was two frames off. Is the issue with the AAC encoding?

Same results seem to happen with v3.0.2 and v2.8.6

Looks like tomorrow I'll be making a 2nd command to offset the video.

Replying to cehoyos:

Replying to enevills:

I see the same thing when making x264 files with AAC audio.

Do I understand correctly that this is only reproducible with x264 and not with -vcodec mpeg4 -qscale 2?

[Carl Eugen Hoyos]

Replying to enevills:

I just gave -vcodec mpeg4 -qscale 2 and it is 1 frame early.

As said, I don't really believe anything can be "one frame early" but much more important: Is the issue cumulative, does it get worse if you repeat the encoding?

[enevills]

Replying to cehoyos:

Replying to enevills:

I just gave -vcodec mpeg4 -qscale 2 and it is 1 frame early.

As said, I don't really believe anything can be "one frame early" but much more important: Is the issue cumulative, does it get worse if you repeat the encoding?

It doesn't seem to get worse if I re-encode the output. Even after a few times.

[enevills]

As a workaround, I'm running this after I transcoder my file. ffmpeg -i x264_file.mov -itsoffset 0.05994 -i x264_file.mov -vcodec copy -acodec copy -map 0:0 -map 1:1 -shortest output.mp4

[enevills]

I know this is old, and solved a bit ago.

This isn't an FFmpeg issue. I found out applications using Apple's deprecated QTKit assume 2112 priming samples rather than reading the header. Most modern encoders seem to use 1024 samples. More info here: ​https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFAppenG/QTFFAppenG.html

I think QTKit has lots of 32-bit code in it so it's finally getting phased out and becoming a non-issue.