Opened 5 months ago
#11041 new defect
Transcode drops frame that should be truncated by editlist
Reported by: | Anne | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avformat |
Version: | git-master | Keywords: | editlist |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
Came across a case where if the edit-list media-time is specified as in between frame timestamps, that frame will get dropped completely in the output of the transcode. The edit-list media-time is specified as 402. This is in between the 400 and 420 frame timestamps. The expectation is that the frame at 400 is still retained and its duration cut short for the transcode output. However, the frame is dropped completely at this line: https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/mov.c#L4167
The iso documentation has a snippet on edit-lists that seems to backup this interpretation: 'Edits are not restricted to fall on sample times. This means that when entering an edit, it can be necessary to (a) back up to a sync point, and pre-roll from there and then (b) be careful about the duration of the first sample — it might have been truncated if the edit enters it during its normal duration.' It seems like the frame should be truncated if edits are in between sample times.
Safari's player, premiere pro, and QuickTime retain the frame, so this behavior results in inconsistent playback of the original vs the transcoded output, where the output is missing a frame.
How to reproduce/output:
ffmpeg -i original.mov -c:v libx264 -y ffmpeg_output.mov ffmpeg version 7.0.1 Copyright (c) 2000-2024 the FFmpeg developers built with Apple clang version 15.0.0 (clang-1500.3.9.4) configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/7.0.1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon libavutil 59. 8.100 / 59. 8.100 libavcodec 61. 3.100 / 61. 3.100 libavformat 61. 1.100 / 61. 1.100 libavdevice 61. 1.100 / 61. 1.100 libavfilter 10. 1.100 / 10. 1.100 libswscale 8. 1.100 / 8. 1.100 libswresample 5. 1.100 / 5. 1.100 libpostproc 58. 1.100 / 58. 1.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'original.mov': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2024-05-04T18:13:43.000000Z com.apple.quicktime.location.accuracy.horizontal: 42.565599 com.apple.quicktime.location.ISO6709: +47.5327-122.1507+229.837/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 11 com.apple.quicktime.software: 16.2 com.apple.quicktime.creationdate: 2024-05-04T11:10:39-0700 Duration: 00:00:12.57, start: 0.000000, bitrate: 8200 kb/s Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 1920x1080, 7639 kb/s, 29.15 fps, 29.97 tbr, 600 tbn (default) Metadata: creation_time : 2024-05-04T18:13:43.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : HEVC Side data: displaymatrix: rotation of -90.00 degrees Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 145 kb/s (default) Metadata: creation_time : 2024-05-04T18:13:43.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2024-05-04T18:13:43.000000Z handler_name : Core Media Metadata Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default) Metadata: creation_time : 2024-05-04T18:13:43.000000Z handler_name : Core Media Metadata Stream mapping: Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) Press [q] to stop, [?] for help [libx264 @ 0x1226107c0] using cpu capabilities: ARMv8 NEON [libx264 @ 0x1226107c0] profile High, level 4.0, 4:2:0, 8-bit [libx264 @ 0x1226107c0] 264 - core 164 r3108 31e19f9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=24 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mov, to 'ffmpeg_output.mov': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt com.apple.quicktime.creationdate: 2024-05-04T11:10:39-0700 com.apple.quicktime.location.accuracy.horizontal: 42.565599 com.apple.quicktime.location.ISO6709: +47.5327-122.1507+229.837/ com.apple.quicktime.make: Apple com.apple.quicktime.model: iPhone 11 com.apple.quicktime.software: 16.2 encoder : Lavf61.1.100 Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1080x1920, q=2-31, 29.97 fps, 30k tbn (default) Metadata: creation_time : 2024-05-04T18:13:43.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : Lavc61.3.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A displaymatrix: rotation of -0.00 degrees Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default) Metadata: creation_time : 2024-05-04T18:13:43.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] encoder : Lavc61.3.100 aac [out#0/mov @ 0x600001e043c0] video:4606KiB audio:197KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.306282% frame= 376 fps=183 q=-1.0 Lsize= 4818KiB time=00:00:12.47 bitrate=3162.7kbits/s speed=6.06x [libx264 @ 0x1226107c0] frame I:2 Avg QP:21.53 size: 82112 [libx264 @ 0x1226107c0] frame P:99 Avg QP:22.87 size: 25951 [libx264 @ 0x1226107c0] frame B:275 Avg QP:24.07 size: 7210 [libx264 @ 0x1226107c0] consecutive B-frames: 0.5% 3.7% 6.4% 89.4% [libx264 @ 0x1226107c0] mb I I16..4: 11.5% 73.7% 14.8% [libx264 @ 0x1226107c0] mb P I16..4: 6.4% 13.0% 0.7% P16..4: 48.5% 6.5% 4.5% 0.0% 0.0% skip:20.3% [libx264 @ 0x1226107c0] mb B I16..4: 0.3% 0.4% 0.0% B16..8: 40.1% 1.1% 0.1% direct: 2.1% skip:55.9% L0:40.8% L1:57.8% BI: 1.3% [libx264 @ 0x1226107c0] 8x8 transform intra:64.6% inter:91.7% [libx264 @ 0x1226107c0] coded y,uvDC,uvAC intra: 27.7% 43.1% 12.0% inter: 10.6% 18.3% 0.2% [libx264 @ 0x1226107c0] i16 v,h,dc,p: 40% 31% 14% 15% [libx264 @ 0x1226107c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 18% 51% 2% 3% 3% 4% 2% 2% [libx264 @ 0x1226107c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 23% 29% 4% 6% 5% 6% 3% 3% [libx264 @ 0x1226107c0] i8c dc,h,v,p: 56% 23% 18% 3% [libx264 @ 0x1226107c0] Weighted P-Frames: Y:17.2% UV:6.1% [libx264 @ 0x1226107c0] ref P L0: 52.6% 26.5% 13.4% 6.8% 0.6% [libx264 @ 0x1226107c0] ref B L0: 92.4% 5.8% 1.8% [libx264 @ 0x1226107c0] ref B L1: 97.8% 2.2% [libx264 @ 0x1226107c0] kb/s:3007.25 [aac @ 0x122684110] Qavg: 234.931