Incorrect frame order when muxing H.264 stream with B-frames

[wdzierzan]

Summary of the bug: The order of frames in an MP4 file is incorrect when an H.264 stream containing B-frames is muxed.

How to reproduce:

% ffmpeg -i blob.h264 -vcodec copy blob.h264.mp4
% ffmpeg -i blob.h264 -vcodec copy -bsf:v dts2pts blob.h264.pts.mp4

ffmpeg version 6.0
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100

Without the dts2pts filter, the initial I-frame is missing in the muxed file. pts_time as shown by ffprobe are non-monotonic (in decoding instead of presentation order?). Playback in QuickTime and popular web browsers is affected.

AFAIU, the dts2pts filter is supposed to help for this case. It helps to some extent: the initial I-frame is preserved and pts_time are almost monotonic. The issue is that they are not exactly monotonic (the order is broken around the middle and end of the file).

Meanwhile, the output of

% MP4Box -add blob.h264 blob.h264.mp4box.mp4

seems fine.

[Balling]

-vcodec option is deprecated (except for ffplay application). You must use -c:v.

[Balling]

[null @ 0000023b03383440] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 38 >= 38

in

ffmpeg.exe -i hnefcva.mp4 -an -vf vfrdet -f null -

[wdzierzan]

Thanks!

Used -c:v copy now with

ffmpeg version N-111800-g5f5a1ccd04

Same result.

[Balling]

The real problem here is that it cannot preserve original framerate of

[trace_headers @ 0000024b9b7cc4c0] 92          num_units_in_tick            00000000000000000000000000000001 = 1
[trace_headers @ 0000024b9b7cc4c0] 124         time_scale                   00000000000000000000000000101000 = 40
[trace_headers @ 0000024b9b7cc4c0] 156         fixed_frame_rate_flag                                       0 = 0

Is that expected that you set the 40 tbr (40/1, time_scale/num_units_in_tick)? That is not a typical tbr.

Also fixed_frame_rate_flag is not set, that is rare.

[wdzierzan]

I'm using the Media Foundation encoder, and I instructed it to set an fps of 20 via MF_MT_FRAME_RATE. Not sure how this ended up as 40 (there's something about "half the tick rate" in https://ffmpeg.org/ffmpeg-all.html#toc-h264_005fmetadata, but frankly I'm not at all sure I understand it correctly).

I agree an fps of 20 (or 40) is not typical, but I didn't think it would matter. Note that the frame durations are consistent with 20 fps.

I also tried a more conventional frame rate of 30, but this doesn't improve things.

[Balling]

there's something about "half the tick rate"

The fixed_frame_rate_flag is not set, half does not apply.

[wdzierzan]

Right.

In any case, different time_scale values don't seem to make a difference for this particular issue.