H.264 VFR timestamp hazards (which means frame duration... or playback experience hazards)

[gdgsdg123]

​123.avi

ffmpeg -y -i "123.avi" -c:v h264_nvenc -r 1 -g 1 -vsync vfr "temp.avi"

The timestamp of the 1st frame is delayed according to -r (by 1/r exactly), while the rest of the timestamps remain unchanged (have verified this with more complex input source).

Which negatively affects the 1st frame's duration. (and the impact persists in further re-encodings)

ffmpeg -y -i "123.avi" -c:v h264_nvenc -r 1 -g 2 -vsync vfr "temp.avi"

The duration of the 2nd frame is influenced by -r.

ffmpeg -y -i "123.avi" -c:v h264_nvenc -r 1 -g 3 -vsync vfr "temp.avi"

Appeared normal. (all timestamps are delayed accordingly)

ffmpeg -y -i "123.avi" -c:v libx264 -r 1 -g 1 -vsync vfr "temp.avi"

Normal.

ffmpeg -y -i "123.avi" -c:v libx264 -r 1 -g 2 -vsync vfr "temp.avi"

The duration of P-frames (the 2nd frame in this case) become 1/r. (absent timestamp information in the packet?..)

ffmpeg -y -i "123.avi" -c:v libx264 -r 10 -g 3 -vsync vfr "temp.avi"

Better demonstration of the above problem.

ffmpeg -y -i "123.avi" -c:v libx264 -r 10 -qp 0 -g 3 -vsync vfr "temp.avi"

Normal. (no P-frame in the output, due to the scenecut decision of x264)

---

Unsure if it's the encoder's fault or FFmpeg's.



Off-Topic

ffprobe -hide_banner -show_entries "frame" -select_streams v:0 -of "xml" "temp.avi"

ffprobe -hide_banner -show_entries "frame=best_effort_timestamp_time" -select_streams v:0 -of "compact=nk=1:p=0" "temp.avi"

...In case you don't know how to use FFprobe.


And in the world of CFR... the duration doesn't have to really exist.

[gdgsdg123]

Replying to gdgsdg123:

Unsure if it's the encoder's fault or FFmpeg's.

I heard that HandBrake (which also use x264) encodes videos in a manner similar to 90000 FPS VFR... (1:90000 time base)

If so, and the problem is on the encoder (x264), then the influence of the problem will be super obvious.


While there doesn't seem to be report on such things happening...

---

I find the problem to be container related...

Cannot reproduce:

ffmpeg -y -i "123.avi" -c:v libx264 -r 10 -g 3 -vsync vfr "temp.mkv"

ffmpeg -y -i "123.avi" -c:v libx264 -r 10 -g 3 -vsync vfr "temp.mp4"

ffmpeg -y -i "123.avi" -c:v h264_nvenc -r 1 -g 1 -vsync vfr "temp.mkv"

ffmpeg -y -i "123.avi" -c:v h264_nvenc -r 1 -g 1 -vsync vfr "temp.mp4"

AVI exclusive?..

---

Replying to gdgsdg123:

ffmpeg -y -i "123.avi" -c:v libx264 -r 1 -g 1 -vsync vfr "temp.avi"

Normal.

ffmpeg -y -i "123.avi" -c:v libx264 -r 1 -g 2 -vsync vfr "temp.avi"

The duration of P-frames (the 2nd frame in this case) become 1/r. (absent timestamp information in the packet?..)

ffmpeg -y -i "123.avi" -c:v libx264 -r 10 -g 3 -vsync vfr "temp.avi"

Better demonstration of the above problem.

ffmpeg -y -i "123.avi" -c:v libx264 -r 10 -qp 0 -g 3 -vsync vfr "temp.avi"

Normal. (no P-frame in the output, due to the scenecut decision of x264)

Just tested with a more common footage (​YouTube Preview):

ffmpeg -i "Fly.mkv" -an -c:v libx264 -r 60 -tune film -crf 19 -x264-params "keyint=150:qcomp=0.75" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -vsync vfr -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "PB_duration_quicktest.avi"

ffmpeg version git-2020-01-10-3d894db Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.2.1 (GCC) 20191125
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 38.100 / 56. 38.100
  libavcodec     58. 65.103 / 58. 65.103
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 70.101 /  7. 70.101
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, matroska,webm, from 'Fly.mkv':
  Metadata:
    ENCODER         : Lavf58.35.101
  Duration: 00:04:42.83, start: 0.000000, bitrate: 157890 kb/s
    Stream #0:0: Video: h264 (High 4:4:4 Predictive), gbrp(tv, gbr/unknown/unknown, progressive), 1920x1080, 60 fps, 60 tbr, 1k tbn, 120 tbc (default)
    Metadata:
      ENCODER         : Lavc58.65.103 libx264rgb
      DURATION        : 00:04:42.817000000
    Stream #0:1: Audio: flac, 48000 Hz, stereo, s16 (default)
    Metadata:
      ENCODER         : Lavc58.65.103 flac
      DURATION        : 00:04:42.829000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 00000000004dacc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 00000000004dacc0] profile High, level 4.2, 4:2:0, 8-bit
Output #0, avi, to 'PB_duration_quicktest.avi':
  Metadata:
    ISFT            : Lavf58.35.101
    Stream #0:0: Video: h264 (libx264) (H264 / 0x34363248), yuv420p(tv, bt470bg/bt470bg/smpte170m), 1920x1080, q=-1--1, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      DURATION        : 00:04:42.817000000
      encoder         : Lavc58.65.103 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 5855 fps= 32 q=-1.0 Lsize=  319142kB time=00:04:42.68 bitrate=9248.5kbits/s speed=1.57x
video:318736kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.127416%
[libx264 @ 00000000004dacc0] frame I:53    Avg QP:12.95  size:173954
[libx264 @ 00000000004dacc0] frame P:1577  Avg QP:15.22  size: 68018
[libx264 @ 00000000004dacc0] frame B:4225  Avg QP:17.46  size: 49681
[libx264 @ 00000000004dacc0] consecutive B-frames:  2.8%  1.7%  3.9% 91.6%
[libx264 @ 00000000004dacc0] mb I  I16..4:  9.5% 60.1% 30.4%
[libx264 @ 00000000004dacc0] mb P  I16..4:  2.8% 23.8%  7.1%  P16..4: 11.6%  7.0%  3.4%  0.0%  0.0%    skip:44.3%
[libx264 @ 00000000004dacc0] mb B  I16..4:  1.0% 12.1%  2.7%  B16..8: 15.2% 10.5%  3.2%  direct: 3.6%  skip:51.6%  L0:46.2% L1:39.0% BI:14.8%
[libx264 @ 00000000004dacc0] 8x8 transform intra:73.2% inter:64.7%
[libx264 @ 00000000004dacc0] coded y,uvDC,uvAC intra: 78.6% 48.4% 19.4% inter: 21.6% 12.7% 1.2%
[libx264 @ 00000000004dacc0] i16 v,h,dc,p: 16% 32%  4% 48%
[libx264 @ 00000000004dacc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 22% 15%  9%  9%  4% 13%  5% 13%
[libx264 @ 00000000004dacc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 27% 12%  6%  9%  6% 10%  4%  7%
[libx264 @ 00000000004dacc0] i8c dc,h,v,p: 60% 24% 13%  3%
[libx264 @ 00000000004dacc0] Weighted P-Frames: Y:3.1% UV:2.7%
[libx264 @ 00000000004dacc0] ref P L0: 46.9% 15.4% 23.1% 14.4%  0.3%
[libx264 @ 00000000004dacc0] ref B L0: 76.4% 18.4%  5.2%
[libx264 @ 00000000004dacc0] ref B L1: 92.4%  7.6%
[libx264 @ 00000000004dacc0] kb/s:9230.79

The problem seemed to only occur on the last 2 frames... * and it influences B-frames too. Check the ​"PB_duration_quicktest.xml".

Highlight (Line 5857 to 5858):

        <frame media_type="video" stream_index="0" key_frame="0" pkt_duration="1" pkt_duration_time="0.016667" pkt_pos="326529470" pkt_size="382" width="1920" height="1080" pix_fmt="yuv420p" pict_type="B" coded_picture_number="5854" display_picture_number="0" interlaced_frame="0" top_field_first="0" repeat_pict="0" color_range="tv" color_space="bt470bg" color_primaries="bt470bg" color_transfer="smpte170m" chroma_location="left"/>
        <frame media_type="video" stream_index="0" key_frame="0" pkt_duration="1" pkt_duration_time="0.016667" pkt_pos="326529356" pkt_size="82" width="1920" height="1080" pix_fmt="yuv420p" pict_type="P" coded_picture_number="5853" display_picture_number="0" interlaced_frame="0" top_field_first="0" repeat_pict="0" color_range="tv" color_space="bt470bg" color_primaries="bt470bg" color_transfer="smpte170m" chroma_location="left"/>

* After comparing the timestamps of the "Fly.mkv" (​workaround'd) and the "PB_duration_quicktest.avi" I also find a lot of discrepancies between them... (yes, it means serious frame PTS shifts) (check the ​"Fly.txt" and the ​"PB_duration_quicktest.txt")


And further re-encoding based on the problematic file:

ffmpeg -i "PB_duration_quicktest.avi" -an -c:v libx264 -preset ultrafast -qp 0 -x264-params "keyint=1" -pix_fmt yuv420p -vsync vfr "PB_duration_quicktest_re.avi"

...ends in rather unpredictable timestamps. (check the ​"PB_duration_quicktest_re.txt")

[gdgsdg123]

Replying to cehoyos:
What info?