HEVC fails to seek on keyframe

[SuRGeoNix]

I'm using av_seek_frame and even if it returns 0 pb will not be at an actual keyframe. The issues seems to happen with any HEVC/mpeg-ts format container.

[Format ] MPEG-TS (MPEG-2 Transport Stream)/mpegts
[Video #0] hevc yuv420p10le 3840x2160 @ 59,94
[Audio #1-eng] aac fltp@0 48KHz stereo

I've tested also with ffplay which fails as well with the following output

C:\Users\Owner\Downloads\ffmpegmb\bin>ffplay.exe -ss 00:01:00 -analyzeduration 100000000000 -probesize 100000000000 "c:\root\down\samples\hd\LG Chess 4K Demo.ts"
ffplay version N-103543-gc655a734b1-20210907 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 10-win32 (GCC) 20210408
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libvmaf --enable-vulkan --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libglslang --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20210907
  libavutil      57.  4.101 / 57.  4.101
  libavcodec     59.  7.102 / 59.  7.102
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  7.101 /  8.  7.101
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[mpegts @ 00000282a0400480] stream 0 : no PTS found at end of file, duration not set
Input #0, mpegts, from 'c:\root\down\samples\hd\LG Chess 4K Demo.ts':
  Duration: 00:01:52.92, start: 1.066722, bitrate: 62572 kb/s
  Program 1
  Stream #0:0[0x101]: Video: hevc (Main 10) ([36][0][0][0] / 0x0024), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn
  Stream #0:1[0x102](und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 383 kb/s
[aac @ 00000282a5beb040] channel element 0.0 is not allocated
[hevc @ 00000282a8b2d040] Could not find ref with POC 15 0B f=0/0
[hevc @ 00000282a8b2d040] Could not find ref with POC 13
[hevc @ 00000282a8b2d040] Could not find ref with POC 17
[hevc @ 00000282a8b2d040] Could not find ref with POC 21
[hevc @ 00000282a8b2d040] Could not find ref with POC 28 0B f=0/0
  62.26 A-V: -0.025 fd=   3 aq=   47KB vq=11446KB sq=    0B f=0/0

[Balling]

I can confirm that, please see: #7950

It happens due to 4 tiles in the file.

[SuRGeoNix]

Found another issue with HEVC (at least with the same LG Chess 4K Demo.ts) that might be related to this one.

I have implemented a reverse playback by decoding from keyframe until the next one and then seeking to the previous keyframe (Manually ensuring that I actually have keyframe to keyframe).

While decoding I'm draining the decoder properly but it seems that sometimes it misses some frames and I have at least 2-3 frames diff between the last draining frame and the next keyframe. I will do further testing and open a new issue if required.

[Balling]

I can give more samples... Most of them are fixed when copying to mp4! So that means the bug is in MPEG-TS part maybe in keyframes marking in TS itself.

Also you can try to revert fbb283cfefb1865375718c4a56ce608d96a4a8ed that marks Clean Random Access recovery points as keyframes.

[SuRGeoNix]

I tried reverting (fbb283cfefb1865375718c4a56ce608d96a4a8ed) but did not fix the issue. I did not notice any difference at all.

[Balling]

Such a problem in mpv. ​https://github.com/mpv-player/mpv/issues/9645

​https://github.com/mpv-player/mpv/issues/9639

[Balling]

Does this fix it? ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20190507032623.80375-1-ffmpeg@tmm1.net/

P.S. No, it does not (. should be changed to ->)

[Balling]

Also apparently the whole thing is so complex, it is crazy. ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220218232001.345826-5-u@pkh.me/
Patset was applied, did not help.

[Balling]

Another such patch touching the same code!!

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20190831101039.2179-2-fumoboy007@me.com/

There are some comments as to how seeking for a real keyframe should work: ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20190831101039.2179-1-fumoboy007@me.com/
The bug is that you need to continue searching for keyframes after finding one, because there may be one closer. In mp4 that all happens correctly.

[Carl Eugen Hoyos]

Is this issue only reproducible with ffplay or also with ffmpeg, the application?

[SuRGeoNix]

Maybe by trying exporting a specific frame (which will be a gray / broken as it will fail to seek on keyframe)

ffmpeg -ss 00:00:20 -analyzeduration 100000000000 -probesize 100000000000 -i "LG Chess 4K Demo.ts" -frames:v 1 out.jpg -loglevel debug

[Balling]

SuRGeoNix, I can confirm that command line produces bad image. I propose to use avif though to preserve 10 bit 420 samples losslessly (aside from top-left chroma siting flag, but that is just metadata): -crf 0 -cpu-used 0 out.avif

[Carl Eugen Hoyos]

Should be tested again once #4888 is fixed - for H.264 several development iterations were necessary.

[Balling]

This is a separate issue! It looks like the artifacts happen on the corners of 4 tiles! Because this is Ateme file with 4 tiles.

[Balling]

In other cases we do have separate issues for hevc and avc.

[Balling]

Still happens