Output of av1_nvenc cannot be processed by nvdec

[cheppizzadrakon]

Summary of the bug:
Output of av1_nvenc cannot be processed by nvdec
Happens only for some files, but I don't know what causes this
How to reproduce:
Windows 10 19045.4170
nvidia-smi produces NVIDIA-SMI 551.76 | Driver Version: 551.76 | CUDA Version: 12.4
Encoding:

% ffmpeg -y -i in.mp4 -an -c:v av1_nvenc out.mp4
ffmpeg version N-114212-gf5441e441f-20240316 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (crosstool-NG 1.26.0.65_ecc5e41)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --enable-libdvdread --enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags='$FF_CFLAGS' --extra-cxxflags='$FF_CXXFLAGS' --extra-ldflags='$FF_LDFLAGS' --extra-ldexeflags='$FF_LDEXEFLAGS' --extra-libs='$FF_LIBS' --extra-version=20240316
  libavutil      59.  1.100 / 59.  1.100
  libavcodec     61.  1.101 / 61.  1.101
  libavformat    61.  0.100 / 61.  0.100
  libavdevice    61.  0.100 / 61.  0.100
  libavfilter    10.  0.100 / 10.  0.100
  libswscale      8.  0.100 /  8.  0.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    58.  0.100 / 58.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2018-03-09T19:22:17.000000Z
  Duration: 00:00:10.01, start: 0.000000, bitrate: 8464 kb/s
  Stream #0:0[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 155 kb/s (default)
      Metadata:
        creation_time   : 2018-03-09T19:22:17.000000Z
        handler_name    : Mainconcept MP4 Sound Media Handler
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 8326 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
      Metadata:
        creation_time   : 2018-03-09T19:22:17.000000Z
        handler_name    : Mainconcept MP4 Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : AVC Coding
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> av1 (av1_nvenc))
Press [q] to stop, [?] for help
Output #0, mp4, to 'out.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf61.0.100
  Stream #0:0(eng): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 29.97 fps, 30k tbn (default)
      Metadata:
        creation_time   : 2018-03-09T19:22:17.000000Z
        handler_name    : Mainconcept MP4 Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.1.101 av1_nvenc
      Side data:
        cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
[out#0/mp4 @ 0000022af96381c0] video:2363KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.062723%
frame=  299 fps=0.0 q=30.0 Lsize=    2365KiB time=00:00:09.97 bitrate=1941.9kbits/s speed=20.9x

Decoding:

% ffmpeg -hwaccel cuda -i out.mp4 -f null -
ffmpeg version N-114212-gf5441e441f-20240316 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (crosstool-NG 1.26.0.65_ecc5e41)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --enable-libdvdread --enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags='$FF_CFLAGS' --extra-cxxflags='$FF_CXXFLAGS' --extra-ldflags='$FF_LDFLAGS' --extra-ldexeflags='$FF_LDEXEFLAGS' --extra-libs='$FF_LIBS' --extra-version=20240316
  libavutil      59.  1.100 / 59.  1.100
  libavcodec     61.  1.101 / 61.  1.101
  libavformat    61.  0.100 / 61.  0.100
  libavdevice    61.  0.100 / 61.  0.100
  libavfilter    10.  0.100 / 10.  0.100
  libswscale      8.  0.100 /  8.  0.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    58.  0.100 / 58.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Lavf61.0.100
  Duration: 00:00:09.98, start: 0.000000, bitrate: 1941 kb/s
  Stream #0:0[0x1](eng): Video: av1 (libdav1d) (Main) (av01 / 0x31307661), yuv420p(tv, bt709, progressive), 1280x720, 1940 kb/s, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 30k tbn (default)
      Metadata:
        handler_name    : Mainconcept MP4 Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.1.101 av1_nvenc
Stream mapping:
  Stream #0:0 -> #0:0 (av1 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[av1 @ 000001dc324421c0] trailing_one_bit out of range: 0, but must be in [1,1].
[av1 @ 000001dc324421c0] Failed to read unit 1 (type 5).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] trailing_one_bit out of range: 0, but must be in [1,1].
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 5).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] Missing reference frame needed for show_existing_frame (frame_to_show_map_idx = 6).
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 3).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] trailing_one_bit out of range: 0, but must be in [1,1].
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 5).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] Missing reference frame needed for show_existing_frame (frame_to_show_map_idx = 3).
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 3).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] trailing_one_bit out of range: 0, but must be in [1,1].
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 5).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] Missing reference frame needed for show_existing_frame (frame_to_show_map_idx = 7).
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 3).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] trailing_one_bit out of range: 0, but must be in [1,1].
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 5).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] Missing reference frame needed for show_existing_frame (frame_to_show_map_idx = 2).
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 3).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] trailing_one_bit out of range: 0, but must be in [1,1].
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 5).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] Missing reference frame needed for show_existing_frame (frame_to_show_map_idx = 6).
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 3).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] trailing_one_bit out of range: 0, but must be in [1,1].
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 5).
[av1 @ 000001dc324421c0] Failed to read packet.

The errors continue like this, check log for full output. The end of the output is

[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[av1 @ 000001dc324421c0] Missing reference frame needed for show_existing_frame (frame_to_show_map_idx = 1).
[av1 @ 000001dc324421c0] Failed to read unit 0 (type 3).
[av1 @ 000001dc324421c0] Failed to read packet.
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Error submitting packet to decoder: Invalid data found when processing input
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Decode error rate 1 exceeds maximum 0.666667
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Task finished with error code: -1145393733 (Error number -1145393733 occurred)
[vist#0:0/av1 @ 000001dc3248fe00] [dec:av1 @ 000001dc32492280] Terminating thread with return code -1145393733 (Error number -1145393733 occurred)
[vost#0:0/wrapped_avframe @ 000001dc32491700] No filtered frames for output stream, trying to initialize anyway.
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Lavf61.0.100
  Stream #0:0(eng): Video: wrapped_avframe, yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 29.97 fps, 30k tbn (default)
      Metadata:
        handler_name    : Mainconcept MP4 Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.1.101 wrapped_avframe
[out#0/null @ 000001dc3245a940] video:0KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
[out#0/null @ 000001dc3245a940] Output file is empty, nothing was encoded(check -ss / -t / -frames parameters if used)
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=N/A bitrate=N/A speed=N/A
Conversion failed!

Software decoding has no issues

% ffmpeg -i out.mp4 -f null -
ffmpeg started on 2024-03-17 at 13:59:29
Report written to "ffmpeg-20240317-135929.log"
Log level: 48
ffmpeg version N-114212-gf5441e441f-20240316 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (crosstool-NG 1.26.0.65_ecc5e41)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --enable-libdvdread --enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags='$FF_CFLAGS' --extra-cxxflags='$FF_CXXFLAGS' --extra-ldflags='$FF_LDFLAGS' --extra-ldexeflags='$FF_LDEXEFLAGS' --extra-libs='$FF_LIBS' --extra-version=20240316
  libavutil      59.  1.100 / 59.  1.100
  libavcodec     61.  1.101 / 61.  1.101
  libavformat    61.  0.100 / 61.  0.100
  libavdevice    61.  0.100 / 61.  0.100
  libavfilter    10.  0.100 / 10.  0.100
  libswscale      8.  0.100 /  8.  0.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    58.  0.100 / 58.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Lavf61.0.100
  Duration: 00:00:09.98, start: 0.000000, bitrate: 1941 kb/s
  Stream #0:0[0x1](eng): Video: av1 (libdav1d) (Main) (av01 / 0x31307661), yuv420p(tv, bt709, progressive), 1280x720, 1940 kb/s, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 30k tbn (default)
      Metadata:
        handler_name    : Mainconcept MP4 Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.1.101 av1_nvenc
Stream mapping:
  Stream #0:0 -> #0:0 (av1 (libdav1d) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomav01iso2mp41
    encoder         : Lavf61.0.100
  Stream #0:0(eng): Video: wrapped_avframe, yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn (default)
      Metadata:
        handler_name    : Mainconcept MP4 Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.1.101 wrapped_avframe
[out#0/null @ 000001c7378a84c0] video:128KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame=  299 fps=0.0 q=-0.0 Lsize=N/A time=00:00:09.97 bitrate=N/A speed=28.6x

[cheppizzadrakon]

HW decode log

[cheppizzadrakon]

Encode log

[cheppizzadrakon]

SW decode log

[cheppizzadrakon]

Input file (cropped with -c copy for filesize limitations)

[cheppizzadrakon]

Output file (transcoded from in.mp4)

[cheppizzadrakon]

Output file (transcoded from in_crop.mp4)

[cheppizzadrakon]

I didn't know of the filesize limitation, in_crop.mp4 presents the same problem but the logs will refer to in.mp4
I have uploaded in.mp4 here ​https://pixeldrain.com/u/ZwKLnwhj, let me know if I need to change anything

[cheppizzadrakon]

As an additional note, it seems that reencoding the input file (i tested with standard libx264) is a workaround

% ffmpeg -y -i in.mp4 -an in_temp.mp4
% ffmpeg -y -i in_temp.mp4 -an -c:v av1_nvenc out.mp4

Decoding out.mp4 has no issues