h264_nvenc / weighted_pred not working with B Frame parameter

[smallishzulu]

Hello,

If B-Frame is set with Weighted Prediction option, FFmpeg fails with:
InitializeEncoder failed: invalid param (8)

/opt/ffmpeg/bin/ffmpeg -loglevel verbose -re -i /home/admintv/big-buck-bunny_294_1280x720.mp4  -aspect 16:9 -s 720x576 -map 0:0  -c:v:0 h264_nvenc -temporal-aq 1 -spatial-aq 1 -aq-strength 15 -2pass 1 -preset slow  -cbr 1 -rc cbr  -profile:v main   -vb 1550k -minrate 1550k -maxrate 1550k -bufsize 1550k -muxrate 1790k    -flags +ilme+ildct -top 1  -r 25 -pix_fmt yuv420p -map 0:1 -c:a:0 aac  -b:a:0 128k -g 50 -f mpegts -bf 2 -weighted_pred 1 'udp://192.168.2.100:5000?pkt_size=188'                                ffmpeg version N-91062-gf995aa8 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --prefix=/opt/ffmpeg --enable-nonfree --enable-gpl --extra-cflags='-I/opt/ffmpeg/include -I/usr/local/include -I/usr/local/cuda/include -I/opt/ndi/sdk/include' --extra-ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/opt/ndi/sdk/lib' --bindir=/opt/ffmpeg/bin --extra-libs=-ldl --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-nvenc --enable-libzvbi --enable-libfdk-aac --enable-libzimg --enable-libzmq --enable-libfreetype --enable-static --enable-shared --enable-hardcoded-tables --enable-vdpau --enable-cuda --enable-cuvid --enable-libvpx --enable-libmp3lame --enable-libndi_newtek --enable-openssl --enable-ffnvcodec --enable-libfontconfig --enable-libfribidi --enable-cuda-sdk --enable-libnpp
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 19.101 / 58. 19.101
  libavformat    58. 13.102 / 58. 13.102
  libavdevice    58.  4.100 / 58.  4.100
  libavfilter     7. 22.100 /  7. 22.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
[h264 @ 0x2544740] Reinit context to 1280x720, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/admintv/big-buck-bunny_294_1280x720.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: mp41
    creation_time   : 1970-01-01T00:00:00.000000Z
    title           : Big Buck Bunny
    encoder         : Lavf52.13.0
  Duration: 00:09:56.46, start: 0.000000, bitrate: 2353 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 2220 kb/s, 24 fps, 24 tbr, 24 tbn, 48 tbc (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[h264 @ 0x25b2740] Reinit context to 1280x720, pix_fmt: yuv420p
[graph_1_in_0_1 @ 0x28a88c0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
[graph 0 input from stream 0:0 @ 0x2ec7b80] w:1280 h:720 pixfmt:yuv420p tb:1/24 fr:24/1 sar:1/1 sws_param:flags=2
[scaler_out_0_0 @ 0x2ec8e80] w:720 h:576 flags:'bicubic' interl:0
[scaler_out_0_0 @ 0x2ec8e80] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:720 h:576 fmt:yuv420p sar:64/45 flags:0x4
[h264_nvenc @ 0x256b800] Loaded Nvenc version 8.1
[h264_nvenc @ 0x256b800] Nvenc initialized successfully
[h264_nvenc @ 0x256b800] 1 CUDA capable devices found
[h264_nvenc @ 0x256b800] [ GPU #0 - < GeForce GTX 1080 > has Compute SM 6.1 ]
[h264_nvenc @ 0x256b800] supports NVENC
[h264_nvenc @ 0x256b800] AQ enabled.
[h264_nvenc @ 0x256b800] Temporal AQ enabled.
[h264_nvenc @ 0x256b800] InitializeEncoder failed: invalid param (8)
[h264_nvenc @ 0x256b800] Nvenc unloaded
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x256e200] Qavg: 188.848
[aac @ 0x256e200] 2 frames left in the queue on closing
Conversion failed!

[Carl Eugen Hoyos]

Why do you think this can be fixed in FFmpeg?

[smallishzulu]

Honestly, for tickets 7300,7301, 7302 & 7303. I opened tickets to NVIDIA side as well.
I opened these tickets here, so these may get attention of Timo and Philip.

I also think this can be a bug coming from implementation.
Afaik, we can use B-Frames with Weighted Prediction algo.

[smallishzulu]

Official NVIDIA reply over their ticket system.

Using weighted prediction along with B-frames is currently not supported. The relevant comment from the EncodeAPI header file is

uint32_t enableWeightedPrediction :1; /< [in]: Set this to 1 to enable weighted prediction. Not supported if encode session is configured for B-Frames( 'frameIntervalP' in NV_ENC_CONFIG is greater than 1).*/

, so the current behaviour is not a bug.

However, when I try the same command without -bf 2:

/opt/ffmpeg/bin/ffmpeg -loglevel verbose -re -i /home/admintv/big-buck-bunny_294_1280x720.mp4 -aspect 16:9 -s 720x576 -map 0:0 -c:v:0 h264_nvenc -temporal-aq 1 -spat ial-aq 1 -aq-strength 15 -2pass 1 -preset slow -cbr 1 -rc cbr -profile:v main -vb 1550k -minrate 1550k -maxrate 1550k -bufsize 1550k -muxrate 1790k -flags +ilme+ildct -top 1 -r 25 -pix_fmt yuv420p -map 0:1 -c:a:0 aac -b:a:0 128k -g 50 -f mpegts -weighted_pred 1 'udp://192.168.2.100:5000?pkt_size=188'
ffmpeg version N-91062-gf995aa8 Copyright (c) 2000-2018 the FFmpeg developers

built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
configuration: --prefix=/opt/ffmpeg --enable-nonfree --enable-gpl --extra-cflags='-I/opt/ffmpeg/include -I/usr/local/include -I/usr/local/cuda/include -I/opt/ndi/sdk/include' --extra-ldflags='-L/ opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/opt/ndi/sdk/lib' --bindir=/opt/ffmpeg/bin --extra-libs=-ldl --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-nvenc --enable-libzvbi --enable-libfdk-aac --enable-libzimg --enable-libzmq --enable-libfreetype --enable-static --enable-shared --enable-hardcoded-tables --enable-vdpau --enable-cuda --enable-cuvid --enable-libvpx --en able-libmp3lame --enable-libndi_newtek --enable-openssl --enable-ffnvcodec --enable-libfontconfig --enable-libfribidi --enable-cuda-sdk --enable-libnpp
libavutil 56. 18.102 / 56. 18.102
libavcodec 58. 19.101 / 58. 19.101
libavformat 58. 13.102 / 58. 13.102
libavdevice 58. 4.100 / 58. 4.100
libavfilter 7. 22.100 / 7. 22.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100

[h264 @ 0x228d680] Reinit context to 1280x720, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/admintv/big-buck-bunny_294_1280x720.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: mp41
creation_time : 1970-01-01T00:00:00.000000Z
title : Big Buck Bunny
encoder : Lavf52.13.0

Duration: 00:09:56.46, start: 0.000000, bitrate: 2353 kb/s

Stream #0:0(und): Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 2220 kb/s, 24 fps, 24 tbr, 24 tbn, 48 tbc (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler

Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler

Stream mapping:

Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))

Press [q] to stop, ? for help
[h264 @ 0x239cf80] Reinit context to 1280x720, pix_fmt: yuv420p
[graph_1_in_0_1 @ 0x25f1ac0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
[graph 0 input from stream 0:0 @ 0x2c10e00] w:1280 h:720 pixfmt:yuv420p tb:1/24 fr:24/1 sar:1/1 sws_param:flags=2
[scaler_out_0_0 @ 0x2c12180] w:720 h:576 flags:'bicubic' interl:0
[scaler_out_0_0 @ 0x2c12180] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:720 h:576 fmt:yuv420p sar:64/45 flags:0x4
[h264_nvenc @ 0x228f440] Loaded Nvenc version 8.1
[h264_nvenc @ 0x228f440] Nvenc initialized successfully
[h264_nvenc @ 0x228f440] 1 CUDA capable devices found
[h264_nvenc @ 0x228f440] [ GPU #0 - < GeForce GTX 1080 > has Compute SM 6.1 ]
[h264_nvenc @ 0x228f440] supports NVENC
[h264_nvenc @ 0x228f440] AQ enabled.
[h264_nvenc @ 0x228f440] Temporal AQ enabled.
[mpegts @ 0x22b6180] muxrate 1790000, pcr every 23 pkts, sdt every 595, pat/pmt every 119 pkts
Output #0, mpegts, to 'udp://192.168.2.100:5000?pkt_size=188':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: mp41
title : Big Buck Bunny
encoder : Lavf58.13.102
Stream #0:0(und): Video: h264 (h264_nvenc) (Main), 1 reference frame, yuv420p(top first, left), 720x576 [SAR 64:45 DAR 16:9], q=-1--1, 1550 kb/s, 25 fps, 90k tbn, 25 tbc (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler
encoder : Lavc58.19.101 h264_nvenc

Side data:

cpb: bitrate max/min/avg: 1550000/0/1550000 buffer size: 1550000 vbv_delay: -1

Stream #0:1(und): Audio: aac (LC), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler
encoder : Lavc58.19.101 aac

[h264_nvenc @ 0x228f440] Failed locking bitstream buffer: invalid param (8)its/s speed=0.479x
Video encoding failed

So waiting for their upcoming reply.

[Balling]

Now b frames are on by default, so should use -b 0.

[Balling]

Works now. [h264_nvenc @ 000001cc049cec00] InitializeEncoder failed: invalid param (8): Weighted Prediction not supported with B-frames.

is printed in unsupported configurations. -b 0 no longer works to disable B-frames. So have to use -preset p2 instead, as -preset p3 already enables B frames, see

​https://github.com/FFmpeg/nv-codec-headers/blob/9934f17316b66ce6de12f3b82203a298bc9351d8/include/ffnvcodec/nvEncodeAPI.h#L2136

enableWeightedPrediction :1; < [in]: Set this to 1 to enable weighted prediction. Not supported if encode session is configured for B-Frames (i.e. NV_ENC_CONFIG::frameIntervalP > 1 or preset >=P3 when tuningInfo = ::NV_ENC_TUNING_INFO_HIGH_QUALITY or tuningInfo = ::NV_ENC_TUNING_INFO_LOSSLESS. This is because preset >=p3 internally enables B frames when tuningInfo = ::NV_ENC_TUNING_INFO_HIGH_QUALITY or ::NV_ENC_TUNING_INFO_LOSSLESS).

ffmpeg -i ​https://trac.ffmpeg.org/raw-attachment/ticket/9573/test.mkv -aspect 16:9 -s 720x576 -map 0:0 -c:v:0 h264_nvenc -temporal-aq 1 -aq-strength 15 -2pass 1 -preset p2 -cbr 1 -rc cbr -profile:v main -vb 1550k -minrate 1550k -maxrate 1550k -bufsize 1550k -muxrate 1790k -r 25 -pix_fmt yuv420p -c:a:0 aac -g 50 -f mpegts -weighted_pred 1 cool.mp4