Abnormal colorspace conversion of BGR -> YUV comparing the RGB variant

[natt]

Input file is bgr24. Output file is yuv420p. If I instruct ffmpeg to convert to rgb24 first (instead of bgr24->yuv420p directly), the colors are slightly different.

Test input, exact command lines, and console output attached.

[natt]

output of testcase executable

[Elon Musk]

Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here

[natt]

Replying to richardpl:

Whatever issue you are reporting I can not reproduce it.
rgb24(->bgr24)->yuv is fine here

The two conversions should be identical, right? Since rgb24<->bgr24 is lossless, bgr->rgb->yuv should produce the same output as bgr->yuv. If it doesn't, it's not working right.

ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
0ba2a7494930baaa64869462d54b85cc

ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw -filter format=rgb24 -sws_flags lanczos -f rawvideo -pix_fmt yuv420p md5:
d5bd2d94675b67d23eb2661ca462966d

[Carl Eugen Hoyos]

If there is an issue, is is reproducible with this command:

$ ./ffmpeg -i tests/lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24 -pix_fmt yuv420p -f crc -
ffmpeg version N-37568-g8162c6f Copyright (c) 2000-2012 the FFmpeg developers
  built on Feb  5 2012 16:31:32 with gcc 4.5.3
  configuration: --cc='/usr/local/gcc-4.5.3/bin/gcc -m32'
  libavutil      51. 37.100 / 51. 37.100
  libavcodec     54.  0.102 / 54.  0.102
  libavformat    54.  0.100 / 54.  0.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 61.100 /  2. 61.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
Input #0, image2, from 'tests/lena.pnm':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc
[buffer @ 0x8dd20a0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x8dd2360] auto-inserting filter 'auto-inserted scale 0' between the filter 'src' and the filter 'out'
[scale @ 0x8dd28e0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4
[buffer @ 0x8de20e0] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x8de1780] auto-inserting filter 'auto-inserted scale 1' between the filter 'Parsed_format_0' and the filter 'out'
[format @ 0x8de0a80] auto-inserting filter 'auto-inserted scale 2' between the filter 'src' and the filter 'Parsed_format_0'
[scale @ 0x8df3960] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4
[scale @ 0x8de0b40] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4
Output #0, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc
Output #1, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm -> rawvideo)
  Stream #0:0 -> #1:0 (ppm -> rawvideo)
Press [q] to stop, [?] for help
CRC=0x7db3d981
CRC=0x74eb2924
frame=    1 fps=  0 q=0.0 Lq=0.0 size=       0kB time=00:00:00.04 bitrate=   3.0kbits/s
video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%

[natt]

Replying to cehoyos:

If there is an issue, is is reproducible with this command:

Yes, I get exactly that.

>ffmpeg -i lena.pnm -pix_fmt yuv420p -f crc - -vf format=bgr24
-pix_fmt yuv420p -f crc -
ffmpeg version N-37541-g670229e Copyright (c) 2000-2012 the FFmpeg developers
  built on Feb  3 2012 20:07:47 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil      51. 37.100 / 51. 37.100
  libavcodec     54.  0.102 / 54.  0.102
  libavformat    54.  0.100 / 54.  0.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 61.100 /  2. 61.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, image2, from 'lena.pnm':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc
[buffer @ 003DA880] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01B9D2A0] auto-inserting filter 'auto-inserted scale 0' between th
e filter 'src' and the filter 'out'
[scale @ 01B9D680] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:yuv420p flags:0x4
[buffer @ 01B9DB20] w:256 h:256 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01C1CCA0] auto-inserting filter 'auto-inserted scale 1' between th
e filter 'Parsed_format_0' and the filter 'out'
[format @ 01C1BE40] auto-inserting filter 'auto-inserted scale 2' between the fi
lter 'src' and the filter 'Parsed_format_0'
[scale @ 01C1AAA0] w:256 h:256 fmt:rgb24 -> w:256 h:256 fmt:bgr24 flags:0x4
[scale @ 01C1BDE0] w:256 h:256 fmt:bgr24 -> w:256 h:256 fmt:yuv420p flags:0x4
Output #0, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31,
200 kb/s, 90k tbn, 25 tbc
Output #1, crc, to 'pipe:':
  Metadata:
    encoder         : Lavf54.0.100
    Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x256, q=2-31,
200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm -> rawvideo)
  Stream #0:0 -> #1:0 (ppm -> rawvideo)
Press [q] to stop, [?] for help
CRC=0x7db3d981
CRC=0x74eb2924
frame=    1 fps=  0 q=0.0 Lq=0.0 size=       0kB time=00:00:00.04 bitrate=   3.0
kbits/s
video:192kB audio:0kB global headers:0kB muxing overhead -99.992371%

[Roger Pack]

anybody have any clues on this one?

[fireattack]

So I assume pix_fmt routine is part of swscale?

The same problem doesn't seem to happen if format conversion is done in {{{colorspace}} using

ffmpeg -i reference.pnm -vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc - -vf format=bgr24,colorspace=iall=bt601-6-625:all=bt709:format=yuv420p -f crc -

[fireattack]

Just some additional info about the impact of this issue. It seems to affect quite a lot downstream software from my observation lately.

I've noticed this "color shifted to yellow" issue in many recording utilities and programs, a noble one being nVidia's ShadowPlay.

Hopefully someone could find the root cause and fix this long standing issue.

[gdgsdg123]

So... the problem can be concluded as -pix_fmt interfering with the format video filter?

Or will ​this be related?.. (one of the swscale hazards)

[fireattack]

They're not interfering anything, -pix_fmt, filter format all have the same discoloration bug as soon as you are converting bgr24->yuv (but NOT when rgb24->yuv, so you can work around by (losslessly) converting to rgb24 first).

format argument in colorspace filter (NOT format filter, two different things), however, does NOT have this issue.

Summary (I tested with additional bt601 to bt709 conversion, but the principle is the same):

-vf scale=out_color_matrix=bt709,format=yuv420p BAD
-vf scale=out_color_matrix=bt709 -pix_fmt yuv420p BAD (-pix_fmt is basically same as format filter from my testing. So you can replace one to the other below)

-vf colormatrix=bt601:bt709,format=yuv420p BAD

-vf colorspace=iall=bt601-6-625:all=bt709:format=yuv420p GOOD (this is using format option in colorspace filter)
-vf colorspace=iall=bt601-6-625:all=bt709,format=yuv420p BAD (this is to chain a format filter)
-vf zscale=matrix=709,format=yuv420p GOOD

[Balling]

This is fixed by -vf scale=flags=accurate_rnd (https://ffmpeg.org/ffmpeg-all.html#Scaler-Options), or more accurately accurate_rnd+whatever algorithm you want. WRONG documentation, BTW, you can use them together.

255, 255, 255 BGR value is converted to (as can be checked by ffplay -i file -vf extractplanes=y) YCbCr values 234, 127, 127 (instead of good 235, 128, 128). So it is just an off-by-one somewhere. And no, it is not out-of-gamut. I suppose the bug is indeed somewhere in second line about special "unscaled" converter, whatever that means (you can print those with +print_info):

[swscaler @ 00000275e7aa00c0] bicubic scaler, from bgr24 to yuv420p using MMXEXT
[swscaler @ 00000275e7aa00c0] using unscaled bgr24 -> yuv420p special converter

as there is no this second line with print_info+accurate_rnd.

What I am concerned with are JPEGs: they are also affected (?), need to check RGB one.

Also see duplicate #8056: bmp is affected too, since it uses BGR. (DPX can be affected too.)

I also clarified what is the bad one: bgr, not rgb.

P.S. -vf scale=out_range=pc is not affected.
P.P.S. If accurate_rnd is bit perfect, we just need to switch as default for only bgr formats. Simple as that.

[Balling]

This is fixed now in 255, 255, 255 color...

But other colors are still broken.

[Balling]

Actually still an issue outside of color white, lol

[MasterQuestionable]

͏    "-sws_flags +accurate_rnd" alike should work, unless there was some recent regression?
͏    Or perhaps passing in this manner somehow failed certain cases..?

[Balling]

-sws_flags +accurate_rnd

sws_flags does not work at all, stop inserting it everywhere, it has no affect. -vf scale=flags=accurate_rnd is not the real fix for this, even if it works fine. The better fix is conversion to rgb24 first if you go from bgr to yuv420p. The reason being accurate_rnd has effect on rgb to yuv420p too.

Also bmp is very much affected, since it uses bgr24.

[MasterQuestionable]

͏    "sws_flags" indeed worked, per:
͏    ​https://github.com/MasterInQuestion/attach/commit/e18e5cd26b9d27bacefa685ba70d0d41bc7d7cbc#diff-75a0130f1814ba26c23e3a6fc6ce5046620163061d392086bc55e2c13c145bf9L264

͏    Probably the "-sws_flags" was problematic..?
͏    ​https://trac.ffmpeg.org/ticket/10993
͏    .
͏    But to my experience it adequately worked.


͏    I know this is somewhat off-topic, but discussion right on the constrained wiki page...
͏    Likely troublesome.

͏    I believe the issue affects everything uses BGR: more than just BMP.


͏    I also tried to add a comment on:
͏    ​https://github.com/FFmpeg/FFmpeg/commit/3e064f52eb368a373ded6e3704fcf29f1db3ff12
͏    .
͏    But somehow failed with obscure error?

͏    I'm even unable to add comment on:
͏    ​https://github.com/FFmpeg/FFmpeg/commit/2f2f73dc3ae3fadbf040eb73f217213b1c99ff39
͏    ; that of my own...

͏    Seems to be sort of auth issue, as I tested in my own repository and succeeded.
͏    But per: ​https://api.github.com/repos/FFmpeg/FFmpeg/comments?per_page=100&page=4
͏    Commit comments should be permitted..?

͏    ----

​https://github.com/FFmpeg/FFmpeg/pull/390
͏    "An owner of this repository has limited the ability to comment to users that are collaborators on this repository."
͏    GitHub issue... The UI for commit comment didn't properly reflect.

[Balling]

Accurate_rnd does not work, that other flag may work.

Commit comments should be permitted..?

People on gothub disabled it after this ​https://github.com/FFmpeg/FFmpeg/commit/8d0fea81c736280ff19283b05cd0ad935d1e35c5

[MasterQuestionable]

͏    "accurate_rnd" passed in alike manner shall work? Per my observation in <colorspace>.
͏    Probably just failed this case..?

͏    Whatsoever would be another bug if you can confirm.


͏    This is the last commit comment indicated from the API:
͏    ​https://github.com/FFmpeg/FFmpeg/commit/7b8445f03d10faf7ed232e6201bf04ba73d980d7#commitcomment-123092236

͏    Rather interesting thing is:
͏    ​https://github.com/FFmpeg/FFmpeg/commit/eb0455d64690eed0068e5cb202f72ecdf899837c#commitcomment-122074348
͏    ; the auto-spam...

͏    Both succeed the mentioned event.

[Balling]

͏>"accurate_rnd" passed in alike manner shall work? Per my observation in <colorspace>. Probably just failed this case..?

Okay, apparently -sws_flags accurate_rnd only works if you use it after -i option. Good to know. Still it is more simple to use it inside vf scale.

Let's have some reference md5 here. I will convert the strange raw file ffmpeg -f rawvideo -video_size 256x256 -pix_fmt bgr24 -i testin.raw gfajfa.bmp and attach this file. This file has the same issue with

ffplay -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p

Let's have some reference md5 here:

ffmpeg -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p -f framemd5 -

says md5 34e64aa3a7268a6002381af53366ad09

Now convert bmp to png, this now has rgb24:
ffmpeg -i gfajfa.bmp anfca.png

and

ffmpeg -i anfca.png -vf scale=out_color_matrix=bt709,format=yuv420p -f framemd5 -

fef01c4f4721b3ab650088081ac6616d

yep, md5 is not the same, this is the bug. But if you add -sws_flags accurate_rnd after -i both md5 become

52c2faebb1c31fba70487f43887b46de.

Now we need to understand where is the problem is and which is closer to 52c2faebb1c31fba70487f43887b46de: 255, 255, 255 color problem is gone, so it is harder to tell where the issue is. Indeed, in older version from 2020 the md5 of png file to yuv420p frame is the same, but bmp to yuv420p differs in old and in new, so this issue is partially fixed. Let's compare the files between accurate_rnd and without (two files without) in hex editor.

INDEED, the accurate_rnd and ffmpeg -i anfca.png -vf scale=out_color_matrix=bt709,format=yuv420p filee.yuv are mostly the same (in the start of the file), yet ffmpeg -i gfajfa.bmp -vf scale=out_color_matrix=bt709,format=yuv420p file.yuv is not even close.

[fireattack]

I don't think even white color is fixed; without accurate_rnd, it still produces yellowish color.

ffmpeg240821.exe -v error -i white.bmp -vf scale=out_color_matrix=bt709,format=yuv420p -color_primaries 1 -color_trc 1 -colorspace 1  -qp 0 "output/white.bmp__ffmpeg240821.exe.mp4" -y
ffmpeg240821.exe -v error -i "output/white.bmp__ffmpeg240821.exe.mp4" "output/img_white.bmp__ffmpeg240821.exe.png" -y

(Note: I convert the image to a proper mp4 video with all the color-related tags first, and then grab the frame from it. Otherwise different versions of FFMPEG often have different implied color matrix/space/etc., and end up with different results.)

I can also confirm there is a hash difference bwtween older version (I use 4.2.1 to test) and current version when converting BGR to YUV (without accurate_rnd) as Balling observed. But perceptually the difference is so small they're visually the same (as in, not fixed at all.)

[Balling]

I don't think even white color is fixed

It is not fixed with your file, indeed, but with a white bmp file of different size it is fixed. Hilarious.

ffplay.exe -i bgr.bmp -vf scale=out_color_matrix=bt709,format=yuv420p

and use a color picker. Same for your file.

[Balling]

I was thinking about this issue, since dither cannot be removed in scale in this without BITDEPTH change transform (sws_dither none indeed has no effect, see Paul comment ​https://trac.ffmpeg.org/ticket/9407#comment:10), only in zscale (by default no dither) that is what causes this issue, I imagine. Off-by-one in colors must be dither. And again even the better result is still

"are mostly the same (in the start of the file), " in the end of the file it is different anyway, so that means it is dither.