decoding yuv420p to rgb shifts luminance down

[burek]

Original post on the forum: ​http://ffmpeg.gusari.org/viewtopic.php?f=11&t=884

Summary of the bug:
"I'm recording video from a grayscale camera into an MPEG4 video file using FFMPEG 1.2. Recently I noticed a weird effect: if I decode the file using FFMPEG or windows media player, the output frames are noticeably darker (by about 10/12 brightness values) than the original source."

How to reproduce:

ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt gray gray.bmp

ffmpeg version 1.2 Copyright (c) 2000-2013 the FFmpeg developers
  built on Apr  4 2013 12:40:58 with gcc 4.6.2 (GCC)
  configuration: --enable-w32threads
  libavutil      52. 18.100 / 52. 18.100
  libavcodec     54. 92.100 / 54. 92.100
  libavformat    54. 63.104 / 54. 63.104
  libavdevice    54.  3.103 / 54.  3.103
  libavfilter     3. 42.103 /  3. 42.103
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
Input #0, avi, from 'video.avi':
  Metadata:
    encoder         : Lavf53.32.100
  Duration: 00:00:07.47, start: 0.000000, bitrate: 16532 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (FMP4 / 0x34504D46), yuv420p, 128
0x720 [SAR 1:1 DAR 16:9], 30 tbr, 30 tbn, 30 tbc
Output #0, image2, to 'gray.bmp':
  Metadata:
    encoder         : Lavf54.63.104
    Stream #0:0: Video: bmp, gray, 1280x680 [SAR 17:18 DAR 16:9], q=2-31, 200 kb
/s, 90k tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 -> bmp)
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.03 bitrate=N/A
video:851kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.002524%

"I thought the encoding step was doing this until I opened the same file in VLC and it gave me back the correct result. I played around with FFMPEG command lines to decode a single frame and realized that if I decode a frame into GRAY8 pixel format, the brightness/luminance values are preserved."

"This happens with every single grayscale video that I encode to MPEG4 with FFMPEG. My best guess so far is it has to do with how the container pixel format gets converted to/from. Since I'm using MPEG4, the file pixel format is YUV420P. I have no idea how ffmpeg encodes from GRAY8 to YUV420P, but maybe it stores just the luminance values in Y... if this happens, then decoding from this to RGB could produce darker pixels due to the shift factor that is applied to the luminance on decoding?

To sum it up:

1) How can encoding a grayscale video to YUV420P with FFMPEG and decoding back produce wrong (darker) brightness values when decoding to RGB versus GRAY8? Presumably once the frames are in YUV420P format it shouldn't matter whether the source is actually grayscale or not so the result should be equivalent, no?

2) How does VLC avoid this situation? I was under the impression that VLC used FFMPEG as well for video decoding, but somehow they managed to figure out how to produce the correct values without requiring me to indicate explicitly that the video was grayscale."

[Michael Niedermayer]

How can this be reproduced ?
I mean like "here is a file, when run with that command here and viewing the output file with that player it is showing this artifact compared to viewing this file with that player"
You say alot but piecing together from that how to reproduce the bug is not as easy as it may seem to you.

[glopes]

Sorry for that. You can download an example video from here:
​https://dl.dropboxusercontent.com/u/3907539/video.avi

Running the following commandline produces a bitmap with the correct luminance values:

ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt gray gray.bmp

Running the following commandline produces a bitmap in which the luminance values are shifted ~10 values darker:

ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt bgr24 rgb.bmp

Also playing this video file in VLC produces the correct luminance values.

The video was acquired from a grayscale camera and encoded as grayscale pixel format to mpeg4 yuv420p using FFMPEG (as you can see from the commandline output shown in the original ticket).

Hope this helps to clarify the issue.

[glopes]

Additional note: as of version 2.0.7 (at least) VLC is no longer able to correctly reproduce the grayscale brightness values.

[Carl Eugen Hoyos]

Is the problem still reproducible with current FFmpeg?

[glopes]

Thanks for asking, actually! I just tried with the latest build (ffmpeg version N-55159-gf118b41) and it turns out now it's not even possible to get the correct luminance values by using the first command line!!

I don't know what changed about pix_fmt gray decoding from yuv420p, but now things are even worse as I have currently no way of recovering the correct luminance from my recorded videos... data loss is now a serious possibility...

Last known working version is ffmpeg version N-51639-g7775992.

Here's the output from the two commands now:

gray pix_fmt:

ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt gray gray.bmp

ffmpeg version N-55159-gf118b41 Copyright (c) 2000-2013 the FFmpeg developers
  built on Aug  1 2013 18:08:30 with gcc 4.7.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp
e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena
ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l
ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp
eex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-
amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --
enable-libxvid --enable-zlib
  libavutil      52. 40.100 / 52. 40.100
  libavcodec     55. 19.100 / 55. 19.100
  libavformat    55. 12.102 / 55. 12.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 82.100 /  3. 82.100
  libswscale      2.  4.100 /  2.  4.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100
Input #0, avi, from 'video.avi':
  Metadata:
    encoder         : Lavf53.22.0
  Duration: 00:00:13.59, start: 0.000000, bitrate: 5596 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (FMP4 / 0x34504D46), yuv420p, 128
0x680 [SAR 1:1 DAR 32:17], 120 fps, 120 tbr, 120 tbn, 120 tbc
File 'gray.bmp' already exists. Overwrite ? [y/N] y
Output #0, image2, to 'gray.bmp':
  Metadata:
    encoder         : Lavf55.12.102
    Stream #0:0: Video: bmp, gray, 1280x680 [SAR 1:1 DAR 32:17], q=2-31, 200 kb/
s, 90k tbn, 120 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 -> bmp)
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=-1.0 Lsize=N/A time=00:00:00.00 bitrate=N/A
video:851kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.002524%

bgr24 pix_fmt:

ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt bgr24 rgb.bmp

ffmpeg version N-55159-gf118b41 Copyright (c) 2000-2013 the FFmpeg developers
  built on Aug  1 2013 18:08:30 with gcc 4.7.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp
e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena
ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l
ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp
eex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-
amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --
enable-libxvid --enable-zlib
  libavutil      52. 40.100 / 52. 40.100
  libavcodec     55. 19.100 / 55. 19.100
  libavformat    55. 12.102 / 55. 12.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 82.100 /  3. 82.100
  libswscale      2.  4.100 /  2.  4.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100
Input #0, avi, from 'video.avi':
  Metadata:
    encoder         : Lavf53.22.0
  Duration: 00:00:13.59, start: 0.000000, bitrate: 5596 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (FMP4 / 0x34504D46), yuv420p, 128
0x680 [SAR 1:1 DAR 32:17], 120 fps, 120 tbr, 120 tbn, 120 tbc
File 'rgb.bmp' already exists. Overwrite ? [y/N] y
Output #0, image2, to 'rgb.bmp':
  Metadata:
    encoder         : Lavf55.12.102
    Stream #0:0: Video: bmp, bgr24, 1280x680 [SAR 1:1 DAR 32:17], q=2-31, 200 kb
/s, 90k tbn, 120 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 -> bmp)
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=-1.0 Lsize=N/A time=00:00:00.00 bitrate=N/A
video:2550kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.000843
%

[Carl Eugen Hoyos]

How did you produce the input video?
Ticket #2684 was about incorrect encoding of gray input, this should now be fixed.

[glopes]

Thanks for the reference. I have to admit I'm not entirely sure I understand what the problem was with encoding gray input...

More importantly, did the fix change anything about the way gray format is decoded?

The video was produced using the AForge framework, which underneath uses some version of FFMPEG prior to 2012. I did not reencode the video with the new version, nor would I want to, since I have 1.5 years of these videos to analyze that just can't be reencoded...

In my case, it turns out I am absolutely sure that the gray encoding is actually correct, since I checked the frame grabber's pixel values myself in real-time while the data was coming through precisely to make sure that it wasn't an encoder problem, back when I found out about this.

This decoding issue now places me at risk of either losing data since the shift actually reduces precious contrast or having to stop updating FFMPEG from now on; neither of them I am particularly fond of...

[Carl Eugen Hoyos]

How did you check the mpeg4 frames encoded by FFmpeg, that sounds difficult?
(And I wonder if the conversion from gray input to yuv420p was correct.)

Please provide the command line including console output of the original encoding, that might help understanding the actual problem.

[Carl Eugen Hoyos]

If you look at the gray input: does it contain values from 0-255 or 16-235?

[glopes]

Yes, I didn't check the actual mpeg4 frames (sorry if I wasn't clear).

What I did check was the raw grayscale pixel intensities of the data coming in through the camera before it was encoded and then compared it to the pixel intensities of the reconstructed frames after decoding.

So if the previous gray encoder/decoder pair was buggy it's interesting why they allowed me to reconstruct the pixel intensities perfectly in the first place.

As to the last question, I just checked and I can find values ranging from 8-255 without much effort. Not sure if there's absolute zeros in there because of lighting conditions, but it definitely looks like I'm full range (which doesn't happen with this current version, where pixels everywhere from [8-14] get squashed to 0).

Don't have access to the setup right now, will try to make do. If I manage to reproduce anything meaningful about encoder differences, I'll let you know.
I'm still learning about pixel format specifications, but I guess what I mean to say is that somehow the bytes that were encoded in the file contain information about the correct pixel intensities. I just want to make sure I can read them back...

[Carl Eugen Hoyos]

Could you confirm that the following patch (against current git head) "fixes" decoding of your samples?

diff --git a/libavcodec/h263dec.c b/libavcodec/h263dec.c
index bf9e072..85ca4b2 100644
--- a/libavcodec/h263dec.c
+++ b/libavcodec/h263dec.c
@@ -770,7 +770,7 @@ const enum AVPixelFormat ff_h263_hwaccel_pixfmt_list_420[] = {
 #if CONFIG_VDPAU
     AV_PIX_FMT_VDPAU,
 #endif
-    AV_PIX_FMT_YUV420P,
+    AV_PIX_FMT_YUVJ420P,
     AV_PIX_FMT_NONE
 };

[Elon Musk]

Provide way to reproduce this bug.
Where is input sample?

[Carl Eugen Hoyos]

Replying to richardpl:

Where is input sample?

​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket2452/

[glopes]

I'm really sorry for not being able to retry the example with the new modified FFmpeg sources, but to say that there is no input sample is not true!

It's in the first comment, I'll repost below. If someone could please just run this video through the patch would be super helpful...

​https://dl.dropboxusercontent.com/u/3907539/video.avi

Running the following commandline produces a bitmap with the correct luminance values:
ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt gray gray.bmp
Running the following commandline produces a bitmap in which the luminance values are shifted ~10 values darker:
ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt bgr24 rgb.bmp
Also playing this video file in VLC produces the correct luminance values.
The video was acquired from a grayscale camera and encoded as grayscale pixel format to mpeg4 yuv420p using FFMPEG (as you can see from the commandline output shown in the original ticket).
Hope this helps to clarify the issue.

I can't seem to modify the original description of the ticket or else I would include it in there...

[Elon Musk]

Video indeed use full color range.

Workaround is to use '-vf extractplanes=y'

[Carl Eugen Hoyos]

Replying to glopes:

I'm really sorry for not being able to retry the example with the new modified FFmpeg sources

Could you test if I am correct that the patch from comment:11 fixes the issue?

[Michael Niedermayer]

The input file is correctly decoded, the fault is in what / how created the input file. I would guess that the wrong pixel format was used mixing jpeg and mpeg luma ranges

[glopes]

Replying to michael:

The input file is correctly decoded, the fault is in what / how created the input file. I would guess that the wrong pixel format was used mixing jpeg and mpeg luma ranges

Well, in regards to "what" created the input file, it was FFmpeg itself, so if there is anything wrong in the encoder it is a bug in FFmpeg as well.

Also, ffmpeg version N-51639-g7775992 was correctly decoding the file, so clearly something changed, would at least be helpful to know what.

Did anyone try the patch in comment:11?

I'll try encoding/decoding grayscale in the latest ffmpeg and report what I find there.

[Carl Eugen Hoyos]

Replying to glopes:

Replying to michael:

The input file is correctly decoded, the fault is in what / how created the input file. I would guess that the wrong pixel format was used mixing jpeg and mpeg luma ranges

Well, in regards to "what" created the input file, it was FFmpeg itself, so if there is anything wrong in the encoder it is a bug in FFmpeg as well.

Also, ffmpeg version N-51639-g7775992 was correctly decoding the file, so clearly something changed, would at least be helpful to know what.

(7775992 is from April 2013.)

FFmpeg behaviour wrt to gray was changed in 1ba01d3d46e8c200a2527ebafc71d5651084586b in July 2013, see ticket #2684. To get the old behaviour, either set input color_range or use the extractplanes filter. The patch in comment:11 is another option.

[Balling]

Hilarious. ffplay video.avi -vf extractplanes=y shows that there are 255 pixels there in Y that means IT CANNOT BE limited range. It must be full range, since in limited YCbCr (and rgb BTW too) [8 bit] 0 and 255 are reserved in all components and should not be used.

I also do not understand how is 1ba01d3d46e8c200a2527ebafc71d5651084586b affecting this since it is SAYING the opposite.

Yet, I do not get it: this is not a bug since range is NOT TAGGED and thus there is no way to say what it should be besides direct bruteforce for reserved values (and even that is not a guarantee, since this is just HDMI requirement, MPEG-4 permits that) or checking for Cb + Cr planes being achromatic (128 that is for 8 bit, 512 for 10 bit). Limited is the default here, since it is not really gray but yuv420p.

A good workaround is ffplay -v debug video.avi -vf zscale=rangein=full

Yet I did not find a way to tag MPEG-4 Visual as full range.