float pixel format

[Björn Sonnenschein]

Hello,
As there are more and more Cameras able to record with more than 8 bit per pixel and there already are programs like Shotcut and Blender which support high-precision internal processing, e.g. blender is unable to handle 10-bit videos properly, because converting a 16 bit buffer from the sws yuv to rgb conversion to its internal 32bit buffer would be too slow and therefore it has to use an 8 bit buffer.
If ffmpeg could convert to 32bit directly, there wouldn't be the need to do two conversion steps, 10 ->16->32 and native high precision video support would be much easier to implement!

[gjdfgh]

What kind of 32 bit? Integer? Float?

[Björn Sonnenschein]

Oh, sorry that I forgot to mention that blender uses float type buffer.

[Björn Sonnenschein]

(Double post)

[gjdfgh]

Float images would be useful for some other things too, but I bet nobody wants to fight libswscale to add conversion for this (which would probably be very non-trivial).

[Cigaes]

When people request float values for pixels (or samples), I always wonder if they really need the features of floats, i.e. more absolute accuracy for numbers near 0. It may make sense for samples, to keep the same relative accuracy regardless of the overall volume. For pixels, maybe issues with gamma will need more accuracy near 0, but I am not really convinced.

Apart from that, I wonder if people realize that 32-bits fixed point is more accurate than floats on 99.21875% of the range.

[Umair]

Hey,
Sorry for commenting on an almost 2 years old ticket.
But, is this still required ?
I'd love to start with this.

[Carl Eugen Hoyos]

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

[Umair]

Replying to cehoyos:

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

I went through some of the code present in utils.c file in libswscale.
I'm not really sure how difficult this feature is.

[Carl Eugen Hoyos]

Replying to omerjerk:

Replying to cehoyos:

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

I went through some of the code present in utils.c file in libswscale.
I'm not really sure how difficult this feature is.

Allow me to repeat my question:
Once you have mastered all difficulties and added the new pixel format to libavutil and libswscale, what usage would the new pix_fmt have within FFmpeg?

[Elon Musk]

To amuse people. like you.

[gjdfgh]

Certainly more usage than "important" formats like MONOWHITE.

[Carl Eugen Hoyos]

I created tickets #5302 and #5303 with floating point samples.

[emilyb]

We would like to have this too.

The use case is high bit depth processing in HDR mode where we want to be able to have super white/black (i.e. < 0.0 and > 1.0 values) at certain stages in the processing. Using floats is simpler to work with than 32bit integers with headroom and still has enough precision. We also do some advanced image processing which requires conversion to float at some point so having the conversion early is beneficial for performance for us.

As one person noted earlier floats have higher precision around 0.0 than 1.0,but in the range [0.5, 1.0] you still have 23 bits per component in precision which is enough.

[mtc]

i guess this means a component color space (RGB/XYZ based, not luminance chroma (YUV) )
RGB16float (16*3 48-bit) or
RGB32float (16*3 48-bit)

also using linear transfer characteristics
(not ​sRGB-gamma or ​HLG or Perceptual-Quantizer ...

floating with support for negative-values
to prevent data-loss (clamping-artifacts) from super-whites and outside of color-primaries'gamut colors
what can occur from conversion from YUV(luminance-chroma) color spaces

importing from exif2.0 sYCC (most modern camera jpegs) can clip overwhites esp blues

also if then thus also support outputting into float-supporting formats e.g. OpenEXR for export into supporting rendering programs

support for linearRGB heps make it easier to support linear-gamma(radiometric) image resizing

[mtc]

gimp 2.9 now supports floating-point RGB formats

[Elon Musk]

There are gray and gbr(a)p float formats now.