FFV1 sometimes encodes invalid values for "ec" (CRC error detection/correction) header parameter field when using 2-pass encoding

[James Johnston]

Summary of the bug:

FFV1 sometimes encodes invalid values for the "ec" header parameter, which declares the CRC error detection/correction type. This results in the file failing validation in both MediaConch and MediaInfo.

How to reproduce:

So far, I have sometimes (but not always) reproduced this on the 2nd pass of a 2-pass encoding. A larger file I am working with seems to do it, but a small size file that I could attach to this issue does not. My guess is there is some kind of undefined behavior causing this.

The behavior occurs on FFmpeg 7.0.2, compiled for Windows 64-bit amd64 (downloaded from ​https://www.gyan.dev/ffmpeg/builds/ )

1. The command I used to encode the file:

# Input file is AVI container with HuffYUV video and PCM audio; probably many other inputs will work
ffmpeg -i input.avi -map 0 -vf setfield=tff -c:a flac -c:v ffv1 -level 3 -g 1 -context 1 -coder 2 -slicecrc 1 -passlogfile input.log -pass 1 -aspect 4:3 -pix_fmt yuv422p output.mkv

# Second pass
ffmpeg -i input.avi -map 0 -vf setfield=tff -c:a flac -c:v ffv1 -level 3 -g 1 -context 1 -coder 2 -slicecrc 1 -passlogfile input.log -pass 2 -aspect 4:3 -pix_fmt yuv422p output.mkv

2. Validate the file using MediaConch.

Expected behavior

FFV1 in MKV container, encoded by FFmpeg, should pass MediaConch validation with flying colors.

Actual behavior

Note that the file fails validation. A similar conformance error is flagged by the MediaInfo utility. The MediaConch validation error screenshot is attached; here is the text version:

FFV1-HEADER-ec Tests run: 1 | Results: ❌ | Fail count: 1
Results: fail ❌ 
Name: ec
Offset: 45503
Value context: /Segment[1]/Tracks[1]/TrackEntry[1]/CodecPrivate[1]/Private data[1]/ConfigurationRecord[1]
Value: 15

Upon further investigation, this validation failure is emitted by code like this: ​https://github.com/MediaArea/MediaInfoLib/blob/abdbb218b07f6cc0d4504c863ac5b42ecfab6fc6/Source/MediaInfo/Video/File_Ffv1.cpp#L1149-L1154

        if (ec>1)
        {
            Param_Error("FFV1-HEADER-ec:1");
            Element_End0();
            return;
        }

and documented here: ​https://github.com/MediaArea/MediaConch/blob/d2a4b62ffbaf5475c5c3e82b3e6fc9883a6be592/MetadataDevelopment/ImplementationChecks/FFV1Registry.xml#L115-L122

The FFV1 specification in section 4.2.16 at ​https://www.ietf.org/rfc/rfc9043.txt states that the "ec" field must be 1 or 0, and it "indicates the error detection/correction type." Other values are "reserved for future use."

Thus, it would appear that FFmpeg has likely written other values than 0 or 1 to the file, which isn't standard, and flagged by this rule.

I notice that FFmpeg doesn't have trouble decoding the file. That may be because the decoder code at ​https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/ffv1dec.c generally seems to assume any non-zero value means CRC is enabled. However, other decoder implementations might be more strict, and have trouble with this file. (And the FFmpeg decoder might display unpredictable values if other values for "ec" are added to the specification, unlikely as this may be.)

[James Johnston]

Screenshot of the validation failure

[James Johnston]

Small segment of file that has an invalid ec header

[James Johnston]

I attached an example MKV file that has the problem. It was created from a 65 GB file with the same problem, encoded using the 2-pass commands seen in the issue description, using the following command:

ffmpeg -t 0.25 -i input.mkv -c copy truncated.mkv

[James Johnston]

MediaConch is available online here: ​https://mediaarea.net/MediaConchOnline/checker -- upload a file and it will check it.

or available for download / offline use here: ​https://mediaarea.net/MediaConch (or from your favorite package manager)

[James Johnston]

I don't know if this is related to #11128 or not. Both only seem to happen when I used 2-pass encoding, but they also seem like very different validation failures that aren't related. So I've opened two issues.

[Michael Niedermayer]

The ec value is 1 in the file, you can see this by using:

./ffmpeg -debug 1  -i ~/tickets/11127/Example\ file\ with\ invalid\ ec\ header.mkv  2>&1 | grep 'ffv1 @'
[ffv1 @ 0x559dd34bb180] global: ver:3.4, coder:2, colorspace: 0 bpr:8 chroma:1(1:0), alpha:0 slices:2x2 qtabs:2 ec:1 intra:1 CRC:0x4333E8AE
[ffv1 @ 0x559dd34bb180] ver:3 keyframe:1 coder:2 ec:1 slices:4 bps:8
[ffv1 @ 0x559dd34bb180] slice 3, CRC: 0xEA3F94B1
[ffv1 @ 0x559dd34bb180] slice 2, CRC: 0x262733FF
[ffv1 @ 0x559dd34bb180] slice 1, CRC: 0xEBFDC675
[ffv1 @ 0x559dd34bb180] slice 0, CRC: 0x62607DF5

So i cannot help, ffmpeg decodes it fine and ffmpeg sees a ec value of 1
If some other code sees a value that is not 1, its really that other code that needs to be looked into. There must be a difference between it and FFmpeg as the same input produces different values.

[James Johnston]

If it's not likely to be an actual FFmpeg issue, then here is a corresponding issue I have raised in the MediaConch project: ​https://github.com/MediaArea/MediaConch_SourceCode/issues/787

There must be a bug in one of these two projects...