Support for Omneon AVC Intra 50/100 MXF OP1a files

[Thomas Mundt]

​http://free-vsts.com/files/MXF/AVCI100_Test.mxf
​http://free-vsts.com/files/MXF/AVCI50_Test.mxf
These AVC-Intra files are created with Omneon Media Port 7600 and cannot be decoded with ffmpeg.

ffmpeg version N-40517-ga548671 Copyright (c) 2000-2012 the FFmpeg developers
  built on May  9 2012 13:00:06 with gcc 4.6.3
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass --enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      51. 50.100 / 51. 50.100
  libavcodec     54. 21.100 / 54. 21.100
  libavformat    54.  4.100 / 54.  4.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 72.105 /  2. 72.105
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 11.100 /  0. 11.100
  libpostproc    52.  0.100 / 52.  0.100
[mxf @ 0225cf80] invalid KAGSize 0 - guessing 1
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[h264 @ 02351300] non-existing PPS referenced
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] non-existing PPS 0 referenced
[h264 @ 02351300] decode_slice_header error
[h264 @ 02351300] no frame!
[mxf @ 0225cf80] decoding for stream 0 failed
[mxf @ 0225cf80] Could not find codec parameters (Video: h264, 1920x1080)
[mxf @ 0225cf80] Estimating duration from bitrate, this may be inaccurate
Input #0, mxf, from 'C:\Dokumente und Einstellungen\a\Desktop\MXFs\AVCI100_Test.mxf':
  Metadata:
    timecode        : 16:48:22:05
  Duration: 00:00:40.00, start: 0.000000, bitrate: 9219 kb/s
    Stream #0:0: Video: h264, 1920x1080, 25 fps, 25 tbr, 25 tbn, 50 tbc
    Stream #0:1: Audio: pcm_s24le, 48000 Hz, 8 channels, s32, 9216 kb/s
At least one output file must be specified

[Carl Eugen Hoyos]

Probably related to ticket #524.

[Thomas Mundt]

I did some testing with the software XFConverter i got new at work. XFConverter wraps broadcast files into several containers. After rewrapping these files with the option "Store AVC-Intra header on each frame" set they can be decoded with ffmpeg.

AVC Intra 100 file after rewrapping:​http://free-vsts.com/files/MXF/AVCI100_Test_xf.mxf

In the documentation of the software it is explained that Panasonic AVC-Intra streams have the SPS and the PPS only on the first frame. The option "Store AVC-Intra header on each frame" is repeating the SPS and the PPS found on the first frame on each frame.

Could ffmpeg do the same trick, using the header from the first frame if the following frames doesn´t have one? The Panasonic AVC-Intra flavour has become standard for many broadcasters and is also used by AVID, Omneon and others...

[Carl Eugen Hoyos]

Maybe also related to ticket #1666.

[reimar]

Are you sure the first frame contains SPS/PPS? Because I think that should be enough to allow playing with FFmpeg as long as you don't seek.
However I suspect that they are instead stored in some place where FFmpeg doesn't find them, or maybe they are not stored at all.
In which case your rewrapped samples might provide us with the reference values we'd just have to copy-and-paste into the source and put into extradata for AVC-Intra streams...

[Thomas Mundt]

I´m not shure that the first frame contains the SPS and PPS.
But somehow XFConverter restores them :-/
If you need more rewrapped samples to get reference values of the Panasonic AVC-Intra header I can help.

[reimar]

So below patch will allow playing the AVCI100 file.
If you try searching for the SPS/PPS you will see there is nothing similar in the original file, so as I suspected the problem is not that SPS/PPS exists only for the first file but that it does not exist at all in the file.
However there are several issues
1) It does not work for AVCI50, so that one uses different SPS/PPS which I don't know.
2) I have no way to detect which files are actually AVC-Intra and which are normal files. Using this patch on normal files has a significant risk of breaking things. I also don't know how to properly distinguish AVCI50 and AVCI100.
3) This probably should be moved into common code, since the mov demuxer needs the same thing to play AVCI files.

diff --git a/libavformat/mxfdec.c b/libavformat/mxfdec.c
index 3ddeb99..d93910f 100644
--- a/libavformat/mxfdec.c
+++ b/libavformat/mxfdec.c
@@ -1487,6 +1489,27 @@ static int mxf_parse_structural_metadata(MXFContext *mxf)
         if (descriptor->extradata) {
             st->codec->extradata = descriptor->extradata;
             st->codec->extradata_size = descriptor->extradata_size;
+        } else {
+            static const uint8_t avc100extra[] = {
+                // SPS
+                0x00, 0x00, 0x00, 0x01, 0x67, 0x7a, 0x00, 0x29,
+                0xb6, 0xd4, 0x20, 0x22, 0x33, 0x19, 0xc6, 0x63,
+                0x23, 0x21, 0x01, 0x11, 0x98, 0xce, 0x33, 0x19,
+                0x18, 0x21, 0x03, 0x3a, 0x46, 0x65, 0x6a, 0x65,
+                0x24, 0xad, 0xe9, 0x12, 0x32, 0x14, 0x1a, 0x26,
+                0x34, 0xad, 0xa4, 0x41, 0x82, 0x23, 0x01, 0x50,
+                0x2b, 0x1a, 0x24, 0x69, 0x48, 0x30, 0x40, 0x2e,
+                0x11, 0x12, 0x08, 0xc6, 0x8c, 0x04, 0x41, 0x28,
+                0x4c, 0x34, 0xf0, 0x1e, 0x01, 0x13, 0xf2, 0xe0,
+                0x3c, 0x60, 0x20, 0x20, 0x28, 0x00, 0x00, 0x03,
+                // PPS
+                0x00, 0x08, 0x00, 0x00, 0x03, 0x01, 0x94, 0x20,
+                0x00, 0x00, 0x00, 0x01, 0x68, 0xce, 0x32, 0x48,
+                0x90
+            };
+            st->codec->extradata = av_mallocz(sizeof(avc100extra) + FF_INPUT_BUFFER_PADDING_SIZE);
+            memcpy(st->codec->extradata, avc100extra, sizeof(avc100extra));
+            st->codec->extradata_size = sizeof(avc100extra);
         }
         if (st->codec->codec_type == AVMEDIA_TYPE_VIDEO) {
             source_track->intra_only = mxf_is_intra_only(descriptor);

[Carl Eugen Hoyos]

Replying to reimar:

2) I have no way to detect which files are actually AVC-Intra and which are normal files. Using this patch on normal files has a significant risk of breaking things. I also don't know how to properly distinguish AVCI50 and AVCI100.

(In some mov cases, the codec_tag allows to differentiate.)

3) This probably should be moved into common code, since the mov demuxer needs the same thing to play AVCI files.

Do you suspect this or were you able to test successfully with the samples from ticket #524 and #1666?

[reimar]

I sent a patch based on that for #524, but #1666 is not the same format.

[Thomas Mundt]

Here you can download the rewrapped AVCI50 file to get the AVCI50 reference SPS/PPS:
​http://free-vsts.com/files/MXF/AVCI50_Test_xf.mxf

The files are 1080i50. I will generate and upload 720p50 files next week.

[reimar]

That file plays with a severely broken aspect, do you have any idea why that might be?
Since the MXF file contains the correct 16:9 I suspect the SPS/PPS are broken and have wrong values.
Is AVCI50 always 1440x1080 or is it possible the tool created incorrect headers for a 1920x1080 AVCI50 file?

[reimar]

Ok, that one is two bugs:
1) FFmpeg does not use the aspect encoded in the MXF file, fix for that sent
2) The SPS in that file has the SAR encoded as 3:4 when it should be 4:3, or in other words: that SPS is wrong.

[Thomas Mundt]

As far as I know, AVCI50 is always 1440x1080 and 960x720 in case of 720p.
It is scaled horizontal 3/4, so SAR should be 4:3 and DAR 16:9.
I hope this is the only fault in the header...

[Thomas Mundt]

The patch works! Also with other AVCI100/1080i files we´ve got on our server. Thanks for that!
Unfortunatly the colors look kind of inconsistant. Since this is also visible with the rewrapped files it could be a general problem of the ffmpeg h264 decoder!?

I´ve uploaded some AVCI 720p samples:
AVCI100 Omneon file: ​http://free-vsts.com/files/MXF/AVCI100_720p.mxf
AVCI100 rewrapped: ​http://free-vsts.com/files/MXF/AVCI100_720p_xf.mxf
AVCI50 Omneon file: ​http://free-vsts.com/files/MXF/AVCI50_720p.mxf
AVCI50 rewrapped: ​http://free-vsts.com/files/MXF/AVCI50_720p_xf.mxf

There seems to be a problem with the AVCI50 header again, but maybe you can find the bug.

Potentially the files can also help with ticket #1666 since it is progressive material. I cannot record in 1080p25 and the wrapper doesn´t accept AVCI in MOV-container.

[reimar]

I don't think I can see what you mean with the colors, but these are 10-bit formats, so there is a chance something is going wrong on the conversion to 8 bit.
Or actually, more likely the wrong colour transformation matrix is used. If you have a current MPlayer version around, you can try -vo gl:colorspace=0..3 to see what the different ones look like (probably lots of other programs that can do that as well, but my own code is easiest for me to remember how to use :-) ).
Due to the problem with the AVCI50 header I'd rather not want to use them directly, but instead properly generate the SPS/PPS. Unfortunately that's a bit more work and I don't know when I'll have time for that.

[Thomas Mundt]

Maybe inconsistant was the wrong word. It looks like some colors from previous frames are still there. It´s slight but visible especially at crossfaded scene changes.
I investigated a little and as I now know where to look for the SPS/PPS, I compared my samples with files from other sources.
AVID Transfermanager and Quantel headers have equal SPS/PPS but are slightly different to the file wrapped by the XFConverter.

When I compile FFmpeg with the AVID/Quantel SPS/PPS the color problems are gone!
So the XFConverter headers are inaccurate. The extracted header of the AVCI50/720p file is even completely wrong, since it has the same SPS/PPS as the AVCI100/720p file.

I´ve seen that you already build a patch at gmane.comp.video.ffmpeg.devel. I extended it a little to additionally support AVCI100/720p and AVCI50/1080i. Since I could not get other AVCI100/720p and AVCI50/1080i files with working header, I used the XFConverter SPS/PPS.
And I found an AVCI100/1080p AVID file with SPS/PPS that also works for the file from ticket #1666.

The patch is attached...

[reimar]

A few small comments:
1) If the SPS is now really correct, the code to set sample_aspect_ratio should not be necessary anymore for these cases (and it should be in a separate patch anyway since it has issues).
2) The "st->codec->codec_tag == MKTAG('a', 'i', '*', '2')" check makes no sense, there is no ai*2 entry in isom.c, so it will never be true. If you want a wildcard behaviour, use something like
uint32_t tmptag = codec_tag | MKTAG(0, 0, 0xff, 0);
if (tmptag == MKTAG('a', 'i', 0xff, '2') ...
But it would be nicer if field_order was set correctly in more than just the 1080p/1080i case.
3) You copied the wrong placement of the PPS comment from my code, the PPS is the part starting with 00 00 00 01 68, so it should be one line down.

I'd still find it nicer if we could generate those headers instead of hard-coding the values, but I admit it's probably too much effort to really be worth it.

[reimar]

About 2):
It would probably be nicer to avoid such hacks and instead do something along the lines of
switch (codec_tag) {
case MKTAG('a', 'i', '1', '2'):
case MKTAG('a', 'i', '5', '2'):
[any other progressive ones]
field_order = PROGRESSIVE;
break;
default:
break
}

[Thomas Mundt]

About 1) I think I´ve found the correct SPS now. But at all AVCI50/1080i samples the DAR is set to 3/4. To me the patch is also necessary because I often had material with wrong aspect ratio in the video header and correct ar in the mxf header.
But I removed it from the avci-patch.

About 2) Thanks for the hint! Im not a good programmer. Actually the patch works without that because the field order was set correct before in the mov-decoder. It´s only necessary for the mxf-decoder.

I´m not shure if the 1080p/25 SPS/PPS will also work for 1080p/50. I wanted to add a framerate check, but st->r_frame_rate didn´t work for mxf files. I will try to get some AVID 1080/p50 files to see if it is necessary at all.

btw: Since I can decode Panasonic AVCI now, is it possible to encode it with ffmpeg by the knowledge of the SPS/PPS?

[reimar]

For frame rate check: Try checking time_base
As to encoding: Figure what parameters the SPS/PPS set and then configure e.g. x264 with these settings and it should work (possibly manually discarding the SPS/PPS it creates).
That would be one more advantage of generating the SPS/PPS manually ourselves, it would serve as documentation of the exact settings that need to be used.

[Thomas Mundt]

st->time_base and st->codec->time_base don´t work. I always get 0.0 with av_q2d(st->time_base). But I think it´s not necessary to check for that since AVCI1080p50 seems not to be used or even does not exist.
Configuring x264 to get AVCI files working on broadcast systems is more complicated than I thought. Maybe I´ll try later when I have more spare time...

[Carl Eugen Hoyos]

The patch was committed, I wonder if there is a problem with the SPS/PPS that makes some samples appear with a codec aspect ratio of 3:4.