mxf with DolbyE and channel_count != 02 is wrongly read by FFMpeg and cannot be remuxed

[Francesco Bucciantini]

Summary of the bug:

mxf files which carry DolbyE 5.1 + 2.0 audio and whose channel_count value in the mxf header is 08 instead of 02 are wrongly read by FFMpeg and therefore are remuxed incorrectly.

Expected behavior:
files are read and remuxed correctly

Actual behavior:
edit unit sync lost on stream 1, jumping from 0 to 1

How to reproduce:

% ffmpeg -i "video.mxf" -map 0:0 -map 0:1 -c:v copy -c:a copy -f mxf -y "output.mxf"

ffmpeg version n6.1-dev
built on 30-05-2023

Explanation:
DolbyE is generally carried as fake stereo pairs in an mxf container so that it can be passed through via SDI.
As such, the channel_count value inside the mxf header is supposed to be 02 to indicate the fake stereo pairs it's supposed to be spoofed as. In other words:

however some muxers like ommcp (the Omneon muxer, from version 7.x 'till version 9.5) write the actual number of channels of the DolbyE stream instead, so for a DolbyE 5.1 + 2.0 (where 5.1 is the surround and 2.0 is the stereo downmix), they wrote 08 (6 channels + 2 channels = 8 channels). In other words:

As such, instead of reading it as fake stereo pairs and remux it correctly:

FFMpeg thinks it's an 8 channel stream and fails to remux it, thus screwing up the whole audio stream:

so much so that the final remuxed file says PCM and it's undecodable:

General
Complete name : A:\MEDIA\temp\remux.mxf
Format : MXF
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 1.23 MiB
Duration : 20 ms
Overall bit rate : 517 Mb/s
Frame rate : 50.000 FPS
Encoded date : 0-00-00 00:00:00.000
Writing application : FFmpeg OP1a Muxer 59.35.100.0.0
Writing library : Lavf (mingw32) 59.35.100.0.0

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High 4:2:2 Intra@L5.2
Format settings, CABAC : No
Format settings, wrapping mode : Frame
Codec ID : 0D01030102106001-0401020201323001
Duration : 20 ms
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 50.000 FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Scan type : Progressive
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : HLG
Matrix coefficients : BT.2020 non-constant

Audio
ID : 3
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (AES)
Codec ID : 0D01030102060300
Duration : 20 ms
Bit rate mode : Constant
Bit rate : 9 216 kb/s
Channel(s) : 8 channels
Sampling rate : 48.0 kHz
Frame rate : 50.000 FPS (960 SPF)
Bit depth : 24 bits
Stream size : 22.5 KiB (2%)
Locked : Yes

Other #1
ID : 1-Material
Type : Time code
Format : MXF TC
Frame rate : 50.000 FPS
Time code of first frame : 10:00:00:00
Time code of last frame : 10:00:00:00
Time code settings : Material Package
Time code, stripped : Yes

Other #2
ID : 1-Source
Type : Time code
Format : MXF TC
Frame rate : 50.000 FPS
Time code of first frame : 10:00:00:00
Time code of last frame : 10:00:00:00
Time code settings : Source Package
Time code, stripped : Yes

instead of saying DolbyE 5.1 + 2.0 as it should:

General
Complete name : A:\MEDIA\temp\original.mxf
Format : MXF
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 1.54 MiB
Duration : 20 ms
Overall bit rate : 645 Mb/s
Frame rate : 50.000 FPS
Encoded date : 2023-05-30 13:58:23.372
Writing application : Omneon Inc. Omneon Media Subsystem 8.3.0.0.1
Writing library : Omneon Media Api (windows)

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High 4:2:2 Intra@L5.2
Format settings, CABAC : No
Format settings, wrapping mode : Frame
Codec ID : 0D01030102106001-0401020201323001
Duration : 20 ms
Maximum bit rate : 500 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 50.000 FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Scan type : Progressive
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : HLG
Matrix coefficients : BT.2020 non-constant
Delay_SystemScheme1 : 0

Audio
ID : 3
Format : Dolby E
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100-0402020203021C00
Duration : 20 ms
Bit rate : 1 152 kb/s
Channel(s) : 8 channels
Sampling rate : 48.0 kHz
Frame rate : 50.000 FPS (960 SPF)
Bit depth : 24 bits
Stream size : 2.81 KiB (0%)
Delay_SystemScheme1 : 0
Locked : Yes

Other #1
ID : 1-Material
Type : Time code
Format : MXF TC
Frame rate : 50.000 FPS
Time code of first frame : 10:00:00:00
Time code of last frame : 10:00:00:00
Time code settings : Material Package
Time code, stripped : Yes

Other #2
ID : 1-Source
Type : Time code
Format : MXF TC
Frame rate : 50.000 FPS
Time code of first frame : 00:00:00:00
Time code of last frame : 00:00:00:00
Time code settings : Source Package
Time code, stripped : Yes

Other #3
Type : Time code
Format : SMPTE TC
Muxing mode : System scheme 1
Time code of first frame : 00:00:00:00

Actual fix: the only current fix is to use a newer version of ommcp like 9.8 to remux the file and force the channel_count value to 02 (fake stereo pairs) and then use FFMpeg to open the file OR use gsar to change the hex value relative to the channel count inside the mxf container and THEN use FFMpeg. Both options are time consuming and really should be avoided.
This should be handled internally by FFMpeg itself instead of choking upon decoding.

[Francesco Bucciantini]

Channel Count value set to 02 (fake stereo pairs) - FFMpeg friendly

[Francesco Bucciantini]

Channel Count value set to 08 (actual number of channels in the DolbyE stream) - bad for FFMpeg

[Francesco Bucciantini]

Channel Count value set to 08 read by FFMpeg before choking

[Francesco Bucciantini]

Channel Count value set to 02 correctly read by FFMpeg

[Francesco Bucciantini]

Original file before being remuxed by FFMpeg (channel_count 08)

[Francesco Bucciantini]

Wrongly remuxed file by FFMpeg

[Tomas Härdin]

The file is broken first of all, and you indicate a way for users to fix their workflows by upgrading ommcp. I'd therefore argue that mxfdec is behaving correctly here and we should not be encouraging users to use broken MXF encoders. It is possible to patch these broken files using something like xxd.

The EditRate is 50/1 and the SourcePackage claims 48 kHz 8 channels 24-bit = 23040 bytes per EditUnit. The EditUnit actually contained in the file is 5760, consistent with 2 channels.

[Francesco Bucciantini]

Hi Tomas,
you're absolutely right, the file is indeed broken and has been wrongly muxed by ommcp, which is why I reached out to Harmonic last year at IBC and I managed to get it fixed on their end.
As results, the new Omneon Media API are creating correctly muxed files once again.
In other words, those who are paying for Harmonic's support and can update their version of the muxer, can easily remux those files and/or make sure they're correct.

Now, the reason why I brought this up is that we won't get rid of those broken files anytime soon, unfortunately.
The reason is that there are many companies using Omneon video server to record feeds via SDI and those hardware encoders can't have their firmware upgraded, so they're gonna be stuck forever producing faulty files.
Unfortunately I'm a victim of this and I've been receiving faulty files on a daily basis.
Luckily I'm also an Omneon user myself, so I can remux them with version 9.8 and call it a day, but at the same time I'm an Avisynth contributor and one of the developers of FFAStrans which stands for (FFMpeg Avisynth Transcoder) which is free and based on the same open source software I contribute to. This means that although I can easily deal with broken files, my very own users won't be able to, nor will the wider open source community.
I told them to use gsar to change the hex of the file which corresponds to the channel_count entry to bring it back from 08 to 02 before using FFMpeg, however that's not just time consuming but also a bit risky 'cause they can't blindly use it or script it.

The best solution would be to be able to manage those broken files within FFMpeg directly.
Would it be possible to introduce a cross check based on the EditUnit to actually make FFMpeg read the file as if it was really 2 channels instead of just blindly trusting the metadata?

[Tomas Härdin]

ProductVersion could potentially be used to identify files like this. For this specific file it is Omneon OmMedia.dll Release jenkins-ommedia_8.3.x.0-COMPILER=vs2010,PLATFORM=win32-10 (unknown),ex={0,1},rng={0,1,0},exPre but I imagine something more generic would be necessary. There's nothing in the metadata indicating Dolby-E I think, that has to be probed elsewhere.

The file has AvgBps = 144000 which works out to 3 bits/sample, which is obviously wrong but could be a thing to trigger on. What do actual 8 channel files output by Omneon look like?

It might be possible to use the fact that the audio packets are smaller than they should be, but this won't work for ClipWrapped files. Can Omneon output ClipWrapped at all?

[Francesco Bucciantini]

Omneon 1 track 8 channels PCM

[Francesco Bucciantini]

Omneon 8 tracks 1 channels PCM

[Francesco Bucciantini]

So, in that very case it was indeed ommedia_8.3, but that's not the only version to be faulty.
After 6.1 they introduced the regression, so 7.x and 8.x are all faulty, 'till you get to 9.x.

About the 8 channel files output by Omneon, no problem at all, I can provide one.
So, I've attached 10 frames of 25fps colorbars in XDCAM-50.
The first one is Omneon_1_track_8_channels.mxf so basically it's 1 video in MPEG-2 and 1 audio track with 8 channels inside, muxed as .mxf by Omneon.
The second one is Omneon_8_track_1_channel.mxf which is 1 video in MPEG-2 again but 8 audio tracks each one with 1 mono channel inside, muxed as .mxf by Omneon.

If you need any other sample, I can provide as many as you need, just let me know. :)

[Francesco Bucciantini]

Ok, I've attached the header dump of the Omneon_1_track_8_channels.mxf and indeed the channel count says 08 like in the faulty DolbyE version:

[ k = ChannelCount
  3d.07, l =     4 (0004) ]
       0  00 00 00 08                                        ....

however here's the average bps value:

[ k = AvgBps
  3d.09, l =     4 (0004) ]
       0  00 11 94 00                                        ....

while for the wrongly muxed one (the one with DolbyE muxed with channel count 08) we have:

 [ k = ChannelCount
  3d.07, l =     4 (0004) ]
       0  00 00 00 08                                        ....

  [ k = AvgBps
  3d.09, l =     4 (0004) ]
       0  00 02 32 80                                        ..2.

[Francesco Bucciantini]

Header Omneon 1 track 8 channels

[Francesco Bucciantini]

Header Omneon faulty DolbyE