Damaged MP4 rejected by ffmpeg, mkvtoolnix can read it fine

[PumaD]

Got a multi GiB file which is unfortunately damaged and all players I tested refuse to play. I almost deleted it but then had the idea to run it though mkvtoolnix. It managed to salvage it almost completely with just minor visual artifacts during playback.

Would be great if ffmpeg could do the same.

How to reproduce:

$ ffmpeg -i slightly-damaged.mp4 out.mp4
ffmpeg version N-115112-g39ce8a9695 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.1 (GCC) 20240417
  configuration: 
  libavutil      59. 17.100 / 59. 17.100
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.103 / 61.  3.103
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x609423feb840] Invalid sample size -1251017134
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x609423feb840] error reading header
[in#0 @ 0x609423feb580] Error opening input: Invalid data found when processing input
Error opening input file slightly-damaged.mp4.
Error opening input files: Invalid data found when processing input

Tested with ffmpeg 6.1.1 as well as git master from yesterday.

[PumaD]

I've uploaded a sample under the name slightly-damaged-refuses-to-play.mp4 to ​https://streams.videolan.org/upload/ since it's too big to attach here.

[MasterQuestionable]

͏    MKVToolNix..?
͏    ​https://gitlab.com/mbunkus/mkvtoolnix#1-introduction

͏    Sure it's MP4?
͏    What's your ​`mkvmerge` command? (if indeed MP4 input)

͏    FFmpeg much relies the filename extension: when ͏"-f" not specified.

[MasterQuestionable]

͏    Indeed MP4:
͏    ​https://streams.videolan.org/ffmpeg/incoming/11005/slightly-damaged-refuses-to-play.mp4 (~ 6.35 MiB)

͏    Note: NSFW

[PumaD]

Replying to MasterQuestionable:

͏    MKVToolNix..?
͏    ​https://gitlab.com/mbunkus/mkvtoolnix#1-introduction

Yes, that one. Sorry, I shouldn't have assumed everyone to be familiar with it.

͏    Sure it's MP4?

Your implication being that MKVToolNix can only read MKV and not MP4? That's not the case. It can only create MKV but read a very wide variety of input formats.

͏    What's your ​`mkvmerge` command? (if indeed MP4 input)

I used the GUI, but that only constructs a command line. A simple mkvmerge --output out.mkv slightly-damaged-refuses-to-play.mp4 works.

͏    FFmpeg much relies the filename extension: when ͏"-f" not specified.

Good to know, I wasn't aware of that. However, you already confirmed in your other comment that it's indeed MP4.

[MasterQuestionable]

͏    Thanks for feedback.
͏    I much figured out the cause. (though unable to provide fix now)

[James]

Should be fixed in 3146b77a7d314f55b8ec5d8ce6fda2c5db049a27.

[PumaD]

Confirmed fixed in master.

The full file reveals further problems which don't show up when remuxing with mkvtoolnix instead. I'll file separate follow up tickets, but for the record:

ffmpeg -i full_file.mp4 -c copy out.mkv

• File size grows by 224 MiB (from 1.9 GiB to 2.2 GiB)
• Video freezes during playback for half a second at 00:42:58
• Audio turns silent until the end of the file at 00:43:05

[MasterQuestionable]

͏    Would this be related?
͏    ​https://trac.ffmpeg.org/ticket/6037#comment:17

[Balling]

I'll file separate follow up tickets

We will check them!

[PumaD]

Replying to MasterQuestionable:

͏    Would this be related?
͏    ​https://trac.ffmpeg.org/ticket/6037#comment:17

No, there are no matroska @ warnings or errors, and all three problems also reproduce when remuxing to MP4 instead.

[MasterQuestionable]

͏    Would the similar effect be reproducible with the ​͏"slightly-damaged-refuses-to-play.mp4" (~ 6.35 MiB) sample?

[PumaD]

No, that's why I said "the full file reveals".

Filled #11030 for the file size increase, #11029 for the freezing video, and #11028 for the audio turning silent. The sample is a lot larger than I'd like but all my attempts to create a smaller one baked in the missing audio. At least all three tickets use the same sample.

[MasterQuestionable]

͏    Well, how was the ~ 6.35 MiB sample generated?
͏    At very least, try separating the audio/video mere remuxing.

͏    Also how did you create the ~ 1 GiB sample?

[PumaD]

Replying to MasterQuestionable:

͏    Well, how was the ~ 6.35 MiB sample generated?

As documented on https://ffmpeg.org/bugreports.html with:
dd if=full_video.mp4 of=slightly-damaged-refuses-to-play.mp4 bs=1024 count=6500

͏    At very least, try separating the audio/video mere remuxing.

Remuxing with mkvtoolnix repairs the file enough that the problem is gone. Remuxing with ffmpeg bakes the problem into the output such that even mkvtoolnix can't repair it anymore.

͏    Also how did you create the ~ 1 GiB sample?

Same as the other one, just with a target size of 1000 MiB, which is luckily enough to cover the spot where the audio goes out.
dd if=full_video.mp4 of=slightly-damaged-audio-stops-at-00-43-05.mp4 bs=1024 count=1024000

[MasterQuestionable]

͏    Would you generate a text file similar to ͏"slightly-damaged-refuses-to-play.mp4.txt" of the original?
͏    See [ ​https://trac.ffmpeg.org/ticket/11002#comment:9 ] for `exiftool`.

͏    See also: ​https://bugzilla.mozilla.org/attachment.cgi?id=9397319
͏    (maybe of use for debugging purpose)

[PumaD]

Replying to MasterQuestionable:

͏    Would you generate a text file similar to ͏"slightly-damaged-refuses-to-play.mp4.txt" of the original?

Attached to #11028
Edit: Oh, "of the original", my bad. Give me a minute.

͏    See [ ​https://trac.ffmpeg.org/ticket/11002#comment:9 ] for `exiftool`.

I've used exiftool 12.76 from the Arch repos. If a newer version is required, let me know.

͏    See also: ​https://bugzilla.mozilla.org/attachment.cgi?id=9397319
͏    (maybe of use for debugging purpose)

You're requesting the additional ffprobe output? Included.

[MasterQuestionable]

͏    The timestamp output for both audio/video shall also help debugging.
͏    You may upload the unprocessed version: I may filter later.
͏    [ The area of interest shall be near the 43:00 (~ 2580). ]

͏    It would be preferable if you update `exiftool` to latest (so less potentially diff noise).

͏    Note:
͏    ​͏"pastebin.com" shall be a better place for temp content.
͏    (optionally, set 6 months Expiration)

[PumaD]

Sorry, I'm not familiar. How do I get the timestamp output you're looking for?

[MasterQuestionable]

͏    E.g.
͏    ffprobe -v quiet -hide_banner -threads 0 -show_entries "frame=best_effort_timestamp_time" -select_streams v:0 -of "compact=nk=1:p=0" "tablet-tab-strip-3.webm"
͏    (as in linked Bugzilla attachment)

͏    "-select_streams a:0" to select the 1st encountered audio.
͏    See also <Seeking> on narrow-down the result. [ Which `ffprobe` doesn't yet seem to support. ]

[PumaD]

Oh, there was I second call to ffprobe in that attachment. Totally missed that, sorry. Updated exiftool to 12.85 as well.

[MasterQuestionable]

͏    You cut too much.
͏    42:58 = 2578 s
͏    The potential area of interest would be 2518 ~ 2638. (± 60 s) [1]
[ [1]
͏    Typical videos mostly won't use larger GOP (Group of Pictures).
͏    Mostly below 15 s. ]

͏    Try below and upload the full XML:
͏    ffprobe -v warning -hide_banner -threads 0 -show_entries "frame" -select_streams v:0 -of "xml" "full_video.mp4" -o "full_video.mp4.xml"

͏    Pastebin seems to lack file upload functionality in whose web interface.
͏    Use [ ​https://streams.videolan.org/upload/ ].

[PumaD]

Uploaded as full_video_timestamps.xml.zip under #11028. Hope zipping it wasn't wrong, the uncompressed XML is ~80 MiB.

[MasterQuestionable]

͏    ​https://streams.videolan.org/ffmpeg/incoming/11028/full_video_timestamps.xml.zip (~ 4.57 MiB: of ~ 79.79 MiB decompressed, ~ 5.73%)
͏    No matter.
͏    Actually you may rely on transport-level compression (e.g. HTTP compression) for this type.
͏    (Brotli shall do better job than Deflate of ZIP)

[MasterQuestionable]

͏    The video freeze near 42:58 shouldn't occur: at least per the frame presentation data.
͏    Unless they're similar-looking frames that felt frozen.

[PumaD]

Remuxing with mkvtoolnix fixes the freeze, otherwise I wouldn't have reported it. Not just similar-looking frames. There's even a bit of movement making it easy to see.

I noticed that ffplay shows these messages with the original file at the time of the freeze:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x712350000c80] Packet corrupt (stream = 1, dts = 113707010).
[aac @ 0x712350008240] channel element 2.0 is not allocated 
[aac @ 0x712350008240] channel element 2.0 is not allocated 
[aac @ 0x712350008240] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x712350008240] channel element 3.1 is not allocated
[h264 @ 0x7123505c8e00] concealing 1127 DC, 1127 AC, 1127 MV errors in P frame

Playing the file remuxed by mkvtoolnix only shows these messages:

[aac @ 0x74e870005480] channel element 2.0 is not allocated 
[aac @ 0x74e870005480] channel element 2.0 is not allocated 
[h264 @ 0x74e870324100] concealing 1127 DC, 1127 AC, 1127 MV errors in P frame

[MasterQuestionable]

͏    ​https://trac.ffmpeg.org/attachment/ticket/11028/full_video.mp4.txt#L188
͏    So it appears audio error may unnecessarily cause video freeze in certain contexts.