Application provided invalid, non monotonically increasing dts to muxer in stream 2: 1372390 >= 1372390 av_interleaved_write_frame(): Invalid argument

[julian]

download the problem file (still uploading, up in 1 hour, size 324.477.128 bytes):

​https://dl.dropbox.com/u/7221986/1.mkv

ffmpeg -i 1.mkv -map 0:0 -map 0:1 -map 0:2 -scodec mov_text -vcodec copy -acodec copy 1.mp4
ffmpeg version 1.1.1-tessus Copyright (c) 2000-2013 the FFmpeg developers
  built on Jan 20 2013 23:17:33 with llvm-gcc 4.2.1 (LLVM build 2336.1.00)
  configuration: --prefix=/Users/tessus/data/ext/ffmpeg/sw --as=yasm --extra-version=tessus --disable-shared --enable-static --disable-ffplay --disable-ffserver --enable-gpl --enable-pthreads --enable-postproc --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-libspeex --enable-bzlib --enable-zlib --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libxavs --enable-version3 --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvpx --enable-libgsm --enable-libopus --enable-fontconfig --enable-libfreetype --enable-libass --enable-filters --enable-runtime-cpudetect
  libavutil      52. 13.100 / 52. 13.100
  libavcodec     54. 86.100 / 54. 86.100
  libavformat    54. 59.106 / 54. 59.106
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 32.100 /  3. 32.100
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[matroska,webm @ 0x102006a00] Unknown entry 0x437E
    Last message repeated 11 times
[matroska,webm @ 0x102006a00] Unknown entry 0x63C9
    Last message repeated 1 times
Input #0, matroska,webm, from '1.mkv':
  Metadata:
    creation_time   : 2012-10-10 23:54:58
    TITLE           : No OP / ED
    TITLE-eng       : No OP / ED
  Duration: 00:23:39.88, start: 0.000000, bitrate: 1828 kb/s
    Chapter #0.0: start 0.000000, end 36.912000
    Metadata:
      title           : Intro
    Chapter #0.1: start 36.952000, end 126.877000
    Metadata:
      title           : Opening
    Chapter #0.2: start 126.917000, end 797.839000
    Metadata:
      title           : Part A
    Chapter #0.3: start 797.879000, end 801.843000
    Metadata:
      title           : Eyecatch
    Chapter #0.4: start 801.883000, end 1324.782000
    Metadata:
      title           : Part B
    Chapter #0.5: start 1324.822000, end 1414.830000
    Metadata:
      title           : Ending
    Chapter #0.6: start 1414.870000, end 1419.835000
    Metadata:
      title           : Next episode preview
    Stream #0:0: Video: h264 (High 10), yuv420p10le, 1280x720, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Metadata:
      title           : H.264
    Stream #0:1(jpn): Audio: aac, 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Japanese 2.0 AAC
    Stream #0:2(eng): Subtitle: ssa (default)
    Metadata:
      title           : English subtitles
Codec 0x18000 is not in the full list.
    Stream #0:3: Attachment: unknown_codec
    Metadata:
      filename        : QuattrocentoSans-BoldItalic.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:4: Attachment: unknown_codec
    Metadata:
      filename        : QuattrocentoSans-Bold.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:5: Attachment: unknown_codec
    Metadata:
      filename        : Jerry_B4s_handwriting.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:6: Attachment: unknown_codec
    Metadata:
      filename        : IwaOMinPro-Bd-Fate.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:7: Attachment: unknown_codec
    Metadata:
      filename        : DK+Crayon+Crumble.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:8: Attachment: unknown_codec
    Metadata:
      filename        : corbelz.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:9: Attachment: unknown_codec
    Metadata:
      filename        : angelina.TTF
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:10: Attachment: unknown_codec
    Metadata:
      filename        : akarinop.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:11: Attachment: unknown_codec
    Metadata:
      filename        : tamayura-handwriting.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:12: Attachment: unknown_codec
    Metadata:
      filename        : Snake.ttf
      mimetype        : application/x-truetype-font
Output #0, mp4, to '1.mp4':
  Metadata:
    TITLE-eng       : No OP / ED
    TITLE           : No OP / ED
    encoder         : Lavf54.59.106
    Chapter #0.0: start 0.000000, end 36.912000
    Metadata:
      title           : Intro
    Chapter #0.1: start 36.952000, end 126.877000
    Metadata:
      title           : Opening
    Chapter #0.2: start 126.917000, end 797.839000
    Metadata:
      title           : Part A
    Chapter #0.3: start 797.879000, end 801.843000
    Metadata:
      title           : Eyecatch
    Chapter #0.4: start 801.883000, end 1324.782000
    Metadata:
      title           : Part B
    Chapter #0.5: start 1324.822000, end 1414.830000
    Metadata:
      title           : Ending
    Chapter #0.6: start 1414.870000, end 1419.835000
    Metadata:
      title           : Next episode preview
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p10le, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 16k tbn, 1k tbc (default)
    Metadata:
      title           : H.264
    Stream #0:1(jpn): Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, stereo (default)
    Metadata:
      title           : Japanese 2.0 AAC
    Stream #0:2(eng): Subtitle: mov_text ([8][0][0][0] / 0x0008) (default)
    Metadata:
      title           : English subtitles
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (ass -> mov_text)
Press [q] to stop, [?] for help
Application provided invalid, non monotonically increasing dts to muxer in stream 2: 1372390 >= 1372390
av_interleaved_write_frame(): Invalid argument

[Carl Eugen Hoyos]

If there is a crash, please provide backtrace etc. as explained on ​http://ffmpeg.org/bugreports.html

[julian]

its not a real crash it just exits with code 01

[Carl Eugen Hoyos]

Partly unrelated: Such conversions need -fix_sub_duration

The following command line works fine:

$ ffmpeg -fix_sub_duration -i 1.mkv -acodec copy -vcodec copy -scodec mov_text -t 00:22:52 out.mp4

The following two commands also produce a working mp4 file:

$ ffmpeg -fix_sub_duration -i 1.mkv -codec copy -map 0 out.mkv
$ ffmpeg -fix_sub_duration -i out.mkv -scodec mov_text -acodec copy -vcodec copy out.mp4

But this fails:

$ ffmpeg -fix_sub_duration -i 1.mkv -scodec mov_text -an -vn -t 00:22:53 out.mp4
ffmpeg version N-49468-ga084884 Copyright (c) 2000-2013 the FFmpeg developers
  built on Jan 31 2013 00:34:25 with gcc 4.7 (SUSE Linux)
  configuration: --disable-indev=jack --disable-optimizations --disable-asm --disable-yasm
  libavutil      52. 17.100 / 52. 17.100
  libavcodec     54. 91.100 / 54. 91.100
  libavformat    54. 61.104 / 54. 61.104
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 34.101 /  3. 34.101
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
[matroska,webm @ 0x2ffbdc0] Unknown entry 0x437E
    Last message repeated 11 times
[matroska,webm @ 0x2ffbdc0] Unknown entry 0x63C9
    Last message repeated 1 times
Input #0, matroska,webm, from '1.mkv':
  Metadata:
    creation_time   : 2012-10-10 23:54:58
    TITLE           : No OP / ED
    TITLE-eng       : No OP / ED
  Duration: 00:23:39.88, start: 0.000000, bitrate: 1828 kb/s
    Chapter #0.0: start 0.000000, end 36.912000
    Metadata:
      title           : Intro
    Chapter #0.1: start 36.952000, end 126.877000
    Metadata:
      title           : Opening
    Chapter #0.2: start 126.917000, end 797.839000
    Metadata:
      title           : Part A
    Chapter #0.3: start 797.879000, end 801.843000
    Metadata:
      title           : Eyecatch
    Chapter #0.4: start 801.883000, end 1324.782000
    Metadata:
      title           : Part B
    Chapter #0.5: start 1324.822000, end 1414.830000
    Metadata:
      title           : Ending
    Chapter #0.6: start 1414.870000, end 1419.835000
    Metadata:
      title           : Next episode preview
    Stream #0:0: Video: h264 (High 10), yuv420p10le, 1280x720, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Metadata:
      title           : H.264
    Stream #0:1(jpn): Audio: aac, 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Japanese 2.0 AAC
    Stream #0:2(eng): Subtitle: ssa (default)
    Metadata:
      title           : English subtitles
Codec 0x18000 is not in the full list.
    Stream #0:3: Attachment: unknown_codec
    Metadata:
      filename        : QuattrocentoSans-BoldItalic.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:4: Attachment: unknown_codec
    Metadata:
      filename        : QuattrocentoSans-Bold.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:5: Attachment: unknown_codec
    Metadata:
      filename        : Jerry_B4s_handwriting.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:6: Attachment: unknown_codec
    Metadata:
      filename        : IwaOMinPro-Bd-Fate.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:7: Attachment: unknown_codec
    Metadata:
      filename        : DK+Crayon+Crumble.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:8: Attachment: unknown_codec
    Metadata:
      filename        : corbelz.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:9: Attachment: unknown_codec
    Metadata:
      filename        : angelina.TTF
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:10: Attachment: unknown_codec
    Metadata:
      filename        : akarinop.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:11: Attachment: unknown_codec
    Metadata:
      filename        : tamayura-handwriting.ttf
      mimetype        : application/x-truetype-font
Codec 0x18000 is not in the full list.
    Stream #0:12: Attachment: unknown_codec
    Metadata:
      filename        : Snake.ttf
      mimetype        : application/x-truetype-font
Output #0, mp4, to 'out.mp4':
  Metadata:
    TITLE-eng       : No OP / ED
    TITLE           : No OP / ED
    encoder         : Lavf54.61.104
    Chapter #0.0: start 0.000000, end 36.912000
    Metadata:
      title           : Intro
    Chapter #0.1: start 36.952000, end 126.877000
    Metadata:
      title           : Opening
    Chapter #0.2: start 126.917000, end 797.839000
    Metadata:
      title           : Part A
    Chapter #0.3: start 797.879000, end 801.843000
    Metadata:
      title           : Eyecatch
    Chapter #0.4: start 801.883000, end 1324.782000
    Metadata:
      title           : Part B
    Chapter #0.5: start 1324.822000, end 1414.830000
    Metadata:
      title           : Ending
    Chapter #0.6: start 1414.870000, end 1419.835000
    Metadata:
      title           : Next episode preview
    Stream #0:0(eng): Subtitle: mov_text ([8][0][0][0] / 0x0008) (default)
    Metadata:
      title           : English subtitles
Stream mapping:
  Stream #0:2 -> #0:0 (ass -> mov_text)
Press [q] to stop, [?] for help
[mp4 @ 0x313b820] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1372390 >= 1372390
av_interleaved_write_frame(): Invalid argument

[julian]

Partly unrelated: Such conversions need -fix_sub_duration

should fix_sub_duration always be used when converting MKV to MP4?

[Cigaes]

Can you extract and attach the subtitles track of your sample? It should be quite small enough to fit into the size limit.

My guess is that you have overlapping ASS events. Matroska+ASS can handle it, but probably not MP4+mov_text, and there is nothing much you can do about it, any conversion will be lossy.

-fix_sub_duration is for subtitles codecs that are terminated by an empty packet instead of having a duration. That is true for dvdsub (and possibly PGS) inside MPEG-TS, but never for anything inside Matroska.

[Carl Eugen Hoyos]

Replying to Cigaes:

-fix_sub_duration is for subtitles codecs that are terminated by an empty packet instead of having a duration. That is true for dvdsub (and possibly PGS) inside MPEG-TS, but never for anything inside Matroska.

For the given sample the following command line produces an output file that contains subtitles with (very) broken timing:
$ ffmpeg -i 1.mkv -strict -2 -qscale 2 -scodec mov_text -t 180 out.mp4
The output of the following command plays fine (the subtitles appear at the right moment):
$ ffmpeg -fix_sub_duration -i 1.mkv -strict -2 -qscale 2 -scodec mov_text -t 180 out.mp4

I uploaded the ass file.

[julian]

Matroska+ASS can handle it, but probably not MP4+mov_text, and there is nothing much you can do about it, any conversion will be lossy.

well i don't mind lossy conversion as long as it won't fail at all (for random input files)

the following command line produces an output file

it seems adding -t 180 prevents the crash from occurring, but of course the conversion only contains the first 3 minutes

[Carl Eugen Hoyos]

Replying to julian:

the following command line produces an output file

it seems adding -t 180 prevents the crash

Which crash?

[julian]

which crash

ok "premature exit", not crash ;)

[Cigaes]

Replying to cehoyos:

For the given sample the following command line produces an output file that contains subtitles with (very) broken timing:
$ ffmpeg -i 1.mkv -strict -2 -qscale 2 -scodec mov_text -t 180 out.mp4
The output of the following command plays fine (the subtitles appear at the right moment):
$ ffmpeg -fix_sub_duration -i 1.mkv -strict -2 -qscale 2 -scodec mov_text -t 180 out.mp4

Thanks for uploading the ASS file. What do you mean, "broken timing"? With your first command, I see the following output packets:

pts_time=177.420000
duration_time=1.980000

pts_time=179.400000
duration_time=0.440000

They correspond to the input ASS lines:

Dialogue: 10,0:02:57.42,0:02:59.40,Default,,0,0,0,,What do you have there?
Dialogue: 0,0:02:58.40,0:03:00.40,op trans,,0,0,0,,
Dialogue: 0,0:02:58.40,0:03:00.40,ed trans,,0,0,0,,
Dialogue: 10,0:02:59.84,0:03:01.86,Default,,0,0,0,,Heilige Wasser!

Note that the Default texts are in the layer 10 while the op/ed trans texts are in the layer 0: they are not supposed to interact.

With -fix_sub_duration, the timestamps become:

pts_time=177.420000
duration_time=0.980000

Notice how the duration has been reduced because of the next op trans text. This is not valid.

ASS can contain events that overlap any random way. My guess is that mov_text can not handle that. There is not much that can be done about it.

[Carl Eugen Hoyos]

Replying to Cigaes:

Replying to cehoyos:

For the given sample the following command line produces an output file that contains subtitles with (very) broken timing:
$ ffmpeg -i 1.mkv -strict -2 -qscale 2 -scodec mov_text -t 180 out.mp4
The output of the following command plays fine (the subtitles appear at the right moment):
$ ffmpeg -fix_sub_duration -i 1.mkv -strict -2 -qscale 2 -scodec mov_text -t 180 out.mp4

Thanks for uploading the ASS file. What do you mean, "broken timing"?

If I transcode the original file with above command line without -fix_sub_duration the subtitles in the resulting output file are approximately 75 seconds late (making the subtitles unusable), with -fix_sub_duration the subtitles in the output file look fine (tested with mplayer and mplayer -ass).

Please note that I originally did not upload the ass file because I am not convinced it allows to reproduce the original problem (see also my original comment that includes a working command line after remuxing the unchanged subtitle stream). I only uploaded it on your request.

[Cigaes]

Ok, I understand what is going on. There are bugs upon bugs.

The first bug, the one that causes the "non monotonically increasing" message, is this pair of lines:

Dialogue: 0,0:22:52.39,0:22:52.43,ed kara other,,0,0,0,fx,{\fad(0,0)(\pos(930,13)}nande dare mo wakatte kunnai no to
Dialogue: 0,0:22:52.39,0:22:52.43,ed trans other,,0,0,0,fx,{\fad(0,0)(\pos(830,713)}"Why doesn't anyone even try to understand me?"

(simultaneous transcription + traduction) because of this:

| + Block group
|  + Block (track number 3, 1 frame(s), timecode 1372.390s = 00:22:52.390)
|   + Frame with size 89
|  + Block duration: 40.000000ms
|+ Cluster
| + Cluster timecode: 1372.288s
| + Block group
|  + Block (track number 3, 1 frame(s), timecode 1372.390s = 00:22:52.390)
|   + Frame with size 104
|  + Block duration: 40.000000ms

In other words, due to some happenstance in the Matroska construction, two simultaneous ASS event packets are in different clusters, and matroskadec does not merge them in that case.

That can be considered a bug in the Matroska demuxer, but Clément and I are working on removing the merging of ASS packet altogether, so fixing it would probably be a waste of time (it does not look trivial).

Now, as for your experiences with -fix_sub_duration and shifted subtitles, the offending line is the first one here:

Dialogue: 10,0:00:36.93,0:02:06.89,Default,,0,0,0,,{OP}
Dialogue: 1,0:00:51.38,0:00:55.61,op kara,,0,0,10,,{\fs41\fnCorbel\b1\fad(200,200)\c&HA26310&\bord3\3c&HFFFFFF&}shibaraku {\c&H6203E2&}mi{\c&HA26310&}tsume atte kara

The first line generates an end packet at 2:06.89. Since that happens inside the muxer, lavf is not aware of it and does not take it into account for "non monotonic timestamps". The next line tries to generate a packet at 0:51.38, but that would go back in time, and therefore reaches the file in a bogus way. That is consistent with the ~75 seconds shift you are reporting.

There are several points that can be considered problems, but the crux of the story is that mov_text-in-MP4 is just not powerful enough to encode complex ASS scripts.

Using -fix_sub_duration will partially hide the problem by clamping the timestamps, but that will produce subtitles with wrong timestamps. Depending on the script, it may be a severe problem or completely invisible; it can not be considered a reliable solution and was not designed as such.

I do not think there is an automatic solution for the problem of converting overlapping subtitles into non-overlapping: deciding what lines can disappear quickly, what lines can be merged together, what lines can be ignored, etc., requires human analysis.

[julian]

Clément and I are working on removing the merging of ASS packet altogether

any ETA?

I do not think there is an automatic solution for the problem of converting overlapping subtitles into non-overlapping

i think the important thing here is that conversion never fails completely like it does now. if the resulting subtitles may be incorrect because the conversion can only happen "lossy", thats just something to accept.

so -fix_sub_duration is useful for MKV to MP4 after all?

[Cigaes]

Replying to julian:

i think the important thing here is that conversion never fails completely like it does now.

I totaly disagree with that statement. A program should NEVER EVER silently discard or downgrade data unless that was specifically requested by the user.

so -fix_sub_duration is useful for MKV to MP4 after all?

No. That is not made for that and may have useful effects only per chance.

[julian]

A program should NEVER EVER silently discard or downgrade data unless that was specifically requested by the user.

then how i can request that?

[Cigaes]

As I said, it is too complex to be automated in a generic way (and in some cases it will always be ugly); therefore there is no option that does the work.

I suggest you try to post on the ffmpeg-user mailing-list to explain exactly what you want to achieve.

[julian]

could someone give me a pointer how i'd add an option to fall back to just not using the subtitle if this problem occurs instead of exiting?

[Carl Eugen Hoyos]

I gave a working command line in comment:4 that allows to copy all relevant subtitles iirc, or do you mean "-sn"?

[julian]

i have an program specifically for easily converting MKVs to MP4s that uses ffmpeg to do the conversion.
of course i want the program to succeed for every possible input file, failing to copy some subtitles if they are problematic would be OK, completely failing to convert some files would not be OK.

a working command line in
$ ffmpeg -fix_sub_duration -i 1.mkv -acodec copy -vcodec copy -scodec mov_text -t 00:22:52 out.mp4

this one relies on specific knowledge about the problem before conversion...

$ ffmpeg -fix_sub_duration -i 1.mkv -codec copy -map 0 out.mkv
$ ffmpeg -fix_sub_duration -i out.mkv -scodec mov_text -acodec copy -vcodec copy out.mp4

this is a multipass solution and would be just as cumbersome to implement in my app as just detecting this type of failure and re-doing transcoding without the subtitles.

i'd be interested to add a flag -fallback_to_not_converting_subtitles_on_problems that would allow ffmpeg to succeed converting every input file and just drop the subtitles if there are problems. this would reduce cumbersome and error prone logic in my app to handle this problem. i just don't know where to start ;)

[julian]

this report is concerned with getting the message (Application provided invalid, non monotonically increasing dts to muxer in stream ) about the subtitlle stream.

i'm also seeing this message for the video stream, what does it mean?

[julian]

this report is concerned with getting the message (Application provided invalid, non monotonically increasing dts to muxer in stream ) about the subtitlle stream.

i'm also seeing this message for the video stream, what does it mean?

[julian]

i can't host the original MKV in the bugreport anymore, its here instead

​http://www.filedropper.com/1_104

[Carl Eugen Hoyos]

The original issue was fixed by Marton Balint in 31bb172b
Whoever considers it a bug that -fix_sub_duration is needed for this conversion, please open a new ticket, sample is in ​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket2212/