Reencoding dvbsub fails

[Carl Eugen Hoyos]

I tested two samples:
​http://samples.ffmpeg.org/MPEG-VOB/ClosedCaptions/Starship_Troopers.vob
​http://samples.ffmpeg.org/MPEG-VOB/transport/dvbsub/dvbsubtest.ts
and tried to reencode them to dvbsub. This works fine for the dvdsub sample, but fails for the dvbsub (input) sample.

$ ffmpeg -i dvbsubtest.ts -qscale 2 -mbd 2 -scodec dvbsub out.ts
ffmpeg version N-47774-g26db653 Copyright (c) 2000-2012 the FFmpeg developers
  built on Dec 12 2012 21:57:33 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 12.100 / 52. 12.100
  libavcodec     54. 79.102 / 54. 79.102
  libavformat    54. 49.101 / 54. 49.101
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 26.102 /  3. 26.102
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x2f23280] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x2f23280] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.49.101
    Stream #0:0(eng): Video: mpeg2video (hq), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
  Stream #0:2 -> #0:2 (dvbsub -> dvbsub)
Press [q] to stop, [?] for help
frame=  310 fps=0.0 q=2.0 size=       0kB time=00:00:12.32 bitrate=   0.0kbits/s dup=10 drop=0Application provided invalid, non monotonically increasing dts to muxer in stream 2: 3005830 >= 1670230
av_interleaved_write_frame(): Invalid argument

Remuxing all strams fails (remuxing only audio and video works fine):

$ ffmpeg -i dvbsubtest.ts -acodec copy -vcodec copy -scodec dvbsub out.ts
ffmpeg version N-47774-g26db653 Copyright (c) 2000-2012 the FFmpeg developers
  built on Dec 12 2012 21:57:33 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 12.100 / 52. 12.100
  libavcodec     54. 79.102 / 54. 79.102
  libavformat    54. 49.101 / 54. 49.101
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 26.102 /  3. 26.102
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x1b92280] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x1b92280] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.49.101
    Stream #0:0(eng): Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 25 fps, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, 256 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (dvbsub -> dvbsub)
Press [q] to stop, [?] for help
[mpegts @ 0x1b9a2e0] Application provided invalid, non monotonically increasing dts to muxer in stream 2: 3005830 >= 1670230
av_interleaved_write_frame(): Invalid argument

Remuxing only the subtitles work, but the resulting file shows no subtitles:

$ ffmpeg -i dvbsubtest.ts -qscale 2 -mbd 2 -scodec copy out.ts
ffmpeg version N-47774-g26db653 Copyright (c) 2000-2012 the FFmpeg developers
  built on Dec 12 2012 21:57:33 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 12.100 / 52. 12.100
  libavcodec     54. 79.102 / 54. 79.102
  libavformat    54. 49.101 / 54. 49.101
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 26.102 /  3. 26.102
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x2ae8280] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x2ae8280] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.49.101
    Stream #0:0(eng): Video: mpeg2video (hq), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
  Stream #0:2 -> #0:2 (copy)
Press [q] to stop, [?] for help
frame=  311 fps=0.0 q=2.0 size=    5961kB time=00:00:13.09 bitrate=3728.0kbits/s dup=10 drop=0frame=  570 fps=569 q=2.0 size=   14614kB time=00:00:23.13 bitrate=5173.9kbits/s dup=10 drop=0frame=  821 fps=546 q=2.0 size=   25327kB time=00:00:32.76 bitrate=6333.3kbits/s dup=10 drop=0PES packet size mismatch
[mpeg2video @ 0x2aec140] invalid mb type in P Frame at 43 16
[mpeg2video @ 0x2aec140] Warning MVs not available
[mpeg2video @ 0x2aec140] concealing 900 DC, 900 AC, 900 MV errors in P frame
[mp2 @ 0x2aecc20] incomplete frame
Error while decoding stream #0:1: Invalid data found when processing input
frame=  866 fps=538 q=2.0 Lsize=   28381kB time=00:00:34.67 bitrate=6704.4kbits/s dup=12 drop=0
video:25613kB audio:535kB subtitle:50 global headers:0kB muxing overhead 8.332572%

[Cigaes]

For the dvbdub → dvbsub case, the input sample is strange, at it has two consecutive subtitles packets with data in them. Normally, dvbsub comes with a packet of data, then an empty packet to clear it.

ffmpeg is still not completely prepared to work with the various kind of subtitles packets, and it is a tricky API design question.

[Carl Eugen Hoyos]

Fails now slightly differently:

$ ffmpeg -i dvbsubtest.ts -qscale 2 -mbd 2 -scodec dvbsub out.ts
ffmpeg version N-48478-g6a9af92 Copyright (c) 2000-2013 the FFmpeg developers
  built on Jan  5 2013 14:56:18 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 13.100 / 52. 13.100
  libavcodec     54. 85.101 / 54. 85.101
  libavformat    54. 59.103 / 54. 59.103
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 32.100 /  3. 32.100
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x226e880] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x226e880] PES packet size mismatch
Input #0, mpegts, from 'dvbsubtest.ts':
  Duration: 00:00:34.60, start: 79677.098467, bitrate: 4848 kb/s
  Program 1
    Stream #0:0[0x1901](eng): Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x19a1](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x19b1](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.59.103
    Stream #0:0(eng): Video: mpeg2video (hq), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1(eng): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
    Stream #0:2(eng): Subtitle: dvb_subtitle
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
  Stream #0:2 -> #0:2 (dvbsub -> dvbsub)
Press [q] to stop, [?] for help
frame=  277 fps=0.0 q=2.0 size=       0kB time=00:00:11.00 bitrate=   0.0kbits/s dup=10 drop=0Application provided invalid, non monotonically increasing dts to muxer in stream 2: 3005830 >= 1670230
av_interleaved_write_frame(): Invalid argument

[Clément Bœsch]

Another sample with a similar issue in #2989. Probably affects any DVB sub sample.

[Wim Vander Schelden]

I've worked out what the problem is (with the help of nevcariel, ubitux and others on #ffmpeg-devel):

• DVB subtitles are encoded as a series of rectangular bitmaps, and they are individually "flushed" to be displayed using a display segment packet.
• The way the DVB subtitles decoder (avcodec/dvbsubdec.c) currently works, is that it keeps track of all the other segments (which define regions, colour lookup tables, and contain the actual bitmaps) until a display segment is encountered, at which point it renders all these things to an AVSubtitle.
• On the encoding side, DVB subtitles differ from other subtitle codecs because it is currently the only codec that needs to have two packets to do its work: one to render the subtitle and one to indicate that it can be removed from the screen. These have (obviously) different PTS'es.
• It's possible that a later AVSubtitle appears before an earlier one dissappears from the screen. In fact, it's quite common for live subtitling, where each word is pushed & displayed as a separate region (have a look at ​http://samples.ffmpeg.org/MPEG-VOB/transport/dvbsub/dvbsubtest.ts for an example of this happening).
• Because the current subtitle encoding happens to write out the display & clear packets for DVB in sequence, a later subtitle can trigger the above error in the muxer.

So this is the problem. As for the solution: I don't have a clue on how to fix this cleanly. There's already quite a bit of DVB subtitles specific code in ffmpeg.c, and I'd rather remove from it than add to it, fixing the issue in the decoder.

I'd love to get this fixed, so I'm hoping that someone has an idea how this could be solved.

[Wim Vander Schelden]

Hi,

I modified ffmpeg to queue the clear packet of DVB subtitles instead of writing it out. I'm not sure if this is acceptable, and I'm pretty sure the code is not upto the required standards.

I'd like some feedback on wether this is an acceptable solution or what I have to do to get this merged.

[Wim Vander Schelden]

A patch that adds queueing to OutputStream to sidestep this bug.

[Carl Eugen Hoyos]

Please send patches to the ffmpeg-devel mailing list where they get more attention.

[Wim Vander Schelden]

Replying to cehoyos:

Please send patches to the ffmpeg-devel mailing list where they get more attention.

I have, I just used the wrong email, so it's awaiting moderation.

[Julian Gardner]

Can I add something in so people will understand how DVBSUBS work.

Dvb subs works on an idea of regions and objects, there are NO CLEAR SUBS commands, what is sent is a new region list with 0 regions, there should be no code to explicitly send clear screen or anything like this.

What will happen is this

New region with 2 objects
New region with 1 object
New region with 0 objects
New region with 1 object

Now all these new regions have a "timeout" which says that if NO NEW DATA arrives then clear the screen as sonething has gone wrong, stream corrupted, out of sync etc.

The current ffmpeg fails in this as it does as posted above and tries to create a "clear screen" packet when it SHOULD NOT.

What should happen is that once encoding it should check the timeout perion and if expired send a "New region with 0 objects", this means we can remove the hack in ffmpeg which makes up 2 packets when it should not.

[mrlika]

"Remuxing only the subtitles work, but the resulting file shows no subtitles" - this issue from head should be fixed with ​https://github.com/FFmpeg/FFmpeg/commit/c917cde9cc52ad1ca89926a617f847bc9861d5a0 commit.

[Carl Eugen Hoyos]

Remuxing the dvb subtitles was fixed by Serhii Marchuk in c917cde9
The problems with transcoding dvb subtitles are still reproducible.

[Anshul]

Problem is due to setting end_display_time using page_time_out in dvbsub decoder, where spec specify decoder should not rely only on page_time_out.

[Carl Eugen Hoyos]

Fixed by Anshul in 7e6cf364