Overlaying / hardcoding subtitles - last subtitle not done correctly

[Adam]

Summary of the bug:

When overlaying subtitles (idx/sub), the very last subtitle is displayed only very briefly and the duration of it appears to be ignored by ffmpeg.

All files attached.
The idx/sub has 3 subtitles at 2 seconds, 4 seconds and 6 seconds.

testhardcode.mkv shows the first and second subtitles with a normal duration, and the last subtitle is very briefly flashed.

How to reproduce:

ffmpeg -report -v 9 -loglevel 99 -i blankvideo.mp4 -i test.idx -filter_complex "[0:v][1:s]overlay=eof_action=pass[vout]" -y -map "[vout]" testhardcode.mkv

ffmpeg version N-60859-g313a6c6 Copyright (c) 2000-2014 the FFmpeg developers

built on Feb 23 2014 22:09:35 with gcc 4.8.2 (GCC)

[Carl Eugen Hoyos]

Do you know how the duration for the last subtitle can be known?

[Adam]

Hi,

I really don't understand this stuff very much, but I did find something that could help you - the source code for Subtitle Edit, which does the calculation of the duration.

The stack trace at the bottom of this comment shows where the calculation of the duration (aka 'Delay') is performed, when a .sub is loaded into Subtitle Edit.

In summary however:

This is where the vob sub packs merging starts:
​https://github.com/SubtitleEdit/subtitleedit/blob/master/src/Forms/VobSubOcr.cs#L954

Which goes into here:
​https://github.com/SubtitleEdit/subtitleedit/blob/master/src/Logic/VobSub/VobSubParser.cs#L156

And then into here which constructs a SubPicture class:
​https://github.com/SubtitleEdit/subtitleedit/blob/master/src/Logic/VobSub/VobSubMergedPack.cs#L15

*
And here is where the calculation of the duration/Delay happens, by parsing the display control commands, and calculating it from the stop display command:

​https://github.com/SubtitleEdit/subtitleedit/blob/master/src/Logic/VobSub/SubPicture.cs#L109

*

Hopefully this helps and will all make sense to you...

(I don't really have an understanding of how subtitles work... And I am not familiar with the ffmpeg code...)

The full stack trace of Subtitle Edit, in case it helps, when loading the .sub and calculating the duration:

SubtitleEdit.exe!Nikse.SubtitleEdit.Logic.VobSub.SubPicture.ParseDisplayControlCommands(bool createBitmap, System.Collections.Generic.List<System.Drawing.Color> colorLookUpTable, System.Collections.Generic.List<System.Drawing.Color> fourColors, bool useCustomColors) Line 109 C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Logic.VobSub.SubPicture.SubPicture(byte[] data) Line 40 + 0x10 bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Logic.VobSub.VobSubMergedPack.VobSubMergedPack(byte[] subPictureData, System.TimeSpan presentationTimeStamp, int streamId, Nikse.SubtitleEdit.Logic.VobSub.IdxParagraph idxLine) Line 15 + 0x18 bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Logic.VobSub.VobSubParser.MergeVobSubPacks() Line 156 + 0x49 bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Forms.VobSubOcr.InitializeSubIdx(string vobSubFileName) Line 954 + 0x10 bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Forms.VobSubOcr.Initialize(string vobSubFileName, Nikse.SubtitleEdit.Logic.VobSubOcrSettings vobSubOcrSettings, bool useNewSubIdxCode, Nikse.SubtitleEdit.Forms.Main main) Line 617 + 0xb bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Forms.Main.ImportAndOcrVobSubSubtitleNew(string fileName, bool showInTaskbar) Line 10050 + 0x36 bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Forms.Main.OpenSubtitle(string fileName, System.Text.Encoding encoding, string videoFileName, string originalFileName) Line 2208 + 0x16 bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Forms.Main.OpenSubtitle(string fileName, System.Text.Encoding encoding) Line 2182 + 0x12 bytes C#
SubtitleEdit.exe!Nikse.SubtitleEdit.Forms.Main.SubtitleListview1_DragDrop(object sender, System.Windows.Forms.DragEventArgs e) Line 9677 + 0xd bytes C#
[External Code]
SubtitleEdit.exe!Nikse.SubtitleEdit.Program.Main() Line 17 + 0x1d bytes C#

[Adam]

For reference, after a quick search, I notice that the code I mentioned in my previous comment for Subtitle Edit, for parsing the subtitles:
​​https://github.com/SubtitleEdit/subtitleedit/blob/master/src/Logic/VobSub/SubPicture.cs#L109

is somewhat similar to the ffmpeg code here:

​https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/dvdsubdec.c#L263

The delay / sub_header->end_display_time calculations are slightly different.

I have no idea if the parsing and calculation routine is the cause of the problem (since all but the last subtitle display for the correct duration), or whether the rendering code somewhere might forget to do something on the last index of a loop somewhere...

thanks.

[Cigaes]

Patch on the mailing-list:
​http://ffmpeg.org/pipermail/ffmpeg-devel/2014-March/155123.html
This is yet another instance of the problem of not having a timestamp for the end of filtered streams.

cehoyos: ffprobe can show the duration:

$ ./ffprobe_g -show_frames /tmp/test.idx
...
[SUBTITLE]
media_type=subtitle
pts=6000000
pts_time=6.000000
format=0
start_display_time=0
end_display_time=2980
num_rects=1
[/SUBTITLE]

[Carl Eugen Hoyos]

Fixed by Nicolas in 521707d4 - thank you for the report!