Opened 10 years ago
Last modified 3 years ago
#4048 new enhancement
segment: Handle WebVTT as per HLS spec
Reported by: | Ridley Combs | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avformat |
Version: | git-master | Keywords: | hls webvtt |
Cc: | mkuron | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
As per the HLS spec:
Subtitle segments MUST be formatted as WebVTT [WebVTT] files. Each subtitle segment MUST contain all subtitle cues that are intended to be displayed during the period indicated by the segment EXTINF duration. The start time offset and end time offset of each cue MUST indicate the total display time for that cue, even if that time range extends beyond the EXTINF duration. A WebVTT segment MAY contain no cues; this indicates that no subtitles are to be displayed during that period.
This means that the segment (or HLS) muxer needs to handle two particular cases, when muxing WebVTT for HLS:
- No frames exist during a segment's duration, so a blank segment should be written.
- A frame spans two or more different segments, so it should be written in all of them.
This behavior could be triggered by an option, or automatically when the underlying format is WebVTT.
Relatedly, the WebVTT muxer should provide an option to write the X-TIMESTAMP-MAP header for HLS synchronization.
Change History (6)
comment:1 by , 10 years ago
comment:2 by , 10 years ago
Keywords: | hls webvtt added |
---|
comment:3 by , 10 years ago
Keywords: | hls webvtt removed |
---|
It's not so much a bug as the lack of support for a newly-specified behavior. But let's use this test.vtt
file as an example:
00:00.000 --> 00:07.000 Test line 1, spans segments 0 and 1 00:07.000 --> 00:08.000 Test line 2, contained in segment 1 00:16.000 --> 00:18.000 Test line 3, should be in segment 3 after an empty segment 2 00:18.000 --> 00:40.000 Test line 4, should span several segments
Let's run ffmpeg -i in.vtt -f segment -segment_time 5 -segment_format webvtt -scodec copy out-%05d.vtt
.
We get:
out-00000.vtt
WEBVTT 00:00.000 --> 00:07.000 Test line 1, spans segments 0 and 1
out-00001.vtt
WEBVTT 00:07.000 --> 00:08.000 Test line 2, contained in segment 1
out-00002.vtt
WEBVTT 00:16.000 --> 00:18.000 Test line 3, should be in segment 3 after an empty segment 2
out-00003.vtt
WEBVTT 00:18.000 --> 00:40.000 Test line 4, should span several segments
The correct output for HLS would be:
out-00000.vtt
WEBVTT 00:00.000 --> 00:07.000 Test line 1, spans segments 0 and 1
out-00001.vtt
WEBVTT 00:00.000 --> 00:07.000 Test line 1, spans segments 0 and 1 00:07.000 --> 00:08.000 Test line 2, contained in segment 1
out-00002.vtt
WEBVTT
out-00003.vtt
WEBVTT 00:18.000 --> 00:40.000 Test line 4, should span several segments
out-00004.vtt
WEBVTT 00:18.000 --> 00:40.000 Test line 4, should span several segments
<etc…, with more duplicates up to out-00007.vtt>
The two behaviors that need adding (writing empty segments, and duplicating packets spanning multiple segments) seem like they could reasonably be separate options. The writing-empty-segments behavior seems sensible for generic subtitle segmenting (in order to have predictable segment durations), whereas duplicating packets is pretty WebVTT-in-HLS-specific.
comment:4 by , 10 years ago
Keywords: | hls webvtt added |
---|
comment:5 by , 7 years ago
I guess this kinda belongs to this ticket:
https://amd.cdn.turner.com/adultswim/episodes/no_uk_geo/bobsburgers/2017/03/bobsburgers_cc_505_Digital_IttyBittyDittyCommittee/hls/bobsburgers_cc_505_Digital_IttyBittyDittyCommittee_stream_webvtt.m3u8
is the webvtt Stream (subtitle only stream) related to https://amd.cdn.turner.com/adultswim/episodes/no_uk_geo/bobsburgers/2017/03/bobsburgers_cc_505_Digital_IttyBittyDittyCommittee/hls/bobsburgers_cc_505_Digital_IttyBittyDittyCommittee_stream_full.m3u8
from http://www.adultswim.com/videos/bobs-burgers/the-itty-bitty-ditty-committee/
Though I'm not sure. If this is only an issue about the muxer but not the demuxer, then say so and I'll open a new ticket.
FFmpeg only saves the first segment instead of merging all segments together, like it's described in this ticket.
ffmpeg -i url -c copy example.vtt
Since the m3u8 won't be up forever, here's the content of it:
#EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:300 #EXT-X-MEDIA-SEQUENCE:1 #EXT-X-PLAYLIST-TYPE:VOD #EXTINF:300.299, bobsburgers_cc_505_Digital_IttyBittyDittyCommittee_stream_webvttvtt_00001.vtt #EXTINF:300.300, bobsburgers_cc_505_Digital_IttyBittyDittyCommittee_stream_webvttvtt_00002.vtt #EXTINF:300.300, bobsburgers_cc_505_Digital_IttyBittyDittyCommittee_stream_webvttvtt_00003.vtt #EXTINF:300.300, bobsburgers_cc_505_Digital_IttyBittyDittyCommittee_stream_webvttvtt_00004.vtt #EXTINF:104.104, bobsburgers_cc_505_Digital_IttyBittyDittyCommittee_stream_webvttvtt_00005.vtt #EXT-X-ENDLIST
FFmpeg only saves the first 5mins, so I guess that's the segment time.
comment:6 by , 3 years ago
Cc: | added |
---|
Iiuc, there is a bug: Our hls muxer does not conform to the specification and produces output that is incorrect.
Is that correct? If yes, please explain how this can be reproduced. (You know what I mean with explain?)
I know that you understand the ticket and I realize that whoever will fix this already understands the issue (and possibly already knows how to fix it). but ideally every ticket should be reproducible for every user. This is particularly important in the future: The commit fixing the ticket might (in theory) introduce another problem and whoever wants to fix it may not know so much about hls...
Or said differently: It is great that you added a link to the specification and quoted the relevant paragraph, it will make somebody's life much easier. But as said previously this analysis (no matter if helpful like in this case or disturbing like in other tickets not created by you) should never replace the actual bug report.