Opened 11 years ago

Last modified 2 years ago

#3356 open enhancement

feature request: Segment HLS streams on SCTE 35 markers

Reported by: Adam Thomas Owned by:
Priority: wish Component: undetermined
Version: git-master Keywords: hls bounty
Cc: o.birkedal@sportradar.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Many proprietary Apple HTTP live streaming or other HTTP streaming encoders accept SCTE 35 markers in the input MPEG stream. Using these streams, these segmenters break the segments at the points described in the SCTE 35 message. In addition to this, a comment is inserted into the M3U8 manifest to indicate that the following chunk occurred after a SCTE 35 message.

This is now a very common practive, but no open source solution exists. The great benefit of this is that it allows a downstream piece of software to swap out chunks when such messages occur by simple text manipulation on the manifest file. The most common use case for this is the insertion of ads between 2 SCTE 35 messages in a live stream.

This is becoming a common feature in commercial encoders and it would be great to see it land in ffmpeg.

Change History (38)

in reply to:  description ; comment:2 by Carl Eugen Hoyos, 11 years ago

Keywords: hls added
Priority: normal → wish
Version: unspecified → git-master

Replying to adamscybot:

Many proprietary Apple HTTP live streaming or other HTTP streaming encoders accept SCTE 35 markers in the input MPEG stream.

Please provide such a sample.

in reply to:  2 ; comment:3 by Adam Thomas, 11 years ago

Replying to cehoyos:

Replying to adamscybot:

Many proprietary Apple HTTP live streaming or other HTTP streaming encoders accept SCTE 35 markers in the input MPEG stream.

Please provide such a sample.

Would you like a sample of a TS stream including the markers or do you mean example products providing this feature?

in reply to:  3 ; comment:4 by Carl Eugen Hoyos, 11 years ago

Replying to adamscybot:

Replying to cehoyos:

Replying to adamscybot:

Many proprietary Apple HTTP live streaming or other HTTP streaming encoders accept SCTE 35 markers in the input MPEG stream.

Please provide such a sample.

Would you like a sample of a TS stream including the markers

Yes.

in reply to:  4 comment:5 by Adam Thomas, 11 years ago

Replying to cehoyos:

Replying to adamscybot:

Replying to cehoyos:

Replying to adamscybot:

Many proprietary Apple HTTP live streaming or other HTTP streaming encoders accept SCTE 35 markers in the input MPEG stream.

Please provide such a sample.

Would you like a sample of a TS stream including the markers

Yes.

Also note that I may be willing to provide a bounty on this.

Here is a sample stream: ​https://drive.google.com/file/d/0B4tWTxKbsp10SlUzV3ZIT25RbHc/edit?usp=sharing

SCTE 35 Cue Out occurs (out_of_network_indicator 1) at approx 75s for an ad break of 242 seconds. If this feature were to be implemented, the chunk would need to be split both when the cue out occurs and at the exact duration offset after that (duration provided in the break_duration structure within the message). This behaviour is accompanied by an "auto_return 1" flag in the scte message. It is my understanding that some encoders dont set the auto return flag and instead send a separate SCTE35 cue in (out_of_network_indicator 0) message at the end of a break.

As far as the comments in the HLS Manifest file are concerned -- there is no set standard. Most solutions have something along the lines of:

chunk.ts
chunk.ts
#EXT-X-CUE-OUT:DURATION=255,scte35uuid
chunk.ts
chunk.ts
chunk.ts
#EXT-X-CUE-IN
chunk.ts

Also just for the hell of it, some example products: (datasheets aren't too clear but if your CTRL+F "SCTE" or "ad insertion" it usually points it out):

Some other products respond to SCTE 104 messages directly and then segment without the SCTE 35 stage. However, these are probably out of scope of ffmpeg as they are servers actively responding to the SCTE104 messages and not reading data embedded in the mpeg stream. It does drive home still though the need to frame accurately segment on some given boundary:

comment:6 by Carl Eugen Hoyos, 11 years ago

Status: new → open

Sample uploaded to ​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket3356/

in reply to:  6 comment:7 by Adam Thomas, 11 years ago

Replying to cehoyos:

Sample uploaded to ​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket3356/

Thanks for the sample upload (I should of read the submission requirements better).

Would it be possible to tag this ticket as "bounty"?

comment:8 by Carl Eugen Hoyos, 11 years ago

Keywords: bounty added

comment:9 by Andrey Utkin, 11 years ago

adamscybot,
Could you please drop a hint - what bounty amount is affordable for you?

in reply to:  9 comment:10 by Adam Thomas, 11 years ago

Replying to Krieger:

adamscybot,
Could you please drop a hint - what bounty amount is affordable for you?

Krieger,
I would be willing to give something around the $500 (£300) mark for a fully working solution tested with some example live streams I can provide. Is this something that interests you?

Last edited 11 years ago by Adam Thomas (previous) (diff)

comment:11 by Adam Thomas, 11 years ago

I should also mention that I currently use the segment.c demuxer as this allows a much higher level of configurability than hlsenc.c.

I am ingesting raw H.264/AVC over UDP so whichever demuxer this is would need to be able to read the SCTE 35 messages.

Last edited 11 years ago by Adam Thomas (previous) (diff)

comment:15 by Carl Eugen Hoyos, 10 years ago

What's wrong with samples.ffmpeg.org?

comment:16 by Soner, 10 years ago

Any updates about the issue? @adamscybot did you work on this with @Krieger?

comment:17 by Anshul, 10 years ago

I am working on it

in reply to:  16 comment:18 by Adam Thomas, 10 years ago

Replying to scaryguy:

Any updates about the issue? @adamscybot did you work on this with @Krieger?

Unfortunately not, I never received contact from a single person regarding this ticket. However, it is telling that this ticket is now the top result for the search term "scte35 HLS." Presumably interest is reasonably high.

Like many others, we continued use of proprietary encoders. Though the completion of this ticket would be a big deal for anyone doing programme insertion in live streams, me included!

comment:19 by Anshul, 10 years ago

@adamscybot

I am able to parse scte data using ffmpeg,
here is debug print of ffmpeg.
a141982112@linux-6jho:~/work/anshul/Project/FFmpeg> ./ffmpeg -loglevel debug -i ~/test_videos/mpegwithscte35.ts -dcodec cue_xml -map 0:0 some.ts
ffmpeg version N-67232-g2ff4edf Copyright (c) 2000-2014 the FFmpeg developers

built on Dec 24 2014 16:06:16 with gcc 4.7 (SUSE Linux)
configuration: --enable-libass --disable-optimizations --enable-libx264 --enable-gpl --enable-libfreetype --disable-optimizations
libavutil 54. 11.100 / 54. 11.100
libavcodec 56. 10.100 / 56. 10.100
libavformat 56. 11.100 / 56. 11.100
libavdevice 56. 2.100 / 56. 2.100
libavfilter 5. 2.100 / 5. 2.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100

Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input file with argument '/home/a141982112/test_videos/mpegwithscte35.ts'.
Reading option '-dcodec' ... matched as option 'dcodec' (force data codec ('copy' to copy stream)) with argument 'cue_xml'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:0'.
Reading option 'some.ts' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file /home/a141982112/test_videos/mpegwithscte35.ts.
Successfully parsed a group of options.
Opening an input file: /home/a141982112/test_videos/mpegwithscte35.ts.
[mpegts @ 0xa21c980] Format mpegts probed with size=2048 and score=100
[mpegts @ 0xa21c980] stream=1 stream_type=1b pid=65 prog_reg_desc=CUEI
[mpegts @ 0xa21c980] stream=2 stream_type=4 pid=100 prog_reg_desc=CUEI
[mpegts @ 0xa21c980] Before avformat_find_stream_info() pos: 0 bytes read:131072 seeks:1
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] decode_slice_header error
[h264 @ 0xa220d00] no frame!
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] decode_slice_header error
[h264 @ 0xa220d00] no frame!
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] decode_slice_header error
[h264 @ 0xa220d00] no frame!
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] decode_slice_header error
[h264 @ 0xa220d00] no frame!
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] decode_slice_header error
[h264 @ 0xa220d00] no frame!
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0xa220d00] non-existing PPS 0 referenced
[h264 @ 0xa220d00] decode_slice_header error
[h264 @ 0xa220d00] no frame!
[h264 @ 0xa220d00] Current profile doesn't provide more RBSP data in PPS, skipping
[h264 @ 0xa220d00] Increasing reorder buffer to 1
[h264 @ 0xa220d00] no picture
[mpegts @ 0xa21c980] All programs have pmt, headers found
[mpegts @ 0xa21c980] All info found
[mpegts @ 0xa21c980] Estimating duration from bitrate, this may be inaccurate
[mpegts @ 0xa21c980] After avformat_find_stream_info() pos: 153032 bytes read:294912 seeks:1 frames:39
Input #0, mpegts, from '/home/a141982112/test_videos/mpegwithscte35.ts':

Duration: 02:08:49.44, start: 89668.801522, bitrate: 120 kb/s
Program 1

Stream #0:0[0x135], 0, 1/90000: Data: scte_35, 0/1
Stream #0:1[0x65], 21, 1/90000: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg, left), 544x576 [SAR 64:33 DAR 544:297], 1/50, 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:2[0x100](eng), 18, 1/90000: Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 120 kb/s

Successfully opened the file.
Parsing a group of options: output file some.ts.
Applying option dcodec (force data codec ('copy' to copy stream)) with argument cue_xml.
Applying option map (set input stream mapping) with argument 0:0.
Successfully parsed a group of options.
Opening an output file: some.ts.
File 'some.ts' already exists. Overwrite ? [y/N] y
Successfully opened the file.
[mpegts @ 0xa2881a0] muxrate VBR, pcr every 9000 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'some.ts':

Metadata:

encoder : Lavf56.11.100
Stream #0:0, 0, 1/90000: Data: cue_xml, 0/1, 128 kb/s
Metadata:

encoder : Lavc56.10.100 cue_xml

Stream mapping:

Stream #0:0 -> #0:0 (scte_35 (native) -> cue_xml (native))

Press [q] to stop, ? for help
[mpegts @ 0xa21c980] Correcting start time by 9223371947185974285
[SCTE 35 Decoder @ 0xa222ec0] NULL packet

Last message repeated 29 times

[SCTE 35 Decoder @ 0xa222ec0] Insert cmd
[SCTE 35 Decoder @ 0xa222ec0] event_id = 296
[SCTE 35 Decoder @ 0xa222ec0] splice_event_cancel_indicator = 0
[SCTE 35 Decoder @ 0xa222ec0] flags = ef
[SCTE 35 Decoder @ 0xa222ec0] out_of_network_indicator = 1
[SCTE 35 Decoder @ 0xa222ec0] program_splice_flag = 1
[SCTE 35 Decoder @ 0xa222ec0] duration_flag = 1
[SCTE 35 Decoder @ 0xa222ec0] splice_immediate_flag = 0
[SCTE 35 Decoder @ 0xa222ec0] time = 1e16a1ab8
[SCTE 35 Decoder @ 0xa222ec0] NULL packet

Last message repeated 119 times

[SCTE 35 Decoder @ 0xa222ec0] Insert cmd
[SCTE 35 Decoder @ 0xa222ec0] event_id = 296
[SCTE 35 Decoder @ 0xa222ec0] splice_event_cancel_indicator = 0
[SCTE 35 Decoder @ 0xa222ec0] flags = 4f
[SCTE 35 Decoder @ 0xa222ec0] out_of_network_indicator = 0
[SCTE 35 Decoder @ 0xa222ec0] program_splice_flag = 1
[SCTE 35 Decoder @ 0xa222ec0] duration_flag = 0
[SCTE 35 Decoder @ 0xa222ec0] splice_immediate_flag = 0
[SCTE 35 Decoder @ 0xa222ec0] time = 1e2b670d8
[SCTE 35 Decoder @ 0xa222ec0] NULL packet

I dont have any tool to cross verify the output.
In the video you have shared, there is only one splice command, which is insert.
I do need more video with more commands in it.

I was also looking for some ready made(may be enterprise) tool output to verify my output.
If you can help me, please.
(please contact me at er.anshul.maheshwari at gmail dot com)

Last edited 10 years ago by Anshul (previous) (diff)

in reply to:  19 comment:20 by Adam Thomas, 10 years ago

Replying to er.anshul.maheshwari@…:

@adamscybot
I dont have any tool to cross verify the output.
In the video you have shared, there is only one splice command, which is insert.
I do need more video with more commands in it.

I was also looking for some ready made(may be enterprise) tool output to verify my output.
If you can help me, please.
(please contact me at er.anshul.maheshwari at gmail dot com)

I should be able to help. My initial impression is that the data outputted there is correct. At the moment, I don't have access to the streams with SCTE markers in, as we are in the process of a rebuild. However, I'm hoping said access will be restored going into january.

With this, I will be able to capture some real broadcast streams with SCTE35 markers in for you (both CUE IN and CUE OUT).

Have you seen TSReader (​http://www.tsreader.com/tsreader/)? With the lite (free) version, you should be able to open the original TS provided on the samples FTP. I *think* you are able to see the decoded scte35 markers with this tool.

Keep in touch. I may send over my skype details if that is useful to you, to discuss provision of samples further?

Good work!

comment:21 by James, 10 years ago

I am very interested in the solution for this issue and would be willing to add an additional bounty on this issue for AUD $300.

I will want to be able to use this from within an existing c++ that uses ffmpeg libraries. I imagine that that is not an issue.

James

comment:22 by Anshul, 10 years ago

I am done with video present in this ticket, but it have only one in/out pair, I do need video with multiple in/out event cue points.
Adam or jamesdarcy can you guys help me with more videos. or live links

comment:23 by Anshul, 10 years ago

patch in process of review and comments

comment:24 by Anshul, 10 years ago

I do need more videos with scte data to make the way of my implementation in the mainstream.
can someone helps me here.

comment:25 by Will Kelleher, 9 years ago

@Anshul any progress here? Do you have a branch somewhere that can be tested?

comment:26 by Ole Andre Birkedal, 8 years ago

Cc: o.birkedal@sportradar.com added

Any updates or partial results on this implementation?

There are a bunch of patches in this thread: https://ffmpeg.org/pipermail/ffmpeg-devel/2015-January/167487.html but it does not seem like it amounted to much.

comment:27 by Carlos Fernandez, 8 years ago

SCTE-35 patches are being sent to the developers' mailing list these days.

Don't know when/if they'll be merged though.

in reply to:  27 comment:28 by evseev, 8 years ago

Replying to cfsmp3:

SCTE-35 patches are being sent to the developers' mailing list these days.

Don't know when/if they'll be merged though.

Thank you for your job!

Here is your patch adopted for ffmpeg-3.2.4:
​https://gist.github.com/ilyaevseev/5bcdfdbb4b503724f9e0df9491534b1c#file-ffmpeg-3-2-4-scte_35-patch

But it seems useless for me because it doesn't generate #EXT-X-SCTE35 tags in playlist.

Trying to add this feature now..

Last edited 8 years ago by evseev (previous) (diff)

comment:29 by TheTroll, 7 years ago

Hi,

any progress on this subject ?

Thanks

in reply to:  29 comment:30 by Vick Kumar, 6 years ago

Replying to TheTroll:

Hi,

any progress on this subject ?

Thanks

Any progress on this ?
Thanks

comment:31 by Mista_D, 6 years ago

Has anyone successfully implemented this?
If not, what type of bounty would sufficiently incentivize to close this ticket please?

in reply to:  21 ; comment:32 by Dennis E. Mungai, 5 years ago

Replying to jamesdarcy:

I am very interested in the solution for this issue and would be willing to add an additional bounty on this issue for AUD $300.

I will want to be able to use this from within an existing c++ that uses ffmpeg libraries. I imagine that that is not an issue.

James

I'll top this up with 600 AUD.
Getting the HLS and DASH muxers to signal the appropriate SCTE35 CUEs, as shown in this example would be a good start: ​https://docs.aws.amazon.com/mediaconvert/latest/ug/sample-manifest-scte-35-enhanced-ad-markers.html

How can one open a bounty on this?

in reply to:  32 ; comment:33 by Adrian, 3 years ago

Replying to Dennis E. Mungai:

Replying to jamesdarcy:

I am very interested in the solution for this issue and would be willing to add an additional bounty on this issue for AUD $300.

I will want to be able to use this from within an existing c++ that uses ffmpeg libraries. I imagine that that is not an issue.

James

I'll top this up with 600 AUD.
Getting the HLS and DASH muxers to signal the appropriate SCTE35 CUEs, as shown in this example would be a good start: ​https://docs.aws.amazon.com/mediaconvert/latest/ug/sample-manifest-scte-35-enhanced-ad-markers.html

How can one open a bounty on this?

You can pay me directly :)
Still interested?
I wrote a hls segmenter that cuts segments on SCTE-35 Cues.

comment:34 by mothership, 2 years ago

Last edited 2 years ago by mothership (previous) (diff)

in reply to:  33 ; comment:35 by mothership, 2 years ago

Replying to Adrian:

Replying to Dennis E. Mungai:

Replying to jamesdarcy:

I am very interested in the solution for this issue and would be willing to add an additional bounty on this issue for AUD $300.

I will want to be able to use this from within an existing c++ that uses ffmpeg libraries. I imagine that that is not an issue.

James

I'll top this up with 600 AUD.
Getting the HLS and DASH muxers to signal the appropriate SCTE35 CUEs, as shown in this example would be a good start: ​https://docs.aws.amazon.com/mediaconvert/latest/ug/sample-manifest-scte-35-enhanced-ad-markers.html

How can one open a bounty on this?

You can pay me directly :)
Still interested?
I wrote a hls segmenter that cuts segments on SCTE-35 Cues.

I'll take you up on that offer. Email me: mothership1101 at gmail

in reply to:  35 comment:36 by Karel, 2 years ago

Replying to mothership:

Replying to Adrian:

Replying to Dennis E. Mungai:

Replying to jamesdarcy:

I am very interested in the solution for this issue and would be willing to add an additional bounty on this issue for AUD $300.

I will want to be able to use this from within an existing c++ that uses ffmpeg libraries. I imagine that that is not an issue.

James

I'll top this up with 600 AUD.
Getting the HLS and DASH muxers to signal the appropriate SCTE35 CUEs, as shown in this example would be a good start: ​https://docs.aws.amazon.com/mediaconvert/latest/ug/sample-manifest-scte-35-enhanced-ad-markers.html

How can one open a bounty on this?

You can pay me directly :)
Still interested?
I wrote a hls segmenter that cuts segments on SCTE-35 Cues.

I'll take you up on that offer. Email me: mothership1101 at gmail

I'd like to hear more; please email me at [hello at raskenlund com]

in reply to:  33 comment:37 by NickBirch, 2 years ago

Replying to Adrian:

Replying to Dennis E. Mungai:

Replying to jamesdarcy:

I am very interested in the solution for this issue and would be willing to add an additional bounty on this issue for AUD $300.

I will want to be able to use this from within an existing c++ that uses ffmpeg libraries. I imagine that that is not an issue.

James

I'll top this up with 600 AUD.
Getting the HLS and DASH muxers to signal the appropriate SCTE35 CUEs, as shown in this example would be a good start: ​https://docs.aws.amazon.com/mediaconvert/latest/ug/sample-manifest-scte-35-enhanced-ad-markers.html

How can one open a bounty on this?

You can pay me directly :)
Still interested?
I wrote a hls segmenter that cuts segments on SCTE-35 Cues.

I'd like to hear more; please email me: nick at bottledlogic.co.uk

in reply to:  33 comment:38 by Karel, 2 years ago

Replying to Adrian:

Replying to Dennis E. Mungai:

Replying to jamesdarcy:

I am very interested in the solution for this issue and would be willing to add an additional bounty on this issue for AUD $300.

I will want to be able to use this from within an existing c++ that uses ffmpeg libraries. I imagine that that is not an issue.

James

I'll top this up with 600 AUD.
Getting the HLS and DASH muxers to signal the appropriate SCTE35 CUEs, as shown in this example would be a good start: ​https://docs.aws.amazon.com/mediaconvert/latest/ug/sample-manifest-scte-35-enhanced-ad-markers.html

How can one open a bounty on this?

You can pay me directly :)
Still interested?
I wrote a hls segmenter that cuts segments on SCTE-35 Cues.

Hi Dennis. Please reach out to me on karel.boek [at] raskenlund.com

Note: See TracTickets for help on using tickets.