Support multi-threaded resolution change

[jyavenard]

Apple provides some samples of HTTP Live Streaming playlist.
In particular, one provides 4 streams at the following bitrates (in bit/s)
stream1 : 200000
stream2 : 311111
stream3 : 484444
stream4 : 37777

​http://devimages.apple.com/iphone/samples/bipbop/gear4/prog_index.m3u8

I implemented a HLS player, allowing to switch streams according to the download bandwidth available. Unfortunately, FFmpeg is unable to play the stream generated when a fallback streams is selected.

The samples attached are made of 10s block made from the 4 streams.
bipbopX.ts is the corresponding TS file where X goes from 1 to 4,

bipbopXY.ts is a stream made of 10s of streamX followed by 10s of streamY

bipbopXYZ.ts is a stream made of 10s of streamX, 10s of streamY and 10s of stream Z
etc...

QuickTime/Safari can play all but VBR streams with stream 1 (eg: bipbop12, bipob21, bipbop1234, bipbop4321)

ffplay only plays the following sequences:
43 and 34...
All the others fail.

This prevent implementing a HLS player that properly handle change of streams on the fly.

[jyavenard]

Should have added that of course, ffplay properly plays bipbopX.ts with 1 <= X <= 4

[Mike]

You should provide more information if you want to get someone else interested in looking at this problem. Your bug description is "ffplay fails." That is too little information.

You don't even say if you are talking about audio or video.

What is the failure you see or hear? The more detailed, the better.

What characteristic changes in these input files (that might cause trouble)? At the very least, give us an 'ffmpeg -i infile' dump for files 1-4.

[jyavenard]

Sorry, I thought it was self-explanatory that ffplay fails to handle the transitions occurring at 10, 20 or 30s (whenever there's a bitrate or resolution change) depending on the sample file played.

The samples are mpeg-ts file with a H264 video stream and a AAC mono audio stream.
Every 10s, the H264 stream change, either just the bitrate change, or the resolution (and as such bitrate) change.

it's the whole avcodec_decode_video2 that fails at the transition between bitstreams change and/or video resolution transition.
avcodec_decode_video2 will just return an error as soon as you reach the transition (exactly occurring at 10, 20 or 30s in the samples provided

I only mentioned ffplay failing, as an example on how to easily reproduce the problem.

This is not an enhancement request.. This is a bug !
The H264 decoder will crap out when the H264 stream changes, and you can't recover from that error.

[h264 @ 0x7fba43843600] Width/height/bit depth/chroma idc changing with threads is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[h264 @ 0x7fba43843600] decode_slice_header error
[h264 @ 0x7fba43843600] Width/height/bit depth/chroma idc changing with threads is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[h264 @ 0x7fba43843600] decode_slice_header error
[h264 @ 0x7fba43843600] Width/height/bit depth/chroma idc changing with threads is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.

It's also not possible to detect the new resolution/bitrate change before avcodec_decode_video2 fails. so one can't take proactive measure to perform some kind of reset.

[Carl Eugen Hoyos]

Is the problem only reproducible with ffplay or also with ffmpeg?
If it is reproducible with ffmpeg, please do not report the problem for ffplay (ffplay depends on an external library that is known to contain bugs, so ffmpeg is the primary application).

In any case: Complete, uncut console output missing.

[jyavenard]

The problem isn't in ffplay, but with ffmpeg libs (precisely libavcodec H264 decoder)

Again, I only mentioned ffplay because I could easily reproduce the problem there too, same symptoms

It would have only taken 30s of your time to reproduce the problem.

The work around for the time being is to detect manually the change of resolution, close the decoder, and re-open it.

e.g.:

    // definition: AVStream *stream;
    if (stream->codec->codec)
        avcodec_close(stream->codec);
    AVCodec *codec = avcodec_find_decoder(stream-codec->codec_id);
    avcodec_open2(stream->codec, codec, NULL);

[reimar]

What you quote quite obviously is _not_ a bug, the message is quite clear: Changing resultions is an unsupported feature in frame-multithreaded decode.
If you need support for resolution changes, you cannot use frame multithreading.
Thus yes, this clearly is a feature request.
It would only be a bug if none of the properties the message says actually changes.

[Carl Eugen Hoyos]

Replying to jyavenard:

QuickTime/Safari can play all but VBR streams with stream 1 (eg: bipbop12, bipob21, bipbop1234, bipbop4321)

Is there a player that can play the streams that contain stream 1?

[jyavenard]

bipbop1.ts on its own ; almost everyone will play it.

For one transitioning from or to bipbop1 ; then I don't know.
Neither mplayer, ffmpeg or Quicktime will play them.

[Carl Eugen Hoyos]

Please allow me to rephrase my badly worded question:
Is there a player that can play the streams that contain stream 1 and another stream?
If not, is there any reason to assume the resulting (combined) stream is valid?

[jyavenard]

then the answer is yes.

This type of stream could theorically occur when you are streaming HTTP live streaming.
This stream for example:
​http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8

on an iPhone or iPad, it starts streaming from the bipbop1 stream, then realise that there is sufficient bandwidth and change to a higher bitrate stream (bipbop2, 3 or 4). So what you see is bipbop1 for a few seconds, and then it switch to another one. it works.

For a player, it is possible to reset the decoder on the fly as you can detect the change of streams, but say you are recording the stream content instead, you would be unable to play back the resulting mpeg-ts.

MythTV now allows to record HLS stream, and can do switch of streams on the fly, according to the available bandwidth.

That no current player is able to manage that mpeg-ts doesn't mean it can't be done :)

[Carl Eugen Hoyos]

Replying to jyavenard:

That no current player is able to manage that mpeg-ts doesn't mean it can't be done :)

I did not try to imply this, on the contrary, afaict, FFmpeg supports changing from 2 to 3 or 4, from 3 to 2 or 4 and from 4 to 2 or 3 fine. Imo, this suggests that there might be a problem with stream 1.

[jyavenard]

If you use the default multi-threaded h264 decoder, ffmpeg doesn't handle any of them...
mplayer does the change on the fly by resetting the decoder, but it's rather heavy handed and shouldn't be necessary

[Tristan Matthews]

Replying to jyavenard:

If you use the default multi-threaded h264 decoder, ffmpeg doesn't handle any of them...
mplayer does the change on the fly by resetting the decoder, but it's rather heavy handed and shouldn't be necessary

Just out of curiosity, where does mplayer do this?

[Carl Eugen Hoyos]

Replying to tmatth:

Replying to jyavenard:

mplayer does the change on the fly by resetting the decoder, but it's rather heavy handed and shouldn't be necessary

Just out of curiosity, where does mplayer do this?

It does not.

[Carl Eugen Hoyos]

Implemented by Janne Grunau.

[jyavenard]

excellent news... any chance to get this ever backported to release/1.0 ?

[reimar]

Personally I think that features should only in very special circumstances ever be backported.
And with such extensive and kind of risky changes I don't think it's a good idea, if you're willing to risk regressions for this feature you can just use git, if you're not willing to take that risk backporting it would be a disservice.