Optimize HTTP request header to workaround simple access blocks

[randomcoder67]

How to reproduce:

ffmpeg -i "https://vs-cmaf-pushb-uk-live.akamaized.net/x=4/i=urn:bbc:pips:service:bbc_one_east/pc_hd_abr_v2.mpd" out.mp4

Version: ffmpeg version n6.1.1 Copyright (c) 2000-2023 the FFmpeg developers

After around 15 seconds ffmpeg will stop downloading with the following errors:

[dash @ 0x5efbff98c580] Failed to open fragment of playlist
[https @ 0x5efc00007880] HTTP error 404 Not Found

The same thing can be seen in mpv, as mpv using ffmpeg for streams. The video will stop after around 15 seconds

It's not an issue with the video stream, as using vlc it will play it, tested for at least 2 minutes and it doesn't stop

Link is from here: ​https://github.com/iptv-org/iptv
(Note, this link may only work in the UK, not 100% sure)

Interestingly it does seem to work using ffplay, although only about half of the time. Half the time ffplay will open in in a tiny window, and it will stop after a couple of seconds. The other half it will be a normal sized window and it seems to work okay

(Note, in attached log, I Ctrl-C killed the process after a few seconds of the error happening)

[randomcoder67]

log

[MasterQuestionable]

͏    Unrecoverable network problem:
͏    HTTP 404 is the typical message. (obscure access denial, or really just "Not Found")

͏    See also:
͏    ​https://trac.ffmpeg.org/ticket/11024#comment:2

͏    Try experimenting the relevant HTTP request headers?
͏    If VLC alike could somehow seemingly properly work.

͏    Excerpt from "log.txt":
[[
͏    User-Agent: Lavf/60.16.100
͏    Accept: */*
͏    Connection: close
͏    Host: vs-cmaf-pushb-uk-live.akamaized.net
͏    Icy-MetaData: 1
]]
͏    For example, you may take a look at:
͏    ​https://browserleaks.com/ip#headers

͏    Make your access pattern more like the expected initiator should workaround.
͏    Note there are channels other than request headers.

͏    See also: ​https://github.com/MasterInQuestion/talk/discussions/10

[randomcoder67]

Replying to MasterQuestionable:

͏    Unrecoverable network problem:
͏    HTTP 404 is the typical message. (obscure access denial, or really just "Not Found")

͏    See also:
͏    ​https://trac.ffmpeg.org/ticket/11024#comment:2

͏    Try experimenting the relevant HTTP request headers?
͏    If VLC alike could somehow seemingly properly work.

͏    Excerpt from "log.txt":
[[
͏    User-Agent: Lavf/60.16.100
͏    Accept: */*
͏    Connection: close
͏    Host: vs-cmaf-pushb-uk-live.akamaized.net
͏    Icy-MetaData: 1
]]
͏    See also:
͏    ​https://browserleaks.com/ip#headers
͏    ​https://github.com/MasterInQuestion/talk/discussions/10

͏    Make your access pattern more like the expected initiator should workaround.
͏    Note there are channels other than request headers.

I tried using the same headers as VLC:

ffmpeg -headers $'Host: vs-cmaf-pushb-uk-live.akamaized.net:443\r\nAccept: */*\r\nAccept-Language: en_GB\r\nUser-Agent: VLC/3.0.21 LibVLC/3.0.21\r\nAccept-Encooding: deflate, gzip\r\nCache-Control: no-cache\r\n' -i "$URL" -y out.mp4

The same issue happens. The headers do seem to be used as they can be seen in the output of ffmpeg with -v debug

You mentioned there are channels other than request headers. What would those be?

[MasterQuestionable]

͏    I don't use FFmpeg alike to directly perform downloading for security considerations.
͏    And I'm not familiar in setting request headers with `ffmpeg`.
͏    ("-headers" appears tricky (buggy?) regardless)

͏    See various tests on BrowserLeaks.
͏    TLS fingerprinting maybe of your interest.
͏    (somehow demonstrated: [ ​https://github.com/curl/curl/issues/13873#issuecomment-2183960844 ])
͏    .
͏    There are far more not directly mentioned.

͏    If the headers are already VLC-like, then it's likely VLC just auto-reconnect...