get_iplayer currently broken...

Wed Jan 13 16:42:32 PST 2016

On Wed Jan 13 21:25:59 GMT 2016, tellyaddict wrote:

> I know you've shown us how to construct an rtmpdump command before
> Vangelis but please can you remind me how to do it
> for the different CDNs and how to find the playlist
> to get the info from?

I'm two timezones ahead of you and with fever...
You appear to be an advanced user of GiP;
I gained much of my understanding of the
inner workings of GiP by producing and saving
verbose/debug logs of GiP downloading sessions,
which where then studied in detail...
Having basic knowledge of rtmpdump and ffmpeg,
with time, perl code blocks start to make sense...

If you MUST download something with imminent
expiration, then I think the hls modes via ffmpeg
are simpler to use manually.

A. Radio example:

http://www.bbc.co.uk/programmes/b06yjbxc

(pid=b06yjbxc, 15min "Newsbeat" of this cursed day)

Load the page in browser (Firefox with NPAPI
Adobe Flash plugin) BUT DO NOT PRESS PLAY
on the embedded player - instead right click
and then click the second line of the context menu;
it is now copied to your clipboard, pasted looks
like this:
b06yjbnk / radioProgramme / b06yjbxc

First string is the vpid one, necessary for stream access;
so, vpid=b06yjbnk.
Load the following mediaselector5 URL:

http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/apple-ipad-hls/vpid/b06yjbnk

This is geo-filtered, it will produce 128kbps
stream data for the UK, 48kbps elsewhere...

What you need is the <connection href=" URL,
the "master" HLS playlist. The token (everything
after the "master.m3u8?" bit) is shortlived, I
think it expires after an hour (and a new one has to be
generated by reloading the page).

Construct and use the following FFmpeg command:
(playlist is for UK stream)

ffmpeg -loglevel error -stats  -i 
"http://cp401492-vh.akamaihd.net/i/prod_af_mp4_aaclc_128/iplayerstream/l2o/b06yjbnk_a573f2d3-f4a2-409f-bba6-63e86875e1d9.mp4/master.m3u8?hdnea=st=1452728177~exp=1452749777~acl=/*b06yjbnk_a573f2d3-f4a2-409f-bba6-63e86875e1d9.mp4*~hmac=d9b14d624b496557923eda824d1c862c627fef011a8b9c4df3c8b4d62e06e18e" 
 -vn -c:a copy radio.ts

Remux .ts file to m4a:

ffmpeg -i radio.ts -vn -c:a copy -bsf:a aac_adtstoasc -f ipod radio.m4a

Rename radio.m4a accordingly to point to the exact
radio programme - no tags, of course!

B. Video example

http://www.bbc.co.uk/iplayer/episode/p03c4z7b/

(pid=p03c4z7b/, 6min clip that expires in 5days...)

Obtain vpid as per radio => vpid=p03c4zkc

Load the following mediaselector5 URL:

http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/p03c4zkc

(works in the UK, only)
Locate block
<media bitrate="2800" encoding="h264" ....
(If for some VOD HD is not available, then locate block
 <media bitrate="1500" encoding="h264" instead).
As with radio, you first need the URL inside
the <connection href=" bit, which is the master HLS playlist.
This needs further manual parsing.
Load in browser:

http://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/,800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033,.mp4.csmil/master.m3u8?hdnea=st=1452729721~exp=1452751321~acl=/*800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033*~hmac=59552c838970bc4ce1fae9587b097b68212453516a99ab7e88ee433e49c7fded

then save as a "master.m3u8" file.
Once saved, open it up in a text editor:

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=796000,RESOLUTION=640x360,CODECS="avc1.77.30, 
mp4a.40.2",CLOSED-CAPTIONS=NONE
http://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/,800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033,.mp4.csmil/index_0_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1496000,RESOLUTION=832x468,CODECS="avc1.77.30, 
mp4a.40.2",CLOSED-CAPTIONS=NONE
http://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/,800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033,.mp4.csmil/index_1_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2393000,RESOLUTION=1280x720,CODECS="avc1.640029, 
mp4a.40.2",CLOSED-CAPTIONS=NONE
http://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/,800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033,.mp4.csmil/index_2_av.m3u8

If you need to grab the hlsvhigh variant (832x468p), copy the second 
playlist;
likewise, if you are after the hlshd (1280x720p, 25FPS) stream, copy the 
third playlist.

As a last step, you still need the token from the master playlist,
so what you need feed ffmpeg is:
(for hlshd: )
http://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/,800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033,.mp4.csmil/index_2_av.m3u8?hdnea=st=1452729721~exp=1452751321~acl=/*800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033*~hmac=59552c838970bc4ce1fae9587b097b68212453516a99ab7e88ee433e49c7fded

Final ffmpeg command:

ffmpeg -loglevel error -stats  -i 
http://cp401489-vh.akamaihd.net/i/iplayerstream/secure_auth/,800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033,.mp4.csmil/index_2_av.m3u8?hdnea=st=1452729721~exp=1452751321~acl=/*800kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452575441,1500kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450452577809,3200kbps/modav/bUnknown-59a63f4c-5fbe-4dd8-84ce-df89edb650bc_p03c4zkc_1450453011033*~hmac=59552c838970bc4ce1fae9587b097b68212453516a99ab7e88ee433e49c7fded - 
c copy video720p.ts

Remux .ts file to mp4:

ffmpeg -i video720p.ts -c copy -bsf:a aac_adtstoasc -f mp4 video720p.mp4

THIS ALL IS NOT THAT COMPLICATED AS IT
MIGHT FIRST LOOK, once you get the logic behind it...
All the RTMP & HLS streams GiP can handle can
be downloaded manually with rtmpdump/ffmpeg - with
a lot more fuss, though - and no, MPEG-DASH can't be
fetched with FFmpeg...

The 30d expiration window is an ally this time;
hopefully dinky will come to the rescue soon:
https://squarepenguin.co.uk/forums/thread-623-post-3392.html#pid3392

Off to bed now... 