BBC Archive - Great Egg race download problems

[Vangelis forthnet]

On Mon Mar 9 12:02:10 GMT 2015, Jimmy Aitken wrote:

> I've been trying to work out how to download episodes of
> the Great Egg Race from the BBC archive site.
> Specifically
> http://www.bbc.co.uk/archive/great_egg_race/10801.shtml
> If anyone can point me to how I can do this, I'd be grateful.

Hi :-)
This archived program is UK-only;
use Mozilla Firefox and its Web Console
to perform URL sniffing;
upon loading the URL in question,
you'll get in the console the playlist URL of the vid:

http://www.bbc.co.uk/archive/xml/10/801.xml

(further sniffing reveals the mediaselector/4 API URL:

http://www.bbc.co.uk/mediaselector/4/gtis/?server=cp47317.edgefcs.net&identifier=archive/white/OAT_BBC_10801&kind=akamai&application=ondemand

- geo-filtered; produces results only from a UK IP -
and from then on you can construct yourself the
rtmpdump download command:

rtmpdump -r "rtmp://cp47317.edgefcs.net:1935/ondemand" -a 
"ondemand?auth=daEaxcmbhakdCaMbzb1bXd1bMb3bBcnb.aG-bu_Cu8-bWG-EpnEHnwqNBvFsxF&aifp=v001&slist=archive/white/OAT_BBC_10801" 
 -f "WIN 16,0,0,305" -W "http://www.bbc.co.uk/emp/10player.swf" -p 
"http://www.bbc.co.uk/archive/great_egg_race/10801.shtml" -C O:1 -C O:0 -y 
"archive/white/OAT_BBC_10801?auth=daEaxcmbhakdCaMbzb1bXd1bMb3bBcnb.aG-bu_Cu8-bWG-EpnEHnwqNBvFsxF&aifp=v001&slist=archive/white/OAT_BBC_10801" 
 -o "OAT_BBC_10801.flv"

RTMPDump v2.4 GIT-2014-03-02 (Compiled by KSV)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
Starting download at: 0.000 kB
INFO: Metadata:
INFO:   duration                2097.04
INFO:   width                   384.00
INFO:   height                  288.00
INFO:   videodatarate           375.00
INFO:   canSeekToEnd            FALSE
INFO:   videocodecid            2.00
INFO:   audiodatarate           128.00
INFO:   audiocodecid            2.00
INFO:   framerate               25.00
INFO:   creationdate            Tue Aug 17 13:58:02 2010
133556.830 kB / 2097.03 sec (99.9%)
Download complete

The FLV file downloaded uses older video/audio codecs
(Video
Format                           : Sorenson Spark

Audio
Format                           : MPEG Audio
Format version                   : Version 1
Format profile                   : Layer 3)

so I am uncertain as to if it could be played back in recent
mobile devices without prior transcoding of the video stream;
a simple remux to the MP4/AVI container just won't work... :-(

At dinkypumpkin (I know you no longer post to the list,
but you are still reading it) :

I have tried with 2.91 to use the CLI command:

get_iplayer --type=tv --url="http://www.bbc.co.uk/archive/xml/10/801.xml" --modes=best 
 --swfurl="http://www.bbc.co.uk/emp/10player.swf?revision=18269_21576" --raw

but it fails:

C:\Program Files\get_iplayer>perl 
get_iplayer-291.pl --url="http://www.bbc.co.uk
/archive/xml/10/801.xml" --modes=best --swfurl="http://www.bbc.co.uk/emp/10playe
r.swf?revision=18269_21576" --raw
get_iplayer v2.91, Copyright (C) 2008-2010 Phil Lewis
  This program comes with ABSOLUTELY NO WARRANTY; for details 
use --warranty.
  This is free software, and you are welcome to redistribute it under 
certain
  conditions; use --conditions for details.

INFO: Trying pid: http://www.bbc.co.uk/archive/xml/10/801.xml using type: tv
INFO: Trying to stream pid using type tv
INFO: pid not found in tv cache
Matches:

INFO: 1 Matching Programmes
INFO: Checking existence of default version
INFO: flashnormal1 modes will be tried for version default
INFO: Trying flashnormal1 mode to record tv:  - -
INFO: File name prefix = 
The_Great_Egg_Race_-__http_www.bbc.co.uk_archive_xml_10
_801.xml_default
RTMPDump v2.4 GIT-2015-01-15 (Compiled by KSV)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
ERROR: rtmp server sent error
ERROR: rtmp server requested close
INFO: Command exit code 1 (raw code = 256)
WARNING: Failed to stream file D:\Vangelis\iPlayer 
Recordings\The_Great_Egg_Race
_-__http_www.bbc.co.uk_archive_xml_10_801.xml_default.partial.flv via RTMP
INFO: skipping flashnormal1 mode
ERROR: Failed to record 'The Great Egg Race  - - 
(http://www.bbc.co.uk/archive/x
ml/10/801.xml)'

C:\Program Files\get_iplayer>

The Akamai server rejects the connection and I initially thought
this was related to VPN blacklisting, but I can stream the vid on
my browser using the same VPN IP; further debugging this using
--debug > dbglog.txt 2>&1
in my previous GiP command reveals that GiP is formulating a wrong
rtmpdump command:

RTMPDump v2.4 GIT-2015-01-15 (Compiled by KSV)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
DEBUG: Protocol : RTMP
DEBUG: Hostname : cp47317.edgefcs.net
DEBUG: Port     : 1935
DEBUG: Playpath : archive/white/OAT_BBC_10801
DEBUG: tcUrl    : 
rtmp://cp47317.edgefcs.net:80/ondemand?_fcs_vhost=cp47317.edgefcs.net&undefined&undefined
DEBUG: swfUrl   : http://www.bbc.co.uk/emp/10player.swf?revision=18269_21576
DEBUG: pageUrl  :
DEBUG: app      : 
ondemand?_fcs_vhost=cp47317.edgefcs.net&undefined&undefined
DEBUG: live     : no
DEBUG: timeout  : 10 sec

so that is probably why the download fails; FWIW, the Akamai host
cp47317.edgefcs.net
is not geo-blocked (hence my download command - with fresh token -
can also work from overseas...).
Your valuable insight on this may be needed - either I am not using the
correct GiP command or GiP needs to be patched (for the "exotic" user
case scenario as the OP's).

Many regards,
Vangelis.