Educating Archie

Thu Jan 3 11:08:12 EST 2013

--- !!! LONG MAIL ALERT !!! ---

On Wed Jan 2 20:59:44 GMT 2013, Chris J Brady wrote:

>Hope you all had a great Xmas and New year.
>Bad start here though - this download fails - any suggestions please?
>http://www.bbc.co.uk/radio/player/b009gc5s
>http://www.bbc.co.uk/iplayer/episode/b009gc5s/Educating_Archie_Archie_in_Australia/

Hi Chris, thanks for the wishes :-) ...
Now, if you go to http://www.bbc.co.uk/iplayer/playlist/b009gc5s , you'll 
find that the
version PID for this show is vPID=b009gc54 (the "identifier" value).
Then, if you are in the UK (or you "appear" as being in the UK), go to this 
URL:
http://www.bbc.co.uk/mediaselector/4/mtis/stream/b009gc54
There you can see all the available streams (in theory, at least) of the 
show you're after.

1. media bitrate="32" encoding="aac" -> low bitrate stream, aimed at mobile 
phones; this is
geo-blocked (the "-acl" bit in the server name) and streamed through "http".
Get_iPlayer picks this up as "flashaaclow2" mode (more about this further 
down...)

2. media bitrate="128" encoding="aac" -> high quality PC stream 
(AAC-LC at 128kbpsABR),
served through "rtmp" protocol; the rtmp server itself is not geo-blocked, 
but access to the
stream is allowed only for UK IPs; Get_iPlayer picks this up as 
"flashaacstd1" mode.

3. media bitrate="48" encoding="aac" -> lower quality PC stream 
(HE-AACv2 at 48kbpsABR),
again served through the "rtmp" protocol and picked up by Get_iPlayer as 
"flashaaclow1" mode.

4. media bitrate="128" encoding="wma9" -> high quality "Windows Media Audio" 
stream,
 aimed at DAB radios (but also Win PCs) and in fact is only 96kbps; but 
sounds almost indistinguishable to
 the "128 aac" stream, especially for spoken content. This stream is not 
picked up by my
2.82-git-64e7c68 version of the get_iplayer.pl script (discussed further 
down...).
The 
http://www.bbc.co.uk/mediaselector/4/asx/b009gc54/iplayer_intl_stream_wma_uk_concrete
URI will fetch you (from a UK IP) an .asx playlist file which, when opened 
with notepad,
will reveal the real stream URLs (main & back-up one):

mms://wm-acl.bbc.co.uk/wms/radio4xcoyopa/radio_4_extra_-_monday_1400.wma
mms://wm-acl.bbc.co.uk/wms2/radio4xcoyopa/radio_4_extra_-_monday_1400.wma

Again, these are geo-blocked at server level (the "-acl" part in the host 
name).

Note: For non-UK IPs, only two choices are offered through
http://www.bbc.co.uk/mediaselector/4/mtis/stream/b009gc54

1. media bitrate="32" encoding="aac" -> this in fact is 48kbps, streamed 
through
the "rtmp" protocol and has the same specs as the "48 aac" UK-only stream;
but they are not identical, this is a dedicated "overseas" rtmp stream.
Get_iPlayer sees this as "flashaaclow1" mode.

2. media bitrate="48" encoding="wma9" -> lower quality "Windows Media Audio" 
stream,
 aimed at DAB radios (but also Win PCs) ; sounds worse than the 
aforementioned
 HE-AACv2 at 48kbpsABR stream. Now, the URI
http://www.bbc.co.uk/mediaselector/4/asx/b009gc54/iplayer_intl_stream_wma_lo_concrete
works exclusively for non-UK IPs and will again fetch you an .asx playlist 
file which,
when opened with notepad, will reveal the real stream URLs (main & back-up 
one):

mms://wm.bbc.co.uk/wms/radio4xcoyopa/radio_4_extra_-_monday_1400.wma
mms://wm.bbc.co.uk/wms2/radio4xcoyopa/radio_4_extra_-_monday_1400.wma

However, these streams themselves are not geo-blocked and can be accessed 
even
from within the UK...

Again, get_iplayer.pl 2.82-git-64e7c68 fails to detect the "wma" radio 
mode...

At Chris J Brady:

I have tried to record all 3 flavours of the theoretically available rtmp 
streams
(i.e. UK @128kbps, UK at 48kbps, non-UK at 48kbps), in every case the
download returns this error:

RTMPDump v2.4-79-g87625d3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
ERROR: Closing connection: NetStream.Play.StreamNotFound
INFO: Command exit code 1 (raw code = 256)
WARNING: Failed to stream file D:\Vangelis\iPlayer Recordings\Educating 
Archie -
 Archie in Australia - 31_12_2012[b009gc5s].partial.aac.flv via RTMP
INFO: skipping flashaacstd1 (/flashaaclow1) mode
ERROR: Failed to record 'Educating Archie - Archie in Australia (b009gc5s)'

which means that rtmpdump cannot find the files (streams) on the rtmp 
servers;
either the beeb have not uploaded them or they somehow got deleted!

BUT THE WMA STREAMS DO WORK!

If you are on a Windows machine and just want to listen to the show,
just open the stream URL in the player of your choice (that is capable of
playing wma streams, that is...).

I am on WinVistaSP2-32bit and WMP11, Winamp 5.57, Real Player SP 1.1.5
play the non-UK stream fine.
And VLC 2.0.5 will also play the stream, if you change the (deprecated)
protocol mms:// to either rtsp:// or mmsh:// (i.e. mms through http):

mmsh://wm.bbc.co.uk/wms/radio4xcoyopa/radio_4_extra_-_monday_1400.wma

If you also want to capture the stream, there exist several methods, but I 
am not
an expert and can only advise on methods I use:
1. VLC will let you record the stream and produce an .asf file (ASF is the 
container
 for windows media streams), in which the stream is stored as is... You can 
enable
recording by choosing "View" -> "Advanced Controls" & then the red "record"
button will appear on the GUI; you then have to go to "Tools" -> 
"Preferences"
-> "Input & Codecs" -> "Record directory or filename" to tell VLC where to
put the recorded files.
The recording can also be achieved through the CLI of VLC, see
http://wiki.videolan.org/Documentation:Streaming_HowTo/Receive_and_Save_a_Stream,
but I am not at all comfortable with that, so other knowledgeable members of 
the list
may intervene,  if needed.
2. Another GUI solution I use is the free SDP Downloader 2.3.0 at
 http://sdp.ppona.com/ ; this has http proxy support. By forcing mms through 
http (rather than TCP)
in the settings and using a UK http proxy, one can grab the elusive UK-only 
96kbps stream.
3. VLC & SDP will save the stream in "real time", i.e. a 30min show will 
need 30min to record.
A third GUI I use is the Japanese "GetASFStream 2.3.0.0c", which can be 
found at
http://tetora.orz.ne.jp/forum/gasdown/download.cgi
The GUI is very poorly translated into English, it is designed only for 
WinXP (and thus has to be run
in WinXP compatibility mode in higher versions of Win), it is very hard to 
master (essentially by
trial & error - all online help is in Japanese), but once you do it will let 
you save a 1 hour long stream
in just 5 minutes (because it does up to 30x parallel downloading).
4. IIRC, get_iplayer used to be able to download wma via Mplayer, but the 
exact command eludes me;
I tried this one for the non-UK stream:
mplayer -bandwidth 9999999 -dumpstream 
mms://wm.bbc.co.uk/wms/radio4xcoyopa/radio_4_extra_-_monday_1400.wma -dumpfile 
radio_4_extra_-_monday_1400.asf -vc null -vo null -ao null
it is slowly downloading it, apparently in "real time":

Playing 
mms://wm.bbc.co.uk/wms/radio4xcoyopa/radio_4_extra_-_monday_1400.wma.
STREAM_ASF, URL: 
mms://wm.bbc.co.uk/wms/radio4xcoyopa/radio_4_extra_-_monday_1400.wma
Resolving wm.bbc.co.uk for AF_INET...
Connecting to server wm.bbc.co.uk[212.58.251.77]: 1755...
Connected
unknown object
unknown object
file object, packet length = 2261 (2261)
unknown object
unknown object
stream object, stream ID: 1
unknown object
data object
mmst packet_length = 2261
Cache size set to 3072 KBytes
Stream not seekable!

but before the file was completed (~ 23% saved),
the download stopped :-( with an error:

read error:: Result too large
pre-header read failed
Core dumped ;)

Exiting... (End of file)

which is reminiscent of the frequent abrupt
terminations I observed back when I tried
wma mode with get_iplayer 2.79 ...

No matter how you saved the wma stream, you must process
the resultant .asf (or .wma) file with AsfBin:
http://www.radioactivepages.com/asfbin.aspx
(WinGUI available, called AsfBinWin.exe), to
re-index it and create a proper, seekable asf file; VLC in particular
creates huge asf files, full of junk, that shrink in size as much
as 40% after treating them with AsfBinWin.
Finally after that, you can safely change the extention from
.asf to .wma, if you like...

At dinkypumpkin:

Two issues arise, as far as GIP is concerned:

1. Why do the wma versions (UK & non-UK) not get detected
as available radiomodes, if they are there?
Surely it must be something on the BBC side, I haven't made
any changes to both the "global" & "user" option files in order
to "hide" the wma radiomode... Nothing is mentioned in the list
about dropping support for wma, and I do remember some months
ago that the wma radiomode did show up as an alternative.

2. Let's focus on the UK-only detected "flashaaclow2" radiomode;
If you try to record this (from a UK IP):

get_iplayer --type=radio --pid=b009gc5s --modes=flashaaclow2 --force --verbose

you'll see that the recording fails:

RTMPDump v2.4-79-g87625d3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
WARNING: Unknown protocol!

DEBUG: Protocol : RTMP
DEBUG: Hostname : 3gp-acl.bbc.co.uk
DEBUG: Port     : 1935
DEBUG: Playpath : 
mp4:mobile/RBN2_radio_4_extra_-_monday_1400_b009gc54_2012_12_3
1_16_21_46
DEBUG: tcUrl    : rtmp://3gp-acl.bbc.co.uk:1935/sdpgen/bbc7/secure_auth
DEBUG: app      : sdpgen/bbc7/secure_auth
DEBUG: live     : no
DEBUG: timeout  : 10 sec
DEBUG: Setting buffer time to: 36000000ms
Connecting ...
ERROR: RTMP_Connect0, failed to connect socket. 10061 (Unknown error)
DEBUG: Closing connection.

INFO: Command exit code 3 (raw code = 768)
WARNING: Failed to stream file D:\Vangelis\iPlayer 
Recordings\Educating_Archie_-
_Archie_in_Australia_b009gc5s_default.partial.aac.flv via RTMP
DEBUG: Record using flashaaclow2 mode return code: 'next'
INFO: skipping flashaaclow2 mode
ERROR: Failed to record 'Educating Archie - Archie in Australia (b009gc5s)'

because GIP tries to use rtmpdump to download a file streamed through the 
http (or rtsp?)
protocol, as discussed in the start of this (very long) email; I don't know 
if GIP is designed
to capture this kind of stream, but to me it looks more of an "rtspaaclow2" 
mode, that VLC
ought to handle... FYI, in my own setup of GIP I haven't installed VLC at 
all (although I am
using a portable [.paf format] version of VLC 2.0.5 on an external HDD), so 
if GIP wanted
for some reason to use VLC, it would complain about the missing VLC binary.
Unless something is terribly wrong at my side, some fixing is needed to GIP 
so it does not
 detect this mobile-phone stream as a "flash" radiomode (and hence assign it 
to rtmpdump
for recording...)

Thanks for bearing with me, everyone :-)
Kind Regards.