peculiar to BBC Radio Wales?

[Vangelis forthnet]

On Mon Nov 17 00:31:24 GMT 2014, artisticforge wrote:

> today, of the three BBC Radio Wales programmes that I normally get
> all are having to be downloaded as liveradio with radiomode=wma
> the three programmes are Roy Noble,
> (snip)
> I have a gut feeling that this is a permanent change.

Greetings, Terry :-)

Unfortunately, your gut feeling is right;
this is another prelude of more changes to come;
this issue of yours has the same root cause as the
one reported in the following support forum thread:

https://squarepenguin.co.uk/forums/topic/downloads-from-bbc-radio-nan-gaidheal-and-radio-ulster-have-failed/

The BBC are testing (?) the new changes starting with the Nations Radios...
You being this highly educated, I'll provide some technical details that I'm 
sure
will puzzle the non-savvy...

Taking as an example the Roy Noble radio programme, with PID=b04nj385,
the "legacy" XML playlist that GiP first tries:

http://www.bbc.co.uk/iplayer/playlist/b04nj385

returns: <noItems reason="preAvailability"/>.
GiP has been recently patched and so then tries
the new JSON playlist:

http://www.bbc.co.uk/programmes/b04nj385/playlist.json

This, when parsed, returns: vPID=b04nj37m.
vPID string is used to rertrieve the stream details.
GiP up to now uses exclusively the "legacy"
mediaselector/4 API (I'll call it ms4 for brevity):

http://www.bbc.co.uk/mediaselector/4/mtis/stream/b04nj37m

The BBC have left but the wma stream data (48kbps bitrate),
the rtmp stream data previously also present there have vanished...
So GiP only "sees" the wma radiomode as being available for
recording...
 The Beeb have in turn implemented the "new" mediaselector/5
(ms5) API for this radio show and, evidently, many others from
the Nations radios.
This new ms5 URL varies according to the client device the stream
is destined for - this is the "mediaset" part of the URL; for desktops
& laptops, mediaset=pc and the URL looks like:

http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/b04nj37m

As you can see, ONLY THE RTMP STREAMS are therein
(radiomodes flashaacstd & flashaaclow). This is because, as they have
already announced, the wma streams will be axed in the end of the year...
For info, other mediasets that I have found browsing their code are:

mediaset/journalism-pc/
mediaset/mobile-phone-basic/
mediaset/mobile-phone-main/
mediaset/mobile-tablet-main/
mediaset/journalism-http-tablet/

and others...
So, for a mobile tablet the ms5 URL would be:

http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/mobile-tablet-main/vpid/b04nj37m

(you'll notice therein the details of the akamai_hls_open streams,
delivered over http - they have also heralded the implementation of
AdobeHDS streams for AOD sometime in 2015).

Now, if you still want the rtmp stream of Roy Noble dumped to disk,
you can:

1. Manually retrieve the stream parameters from the ms5 page and
then manually compose a working rtmpdump command - I have posted
templates for both ak & ll CDNs some weeks ago in the list:

http://lists.infradead.org/pipermail/get_iplayer/attachments/20141007/8253d96d/attachment.txt

2. patch your local copy of GiP to use the ms5 URL -
Make a copy of your version of the script and edit it with an editor.
Sadly I am not a coder, so I had to apply a "butcher" method myself
(this is because from the start of September, the majority of visual content
(usually clips of TV programmes) offered to non-UK users of iplayer
is using exclusively the ms5 infrastructure).

In the  get_stream_data subroutine, change
my $media_stream_data_prefix = 
'http://www.bbc.co.uk/mediaselector/4/mtis/stream/'; # $verpid
to
my $media_stream_data_prefix = 
'http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/'; 
# $verpid

Also, the ms5 pages do not contain the "kind=" entry, but the "supplier=" 
entry
with equal value can be used in its place.
So, some

$cattribs->{kind}

entries in the GiP code (inside the get_stream_data_cdn subroutine)
for flash on demand streams must be changed to

$cattribs->{supplier}

BUT FIRST, comment out the following code:

   # Get authstring from more specific mediaselector if this mode is 
specified - fails sometimes otherwise
# if ( $cattribs->{authString} && $cattribs->{supplier} =~ 
/^(limelight|akamai|level3|sis|iplayertok)$/ && (grep /^$mode$/, (split /,/, 
$mattribs->{modelist})) ) {
# Build URL
#   my $media_stream_data_prefix = 
'http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/';
#   my $url = 
$media_stream_data_prefix."$mattribs->{verpid}/$mattribs->{service}/$cattribs->{supplier}?cb=".( 
sprintf "%05.0f", 99999*rand(0) );
#   my $xml = main::request_url_retry( main::create_ua( 'desktop' ), $url, 
3, undef, undef, 1 );
#   main::logger "\n$xml\n" if $opt->{debug};
    # get new set of connection attributes from the new xml data
#   my $new_mattribs = (parse_metadata( $xml ))[0];
#   my $new_cattribs = $new_mattribs->{connections}[0];
    # Override elements from more specific connection attribs if present
#   for my $element ( keys %{ $new_cattribs } ) {
#     $cattribs->{$element} = $new_cattribs->{$element} if 
$new_cattribs->{$element};
#   }
# }

BECAUSE THE LOGIC BEHIND IT DOESN'T WORK WITH MS5 API.
This last bit troubled me for quite a while, because I couldn't figure out 
why my patched code
wasn't working...
I have named this "patched" version of my script get_iplayer-ms5.pl and I am 
invoking it
(on Windows) as perl get_iplayer_ms5.pl, leaving intact the original script 
for other functions...
So:
perl get_iplayer_ms5.pl --type=radio --modes=flashaac --pid=b04nj385 
produces:

INFO: Checking existence of default version
INFO: flashaacstd1,flashaacstd2,flashaaclow1,flashaaclow2 modes will be 
tried fo
r version default
INFO: Trying flashaacstd1 mode to record radio: BBC iPlayer Feeds - -
INFO: File name prefix = BBC_iPlayer_Feeds_-_-_b04nj385_default

RTMPDump v2.4-78-g26c4637-4GB
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
Starting download at: 0.000 kB
INFO: Metadata:
INFO:   duration              5414.40
INFO:   moovPosition          72.00
INFO:   audiocodecid          mp4a
INFO:   aacaot                2.00
INFO:   audiosamplerate       48000.00
INFO:   audiochannels         2.00
INFO: trackinfo:
INFO:   length                259891200.00
INFO:   timescale             48000.00
INFO:   language              und
INFO: sampledescription:
INFO:   sampletype            mp4a
24050.560 kB / 1466.20 sec (27.0%)

3. YOU SHOULD WAIT FOR dinkypumpkin TO PROPERLY PATCH GiP
to try ms5 URLs if the ms4 fail; I have observed that only using MS5 API
with mediaset=pc will not only ignore the wma while still available, but 
would
cause the loss of some lower quality tv modes, too (< flashstd).

Hope I haven't done anything wrong by posting my findings...

V.