peculiar to BBC Radio Wales?

Mon Nov 17 06:33:02 PST 2014

Hello

The patch suggested works.
The work now is to incorporate this into get_iplayer so that a ms4
failure falls over to try ms5.

The patched get_iplayer has downloaded Roy Noble and Beverley's World
of Music from BBC radio Wales.

On Mon, Nov 17, 2014 at 1:22 AM, Vangelis forthnet
<northmedia1 at the.forthnet.gr> wrote:
> On Mon Nov 17 00:31:24 GMT 2014, artisticforge wrote:
>
>> today, of the three BBC Radio Wales programmes that I normally get
>> all are having to be downloaded as liveradio with radiomode=wma
>> the three programmes are Roy Noble,
>> (snip)
>> I have a gut feeling that this is a permanent change.
>
>
> Greetings, Terry :-)
>
> Unfortunately, your gut feeling is right;
> this is another prelude of more changes to come;
> this issue of yours has the same root cause as the
> one reported in the following support forum thread:
>
> https://squarepenguin.co.uk/forums/topic/downloads-from-bbc-radio-nan-gaidheal-and-radio-ulster-have-failed/
>
> The BBC are testing (?) the new changes starting with the Nations Radios...
> You being this highly educated, I'll provide some technical details that I'm
> sure
> will puzzle the non-savvy...
>
> Taking as an example the Roy Noble radio programme, with PID=b04nj385,
> the "legacy" XML playlist that GiP first tries:
>
> http://www.bbc.co.uk/iplayer/playlist/b04nj385
>
> returns: <noItems reason="preAvailability"/>.
> GiP has been recently patched and so then tries
> the new JSON playlist:
>
> http://www.bbc.co.uk/programmes/b04nj385/playlist.json
>
> This, when parsed, returns: vPID=b04nj37m.
> vPID string is used to rertrieve the stream details.
> GiP up to now uses exclusively the "legacy"
> mediaselector/4 API (I'll call it ms4 for brevity):
>
> http://www.bbc.co.uk/mediaselector/4/mtis/stream/b04nj37m
>
> The BBC have left but the wma stream data (48kbps bitrate),
> the rtmp stream data previously also present there have vanished...
> So GiP only "sees" the wma radiomode as being available for
> recording...
> The Beeb have in turn implemented the "new" mediaselector/5
> (ms5) API for this radio show and, evidently, many others from
> the Nations radios.
> This new ms5 URL varies according to the client device the stream
> is destined for - this is the "mediaset" part of the URL; for desktops
> & laptops, mediaset=pc and the URL looks like:
>
> http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/b04nj37m
>
> As you can see, ONLY THE RTMP STREAMS are therein
> (radiomodes flashaacstd & flashaaclow). This is because, as they have
> already announced, the wma streams will be axed in the end of the year...
> For info, other mediasets that I have found browsing their code are:
>
> mediaset/journalism-pc/
> mediaset/mobile-phone-basic/
> mediaset/mobile-phone-main/
> mediaset/mobile-tablet-main/
> mediaset/journalism-http-tablet/
>
> and others...
> So, for a mobile tablet the ms5 URL would be:
>
> http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/mobile-tablet-main/vpid/b04nj37m
>
> (you'll notice therein the details of the akamai_hls_open streams,
> delivered over http - they have also heralded the implementation of
> AdobeHDS streams for AOD sometime in 2015).
>
> Now, if you still want the rtmp stream of Roy Noble dumped to disk,
> you can:
>
> 1. Manually retrieve the stream parameters from the ms5 page and
> then manually compose a working rtmpdump command - I have posted
> templates for both ak & ll CDNs some weeks ago in the list:
>
> http://lists.infradead.org/pipermail/get_iplayer/attachments/20141007/8253d96d/attachment.txt
>
> 2. patch your local copy of GiP to use the ms5 URL -
> Make a copy of your version of the script and edit it with an editor.
> Sadly I am not a coder, so I had to apply a "butcher" method myself
> (this is because from the start of September, the majority of visual content
> (usually clips of TV programmes) offered to non-UK users of iplayer
> is using exclusively the ms5 infrastructure).
>
> In the  get_stream_data subroutine, change
> my $media_stream_data_prefix =
> 'http://www.bbc.co.uk/mediaselector/4/mtis/stream/'; # $verpid
> to
> my $media_stream_data_prefix =
> 'http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/';
> # $verpid
>
> Also, the ms5 pages do not contain the "kind=" entry, but the "supplier="
> entry
> with equal value can be used in its place.
> So, some
>
> $cattribs->{kind}
>
> entries in the GiP code (inside the get_stream_data_cdn subroutine)
> for flash on demand streams must be changed to
>
> $cattribs->{supplier}
>
> BUT FIRST, comment out the following code:
>
>   # Get authstring from more specific mediaselector if this mode is
> specified - fails sometimes otherwise
> # if ( $cattribs->{authString} && $cattribs->{supplier} =~
> /^(limelight|akamai|level3|sis|iplayertok)$/ && (grep /^$mode$/, (split /,/,
> $mattribs->{modelist})) ) {
> # Build URL
> #   my $media_stream_data_prefix =
> 'http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/';
> #   my $url =
> $media_stream_data_prefix."$mattribs->{verpid}/$mattribs->{service}/$cattribs->{supplier}?cb=".(
> sprintf "%05.0f", 99999*rand(0) );
> #   my $xml = main::request_url_retry( main::create_ua( 'desktop' ), $url,
> 3, undef, undef, 1 );
> #   main::logger "\n$xml\n" if $opt->{debug};
>    # get new set of connection attributes from the new xml data
> #   my $new_mattribs = (parse_metadata( $xml ))[0];
> #   my $new_cattribs = $new_mattribs->{connections}[0];
>    # Override elements from more specific connection attribs if present
> #   for my $element ( keys %{ $new_cattribs } ) {
> #     $cattribs->{$element} = $new_cattribs->{$element} if
> $new_cattribs->{$element};
> #   }
> # }
>
> BECAUSE THE LOGIC BEHIND IT DOESN'T WORK WITH MS5 API.
> This last bit troubled me for quite a while, because I couldn't figure out
> why my patched code
> wasn't working...
> I have named this "patched" version of my script get_iplayer-ms5.pl and I am
> invoking it
> (on Windows) as perl get_iplayer_ms5.pl, leaving intact the original script
> for other functions...
> So:
> perl get_iplayer_ms5.pl --type=radio --modes=flashaac --pid=b04nj385
> produces:
>
> INFO: Checking existence of default version
> INFO: flashaacstd1,flashaacstd2,flashaaclow1,flashaaclow2 modes will be
> tried fo
> r version default
> INFO: Trying flashaacstd1 mode to record radio: BBC iPlayer Feeds - -
> INFO: File name prefix = BBC_iPlayer_Feeds_-_-_b04nj385_default
>
> RTMPDump v2.4-78-g26c4637-4GB
> (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
> Connecting ...
> INFO: Connected...
> Starting download at: 0.000 kB
> INFO: Metadata:
> INFO:   duration              5414.40
> INFO:   moovPosition          72.00
> INFO:   audiocodecid          mp4a
> INFO:   aacaot                2.00
> INFO:   audiosamplerate       48000.00
> INFO:   audiochannels         2.00
> INFO: trackinfo:
> INFO:   length                259891200.00
> INFO:   timescale             48000.00
> INFO:   language              und
> INFO: sampledescription:
> INFO:   sampletype            mp4a
> 24050.560 kB / 1466.20 sec (27.0%)
>
> 3. YOU SHOULD WAIT FOR dinkypumpkin TO PROPERLY PATCH GiP
> to try ms5 URLs if the ms4 fail; I have observed that only using MS5 API
> with mediaset=pc will not only ignore the wma while still available, but
> would
> cause the loss of some lower quality tv modes, too (< flashstd).
>
> Hope I haven't done anything wrong by posting my findings...
>
> V.
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

-- 
terry l. ridder ><>