get_iplayer currently broken...

[Dirk Husemann]

PIDs can still be retrieved via the
http://www.bbc.co.uk/iplayer/guide/bbc/20160113 URLs, then fishing for
^/iplayer/episode/[^/]+/.*%s-*' URL in that response and using the
data-id to get the PID --- http://www.bbc.co.uk/iplayer/js/episode/PID
gives some meta data. A sample python script is available at
https://github.com/dirkhusemann/beebit/blob/master/beebit.py --- that
one will go through the last 14 days to search for stuff.

However, that doesn't give us the actual download URLs...

cheers,
Dirk

On 2016-01-13 21:13, tellyaddict wrote:
> Before when the feeds went for the 1st time, scraping the site was one possible option and although it can be a long process, a couple of people who had the patience to let the process run posted their cache files on the forums for others to use. I wonder if this scraping could possibly also scrape the vpids to go with the cache entrys so that GiP could then assemble the command using those to trigger the downloads.
>
> There must easier ways than that though.
>
>> Sent: Wednesday, January 13, 2016 at 7:32 PM
>> From: "Vangelis forthnet" <northmedia1 at the.forthnet.gr>
>> To: get_iplayer at lists.infradead.org
>> Subject: get_iplayer currently broken...
>>
>> Sadly, from ca. 16:00 GMT today (13/01/2016),
>> GiP is belly up :-(
>>
>> Manifestation:
>>
>> 1. TV cache via the schedule feeds can't be refreshed
>> (Radio cache refreshes normally).
>>
>> The reason for the TV cache refresh failure is that URLs
>> of the following pattern:
>>
>> http://www.bbc.co.uk/<channel>/programmes/schedules/this_week.xml
>> http://www.bbc.co.uk/<channel>/programmes/schedules/last_week.xml
>>
>> do not fetch expected schedule feeds, but the message posted already:
>>
>>> This service is currently undergoing a period of downtime for essential 
>>> maintenance.
>>> Please be advised that the programme information service
>>> which is currently available at this URL
>>> will soon be replaced by a new service from the BBC,
>>> at a new set of URLs.
>>> Please contact the following email address for further information:
>>> 'programmes-devel at lists.forge.bbc.co.uk'
>> Refresh of the TV cache was erratic, at best, during the
>> last few days, now not working at all.
>>
>> 2. Download attempts fail completely,
>> even with the "--pid" method
>>
>> This is because GiP loads URLs of the following template:
>>
>> http://www.bbc.co.uk/programmes/<pid>.xml
>> http://www.bbc.co.uk/programmes/<pid>.rdf
>>
>> to retrieve the vpid string (that grants access to
>> available streams) and grab other available metadata;
>> those URLs now also return the above message...
>>
>> Over on the iPlayer site itself, using Firefox,
>> both TV & Radio do seem to play OK;
>> the TV programmes's vpid string is, I believe,
>> buried deep inside page source, the default
>> stream delivery method on desktop browser
>> is AdobeHDS.
>>
>> For radio, the JSON playlist is being used
>> by the embedded player, which is of the template:
>>
>> http://www.bbc.co.uk/programmes/<pid>/playlist.json
>>
>> that seems to load more reliably, but it does
>> sometimes return (at least from my location)
>> the infamous "downtime" message...
>> Again, default stream delivery
>> for audio is AdobeHDS.
>>
>> The "--playlist-metadata" switch was "ignored" in 2.94
>> (and removed in 2.95dev) in this commit:
>>
>> https://github.com/get-iplayer/get_iplayer/commit/116d272
>>
>> "--playlist-metadata" was supposed to
>>
>>> Force use of playlists (XML and JSON) for programme metadata
>>> instead of /programmes data endpoints
>> however, after reverting the commit on my local
>> copy of GiP 2.94 and issuing
>>
>> perl 
>> get_iplayer-294.pl --type=radio --pid=b06vf205 --playlist-metadata --modes=flashaac 
>>  --force --raw -v
>>
>> the script only tried the long-gone XML playlist :
>>
>> ====================================
>> INFO: pid found in cache
>> Matches:
>> 11363:  Greg James - Greg has the latest Chart Update, BBC Radio 1,
>>
>> INFO: 1 Matching Programmes
>> INFO: iPlayer metadata URL = http://www.bbc.co.uk/iplayer/playlist/b06vf205
>> INFO: Getting page http://www.bbc.co.uk/iplayer/playlist/b06vf205
>> ERROR: Failed to get version pid metadata from iplayer site
>>
>> ERROR: Could not get version pid metadata
>> ERROR: Could not get programme metadata
>> ====================================
>>
>> and not the JSON playlist that appears to still work...
>>
>> If the XML/RDF metadata pages are gone for good
>> (possibly to be replaced by Nitro URLs), this will
>> constitute a major setback to GiP - the JSON playlists
>> (not available for all VOD as I recall) could prove a stopgap,
>> but their future may also be short (given that they, too,
>> produce the "replacement" notice).
>>
>> Another case where the existence of a "--vpid" switch
>> in GiP could've proved a lifesaver; I can find manually
>> the vpid (right-clicking the embedded player, via page
>> source, via JSON playlist etc.), the mediaselector/5 URLs
>> do still function normally, so stream access is still feasible
>> in this emergency, alas not for GiP.
>> Time to get back to composing manually rtmpdump/ffmpeg
>> commands... and AdobeHDS.php is of invaluable help, too!
>>
>> As tellyaddict (tvfan) advised, better keep an eye
>> on the support forums for any developments:
>> https://squarepenguin.co.uk/forums/
>> No point in "me, too" posts either here in the
>> list or the forums, though...
>>
>> Let's hope this thing is just
>> a temporary mishap on the beeb's side...
>>
>> V. 
>>
>>
>> _______________________________________________
>> get_iplayer mailing list
>> get_iplayer at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/get_iplayer
>>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer