get_iplayer currently broken...

Wed Jan 13 13:25:59 PST 2016

When we download news video's etc, GiP stopped being able to work it out from the page url so we have to sift through the source of the page to find a .xml link which would work to download the video.

Is it possible to do something similar in the mean time with programmes while we wait for GiP to be repaired?

I know you've shown us how to construct an rtmpdump command before Vangelis but please can you remind me how to do it for the different CDNs and how to find the playlist to get the info from?

> Sent: Wednesday, January 13, 2016 at 8:18 PM
> From: "Dirk Husemann" <dirk+getiplayer at d2h.net>
> To: tellyaddict <tellyaddict21 at gmx.co.uk>, get_iplayer <get_iplayer at lists.infradead.org>
> Subject: Re: get_iplayer currently broken...
>
> PIDs can still be retrieved via the
> http://www.bbc.co.uk/iplayer/guide/bbc/20160113 URLs, then fishing for
> ^/iplayer/episode/[^/]+/.*%s-*' URL in that response and using the
> data-id to get the PID --- http://www.bbc.co.uk/iplayer/js/episode/PID
> gives some meta data. A sample python script is available at
> https://github.com/dirkhusemann/beebit/blob/master/beebit.py --- that
> one will go through the last 14 days to search for stuff.
> 
> However, that doesn't give us the actual download URLs...
> 
> cheers,
> Dirk
> 
> On 2016-01-13 21:13, tellyaddict wrote:
> > Before when the feeds went for the 1st time, scraping the site was one possible option and although it can be a long process, a couple of people who had the patience to let the process run posted their cache files on the forums for others to use. I wonder if this scraping could possibly also scrape the vpids to go with the cache entrys so that GiP could then assemble the command using those to trigger the downloads.
> >
> > There must easier ways than that though.
> >
> >> Sent: Wednesday, January 13, 2016 at 7:32 PM
> >> From: "Vangelis forthnet" <northmedia1 at the.forthnet.gr>
> >> To: get_iplayer at lists.infradead.org
> >> Subject: get_iplayer currently broken...
> >>
> >> Sadly, from ca. 16:00 GMT today (13/01/2016),
> >> GiP is belly up :-(
> >>
> >> Manifestation:
> >>
> >> 1. TV cache via the schedule feeds can't be refreshed
> >> (Radio cache refreshes normally).
> >>
> >> The reason for the TV cache refresh failure is that URLs
> >> of the following pattern:
> >>
> >> http://www.bbc.co.uk/<channel>/programmes/schedules/this_week.xml
> >> http://www.bbc.co.uk/<channel>/programmes/schedules/last_week.xml
> >>
> >> do not fetch expected schedule feeds, but the message posted already:
> >>
> >>> This service is currently undergoing a period of downtime for essential 
> >>> maintenance.
> >>> Please be advised that the programme information service
> >>> which is currently available at this URL
> >>> will soon be replaced by a new service from the BBC,
> >>> at a new set of URLs.
> >>> Please contact the following email address for further information:
> >>> 'programmes-devel at lists.forge.bbc.co.uk'
> >> Refresh of the TV cache was erratic, at best, during the
> >> last few days, now not working at all.
> >>
> >> 2. Download attempts fail completely,
> >> even with the "--pid" method
> >>
> >> This is because GiP loads URLs of the following template:
> >>
> >> http://www.bbc.co.uk/programmes/<pid>.xml
> >> http://www.bbc.co.uk/programmes/<pid>.rdf
> >>
> >> to retrieve the vpid string (that grants access to
> >> available streams) and grab other available metadata;
> >> those URLs now also return the above message...
> >>
> >> Over on the iPlayer site itself, using Firefox,
> >> both TV & Radio do seem to play OK;
> >> the TV programmes's vpid string is, I believe,
> >> buried deep inside page source, the default
> >> stream delivery method on desktop browser
> >> is AdobeHDS.
> >>
> >> For radio, the JSON playlist is being used
> >> by the embedded player, which is of the template:
> >>
> >> http://www.bbc.co.uk/programmes/<pid>/playlist.json
> >>
> >> that seems to load more reliably, but it does
> >> sometimes return (at least from my location)
> >> the infamous "downtime" message...
> >> Again, default stream delivery
> >> for audio is AdobeHDS.
> >>
> >> The "--playlist-metadata" switch was "ignored" in 2.94
> >> (and removed in 2.95dev) in this commit:
> >>
> >> https://github.com/get-iplayer/get_iplayer/commit/116d272
> >>
> >> "--playlist-metadata" was supposed to
> >>
> >>> Force use of playlists (XML and JSON) for programme metadata
> >>> instead of /programmes data endpoints
> >> however, after reverting the commit on my local
> >> copy of GiP 2.94 and issuing
> >>
> >> perl 
> >> get_iplayer-294.pl --type=radio --pid=b06vf205 --playlist-metadata --modes=flashaac 
> >>  --force --raw -v
> >>
> >> the script only tried the long-gone XML playlist :
> >>
> >> ====================================
> >> INFO: pid found in cache
> >> Matches:
> >> 11363:  Greg James - Greg has the latest Chart Update, BBC Radio 1,
> >>
> >> INFO: 1 Matching Programmes
> >> INFO: iPlayer metadata URL = http://www.bbc.co.uk/iplayer/playlist/b06vf205
> >> INFO: Getting page http://www.bbc.co.uk/iplayer/playlist/b06vf205
> >> ERROR: Failed to get version pid metadata from iplayer site
> >>
> >> ERROR: Could not get version pid metadata
> >> ERROR: Could not get programme metadata
> >> ====================================
> >>
> >> and not the JSON playlist that appears to still work...
> >>
> >> If the XML/RDF metadata pages are gone for good
> >> (possibly to be replaced by Nitro URLs), this will
> >> constitute a major setback to GiP - the JSON playlists
> >> (not available for all VOD as I recall) could prove a stopgap,
> >> but their future may also be short (given that they, too,
> >> produce the "replacement" notice).
> >>
> >> Another case where the existence of a "--vpid" switch
> >> in GiP could've proved a lifesaver; I can find manually
> >> the vpid (right-clicking the embedded player, via page
> >> source, via JSON playlist etc.), the mediaselector/5 URLs
> >> do still function normally, so stream access is still feasible
> >> in this emergency, alas not for GiP.
> >> Time to get back to composing manually rtmpdump/ffmpeg
> >> commands... and AdobeHDS.php is of invaluable help, too!
> >>
> >> As tellyaddict (tvfan) advised, better keep an eye
> >> on the support forums for any developments:
> >> https://squarepenguin.co.uk/forums/
> >> No point in "me, too" posts either here in the
> >> list or the forums, though...
> >>
> >> Let's hope this thing is just
> >> a temporary mishap on the beeb's side...
> >>
> >> V. 
> >>
> >>
> >> _______________________________________________
> >> get_iplayer mailing list
> >> get_iplayer at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/get_iplayer
> >>
> > _______________________________________________
> > get_iplayer mailing list
> > get_iplayer at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/get_iplayer
> 
> 
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer
> 