A message from Auntie

[Ross]

The feeds are gone, but at least for now the bbc schedules are still there.  I am not sure you need scraping … yet.

Here is one of the schedules:  http://www.bbc.co.uk/bbctwo/programmes/schedules/hd/this_week.xml

This gives an option for an immediate workaround to get the pvr going again (as already pointed out by User name), and also potentially gives a longer term solution.

As a side note I looked at this stuff a week ago to see how easy it would be to extend the search for 30 days, and I believe the schedule data will allow us to do that.

WORKAROUND:

I will quote User Name.  I also have this running on my machine, only took a couple of minutes to patch and seems to work OK (barring a small issue getting the episode numbers).

—— START
I found that by co-opting (and modifying slightly the behaviour of) the --refreshfuture cmdline option, I was able to re-populate the cache with a fair amount of stuff. I have modified the 2.86 git version of the get_iplayer.pl script I have on Windows (precisely, from git branch f109d22).


In the get_links sub, I changed schedule_feeds to:

   my @schedule_feeds = (
    "
http://www.bbc.co.uk/${channel_id
}/this_week.xml",
    "
http://www.bbc.co.uk/${channel_id
}/last_week.xml",
   );


and then reversed the sense of:

     # Don't create this prog instance if the availablity is in the past 
     # this prevents programmes which never appear in iPlayer from being indexed
     next if Programme::get_time_string( $available ) < $now;


to:


     next if Programme::get_time_string( $available )> $now;


Now when I perform, e.g.:


"%Windir%\system32\cmd.exe" /c get_iplayer --refresh --type radio --refreshfuture

or

"%Windir%\system32\cmd.exe" /c get_iplayer --refresh --type tv --refreshfuture


my radio.cache (or tv.cache) is repopulated with a wealth of metadata.

Subsequently downloading content with this cache (at least I've tried for flashaaclow radio) leaves me with files populated with a (albeit wrong/generic iPlayer) thumbnail and what appears to be at least some of the correct tagging.


The only down-side so far seems to be that some progs are missing from before (e.g. Doctor Who Extra in the tv.cache). Presumably these are non-broadcast-schedule files, e.g. red-button-esque web-only iPlayer features.

——— END 

Just add —refresh future to your commands.  e.g. get_iplayer —pvr —whitespace —refreshfuture

Once the cache is populated you can see it in the web gui, search, save new pvr (just do not press refresh cache!)

LONGTERM FIX

This is what I would do to fix it.

1) Split the existing schedule logic off to its own method, so that it can be called twice (to get past programs and to get future programs).
2) Remove the broken feed logic
3) Replace it with a call to the schedule logic method.  (NB: Doing this pulls this logic out into its own method, so it means that if the schedule xml disappears in the future and we need to do some web scraping, then we just need a new method to call instead)
4) Add the extract for episodenum from <position> in the xml.  (Not a biggy, but quick to implement).
5) Add some date logic to get the current date and look back further than the last_week.xml feed (this way we could get the full 30 days).

That is probably it.  Then the pvr and search functions should work fine.  The web gui will also work.

That is my suggestion to fix it, not too hard, probably a few bugs or issues would need to be flushed out later, but certainly workable.
Please reply if you have some other wizardry in mind.

So any perl gurus out there who want to have a go.  If no one else does it I will probably have a look when I get a moment, not for a week or so though  I might have to bring out the perl cookbook though.  Not great at perl.


On 31/10/2014, at 22:34, Roger Bell_West <roger at firedrake.org> wrote:

> On Fri, Oct 31, 2014 at 09:08:26AM -0000, Trevor Campbell Davis wrote:
>> It’s a pain, but I can easily manually source the PIDs of the
>> programmes that I want on a daily basis.  What I don’t want (or have
>> time) to do is to manually enter single command line instructions for
>> each programme when the previous one has finished downloading.  So a
>> script (resembling a queue) where I can enter multiple PIDs for
>> automatic sequential download would be really helpful.
> 
> At the most basic level, you can just use the shell (this is for
> bash):
> 
> for P in b00hslb0 b04mb11l b04mb6ck; do get_iplayer --type radio --metadata generic --pid $P --get; done
> 
> I have written a program to automate scraping, but I'm working on
> getting it into a suitable form for release.
> 
> R
> 
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer