From richard22j at zoho.com  Sat Mar  3 02:53:00 2018
From: richard22j at zoho.com (RS)
Date: Sat, 3 Mar 2018 10:53:00 +0000
Subject: Format of options file
Message-ID: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>

Is the format of the options file different in Windows and Linux?  I am 
setting up my machines to dual boot between Windows 10 and ubuntu 16.04. 
  On my main machine I wanted to copy the history file.  I thought while 
I was doing that I might as well copy the whole hidden .get_iplayer 
directory including the cache and options files. I expected to have to 
change the --output option since the path is different, but some of the 
options don't work, and others do.

The criterion seems to be that options with a numeric or boolean value 
like --subtitles or --expiry work.

Options with a string value like --tvmode, --radiomode or 
--ffmpegloglevel don't.  For example with --tvmode=hlshd I get

INFO: Downloading tv: 'Collateral: Series 1 - 2. Episode 2 (b09sym6m) 
[original]'
) available for this programme with version 'original'
INFO: Available modes: 
hlshd,hlsvhigh,hvfhd,hvfsd,hvfxsd,hvfhigh,hvfxhigh,hvfstd,hvflow

When I re-enter --prefs-add --tvmode=hlshd
--ffmpeg-loglevel=info gives an error

INFO: Downloading tv: 'Collateral: Series 1 - 2. Episode 2 (b09sym6m) 
[original]'
') - using defaultalue for --ffmpeg-loglevel ('info

If I enter the options again with --prefs-add they work correctly, so it 
is no big deal.  Any ideas what is happening?



From ralph at inputplus.co.uk  Sat Mar  3 03:09:49 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sat, 03 Mar 2018 11:09:49 +0000
Subject: Format of options file
In-Reply-To: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
Message-ID: <20180303110949.B91381FE27@orac.inputplus.co.uk>

Hi Richard,

> ') - using defaultalue for --ffmpeg-loglevel ('info
>
> If I enter the options again with --prefs-add they work correctly, so
> it is no big deal.  Any ideas what is happening?

That line above seems to be overprinting the ?')? at the start of the
line suggesting there's an ASCII CR, carriage return, after the `info'
as if a Unix system was reading a POSIX text file of lines ending in
ASCII LF, linefeed, but being fed a DOS one with CR LF pairs.

`od -c foo' will show them, and they can be removed with `sed -i
's/\r$//' foo'.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From dave.widgery at gmail.com  Sat Mar  3 03:40:08 2018
From: dave.widgery at gmail.com (Dave Widgery)
Date: Sat, 03 Mar 2018 11:40:08 +0000
Subject: The Coroner series 2 Episode 8 not available
Message-ID: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>

Hi,
Has anyone any idea why Episode 8 of the latest series of The Coroner is 
not available? it isn't available on bbc iplayer either?

Dave



From roger at tricker.co.uk  Sat Mar  3 04:00:55 2018
From: roger at tricker.co.uk (Roger Tricker)
Date: Sat, 3 Mar 2018 12:00:55 -0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
Message-ID: <008101d3b2e7$4bb1a840$e314f8c0$@tricker.co.uk>

Dave,

If you check the showing dates, which were on a daily basis, the showings
went straight from Ep7 to Ep9 with no missing day in between.
The episode is available on Youtube.

Roger

-----Original Message-----
From: get_iplayer <get_iplayer-bounces at lists.infradead.org> On Behalf Of
Dave Widgery
Sent: 03 March 2018 11:40
To: get_iplayer <get_iplayer at lists.infradead.org>
Subject: The Coroner series 2 Episode 8 not available

Hi,
Has anyone any idea why Episode 8 of the latest series of The Coroner is not
available? it isn't available on bbc iplayer either?

Dave


_______________________________________________
get_iplayer mailing list
get_iplayer at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer



From roger at tricker.co.uk  Sat Mar  3 04:49:11 2018
From: roger at tricker.co.uk (Roger Tricker)
Date: Sat, 3 Mar 2018 12:49:11 -0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <em4875cdf8-7bda-44fe-bf3d-f05537dd1469@owner-pc>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <008101d3b2e7$4bb1a840$e314f8c0$@tricker.co.uk>
 <em4875cdf8-7bda-44fe-bf3d-f05537dd1469@owner-pc>
Message-ID: <009301d3b2ee$07cfdd20$176f9760$@tricker.co.uk>

Not sure. I've not downloaded from YouTube. I'm sure that there is plenty of freeware out there.

Roger

-----Original Message-----
From: Dave Widgery <dave.widgery at gmail.com> 
Sent: 03 March 2018 12:44
To: Roger Tricker <roger at tricker.co.uk>; get_iplayer <get_iplayer at lists.infradead.org>
Subject: Re[2]: The Coroner series 2 Episode 8 not available

Hi Roger,

Thanks for the reply, I didn't notice the dates but it is still odd that it wasn't shown, unless it had a topical subject to something that has just happened in real life.

What is the best way to download it from youtube?

Dave

------ Original Message ------
From: "Roger Tricker" <roger at tricker.co.uk>
To: "Dave Widgery" <dave.widgery at gmail.com>; "get_iplayer" 
<get_iplayer at lists.infradead.org>
Sent: 03/03/2018 13:00:55
Subject: RE: The Coroner series 2 Episode 8 not available

>Dave,
>
>If you check the showing dates, which were on a daily basis, the 
>showings went straight from Ep7 to Ep9 with no missing day in between.
>The episode is available on Youtube.
>
>Roger
>
>-----Original Message-----
>From: get_iplayer <get_iplayer-bounces at lists.infradead.org> On Behalf 
>Of Dave Widgery
>Sent: 03 March 2018 11:40
>To: get_iplayer <get_iplayer at lists.infradead.org>
>Subject: The Coroner series 2 Episode 8 not available
>
>Hi,
>Has anyone any idea why Episode 8 of the latest series of The Coroner 
>is not available? it isn't available on bbc iplayer either?
>
>Dave
>
>
>_______________________________________________
>get_iplayer mailing list
>get_iplayer at lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/get_iplayer
>



From dave.widgery at gmail.com  Sat Mar  3 05:00:51 2018
From: dave.widgery at gmail.com (Dave Widgery)
Date: Sat, 03 Mar 2018 13:00:51 +0000
Subject: Message with suspicious header
Message-ID: <em3d0ce6e3-a614-4d6f-831b-f5d8006ca65b@owner-pc>

Hi

I have yet again had a message held due to a suspicious header, all I 
did was press reply?

Dave



From richard22j at zoho.com  Sat Mar  3 05:30:58 2018
From: richard22j at zoho.com (RS)
Date: Sat, 3 Mar 2018 13:30:58 +0000
Subject: Format of options file
In-Reply-To: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
Message-ID: <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>



On 03/03/18 10:53, I wrote:
> Is the format of the options file different in Windows and Linux?  
...
> The criterion seems to be that options with a numeric or boolean value 
> like --subtitles or --expiry work.
> 
> Options with a string value like --tvmode, --radiomode or 
> --ffmpegloglevel don't.

Looking at the files with xxd, the lines in the Windows file end with 
CRLF (hex 0D0A).  In file copied to Linux with re-entering of some of 
the options with --prefs-add there is a mixture of lines ending CRLF and 
LF, even though the lines have been shuffled by --prefs-add.  There does 
not seem to be any correlation between the errors and the presence of CR 
characters.

I have not yet looked at the code to see how the options file is parsed. 
  What is puzzling is that the options display correctly with --prefs-show.

The Windows download_history file seems to work correctly.  I hope I 
don't need to edit it.



From richard22j at zoho.com  Sat Mar  3 05:59:52 2018
From: richard22j at zoho.com (RS)
Date: Sat, 3 Mar 2018 13:59:52 +0000
Subject: Format of options file
In-Reply-To: <20180303110949.B91381FE27@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
Message-ID: <28f7bc80-30bf-3417-0bbc-5eb9034b18c6@zoho.com>

On 03/03/18 11:09, Ralph Corderoy wrote:
> Hi Richard,
> 
>> ') - using defaultalue for --ffmpeg-loglevel ('info
>>
>> If I enter the options again with --prefs-add they work correctly, so
>> it is no big deal.  Any ideas what is happening?
> 
> That line above seems to be overprinting the ?')? at the start of the
> line suggesting there's an ASCII CR, carriage return, after the `info'
> as if a Unix system was reading a POSIX text file of lines ending in
> ASCII LF, linefeed, but being fed a DOS one with CR LF pairs.
> 
> `od -c foo' will show them, and they can be removed with `sed -i
> 's/\r$//' foo'.
> 

Thanks, Ralph.  That does indeed seem to be the problem.  Thanks also 
for the command to remove the unwanted CR characters.  That seems to 
have cleared everything up.

Best wishes
Richard



From specialk99 at blueyonder.co.uk  Sat Mar  3 07:02:39 2018
From: specialk99 at blueyonder.co.uk (Kevin McCarthy)
Date: Sat, 3 Mar 2018 15:02:39 -0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <008101d3b2e7$4bb1a840$e314f8c0$@tricker.co.uk>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <008101d3b2e7$4bb1a840$e314f8c0$@tricker.co.uk>
Message-ID: <000301d3b300$ac290e20$047b2a60$@co.uk>

It's probably the music used in the episode.
In the credits it states Blur's "Girls and Boys" is featured in the episode
yet no other episode in Season 2 features any similar music.
I imagine the BBC only had broadcast rights for the music at the time of the
original airing in 2016.

Regards
Special-K

-----Original Message-----
From: get_iplayer [mailto:get_iplayer-bounces at lists.infradead.org] On Behalf
Of Roger Tricker
Sent: 03 March 2018 12:01
To: 'Dave Widgery'; 'get_iplayer'
Subject: RE: The Coroner series 2 Episode 8 not available

Dave,

If you check the showing dates, which were on a daily basis, the showings
went straight from Ep7 to Ep9 with no missing day in between.
The episode is available on Youtube.

Roger

-----Original Message-----
From: get_iplayer <get_iplayer-bounces at lists.infradead.org> On Behalf Of
Dave Widgery
Sent: 03 March 2018 11:40
To: get_iplayer <get_iplayer at lists.infradead.org>
Subject: The Coroner series 2 Episode 8 not available

Hi,
Has anyone any idea why Episode 8 of the latest series of The Coroner is not
available? it isn't available on bbc iplayer either?

Dave


_______________________________________________
get_iplayer mailing list
get_iplayer at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer


_______________________________________________
get_iplayer mailing list
get_iplayer at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer



From james at jls-radio.com  Sat Mar  3 09:33:35 2018
From: james at jls-radio.com (James Scholes)
Date: Sat, 3 Mar 2018 11:33:35 -0600
Subject: Format of options file
In-Reply-To: <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
Message-ID: <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>

RS wrote:
> The Windows download_history file seems to work correctly.? I hope I 
> don't need to edit it.

You've probably edited your options file by hand in the past, resulting 
in a mixture of weird line endings from a Windows editor, and 
appropriate ones when get_iplayer itself has updated it.  That's just my 
guess after going through exactly the same thing recently.  I'd bet you 
probably don't go editing your download_history file by hand very often 
so it's most likely fine.

> 
> 
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer
> 



Regards,

James Scholes
https://twitter.com/JamesScholes


From bap at shrdlu.com  Sat Mar  3 14:54:43 2018
From: bap at shrdlu.com (Bernard Peek)
Date: Sat, 3 Mar 2018 22:54:43 +0000
Subject: Format of options file
In-Reply-To: <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
Message-ID: <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>



On 03/03/18 17:33, James Scholes wrote:
> RS wrote:
>> The Windows download_history file seems to work correctly.? I hope I 
>> don't need to edit it.
>
> You've probably edited your options file by hand in the past, 
> resulting in a mixture of weird line endings from a Windows editor, 
> and appropriate ones when get_iplayer itself has updated it.? That's 
> just my guess after going through exactly the same thing recently.? 
> I'd bet you probably don't go editing your download_history file by 
> hand very often so it's most likely fine.

For what it's worth I noticed that under Windows the Wordpad program 
seems to understand UNIX style terminations and retains them.


-- 
Bernard Peek

   bap at shrdlu.com



From c.e.macfarlane at macfh.co.uk  Sat Mar  3 15:44:37 2018
From: c.e.macfarlane at macfh.co.uk (MacFH - C E Macfarlane)
Date: Sat, 3 Mar 2018 23:44:37 +0000
Subject: Format of options file
In-Reply-To: <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
Message-ID: <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>

Please see below ...

On 03/03/2018 22:54, Bernard Peek wrote:
>
> On 03/03/18 17:33, James Scholes wrote:
>> RS wrote:
>>> The Windows download_history file seems to work correctly.? I hope I 
>>> don't need to edit it.
>>
>> You've probably edited your options file by hand in the past, 
>> resulting in a mixture of weird line endings from a Windows editor, 
>> and appropriate ones when get_iplayer itself has updated it.? That's 
>> just my guess after going through exactly the same thing recently.? 
>> I'd bet you probably don't go editing your download_history file by 
>> hand very often so it's most likely fine.
>
> For what it's worth I noticed that under Windows the Wordpad program 
> seems to understand UNIX style terminations and retains them.

Yes, so does the excellent tryit-ware program Textpad, which is 
relatively cheap to license fully anyway, and for that you get just 
about the best Windows programming editor there is.

But as the OP is changing over to Linux, perhaps the de facto standard 
'dos2unix' would be a more appropriate recommendation. There are Windows 
versions, but almost certainly it would be available from the original 
Linux distro repository.? I can't recall what flavour of Linux the OP is 
running, but in Ubuntu and similar distros ...
 ??? ??? sudo apt-get install dos2unix
... should install it, and ...
 ??? ??? man dos2unix
... should give instructions for using it.



From richard22j at zoho.com  Sat Mar  3 16:24:12 2018
From: richard22j at zoho.com (RS)
Date: Sun, 4 Mar 2018 00:24:12 +0000
Subject: Format of options file
In-Reply-To: <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
Message-ID: <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>



On 03/03/18 23:44, MacFH - C E Macfarlane wrote:
> Please see below ...
> 
> On 03/03/2018 22:54, Bernard Peek wrote:
>>
>> On 03/03/18 17:33, James Scholes wrote:
>>> RS wrote:
>>>> The Windows download_history file seems to work correctly.? I hope I 
>>>> don't need to edit it.
>>>
>>> You've probably edited your options file by hand in the past, 
>>> resulting in a mixture of weird line endings from a Windows editor, 
>>> and appropriate ones when get_iplayer itself has updated it.? That's 
>>> just my guess after going through exactly the same thing recently. 
>>> I'd bet you probably don't go editing your download_history file by 
>>> hand very often so it's most likely fine.

Thanks for all the comments.  I don't remember ever having edited the 
options file manually.  Until today I did not even know the format.  In 
any event I have now looked at the original Windows options file.  It 
uses 0x0D0A (CRLF) as a line terminator throughout.  The 0x0A 
terminators have been introduced by running --prefs-add in Linux but it 
does not replace all the 0x0D0A terminators.

I have also just looked at the last few entries in the Windows 
download_history. They have 0x0A as a line terminator.  I used gedit to 
save a manageable number of entries, so it is possible gedit changed the 
line terminators.  To check that I edited and saved the Windows options 
file with gedit.  The 0x0A0D line terminators were preserved.

If download_history can use 0x0A as a line terminator in both Linux and 
Windows I can't see why the options file should not also.  The options 
file is internal to get_iplayer.  Users do not need to edit it directly.

Ralph mentioned POSIX but as far as I can see from the Wikipedia article 
there is a lot more to POSIX than converting 0x0A0D to 0x0A.  Cygwin is 
given as one example.
>>

> But as the OP is changing over to Linux, perhaps the de facto standard 
> 'dos2unix' would be a more appropriate recommendation. 

Thanks for the suggestion but I don't think I need to convert the 
download_history file (although gedit complained about invalid UTF-8 
characters) and thanks to Ralph's command I now have an options file 
which consistently uses 0x0D as a line terminator.

Best wishes
Richard




From njhp94 at gmail.com  Sat Mar  3 18:31:13 2018
From: njhp94 at gmail.com (Nick Payne)
Date: Sun, 4 Mar 2018 13:31:13 +1100
Subject: Format of options file
In-Reply-To: <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
Message-ID: <6ded29d5-af94-75bf-bcc7-27ab3c8dd73b@gmail.com>

On 4/03/2018 9:54 AM, Bernard Peek wrote:
>
> On 03/03/18 17:33, James Scholes wrote:
>> RS wrote:
>>> The Windows download_history file seems to work correctly.? I hope I
>>> don't need to edit it.
>>
>> You've probably edited your options file by hand in the past,
>> resulting in a mixture of weird line endings from a Windows editor,
>> and appropriate ones when get_iplayer itself has updated it.? That's
>> just my guess after going through exactly the same thing recently.?
>> I'd bet you probably don't go editing your download_history file by
>> hand very often so it's most likely fine.
>
> For what it's worth I noticed that under Windows the Wordpad program
> seems to understand UNIX style terminations and retains them.
The free Notepad++ editor does also. In fact, you can set in the program
preferences whether you want line endings to be formatted as the Windows
CRLF, Unix LF, or Mac CR.

Nick


From richard22j at zoho.com  Sun Mar  4 01:25:05 2018
From: richard22j at zoho.com (RS)
Date: Sun, 4 Mar 2018 09:25:05 +0000
Subject: Format of options file
In-Reply-To: <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
Message-ID: <a3341ce0-dc8f-8b6b-c625-727953f41f87@zoho.com>

On 04/03/18 00:24, I wrote:
> 
> If download_history can use 0x0A as a line terminator in both Linux and 
> Windows I can't see why the options file should not also.? The options 
> file is internal to get_iplayer.? Users do not need to edit it directly.
> 
I would add that now Linux fully supports NTFS it is not unreasonable to 
want to share data between Windows and Linux versions of programs and to 
go backwards and forwards between whatever happens to be convenient at 
the moment.  Thunderbird, Libre Office, Softmaker Teamviewer and 
browsers all make it possible, as long as you don't try to do it on the 
Windows c: drive.

I am writing this with Thunderbird sharing its message store between 
Windows 10 and ubuntu 16.04.  All I needed to do to set it up was to run 
thunderbird -p
to tell it where to store the messages.

Best wishes
Richard




From michaeltnorman at gmail.com  Sun Mar  4 04:40:23 2018
From: michaeltnorman at gmail.com (michael norman)
Date: Sun, 4 Mar 2018 12:40:23 +0000
Subject: Format of options file
In-Reply-To: <a3341ce0-dc8f-8b6b-c625-727953f41f87@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <a3341ce0-dc8f-8b6b-c625-727953f41f87@zoho.com>
Message-ID: <78eab836-d8da-64ed-1e4b-8fd4aeb6fc48@gmail.com>

On 04/03/18 09:25, RS wrote:
> On 04/03/18 00:24, I wrote:
>>
>> If download_history can use 0x0A as a line terminator in both Linux 
>> and Windows I can't see why the options file should not also.? The 
>> options file is internal to get_iplayer.? Users do not need to edit it 
>> directly.
>>
> I would add that now Linux fully supports NTFS it is not unreasonable to 
> want to share data between Windows and Linux versions of programs and to 
> go backwards and forwards between whatever happens to be convenient at 
> the moment.? Thunderbird, Libre Office, Softmaker Teamviewer and 
> browsers all make it possible, as long as you don't try to do it on the 
> Windows c: drive.
> 
> I am writing this with Thunderbird sharing its message store between 
> Windows 10 and ubuntu 16.04.? All I needed to do to set it up was to run 
> thunderbird -p
> to tell it where to store the messages.
> 
> Best wishes
> Richard

This is OT but the other way of running Linux and Windows on the same 
box is to use virtualisation.  Virtualbox

https://www.virtualbox.org/wiki/Downloads

is a free way of doing that and certainly works with Linux.  My 
experience is using Linux as host and Windows as guest.  Long time since 
I did it but I do know W7 works, I have not used W10.  The advantage 
over dual boot is that you can access both OS's at the same time, and 
you can certainly share directories between the two.  The disadvantage 
is that the virtual machine may lose some functionality ie its not 
suitable IME for playing video games or doing heavy processing of things 
like video files.  But for pretty much anything else it works fine.

Disclaimer, I am by no means an expert in this so please don't direct 
questions to me.  VB is very well documented and IME both openSUSE and 
Linuxmint have extensive community support where you can ask questions. 
I will assome that Windows users will be able to access similar forms of 
support.

As I said OT for this list but maybe germane to what has been said thus far.

M


From richard22j at zoho.com  Sun Mar  4 04:49:14 2018
From: richard22j at zoho.com (RS)
Date: Sun, 4 Mar 2018 12:49:14 +0000
Subject: Format of options file
In-Reply-To: <a3341ce0-dc8f-8b6b-c625-727953f41f87@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <a3341ce0-dc8f-8b6b-c625-727953f41f87@zoho.com>
Message-ID: <88bd3a49-5c17-d59a-2a54-b4e880112314@zoho.com>



On 04/03/18 09:25, I wrote:
> On 04/03/18 00:24, I wrote:
>>
>> If download_history can use 0x0A as a line terminator in both Linux 
>> and Windows I can't see why the options file should not also.? The 
>> options file is internal to get_iplayer.? Users do not need to edit it 
>> directly.
>>
> I would add that now Linux fully supports NTFS it is not unreasonable to 
> want to share data between Windows and Linux versions of programs and to 
> go backwards and forwards between whatever happens to be convenient at 
> the moment.? Thunderbird, Libre Office, Softmaker Teamviewer and 
> browsers all make it possible, as long as you don't try to do it on the 
> Windows c: drive.
> 
The discussion in this thread
https://forums.squarepenguin.co.uk/showthread.php?tid=1690
suggests it may be possible to put the profile directory somewhere it 
could be read and written to by both Windows and Linux (not in c: or 
/home).  The incompatibility in file formats for the options file 
between Windows and Linux would prevent that idea from working.

Best wishes
Richard




From bap at shrdlu.com  Sun Mar  4 05:22:59 2018
From: bap at shrdlu.com (Bernard Peek)
Date: Sun, 4 Mar 2018 13:22:59 +0000
Subject: Format of options file
In-Reply-To: <78eab836-d8da-64ed-1e4b-8fd4aeb6fc48@gmail.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <a3341ce0-dc8f-8b6b-c625-727953f41f87@zoho.com>
 <78eab836-d8da-64ed-1e4b-8fd4aeb6fc48@gmail.com>
Message-ID: <5286bab8-f0bb-2cb0-9e01-399721d21649@shrdlu.com>



On 04/03/18 12:40, michael norman wrote:
> On 04/03/18 09:25, RS wrote:
>> On 04/03/18 00:24, I wrote:
>>>
>>> If download_history can use 0x0A as a line terminator in both Linux 
>>> and Windows I can't see why the options file should not also.? The 
>>> options file is internal to get_iplayer.? Users do not need to edit 
>>> it directly.
>>>
>> I would add that now Linux fully supports NTFS it is not unreasonable 
>> to want to share data between Windows and Linux versions of programs 
>> and to go backwards and forwards between whatever happens to be 
>> convenient at the moment.? Thunderbird, Libre Office, Softmaker 
>> Teamviewer and browsers all make it possible, as long as you don't 
>> try to do it on the Windows c: drive.
>>
>> I am writing this with Thunderbird sharing its message store between 
>> Windows 10 and ubuntu 16.04.? All I needed to do to set it up was to 
>> run thunderbird -p
>> to tell it where to store the messages.
>>
>> Best wishes
>> Richard
>
> This is OT but the other way of running Linux and Windows on the same 
> box is to use virtualisation.? Virtualbox
>
> https://www.virtualbox.org/wiki/Downloads

I mainly use Linux with get_iplayer running on a SuSE server but also 
use a Mint workstation accessing the server using SSH. I have virtual 
machines on the workstation using VB. I mostly run virtual XP machines 
but I have tried W7, W8 and W10 successfully. I'm thinking of setting up 
some virtual NT4 systems as they are likely to use less resources and 
will run most of the programs I am interested in.



-- 

Bernard Peek

   bap at shrdlu.com



From ralph at inputplus.co.uk  Sun Mar  4 05:39:36 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sun, 04 Mar 2018 13:39:36 +0000
Subject: FAO BBC: Double-encoded UTF-8 in Programme's JSON.
Message-ID: <20180304133936.D29BD207ED@orac.inputplus.co.uk>

Hi,

I noticed get_iplayer showing

    Rotha?? M??ra an tSaoil: Series 1

and wondered if it was a bug, but the BBC's JSON has

    $ curl -sS https://www.bbc.co.uk/programmes/b09w6dhm.json |
    > grep -o '"Roth[^"]*"'
    "Rotha\u00c3\u00ad M\u00c3\u00b3ra an tSaoil"
    "Rotha\u00c3\u00ad M\u00c3\u00b3ra an tSaoil"
    $

and get_iplayer is correctly showing U+c3 and U+ad after `Rotha'.

The problem is the BBC have taken a UTF-8 encoding of the intended rune
and encoded it again as UTF-8.

    $ iconv -f utf-8 -t ucs-2be <<<$'\xc3\xad \xc3\xb3' |
    > od --endian=big -tx2
    0000000 00ed 0020 00f3 000a
    0000010
    $

Thus the title is meant to be

    $ printf 'Rotha\u00ed M\u00f3ra an tSaoil\n'
    Rotha? M?ra an tSaoil
    $

Can a BBC lurker please see if they can stop it happening.  Thanks.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From c.e.macfarlane at macfh.co.uk  Sun Mar  4 07:52:31 2018
From: c.e.macfarlane at macfh.co.uk (MacFH - C E Macfarlane)
Date: Sun, 4 Mar 2018 15:52:31 +0000
Subject: Format of options file
In-Reply-To: <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
Message-ID: <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>

Please see below ...

On 04/03/2018 00:24, RS wrote:
>
> On 03/03/18 23:44, MacFH - C E Macfarlane wrote:
>
>> But as the OP is changing over to Linux, perhaps the de facto 
>> standard 'dos2unix' would be a more appropriate recommendation. 
>
> Thanks for the suggestion but I don't think I need to convert the 
> download_history file (although gedit complained about invalid UTF-8 
> characters) and thanks to Ralph's command I now have an options file 
> which consistently uses 0x0D as a line terminator.

You seem to be confused about which line terminators are used by each 
operating system, so to confirm:
 ??? OS??? ??? Mnem??? Hex code
 ??? ---------------------------------
 ??? Linux??? LF ??? ????? 0x0A
 ??? Mac???? CR?????????? 0x0D (I believe)
 ??? Win ? ? CRLF ????? 0x0D0x0A

I can't recall ever trying to copy windows GiP data over to Linux, so I 
can't comment on whether the 'wrong' line terminator will make any 
difference, but, as the engine interpreting the files is Perl rather 
than the underlying OS, perhaps it won't matter to GiP at all.? However, 
if you try to edit the file in a native editor such as GEdit, then yes, 
you should expect problems.?? I think I'd be inclined to make a backup 
copy of the entire GiP data directory, I mean the one usually called 
.get_iplayer containing the options and download_history files that 
we've been discussing, and then run dos2unix on options, 
download_history, and either delete the caches so that they are remade, 
or run dos2unix on those as well.

Regards Charles.


From richard22j at zoho.com  Sun Mar  4 11:59:52 2018
From: richard22j at zoho.com (RS)
Date: Sun, 4 Mar 2018 19:59:52 +0000
Subject: Format of options file
In-Reply-To: <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
Message-ID: <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>



On 04/03/18 15:52, MacFH - C E Macfarlane wrote:
> Please see below ...
> 
> On 04/03/2018 00:24, RS wrote:
>>
>> On 03/03/18 23:44, MacFH - C E Macfarlane wrote:
>>
>>> But as the OP is changing over to Linux, perhaps the de facto 
>>> standard 'dos2unix' would be a more appropriate recommendation. 
>>
>> Thanks for the suggestion but I don't think I need to convert the 
>> download_history file (although gedit complained about invalid UTF-8 
>> characters) and thanks to Ralph's command I now have an options file 
>> which consistently uses 0x0D as a line terminator.
> 
> You seem to be confused about which line terminators are used by each 
> operating system, so to confirm:
>  ??? OS??? ??? Mnem??? Hex code
>  ??? ---------------------------------
>  ??? Linux??? LF ??? ????? 0x0A
>  ??? Mac???? CR?????????? 0x0D (I believe)
>  ??? Win ? ? CRLF ????? 0x0D0x0A
> 

You're right I am a bit confused, but not about the line termination 
conventions for each OS, although I can't speak for the Mac.

What I said the download_history file was wrong.  I have now looked at 
it with a hex editor which allows me to view the whole file.  Most of it 
was written in Windows and has CRLF as a line terminator.  The most 
recent records, appended to the end, written in Linux, have LF as a line 
terminator.  The important thing is that the discrepancy does not stop 
it working.  The entries are indexed by pid.  The script is able to find 
the pid whether it is preceded by LF or CRLF, and identify the programme 
as already in history.  The mixed file works both in Linux and in Windows.

In the case of the options file the discrepancy in line termination does 
prevent some options from being interpreted correctly.  To find out why 
I need to compare the Perl code for parsing the download_history file 
with that for parsing the options file.  I have not yet managed to find it.

As for using win2unix to convert, if you look at the win2unix 
documentation you will see it is not that simple.  It is possible to use 
it just to change the line terminations.  You also need to think about 
character sets.  Some of the comments in the code say UTF-8 is used for 
Linux and cp850 for Windows.  If I remember from 30 odd years ago that 
was the oldest MSDOS code page.  Should that be converted too?  If I 
want to go backwards and forwards I don't want to keep converting.

Best wishes
Richard



From ralph at inputplus.co.uk  Sun Mar  4 13:34:37 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sun, 04 Mar 2018 21:34:37 +0000
Subject: Format of options file
In-Reply-To: <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
Message-ID: <20180304213437.0FA5B20A28@orac.inputplus.co.uk>

Hi Richard,

> What I said the download_history file was wrong.  I have now looked at
> it with a hex editor which allows me to view the whole file.  Most of
> it was written in Windows and has CRLF as a line terminator.  The most
> recent records, appended to the end, written in Linux, have LF as a
> line terminator.  The important thing is that the discrepancy does not
> stop it working.  The entries are indexed by pid.  The script is able
> to find the pid whether it is preceded by LF or CRLF, and identify the
> programme as already in history.  The mixed file works both in Linux
> and in Windows.

It probably doesn't.

Let's forget Mac for the moment.  Linux text files are POSIX text files;
zero or more lines, each terminated by a LF.  See ascii(7).  DOS ones
use CR followed by LF at the end of each line.

Thus a DOS text file looks like a text file to Linux, but one where the
last character at the end of each line, just before the LF, is a CR,
which is just another possible byte that could be within the line.  If
the operation you're doing doesn't mind that the CR is present then it
works;  reading the PID from the start of the line and using it as a
key.  But if you're using the data that runs to the end of the line then
you'll pick up the unwanted CR and that may do things like corrupt the
output.

    $ seq 3
    1
    2
    3
    $ seq 3 | grep '^2$'
    2
    $ seq 3 | sed 's/$/\r/'
    1
    2
    3
    $ seq 3 | sed 's/$/\r/' | grep '^2$'
    $ 
    $ seq 3 | sed 's/$/\r/' | sed -n l
    1\r$
    2\r$
    3\r$
    $ 

The first grep works.  Appending CR with sed doesn't change the apparent
appearance, but it's there as a byte so the second grep fails to find
the line that contains just `2'.  sed's `l' command shows up the
problem.

So the new lines added by Linux to the end of the history file might
well look like one long, unterminated as there's no CR LF, line to
Windows.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From c.e.macfarlane at macfh.co.uk  Sun Mar  4 13:52:00 2018
From: c.e.macfarlane at macfh.co.uk (MacFH - C E Macfarlane)
Date: Sun, 4 Mar 2018 21:52:00 +0000
Subject: Format of options file
In-Reply-To: <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
Message-ID: <9a53641f-5510-f3e3-f0b2-470d1bc353f4@macfh.co.uk>

Please see below ...

On 04/03/2018 19:59, RS wrote:
>
> On 04/03/18 15:52, MacFH - C E Macfarlane wrote:
>>
>> On 04/03/2018 00:24, RS wrote:
>>>
>>> On 03/03/18 23:44, MacFH - C E Macfarlane wrote:
>>>
>>>> But as the OP is changing over to Linux, perhaps the de facto 
>>>> standard 'dos2unix' would be a more appropriate recommendation. 
>>>
>>> Thanks for the suggestion but I don't think I need to convert the 
>>> download_history file (although gedit complained about invalid UTF-8 
>>> characters) and thanks to Ralph's command I now have an options file 
>>> which consistently uses 0x0D as a line terminator.
>>
>> You seem to be confused about which line terminators are used by each 
>> operating system, so to confirm:
>> ???? OS??? ??? Mnem??? Hex code
>> ???? ---------------------------------
>> ???? Linux??? LF ??? ????? 0x0A
>> ???? Mac???? CR?????????? 0x0D (I believe)
>> ???? Win ? ? CRLF ????? 0x0D0x0A
>
> You're right I am a bit confused, but not about the line termination 
> conventions for each OS, although I can't speak for the Mac.
>
> What I said the download_history file was wrong.? I have now looked at 
> it with a hex editor which allows me to view the whole file.? Most of 
> it was written in Windows and has CRLF as a line terminator.? The most 
> recent records, appended to the end, written in Linux, have LF as a 
> line terminator.? The important thing is that the discrepancy does not 
> stop it working.? The entries are indexed by pid.? The script is able 
> to find the pid whether it is preceded by LF or CRLF, and identify the 
> programme as already in history.? The mixed file works both in Linux 
> and in Windows.
>
> In the case of the options file the discrepancy in line termination 
> does prevent some options from being interpreted correctly.? To find 
> out why I need to compare the Perl code for parsing the 
> download_history file with that for parsing the options file.? I have 
> not yet managed to find it.

Most probably it is those options which are passed to an external 
program that cause difficulty? -? my guess, I stress that it is a guess? 
-? the program is getting confused by the unexpected CR 0x0D when it is 
expecting only LF 0x0A, and may be trying to interpret the CR as part of 
the parameter it is being given.

> As for using win2unix to convert, if you look at the win2unix 
> documentation you will see it is not that simple.? It is possible to 
> use it just to change the line terminations.? You also need to think 
> about character sets.? Some of the comments in the code say UTF-8 is 
> used for Linux and cp850 for Windows.? If I remember from 30 odd years 
> ago that was the oldest MSDOS code page.? Should that be converted 
> too?? If I want to go backwards and forwards I don't want to keep 
> converting.

The Windows ANSI and UTF-8 character sets are sufficiently close that I 
wouldn't worry about any need to convert, just do the line-termination.? 
Given two directories, .get_iplayer.bak and .get_iplayer, beneath the 
present working directory HOME aka ~, this command should do what you 
want (I've doubled up the spaces for better clarity of the syntax):
 ??? for? F? in? options? download_history? *.cache? ;? do dos2unix -n? 
.get_iplayer.bak/$F? .get_iplayer/$F? ;? done

Regards Charles.



From c.e.macfarlane at macfh.co.uk  Sun Mar  4 14:18:27 2018
From: c.e.macfarlane at macfh.co.uk (MacFH - C E Macfarlane)
Date: Sun, 4 Mar 2018 22:18:27 +0000
Subject: Format of options file
In-Reply-To: <9a53641f-5510-f3e3-f0b2-470d1bc353f4@macfh.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <9a53641f-5510-f3e3-f0b2-470d1bc353f4@macfh.co.uk>
Message-ID: <40c3e58b-c36b-299a-77bd-005fd65ff7e4@macfh.co.uk>

Please see correction below ...

On 04/03/2018 21:52, MacFH - C E Macfarlane wrote:
>
> ??? for? F? in? options? download_history? *.cache? ;? do dos2unix -n? 
> .get_iplayer.bak/$F? .get_iplayer/$F? ;? done

I've just realised that this may no do the *.cache files because they 
are in a different directory than the one where you are giving the 
command, so safer would be:
 ??? for? F? in? options? download_history? radio.cache? tv.cache? ; do 
dos2unix -n? .get_iplayer.bak/$F? .get_iplayer/$F? ;? done

Apologies for the slight error.
Regards Charles.



From richard22j at zoho.com  Mon Mar  5 01:52:53 2018
From: richard22j at zoho.com (RS)
Date: Mon, 5 Mar 2018 09:52:53 +0000
Subject: Format of options file
In-Reply-To: <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
Message-ID: <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>

On 04/03/18 21:34, Ralph Corderoy wrote:
> 
> 
> It probably doesn't.
> 
> Let's forget Mac for the moment.  Linux text files are POSIX text files;
> zero or more lines, each terminated by a LF.  See ascii(7).  DOS ones
> use CR followed by LF at the end of each line.
> 
> Thus a DOS text file looks like a text file to Linux, but one where the
> last character at the end of each line, just before the LF, is a CR,
> which is just another possible byte that could be within the line.  If
> the operation you're doing doesn't mind that the CR is present then it
> works;  reading the PID from the start of the line and using it as a
> key.  But if you're using the data that runs to the end of the line then
> you'll pick up the unwanted CR and that may do things like corrupt the
> output.

Hi Ralph

Essentially you seem to be saying I can't have a CR in a Linux text file 
because it is non-standard.  It seems a strange standard that prevents 
data interchange rather than facilitates it. My approach would be to 
ignore the CR or to recognise three possible end of record markers, but 
it seems that is not allowed.

One article I read mentioned that errors can be caused by strict 
adherence to a standard so that the last line of data is ignored if it 
does not end with a newline.

I can exchange data in elaborate formats like DOCX, ODT or H.264 but I 
can't as a text file.

What is the answer?  How can Windows and Linux exchange data in text 
files without requiring the user to run them through external conversion 
programs?

Best wishes
Richard



From alan at soulman1949.com  Mon Mar  5 02:25:58 2018
From: alan at soulman1949.com (Alan Milewczyk)
Date: Mon, 5 Mar 2018 18:25:58 +0800
Subject: Format of options file
In-Reply-To: <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
Message-ID: <08027b7d-d266-225e-c577-c15f5b33fb33@soulman1949.com>

On 05/03/2018 17:52, RS wrote:
>
> What is the answer?? How can Windows and Linux exchange data in text 
> files without requiring the user to run them through external 
> conversion programs?

I've been following the gist of the thread but not the detail. I am 
wondering whether the answer to your second question is that they 
can't!!! Although GIP can be run on Windows, MacOS or Linux, each one 
looks like it treats certain issues differently.? Not a problem when 
you're sticking with the one OS but if you're trying to share the data 
on different OSs, maybe you're trying to do something that cannot be 
done seamlessly and that you have to use a conversion program.

Just my pennyworth..

Alan


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus



From roger at firedrake.org  Mon Mar  5 03:07:40 2018
From: roger at firedrake.org (Roger Bell_West)
Date: Mon, 5 Mar 2018 11:07:40 +0000
Subject: Format of options file
In-Reply-To: <08027b7d-d266-225e-c577-c15f5b33fb33@soulman1949.com>
References: <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <08027b7d-d266-225e-c577-c15f5b33fb33@soulman1949.com>
Message-ID: <20180305110740.xzdgrijkpj2mfnqe@firedrake.org>

On Mon, Mar 05, 2018 at 06:25:58PM +0800, Alan Milewczyk wrote:
>I've been following the gist of the thread but not the detail. I am wondering
>whether the answer to your second question is that they can't!!!

It would be possible to rewrite get_iplayer to make it agnostic with
respect to line endings.

This is, of course, only useful for those users who transfer its
working files between Windows and Linux machines.

If you are one of those users, feel free to write a patch.


From clanlaw at gmail.com  Mon Mar  5 03:38:15 2018
From: clanlaw at gmail.com (Colin Law)
Date: Mon, 5 Mar 2018 11:38:15 +0000
Subject: Format of options file
In-Reply-To: <20180303110949.B91381FE27@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
Message-ID: <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>

On 3 March 2018 at 11:09, Ralph Corderoy <ralph at inputplus.co.uk> wrote:
>
> Hi Richard,
>
> > ') - using defaultalue for --ffmpeg-loglevel ('info
> >
> > If I enter the options again with --prefs-add they work correctly, so
> > it is no big deal.  Any ideas what is happening?
>
> That line above seems to be overprinting the ?')? at the start of the
> line suggesting there's an ASCII CR, carriage return, after the `info'
> as if a Unix system was reading a POSIX text file of lines ending in
> ASCII LF, linefeed, but being fed a DOS one with CR LF pairs.


Are we certain that the problem has been correctly analysed?  The line
shown above appears to have a CR after info and before the quote sign,
which would not be valid on either system.  In practice it may work on
Windows if the code just ignores CR characters wherever they are.  Is
it certain that a correctly formatted windows file will definitely not
work on Linux?

On 3 March 2018 at 11:09, Ralph Corderoy <ralph at inputplus.co.uk> wrote:
> Hi Richard,
>
>> ') - using defaultalue for --ffmpeg-loglevel ('info
>>
>> If I enter the options again with --prefs-add they work correctly, so
>> it is no big deal.  Any ideas what is happening?
>
> That line above seems to be overprinting the ?')? at the start of the
> line suggesting there's an ASCII CR, carriage return, after the `info'
> as if a Unix system was reading a POSIX text file of lines ending in
> ASCII LF, linefeed, but being fed a DOS one with CR LF pairs.
>
> `od -c foo' will show them, and they can be removed with `sed -i
> 's/\r$//' foo'.
>
> --
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


From ralph at inputplus.co.uk  Mon Mar  5 04:02:39 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Mon, 05 Mar 2018 12:02:39 +0000
Subject: Format of options file
In-Reply-To: <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
Message-ID: <20180305120239.13862208E9@orac.inputplus.co.uk>

Hi Colin,

> > > ') - using defaultalue for --ffmpeg-loglevel ('info
> >
> > That line above seems to be overprinting the ?')? at the start of
> > the line suggesting there's an ASCII CR, carriage return, after the
> > `info' as if a Unix system was reading a POSIX text file of lines
> > ending in ASCII LF, linefeed, but being fed a DOS one with CR LF
> > pairs.
>
> Are we certain that the problem has been correctly analysed?

Yes, I did.  :-)

> The line shown above appears to have a CR after info and before the
> quote sign, which would not be valid on either system.

Because the value is ?info\r? and it's being printed surrounded by
?('...')?.

    WARNING: invalid value for --ffmpeg-loglevel ('$opt->{ffmpegloglevel}') - using default\n

becomes

    WARNING: invalid value for --ffmpeg-loglevel ('info
    ') - using default

becomes

    ') - using defaultalue for --ffmpeg-loglevel ('info

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From clanlaw at gmail.com  Mon Mar  5 04:14:18 2018
From: clanlaw at gmail.com (Colin Law)
Date: Mon, 5 Mar 2018 12:14:18 +0000
Subject: Format of options file
In-Reply-To: <20180305120239.13862208E9@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
 <20180305120239.13862208E9@orac.inputplus.co.uk>
Message-ID: <CAL=0gLtT+1HHLQCL=5Ae86-Wj6Qy5Zr1dD_H13rGGee5+CBoUw@mail.gmail.com>

On 5 March 2018 at 12:02, Ralph Corderoy <ralph at inputplus.co.uk> wrote:
> Hi Colin,
>
>> > > ') - using defaultalue for --ffmpeg-loglevel ('info
>> >
>> > That line above seems to be overprinting the ?')? at the start of
>> > the line suggesting there's an ASCII CR, carriage return, after the
>> > `info' as if a Unix system was reading a POSIX text file of lines
>> > ending in ASCII LF, linefeed, but being fed a DOS one with CR LF
>> > pairs.
>>
>> Are we certain that the problem has been correctly analysed?
>
> Yes, I did.  :-)
>
>> The line shown above appears to have a CR after info and before the
>> quote sign, which would not be valid on either system.
>
> Because the value is ?info\r? and it's being printed surrounded by
> ?('...')?.
>
>     WARNING: invalid value for --ffmpeg-loglevel ('$opt->{ffmpegloglevel}') - using default\n
>
> becomes
>
>     WARNING: invalid value for --ffmpeg-loglevel ('info
>     ') - using default
>
> becomes
>
>     ') - using defaultalue for --ffmpeg-loglevel ('info

Yes, I agree.  For anyone who knows the code it should not be
difficult to ignore CR characters after the text. I expect it is
already ignoring white space after it.  Any patch takes time to work
out and to test though.

Colin

>
> --
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


From alan at soulman1949.com  Mon Mar  5 04:20:42 2018
From: alan at soulman1949.com (Alan Milewczyk)
Date: Mon, 5 Mar 2018 20:20:42 +0800
Subject: Format of options file
In-Reply-To: <20180305110740.xzdgrijkpj2mfnqe@firedrake.org>
References: <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <08027b7d-d266-225e-c577-c15f5b33fb33@soulman1949.com>
 <20180305110740.xzdgrijkpj2mfnqe@firedrake.org>
Message-ID: <f56487e8-8394-35d6-df28-b7b97027bb83@soulman1949.com>

On 05/03/2018 19:07, Roger Bell_West wrote:
> On Mon, Mar 05, 2018 at 06:25:58PM +0800, Alan Milewczyk wrote:
>> I've been following the gist of the thread but not the detail. I am wondering
>> whether the answer to your second question is that they can't!!!
Sorry my statement should be qualified? by the caveat "with GIP as it 
currently stands".
> It would be possible to rewrite get_iplayer to make it agnostic with
> respect to line endings.
>
> This is, of course, only useful for those users who transfer its
> working files between Windows and Linux machines.
>
> If you are one of those users, feel free to write a patch.

This could always be raised as a request in the main GIP forum, but I 
wouldn't hold your breath - I've seen many other requests receive short 
shrift.... but you won't know unless you ask.

A

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus



From c.e.macfarlane at macfh.co.uk  Mon Mar  5 04:55:43 2018
From: c.e.macfarlane at macfh.co.uk (MacFH - C E Macfarlane)
Date: Mon, 5 Mar 2018 12:55:43 +0000
Subject: Format of options file
In-Reply-To: <20180305120239.13862208E9@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
 <20180305120239.13862208E9@orac.inputplus.co.uk>
Message-ID: <7c952a46-929f-6815-f803-821527c6e93d@macfh.co.uk>

Please see below ...

On 05/03/2018 12:02, Ralph Corderoy wrote:
> Hi Colin,
>
>>>> ') - using defaultalue for --ffmpeg-loglevel ('info
>>> That line above seems to be overprinting the ?')? at the start of
>>> the line suggesting there's an ASCII CR, carriage return, after the
>>> `info' as if a Unix system was reading a POSIX text file of lines
>>> ending in ASCII LF, linefeed, but being fed a DOS one with CR LF
>>> pairs.
>> Are we certain that the problem has been correctly analysed?
> Yes, I did.  :-)
>
>> The line shown above appears to have a CR after info and before the
>> quote sign, which would not be valid on either system.
> Because the value is ?info\r? and it's being printed surrounded by
> ?('...')?.
>
>      WARNING: invalid value for --ffmpeg-loglevel ('$opt->{ffmpegloglevel}') - using default\n
>
> becomes
>
>      WARNING: invalid value for --ffmpeg-loglevel ('info
>      ') - using default
>
> becomes
>
>      ') - using defaultalue for --ffmpeg-loglevel ('info

Seems valid reasoning to me.

Some people may be misunderstanding the precise meaning of 'Carriage 
Return'.? I'm not 100% up on my telecoms history, but AIUI ...

CR, LF, etc are ASCII acronyms, it itself being one for American 
Standard Code for Information Interchange, 
https://en.wikipedia.org/wiki/ASCII, which was a coding designed 
originally to allow early telecoms equipment to inter-operate. 'Carriage 
Return', CR, meant precisely that? -? in other words, move the slidable 
teleprinter carriage holding the paper to the right so that the fixed 
print head is at the left margin position.? LineFeed, LF, also meant 
exactly that, rotate the carriage roller to move the printing position 
to the next line.? Thus to print a new line of teleprinter output, both 
a CR and a LF are required.? DOS followed by Windows have always stuck 
by the original meaning of these codes, and thus both use both codes at 
the end of lines of text.? Arguably the oddballs are Linux and perhaps 
Macs, using only one.? This would mean that if their text was sent 
directly to a teleprinter, the output would not be what is required, but 
of course these days that would seldom if ever be done.? However, when 
printing CRs to the Linux console, as Ralph has explained above, Linux 
still retains the original meaning of CR, and moves the cursor back to 
the beginning of the line, and in the absence of a line feed immediate 
following, any other text following overprints? the beginning of the 
same line.? Linux could be argued to have subverted both codes, because 
not only does a LF move the cursor to the next line of text, but an 
implicit CR is also performed, which is why having only a LF at the end 
of each line of text works in Linux, but what really throws the spanner 
in the works is that Linux counts a CR preceding a LF as part of the 
text of the line, not as part of the line termination, hence the 
original problem and subsequent rather convoluted thread.



From richard22j at zoho.com  Mon Mar  5 05:15:22 2018
From: richard22j at zoho.com (RS)
Date: Mon, 5 Mar 2018 13:15:22 +0000
Subject: Format of options file
In-Reply-To: <CAL=0gLtT+1HHLQCL=5Ae86-Wj6Qy5Zr1dD_H13rGGee5+CBoUw@mail.gmail.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
 <20180305120239.13862208E9@orac.inputplus.co.uk>
 <CAL=0gLtT+1HHLQCL=5Ae86-Wj6Qy5Zr1dD_H13rGGee5+CBoUw@mail.gmail.com>
Message-ID: <6c1db948-df82-d17f-dc04-aa44759d9933@zoho.com>

On 05/03/2018 12:14, Colin Law wrote:

> 
> Yes, I agree.  For anyone who knows the code it should not be
> difficult to ignore CR characters after the text. I expect it is
> already ignoring white space after it.  Any patch takes time to work
> out and to test though.
> 
For history (I haven't yet found where it processes the options file) it 
uses chomp at line 3717 (v3.12) to strip off any unwanted characters. 
If I have understood this article correctly
http://perldoc.perl.org/perlport.html#Newlines
chomp should work portably with default settings.  I am not sure I have 
fully understood the article it links to about chomp VARIABLE,  but I 
can't see anywhere the variable is set.

Best wishes
Richard




From ralph at inputplus.co.uk  Mon Mar  5 05:19:53 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Mon, 05 Mar 2018 13:19:53 +0000
Subject: Format of options file
In-Reply-To: <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
Message-ID: <20180305131953.BA23B208E9@orac.inputplus.co.uk>

Hi Richard,

> > Let's forget Mac for the moment.  Linux text files are POSIX text
> > files; zero or more lines, each terminated by a LF.  See ascii(7).
> > DOS ones use CR followed by LF at the end of each line.
> > 
> > Thus a DOS text file looks like a text file to Linux, but one where
> > the last character at the end of each line, just before the LF, is a
> > CR, which is just another possible byte that could be within the
> > line.  If the operation you're doing doesn't mind that the CR is
> > present then it works;  reading the PID from the start of the line
> > and using it as a key.  But if you're using the data that runs to
> > the end of the line then you'll pick up the unwanted CR and that may
> > do things like corrupt the output.
>
> Essentially you seem to be saying I can't have a CR in a Linux text
> file because it is non-standard.

Nope, I'm not saying that.  `Linux text files are POSIX text files; zero
or more lines, each terminated by a LF.'  Here's a one-line POSIX text
file.

    $ sed -n l oneline
    one\rtwo\rthree$
    $ cat oneline
    three
    $

CR is just another character, a perfectly valid one, that's data as
opposed to the LF that's the line's terminator and meta-data.

> It seems a strange standard that prevents data interchange rather than
> facilitates it.

It is an excellent standard and ensures data interchange between the
hundreds of Unix text-processing tools that made Unix very popular.

> One article I read mentioned that errors can be caused by strict
> adherence to a standard so that the last line of data is ignored if it
> does not end with a newline.

Emacs is a non-Unix text editor that was ported to Unix, but kept its
foreign text-file format that uses line separators, not terminators.
Emacs users pollute the Unix filesystem with non-text files that have
trailing bytes beyond the zero-or-more Unix text lines unless they
change Emacs's defaults.  Unix programs are not at fault here, nor is
their `strict' adherence;  why bother otherwise?

> My approach would be to ignore the CR or to recognise three possible
> end of record markers, but it seems that is not allowed.

It certainly is allowed.  It would be the Richard Text File Format.
Programs could code to handle it.  More precise specification would be
required.  Is the first line-ending encountered from the start of the
file the one that must be used thence?  Or may each line differ?  RTTF
is incompatible with reading the POSIX `oneline' text file above so
conversion would be required, presumably by some means of escaping the
CR?  And escaping the escape character?  And converting files that were
already using the escape character to not mean escape.
https://xkcd.com/927/

> I can exchange data in elaborate formats like DOCX, ODT or H.264 but I
> can't as a text file.

They are file formats designed for an application, or a standard for
interchange.  A text file is an operating-system file format, and they
show their lineage.  CPM and DOS let the needs of the hardware leak into
the file format, wasting a byte each time.  Before them, Unix, really
Multics, bettered that by having a logical representation and leaving
the hardware's needs to the device driver.

I understand macOS, the latest name for it, has moved to POSIX's format.

> What is the answer?  How can Windows and Linux exchange data in text
> files without requiring the user to run them through external
> conversion programs?

A program can code for one format everywhere, e.g. POSIX, but that then
depends on editors that stick with POSIX format on non-POSIX systems.
They exist and are becoming ever more common.  Or it can do as Python as
done and try to adapt on the fly, but that corrupts some POSIX text
files on reading, which is why Python lets you disable it.

Because C has lines terminated by LF, like Unix, C libraries on Windows
convert CRLF to LF on reading a text file, and do the reverse on
writing.  Perl implements that.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From bap at shrdlu.com  Mon Mar  5 05:33:10 2018
From: bap at shrdlu.com (Bernard Peek)
Date: Mon, 5 Mar 2018 13:33:10 +0000
Subject: Format of options file
In-Reply-To: <7c952a46-929f-6815-f803-821527c6e93d@macfh.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
 <20180305120239.13862208E9@orac.inputplus.co.uk>
 <7c952a46-929f-6815-f803-821527c6e93d@macfh.co.uk>
Message-ID: <7120ef33-a03f-2116-3b81-c405265e7f5f@shrdlu.com>



On 05/03/18 12:55, MacFH - C E Macfarlane wrote:
> Seems valid reasoning to me.
>
> Some people may be misunderstanding the precise meaning of 'Carriage 
> Return'.? I'm not 100% up on my telecoms history, but AIUI ...
>

This is a long-standing problem for any organisation that exchanges text 
files between systems using different formats. When I wrote interface 
formats I mandated that a receiving program will handle any permutation 
of <CR> <LF> in any permutation. It's even a problem for places that 
think XML solves all standards problems.

-- 

Bernard Peek

   bap at shrdlu.com



From ralph at inputplus.co.uk  Mon Mar  5 05:36:49 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Mon, 05 Mar 2018 13:36:49 +0000
Subject: Format of options file
In-Reply-To: <7c952a46-929f-6815-f803-821527c6e93d@macfh.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
 <20180305120239.13862208E9@orac.inputplus.co.uk>
 <7c952a46-929f-6815-f803-821527c6e93d@macfh.co.uk>
Message-ID: <20180305133649.DAEC4208E9@orac.inputplus.co.uk>

Hi MacFH,

> 'Carriage Return', CR, meant precisely that? -? in other words, move
> the slidable teleprinter carriage holding the paper to the right so
> that the fixed print head is at the left margin position.? LineFeed,
> LF, also meant exactly that, rotate the carriage roller to move the
> printing position to the next line.

Those of us that used manual typewriters, that pre-date fangled
teleprinters, will recognise the two actions that could be done by the
lever to the left.  It would feed the paper to the next line when
pressed, and when continued to be pushed it would slide the carriage
along so it returned to the start.  It was the `Line feed/Carriage
return lever'.

> Arguably the oddballs are Linux and perhaps Macs, using only one.?
> This would mean that if their text was sent directly to a teleprinter,
> the output would not be what is required, but of course these days
> that would seldom if ever be done.

But in those days of Multics and Unix that's precisely what was done
with Unix text files, terminated by just LF, and yet the text didn't
stagger across the page because a device driver was correctly the thing
that knew what a particular device needed to be sent as bytes;  the file
format wasn't the place for that.  Other characteristics include a
timing delay to allow for the carriage to return before sending more
bytes.

> Linux could be argued to have subverted both codes, because not only
> does a LF move the cursor to the next line of text, but an implicit CR
> is also performed

This happens in a terminal window for the same reason it happens on a
teleprinter;  the TTY is so configured and the device driver obeys.  I
can say this particular device doesn't need a CR sent to it before every
LF.

    $ stty onlcr; printf 'foo\nbar\n'; stty -onlcr; printf 'foo\nbar\n'
    foo
    bar
    foo
       bar
	  $

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From richard22j at zoho.com  Mon Mar  5 05:46:10 2018
From: richard22j at zoho.com (RS)
Date: Mon, 5 Mar 2018 13:46:10 +0000
Subject: Format of options file
In-Reply-To: <20180305131953.BA23B208E9@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
Message-ID: <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>

On 05/03/2018 13:19, Ralph Corderoy wrote:
> 
>> My approach would be to ignore the CR or to recognise three possible
>> end of record markers, but it seems that is not allowed.
> 
> It certainly is allowed.  It would be the Richard Text File Format.
> Programs could code to handle it.  More precise specification would be
> required.  Is the first line-ending encountered from the start of the
> file the one that must be used thence?  Or may each line differ? 

Hi Ralph

I suggested ignoring CR because they are there.  Ideally they would not 
be there.  The files are internal to get_iplayer so they can be in any 
format.  I would have thought it sensible to use LF as a line terminator 
for get_iplayer's internal files across all operating systems.  The 
question is how the CR are getting into the files in Windows.

Since the script uses
print \n
the answer must be that Perl is translating \n as CR LF in Windows.  If 
Perl can then handle the variants portably with chomp VARIABLE that's 
fine, but it doesn't seem to be working properly at the moment. 
Otherwise the solution may be to replace \n with \015.

Best wishes
Richard



From ralph at inputplus.co.uk  Mon Mar  5 06:19:13 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Mon, 05 Mar 2018 14:19:13 +0000
Subject: Format of options file
In-Reply-To: <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
 <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
Message-ID: <20180305141913.C5EAA208E9@orac.inputplus.co.uk>

Hi Richard,

> I suggested ignoring CR because they are there.  Ideally they would
> not be there.  The files are internal to get_iplayer so they can be in
> any format.

Only if you don't want them to be native text files, editable on that
system with a text editor by any user.  And I thought you do because you
have been editing them.

> I would have thought it sensible to use LF as a line terminator for
> get_iplayer's internal files across all operating systems.  The
> question is how the CR are getting into the files in Windows.

I pre-empted this in the email you're replying to.

    Because C has lines terminated by LF, like Unix, C libraries on
    Windows convert CRLF to LF on reading a text file, and do the
    reverse on writing.  Perl implements that.

> If Perl can then handle the variants portably with chomp VARIABLE
> that's fine, but it doesn't seem to be working properly at the moment. 

That's not what chomp() does.  It removes a single trailing LF if it's
present.  The conversion of CRLF to LF on reading a text file is
PerlIO(3perl)'s `:crlf' layer, stacked on Windows systems.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From david at cantrell.org.uk  Mon Mar  5 07:34:11 2018
From: david at cantrell.org.uk (David Cantrell)
Date: Mon, 5 Mar 2018 15:34:11 +0000
Subject: Format of options file
In-Reply-To: <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
Message-ID: <20180305153411.GA11882@bytemark.barnyard.co.uk>

On Sun, Mar 04, 2018 at 07:59:52PM +0000, RS wrote:

> You're right I am a bit confused, but not about the line termination 
> conventions for each OS, although I can't speak for the Mac.

Macs use \n like normal Unix machines. They used to use \r in the bad
old days before they went Unixy.

-- 
David Cantrell | Bourgeois reactionary pig

      You know you're getting old when you fancy the
      teenager's parent and ignore the teenager
        -- Paul M in uknot


From david at cantrell.org.uk  Mon Mar  5 07:41:50 2018
From: david at cantrell.org.uk (David Cantrell)
Date: Mon, 5 Mar 2018 15:41:50 +0000
Subject: Format of options file
In-Reply-To: <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
Message-ID: <20180305154150.GB11882@bytemark.barnyard.co.uk>

On Mon, Mar 05, 2018 at 09:52:53AM +0000, RS wrote:

> Essentially you seem to be saying I can't have a CR in a Linux text file 
> because it is non-standard.  It seems a strange standard that prevents 
> data interchange rather than facilitates it.

Yes, it is strange that Microsoft chose a non-standard line ending
convention. In this case though we can't really blame them as they were
just copying CP/M. CP/M chose its line ending convention in 1974, a few
years after Unix chose its convention.

-- 
David Cantrell | semi-evolved ape-thing

    Immigration: making Britain great since AD43


From richard22j at zoho.com  Mon Mar  5 08:04:06 2018
From: richard22j at zoho.com (RS)
Date: Mon, 5 Mar 2018 16:04:06 +0000
Subject: Format of options file
In-Reply-To: <20180305141913.C5EAA208E9@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
 <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
 <20180305141913.C5EAA208E9@orac.inputplus.co.uk>
Message-ID: <da690c09-1a47-5e8c-78e0-ab409c769185@zoho.com>

On 05/03/2018 14:19, Ralph Corderoy wrote:

>> I suggested ignoring CR because they are there.  Ideally they would
>> not be there.  The files are internal to get_iplayer so they can be in
>> any format.
> 
> Only if you don't want them to be native text files, editable on that
> system with a text editor by any user.  And I thought you do because you
> have been editing them.

Hi Ralph

I have only ever edited the options file by running the script you gave 
me to convert CR LF to LF.  There is no need to edit the file directly 
because it can be displayed with --prefs-show.  Entries can be changed 
and added with --prefs-add and removed with --prefs-del.  The format of 
those commands and the display are different from each other and from 
the file format, so editing directly would be a recipe for confusion.
> 
>> I would have thought it sensible to use LF as a line terminator for
>> get_iplayer's internal files across all operating systems.  The
>> question is how the CR are getting into the files in Windows.
> 
> I pre-empted this in the email you're replying to.
> 
>      Because C has lines terminated by LF, like Unix, C libraries on
>      Windows convert CRLF to LF on reading a text file, and do the
>      reverse on writing.  Perl implements that.
> 
If the conversion only occurs in Windows, that's no good.  I have cache, 
history and options files created in Windows with CR LF.  I want 
get_iplayer to be able to read and update those files in Linux.  Ideally 
I want  to be able to put them on a shared drive so that both 
installations can update them.  I thought from the article I read chomp 
VARIABLE could do that, but clearly I have misunderstood.

Best wishes
Richard




From ralph at inputplus.co.uk  Mon Mar  5 08:08:14 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Mon, 05 Mar 2018 16:08:14 +0000
Subject: Format of options file
In-Reply-To: <20180305153411.GA11882@bytemark.barnyard.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180305153411.GA11882@bytemark.barnyard.co.uk>
Message-ID: <20180305160814.ACF4C21A0B@orac.inputplus.co.uk>

Hi David,

> Macs use \n like normal Unix machines. They used to use \r in the bad
> old days before they went Unixy.

Yes, you're right.  Earlier in the thread I said I thought they used CR,
but that was much earlier up to Mac OS 9.  Mac OS X came along around
2000, was based on Unix, unlike 9, and switched to its LF line
terminator.  macOS is still version 10, the X Roman numeral, but has
changed name to OS X, and now macOS along the way.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From c.e.macfarlane at macfh.co.uk  Mon Mar  5 11:01:24 2018
From: c.e.macfarlane at macfh.co.uk (MacFH - C E Macfarlane)
Date: Mon, 5 Mar 2018 19:01:24 +0000
Subject: Format of options file
In-Reply-To: <20180305133649.DAEC4208E9@orac.inputplus.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
 <20180305120239.13862208E9@orac.inputplus.co.uk>
 <7c952a46-929f-6815-f803-821527c6e93d@macfh.co.uk>
 <20180305133649.DAEC4208E9@orac.inputplus.co.uk>
Message-ID: <7ad17483-c003-8739-c28b-8f7f025d94a0@macfh.co.uk>

We could argue about this for hours, but I have no desire to do so? -? 
there are things I like and others I detest about both OSs, and the 
thread's becoming increasingly OT.? I answer one specific point below, 
but anyone not interested in computer history and debugging programs can 
safely ignore ...

On 05/03/2018 13:36, Ralph Corderoy wrote:
>
>> Arguably the oddballs are Linux and perhaps Macs, using only one.
>> This would mean that if their text was sent directly to a teleprinter,
>> the output would not be what is required, but of course these days
>> that would seldom if ever be done.
> But in those days of Multics and Unix that's precisely what was done
> with Unix text files, terminated by just LF, and yet the text didn't
> stagger across the page because a device driver was correctly the thing
> that knew what a particular device needed to be sent as bytes;  the file
> format wasn't the place for that.  Other characteristics include a
> timing delay to allow for the carriage to return before sending more
> bytes.
Don't forget that the earliest teleprinter style devices date back as 
far as at latest WW2, for example the German Lorenz machine which also 
applied cryptography? -? and I don't believe the concept of 
'device-driver' existed then; nor am I questioning the use of device 
drivers, obviously they make a lot of programming sense.? What I'm 
questioning is the wisdom, perhaps that should be rank stupidity, of 
using LF as the line-terminator, while treating a preceding CR as part 
of the line text.? That completely f*d backward compatibility with what 
had gone before.? You say that it is C based, but my recollection of C's 
internal representation is that strings are NULL terminated, and that 
also might have been a more logical choice of line-terminator for Unix.

At any rate, we have the mess we have, and I join with Bernard in 
deploring it.? I can think of at least two baffling examples that have 
afflicted me in the past, one in each direction ...

The first concerned a Psion Gold modem PCMCIA card, installation of the 
Windows drivers for which broke not only use of that card, but any 
previously similar card using the same INI file.? I wasn't the person 
who wrote this original installation script, from what I can recall that 
had been, shudder, a contractor, but the mess was given to me to sort 
out.? After I'd corrected all the usual contractor horrors, like failing 
to get the Windows and Temp directory names from the environment, and 
the hard-coding of them and other non-constant-across-different-machines 
values repeatedly all the way through the script, I was still left with 
a script that didn't work.? Now that I had cleaned it up, the script 
appeared to be doing everything it should have, and should have worked 
on any machine in the firm, and the resulting INI file looked alright, 
yet nothing was working.? It occurred to me that perhaps there was some 
hidden character in the INI file supplied by Psion, which we were 
editing via the script, so I loaded it as hex into TextPad and, lo and 
behold, Psion had supplied a *Windows* INI file with *Unix* line 
endings!? Obviously, I could have just loaded the file as text into 
Textpad and saved it with Windows line-endings, but this would not have 
helped the many people who had already tried the installation and had it 
fail.? I had to adapt the script further to clean up any pre-existing 
mess from a previous installation by correcting the line endings in 
their existing INI file.? Thus this incompatibility between line endings 
created a ridiculous amount of lost productivity and extra work.

The second occasion was that I created a Linux bash script on a Windows 
machine using TextPad, but failed to remember to save it with Unix line 
endings.? Because it was interpreting the CRs as part of the line, the 
errors given were baffling.? I tried posting the script in one or more 
Linux newsgroups, but nobody could suggest what might be going wrong, 
and I moved on to other things for a while.? It was only when I came 
back to it and started trying things almost at random, that after a few 
hours I came to realise what the problem was.? Again, so much time 
wasted, for so little a reason.



From ralph at inputplus.co.uk  Mon Mar  5 11:25:06 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Mon, 05 Mar 2018 19:25:06 +0000
Subject: Format of options file
In-Reply-To: <7ad17483-c003-8739-c28b-8f7f025d94a0@macfh.co.uk>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <20180303110949.B91381FE27@orac.inputplus.co.uk>
 <CAL=0gLty8TLCYDc0pYWvs+Hapm_tjvWJxFL150KNAVgoXxfw-Q@mail.gmail.com>
 <20180305120239.13862208E9@orac.inputplus.co.uk>
 <7c952a46-929f-6815-f803-821527c6e93d@macfh.co.uk>
 <20180305133649.DAEC4208E9@orac.inputplus.co.uk>
 <7ad17483-c003-8739-c28b-8f7f025d94a0@macfh.co.uk>
Message-ID: <20180305192506.70B6B21A0B@orac.inputplus.co.uk>

Hi MacFH,

> nor am I questioning the use of device drivers, obviously they make a
> lot of programming sense.? What I'm questioning is the wisdom, perhaps
> that should be rank stupidity,

You're saying Unix's core designers are stupid for choosing to stick
with text files as lines terminated by LF.  There's some Sherlock Holmes
quote about `It takes talent to recognise genius...'.  :-)

> of using LF as the line-terminator, while treating a preceding CR as
> part of the line text.? That completely f*d backward compatibility
> with what had gone before.

Much of Unix broke orthodoxy, and Unix wasn't the first to punt device
dependencies out of the logical file format and into a device driver
that needed to know about the exceptions and oddities of the physical
world.  Nor was it the first major OS to use LF-terminated lines.

> You say that it is C based, but my recollection of C's internal
> representation is that strings are NULL terminated,

They are, but that it not relevant.

> and that also might have been a more logical choice of line-terminator
> for Unix.

Only if a C string was to be prohibited from holding more than one line
of text.  C allows the bytes "line 1\nline 2\n\0" as a NUL-terminated
string.  And only if a C string "foobar" could no long be split into
two, "foo", "bar", without losing the fact that they are not both lines
that have been read.

C's standard library widely expects LF-terminated lines, and translates,
as I've said twice now in replies to Richard, from the host's
representation to that on reading and back on writing where it differs,
e.g. CP/M and its descendants.

You're drawing conclusions from sketchy knowledge and thus not able to
follow through their implications.  Sorry, but there's no point to this
conversation.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From richard22j at zoho.com  Mon Mar  5 12:04:09 2018
From: richard22j at zoho.com (RS)
Date: Mon, 5 Mar 2018 20:04:09 +0000
Subject: Format of options file
In-Reply-To: <da690c09-1a47-5e8c-78e0-ab409c769185@zoho.com>
References: <bdc80934-8d70-d89a-185a-9e97dd4a53c3@zoho.com>
 <bebb4415-d1e2-950d-e076-6a4c1f77454d@zoho.com>
 <9949833b-df74-791a-c861-a1ec24ccf4af@jls-radio.com>
 <221cb3c2-5e38-5f62-9a05-25c6f136d50f@shrdlu.com>
 <669265cb-95c3-cc5b-231f-69acdbbbd1ee@macfh.co.uk>
 <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
 <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
 <20180305141913.C5EAA208E9@orac.inputplus.co.uk>
 <da690c09-1a47-5e8c-78e0-ab409c769185@zoho.com>
Message-ID: <14c54bfd-54d0-4b35-ef17-1558f17ff7c6@zoho.com>

On 05/03/2018 16:04, I wrote:

>>
>>> I would have thought it sensible to use LF as a line terminator for
>>> get_iplayer's internal files across all operating systems.? The
>>> question is how the CR are getting into the files in Windows.
>>
>> I pre-empted this in the email you're replying to.
>>
>> ???? Because C has lines terminated by LF, like Unix, C libraries on
>> ???? Windows convert CRLF to LF on reading a text file, and do the
>> ???? reverse on writing.? Perl implements that.
>>
> If the conversion only occurs in Windows, that's no good.? I have cache, 
> history and options files created in Windows with CR LF.? I want 
> get_iplayer to be able to read and update those files in Linux.? Ideally 
> I want? to be able to put them on a shared drive so that both 
> installations can update them.? I thought from the article I read chomp 
> VARIABLE could do that, but clearly I have misunderstood.
> 
It's worse than I thought.  I had got the impression from the perlport 
perldoc that if replaced \n with \012 in a print statement I would get a 
LF on its own in Windows.  I don't.  If I insert \015 I can have a CR on 
its own, but \012 is still replaced with CR LF.

This article
http://www.perlmonks.org/?node=binmode
says I can use binmode, so that may be an answer.  It's quite old, so it 
may no longer apply.

Best wishes
Richard




From dwmw2 at infradead.org  Tue Mar  6 01:50:36 2018
From: dwmw2 at infradead.org (David Woodhouse)
Date: Tue, 06 Mar 2018 09:50:36 +0000
Subject: Message with suspicious header
In-Reply-To: <em3d0ce6e3-a614-4d6f-831b-f5d8006ca65b@owner-pc>
References: <em3d0ce6e3-a614-4d6f-831b-f5d8006ca65b@owner-pc>
Message-ID: <1520329836.31298.12.camel@infradead.org>



On Sat, 2018-03-03 at 13:00 +0000, Dave Widgery wrote:
> I have yet again had a message held due to a suspicious header, all I?
> did was press reply?

For messages which are in response to a thread, the system is checking
that they start with either 'Re:' or 'Aw:', or some other things.

For some reason your latest one started 'Re[2]:' instead. I'll look at
fixing the checks to tolerate that too, bizarre though it is.

Apologies for the inconvenience, but across the full set of mailing
lists this is a useful tool to... "encourage" people to behave
correctly and not do bad things which detract from the communication.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5213 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/get_iplayer/attachments/20180306/cf5ff8c3/attachment.bin>

From david at cantrell.org.uk  Tue Mar  6 05:19:36 2018
From: david at cantrell.org.uk (David Cantrell)
Date: Tue, 6 Mar 2018 13:19:36 +0000
Subject: Format of options file
In-Reply-To: <14c54bfd-54d0-4b35-ef17-1558f17ff7c6@zoho.com>
References: <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
 <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
 <20180305141913.C5EAA208E9@orac.inputplus.co.uk>
 <da690c09-1a47-5e8c-78e0-ab409c769185@zoho.com>
 <14c54bfd-54d0-4b35-ef17-1558f17ff7c6@zoho.com>
Message-ID: <20180306131936.GE11077@bytemark.barnyard.co.uk>

On Mon, Mar 05, 2018 at 08:04:09PM +0000, RS wrote:

> It's worse than I thought.  I had got the impression from the perlport 
> perldoc that if replaced \n with \012 in a print statement I would get a 
> LF on its own in Windows.  I don't.  If I insert \015 I can have a CR on 
> its own, but \012 is still replaced with CR LF.
> 
> This article
> http://www.perlmonks.org/?node=binmode
> says I can use binmode, so that may be an answer.  It's quite old, so it 
> may no longer apply.

binmode does still work AFAIK, but a more modern and flexible method is
to use the crlf I/O layer, which is documented here:
  https://perldoc.perl.org/PerlIO.html

Note however that an awful lot of perl code just doesn't bother. Windows
is very much a second-class citizen in the perl world.

-- 
David Cantrell | Nth greatest programmer in the world

    If you can read this, thank a teacher.
    If you're reading it in English, thank Chaucer.


From ralph at inputplus.co.uk  Tue Mar  6 05:29:51 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Tue, 06 Mar 2018 13:29:51 +0000
Subject: Format of options file
In-Reply-To: <20180306131936.GE11077@bytemark.barnyard.co.uk>
References: <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
 <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
 <20180305141913.C5EAA208E9@orac.inputplus.co.uk>
 <da690c09-1a47-5e8c-78e0-ab409c769185@zoho.com>
 <14c54bfd-54d0-4b35-ef17-1558f17ff7c6@zoho.com>
 <20180306131936.GE11077@bytemark.barnyard.co.uk>
Message-ID: <20180306132951.3659D1F96E@orac.inputplus.co.uk>

Hi David,

> binmode does still work AFAIK, but a more modern and flexible method
> is to use the crlf I/O layer, which is documented here:
>   https://perldoc.perl.org/PerlIO.html
>
> Note however that an awful lot of perl code just doesn't bother.

Windows stacks the `:crlf' layer by default.  I *think* Richard's trying
to avoid that because he wants his get_iplayer to use POSIX text files
on Linux, as normal, and Windows.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From ilainftw at gmx.com  Tue Mar  6 06:09:55 2018
From: ilainftw at gmx.com (iz)
Date: Tue, 6 Mar 2018 15:09:55 +0100
Subject: Format of options file
In-Reply-To: <20180306132951.3659D1F96E@orac.inputplus.co.uk>
References: <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
 <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
 <20180305141913.C5EAA208E9@orac.inputplus.co.uk>
 <da690c09-1a47-5e8c-78e0-ab409c769185@zoho.com>
 <14c54bfd-54d0-4b35-ef17-1558f17ff7c6@zoho.com>
 <20180306131936.GE11077@bytemark.barnyard.co.uk>
 <20180306132951.3659D1F96E@orac.inputplus.co.uk>
Message-ID: <trinity-89df0fb8-5c0d-4108-966b-4e1033959b73-1520345395157@3c-app-mailcom-bs01>

> > binmode does still work AFAIK, but a more modern and flexible method
> > is to use the crlf I/O layer, which is documented here:
> >   https://perldoc.perl.org/PerlIO.html
> >
> > Note however that an awful lot of perl code just doesn't bother.
> 
> Windows stacks the `:crlf' layer by default.  I *think* Richard's trying
> to avoid that because he wants his get_iplayer to use POSIX text files
> on Linux, as normal, and Windows.

PerlIO enables you to go either way, and no code changes are necessary. The simplest solution is to run the equivalent of 'PERLIO=crlf get_iplayer [...]' on Linux, perhaps via an alias in your shell. Probably best not to set PERLIO=crlf permanently if you run other Perl programs. You may not like having files with CRLF line endings, but they will still work on both platforms if you screw up editing and get mixed line endings. If you really can't abide CRLF, use PERLIO=perlio on Windows instead to produce LF line endings. Probably simplest to set it locally in the get_iplayer batch file. You just have to be careful not to inject CRLF line endings if you edit any files directly, though I would expect any decent text editor on both platforms can avoid that.


From rob.dixon at gmx.com  Tue Mar  6 07:37:36 2018
From: rob.dixon at gmx.com (rob.dixon at gmx.com)
Date: Tue, 6 Mar 2018 15:37:36 +0000
Subject: Format of options file
In-Reply-To: <20180306132951.3659D1F96E@orac.inputplus.co.uk>
References: <83bfe459-df79-8659-9f77-acecd9381ec4@zoho.com>
 <39757a1e-fd07-9f02-1788-3bf057bc30ea@macfh.co.uk>
 <5415cf99-7902-cc75-8218-20e5d00cfb22@zoho.com>
 <20180304213437.0FA5B20A28@orac.inputplus.co.uk>
 <5c087793-a7c4-bccd-33b4-68b5c697e5f3@zoho.com>
 <20180305131953.BA23B208E9@orac.inputplus.co.uk>
 <e10f8341-b415-47fa-ebfe-9c3732ce4bd5@zoho.com>
 <20180305141913.C5EAA208E9@orac.inputplus.co.uk>
 <da690c09-1a47-5e8c-78e0-ab409c769185@zoho.com>
 <14c54bfd-54d0-4b35-ef17-1558f17ff7c6@zoho.com>
 <20180306131936.GE11077@bytemark.barnyard.co.uk>
 <20180306132951.3659D1F96E@orac.inputplus.co.uk>
Message-ID: <2e95419a-b86d-4f1b-b6d5-ca71d526e9b0.maildroid@localhost>

I don't know the history of this thread, but a Windows perl will use the `:crlf` layer which is fine for both Windows (CR LF) and Linux (LF) line terminations. The CR will be removed, and every line read will end with LF ("\n")

The most common problem arises when a Linux perl attempts to read a Windows text file. The default PerlIO stack on Linux is an empty one, so the CR LF characters appear intact in every line read

There are a few solutions:

1/ Change the default PerlIO layers

    use open qw/ :std :crlf /

will cause every file to be opened with the Windows `:crlf` layer, which removes any CR if it is there. It doesn't affect Linux files at all

2/ If you *always* `chomp` every line, then use

    s/\R\z//

instead, which will remove any line termination character or character pair

3/ If you *always* `split` every line using the default of `split ' ', $_` then there is nothing to do: CR is considered to be a whitespace character and will be removed from the fields

There are clearly variations on this idea, but it shouldn't be hard to resolve from here

Can someone please tell me whether I have answered the original question, or if I'm miles off?

Rob Dixon
Norfolk
England


-----Original Message-----
From: Ralph Corderoy <ralph at inputplus.co.uk>
To: get_iplayer at lists.infradead.org
Sent: Tue, 06 Mar 2018 13:30
Subject: Re: Format of options file

Hi David,

> binmode does still work AFAIK, but a more modern and flexible method
> is to use the crlf I/O layer, which is documented here:
>   https://perldoc.perl.org/PerlIO.html
>
> Note however that an awful lot of perl code just doesn't bother.

Windows stacks the `:crlf' layer by default.  I *think* Richard's trying
to avoid that because he wants his get_iplayer to use POSIX text files
on Linux, as normal, and Windows.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

_______________________________________________
get_iplayer mailing list
get_iplayer at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer


From artisticforge at gmail.com  Tue Mar  6 22:42:09 2018
From: artisticforge at gmail.com (artisticforge .)
Date: Wed, 7 Mar 2018 00:42:09 -0600
Subject: more creatures gret and small question
Message-ID: <CANqy083XaBb-Prmt5rwWsJVcnMYXuuoRb95F5NUWX8jn-HZLmw@mail.gmail.com>

hello

More Creatures Great and Small Episode 9 in in the tv.cache

4648|tv|More Creatures Great and
Small|b0400tyf|2018-03-02T16:20:00+00:00|1522599600|Episode
9||9|default|1800|Vet Megan is on a mission in Fife to help rehome 80
commercial caged hens.|BBC
Two|||1519047272||http://www.bbc.co.uk/programmes/b0400tyf|

when attempting to fetch this episode I received the below error:

<begin quote>
INFO: Search args: ''
INFO: Loaded history for first check.
INFO: Loading recordings history
INFO: Programme not in history
INFO: Got 8814 file cache entries for tv
INFO: Got 18545 file cache entries for radio
INFO: Cleaning PID - old: 'b0400tyf' new: 'b0400tyf'
INFO: Getting URL: http://www.bbc.co.uk/programmes/b0400tyf.json
INFO: tv episode PID detected (b0400tyf)
Matches:
4648:   More Creatures Great and Small - Episode 9, BBC Two, b0400tyf
INFO: 1 matching programmes
INFO: Programme not in history
INFO: Getting URL: http://www.bbc.co.uk/programmes/b0400tyf.json
INFO: Getting URL:
http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/b0400tyc/transferformat/dash?cb=32849
ERROR: Response: 404 Not Found
INFO: Getting URL:
http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/b0400tyc/transferformat/dash?cb=57114
ERROR: Response: 404 Not Found
INFO: Getting URL:
http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/b0400tyc/transferformat/hls?cb=09524
ERROR: Response: 404 Not Found
INFO: Getting URL:
http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/apple-ipad-hls/vpid/b0400tyc/transferformat/hls?cb=40092
ERROR: Response: 404 Not Found
INFO: No streams available for 'original' version (b0400tyc) - skipping
INFO: No streams found for 'original' version (b0400tyc) - deleting
WARNING: No media streams found for requested programme versions and
recording modes.
ERROR: Could not get programme metadata
<end quote>

It appears that the episode was scheduled to be aired and for whatever
reason was skipped.

This also happened with Episode 1
Never figured that one out.

thoughts, comments, hints welcomed.

-- 
terry l. ridder ><>


From alan at soulman1949.com  Wed Mar  7 17:06:43 2018
From: alan at soulman1949.com (Alan Milewczyk)
Date: Thu, 8 Mar 2018 09:06:43 +0800
Subject: more creatures gret and small question
In-Reply-To: <CANqy083XaBb-Prmt5rwWsJVcnMYXuuoRb95F5NUWX8jn-HZLmw@mail.gmail.com>
References: <CANqy083XaBb-Prmt5rwWsJVcnMYXuuoRb95F5NUWX8jn-HZLmw@mail.gmail.com>
Message-ID: <9b8da472-2b91-aaba-f37c-6a280fc626b5@soulman1949.com>

On 07/03/2018 14:42, artisticforge . wrote:
> hello
>
> More Creatures Great and Small Episode 9 in in the tv.cache

It's not the first time programmes have been rescheduled but if you look 
at the BBC two schedule page for that day (Friday 2nd March) at 
https://www.bbc.co.uk/schedules/p00fzl97/2018/03/02, you will see that 
the Athletics was on at 09:00 and the Daily Politics at 13:30 (special 
coverage of the Maybot's speech about the latest position on Brexit). 
Daily Politics is normally on for an hour 12:00-13:00.

I don't have access to the original schedule but I would hazard a guess 
that when the Maybot announced she would be making her speech, the Beeb 
decided to reschedule Daily Politics, extend the Athletics and as a 
result More Creatures got bumped.

I wouldn't worry, I find that a lot of the BBC Two afternoon programmes 
are rebroadcast periodically (the iplayer page seems to indicate this 
series was broadcast in 2014).

HTH.

Alan


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus



From ralph at inputplus.co.uk  Thu Mar  8 02:02:30 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Thu, 08 Mar 2018 10:02:30 +0000
Subject: FAO BBC: Same Series and Episode, Differents PIDs and Description.
Message-ID: <20180308100230.70AA2208E8@orac.inputplus.co.uk>

Hi,

There's been much weirdness recently in get_iplayer's tv.cache due to
what's being published by the BBC AFAICS.

The latest example, I have some older more complex ones to write up, is
PIDs b09wc2m1 and b09wbylq are both S02E01 of _The Repair Shop_, but
with different descriptions.  The `short' one is

    b09wc2m1: Antique photography expert Brenton West repairs a camera
    that survived World War I.

    b09wbylq: A Boulle-work clock, a much-loved wheeled elephant and a
    300-year-old desk are in the shop

This confusion is also present on the BBC's site.

    http://www.bbc.co.uk/programmes/b09wc2m1
    http://www.bbc.co.uk/programmes/b09wbylq

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From alan at soulman1949.com  Thu Mar  8 03:03:38 2018
From: alan at soulman1949.com (Alan Milewczyk)
Date: Thu, 8 Mar 2018 19:03:38 +0800
Subject: FAO BBC: Same Series and Episode, Differents PIDs and Description.
In-Reply-To: <20180308100230.70AA2208E8@orac.inputplus.co.uk>
References: <20180308100230.70AA2208E8@orac.inputplus.co.uk>
Message-ID: <ccc3004d-c96e-7262-8bc9-8978fb2db1eb@soulman1949.com>

On 08/03/2018 18:02, Ralph Corderoy wrote:
> Hi,
>
> There's been much weirdness recently in get_iplayer's tv.cache due to
> what's being published by the BBC AFAICS.
>
> The latest example, I have some older more complex ones to write up, is
> PIDs b09wc2m1 and b09wbylq are both S02E01 of _The Repair Shop_, but
> with different descriptions.  The `short' one is
>
>      b09wc2m1: Antique photography expert Brenton West repairs a camera
>      that survived World War I.
>
>      b09wbylq: A Boulle-work clock, a much-loved wheeled elephant and a
>      300-year-old desk are in the shop
>
> This confusion is also present on the BBC's site.
>
>      http://www.bbc.co.uk/programmes/b09wc2m1
>      http://www.bbc.co.uk/programmes/b09wbylq
>
There's more of the same sort of confusion at 
https://www.bbc.co.uk/programmes/b08l581p/episodes/guide where there 
appear to be duplicated numbered episodes which have different PIDs. 
Very odd.


A


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus



From david at cantrell.org.uk  Thu Mar  8 05:27:51 2018
From: david at cantrell.org.uk (David Cantrell)
Date: Thu, 8 Mar 2018 13:27:51 +0000
Subject: FAO BBC: Same Series and Episode, Differents PIDs and Description.
In-Reply-To: <20180308100230.70AA2208E8@orac.inputplus.co.uk>
References: <20180308100230.70AA2208E8@orac.inputplus.co.uk>
Message-ID: <20180308132750.GA16024@bytemark.barnyard.co.uk>

On Thu, Mar 08, 2018 at 10:02:30AM +0000, Ralph Corderoy wrote:

> There's been much weirdness recently in get_iplayer's tv.cache due to
> what's being published by the BBC AFAICS.
> 
> The latest example, I have some older more complex ones to write up, is
> PIDs b09wc2m1 and b09wbylq are both S02E01 of _The Repair Shop_, but
> with different descriptions.  The `short' one is
> 
>     b09wc2m1: Antique photography expert Brenton West repairs a camera
>     that survived World War I.
> 
>     b09wbylq: A Boulle-work clock, a much-loved wheeled elephant and a
>     300-year-old desk are in the shop

My assumption would be that they are different versions of the same
episode, with one cut for length for eg a repeat.

-- 
David Cantrell | Nth greatest programmer in the world

You may now start misinterpreting what I just
wrote, and attacking that misinterpretation.


From ralph at inputplus.co.uk  Thu Mar  8 05:41:28 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Thu, 08 Mar 2018 13:41:28 +0000
Subject: FAO BBC: Same Series and Episode, Differents PIDs and Description.
In-Reply-To: <20180308132750.GA16024@bytemark.barnyard.co.uk>
References: <20180308100230.70AA2208E8@orac.inputplus.co.uk>
 <20180308132750.GA16024@bytemark.barnyard.co.uk>
Message-ID: <20180308134128.6B57921A85@orac.inputplus.co.uk>

Hi David,

> >     b09wc2m1: Antique photography expert Brenton West repairs a
> >     camera that survived World War I.
> > 
> >     b09wbylq: A Boulle-work clock, a much-loved wheeled elephant and
> >     a 300-year-old desk are in the shop
>
> My assumption would be that they are different versions of the same
> episode, with one cut for length for eg a repeat.

You assume too much!  :-)  They're both 30 minutes, and completely
different content.

    b09wc2m1
    duration: 1800
    desclong: Today in the Repair Shop, Jay Blades and the team bring
    three treasured family heirlooms, and the memories they hold, back
    to life.  Resident antique photography expert Brenton West brings
    his expertise to bear on a camera that survived the First World War
    but hasn't taken a picture in over a hundred years, while leather
    expert Suzie adds a surprise touch that leaves the camera's owner
    Phil lost for words. Furniture restorer Will Kirk and horologist
    Steve Fletcher joins forces to get a 20th-century designer wooden
    screen back on its feet. And violin restorer John Dilworth works on
    a treasured instrument that spent World War II in Auschwitz
    concentration camp.

    b09wbylq
    duration: 1800
    desclong: Today in the Repair Shop, Jay Blades and the team bring
    three treasured family heirlooms, and the memories they hold, back
    to life.  Resident clockmaker Steve tackles an intricate Boulle-work
    clock that hasn't ticked for over 15 years. Soft toy restorers Julie
    and Amanda take on a jumbo-sized project in the shape of a
    threadbare, but much loved, wheeled elephant. And woodwork
    specialist Will get to grips with one of the Repair Shop's oldest
    ever assignments - a 300-year-old Georgian desk that is showing its
    age.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From ralph at inputplus.co.uk  Thu Mar  8 07:20:30 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Thu, 08 Mar 2018 15:20:30 +0000
Subject: more creatures gret and small question
In-Reply-To: <9b8da472-2b91-aaba-f37c-6a280fc626b5@soulman1949.com>
References: <CANqy083XaBb-Prmt5rwWsJVcnMYXuuoRb95F5NUWX8jn-HZLmw@mail.gmail.com>
 <9b8da472-2b91-aaba-f37c-6a280fc626b5@soulman1949.com>
Message-ID: <20180308152030.3900E21A85@orac.inputplus.co.uk>

Hi Alan,

> I don't have access to the original schedule

I've quite a few tv.cache, backing it up every time it gets written.
They've all got a gap in them that morning, which is quite odd, before
11:45.  I've marked those that start at the same time as another entry,
showing schedules change, with their group number.

    $ cat ~/.get_iplayer/tv.cache* |
    > awk -F\| '$5 ~ /^2018-03-02T[01]/ && $13 == "BBC Two" {print $5, $4, $3}' |
    > sort
    2018-03-02T00:05:00+00:00 b09tmqlv Snooker: Welsh Open Highlights: 2018
    2018-03-02T00:20:00+00:00 b017yznd Coast: Series 5 Reversions
    2018-03-02T02:05:00+00:00 b09gsz3b Royal Recipes: Series 2
    2018-03-02T11:45:00+00:00 b09trgmb Wanted Down Under Revisited: Series 11
    2018-03-02T12:15:00+00:00 b09tn3l1 Caught Red Handed: Series 6
    2018-03-02T13:00:00+00:00 b09tr342 Athletics: 2018
    2018-03-02T14:15:00+00:00 b039sp5t Plan It, Build It
    2018-03-02T14:45:00+00:00 b01406xp Coast: Series 5 Reversions
    2018-03-02T14:45:00+00:00 b09w4403 Coast
    2018-03-02T15:00:00+00:00 b07v7b6l Yes Chef: Series 1
    2018-03-02T15:50:00+00:00 b086y32c A Place to Call Home: Series 4
 1? 2018-03-02T16:20:00+00:00 b0400tyf More Creatures Great and Small
 1  2018-03-02T16:20:00+00:00 b07v6hdt Grand Tours of the Scottish Islands: Shorts
 2  2018-03-02T16:30:00+00:00 b09tmqw5 Snooker: Welsh Open: 2018
 2  2018-03-02T16:30:00+00:00 b09tr21c Get Away for Winter: Series 1
 2  2018-03-02T16:30:00+00:00 b09w8sgs Snooker: Welsh Open: 2018
    2018-03-02T17:20:00+00:00 p03gk861 Greece with Simon Reeve
 3  2018-03-02T17:50:00+00:00 b04871bz Coast: Series 5 Reversions
 3  2018-03-02T17:50:00+00:00 b088rxvh Great British Railway Journeys: Series 8
    2018-03-02T18:00:00+00:00 b0079285 Coast: Shorts
    2018-03-02T19:00:00+00:00 b09tr344 Athletics: 2018
    2018-03-02T19:30:00+00:00 b09tmrhv Snooker: Welsh Open: 2018
    2018-03-02T19:57:00+00:00 b09ts2db MasterChef: Series 14
    $

I wonder if that means obtaining the TV listings for the PVR is quite
sketchy, leaving gaps?

> but I would hazard a guess that when the Maybot announced she would be
> making her speech, the Beeb decided to reschedule Daily Politics,
> extend the Athletics and as a result More Creatures got bumped.

You're probably right.  If they don't simply slide all the episodes
along one, but actually drop one episode from broadcast leaving a hole,
you'd think they would *still* make it available on iPlayer for those
that want continuity.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From ralph at inputplus.co.uk  Thu Mar  8 07:53:17 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Thu, 08 Mar 2018 15:53:17 +0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
Message-ID: <20180308155317.F386121A85@orac.inputplus.co.uk>

Hi Dave,

> Has anyone any idea why Episode 8 of the latest series of The Coroner
> is not available? it isn't available on bbc iplayer either?

The published data for that programme went very wrong.  This is what my
tv.cache saw over time recently.

    $ grep -h The.Coroner ~/.get_iplayer/tv.cache* |
    > gawk -F\| '{print sprintf("s%de%02d", $8, $9), $5, $4, strftime("%Y-%m-%d %T", $16), $3, $7}' |
    > sort -u | sed 's/T/ /; s/:00+00:00//'
    s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 1 First Love
    s1e01 2018-02-12 15:00 b06q54pd 2018-03-08 09:14:57 The Coroner: Series 1 First Love
    s1e02 2018-02-13 15:00 b06q553j 2018-02-05 11:34:54 The Coroner: Series 1 How to Catch a Lobster
    s1e03 2018-02-14 15:00 b06q588j 2018-02-05 11:34:54 The Coroner: Series 1 That's the Way To Do It
    s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 1 The Fisherman's Tale
    s1e05 2018-02-16 15:00 b06q5rq3 2018-02-06 11:24:56 The Coroner: Series 1 Gilt
    s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 1 Capsized
    s1e07 2018-02-20 15:00 b06qjfq0 2018-02-12 07:30:53 The Coroner: Series 1 The Salcombe Selkie
    s1e08 2018-02-21 15:00 b06qjjlh 2018-02-12 07:30:53 The Coroner: Series 1 Napoleon's Violin
    s1e09 2018-02-22 15:00 b06qjkc0 2018-02-12 07:30:53 The Coroner: Series 1 The Deep Freeze
    s1e10 2018-02-23 15:00 b06qjl6j 2018-02-12 07:30:53 The Coroner: Series 1 Dirty Dancing
    s2e01 2018-02-13 15:00 b0840227 2018-02-13 12:34:30 The Coroner: Series 2 The Drop Zone
    s2e02 2018-02-14 15:00 b084022y 2018-02-14 08:47:52 The Coroner: Series 2 Perfectly Formed
    s2e03 2018-02-15 15:00 b084024r 2018-02-14 08:47:52 The Coroner: Series 2 Those in Peril
    s2e04 2018-02-16 15:00 b084025z 2018-02-14 08:47:52 The Coroner: Series 2 The Beast of Lighthaven
    s2e05 2018-02-19 15:00 b0840277 2018-02-14 08:47:52 The Coroner: Series 2 The Captain's Pipe
    s2e06 2018-02-20 15:00 b084dxth 2018-02-14 08:47:52 The Coroner: Series 2 Life
    s2e07 2018-02-21 15:00 b084dzjy 2018-02-14 08:47:52 The Coroner: Series 2 Perfect Pair
    s2e09 2018-02-22 15:00 b084f1zy 2018-02-14 08:47:52 The Coroner: Series 2 Pieces of Eight
    s2e10 2018-02-23 15:00 b084f3nk 2018-02-14 08:47:52 The Coroner: Series 2 Crash
    $

The first date-time is when it was to be broadcast;  always 3pm.  The
second is when that entry was added to tv.cache.  s1e01 did actually
broadcast, but then all of the rest of s1 didn't.  Not good if you only
invested time watching it on the basis of watching the series that was
already scheduled.  (iPlayer support reference CAS-4806288-B3HT3L.)
s2e08 is missing from that list;  they never scheduled it in this run of
repeats.

Ordering that list by scheduled broadcast time, and eliding duplicate
dates, gives

    s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 1 First Love
    s1e01      ""          b06q54pd 2018-03-08 09:14:57 The Coroner: Series 1 First Love
    s1e02 2018-02-13 15:00 b06q553j 2018-02-05 11:34:54 The Coroner: Series 1 How to Catch a Lobster
    s2e01      ""          b0840227 2018-02-13 12:34:30 The Coroner: Series 2 The Drop Zone
    s1e03 2018-02-14 15:00 b06q588j 2018-02-05 11:34:54 The Coroner: Series 1 That's the Way To Do It
    s2e02      ""          b084022y 2018-02-14 08:47:52 The Coroner: Series 2 Perfectly Formed
    s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 1 The Fisherman's Tale
    s2e03      ""          b084024r 2018-02-14 08:47:52 The Coroner: Series 2 Those in Peril
    s1e05 2018-02-16 15:00 b06q5rq3 2018-02-06 11:24:56 The Coroner: Series 1 Gilt
    s2e04      ""          b084025z 2018-02-14 08:47:52 The Coroner: Series 2 The Beast of Lighthaven
    s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 1 Capsized
    s2e05      ""          b0840277 2018-02-14 08:47:52 The Coroner: Series 2 The Captain's Pipe
    s1e07 2018-02-20 15:00 b06qjfq0 2018-02-12 07:30:53 The Coroner: Series 1 The Salcombe Selkie
    s2e06      ""          b084dxth 2018-02-14 08:47:52 The Coroner: Series 2 Life
    s1e08 2018-02-21 15:00 b06qjjlh 2018-02-12 07:30:53 The Coroner: Series 1 Napoleon's Violin
    s2e07      ""          b084dzjy 2018-02-14 08:47:52 The Coroner: Series 2 Perfect Pair
    s1e09 2018-02-22 15:00 b06qjkc0 2018-02-12 07:30:53 The Coroner: Series 1 The Deep Freeze
    s2e09      ""          b084f1zy 2018-02-14 08:47:52 The Coroner: Series 2 Pieces of Eight
    s1e10 2018-02-23 15:00 b06qjl6j 2018-02-12 07:30:53 The Coroner: Series 1 Dirty Dancing
    s2e10      ""          b084f3nk 2018-02-14 08:47:52 The Coroner: Series 2 Crash

These were all `BBC One'.  So it looks like they scheduled and broadcast
s1e01 by mistake, and then switched to s2e01, replacing s1e02, and so
one for the remaining nine slots.  But that meant something had to go
and that was s2e08.

One more ordering, by the date-time it was added to tv.cache, split by
series.

    s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 1 First Love
    s1e02 2018-02-13 15:00 b06q553j      ""             The Coroner: Series 1 How to Catch a Lobster
    s1e03 2018-02-14 15:00 b06q588j      ""             The Coroner: Series 1 That's the Way To Do It
    s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 1 The Fisherman's Tale
    s1e05 2018-02-16 15:00 b06q5rq3      ""             The Coroner: Series 1 Gilt
    s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 1 Capsized
    s1e07 2018-02-20 15:00 b06qjfq0      ""             The Coroner: Series 1 The Salcombe Selkie
    s1e08 2018-02-21 15:00 b06qjjlh      ""             The Coroner: Series 1 Napoleon's Violin
    s1e09 2018-02-22 15:00 b06qjkc0      ""             The Coroner: Series 1 The Deep Freeze
    s1e10 2018-02-23 15:00 b06qjl6j      ""             The Coroner: Series 1 Dirty Dancing

    s2e01 2018-02-13 15:00 b0840227 2018-02-13 12:34:30 The Coroner: Series 2 The Drop Zone
    s2e02 2018-02-14 15:00 b084022y 2018-02-14 08:47:52 The Coroner: Series 2 Perfectly Formed
    s2e03 2018-02-15 15:00 b084024r      ""             The Coroner: Series 2 Those in Peril
    s2e04 2018-02-16 15:00 b084025z      ""             The Coroner: Series 2 The Beast of Lighthaven
    s2e05 2018-02-19 15:00 b0840277      ""             The Coroner: Series 2 The Captain's Pipe
    s2e06 2018-02-20 15:00 b084dxth      ""             The Coroner: Series 2 Life
    s2e07 2018-02-21 15:00 b084dzjy      ""             The Coroner: Series 2 Perfect Pair
    s2e09 2018-02-22 15:00 b084f1zy      ""             The Coroner: Series 2 Pieces of Eight
    s2e10 2018-02-23 15:00 b084f3nk      ""             The Coroner: Series 2 Crash

    s1e01 2018-02-12 15:00 b06q54pd 2018-03-08 09:14:57 The Coroner: Series 1 First Love

I tend to refresh tv.cache daily, so the added s2 around the time they
started to broadcast them instead of s1.

No wonder Andrew Neil is always complaining about the BBC schedulers!

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From skymap62 at gmail.com  Thu Mar  8 08:09:53 2018
From: skymap62 at gmail.com (Chris Marriott)
Date: Thu, 8 Mar 2018 16:09:53 -0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <20180308155317.F386121A85@orac.inputplus.co.uk>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <20180308155317.F386121A85@orac.inputplus.co.uk>
Message-ID: <F287A58A83FC49FCBA0A3677EB88B8F7@DESKTOPFPV36E8>

It's perhaps worth noting that repeats of daytime TV quite commonly seem to 
miss out episodes at random. I guess they show however many episodes they 
need to fill a gap until the next new series starts. Looking back, I see 
that I recorded S2E8 of "The Coroner" (which was called "The Foxby Affair") 
on 2nd December 2016.

Chris


-----Original Message----- 
From: Ralph Corderoy
Sent: Thursday, March 8, 2018 3:53 PM
To: Dave Widgery
Cc: get_iplayer
Subject: Re: The Coroner series 2 Episode 8 not available

Hi Dave,

> Has anyone any idea why Episode 8 of the latest series of The Coroner
> is not available? it isn't available on bbc iplayer either?

The published data for that programme went very wrong.  This is what my
tv.cache saw over time recently.

    $ grep -h The.Coroner ~/.get_iplayer/tv.cache* |
    > gawk -F\| '{print sprintf("s%de%02d", $8, $9), $5, $4, 
strftime("%Y-%m-%d %T", $16), $3, $7}' |
    > sort -u | sed 's/T/ /; s/:00+00:00//'
    s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 
1 First Love
    s1e01 2018-02-12 15:00 b06q54pd 2018-03-08 09:14:57 The Coroner: Series 
1 First Love
    s1e02 2018-02-13 15:00 b06q553j 2018-02-05 11:34:54 The Coroner: Series 
1 How to Catch a Lobster
    s1e03 2018-02-14 15:00 b06q588j 2018-02-05 11:34:54 The Coroner: Series 
1 That's the Way To Do It
    s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 
1 The Fisherman's Tale
    s1e05 2018-02-16 15:00 b06q5rq3 2018-02-06 11:24:56 The Coroner: Series 
1 Gilt
    s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 
1 Capsized
    s1e07 2018-02-20 15:00 b06qjfq0 2018-02-12 07:30:53 The Coroner: Series 
1 The Salcombe Selkie
    s1e08 2018-02-21 15:00 b06qjjlh 2018-02-12 07:30:53 The Coroner: Series 
1 Napoleon's Violin
    s1e09 2018-02-22 15:00 b06qjkc0 2018-02-12 07:30:53 The Coroner: Series 
1 The Deep Freeze
    s1e10 2018-02-23 15:00 b06qjl6j 2018-02-12 07:30:53 The Coroner: Series 
1 Dirty Dancing
    s2e01 2018-02-13 15:00 b0840227 2018-02-13 12:34:30 The Coroner: Series 
2 The Drop Zone
    s2e02 2018-02-14 15:00 b084022y 2018-02-14 08:47:52 The Coroner: Series 
2 Perfectly Formed
    s2e03 2018-02-15 15:00 b084024r 2018-02-14 08:47:52 The Coroner: Series 
2 Those in Peril
    s2e04 2018-02-16 15:00 b084025z 2018-02-14 08:47:52 The Coroner: Series 
2 The Beast of Lighthaven
    s2e05 2018-02-19 15:00 b0840277 2018-02-14 08:47:52 The Coroner: Series 
2 The Captain's Pipe
    s2e06 2018-02-20 15:00 b084dxth 2018-02-14 08:47:52 The Coroner: Series 
2 Life
    s2e07 2018-02-21 15:00 b084dzjy 2018-02-14 08:47:52 The Coroner: Series 
2 Perfect Pair
    s2e09 2018-02-22 15:00 b084f1zy 2018-02-14 08:47:52 The Coroner: Series 
2 Pieces of Eight
    s2e10 2018-02-23 15:00 b084f3nk 2018-02-14 08:47:52 The Coroner: Series 
2 Crash
    $

The first date-time is when it was to be broadcast;  always 3pm.  The
second is when that entry was added to tv.cache.  s1e01 did actually
broadcast, but then all of the rest of s1 didn't.  Not good if you only
invested time watching it on the basis of watching the series that was
already scheduled.  (iPlayer support reference CAS-4806288-B3HT3L.)
s2e08 is missing from that list;  they never scheduled it in this run of
repeats.

Ordering that list by scheduled broadcast time, and eliding duplicate
dates, gives

    s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 
1 First Love
    s1e01      ""          b06q54pd 2018-03-08 09:14:57 The Coroner: Series 
1 First Love
    s1e02 2018-02-13 15:00 b06q553j 2018-02-05 11:34:54 The Coroner: Series 
1 How to Catch a Lobster
    s2e01      ""          b0840227 2018-02-13 12:34:30 The Coroner: Series 
2 The Drop Zone
    s1e03 2018-02-14 15:00 b06q588j 2018-02-05 11:34:54 The Coroner: Series 
1 That's the Way To Do It
    s2e02      ""          b084022y 2018-02-14 08:47:52 The Coroner: Series 
2 Perfectly Formed
    s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 
1 The Fisherman's Tale
    s2e03      ""          b084024r 2018-02-14 08:47:52 The Coroner: Series 
2 Those in Peril
    s1e05 2018-02-16 15:00 b06q5rq3 2018-02-06 11:24:56 The Coroner: Series 
1 Gilt
    s2e04      ""          b084025z 2018-02-14 08:47:52 The Coroner: Series 
2 The Beast of Lighthaven
    s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 
1 Capsized
    s2e05      ""          b0840277 2018-02-14 08:47:52 The Coroner: Series 
2 The Captain's Pipe
    s1e07 2018-02-20 15:00 b06qjfq0 2018-02-12 07:30:53 The Coroner: Series 
1 The Salcombe Selkie
    s2e06      ""          b084dxth 2018-02-14 08:47:52 The Coroner: Series 
2 Life
    s1e08 2018-02-21 15:00 b06qjjlh 2018-02-12 07:30:53 The Coroner: Series 
1 Napoleon's Violin
    s2e07      ""          b084dzjy 2018-02-14 08:47:52 The Coroner: Series 
2 Perfect Pair
    s1e09 2018-02-22 15:00 b06qjkc0 2018-02-12 07:30:53 The Coroner: Series 
1 The Deep Freeze
    s2e09      ""          b084f1zy 2018-02-14 08:47:52 The Coroner: Series 
2 Pieces of Eight
    s1e10 2018-02-23 15:00 b06qjl6j 2018-02-12 07:30:53 The Coroner: Series 
1 Dirty Dancing
    s2e10      ""          b084f3nk 2018-02-14 08:47:52 The Coroner: Series 
2 Crash

These were all `BBC One'.  So it looks like they scheduled and broadcast
s1e01 by mistake, and then switched to s2e01, replacing s1e02, and so
one for the remaining nine slots.  But that meant something had to go
and that was s2e08.

One more ordering, by the date-time it was added to tv.cache, split by
series.

    s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 
1 First Love
    s1e02 2018-02-13 15:00 b06q553j      ""             The Coroner: Series 
1 How to Catch a Lobster
    s1e03 2018-02-14 15:00 b06q588j      ""             The Coroner: Series 
1 That's the Way To Do It
    s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 
1 The Fisherman's Tale
    s1e05 2018-02-16 15:00 b06q5rq3      ""             The Coroner: Series 
1 Gilt
    s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 
1 Capsized
    s1e07 2018-02-20 15:00 b06qjfq0      ""             The Coroner: Series 
1 The Salcombe Selkie
    s1e08 2018-02-21 15:00 b06qjjlh      ""             The Coroner: Series 
1 Napoleon's Violin
    s1e09 2018-02-22 15:00 b06qjkc0      ""             The Coroner: Series 
1 The Deep Freeze
    s1e10 2018-02-23 15:00 b06qjl6j      ""             The Coroner: Series 
1 Dirty Dancing

    s2e01 2018-02-13 15:00 b0840227 2018-02-13 12:34:30 The Coroner: Series 
2 The Drop Zone
    s2e02 2018-02-14 15:00 b084022y 2018-02-14 08:47:52 The Coroner: Series 
2 Perfectly Formed
    s2e03 2018-02-15 15:00 b084024r      ""             The Coroner: Series 
2 Those in Peril
    s2e04 2018-02-16 15:00 b084025z      ""             The Coroner: Series 
2 The Beast of Lighthaven
    s2e05 2018-02-19 15:00 b0840277      ""             The Coroner: Series 
2 The Captain's Pipe
    s2e06 2018-02-20 15:00 b084dxth      ""             The Coroner: Series 
2 Life
    s2e07 2018-02-21 15:00 b084dzjy      ""             The Coroner: Series 
2 Perfect Pair
    s2e09 2018-02-22 15:00 b084f1zy      ""             The Coroner: Series 
2 Pieces of Eight
    s2e10 2018-02-23 15:00 b084f3nk      ""             The Coroner: Series 
2 Crash

    s1e01 2018-02-12 15:00 b06q54pd 2018-03-08 09:14:57 The Coroner: Series 
1 First Love

I tend to refresh tv.cache daily, so the added s2 around the time they
started to broadcast them instead of s1.

No wonder Andrew Neil is always complaining about the BBC schedulers!

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

_______________________________________________
get_iplayer mailing list
get_iplayer at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer 



From alan at soulman1949.com  Thu Mar  8 16:11:54 2018
From: alan at soulman1949.com (Alan Milewczyk)
Date: Fri, 9 Mar 2018 08:11:54 +0800
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <F287A58A83FC49FCBA0A3677EB88B8F7@DESKTOPFPV36E8>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <20180308155317.F386121A85@orac.inputplus.co.uk>
 <F287A58A83FC49FCBA0A3677EB88B8F7@DESKTOPFPV36E8>
Message-ID: <dc3d409d-0c54-440f-5016-3a79c202ab64@soulman1949.com>

On 09/03/2018 00:09, Chris Marriott wrote:
> It's perhaps worth noting that repeats of daytime TV quite commonly 
> seem to miss out episodes at random. I guess they show however many 
> episodes they need to fill a gap until the next new series starts. 
> Looking back, I see that I recorded S2E8 of "The Coroner" (which was 
> called "The Foxby Affair") on 2nd December 2016.
>
> Chris
>

I get the impression that where repeat series are being broadcast they 
don't worry about missing the odd programme. Generally, the only time 
they take steps to broadcast a missed show is if it is the original 
airing especially BBC One in the evenings. Other programmes take lower 
priority, I guess.

A

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus



From dave.widgery at gmail.com  Fri Mar  9 10:00:05 2018
From: dave.widgery at gmail.com (Dave Widgery)
Date: Fri, 09 Mar 2018 18:00:05 +0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <dc3d409d-0c54-440f-5016-3a79c202ab64@soulman1949.com>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <20180308155317.F386121A85@orac.inputplus.co.uk>
 <F287A58A83FC49FCBA0A3677EB88B8F7@desktopfpv36e8>
 <dc3d409d-0c54-440f-5016-3a79c202ab64@soulman1949.com>
Message-ID: <emd15884c3-f269-4e35-9bdf-dcccb6c702b5@owner-pc>

Hi
Thanks for the reply, until somebody pointed it out I hadn't realised 
that it had not been broadcast at all (this time around) and wondered 
why it wasn't available on iplayer, but even if they hadn?t broadcast it 
live you would think that it could still be available on iplayer.

Having said that I wonder why they picked episode 8?  maybe it was the 
one episode that didn't effect the ongoing background story too much. 
(or maybe that it giving the BBC too much credit).

Dave

------ Original Message ------
From: "Alan Milewczyk" <alan at soulman1949.com>
To: get_iplayer at lists.infradead.org
Sent: 09/03/2018 01:11:54
Subject: Re: The Coroner series 2 Episode 8 not available

>On 09/03/2018 00:09, Chris Marriott wrote:
>>It's perhaps worth noting that repeats of daytime TV quite commonly 
>>seem to miss out episodes at random. I guess they show however many 
>>episodes they need to fill a gap until the next new series starts. 
>>Looking back, I see that I recorded S2E8 of "The Coroner" (which was 
>>called "The Foxby Affair") on 2nd December 2016.
>>
>>Chris
>>
>
>I get the impression that where repeat series are being broadcast they 
>don't worry about missing the odd programme. Generally, the only time 
>they take steps to broadcast a missed show is if it is the original 
>airing especially BBC One in the evenings. Other programmes take lower 
>priority, I guess.
>
>A
>
>---
>This email has been checked for viruses by Avast antivirus software.
>https://www.avast.com/antivirus
>
>
>_______________________________________________
>get_iplayer mailing list
>get_iplayer at lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/get_iplayer



From tellyaddict21 at gmx.co.uk  Fri Mar  9 11:46:47 2018
From: tellyaddict21 at gmx.co.uk (tellyaddict)
Date: Fri, 9 Mar 2018 20:46:47 +0100
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <emd15884c3-f269-4e35-9bdf-dcccb6c702b5@owner-pc>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <20180308155317.F386121A85@orac.inputplus.co.uk>
 <F287A58A83FC49FCBA0A3677EB88B8F7@desktopfpv36e8>
 <dc3d409d-0c54-440f-5016-3a79c202ab64@soulman1949.com>
 <emd15884c3-f269-4e35-9bdf-dcccb6c702b5@owner-pc>
Message-ID: <trinity-127e2989-1556-4207-8fdf-6768e82e07be-1520624806983@3c-app-mailcom-bs02>

Hi Dave,

> but even if they hadn?t broadcast it live you would think that it could still be available on iplayer.

The BBC would probably say that rights restrictions stop them doing that if they haven't broadcast it. Having said that, they already do this with a lot of programmes.

> Having said that I wonder why they picked episode 8?  maybe it was the 
> one episode that didn't effect the ongoing background story too much. 
> (or maybe that it giving the BBC too much credit).

Nice idea but I'm not sure they would have thought too much about it. The strange thing here is they obviously had space to broadcast it because they had to fill the final slot on the friday with an episode from series 1 because they had 10 slots and after they scrapped episode 8 they had 9 episodes to fill them.

> Dave


From roger at tricker.co.uk  Fri Mar  9 14:07:03 2018
From: roger at tricker.co.uk (Roger Tricker)
Date: Fri, 9 Mar 2018 22:07:03 -0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <trinity-127e2989-1556-4207-8fdf-6768e82e07be-1520624806983@3c-app-mailcom-bs02>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <20180308155317.F386121A85@orac.inputplus.co.uk>
 <F287A58A83FC49FCBA0A3677EB88B8F7@desktopfpv36e8>
 <dc3d409d-0c54-440f-5016-3a79c202ab64@soulman1949.com>
 <emd15884c3-f269-4e35-9bdf-dcccb6c702b5@owner-pc>
 <trinity-127e2989-1556-4207-8fdf-6768e82e07be-1520624806983@3c-app-mailcom-bs02>
Message-ID: <017501d3b7f2$f58c2f10$e0a48d30$@tricker.co.uk>

Didn't they broadcast S1 Ep 1 instead of S2 Ep1, then followed by S2 Ep 2-7/9-10, so there were 10 programmes for 10 slots?

Roger

-----Original Message-----
From: get_iplayer <get_iplayer-bounces at lists.infradead.org> On Behalf Of tellyaddict
Sent: 09 March 2018 19:47
To: get_iplayer <get_iplayer at lists.infradead.org>
Subject: Re: The Coroner series 2 Episode 8 not available

Hi Dave,

> but even if they hadn?t broadcast it live you would think that it could still be available on iplayer.

The BBC would probably say that rights restrictions stop them doing that if they haven't broadcast it. Having said that, they already do this with a lot of programmes.

> Having said that I wonder why they picked episode 8?  maybe it was the 
> one episode that didn't effect the ongoing background story too much.
> (or maybe that it giving the BBC too much credit).

Nice idea but I'm not sure they would have thought too much about it. The strange thing here is they obviously had space to broadcast it because they had to fill the final slot on the friday with an episode from series 1 because they had 10 slots and after they scrapped episode 8 they had 9 episodes to fill them.

> Dave

_______________________________________________
get_iplayer mailing list
get_iplayer at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer



From chrisjbrady at gmail.com  Sat Mar 10 00:52:19 2018
From: chrisjbrady at gmail.com (CJB)
Date: Sat, 10 Mar 2018 08:52:19 +0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <trinity-127e2989-1556-4207-8fdf-6768e82e07be-1520624806983@3c-app-mailcom-bs02>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <20180308155317.F386121A85@orac.inputplus.co.uk>
 <F287A58A83FC49FCBA0A3677EB88B8F7@desktopfpv36e8>
 <dc3d409d-0c54-440f-5016-3a79c202ab64@soulman1949.com>
 <emd15884c3-f269-4e35-9bdf-dcccb6c702b5@owner-pc>
 <trinity-127e2989-1556-4207-8fdf-6768e82e07be-1520624806983@3c-app-mailcom-bs02>
Message-ID: <CAPZzCnPe46t31TXJve3PgG_2E+HBMnfBCcgb1gr=cE2tZKMR3w@mail.gmail.com>

Sure its on TV Chaos - take a look CJB

On 09/03/2018, tellyaddict <tellyaddict21 at gmx.co.uk> wrote:
> Hi Dave,
>
>> but even if they hadn?t broadcast it live you would think that it could
>> still be available on iplayer.
>
> The BBC would probably say that rights restrictions stop them doing that if
> they haven't broadcast it. Having said that, they already do this with a lot
> of programmes.
>
>> Having said that I wonder why they picked episode 8?  maybe it was the
>> one episode that didn't effect the ongoing background story too much.
>> (or maybe that it giving the BBC too much credit).
>
> Nice idea but I'm not sure they would have thought too much about it. The
> strange thing here is they obviously had space to broadcast it because they
> had to fill the final slot on the friday with an episode from series 1
> because they had 10 slots and after they scrapped episode 8 they had 9
> episodes to fill them.
>
>> Dave
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer
>


From ralph at inputplus.co.uk  Sat Mar 10 04:19:46 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sat, 10 Mar 2018 12:19:46 +0000
Subject: The Coroner series 2 Episode 8 not available
In-Reply-To: <017501d3b7f2$f58c2f10$e0a48d30$@tricker.co.uk>
References: <em358ab284-9151-4a33-bbb3-7f217fa2856c@owner-pc>
 <20180308155317.F386121A85@orac.inputplus.co.uk>
 <F287A58A83FC49FCBA0A3677EB88B8F7@desktopfpv36e8>
 <dc3d409d-0c54-440f-5016-3a79c202ab64@soulman1949.com>
 <emd15884c3-f269-4e35-9bdf-dcccb6c702b5@owner-pc>
 <trinity-127e2989-1556-4207-8fdf-6768e82e07be-1520624806983@3c-app-mailcom-bs02>
 <017501d3b7f2$f58c2f10$e0a48d30$@tricker.co.uk>
Message-ID: <20180310121946.D9EED1FE27@orac.inputplus.co.uk>

Hi Roger,

> tellyadict wrote:
> > Nice idea but I'm not sure they would have thought too much about
> > it. The strange thing here is they obviously had space to broadcast
> > it because they had to fill the final slot on the friday with an
> > episode from series 1 because they had 10 slots and after they
> > scrapped episode 8 they had 9 episodes to fill them.
>
> Didn't they broadcast S1 Ep 1 instead of S2 Ep1, then followed by S2
> Ep 2-7/9-10, so there were 10 programmes for 10 slots?

Exactly.
http://lists.infradead.org/pipermail/get_iplayer/2018-March/011592.html
has the gore.

They published the schedule for S1 E1-10 up to a week in advance, then
*after* E1 had broadcast they scheduled S2 E1-7,9-10 for the same
remaining time slots as S1 E2-E10.

Perhaps the BBC either consider it junk that needs no continuity, or the
afternoon audience incapable of noticing.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From richard22j at zoho.com  Tue Mar 13 05:06:22 2018
From: richard22j at zoho.com (RS)
Date: Tue, 13 Mar 2018 12:06:22 +0000
Subject: --audio-only
Message-ID: <95caa2bf-67e6-c3df-88e3-c25092e0b0da@zoho.com>

I was pleased that Dinky implemented the --audio-only option because it 
can be can be useful for discussion programmes on a slow or metered 
connection.

Is there any way to force it to 128kbit/s?  I have just downloaded 
Sunday's Andrew Marr, b09w2zj2, with --tvmode=hvflow --audio-only.  I 
got a bit rate 317kbit/s and a file nearly three times as large as I wanted.



From richard22j at zoho.com  Tue Mar 13 05:21:57 2018
From: richard22j at zoho.com (RS)
Date: Tue, 13 Mar 2018 12:21:57 +0000
Subject: --audio-only
In-Reply-To: <95caa2bf-67e6-c3df-88e3-c25092e0b0da@zoho.com>
References: <95caa2bf-67e6-c3df-88e3-c25092e0b0da@zoho.com>
Message-ID: <ecaa4837-80c5-046f-cfad-0ba81542f6f8@zoho.com>

On 13/03/18 12:06, I wrote:
> I was pleased that Dinky implemented the --audio-only option because it 
> can be can be useful for discussion programmes on a slow or metered 
> connection.
> 
> Is there any way to force it to 128kbit/s?? I have just downloaded 
> Sunday's Andrew Marr, b09w2zj2, with --tvmode=hvflow --audio-only.? I 
> got a bit rate 317kbit/s and a file nearly three times as large as I 
> wanted.
> 
It seems I can get a smaller download with --tvmode=dvflow --audio-only. 
  I then get 94kbit/s.  I also get the SBR extension (HE-AAC v1) which I 
did not really want either since most of my players do not support it.




From richard22j at zoho.com  Tue Mar 13 05:34:23 2018
From: richard22j at zoho.com (RS)
Date: Tue, 13 Mar 2018 12:34:23 +0000
Subject: --audio-only
In-Reply-To: <ecaa4837-80c5-046f-cfad-0ba81542f6f8@zoho.com>
References: <95caa2bf-67e6-c3df-88e3-c25092e0b0da@zoho.com>
 <ecaa4837-80c5-046f-cfad-0ba81542f6f8@zoho.com>
Message-ID: <afba26c0-f5cd-65d7-c87b-7a6e641aa1f4@zoho.com>



On 13/03/18 12:21, RS wrote:
> On 13/03/18 12:06, I wrote:
>> I was pleased that Dinky implemented the --audio-only option because 
>> it can be can be useful for discussion programmes on a slow or metered 
>> connection.
>>
>> Is there any way to force it to 128kbit/s?? I have just downloaded 
>> Sunday's Andrew Marr, b09w2zj2, with --tvmode=hvflow --audio-only.? I 
>> got a bit rate 317kbit/s and a file nearly three times as large as I 
>> wanted.
>>
> It seems I can get a smaller download with --tvmode=dvflow --audio-only. 
>  ?I then get 94kbit/s.? I also get the SBR extension (HE-AAC v1) which I 
> did not really want either since most of my players do not support it.
> 
The answer seems to be --tvmode=dvfhigh --audio-only.  That gives me 
126kbit/s without SBR.




From richard22j at zoho.com  Thu Mar 22 10:57:47 2018
From: richard22j at zoho.com (RS)
Date: Thu, 22 Mar 2018 17:57:47 +0000
Subject: Unable to refresh
Message-ID: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>

Am I doing something stupid, or is there a problem refreshing the cache?

I have just refreshed with
get_iplayer --refresh --type all
and I get told the number of tv and radio programmes added is 0.

get_player .* --since 70
shows 67 programmes

get_iplayer .* --since 60
show 0 matching programmes



From gasssmith at gmail.com  Thu Mar 22 11:15:56 2018
From: gasssmith at gmail.com (Geoff Smith)
Date: Thu, 22 Mar 2018 18:15:56 +0000
Subject: Unable to refresh
In-Reply-To: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
Message-ID: <CAJdt=oHXf5L1yShi-Es57q_DoRuCsd6QwWVPNzrMAUvYj0X2MA@mail.gmail.com>

https://forums.squarepenguin.co.uk/showthread.php?tid=1707

On 22/03/2018, RS <richard22j at zoho.com> wrote:
> Am I doing something stupid, or is there a problem refreshing the cache?
>
> I have just refreshed with
> get_iplayer --refresh --type all
> and I get told the number of tv and radio programmes added is 0.
>
> get_player .* --since 70
> shows 67 programmes
>
> get_iplayer .* --since 60
> show 0 matching programmes
>
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer
>


From ralph at inputplus.co.uk  Thu Mar 22 11:24:06 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Thu, 22 Mar 2018 18:24:06 +0000
Subject: Unable to refresh
In-Reply-To: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
Message-ID: <20180322182406.569C0208E9@orac.inputplus.co.uk>

Hi Richard,

> Am I doing something stupid, or is there a problem refreshing the cache?
...
> get_player .* --since 70
> shows 67 programmes

If you are typing `get_player .* --since 70' into a Linux shell then it
will glob the `.*' and replace it with the expansion, e.g. `. ..',
unless it's quoted.  I don't see how that will be affecting your lack of
additions in the last 60 hours, but it clouds the problem.  If you are
quoting it, then paste what you run to stop wasting our time.  :-)

This will show when PIDs were added to your cache, in order.

    sort -t\| -k16,16n -k1,1 ~/.get_iplayer/tv.cache |
    gawk -F\| '
	!/^#/ {
	    print strftime("%Y-%m-%d %T %z %a", $16, 1) "  " $4 "  " $3
	}
    '

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From richard22j at zoho.com  Thu Mar 22 11:57:32 2018
From: richard22j at zoho.com (RS)
Date: Thu, 22 Mar 2018 18:57:32 +0000
Subject: Unable to refresh
In-Reply-To: <CAJdt=oHXf5L1yShi-Es57q_DoRuCsd6QwWVPNzrMAUvYj0X2MA@mail.gmail.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <CAJdt=oHXf5L1yShi-Es57q_DoRuCsd6QwWVPNzrMAUvYj0X2MA@mail.gmail.com>
Message-ID: <5dc0649b-6e68-0aad-ee16-668181ebb407@zoho.com>

On 22/03/18 18:15, Geoff Smith wrote:
> https://forums.squarepenguin.co.uk/showthread.php?tid=1707
> 
> On 22/03/2018, RS <richard22j at zoho.com> wrote:
>> Am I doing something stupid, or is there a problem refreshing the cache?
>>
>> I have just refreshed with
>> get_iplayer --refresh --type all
>> and I get told the number of tv and radio programmes added is 0.
>>
>> get_player .* --since 70
>> shows 67 programmes
>>
>> get_iplayer .* --since 60
>> show 0 matching programmes
>>

Thanks for pointing me to that.  I had a gap in my cache as I had been 
away, and I wondered if that was causing a problem.  Thanks for saving 
me an unnecessary investigation.

Best wishes
Richard




From mail at john-eason.co.uk  Thu Mar 22 12:09:00 2018
From: mail at john-eason.co.uk (J K.Eason)
Date: Thu, 22 Mar 2018 19:09 +0000 (GMT Standard Time)
Subject: Unable to refresh
In-Reply-To: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
Message-ID: <memo.20180322190914.5456P@jeason.cix.co.uk>

> *From:* RS <richard22j at zoho.com>
> *To:* get_iplayer at lists.infradead.org
> *Date:* Thu, 22 Mar 2018 17:57:47 +0000
> 
> Am I doing something stupid, or is there a problem refreshing the 
> cache?
> 
> I have just refreshed with
> get_iplayer --refresh --type all
> and I get told the number of tv and radio programmes added is 0.
> 
> get_player .* --since 70
> shows 67 programmes
> 
> get_iplayer .* --since 60
> show 0 matching programmes

The BBC have broken GIP by changing the page layout according to a post
by Dinky a short while ago at
https://forums.squarepenguin.co.uk/thread-1707.html
 
"The layout of BBC schedule pages has just changed, so get_iplayer cache
updates are broken at the moment. Download programmes via --pid or --url
until things are sorted."

-- 
Regards
       John


From richard22j at zoho.com  Thu Mar 22 12:25:14 2018
From: richard22j at zoho.com (RS)
Date: Thu, 22 Mar 2018 19:25:14 +0000
Subject: Unable to refresh
In-Reply-To: <20180322182406.569C0208E9@orac.inputplus.co.uk>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
Message-ID: <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>



On 22/03/18 18:24, Ralph Corderoy wrote:
> Hi Richard,
> 
>> Am I doing something stupid, or is there a problem refreshing the cache?
> ...
>> get_player .* --since 70
>> shows 67 programmes
> 
> If you are typing `get_player .* --since 70' into a Linux shell then it
> will glob the `.*' and replace it with the expansion, e.g. `. ..',
> unless it's quoted.  I don't see how that will be affecting your lack of
> additions in the last 60 hours, but it clouds the problem.  If you are
> quoting it, then paste what you run to stop wasting our time.  :-)
> 
> This will show when PIDs were added to your cache, in order.
> 
>      sort -t\| -k16,16n -k1,1 ~/.get_iplayer/tv.cache |
>      gawk -F\| '
> 	!/^#/ {
> 	    print strftime("%Y-%m-%d %T %z %a", $16, 1) "  " $4 "  " $3
> 	}
>      '
Hi Ralph

Sorry if I've got the command wrong.

get_iplayer .* --since 70
get_iplayer '.*' --since 70
get_iplayer ".*" --since 70

all give 67 programmes

I don't know how to insert your left back quote ` in a Linux Terminal.

I need to go back to the documentation to check what the correct format 
is to use with --since

Anyway Geoff Smith has confirmed there is a problem because the layout 
of the schedule pages has changed.

Best wishes
Richard



From mike.ralphson at gmx.com  Thu Mar 22 12:29:05 2018
From: mike.ralphson at gmx.com (Mike Ralphson)
Date: Thu, 22 Mar 2018 20:29:05 +0100
Subject: Unable to refresh
In-Reply-To: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
Message-ID: <trinity-32e3b2ea-a243-43a3-a47c-7de3a2ffb83a-1521746945831@3c-app-mailcom-bs07>

Just in case there is an extended period until get-iplayer can refresh its cache again, daily radio and tv cache files are still available from

https://schedules.github.io/ess/
?
Mike


From richard22j at zoho.com  Thu Mar 22 12:49:17 2018
From: richard22j at zoho.com (RS)
Date: Thu, 22 Mar 2018 19:49:17 +0000
Subject: Unable to refresh
In-Reply-To: <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
Message-ID: <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>



On 22/03/18 19:25, I wrote:
> 
> 
> On 22/03/18 18:24, Ralph Corderoy wrote:

>> If you are typing `get_player .* --since 70' into a Linux shell then it
>> will glob the `.*' and replace it with the expansion, e.g. `. ..',
>> unless it's quoted. 

> Sorry if I've got the command wrong.
> 
> get_iplayer .* --since 70
> get_iplayer '.*' --since 70
> get_iplayer ".*" --since 70
> 
> all give 67 programmes
> 
> I don't know how to insert your left back quote ` in a Linux Terminal.
> 
> I need to go back to the documentation to check what the correct format 
> is to use with --since
> 
The 3.09 release notes say,
"get_iplayer no longer lists all programmes when invoked without a 
search argument. If you wish to list all programmes, you must now 
explicitly specify a wildcard search: get_iplayer ".*" - note the 
quotes. The Web PVR Manager does that by default. Also remember to use 
--refresh to force ad hoc cache updates if you don't supply a search 
argument."

"note the quotes" is in bold, but in this case it seems to give the same 
results without.

Best wishes
Richard



From ralph at inputplus.co.uk  Thu Mar 22 16:40:17 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Thu, 22 Mar 2018 23:40:17 +0000
Subject: Unable to refresh
In-Reply-To: <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
Message-ID: <20180322234017.E0068208E9@orac.inputplus.co.uk>

Hi Richard,

> > > If you are typing `get_player .* --since 70' into a Linux shell
> > > then it will glob the `.*' and replace it with the expansion, e.g.
> > > `. ..', unless it's quoted. 
>
> The 3.09 release notes say, "get_iplayer no longer lists all
> programmes when invoked without a search argument. If you wish to list
> all programmes, you must now explicitly specify a wildcard search:
> get_iplayer ".*" - note the quotes.
>
> "note the quotes" is in bold, but in this case it seems to give the
> same results without.

The instructions here say to add two and two, in bold, but I find
multiplying them works just as well, and raising one to the power of the
other.  :-)

?.*? glob'd by the shell to ?.? and ?..?, and perhaps other things, then
gives get_iplayer two or more regexps and it tries to match any of them.
It's not your intent.  Convention is to use the strongest quotes
possible to ease the interpretation by the readers.  For regexps, that's
single quotes, so ?get_iplayer '.*'?.  But ?get_iplayer ^? will give the
same results and needs no quotes.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From richard22j at zoho.com  Fri Mar 23 05:48:18 2018
From: richard22j at zoho.com (RS)
Date: Fri, 23 Mar 2018 12:48:18 +0000
Subject: Unable to refresh
In-Reply-To: <20180322234017.E0068208E9@orac.inputplus.co.uk>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
Message-ID: <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>



On 22/03/18 23:40, Ralph Corderoy wrote:
> Hi Richard,
> 
>>>> If you are typing `get_player .* --since 70' into a Linux shell
>>>> then it will glob the `.*' and replace it with the expansion, e.g.
>>>> `. ..', unless it's quoted.
>>
>> The 3.09 release notes say, "get_iplayer no longer lists all
>> programmes when invoked without a search argument. If you wish to list
>> all programmes, you must now explicitly specify a wildcard search:
>> get_iplayer ".*" - note the quotes.
>>
>> "note the quotes" is in bold, but in this case it seems to give the
>> same results without.
> 
> The instructions here say to add two and two, in bold, but I find
> multiplying them works just as well, and raising one to the power of the
> other.  :-)
> 
> ?.*? glob'd by the shell to ?.? and ?..?, and perhaps other things, then
> gives get_iplayer two or more regexps and it tries to match any of them.
> It's not your intent.  Convention is to use the strongest quotes
> possible to ease the interpretation by the readers.  For regexps, that's
> single quotes, so ?get_iplayer '.*'?.  But ?get_iplayer ^? will give the
> same results and needs no quotes.
> 

Hi Ralph

It can be easy to over-complicate problems.  I am as guilty of that as 
anyone.

The primary symptom was that when I refreshed the cache I was told 0 
programmes had been added.  I was aware that the algorithm for scraping 
the schedules had been described as fragile.

When I made my original post I was hoping someone else would try it to 
confirm whether there was general problem, or one that only affected me. 
  If I had looked at the forum I would have seen that had already been done.

My use of --since was by way of confirmation.  As it happened I 
mis-remembered the command and was too lazy to check the documentation. 
Even so I found that with a --since argument less than 60, no programmes 
were found, and with an argument over 70, programmes were found.  That 
was consistent with the dates in some news programmes found which were 3 
days earlier.

I got exactly the same results when I inserted the missing quotes.  Why 
I got the same results is beyond my level of knowledge.  If I had 
remembered that the search string was a regex I agree I could have used ^

I have come across this in relation to bash
"The characters *, ? and [ are called glob characters or wild card 
characters. If an unquoted argument contains one or more glob 
characters, the shell processes the argument for file name generation. 
The glob characters are part of glob patterns which represent file and 
directory names. These patterns are similar to regular expressions, but 
differ in syntax, since they are intended to match file names and words 
(not arbitrary strings). The special constructions that may appear in 
glob patterns are: ... "

What that seems to mean is that without quotes around *. a file name or 
word can be matched by bash and with quotes an arbitrary string can be 
matched as a regex.  It is not clear to me why that matters.  One thing 
that does not appear to have happened is infinite recursion, or even 
matching of additional programmes.

As for using a single quote ' as the strongest quote, I suspect the 
documentation has used double quotes " for compatibility with Windows.

Best wishes
Richard



From ralph at inputplus.co.uk  Sat Mar 24 02:43:44 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sat, 24 Mar 2018 09:43:44 +0000
Subject: Unable to refresh
In-Reply-To: <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
Message-ID: <20180324094344.CA1A7208E9@orac.inputplus.co.uk>

Hi Richard,

> > > > If you are typing `get_player .* --since 70' into a Linux shell
>
> I have come across this in relation to bash "The characters *, ? and [
> are called glob characters or wild card characters. If an unquoted
> argument contains one or more glob characters, the shell processes the
> argument for file name generation.  The glob characters are part of
> glob patterns which represent file and directory names. These patterns
> are similar to regular expressions, but differ in syntax, since they
> are intended to match file names and words (not arbitrary strings).
> The special constructions that may appear in glob patterns are: ... "
>
> What that seems to mean is that without quotes around *. a file name
> or word can be matched by bash and with quotes an arbitrary string can
> be matched as a regex.  It is not clear to me why that matters.

The shell is expanding globs before invoking get_iplayer, thus they're
not seen by get_iplayer if they match anything.  If they don't match
anything then they normally remain and are passed to get_iplayer anyway.
For the arguments get_iplayer does see, it decides to interpret some of
them as regexps.

?get_iplayer Railway? has no glob metacharacters to expand so one
argument is passed to get_iplayer, it uses it as a regexp, it has no
regexp metacharacters so effectively is a substring search of the
titles.

?get_iplayer R.*way? has a glob metacharacter, the ?*?, the shell looks
at the current directory for entries starting ?R.? and ending ?way?.
There are none.  The glob remains, unexpanded.  get_iplayer has one
argument, ?R.*way? that it uses as a regexp.  There's two regexp
metacharacters, ?.*?, meaning zero or more of any character, used in the
search.

?get_iplayer R.*way? is run again, and again has a glob, the ?*?.  This
time, the current directory has ?R.steinway? in it.  The argument with
the glob is expanded into that and get_iplayer has one argument,
?R.steinway?, that's used as a regexp.  It's unlikely to match any
titles, e.g. ?Resteinway?.

To avoid glob expansion, quote the glob metacharacters, ?get_iplayer
'R.*way'?, and get_iplayer sees the regexp ?R.*way?.

> One thing that does not appear to have happened is infinite recursion,
> or even matching of additional programmes.

Your unquoted ?.*? on Linux would often expand to ?. ..?, and perhaps
more if you've other `dot' files present.  These are two regexps
interpreted by get_iplayer.  It prints titles matching either.  Since
anything matching the second is also matched by the first, you are
seeing any title at least one character long.  That's almost like ?.*?
and ?^? except that a zero-length title won't be matched.

> As for using a single quote ' as the strongest quote, I suspect the
> documentation has used double quotes " for compatibility with Windows.

Yes, I expect you're right.  Fortunately, I've only had a little
exposure to that in the days of DOS.  :-)  If Windows doesn't treat ?^?
specially then that could be used instead with no quoting needed.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From jn.ml.gti.91 at wingsandbeaks.org.uk  Sat Mar 24 02:56:29 2018
From: jn.ml.gti.91 at wingsandbeaks.org.uk (Jeremy Nicoll - ml gip)
Date: Sat, 24 Mar 2018 09:56:29 +0000
Subject: Unable to refresh
In-Reply-To: <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
 <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
Message-ID: <26c36b658b92cd498d2036815fc08a0b@wingsandbeaks.org.uk>

On 2018-03-24 09:43, Ralph Corderoy wrote:

> Yes, I expect you're right.  Fortunately, I've only had a little
> exposure to that in the days of DOS.  :-)  If Windows doesn't treat
> ?^? specially then that could be used instead with no quoting needed.

The ^ character is Windows' escape character for command-line use.

-- 
Jeremy Nicoll - my opinions are my own


From richard22j at zoho.com  Sat Mar 24 04:17:55 2018
From: richard22j at zoho.com (RS)
Date: Sat, 24 Mar 2018 11:17:55 +0000
Subject: Unable to refresh
In-Reply-To: <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
 <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
Message-ID: <adde81f6-d058-23fd-b003-cc2b2b8ed6ff@zoho.com>

On 24/03/18 09:43, Ralph Corderoy wrote:

>>>>> If you are typing `get_player .* --since 70' into a Linux shell
>>

> 
> The shell is expanding globs before invoking get_iplayer, thus they're
> not seen by get_iplayer if they match anything.  If they don't match
> anything then they normally remain and are passed to get_iplayer anyway.
> For the arguments get_iplayer does see, it decides to interpret some of
> them as regexps.
> 
> ?get_iplayer Railway? has no glob metacharacters to expand so one
> argument is passed to get_iplayer, it uses it as a regexp, it has no
> regexp metacharacters so effectively is a substring search of the
> titles.
> 
> ?get_iplayer R.*way? has a glob metacharacter, the ?*?, the shell looks
> at the current directory for entries starting ?R.? and ending ?way?.
> There are none.  The glob remains, unexpanded.  get_iplayer has one
> argument, ?R.*way? that it uses as a regexp.  There's two regexp
> metacharacters, ?.*?, meaning zero or more of any character, used in the
> search.
> 
> ?get_iplayer R.*way? is run again, and again has a glob, the ?*?.  This
> time, the current directory has ?R.steinway? in it.  The argument with
> the glob is expanded into that and get_iplayer has one argument,
> ?R.steinway?, that's used as a regexp.  It's unlikely to match any
> titles, e.g. ?Resteinway?.
> 
> To avoid glob expansion, quote the glob metacharacters, ?get_iplayer
> 'R.*way'?, and get_iplayer sees the regexp ?R.*way?.
> 
>> One thing that does not appear to have happened is infinite recursion,
>> or even matching of additional programmes.
> 
> Your unquoted ?.*? on Linux would often expand to ?. ..?, and perhaps
> more if you've other `dot' files present.  These are two regexps
> interpreted by get_iplayer.  It prints titles matching either.  Since
> anything matching the second is also matched by the first, you are
> seeing any title at least one character long.  That's almost like ?.*?
> and ?^? except that a zero-length title won't be matched.
> 
Hi Ralph

I can see that in some special cases leaving out the quotes will give 
wrong results.  I only wanted a crude indication of how long it had been 
since refreshing the cache had been working, but that is no excuse for 
getting it wrong.

I clearly still need to think it through further.  I would have expected
get_iplayer * --since 110
to match 67 programmes, but it matches 0.

I would have expected
get_iplayer '*' --since 110
to match 0 programmes but it matches 67.

If I put in an invalid regex

get_iplayer *. --since 110
get_iplayer '*.' --since 110

both seem to reach Perl as a regex even though the first has a bash 
wildcard.

Quantifier follows nothing in regex; marked by <-- HERE in m/* <-- HERE 
./ at /usr/bin/get_iplayer line 1245.

Best wishes
Richard






From mtbc at ixod.org  Sat Mar 24 04:26:28 2018
From: mtbc at ixod.org (Mark Carroll)
Date: Sat, 24 Mar 2018 11:26:28 +0000
Subject: Unable to refresh
In-Reply-To: <adde81f6-d058-23fd-b003-cc2b2b8ed6ff@zoho.com> (RS's message of
 "Sat, 24 Mar 2018 11:17:55 +0000")
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
 <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
 <adde81f6-d058-23fd-b003-cc2b2b8ed6ff@zoho.com>
Message-ID: <874ll54suz.fsf@ixod.org>

On 24 Mar 2018, RS wrote:

> I can see that in some special cases leaving out the quotes will give 
> wrong results.  I only wanted a crude indication of how long it had been 
> since refreshing the cache had been working, but that is no excuse for 
> getting it wrong.
>
> I clearly still need to think it through further.  I would have expected
> get_iplayer * --since 110
> to match 67 programmes, but it matches 0.
(snip)

You can type 'echo' first to see what get_iplayer is actually getting:

$ ls 
another  thing
$ echo get_iplayer * --since 110
get_iplayer another thing --since 110

bash won't expand unmatched wildcards:

$ echo a* b*
another b*

-- Mark


From richard22j at zoho.com  Sat Mar 24 04:29:07 2018
From: richard22j at zoho.com (RS)
Date: Sat, 24 Mar 2018 11:29:07 +0000
Subject: Unable to refresh
In-Reply-To: <26c36b658b92cd498d2036815fc08a0b@wingsandbeaks.org.uk>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
 <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
 <26c36b658b92cd498d2036815fc08a0b@wingsandbeaks.org.uk>
Message-ID: <802776e8-d0a4-3c52-4809-2bec6c387d59@zoho.com>

On 24/03/18 09:56, Jeremy Nicoll - ml gip wrote:
> On 2018-03-24 09:43, Ralph Corderoy wrote:
> 
>> Yes, I expect you're right.? Fortunately, I've only had a little
>> exposure to that in the days of DOS.? :-)? If Windows doesn't treat
>> ?^? specially then that could be used instead with no quoting needed.
> 
> The ^ character is Windows' escape character for command-line use.
> 
If you look at
https://github.com/get-iplayer/get_iplayer/wiki/custcmd
and
https://github.com/get-iplayer/get_iplayer/wiki/aactomp3
there are examples of the different quoting needed for Unix/macOS and 
Windows.

Best wishes
Richard




From chrisjbrady at gmail.com  Sat Mar 24 04:44:26 2018
From: chrisjbrady at gmail.com (CJB)
Date: Sat, 24 Mar 2018 11:44:26 +0000
Subject: Unable to refresh
In-Reply-To: <802776e8-d0a4-3c52-4809-2bec6c387d59@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
 <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
 <26c36b658b92cd498d2036815fc08a0b@wingsandbeaks.org.uk>
 <802776e8-d0a4-3c52-4809-2bec6c387d59@zoho.com>
Message-ID: <CAPZzCnOEoWPCOm0ms=N0eaigsQ43X0nUt90_VZeWTd1Kb2QsGA@mail.gmail.com>

So can we expect to see a new version of GiP if the Beeb has rejigged
its schedule pages? Thanks - CJB


From ralph at inputplus.co.uk  Sat Mar 24 04:54:34 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sat, 24 Mar 2018 11:54:34 +0000
Subject: Unable to refresh
In-Reply-To: <adde81f6-d058-23fd-b003-cc2b2b8ed6ff@zoho.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
 <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
 <adde81f6-d058-23fd-b003-cc2b2b8ed6ff@zoho.com>
Message-ID: <20180324115434.7C0EE208F1@orac.inputplus.co.uk>

Hi Richard,

> > If they don't match anything then they normally remain and are
> > passed to get_iplayer anyway.
...
> > The argument with the glob is expanded into that and get_iplayer has
> > one argument, ?R.steinway?, that's used as a regexp.  It's unlikely
> > to match any titles, e.g. ?Resteinway?.
...
> I clearly still need to think it through further.

Or, study my worked example above until it's understood.  :-)

> I would have expected
>     get_iplayer * --since 110
> to match 67 programmes, but it matches 0.

?*? is probably being expanded by the shell based on entries in the
current directory.

> I would have expected
>     get_iplayer '*' --since 110
> to match 0 programmes but it matches 67.

get_iplayer, for some unknown reason, has

    my @search_args = map { $_ eq "*" ? ".*" : $_ } @ARGV;

to break its consistency over argument handling.  It treats these two
the same.  A bad idea IMO.

    get_iplayer '*'
    get_iplayer '.*'

> If I put in an invalid regex
>     get_iplayer *. --since 110
>     get_iplayer '*.' --since 110
> both seem to reach Perl as a regex even though the first has a bash
> wildcard.

Again, the worked example predicts this and covers it.  Create ?a-dot-.?
in the current directory and try again.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy


From ralph at inputplus.co.uk  Sat Mar 24 04:56:08 2018
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sat, 24 Mar 2018 11:56:08 +0000
Subject: Unable to refresh
In-Reply-To: <CAPZzCnOEoWPCOm0ms=N0eaigsQ43X0nUt90_VZeWTd1Kb2QsGA@mail.gmail.com>
References: <873f9891-334e-a56a-ad0e-17fbea77a5fd@zoho.com>
 <20180322182406.569C0208E9@orac.inputplus.co.uk>
 <168de9b7-51a5-378e-0035-4a047cb68795@zoho.com>
 <fded3d35-ae59-4bb3-a05b-6507444dc7b6@zoho.com>
 <20180322234017.E0068208E9@orac.inputplus.co.uk>
 <2f6a6465-08f4-08af-eab9-0ed63bf76c0a@zoho.com>
 <20180324094344.CA1A7208E9@orac.inputplus.co.uk>
 <26c36b658b92cd498d2036815fc08a0b@wingsandbeaks.org.uk>
 <802776e8-d0a4-3c52-4809-2bec6c387d59@zoho.com>
 <CAPZzCnOEoWPCOm0ms=N0eaigsQ43X0nUt90_VZeWTd1Kb2QsGA@mail.gmail.com>
Message-ID: <20180324115608.D63E6208F1@orac.inputplus.co.uk>

> So can we expect to see a new version of GiP if the Beeb has rejigged
> its schedule pages?

Yes.


From richard22j at zoho.com  Sat Mar 24 12:06:45 2018
From: richard22j at zoho.com (RS)
Date: Sat, 24 Mar 2018 19:06:45 +0000
Subject: v3.13 released
Message-ID: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>

v3.13 has been released.
https://github.com/get-iplayer/get_iplayer/wiki/release310to319#release313



From ajebay at errichel.co.uk  Sun Mar 25 10:21:57 2018
From: ajebay at errichel.co.uk (Budge)
Date: Sun, 25 Mar 2018 18:21:57 +0100
Subject: Editing Prefs (Options)
Message-ID: <1e51561f-1a19-799d-32ff-10738c676166@errichel.co.uk>

I have been using a version of ffmpeg which I downloaded when I had 
problems more than a year ago.  I assume that the distribution repo site 
version will by now meet the requirements so wish to take out the ffmpeg 
path line in my options file.

Rather than edit the file I thought I would use --prefs-del.  Is that 
the correct approach and what is syntax to specify the relevant line.  I 
tried using the line as in the file but nothing happened.  Should it be 
in quotes or what?


From richard22j at zoho.com  Sun Mar 25 14:01:56 2018
From: richard22j at zoho.com (RS)
Date: Sun, 25 Mar 2018 22:01:56 +0100
Subject: v3.13 released - question
Message-ID: <1b76121b-4dfd-4943-bff9-035558ebc408@zoho.com>

I'll have a third try at sending this reply.  As I said at the previous 
attempt, that may have failed because I replied to a message which had 
been sent to me instead of the list.  I then tried replying to my 
original message.  I am now trying to send it as a new message.

This reply seems to have gone astray, possibly because the message was 
sent to me rather than the list.

On 25/03/18 17:05, Amro Bilal wrote:
 > Has this release solved the refresh issue? Thanks.
 >
 >
 > Amro
 >
 >
 > On 24/03/2018 07:06 PM, I wrote:
 >> v3.13 has been released.
 >> 
https://github.com/get-iplayer/get_iplayer/wiki/release310to319#release313
 >>

The release notes say it has (see the last item).  As far as I can tell, 
it has.

Best wishes
Richard



From richard22j at zoho.com  Sun Mar 25 10:30:05 2018
From: richard22j at zoho.com (RS)
Date: Sun, 25 Mar 2018 18:30:05 +0100
Subject: v3.13 released
In-Reply-To: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
Message-ID: <561c8426-2dfa-79c8-30a1-76a035ffb567@zoho.com>

This reply seems to have gone astray, possibly because the message was 
sent to me rather than the list.

On 25/03/18 17:05, Amro Bilal wrote:
 > Has this release solved the refresh issue? Thanks.
 >
 >
 > Amro
 >
 >
 > On 24/03/2018 07:06 PM, I wrote:
 >> v3.13 has been released.
 >> 
https://github.com/get-iplayer/get_iplayer/wiki/release310to319#release313
 >>

The release notes say it has (see the last item).  As far as I can tell, 
it has.

Best wishes
Richard




From richard22j at zoho.com  Sun Mar 25 09:46:58 2018
From: richard22j at zoho.com (RS)
Date: Sun, 25 Mar 2018 17:46:58 +0100
Subject: v3.13 released
In-Reply-To: <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
Message-ID: <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>

On 25/03/18 17:05, Amro Bilal wrote:
> Has this release solved the refresh issue? Thanks.
> 
> 
> Amro
> 
> 
> On 24/03/2018 07:06 PM, RS wrote:
>> v3.13 has been released.
>> https://github.com/get-iplayer/get_iplayer/wiki/release310to319#release313 
>>
>>

The release notes say it has (see the last item).  As far as I can tell, 
it has.

Best wishes
Richard





From richard22j at zoho.com  Sun Mar 25 16:19:31 2018
From: richard22j at zoho.com (RS)
Date: Mon, 26 Mar 2018 00:19:31 +0100
Subject: Editing Prefs (Options)
In-Reply-To: <1e51561f-1a19-799d-32ff-10738c676166@errichel.co.uk>
References: <1e51561f-1a19-799d-32ff-10738c676166@errichel.co.uk>
Message-ID: <dd212028-b6ad-a4f8-7181-54bf4c3fc24f@zoho.com>

On 25/03/18 18:21, Budge wrote:
> I have been using a version of ffmpeg which I downloaded when I had 
> problems more than a year ago.? I assume that the distribution repo site 
> version will by now meet the requirements so wish to take out the ffmpeg 
> path line in my options file.
> 
> Rather than edit the file I thought I would use --prefs-del.? Is that 
> the correct approach and what is syntax to specify the relevant line.? I 
> tried using the line as in the file but nothing happened.? Should it be 
> in quotes or what?
> 
The format in the options file and the command are different.  I suggest 
you use --prefs-show to get an idea of the format, although there are 
still differences between the display and the command, so you'll need to 
experiment a bit.  Yes, you do need quotes for most of the --prefs 
options, otherwise what you put in will be interpreted as a search string.

After writing that I tried deleting some options.  There seems to be a 
bug in that it will not accept a space between the option and its 
argument.  There has to be an = character.  I think you probably need

--prefs-del 'ffmpeg=/usr/bin/'
or whatever your path is.

Best wishes
Richard





From info at amrobilal.net  Sun Mar 25 21:47:14 2018
From: info at amrobilal.net (Amro Bilal)
Date: Mon, 26 Mar 2018 05:47:14 +0100
Subject: v3.13 released
In-Reply-To: <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
Message-ID: <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>

Apologies, yes it looks I mistakenly replied to you directly rather than the list. And thanks a lot for answering my question.
All the best
Amro 

Sent from my iPhone

> On 25 Mar 2018, at 5:46 pm, RS <richard22j at zoho.com> wrote:
> 
>> On 25/03/18 17:05, Amro Bilal wrote:
>> Has this release solved the refresh issue? Thanks.
>> Amro
>>> On 24/03/2018 07:06 PM, RS wrote:
>>> v3.13 has been released.
>>> https://github.com/get-iplayer/get_iplayer/wiki/release310to319#release313 
>>> 
> 
> The release notes say it has (see the last item).  As far as I can tell, it has.
> 
> Best wishes
> Richard
> 
> 
> 
> 
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer



From ajebay at errichel.co.uk  Mon Mar 26 01:30:40 2018
From: ajebay at errichel.co.uk (Budge)
Date: Mon, 26 Mar 2018 09:30:40 +0100
Subject: Editing Prefs (Options)
In-Reply-To: <dd212028-b6ad-a4f8-7181-54bf4c3fc24f@zoho.com>
References: <1e51561f-1a19-799d-32ff-10738c676166@errichel.co.uk>
 <dd212028-b6ad-a4f8-7181-54bf4c3fc24f@zoho.com>
Message-ID: <701137ff-5562-78e9-f361-7f2df7b74882@errichel.co.uk>

On 26/03/18 00:19, RS wrote:
> On 25/03/18 18:21, Budge wrote:
>> I have been using a version of ffmpeg which I downloaded when I had 
>> problems more than a year ago.? I assume that the distribution repo 
>> site version will by now meet the requirements so wish to take out the 
>> ffmpeg path line in my options file.
>>
>> Rather than edit the file I thought I would use --prefs-del.? Is that 
>> the correct approach and what is syntax to specify the relevant line.  
>> I tried using the line as in the file but nothing happened.? Should it 
>> be in quotes or what?
>>
> The format in the options file and the command are different.? I suggest 
> you use --prefs-show to get an idea of the format, although there are 
> still differences between the display and the command, so you'll need to 
> experiment a bit.? Yes, you do need quotes for most of the --prefs 
> options, otherwise what you put in will be interpreted as a search string.
> 
> After writing that I tried deleting some options.? There seems to be a 
> bug in that it will not accept a space between the option and its 
> argument.? There has to be an = character.? I think you probably need
> 
> --prefs-del 'ffmpeg=/usr/bin/'
> or whatever your path is.
> 
> Best wishes
> Richard

Hi Richard,
That was it.  Many thanks.
Budge


From roadcone at gmx.com  Tue Mar 27 13:19:43 2018
From: roadcone at gmx.com (Clive)
Date: Tue, 27 Mar 2018 21:19:43 +0100
Subject: v3.13 released
In-Reply-To: <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
Message-ID: <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>

I've been away so my apologies if this has been covered. I understand 
that scraping the BBC pages was broken but is fixed in v3.13 - well, not 
for me it seems. I am using get_iplayer in a Terminal within Linux Mint. 
For example, there are three episodes of Hitchhiker available but only 
Episodes 1 & 2 are found by the term: get_iplayer hitch. I can get the 
third episode from the pid. Can anyone offer any guidance please?

Thank you.

Clive


From roger at firedrake.org  Tue Mar 27 13:34:00 2018
From: roger at firedrake.org (Roger Bell_West)
Date: Tue, 27 Mar 2018 21:34:00 +0100
Subject: v3.13 released
In-Reply-To: <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
Message-ID: <20180327203400.jft3a2romeone7t2@firedrake.org>

On Tue, Mar 27, 2018 at 09:19:43PM +0100, Clive wrote:
>I've been away so my apologies if this has been covered. I understand that
>scraping the BBC pages was broken but is fixed in v3.13 - well, not for me it
>seems. I am using get_iplayer in a Terminal within Linux Mint. For example,
>there are three episodes of Hitchhiker available but only Episodes 1 & 2 are
>found by the term: get_iplayer hitch. I can get the third episode from the
>pid. Can anyone offer any guidance please?

Since I have successfully downloaded that same programme with 3.13,
can you confirm absolutely that you're running the new version?

(Yes, I know it sounds stupid. I've done it before.)

R


From roadcone at gmx.com  Tue Mar 27 13:36:55 2018
From: roadcone at gmx.com (Clive)
Date: Tue, 27 Mar 2018 21:36:55 +0100
Subject: v3.13 released
In-Reply-To: <20180327203400.jft3a2romeone7t2@firedrake.org>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
Message-ID: <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>

On 27/03/18 21:34, Roger Bell_West wrote:
> On Tue, Mar 27, 2018 at 09:19:43PM +0100, Clive wrote:
>> I've been away so my apologies if this has been covered. I understand that
>> scraping the BBC pages was broken but is fixed in v3.13 - well, not for me it
>> seems. I am using get_iplayer in a Terminal within Linux Mint. For example,
>> there are three episodes of Hitchhiker available but only Episodes 1 & 2 are
>> found by the term: get_iplayer hitch. I can get the third episode from the
>> pid. Can anyone offer any guidance please?
> Since I have successfully downloaded that same programme with 3.13,
> can you confirm absolutely that you're running the new version?
>
> (Yes, I know it sounds stupid. I've done it before.)
>
> R
>
get_iplayer > x.txt produces:

get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis
 ? This program comes with ABSOLUTELY NO WARRANTY; for details use 
--warranty.
 ? This is free software, and you are welcome to redistribute it under 
certain
 ? conditions; use --conditions for details.


From roger at firedrake.org  Tue Mar 27 13:42:48 2018
From: roger at firedrake.org (Roger Bell_West)
Date: Tue, 27 Mar 2018 21:42:48 +0100
Subject: v3.13 released
In-Reply-To: <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
Message-ID: <20180327204248.tkrnepdi3mv4irnp@firedrake.org>

On Tue, Mar 27, 2018 at 09:36:55PM +0100, Clive wrote:
>get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis

OK, that's not a "pure" 3.13, and maybe that's your problem. I'm
running directly off the github download. Anyone else using this PPA
version?


From roadcone at gmx.com  Tue Mar 27 13:45:59 2018
From: roadcone at gmx.com (Clive)
Date: Tue, 27 Mar 2018 21:45:59 +0100
Subject: v3.13 released
In-Reply-To: <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
Message-ID: <b69ecd95-9e63-c8dc-3b31-40fb5f6fe468@gmx.com>


On 27/03/18 21:42, Roger Bell_West wrote:
> On Tue, Mar 27, 2018 at 09:36:55PM +0100, Clive wrote:
>> get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis
> OK, that's not a "pure" 3.13, and maybe that's your problem. I'm
> running directly off the github download. Anyone else using this PPA
> version?
>
If you were to email me the .pl script directly I could rename the one I 
have and try your copy in its place.


From getiplayer at squarepenguin.co.uk  Tue Mar 27 13:48:09 2018
From: getiplayer at squarepenguin.co.uk (SquarePenguin)
Date: Tue, 27 Mar 2018 21:48:09 +0100
Subject: v3.13 released
In-Reply-To: <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
Message-ID: <1522183689.22679.5.camel@squarepenguin.co.uk>

On Tue, 2018-03-27 at 21:42 +0100, Roger Bell_West wrote:
> OK, that's not a "pure" 3.13, and maybe that's your problem.

Unlikely. That's just the Ubuntu PPA which basically installs the exact same
files as if you do it manually. 

I'm using it and see episodes 2 and 3 just fine. (I don't see 1 because I
haven't refreshed my cache in a while).



From graeme.thorn at gmail.com  Tue Mar 27 13:54:09 2018
From: graeme.thorn at gmail.com (Graeme Thorn)
Date: Tue, 27 Mar 2018 21:54:09 +0100
Subject: v3.13 released
In-Reply-To: <1522183689.22679.5.camel@squarepenguin.co.uk>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <1522183689.22679.5.camel@squarepenguin.co.uk>
Message-ID: <CACgf1F2NRACkZ_EQNuazCpcrdrCVHRwe47M_m2es3FHdkv70FQ@mail.gmail.com>

Hi,

I would suggest doing a full cache rebuild using get_iplayer
--cache-rebuild --type="radio", as it could be that the days then g_ip
couldn't download are missing from them.

G

On 27 March 2018 at 21:48, SquarePenguin <getiplayer at squarepenguin.co.uk> wrote:
> On Tue, 2018-03-27 at 21:42 +0100, Roger Bell_West wrote:
>> OK, that's not a "pure" 3.13, and maybe that's your problem.
>
> Unlikely. That's just the Ubuntu PPA which basically installs the exact same
> files as if you do it manually.
>
> I'm using it and see episodes 2 and 3 just fine. (I don't see 1 because I
> haven't refreshed my cache in a while).
>
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


From clanlaw at gmail.com  Tue Mar 27 13:54:28 2018
From: clanlaw at gmail.com (Colin Law)
Date: Tue, 27 Mar 2018 21:54:28 +0100
Subject: v3.13 released
In-Reply-To: <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
Message-ID: <CAL=0gLv39m5zEiKrPY8LE5HTUx7Pe7mcCKRyUYk1styuLdo6PQ@mail.gmail.com>

On 27 March 2018 at 21:42, Roger Bell_West <roger at firedrake.org> wrote:
> On Tue, Mar 27, 2018 at 09:36:55PM +0100, Clive wrote:
>>get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis
>
> OK, that's not a "pure" 3.13, and maybe that's your problem. I'm
> running directly off the github download. Anyone else using this PPA
> version?

I am seeing the same issue (only the first two episodes) running that
version on a pi.
$ apt-cache policy get-iplayer
get-iplayer:
  Installed: 3.13-ppa31d~stable
  Candidate: 3.13-ppa31d~stable
  Version table:
 *** 3.13-ppa31d~stable 0
        500 http://packages.hedgerows.org.uk/raspbian/ stable/ Packages
        100 /var/lib/dpkg/status


From getiplayer at squarepenguin.co.uk  Tue Mar 27 14:01:11 2018
From: getiplayer at squarepenguin.co.uk (SquarePenguin)
Date: Tue, 27 Mar 2018 22:01:11 +0100
Subject: v3.13 released
In-Reply-To: <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
Message-ID: <1522184471.22679.15.camel@squarepenguin.co.uk>

On Tue, 2018-03-27 at 21:19 +0100, Clive wrote: 
> Episodes 1 & 2 are found by the term: get_iplayer hitch. I can get the 
> third episode from the pid. Can anyone offer any guidance please?

Unless my installation is broken too I'm surprised you can see anything with
`get_iplayer hitch`. 

Those episodes are radio so I'd expect to have to use get_iplayer --type=radio
hitch` to see those episodes. 

Perhaps a typo (or some other option preference saved), but nonetheless I'm also
surprised you get episodes 1 and 2 but not 3. Being the oldest episode I'd have
guessed at episode 1 not being available simply by dint of not updating the
cache over the last 30 days. 

Try forcing a cache update `get_iplayer --type=all --refresh`

After the above, could you post the output of `get_iplayer --type=radio hitch` ?





From roadcone at gmx.com  Tue Mar 27 14:01:51 2018
From: roadcone at gmx.com (Clive)
Date: Tue, 27 Mar 2018 22:01:51 +0100
Subject: v3.13 released
In-Reply-To: <CACgf1F2NRACkZ_EQNuazCpcrdrCVHRwe47M_m2es3FHdkv70FQ@mail.gmail.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <1522183689.22679.5.camel@squarepenguin.co.uk>
 <CACgf1F2NRACkZ_EQNuazCpcrdrCVHRwe47M_m2es3FHdkv70FQ@mail.gmail.com>
Message-ID: <8028bee0-42e4-b9a9-33ed-e9099119eb16@gmx.com>

On 27/03/18 21:54, Graeme Thorn wrote:
> get_iplayer
> --cache-rebuild --type="radio"
That worked. Wow, never seen so many lines of dots on the terminal 
screen. Thank you, I now have three episodes of hitchhiker showing. Must 
do that for TV also.

Thank you again.

Clive



From richard22j at zoho.com  Tue Mar 27 14:03:03 2018
From: richard22j at zoho.com (RS)
Date: Tue, 27 Mar 2018 22:03:03 +0100
Subject: v3.13 released
In-Reply-To: <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
Message-ID: <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>



On 27/03/18 21:42, Roger Bell_West wrote:
> On Tue, Mar 27, 2018 at 09:36:55PM +0100, Clive wrote:
>> get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis
> 
> OK, that's not a "pure" 3.13, and maybe that's your problem. I'm
> running directly off the github download. Anyone else using this PPA
> version?
> 

I too can see three episodes.  I didn't wait for the PPA (for ubuntu 
16.04.3). I took the new get_iplayer Perl file from the .zip archive and 
replaced it in /usr/bin/.  Curiously it is now showing
get_iplayer 3.13-ppa31d
which I don't think it did when I first started using the new version. 
Maybe the PPA has been updated in a software update.

Best wishes
Richard




From roadcone at gmx.com  Tue Mar 27 14:09:24 2018
From: roadcone at gmx.com (Clive)
Date: Tue, 27 Mar 2018 22:09:24 +0100
Subject: v3.13 released
In-Reply-To: <1522184471.22679.15.camel@squarepenguin.co.uk>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <1522184471.22679.15.camel@squarepenguin.co.uk>
Message-ID: <dfa28e6a-cf41-9e91-ec13-45d89ba39bb1@gmx.com>

On 27/03/18 22:01, SquarePenguin wrote:
> On Tue, 2018-03-27 at 21:19 +0100, Clive wrote:
>> Episodes 1 & 2 are found by the term: get_iplayer hitch. I can get the
>> third episode from the pid. Can anyone offer any guidance please?
> Unless my installation is broken too I'm surprised you can see anything with
> `get_iplayer hitch`.
>
> Those episodes are radio so I'd expect to have to use get_iplayer --type=radio
> hitch` to see those episodes.
>
> Perhaps a typo (or some other option preference saved), but nonetheless I'm also
> surprised you get episodes 1 and 2 but not 3. Being the oldest episode I'd have
> guessed at episode 1 not being available simply by dint of not updating the
> cache over the last 30 days.
>
> Try forcing a cache update `get_iplayer --type=all --refresh`
>
> After the above, could you post the output of `get_iplayer --type=radio hitch` ?
I download much more radio than TV so I have radio set in the prefs 
file. I had done a forced --refresh previously (albeit only on radio) 
and noted that many programs were added to the list so thought nothing 
of it. As requested:

get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis
 ? This program comes with ABSOLUTELY NO WARRANTY; for details use 
--warranty.
 ? This is free software, and you are welcome to redistribute it under 
certain
 ? conditions; use --conditions for details.


Matches:
44741:??? The Hitchhiker's Guide to the Galaxy: Hexagonal Phase - 
Episode 1, BBC Radio 4, b09th4hf
44742:??? The Hitchhiker's Guide to the Galaxy: Hexagonal Phase - 
Episode 2, BBC Radio 4, b09v734q
44743:??? The Hitchhiker's Guide to the Galaxy: Hexagonal Phase - 
Episode 3, BBC Radio 4, b09w16mn
INFO: 3 matching programmes

You'll see from an earlier response that the problem has now been 
resolved with a full rebuild of the cache.

Regards

Clive



From clanlaw at gmail.com  Tue Mar 27 14:20:41 2018
From: clanlaw at gmail.com (Colin Law)
Date: Tue, 27 Mar 2018 22:20:41 +0100
Subject: v3.13 released
In-Reply-To: <8028bee0-42e4-b9a9-33ed-e9099119eb16@gmx.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <1522183689.22679.5.camel@squarepenguin.co.uk>
 <CACgf1F2NRACkZ_EQNuazCpcrdrCVHRwe47M_m2es3FHdkv70FQ@mail.gmail.com>
 <8028bee0-42e4-b9a9-33ed-e9099119eb16@gmx.com>
Message-ID: <CAL=0gLtuhaeeYKw+SViP0yZpq8ToV01xfJD0J9b7MR9-O=x2wA@mail.gmail.com>

On 27 March 2018 at 22:01, Clive <roadcone at gmx.com> wrote:
> On 27/03/18 21:54, Graeme Thorn wrote:
>>
>> get_iplayer
>> --cache-rebuild --type="radio"
>
> That worked. Wow, never seen so many lines of dots on the terminal screen.
> Thank you, I now have three episodes of hitchhiker showing. Must do that for
> TV also.

cache-rebuild fixed it for me too (I was already using --type=radio).

Thanks

Colin


From richard22j at zoho.com  Tue Mar 27 14:38:58 2018
From: richard22j at zoho.com (RS)
Date: Tue, 27 Mar 2018 22:38:58 +0100
Subject: v3.13 released
In-Reply-To: <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
Message-ID: <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>



On 27/03/18 22:03, I wrote:
> 
> 
> On 27/03/18 21:42, Roger Bell_West wrote:
>> On Tue, Mar 27, 2018 at 09:36:55PM +0100, Clive wrote:
>>> get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis
>>
>> OK, that's not a "pure" 3.13, and maybe that's your problem. I'm
>> running directly off the github download. Anyone else using this PPA
>> version?
>>
> 
> I too can see three episodes.? I didn't wait for the PPA (for ubuntu 
> 16.04.3). I took the new get_iplayer Perl file from the .zip archive and 
> replaced it in /usr/bin/.? Curiously it is now showing
> get_iplayer 3.13-ppa31d
> which I don't think it did when I first started using the new version. 
> Maybe the PPA has been updated in a software update.
> 
> Best wishes
> Richard
> 
> 
The get_iplayer script now in /usr/bin/ is not the one I copied there.
my $version_text;
is now
my $version_text = '3.13-ppa31d';

I have just tried on another machine.  I refreshed the cache.
A search gave me two episodes.
I installed the PPA and refreshed the cache again.  I did not rebuild 
the cache, but nor did I after copying the get_iplayer script to 
/usr/bin/ on my main machine  I can still only see the first two episodes.

Best wishes
Richard



From richard22j at zoho.com  Tue Mar 27 15:36:49 2018
From: richard22j at zoho.com (RS)
Date: Tue, 27 Mar 2018 23:36:49 +0100
Subject: v3.13 released
In-Reply-To: <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
 <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
Message-ID: <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>



On 27/03/18 22:38, I wrote:
> 
> 
> On 27/03/18 22:03, I wrote:
>>
>>
>> On 27/03/18 21:42, Roger Bell_West wrote:
>>> On Tue, Mar 27, 2018 at 09:36:55PM +0100, Clive wrote:
>>>> get_iplayer 3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis
>>>
>>> OK, that's not a "pure" 3.13, and maybe that's your problem. I'm
>>> running directly off the github download. Anyone else using this PPA
>>> version?
>>>
>>
>> I too can see three episodes.? I didn't wait for the PPA (for ubuntu 
>> 16.04.3). I took the new get_iplayer Perl file from the .zip archive 
>> and replaced it in /usr/bin/.? Curiously it is now showing
>> get_iplayer 3.13-ppa31d
>> which I don't think it did when I first started using the new version. 
>> Maybe the PPA has been updated in a software update.
>>

>>
> The get_iplayer script now in /usr/bin/ is not the one I copied there.
> my $version_text;
> is now
> my $version_text = '3.13-ppa31d';
> 
> I have just tried on another machine.? I refreshed the cache.
> A search gave me two episodes.
> I installed the PPA and refreshed the cache again.? I did not rebuild 
> the cache, but nor did I after copying the get_iplayer script to 
> /usr/bin/ on my main machine? I can still only see the first two episodes.
> 
I have run diff to compare the get_iplayer in the .zip archive with the 
PPA version.
my $version_text = '3.13-ppa31d';
is the only difference reported.

A more likely explanation is that the searches were done at different 
times.  As I understand it, and I haven't checked the code, --refresh 
only goes back 7 days, and maybe it is now too old.  That doesn't quite 
work as an explanation because episode 3 was broadcast on Radio 4 at 
1830 on 22 March and won't be repeated on Radio 4 Extra until 29 March. 
Maybe the 7 days has been shortened.  I have a vague recollection that 
the 7 days runs from the start of the week, and that would fit.

Best wishes
Richard




From richard22j at zoho.com  Wed Mar 28 02:34:57 2018
From: richard22j at zoho.com (RS)
Date: Wed, 28 Mar 2018 10:34:57 +0100
Subject: v3.13 released
In-Reply-To: <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
 <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
 <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>
Message-ID: <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>

On 27/03/18 23:36, I wrote:
> 

> 
> A more likely explanation is that the searches were done at different 
> times.? As I understand it, and I haven't checked the code, --refresh 
> only goes back 7 days, and maybe it is now too old.? That doesn't quite 
> work as an explanation because episode 3 was broadcast on Radio 4 at 
> 1830 on 22 March and won't be repeated on Radio 4 Extra until 29 March. 
> Maybe the 7 days has been shortened.? I have a vague recollection that 
> the 7 days runs from the start of the week, and that would fit.
> 

The way in which the cache is refreshed is explained here.
https://github.com/get-iplayer/get_iplayer/wiki/cache

It says,
"get_iplayer does NOT index 30 days of programme listings every time the 
cache is updated - it indexes no further back than the beginning of the 
previous calendar week. This means that you must update your cache each 
calendar week for a month after installation in order to build up a full 
30 days of programme data. From then on, the 30-day buffer should be 
maintained as long as you update the cache at least once per calendar week."

According to that description, a programme broadcast less than 6 days 
previously ought to have been indexed without the need to re-build the 
cache.  It may be that the new scraping algorithm works differently and 
the documentation needs to be updated.

Has anyone come across any other examples?

Best wishes
Richard



From nicsiddle at gmail.com  Wed Mar 28 02:58:44 2018
From: nicsiddle at gmail.com (Nic Siddle)
Date: Wed, 28 Mar 2018 10:58:44 +0100
Subject: v3.13 released
In-Reply-To: <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
 <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
 <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>
 <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>
Message-ID: <2b113a1a-603b-bdf3-2b9e-4af2521edbc9@gmail.com>

Similar apparent problem with Chain Reaction. Not picking up latest in 
Series 3 ep 6 (Jack Dee and Jeremy Hardy). Running Windows 10 Home 64 
bit. Updated to get i-player 3.13. Current PVR search includes the 
following lines:-

_Chain_Reaction_name_radio

_Chain_Reaction_Series_11_name_radio

_Chain_Reaction_Series_3_name_radio

However, when I put the PID into the quick URL search, it finds and 
records OK.

I am non-tech and rarely understand the list messages - but would 
welcome any advice

Nic

On 28/03/2018 10:34, RS wrote:
> On 27/03/18 23:36, I wrote:
>>
>
>>
>> A more likely explanation is that the searches were done at different 
>> times.? As I understand it, and I haven't checked the code, --refresh 
>> only goes back 7 days, and maybe it is now too old.? That doesn't 
>> quite work as an explanation because episode 3 was broadcast on Radio 
>> 4 at 1830 on 22 March and won't be repeated on Radio 4 Extra until 29 
>> March. Maybe the 7 days has been shortened.? I have a vague 
>> recollection that the 7 days runs from the start of the week, and 
>> that would fit.
>>
>
> The way in which the cache is refreshed is explained here.
> https://github.com/get-iplayer/get_iplayer/wiki/cache
>
> It says,
> "get_iplayer does NOT index 30 days of programme listings every time 
> the cache is updated - it indexes no further back than the beginning 
> of the previous calendar week. This means that you must update your 
> cache each calendar week for a month after installation in order to 
> build up a full 30 days of programme data. From then on, the 30-day 
> buffer should be maintained as long as you update the cache at least 
> once per calendar week."
>
> According to that description, a programme broadcast less than 6 days 
> previously ought to have been indexed without the need to re-build the 
> cache.? It may be that the new scraping algorithm works differently 
> and the documentation needs to be updated.
>
> Has anyone come across any other examples?
>
> Best wishes
> Richard
>
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer



From ilainftw at gmail.com  Wed Mar 28 03:03:07 2018
From: ilainftw at gmail.com (iz)
Date: Wed, 28 Mar 2018 11:03:07 +0100
Subject: v3.13 released
In-Reply-To: <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
 <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
 <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>
 <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>
Message-ID: <CAF_ssT9fznZDyxEytGaf=2TwDvbZNGqaWdJiaA9n12Y71LrBFw@mail.gmail.com>

Once get_iplayer updates the cache in any given week, it doesn't go
back to the previous week. Ordinarily that would be unnecessary, but
you can force it with the use of --refresh-limit or --cache-rebuild,
as has already been suggested. It's likely that people did their first
update this week with the old broken get_iplayer, and thus subsequent
updates with the new get_iplayer didn't go back to last week.


From richard22j at zoho.com  Wed Mar 28 03:33:16 2018
From: richard22j at zoho.com (RS)
Date: Wed, 28 Mar 2018 11:33:16 +0100
Subject: v3.13 released
In-Reply-To: <CAF_ssT9fznZDyxEytGaf=2TwDvbZNGqaWdJiaA9n12Y71LrBFw@mail.gmail.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
 <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
 <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>
 <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>
 <CAF_ssT9fznZDyxEytGaf=2TwDvbZNGqaWdJiaA9n12Y71LrBFw@mail.gmail.com>
Message-ID: <7f4f50da-f0af-1680-8ff1-513627536ab4@zoho.com>

On 28/03/18 11:03, iz wrote:
 > Once get_iplayer updates the cache in any given week, it doesn't go
 > back to the previous week. Ordinarily that would be unnecessary, but
 > you can force it with the use of --refresh-limit or --cache-rebuild,
 > as has already been suggested. It's likely that people did their first
 > update this week with the old broken get_iplayer, and thus subsequent
 > updates with the new get_iplayer didn't go back to last week.
 >
You may be right, but both machines I have tried had the cache refreshed 
during the last calendar week.  The one I updated to v3.13 on Saturday 
evening has indexed the Jack Dee/Jeremy Hardy episode of Chain Reaction 
broadcast on Radio 4 Extra at 2330 on 23 March.  The one I updated last 
night has not.

That suggests that with the new algorithm it is going to be necessary to 
refresh more than once a week to maintain a complete cache.

Best wishes
Richard




From ilainftw at gmail.com  Wed Mar 28 04:04:48 2018
From: ilainftw at gmail.com (iz)
Date: Wed, 28 Mar 2018 12:04:48 +0100
Subject: v3.13 released
In-Reply-To: <7f4f50da-f0af-1680-8ff1-513627536ab4@zoho.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
 <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
 <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>
 <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>
 <CAF_ssT9fznZDyxEytGaf=2TwDvbZNGqaWdJiaA9n12Y71LrBFw@mail.gmail.com>
 <7f4f50da-f0af-1680-8ff1-513627536ab4@zoho.com>
Message-ID: <CAF_ssT9zMFvaCj2Aj=J+yszxasekR2vKbFvzkWb_0xq7Or8nYw@mail.gmail.com>

On 28 March 2018 at 11:33, RS <richard22j at zoho.com> wrote:
> That suggests that with the new algorithm it is going to be necessary to
> refresh more than once a week to maintain a complete cache.

There is no "new algorithm", and you don't have to refresh more often
than you did before. If you can't live with a hole in your cache, just
rebuild it.


From john.reay at gmail.com  Wed Mar 28 08:22:48 2018
From: john.reay at gmail.com (John Reay)
Date: Wed, 28 Mar 2018 11:22:48 -0400
Subject: v3.13 released
In-Reply-To: <1522183689.22679.5.camel@squarepenguin.co.uk>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <1522183689.22679.5.camel@squarepenguin.co.uk>
Message-ID: <CAEUCKTwgmLYb8b-JET9KsPODFctCcTWXRPh8MnAxnTibQJ6HDw@mail.gmail.com>

I installed 3.13 from the Ubuntu PPA, replacing 3.12.

if I run get_iplayer from the command line it shows >get_iplayer
3.13-ppa31d, Copyright (C) 2008-2010 Phil Lewis

However the Web PVR screens still say 3.12. I think maybe it's just
the text that wasn't updated, but not 100% sure

On 27 March 2018 at 16:48, SquarePenguin <getiplayer at squarepenguin.co.uk> wrote:
> On Tue, 2018-03-27 at 21:42 +0100, Roger Bell_West wrote:
>> OK, that's not a "pure" 3.13, and maybe that's your problem.
>
> Unlikely. That's just the Ubuntu PPA which basically installs the exact same
> files as if you do it manually.
>
> I'm using it and see episodes 2 and 3 just fine. (I don't see 1 because I
> haven't refreshed my cache in a while).
>
>
> _______________________________________________
> get_iplayer mailing list
> get_iplayer at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


From richard22j at zoho.com  Wed Mar 28 13:06:56 2018
From: richard22j at zoho.com (RS)
Date: Wed, 28 Mar 2018 21:06:56 +0100
Subject: v3.13 released
In-Reply-To: <CAF_ssT9zMFvaCj2Aj=J+yszxasekR2vKbFvzkWb_0xq7Or8nYw@mail.gmail.com>
References: <79692fa8-9df8-586a-dca3-23f2b49b392b@zoho.com>
 <1c6e655d-5c0c-72da-94b8-34910007b69a@amrobilal.net>
 <a9e6cfce-8f29-4574-5a9e-f0f5e3d4cff6@zoho.com>
 <D3E4D9FC-BE5B-41F8-9062-117A45B4D55F@amrobilal.net>
 <0defb304-4812-3c04-0514-535f08ed4fb0@gmx.com>
 <20180327203400.jft3a2romeone7t2@firedrake.org>
 <a3df91e4-a2c5-2013-027f-83ee7326f426@gmx.com>
 <20180327204248.tkrnepdi3mv4irnp@firedrake.org>
 <eac0db4c-e8cf-24d6-bc90-e8b401c95267@zoho.com>
 <94b14e32-1452-4683-5760-9f564492bfac@zoho.com>
 <b959cd60-af19-7052-d247-d9e95223715c@zoho.com>
 <7be212bf-26a6-f027-ccd7-01c98e0b3c4d@zoho.com>
 <CAF_ssT9fznZDyxEytGaf=2TwDvbZNGqaWdJiaA9n12Y71LrBFw@mail.gmail.com>
 <7f4f50da-f0af-1680-8ff1-513627536ab4@zoho.com>
 <CAF_ssT9zMFvaCj2Aj=J+yszxasekR2vKbFvzkWb_0xq7Or8nYw@mail.gmail.com>
Message-ID: <1b5ee1e0-a953-20aa-8eb6-46cc9b095939@zoho.com>

On 28/03/18 12:04, iz wrote:

> There is no "new algorithm", and you don't have to refresh more often
> than you did before. If you can't live with a hole in your cache, just
> rebuild it.
> 
I have another machine which had not been refreshed in the last week. 
It has just been updated to v3.13 and refreshed but not re-built.  It 
has indexed episodes 2 and 3 of the Hithchhiker's Guide to the Galaxy 
and the Jack Dee/Jeremy Hardy episode of Chain Reaction, so you are right.

Although I have not seen it mentioned in the documentation, I did not 
need to re-install the PPA.  It was updated to v3.13 automatically by a 
software update.

Best wishes
Richard



