Format of options file

[MacFH - C E Macfarlane]

Please see below ...

On 04/03/2018 19:59, RS wrote:
>
> On 04/03/18 15:52, MacFH - C E Macfarlane wrote:
>>
>> On 04/03/2018 00:24, RS wrote:
>>>
>>> On 03/03/18 23:44, MacFH - C E Macfarlane wrote:
>>>
>>>> But as the OP is changing over to Linux, perhaps the de facto 
>>>> standard 'dos2unix' would be a more appropriate recommendation. 
>>>
>>> Thanks for the suggestion but I don't think I need to convert the 
>>> download_history file (although gedit complained about invalid UTF-8 
>>> characters) and thanks to Ralph's command I now have an options file 
>>> which consistently uses 0x0D as a line terminator.
>>
>> You seem to be confused about which line terminators are used by each 
>> operating system, so to confirm:
>>      OS        Mnem    Hex code
>>      ---------------------------------
>>      Linux    LF           0x0A
>>      Mac     CR           0x0D (I believe)
>>      Win     CRLF       0x0D0x0A
>
> You're right I am a bit confused, but not about the line termination 
> conventions for each OS, although I can't speak for the Mac.
>
> What I said the download_history file was wrong.  I have now looked at 
> it with a hex editor which allows me to view the whole file.  Most of 
> it was written in Windows and has CRLF as a line terminator.  The most 
> recent records, appended to the end, written in Linux, have LF as a 
> line terminator.  The important thing is that the discrepancy does not 
> stop it working.  The entries are indexed by pid.  The script is able 
> to find the pid whether it is preceded by LF or CRLF, and identify the 
> programme as already in history.  The mixed file works both in Linux 
> and in Windows.
>
> In the case of the options file the discrepancy in line termination 
> does prevent some options from being interpreted correctly.  To find 
> out why I need to compare the Perl code for parsing the 
> download_history file with that for parsing the options file.  I have 
> not yet managed to find it.

Most probably it is those options which are passed to an external 
program that cause difficulty  -  my guess, I stress that it is a guess  
-  the program is getting confused by the unexpected CR 0x0D when it is 
expecting only LF 0x0A, and may be trying to interpret the CR as part of 
the parameter it is being given.

> As for using win2unix to convert, if you look at the win2unix 
> documentation you will see it is not that simple.  It is possible to 
> use it just to change the line terminations.  You also need to think 
> about character sets.  Some of the comments in the code say UTF-8 is 
> used for Linux and cp850 for Windows.  If I remember from 30 odd years 
> ago that was the oldest MSDOS code page.  Should that be converted 
> too?  If I want to go backwards and forwards I don't want to keep 
> converting.

The Windows ANSI and UTF-8 character sets are sufficiently close that I 
wouldn't worry about any need to convert, just do the line-termination.  
Given two directories, .get_iplayer.bak and .get_iplayer, beneath the 
present working directory HOME aka ~, this command should do what you 
want (I've doubled up the spaces for better clarity of the syntax):
     for  F  in  options  download_history  *.cache  ;  do dos2unix -n  
.get_iplayer.bak/$F  .get_iplayer/$F  ;  done

Regards Charles.