[LEDE-DEV] [PATCH 2/3] procd: Don't use syslog before its initialization

[David Lang]

On Sat, 11 Mar 2017, Michal Sojka wrote:

> On Sat, Mar 11 2017, John Crispin wrote:
>> On 11/03/17 01:48, Michal Sojka wrote:
>>> When procd starts a rcS script, it captures its stdout and stderr and
>>> logs them to syslog. It could happen (and I don't know exactly why),
>>> that a rcS scripts produces some output before openlog() is called in
>>> procd. The result is that all subsequent logs from procd are logged
>>> with ident and PID of some other service that happens to be started at
>>> that time. I also have no clue why this is happening (bug in musl?),
>>> but it is very confusing when one tries to debug boot problems.
>>
>> is there a way to reliably reproduce this issue ?
>
> I can reproduce it reliably with handful of custom packages. I'll try to
> prepare a more simpler setup where this can be seen.

This sounds like a problem with procd, capturing stdout and stderr and feeding 
them to syslog could cause a serious problem if the error messages are from the 
syslog daemon.

it sounds like they are trying to emulate systemd, and that's not always a good 
thing :-)

sending a bunch of things to syslog before the system is up can also run into 
problems if the syslog daemon is configured to send the logs to a remote server, 
and can't reach it yet (all sorts of possible reasons for that).

all sorts of programs spit various garbaage out to stdout or stderr that is 
meaningless/debug stuff that really should not be turned on for production use, 
but was missed, this can cause a lot of log messages (and stderr messages in 
particular tend to be very badly formed)

this sort of thing in the init system needs safeguards, and those need to be 
thought about.

possibly try to deliver the log to syslog, but throw them away if the attempt 
blocks

possibly try to detect that it's starting the syslog daemon and skip delivering 
logs

poosibly have the startup script for the logging daemon set some lock to turn on 
this functionality (and leave it up to that script to figure out when to do 
that)

and it should be configurable so that people can turn on/off stdout and/or 
stderr capturing either per-script and/or globally

David Lang