[LEDE-DEV] [PATCH ubox v2] Add template support to logread

[Henry Chang]

Hi Petr,

> From my point of view and experience, logd is good for simple use cases. For
anything else I would recommend you to use syslogd from Busybox for example.

The reason I chose to stay with logd is I wanna keep basing on ubus.

> syslogd from Busybox handles log rotation very well, including multiple old
files. AFAIK logd doesn't support more then one rotated file.

I don't need file rotation. If I need, I can simply use logrotate.

> What about log uploading over HTTPS, using curl and a few lines of shell
script?  I bet, that some of those cloud log services support such input
source, you can tag and format it as needed also.

This is definitely doable. And this is also what I'm doing currently.
I use stunnel for securing the connection. The thing is, string
parsing is more expensive than object parsing. So if we can do object
parsing, why not?

> It seems to me, that you're just trying to bend actual and simple tools for
your use case, instead of using right tools for the job.

I didn't change anything in logd, instead, I added a customizable
output for its reader logread. There's a regex filtering feature in
logread, I think the reason it's there is very similar to why I added
this template.

I can write a separate reader for logd to match my own use case. But
while I was thinking of that, I also came up an idea that I should
make it more general so that future people don't have to worry about
the same stuff. And I really don't think this is a special case.

Regards,

Henry

On Tue, May 2, 2017 at 3:11 AM, Petr Štetiar <ynezz at true.cz> wrote:
> Henry Chang <mr.changyuheng at gmail.com> [2017-04-28 14:14:57]:
>
>> 1. logd is built-in logd is a built-in feature of ubox, and ubox is a heart
>> of OpenWrt/LEDE. If ubox can achieve all the use cases, we should simply
>> just use it. Use rsyslog instead will create additional footprint in terms
>> of resource while a lot of features in rsyslog is not going to be used.
>
> From my point of view and experience, logd is good for simple use cases. For
> anything else I would recommend you to use syslogd from Busybox for example.
>
>> "Simple plain text files also require log rotation to prevent them
>> from becoming too large. In log rotation, existing log files are
>
> syslogd from Busybox handles log rotation very well, including multiple old
> files. AFAIK logd doesn't support more then one rotated file.
>
>> 3. To make a working and secure remote logging pipeline, what we need
>> is just logread, stunnel, and a log server. A log server will probably
>> need some sort of special format in order to do the authentication or
>> classification. So without template feature in logread, we'll need
>> another service in between logread and stunnel for just manipulating
>> every line of the log or we write another logread to achieve this.
>> While logread already has some sort of object of the log, doing this
>> in logread will be easier for both programmers and the computer.
>
> What about log uploading over HTTPS, using curl and a few lines of shell
> script?  I bet, that some of those cloud log services support such input
> source, you can tag and format it as needed also.
>
>> 4. The template feature in this patch is very efficient. It doesn't
>> create any overhead if the user doesn't enable this feature by passing
>> the corresponding argument. The string replacement is also efficient,
>> we can improve it by optimizing some strlen() function calls for sure
>> but basically it's efficient enough.
>
> It seems to me, that you're just trying to bend actual and simple tools for
> your use case, instead of using right tools for the job.
>
> -- ynezz