[PATCH v0] mkfs.ubifs: Add support for lz4hc compressor in mkfs.ubifs

[Elie De Brauwer]

On Sat, Oct 5, 2013 at 4:17 PM, Konstantin Tokarev <annulen at yandex.ru> wrote:
>
>
> 05.10.2013, 17:52, "Konstantin Tokarev" <annulen at yandex.ru>:
>> 05.10.2013, 17:25, "Konstantin Tokarev" <annulen at yandex.ru>:
>>
>>>  05.10.2013, 16:56, "Elie De Brauwer" <eliedebrauwer at gmail.com>:
>>>>   That's odd, yesterday I started by cloning mtd-utils.git and when I
>>>>   try to repeat my own steps and apply my patch it does seem to work.
>>>  I've figured it out: you've included patch into the mail instead of attaching
>>>  it as a file, so I copied it, and tab characters got lost. patch -p1 -l
>>>  works successfully.
>>
>> When trying to compress my rootfs, I got message "32866 compression errors occurred".
>> Resulting image is much larger than LZO one (133M for lz4hc, 86M for lzo, 79M for zlib)
>
> The next patch over your patch fixes the issue:
>
> diff --git a/mkfs.ubifs/compr.c b/mkfs.ubifs/compr.c
> index 2030042..65c7dbe 100644
> --- a/mkfs.ubifs/compr.c
> +++ b/mkfs.ubifs/compr.c
> @@ -109,9 +109,10 @@ static int lz4hc_compress(void *in_buf, size_t in_len, void *out_buf,
>             size_t *out_len)
>  {
>     int ret;
> -   ret = LZ4_compressHC_limitedOutput(in_buf, out_buf, in_len, *out_len);
> +   ret = LZ4_compressHC(in_buf, out_buf, in_len);
> +   *out_len = ret;
>
> -   if (ret != 0) {
> +   if (ret == 0) {
>         errcnt += 1;
>         return -1;
>     }
>
>
> I've replaced LZ4_compressHC_limitedOutput with LZ4_compressHC to match lzo_compress logic which changes out_len instead of limiting it. No errors are reported now. However, result is still larger than LZO (93M).
>
>

Oops, guessed our e-mails crossed each-other, the reason why I
initially choose limitedOutput is because compressHC will assume the
output buffer is large enough (which means it could potentially
outgrow the input buffer in size, it seemed safer to use the
limitedOutput option now rather than to introduce a potential buffer
overflow).

But apparently our findings are the same, lz4hc also produces a
slightly larger resulting binary, but it is the fastet one (on my x86
laptop):

edb at lapelidb:/tmp/l/mtd-utils$ time sudo mkfs.ubifs/mkfs.ubifs  -x
none -m 2048 -e 126976 -c 744 -d /tmp/rootfs/ /tmp/imag.zlib

real 0m0.452s
user 0m0.108s
sys 0m0.044s
edb at lapelidb:/tmp/l/mtd-utils$ time sudo mkfs.ubifs/mkfs.ubifs  -x
lz4hc -m 2048 -e 126976 -c 744 -d /tmp/rootfs/ /tmp/imag.lz4hc

real 0m0.805s
user 0m0.768s
sys 0m0.032s
edb at lapelidb:/tmp/l/mtd-utils$ time sudo mkfs.ubifs/mkfs.ubifs  -x
zlib -m 2048 -e 126976 -c 744 -d /tmp/rootfs/ /tmp/imag.zlib

real 0m1.834s
user 0m1.792s
sys 0m0.036s
edb at lapelidb:/tmp/l/mtd-utils$ time sudo mkfs.ubifs/mkfs.ubifs  -x lzo
-m 2048 -e 126976 -c 744 -d /tmp/rootfs/ /tmp/imag.lzo

real 0m3.341s
user 0m3.296s
sys 0m0.044s

But it's at the decompression side that the real benefit probably lies
(as in flash performance vs decompression speed).

my 2 cents
E.





-- 
Elie De Brauwer