Kernel-4.14: With ubuntu-18.04 building rootfs images and booting gives SQUASHFS error: xz decompression failed, data probably corrupt

[Pintu Agarwal]

+ Adding squashfs-devel to get opinion from squashfs side.

On Fri, 12 Nov 2021 at 12:48, Pintu Agarwal <pintu.ping at gmail.com> wrote:
>
> Hi,
>
> On Tue, 9 Nov 2021 at 21:04, Pintu Agarwal <pintu.ping at gmail.com> wrote:
>
> > > > We only get these squashfs errors flooded in the boot logs:
> > > > {{{
> > > > ....
> > > > [    5.153479] device-mapper: init: dm-0 is ready
> > > > [    5.334282] VFS: Mounted root (squashfs filesystem) readonly on device 253:0.
> > > > ....
> > > > [    8.954120] SQUASHFS error: xz decompression failed, data probably corrupt
> > > > [    8.954153] SQUASHFS error: squashfs_read_data failed to read block 0x1106
> > > > [    8.970316] SQUASHFS error: Unable to read data cache entry [1106]
> > > > [    8.970349] SQUASHFS error: Unable to read page, block 1106, size 776c
> > > > [    8.980298] SQUASHFS error: Unable to read data cache entry [1106]
> > > > [    8.981911] SQUASHFS error: Unable to read page, block 1106, size 776c
> > > > [    8.988280] SQUASHFS error: Unable to read data cache entry [1106]
> > > > ....
> > > > }}}
> > > >
>
> One more observation:
> When I disable FEC flag in bootloader, I see the below error:
> [    8.360791] device-mapper: verity: 253:0: data block 2 is corrupted
> [    8.361134] device-mapper: verity: 253:0: data block 3 is corrupted
> [    8.366016] SQUASHFS error: squashfs_read_data failed to read block 0x1106
> [    8.379652] SQUASHFS error: Unable to read data cache entry [1106]
> [    8.379680] SQUASHFS error: Unable to read page, block 1106, size 7770
>
> Also, now I see that the decompress error is gone, but the read error
> is still there.
>
> This seems to me that dm-verity detects some corrupted blocks but with
> FEC it auto corrects itself, how when dm-verity auto corrects itself,
> the squashfs decompression algorithm somehow could not understand it.
>
> So, it seems like there is some mis-match between the way FEC
> correction and the squashfs decompression happens ?
>
> Is this issue seen by anybody else here ?
>

The squashfs version used by Kernel:
[    0.355958] squashfs: version 4.0 (2009/01/31) Phillip Lougher

The squashfs version available on Ubuntu:
mksquashfs version 4.3-git (2014/06/09)

The squashfs version used by Yocto 2.6:
squashfs-tools/0001-squashfs-tools-Allow-setting-selinux-xattrs-through-.patch:61:
   printf("mksquashfs version 4.3-git (2014/09/12)\n");

We create dm-verity squashfs image using version 4.3 whereas, the
kernel uses 4.0 version to decompress it.
Is there something missing here?

When FEC (Forward Error Correction) comes into picture, then squashfs
decompress fails.
When we remove FEC flag from dm-verity then decompress works but read
error still occurs.
This seems as if something is missing either in FEC handling or either
in squashfs decompress logic.

Just wanted to know if there are any fixes already available in the
mainline for this ?


Thanks,
Pintu