Arm + KASAN + syzbot

[Dmitry Vyukov]

On Tue, Jan 19, 2021 at 8:48 PM Russell King - ARM Linux admin
<linux at armlinux.org.uk> wrote:
>
> On Tue, Jan 19, 2021 at 07:57:16PM +0100, Dmitry Vyukov wrote:
> > Using "-kernel arch/arm/boot/zImage -dtb
> > arch/arm/boot/dts/vexpress-v2p-ca15-tc1.dtb" fully works.
>
> Good.
>
> > Using just "-kernel arch/arm/boot/zImage" does not work, not output
> > from qemu whatsoever (expected).
>
> Yep.
>
> > But using just "-kernel arch/arm/boot/zImage.dtb" gives an interesting
> > effect. Kernel starts booting, I see console output up to late init
> > stages, but then it can't find the root device.
> > So appended dtb works... but only in half. Is names of block devices
> > something that's controlled by dtb?
>
> My knowledge about this is limited to qemu being used for KVM.
>
> Firstly, there is are no block devices except for MTD, USB, or CF
> based block devices in the Versatile Express hardware. So, the DTB
> contains no block devices.
>
> In your first case above, it is likely that QEMU modifies the passed
> DTB to add PCIe devices to describe a virtio block device.
>
> In this case, because QEMU has no visibility of the appended DTB, it
> can't modify it, so the kernel only knows about devices found on the
> real hardware. Hence, any of the "special" virtio devices that QEMU
> use likely won't be found.
>
> I'm not sure how QEMU adds those (you're probably in a better position
> than I to boot using your first method, grab a copy of the DTB that
> the booted kernel used from /sys/firmware/fdt, and use dtc to turn it
> back into a dts and see what the changes are.
>
> I suspect you'll find that there's a new PCIe controller been added
> by QEMU, behind which will be a load of virtio devices for things like
> network and the "vda" block device.

Thanks, Russell. This makes perfect sense.

I think allowing qemu to modify dtb on the fly (rather than appending
it to the kernel) may be useful for testing purposes. In future we
will probably want to make qemu emulate as many devices as possible to
increase testing coverage. Passing dtb separately will allow qemu to
emulate all kinds of devices that are not originally on the board.

However, I hit the next problem.
If I build a kernel with KASAN, binaries built from Go sources don't
work. dhcpd/sshd/etc start fine, but any Go binaries just consume 100%
of CPU and do nothing. The process state is R and it manages to create
2 child threads and mmap ~800MB of virtual memory, which I suspect may
be the root cause (though, actual memory consumption is much smaller,
dozen of MB or so). The binary cannot be killed with kill -9. I tried
to give VM 2GB and 8GB, so it should have plenty of RAM. These
binaries run fine on non-KASAN kernel...