Arm + KASAN + syzbot

[Linus Walleij]

On Mon, Jan 18, 2021 at 5:31 PM Dmitry Vyukov <dvyukov at google.com> wrote:

> We are considering setting up an Arm 32-bit instance on syzbot for
> continuous testing using qemu emulation and I have several questions
> related to that.

That's interesting. I don't know much about syzbot but it reminds me
of syzcaller.

> 1. Is there interest in this on your end? What git tree/branch should
> be used for testing (contains latest development and is regularly
> updated with fixes)?

The most important would be Russell's branch I think, that is where
the core architecture changes end up. They also land in linux-next.

I think for the core developers this is the interesting tree,
the corporate users mostly use KASAN for fuzzing their
out-of-tree codebase and that is not of our concern. There can
be some specific platforms we want to test but they mostly
require real hardware because the interesting bugs tend to be
in drivers and driver subsystems that only gets exercised on
real hardware (not Qemu).

> 2. I see KASAN has just become supported for Arm, which is very
> useful, but I can't boot a kernel with KASAN enabled. I am using
> v5.11-rc4 and this config without KASAN boots fine:
> https://gist.githubusercontent.com/dvyukov/12de2905f9479ba2ebdcc603c2fec79b/raw/c8fd3f5e8328259fe760ce9a57f3e6c6f5a95c8f/gistfile1.txt
> using the following qemu command line:
> qemu-system-arm \
>   -machine vexpress-a15 -cpu max -smp 2 -m 2G \
>   -device virtio-blk-device,drive=hd0 \
>   -drive if=none,format=raw,id=hd0,file=image-arm -snapshot \
>   -kernel arch/arm/boot/zImage \
>   -dtb arch/arm/boot/dts/vexpress-v2p-ca15-tc1.dtb \
>   -nographic \
>   -netdev user,host=10.0.2.10,hostfwd=tcp::10022-:22,id=net0 -device
> virtio-net-device,netdev=net0 \
>   -append "root=/dev/vda earlycon earlyprintk=serial console=ttyAMA0
> oops=panic panic_on_warn=1 panic=86400 vmalloc=512M"
>
> However, when I enable KASAN and get this config:
> https://gist.githubusercontent.com/dvyukov/a7e3edd35cc39a1b69b11530c7d2e7ac/raw/7cbda88085d3ccd11227224a1c9964ccb8484d4e/gistfile1.txt
>
> kernel does not boot, qemu only prints the following output and then silence:
> pulseaudio: set_sink_input_volume() failed
> pulseaudio: Reason: Invalid argument
> pulseaudio: set_sink_input_mute() failed
> pulseaudio: Reason: Invalid argument
>
> What am I doing wrong?

I tried it with both KASAN_INLINE and KASAN_OUTLINE this
morning on Torvald's tree and it works fine for me.
I brought it up with this and it booted (takes ~30 seconds to come up
on an i7).

Here is my config:
https://dflund.se/~triad/krad/vexpress_config.txt

> 3. CONFIG_KCOV does not seem to fully work.
> It seems to work except for when the kernel crashes, and that's the
> most interesting scenario for us. When the kernel crashes for other
> reasons, crash handlers re-crashe in KCOV making all crashes
> unactionable and indistinguishable.
> Here are some samples (search for __sanitizer_cov_trace):
> https://gist.githubusercontent.com/dvyukov/c8a7ff1c00a5223c5143fd90073f5bc4/raw/c0f4ac7fd7faad7253843584fed8620ac6006338/gistfile1.txt
> Perhaps some additional Makefiles in arch/arm need KCOV_INSTRUMENT :=
> n to fix this.
> And LKDTM can be used for testing:
> https://www.kernel.org/doc/html/latest/fault-injection/provoke-crashes.html

I have never use CONFIG_KCOV really, it's yet another universe
that I haven't looked into.

Yours,
Linus Walleij