RISC-V Linux kernel not booting up with KASAN enabled

Thu Mar 2 21:44:13 PST 2023

On Thu, 2 Mar 2023 at 21:11, Alexandre Ghiti <alex at ghiti.fr> wrote:
>
> +cc Dmitry and kasan-dev, in case they know about this but I did not
> find anything related

Hard to say anything w/o commit/symbolized report.
If it's stack unwinder and it's supposed to be precise, then it may be
a bug in the unwinder where it reads a wrong location and is imprecise
(not the frame pointer).
If it's supposed to be imprecise, then it should use READ_ONCE_NOCHECK
to read random stack locations.

> On 3/2/23 19:01, Chathura Rajapaksha wrote:
> > Hi Alex/All,
> >
> > Kernel is booting now but I get the following KASAN failure in the
> > bootup itself.
> > I didn't see this bug was reported before anywhere.
> >
> > [    0.000000] Memory: 63436K/129024K available (20385K kernel code,
> > 7120K rwdata, 4096K rodata, 2138K init, 476K bss, 65588K reserved, 0K
> > cma-reserved)
> > [    0.000000] ==================================================================
> > [    0.000000] BUG: KASAN: stack-out-of-bounds in walk_stackframe+0x1b2/0x1e2
> > [    0.000000] Read of size 8 at addr ffffffff81e07c40 by task swapper/0
> > [    0.000000]
> > [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
> > 6.2.0-gae3419fbac84-dirty #7
> > [    0.000000] Hardware name: riscv-virtio,qemu (DT)
> > [    0.000000] Call Trace:
> > [    0.000000] [<ffffffff8000ab9e>] walk_stackframe+0x0/0x1e2
> > [    0.000000] [<ffffffff80108508>] init_param_lock+0x26/0x2a
> > [    0.000000] [<ffffffff8000ad4c>] walk_stackframe+0x1ae/0x1e2
> > [    0.000000] [<ffffffff813d86e0>] dump_stack_lvl+0x22/0x36
> > [    0.000000] [<ffffffff813bd17a>] print_report+0x198/0x4a8
> > [    0.000000] [<ffffffff80108508>] init_param_lock+0x26/0x2a
> > [    0.000000] [<ffffffff8000ad4c>] walk_stackframe+0x1ae/0x1e2
> > [    0.000000] [<ffffffff8023bd52>] kasan_report+0x9a/0xc8
> > [    0.000000] [<ffffffff8000ad4c>] walk_stackframe+0x1ae/0x1e2
> > [    0.000000] [<ffffffff8000ad4c>] walk_stackframe+0x1ae/0x1e2
> > [    0.000000] [<ffffffff80108748>] stack_trace_save+0x88/0xa6
> > [    0.000000] [<ffffffff801086bc>] filter_irq_stacks+0x8a/0x8e
> > [    0.000000] [<ffffffff800b65e2>] devkmsg_read+0x3f8/0x3fc
> > [    0.000000] [<ffffffff8023b2de>] kasan_save_stack+0x2c/0x56
> > [    0.000000] [<ffffffff80108744>] stack_trace_save+0x84/0xa6
> > [    0.000000] [<ffffffff8023b31a>] kasan_set_track+0x12/0x20
> > [    0.000000] [<ffffffff8023b8f6>] __kasan_slab_alloc+0x58/0x5e
> > [    0.000000] [<ffffffff8023aeae>] __kmem_cache_create+0x21e/0x39a
> > [    0.000000] [<ffffffff8141623e>] create_boot_cache+0x70/0x9c
> > [    0.000000] [<ffffffff8141b5f6>] kmem_cache_init+0x6c/0x11e
> > [    0.000000] [<ffffffff8140125a>] mm_init+0xd8/0xfe
> > [    0.000000] [<ffffffff8140145c>] start_kernel+0x190/0x3ca
> > [    0.000000]
> > [    0.000000] The buggy address belongs to stack of task swapper/0
> > [    0.000000]  and is located at offset 0 in frame:
> > [    0.000000]  stack_trace_save+0x0/0xa6
> > [    0.000000]
> > [    0.000000] This frame has 1 object:
> > [    0.000000]  [32, 56) 'c'
> > [    0.000000]
> > [    0.000000] The buggy address belongs to the physical page:
> > [    0.000000] page:(____ptrval____) refcount:1 mapcount:0
> > mapping:0000000000000000 index:0x0 pfn:0x82007
> > [    0.000000] flags: 0x1000(reserved|zone=0)
> > [    0.000000] raw: 0000000000001000 ff60000007ca5090 ff60000007ca5090
> > 0000000000000000
> > [    0.000000] raw: 0000000000000000 0000000000000000 00000001ffffffff
> > [    0.000000] page dumped because: kasan: bad access detected
> > [    0.000000]
> > [    0.000000] Memory state around the buggy address:
> > [    0.000000]  ffffffff81e07b00: 00 00 00 00 00 00 00 00 00 00 00 00
> > 00 00 00 00
> > [    0.000000]  ffffffff81e07b80: 00 00 00 00 00 00 00 00 00 00 00 00
> > 00 00 00 00
> > [    0.000000] >ffffffff81e07c00: 00 00 00 00 00 00 00 00 f1 f1 f1 f1
> > 00 00 00 f3
> > [    0.000000]                                            ^
> > [    0.000000]  ffffffff81e07c80: f3 f3 f3 f3 00 00 00 00 00 00 00 00
> > 00 00 00 00
> > [    0.000000]  ffffffff81e07d00: 00 00 00 00 00 00 00 00 00 00 00 00
> > 00 00 00 00
> > [    0.000000] ==================================================================
>
>
> I was able to reproduce the exact same trace, I'll debug that tomorrow,
> I hope it is a real bug :)
>
> Thanks for the report Chatura,
>
> Alex
>
>
> >
> > Best,
> > Chath
> >
> > On Thu, Mar 2, 2023 at 11:25 AM Chathura Rajapaksha
> > <chathura.abeyrathne.lk at gmail.com> wrote:
> >> Hi Alex,
> >>
> >> Thank you very much, kernel booted up with the patches you mentioned.
> >> Bootup was pretty slow compared to before though (on a dev board).
> >> I guess that is kind of expected with KASAN enabled.
> >> Thanks again.
> >>
> >> Regards,
> >> Chath
> >>
> >> On Thu, Mar 2, 2023 at 2:50 AM Alexandre Ghiti <alex at ghiti.fr> wrote:
> >>> Hi Chatura,
> >>>
> >>> On 3/2/23 04:13, Chathura Rajapaksha wrote:
> >>>> Hi All,
> >>>>
> >>>> I observed that RISC-V Linux hangs when I enable KASAN.
> >>>> Without KASAN it works fine with QEMU.
> >>>> I am using the commit ae3419fbac845b4d3f3a9fae4cc80c68d82cdf6e
> >>>>
> >>>> When KASAN is enabled, QEMU hangs after OpenSBI prints.
> >>>>
> >>>> I noticed a similar issue was reported before in
> >>>> https://lore.kernel.org/lkml/CACT4Y+ZmuOpyf_0vHTT4t3wkmJuW8Ezvcg7v6yDVd8YOViS=GA@mail.gmail.com/t/
> >>>> But I believe I have the patch mentioned in that thread.
> >>>
> >>> I proposed a series that will be included in 6.3 regarding KASAN issues
> >>> here: https://patchwork.kernel.org/project/linux-riscv/list/?series=718458
> >>>
> >>> Can you give it a try and tell me if it works better?
> >>>
> >>> Thanks,
> >>>
> >>> Alex
> >>>
> >>>
> >>>> My kernel config:
> >>>> https://drive.google.com/file/d/1j9nU7f9MxCc_i-UHUCTvo7o6nDrcUz0w/view?usp=sharing
> >>>>
> >>>> Best regards,
> >>>> Chath
> >>>>
> >>>> _______________________________________________
> >>>> linux-riscv mailing list
> >>>> linux-riscv at lists.infradead.org
> >>>> http://lists.infradead.org/mailman/listinfo/linux-riscv
> >
> >