[PATCH 0/6 v14] KASan for Arm

Florian Fainelli f.fainelli at gmail.com
Sun Oct 4 16:24:53 EDT 2020



On 10/4/2020 2:09 AM, Ard Biesheuvel wrote:
> On Sun, 4 Oct 2020 at 10:41, Ard Biesheuvel <ardb at kernel.org> wrote:
>>
>> On Sun, 4 Oct 2020 at 10:06, Ard Biesheuvel <ardb at kernel.org> wrote:
>>>
>>> On Sat, 3 Oct 2020 at 17:50, Ard Biesheuvel <ardb at kernel.org> wrote:
>>>>
>>>> On Thu, 1 Oct 2020 at 21:19, Florian Fainelli <f.fainelli at gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 10/1/2020 8:22 AM, Linus Walleij wrote:
>>>>>> This is the 14th iteration of KASan for ARM/Aarch32.
>>>>>>
>>>>>> I have added one patch in the beginning of the series to
>>>>>> fix the issue when the DTB (often attached DTB) ends up
>>>>>> in lowmem. It also amends ARM to copy the device tree
>>>>>> instead of just unflattening it and using it from where
>>>>>> it is.
>>>>>>
>>>>>> This fixes my particular issue on the Qualcomm APQ8060
>>>>>> and I hope it may also solve Florian's issue and what
>>>>>> Ard has been seeing. If you inspect patch 1/6 you can
>>>>>> see what has been going on for me. My hypothesis about
>>>>>> what was going on was mostly right.
>>>>>>
>>>>>> You are encouraged to test this patch set to find memory out
>>>>>> of bounds bugs with ARM32 platforms and drivers.
>>>>>>
>>>>>> There is a git branch you can pull in:
>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-integrator.git/log/?h=kasan
>>>>>
>>>>> It does appear to be slight better, although all platforms that I have
>>>>> where memory starts at physical address 0 cannot boot, attached logs
>>>>> which are all more or less the same.
>>>>>
>>>>> The physical memory map looks like this:
>>>>>
>>>>> 0..3GB -> DRAM
>>>>> 3GB..4GB -> Registers, Boot ROM, Boot SRAM
>>>>> 4GB..12GB -> DRAM extension
>>>>>
>>>>> Do any of the platforms you use for testing have a similar memory map?
>>>>> Could you try to contrive a QEMU machine to have something similar in
>>>>> case that helps reproducing these failures?
>>>>>
>>>>
>>>> I am getting very similar failures on a Raspberry Pi4 booting in
>>>> 32-bit mode from U-boot+EFI
>>>>
>>>> Full log attached.
>>>>
>>>> I will try to dig a bit deeper.
>>>
>>> OK, one obvious issue with the code as-is is that the following routine
>>>
>>> static __init void *kasan_alloc_block(size_t size)
>>> {
>>>    return memblock_alloc_try_nid(size, size, __pa(MAX_DMA_ADDRESS),
>>>                                  MEMBLOCK_ALLOC_KASAN, NUMA_NO_NODE);
>>> }
>>>
>>> is called after the early shadow is unmapped, but before the permanent
>>> shadow is in place. memblock_alloc_try_nid() clears the newly
>>> allocated memory using memset(), which checks the associated shadow,
>>> which is unmapped -> BOOM.
>>>
>>> With the following implementation, I can avoid the crash similar to
>>> the one Florian is reporting:
>>>
>>> static __init void *kasan_alloc_block(size_t size)
>>> {
>>>    void *p = memblock_alloc_try_nid_raw(size, size,
>>>      __pa(MAX_DMA_ADDRESS), MEMBLOCK_ALLOC_KASAN,
>>>      NUMA_NO_NODE);
>>>
>>>    if (p)
>>>      __memset(p, 0, size);
>>>    return p;
>>> }
>>>
>>> However, I still get a hang a bit later, and I haven't tracked that down yet.
>>
>> The above issue appears to be related to TLB maintenance. So keeping
>> kasan_alloc_block() as is, and doing
>>
>> --- a/arch/arm/mm/kasan_init.c
>> +++ b/arch/arm/mm/kasan_init.c
>> @@ -223,6 +223,8 @@ void __init kasan_init(void)
>>                  __pgd(__pa(tmp_pmd_table) | PMD_TYPE_TABLE | L_PGD_SWAPPER));
>>   #endif
>>          cpu_switch_mm(tmp_pgd_table, &init_mm);
>> +       local_flush_tlb_all();
>> +
>>          clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END);
>>
>>          kasan_populate_early_shadow(kasan_mem_to_shadow((void *)VMALLOC_START),
>>
>> instead fixes the crash as well.
>>
>> Still have the hang right after though ..
> 
> OK, booting now - turns out the switch back to swapper_pg_dir needs a
> proper TLB flush as well.
> 
> Full patch below - with that applied, I can boot the RPi4 to the point
> where it wants to mount the rootfs (but I don't have a 32-bit rootfs
> at hand)
> 
> 
> The first change avoids reallocating KASAN blocks when a range gets
> mapped twice - this occurs when mapping the DTB space explicitly
> (although I am not sure that that is still needed now that you move
> the DTB)

We are almost there it seems. There are only 3 boards that do not boot 
(they are all similar in configuration, amounts of memory and placement 
of reserved regions), see the attached log. Every other platform now 
boots correctly so this is definitively a big improvement!
-- 
Florian
-------------- next part --------------
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 5.9.0-rc7-g6e04af38fd04 (fainelli at fainelli-desktop) (arm-linux-gcc (GCC) 8.3.0, GNU ld (GNU Binutils) 2.32) #12 SMP Sun Oct 4 13:06:11 PDT 2020
[    0.000000] CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), cr=30c5383d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[    0.000000] OF: fdt: Machine model: BCM972112SV
[    0.000000] earlycon: pl11 at MMIO 0x000000047e201000 (options '115200')
[    0.000000] printk: bootconsole [pl11] enabled
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] cma: Reserved 16 MiB at 0x000000007f000000
[    0.000000] ATAGs/DTB found in lowmem, skip clearing PMD @0xce5fe000
[    0.000000] ATAGs/DTB found in lowmem, skip clearing PMD @0xce7fe000
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000002fffffff]
[    0.000000]   Normal   empty
[    0.000000]   HighMem  [mem 0x0000000030000000-0x000000007fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x00000000063fdfff]
[    0.000000]   node   0: [mem 0x0000000006400000-0x000000000fffffff]
[    0.000000]   node   0: [mem 0x0000000010400000-0x000000007fffffff]
[    0.000000] Zeroed struct page in unavailable ranges: 2 pages
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000007fffffff]
[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x000000007fbfe000 reserved size = 0x00000000053ef6e0
[    0.000000]  memory.cnt  = 0x3
[    0.000000]  memory[0x0]	[0x0000000000000000-0x00000000063fdfff], 0x00000000063fe000 bytes flags: 0x0
[    0.000000]  memory[0x1]	[0x0000000006400000-0x000000000fffffff], 0x0000000009c00000 bytes flags: 0x0
[    0.000000]  memory[0x2]	[0x0000000010400000-0x000000007fffffff], 0x000000006fc00000 bytes flags: 0x0
[    0.000000]  reserved.cnt  = 0xb
[    0.000000]  reserved[0x0]	[0x0000000000000000-0x0000000000000fff], 0x0000000000001000 bytes flags: 0x0
[    0.000000]  reserved[0x1]	[0x0000000000003000-0x0000000000007fff], 0x0000000000005000 bytes flags: 0x0
[    0.000000]  reserved[0x2]	[0x0000000000200000-0x0000000002fdcf9f], 0x0000000002ddcfa0 bytes flags: 0x0
[    0.000000]  reserved[0x3]	[0x00000000061ff000-0x00000000061fffff], 0x0000000000001000 bytes flags: 0x0
[    0.000000]  reserved[0x4]	[0x0000000006400000-0x00000000065fffff], 0x0000000000200000 bytes flags: 0x0
[    0.000000]  reserved[0x5]	[0x000000000e415000-0x000000000e41960f], 0x0000000000004610 bytes flags: 0x0
[    0.000000]  reserved[0x6]	[0x000000002e980000-0x000000002fd7ffff], 0x0000000001400000 bytes flags: 0x0
[    0.000000]  reserved[0x7]	[0x000000002fff8000-0x000000002fffdfff], 0x0000000000006000 bytes flags: 0x0
[    0.000000]  reserved[0x8]	[0x000000002fffeec0-0x000000002fffefbf], 0x0000000000000100 bytes flags: 0x0
[    0.000000]  reserved[0x9]	[0x000000002fffefd0-0x000000002fffffff], 0x0000000000001030 bytes flags: 0x0
[    0.000000]  reserved[0xa]	[0x000000007f000000-0x000000007fffffff], 0x0000000001000000 bytes flags: 0x0
[    0.000000] kasan: Mapping kernel virtual memory block: c0000000-c63fe000 at shadow: b7000000-b7c7fc00
[    0.000000] Kernel panic - not syncing: kasan_pte_populate failed to alloc pte for address 0xe2806000
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.9.0-rc7-g6e04af38fd04 #12
[    0.000000] Hardware name: Broadcom STB (Flattened Device Tree)
[    0.000000] Backtrace: 
[    0.000000] [<c02120b4>] (dump_backtrace) from [<c02123d8>] (show_stack+0x20/0x24)
[    0.000000]  r9:ffffffff r8:00000080 r7:c298e900 r6:600000d3 r5:00000000 r4:c298e900
[    0.000000] [<c02123b8>] (show_stack) from [<c08869e0>] (dump_stack+0xbc/0xe0)
[    0.000000] [<c0886924>] (dump_stack) from [<c022fc6c>] (panic+0x19c/0x3e4)
[    0.000000]  r10:e2806000 r9:c2b790e0 r8:c166b5bc r7:c2803d60 r6:00000000 r5:c2b7df00
[    0.000000]  r4:c2b78e20 r3:00000001
[    0.000000] [<c022fad0>] (panic) from [<c180bae4>] (kasan_pgd_populate+0x1ac/0x26c)
[    0.000000]  r3:00000000 r2:e2806000 r1:c1212734 r0:c166b5bc
[    0.000000]  r7:b7c7fc00
[    0.000000] [<c180b938>] (kasan_pgd_populate) from [<c180bbfc>] (create_mapping+0x58/0x64)
[    0.000000]  r10:c166b690 r9:00000000 r8:063fe000 r7:c2ba0ac0 r6:c28a29e0 r5:b7000000
[    0.000000]  r4:b7c7fc00
[    0.000000] [<c180bba4>] (create_mapping) from [<c180bee0>] (kasan_init+0x270/0x3c0)
[    0.000000]  r5:00000000 r4:c0000000
[    0.000000] [<c180bc70>] (kasan_init) from [<c1805684>] (setup_arch+0x848/0x1048)
[    0.000000]  r10:c1861450 r9:c0226668 r8:e12fff1e r7:c0008000 r6:c1873a40 r5:c2fdcfa0
[    0.000000]  r4:c28a2780
[    0.000000] [<c1804e3c>] (setup_arch) from [<c1801010>] (start_kernel+0x88/0x3e4)
[    0.000000]  r10:c2806d40 r9:410fd083 r8:0e415000 r7:ffffffff r6:30c0387d r5:c2806d48
[    0.000000]  r4:00007000
[    0.000000] [<c1800f88>] (start_kernel) from [<00000000>] (0x0)
[    0.000000]  r10:30c5387d r9:410fd083 r8:0e415000 r7:ffffffff r6:30c0387d r5:00000000
[    0.000000]  r4:c1800334
[    0.000000] ---[ end Kernel panic - not syncing: kasan_pte_populate failed to alloc pte for address 0xe2806000 ]---


More information about the linux-arm-kernel mailing list