[PATCH v3 0/5] arch_topology: Build cacheinfo from primary CPU
Krzysztof Kozlowski
krzysztof.kozlowski at linaro.org
Thu Dec 29 08:47:44 PST 2022
On 09/12/2022 11:31, Pierre Gondois wrote:
> v2:
> - Applied renaming/formatting comments from v1.
> - Check CACHE_TYPE_VALID flag in pppt.c.
> v3:
> - Applied Sudeep's suggestions (for patch 5/5):
> - Renaming allocate_cache_info() -> fecth_cache_info()
> - Updated error message
> - Extract an inline allocate_cache_info() function
> - Re-run checkpatch with --strict option
>
> Note:
> This patchset requires the following patch to be applied first in
> order to avoid the same bug described in the commit message:
> https://lore.kernel.org/all/20221116094958.2141072-1-pierre.gondois@arm.com/
>
> [1] and [2] build the CPU topology from the cacheinfo information for
> both DT/ACPI based systems and remove (struct cpu_topology).llc_id
> which was used by ACPI only.
>
> Creating the cacheinfo for secondary CPUs is done during early boot.
> Preemption and interrupts are disabled at this stage. On PREEMPT_RT
> kernels, allocating memory (and parsing the PPTT table for ACPI based
> systems) triggers a:
> 'BUG: sleeping function called from invalid context' [4]
>
> To prevent this bug, allocate the cacheinfo from the primary CPU when
> preemption and interrupts are enabled and before booting secondary
> CPUs. The cache levels/leaves are computed from DT/ACPI PPTT information
> only, without relying on the arm64 CLIDR_EL1 register.
> If no cache information is found in the DT/ACPI PPTT, then fallback
> to the current state, triggering [4] on PREEMPT_RT kernels.
>
> Patches to update the arm64 device trees that have incomplete cacheinfo
> (mostly for missing the 'cache-level' or 'cache-unified' property)
> have been sent at [3].
>
> Tested platforms:
> - ACPI + PPTT: Ampere Altra, Ampere eMAG, Cavium ThunderX2,
> Kunpeng 920, Juno-r2
> - DT: rb5, db845c, Juno-r2
>
I gave the patchset a try with DTS fixes for cache topology on Qualcomm
RB5 board (SM8250 SoC) and with KASAN it produces:
BUG: KASAN: slab-out-of-bounds in populate_cache_leaves+0x84/0x15c
[ 0.633014] dump_backtrace.part.0+0xe0/0xf0
[ 0.633035] show_stack+0x18/0x40
[ 0.633050] dump_stack_lvl+0x8c/0xb8
[ 0.633085] print_report+0x188/0x488
[ 0.633106] kasan_report+0xac/0xf0
[ 0.633136] __asan_store4+0x80/0xa4
[ 0.633158] populate_cache_leaves+0x84/0x15c
[ 0.633181] detect_cache_attributes+0xc0/0x8c4
[ 0.633213] update_siblings_masks+0x28/0x43c
[ 0.633235] store_cpu_topology+0x98/0xc0
[ 0.633251] smp_prepare_cpus+0x2c/0x15c
[ 0.633281] kernel_init_freeable+0x22c/0x424
[ 0.633310] kernel_init+0x24/0x13c
[ 0.633328] ret_from_fork+0x10/0x20
[ 0.633388]
[ 0.708729] Allocated by task 1:
[ 0.712078] kasan_save_stack+0x2c/0x60
[ 0.716066] kasan_set_track+0x2c/0x40
[ 0.719959] kasan_save_alloc_info+0x24/0x3c
[ 0.724387] __kasan_kmalloc+0xa0/0xbc
[ 0.728278] __kmalloc+0x74/0x110
[ 0.731740] fetch_cache_info+0x170/0x210
[ 0.735902] init_cpu_topology+0x254/0x2bc
[ 0.740171] smp_prepare_cpus+0x20/0x15c
[ 0.744272] kernel_init_freeable+0x22c/0x424
[ 0.748791] kernel_init+0x24/0x13c
[ 0.752420] ret_from_fork+0x10/0x20
Best regards,
Krzysztof
-------------- next part --------------
[ 0.000000] arch_timer: cp15 and mmio timer(s) running at 19.20MHz (virt/virt).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[ 0.000001] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[ 0.005051] Console: colour dummy device 80x25
[ 0.478980] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
[ 0.478992] ... MAX_LOCKDEP_SUBCLASSES: 8
[ 0.479002] ... MAX_LOCK_DEPTH: 48
[ 0.479011] ... MAX_LOCKDEP_KEYS: 8192
[ 0.479019] ... CLASSHASH_SIZE: 4096
[ 0.479027] ... MAX_LOCKDEP_ENTRIES: 32768
[ 0.479035] ... MAX_LOCKDEP_CHAINS: 65536
[ 0.479043] ... CHAINHASH_SIZE: 32768
[ 0.479052] memory used by lock dependency info: 6365 kB
[ 0.479061] memory used for stack traces: 4224 kB
[ 0.479069] per task-struct memory footprint: 1920 bytes
[ 0.479976] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=19200)
[ 0.480007] pid_max: default: 32768 minimum: 301
[ 0.482256] LSM: Security Framework initializing
[ 0.484629] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[ 0.484692] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[ 0.518295] ==================================================================
[ 0.617001] BUG: KASAN: slab-out-of-bounds in populate_cache_leaves+0x84/0x15c
[ 0.624489] Write of size 4 at addr ffff0d83400366c8 by task swapper/0/1
[ 0.631400]
[ 0.632973] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.1.0-rt5-00372-ga6339d0b4e8e #45
[ 0.632995] Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT)
[ 0.633006] Call trace:
[ 0.633014] dump_backtrace.part.0+0xe0/0xf0
[ 0.633035] show_stack+0x18/0x40
[ 0.633050] dump_stack_lvl+0x8c/0xb8
[ 0.633085] print_report+0x188/0x488
[ 0.633106] kasan_report+0xac/0xf0
[ 0.633136] __asan_store4+0x80/0xa4
[ 0.633158] populate_cache_leaves+0x84/0x15c
[ 0.633181] detect_cache_attributes+0xc0/0x8c4
[ 0.633213] update_siblings_masks+0x28/0x43c
[ 0.633235] store_cpu_topology+0x98/0xc0
[ 0.633251] smp_prepare_cpus+0x2c/0x15c
[ 0.633281] kernel_init_freeable+0x22c/0x424
[ 0.633310] kernel_init+0x24/0x13c
[ 0.633328] ret_from_fork+0x10/0x20
[ 0.633388]
[ 0.708729] Allocated by task 1:
[ 0.712078] kasan_save_stack+0x2c/0x60
[ 0.716066] kasan_set_track+0x2c/0x40
[ 0.719959] kasan_save_alloc_info+0x24/0x3c
[ 0.724387] __kasan_kmalloc+0xa0/0xbc
[ 0.728278] __kmalloc+0x74/0x110
[ 0.731740] fetch_cache_info+0x170/0x210
[ 0.735902] init_cpu_topology+0x254/0x2bc
[ 0.740171] smp_prepare_cpus+0x20/0x15c
[ 0.744272] kernel_init_freeable+0x22c/0x424
[ 0.748791] kernel_init+0x24/0x13c
[ 0.752420] ret_from_fork+0x10/0x20
[ 0.756131]
[ 0.757726] The buggy address belongs to the object at ffff0d8340036600
[ 0.757726] which belongs to the cache kmalloc-256 of size 256
[ 0.770607] The buggy address is located 200 bytes inside of
[ 0.770607] 256-byte region [ffff0d8340036600, ffff0d8340036700)
[ 0.782690]
[ 0.784256] The buggy address belongs to the physical page:
[ 0.790008] page:(____ptrval____) refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x100034
[ 0.799686] head:(____ptrval____) order:2 compound_mapcount:0 compound_pincount:0
[ 0.807405] flags: 0x800000000010200(slab|head|node=0|zone=2)
[ 0.813365] raw: 0800000000010200 0000000000000000 dead000000000122 ffff0d8340002480
[ 0.821349] raw: 0000000000000000 0000000080200020 00000001ffffffff 0000000000000000
[ 0.829364] page dumped because: kasan: bad access detected
[ 0.835117]
[ 0.836679] Memory state around the buggy address:
[ 0.841639] ffff0d8340036580: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 0.849085] ffff0d8340036600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[ 0.856562] >ffff0d8340036680: 00 00 00 00 00 00 00 00 fc fc fc fc fc fc fc fc
[ 0.864005] ^
[ 0.869760] ffff0d8340036700: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 0.877206] ffff0d8340036780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 0.884650] ==================================================================
[ 0.892100] Disabling lock debugging due to kernel taint
[ 0.904200] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.904241] cblist_init_generic: Setting shift to 3 and lim to 1.
[ 0.918718] cblist_init_generic: Setting shift to 3 and lim to 1.
[ 0.925721] Running RCU-tasks wait API self tests
[ 1.036108] rcu: Hierarchical SRCU implementation.
[ 1.036118] rcu: Max phase no-delay instances is 400.
[ 1.036756] printk: bootconsole [qcom_geni0] printing thread started
[ 1.048808] Callback from call_rcu_tasks_trace() invoked.
[ 1.081711] EFI services will not be available.
[ 1.091529] smp: Bringing up secondary CPUs ...
[ 1.103585] Detected VIPT I-cache on CPU1
[ 1.103747] GICv3: CPU1: found redistributor 100 region 0:0x0000000017a80000
[ 1.103830] CPU1: Booted secondary processor 0x0000000100 [0x51df805e]
[ 1.129767] Detected VIPT I-cache on CPU2
[ 1.129911] GICv3: CPU2: found redistributor 200 region 0:0x0000000017aa0000
[ 1.129979] CPU2: Booted secondary processor 0x0000000200 [0x51df805e]
[ 1.155742] Detected VIPT I-cache on CPU3
[ 1.155876] GICv3: CPU3: found redistributor 300 region 0:0x0000000017ac0000
[ 1.155936] CPU3: Booted secondary processor 0x0000000300 [0x51df805e]
[ 1.182791] CPU features: detected: Spectre-v4
[ 1.182824] CPU features: detected: Spectre-BHB
[ 1.182854] CPU features: detected: ARM erratum 1508412 (kernel portion)
[ 1.182900] Detected PIPT I-cache on CPU4
[ 1.183187] GICv3: CPU4: found redistributor 400 region 0:0x0000000017ae0000
[ 1.183294] CPU4: Booted secondary processor 0x0000000400 [0x411fd0d0]
[ 1.225640] Detected PIPT I-cache on CPU5
[ 1.226000] GICv3: CPU5: found redistributor 500 region 0:0x0000000017b00000
[ 1.226099] CPU5: Booted secondary processor 0x0000000500 [0x411fd0d0]
[ 1.252358] Detected PIPT I-cache on CPU6
[ 1.252722] GICv3: CPU6: found redistributor 600 region 0:0x0000000017b20000
[ 1.252821] CPU6: Booted secondary processor 0x0000000600 [0x411fd0d0]
[ 1.266024] Callback from call_rcu_tasks() invoked.
[ 1.284303] Detected PIPT I-cache on CPU7
[ 1.284475] GICv3: CPU7: found redistributor 700 region 0:0x0000000017b40000
[ 1.284525] CPU7: Booted secondary processor 0x0000000700 [0x411fd0d0]
[ 1.284926] smp: Brought up 1 node, 8 CPUs
[ 1.284943] SMP: Total of 8 processors activated.
[ 1.284954] CPU features: detected: 32-bit EL0 Support
[ 1.284963] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[ 1.284975] CPU features: detected: Common not Private translations
[ 1.284985] CPU features: detected: CRC32 instructions
[ 1.284999] CPU features: detected: RCpc load-acquire (LDAPR)
[ 1.285009] CPU features: detected: LSE atomic instructions
[ 1.285019] CPU features: detected: Privileged Access Never
[ 1.285029] CPU features: detected: RAS Extension Support
[ 1.285044] CPU features: detected: Speculative Store Bypassing Safe (SSBS)
[ 1.296826] CPU: All CPU(s) started at EL1
[ 1.296891] alternatives: applying system-wide alternatives
[ 1.315641] devtmpfs: initialized
[ 1.670123] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 1.670327] futex hash table entries: 2048 (order: 6, 393216 bytes, linear)
[ 1.673537] pinctrl core: initialized pinctrl subsystem
[ 1.684021] DMI not present or invalid.
[ 1.687165] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 1.695595] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[ 1.696373] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[ 1.698379] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[ 1.698742] audit: initializing netlink subsys (disabled)
[ 1.699628] audit: type=2000 audit(1.628:1): state=initialized audit_enabled=0 res=1
[ 1.708543] thermal_sys: Registered thermal governor 'step_wise'
[ 1.708570] thermal_sys: Registered thermal governor 'power_allocator'
[ 1.708961] cpuidle: using governor ladder
[ 1.709035] cpuidle: using governor menu
[ 1.709849] NET: Registered PF_QIPCRTR protocol family
[ 1.711345] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 1.719297] ASID allocator initialised with 32768 entries
[ 1.733197] Serial: AMBA PL011 UART driver
[ 1.855339] platform 1d87000.phy: Fixing up cyclic dependency with 1d84000.ufshc
[ 2.041279] KASLR enabled
[ 2.174612] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 2.174626] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 2.174634] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
[ 2.174640] HugeTLB: 508 KiB vmemmap can be freed for a 32.0 MiB page
[ 2.174647] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 2.174652] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 2.174660] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
[ 2.174665] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
[ 2.188583] ACPI: Interpreter disabled.
[ 2.306645] iommu: Default domain type: Translated
[ 2.306660] iommu: DMA domain TLB invalidation policy: strict mode
[ 2.310343] SCSI subsystem initialized
[ 2.313611] usbcore: registered new interface driver usbfs
[ 2.313987] usbcore: registered new interface driver hub
[ 2.314314] usbcore: registered new device driver usb
[ 2.321138] pps_core: LinuxPPS API ver. 1 registered
[ 2.321144] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti at linux.it>
[ 2.321246] PTP clock support registered
[ 2.321452] EDAC MC: Ver: 3.0.0
[ 2.324853] CPUidle PSCI: Initialized CPU PM domain topology
[ 2.327564] qcom_scm: convention: smc arm 64
[ 2.335862] FPGA manager framework
More information about the linux-riscv
mailing list