[INTERNAL REVIEW] [RFC PATCH v3 01/20] mm: change behavior of pXdp_get()/pXd_page() in compile-time folded pgtable
Yeoreum Yun
yeoreum.yun at arm.com
Thu Jul 23 01:41:33 PDT 2026
Sorry for making a noise. I've posted wrongly.
Please, ignore this. Thanks!
> Using ptep_get() and its counterparts in common code is suboptimal on
> kernel configurations with generic compile-time folded page tables.
> By default, ptep_get() and its friends expands to READ_ONCE(),
> forcing the compiler to emit a load even when the value is not used afterwards.
>
> This issue was recently reported by Christophe Leroy [1] for ppc32
> preventing futher code conversion to ptep_get()/pmdp_get()/... helper
> and the same behavior can also be observed on arm64 when built with
> 2- or 3-level page tables
>
> e.g) perf_get_page_size() in arm64 with CONFIG_PGTABLE_LEVEL=3:
>
> 00000000000052a0 <perf_get_page_size>:
> ...
> 52dc: d53b4234 mrs x20, DAIF
> 52e0: d50343df msr DAIFSet, #0x3
> ...
> 52fc: d35e9a69 ubfx x9, x19, #30, #9 /* pud_offset_lockless() */
> 5300: f9403508 ldr x8, [x8, #0x68]
> 5304: f869790a ldr x10, [x8, x9, lsl #3] /* pudp_get() */
> 5308: f90007ea str x10, [sp, #0x8]
> 530c: f8697908 ldr x8, [x8, x9, lsl #3] /* pudp_get() */
> ...
> 5360: 90000009 adrp x9, 0x5000 <perf_prepare_sample+0x548>
> 5364: 92746908 and x8, x8, #0x7ffffff000
> 5368: d3557675 ubfx x21, x19, #21, #9 /* pmd_offset_lockless() */
> ...
> 5394: f8757ac8 ldr x8, [x22, x21, lsl #3] /* pmdp_get() */
>
> Though PGTABLE_LEVEL=3, since the pudp_get() still remain with
> READ_ONCE(), there's redundant load for the pud which is folded.
>
> To prevent generating suboptimal code, make pXdp_get() return a dummy
> entry for compile-time folded page tables and prohibit calls to
> pXd_page() in pgtable-nopXd.h.
>
> Also make the helpers in pgtable-nopXd.h, such as pXd_offset_lockless(),
> validate dummy entries at compile time to catch the wrong usage.
>
> As the pXdp_get() can return *dummy* entry, some of code using
> the stack value where saves the pXdp_get() could be a problematic:
>
> 1. Passing address of stack value where saves the pXdp_get() result
> to pXd_offset() for example:
>
> pud_t *pudp, pud;
> pmd_t *pmdp;
>
> pud = pudp_get(pudp, address);
> pmdp = pmd_offset(&pud, pud, address);
>
> (e.g. host_pfn_mapping_level() in loongarch).
>
> 2. Using the pXdp_get() result to use as argument of pXd_val() and
> to check prot without checking pgtable is folded.
> for example, x86's effective_prot().
>
> 3. Using set_pXd() with pXdp_get() will set problematic dummy entry
> in folded page table like:
>
> set_pXd(pxdp, pXdp_get(pxdp_k));
>
> 4. Using pgd_page_vaddr() to get the first-level pgtable.
> passing dummy pxdp_get() for pgd_page_vaddr() will return wrong
> address. Therefore, make pgd_page_vaddr() and pXd_pgtable() to
> trigger the error for improper usage with folded dummy entry in the
> generic compile-time folded pgtable.
>
> Thanksfully, above cases are rare since (1) most of usage using
> pXd_offset() with result of upper pXd_offset(), (2) it's extreamely
> rare to use pXd_val() for non-leaf entry in the kernel,
> (3) is to handle the vmalloc_fault or set the first level of page table
> and (4) to setup early page table and etc.
>
> Therefore, convert this kind of problematic rare pattern properly.
>
> Furthermore, passing the ptep_get() (or its counterparts) as argument
> directly to pte_present() and related helpers can generate suboptimal code,
> in arm64 as the current macro implementation may evaluate its argument
> more than once like:
>
> !pte_val(READ_ONCE(*pte) || pte_present_invalid(READ_ONCE(*pte))
>
> resulting in redundant loads.
>
> A typical example is pud_free_pmd_page(), where the expansion of
> pmd_present() generates:
> ...
> /* pmd_present() (x20 = pmdp) */
> 1b88: f9400288 ldr x8, [x20] // read pmdp.
> 1b8c: f9000fa8 str x8, [x29, #0x18]
> 1b90: 3707fec8 tbnz w8, #0x0, 0x1b68 <pud_free_pmd_page+0xd0>
> 1b94: f9400288 ldr x8, [x20] // redundant read of pmdp.
> 1b98: 8a170109 and x9, x8, x23
> 1b9c: f9000fa8 str x8, [x29, #0x18]
> 1ba0: f120013f cmp x9, #0x800
> 1ba4: 54fffe20 b.eq 0x1b68 <pud_free_pmd_page+0xd0>
> 1ba8: 17fffff4 b 0x1b78 <pud_free_pmd_page+0xe0>
> ...
>
> To address this in arm64, convert pte_present() macro and its friends
> to static inline function.
>
> This patch is based on mm-unstable.
>
> Future work
> ===========
> - print_bad_page_map() and show_pte() still prints dummy values
> instead of printing the same content for all generic compile-time
> folded page tables. We might want to skip printing dummy values later.
>
> - We currently catch abuse of dummy values on the stack at compile-time by
> relying on constant propagation by the compiler. Usama's work [3] on using
> distinct types for sw vs. hw PTEs could help here as well."
>
> Patch History
> =============
> from v2 to v3:
> - repasre commit message
> - move ptdump_pgtable_first_level() into pgalloc.h
> - skip the huge pud operation when CONFIG_X86_DIRECT_GBPAGES is
> disabled
> - https://lore.kernel.org/all/20260722-dummy_ptxp3-v2-0-d9e4bad31e0a@arm.com/
>
> from v1 to v2:
> - Restore slient fallback to next pXd in set_pXd() and pXd_pgtable()
> and add check whether they're called with dummy entry.
> - Add some comment for returning first entry of pgd in arm with
> 2 pgtable-level
> - https://lore.kernel.org/all/20260713135614.1618183-1-yeoreum.yun@arm.com/
>
> To: Russell King <linux at armlinux.org.uk>
> To: Huacai Chen <chenhuacai at kernel.org>
> To: WANG Xuerui <kernel at xen0n.name>
> To: Thomas Bogendoerfer <tsbogend at alpha.franken.de>
> To: Catalin Marinas <catalin.marinas at arm.com>
> To: Will Deacon <will at kernel.org>
> To: Arnd Bergmann <arnd at arndb.de>
> To: Andrew Morton <akpm at linux-foundation.org>
> To: Kairui Song <kasong at tencent.com>
> To: Qi Zheng <qi.zheng at linux.dev>
> To: Shakeel Butt <shakeel.butt at linux.dev>
> To: Barry Song <baohua at kernel.org>
> To: Axel Rasmussen <axelrasmussen at google.com>
> To: Yuanchu Xie <yuanchu at google.com>
> To: Wei Xu <weixugc at google.com>
> To: Johannes Weiner <hannes at cmpxchg.org>
> To: David Hildenbrand <david at kernel.org>
> To: Michal Hocko <mhocko at kernel.org>
> To: Lorenzo Stoakes <ljs at kernel.org>
> To: Tianrui Zhao <zhaotianrui at loongson.cn>
> To: Bibo Mao <maobibo at loongson.cn>
> To: Anup Patel <anup at brainfault.org>
> To: Atish Patra <atish.patra at linux.dev>
> To: Paul Walmsley <pjw at kernel.org>
> To: Palmer Dabbelt <palmer at dabbelt.com>
> To: Albert Ou <aou at eecs.berkeley.edu>
> To: Alexandre Ghiti <alex at ghiti.fr>
> To: Dave Hansen <dave.hansen at linux.intel.com>
> To: Andy Lutomirski <luto at kernel.org>
> To: Peter Zijlstra <peterz at infradead.org>
> To: Thomas Gleixner <tglx at kernel.org>
> To: Ingo Molnar <mingo at redhat.com>
> To: Borislav Petkov <bp at alien8.de>
> To: x86 at kernel.org
> To: H. Peter Anvin <hpa at zytor.com>
> To: Liam R. Howlett <liam at infradead.org>
> To: Vlastimil Babka <vbabka at kernel.org>
> To: Mike Rapoport <rppt at kernel.org>
> To: Suren Baghdasaryan <surenb at google.com>
> To: Michal Hocko <mhocko at suse.com>
> To: Jonas Bonn <jonas at southpole.se>
> To: Stefan Kristiansson <stefan.kristiansson at saunalahti.fi>
> To: Stafford Horne <shorne at gmail.com>
> Cc: linux-arm-kernel at lists.infradead.org
> Cc: linux-kernel at vger.kernel.org
> Cc: loongarch at lists.linux.dev
> Cc: linux-mips at vger.kernel.org
> Cc: linux-arch at vger.kernel.org
> Cc: linux-mm at kvack.org
> Cc: kvm at vger.kernel.org
> Cc: kvm-riscv at lists.infradead.org
> Cc: linux-riscv at lists.infradead.org
> Cc: linux-openrisc at vger.kernel.org
> Link: [1] https://lore.kernel.org/all/0019d675-ce3d-4a5c-89ed-f126c45145c9@kernel.org/
> Link: [2] https://lore.kernel.org/all/20251113014656.2605447-1-samuel.holland@sifive.com/
> Link: [3] https://lore.kernel.org/r/74182e50-b54f-4d2d-a27f-3a59a538d6bc@arm.com
>
> --- b4-submit-tracking ---
> # This section is used internally by b4 prep for tracking purposes.
> {
> "series": {
> "revision": 3,
> "change-id": "20260722-dummy_ptxp3-3741d78cc70f",
> "prefixes": [
> "RFC"
> ],
> "history": {
> "v2": [
> "20260722-dummy_ptxp3-v2-0-d9e4bad31e0a at arm.com"
> ]
> }
> }
> }
>
> Signed-off-by: Yeoreum Yun <yeoreum.yun at arm.com>
> --
> LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7}
>
--
Sincerely,
Yeoreum Yun
More information about the linux-riscv
mailing list