[PATCH] arm64: mm: Fix lockless walks with static and dynamic page-table folding

[Ard Biesheuvel]

On Thu, 25 Jul 2024 at 11:03, Will Deacon <will at kernel.org> wrote:
>
> Lina reports random oopsen originating from the fast GUP code when
> 16K pages are used with 4-level page-tables, the fourth level being
> folded at runtime due to lack of LPA2.
>
> In this configuration, the generic implementation of
> p4d_offset_lockless() will return a 'p4d_t *' corresponding to the
> 'pgd_t' allocated on the stack of the caller, gup_fast_pgd_range().
> This is normally fine, but when the fourth level of page-table is folded
> at runtime, pud_offset_lockless() will offset from the address of the
> 'p4d_t' to calculate the address of the PUD in the same page-table page.
> This results in a stray stack read when the 'p4d_t' has been allocated
> on the stack and can send the walker into the weeds.
>
> Fix the problem by providing our own definition of p4d_offset_lockless()
> when CONFIG_PGTABLE_LEVELS <= 4 which returns the real page-table
> pointer rather than the address of the local stack variable.
>
> Cc: Catalin Marinas <catalin.marinas at arm.com>
> Cc: Ard Biesheuvel <ardb at kernel.org>
> Cc: <stable at vger.kernel.org>
> Link: https://lore.kernel.org/r/50360968-13fb-4e6f-8f52-1725b3177215@asahilina.net
> Fixes: 0dd4f60a2c76 ("arm64: mm: Add support for folding PUDs at runtime")
> Reported-by: Asahi Lina <lina at asahilina.net>
> Signed-off-by: Will Deacon <will at kernel.org>
> ---
>  arch/arm64/include/asm/pgtable.h | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>

Reviewed-by: Ard Biesheuvel <ardb at kernel.org>