[RESEND] [PATCH v2] arm64/mm: Elide TLB flush in certain pte protection transitions

[Barry Song]

On Sun, Oct 19, 2025 at 12:36 PM Dev Jain <dev.jain at arm.com> wrote:
>
> Currently arm64 does an unconditional TLB flush in mprotect(). This is not
> required for some cases, for example, when changing from PROT_NONE to
> PROT_READ | PROT_WRITE (a real usecase - glibc malloc does this to emulate
> growing into the non-main heaps), and unsetting uffd-wp in a range.

I recall seeing this pattern frequently in multi-threaded programs. The stacks
for the threads can be observed by running strace on a multi-threaded app:

mmap(NULL, 20480, PROT_NONE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fbfcd14c000
mprotect(0x7fbfcd14d000, 16384, PROT_READ|PROT_WRITE) = 0

The stack guard page in the mmap region remains non-READ/WRITE, while the
rest of the area is set to RW after initially being PROT_NONE.

>
> Therefore, implement pte_needs_flush() for arm64, which is already
> implemented by some other arches as well.
>
> Running a userspace program changing permissions back and forth between
> PROT_NONE and PROT_READ | PROT_WRITE, and measuring the average time taken
> for the none->rw transition, I get a reduction from 3.2 microseconds to
> 2.85 microseconds, giving a 12.3% improvement.
>
> Reviewed-by: Kefeng Wang <wangkefeng.wang at huawei.com>
> Signed-off-by: Dev Jain <dev.jain at arm.com>

The patch seems to make a lot of sense to me, but I’m not an ARM architecture
expert and cannot judge whether __pte_flags_need_flush() is correct. I’ll leave
that to Will and Catalin.

> ---
> mm-selftests pass. Based on 6.18-rc1.
>
> v1->v2:
>  - Drop PTE_PRESENT_INVALID and PTE_AF checks, use ptdesc_t instead of
>    pteval_t, return !!diff (Ryan)
>
>  arch/arm64/include/asm/tlbflush.h | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
> index 18a5dc0c9a54..40df783ba09a 100644
> --- a/arch/arm64/include/asm/tlbflush.h
> +++ b/arch/arm64/include/asm/tlbflush.h
> @@ -524,6 +524,33 @@ static inline void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *b
>  {
>         __flush_tlb_range_nosync(mm, start, end, PAGE_SIZE, true, 3);
>  }
> +
> +static inline bool __pte_flags_need_flush(ptdesc_t oldval, ptdesc_t newval)
> +{
> +       ptdesc_t diff = oldval ^ newval;
> +
> +       /* invalid to valid transition requires no flush */
> +       if (!(oldval & PTE_VALID))
> +               return false;
> +
> +       /* Transition in the SW bits requires no flush */
> +       diff &= ~PTE_SWBITS_MASK;
> +
> +       return !!diff;
> +}

Thanks
Barry