[PATCH v2 13/20] arm64: percpu: Add infrastructure for preemptible this_cpu_*() ops

[Mark Rutland]

On Wed, Aug 05, 2026 at 08:47:08AM +0200, David Hildenbrand (Arm) wrote:
> On 8/5/26 08:45, David Hildenbrand (Arm) wrote:
> >> Atop v7.2-rc4, with GCC 15.2.0 and defconfig, this is compiled as:
> >>
> >> | <outline_this_cpu_add_u64>:
> >> |        paciasp
> >> |        stp     x29, x30, [sp, #-16]!
> >> |        mrs     x2, sp_el0
> >> |        mov     x29, sp
> >> |        ldr     w3, [x2, #8]
> >> |        add     w3, w3, #0x1
> >> |        str     w3, [x2, #8]
> >> |        mrs     x3, tpidr_el1
> >> |        add     x0, x0, x3
> >> | 1:     ldxr    x5, [x0]
> >> |        add     x5, x5, x1
> >> |        stxr    w4, x5, [x0]
> >> |        cbnz    w4, 1b
> >> |        ldr     x0, [x2, #8]
> >> |        sub     x0, x0, #0x1
> >> |        str     w0, [x2, #8]
> >> |        cbz     x0, 2f
> >> |        ldr     x0, [x2, #8]
> >> |        cbnz    x0, 3f
> >> | 2:     bl      preempt_schedule_notrace
> >> | 3:     ldp     x29, x30, [sp], #16
> >> |        autiasp
> >> |        ret
> > 
> > FWIW, in a recent discussion on some prototype hacking [1] we saw some overhead
> > in micro-benchmarks that would really hammer on a path that would now do a
> > preempt_disable()+preempt_enable().
> > 
> > Switching from preempt_disable() to preempt_enable_no_resched() made it turn to
> > noise. Of course, that has other undesirable impacts, and I am not sure if we
> > are in the territory of code layout changes affecting the numbers.
> > 
> > Just mentioning it as some data point.
> 
> [1] https://lore.kernel.org/linux-mm/20260630174852-mutt-send-email-mst@kernel.org/

Thanks for the pointer.

IIUC in those cases you're using preempt_disable() .. preempt_enable()
directly, not this_cpu_*(), right?

If so, patches 5 and 6 of this series [2,3] might have an impact, but I
wouldn't expect a significant change unless you're calling
preempt_enable a lot.

Please beware that it's not safe to use preempt_enable_no_resched()
UNLESS it is immediately followed by a call to schedule(). That's not
documented today (and I couldn't find a good reference), so more folk
are likely to be tempted to use it...

I'll send a patch to clarify that (or at least start the discussion with
scheduler/preempt folk).

Mark.

[2] https://lore.kernel.org/linux-arm-kernel/20260804170503.3513916-6-mark.rutland@arm.com/
[3] https://lore.kernel.org/linux-arm-kernel/20260804170503.3513916-7-mark.rutland@arm.com/