[PATCH v3 7/7] arm64: support PREEMPT_DYNAMIC

[Catalin Marinas]

On Wed, Feb 09, 2022 at 03:35:35PM +0000, Mark Rutland wrote:
> This patch enables support for PREEMPT_DYNAMIC on arm64, allowing the
> preemption model to be chosen at boot time.
> 
> Specifically, this patch selects HAVE_PREEMPT_DYNAMIC_KEY, so that each
> preemption function is an out-of-line call with an early return
> depending upon a static key. This leaves almost all the codegen up to
> the compiler, and side-steps a number of pain points with static calls
> (e.g. interaction with CFI schemes). This should have no worse overhead
> than using non-inline static calls, as those use out-of-line trampolines
> with early returns.
> 
> For example, the dynamic_cond_resched() wrapper looks as follows when
> enabled. When disabled, the first `B` is replaced with a `NOP`,
> resulting in an early return.
> 
> | <dynamic_cond_resched>:
> |        bti     c
> |        b       <dynamic_cond_resched+0x10>     // or `nop`
> |        mov     w0, #0x0
> |        ret
> |        mrs     x0, sp_el0
> |        ldr     x0, [x0, #8]
> |        cbnz    x0, <dynamic_cond_resched+0x8>
> |        paciasp
> |        stp     x29, x30, [sp, #-16]!
> |        mov     x29, sp
> |        bl      <preempt_schedule_common>
> |        mov     w0, #0x1
> |        ldp     x29, x30, [sp], #16
> |        autiasp
> |        ret
> 
> ... compared to the regular form of the function:
> 
> | <__cond_resched>:
> |        bti     c
> |        mrs     x0, sp_el0
> |        ldr     x1, [x0, #8]
> |        cbz     x1, <__cond_resched+0x18>
> |        mov     w0, #0x0
> |        ret
> |        paciasp
> |        stp     x29, x30, [sp, #-16]!
> |        mov     x29, sp
> |        bl      <preempt_schedule_common>
> |        mov     w0, #0x1
> |        ldp     x29, x30, [sp], #16
> |        autiasp
> |        ret
> 
> Since arm64 does not yet use the generic entry code, we must define our
> own `sk_dynamic_irqentry_exit_cond_resched`, which will be
> enabled/disabled by the common code in kernel/sched/core.c. All other
> preemption functions and associated static keys are defined there.
> 
> Note that PREEMPT_DYNAMIC is `def bool y`, so this will default to
> enabled.
> 
> Signed-off-by: Mark Rutland <mark.rutland at arm.com>
> Cc: Ard Biesheuvel <ardb at kernel.org>
> Cc: Catalin Marinas <catalin.marinas at arm.com>
> Cc: Frederic Weisbecker <frederic at kernel.org>
> Cc: Ingo Molnar <mingo at redhat.com>
> Cc: Joey Gouly <joey.gouly at arm.com>
> Cc: Juri Lelli <juri.lelli at redhat.com>
> Cc: Peter Zijlstra <peterz at infradead.org>
> Cc: Valentin Schneider <valentin.schneider at arm.com>
> Cc: Will Deacon <will at kernel.org>

Acked-by: Catalin Marinas <catalin.marinas at arm.com>