[PATCH 19/21] arm64: entry: The great stack switcheroo

[Will Deacon]

On Mon, Sep 07, 2026 at 05:42:44PM +0100, Will Deacon wrote:
> With the kernel stack pointer in SP_EL1 and the overflow stack pointer
> in SP_EL0, it is now straightforward to switch between the two on
> exception entry from EL1 by writing to SPSel. However, since exception
> entry sets PSTATE.SP to 1 (selecting SP_EL1 as the stack pointer),
> repurposing the overflow stack as a more general kernel exception stack
> would require writing to SPSel on every exception entry from the kernel.
> 
> Switch things around so that the overflow stack resides in SP_EL1, with
> the kernel stack residing in SP_EL0.
> 
> Signed-off-by: Will Deacon <will at kernel.org>
> ---
>  arch/arm64/include/asm/kvm_asm.h | 16 +++++++++++++++
>  arch/arm64/kernel/entry-common.c | 34 ++++++++++++++++----------------
>  arch/arm64/kernel/entry.S        | 26 +++++++++++++-----------
>  arch/arm64/kernel/head.S         |  5 ++++-
>  arch/arm64/kernel/smp.c          |  8 +++++---
>  arch/arm64/kvm/hyp/entry.S       |  3 +++
>  arch/arm64/mm/proc.S             |  3 +++
>  7 files changed, 63 insertions(+), 32 deletions(-)

[...]

> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index 12aacc74f764..0811fa569100 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -98,6 +98,7 @@ SYM_FUNC_START(cpu_do_suspend)
>  	mrs	x10, oslsr_el1
>  	mrs	x11, sctlr_el1
>  	get_this_cpu_offset x12
> +	msr	spsel, #1
>  	mrs	x13, sp_el0
>  	stp	x2, x3, [x0]
>  	stp	x4, x5, [x0, #16]
> @@ -156,6 +157,8 @@ alternative_else_nop_endif
>  	msr	sctlr_el1, x12
>  	set_this_cpu_offset x13
>  	msr	sp_el0, x14
> +	msr	spsel, #0

Sashiko rightfully points out that I've got this slightly wrong because
I've pushed the spsel toggle too deep in the callchain for the
suspend/resume paths. My original rationale was that the current code
already saves/restores SP_EL{0,1}, so we shouldn't need to change
anything. However, that's not quite right because it's the caller (e.g.
__cpu_suspend_enter()) which saves SP_EL1 as 'sp', so the stack toggle
should happen there.

I'll fix that in v2.

Will