[PATCH v2 2/2] KVM: arm64: nv: Don't save/restore FP register during a nested ERET or exception

[Mark Rutland]

On Wed, May 20, 2026 at 09:50:36AM +0100, Marc Zyngier wrote:
> When switching between L1 and L2, we save the old state using
> kvm_arch_vcpu_put(), mutate the state in memory, then load the new
> state using kvm_arch_vcpu_load(). Any live FPSIMD/SVE state is saved
> and unbound, such that it can be lazily restored on a subsequent trap.
> 
> The FPSIMD/SVE state is shared by exception levels, and only a handful
> of related control registers need to be changed when transitioning
> between L1 and L2. The save/restore of the common state is needless
> overhead, especially as trapping becomes exponentially more expensive
> with nesting.
> 
> Avoid this overhead by leaving the common FPSIMD/SVE state live on the
> CPU, and only switching the state that is distinct for L1 and L2:
> 
> - the trap controls: the effective values are recomputed on each entry
>   into the guest to take the EL into account and merge the L0 and L1
>   configuration if in a nested context, or directly use the L0 configuration
>   in non-nested context (see __activate_traps()).
> 
> - the VL settings: the effective values are are also recomputed on each
>   entry into the guest (see fpsimd_lazy_switch_to_guest()).
>
> Since we appear to cover all bases, use the vcpu flags indicating the
> handling of a nested ERET or exception delivery to avoid the whole FP
> save/restore shenanigans. SME will have to be similarly dealt with when
> it eventually gets supported.
> 
> For an EL1 L3 guest where L1 and L2 have this optimisation, this
> results in at least a 10% wall clock reduction when running an I/O
> heavy workload, generating a high rate of nested exceptions.

There's on additional thing that's important, but I forgot to mention
last time: in the window between kvm_arch_vcpu_put() and
kvm_arch_vcpu_load(), it's possible to take an interrupt, and for a
softirq handler to try to use kernel mode NEON.

Due to that, kvm_arch_vcpu_put() must leave the L1 guest's maximum VL
configured in the host's ZCR_ELx, such that the guest's state can be
saved.

That value is configured by fpsimd_lazy_switch_to_host(), so we just
need to make sure that kvm_arch_vcpu_put() doesn't clobber it. I *think*
that's fine today, but maybe that warrants a comment somewhere.

Other than that, this all looks good to me:

Acked-by: Mark Rutland <mark.rutland at arm.com>

Mark.

> Signed-off-by: Marc Zyngier <maz at kernel.org>
> ---
>  arch/arm64/kvm/fpsimd.c | 23 +++++++++++++++++++++++
>  1 file changed, 23 insertions(+)
> 
> diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
> index 15e17aca1dec0..aca98752a6e42 100644
> --- a/arch/arm64/kvm/fpsimd.c
> +++ b/arch/arm64/kvm/fpsimd.c
> @@ -28,6 +28,20 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
>  	if (!system_supports_fpsimd())
>  		return;
>  
> +	/*
> +	 * Avoid needless save/restore of the guest's common
> +	 * FPSIMD/SVE/SME regs during transitions between L1/L2.
> +	 *
> +	 * These transitions only happens in a non-preemptible context
> +	 * where the host regs have already been saved and unbound. The
> +	 * live registers are either free or owned by the guest.
> +	 */
> +	if (vcpu_get_flag(vcpu, IN_NESTED_ERET) ||
> +	    vcpu_get_flag(vcpu, IN_NESTED_EXCEPTION)) {
> +		WARN_ON_ONCE(host_owns_fp_regs());
> +		return;
> +	}
> +
>  	/*
>  	 * Ensure that any host FPSIMD/SVE/SME state is saved and unbound such
>  	 * that the host kernel is responsible for restoring this state upon
> @@ -102,6 +116,15 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
>  {
>  	unsigned long flags;
>  
> +	/*
> +	 * See comment in kvm_arch_vcpu_load_fp().
> +	 */
> +	if (vcpu_get_flag(vcpu, IN_NESTED_ERET) ||
> +	    vcpu_get_flag(vcpu, IN_NESTED_EXCEPTION)) {
> +		WARN_ON_ONCE(host_owns_fp_regs());
> +		return;
> +	}
> +
>  	local_irq_save(flags);
>  
>  	if (guest_owns_fp_regs()) {
> -- 
> 2.47.3
>