[PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H

[Mark Rutland]

On Mon, Apr 15, 2024 at 09:54:15AM +0200, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <ardb at kernel.org>
> 
> Even though the boot protocol stipulates otherwise, an exception has
> been made for the EFI stub, and entering the core kernel with the MMU
> enabled is permitted. This allows a substantial amount of cache
> maintenance to be elided, wich is significant when fast boot times are
> critical (e.g., for booting micro-VMs)
> 
> Once the initial ID map has been populated, the MMU is disabled as part
> of the logic sequence that puts all system registers into a known state.
> Any code that needs to execute within the window where the MMU is off is
> cleaned to the PoC explicitly, which includes all of HYP text when
> entering at EL2.
> 
> However, the current sequence of initializing the EL2 system registers
> is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2
> is reprogrammed, and this means that a VHE-to-nVHE switch may occur
> while the MMU is enabled. This switch causes some system registers as
> well as page table descriptors to be interpreted in a different way,
> potentially resulting in spurious exceptions relating to MMU
> translation.
> 
> So disable the MMU explicitly first when entering in EL2 with the MMU
> and caches enabled.
> 
> Signed-off-by: Ard Biesheuvel <ardb at kernel.org>

Acked-by: Mark Rutland <mark.rutland at arm.com>

Mark.

> ---
>  arch/arm64/kernel/head.S | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index b8bbd72cb194..cb68adcabe07 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
>  	adr_l	x1, __hyp_text_end
>  	adr_l	x2, dcache_clean_poc
>  	blr	x2
> +
> +	mov_q	x0, INIT_SCTLR_EL2_MMU_OFF
> +	pre_disable_mmu_workaround
> +	msr	sctlr_el2, x0
> +	isb
>  0:
>  	mov_q	x0, HCR_HOST_NVHE_FLAGS
>  
> -- 
> 2.44.0.683.g7961c838ac-goog
>