[PATCH v2 09/12] KVM: arm64: Support up to 5 levels of translation in kvm_pgtable

[Ryan Roberts]

On 06/03/2023 19:54, Ryan Roberts wrote:
> FEAT_LPA2 increases the maximum levels of translation from 4 to 5 for
> the 4KB page case, when IA is >48 bits. While we can still use 4 levels
> for stage2 translation in this case (due to stage2 allowing concatenated
> page tables for first level lookup), the same kvm_pgtable library is
> used for the hyp stage1 page tables and stage1 does not support
> concatenation.
> 
> Therefore, modify the library to support up to 5 levels. Previous
> patches already laid the groundwork for this by refactoring code to work
> in terms of KVM_PGTABLE_FIRST_LEVEL and KVM_PGTABLE_LAST_LEVEL. So we
> just need to change these macros.
> 
> The hardware sometimes encodes the new level differently from the
> others: One such place is when reading the level from the FSC field in
> the ESR_EL2 register. We never expect to see the lowest level (-1) here
> since the stage 2 page tables always use concatenated tables for first
> level lookup and therefore only use 4 levels of lookup. So we get away
> with just adding a comment to explain why we are not being careful about
> decoding level -1.
> 
> For stage2 VTCR_EL2.SL2 is introduced to encode the new start level.
> However, since we always use concatenated page tables for first level
> look up at stage2 (and therefore we will never need the new extra level)
> we never touch this new field.
> 
> Signed-off-by: Ryan Roberts <ryan.roberts at arm.com>
> ---
>  arch/arm64/include/asm/kvm_emulate.h | 10 ++++++++++
>  arch/arm64/include/asm/kvm_pgtable.h |  2 +-
>  arch/arm64/kvm/hyp/pgtable.c         | 10 ++++++++++
>  3 files changed, 21 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 26666a623fa8..542575aad159 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -405,6 +405,16 @@ static __always_inline u8 kvm_vcpu_trap_get_fault_type(const struct kvm_vcpu *vc
>  
>  static __always_inline s8 kvm_vcpu_trap_get_fault_level(const struct kvm_vcpu *vcpu)
>  {
> +	/*
> +	 * Note: With the introduction of FEAT_LPA2 an extra level of
> +	 * translation (level -1) is added. This level (obviously) doesn't
> +	 * follow the previous convention of encoding the 4 levels in the 2 LSBs
> +	 * of the FSC so this function breaks if the fault is for level -1.
> +	 *
> +	 * However, stage2 tables always use concatenated tables for first level
> +	 * lookup and therefore it is guaranteed that the level will be between
> +	 * 0 and 3, and this function continues to work.
> +	 */
>  	return kvm_vcpu_get_esr(vcpu) & ESR_ELx_FSC_LEVEL;
>  }
>  
> diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
> index 8c7e65147777..a5d96592c658 100644
> --- a/arch/arm64/include/asm/kvm_pgtable.h
> +++ b/arch/arm64/include/asm/kvm_pgtable.h
> @@ -11,7 +11,7 @@
>  #include <linux/kvm_host.h>
>  #include <linux/types.h>
>  
> -#define KVM_PGTABLE_FIRST_LEVEL		0
> +#define KVM_PGTABLE_FIRST_LEVEL		-1
>  #define KVM_PGTABLE_LAST_LEVEL		3
>  
>  /*
> diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
> index 939eab245205..a09c17242da7 100644
> --- a/arch/arm64/kvm/hyp/pgtable.c
> +++ b/arch/arm64/kvm/hyp/pgtable.c
> @@ -612,6 +612,15 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift)
>  	lvls = stage2_pgtable_levels(phys_shift);
>  	if (lvls < 2)
>  		lvls = 2;
> +
> +	/*
> +	 * When LPA2 is enabled, the HW supports an extra level of translation
> +	 * (for 5 in total) when using 4K pages. It also introduces VTCR_EL2.SL2
> +	 * to as an addition to SL0 to enable encoding this extra start level.
> +	 * However, since we always use concatenated pages for the first level
> +	 * lookup, we will never need this extra level and therefore do not need
> +	 * to touch SL2.
> +	 */
>  	vtcr |= VTCR_EL2_LVLS_TO_SL0(lvls);
>  
>  #ifdef CONFIG_ARM64_HW_AFDBM
> @@ -1053,6 +1062,7 @@ struct stage2_attr_data {
>  	kvm_pte_t			attr_clr;
>  	kvm_pte_t			pte;
>  	s8				level;
> +	struct kvm_pgtable_mm_ops	*mm_ops;

Naturally, as soon as I sent this out, I noticed this, which clearly should not
be here. I suspect it's an artifact from rebasing. Sorry about that, please
ignore and I'll fix it in the next version.


>  };
>  
>  static int stage2_attr_walker(const struct kvm_pgtable_visit_ctx *ctx,