[PATCH v5 12/14] KVM: arm64: Make leaf->leaf PTE changes parallel-aware

[Ben Gardon]

On Mon, Nov 7, 2022 at 1:59 PM Oliver Upton <oliver.upton at linux.dev> wrote:
>
> Convert stage2_map_walker_try_leaf() to use the new break-before-make
> helpers, thereby making the handler parallel-aware. As before, avoid the
> break-before-make if recreating the existing mapping. Additionally,
> retry execution if another vCPU thread is modifying the same PTE.
>
> Signed-off-by: Oliver Upton <oliver.upton at linux.dev>

Reviewed-by: Ben Gardon <bgardon at google.com>


> ---
>  arch/arm64/kvm/hyp/pgtable.c | 26 ++++++++++++--------------
>  1 file changed, 12 insertions(+), 14 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
> index b9f0d792b8d9..238f29389617 100644
> --- a/arch/arm64/kvm/hyp/pgtable.c
> +++ b/arch/arm64/kvm/hyp/pgtable.c
> @@ -804,18 +804,17 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx,
>         else
>                 new = kvm_init_invalid_leaf_owner(data->owner_id);
>
> -       if (stage2_pte_is_counted(ctx->old)) {
> -               /*
> -                * Skip updating the PTE if we are trying to recreate the exact
> -                * same mapping or only change the access permissions. Instead,
> -                * the vCPU will exit one more time from guest if still needed
> -                * and then go through the path of relaxing permissions.
> -                */
> -               if (!stage2_pte_needs_update(ctx->old, new))
> -                       return -EAGAIN;
> +       /*
> +        * Skip updating the PTE if we are trying to recreate the exact
> +        * same mapping or only change the access permissions. Instead,
> +        * the vCPU will exit one more time from guest if still needed
> +        * and then go through the path of relaxing permissions.
> +        */
> +       if (!stage2_pte_needs_update(ctx->old, new))
> +               return -EAGAIN;
>
> -               stage2_put_pte(ctx, data->mmu, mm_ops);
> -       }
> +       if (!stage2_try_break_pte(ctx, data->mmu))
> +               return -EAGAIN;
>
>         /* Perform CMOs before installation of the guest stage-2 PTE */
>         if (mm_ops->dcache_clean_inval_poc && stage2_pte_cacheable(pgt, new))
> @@ -825,9 +824,8 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx,
>         if (mm_ops->icache_inval_pou && stage2_pte_executable(new))
>                 mm_ops->icache_inval_pou(kvm_pte_follow(new, mm_ops), granule);
>
> -       smp_store_release(ctx->ptep, new);
> -       if (stage2_pte_is_counted(new))
> -               mm_ops->get_page(ctx->ptep);
> +       stage2_make_pte(ctx, new);
> +
>         if (kvm_phys_is_valid(phys))
>                 data->phys += granule;
>         return 0;
> --
> 2.38.1.431.g37b22c650d-goog
>