[PATCH v1 2/2] KVM: arm64: Make stage2_split_walker() skip unnecessary walks

[Leonardo Bras]

Currently, when splitting a hugepage, all it's child and sibling nodes
will be walked, with the walker just returning earlier if there is nothing
to do. This means all pagetable entries in the splitting range get a
callback from the walker function, even if it was a level-3 entry.

Optimize splitting by skipping all level-3 entries, as they are already the
smallest block size and can't be split any further.
(i.e. set flag KVM_PGTABLE_WALK_SKIP_LEVEL3)

Optimization measured on two scenarios involving eager-splitting on a
VM with 1 memslot of 64GB:
- Scenario 1: No manual protect, whole memslot split at dirty-track enable
  (KVM_SET_USER_MEMORY_REGION2 ioctl with KVM_MEM_LOG_DIRTY_PAGES)
- Scenario 2: Manual protect, split happens during dirty-bit clean
  (KVM_CLEAR_DIRTY_LOG ioctl), average for 2 iterations.

Scenario 1, improvement on dirty-track enable for the memslot:
- Memory was already split (4k pages):  -35.47% runtime
- THP backed memory:                    -11.94% runtime
- 64x1GB hugetlb memory:                -14.46% runtime

Scenario 2, improvement on dirty-log clean for the memslot:
- Memory was already split (4k pages):  -26.36% runtime
- THP backed memory:                    -12.05% runtime
- 64x1GB hugetlb memory:                -13.87% runtime

Signed-off-by: Leonardo Bras <leo.bras at arm.com>
---
 arch/arm64/kvm/hyp/pgtable.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 48d88a290a53..70103934a04a 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -1565,21 +1565,22 @@ static int stage2_split_walker(const struct kvm_pgtable_visit_ctx *ctx,
 	new = kvm_init_table_pte(childp, mm_ops);
 	stage2_make_pte(ctx, new);
 	return 0;
 }
 
 int kvm_pgtable_stage2_split(struct kvm_pgtable *pgt, u64 addr, u64 size,
 			     struct kvm_mmu_memory_cache *mc)
 {
 	struct kvm_pgtable_walker walker = {
 		.cb	= stage2_split_walker,
-		.flags	= KVM_PGTABLE_WALK_LEAF,
+		.flags	= KVM_PGTABLE_WALK_LEAF |
+			  KVM_PGTABLE_WALK_SKIP_LEVEL3,
 		.arg	= mc,
 	};
 	int ret;
 
 	ret = kvm_pgtable_walk(pgt, addr, size, &walker);
 	dsb(ishst);
 	return ret;
 }
 
 int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_s2_mmu *mmu,
-- 
2.54.0