[PATCH] RISC-V: KVM: Fix double free of the split page cache

[Li Jinyue]

kvm_riscv_mmu_free_pgd() frees pgd_split_page_cache after dropping
mmu_lock. It is reached from kvm_arch_flush_shadow_all(), i.e. from the
MMU notifier ->release() callback, which can run concurrently for the
same VM from exit_mmap() and from mmu_notifier_unregister() in
kvm_destroy_vm(). The stage-2 teardown is serialized by mmu_lock and the
second caller finds kvm->arch.pgd already cleared, but both callers then
go on to free the cache, and kvm_mmu_free_memory_cache() does not
tolerate concurrent callers:

  BUG: KASAN: double-free in kvfree+0x28/0x32
  ...
   kfree+0xc4/0x428
   kvfree+0x28/0x32
   kvm_mmu_free_memory_cache+0x178/0x246
   kvm_riscv_mmu_free_pgd+0x20e/0x2b4
   kvm_arch_flush_shadow_all+0x1a/0x24
   kvm_mmu_notifier_release+0x36/0x7c
   __mmu_notifier_release+0x182/0x66a
   exit_mmap+0x7f8/0x8c4
   mmput+0x7e/0x2fc
   do_exit+0x57a/0x22b4
  ...
  Freed by task 15956 on cpu 3 at 8926.103524s:
  ...
   kvm_mmu_free_memory_cache+0x178/0x246
   kvm_riscv_mmu_free_pgd+0x20e/0x2b4
   kvm_arch_flush_shadow_all+0x1a/0x24
   kvm_mmu_notifier_release+0x36/0x7c
   mmu_notifier_unregister+0xf0/0x328
   kvm_put_kvm+0x32a/0x7b8
   kvm_vm_release+0x3a/0x50
   __fput+0x330/0x974

The cache only feeds huge page splitting from VM ioctls, so it does not
need to be released together with the page table. Free it from
kvm_arch_destroy_vm() instead, which runs once after the MMU notifier
has been unregistered, as arm64 and x86 already do. The error path in
kvm_arch_init_vm() is not affected because the cache cannot have been
filled at that point.

Found by syzkaller.

Fixes: 6c7f75ef096a ("RISC-V: KVM: Add the split page cache for ioctl context")
Cc: stable at vger.kernel.org
Signed-off-by: Li Jinyue <jinyue.li at linux.alibaba.com>
---
 arch/riscv/kvm/mmu.c | 2 --
 arch/riscv/kvm/vm.c  | 2 ++
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 3e955d808743..74dece22d172 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -798,8 +798,6 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
 
 	if (pgd)
 		free_pages((unsigned long)pgd, get_order(kvm_riscv_gstage_pgd_size));
-
-	kvm_mmu_free_memory_cache(&kvm->arch.pgd_split_page_cache);
 }
 
 void kvm_riscv_mmu_update_hgatp(struct kvm_vcpu *vcpu)
diff --git a/arch/riscv/kvm/vm.c b/arch/riscv/kvm/vm.c
index a9f083feeb76..a7e1b518257a 100644
--- a/arch/riscv/kvm/vm.c
+++ b/arch/riscv/kvm/vm.c
@@ -54,6 +54,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 	kvm_destroy_vcpus(kvm);
 
 	kvm_riscv_aia_destroy_vm(kvm);
+
+	kvm_mmu_free_memory_cache(&kvm->arch.pgd_split_page_cache);
 }
 
 int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irql,

base-commit: c03c1a14507dfaf8b856c2608f9ac216021fc222
-- 
2.25.1