[PATCH v10 00/59] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support
Ganapatrao Kulkarni
gankulkarni at os.amperecomputing.com
Tue Jun 27 23:45:55 PDT 2023
Hi Marc,
On 15-05-2023 11:00 pm, Marc Zyngier wrote:
> This is the 4th drop of NV support on arm64 for this year.
>
> For the previous episodes, see [1].
>
> What's changed:
>
> - New framework to track system register traps that are reinjected in
> guest EL2. It is expected to replace the discrete handling we have
> enjoyed so far, which didn't scale at all. This has already fixed a
> number of bugs that were hidden (a bunch of traps were never
> forwarded...). Still a work in progress, but this is going in the
> right direction.
>
> - Allow the L1 hypervisor to have a S2 that has an input larger than
> the L0 IPA space. This fixes a number of subtle issues, depending on
> how the initial guest was created.
>
> - Consequently, the patch series has gone longer again. Boo. But
> hopefully some of it is easier to review...
>
I am facing issue in booting NestedVM with V9 as well with 10 patchset.
I have tried V9/V10 on Ampere platform using kvmtool and I could boot
Guest-Hypervisor and then NestedVM without any issue.
However when I try to boot using QEMU(not using EDK2/EFI),
Guest-Hypervisor is booted with Fedora 37 using virtio disk. From
Guest-Hypervisor console(or ssh shell), If I try to boot NestedVM, boot
hangs very early stage of the boot.
I did some debug using ftrace and it seems the Guest-Hypervisor is
getting very high rate of arch-timer interrupts,
due to that all CPU time is going on in serving the Guest-Hypervisor
and it is never going back to NestedVM.
I am using QEMU vanilla version v7.2.0 with top-up patches for NV [1]
[1]
https://lore.kernel.org/all/20230227163718.62003-1-miguel.luis@oracle.com/
> [1] https://lore.kernel.org/r/20230405154008.3552854-1-maz@kernel.org
>
> Andre Przywara (1):
> KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
>
> Christoffer Dall (5):
> KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> KVM: arm64: nv: vgic: Emulate the HW bit in software
> KVM: arm64: nv: Sync nested timer state with FEAT_NV2
>
> Jintack Lim (7):
> KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
> KVM: arm64: nv: Respect virtual HCR_EL2.{NV,TSC) settings
> KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> KVM: arm64: nv: Nested GICv3 Support
>
> Marc Zyngier (46):
> KVM: arm64: Move VTCR_EL2 into struct s2_mmu
> arm64: Add missing Set/Way CMO encodings
> arm64: Add missing VA CMO encodings
> arm64: Add missing ERXMISCx_EL1 encodings
> arm64: Add missing DC ZVA/GVA/GZVA encodings
> arm64: Add TLBI operation encodings
> arm64: Add AT operation encodings
> KVM: arm64: Add missing HCR_EL2 trap bits
> KVM: arm64: nv: Add trap forwarding infrastructure
> KVM: arm64: nv: Add trap forwarding for HCR_EL2
> KVM: arm64: nv: Expose FEAT_EVT to nested guests
> KVM: arm64: nv: Add trap forwarding for MDCR_EL2
> KVM: arm64: nv: Add trap forwarding for CNTHCTL_EL2
> KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
> KVM: arm64: nv: Handle virtual EL2 registers in
> vcpu_read/write_sys_reg()
> KVM: arm64: nv: Handle SPSR_EL2 specially
> KVM: arm64: nv: Handle HCR_EL2.E2H specially
> KVM: arm64: nv: Save/Restore vEL2 sysregs
> KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
> KVM: arm64: nv: Handle shadow stage 2 page faults
> KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
> KVM: arm64: nv: Set a handler for the system instruction traps
> KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
> KVM: arm64: nv: Hide RAS from nested guests
> KVM: arm64: nv: Add handling of EL2-specific timer registers
> KVM: arm64: nv: Load timer before the GIC
> KVM: arm64: nv: Don't load the GICv4 context on entering a nested
> guest
> KVM: arm64: nv: Implement maintenance interrupt forwarding
> KVM: arm64: nv: Deal with broken VGIC on maintenance interrupt
> delivery
> KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
> KVM: arm64: nv: Add handling of FEAT_TTL TLB invalidation
> KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
> information
> KVM: arm64: nv: Tag shadow S2 entries with nested level
> KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
> KVM: arm64: nv: Map VNCR-capable registers to a separate page
> KVM: arm64: nv: Move nested vgic state into the sysreg file
> KVM: arm64: Add FEAT_NV2 cpu feature
> KVM: arm64: nv: Fold GICv3 host trapping requirements into guest setup
> KVM: arm64: nv: Publish emulated timer interrupt state in the
> in-memory state
> KVM: arm64: nv: Allocate VNCR page when required
> KVM: arm64: nv: Enable ARMv8.4-NV support
> KVM: arm64: nv: Fast-track 'InHost' exception returns
> KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
> KVM: arm64: nv: Use FEAT_ECV to trap access to EL0 timers
> KVM: arm64: nv: Accelerate EL0 timer read accesses when FEAT_ECV is on
>
> .../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
> arch/arm64/include/asm/esr.h | 1 +
> arch/arm64/include/asm/kvm_arm.h | 14 +
> arch/arm64/include/asm/kvm_asm.h | 4 +
> arch/arm64/include/asm/kvm_emulate.h | 93 +-
> arch/arm64/include/asm/kvm_host.h | 181 +++-
> arch/arm64/include/asm/kvm_hyp.h | 2 +
> arch/arm64/include/asm/kvm_mmu.h | 20 +-
> arch/arm64/include/asm/kvm_nested.h | 133 +++
> arch/arm64/include/asm/stage2_pgtable.h | 4 +-
> arch/arm64/include/asm/sysreg.h | 196 ++++
> arch/arm64/include/asm/vncr_mapping.h | 74 ++
> arch/arm64/include/uapi/asm/kvm.h | 1 +
> arch/arm64/kernel/cpufeature.c | 11 +
> arch/arm64/kvm/Makefile | 4 +-
> arch/arm64/kvm/arch_timer.c | 98 +-
> arch/arm64/kvm/arm.c | 33 +-
> arch/arm64/kvm/at.c | 219 ++++
> arch/arm64/kvm/emulate-nested.c | 934 ++++++++++++++++-
> arch/arm64/kvm/handle_exit.c | 29 +-
> arch/arm64/kvm/hyp/include/hyp/switch.h | 8 +-
> arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 5 +-
> arch/arm64/kvm/hyp/nvhe/mem_protect.c | 8 +-
> arch/arm64/kvm/hyp/nvhe/pkvm.c | 4 +-
> arch/arm64/kvm/hyp/nvhe/switch.c | 2 +-
> arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
> arch/arm64/kvm/hyp/pgtable.c | 2 +-
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 6 +-
> arch/arm64/kvm/hyp/vhe/switch.c | 206 +++-
> arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 124 ++-
> arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
> arch/arm64/kvm/mmu.c | 255 ++++-
> arch/arm64/kvm/nested.c | 799 ++++++++++++++-
> arch/arm64/kvm/pkvm.c | 2 +-
> arch/arm64/kvm/reset.c | 7 +
> arch/arm64/kvm/sys_regs.c | 958 +++++++++++++++++-
> arch/arm64/kvm/trace_arm.h | 19 +
> arch/arm64/kvm/vgic/vgic-init.c | 33 +
> arch/arm64/kvm/vgic/vgic-kvm-device.c | 32 +-
> arch/arm64/kvm/vgic/vgic-v3-nested.c | 248 +++++
> arch/arm64/kvm/vgic/vgic-v3.c | 43 +-
> arch/arm64/kvm/vgic/vgic.c | 29 +
> arch/arm64/kvm/vgic/vgic.h | 10 +
> arch/arm64/tools/cpucaps | 1 +
> include/clocksource/arm_arch_timer.h | 4 +
> include/kvm/arm_arch_timer.h | 1 +
> include/kvm/arm_vgic.h | 17 +
> include/uapi/linux/kvm.h | 1 +
> tools/arch/arm/include/uapi/asm/kvm.h | 1 +
> 49 files changed, 4790 insertions(+), 183 deletions(-)
> create mode 100644 arch/arm64/include/asm/vncr_mapping.h
> create mode 100644 arch/arm64/kvm/at.c
> create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c
>
Thanks,
Ganapat
More information about the linux-arm-kernel
mailing list