Any way to disable KVM VHE extension?

[Robin Murphy]

On 2021-07-15 09:55, Qu Wenruo wrote:
> Hi,
> 
> Recently I'm playing around the Nvidia Xavier AGX board, which has VHE 
> extension support.
> 
> In theory, considering the CPU and memory, it should be pretty powerful 
> compared to boards like RPI CM4.
> 
> But to my surprise, KVM runs pretty poor on Xavier.
> 
> Just booting the edk2 firmware could take over 10s, and 20s to fully 
> boot the kernel.
> Even my VM on RPI CM4 has way faster boot time, even just running on 
> PCIE2.0 x1 lane NVME, and just 4 2.1Ghz A72 core.
> 
> This is definitely out of my expectation, I double checked to be sure 
> that it's running in KVM mode.
> 
> But further digging shows that, since Xavier AGX CPU supports VHE, kvm 
> is running in VHE mode other than HYP mode on CM4.
> 
> Is there anyway to manually disable VHE mode to test the more common HYP 
> mode on Xavier?

According to kernel-parameters.txt, "kvm-arm.mode=nvhe" (or its 
low-level equivalent "id_aa64mmfr1.vh=0") on the command line should do 
that.

However I'd imagine the discrepancy is likely to be something more 
fundamental to the wildly different microarchitectures. There's 
certainly no harm in giving non-VHE a go for comparison, but I wouldn't 
be surprised if it turns out even slower...

Robin.

> BTW, this is the dmesg related to KVM on Xavier, running v5.13 upstream 
> kernel, with 64K page size:
> [    0.852357] kvm [1]: IPA Size Limit: 40 bits
> [    0.857378] kvm [1]: vgic interrupt IRQ9
> [    0.862122] kvm: pmu event creation failed -2
> [    0.866734] kvm [1]: VHE mode initialized successfully
> 
> While on CM4, the host runs v5.12.10 upstream kernel (with downstream 
> dtb), with 4K page size:
> [    1.276818] kvm [1]: IPA Size Limit: 44 bits
> [    1.278425] kvm [1]: vgic interrupt IRQ9
> [    1.278620] kvm [1]: Hyp mode initialized successfully
> 
> Could it be the PAGE size causing problem?
> 
> Thanks,
> Qu
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel