[PATCH v19 05/20] KVM: arm64: Track the type of VM in kvm_arch
Gavin Shan
gshan at redhat.com
Tue Sep 22 23:19:27 PDT 2026
On 9/23/26 4:05 PM, Gavin Shan wrote:
> On 9/21/26 7:28 AM, Suzuki K Poulose wrote:
>> KVM arm64 has different types of VMs with all the different modes in which
>> the hypervisor code can be run. e.g., VHE, nVHE, pKVM etc. Then there is
>> protected VM and normal VMs with pKVM. We might soon add other types,
>> e.g., Arm CCA Realm. So in an effort to make the handling of these
>> different types of VMs a bit more friendly to the eyes, add a VM flavor to
>> the kvm_arch and we could then add handlers for different operations based
>> on the VM type.
>>
>> Keep the flavor initialisation at the beginning to allow for the detection
>> early enough and fail out on any unsupported requests.
>>
>> With that, add wrappers for checking the "type" of a VM and replace the
>> existing users with the new wrappers.
>>
>> Given we already have the construct of "kvm_vm_is_protected" in the core
>> KVM code, use that for all confidential compute guests including Realms
>> that we are about to add.
>>
>> Adds __VM_PROTECTED marker vm flavor to generalize kvm_vm_is_protected()
>> to predicate all confidential guests running on KVM. In later patches, we
>> would add Realm VMs, which would also be classified as protected.
>>
>> Add explicit helper to detect if a given VM is a "protected" VM under pKVM.
>> Change the existing users that precisely want to check the VM type. These
>> include :
>> - kvm_arch_prepare_memory_region - For preventing memslot changes after
>> pVM creation.
>>
>> All the others are retained as a wider check for confidential guest VMs.
>> These are:
>> - kvm_vm_ioctl_set_counter_offset - For disallowing timer offset
>> configuration
>> - io_mem_abort for dabt handling without valid syndrome information
>>
>> Both of which are true for Realms too.
>>
>> Realms support is restricted to VHE host and thus "kvm_vm_is_protected()"
>> checks in the pkvm hyp specific code doesn't need to change, as the only
>> protected guests it deals with is "protected pKVM" guests. To tighten this
>> init_pkvm_hyp_vm() restricts the hyp copy of the vm_flavor to the ones it
>> supports.
>>
>> Suggested-by: Marc Zyngier <maz at kernel.org>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose at arm.com>
>> ---
>> Changes since v18:
>> - Merge the __VM_PROTECTED marker and the widening of kvm_vm_is_protected()
>> to this patch.
>> - Merge the use of kvm_vm_is_unprotected_pkvm() for !kvm_vm_is_protected()
>> given the scope changes here.
>> - Drop Fuad's review tag, as this patch has multiple merges
>> - Restrict the VM flavors to the supported types in init_pkvm_hyp_vm().
>> - Drop kvm_vm_hyp_is_pkvm() and revert to is_protected_kvm_enabled()
>> - Use is_protected_kvm_enabled() to make the pKVM guest flavor checks.
>> - s/PKVM/pKVM for commit descriptions too
>>
>> Changes since v17:
>> * s/PKVM/pKVM for the comments
>> * Drop type argument for pkvm_init_host_vm and also drop protected variable.
>> * Add helpers for checking if the VM is running on pKVM (kvm_vm_hyp_is_pkvm())
>> * Use kvm_vm_hyp_is_pkvm() to replace is_protected_kvm_enabled() with valid
>> kvm instance
>> ---
>> arch/arm64/include/asm/kvm_host.h | 22 +++++++++++++++++++---
>> arch/arm64/include/asm/kvm_pkvm.h | 4 ++--
>> arch/arm64/kvm/arm.c | 31 ++++++++++++++++++++++++++-----
>> arch/arm64/kvm/handle_exit.c | 2 +-
>> arch/arm64/kvm/hyp/nvhe/pkvm.c | 6 +++++-
>> arch/arm64/kvm/mmu.c | 2 +-
>> arch/arm64/kvm/pkvm.c | 6 ++----
>> 7 files changed, 56 insertions(+), 17 deletions(-)
>>
>
> This causes broken nVHE mode. I applied PATCH[01-05] to v7.3.rc4 whose head commit
> is f0100363d8c3, and kselftests/kvm/guest_print causes host crash (as below). I don't
> see the crash if only PATCH[01-04] are applied.
>
> host$ cat /proc/cmdline | grep kvm-arm\.mode
> BOOT_IMAGE=(hd0,gpt2)/vmlinuz-7.3.0-rc4-gavin+ root=/dev/mapper/rhel_nvidia--grace--hopper--01-root ro crashkernel=2G-4G:406M,4G-64G:470M,64G-:726M rd.lvm.lv=rhel_nvidia-grace-hopper-01/root rd.lvm.lv=rhel_nvidia-grace-hopper-01/swap video=simplefb:off kvm-arm.mode=nvhe
>
> host$ cd linux/tools/testing/selftests/kvm
> host$ ./guest_print_test
> Random seed: 0x193a0ed3
> [ 192.754328] kvm [6674]: nVHE hyp panic at: [<ffff8000811da044>] __kvm_nvhe___timer_enable_traps+0x4/0x160!
> [ 192.754338] kvm [6674]: nVHE call trace:
> [ 192.754339] kvm [6674]: [<ffff8000811dcf34>] __kvm_nvhe_hyp_panic+0xb4/0xe0
> [ 192.754342] kvm [6674]: [<ffff8000811dcba4>] __kvm_nvhe___kvm_vcpu_run+0x164/0x440
> [ 192.754344] kvm [6674]: [<ffff8000811dfe90>] __kvm_nvhe_handle___kvm_vcpu_run+0x40/0x1f0
> [ 192.754346] kvm [6674]: [<ffff8000811e0238>] __kvm_nvhe_handle_trap+0x158/0x280
> [ 192.754347] kvm [6674]: [<ffff8000811dd8fc>] __kvm_nvhe___skip_pauth_save+0x4/0x4
> [ 192.754348] kvm [6674]: ---[ end nVHE call trace ]---
> [ 192.754350] Code: d2818002 17fffff7 d503201f f9400001 (b94a6821)
> [ 192.754350] kvm [6674]: Hyp Offset: 0xfffeb0d7fe2e0000
> [ 192.754351] Kernel panic - not syncing: HYP panic:
> [ 192.754351] PS:834003c9 PC:0000cf2882efa044 ESR:0000000096000004
> [ 192.754351] FAR:ffff00009b286a68 HPFAR:8000000000000000 PAR:1d00ec7edbadc8de
> [ 192.754351] VCPU:0000cf011848a350
> [ 192.845154] CPU: 12 UID: 0 PID: 6674 Comm: guest_print_tes Kdump: loaded Not tainted 7.3.0-rc4-gavin+ #7 PREEMPT(full)
> [ 192.856182] Hardware name: GH200 P5042, BIOS 02.04.01 20250422
> [ 192.862231] Call trace:
> [ 192.864725] show_stack+0x20/0x38 (C)
> [ 192.868471] dump_stack_lvl+0x88/0xb8
> [ 192.872215] dump_stack+0x18/0x30
> [ 192.875598] vpanic+0x280/0x498
> [ 192.878806] panic+0x68/0x70
> [ 192.881745] nvhe_hyp_panic_handler+0x184/0x190
> [ 192.886372] kvm_arm_vcpu_enter_exit+0x24/0x100
> [ 192.891003] kvm_arch_vcpu_ioctl_run+0x254/0x7c0
> [ 192.895726] kvm_vcpu_ioctl+0x174/0xb40
> [ 192.899645] __arm64_sys_ioctl+0xb0/0x120
> [ 192.903745] invoke_syscall.constprop.0+0xa8/0x100
> [ 192.908639] do_el0_svc+0xb8/0xe0
> [ 192.912022] el0_svc+0x48/0x1f8
> [ 192.915228] el0t_64_sync_handler+0xa0/0xe8
> [ 192.919500] el0t_64_sync+0x1ac/0x1b0
> [ 192.923243] SMP: stopping secondary CPUs
> [ 192.927653] Starting crashdump kernel...
> [ 192.931658] Bye!
>
With the following changes applied after PATCH[01-05] on top of v7.3.rc4, the crash
is avoided.
In arch/arm64/include/asm/kvm_host.h:
-#define kvm_vm_is_protected(kvm) ((kvm)->arch.vm_flavor >= __VM_PROTECTED)
+#define kvm_vm_is_protected(kvm) \
+ (is_protected_kvm_enabled() && (kvm)->arch.vm_flavor >= __VM_PROTECTED)
Thanks,
Gavin
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>> index 286489a69dff5..9b1cf9c59e81f 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -257,7 +257,6 @@ struct kvm_protected_vm {
>> pkvm_handle_t handle;
>> struct kvm_hyp_memcache teardown_mc;
>> struct kvm_hyp_memcache stage2_teardown_mc;
>> - bool is_protected;
>> bool is_created;
>> /*
>> @@ -306,9 +305,19 @@ enum fgt_group_id {
>> __NR_FGT_GROUP_IDS__
>> };
>> +enum kvm_arm_vm_flavor {
>> + VM_NVHE,
>> + VM_VHE,
>> + VM_PKVM, /* Normal guests on pKVM */
>> + MARKER(__VM_PROTECTED),
>> + VM_PROTECTED_PKVM, /* Protected VM */
>> + VM_FLAVOR_MAX,
>> +};
>> +
>> struct kvm_arch {
>> struct kvm_s2_mmu mmu;
>> + enum kvm_arm_vm_flavor vm_flavor;
>> /*
>> * Fine-Grained UNDEF, mimicking the FGT layout defined by the
>> * architecture. We track them globally, as we present the
>> @@ -1504,10 +1513,17 @@ struct kvm *kvm_arch_alloc_vm(void);
>> #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE
>> -#define kvm_vm_is_protected(kvm) (is_protected_kvm_enabled() && (kvm)->arch.pkvm.is_protected)
>> -
>> +#define kvm_vm_is_protected(kvm) ((kvm)->arch.vm_flavor >= __VM_PROTECTED)
>> #define vcpu_is_protected(vcpu) kvm_vm_is_protected((vcpu)->kvm)
>> +#define kvm_vm_is_protected_pkvm(kvm) \
>> + (is_protected_kvm_enabled() && ((kvm)->arch.vm_flavor == VM_PROTECTED_PKVM))
>> +#define vcpu_is_protected_pkvm(vcpu) kvm_vm_is_protected_pkvm(vcpu->kvm)
>> +
>> +#define kvm_vm_is_unprotected_pkvm(kvm) \
>> + (is_protected_kvm_enabled() && ((kvm)->arch.vm_flavor == VM_PKVM))
>> +
>> +
>> int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature);
>> bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu);
>> diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
>> index 54a618d887fa4..e4ea80711bec6 100644
>> --- a/arch/arm64/include/asm/kvm_pkvm.h
>> +++ b/arch/arm64/include/asm/kvm_pkvm.h
>> @@ -17,7 +17,7 @@
>> #define HYP_MEMBLOCK_REGIONS 128
>> -int pkvm_init_host_vm(struct kvm *kvm, unsigned long type);
>> +int pkvm_init_host_vm(struct kvm *kvm);
>> int pkvm_create_hyp_vm(struct kvm *kvm);
>> bool pkvm_hyp_vm_is_created(struct kvm *kvm);
>> void pkvm_destroy_hyp_vm(struct kvm *kvm);
>> @@ -49,7 +49,7 @@ static inline bool kvm_pkvm_ext_allowed(struct kvm *kvm, long ext)
>> case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES:
>> return false;
>> default:
>> - return !kvm || !kvm_vm_is_protected(kvm);
>> + return !kvm || kvm_vm_is_unprotected_pkvm(kvm);
>> }
>> }
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index db36815630790..8c784b266a8e8 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -214,6 +214,26 @@ static int kvm_arm_default_max_vcpus(void)
>> return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
>> }
>> +static int kvm_init_vm_flavor(struct kvm *kvm, unsigned long type)
>> +{
>> + bool protected = type & KVM_VM_TYPE_ARM_PROTECTED;
>> +
>> + if (is_protected_kvm_enabled()) {
>> + if (protected)
>> + kvm->arch.vm_flavor = VM_PROTECTED_PKVM;
>> + else
>> + kvm->arch.vm_flavor = VM_PKVM;
>> + } else if (protected) {
>> + return -EINVAL;
>> + } else if (has_vhe()) {
>> + kvm->arch.vm_flavor = VM_VHE;
>> + } else {
>> + kvm->arch.vm_flavor = VM_NVHE;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> /**
>> * kvm_arch_init_vm - initializes a VM data structure
>> * @kvm: pointer to the KVM struct
>> @@ -236,6 +256,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>> mutex_unlock(&kvm->lock);
>> #endif
>> + ret = kvm_init_vm_flavor(kvm, type);
>> + if (ret)
>> + return ret;
>> +
>> kvm_init_nested(kvm);
>> ret = kvm_share_hyp(kvm, kvm + 1);
>> @@ -257,12 +281,9 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
>> * If any failures occur after this is successful, make sure to
>> * call __pkvm_unreserve_vm to unreserve the VM in hyp.
>> */
>> - ret = pkvm_init_host_vm(kvm, type);
>> + ret = pkvm_init_host_vm(kvm);
>> if (ret)
>> goto err_uninit_mmu;
>> - } else if (type & KVM_VM_TYPE_ARM_PROTECTED) {
>> - ret = -EINVAL;
>> - goto err_uninit_mmu;
>> }
>> kvm_vgic_early_init(kvm);
>> @@ -985,7 +1006,7 @@ int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)
>> if (is_protected_kvm_enabled()) {
>> /* Start with the vcpu in a dirty state */
>> - if (!kvm_vm_is_protected(vcpu->kvm))
>> + if (kvm_vm_is_unprotected_pkvm(vcpu->kvm))
>> vcpu_set_flag(vcpu, PKVM_HOST_STATE_DIRTY);
>> ret = pkvm_create_hyp_vm(kvm);
>> if (ret)
>> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
>> index db37678dcb05c..384c5d258c7f8 100644
>> --- a/arch/arm64/kvm/handle_exit.c
>> +++ b/arch/arm64/kvm/handle_exit.c
>> @@ -490,7 +490,7 @@ static void handle_exit_pkvm_state(struct kvm_vcpu *vcpu, int exception_index)
>> {
>> int exception_code = ARM_EXCEPTION_CODE(exception_index);
>> - if (!is_protected_kvm_enabled() || kvm_vm_is_protected(vcpu->kvm))
>> + if (!kvm_vm_is_unprotected_pkvm(vcpu->kvm))
>> return;
>> /*
>> diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
>> index 459bd9eb7e4bc..57e2eef6d7426 100644
>> --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
>> +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
>> @@ -432,7 +432,11 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, struct pkvm_hyp_vm *hyp_vm,
>> hyp_vm->host_kvm = host_kvm;
>> hyp_vm->kvm.created_vcpus = nr_vcpus;
>> - hyp_vm->kvm.arch.pkvm.is_protected = READ_ONCE(host_kvm->arch.pkvm.is_protected);
>> + if (kvm_vm_is_protected(host_kvm))
>> + hyp_vm->kvm.arch.vm_flavor = VM_PROTECTED_PKVM;
>> + else
>> + hyp_vm->kvm.arch.vm_flavor = VM_PKVM;
>> +
>> hyp_vm->kvm.arch.flags = 0;
>> pkvm_init_features_from_host(hyp_vm, host_kvm);
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 9ba86450fe4af..0f4e8b71fa85d 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
>> @@ -2624,7 +2624,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
>> hva_t hva, reg_end;
>> int ret = 0;
>> - if (kvm_vm_is_protected(kvm)) {
>> + if (kvm_vm_is_protected_pkvm(kvm)) {
>> /* Cannot modify memslots once a pVM has run. */
>> if (pkvm_hyp_vm_is_created(kvm) &&
>> (change == KVM_MR_DELETE || change == KVM_MR_MOVE)) {
>> diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
>> index 8e4c6e4bec123..8e9176a700926 100644
>> --- a/arch/arm64/kvm/pkvm.c
>> +++ b/arch/arm64/kvm/pkvm.c
>> @@ -229,10 +229,9 @@ void pkvm_destroy_hyp_vm(struct kvm *kvm)
>> mutex_unlock(&kvm->arch.config_lock);
>> }
>> -int pkvm_init_host_vm(struct kvm *kvm, unsigned long type)
>> +int pkvm_init_host_vm(struct kvm *kvm)
>> {
>> int ret;
>> - bool protected = type & KVM_VM_TYPE_ARM_PROTECTED;
>> /* Reserve the VM in hyp and obtain a hyp handle for the VM. */
>> ret = kvm_call_hyp_nvhe(__pkvm_reserve_vm);
>> @@ -240,8 +239,7 @@ int pkvm_init_host_vm(struct kvm *kvm, unsigned long type)
>> return ret;
>> kvm->arch.pkvm.handle = ret;
>> - kvm->arch.pkvm.is_protected = protected;
>> - if (protected) {
>> + if (kvm_vm_is_protected(kvm)) {
>> pr_warn_once("kvm: protected VMs are experimental and for development only, tainting kernel\n");
>> add_taint(TAINT_USER, LOCKDEP_STILL_OK);
>> }
>
More information about the linux-arm-kernel
mailing list