[PATCH v16 00/45] arm64: Support for Arm CCA in KVM
Gavin Shan
gshan at redhat.com
Tue Aug 11 20:07:50 PDT 2026
On 8/11/26 9:12 PM, Suzuki K Poulose wrote:
> On 11/08/2026 05:44, Gavin Shan wrote:
>> On 8/7/26 8:05 AM, Suzuki K Poulose wrote:
>>
>> [...]
>>
>>>
>>> Here is a cleaned up version, rebased on to Will's kvmtool master
>>> branch:
>>>
>>> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/kvm-v16
>>>
>>> (This works with the v15 of the KVM series too)
>>> Build on top of Fuad's Guest memfd support patches.
>>>
>>
>> With the following combination, I'm able boot up the realm guest.
>>
>> tf-rmm: https://git.trustedfirmware.org/TF-RMM/tf- rmm.git (branch: topics/rmm-v2.0-poc_2)
>
>
> Please be aware that CCA KVM v15 onwards, the above branch is not compatible. You should be using rmm-v2.0-poc_3. Please could you
> confirm if this is still an issue ?
>
With tf-rmm/topics/rmm-v2.0-poc_2 + cca/host-v16 + kvmtool/cca/v16, there is no issue
and the realm guest can boot up successfully.
When tf-rmm/topics/rmm-v2.0-poc_3 is used, the realm guest boot gets stuck as I reported
earlier. Note the host is emulated by QEMU (TCG mode).
As the following calltrace indicates, -EAGAIN is returned from tf-rmm::update_ripas()
because true is returned from s2tte_drain_pending() for the S2TTE corresponding to
IPA 0x80000000. Linux host received error (RMI_ERROR_RTT, level=3) in ripas_change().
Upon this specific error and the IPA range [0x80000000 0x90000000], find_map_level()
returns level of 2, and realm_create_rtt_levels() returns 0 without populating any
RTTs. After that, rmi_rtt_set_ripas() is re-executed and the above loop starts over
again.
Linux host
==========
kvm_arch_vcpu_ioctl_run // cca/host-v16
check_vcpu_requests
kvm_check_request
kvm_rec_handle_request
kvm_complete_ripas_change
realm_set_ipa_state
ripas_change
rmi_rtt_set_ripas
SMC_RMI_RTT_SET_RIPAS
TF-RMM
======
SMC_RMI_RTT_SET_RIPAS // tf-rmm/topics/rmm-v2.0-poc_3
smc_rtt_set_ripas
s2tt_walk_lock_unlock
rtt_set_ripas_range
update_ripas
s2tte_drain_pending // true, returns -EAGAIN
The problem is the pending-bit for RTE corresponding IPA address 0x80000000 isn't cleared
when SMC_RMI_RTT_SET_RIPAS is invoked. I didn't figure out how this bit is set and why
it's not cleared in time.
Thanks,
Gavin
>
> Cheers
> Suzuki
>
>
>> tf-a: https://git.trustedfirmware.org/TF-A/trusted-firmware- a.git (branch: master)
>> host: https://git.gitlab.arm.com/linux-arm/linux- cca.git (branch: cca-host/v16)
>> kvmtool: https://gitlab.arm.com/linux-arm/kvmtool- cca (branch: cca/kvm-v16)
>>
>> However, the guest can't boot up and become stuck in SMC_RSI_IPA_STATE_SET request,
>> which can't completed by the host.
>>
>> tf-rmm: https://git.trustedfirmware.org/TF-RMM/tf- rmm.git (branch: topics/rmm-v2.0-poc_3)
>>
>> (1) Login the emulated host
>>
>> machine$ ssh -o StrictHostKeyChecking=no root at 10.26.1.240
>>
>> (2) Start realm guest using kvmtool
>>
>> root at host:~# lkvm run --realm -c 1 -m 256 \
>> -k /mnt/linux/arch/arm64/boot/Image \
>> -i /mnt/buildroot/output/images/rootfs.cpio.xz \
>> -p earlycon=uart,mmio,0x101000000
>> :
>> [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510]
>> [ 0.000000] Linux version 7.2.0-rc5-gavin-gf5098b6bae76 (gshan at nvidia-grace-hopper-01.khw.eng.bos2.dc.redhat.com) (gcc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4), GNU ld version 2.41-65.el10) #47 SMP PREEMPT Mon Jul 27 02:29:03 EDT 2026
>> [ 0.000000] KASLR enabled
>> [ 0.000000] Machine model: linux,dummy-virt
>> [ 0.000000] earlycon: uart0 at MMIO 0x0000000101000000 (options '')
>> [ 0.000000] printk: legacy bootconsole [uart0] enabled
>> [ 0.000000] efi: UEFI not found.
>> [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT
>> [ 0.000000] NUMA: Faking a node at [mem 0x0000000080000000-0x000000008fffffff]
>> [ 0.000000] NODE_DATA(0) allocated [mem 0x8ff76dc0-0x8ff7ac7f]
>> [ 0.000000] psci: probing for conduit method from DT.
>> [ 0.000000] psci: PSCIv1.1 detected in firmware.
>> [ 0.000000] psci: Using standard PSCI v0.2 function IDs
>> [ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
>> [ 0.000000] psci: SMC Calling Convention v1.2
>> [ 0.000000] RME: Using RSI version 1.0
>> <... no more output from the guest ...>
>>
>>
>> (3) The output from host's serial console
>>
>> SMC_RMI_REALM_CREATE 12a4c7000 12a4c6000 > RMI_INCOMPLETE 0 24 0 0
>> SMC_RMI_OP_MEM_DONATE 0 1002c7098 9 > RMI_INCOMPLETE 9 0 0 0
>> SMC_RMI_OP_CONTINUE 0 0 > RMI_SUCCESS 0 0
>> SMC_RMI_RTT_UNPROT_UNMAP 12a4c7000 18f85c000 18fe00000 0 0 > RMI_ERROR_RTT 2 0 0 0 0
>> SMC_RMI_RTT_UNPROT_UNMAP 12a4c7000 18fe00000 18fe10000 0 0 > RMI_ERROR_RTT 2 0 0 0 0
>> SMC_RMI_RTT_UNPROT_UNMAP 12a4c7000 18fe00000 18fe10000 0 0 > RMI_ERROR_RTT 2 0 0 0 0
>> SMC_RMI_REC_CREATE 12a4c7000 12a392000 12a391000 > RMI_INCOMPLETE 0 40 0 0
>> SMC_RMI_OP_MEM_DONATE 0 106406098 10 > RMI_INCOMPLETE 10 0 0 0
>> SMC_RMI_OP_CONTINUE 0 0 > RMI_SUCCESS 0 0
>> SMC_RMI_REALM_ACTIVATE 12a4c7000 > RMI_SUCCESS
>> Unhandled write S2_0_C0_C2_2
>> SMC_RMI_RTT_DATA_MAP 12a4c7000 8ffb4000 8ffb5000 1 129f1d004 > RMI_INCOMPLETE 0 0 0 1
>> SMC_RMI_OP_CONTINUE 0 0 > RMI_SUCCESS 8ffb5000 0
>> PSCI_84000000 0 0 0 0 0 0 0 > 10001 0 0 0
>> PSCI_84000006 0 0 0 0 0 0 0 > ffffffffffffffff 0 0 0
>> PSCI_8400000a 80000000 0 0 0 0 0 0 > 0 0 0 0
>> SMC_80000000 0 0 0 0 0 0 0 > 10002 0 0 0
>> SMC_84000050 1 0 0 ffffacd67d1e8000 ffffacd67d124000 0 0 > ffffffffffffffff 0 0 0
>> SMC_80000001 80000002 ffff 0 ffffacd67d1e8000 ffffacd67d124000 ffffacd67cd5d000 ffffacd67cd5dd38 > ffffffffffffffff 0 0 0
>> PSCI_8400000a c4000001 0 0 0 0 0 0 > 0 0 0 0
>> PSCI_8400000a c4000012 0 0 0 0 0 0 > ffffffffffffffff 0 0 0
>> PSCI_8400000a c4000015 0 0 0 0 0 0 > ffffffffffffffff 0 0 0
>> SMC_8600ff01 ffffacd67cfc0740 10002 0 0 0 0 ffffacd67cfb3cc8 > ffffffffffffffff 0 0 0
>> SMC_RSI_VERSION 10000 > RSI_SUCCESS 10000 10001
>> SMC_RSI_REALM_CONFIG 81385000 > RSI_SUCCESS
>> SMC_RSI_IPA_STATE_SET 80000000 90000000 1 0
>> SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
>> SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
>> SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
>> SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
>> SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
>> SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
>> <... the last message repeats ...>
>>
>> Thanks,
>> Gavin
>>
>>
>
More information about the linux-arm-kernel
mailing list