Question about the "TLBs and I-cache are private to each vCPU" guarantee with VTTBR_EL2.CnP

[Tangnianyao]

On 7/6/2026 16:44, Marc Zyngier wrote:
> On Mon, 06 Jul 2026 09:25:46 +0100,
> Tangnianyao <tangnianyao at huawei.com> wrote:
>>
>>
>> On 7/6/2026 15:25, Marc Zyngier wrote:
>>> On Mon, 06 Jul 2026 04:30:30 +0100,
>>> Tangnianyao <tangnianyao at huawei.com> wrote:
>>>>
>>>> On 7/6/2026 1:28, Wei-Lin Chang wrote:
>>>>> Hi,
>>>>>
>>>>> Let me try to answer this:
>>>>>
>>>>> On Sat, Jul 04, 2026 at 03:45:56PM +0800, Tangnianyao wrote:
>>>>>> Hi, all
>>>>>>
>>>>>> I'm trying to understand the TLB and I-cache invalidation in
>>>>>> `kvm_arch_vcpu_load()` that is intended to "guarantee that both TLBs and
>>>>>> I-cache are private to each vCPU".
>>>>>>
>>>>>> As I understand it, when `VTTBR_EL2.CnP == 1`, `__kvm_flush_cpu_context()`
>>>>>> only performs a local TLB and I-cache invalidation, which does not seem
>>>>>> sufficient to guarantee that property.
>>>>>>
>>>>>> In fact, even if the invalidation were extended to the Inner Shareable
>>>>>> domain, it still seems difficult to guarantee “TLBs and I-cache are
>>>>>> private to each vCPU”, when `VTTBR_EL2.CnP == 1`, as long as multiple
>>>>>> vCPUs from the same VM may be running concurrently on different PEs.
>>>>> I think you have missed that when 2 stages are involved, both stages
>>>>> have to set CnP == 1 in order to share TLB entries (Arm ARM R_ZVRZW).
>>>>> So if TLB entry sharing happens, the guest kernel must have allowed it
>>>>> in the first place (by setting TTBR0/1_EL1.CnP == 1), hence accidental
>>>>> sharing that you are worried about won't happen.
>>>>>
>>>>> __kvm_flush_cpu_context() is solving problems that occur when multiple
>>>>> vCPUs of a VM are multiplexed on a single physical CPU.
>>>> Thanks for you answer.
>>>>
>>>> If guest kernel allow TLB shared across CPUs by setting TTBR0/1_EL1.CnP == 1,
>>>> does kvm still need to guarantee that TLBs are private to each vCPU?
>>> Yes, because there is nothing that describes which physical CPUs
>>> actually share TLBs. So the only possible course of action is to
>>> ignore what the guest says and fallback to something that is safe.
>>>
>>>>>> So I have two questions:
>>>>>>
>>>>>> 1. What is the rationale behind the comment that "guarantee that both TLBs
>>>>>> and I-cache are private to each vCPU"?
>>>>> I assume you are asking why keeping both TLBs and I-cache private per
>>>>> each vCPU is required. The fundamental answer is that each physical CPU
>>>>> is expected to have its own TLB and I-cache, so we must uphold that
>>>>> property for vCPUs as well. vCPUs can be scheduled on the same physical
>>>>> CPU, and use the same physical TLB/I-cache, obviously, so extra
>>>>> invalidations need to be done.
>>>> Let's assume that both Stage-1 CnP and Stage-2 CnP are enabled.
>>>>
>>>> As I understand it, the architecture permits TLB to be shared by multiple
>>>> PEs within an Inner Shareable domain. Right?
>>>>
>>>> If an implementation allows TLB entries to be shared in this way, it seems
>>>> that the current invalidation performed by kvm would no longer be sufficient
>>>> to guarantee that TLBs are private to each vCPU.
>>> Care to explain why?
>>>
>>> The core assumption is that a TLBI take effect on all the PEs the TLB
>>> is shared with. If this doesn't work, then CnP is unusable, because it
>>> is then impossible to guarantee that a translation will be refetched
>>> (you could always hit in another PEs TLBs). Such an implementation
>>> would be terminally broken.
>>>
>>> 	M.
>>>
>> For example:
>> Sharing the TLB between the two SMT threads of the same physical core can
>> reduce hardware cost while increasing the effective TLB coverage.
> I have a precise idea of what TLB sharing can achieve.
>
>> A local TLBI take effect on the whole TLB shared by the two SMT threads,
>> with the sharing enabled by CnP.
>>
>> In this scenario, enabling CnP in KVM appears to break the guarantee that
>> TLBs are private to each vCPU when multiple vCPUs of the same VM run
>> concurrently on different SMT threads of the same PE.
> Please stop inventing your own terminology. A PE *is* a thread in the
> architecture. There is no such thing as "threads of the same PE".
>
> But more to the point: if TLBI invalidates the relevant TLBs for all
> the PEs that share them, *why* isn't the current KVM behaviour not
> enough to ensure that the vcpu will not hit old TLBs that are there as
> a result of a vcpu having run there previously?
>
> Please explain.
>
> 	M.
>

Two SMT threads(PE0,PE1) on the same physical core share TLB.

VM0 has 2 vcpus, vcpu0 and vcpu1 that share all architectural context
except the address translation context.

Vcpu0 may observe TLB entries that are supposed to be private to vcpu1
in the following case:

    PE0(core0,smt0)         PE1(core0,smt1)
vcpu0 load           
vcpu0 va->pa0        
vcpu0 put            
                                       vcpu1 load
                                       vcpu1 flush local tlb
                                       vcpu1 modify desc to va->pa1
vcpu0 load
vcpu0 hit *va->pa1*


Thanks
Nianyao Tang