[RFC] KVM: arm64: Don't force broadcast tlbi when guest is running

Marc Zyngier maz at kernel.org
Tue Nov 3 08:26:18 EST 2020


On 2020-11-03 11:31, Shaokun Zhang wrote:

[...]

>>> Further, we consider restricting tlbi broadcast range and make
>>> tlbi more accurate.
>>> One scheme is replacing tlbiis with ipi + tlbi + HCR_EL2.FB=0.
>> 
>> Ah. That old thing again. No, thank you. The right thing to do
>> is to build CPUs and interconnects that properly deals with
>> IS invalidations by not sending DVM messages to places where
>> things don't run (snoop filters?). I also doubt the arm64
>> maintainers would be sympathetic to the idea anyway.
> 
> We also do the same test on intel 6248 and get better result,
> less performance drop in multi-vm case compare to single vm.
> Intel use ipi + flush tlb scenario, do you think this scenario is
> meaningful on Arm architect hardware?

I can't see how you can compare the two architectures: one only
has local TLB invalidation and *must* rely on IPI, while the other
supports broadcast TLB invalidation which can lead to better performance
and lower latencies if the HW is well designed.

>>> Consider I-cache invalidation, KVM also needs to invalid icache
>>> when moving vcpu to a new pcpu.
>>> Do we miss any cases that need HCR_EL2.FB == 1?
>>> Eventually we expect HCR_EL2.FB == 0 if it is possible.
>> 
>> I have no intention to ever set FB to 0, as this would resu> in 
>> over-invalidation in the general case, and worse performance
> 
> The reason that we want to disable FB is that IPI solution
> is needed if it is accepted.

Let me repeat it: setting FB=0 may improve bad implementations, but
has will have a negative impact on *good* HW implementations.
It results in extra TLB and I-cache invalidations in the general case.

You are only considering your narrow use case where vcpus are pinned
to physical CPUs, where you indeed don't need invalidations to be
broadcast. But that's not everybody has your use case.

*IF* you come up with a way to find that a vcpu is pinned and cannot
be migrated, then that's an avenue for optimisation. But otherwise, no.

         M.
-- 
Jazz is not dead. It just smells funny...



More information about the linux-arm-kernel mailing list