[PATCH v2] arm64: tlbflush: Don't broadcast if mm was only active on local cpu

[Yao Yuan]

On Fri, Oct 30, 2026 at 12:18:39PM +0800, Catalin Marinas wrote:
> On Fri, Oct 30, 2026 at 11:20:40AM +0000, Will Deacon wrote:
> > On Wed, Oct 28, 2026 at 02:47:19PM +0000, Usama Anjum wrote:
> > > On 23/05/2026 2:47 pm, Linu Cherian wrote:
> > > > From: Ryan Roberts <ryan.roberts at arm.com>
> > > >
> > > > There are 3 variants of tlb flush that invalidate user mappings:
> > > > flush_tlb_mm(), flush_tlb_page() and __flush_tlb_range(). All of these
> > > > would previously unconditionally broadcast their tlbis to all cpus in
> > > > the inner shareable domain.
> > > >
> > > > But this is a waste of effort if we can prove that the mm for which we
> > > > are flushing the mappings has only ever been active on the local cpu. In
> > > > that case, it is safe to avoid the broadcast and simply invalidate the
> > > > current cpu.
> > > >
> > > > So let's track in mm_context_t::active_cpu either the mm has never been
> > > > active on any cpu, has been active on more than 1 cpu, or has been
> > > > active on precisely 1 cpu - and in that case, which one. We update this
> > > > when switching context, being careful to ensure that it gets updated
> > > > *before* installing the mm's pgtables. On the reader side, we ensure we
> > > > read *after* the previous write(s) to the pgtable(s) that necessitated
> > > > the tlb flush have completed. This guarrantees that if a cpu that is
> > > > doing a tlb flush sees it's own id in active_cpu, then the old pgtable
> > > > entry cannot have been seen by any other cpu and we can flush only the
> > > > local cpu.
> > > >
> > > > Signed-off-by: Ryan Roberts <ryan.roberts at arm.com>
> > > > Reviewed-by: Catalin Marinas <catalin.marinas at arm.com>
> > > > Tested-by: Huang Ying <ying.huang at linux.alibaba.com>
> > > > [linu.cherian at arm.com: Adapted for v7.1 flush tlb API changes]
> > > > Signed-off-by: Linu Cherian <linu.cherian at arm.com>
> > > > ---
> > > > Changelog from RFC v1:
> > > > - Adapted for v7.1 flush tlb API changes
> > > >   No changes in core logic
> > > > - Collected Rb and Tb tags
> > > > - lat_mmap benchmark showed dsb(ishst) performs better than dsb(ish),
> > > >   hence retained dsb(ishst) in flush_tlb_user_pre
> > > >
> > > >
> > > > Testing with 7.1-rc4 :
> > > > +-----------------------+---------------------------------------------------+-------------+
> > > > | Benchmark             | Result Class                                      |  Improvement|
> > > > +=======================+===================================================+=============+
> > > > | perf/syscall          | fork (ops/sec)                                    |   (I) 3.25% |
> > > > +-----------------------+---------------------------------------------------+-------------+
> > > > | pts/memtier-benchmark | Protocol: Redis Clients: 100 Ratio: 1:5 (Ops/sec) |   (I) 2.70% |
> > > > | 			| Protocol: Redis Clients: 100 Ratio: 5:1 (Ops/sec) |   (I) 2.13% |
> > > > +-----------------------+---------------------------------------------------+-------------+
> > >
> > > I tested this patch using a synthetic Firecracker benchmark on a 192-CPU
> > > arm64 host, with Linux v7.3 on both the host and guests. Each worker
> > > repeatedly creates a fresh 1-vCPU microVM, boots Ubuntu, verifies an echo
> > > response over the serial console, and shuts the guest down, waiting for
> > > Firecracker to exit before starting the next VM. Each guest has 256 MiB of
> > > RAM, with two VM slots per workload CPU.
> > >
> > > With 4 KiB guests, the unpatched workload stops scaling between 64 and 96
> > > CPUs: throughput falls from approximately 68 to 65 completed lifecycles/s
> > > despite the additional CPUs.
> > >
> > > Applying the patch to the guest substantially improves throughput at higher
> > > concurrency. At 128 workload CPUs, I measured:
> > >
> > >   Guest pages  Unpatched  Guest patched only  Host patched only
> > >   -----------  ---------  ------------------  -----------------
> > >   4 KiB        73.83/s    108.55/s (+47.0%)    74.09/s (+0.4%)
> > >   64 KiB       86.46/s    105.04/s (+21.5%)    90.15/s (+4.3%)
> > >
> > > The host uses 4 KiB pages throughout. The host-only difference with 4 KiB
> > > guests is small enough that I would regard it as essentially unchanged.
> >
> > Sorry, but I haven't understood the setup here. If each guest ("microVM")
> > only has 1 vCPU, then why does this patch have any effect in the "guest
> > patched only" case?
>
> I wonder whether it's the repeat TLBI workaround just doing more DVM
> traffic. Turned off when no broadcast is needed.
>
> > Additionally, the non-broadcast TLB invalidation will
> > be upgraded to broadcast by the hypervisor.
>
> I have a hack that makes things even faster (opportunistic
> HCR_EL2.FB=0). I may post it once I understand the implications
> properly.

Hi Catalin,

I tested HCR_EL.{FB,BSU_IS} = 0 w/ this patch on guest
kernel before, here is my result:

The in VM micro benchmark is mprotect() on one 4K private
page, change READ <-> WRITE in total 1000000 times.

The testing runs on system w/ 2 physical NUMAs, each NUMA
has 62 physical cpus.

The VM has 2 virtual NUMAs, 1:1 map to the physical NUMA
above, the vcpu 1:1 pin on pcpu, total 120 vcpus in VM
(yes not 124, it's 120), QEMU's other threads are pinned
to 2 free cpus in each NUMA.

The benchmark runs on 120 vcpu at the same time, the
average cost time is used as performance indicator.

W/o this patch on guest kernel:
+------------------------+------------------------+
| HCR_EL.FB and BSU_IS=0 | HCR_EL.FB and BSU_IS=1 |
+------------------------+------------------------+
| average: 26.227070s    | average: 26.182552s    |
+------------------------+------------------------+

W/ this patch on guest kernel:
+------------------------+-----------------------+
| HCR_EL.FB and BSU_IS=0 | HCR_EL.FB and BSU_IS=1|
+------------------------+-----------------------+
|  average 0.478060s     | average 25.836823s    |
+------------------------+-----------------------+

IIUC the improvement should come from the inside guest
non-broadcast TLBi by this patch + hardware doesn't
"upgrade" such non-broadcast TLBi and barrier to broadcast
scope in such across NUMA scenario, the later is controlled
by FB and BSU_IS for guest.

>
> --
> Catalin