[PATCH 0/2] arm64: tlb: Optimize ARM64_WORKAROUND_REPEAT_TLBI

[Mark Rutland]

Hi all,

Some Arm partners have complained that the overhead of
ARM64_WORKAROUND_REPEAT_TLBI is too large, and despite the relevant
errata being categorized as "rare", they still want to use the
workaround in some deployments.

For historical reasons, the current workaround is far stronger (and
consequently far more expensive) than necessary. In part, the SDENs had
somewhat misleading descriptions, which have recently been clarified:

* Arm Cortex-A76 erratum #1286807
  SDEN v33: https://developer.arm.com/documentation/SDEN-885749/33-0/

* Arm Cortex-A55 erratum #2441007
  SDEN v16: https://developer.arm.com/documentation/SDEN-859338/1600/

* Arm Cortex-A510 erratum #2441009
  SDEN v19: https://developer.arm.com/documentation/SDEN-1873351/1900/

Patch 1 allows the __TLBI*() helpers to generate XZR as an argument.
I've split this out as its own patch to make bisection easier in case we
see any problems due to incorrect trap+emulation handling of XZR.
Otherwise this shouldn't have any functional change.

Patch 2 is the actual optimization, spelled out in detail in the commit
message. The gist is that it's not necessary to duplicate every
individual TLBI, and it's sufficient to have a single arbitrary TLBI;DSB
after any number of batched TLBIs;DSB.

As mentioned in the commit message for patch 2, this results in fewer
alternatives and better code generation whenever
ARM64_WORKAROUND_REPEAT_TLBI is built into the kernel, so it's a
(trivial) win on hardware that isn't affected by the relevant errata.

Mark.

Mark Rutland (2):
  arm64: tlb: Allow XZR argument to TLBI ops
  arm64: tlb: Optimize ARM64_WORKAROUND_REPEAT_TLBI

 arch/arm64/include/asm/tlbflush.h | 63 ++++++++++++++++++-------------
 arch/arm64/kernel/sys_compat.c    |  2 +-
 arch/arm64/kvm/hyp/nvhe/mm.c      |  2 +-
 arch/arm64/kvm/hyp/nvhe/tlb.c     |  8 ++--
 arch/arm64/kvm/hyp/pgtable.c      |  2 +-
 arch/arm64/kvm/hyp/vhe/tlb.c      | 10 ++---
 6 files changed, 49 insertions(+), 38 deletions(-)

-- 
2.30.2