[PATCH] arm64: mte: switch GCR_EL1 on task switch rather than entry/exit
Catalin Marinas
catalin.marinas at arm.com
Mon Jul 5 07:17:35 PDT 2021
On Fri, Jul 02, 2021 at 11:39:33AM -0700, Peter Collingbourne wrote:
> On Fri, Jul 2, 2021 at 10:44 AM Catalin Marinas <catalin.marinas at arm.com> wrote:
> > On Thu, Jul 01, 2021 at 08:19:22PM -0700, Peter Collingbourne wrote:
> > > Accessing GCR_EL1 and issuing an ISB can be expensive on some
> > > microarchitectures. To avoid taking this performance hit on every
> > > kernel entry/exit, switch GCR_EL1 on task switch rather than
> > > entry/exit.
> >
> > Is it the ISB that's causing issues or the MRS/MSR as well? I think we
> > can avoid the ISB when PtrAuth is enabled by shuffling the entry code a
> > bit. We can also simplify the mte_set_gcr macro to avoid an MRS.
>
> This was the first thing that I tried on our hardware:
>
> diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
> index 7312eafec946..8699ab28a924 100644
> --- a/arch/arm64/kernel/entry.S
> +++ b/arch/arm64/kernel/entry.S
> @@ -204,7 +204,6 @@ alternative_else_nop_endif
> ldr_l \tmp, gcr_kernel_excl
>
> mte_set_gcr \tmp, \tmp2
> - isb
> 1:
> #endif
> .endm
> @@ -277,13 +276,13 @@ alternative_if ARM64_HAS_ADDRESS_AUTH
> orr x0, x0, SCTLR_ELx_ENIA
> msr sctlr_el1, x0
> 2:
> - isb
> alternative_else_nop_endif
> #endif
>
> apply_ssbd 1, x22, x23
>
> mte_set_kernel_gcr x22, x23
> + isb
>
> scs_load tsk, x20
> .else
>
> However, on most of the cores this led to only around half of the
> performance improvement of the patch that I sent. Which is somewhat
> surprising, but it is what it is.
BTW, can you also modify mte_set_kernel_gcr to only do a write to the
GCR_EL1 register rather than a read-modify-write?
--
Catalin
More information about the linux-arm-kernel
mailing list