[PATCH 6/7] arm64: start using 'asm goto' for put_user() when available

[Nathan Chancellor]

Hi Linus,

On Mon, Jun 10, 2024 at 01:48:20PM -0700, Linus Torvalds wrote:
> This generates noticeably better code with compilers that support it,
> since we don't need to test the error register etc, the exception just
> jumps to the error handling directly.
> 
> Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
> ---
>  arch/arm64/include/asm/uaccess.h | 77 +++++++++++++++++++-------------
>  1 file changed, 46 insertions(+), 31 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
> index 23c2edf517ed..4ab3938290ab 100644
> --- a/arch/arm64/include/asm/uaccess.h
> +++ b/arch/arm64/include/asm/uaccess.h
> @@ -294,29 +294,41 @@ do {									\
>  	} while (0);							\
>  } while (0)
>  
> -#define __put_mem_asm(store, reg, x, addr, err, type)			\
> +#ifdef CONFIG_CC_HAS_ASM_GOTO

This symbol was eliminated two years ago with commit a0a12c3ed057 ("asm
goto: eradicate CC_HAS_ASM_GOTO") since all supported compilers have
support for it.

> +#define __put_mem_asm(store, reg, x, addr, label, type)			\
> +	asm goto(							\
> +	"1:	" store "	" reg "0, [%1]\n"			\
> +	"2:\n"								\
> +	_ASM_EXTABLE_##type##ACCESS_ZERO(1b, %l2)			\
> +	: : "rZ" (x), "r" (addr) : : label)
> +#else
> +#define __put_mem_asm(store, reg, x, addr, label, type) do {		\
> +	int __pma_err = 0;						\
>  	asm volatile(							\
>  	"1:	" store "	" reg "1, [%2]\n"			\
>  	"2:\n"								\
>  	_ASM_EXTABLE_##type##ACCESS_ERR(1b, 2b, %w0)			\
> -	: "+r" (err)							\
> -	: "rZ" (x), "r" (addr))
> +	: "+r" (__pma_err)						\
> +	: "rZ" (x), "r" (addr));					\
> +	if (__pma_err) goto label;					\
> +} while (0)
> +#endif

Cheers,
Nathan