[PATCH 2/5] arm64: atomics lse: define SUBs in terms of ADDs

[Will Deacon]

On Fri, Dec 10, 2021 at 03:14:07PM +0000, Mark Rutland wrote:
> The FEAT_LSE atomic instructions include atomic ADD instructions
> (`stadd*` and `ldadd*`), but do not include atomic SUB instructions, so
> we must build all of the SUB operations using the ADD instructions. We
> open-code these today, with each SUB op implemented as a copy of the
> corresponding ADD op with a leading `neg` instruction in the inline
> assembly to negate the `i` argument.
> 
> As the compiler has no visibility of the `neg`, this leads to less than
> optimal code generation when generating `i` into a register. For
> example, __les_atomic_fetch_sub(1, v) can be compiled to:
> 
> 	mov     w1, #0x1
> 	neg     w1, w1
> 	ldaddal w1, w1, [x2]
> 
> This patch improves this by replacing the `neg` with negation in C
> before the inline assembly block, e.g.
> 
> 	i = -i;
> 
> This allows the compiler to generate `i` into a register more optimally,
> e.g.
> 
> 	mov     w1, #0xffffffff
> 	ldaddal w1, w1, [x2]
> 
> With this change the assembly for each SUB op is identical to the
> corresponding ADD op (including barriers and clobbers), so I've removed
> the inline assembly and rewritten each SUB op in terms of the
> corresponding ADD op, e.g.
> 
> | static inline void __lse_atomic_sub(int i, atomic_t *v)
> | {
> | 	__lse_atomic_add(-i, v);
> | }
> 
> For clarity I've moved the definition of each SUB op immediately after
> the corresponding ADD op, and used a single macro to create the RETURN
> forms of both ops.
> 
> This is intended as an optimization and cleanup.
> There should be no functional change as a result of this patch.
>
> Signed-off-by: Mark Rutland <mark.rutland at arm.com>
> Cc: Boqun Feng <boqun.feng at gmail.com>
> Cc: Catalin Marinas <catalin.marinas at arm.com>
> Cc: Peter Zijlstra <peterz at infradead.org>
> Cc: Will Deacon <will at kernel.org>
> ---
>  arch/arm64/include/asm/atomic_lse.h | 180 +++++++++-------------------
>  1 file changed, 58 insertions(+), 122 deletions(-)

Great diffstat!

Acked-by: Will Deacon <will at kernel.org>

Will