[PATCH 0/2] arm64: csum: Add fused copy and Internet checksum

[David Laight]

On Sun, 1 Nov 2026 14:33:49 +0100
Demian Shulhan <demyansh at gmail.com> wrote:

> Hi David,
> 
> Sorry for the delay, I was in the middle of a company change.
> 
> You are right about SG and TX csum, but only when the traffic goes
> straight to a NIC that offloads them. There is another side: software
> GSO (skb_segment() with !sg && !csum) hits this path on 100% of TCP
> payload over WireGuard (wg_xmit() -> skb_gso_segment(skb, 0)), and on
> drivers without SG/TX csum (u_ether/f_ncm, cdc_ncm, mt76, rtw88/89,
> ath9k).
> 
> So in general the software copy+csum path is still alive.

I was partially worried that someone had just looked at the code and
noticed that arm didn't have the code to do checksum+copy and decided
to write it.

David

> 
> On the other hand, NEON is useless here. skb_segment() splits GSO
> packets into 1-MSS segments before copying, so on these paths calls
> never exceed 1448 bytes (1368 on WireGuard). At that size
> kernel_neon_begin()/end() costs more than NEON saves, on real hardware
> v1 was ~3% slower in cycles/byte than the scalar loop on the slower
> cores. So that part can go.
> 
> I would prefer to keep only the scalar version of the fused copy+csum.
> It reads the data once instead of twice, and on real Armv9 hardware it
> cuts end-to-end cycles/byte for bulk TCP over WireGuard by 3-12%
> depending on the core (+6-8% throughput). It uses only general-purpose
> registers, so there is no FPSIMD state to save and it is safe in any
> context.
> 
> If you are OK with the scalar path, I can send v2 with NEON dropped,
> the scalar path only, and the KUnit fix for the x86 zero-length case
> Jakub reported.
> 
> Thanks,
> Demian
> 
> нд, 27 вер. 2026 р. о 19:44 David Laight <david.laight.linux at gmail.com> пише:
> >
> > On Sun, 27 Sep 2026 15:17:56 +0200
> > Demian Shulhan <demyansh at gmail.com> wrote:
> >  
> > > arm64 currently uses the generic csum_partial_copy_nocheck(), which
> > > performs memcpy() followed by a second pass for csum_partial(). This
> > > double pass exerts unnecessary pressure on the L1 cache.  
> >
> > Which workload actually needs this?
> > Most modern ethernet MAC support checksum setting on transmit and
> > checking on receive.
> > So the software checksum shouldn't be needed very often.
> >
> > IIRC there is also code to defer UDP checksum validation until the
> > copy_to_user().
> > I'd bet (a few pints of beer) that the complication this adds isn't
> > actually worth while.
> > Even Linus can't remember why it was done, my guess is it improved the
> > performance of the userspace NFS (over UDP) daemon that would be
> > doing 8k UDP send/receive (fragmented by IP).
> >
> > There is certainly still code to checksum data during copy_from_user()
> > in send().
> > Last time I looked I couldn't see why send on TCP sockets didn't go through it.
> > On x86 (in particular) copies can be done far faster than ones that
> > include a checksum.
> >
> > David
> >  
> > >
> > > Replace it with a single-pass implementation. The new implementation
> > > provides a general-purpose register path for short buffers and atomic
> > > contexts, and a kernel-mode NEON path for lengths >= 1024 bytes.
> > >
> > > Measured in-kernel on an Ampere Altra (Neoverse-N1):
> > > - Scalar path: 1.2x-1.6x faster for lengths < 1024 bytes.
> > > - NEON path: 1.2x faster at 1024 bytes, scaling up to 1.6x-1.8x at
> > >   4096 bytes.
> > > On Apple M-series cores, gains are 1.3-1.7x below 1024 bytes and
> > > 1.6-2.4x above. No length or alignment regresses on either
> > > microarchitecture.
> > >
> > > Patch 1 implements the fused routines and the dispatcher.
> > > Patch 2 adds KUnit test coverage for the new API and internal paths.
> > >
> > > Tested: in-kernel benchmark module on Neoverse-N1 with both
> > > implementations cross-checked (0 mismatches); KUnit suite under QEMU
> > > (with/without KASAN, with PREEMPT_RT), exhaustive and random userspace
> > > testing of both routines against a naive reference with PROT_NONE guard
> > > pages, gcc 13 and clang 18 W=1 builds, checkpatch --strict.
> > >
> > > Demian Shulhan (2):
> > >   arm64: csum: Add fused copy and Internet checksum
> > >   lib/tests: checksum: Add KUnit tests for csum_partial_copy_nocheck()
> > >
> > >  arch/arm64/include/asm/checksum.h |   3 +
> > >  arch/arm64/lib/Makefile           |   7 +-
> > >  arch/arm64/lib/csum-copy-neon.c   | 168 +++++++++++
> > >  arch/arm64/lib/csum-copy.c        | 108 +++++++
> > >  arch/arm64/lib/csum-copy.h        |  84 ++++++
> > >  arch/arm64/lib/csum.c             |  54 ++++
> > >  lib/Kconfig.debug                 |  10 +
> > >  lib/tests/checksum_kunit.c        | 463 ++++++++++++++++++++++++++++++
> > >  8 files changed, 896 insertions(+), 1 deletion(-)
> > >  create mode 100644 arch/arm64/lib/csum-copy-neon.c
> > >  create mode 100644 arch/arm64/lib/csum-copy.c
> > >  create mode 100644 arch/arm64/lib/csum-copy.h
> > >
> > >
> > > base-commit: 93f51579e7df248780214094418f205253383cc5  
> >