MACsec interface eventually stops receiving packets when hardware offload is enabled

[Christian Schürmann]

Hi everyone,

I'm trying to get MACsec with hardware offloading to work reliably. I'm using two NVIDIA ConnectX-7 MCX713106AC-CEAT (latest firmware 28.47.1088), kernel 6.19.0-301.fc44.x86_64, and wpa_supplicant 273e28b (latest from main branch as of writing).

wpa_supplicant -i ens3f0np0 -Dmacsec_linux -c wpa_supplicant.conf

with the configuration file reading:

ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=wheel
eapol_version=3
ap_scan=0
fast_reauth=1
network={
        key_mgmt=NONE
        eapol_flags=0
        macsec_policy=1
        mka_ckn=DEAD
        mka_cak=e9720c69644c93f4386a0c471cca409183686b2b3ec22904a3963238586ce3ac
        mka_priority=1
        macsec_csindex=0
        macsec_integ_only=0
        macsec_offload=2
}

(mka_priority=2 on the second host)
All seems to work fine:

# ip macsec show
23: macsec0: protect on validate strict sc off sa off encrypt on send_sci on end_station off scb off replay off 
    cipher suite: GCM-AES-128, using ICV length 16
    TXSC: 58a2e1fa88a80001 on SA 0
        0: PN 1, state on, key 3c939aebc18a3ff54253344001000000
    RXSC: 58a2e1fa88980001, state on
        0: PN 1, state on, key 3c939aebc18a3ff54253344001000000
    offload: mac 

However, after exactly 3.221.225.471 received packets (i.e. 0xC0000000 - 1 or 75% of 2^32-1) as reported by:

ethtool -S ens3f0np0 | grep "macsec_rx_pkts:"

all communication on the MACsec interface of the receiving host stops, e.g. "tcpdump -i macsec0" shows absolutely no output any more.
Terminating wpa_supplicant and starting it again let's me transfer another 0xC0000000 - 1 packets.

I'm not entirely sure this is on topic here but I'd guess it has something so do with PN exhaustion and SAK rekey not working correctly. Can I only use the MACsec Key Agreement protocol when hardware offloading is disabled?
When hardware offloading is disabled this does not happen and SAK rekeying seems to work correctly:

...
KaY: My OLPN exhaustion
KaY: Life time has not elapsed since prior SAK distributed

Tangentially related: I've tried using the Extended Packet Numbering via gcm-aes-xpn as described in https://man7.org/linux/man-pages/man8/ip-macsec.8.html#EXTENDED_PACKET_NUMBER_NOTES
This does work but also dies eventually (traffic stops, tcpdump shows no more packets being received).

Any pointers are greatly appreciated.

Cheers,
Christian