[LSF/MM/BPF TOPIC] NVMe over MPTCP: Multi-Fold Acceleration for NVMe over TCP in Multi-NIC Environments

[Nilay Shroff]

On 1/29/26 9:43 AM, Geliang Tang wrote:
> 3. Performance Benefits
> 
> This new feature has been evaluated in different environments:
> 
> I conducted 'NVMe over MPTCP' tests between two PCs, each equipped with
> two Gigabit NICs and directly connected via Ethernet cables. Using
> 'NVMe over TCP', the fio benchmark showed a speed of approximately 100
> MiB/s. In contrast, 'NVMe over MPTCP' achieved about 200 MiB/s with
> fio, doubling the throughput.
> 
> In a virtual machine test environment simulating four NICs on both
> sides, 'NVMe over MPTCP' delivered bandwidth up to four times that of
> standard TCP.

This is interesting. Did you try using an NVMe multipath iopolicy other
than the default numa policy? Assuming both the host and target are multihomed,
configuring round-robin or queue-depth may provide performance comparable
to what you are seeing with MPTCP.

I think MPTCP shall distribute traffic using transport-level metrics such as
RTT, cwnd, and packet loss, whereas the NVMe multipath layer makes decisions
based on ANA state, queue depth, and NUMA locality. In a setup with multiple
active paths, switching the iopolicy from numa to round-robin or queue-depth
could improve load distribution across controllers and thus improve performance.

IMO, it would be useful to test with those policies and compare the results
against the MPTCP setup.

Thanks,
--Nilay