[LSF/MM/BPF TOPIC] NVMe over MPTCP: Multi-Fold Acceleration for NVMe over TCP in Multi-NIC Environments

[Ming Lei]

Hi Geliang,

Looks one interesting topic!

On Thu, Jan 29, 2026 at 12:13:25PM +0800, Geliang Tang wrote:
> As one of the MPTCP upstream developers, I'm recently working on adding
> MPTCP support to 'NVMe over TCP'. This approach achieves a multi-fold
> performance improvement over using standard TCP. The implementation and
> testing phases are largely complete. The code is currently in the RFC
> stage and has undergone several rounds of discussion and iteration on
> the MPTCP mailing list [1]. It will be sent to the NVMe mailing list
> shortly.
> 
> 1. Introduction to MPTCP
> 
> Multipath TCP (MPTCP), standardized in RFC 8684, represents a major
> evolution of the TCP protocol. It enables a single transport connection
> to utilize multiple network paths simultaneously, providing benefits in
> redundancy, resilience, and bandwidth aggregation. Since its
> introduction in Linux kernel v5.6, it has become a key technology for
> modern networking, particularly in multi-NIC environments.
> 
> On a supported system such as Linux, an MPTCP socket is created by
> specifying the IPPROTO_MPTCP protocol in the socket() system call:
> 
> 	int fd = socket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP);
> 
> This creates a socket that appears as a standard TCP socket to the
> application but uses the MPTCP protocol stack underneath.
> 
> For more details, please visit the project website: https://mptcp.dev.
> 
> 2. Implementation
> 
> 'NVMe over TCP' establishes multiple TCP connections between the target
> and host for data transfer. This includes one admin queue connection
> for management traffic and multiple I/O queue connections for data
> traffic, with the number typically scaling with available CPU cores.
> While these multiple TCP connections (using the same IP address but
> different ports) help distribute computational load across CPUs, all
> data traffic still flows through a single network interface card (NIC),
> even in multi-NIC environments.
> 
> The 'NVMe over MPTCP' solution enhances 'NVMe over TCP' by replacing
> the multiple TCP connections with multiple MPTCP connections, leaving
> other mechanisms unchanged. Internally, each MPTCP connection can
> establish multiple subflows based on the number of configured NICs.
> This distributes data traffic across all available NICs, thereby
> increasing aggregate transmission speed.

NVMe supports multipath, which can apply load balance or sort of algorithm
to maximize network link/bandwidth too.

Maybe you can compare mptcp with multipath in this viewpoint.



Thanks,
Ming