[PATCH v4 0/3] nvme: introduce the service-time iopolicy
Guixin Liu
kanie at linux.alibaba.com
Mon Oct 26 04:10:15 PDT 2026
The queue-depth iopolicy counts requests, so a 128K I/O weighs the same
as a 4K one, and it cannot express that one path is simply faster than
another. This series adds a service-time iopolicy for NVMe native
multipath, adapted from dm-ps-service-time: it picks the path with the
lowest estimated service time, in_flight_bytes / relative_throughput,
so balancing follows actual bytes in flight and per-path throughput
weights.
Weights live in a per-path relative_throughput sysfs attribute (0-100,
default 1, 0 makes the path a standby). Patch 3 documents the new
attributes in the stable sysfs ABI; patch 1 carries the benchmark data
in its commit log.
This version is rebased onto the nvme-7.4 branch of
git://git.infradead.org/nvme.git.
Changes since v3:
- Rebased onto nvme-7.4, which adds the iopolicies and path_state
attributes and the discovery-subsystem "none" handling.
- The iopolicies ABI entry now lists service-time too. nvme-7.4
already documents the attribute (with a hardcoded list of the
three older policies); patch 3 extends that list along with the
iopolicy accepted values.
Changes since v2:
- in_flight_bytes no longer leaks on failover: the accounted bytes
are now released in nvme_failover_req() before blk_steal_bios()
clears the request data length. Before, every failover (ANA
transition, path error, queue_rq unwind) permanently left the
request's bytes in the counter, skewing path selection until a
controller reset. Reproduced with an ANA transition during load on
the NVMe-oF/TCP test setup: the counter kept 512K of phantom bytes
after all I/O completed; with the fix it returns to zero.
- The release clamps at zero instead of a plain subtraction. A
request requeued across a controller reset is accounted against the
pre-reset counter but completes after nvme_mpath_init_identify()
re-zeroed it, and the plain subtraction would wrap negative, making
the comparator treat the path as pathologically overloaded. This
mirrors atomic_dec_if_positive() in the queue-depth accounting.
- The comparison operates on u64 instead of size_t, so the byte
counts and the shift guard keep their magnitude on 32-bit builds,
where size_t would truncate the atomic64 reads.
- Added "service-time" to the accepted values of the subsystem
iopolicy ABI entry and to the policies listed by the iopolicies
ABI entry.
No public v3 was posted; this version folds the review feedback from
the v2 discussion and a rebase onto nvme-7.4.
On the adaptive I/O policy dependency Hannes raised during v1: that
series has become the latency I/O policy and is still under
revision. The two are complementary: the latency policy infers path
speed from observed completions, service-time takes configured
weights and byte counts.
Guixin Liu (3):
nvme-multipath: implement service-time iopolicy
nvme-multipath: document service-time iopolicy
nvme: document the service-time iopolicy sysfs attributes
Documentation/ABI/stable/sysfs-nvme | 35 +++-
Documentation/admin-guide/nvme-multipath.rst | 31 +++-
drivers/nvme/host/multipath.c | 189 ++++++++++++++++++-
drivers/nvme/host/nvme.h | 6 +
drivers/nvme/host/sysfs.c | 7 +-
5 files changed, 256 insertions(+), 12 deletions(-)
v2 of this series:
https://lore.kernel.org/r/20261019075607.808947-1-kanie@linux.alibaba.com
v1 of this series:
https://lore.kernel.org/r/20260617114602.2224074-1-kanie@linux.alibaba.com
latency I/O policy v7:
https://lore.kernel.org/r/20260809100825.2014133-1-nilay@linux.ibm.com
--
2.43.7
More information about the Linux-nvme
mailing list