[PATCH v4 0/3] nvme: introduce the service-time iopolicy

[Guixin Liu]

Hi guys,
  All the issues Sashiko raised are addressed in this v4, and it looks
like Sashiko has no further comments on the current version now.
  Please review this series, all comments are welcome. Or if can tell
me this approach is reasonable, I'll keep iterating on the implementation
details.

Best Regards,
Guixin Liu


在 2026/10/26 19:10, Guixin Liu 写道:
> The queue-depth iopolicy counts requests, so a 128K I/O weighs the same
> as a 4K one, and it cannot express that one path is simply faster than
> another. This series adds a service-time iopolicy for NVMe native
> multipath, adapted from dm-ps-service-time: it picks the path with the
> lowest estimated service time, in_flight_bytes / relative_throughput,
> so balancing follows actual bytes in flight and per-path throughput
> weights.
>
> Weights live in a per-path relative_throughput sysfs attribute (0-100,
> default 1, 0 makes the path a standby). Patch 3 documents the new
> attributes in the stable sysfs ABI; patch 1 carries the benchmark data
> in its commit log.
>
> This version is rebased onto the nvme-7.4 branch of
> git://git.infradead.org/nvme.git.
>
> Changes since v3:
>
> - Rebased onto nvme-7.4, which adds the iopolicies and path_state
>   attributes and the discovery-subsystem "none" handling.
>
> - The iopolicies ABI entry now lists service-time too. nvme-7.4
>   already documents the attribute (with a hardcoded list of the
>   three older policies); patch 3 extends that list along with the
>   iopolicy accepted values.
>
> Changes since v2:
>
> - in_flight_bytes no longer leaks on failover: the accounted bytes
>   are now released in nvme_failover_req() before blk_steal_bios()
>   clears the request data length. Before, every failover (ANA
>   transition, path error, queue_rq unwind) permanently left the
>   request's bytes in the counter, skewing path selection until a
>   controller reset. Reproduced with an ANA transition during load on
>   the NVMe-oF/TCP test setup: the counter kept 512K of phantom bytes
>   after all I/O completed; with the fix it returns to zero.
>
> - The release clamps at zero instead of a plain subtraction. A
>   request requeued across a controller reset is accounted against the
>   pre-reset counter but completes after nvme_mpath_init_identify()
>   re-zeroed it, and the plain subtraction would wrap negative, making
>   the comparator treat the path as pathologically overloaded. This
>   mirrors atomic_dec_if_positive() in the queue-depth accounting.
>
> - The comparison operates on u64 instead of size_t, so the byte
>   counts and the shift guard keep their magnitude on 32-bit builds,
>   where size_t would truncate the atomic64 reads.
>
> - Added "service-time" to the accepted values of the subsystem
>   iopolicy ABI entry and to the policies listed by the iopolicies
>   ABI entry.
>
> No public v3 was posted; this version folds the review feedback from
> the v2 discussion and a rebase onto nvme-7.4.
>
> On the adaptive I/O policy dependency Hannes raised during v1: that
> series has become the latency I/O policy and is still under
> revision. The two are complementary: the latency policy infers path
> speed from observed completions, service-time takes configured
> weights and byte counts.
>
> Guixin Liu (3):
>   nvme-multipath: implement service-time iopolicy
>   nvme-multipath: document service-time iopolicy
>   nvme: document the service-time iopolicy sysfs attributes
>
>  Documentation/ABI/stable/sysfs-nvme          |  35 +++-
>  Documentation/admin-guide/nvme-multipath.rst |  31 +++-
>  drivers/nvme/host/multipath.c                | 189 ++++++++++++++++++-
>  drivers/nvme/host/nvme.h                     |   6 +
>  drivers/nvme/host/sysfs.c                    |   7 +-
>  5 files changed, 256 insertions(+), 12 deletions(-)
>
> v2 of this series:
> https://lore.kernel.org/r/20261019075607.808947-1-kanie@linux.alibaba.com
>
> v1 of this series:
> https://lore.kernel.org/r/20260617114602.2224074-1-kanie@linux.alibaba.com
>
> latency I/O policy v7:
> https://lore.kernel.org/r/20260809100825.2014133-1-nilay@linux.ibm.com