[PATCH 0/4] nvme-tcp: improve scalability

Hannes Reinecke hare at kernel.org
Wed Jul 3 06:50:17 PDT 2024


Hi all,

for workloads with a lot of controllers we run into workqueue contention,
where the single workqueue is not able to service requests fast enough,
leading to spurious I/O errors and connect resets during high load.
This patchset improves the situation by introducing per-controller workqueues,
and modifies the 'wq_unbound' module parameter to include the blk-mq
cpu information for improving the locality.
With this we reduce the spurious I/O errors and improve the overall
performance for highly contended workloads.

Performance comparisons are hard to get as the original code would
abort with I/O timeouts such that fio has a hard time completing...

As usual, comments and reviews are welcome.

Hannes Reinecke (4):
  nvme-tcp: per-controller I/O workqueues
  nvme-tcp: align I/O cpu with blk-mq mapping
  workqueue: introduce helper workqueue_unbound_affinity_scope()
  nvme-tcp: switch to 'cpu' affinity scope for unbound workqueues

 drivers/nvme/host/tcp.c   | 84 +++++++++++++++++++++++++--------------
 include/linux/workqueue.h |  1 +
 kernel/workqueue.c        | 47 +++++++++++++++++-----
 3 files changed, 93 insertions(+), 39 deletions(-)

-- 
2.35.3




More information about the Linux-nvme mailing list