[PATCH v2] nvme-tcp: Fix I/O queue cpu spreading for multiple controllers

[Keith Busch]

On Sat, Jan 04, 2025 at 11:27:11PM +0200, Sagi Grimberg wrote:
> Since day-1 we are assigning the queue io_cpu very naively. We always
> base the queue id (controller scope) and assign it its matching cpu
> from the online mask. This works fine when the number of queues match
> the number of cpu cores.
> 
> The problem starts when we have less queues than cpu cores. First, we
> should take into account the mq_map and select a cpu within the cpus
> that are assigned to this queue by the mq_map in order to minimize cross
> numa cpu bouncing.
> 
> Second, even worse is that we don't take into account multiple
> controllers may have assigned queues to a given cpu. As a result we may
> simply compund more and more queues on the same set of cpus, which is
> suboptimal.
> 
> We fix this by introducing global per-cpu counters that tracks the
> number of queues assigned to each cpu, and we select the least used cpu
> based on the mq_map and the per-cpu counters, and assign it as the queue
> io_cpu.

Thanks, applied to nvme-6.14.