[PATCH v2 2/2] nvme-tcp: parallelize I/O queue allocation and startup

[Surabhi Gogte (she/her)]

On Sun, Aug 30, 2026 at 3:00 PM Sagi Grimberg <sagi at grimberg.me> wrote:
>
>
>
> On 25/08/2026 1:56, Surabhi Gogte wrote:
> > Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
> > allocation and startup"), refactor nvme tcp I/O queue setup to use async
> > API, combining allocation and startup into a single parallel operation
> > per queue. This reduces connection and reconnection setup time when
> > there are delays in establishing connections, which is especially
> > important for high-core-count hosts.
> >
> > Key changes:
> > - Use async API to facilitate parallel calls for io queue setup.
> > - Add nvme_tcp_setup_ctx for propagating errors from async workers.
> > - Remove nvme_tcp_start_io_queues() and __nvme_tcp_alloc_io_queues();
> >    their logic is folded into nvme_tcp_setup_io_queues() and
> >    nvme_tcp_configure_io_queues().
> > - Allocate the io tag set before the queues so that the queue range is
> >    known, and only set up the reconnect grow case if the queue count
> >    actually increased.
> > - Serialize the cpu scan and claim in nvme_tcp_set_queue_io_cpu() with a
> >    spinlock, as concurrent callers would otherwise select the same cpu.
> >    The per-cpu counters no longer need to be atomics.
> > - Use init_net in nvme_tcp_alloc_queue() instead of the namespace of
> >    current, which is no longer the connecting task once the allocation
> >    runs from a worker. A controller is not guaranteed to be tied to a
> >    namespace, as the reconnect and error recovery paths already run from
> >    a workqueue in init_net.
>
> Well, I think this is breaking 1be52169c3488ef98582ed553ab35cefa3978817
>
Trying to understand how this breaks the commit 1be52169c348. Added my
understanding later in the thread.

> > @@ -154,6 +156,12 @@ struct nvme_tcp_queue {
> >   static DEFINE_MUTEX(nvme_tcp_ctrl_mutex);
> >   static LIST_HEAD_GUARDED(nvme_tcp_ctrl_list, nvme_tcp_ctrl_mutex);
> >
> > +struct nvme_tcp_setup_ctx {
>
> nvme_tcp_queue_setup_ctx?
>

Yeah, can rename it.

> > +     spin_unlock(&nvme_tcp_cpu_queues_lock);
> >   out:
> >       dev_dbg(ctrl->ctrl.device, "queue %d: using cpu %d\n",
> >               qid, queue->io_cpu);
> > @@ -1846,7 +1856,7 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
> >               queue->cmnd_capsule_len = sizeof(struct nvme_command) +
> >                                               NVME_TCP_ADMIN_CCSZ;
> >
> > -     ret = sock_create_kern(current->nsproxy->net_ns,
> > +     ret = sock_create_kern(&init_net,
>
> I don't think we can just change this...

Passing init_net explicitly does change behavior on the synchronous connect
path. However, with queue allocation now moved into an async worker,
current->nsproxy->net_ns resolves to init_net in that context anyway, so
the initial connect path already loses the caller's netns regardless of
which argument is passed.

If the goal is to actually preserve the caller's netns through the full
ctrl lifecycle - initial connect, reconnect, and error recovery; it can
be pinned at ctrl creation like:

    nvme_tcp_alloc_ctrl():
            to_tcp_ctrl(ctrl)->net = get_net(current->nsproxy->net_ns);

    nvme_tcp_free_ctrl():
            put_net(to_tcp_ctrl(ctrl)->net);

    nvme_tcp_alloc_queue():
            sock_create_kern(to_tcp_ctrl(ctrl)->net, ...);

This captures the caller's netns while still in userspace context, then
carries it through all async paths — so reconnect and error recovery
also honor the original netns rather than falling back to the kworker's
init_net.

Is this approach preferred?