[PATCH 11/19] nvmet-rdma: treat a failed connect as a disconnect

[Keith Busch]

From: Chris Mason <mason at kernel.org>

Once a connect request has been accepted, a queue is only freed by its
release_work, so that must be queued once. The disconnect paths take
the queue off nvmet_rdma_queue_list first and queue the work only if it
was still on it. nvmet_rdma_queue_connect_fail(), for REJECTED,
UNREACHABLE and CONNECT_ERROR, queues it unconditionally and only warns
if the queue isn't connecting. But the IB CM delivers a REJ from the
peer while the connection is established and even after we've sent a
DREQ, so a peer that causes a disconnect with a short capsule and then
sends a REJ while the release work is running queues it a second time
and the work runs on a freed queue. No credentials needed, just RoCE or
IB reachability, reproduced over rxe with a UDP socket.

Route connect failures through nvmet_rdma_queue_disconnect(), so only
the path that takes the queue off the list queues the work. A REJ on an
established connection still frees the queue, the RDMA CM reports no
disconnect after one.

Fixes: 8f000cac6e7a ("nvmet-rdma: add a NVMe over Fabrics RDMA target driver")
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Chaitanya Kulkarni <kch at nvidia.com>
Signed-off-by: Chris Mason <mason at kernel.org>
Signed-off-by: Jens Axboe <axboe at kernel.dk>
Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/target/rdma.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index b9e39d1657d29..fb20ba1b84e7f 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -1745,15 +1745,14 @@ static void nvmet_rdma_queue_disconnect(struct nvmet_rdma_queue *queue)
 static void nvmet_rdma_queue_connect_fail(struct rdma_cm_id *cm_id,
 		struct nvmet_rdma_queue *queue)
 {
-	WARN_ON_ONCE(queue->state != NVMET_RDMA_Q_CONNECTING);
-
-	mutex_lock(&nvmet_rdma_queue_mutex);
-	if (!list_empty(&queue->queue_list))
-		list_del_init(&queue->queue_list);
-	mutex_unlock(&nvmet_rdma_queue_mutex);
-
 	pr_err("failed to connect queue %d\n", queue->idx);
-	queue_work(nvmet_wq, &queue->release_work);
+
+	/*
+	 * The peer decides when these events arrive: a REJ is also delivered
+	 * for a connection that is established, or that we are disconnecting.
+	 * Whoever takes the queue off the list queues release_work, once.
+	 */
+	nvmet_rdma_queue_disconnect(queue);
 }
 
 /**
-- 
2.52.0