[PATCH 03/19] nvme-fabrics: reconnect if the association is gone

[Keith Busch]

From: Josef Bacik <josef at toxicpanda.com>

A reconnect connects the admin queue, gets a controller ID, and connects
the I/O queues with it. If the controller goes away in between, the
Connect of an I/O queue fails with Connect Invalid Parameters pointing
at the cntlid, with DNR set:

  nvme nvme0: Connect Invalid Data Parameter, cntlid: 1
  nvme nvme0: Failed reconnect attempt 1/600
  nvme nvme0: Removing controller (16770)...

nvmf_should_reconnect() honours DNR and the host deletes the controller
with 599 attempts to go. DNR is right in that resending that command is
pointless, but the next attempt would create a new association with a
new cntlid. This is not a verdict on the host like a failed admin
Connect is.

Treat it as a lost association and return -ENOTCONN, so the retries are
bounded by ctrl_loss_tmo like after any other loss.

Fixes: adfde7ed0b30 ("nvme-fabrics: short-circuit reconnect retries")
Cc: Keith Busch <kbusch at kernel.org>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Josef Bacik <josef at toxicpanda.com>
Signed-off-by: Jens Axboe <axboe at kernel.dk>
Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/fabrics.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index 3d7ad86c1955f..aeff98e16d6d5 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -542,8 +542,19 @@ int nvmf_connect_io_queue(struct nvme_ctrl *ctrl, u16 qid)
 			NVME_SUBMIT_RESERVED |
 			NVME_SUBMIT_NOWAIT);
 	if (ret) {
-		nvmf_log_connect_error(ctrl, ret, le32_to_cpu(res.u32),
-				       &cmd, data);
+		result = le32_to_cpu(res.u32);
+		nvmf_log_connect_error(ctrl, ret, result, &cmd, data);
+		/*
+		 * The controller ID we got from the admin queue's Connect is
+		 * not known any more: the association has gone since. DNR is
+		 * right about this command and says nothing about the next
+		 * association. Don't let nvmf_should_reconnect() see it.
+		 */
+		if (ret > 0 &&
+		    (ret & NVME_SCT_SC_MASK) == NVME_SC_CONNECT_INVALID_PARAM &&
+		    result >> 16 && (result & 0xffff) ==
+				offsetof(struct nvmf_connect_data, cntlid))
+			ret = -ENOTCONN;
 		goto out_free_data;
 	}
 	result = le32_to_cpu(res.u32);
-- 
2.52.0