[PATCH] nvme-pci: skip FLR after a failed controller reset

[Haowen Bai]

nvme_disable_ctrl() already waits up to CAP.TO for CSTS.RDY to clear.
If that times out, nvme_pci_configure_admin_queue() currently issues a
PCIe Function Level Reset and retries.

FLR is performed with PCI config cycles. Those cycles take
pci_config_lock, a raw spinlock, and wait for the endpoint to complete
the transaction. A wedged NVMe function can stall that completion.
Other CPUs then spin in pci_conf1_read() -- including ACPI PCI config
from an unrelated device -- and the NMI watchdog reports a hard lockup.

This was observed on an x86_64 UGREEN DXP4800 (kernel 6.18.15) with two
ZHITAI Ti600 NVMe devices used as bcache. Each disk independently:

  nvme: I/O timeout, reset controller
  nvme: Device not ready; aborting reset, CSTS=0x1
  nvme: Device not ready; aborting reset, CSTS=0x1
  watchdog: Watchdog detected hard LOCKUP
  RIP: native_queued_spin_lock_slowpath
  pci_conf1_read -> acpi_pci_set_power_state -> mmc runtime resume

The two "aborting reset" messages are nvme_wait_ready() timeouts, 128s
apart, matching (CAP.TO+1)/2. The lockup is ~11s after the second
timeout, i.e. on the post-FLR cleanup path, not in the wait loop.
Linux 6.12 has no FLR fallback here; 6.18 and 7.3 still do.

Skip FLR when the controller is already in NVME_CTRL_RESETTING (I/O
timeout recovery). Keep the FLR hammer for initial probe, where the
device may simply have been left enabled by firmware. Reset work then
marks namespaces dead instead of hard-locking the host.

Cc: linux-nvme at lists.infradead.org
Cc: linux-pci at vger.kernel.org
Cc: Keith Busch <kbusch at kernel.org>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Haowen Bai <calvin.bai at ugreen.com>
---
 drivers/nvme/host/pci.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 5440cf18b55b..1df475fab950 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2373,12 +2373,18 @@ static int nvme_pci_configure_admin_queue(struct nvme_dev *dev)
 		struct pci_dev *pdev = to_pci_dev(dev->dev);
 
 		/*
-		 * The NVMe Controller Reset method did not get an expected
-		 * CSTS.RDY transition, so something with the device appears to
-		 * be stuck. Use the lower level and bigger hammer PCIe
-		 * Function Level Reset to attempt restoring the device to its
-		 * initial state, and try again.
+		 * Controller Reset did not clear CSTS.RDY. FLR can recover
+		 * some devices, but it issues PCI config cycles with
+		 * pci_config_lock held. A wedged function can stall those
+		 * cycles and hard-lock unrelated PCI users.
+		 *
+		 * Only try FLR during initial probe. On I/O-timeout reset
+		 * the controller is already known stuck; fail the reset
+		 * instead of risking a host lockup.
 		 */
+		if (nvme_ctrl_state(&dev->ctrl) == NVME_CTRL_RESETTING)
+			return result;
+
 		result = pcie_reset_flr(pdev, false);
 		if (result < 0)
 			return result;
-- 
2.47.3