[PATCH 0/6] fix possible controller reset hangs in nvme-tcp/nvme-rdma
Sagi Grimberg
sagi at grimberg.me
Mon Aug 3 02:58:46 EDT 2020
When a controller reset runs during I/O we may hang if the controller
suddenly becomes unresponsive during the reset and/or the reconnection
stages. This is due to how the timeout handler did not fail inflight
commands properly and also not being able to abort the controller reset
sequence when the controller becomes unresponsive (hence can't ever
recover even if the controller ever becomes responsive again).
This set fixes nvme-tcp and nvme-rdma for exactly the same scenarios.
Patch 1 prevents commands being queued fora live queued, making
commands mistakenly getting requeued forever while we are either
resetting or connecting to a controller.
Patches 2,4,6 address the case when a controller stops responding when
we are in the middle of a connection establishment stage (tcp and rdma).
Patches 3,5 rework the timeout handler to fail commands (and allow them
to either requeue or fail) in case the controller is not responsive when
we are in the middle of reset (teardown) or establishment (connect
sequence).
James, please have a look to patch 1, this relates to the discussions
we had recently. We still keep the admin commands with a guard, but
that would be addressed in a follow-up set.
Sagi Grimberg (6):
nvme-fabrics: allow to queue requests for live queues
nvme: have nvme_wait_freeze_timeout return if it timed out
nvme-tcp: fix timeout handler
nvme-tcp: fix reset hang if controller died in the middle of a reset
nvme-rdma: fix timeout handler
nvme-rdma: fix reset hang if controller died in the middle of a reset
drivers/nvme/host/core.c | 3 +-
drivers/nvme/host/fabrics.c | 13 +++---
drivers/nvme/host/nvme.h | 2 +-
drivers/nvme/host/rdma.c | 78 ++++++++++++++++++++++++++---------
drivers/nvme/host/tcp.c | 81 +++++++++++++++++++++++++++----------
5 files changed, 130 insertions(+), 47 deletions(-)
--
2.25.1
More information about the Linux-nvme
mailing list