[PATCH v3 0/2] resolve controller delete hang due to ongoing mpath I/O
Sagi Grimberg
sagi at grimberg.me
Wed Jul 22 19:32:17 EDT 2020
Changes from v2:
- remove nvme_remove_namespaces split (Christoph)
- tweaked nvme_path_is_disabled to look better (Christoph)
Changes from v1:
- Rename states to NVME_CTRL_DELETING and NVME_CTRL_DELETING_NOIO to better
describe the states
- Added prep patch to split nvme_remove_namespaces to _prep_ and _do_
- Added prep patch to provide some documentation about the states
--------------------------------------------------------------------------
A deadlock happens in the following scenario with multipath:
1) scan_work(nvme0) detects a new nsid while nvme0
is an optimized path to it, path nvme1 happens to be
inaccessible.
2) Before scan_work is complete nvme0 disconnect is initiated
nvme_delete_ctrl_sync() sets nvme0 state to NVME_CTRL_DELETING
3) scan_work(1) attempts to submit IO,
but nvme_path_is_optimized() observes nvme0 is not LIVE.
Since nvme1 is a possible path IO is requeued and scan_work hangs.
4) Delete also hangs in flush_work(ctrl->scan_work)
from nvme_remove_namespaces().
Similiarly a deadlock with ana_work may happen: if ana_work has started
and calls nvme_mpath_set_live and device_add_disk, it will
trigger I/O. When we trigger disconnect I/O will block because
our accessible (optimized) path is disconnecting, but the alternate
path is inaccessible, so I/O blocks. Then disconnect tries to flush
the ana_work and hangs.
This patchset alters the nvme states to address this deadlock condition.
Sagi Grimberg (2):
nvme: document nvme controller states
nvme-core: fix deadlock in disconnect during scan_work and/or ana_work
drivers/nvme/host/core.c | 15 +++++++++++++++
drivers/nvme/host/fc.c | 1 +
drivers/nvme/host/multipath.c | 18 +++++++++++++++---
drivers/nvme/host/nvme.h | 20 ++++++++++++++++++++
drivers/nvme/host/rdma.c | 10 ++++++----
drivers/nvme/host/tcp.c | 15 +++++++++------
6 files changed, 66 insertions(+), 13 deletions(-)
--
2.25.1
More information about the Linux-nvme
mailing list