[PATCH 1/3] nvme-core: fix io interrupt caused by non path error

[Chao Leng]

On 2020/8/12 23:09, Christoph Hellwig wrote:
> On Wed, Aug 12, 2020 at 04:18:37PM +0800, Chao Leng wrote:
>> For nvme multipath configured, just fail over to retry IO for path error,
>> but maybe blk_queue_dying return true, IO can not be retry at current
>> path, thus IO will interrupted.
>>
>> For nvme multipath configured, blk_queue_dying and path error both need
>> fail over to retry. We need check whether path-related errors first, and
>> then retry local or fail over to retry.
> 
> Err, no.  None of this really makes any sense.  The existing code
> actually works perfectly well unless you really insist on trying to
> use a completley unsupported multipathing configuration.  I would
> storngly recommend to not use dm-multipath with nvme, but if you
> insist please fix your problems without impacting the fully supported
> native path.
The scenario: IO already return with non path error(such as
NVME_SC_CMD_INTERRUPTED or NVME_SC_DATA_XFER_ERROR etc.), but is waiting
to be processed, at the same time, delete ctrl happens, delete ctrl may
set queue flag: QUEUE_FLAG_DYING when call nvme_remove_namespaces. Then
for example, if fabric is rdma, delete ctrl will call
nvme_rdma_delete_ctrl, nvme_rdma_delete_ctrl will drain qp first, thus
the IO, which return with non path error, can not be failover retry,
and also can not retry local, IO will interrupt.
Another solution can also avoid it: nvme_rdma_delete_ctrl first disable
irq instead of drain qp, then cancel all io(set path error), thus
nvme multipath will failover retry. There may be a little flaw: if
io complete success, which is waiting to be processed, will also be
canceled and failover retry.