[PATCHv2] nvme-mpath: delete disk after last connection

[Hannes Reinecke]

On 4/20/21 4:14 PM, Keith Busch wrote:
> On Tue, Apr 20, 2021 at 03:19:10PM +0200, Hannes Reinecke wrote:
>> On 4/20/21 10:05 AM, Christoph Hellwig wrote:
>>> On Fri, Apr 16, 2021 at 08:24:11AM +0200, Hannes Reinecke wrote:
>>>> With the proposed patch, the following messages appear:
>>>>
>>>>    [  227.516807] md/raid1:md0: Disk failure on nvme3n1, disabling device.
>>>>    [  227.516807] md/raid1:md0: Operation continuing on 1 devices.
>>>
>>> So how is this going to work for e.g. a case where the device
>>> disappear due to resets or fabrics connection problems?  This now
>>> directly teards down the device.
>>>
>> Yes, that is correct; the nshead will be removed once the last path is
>> _removed_.
> 
> The end result is also how non-multipath nvme behaves, so I think that's
> what users have come to expect.
> 
Nope. It doesn't.

This patch precisely _aligns_ the behaviour between multipathed (or, to 
be precise, namespaces with CMIC != 0) and non-multipathed ones.

When CMIC == 0 this patch doesn't change the behaviour at all.

>> But key point here is that once the system finds itself in that
>> situation it's impossible to recover, as the refcounts are messed.
>> Even a manual connect call with the same parameter will _not_ restore
>> operation, but rather result in a new namespace.
> 
> I haven't looked at this yet, but is it really not possible to restore
> the original namespace upon the reestablished connection?
> 

No. As mentioned, refcount is messed up.

With the current behaviour, once the last path drops, the namespace will 
_vanish_ from the list of namespaces.
Once you reconnect the original namespace 'magically' reappears in 'nvme 
list', but with size '0', _and_ the new namespace beside it:

Needless to say, the current opener (mount, MD, you name it) still refer 
to the original namespace, and you can't correct the situation as I/O is 
still pending you have no way to abort it.

(Funny, though; it was you who suggested the patch in the first place; 
feels kinda strange to explain the patch to its author ...)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer