NVMe fabric multipathing

[Mark Syms]

On Thu, 5 Oct 2023 at 09:25, Christoph Hellwig <hch at infradead.org> wrote:
>
> Hi Mark,
>
> On Mon, Oct 02, 2023 at 01:22:59PM +0100, Mark Syms wrote:
> > We have a requirement to report information about the status of NVMe
> > multipath controllers on an NVMe over fabrics deployment. In
> > particular the total number of "known" paths and the number of
> > "currently active" paths. As far as we can see right now, when a path
> > to an NVMe target device, when using Fibre Channel, is blocked, all
> > local information about the affected controller is flushed leaving
> > just the currently active controller(s) present.
>
> What do you mean with "blocked"?
>
Made unavailable for any reason. So failed switch port, failed HBA,
failed SAN controller, etc. We've been "emulating" this for testing
purposes by unbinding the PCI device for the HBA on the NVMe target
device but I expect the same happens for any of those reasons.

We basically want to be able to report to the end user something along
the lines of

* Multipath active, 4 active of total 4 paths

vs

* Multipath active 2 active of total 4 path

(and variants of that).

For SCSI based storage (iSCSI, FC, FCoE) using Device Mapper Multipath
for the multipathing this information is readily available from the
system tools (at least for paths which were present at bootime or
after a rescan) and shows active and faulty paths so that the above
style status message can be produced. With the entire NVMe controller
and all knowledge about it seemingly disappearing without trace it
means that to achieve the same results we'd need to track externally
how many paths we've ever seen (and some way of uniquely identifying
them, traddr and host_traddr probably) and what the current state of
those paths, at least for any which we've seen but are not currently
present. Doable of course but potentially a bit fragile so if
avoidable that would be preferable.