Native multipath across multiple subsystem NQNs

[Randy Jennings]

> The fact that NVMe with ANA required namespaces
> to be be attached to the same subsystem is good and important.
> TPE 4034 is retrograde and completely broken in this respect and should
> have never been ratifier.  Please fix your storage system to export
> virtual subsystems like everyone else and don't break the nice NVMe
> architecture.
Supposing we did implement virtual subsystems to handle exposing
namespaces on multiple arrays.

In addition to other considerations, migrating a namespace with hot data
from a different storage vendor array nondisruptive to client i/o would
be difficult to do without having a namespace exist under multiple NQNs.
One approach that has been used for SCSI is by multipathing to both
targets & a proxy mirroring writes; on cut-over, the other path
disconnects.  If the filehandle has to change, that is disruptive to the
client software.

Unifying different storage arrays behind the same NQN/virtual subsystem
requires coordination of nvme ctrl_ids at least, and doing that between
different storage vendor arrays is unlikely.  Having a mechanism to
migrate between different JBOF devices non-disruptively would be helpful
regardless of the source/destination vendors.  Such devices will
probably not have the option of virtual subsystems.

Additionally, the granularity at which NQNs/subsystems are exposed to
hosts affects how many connections the host creates with the controller.
Each connection has a cost in hardware & software.

In other words, even with implementing virtual subsystems, we still have
use for non-disruptively moving a namespace between subsystems.  How
will this usecase be supported on Linux?

Sincerely,
Randy Jennings