Native multipath across multiple subsystem NQNs

[Christoph Hellwig]

On Thu, Feb 24, 2022 at 05:53:06PM -0700, Randy Jennings wrote:
> In addition to other considerations, migrating a namespace with hot data
> from a different storage vendor array nondisruptive to client i/o would
> be difficult to do without having a namespace exist under multiple NQNs.
> One approach that has been used for SCSI is by multipathing to both
> targets & a proxy mirroring writes; on cut-over, the other path
> disconnects.  If the filehandle has to change, that is disruptive to the
> client software.
> 
> Unifying different storage arrays behind the same NQN/virtual subsystem
> requires coordination of nvme ctrl_ids at least, and doing that between
> different storage vendor arrays is unlikely.  Having a mechanism to
> migrate between different JBOF devices non-disruptively would be helpful
> regardless of the source/destination vendors.  Such devices will
> probably not have the option of virtual subsystems.

Yes, it does require coordination, so please coordinate.

> In other words, even with implementing virtual subsystems, we still have
> use for non-disruptively moving a namespace between subsystems.  How
> will this usecase be supported on Linux?

You can create a mirror using dm (or md but not as easily), resync and
then cut off a leg.   If you want to do that transparently on an
already existing device node you'll need to pick up the block device
interposer patchset and help to get it upstream.