Native multipath across multiple subsystem NQNs

[Christoph Hellwig]

Hi Prashanth,

thanks for adding Fred who apparently complained to the NVMe BOD about
this issue, and who really needs some shaming from Greg about trying to
use corporate means for forcing Linux developers into implementing broken
standards.

On Thu, Mar 03, 2022 at 11:04:48AM +0000, Adurthi, Prashanth wrote:
> Implementing the same subsystem NQN on multiple storage arrays imposes excessive
> architecture/design costs on the target.

While on the other hand implementing TP4034 forces excessive architecture,
design, maintainance and debugging costs on the host.  That's why I've
told the NVMe technical working group on the very day that this TPAR
was brought in that we are not going to support it, and I've also heard
feeback from various other host implementators that there is very little
interest in it.

>
> In addition to implementing
> the same subsystem NQN, the target has to ensure that all components
> that make up the virtual subsystem (across multiple arrays) have
> the same inventory (same namespaces with matching NSIDs and ANA
> groups). This needs to be kept consistent even when there are communication
> failures between those components of the virtual subsystem.
> The virtual subsystem would also be required to have the same host
> access control settings and similar QoS settings etc on all of the
> components that make up the virtual subsystem. This coordination, in
> addition to what is required for controller IDs, is extremely complicated when both
> the arrays belong to the same vendor and next to impossible, if in the future,
> a multi-vendor solution is developed. 

Yes, and that is the whole point.

> Can you please shed some light on why you consider TP 4034 retrograde? What issues are you concerned about if Linux nvme were to handle namespaces shared across subsystems?

NVMe has a sensible object model that allows us for easy sanity checking
of duplicate ids and more importantly for proper discovery.  We do not
want to lose that, nor do want to break the sysfs hierachy that directly
results from encoding the nvme architecture model in the Linux device
model.