[PATCH 1/3] nvmet: expose discovery subsystem in sysfs

[Hannes Reinecke]

On 3/15/22 11:49, Sagi Grimberg wrote:
> 
>>>> The core question really is: do we _want_ to expose the discovery 
>>>> subsystem
>>>> in configfs?
>>>
>>> Well, if you want a freely configurable one we kinda have to, right?
>>>
>>>> Unfortunately, exposing the discovery subsystem and trying to 
>>>> configure it
>>>> with configfs does _not_ match with the way discovery is implemented 
>>>> today.
>>>> While we currently only have a single discovery subsystem, it will only
>>>> ever return the subsystems visible from this particular port.
>>>
>>> Well.  The original Fabrics spec had this concept of that magic 
>>> discovery
>>> NQN, which implies that there is one subsystem (or many pretending to be
>>> one).  And that is what the implementation followed.  The varipus 
>>> 80?? TPs
>>> then made a complete mess off that.
>>>
>>>> Hence this rather simple approach, having the 'normal' discovery 
>>>> subsystem
>>>> exposed, and let the admin configure it accordingly.
>>>>
>>>> I can look at keeping the internal implementation, and only expose 
>>>> unique
>>>> discovery controller (ie those with a unique subsystem NQN).
>>>> That would remove the need to having the 'discovery_nqn' attribute, and
>>>> address Christophs concerns.
>>>
>>> I suspect if we want to support all the new mess from the FMDS group
>>> (and maybe we need to question the why a little more), then we should
>>> so something like:
>>>
>>>   (1) keep the existing global NQN-based discovery as-is.
>>>   (2) maybe add a per-port known to allow disabling it if people 
>>> really care
>>>   (3) allow creating additional discovery subsystems with non-default
>>>       NQNs that do not automatically get anything added to them and will
>>>       just be configured as needed through configfs
>>>
>> Yeah, that's the line I'm following.
>>
>>> But maybe first we should take a step back and figure out what 
>>> supporting
>>> TPAR8013 even buys us?
>>
>> Properly supporting persistent discovery controllers.
>>
>> The current problem we're having (on linux) is that we cannot identify 
>> discovery controllers. Each discovery controller has the same 
>> subsystem NQN, and hence that's not sufficient to identify the subsystem
>> Which resulted in linux to always create a new 'nvme_subsystem' 
>> instance when creating a new nvme discover controller.
> 
> That is the case regardless of naming... there is no multipathing with
> discovery subsystems so there will be a subsystem for every discovery
> controller...
>
Well, technically, no, but in practice there is, as one can easily 
specify more than one discovery port (to the same discovery subsystem).
And quite some target implementations do.
Discussion is probably moot anyway as multipathing would only affect 
I/O, which won't be done anyway.

>> Problem now starts when you try to use persistent discovery 
>> controllers; it's quite easy to _create_ a persistent discovery 
>> controller, but less easy to _use_ an existing one; how would you know 
>> which one to pick?
> 
> You are not supposed to "use" an existing one, the whole point of the
> persistent discovery controllers is that you don't need to worry about
> it. We already perfectly handle that with udev using the kernel of
> the uevent.
> 

And my point is that you _do_. Yes, udev does work, but only in so far 
AENs are involved. When you want to retrieve the discovery log manually 
_and_ have existing persistent discovery controllers you'll need to know 
which device to pick.

Which is getting especially interesting if you have several different 
subsystems to contend with.

>> More importantly, as linux will always create a new nvme_subsystem for 
>> each discovery controller you'll end up with several 'nvme_subsystem' 
>> instances which in fact all refer to the very same subsystem.
> 
> OK, still don't understand why that is a problem.
> 
>> nvme-cli tries to work around that problem by requiring the user to 
>> specify the device name, but that requires quite some knowledge from 
>> the admin side, and making that work in a scripted fashion is a pain.
> 
> I am not aware at all on what scripting are you referring to. But udev
> scripts are written already. Can you explain?
> 
>> So with TPAR8013 we now _have_ distinct discovery subsystems, can 
>> create unique subsystem, won't have duplicate subsystems, and can 
>> automate nvme-cli to pick the correct existing persistent discovery 
>> controller even if the user does _not_ specify the device name.
> 
> And instead specify the subsysnqn? I am not aware of this reasoning you
> are referring to at all, but I was probably absent from these
> discussions... I thought that this TP was solely related to 8010 stuff
> and also solves an issue that came up with the discovery subsystem and
> the authentication TPs.

True. authentication requires a unique NQN, and hence can't be used 
without TP8013.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		           Kernel Storage Architect
hare at suse.de			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), GF: Felix Imendörffer