NVMe over Fabrics host: behavior on presence of ANA Group in "change" state

[Knight, Frederick]

> 
> On 2/11/22 10:21, Alex Talker wrote:
> >  > [FK> ] I think I'm missing a bunch of context here. What is the  >
> > original question? I take a stab at some assumptions: What is an  >
> > empty ANA group? That is an ANA Group with NO NSIDs associated with  >
> > that group. Meaning the "Number of NSID Values" field is cleared to  >
> > '0h' in the ANA Group Descriptor. That descriptor can be used to  >
> > update some host internal state information related to that ANA  >
> > group, but it has no impact on any I/O because there can be no I/O  >
> > (since there are no NSID values). So I'm not sure where that is  >
> > going (because RGO=1 also can return ANA Groups that have state, but
> > > no attached namespaces (it's a way to get group state without any  >
> > NSID inventory requirements)).
> >
> > That's exactly right, "nnsids=0" case. I/O is not a problem for such a
> > group, for sure.
> > I suppose the main argument we're having here is that when such a
> > group has a "change" ANA state, the host("nvme-core" module) starts a
> > timer for ANATT which upon expiration resets the controller.
> > Now, I do not disagree that having such a group is "ugly" but rather
> > argue that ANATT-related functionality could be only invoked for
> > "nnsids>0" case, since only then there's a relation between "change"
> > state and a namespace via "ANAGRPID".
> >
> Ah. So now I get where you are coming from.
> 
> The problem seems to be around a static ANA group with status 'change'.
> The whole idea behind the 'change' ANA status (and ANA groups in
> general) was that ANA groups could _change_ the ANA status, and such a
> change would affect all namespaces in that group.
>  From that angle having a 'change' state makes sense, as one could use them
> to signal "hey, I'm busy transitioning to another state" to the host.
> 
> But when using _static_ ANA groups the whole concept become shaky.
> While it's perfectly okay to have static ANA groups with 'normal' states (ie
> excluding the 'change' state), things become iffy if you have a static 'change'
> state.
> Thing is, having a 'change' state indicates to the host that a controller will
> need time to transition between states. And this transitioning out of
> necessity will always be between 'normal' ANA states (ie excluding the
> 'change' state).
> The spec doesn't allow the controller to specify "I need time to transition
> _to_ the 'change' state"; the transition to and from 'change'
> is always assumed to be atomic.
> (Because the 'change' state is used to signal precisely that condition.)
> 
> Exec summary: having static ANA groups is okay, having a static ANA group in
> 'change' state questionable and not something I would recommend.

[FK> ]  That is the point of ANATT.  If you get a status code of Asymmetric Access Transition on any I/O,
Then all namespaces in the ANA Group that contains that namespace are in CHANGE state.  All of those
Namespaces (the ones in the same ANA Group as the namespace that returned a status code of Asymmetric
Access Transition) are allowed to stay in that state for ANATT seconds.  So YES, that ANA Group may
Stay in the CHANGE state (but only for ANATT seconds).  If it stays in the CHANGE state for longer than
ANATT Seconds, then there is a problem.  The definition of ANATT = the controller will NOT stay in the
ANA Change state longer than this amount of time.

> 
> Cheers,
> 
> Hannes
> --
> Dr. Hannes Reinecke                        Kernel Storage Architect
> hare at suse.de                                      +49 911 74053 688
> SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
> HRB 36809 (AG Nürnberg), GF: Felix Imendörffer