Issue with AWUPF when using multiple controllers in a subsystem

[Christoph Hellwig]

On Fri, Apr 04, 2025 at 03:42:50PM -0700, alan.adamson at oracle.com wrote:
> When dumping the awupf value with nvme-cli, the values match what was setup
> in qemu, but when each devices atomic queue limits was displayed, it didn't
> match its awupf. Currently the awupf is saved in the nvme_subsystem, but the
> awupf is specific to the controller, not the subsystem. Two controllers in a
> subsystem can have different atomic parameters.
> 
> The commit 81adb8633491 ("nvme: set physical block size and optimal I/O
> size") added subsys->awupf.
> 
> I think the awupf belongs in the nvme_ctrl structure.

No, that would be extremely dangerous.  But as NVMe has not explicit
language to forbid different values for different controllers (even
if that is really stupid) we'll need to reject adding controllers
that have lower than the initial value to not break setups.

Sigh..  I'll also reach out to NVMe to see if we can ECN this, but
we'll still need to work around this.

Alternatively we could stop supporting AWUPF entirely and require
NAWUPF which must be the same for all attachments of the controller.