What should we do about the nvme atomics mess?

[Nilay Shroff]

On 12/9/25 1:56 PM, John Garry wrote:
> On 08/12/2025 12:11, Nilay Shroff wrote:
>>>> Do you think that you could check with the OEM for updated firmware (that reports NAWUPF)?
>>> Yes, certainly — we can check with the disk vendor for updated firmware that correctly reports
>>> NAWUPF. I’ve already discussed this with my manager, and he’ll arrange a call with the vendor
>>> so that I can directly explain the issue and what we want from the kernel perspective.
>>> I’ll follow up with you once I’ve spoken to the vendor.
>>>
>> Just an update...
>>
>> We met with the vendor team and explained the current situation regarding atomic
>> writes and the NVMe Linux kernel driver. I’m happy to report that they acknowledged
>> the issue and agreed to provide updated firmware that advertises atomic write support
>> using NAWUPF instead of AWUPF.
>>
>> We shall first receive a test firmware build with this change. Once our test team
>> verifies that it behaves as expected, the vendor will proceed with a formal firmware
>> release. I’ll keep you posted once we receive the updated firmware.
> 
> Thanks for the update. It's good to know that this vendor will provide the required FW update.
> 
An update...

IBM has now formally received a firmware update that adheres to the (proposed) Linux
NVMe driver requirements for atomic write support. With this updated firmware, the
controller now reports AWUPF = 0 in the Identify Controller data structure. All 
namespace-specific atomic write limits, including NAWUPF, are reported exclusively
via the Identify Namespace data structure.
Additionally, the atomic write limits are now correctly scaled based on the LBA format,
as expected. The results below illustrate this behavior:

$ nvme list
Node                  Generic               SN                   Model                                    Namespace  Usage                      Format           FW Rev  
--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- --------
/dev/nvme5n2          /dev/ng5n2            3D60A04906N1         1.6TB NVMe Gen4 U.2 SSD IV               0x1         12.44  GB / 204.84  GB      4 KiB +  0 B   REV.BAS8
/dev/nvme5n3          /dev/ng5n3            3D60A04906N1         1.6TB NVMe Gen4 U.2 SSD IV               0x3          0.00   B /  25.61  GB    512   B +  0 B   REV.BAS8

$ nvme id-ctrl /dev/nvme5 | grep awupf
awupf     : 0

$ nvme id-ns /dev/nvme5n2 | grep -A4 nawupf
nawupf  : 63
nacwu   : 63
nabsn   : 0
nabo    : 0
nabspf  : 0

nvme id-ns /dev/nvme5n3 | grep -A4 nawupf
nawupf  : 511		==>  As expected, the atomic write limit scales naturally for 512B LBA
nacwu   : 511
nabsn   : 0
nabo    : 0
nabspf  : 0

With this update, atomic write support is now clearly and unambiguously expressed
at the namespace level, while the controller-level AWUPF is set to zero, avoiding
ambiguity and aligning with the Linux NVMe driver expectations. Based on this
outcome, we could consider:

- Recommending other vendors adopt the same reporting model, and
- As it was already proposed earlier, updating the Linux NVMe kernel driver to disregard
  atomic write support for devices that report a non-zero AWUPF at the controller level.

Thanks,
--Nilay