"nvme nvmeX: IO queues not created" with "Amazon.com, Inc. NVMe SSD Controller" from 5.19.y (issue bisected)

[Sironi, Filippo]

On 20.09.22, 21:57, "Keith Busch" <kbusch at kernel.org> wrote:
> On Tue, Sep 20, 2022 at 11:41:58AM +0200, Thorsten Leemhuis wrote:
>  > Hi, this is your Linux kernel regression tracker.
>  >
>  > On 07.09.22 11:58, Sironi, Filippo wrote:
>  > > Adding more folks (Soumyaroop, Valentin, and Zorik) involved in the investigation, which is by now concluded.
>  >
>  > Has any progress been made to get this regression fixed? I might be
>  > missing something, but from here it looks like nothing happened since
>  > two weeks. Thing is: ideally it shouldn't take this long to fix
>  > regressions in production releases, as explained in
>  > https://docs.kernel.org/process/handling-regressions.html
>
>  This is a device bug, not a kernel one. The ideal fix will come from the
>  device's vendor to bring itself into protocol complaince.
>
>  We do work around these types of problems when necessary, but we usually want a
>  statement from the vendor that they can't/won't fix it before we create new
>  quirks to maintain. The vendor has said they are investigating this and will
>  update with their conclusions. As far as I know, the ball is still in their
>  court.

Soumyaroop replied a few minutes before your email.

This has been root caused to an NVMe controller firmware latent issue
that was uncovered with the recent changes in Linux 5.19. We identified
a fix for this issue and it is rolling out in our fleet as we speak. The
rollout will finish in the next few weeks.




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879