"nvme nvmeX: IO queues not created" with "Amazon.com, Inc. NVMe SSD Controller" from 5.19.y (issue bisected) #forregzbot

[Thorsten Leemhuis]

On 08.09.22 12:41, Thorsten Leemhuis wrote:
> TWIMC: this mail is primarily send for documentation purposes and for
> regzbot, my Linux kernel regression tracking bot. These mails usually
> contain '#forregzbot' in the subject, to make them easy to spot and filter.
> 
> On 07.09.22 08:05, Jaroslav Pulchart wrote:
>> Hello,
>>
>> I would like to report a regression issue in 5.19.y in NVMe driver.
>>
>> The issue is reproducible at AWS EC2 instances with local NVMe storage
>> like "r5d.*". Kernel report "IO queues not created":
>> [    2.936641] nvme nvme0: 2/0/0 default/read/poll queues  <- EBS volume
>> [    2.939493] nvme nvme1: 2/0/0 default/read/poll queues  <- EBS volume
>> [    2.940797] nvme nvme2: IO queues not created              <- Local volume
>> with 5.19.y (kernel 5.19) and the nvme storage cannot be used.
>>
>> I bisected the issue to commit
>> "aa41d2fe60ee2e4452b0f9ca9f0f6d80a4ff9f9d" (nvme: set controller
>> enable bit in a separate write). Reverting it makes the nvme device
>> working again:
>> [    3.025599] nvme nvme0: 2/0/0 default/read/poll queues
>> [    3.032467] nvme nvme2: 8/0/0 default/read/poll queues
>> [    3.040040] nvme nvme1: 2/0/0 default/read/poll queues
> 
> #regzbot ^introduced aa41d2fe60ee2e4
> #regzbot title block: nvme: AWS EC2 instances with local NVMe storage fail
> #regzbot ignore-activity

#regzbot invalid: will be fixed by a rollout of a new firmware, for
details see the answers to this mail:
https://lore.kernel.org/linux-nvme/18e67378-1365-5e36-981b-13cada73bcda@leemhuis.info/