[bug report] nvme/063 failure (tcp transport)

[Hannes Reinecke]

On 5/19/25 04:53, Shinichiro Kawasaki wrote:
> On May 18, 2025 / 13:25, Hannes Reinecke wrote:
>> On 5/18/25 12:01, Sagi Grimberg wrote:
>>>
>>>
>>> On 16/05/2025 15:31, Shinichiro Kawasaki wrote:
>>>> Hello all,
>>>>
>>>> Using the kernel v6.15-rc6 and the latest blktests (git hash
>>>> 613b8377e4d3), I
>>>> observe the test case nvme/063 fails with tcp transport. Kernel
>>>> reported WARN in
>>>> blk_mq_unquiesce_queue and KASAN sauf in blk_mq_queue_tag_busy_iter
>>>> [1]. The
>>>> failure is recreated in stable manner on my test nodes.
>>>>
>>>> The test case script had a bug then this failure was not found until
>>>> the bug get
>>>> fixed. I tried the kernel v6.15-rc1, and observed the same failure
>>>> symptom. This
>>>> test case cannot be run with the kernel v6.14, since it does not
>>>> have secure
>>>> concatenation feature.
>>>>
>>>> Actions for fix will be appreciated.
>>>
>>> Hannes, did you encounter this?
>>>
>> No; I would think it's an artifact due to multipath not being enabled.
>> Shin'ichiro, can you reproduce it with CONFIG_NVME_MULTIPATH on?
> 
> I tried both CONFIG_NVME_MULTIPATH on and off, and the failure was recreated
> regardless of the config value. Of note is that sometimes it is recreated at
> the first test case run, and sometimes it is required to repeat the test case a
> few times.
> 
> FYI, I attached the kernel config which I used to recreate the failure.

Hmm.
Can you check with this patch (on top of the previous one):


diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 55569eb7770b..43d86e8c6df3 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2197,6 +2197,7 @@ static int nvme_tcp_configure_io_queues(struct 
nvme_ctrl *ctrl, bool new)
                 blk_mq_update_nr_hw_queues(ctrl->tagset,
                         ctrl->queue_count - 1);
                 nvme_unfreeze(ctrl);
+               nvme_quiesce_io_queues(ctrl);
         }

         /*

Thanks.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare at suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich