6.13/regression/bisected - new nvme timeout errors

[Maurizio Lombardi]

On Sun Mar 23, 2025 at 10:49 AM CET, Mikhail Gavrilov wrote:
> On Thu, Mar 6, 2025 at 8:19 PM Keith Busch <kbusch at kernel.org> wrote:
>>
>> >
>> >     nvme-pci: reverse request order in nvme_queue_rqs
>>
>> The patch here uses the order recieved to dispatch commands in
>> consequetive submission queue entries, which is supposed to be the
>> desired behavior for any device. I did some testing on mailine, and it
>> sure looks like the order the driver does this is optimal, so I'm not
>> sure what's going on with your observation.
>>
>> Do you have a scheduler enabled on your device?
>>
>> How are you generating IO? Is it a pattern I should be able to replicate
>> with 'fio'?
>
> Sorry for not answering for a long time, but all because I was testing
> the off-list patch that was sent to me by Maurizio Lombardi.
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 950289405ef2..990d9ef1eef9 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -989,6 +989,9 @@ static void nvme_submit_cmds(struct nvme_queue
> *nvmeq, struct rq_list *rqlist)
>  {
>         struct request *req;
>
> +       if (rq_list_empty(rqlist))
> +               return;
> +
>         spin_lock(&nvmeq->sq_lock);
>         while ((req = rq_list_pop(rqlist))) {
>                 struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
>
> And I want to say right away that this patch solved my problem. I
> double checked, and both times the system worked for 4 days without
> errors:
>

Glad to hear it worked!

Keith, I think the problem here is simply that nvme_write_sq_db() ends up
being called even when the rqlist is empty and this couldn't happen before
the changes introduced by commit "nvme-pci: reverse request order in
nvme_queue_rqs".

If you say that this change makes sense I can submit a formal patch.

Maurizio