[PATCH] nvme: reject completions for requests that are not in flight

[K]

Thanks, pls let me take a look, it looks like all AI generated. May I
know what kind of test you run or LLM run?

Regards,
Chao

On Tue, Nov 3, 2026 at 2:17 PM Mauricio Faria de Oliveira
<mfo at igalia.com> wrote:
>
> From: Chao Shi <coshi036 at gmail.com>
>
> There are a few kernel crash reports from the Steam Deck where incorrect
> completions received by the nvme driver have a command id/tag associated
> with a freed request (rq->mq_hctx = NULL), dereferencing a NULL pointer.
>
>   nvme_handle_cqe()
>   -> nvme_try_complete_req(req, ...)
>      -> blk_mq_complete_request_remote(req)
>         -> if ((rq->mq_hctx->nr_ctx == 1 &&   /* Oops! */
>
> Although the usual suspect is the nvme device firmware, one report shows
> incorrect completions also received by the ath11k wireless driver at the
> same time, which suggests the underlying issue might be something else.
>
>   [ 2585.223155] ath11k_pci 0000:03:00.0: tx completion for unknown msdu_id 0
>   [ 2585.223163] ath11k_pci 0000:03:00.0: tx completion for unknown msdu_id 0
>   ...
>   [ 2585.228077] nvme nvme0: request 0xec genctr mismatch (got 0x2 expected 0xa)
>   [ 2585.228079] nvme nvme0: invalid id 8428 completed on queue 3
>   [ 2585.228205] nvme nvme0: request 0xe2 genctr mismatch (got 0xe expected 0x2)
>   [ 2585.228208] nvme nvme0: invalid id 57570 completed on queue 3
>   ...
>   [ 2585.228726] BUG: kernel NULL pointer dereference, address: 00000000000000fe
>   ...
>   [ 2585.228752] RIP: 0010:blk_mq_complete_request_remote+0x1c/0x160
>   ...
>   [ 2585.228787] Call Trace:
>   [ 2585.228790]  <IRQ>
>   [ 2585.228828]  nvme_poll_cq+0x14b/0x300
>   [ 2585.228838]  nvme_irq+0x41/0x80
>   [ 2585.228845]  __handle_irq_event_percpu+0x4a/0x180
>   [ 2585.228850]  handle_irq_event+0x38/0x90
>   [ 2585.228853]  handle_edge_irq+0x8b/0x230
>   [ 2585.228858]  __common_interrupt+0x3e/0x90
>   [ 2585.228862]  common_interrupt+0x80/0xa0
>   [ 2585.228866]  </IRQ>
>   [ 2585.228868]  <TASK>
>   ...
>
> However, even when the nvme device firmware is at fault and need an update,
> it is interesting for the kernel not to crash, so that it is even possible
> to run the firmware update where the issue happens frequently or early on,
> and while waiting for the update to be made available by the device vendor.
>
> The fix is to check that a request is actually in-flight before completing
> it. This has been previously reported and proposed by Chao Shi, found with
> fuzzing, but without further feedback since July. I happened to learn of it
> only after root-cause analysis and reaching a similar conclusion, checking
> for prior work/feedback, thus keeping them as Author/Signed-off-by, adding
> myself as Co-developed-by for the commit message and root cause analysis.
>
> In Chao Shi's patch, there's been feedback/question of whether this should
> be done in blk-mq, as this is not specific to nvme. I'd say no, at first:
> - Some blk-mq calls for completion have no return value (e.g., completion
>   on other CPUs), which suggests maybe an API change and updates to many
>   callers, all for an edge case.
> - An alternative available currently might be blk_mq_ops.complete(), but
>   that is handled in the driver anyway, with risk of hitting an error in
>   blk-mq (like this) as the code changes, and seems semantically wrong,
>   i.e., completing a request that should not have been completed.
> - Keeping in mind a small fix for stable, for now, should this be merged.
> - But happy to take suggestions if blk-mq is indeed the way to go.
>
> This has been tested with a synthetic reproducer produced by an LLM.
>
> Originally-by: Chao Shi <coshi036 at gmail.com>
> Signed-off-by: Chao Shi <coshi036 at gmail.com>
> Link: https://lore.kernel.org/linux-nvme/20260522153034.2168862-1-coshi036@gmail.com/
> Link: https://lore.kernel.org/linux-nvme/20260715005105.1115958-1-coshi036@gmail.com/
> Co-developed-by: Mauricio Faria de Oliveira <mfo at igalia.com>
> Signed-off-by: Mauricio Faria de Oliveira <mfo at igalia.com>
> ---
>  drivers/nvme/host/nvme.h | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 2d8f681079e85601a92624d196e77f1ba3ace34a..3ba0666da7be6a3de2e5f0fd50155af115b4e1f6 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -708,6 +708,11 @@ static inline struct request *nvme_find_rq(struct blk_mq_tags *tags,
>                         tag, genctr, nvme_genctr_mask(nvme_req(rq)->genctr));
>                 return NULL;
>         }
> +       if (unlikely(blk_mq_rq_state(rq) != MQ_RQ_IN_FLIGHT)) {
> +               dev_err(nvme_req(rq)->ctrl->device,
> +                       "completion for request %#x not in flight\n", tag);
> +               return NULL;
> +       }
>         return rq;
>  }
>
>
> ---
> base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
> change-id: 20261102-nvme-double-cmpl-uaf-blk-mq-e3d28f74cdaf
>
> Best regards,
> --
> Mauricio Faria de Oliveira <mfo at igalia.com>
>