Question: NVMe AER completion delivery during controller removal

[Mohamed Khalfella]

On Mon 2026-11-09 11:08:54 -0800, Mohamed Khalfella wrote:
> On Wed 2026-11-04 12:50:42 +0800, zhang san wrote:
> > Hello,
> > 
> > We are seeking clarification about asynchronous event request (AER)
> > completion behavior during NVMe controller removal, before treating an
> > observed test failure as a production-reachable driver bug.
> > 
> > Our source analysis suggests a possible lifetime race between the AEN
> > completion path, which can requeue nvme_async_event_work, and the
> > controller removal path. We observed KASAN use-after-free in an
> > emulated environment on Linux v7.4-rc1 (6628e7d4c6b2).
> 
> 
> My reading of the code suggests this can happen. nvme_stop_ctrl()
> flushes async_event_work, but the admin queue can still deliver an AER
> completion afterwards, either from the admin queue interrupt handler or
> from the final admin CQ poll in nvme_dev_disable():
> 
> nvme_remove() ->
>   nvme_dev_disable() ->
>     nvme_poll_irqdisable(&dev->queues[0]) ->
>       nvme_poll_cq() ->
>         nvme_handle_cqe() ->
>           nvme_complete_async_event() ->
> 	    queue_work(nvme_wq, &ctrl->async_event_work);
> 
> 
> Nothing flushes async_event_work after that, so the work can run after
> nvme_uninit_ctrl() drops the last reference and the nvme_dev is freed.
> 
> > 
> > However, our test induced an ERROR AEN through an out-of-specification
> > MMIO doorbell write while the device was being removed. This
> > deliberately synthetic trigger does not establish that a compliant
> > controller would generate the same completion.
> 
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -3909,20 +3909,21 @@ static void nvme_remove(struct pci_dev *pdev)
> 
>         if (!pci_device_is_present(pdev)) {
>                 nvme_change_ctrl_state(&dev->ctrl, NVME_CTRL_DEAD);
>                 nvme_dev_disable(dev, true);
>         }
> 
>         flush_work(&dev->ctrl.reset_work);
>         nvme_stop_ctrl(&dev->ctrl);
>         nvme_remove_namespaces(&dev->ctrl);
>         nvme_dev_disable(dev, true);
> +       flush_work(&dev->ctrl.async_event_work);
>         nvme_free_host_mem(dev);
>         nvme_dev_remove_admin(dev);
>         nvme_dbbuf_dma_free(dev);
>         nvme_free_queues(dev, 0);
>         mempool_destroy(dev->dmavec_mempool);
>         nvme_release_descriptor_pools(dev);
>         nvme_dev_unmap(dev);
>         nvme_uninit_ctrl(&dev->ctrl);
>  }
> 
> Since you are able to reproduce this issue reliabely, how about you try
> the diff above and see if it fixes the issue.

Looks like it has been already fixed in [1].

1- https://lore.kernel.org/all/20261105114149.970315-1-shunsheng1218@gmail.com/


> 
> > 
> > Our main question is: can a specification-compliant controller deliver
> > a completion for a previously submitted AER during the interval in
> > which the host is disabling the controller and tearing down the admin
> > queue, or must such completions be quiesced before disable completes?
> > We would also welcome guidance on the exact point after which the host
> > may safely assume that no further AER completions can be delivered.
> 
> If the above diff fixes the issue for you, then I think this is a driver
> bug.