[PATCH] block: Fix blk_sync_queue() to properly stop timeout timer

[Keith Busch]

On Thu, May 29, 2025 at 03:49:28PM -0600, Mohamed Khalfella wrote:
> nvme-fc initiator hit hung_task with stacktrace above while handling
> request timeout call. The work thread is waiting for itself to finish
> which is never going to happen. From the stacktrace the nvme controller
> was in NVME_CTRL_CONNECTING state when nvme_fc_timeout() was called.
> We do not expect to get IO timeout call in NVME_CTRL_CONNECTING state
> because blk_sync_queue() must have been called on this queue before
> switching from NVME_CTRL_RESETTING to NVME_CTRL_CONNECTING.
> 
> It turned out that blk_sync_queue() did not stop q->timeout_work from
> running as expected. nvme_fc_timeout() returned BLK_EH_RESET_TIMER
> causing q->timeout to be rearmed after it was canceled earlier.
> q->timeout queued q->timeout_work after the controller switched to
> NVME_CTRL_CONNECTING state causing deadlock above.
> 
> Add QUEUE_FLAG_NOTIMEOUT queue flag to tell q->timeout not to queue
> q->timeout_work while queue is being synced. Update blk_sync_queue() to
> cancel q->timeout_work first and then cancel q->timeout.

I feel like this is a nvme-fc problem that doesn't need the block layer
to handle. Just don't sync the queues within the timeout workqueue
context.