[PATCH rfc v2 3/4] nvme-core: reduce io failover time

[Chao Leng]

On 2020/7/25 7:06, Sagi Grimberg wrote:
> From: Chao Leng <lengchao at huawei.com>
>
> We test nvme over roce fail over with multipath when 1000 namespaces
> configured, io pause more than 10 seconds. The reason: nvme_stop_queues
> will quiesce all queues for each namespace when io timeout cause path
> error. Quiesce queue wait all ongoing dispatches finished through
> synchronize_rcu, need more than 10 milliseconds for each wait,
> thus io pause more than 10 seconds.
>
> To reduce io pause time, nvme_stop_queues use
> blk_mq_quiesce_queue_nowait to quiesce the queue, nvme_stop_queues wait
> all ongoing dispatches completed after all queues has been quiesced.
>
> Signed-off-by: Chao Leng <lengchao at huawei.com>
> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
> ---
>   drivers/nvme/host/core.c | 7 ++++++-
>   1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 2ae8caa4e25f..e3fae68f7de6 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -4548,8 +4548,13 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
>   
>   	down_read(&ctrl->namespaces_rwsem);
>   	list_for_each_entry(ns, &ctrl->namespaces, list)
> -		blk_mq_quiesce_queue(ns->queue);
> +		blk_mq_quiesce_queue_nowait(ns->queue);
>   	up_read(&ctrl->namespaces_rwsem);
> +	/*
> +	 * BLK_MQ_F_BLOCKING drivers should never call us
> +	 */
> +	WARN_ON_ONCE(ctrl->tagset.flags & BLK_MQ_F_BLOCKING);

There is a little compile bug, ctrl->tagset is a pointer. it should:

WARN_ON_ONCE(ctrl->tagset->flags & BLK_MQ_F_BLOCKING);


> +	synchronize_rcu();
>   }
>   EXPORT_SYMBOL_GPL(nvme_stop_queues);
>