From wenbo.wang at memblaze.com  Mon Feb  1 07:01:09 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Mon, 1 Feb 2016 15:01:09 +0000
Subject: [PATCH 1/2] NVMe: Make surprise removal work again
In-Reply-To: <1453757017-13640-1-git-send-email-keith.busch@intel.com>
References: <1453757017-13640-1-git-send-email-keith.busch@intel.com>
Message-ID: <BJXPR01MB199DDC40C3949335FD70DEBE0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

>		blk_mq_abort_requeue_list(ns->queue);
>+		__nvme_start_queue_locked(ns);

Why is the queue stopped? Is it due to unsuccessful device reset (since device is hot removed)?

-----Original Message-----
From: Linux-nvme [mailto:linux-nvme-bounces at lists.infradead.org] On Behalf Of Keith Busch
Sent: Tuesday, January 26, 2016 5:24 AM
To: linux-nvme at lists.infradead.org; Jens Axboe
Cc: Christoph Hellwig; Keith Busch
Subject: [PATCH 1/2] NVMe: Make surprise removal work again

Ends all IO on disk removal when the controller can't respond. For device failure or surprise removal, the driver ends new requests after disabling the controller and setting the queue to dying. The h/w queue is restarted to flush pending commands so they can be failed.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 19 ++++++++++++++-----  drivers/nvme/host/pci.c  | 13 +++++++++++++
 2 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index c5bf001..37815c9 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1178,6 +1178,13 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	kfree(ns);
 }
 
+static void __nvme_start_queue_locked(struct nvme_ns *ns) {
+	queue_flag_clear_unlocked(QUEUE_FLAG_STOPPED, ns->queue);
+	blk_mq_start_stopped_hw_queues(ns->queue, true);
+	blk_mq_kick_requeue_list(ns->queue);
+}
+
 static void nvme_ns_remove(struct nvme_ns *ns)  {
 	bool kill = nvme_io_incapable(ns->ctrl) && @@ -1187,15 +1194,20 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 
 	if (kill) {
 		blk_set_queue_dying(ns->queue);
+		mb();
 
 		/*
 		 * The controller was shutdown first if we got here through
 		 * device removal. The shutdown may requeue outstanding
 		 * requests. These need to be aborted immediately so
 		 * del_gendisk doesn't block indefinitely for their completion.
+		 * The queue needs to be restarted to let pending requests
+		 * fail.
 		 */
 		blk_mq_abort_requeue_list(ns->queue);
+		__nvme_start_queue_locked(ns);
 	}
+
 	if (ns->disk->flags & GENHD_FL_UP) {
 		if (blk_get_integrity(ns->disk))
 			blk_integrity_unregister(ns->disk);
@@ -1424,11 +1436,8 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
 	struct nvme_ns *ns;
 
 	mutex_lock(&ctrl->namespaces_mutex);
-	list_for_each_entry(ns, &ctrl->namespaces, list) {
-		queue_flag_clear_unlocked(QUEUE_FLAG_STOPPED, ns->queue);
-		blk_mq_start_stopped_hw_queues(ns->queue, true);
-		blk_mq_kick_requeue_list(ns->queue);
-	}
+	list_for_each_entry(ns, &ctrl->namespaces, list)
+		__nvme_start_queue_locked(ns);
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 72ef832..bdf148e 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -640,6 +640,10 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 	struct nvme_command cmnd;
 	int ret = BLK_MQ_RQ_QUEUE_OK;
 
+	if (unlikely(blk_queue_dying(req->q))) {
+		blk_mq_end_request(req, -EIO);
+		return BLK_MQ_RQ_QUEUE_OK;
+	}
 	/*
 	 * If formated with metadata, require the block layer provide a buffer
 	 * unless this namespace is formated such that the metadata can be @@ -2118,6 +2122,15 @@ static void nvme_remove(struct pci_dev *pdev)
 	pci_set_drvdata(pdev, NULL);
 	flush_work(&dev->reset_work);
 	flush_work(&dev->scan_work);
+
+	/*
+	 * If the controller can't do IO (surprise removal, for example), we
+	 * need to quiesce prior to deleting namespaces. This ends outstanding
+	 * requests and prevents attempts to sync dirty data.
+	 */
+	if (nvme_io_incapable(&dev->ctrl))
+		nvme_dev_disable(dev, true);
+
 	nvme_remove_namespaces(&dev->ctrl);
 	nvme_uninit_ctrl(&dev->ctrl);
 	nvme_dev_disable(dev, true);
--
2.6.2.307.g37023ba


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

From keith.busch at intel.com  Mon Feb  1 07:27:24 2016
From: keith.busch at intel.com (Busch, Keith)
Date: Mon, 1 Feb 2016 15:27:24 +0000
Subject: [PATCH 1/2] NVMe: Make surprise removal work again
In-Reply-To: <BJXPR01MB199DDC40C3949335FD70DEBE0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1453757017-13640-1-git-send-email-keith.busch@intel.com>
 <BJXPR01MB199DDC40C3949335FD70DEBE0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <B58D82457FDA0744A320A2FC5AC253B93D3D7E4F@fmsmsx104.amr.corp.intel.com>

> Subject: RE: [PATCH 1/2] NVMe: Make surprise removal work again
> 
> >		blk_mq_abort_requeue_list(ns->queue);
> >+		__nvme_start_queue_locked(ns);
> 
> Why is the queue stopped? Is it due to unsuccessful device reset (since device is hot removed)?

It could be from an unsuccessful reset if that's the path we took. We also could go straight to the removal case if the pci hotplug driver notifies the driver, in which case this patch will disables the controller first to halt new IO and reap outstanding requests. This also stops queues, so needed to restart it to flush out the pending requests.

Anyway, the direction I was given was to move the request ending to the block layer when we kill it, so this won't be necessary in the next revision (will be sent out today).

From mail_weber_wang at 163.com  Mon Feb  1 07:42:04 2016
From: mail_weber_wang at 163.com (Wenbo Wang)
Date: Mon,  1 Feb 2016 10:42:04 -0500
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended
Message-ID: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>

If __nvme_submit_cmd races with nvme_dev_disable, nvmeq
could have been suspended and dev->bar could have been
unmapped. Do not touch sq door bell in this case.

Signed-off-by: Wenbo Wang <wenbo.wang at memblaze.com>
Reviewed-by: Wenwei Tao <wenwei.tao at memblaze.com>
CC: linux-nvme at lists.infradead.org
---
 drivers/nvme/host/pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 8b1a725..2288712 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -325,7 +325,8 @@ static void __nvme_submit_cmd(struct nvme_queue *nvmeq,
 
 	if (++tail == nvmeq->q_depth)
 		tail = 0;
-	writel(tail, nvmeq->q_db);
+	if (likely(nvmeq->cq_vector >= 0))
+		writel(tail, nvmeq->q_db);
 	nvmeq->sq_tail = tail;
 }
 
-- 
1.8.3.1



From keith.busch at intel.com  Mon Feb  1 07:59:42 2016
From: keith.busch at intel.com (Busch, Keith)
Date: Mon, 1 Feb 2016 15:59:42 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
Message-ID: <B58D82457FDA0744A320A2FC5AC253B93D3D7F52@fmsmsx104.amr.corp.intel.com>

Does this ever happen? The queue should be stopped before the bar is unmapped. If that's insufficient to guard against this, we've another problem this patch does not cover. That command will just timeout since it was accepted by the driver, but not actually submitted to anything. The request needs to be requeued or return BLK_MQ_RQ_QUEUE_BUSY.

> -----Original Message-----
> From: Wenbo Wang [mailto:mail_weber_wang at 163.com]
> Sent: Monday, February 01, 2016 8:42 AM
> To: axboe at fb.com; Busch, Keith
> Cc: linux-kernel at vger.kernel.org; Wenbo Wang; Wenbo Wang; linux-nvme at lists.infradead.org
> Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended
> 
> If __nvme_submit_cmd races with nvme_dev_disable, nvmeq
> could have been suspended and dev->bar could have been
> unmapped. Do not touch sq door bell in this case.
> 
> Signed-off-by: Wenbo Wang <wenbo.wang at memblaze.com>
> Reviewed-by: Wenwei Tao <wenwei.tao at memblaze.com>
> CC: linux-nvme at lists.infradead.org
> ---
>  drivers/nvme/host/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 8b1a725..2288712 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -325,7 +325,8 @@ static void __nvme_submit_cmd(struct nvme_queue *nvmeq,
> 
>  	if (++tail == nvmeq->q_depth)
>  		tail = 0;
> -	writel(tail, nvmeq->q_db);
> +	if (likely(nvmeq->cq_vector >= 0))
> +		writel(tail, nvmeq->q_db);
>  	nvmeq->sq_tail = tail;
>  }
> 
> --
> 1.8.3.1



From wenbo.wang at memblaze.com  Mon Feb  1 08:17:09 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Mon, 1 Feb 2016 16:17:09 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <B58D82457FDA0744A320A2FC5AC253B93D3D7F52@fmsmsx104.amr.corp.intel.com>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <B58D82457FDA0744A320A2FC5AC253B93D3D7F52@fmsmsx104.amr.corp.intel.com>
Message-ID: <BJXPR01MB1991E3BE20DC3FF9EC4A856E0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

No, this issue has not been seen yet. 

Since blk_mq_run_hw_queue tests queue stopped in the very beginning. It should be possible to race with nvme_dev_disable.

I agree that the request shall be re-queued.

-----Original Message-----
From: Busch, Keith [mailto:keith.busch at intel.com] 
Sent: Tuesday, February 2, 2016 12:00 AM
To: Wenbo Wang; axboe at fb.com
Cc: linux-kernel at vger.kernel.org; Wenbo Wang; linux-nvme at lists.infradead.org
Subject: RE: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended

Does this ever happen? The queue should be stopped before the bar is unmapped. If that's insufficient to guard against this, we've another problem this patch does not cover. That command will just timeout since it was accepted by the driver, but not actually submitted to anything. The request needs to be requeued or return BLK_MQ_RQ_QUEUE_BUSY.

> -----Original Message-----
> From: Wenbo Wang [mailto:mail_weber_wang at 163.com]
> Sent: Monday, February 01, 2016 8:42 AM
> To: axboe at fb.com; Busch, Keith
> Cc: linux-kernel at vger.kernel.org; Wenbo Wang; Wenbo Wang; 
> linux-nvme at lists.infradead.org
> Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been 
> suspended
> 
> If __nvme_submit_cmd races with nvme_dev_disable, nvmeq could have 
> been suspended and dev->bar could have been unmapped. Do not touch sq 
> door bell in this case.
> 
> Signed-off-by: Wenbo Wang <wenbo.wang at memblaze.com>
> Reviewed-by: Wenwei Tao <wenwei.tao at memblaze.com>
> CC: linux-nvme at lists.infradead.org
> ---
>  drivers/nvme/host/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 
> 8b1a725..2288712 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -325,7 +325,8 @@ static void __nvme_submit_cmd(struct nvme_queue 
> *nvmeq,
> 
>  	if (++tail == nvmeq->q_depth)
>  		tail = 0;
> -	writel(tail, nvmeq->q_db);
> +	if (likely(nvmeq->cq_vector >= 0))
> +		writel(tail, nvmeq->q_db);
>  	nvmeq->sq_tail = tail;
>  }
> 
> --
> 1.8.3.1


From wenbo.wang at memblaze.com  Mon Feb  1 08:36:00 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Mon, 1 Feb 2016 16:36:00 +0000
Subject: Is nvme_clear_queue() correct?
Message-ID: <BJXPR01MB1993DAAA2A2F862E41AA849E0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Hi,

nvme_clear_queue() is invoked by nvme_dev_disable. It finds all inuse tags and cancels the corresponding requests. However, it seems to have some issues:

1. Some requests might be in q->request_list, ctx->rq_list or hctx->dispatch. nvme_clear_queue does not remove them from these lists, it may cause problems.
2. Some blk_mq_run_hw_queue might be running concurrently with nvme_dev_disable, they are not aware that the requests have been canceled.

Looking forward to your comments. Thanks.

From axboe at fb.com  Mon Feb  1 08:54:13 2016
From: axboe at fb.com (Jens Axboe)
Date: Mon, 1 Feb 2016 09:54:13 -0700
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
Message-ID: <56AF8DB5.70206@fb.com>

On 02/01/2016 08:42 AM, Wenbo Wang wrote:
> If __nvme_submit_cmd races with nvme_dev_disable, nvmeq
> could have been suspended and dev->bar could have been
> unmapped. Do not touch sq door bell in this case.
>
> Signed-off-by: Wenbo Wang <wenbo.wang at memblaze.com>
> Reviewed-by: Wenwei Tao <wenwei.tao at memblaze.com>
> CC: linux-nvme at lists.infradead.org
> ---
>   drivers/nvme/host/pci.c | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 8b1a725..2288712 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -325,7 +325,8 @@ static void __nvme_submit_cmd(struct nvme_queue *nvmeq,
>
>   	if (++tail == nvmeq->q_depth)
>   		tail = 0;
> -	writel(tail, nvmeq->q_db);
> +	if (likely(nvmeq->cq_vector >= 0))
> +		writel(tail, nvmeq->q_db);
>   	nvmeq->sq_tail = tail;

What Keith said (this should not happen), and additionally, this won't 
work for a polled CQ without a vector.

-- 
Jens Axboe



From keith.busch at intel.com  Mon Feb  1 17:17:52 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 01:17:52 +0000
Subject: [PATCH 1/2] NVMe: Make surprise removal work again
In-Reply-To: <B58D82457FDA0744A320A2FC5AC253B93D3D7E57@fmsmsx104.amr.corp.intel.com>
References: <1453757017-13640-1-git-send-email-keith.busch@intel.com>
 <BJXPR01MB199DDC40C3949335FD70DEBE0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <B58D82457FDA0744A320A2FC5AC253B93D3D7E57@fmsmsx104.amr.corp.intel.com>
Message-ID: <20160202011752.GB4679@localhost.localdomain>

On Mon, Feb 01, 2016 at 07:27:23AM -0800, Busch, Keith wrote:
> the direction I was given was to move the request ending to the block layer
> when we kill it, so this won't be necessary in the next revision (will be
> sent out today).

After merging and moving io ending to the block layer, something appears
broken. Not sure what's going on, so just posting here in case there's
better ideas.

The test runs buffered writes to an nvme drive (ex: dd if=/dev/zero
of=/dev/nvme0n1 bs=16M), then yank the drive when that ramps up.

Device removal completes after a few seconds, and /dev/nvme0n1 is no
longer present. However, the 'dd' task never completes, with kernel
stack trace:

[<ffffffff81093fb0>] __mod_timer+0xd4/0xe6
[<ffffffff810939bf>] process_timeout+0x0/0xc
[<ffffffff810fcad7>] balance_dirty_pages_ratelimited+0x8b1/0xa05
[<ffffffff8116a2a3>] __set_page_dirty.constprop.61+0x81/0x9f
[<ffffffff810f3457>] generic_perform_write+0x15a/0x1d1
[<ffffffff81157095>] generic_update_time+0x9f/0xaa
[<ffffffff810f4965>] __generic_file_write_iter+0xea/0x146
[<ffffffff8116d2a3>] blkdev_write_iter+0x78/0xf5
[<ffffffff811429b4>] __vfs_write+0x83/0xab
[<ffffffff81143cda>] vfs_write+0x87/0xdd
[<ffffffff81143ed7>] SyS_write+0x56/0x8a
[<ffffffff81472657>] entry_SYSCALL_64_fastpath+0x12/0x6a
[<ffffffffffffffff>] 0xffffffffffffffff

If driver ends all IO's it knows about and blk_cleanup_queue returns,
then the driver did it's part as far as I know. Not sure how to get this
to end with the expected IO error, but I'm pretty sure this used to work.


From wenbo.wang at memblaze.com  Mon Feb  1 18:41:53 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Tue, 2 Feb 2016 02:41:53 +0000
Subject: [PATCH 1/2] NVMe: Make surprise removal work again
In-Reply-To: <20160202011752.GB4679@localhost.localdomain>
References: <1453757017-13640-1-git-send-email-keith.busch@intel.com>
 <BJXPR01MB199DDC40C3949335FD70DEBE0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <B58D82457FDA0744A320A2FC5AC253B93D3D7E57@fmsmsx104.amr.corp.intel.com>
 <20160202011752.GB4679@localhost.localdomain>
Message-ID: <BJXPR01MB199DD66107F6C2D86BB3F83E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Is it confirmed that blk_cleanup_queue has returned? It is likely the driver stuck in del_gendisk.

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Tuesday, February 2, 2016 9:18 AM
To: Wenbo Wang; linux-nvme at lists.infradead.org; Jens Axboe
Cc: Christoph Hellwig
Subject: Re: [PATCH 1/2] NVMe: Make surprise removal work again

On Mon, Feb 01, 2016 at 07:27:23AM -0800, Busch, Keith wrote:
> the direction I was given was to move the request ending to the block 
> layer when we kill it, so this won't be necessary in the next revision 
> (will be sent out today).

After merging and moving io ending to the block layer, something appears broken. Not sure what's going on, so just posting here in case there's better ideas.

The test runs buffered writes to an nvme drive (ex: dd if=/dev/zero
of=/dev/nvme0n1 bs=16M), then yank the drive when that ramps up.

Device removal completes after a few seconds, and /dev/nvme0n1 is no longer present. However, the 'dd' task never completes, with kernel stack trace:

[<ffffffff81093fb0>] __mod_timer+0xd4/0xe6 [<ffffffff810939bf>] process_timeout+0x0/0xc [<ffffffff810fcad7>] balance_dirty_pages_ratelimited+0x8b1/0xa05
[<ffffffff8116a2a3>] __set_page_dirty.constprop.61+0x81/0x9f
[<ffffffff810f3457>] generic_perform_write+0x15a/0x1d1 [<ffffffff81157095>] generic_update_time+0x9f/0xaa [<ffffffff810f4965>] __generic_file_write_iter+0xea/0x146
[<ffffffff8116d2a3>] blkdev_write_iter+0x78/0xf5 [<ffffffff811429b4>] __vfs_write+0x83/0xab [<ffffffff81143cda>] vfs_write+0x87/0xdd [<ffffffff81143ed7>] SyS_write+0x56/0x8a [<ffffffff81472657>] entry_SYSCALL_64_fastpath+0x12/0x6a
[<ffffffffffffffff>] 0xffffffffffffffff

If driver ends all IO's it knows about and blk_cleanup_queue returns, then the driver did it's part as far as I know. Not sure how to get this to end with the expected IO error, but I'm pretty sure this used to work.

From keith.busch at intel.com  Mon Feb  1 21:29:16 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 05:29:16 +0000
Subject: [PATCH 1/2] NVMe: Make surprise removal work again
In-Reply-To: <BJXPR01MB199DD66107F6C2D86BB3F83E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1453757017-13640-1-git-send-email-keith.busch@intel.com>
 <BJXPR01MB199DDC40C3949335FD70DEBE0DE0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <B58D82457FDA0744A320A2FC5AC253B93D3D7E57@fmsmsx104.amr.corp.intel.com>
 <20160202011752.GB4679@localhost.localdomain>
 <BJXPR01MB199DD66107F6C2D86BB3F83E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160202052915.GA6696@localhost.localdomain>

On Tue, Feb 02, 2016 at 02:41:53AM +0000, Wenbo Wang wrote:
> Is it confirmed that blk_cleanup_queue has returned? It is likely the driver stuck in del_gendisk.

Yes, the entire driver's remove completes, which includes
blk_cleanup_queue and del_gendisk.

I don't know what the task is waiting for, but recall this worked
before. Might bisect.


From wenbo.wang at memblaze.com  Mon Feb  1 23:15:57 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Tue, 2 Feb 2016 07:15:57 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <56AF8DB5.70206@fb.com>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
Message-ID: <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Jens,

I did the following test to validate the issue.

1. Modify code as below to increase the chance of races.
	Add 10s delay after nvme_dev_unmap() in nvme_dev_disable()
	Add 10s delay before __nvme_submit_cmd()
2. Run dd and at the same time, echo 1 to reset_controller to trigger device reset. Finally kernel crashes due to accessing unmapped door bell register.

Following is the execution order of the two code paths:
__blk_mq_run_hw_queue
  Test BLK_MQ_S_STOPPED
					nvme_dev_disable()
					     nvme_stop_queues()  <-- set BLK_MQ_S_STOPPED
					     nvme_dev_unmap(dev)  <-- unmap door bell
  nvme_queue_rq()
      Touch door bell	<-- panic here

-----Original Message-----
From: Jens Axboe [mailto:axboe at fb.com] 
Sent: Tuesday, February 2, 2016 12:54 AM
To: Wenbo Wang; keith.busch at intel.com
Cc: linux-kernel at vger.kernel.org; Wenbo Wang; linux-nvme at lists.infradead.org
Subject: Re: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended

On 02/01/2016 08:42 AM, Wenbo Wang wrote:
> If __nvme_submit_cmd races with nvme_dev_disable, nvmeq could have 
> been suspended and dev->bar could have been unmapped. Do not touch sq 
> door bell in this case.
>
> Signed-off-by: Wenbo Wang <wenbo.wang at memblaze.com>
> Reviewed-by: Wenwei Tao <wenwei.tao at memblaze.com>
> CC: linux-nvme at lists.infradead.org
> ---
>   drivers/nvme/host/pci.c | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 
> 8b1a725..2288712 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -325,7 +325,8 @@ static void __nvme_submit_cmd(struct nvme_queue 
> *nvmeq,
>
>   	if (++tail == nvmeq->q_depth)
>   		tail = 0;
> -	writel(tail, nvmeq->q_db);
> +	if (likely(nvmeq->cq_vector >= 0))
> +		writel(tail, nvmeq->q_db);
>   	nvmeq->sq_tail = tail;

What Keith said (this should not happen), and additionally, this won't work for a polled CQ without a vector.

--
Jens Axboe


From sagig at dev.mellanox.co.il  Tue Feb  2 04:41:37 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Tue, 2 Feb 2016 14:41:37 +0200
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <56B0A401.30306@dev.mellanox.co.il>


> Jens,
>
> I did the following test to validate the issue.
>
> 1. Modify code as below to increase the chance of races.
> 	Add 10s delay after nvme_dev_unmap() in nvme_dev_disable()
> 	Add 10s delay before __nvme_submit_cmd()
> 2. Run dd and at the same time, echo 1 to reset_controller to trigger device reset. Finally kernel crashes due to accessing unmapped door bell register.
>
> Following is the execution order of the two code paths:
> __blk_mq_run_hw_queue
>    Test BLK_MQ_S_STOPPED
> 					nvme_dev_disable()
> 					     nvme_stop_queues()  <-- set BLK_MQ_S_STOPPED
> 					     nvme_dev_unmap(dev)  <-- unmap door bell
>    nvme_queue_rq()
>        Touch door bell	<-- panic here

First of all, I think we need to cancel all
inflight requests before nvme_dev_unmap.

With my patches that move I/O termination to the
nvme core ([PATCH v1 0/3] Move active IO termination to the core)
the change needed is:
--
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e921165..2288bdb 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1890,10 +1890,11 @@ static void nvme_dev_shutdown(struct nvme_dev *dev)
                 nvme_shutdown_ctrl(&dev->ctrl);
                 nvme_disable_queue(dev, 0);
         }
-       nvme_dev_unmap(dev);

         blk_mq_tagset_busy_iter(&dev->tagset, nvme_cancel_io, dev);
         blk_mq_tagset_busy_iter(&dev->admin_tagset, nvme_cancel_io, dev);
+
+       nvme_dev_unmap(dev);
  }

  static int nvme_setup_prp_pools(struct nvme_dev *dev)
--

But still we need a way to wait out for all active
queue_rq to end. It seems like we need to maintain
the request_fn_active for blk-mq and provide an
API (blk_mq_wait_for_active_requests ?) that waits for
it to drop to zero.

Thoughts?


From sagig at mellanox.com  Tue Feb  2 04:44:58 2016
From: sagig at mellanox.com (Sagi Grimberg)
Date: Tue,  2 Feb 2016 14:44:58 +0200
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
Message-ID: <1454417099-19332-3-git-send-email-sagig@mellanox.com>

blk-mq offers a tagset iterator so let's use that
instead of using nvme_clear_queues.

Note, we changed nvme_queue_cancel_ios name to nvme_cancel_io
as there is no concept of a queue now in this function (we
also lost the print).

Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Acked-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 6e80390d8bd5..7872dd884c6b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -991,16 +991,15 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved)
 	return BLK_EH_RESET_TIMER;
 }
 
-static void nvme_cancel_queue_ios(struct request *req, void *data, bool reserved)
+static void nvme_cancel_io(struct request *req, void *data, bool reserved)
 {
-	struct nvme_queue *nvmeq = data;
+	struct nvme_dev *dev = data;
 	int status;
 
 	if (!blk_mq_request_started(req))
 		return;
 
-	dev_warn(nvmeq->q_dmadev,
-		 "Cancelling I/O %d QID %d\n", req->tag, nvmeq->qid);
+	dev_warn(dev->dev, "Cancelling I/O %d\n", req->tag);
 
 	status = NVME_SC_ABORT_REQ;
 	if (blk_queue_dying(req->q))
@@ -1057,14 +1056,6 @@ static int nvme_suspend_queue(struct nvme_queue *nvmeq)
 	return 0;
 }
 
-static void nvme_clear_queue(struct nvme_queue *nvmeq)
-{
-	spin_lock_irq(&nvmeq->q_lock);
-	if (nvmeq->tags && *nvmeq->tags)
-		blk_mq_all_tag_busy_iter(*nvmeq->tags, nvme_cancel_queue_ios, nvmeq);
-	spin_unlock_irq(&nvmeq->q_lock);
-}
-
 static void nvme_disable_admin_queue(struct nvme_dev *dev, bool shutdown)
 {
 	struct nvme_queue *nvmeq = dev->queues[0];
@@ -1836,8 +1827,8 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 	}
 	nvme_dev_unmap(dev);
 
-	for (i = dev->queue_count - 1; i >= 0; i--)
-		nvme_clear_queue(dev->queues[i]);
+	blk_mq_tagset_busy_iter(&dev->tagset, nvme_cancel_io, dev);
+	blk_mq_tagset_busy_iter(&dev->admin_tagset, nvme_cancel_io, dev);
 	mutex_unlock(&dev->shutdown_lock);
 }
 
-- 
1.8.4.3



From sagig at mellanox.com  Tue Feb  2 04:44:56 2016
From: sagig at mellanox.com (Sagi Grimberg)
Date: Tue,  2 Feb 2016 14:44:56 +0200
Subject: [PATCH RESEND v1 0/3] Move active IO termination to the core
Message-ID: <1454417099-19332-1-git-send-email-sagig@mellanox.com>

Resending with Jens on the CC list.

This patchset adds helper to active IO termination to the nvme
core based on the pci nvme_clear_queues() functionality. This
is needed for live shutdowns and resets during live IO and it's
not pci specific and will be needed for other transports as well.

This patchset applies on top of Keith's "NVMe fixes and updates for 4.5"
patchset.

Changes from v0:
- Removed the nvme core helpers for IO termination and have pci
  use blk-mq helper directly
- Rebased on top of Keith and Christoph's patches

Sagi Grimberg (3):
  blk-mq: Export tagset iter function
  nvme: Use blk-mq helper for IO termination
  blk-mq: Make blk_mq_all_tag_busy_iter static

 block/blk-mq-tag.c      | 15 ++++++++++++---
 drivers/nvme/host/pci.c | 19 +++++--------------
 include/linux/blk-mq.h  |  4 ++--
 3 files changed, 19 insertions(+), 19 deletions(-)

-- 
1.8.4.3



From sagig at mellanox.com  Tue Feb  2 04:44:57 2016
From: sagig at mellanox.com (Sagi Grimberg)
Date: Tue,  2 Feb 2016 14:44:57 +0200
Subject: [PATCH v1 1/3] blk-mq: Export tagset iter function
In-Reply-To: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
Message-ID: <1454417099-19332-2-git-send-email-sagig@mellanox.com>

Its useful to iterate on all the active tags in cases
where we will need to fail all the queues IO.

Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
---
 block/blk-mq-tag.c     | 10 ++++++++++
 include/linux/blk-mq.h |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index abdbb47405cb..39660ca8945b 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -474,6 +474,16 @@ void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags, busy_tag_iter_fn *fn,
 }
 EXPORT_SYMBOL(blk_mq_all_tag_busy_iter);
 
+void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset,
+		busy_tag_iter_fn *fn, void *priv)
+{
+	int i;
+
+	for (i = 0; i < tagset->nr_hw_queues; i++)
+		blk_mq_all_tag_busy_iter(tagset->tags[i], fn, priv);
+}
+EXPORT_SYMBOL(blk_mq_tagset_busy_iter);
+
 void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn,
 		void *priv)
 {
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 7fc9296b5742..ecab7b2b6a06 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -240,6 +240,8 @@ void blk_mq_run_hw_queues(struct request_queue *q, bool async);
 void blk_mq_delay_queue(struct blk_mq_hw_ctx *hctx, unsigned long msecs);
 void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags, busy_tag_iter_fn *fn,
 		void *priv);
+void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset,
+		busy_tag_iter_fn *fn, void *priv);
 void blk_mq_freeze_queue(struct request_queue *q);
 void blk_mq_unfreeze_queue(struct request_queue *q);
 void blk_mq_freeze_queue_start(struct request_queue *q);
-- 
1.8.4.3



From sagig at mellanox.com  Tue Feb  2 04:44:59 2016
From: sagig at mellanox.com (Sagi Grimberg)
Date: Tue,  2 Feb 2016 14:44:59 +0200
Subject: [PATCH v1 3/3] blk-mq: Make blk_mq_all_tag_busy_iter static
In-Reply-To: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
Message-ID: <1454417099-19332-4-git-send-email-sagig@mellanox.com>

No caller outside the blk-mq code so we can settle
with it static.

Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
---
 block/blk-mq-tag.c     | 5 ++---
 include/linux/blk-mq.h | 2 --
 2 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index 39660ca8945b..be635faaeae8 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -464,15 +464,14 @@ static void bt_tags_for_each(struct blk_mq_tags *tags,
 	}
 }
 
-void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags, busy_tag_iter_fn *fn,
-		void *priv)
+static void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags,
+		busy_tag_iter_fn *fn, void *priv)
 {
 	if (tags->nr_reserved_tags)
 		bt_tags_for_each(tags, &tags->breserved_tags, 0, fn, priv, true);
 	bt_tags_for_each(tags, &tags->bitmap_tags, tags->nr_reserved_tags, fn, priv,
 			false);
 }
-EXPORT_SYMBOL(blk_mq_all_tag_busy_iter);
 
 void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset,
 		busy_tag_iter_fn *fn, void *priv)
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index ecab7b2b6a06..60a19aaf51ad 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -238,8 +238,6 @@ void blk_mq_start_hw_queues(struct request_queue *q);
 void blk_mq_start_stopped_hw_queues(struct request_queue *q, bool async);
 void blk_mq_run_hw_queues(struct request_queue *q, bool async);
 void blk_mq_delay_queue(struct blk_mq_hw_ctx *hctx, unsigned long msecs);
-void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags, busy_tag_iter_fn *fn,
-		void *priv);
 void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset,
 		busy_tag_iter_fn *fn, void *priv);
 void blk_mq_freeze_queue(struct request_queue *q);
-- 
1.8.4.3



From keith.busch at intel.com  Tue Feb  2 06:27:57 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 14:27:57 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <56B0A401.30306@dev.mellanox.co.il>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <56B0A401.30306@dev.mellanox.co.il>
Message-ID: <20160202142756.GA10690@localhost.localdomain>

On Tue, Feb 02, 2016 at 02:41:37PM +0200, Sagi Grimberg wrote:
> First of all, I think we need to cancel all
> inflight requests before nvme_dev_unmap.

IO cancelling is where it is because it protects against host memory
corruption. If you're going to mess with the ordering, just make sure
the PCI device is disabled from bus mastering first.


From sagig at dev.mellanox.co.il  Tue Feb  2 06:33:10 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Tue, 2 Feb 2016 16:33:10 +0200
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <20160202142756.GA10690@localhost.localdomain>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <56B0A401.30306@dev.mellanox.co.il>
 <20160202142756.GA10690@localhost.localdomain>
Message-ID: <56B0BE26.6070208@dev.mellanox.co.il>

Hey Keith,

>> First of all, I think we need to cancel all
>> inflight requests before nvme_dev_unmap.
>
> IO cancelling is where it is because it protects against host memory
> corruption. If you're going to mess with the ordering, just make sure
> the PCI device is disabled from bus mastering first.

Little help? :)

What corruption is the ordering protecting against?


From keith.busch at intel.com  Tue Feb  2 06:46:48 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 14:46:48 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <56B0BE26.6070208@dev.mellanox.co.il>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <56B0A401.30306@dev.mellanox.co.il>
 <20160202142756.GA10690@localhost.localdomain>
 <56B0BE26.6070208@dev.mellanox.co.il>
Message-ID: <20160202144648.GA10728@localhost.localdomain>

On Tue, Feb 02, 2016 at 04:33:10PM +0200, Sagi Grimberg wrote:
> Hey Keith,
> 
> >>First of all, I think we need to cancel all
> >>inflight requests before nvme_dev_unmap.
> >
> >IO cancelling is where it is because it protects against host memory
> >corruption. If you're going to mess with the ordering, just make sure
> >the PCI device is disabled from bus mastering first.
> 
> Little help? :)
> 
> What corruption is the ordering protecting against?

Sure thing. :)

We free the transfer buffers when a command is cancelled. The controller,
however, may still own the command and may try to write to them. We
have to fence the controller off from being able to do that, so we can't
cancel inflight commands while the PCI device is still bus master enabled.

In a perfect world, we could trust in disabling with NVMe registers,
but sometimes we can't rely on that.

This was commit 07836e659c81ec6b0d683dfbf7958339a22a7b69, which might
explain the scenario a little better, and was reported by end user.


From wenbo.wang at memblaze.com  Tue Feb  2 08:32:33 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Tue, 2 Feb 2016 16:32:33 +0000
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <1454417099-19332-3-git-send-email-sagig@mellanox.com>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
 <1454417099-19332-3-git-send-email-sagig@mellanox.com>
Message-ID: <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

If some requests are in q->request_list or ctx->rq_list, does nvme_cancel_io remove them from these lists? If not, will it cause any issue?

-----Original Message-----
From: Linux-nvme [mailto:linux-nvme-bounces at lists.infradead.org] On Behalf Of Sagi Grimberg
Sent: Tuesday, February 2, 2016 8:45 PM
To: Jens Axboe
Cc: linux-nvme at lists.infradead.org
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination

blk-mq offers a tagset iterator so let's use that instead of using nvme_clear_queues.

Note, we changed nvme_queue_cancel_ios name to nvme_cancel_io as there is no concept of a queue now in this function (we also lost the print).

Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Acked-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 6e80390d8bd5..7872dd884c6b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -991,16 +991,15 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved)
 	return BLK_EH_RESET_TIMER;
 }
 
-static void nvme_cancel_queue_ios(struct request *req, void *data, bool reserved)
+static void nvme_cancel_io(struct request *req, void *data, bool 
+reserved)
 {
-	struct nvme_queue *nvmeq = data;
+	struct nvme_dev *dev = data;
 	int status;
 
 	if (!blk_mq_request_started(req))
 		return;
 
-	dev_warn(nvmeq->q_dmadev,
-		 "Cancelling I/O %d QID %d\n", req->tag, nvmeq->qid);
+	dev_warn(dev->dev, "Cancelling I/O %d\n", req->tag);
 
 	status = NVME_SC_ABORT_REQ;
 	if (blk_queue_dying(req->q))
@@ -1057,14 +1056,6 @@ static int nvme_suspend_queue(struct nvme_queue *nvmeq)
 	return 0;
 }
 
-static void nvme_clear_queue(struct nvme_queue *nvmeq) -{
-	spin_lock_irq(&nvmeq->q_lock);
-	if (nvmeq->tags && *nvmeq->tags)
-		blk_mq_all_tag_busy_iter(*nvmeq->tags, nvme_cancel_queue_ios, nvmeq);
-	spin_unlock_irq(&nvmeq->q_lock);
-}
-
 static void nvme_disable_admin_queue(struct nvme_dev *dev, bool shutdown)  {
 	struct nvme_queue *nvmeq = dev->queues[0]; @@ -1836,8 +1827,8 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 	}
 	nvme_dev_unmap(dev);
 
-	for (i = dev->queue_count - 1; i >= 0; i--)
-		nvme_clear_queue(dev->queues[i]);
+	blk_mq_tagset_busy_iter(&dev->tagset, nvme_cancel_io, dev);
+	blk_mq_tagset_busy_iter(&dev->admin_tagset, nvme_cancel_io, dev);
 	mutex_unlock(&dev->shutdown_lock);
 }
 
--
1.8.4.3


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

From keith.busch at intel.com  Tue Feb  2 08:49:15 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 16:49:15 +0000
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
 <1454417099-19332-3-git-send-email-sagig@mellanox.com>
 <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160202164915.GB10728@localhost.localdomain>

On Tue, Feb 02, 2016 at 04:32:33PM +0000, Wenbo Wang wrote:
> If some requests are in q->request_list or ctx->rq_list, does nvme_cancel_io remove them from these lists? If not, will it cause any issue?

It's fine to let them queue there if we expect to restart the
controller. We're going to restart the queues shortly.

If we're about to kill the queue, blk-mq's timeout handler kills
unstarted requests.


From sagig at dev.mellanox.co.il  Tue Feb  2 09:20:01 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Tue, 2 Feb 2016 19:20:01 +0200
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <20160202144648.GA10728@localhost.localdomain>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <56B0A401.30306@dev.mellanox.co.il>
 <20160202142756.GA10690@localhost.localdomain>
 <56B0BE26.6070208@dev.mellanox.co.il>
 <20160202144648.GA10728@localhost.localdomain>
Message-ID: <56B0E541.1000907@dev.mellanox.co.il>


> We free the transfer buffers when a command is cancelled. The controller,
> however, may still own the command and may try to write to them. We
> have to fence the controller off from being able to do that, so we can't
> cancel inflight commands while the PCI device is still bus master enabled.
>
> In a perfect world, we could trust in disabling with NVMe registers,
> but sometimes we can't rely on that.

OK, I wasn't aware that we cannot rely on that.

So it looks like we cannot change the ordering. So this leaves us with
the need to guarantee that no queue_rq is inflight before we unmap.


From keith.busch at intel.com  Tue Feb  2 09:25:03 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 17:25:03 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160202172503.GC10728@localhost.localdomain>

On Tue, Feb 02, 2016 at 07:15:57AM +0000, Wenbo Wang wrote:
> Jens,
> 
> I did the following test to validate the issue.
> 
> 1. Modify code as below to increase the chance of races.
> 	Add 10s delay after nvme_dev_unmap() in nvme_dev_disable()
> 	Add 10s delay before __nvme_submit_cmd()

If running sync IO, preempt is disabled. You can't just put a 10 second
delay there. Wouldn't you hit a "scheduling while atomic" bug instead?

If blk-mq is running the h/w context from its work queue, that might
be a different issue. Maybe we can change the "cancel_delayed_work" to
"cancel_delayed_work_sync" in blk_mq_stop_hw_queues.

If there's still a window where blk-mq can insert a request after the
driver requested to stop queues, I think we should try to close it with
the block layer.


From mark_jacobson at stackvelocity.com  Tue Feb  2 15:31:22 2016
From: mark_jacobson at stackvelocity.com (Mark Jacobson)
Date: Wed, 3 Feb 2016 00:31:22 +0100
Subject: NVMe and IRQ Affinity
Message-ID: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>

Hello there,

I had a question regarding the NVMe driver and IRQ affinity. For
context, I'm running Centos 7.1 with the 4.2.3-1.el7.elrepo.x86_64
kernel on a system with two E5-2650 v3 CPUs.

I've been battling soft core lockups in a system with 24 NVMe drives
and noticed that the drives I'm working with (Samsung PM953) will by
default only route interrupts to CPU0 despite having affinity for all
cores and I figured I'd ask here since that seemed like a driver
issue.

I've tried adjusting the smp_affinity for the various IRQ handlers,
but 1) That seems to essentially force (most of the) interrupts up to
the higher MSI-X vectors that do still have CPU 0 available, and 2)
there seems to be a huge performance hit swapping to another CPU (say
CPU2).  Is this something that makes any kind of sense from a driver
perspective? (I know high-speed NICs commonly distribute interrupts
across many cores, so it doesn't seem normal to me.) Is there a newer
kernel version that would help with this problem? Or barring any of
that, does this seem like a hardware issue?

(And yes, I realize eventually my IRQ masks should only point to cores
associated with the NUMA node they're connected to. I've been ignoring
that for debugging.)

Trimmed interrupt table (forgive the many, many wraps) below. The
spikes on cores on interrupt 159 are for when I forced interrupts away
from CPU0 by masking off some of the lower order irq affinity bits.

 159:  145078914          0          0          0   11994376
0          0          0    5139627          0          0          0
      0          0          0          0          0          0
 0          0    2766162
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058624-edge
   nvme7q0, nvme7q1
 160:   12514715          0          0          0          0
0          0          0          0          0          0          0
      0          0          0          0          0          0
 0          0          0
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058625-edge
   nvme7q2
 161:     985758          0          0          0          0
0          0          0          0          0          0          0
      0          0          0          0          0          0
 0          0          0
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058626-edge
   nvme7q3
 162:    6155130          0          0          0          0
0          0          0          0          0          0          0
      0          0          0          0          0          0
 0          0          0
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058627-edge
   nvme7q4
 163:      61111          0          0          0          0
0          0          0          0          0          0          0
      0          0          0          0          0          0
 0          0          0
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058628-edge
   nvme7q5
 164:      53257          0          0          0          0
0          0          0          0          0          0          0
      0          0          0          0          0          0
 0          0          0
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058629-edge
   nvme7q6
 165:      52186          0          0          0          0
0          0          0          0          0          0          0
      0          0          0          0          0          0
 0          0          0
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058630-edge
   nvme7q7
 166:      28943          0          0          0          0
0          0          0          0          0          0          0
      0          0          0          0          0          0
 0          0          0
         0          0          0          0          0          0
    0          0          0          0          0          0
0          0          0          0          0          0          0
IR-PCI-MSI 12058631-edge
   nvme7q8

Any help would be appreciated!

Thank you,

Mark Jacobson
Software Test Engineer
Stack Velocity


From keith.busch at intel.com  Tue Feb  2 15:45:40 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 23:45:40 +0000
Subject: NVMe and IRQ Affinity
In-Reply-To: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
Message-ID: <20160202234540.GA21046@localhost.localdomain>

On Wed, Feb 03, 2016 at 12:31:22AM +0100, Mark Jacobson wrote:
> and noticed that the drives I'm working with (Samsung PM953) will by
> default only route interrupts to CPU0 despite having affinity for all
> cores and I figured I'd ask here since that seemed like a driver
> issue.

Sounds like the affinity hints are either messed up in this distro, or
just not being used by irqbalance. Could you run the following script
and send the output?

---
cat /sys/block/nvme0n1/mq/*/cpu_list

for i in $(grep nvme0q /proc/interrupts  | cut -d":" -f1 | sed "s/ //g"); do
  echo "IRQ:  $i";
  echo -n "HINT: " && cat /proc/irq/$i/affinity_hint
  echo -n "SMP:  " && cat /proc/irq/$i/smp_affinity
done


From mark_jacobson at stackvelocity.com  Tue Feb  2 15:50:06 2016
From: mark_jacobson at stackvelocity.com (Mark Jacobson)
Date: Wed, 3 Feb 2016 00:50:06 +0100
Subject: NVMe and IRQ Affinity
In-Reply-To: <20160202234540.GA21046@localhost.localdomain>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
 <20160202234540.GA21046@localhost.localdomain>
Message-ID: <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>

Output is below. I'm aware the distro hints are fairly invalid.
Luckily, I've had to implement PCIe endpoints (in FPGAs) in the past,
so I knew roughly where to look. Note that despite the 00,3ff003ff,
only CPU0 ever gets hit unless I force-disable that bit.

root# cat /sys/block/nvme0n1/mq/*/cpu_list
0, 1, 2, 20, 21, 22
3, 4, 23, 24
5, 6, 7, 25, 26, 27
8, 9, 28, 29
10, 11, 12, 30, 31, 32
13, 14, 33, 34
15, 16, 17, 35, 36, 37
18, 19, 38, 39
root#
root# for i in $(grep nvme0q /proc/interrupts  | cut -d":" -f1 | sed
"s/ //g"); do
>   echo "IRQ:  $i";
>   echo -n "HINT: " && cat /proc/irq/$i/affinity_hint
>   echo -n "SMP:  " && cat /proc/irq/$i/smp_affinity
> done
IRQ:  87
HINT: ff,ffffffff
SMP:  ff,ffffffff
IRQ:  88
HINT: 00,00000000
SMP:  00,3ff003ff
IRQ:  89
HINT: 00,00000000
SMP:  00,3ff003ff
IRQ:  90
HINT: 00,00000000
SMP:  00,3ff003ff
IRQ:  91
HINT: 00,00000000
SMP:  00,3ff003ff
IRQ:  92
HINT: 00,00000000
SMP:  00,3ff003ff
IRQ:  93
HINT: 00,00000000
SMP:  00,3ff003ff
IRQ:  94
HINT: 00,00000000
SMP:  00,3ff003ff
Thank you,

Mark Jacobson
Software Test Engineer
Stack Velocity


On Wed, Feb 3, 2016 at 12:45 AM, Keith Busch <keith.busch at intel.com> wrote:
> On Wed, Feb 03, 2016 at 12:31:22AM +0100, Mark Jacobson wrote:
>> and noticed that the drives I'm working with (Samsung PM953) will by
>> default only route interrupts to CPU0 despite having affinity for all
>> cores and I figured I'd ask here since that seemed like a driver
>> issue.
>
> Sounds like the affinity hints are either messed up in this distro, or
> just not being used by irqbalance. Could you run the following script
> and send the output?
>
> ---
> cat /sys/block/nvme0n1/mq/*/cpu_list
>
> for i in $(grep nvme0q /proc/interrupts  | cut -d":" -f1 | sed "s/ //g"); do
>   echo "IRQ:  $i";
>   echo -n "HINT: " && cat /proc/irq/$i/affinity_hint
>   echo -n "SMP:  " && cat /proc/irq/$i/smp_affinity
> done


From keith.busch at intel.com  Tue Feb  2 15:58:19 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 2 Feb 2016 23:58:19 +0000
Subject: NVMe and IRQ Affinity
In-Reply-To: <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
 <20160202234540.GA21046@localhost.localdomain>
 <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>
Message-ID: <20160202235818.GB21046@localhost.localdomain>

On Wed, Feb 03, 2016 at 12:50:06AM +0100, Mark Jacobson wrote:
> Output is below. I'm aware the distro hints are fairly invalid.

They're all invalid. This kernel must have forked before the affinity
hints were fixed for a blk-mq nvme driver. A more optimal affinity hint
would match the mq's cpu_list, which is how it looks upstream.

I guess your platform strongly prefers CPU 0 when allowed. You can
either manually override the smp_affinity, use an out-of-tree driver
with a fix and let irqbalance handle it.


From mark_jacobson at stackvelocity.com  Tue Feb  2 16:13:23 2016
From: mark_jacobson at stackvelocity.com (Mark Jacobson)
Date: Wed, 3 Feb 2016 01:13:23 +0100
Subject: NVMe and IRQ Affinity
In-Reply-To: <20160202235818.GB21046@localhost.localdomain>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
 <20160202234540.GA21046@localhost.localdomain>
 <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>
 <20160202235818.GB21046@localhost.localdomain>
Message-ID: <CACo6K_Zry4303xcvjoqMCCnzmxAVrWgT1+JPWGqXObGu9LnU1g@mail.gmail.com>

In that case, please forgive the silly questions, as I am not an
experienced kernel developer by any means...  (I'm just looking for
enough information go Googling. I won't ask much more down that line
of questioning, as I know this list is not for that purpose.)

1. When you say out-of-tree, do you mean a loadable kernel module?
(My understanding is that the NVMe driver is now part of the mainline
Linux kernel source tree, so I'm a bit confused as to where to nab
that from.)
2. Does the upstream 4.4.1 kernel have any of these fixes if I were to
build it myself with the appropriate support ticked off?

Also, thank you very much for the quick response and assistance. I
really appreciate the help. :)
Thank you,

Mark Jacobson
Software Test Engineer
Stack Velocity


On Wed, Feb 3, 2016 at 12:58 AM, Keith Busch <keith.busch at intel.com> wrote:
> On Wed, Feb 03, 2016 at 12:50:06AM +0100, Mark Jacobson wrote:
>> Output is below. I'm aware the distro hints are fairly invalid.
>
> They're all invalid. This kernel must have forked before the affinity
> hints were fixed for a blk-mq nvme driver. A more optimal affinity hint
> would match the mq's cpu_list, which is how it looks upstream.
>
> I guess your platform strongly prefers CPU 0 when allowed. You can
> either manually override the smp_affinity, use an out-of-tree driver
> with a fix and let irqbalance handle it.


From wenbo.wang at memblaze.com  Tue Feb  2 16:19:46 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Wed, 3 Feb 2016 00:19:46 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <20160202172503.GC10728@localhost.localdomain>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160202172503.GC10728@localhost.localdomain>
Message-ID: <BJXPR01MB19992B07FA623851AC93FB7E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

I used mdelay but not sleep.



-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Wednesday, February 3, 2016 1:25 AM
To: Wenbo Wang
Cc: Jens Axboe; Wenbo Wang; linux-kernel at vger.kernel.org; linux-nvme at lists.infradead.org; Wenwei.Tao
Subject: Re: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended

On Tue, Feb 02, 2016 at 07:15:57AM +0000, Wenbo Wang wrote:
> Jens,
> 
> I did the following test to validate the issue.
> 
> 1. Modify code as below to increase the chance of races.
> 	Add 10s delay after nvme_dev_unmap() in nvme_dev_disable()
> 	Add 10s delay before __nvme_submit_cmd()

If running sync IO, preempt is disabled. You can't just put a 10 second delay there. Wouldn't you hit a "scheduling while atomic" bug instead?

If blk-mq is running the h/w context from its work queue, that might be a different issue. Maybe we can change the "cancel_delayed_work" to "cancel_delayed_work_sync" in blk_mq_stop_hw_queues.

If there's still a window where blk-mq can insert a request after the driver requested to stop queues, I think we should try to close it with the block layer.

From wenbo.wang at memblaze.com  Tue Feb  2 16:35:55 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Wed, 3 Feb 2016 00:35:55 +0000
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <20160202164915.GB10728@localhost.localdomain>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
 <1454417099-19332-3-git-send-email-sagig@mellanox.com>
 <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160202164915.GB10728@localhost.localdomain>
Message-ID: <BJXPR01MB1999CBB2644C70CE07E4B28E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

After canceling these requests, the requests and tags are freed. If queues are restarted shortly, these already freed requests will be queue_rq again? This seems not correct.

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Wednesday, February 3, 2016 12:49 AM
To: Wenbo Wang
Cc: Sagi Grimberg; Jens Axboe; linux-nvme at lists.infradead.org
Subject: Re: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination

On Tue, Feb 02, 2016 at 04:32:33PM +0000, Wenbo Wang wrote:
> If some requests are in q->request_list or ctx->rq_list, does nvme_cancel_io remove them from these lists? If not, will it cause any issue?

It's fine to let them queue there if we expect to restart the controller. We're going to restart the queues shortly.

If we're about to kill the queue, blk-mq's timeout handler kills unstarted requests.

From wenbo.wang at memblaze.com  Tue Feb  2 16:49:14 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Wed, 3 Feb 2016 00:49:14 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <56B0E541.1000907@dev.mellanox.co.il>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <56B0A401.30306@dev.mellanox.co.il>
 <20160202142756.GA10690@localhost.localdomain>
 <56B0BE26.6070208@dev.mellanox.co.il>
 <20160202144648.GA10728@localhost.localdomain>
 <56B0E541.1000907@dev.mellanox.co.il>
Message-ID: <BJXPR01MB199FB54890F656169E92617E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

nvme interrupt handler touches cq door bell, so actually before unmap it must be taken care of too. Currently it checks cq_vector >= 0 to avoid touching unmapped door bell. 

I think it is better to let queue_rq and interrupt handler use the same mechanism to detect unmap, by driver itself (e.g. cq_vector) or by block layer (e.g. a new API to guarantee no inflight IO under hw queue).


-----Original Message-----
From: Sagi Grimberg [mailto:sagig at dev.mellanox.co.il] 
Sent: Wednesday, February 3, 2016 1:20 AM
To: Keith Busch
Cc: Wenbo Wang; Jens Axboe; Wenbo Wang; Wenwei.Tao; linux-kernel at vger.kernel.org; linux-nvme at lists.infradead.org
Subject: Re: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended


> We free the transfer buffers when a command is cancelled. The 
> controller, however, may still own the command and may try to write to 
> them. We have to fence the controller off from being able to do that, 
> so we can't cancel inflight commands while the PCI device is still bus master enabled.
>
> In a perfect world, we could trust in disabling with NVMe registers, 
> but sometimes we can't rely on that.

OK, I wasn't aware that we cannot rely on that.

So it looks like we cannot change the ordering. So this leaves us with the need to guarantee that no queue_rq is inflight before we unmap.

From keith.busch at intel.com  Tue Feb  2 17:26:39 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 3 Feb 2016 01:26:39 +0000
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <BJXPR01MB1999CBB2644C70CE07E4B28E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
 <1454417099-19332-3-git-send-email-sagig@mellanox.com>
 <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160202164915.GB10728@localhost.localdomain>
 <BJXPR01MB1999CBB2644C70CE07E4B28E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160203012638.GC21046@localhost.localdomain>

On Wed, Feb 03, 2016 at 12:35:55AM +0000, Wenbo Wang wrote:
> After canceling these requests, the requests and tags are freed. If queues are restarted shortly, these already freed requests will be queue_rq again? This seems not correct.

That's not true. We only set DNR if the queue is dead. Otherwise the
requests are eligible for requeuing if they haven't exceeded total time.


From keith.busch at intel.com  Wed Feb  3 06:30:56 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 3 Feb 2016 14:30:56 +0000
Subject: NVMe and IRQ Affinity
In-Reply-To: <CACo6K_Zry4303xcvjoqMCCnzmxAVrWgT1+JPWGqXObGu9LnU1g@mail.gmail.com>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
 <20160202234540.GA21046@localhost.localdomain>
 <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>
 <20160202235818.GB21046@localhost.localdomain>
 <CACo6K_Zry4303xcvjoqMCCnzmxAVrWgT1+JPWGqXObGu9LnU1g@mail.gmail.com>
Message-ID: <20160203143055.GA23910@localhost.localdomain>

On Wed, Feb 03, 2016 at 01:13:23AM +0100, Mark Jacobson wrote:
> 1. When you say out-of-tree, do you mean a loadable kernel module?
> (My understanding is that the NVMe driver is now part of the mainline
> Linux kernel source tree, so I'm a bit confused as to where to nab
> that from.)

Yes, a loadable kernel module or a package that can build one. I am not
aware of any publicly available. Perhaps your vendor provides one.

> 2. Does the upstream 4.4.1 kernel have any of these fixes if I were to
> build it myself with the appropriate support ticked off?

The 4.3 kernel has the fixes, and anything newer than that should also
work. I think it was just the 4.2 kernel that had this wrong (will see
if there's are any stable patch candidates).


From keith.busch at intel.com  Wed Feb  3 06:41:24 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 3 Feb 2016 14:41:24 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160203144123.GB23910@localhost.localdomain>

On Tue, Feb 02, 2016 at 07:15:57AM +0000, Wenbo Wang wrote:
> I did the following test to validate the issue.
> 
> 1. Modify code as below to increase the chance of races.
> 	Add 10s delay after nvme_dev_unmap() in nvme_dev_disable()
> 	Add 10s delay before __nvme_submit_cmd()
> 2. Run dd and at the same time, echo 1 to reset_controller to trigger device reset. Finally kernel crashes due to accessing unmapped door bell register.
> 
> Following is the execution order of the two code paths:
> __blk_mq_run_hw_queue
>   Test BLK_MQ_S_STOPPED
> 					nvme_dev_disable()
> 					     nvme_stop_queues()  <-- set BLK_MQ_S_STOPPED
> 					     nvme_dev_unmap(dev)  <-- unmap door bell
>   nvme_queue_rq()
>       Touch door bell	<-- panic here

Does the following force the first to complete before the unmap?

---
@@ -1415,10 +1421,21 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
 
 		blk_mq_cancel_requeue_work(ns->queue);
 		blk_mq_stop_hw_queues(ns->queue);
+		blk_sync_queue(ns->queue);
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
--


From keith.busch at intel.com  Wed Feb  3 08:05:39 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed,  3 Feb 2016 09:05:39 -0700
Subject: [PATCH 0/4] NVMe: Surprise removal fixes
Message-ID: <1454515543-21683-1-git-send-email-keith.busch@intel.com>

First on the 'dd' experiements, I did not find kernel that "worked"
as expected on a surprise removal. The IO process runs until SIGKILL
is recieved. I don't think that's right, and it causes noticable system
performance issues for unrelated tasks.

This series is just focusing on getting the driver to cleanup its part
so it can unbind from a controller.

Earlier feedback suggested this functionality be in the block layer. I
don't have the devices to test what happens with other drivers, and
the desired sequence seems unique to NVMe. Maybe that's an indication
that the driver's flow could benefit from some redesign, or maybe it's
an artifact from having the controller and storage being the same device.

In any case, I would like to isolate this fix to the NVMe driver in
the interest of time, and flush out the block layer before the next
merge window.

Keith Busch (4):
  NVMe: Fix io incapable return values
  NVMe: Sync stopped queues with block layer
  NVMe: Surprise removal fixes
  blk-mq: End unstarted requests on dying queue

 block/blk-mq.c           |  6 ++++--
 drivers/nvme/host/core.c | 14 ++++++++------
 drivers/nvme/host/nvme.h |  4 ++--
 drivers/nvme/host/pci.c  | 14 ++++++++++++++
 4 files changed, 28 insertions(+), 10 deletions(-)

-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb  3 08:05:43 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed,  3 Feb 2016 09:05:43 -0700
Subject: [PATCH 4/4] blk-mq: End unstarted requests on dying queue
In-Reply-To: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
Message-ID: <1454515543-21683-5-git-send-email-keith.busch@intel.com>

Go directly to ending a request if it wasn't started. Previously, completing a
request may invoke a driver callback for a request it didn't initialize.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 block/blk-mq.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 4c0622f..56c0a72 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -599,8 +599,10 @@ static void blk_mq_check_expired(struct blk_mq_hw_ctx *hctx,
 		 * If a request wasn't started before the queue was
 		 * marked dying, kill it here or it'll go unnoticed.
 		 */
-		if (unlikely(blk_queue_dying(rq->q)))
-			blk_mq_complete_request(rq, -EIO);
+		if (unlikely(blk_queue_dying(rq->q))) {
+			rq->errors = -EIO;
+			blk_mq_end_request(rq, rq->errors);
+		}
 		return;
 	}
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb  3 08:05:41 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed,  3 Feb 2016 09:05:41 -0700
Subject: [PATCH 2/4] NVMe: Sync stopped queues with block layer
In-Reply-To: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
Message-ID: <1454515543-21683-3-git-send-email-keith.busch@intel.com>

Make sure no work will submit requests to a stopped h/w queue.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c5bf001..9d415489 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1415,6 +1415,7 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
 
 		blk_mq_cancel_requeue_work(ns->queue);
 		blk_mq_stop_hw_queues(ns->queue);
+		blk_sync_queue(ns->queue);
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb  3 08:05:42 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed,  3 Feb 2016 09:05:42 -0700
Subject: [PATCH 3/4] NVMe: Surprise removal fixes
In-Reply-To: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
Message-ID: <1454515543-21683-4-git-send-email-keith.busch@intel.com>

Temporarily halts IO queues on surprise removal prior to flushing
pending requests to an error completion.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 13 +++++++------
 drivers/nvme/host/pci.c  | 14 ++++++++++++++
 2 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 9d415489..f11cb5a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1186,15 +1186,16 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
 
 	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
 		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
+		 * The request queue is currently stoppped if we got here.
+		 * Prevent new requests from entering before flushing the rest.
+		 * New requests will fail to enter a dying frozen queue, which
+		 * allows blk_cleanup_queue to complete.
 		 */
+		blk_mq_freeze_queue_start(ns->queue);
+		blk_set_queue_dying(ns->queue);
 		blk_mq_abort_requeue_list(ns->queue);
+		blk_mq_start_hw_queues(ns->queue);
 	}
 	if (ns->disk->flags & GENHD_FL_UP) {
 		if (blk_get_integrity(ns->disk))
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 72ef832..4cc7398 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -640,6 +640,11 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 	struct nvme_command cmnd;
 	int ret = BLK_MQ_RQ_QUEUE_OK;
 
+	if (unlikely(blk_queue_dying(req->q))) {
+		blk_mq_end_request(req, -EFAULT);
+		return BLK_MQ_RQ_QUEUE_OK;
+	}
+
 	/*
 	 * If formated with metadata, require the block layer provide a buffer
 	 * unless this namespace is formated such that the metadata can be
@@ -2118,6 +2123,15 @@ static void nvme_remove(struct pci_dev *pdev)
 	pci_set_drvdata(pdev, NULL);
 	flush_work(&dev->reset_work);
 	flush_work(&dev->scan_work);
+
+	/*
+	 * If the controller can't do IO (surprise removal, for example), we
+	 * need to halt queues prior to deleting namespaces. This ends
+	 * outstanding requests and prevents attempts to sync dirty data.
+	 */
+	if (nvme_io_incapable(&dev->ctrl))
+		nvme_dev_disable(dev, true);
+
 	nvme_remove_namespaces(&dev->ctrl);
 	nvme_uninit_ctrl(&dev->ctrl);
 	nvme_dev_disable(dev, true);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb  3 08:05:40 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed,  3 Feb 2016 09:05:40 -0700
Subject: [PATCH 1/4] NVMe: Fix io incapable return values
In-Reply-To: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
Message-ID: <1454515543-21683-2-git-send-email-keith.busch@intel.com>

The function returns true when the controller can't handle IO.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/nvme.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4fb5bb7..9664d07 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -139,9 +139,9 @@ static inline bool nvme_io_incapable(struct nvme_ctrl *ctrl)
 	u32 val = 0;
 
 	if (ctrl->ops->io_incapable(ctrl))
-		return false;
+		return true;
 	if (ctrl->ops->reg_read32(ctrl, NVME_REG_CSTS, &val))
-		return false;
+		return true;
 	return val & NVME_CSTS_CFS;
 }
 
-- 
2.6.2.307.g37023ba



From kim1158 at gmail.com  Wed Feb  3 08:14:29 2016
From: kim1158 at gmail.com (Kim Kyungsan)
Date: Thu, 4 Feb 2016 01:14:29 +0900
Subject: NVMe and IRQ Affinity
In-Reply-To: <CACo6K_Zry4303xcvjoqMCCnzmxAVrWgT1+JPWGqXObGu9LnU1g@mail.gmail.com>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
 <20160202234540.GA21046@localhost.localdomain>
 <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>
 <20160202235818.GB21046@localhost.localdomain>
 <CACo6K_Zry4303xcvjoqMCCnzmxAVrWgT1+JPWGqXObGu9LnU1g@mail.gmail.com>
Message-ID: <CAARenASB5+vdwBGq7iwrYLur6MzGSdvK0xX81K3GUU0dmtMYCg@mail.gmail.com>

Hi, I had a similar experience from nvme interrupts.
As you said, kernel basically allocates cpu0 for interrupt handling.
Without setting irq affinity or irqbalance daemon, it could cause the
decrease in performance and soft-lock CPU bug.
The symptom was noticed on systems under high workload with multiple
nvme devices.

How we solved was that setting irq affinity to evenly distribute cpus
to handle interrupts from an IRQ like below.

       Cpu0 - nvme irq0
       Cpu1 - nvme irq1
       Cpu2 - nvme irq2
       Cpu3 - nvme irq3
       Cpu4 - nvme irq4
       Cpu5 - nvme irq5
       Cpu6 - nvme irq6
       Cpu7 - nvme irq7
       ....

And there are two ways to set irq affinity. First one is using proc
interface as you mentioned, another one is using inbox driver higher
than kernel 4.3 . In fact, nvme driver under 4.3 kernel also tried to
set irq affinity hint during device initialization, however, it hadn't
worked due to a bug which is fixed by Keith Busch on 4.3 kernel.
>From kernel 4.3, nvme driver sets irq affinity as well as
affinity_hint during device initialization by calling
irq_set_affinity_hint()

Please refer below.

/* kernel 4.3 nvme-core.c */
nvme_dev_scan()
     + nvme_set_irq_hints()
         +irq_set_affinity_hint(dev->entry[nvmeq->cq_vector].vector,
blk_mq_tags_cpumask(*nvmeq->tags));


/* kernel/irq/manage.c */
int irq_set_affinity_hint(unsigned int irq, const struct cpumask *m)
{
        unsigned long flags;
        struct irq_desc *desc = irq_get_desc_lock(irq, &flags,
IRQ_GET_DESC_CHECK_GLOBAL);

         if (!desc)
                return -EINVAL;
        desc->affinity_hint = m;
        irq_put_desc_unlock(desc, flags);
        /* set the initial affinity to prevent every interrupt being on CPU0 */
        if (m)
               __irq_set_affinity(irq, m, false);
        return 0;
}
EXPORT_SYMBOL_GPL(irq_set_affinity_hint);

The last thing i want to note is you better disable irqbalance daemon
after setting irq affinity by yourself because the daemon adjust irq
affinity again which can cause unbalanced interrupts handling again.


On Wed, Feb 3, 2016 at 9:13 AM, Mark Jacobson
<mark_jacobson at stackvelocity.com> wrote:
> In that case, please forgive the silly questions, as I am not an
> experienced kernel developer by any means...  (I'm just looking for
> enough information go Googling. I won't ask much more down that line
> of questioning, as I know this list is not for that purpose.)
>
> 1. When you say out-of-tree, do you mean a loadable kernel module?
> (My understanding is that the NVMe driver is now part of the mainline
> Linux kernel source tree, so I'm a bit confused as to where to nab
> that from.)
> 2. Does the upstream 4.4.1 kernel have any of these fixes if I were to
> build it myself with the appropriate support ticked off?
>
> Also, thank you very much for the quick response and assistance. I
> really appreciate the help. :)
> Thank you,
>
> Mark Jacobson
> Software Test Engineer
> Stack Velocity
>
>
> On Wed, Feb 3, 2016 at 12:58 AM, Keith Busch <keith.busch at intel.com> wrote:
>> On Wed, Feb 03, 2016 at 12:50:06AM +0100, Mark Jacobson wrote:
>>> Output is below. I'm aware the distro hints are fairly invalid.
>>
>> They're all invalid. This kernel must have forked before the affinity
>> hints were fixed for a blk-mq nvme driver. A more optimal affinity hint
>> would match the mq's cpu_list, which is how it looks upstream.
>>
>> I guess your platform strongly prefers CPU 0 when allowed. You can
>> either manually override the smp_affinity, use an out-of-tree driver
>> with a fix and let irqbalance handle it.
>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme



-- 
------------------------------------------------------------
the person who practices a truth goes toward light.


From wenbo.wang at memblaze.com  Wed Feb  3 08:35:03 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Wed, 3 Feb 2016 16:35:03 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <20160203144123.GB23910@localhost.localdomain>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203144123.GB23910@localhost.localdomain>
Message-ID: <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

For async io (executed by run_work worker) it should work. However for sync io in blk_mq_run_hw_queue, this seems not help, there is still a window.

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Wednesday, February 3, 2016 10:41 PM
To: Wenbo Wang
Cc: Jens Axboe; Wenbo Wang; linux-kernel at vger.kernel.org; linux-nvme at lists.infradead.org; Wenwei.Tao
Subject: Re: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended

On Tue, Feb 02, 2016 at 07:15:57AM +0000, Wenbo Wang wrote:
> I did the following test to validate the issue.
> 
> 1. Modify code as below to increase the chance of races.
> 	Add 10s delay after nvme_dev_unmap() in nvme_dev_disable()
> 	Add 10s delay before __nvme_submit_cmd() 2. Run dd and at the same 
> time, echo 1 to reset_controller to trigger device reset. Finally kernel crashes due to accessing unmapped door bell register.
> 
> Following is the execution order of the two code paths:
> __blk_mq_run_hw_queue
>   Test BLK_MQ_S_STOPPED
> 					nvme_dev_disable()
> 					     nvme_stop_queues()  <-- set BLK_MQ_S_STOPPED
> 					     nvme_dev_unmap(dev)  <-- unmap door bell
>   nvme_queue_rq()
>       Touch door bell	<-- panic here

Does the following force the first to complete before the unmap?

---
@@ -1415,10 +1421,21 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
 
 		blk_mq_cancel_requeue_work(ns->queue);
 		blk_mq_stop_hw_queues(ns->queue);
+		blk_sync_queue(ns->queue);
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
--

From keith.busch at intel.com  Wed Feb  3 08:38:09 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 3 Feb 2016 16:38:09 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203144123.GB23910@localhost.localdomain>
 <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160203163808.GC23910@localhost.localdomain>

On Wed, Feb 03, 2016 at 04:35:03PM +0000, Wenbo Wang wrote:
> For async io (executed by run_work worker) it should work. However for sync io in blk_mq_run_hw_queue, this seems not help, there is still a window.

Alright, for lack of a better way to sync a stopped queue, the driver's
indication looks like the best we can do at this point.


From snitzer at redhat.com  Wed Feb  3 10:04:06 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Wed, 3 Feb 2016 13:04:06 -0500
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <56AEFF63.7050606@suse.de>
References: <20160125214016.GA10060@redhat.com>
 <20160125233717.GQ24960@octiron.msp.redhat.com>
 <20160126132939.GA23967@redhat.com>
 <56A8A6A8.9090003@dev.mellanox.co.il>
 <20160127174828.GA31802@redhat.com>
 <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
Message-ID: <20160203180406.GA11591@redhat.com>

On Mon, Feb 01 2016 at  1:46am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 01/30/2016 08:12 PM, Mike Snitzer wrote:
> > On Sat, Jan 30 2016 at  3:52am -0500,
> > Hannes Reinecke <hare at suse.de> wrote:
> > > 
> >> So nearly on par with your null-blk setup. but with real hardware.
> >> (Which in itself is pretty cool. You should get faster RAM :-)
> > 
> > You've misunderstood what I said my null_blk (RAM) performance is.
> > 
> > My null_blk test gets ~1900K read IOPs.  But dm-mpath ontop only gets
> > between 600K and 1000K IOPs depending on $FIO_QUEUE_DEPTH and if I
> > use multiple $NULL_BLK_HW_QUEUES.
> > 
> Right.
> We're using two 16G FC links, each talking to 4 LUNs.
> With dm-mpath on top. The FC HBAs have a hardware queue depth
> of roughly 2000, so we might need to tweak the queue depth of the
> multipath devices, too.
> 
> 
> Will be having a look at your patches.

I have staged quite a few patches in linux-next for the 4.6 merge window:
https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/log/?h=dm-4.6

I'm open to posting them to dm-devel if it would ease review.  Let me
know.

These changes range from:
- defaulting to queue_depth of 2048 (rather than 64) request per blk-mq
  hw queue -- fixed stalls waiting for finite amount of tags (in bt_get)
- making additional use of the DM-multipath blk-mq device's pdu for
  mpath per-io data structures
- using blk-mq interfaces rather than generic wrappers (mainly just
  helps document the nature of the requests in blk-mq specific code
  paths)
- avoiding running the blk-mq hw queues on request completion (doesn't
  seem to help like it does for .request_fn multipath; only serves to
  generate extra kblockd work for no observed gain)
- optimize both .request_fn (dm_request_fn) and blk-mq (dm_mq_queue_rq)
  so they don't bother with the bio-based DM pattern of finding which
  target is used to map IO at the particular offset -- request-based DM
  only ever has a single immutable target associated with it
- removal of dead code and code comment improvements

I've seen blk-mq DM-multipath performance improvement but _not_ enough
to consider this line of work "done".  I'd be very interested to see
what kind of improvements you (Hannes) and Sagi can realize with your
respective testbeds.

I'm still not clear on where the considerable performance loss is coming
from (on null_blk devices I see ~1900K read IOPs but I'm still only
seeing ~1000K read IOPs when blk-mq DM-multipath is layered ontop).
What is very much apparent is layering dm-mq multipath ontop of null_blk
results in a HUGE amount of additional context switches.  I can only
infer that the request completion for this stacked device (blk-mq queue
ontop of blk-mq queue, with 2 completions: 1 for clone completing on
underlying device and 1 for original request completing) is the reason
for all the extra context switches.

Here are pictures of 'perf report' for perf datat collected using
'perf record -ag -e cs'.

Against null_blk:
http://people.redhat.com/msnitzer/perf-report-cs-null_blk.png
Against dm-mpath ontop of the same null_blk:
http://people.redhat.com/msnitzer/perf-report-cs-dm_mq.png

Looks like there may be some low-hanging fruit associated with steering
completion to reduce all the excessive ksoftirq and kworker context
switching.  Pin-pointing the reason these tasks are context switching is
my next focus.

I've yet to actually test on DM-multipath device with more than one
path.  Hannes, Sagi, and/or others: on such a setup it would be
interesting to see if increasing the 'blk_mq_nr_hw_queues' helps at all.
Any 'perf report' traces that shed light on bottlenecks you might be
experiencing would obviously be appreciated.  I'm skeptical there is
enough parallelism in the dm-mpath.c code to allow for proper scaling --
switching to RCU could help this.

Mike

p.s.
I experimented with using the top-level DM multipath blk-mq queue's
pdu for the underlying clone 'struct request' that is implicitly needed
when issuing the request to the underlying path -- by (ab)using
blk_mq_tag_set_rq that is used by blk-flush.c.  blk-mq hated me for
trying this.  I kept getting list corruption on unplug with this (and
many variants on work along these lines):
http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel2&id=7b7203c93cec7ad3a0ae2a2da567d45f46fe8098

I stopped that line of work due to inability to make it function.. but
it was a skunk-works experiment that needed to die anyway (as I'm sure
Jens will agree).


From snitzer at redhat.com  Wed Feb  3 10:24:24 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Wed, 3 Feb 2016 13:24:24 -0500
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <20160203180406.GA11591@redhat.com>
References: <20160125233717.GQ24960@octiron.msp.redhat.com>
 <20160126132939.GA23967@redhat.com>
 <56A8A6A8.9090003@dev.mellanox.co.il>
 <20160127174828.GA31802@redhat.com>
 <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
Message-ID: <20160203182423.GA12913@redhat.com>

On Wed, Feb 03 2016 at  1:04pm -0500,
Mike Snitzer <snitzer at redhat.com> wrote:
 
> I'm still not clear on where the considerable performance loss is coming
> from (on null_blk device I see ~1900K read IOPs but I'm still only
> seeing ~1000K read IOPs when blk-mq DM-multipath is layered ontop).
> What is very much apparent is: layering dm-mq multipath ontop of null_blk
> results in a HUGE amount of additional context switches.  I can only
> infer that the request completion for this stacked device (blk-mq queue
> ontop of blk-mq queue, with 2 completions: 1 for clone completing on
> underlying device and 1 for original request completing) is the reason
> for all the extra context switches.

Starts to explain, certainly not the "reason"; that is still very much
TBD...

> Here are pictures of 'perf report' for perf datat collected using
> 'perf record -ag -e cs'.
> 
> Against null_blk:
> http://people.redhat.com/msnitzer/perf-report-cs-null_blk.png

if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
  cpu          : usr=25.53%, sys=74.40%, ctx=1970, majf=0, minf=474
if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
  cpu          : usr=26.79%, sys=73.15%, ctx=2067, majf=0, minf=479

> Against dm-mpath ontop of the same null_blk:
> http://people.redhat.com/msnitzer/perf-report-cs-dm_mq.png

if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
  cpu          : usr=11.07%, sys=33.90%, ctx=667784, majf=0, minf=466
if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
  cpu          : usr=15.22%, sys=48.44%, ctx=2314901, majf=0, minf=466

So yeah, the percentages reflected in these respective images didn't do
the huge increase in context switches justice... we _must_ figure out
why we're seeing so many context switches with dm-mq.

The same fio job is ran to measure these context switches, e.g.:

fio --cpus_allowed_policy=split --group_reporting --rw=randread --bs=4k
--numjobs=12 --iodepth=32 --runtime=10 --time_based --loops=1
--ioengine=libaio --direct=1 --invalidate=1 --randrepeat=1 --norandommap
--exitall --name task_nullb0 --filename=/dev/nullb0

fio --cpus_allowed_policy=split --group_reporting --rw=randread --bs=4k
--numjobs=12 --iodepth=32 --runtime=10 --time_based --loops=1
--ioengine=libaio --direct=1 --invalidate=1 --randrepeat=1 --norandommap
--exitall --name task_dm_mq --filename=/dev/mapper/dm_mq


From azher at hep.caltech.edu  Wed Feb  3 10:31:45 2016
From: azher at hep.caltech.edu (Azher Mughal)
Date: Wed, 3 Feb 2016 10:31:45 -0800
Subject: NVMe and IRQ Affinity
In-Reply-To: <20160203143055.GA23910@localhost.localdomain>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
 <20160202234540.GA21046@localhost.localdomain>
 <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>
 <20160202235818.GB21046@localhost.localdomain>
 <CACo6K_Zry4303xcvjoqMCCnzmxAVrWgT1+JPWGqXObGu9LnU1g@mail.gmail.com>
 <20160203143055.GA23910@localhost.localdomain>
Message-ID: <56B24791.7040807@hep.caltech.edu>

Hi Keith,

I have similar affinity issues in CentOS 7.2. The nvme module is generic
that came along with this distro. Is there a git repo which can be used
with the kernel "3.10.0-327.4.5" ?

Thanks
-Azher

# modinfo nvme
filename:      
/lib/modules/3.10.0-327.4.5.el7.x86_64/kernel/drivers/block/nvme.ko
version:        1.0
license:        GPL
author:         Matthew Wilcox <willy at linux.intel.com>
rhelversion:    7.2
srcversion:     6FE34EC5F6A703F8EDE6C77
alias:          pci:v*d*sv*sd*bc01sc08i02*
depends:       
intree:         Y
vermagic:       3.10.0-327.4.5.el7.x86_64 SMP mod_unload modversions
signer:         CentOS Linux kernel signing key
sig_key:        10:5D:A1:3D:CA:AA:74:AE:50:00:17:E7:D5:2C:DA:9B:7C:C5:10:93
sig_hashalgo:   sha256
parm:           admin_timeout:timeout in seconds for admin commands (byte)
parm:           io_timeout:timeout in seconds for I/O (byte)
parm:           shutdown_timeout:timeout in seconds for controller
shutdown (byte)
parm:           nvme_major:int
parm:           nvme_char_major:int
parm:           use_threaded_interrupts:int


On 2/3/2016 6:30 AM, Keith Busch wrote:
> On Wed, Feb 03, 2016 at 01:13:23AM +0100, Mark Jacobson wrote:
>> 1. When you say out-of-tree, do you mean a loadable kernel module?
>> (My understanding is that the NVMe driver is now part of the mainline
>> Linux kernel source tree, so I'm a bit confused as to where to nab
>> that from.)
> Yes, a loadable kernel module or a package that can build one. I am not
> aware of any publicly available. Perhaps your vendor provides one.
>
>> 2. Does the upstream 4.4.1 kernel have any of these fixes if I were to
>> build it myself with the appropriate support ticked off?
> The 4.3 kernel has the fixes, and anything newer than that should also
> work. I think it was just the 4.2 kernel that had this wrong (will see
> if there's are any stable patch candidates).
>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
>



From keith.busch at intel.com  Wed Feb  3 10:58:30 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 3 Feb 2016 18:58:30 +0000
Subject: NVMe and IRQ Affinity
In-Reply-To: <56B24791.7040807@hep.caltech.edu>
References: <CACo6K_bSDJwt61mnKDpYvcXjCVA9UKGO17pythzcJ8Rnjh19sA@mail.gmail.com>
 <20160202234540.GA21046@localhost.localdomain>
 <CACo6K_YGiH2ER4s21OzmD69LDL+ES0PO54jJ_efcSFw7pHADvQ@mail.gmail.com>
 <20160202235818.GB21046@localhost.localdomain>
 <CACo6K_Zry4303xcvjoqMCCnzmxAVrWgT1+JPWGqXObGu9LnU1g@mail.gmail.com>
 <20160203143055.GA23910@localhost.localdomain>
 <56B24791.7040807@hep.caltech.edu>
Message-ID: <20160203185829.GA28747@localhost.localdomain>

On Wed, Feb 03, 2016 at 10:31:45AM -0800, Azher Mughal wrote:
> I have similar affinity issues in CentOS 7.2. The nvme module is generic
> that came along with this distro.

Yes, it's the same issue.

> Is there a git repo which can be used
> with the kernel "3.10.0-327.4.5" ?

Not that I know of.


From snitzer at redhat.com  Wed Feb  3 11:22:57 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Wed, 3 Feb 2016 14:22:57 -0500
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <20160203182423.GA12913@redhat.com>
References: <20160126132939.GA23967@redhat.com>
 <56A8A6A8.9090003@dev.mellanox.co.il>
 <20160127174828.GA31802@redhat.com>
 <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com>
Message-ID: <20160203192257.GA13076@redhat.com>


On Wed, Feb 03 2016 at  1:24pm -0500,
Mike Snitzer <snitzer at redhat.com> wrote:
 
> > Here are pictures of 'perf report' for perf datat collected using
> > 'perf record -ag -e cs'.
> > 
> > Against null_blk:
> > http://people.redhat.com/msnitzer/perf-report-cs-null_blk.png
> 
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
>   cpu          : usr=25.53%, sys=74.40%, ctx=1970, majf=0, minf=474
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
>   cpu          : usr=26.79%, sys=73.15%, ctx=2067, majf=0, minf=479
> 
> > Against dm-mpath ontop of the same null_blk:
> > http://people.redhat.com/msnitzer/perf-report-cs-dm_mq.png
> 
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
>   cpu          : usr=11.07%, sys=33.90%, ctx=667784, majf=0, minf=466
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
>   cpu          : usr=15.22%, sys=48.44%, ctx=2314901, majf=0, minf=466

I promise, this is my last reply to myself ;)

The above dm-mq results were _without_ using this commit:
"dm: don't blk_mq_run_hw_queues in blk-mq request completion"
https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=cc6ca783e8f0669112c5f4154f51a7cb17b76006

But with that commit I'm still seeing the high context switches:

if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
   cpu          : usr=11.78%, sys=36.11%, ctx=690262, majf=0, minf=470
if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
  cpu          : usr=15.62%, sys=49.95%, ctx=2425084, majf=0, minf=466

So running blk_mq_run_hw_queues (async to punt to kblockd) on dm-mq
request completion isn't the source of any of the accounted context
switches...


From wenbo.wang at memblaze.com  Wed Feb  3 19:25:50 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Thu, 4 Feb 2016 03:25:50 +0000
Subject: [PATCH 0/4] NVMe: Surprise removal fixes
In-Reply-To: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
Message-ID: <BJXPR01MB1997AE807A93FA2EBB32076E0D10@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Isn't the dd behavior correct? For non-direct dd, even if a surprise remove happened, it simply keeps dirtying the page cache and is not aware of any errors below.

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Thursday, February 4, 2016 12:06 AM
To: linux-nvme at lists.infradead.org; Jens Axboe
Cc: Christoph Hellwig; Sagi Grimberg; Wenbo Wang; Keith Busch
Subject: [PATCH 0/4] NVMe: Surprise removal fixes

First on the 'dd' experiements, I did not find kernel that "worked"
as expected on a surprise removal. The IO process runs until SIGKILL is recieved. I don't think that's right, and it causes noticable system performance issues for unrelated tasks.

This series is just focusing on getting the driver to cleanup its part so it can unbind from a controller.

Earlier feedback suggested this functionality be in the block layer. I don't have the devices to test what happens with other drivers, and the desired sequence seems unique to NVMe. Maybe that's an indication that the driver's flow could benefit from some redesign, or maybe it's an artifact from having the controller and storage being the same device.

In any case, I would like to isolate this fix to the NVMe driver in the interest of time, and flush out the block layer before the next merge window.

Keith Busch (4):
  NVMe: Fix io incapable return values
  NVMe: Sync stopped queues with block layer
  NVMe: Surprise removal fixes
  blk-mq: End unstarted requests on dying queue

 block/blk-mq.c           |  6 ++++--
 drivers/nvme/host/core.c | 14 ++++++++------  drivers/nvme/host/nvme.h |  4 ++--  drivers/nvme/host/pci.c  | 14 ++++++++++++++
 4 files changed, 28 insertions(+), 10 deletions(-)

--
2.6.2.307.g37023ba


From hare at suse.de  Wed Feb  3 22:54:52 2016
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2016 07:54:52 +0100
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <20160203182423.GA12913@redhat.com>
References: <20160125233717.GQ24960@octiron.msp.redhat.com>
 <20160126132939.GA23967@redhat.com> <56A8A6A8.9090003@dev.mellanox.co.il>
 <20160127174828.GA31802@redhat.com> <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
Message-ID: <56B2F5BC.1010700@suse.de>

On 02/03/2016 07:24 PM, Mike Snitzer wrote:
> On Wed, Feb 03 2016 at  1:04pm -0500,
> Mike Snitzer <snitzer at redhat.com> wrote:
>  
>> I'm still not clear on where the considerable performance loss is coming
>> from (on null_blk device I see ~1900K read IOPs but I'm still only
>> seeing ~1000K read IOPs when blk-mq DM-multipath is layered ontop).
>> What is very much apparent is: layering dm-mq multipath ontop of null_blk
>> results in a HUGE amount of additional context switches.  I can only
>> infer that the request completion for this stacked device (blk-mq queue
>> ontop of blk-mq queue, with 2 completions: 1 for clone completing on
>> underlying device and 1 for original request completing) is the reason
>> for all the extra context switches.
> 
> Starts to explain, certainly not the "reason"; that is still very much
> TBD...
> 
>> Here are pictures of 'perf report' for perf datat collected using
>> 'perf record -ag -e cs'.
>>
>> Against null_blk:
>> http://people.redhat.com/msnitzer/perf-report-cs-null_blk.png
> 
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
>   cpu          : usr=25.53%, sys=74.40%, ctx=1970, majf=0, minf=474
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
>   cpu          : usr=26.79%, sys=73.15%, ctx=2067, majf=0, minf=479
> 
>> Against dm-mpath ontop of the same null_blk:
>> http://people.redhat.com/msnitzer/perf-report-cs-dm_mq.png
> 
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
>   cpu          : usr=11.07%, sys=33.90%, ctx=667784, majf=0, minf=466
> if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
>   cpu          : usr=15.22%, sys=48.44%, ctx=2314901, majf=0, minf=466
> 
> So yeah, the percentages reflected in these respective images didn't do
> the huge increase in context switches justice... we _must_ figure out
> why we're seeing so many context switches with dm-mq.
> 
Well, the most obvious one being that you're using 1 dm-mq queue vs
4 null_blk queues.
So you will have have to do an additional context switch for 75% of
the total I/Os submitted.

Have you tested with 4 dm-mq hw queues?

To avoid context switches we would have to align the dm-mq queues to
the underlying blk-mq layout for the paths.

And we need to look at making the main submission path lockless;
I was wondering if we really need to take the lock if we don't
switch priority groups; maybe we can establish a similar algorithm
blk-mq does; if we were to have a queue per valid path in any given
priority group we should be able to run lockless and only take the
lock if we need to switch priority groups.

But anyway, I'll be looking at your patches.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From snitzer at redhat.com  Thu Feb  4 05:54:20 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Thu, 4 Feb 2016 08:54:20 -0500
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <56B2F5BC.1010700@suse.de>
References: <56A8A6A8.9090003@dev.mellanox.co.il>
 <20160127174828.GA31802@redhat.com>
 <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
Message-ID: <20160204135420.GA18227@redhat.com>

On Thu, Feb 04 2016 at  1:54am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/03/2016 07:24 PM, Mike Snitzer wrote:
> > On Wed, Feb 03 2016 at  1:04pm -0500,
> > Mike Snitzer <snitzer at redhat.com> wrote:
> >  
> >> I'm still not clear on where the considerable performance loss is coming
> >> from (on null_blk device I see ~1900K read IOPs but I'm still only
> >> seeing ~1000K read IOPs when blk-mq DM-multipath is layered ontop).
> >> What is very much apparent is: layering dm-mq multipath ontop of null_blk
> >> results in a HUGE amount of additional context switches.  I can only
> >> infer that the request completion for this stacked device (blk-mq queue
> >> ontop of blk-mq queue, with 2 completions: 1 for clone completing on
> >> underlying device and 1 for original request completing) is the reason
> >> for all the extra context switches.
> > 
> > Starts to explain, certainly not the "reason"; that is still very much
> > TBD...
> > 
> >> Here are pictures of 'perf report' for perf datat collected using
> >> 'perf record -ag -e cs'.
> >>
> >> Against null_blk:
> >> http://people.redhat.com/msnitzer/perf-report-cs-null_blk.png
> > 
> > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
> >   cpu          : usr=25.53%, sys=74.40%, ctx=1970, majf=0, minf=474
> > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
> >   cpu          : usr=26.79%, sys=73.15%, ctx=2067, majf=0, minf=479
> > 
> >> Against dm-mpath ontop of the same null_blk:
> >> http://people.redhat.com/msnitzer/perf-report-cs-dm_mq.png
> > 
> > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
> >   cpu          : usr=11.07%, sys=33.90%, ctx=667784, majf=0, minf=466
> > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
> >   cpu          : usr=15.22%, sys=48.44%, ctx=2314901, majf=0, minf=466
> > 
> > So yeah, the percentages reflected in these respective images didn't do
> > the huge increase in context switches justice... we _must_ figure out
> > why we're seeing so many context switches with dm-mq.
> > 
> Well, the most obvious one being that you're using 1 dm-mq queue vs
> 4 null_blk queues.
> So you will have have to do an additional context switch for 75% of
> the total I/Os submitted.

Right, that case is certainly prone to more context switches.  But I'm
initially most concerned about the case where both only have 1 queue.

> Have you tested with 4 dm-mq hw queues?

Yes, it makes performance worse.  This is likely rooted in dm-mpath IO
path not being lockless.  But I also have concern about whether the
clone, sent to the underlying path, is completing on a different cpu
than dm-mq's original request.

I'll be using ftrace to try to dig into the various aspects of this
(perf, as I know how to use it, isn't giving me enough precision in its
reporting).

> To avoid context switches we would have to align the dm-mq queues to
> the underlying blk-mq layout for the paths.

Right, we need to take more care (how remains TBD).  But for now I'm
just going to focus on the case where both dm-mq and null_blk have 1 for
nr_hw_queues.  As you can see even in that config the number of context
switches goes from 1970 to 667784 (and there is a huge loss of system
cpu utilization) once dm-mq w/ 1 hw_queue is stacked ontop on the
null_blk device.

Once we understand the source of all the additional context switching
for this more simplistic stacked configuration we can look closer at
scaling as we add more underlying paths.

> And we need to look at making the main submission path lockless;
> I was wondering if we really need to take the lock if we don't
> switch priority groups; maybe we can establish a similar algorithm
> blk-mq does; if we were to have a queue per valid path in any given
> priority group we should be able to run lockless and only take the
> lock if we need to switch priority groups.

I'd like to explore this further with you once I come back up from this
frustrating deep dive on "what is causing all these context switches!?"
 
> But anyway, I'll be looking at your patches.

Thanks, sadly none of the patches are going to fix the performance
problems but I do think they are a step forward.


From hare at suse.de  Thu Feb  4 05:58:06 2016
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2016 14:58:06 +0100
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <20160204135420.GA18227@redhat.com>
References: <56A8A6A8.9090003@dev.mellanox.co.il>
 <20160127174828.GA31802@redhat.com> <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
 <56B2F5BC.1010700@suse.de> <20160204135420.GA18227@redhat.com>
Message-ID: <56B358EE.6000007@suse.de>

On 02/04/2016 02:54 PM, Mike Snitzer wrote:
> On Thu, Feb 04 2016 at  1:54am -0500,
> Hannes Reinecke <hare at suse.de> wrote:
> 
[ .. ]
>> But anyway, I'll be looking at your patches.
> 
> Thanks, sadly none of the patches are going to fix the performance
> problems but I do think they are a step forward.
> 
Hmm. I've got a slew of patches converting dm-mpath to use atomic_t
and bitops; with that we should be able to move to rcu for path
lookup and do away with most of the locking.
Quite raw, though; drop me a mail if you're interested.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From snitzer at redhat.com  Thu Feb  4 06:09:59 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Thu, 4 Feb 2016 09:09:59 -0500
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <56B358EE.6000007@suse.de>
References: <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com> <56B358EE.6000007@suse.de>
Message-ID: <20160204140959.GA18328@redhat.com>

On Thu, Feb 04 2016 at  8:58am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/04/2016 02:54 PM, Mike Snitzer wrote:
> > On Thu, Feb 04 2016 at  1:54am -0500,
> > Hannes Reinecke <hare at suse.de> wrote:
> > 
> [ .. ]
> >> But anyway, I'll be looking at your patches.
> > 
> > Thanks, sadly none of the patches are going to fix the performance
> > problems but I do think they are a step forward.
> > 
> Hmm. I've got a slew of patches converting dm-mpath to use atomic_t
> and bitops; with that we should be able to move to rcu for path
> lookup and do away with most of the locking.
> Quite raw, though; drop me a mail if you're interested.

Hmm, ok I just switched m->lock from spinlock_t to rwlock_t, see:
https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5226e23a6958ac9b7ade13a983604c43d232c7d

So any patch you'd have in this area would need rebasing.  I'll gladly
look at what you have (even if it isn't rebased).  So yes please share.

(it could be that there isn't a big enough win associated with switching
to rwlock_t -- that we could get away without doing that particular
churn.. open to that if you think rwlock_t pointless given we'll take
the write lock after repeat_count drops to 0)


From wenbo.wang at memblaze.com  Thu Feb  4 06:28:24 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Thu, 4 Feb 2016 14:28:24 +0000
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <20160203012638.GC21046@localhost.localdomain>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
 <1454417099-19332-3-git-send-email-sagig@mellanox.com>
 <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160202164915.GB10728@localhost.localdomain>
 <BJXPR01MB1999CBB2644C70CE07E4B28E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203012638.GC21046@localhost.localdomain>
Message-ID: <BJXPR01MB19990484665533561B21DB3E0D10@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Is the following execution valid?

1. request A is linked in q->request_list
2. device reset
3. all requests (including request A) are cancelled by nvme_dev_disable()->nvme_clear_queue()
4. device restarted
5. flush q->request_list, and request A is again running in queue_rq.         <-- since request A has been cancelled, it shall not be running again

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Wednesday, February 3, 2016 9:27 AM
To: Wenbo Wang
Cc: Sagi Grimberg; Jens Axboe; linux-nvme at lists.infradead.org
Subject: Re: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination

On Wed, Feb 03, 2016 at 12:35:55AM +0000, Wenbo Wang wrote:
> After canceling these requests, the requests and tags are freed. If queues are restarted shortly, these already freed requests will be queue_rq again? This seems not correct.

That's not true. We only set DNR if the queue is dead. Otherwise the requests are eligible for requeuing if they haven't exceeded total time.

From hare at suse.de  Thu Feb  4 06:32:03 2016
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2016 15:32:03 +0100
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <20160204140959.GA18328@redhat.com>
References: <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
 <56B2F5BC.1010700@suse.de> <20160204135420.GA18227@redhat.com>
 <56B358EE.6000007@suse.de> <20160204140959.GA18328@redhat.com>
Message-ID: <56B360E3.2020806@suse.de>

On 02/04/2016 03:09 PM, Mike Snitzer wrote:
> On Thu, Feb 04 2016 at  8:58am -0500,
> Hannes Reinecke <hare at suse.de> wrote:
> 
>> On 02/04/2016 02:54 PM, Mike Snitzer wrote:
>>> On Thu, Feb 04 2016 at  1:54am -0500,
>>> Hannes Reinecke <hare at suse.de> wrote:
>>>
>> [ .. ]
>>>> But anyway, I'll be looking at your patches.
>>>
>>> Thanks, sadly none of the patches are going to fix the performance
>>> problems but I do think they are a step forward.
>>>
>> Hmm. I've got a slew of patches converting dm-mpath to use atomic_t
>> and bitops; with that we should be able to move to rcu for path
>> lookup and do away with most of the locking.
>> Quite raw, though; drop me a mail if you're interested.
> 
> Hmm, ok I just switched m->lock from spinlock_t to rwlock_t, see:
> https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5226e23a6958ac9b7ade13a983604c43d232c7d
> 
> So any patch you'd have in this area would need rebasing.  I'll gladly
> look at what you have (even if it isn't rebased).  So yes please share.
> 
> (it could be that there isn't a big enough win associated with switching
> to rwlock_t -- that we could get away without doing that particular
> churn.. open to that if you think rwlock_t pointless given we'll take
> the write lock after repeat_count drops to 0)
> 
personally, I don't think the switching to a rwlock_t will buy us
anything; for a decent performance you have to set rq_min_io to 1
anyway, thereby defeating the purpose of the rwlock.

My thinking was rather a different direction:
Move the crucial bits of the multipath structure to atomics, and
split off the path selection code into one bit for selecting the
path within a path group, and another which switches the path groups.
When we do that we could use rcus for the paths themselves, and
would only need to take the spinlock if we need to switch path
groups. Which should be okay as switching path groups is
(potentially) a rather slow operation anyway.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dm-atomic.tar.gz
Type: application/x-gzip
Size: 10584 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20160204/b81a6d8b/attachment.bin>

From snitzer at redhat.com  Thu Feb  4 06:44:12 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Thu, 4 Feb 2016 09:44:12 -0500
Subject: dm-multipath low performance with blk-mq
In-Reply-To: <56B360E3.2020806@suse.de>
References: <56AC79D0.5060104@suse.de> <20160130191238.GA18686@redhat.com>
 <56AEFF63.7050606@suse.de> <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com> <56B358EE.6000007@suse.de>
 <20160204140959.GA18328@redhat.com> <56B360E3.2020806@suse.de>
Message-ID: <20160204144412.GA18529@redhat.com>

On Thu, Feb 04 2016 at  9:32am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/04/2016 03:09 PM, Mike Snitzer wrote:
> > On Thu, Feb 04 2016 at  8:58am -0500,
> > Hannes Reinecke <hare at suse.de> wrote:
> > 
> >> On 02/04/2016 02:54 PM, Mike Snitzer wrote:
> >>> On Thu, Feb 04 2016 at  1:54am -0500,
> >>> Hannes Reinecke <hare at suse.de> wrote:
> >>>
> >> [ .. ]
> >>>> But anyway, I'll be looking at your patches.
> >>>
> >>> Thanks, sadly none of the patches are going to fix the performance
> >>> problems but I do think they are a step forward.
> >>>
> >> Hmm. I've got a slew of patches converting dm-mpath to use atomic_t
> >> and bitops; with that we should be able to move to rcu for path
> >> lookup and do away with most of the locking.
> >> Quite raw, though; drop me a mail if you're interested.
> > 
> > Hmm, ok I just switched m->lock from spinlock_t to rwlock_t, see:
> > https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5226e23a6958ac9b7ade13a983604c43d232c7d
> > 
> > So any patch you'd have in this area would need rebasing.  I'll gladly
> > look at what you have (even if it isn't rebased).  So yes please share.
> > 
> > (it could be that there isn't a big enough win associated with switching
> > to rwlock_t -- that we could get away without doing that particular
> > churn.. open to that if you think rwlock_t pointless given we'll take
> > the write lock after repeat_count drops to 0)
> > 
> personally, I don't think the switching to a rwlock_t will buy us
> anything; for a decent performance you have to set rq_min_io to 1
> anyway, thereby defeating the purpose of the rwlock.

OK, I'll drop the rwlock_t commit.
 
> My thinking was rather a different direction:
> Move the crucial bits of the multipath structure to atomics, and
> split off the path selection code into one bit for selecting the
> path within a path group, and another which switches the path groups.
> When we do that we could use rcus for the paths themselves, and
> would only need to take the spinlock if we need to switch path
> groups. Which should be okay as switching path groups is
> (potentially) a rather slow operation anyway.

If you were to put focus to dusting your work off and helping me get
switched over to RCU I'd be very thankful.


From keith.busch at intel.com  Thu Feb  4 07:44:54 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 4 Feb 2016 15:44:54 +0000
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <BJXPR01MB19990484665533561B21DB3E0D10@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
 <1454417099-19332-3-git-send-email-sagig@mellanox.com>
 <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160202164915.GB10728@localhost.localdomain>
 <BJXPR01MB1999CBB2644C70CE07E4B28E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203012638.GC21046@localhost.localdomain>
 <BJXPR01MB19990484665533561B21DB3E0D10@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160204154454.GB28747@localhost.localdomain>

On Thu, Feb 04, 2016 at 02:28:24PM +0000, Wenbo Wang wrote:
> Is the following execution valid?
> 
> 1. request A is linked in q->request_list

q->request_list? Do you mean q->requeue_list?

> 2. device reset
> 3. all requests (including request A) are cancelled by nvme_dev_disable()->nvme_clear_queue()

If the request is on some list owned by the request_queue, it was not
started by the driver, and nvme_clear_queue doesn't do anything to it.

> 4. device restarted
> 5. flush q->request_list, and request A is again running in queue_rq.         <-- since request A has been cancelled, it shall not be running again

Request A was not cancelled if it was never started. It will be submitted
when the driver restarts the queues h/w contexts.

Cancelling a command with the controller does not mean ending the request
in the block layer.


From keith.busch at intel.com  Thu Feb  4 07:52:22 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 4 Feb 2016 15:52:22 +0000
Subject: [PATCH 0/4] NVMe: Surprise removal fixes
In-Reply-To: <BJXPR01MB1997AE807A93FA2EBB32076E0D10@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <BJXPR01MB1997AE807A93FA2EBB32076E0D10@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160204155221.GC28747@localhost.localdomain>

On Thu, Feb 04, 2016 at 03:25:50AM +0000, Wenbo Wang wrote:
> Isn't the dd behavior correct? For non-direct dd, even if a surprise remove happened, it simply keeps dirtying the page cache and is not aware of any errors below.

Try it with a usb drive. It ends with ENOSPC error.


From sagig at dev.mellanox.co.il  Thu Feb  4 07:55:05 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 4 Feb 2016 17:55:05 +0200
Subject: [PATCH 0/4] NVMe: Surprise removal fixes
In-Reply-To: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
Message-ID: <56B37459.3080602@dev.mellanox.co.il>

Hi Keith,

> Earlier feedback suggested this functionality be in the block layer. I
> don't have the devices to test what happens with other drivers, and
> the desired sequence seems unique to NVMe.

Umm, it actually isn't. I think this is what all the block drivers
want.

> Maybe that's an indication
> that the driver's flow could benefit from some redesign, or maybe it's
> an artifact from having the controller and storage being the same device.
>
> In any case, I would like to isolate this fix to the NVMe driver in
> the interest of time, and flush out the block layer before the next
> merge window.

I'm fine with that. But we definitely want to avoid the queue_rq
conditional in the long run...


From sagig at dev.mellanox.co.il  Thu Feb  4 07:57:07 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 4 Feb 2016 17:57:07 +0200
Subject: [PATCH 2/4] NVMe: Sync stopped queues with block layer
In-Reply-To: <1454515543-21683-3-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-3-git-send-email-keith.busch@intel.com>
Message-ID: <56B374D3.9010003@dev.mellanox.co.il>


> Make sure no work will submit requests to a stopped h/w queue.

Not sure what that buys us, and it's not documented. But
I don't mind having it. Can you provide a better explanation
why it's added?


From sagig at dev.mellanox.co.il  Thu Feb  4 07:59:41 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 4 Feb 2016 17:59:41 +0200
Subject: [PATCH 4/4] blk-mq: End unstarted requests on dying queue
In-Reply-To: <1454515543-21683-5-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-5-git-send-email-keith.busch@intel.com>
Message-ID: <56B3756D.7040500@dev.mellanox.co.il>


> Go directly to ending a request if it wasn't started. Previously, completing a
> request may invoke a driver callback for a request it didn't initialize.

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From keith.busch at intel.com  Thu Feb  4 08:00:44 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 4 Feb 2016 16:00:44 +0000
Subject: [PATCH 2/4] NVMe: Sync stopped queues with block layer
In-Reply-To: <56B374D3.9010003@dev.mellanox.co.il>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-3-git-send-email-keith.busch@intel.com>
 <56B374D3.9010003@dev.mellanox.co.il>
Message-ID: <20160204160044.GD28747@localhost.localdomain>

On Thu, Feb 04, 2016 at 05:57:07PM +0200, Sagi Grimberg wrote:
> 
> >Make sure no work will submit requests to a stopped h/w queue.
> 
> Not sure what that buys us, and it's not documented. But
> I don't mind having it. Can you provide a better explanation
> why it's added?

This was trying to close that gap with requests queueing on stopped h/w
queues and writing the queue doorbell. It works for requests through
the kblock_workqueue, but that's not always the case, so it's only a
partial solution.


From sagig at dev.mellanox.co.il  Thu Feb  4 07:55:28 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 4 Feb 2016 17:55:28 +0200
Subject: [PATCH 1/4] NVMe: Fix io incapable return values
In-Reply-To: <1454515543-21683-2-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-2-git-send-email-keith.busch@intel.com>
Message-ID: <56B37470.4090009@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From wenbo.wang at memblaze.com  Thu Feb  4 22:53:34 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Fri, 5 Feb 2016 06:53:34 +0000
Subject: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination
In-Reply-To: <20160204154454.GB28747@localhost.localdomain>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
 <1454417099-19332-3-git-send-email-sagig@mellanox.com>
 <BJXPR01MB1992E35D667BC3289F424CAE0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160202164915.GB10728@localhost.localdomain>
 <BJXPR01MB1999CBB2644C70CE07E4B28E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203012638.GC21046@localhost.localdomain>
 <BJXPR01MB19990484665533561B21DB3E0D10@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160204154454.GB28747@localhost.localdomain>
Message-ID: <BJXPR01MB19999BF4E16A2752494C2F3E0D20@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Thanks.

The problem is that just after nvme_cancel_queue_ios checks the request is started, someone else move the request to q->requeue_list. 
The following patch I proposed has this issue. Queue_rq returns BLK_MQ_RQ_QUEUE_BUSY to __blk_mq_run_hw_queue, which re-queues the request to q->queue_list.
The solution may be call blk_mq_start_request after checking cq_vector. However I suspect some other code may break nvme_cancel_queue_ios too.

Is it a general solution to add a bit lock for each request and use that lock to synchronize stuffs?

Subject: [PATCH v2] NVMe: do not touch sq door bell if nvmeq has been suspended
>@@ -678,6 +678,11 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
> 	blk_mq_start_request(req);
> 
> 	spin_lock_irq(&nvmeq->q_lock);
>+	if (unlikely(nvmeq->cq_vector == -1)) {
>+		ret = BLK_MQ_RQ_QUEUE_BUSY;
>+		spin_unlock_irq(&nvmeq->q_lock);
>+		goto out;
>+	}
> 	__nvme_submit_cmd(nvmeq, &cmnd);
> 	nvme_process_cq(nvmeq);
> 	spin_unlock_irq(&nvmeq->q_lock);
>--



-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Thursday, February 4, 2016 11:45 PM
To: Wenbo Wang
Cc: Sagi Grimberg; Jens Axboe; linux-nvme at lists.infradead.org
Subject: Re: [PATCH v1 2/3] nvme: Use blk-mq helper for IO termination

On Thu, Feb 04, 2016 at 02:28:24PM +0000, Wenbo Wang wrote:
> Is the following execution valid?
> 
> 1. request A is linked in q->request_list

q->request_list? Do you mean q->requeue_list?

> 2. device reset
> 3. all requests (including request A) are cancelled by 
> nvme_dev_disable()->nvme_clear_queue()

If the request is on some list owned by the request_queue, it was not started by the driver, and nvme_clear_queue doesn't do anything to it.

> 4. device restarted
> 5. flush q->request_list, and request A is again running in queue_rq.         <-- since request A has been cancelled, it shall not be running again

Request A was not cancelled if it was never started. It will be submitted when the driver restarts the queues h/w contexts.

Cancelling a command with the controller does not mean ending the request in the block layer.

From snitzer at redhat.com  Fri Feb  5 07:13:35 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Fri, 5 Feb 2016 10:13:35 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160204135420.GA18227@redhat.com>
References: <20160127174828.GA31802@redhat.com>
 <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com>
Message-ID: <20160205151334.GA82754@redhat.com>

On Thu, Feb 04 2016 at  8:54P -0500,
Mike Snitzer <snitzer at redhat.com> wrote:

> On Thu, Feb 04 2016 at  1:54am -0500,
> Hannes Reinecke <hare at suse.de> wrote:
> 
> > On 02/03/2016 07:24 PM, Mike Snitzer wrote:
> > > On Wed, Feb 03 2016 at  1:04pm -0500,
> > > Mike Snitzer <snitzer at redhat.com> wrote:
> > >  
> > >> I'm still not clear on where the considerable performance loss is coming
> > >> from (on null_blk device I see ~1900K read IOPs but I'm still only
> > >> seeing ~1000K read IOPs when blk-mq DM-multipath is layered ontop).
> > >> What is very much apparent is: layering dm-mq multipath ontop of null_blk
> > >> results in a HUGE amount of additional context switches.  I can only
> > >> infer that the request completion for this stacked device (blk-mq queue
> > >> ontop of blk-mq queue, with 2 completions: 1 for clone completing on
> > >> underlying device and 1 for original request completing) is the reason
> > >> for all the extra context switches.
> > > 
> > > Starts to explain, certainly not the "reason"; that is still very much
> > > TBD...
> > > 
> > >> Here are pictures of 'perf report' for perf datat collected using
> > >> 'perf record -ag -e cs'.
> > >>
> > >> Against null_blk:
> > >> http://people.redhat.com/msnitzer/perf-report-cs-null_blk.png
> > > 
> > > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
> > >   cpu          : usr=25.53%, sys=74.40%, ctx=1970, majf=0, minf=474
> > > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
> > >   cpu          : usr=26.79%, sys=73.15%, ctx=2067, majf=0, minf=479
> > > 
> > >> Against dm-mpath ontop of the same null_blk:
> > >> http://people.redhat.com/msnitzer/perf-report-cs-dm_mq.png
> > > 
> > > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=1
> > >   cpu          : usr=11.07%, sys=33.90%, ctx=667784, majf=0, minf=466
> > > if dm-mq nr_hw_queues=1 and null_blk nr_hw_queues=4
> > >   cpu          : usr=15.22%, sys=48.44%, ctx=2314901, majf=0, minf=466
> > > 
> > > So yeah, the percentages reflected in these respective images didn't do
> > > the huge increase in context switches justice... we _must_ figure out
> > > why we're seeing so many context switches with dm-mq.
> > > 
> > Well, the most obvious one being that you're using 1 dm-mq queue vs
> > 4 null_blk queues.
> > So you will have have to do an additional context switch for 75% of
> > the total I/Os submitted.
> 
> Right, that case is certainly prone to more context switches.  But I'm
> initially most concerned about the case where both only have 1 queue.
> 
> > Have you tested with 4 dm-mq hw queues?
> 
> Yes, it makes performance worse.  This is likely rooted in dm-mpath IO
> path not being lockless.  But I also have concern about whether the
> clone, sent to the underlying path, is completing on a different cpu
> than dm-mq's original request.
> 
> I'll be using ftrace to try to dig into the various aspects of this
> (perf, as I know how to use it, isn't giving me enough precision in its
> reporting).
> 
> > To avoid context switches we would have to align the dm-mq queues to
> > the underlying blk-mq layout for the paths.
> 
> Right, we need to take more care (how remains TBD).  But for now I'm
> just going to focus on the case where both dm-mq and null_blk have 1 for
> nr_hw_queues.  As you can see even in that config the number of context
> switches goes from 1970 to 667784 (and there is a huge loss of system
> cpu utilization) once dm-mq w/ 1 hw_queue is stacked ontop on the
> null_blk device.
> 
> Once we understand the source of all the additional context switching
> for this more simplistic stacked configuration we can look closer at
> scaling as we add more underlying paths.

Following is RFC because it really speaks to dm-mq _needing_ a variant
of blk_mq_complete_request() that supports partial completions.  Not
supporting partial completions really isn't an option for DM multipath.

From: Mike Snitzer <snitzer at redhat.com>
Date: Fri, 5 Feb 2016 08:49:01 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching

Request-based DM's blk-mq support (dm-mq) was reported to be 50% slower
than if an underlying null_blk device were used directly.  This biggest
reason for this drop in performance is that blk_insert_clone_request()
was calling blk_mq_insert_request() with @async=true.  This forced the
use of kblockd_schedule_delayed_work_on() to run the queues which
ushered in ping-ponging between process context (fio in this case) and
kblockd's kworker to submit the cloned request.  The ftrace
function_graph tracer showed:

  kworker-2013  =>   fio-12190
  fio-12190    =>  kworker-2013
  ...
  kworker-2013  =>   fio-12190
  fio-12190    =>  kworker-2013
  ...

Fixing blk_mq_insert_request() to _not_ use kblockd to submit the cloned
requests isn't enough to fix eliminated the oberved context switches.

In addition to this dm-mq specific blk-core fix, there were 2 DM core
fixes to dm-mq that (when paired with the blk-core fix) completely
eliminate the observed context switching:

1)  don't blk_mq_run_hw_queues in blk-mq request completion

    Motivated by desire to reduce overhead of dm-mq, punting to kblockd
    just increases context switches.

    In my testing against a really fast null_blk device there was no benefit
    to running blk_mq_run_hw_queues() on completion (and no other blk-mq
    driver does this).  So hopefully this change doesn't induce the need for
    yet another revert like commit 621739b00e16ca2d !

2)  use blk_mq_complete_request() in dm_complete_request()

    blk_complete_request() doesn't offer the traditional q->mq_ops vs
    .request_fn branching pattern that other historic block interfaces
    do (e.g. blk_get_request).  Using blk_mq_complete_request() for
    blk-mq requests is important for performance but it doesn't handle
    partial completions -- which is a pretty big problem given the
    potential for partial completions with DM multipath due to path
    failure(s).  As such this makes this entire patch only RFC-worthy.

dm-mq "fix" #2 is _much_ more important than #1 for eliminating the
excessive context switches.
Before: cpu          : usr=15.10%, sys=59.39%, ctx=7905181, majf=0, minf=475
After:  cpu          : usr=20.60%, sys=79.35%, ctx=2008, majf=0, minf=472

With these changes the multithreaded async read IOPs improved from ~950K
to ~1350K for this dm-mq stacked on null_blk test-case.  The raw read
IOPs of the underlying null_blk device for the same workload is ~1950K.

Reported-by: Sagi Grimberg <sagig at dev.mellanox.co.il>
Cc: Jens Axboe <axboe at kernel.dk>
Signed-off-by: Mike Snitzer <snitzer at redhat.com>
---
 block/blk-core.c |  2 +-
 drivers/md/dm.c  | 13 ++++++-------
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index ab51685..c60e233 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -2198,7 +2198,7 @@ int blk_insert_cloned_request(struct request_queue *q, struct request *rq)
 	if (q->mq_ops) {
 		if (blk_queue_io_stat(q))
 			blk_account_io_start(rq, true);
-		blk_mq_insert_request(rq, false, true, true);
+		blk_mq_insert_request(rq, false, true, false);
 		return 0;
 	}
 
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index c683f6d..a618477 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1119,12 +1119,8 @@ static void rq_completed(struct mapped_device *md, int rw, bool run_queue)
 	 * back into ->request_fn() could deadlock attempting to grab the
 	 * queue lock again.
 	 */
-	if (run_queue) {
-		if (md->queue->mq_ops)
-			blk_mq_run_hw_queues(md->queue, true);
-		else
-			blk_run_queue_async(md->queue);
-	}
+	if (!md->queue->mq_ops && run_queue)
+		blk_mq_run_hw_queues(md->queue, true);
 
 	/*
 	 * dm_put() must be at the end of this function. See the comment above
@@ -1344,7 +1340,10 @@ static void dm_complete_request(struct request *rq, int error)
 	struct dm_rq_target_io *tio = tio_from_request(rq);
 
 	tio->error = error;
-	blk_complete_request(rq);
+	if (!rq->q->mq_ops)
+		blk_complete_request(rq);
+	else
+		blk_mq_complete_request(rq, rq->errors);
 }
 
 /*
-- 
2.5.4 (Apple Git-61)



From snitzer at redhat.com  Fri Feb  5 10:05:15 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Fri, 5 Feb 2016 13:05:15 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160205151334.GA82754@redhat.com>
References: <56A904B6.50407@dev.mellanox.co.il>
 <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com>
Message-ID: <20160205180515.GA25808@redhat.com>

On Fri, Feb 05 2016 at 10:13am -0500,
Mike Snitzer <snitzer at redhat.com> wrote:
 
> Following is RFC because it really speaks to dm-mq _needing_ a variant
> of blk_mq_complete_request() that supports partial completions.  Not
> supporting partial completions really isn't an option for DM multipath.
> 
> From: Mike Snitzer <snitzer at redhat.com>
> Date: Fri, 5 Feb 2016 08:49:01 -0500
> Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
> 
> Request-based DM's blk-mq support (dm-mq) was reported to be 50% slower
> than if an underlying null_blk device were used directly.  This biggest
> reason for this drop in performance is that blk_insert_clone_request()
> was calling blk_mq_insert_request() with @async=true.  This forced the
> use of kblockd_schedule_delayed_work_on() to run the queues which
> ushered in ping-ponging between process context (fio in this case) and
> kblockd's kworker to submit the cloned request.  The ftrace
> function_graph tracer showed:
> 
>   kworker-2013  =>   fio-12190
>   fio-12190    =>  kworker-2013
>   ...
>   kworker-2013  =>   fio-12190
>   fio-12190    =>  kworker-2013
>   ...
> 
> Fixing blk_mq_insert_request() to _not_ use kblockd to submit the cloned
> requests isn't enough to fix eliminated the oberved context switches.
> 
> In addition to this dm-mq specific blk-core fix, there were 2 DM core
> fixes to dm-mq that (when paired with the blk-core fix) completely
> eliminate the observed context switching:
> 
> 1)  don't blk_mq_run_hw_queues in blk-mq request completion
> 
>     Motivated by desire to reduce overhead of dm-mq, punting to kblockd
>     just increases context switches.
> 
>     In my testing against a really fast null_blk device there was no benefit
>     to running blk_mq_run_hw_queues() on completion (and no other blk-mq
>     driver does this).  So hopefully this change doesn't induce the need for
>     yet another revert like commit 621739b00e16ca2d !
> 
> 2)  use blk_mq_complete_request() in dm_complete_request()
> 
>     blk_complete_request() doesn't offer the traditional q->mq_ops vs
>     .request_fn branching pattern that other historic block interfaces
>     do (e.g. blk_get_request).  Using blk_mq_complete_request() for
>     blk-mq requests is important for performance but it doesn't handle
>     partial completions -- which is a pretty big problem given the
>     potential for partial completions with DM multipath due to path
>     failure(s).  As such this makes this entire patch only RFC-worthy.

> diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> index c683f6d..a618477 100644
> --- a/drivers/md/dm.c
> +++ b/drivers/md/dm.c
> @@ -1344,7 +1340,10 @@ static void dm_complete_request(struct request *rq, int error)
>  	struct dm_rq_target_io *tio = tio_from_request(rq);
>  
>  	tio->error = error;
> -	blk_complete_request(rq);
> +	if (!rq->q->mq_ops)
> +		blk_complete_request(rq);
> +	else
> +		blk_mq_complete_request(rq, rq->errors);
>  }
>  
>  /*

Looking closer, DM is very likely OK just using blk_mq_complete_request.

blk_complete_request() also doesn't provide native partial completion
support (it relies on the driver to do it, which DM core does):

/**
 * blk_complete_request - end I/O on a request
 * @req:      the request being processed
 *
 * Description:
 *     Ends all I/O on a request. It does not handle partial completions,
 *     unless the driver actually implements this in its completion callback
 *     through requeueing. The actual completion happens out-of-order,
 *     through a softirq handler. The user must have registered a completion
 *     callback through blk_queue_softirq_done().
 **/

blk_mq_complete_request() is effectively implemented in a comparable
fashion to blk_complete_request().  Given that DM core is providing
partial completion support by dm.c:end_clone_bio() triggering requeueing
of the request via dm-mpath.c:multipath_end_io()'s return of
DM_ENDIO_REQUEUE.

So I'm thinking I can drop the "RFC" for this patch and run with
it.. once I get Jens' feedback (hopefully) confirming my understanding.

Jens, please advise.  If you're comfortable providing your Acked-by I
can get this fix in for 4.5-rc4 or so...

Thanks!

Mike


From snitzer at redhat.com  Fri Feb  5 11:19:10 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Fri, 5 Feb 2016 14:19:10 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160205180515.GA25808@redhat.com>
References: <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com>
Message-ID: <20160205191909.GA25982@redhat.com>

On Fri, Feb 05 2016 at  1:05pm -0500,
Mike Snitzer <snitzer at redhat.com> wrote:

> On Fri, Feb 05 2016 at 10:13am -0500,
> Mike Snitzer <snitzer at redhat.com> wrote:
>  
> > Following is RFC because it really speaks to dm-mq _needing_ a variant
> > of blk_mq_complete_request() that supports partial completions.  Not
> > supporting partial completions really isn't an option for DM multipath.
> > 
> > From: Mike Snitzer <snitzer at redhat.com>
> > Date: Fri, 5 Feb 2016 08:49:01 -0500
> > Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
> > 
> > Request-based DM's blk-mq support (dm-mq) was reported to be 50% slower
> > than if an underlying null_blk device were used directly.  This biggest
> > reason for this drop in performance is that blk_insert_clone_request()
> > was calling blk_mq_insert_request() with @async=true.  This forced the
> > use of kblockd_schedule_delayed_work_on() to run the queues which
> > ushered in ping-ponging between process context (fio in this case) and
> > kblockd's kworker to submit the cloned request.  The ftrace
> > function_graph tracer showed:
> > 
> >   kworker-2013  =>   fio-12190
> >   fio-12190    =>  kworker-2013
> >   ...
> >   kworker-2013  =>   fio-12190
> >   fio-12190    =>  kworker-2013
> >   ...
> > 
> > Fixing blk_mq_insert_request() to _not_ use kblockd to submit the cloned
> > requests isn't enough to fix eliminated the oberved context switches.
> > 
> > In addition to this dm-mq specific blk-core fix, there were 2 DM core
> > fixes to dm-mq that (when paired with the blk-core fix) completely
> > eliminate the observed context switching:
> > 
> > 1)  don't blk_mq_run_hw_queues in blk-mq request completion
> > 
> >     Motivated by desire to reduce overhead of dm-mq, punting to kblockd
> >     just increases context switches.
> > 
> >     In my testing against a really fast null_blk device there was no benefit
> >     to running blk_mq_run_hw_queues() on completion (and no other blk-mq
> >     driver does this).  So hopefully this change doesn't induce the need for
> >     yet another revert like commit 621739b00e16ca2d !
> > 
> > 2)  use blk_mq_complete_request() in dm_complete_request()
> > 
> >     blk_complete_request() doesn't offer the traditional q->mq_ops vs
> >     .request_fn branching pattern that other historic block interfaces
> >     do (e.g. blk_get_request).  Using blk_mq_complete_request() for
> >     blk-mq requests is important for performance but it doesn't handle
> >     partial completions -- which is a pretty big problem given the
> >     potential for partial completions with DM multipath due to path
> >     failure(s).  As such this makes this entire patch only RFC-worthy.
> 
> > diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> > index c683f6d..a618477 100644
> > --- a/drivers/md/dm.c
> > +++ b/drivers/md/dm.c
> > @@ -1344,7 +1340,10 @@ static void dm_complete_request(struct request *rq, int error)
> >  	struct dm_rq_target_io *tio = tio_from_request(rq);
> >  
> >  	tio->error = error;
> > -	blk_complete_request(rq);
> > +	if (!rq->q->mq_ops)
> > +		blk_complete_request(rq);
> > +	else
> > +		blk_mq_complete_request(rq, rq->errors);
> >  }
> >  
> >  /*
> 
> Looking closer, DM is very likely OK just using blk_mq_complete_request.
> 
> blk_complete_request() also doesn't provide native partial completion
> support (it relies on the driver to do it, which DM core does):
> 
> /**
>  * blk_complete_request - end I/O on a request
>  * @req:      the request being processed
>  *
>  * Description:
>  *     Ends all I/O on a request. It does not handle partial completions,
>  *     unless the driver actually implements this in its completion callback
>  *     through requeueing. The actual completion happens out-of-order,
>  *     through a softirq handler. The user must have registered a completion
>  *     callback through blk_queue_softirq_done().
>  **/
> 
> blk_mq_complete_request() is effectively implemented in a comparable
> fashion to blk_complete_request().  Given that DM core is providing
> partial completion support by dm.c:end_clone_bio() triggering requeueing
> of the request via dm-mpath.c:multipath_end_io()'s return of
> DM_ENDIO_REQUEUE.
> 
> So I'm thinking I can drop the "RFC" for this patch and run with
> it.. once I get Jens' feedback (hopefully) confirming my understanding.
> 
> Jens, please advise.  If you're comfortable providing your Acked-by I
> can get this fix in for 4.5-rc4 or so...

FYI, here is the latest revised patch:
https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5b835282422ec41991c1dbdb88daa4af7d166d2

(revised patch header and fixed a thinko in the dm.c:rq_completed()
change from the RFC patch I posted earlier)


From wenbo.wang at memblaze.com  Sat Feb  6 06:32:24 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Sat, 6 Feb 2016 14:32:24 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <20160203163808.GC23910@localhost.localdomain>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203144123.GB23910@localhost.localdomain>
 <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203163808.GC23910@localhost.localdomain>
Message-ID: <BJXPR01MB19916231021454BD8731EBDE0D30@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Keith,

Is the following solution OK?
synchronize_rcu guarantee that no queue_rq is running concurrently with device disable code. Together with your another patch (adding blk_sync_queue), both sync/async path shall be handled correctly.

Do you think synchronize_rcu shall be added to blk_sync_queue?

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 4c0622f..bfe9132 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -865,7 +865,9 @@ void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async)
        if (!async) {
                int cpu = get_cpu();
                if (cpumask_test_cpu(cpu, hctx->cpumask)) {
+                       rcu_read_lock();
                        __blk_mq_run_hw_queue(hctx);
+                       rcu_read_unlock();
                        put_cpu();
                        return;
                }

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 8b1a725..d1e5e63 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1861,6 +1861,7 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
                nvme_disable_io_queues(dev);
                nvme_disable_admin_queue(dev, shutdown);
        }
+       synchronize_rcu();
        nvme_dev_unmap(dev);

        for (i = dev->queue_count - 1; i >= 0; i--)

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Thursday, February 4, 2016 12:38 AM
To: Wenbo Wang
Cc: Jens Axboe; Wenbo Wang; linux-kernel at vger.kernel.org; linux-nvme at lists.infradead.org; Wenwei.Tao
Subject: Re: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended

On Wed, Feb 03, 2016 at 04:35:03PM +0000, Wenbo Wang wrote:
> For async io (executed by run_work worker) it should work. However for sync io in blk_mq_run_hw_queue, this seems not help, there is still a window.

Alright, for lack of a better way to sync a stopped queue, the driver's indication looks like the best we can do at this point.

From James.Bottomley at HansenPartnership.com  Sat Feb  6 10:33:44 2016
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Sat, 06 Feb 2016 10:33:44 -0800
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO support
 optional
Message-ID: <1454783624.2809.6.camel@HansenPartnership.com>

The reason is fairly obvious: the default for the new option
 BLK_DEV_NVME_SCSI is N and all the distribution kernels (and me when
testing) take the default options (I checked in the OBS kernel builds
and this is true).

The net result is that scsi_id from udev no longer works on nvme disks
and that means that the /dev/disk/by-id links are all gone in 4.5-rc1. 
 If this happens to be how you name your disks in fstab or crypttab,
you can't boot.

If you're going to break an ABI in this way, you really have to plan
for it in userspace.  How are NVMe disk ids supposed to be exported
now?  Does udev need a nvme_id program to do this?  Until there's an
infrastructure ready to work in this way, we need to unconditionally
enable BLK_DEV_NVME_SCSI.

James



From hch at lst.de  Sun Feb  7 01:22:41 2016
From: hch at lst.de (Christoph Hellwig)
Date: Sun, 7 Feb 2016 10:22:41 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <1454783624.2809.6.camel@HansenPartnership.com>
References: <1454783624.2809.6.camel@HansenPartnership.com>
Message-ID: <20160207092241.GA15331@lst.de>

Keith said it should be on by default, and I promised him to change
it once we run into problems, which I guess this counts as.

But just curious:  what distro are you using?  Upstream systemd
explicitly rejected using scsi_id for NVMe here:

	https://github.com/systemd/systemd/issues/1453

and all my test systems don't do this either.


From sagig at dev.mellanox.co.il  Sun Feb  7 05:41:53 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 7 Feb 2016 15:41:53 +0200
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <BJXPR01MB19916231021454BD8731EBDE0D30@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203144123.GB23910@localhost.localdomain>
 <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203163808.GC23910@localhost.localdomain>
 <BJXPR01MB19916231021454BD8731EBDE0D30@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <56B749A1.8030504@dev.mellanox.co.il>


> Keith,
>
> Is the following solution OK?
> synchronize_rcu guarantee that no queue_rq is running concurrently with device disable code.
> Together with your another patch (adding blk_sync_queue), both sync/async path shall be handled correctly.

This can be acceptable I think.

> Do you think synchronize_rcu shall be added to blk_sync_queue?

Or, we'll add it to blk_mq_stop_hw_queues() and then scsi
will enjoy it as well.

> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 4c0622f..bfe9132 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -865,7 +865,9 @@ void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async)
>          if (!async) {
>                  int cpu = get_cpu();
>                  if (cpumask_test_cpu(cpu, hctx->cpumask)) {
> +                       rcu_read_lock();
>                          __blk_mq_run_hw_queue(hctx);
> +                       rcu_read_unlock();

I think the rcu is better folded into __blk_mq_run_hw_queue
to cover all the call sites.


From sagig at dev.mellanox.co.il  Sun Feb  7 07:41:16 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 7 Feb 2016 17:41:16 +0200
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160205191909.GA25982@redhat.com>
References: <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
 <56B2F5BC.1010700@suse.de> <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com> <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com>
Message-ID: <56B7659C.8040601@dev.mellanox.co.il>


> FYI, here is the latest revised patch:
> https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5b835282422ec41991c1dbdb88daa4af7d166d2
>
> (revised patch header and fixed a thinko in the dm.c:rq_completed()
> change from the RFC patch I posted earlier)

Hi Mike,

So I gave your patches a go (dm-4.6) but I still don't see the
improvement you reported (while I do see a minor improvement).

null_blk queue_mode=2 submit_queues=24
dm_mod blk_mq_nr_hw_queues=24 blk_mq_queue_depth=4096 use_blk_mq=Y

I see 620K IOPs on dm_mq vs. 1750K IOPs on raw nullb0.

Is there something I'm missing?


From James.Bottomley at HansenPartnership.com  Sun Feb  7 08:04:00 2016
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Sun, 07 Feb 2016 08:04:00 -0800
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <20160207092241.GA15331@lst.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
Message-ID: <1454861040.2329.4.camel@HansenPartnership.com>

On Sun, 2016-02-07 at 10:22 +0100, Christoph Hellwig wrote:
> Keith said it should be on by default, and I promised him to change
> it once we run into problems, which I guess this counts as.
> 
> But just curious:  what distro are you using?  Upstream systemd
> explicitly rejected using scsi_id for NVMe here:
> 
> 	https://github.com/systemd/systemd/issues/1453
> 
> and all my test systems don't do this either.

This was SUSE (in my case, openSUSE Leap).  I just checked the source
package; they patch the by-id rules back in for NVME:

# PATCH-FIX-SUSE 1101-rules-persistent-device-names-for-NVMe-devices.patch (bsc#944132)
Patch1101:      1101-rules-persistent-device-names-for-NVMe-devices.patch

The bugzilla is giving access denied for bug id 944132, so it's likely
some proprietary vendor problem.  The patch has no preamble, so it's
hard to tell what they were thinking.

James



From snitzer at redhat.com  Sun Feb  7 08:07:12 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Sun, 7 Feb 2016 11:07:12 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B7659C.8040601@dev.mellanox.co.il>
References: <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il>
Message-ID: <20160207160712.GA6017@redhat.com>

On Sun, Feb 07 2016 at 10:41am -0500,
Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:

> 
> >FYI, here is the latest revised patch:
> >https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5b835282422ec41991c1dbdb88daa4af7d166d2
> >
> >(revised patch header and fixed a thinko in the dm.c:rq_completed()
> >change from the RFC patch I posted earlier)
> 
> Hi Mike,
> 
> So I gave your patches a go (dm-4.6) but I still don't see the
> improvement you reported (while I do see a minor improvement).
> 
> null_blk queue_mode=2 submit_queues=24
> dm_mod blk_mq_nr_hw_queues=24 blk_mq_queue_depth=4096 use_blk_mq=Y
> 
> I see 620K IOPs on dm_mq vs. 1750K IOPs on raw nullb0.

blk_mq_nr_hw_queues=24 isn't likely to help you (but with these patches,
the first being the most important, it shouldn't hurt either provided
you have 24 cpus).

Could be you have multiple NUMA nodes and are seeing problems from that?

I have 12 cpus (in the same physical cpu) and only a single NUMA node.
I get the same results as blk_mq_nr_hw_queues=12 with
blk_mq_nr_hw_queues=4 (same goes for null_blk submit_queues).
I've seen my IOPs go from ~950K to ~1400K.  The peak null_blk can get on
my setup is ~1950K.  So I'm still seeing a ~25% drop with dm-mq (but
that is much better than the over 50% drop I saw seeing).

> Is there something I'm missing?

Not sure, I just emailed out all my patches (and cc'd you).  Please
verify you're using the latest here (same as 'dm-4.6' branch):
https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/log/?h=for-next

I rebased a couple times... so please diff what you have tested against
this latest 'dm-4.6' branch.


From bart.vanassche at sandisk.com  Sun Feb  7 08:37:42 2016
From: bart.vanassche at sandisk.com (Bart Van Assche)
Date: Sun, 7 Feb 2016 08:37:42 -0800
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B7659C.8040601@dev.mellanox.co.il>
References: <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
 <56B2F5BC.1010700@suse.de> <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com> <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com> <56B7659C.8040601@dev.mellanox.co.il>
Message-ID: <56B772D6.2090403@sandisk.com>

On 02/07/16 07:41, Sagi Grimberg wrote:
>
>> FYI, here is the latest revised patch:
>> https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-4.6&id=a5b835282422ec41991c1dbdb88daa4af7d166d2
>>
>> (revised patch header and fixed a thinko in the dm.c:rq_completed()
>> change from the RFC patch I posted earlier)
>
> Hi Mike,
>
> So I gave your patches a go (dm-4.6) but I still don't see the
> improvement you reported (while I do see a minor improvement).
>
> null_blk queue_mode=2 submit_queues=24
> dm_mod blk_mq_nr_hw_queues=24 blk_mq_queue_depth=4096 use_blk_mq=Y
>
> I see 620K IOPs on dm_mq vs. 1750K IOPs on raw nullb0.
>
> Is there something I'm missing?

Hello Sagi,

Did you run your test on a NUMA system ? If so, can you check with e.g. 
perf record -ags -e LLC-load-misses sleep 10 && perf report whether this 
workload triggers perhaps lock contention ? What you need to look for in 
the perf output is whether any functions occupy more than 10% CPU time.

Bart.




From sagig at dev.mellanox.co.il  Sun Feb  7 08:42:43 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 7 Feb 2016 18:42:43 +0200
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160207160712.GA6017@redhat.com>
References: <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
 <56B2F5BC.1010700@suse.de> <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com> <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com> <56B7659C.8040601@dev.mellanox.co.il>
 <20160207160712.GA6017@redhat.com>
Message-ID: <56B77403.40108@dev.mellanox.co.il>


>> Hi Mike,
>>
>> So I gave your patches a go (dm-4.6) but I still don't see the
>> improvement you reported (while I do see a minor improvement).
>>
>> null_blk queue_mode=2 submit_queues=24
>> dm_mod blk_mq_nr_hw_queues=24 blk_mq_queue_depth=4096 use_blk_mq=Y
>>
>> I see 620K IOPs on dm_mq vs. 1750K IOPs on raw nullb0.
>
> blk_mq_nr_hw_queues=24 isn't likely to help you (but with these patches,
> the first being the most important, it shouldn't hurt either provided
> you have 24 cpus).

I tried with less but as you said, it didn't have an impact...

> Could be you have multiple NUMA nodes and are seeing problems from that?

I am running on a dual socket server, this can most likely be the
culprit...

> I have 12 cpus (in the same physical cpu) and only a single NUMA node.
> I get the same results as blk_mq_nr_hw_queues=12 with
> blk_mq_nr_hw_queues=4 (same goes for null_blk submit_queues).
> I've seen my IOPs go from ~950K to ~1400K.  The peak null_blk can get on
> my setup is ~1950K.  So I'm still seeing a ~25% drop with dm-mq (but
> that is much better than the over 50% drop I saw seeing).

That's what I was planning on :(

>> Is there something I'm missing?
>
> Not sure, I just emailed out all my patches (and cc'd you).  Please
> verify you're using the latest here (same as 'dm-4.6' branch):
> https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/log/?h=for-next
>
> I rebased a couple times... so please diff what you have tested against
> this latest 'dm-4.6' branch.

I am. I'll try to instrument what's going on...


From sagig at dev.mellanox.co.il  Sun Feb  7 08:43:48 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 7 Feb 2016 18:43:48 +0200
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B772D6.2090403@sandisk.com>
References: <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
 <56B2F5BC.1010700@suse.de> <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com> <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com> <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
Message-ID: <56B77444.3030106@dev.mellanox.co.il>


> Hello Sagi,

Hey Bart,

> Did you run your test on a NUMA system ?

I did.

> If so, can you check with e.g.
> perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
> workload triggers perhaps lock contention ? What you need to look for in
> the perf output is whether any functions occupy more than 10% CPU time.

I will, thanks for the tip!


From mike at fireburn.co.uk  Sun Feb  7 08:43:35 2016
From: mike at fireburn.co.uk (Mike Lothian)
Date: Sun, 7 Feb 2016 16:43:35 +0000
Subject: NVME Issues with Suspend
In-Reply-To: <CAHbf0-GDmbx3xO+Kmb8y8X-41TxnaetQ=k4FR-NonMZoL-U2UA@mail.gmail.com>
References: <CAHbf0-GSRV+SH4FvDj8_gfLFjEO+_KUBXZQ2VKjrFzszwcY-ww@mail.gmail.com>
 <CAHbf0-GDmbx3xO+Kmb8y8X-41TxnaetQ=k4FR-NonMZoL-U2UA@mail.gmail.com>
Message-ID: <CAHbf0-GWxOd2E+yPu9L_EXYHfJ_7C02=ybXYwkHnCw_6dPzj4A@mail.gmail.com>

Hi

I just thought I'd pop in an update

I've just managed to fix this by changing my PCIe options in my kernel config

The problem showed up with:

[*] PCI support

      ? ?
 ? ?
           [*]   Support mmconfig PCI config space access

                 ? ?
 ? ?
           [*]   PCI Express Port Bus support

                 ? ?
 ? ?
           [*]     PCI Express Hotplug driver

                 ? ?
 ? ?
           [*]     Root Port Advanced Error Reporting support

                 ? ?
 ? ?
           [*]       PCI Express ECRC settings control

                 ? ?
 ? ?
           < >       PCIe AER error injector support

                 ? ?
 ? ?
           -*-     PCI Express ASPM control

                 ? ?
 ? ?
           [ ]       Debug PCI Express ASPM

                 ? ?
 ? ?
                     Default ASPM policy (BIOS default)  --->

                 ? ?
 ? ?
           [*]   Message Signaled Interrupts (MSI and MSI-X)

                 ? ?
 ? ?
           [ ]   PCI Debugging

                 ? ?
 ? ?
           [*]   Enable PCI resource re-allocation detection

                 ? ?
 ? ?
           < >   PCI Stub driver

                 ? ?
 ? ?
           [*]   Interrupts on hypertransport devices

                 ? ?
 ? ?
           [ ] PCI IOV support

                 ? ?
 ? ?
           [*] PCI PRI support

                 ? ?
 ? ?
           -*- PCI PASID support

                 ? ?
 ? ?
               PCI host controller drivers  ----

                 ? ?
 ? ?
           < > PCCard (PCMCIA/CardBus) support  ----

                 ? ?
 ? ?
           [*] Support for PCI Hotplug  --->

                 ? ?
 ? ?
           < > RapidIO support

                 ? ?


This is what I have now:

[*] PCI support

      ? ?
 ? ?
           [*]   Support mmconfig PCI config space access

                 ? ?
 ? ?
           [*]   PCI Express Port Bus support

                 ? ?
 ? ?
           [ ]     Root Port Advanced Error Reporting support

                 ? ?
 ? ?
           -*-     PCI Express ASPM control

                 ? ?
 ? ?
           [ ]       Debug PCI Express ASPM

                 ? ?
 ? ?
                     Default ASPM policy (BIOS default)  --->

                 ? ?
 ? ?
           [*]   Message Signaled Interrupts (MSI and MSI-X)

                 ? ?
 ? ?
           [*]   PCI Debugging

                 ? ?
 ? ?
           [ ]   Enable PCI resource re-allocation detection

                 ? ?
 ? ?
           < >   PCI Stub driver

                 ? ?
 ? ?
           [*]   Interrupts on hypertransport devices

                 ? ?
 ? ?
           [ ] PCI IOV support

                 ? ?
 ? ?
           [ ] PCI PRI support

                 ? ?
 ? ?
           [ ] PCI PASID support

                 ? ?
 ? ?
               PCI host controller drivers  ----

                 ? ?
 ? ?
           < > PCCard (PCMCIA/CardBus) support  ----

                 ? ?
 ? ?
           [ ] Support for PCI Hotplug  ----

                 ? ?
 ? ?
           < > RapidIO support

So I guess the problem probably lies either in the PCIe subsystem or
something it interacts with in the "bios"

On 25 January 2016 at 01:30, Mike Lothian <mike at fireburn.co.uk> wrote:
> Hi
>
> I managed to capture the dmesg by copying the dmesg and sync binaries
> to a usb disk and executing and writing to there after the suspend
>
> I hope this helps in some way and I'll happily raise a proper bug if
> you tell me which product / component I should use for nvme
>
> I also tried disabling IOMMU but that had no affect
>
> Cheers
>
> Mike
>
> On 23 January 2016 at 17:02, Mike Lothian <mike at fireburn.co.uk> wrote:
>> Hi
>>
>> I currently have a 3 partition NVME setup, however when I suspend and
>> resume my laptop the drive isn't re-initialised correctly, the console
>> is filled with access errors and I can't run any commands as they
>> can't be found. After hard rebooting my machine sometimes it can't
>> find the GPD partition and I have to use testdisk to restore the
>> partition table
>>
>> I have an Alienware 15 laptop with a 1TB NVME harddisk running kernel
>> 4.5-rc0. As I don't suspend my laptop often I'm not sure if this is a
>> regression brought in from the merge window or an existing issue
>>
>> I can bisect if needed
>>
>> 3e:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co
>> Ltd Device [144d:a802] (rev 01)
>>        Subsystem: Samsung Electronics Co Ltd Device a801
>>        Kernel driver in use: nvme
>>
>> [    1.050096]  nvme0n1: p1 p2 p3
>>
>> I was going to submit a bug on bugzilla.kernel.org but I wasn't sure
>> which Product / Component it should be filed under
>>
>> I haven't figured out a good way of collecting logs when the system
>> stops responding so please let me know if you know how I can collect
>> logs - I don't think SSH sessions survive suspend / resume cycles
>>
>> Thanks
>>
>> Mike


From snitzer at redhat.com  Sun Feb  7 08:53:40 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Sun, 7 Feb 2016 11:53:40 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B77444.3030106@dev.mellanox.co.il>
References: <20160203180406.GA11591@redhat.com>
 <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il>
Message-ID: <20160207165340.GA6298@redhat.com>

On Sun, Feb 07 2016 at 11:43am -0500,
Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:

> 
> >Hello Sagi,
> 
> Hey Bart,
> 
> >Did you run your test on a NUMA system ?
> 
> I did.
> 
> >If so, can you check with e.g.
> >perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
> >workload triggers perhaps lock contention ? What you need to look for in
> >the perf output is whether any functions occupy more than 10% CPU time.
> 
> I will, thanks for the tip!

Also, I found ftrace's function_graph tracer very helpful (it is how I
found the various issues fixed by the first context switch patch).  Here
is my latest script:

#!/bin/sh

set -xv

NULL_BLK_HW_QUEUES=4
NULL_BLK_QUEUE_DEPTH=4096

DM_MQ_HW_QUEUES=4
DM_MQ_QUEUE_DEPTH=2048

FIO=/root/snitm/git/fio/fio
FIO_QUEUE_DEPTH=32
FIO_RUNTIME=10
FIO_NUMJOBS=12

PERF=perf
#PERF=/root/snitm/git/linux/tools/perf/perf

run_fio() {
    DEVICE=$1
    TASK_NAME=$(basename ${DEVICE})
    PERF_RECORD=$2
    RUN_CMD="${FIO} --cpus_allowed_policy=split --group_reporting --rw=randread --bs=4k --numjobs=${FIO_NUMJOBS} \
              --iodepth=${FIO_QUEUE_DEPTH} --runtime=${FIO_RUNTIME} --time_based --loops=1 --ioengine=libaio \
              --direct=1 --invalidate=1 --randrepeat=1 --norandommap --exitall --name task_${TASK_NAME} --filename=${DEVICE}"

    if [ ! -z "${PERF_RECORD}" ]; then
    ${PERF_RECORD} ${RUN_CMD}
    mv perf.data perf.data.${TASK_NAME}
    else
    ${RUN_CMD}
    fi
}

run_fio_with_ftrace() {
    DEVICE=$1
    TASK_NAME=$(basename ${DEVICE})

    echo > /sys/kernel/debug/tracing/trace
    echo 0 > /sys/kernel/debug/tracing/tracing_on
    echo function_graph > /sys/kernel/debug/tracing/current_tracer
    echo 1 > /sys/kernel/debug/tracing/tracing_on
    run_fio $DEVICE
    echo 0 > /sys/kernel/debug/tracing/tracing_on
    cat /sys/kernel/debug/tracing/trace > trace.${TASK_NAME}
    echo nop > /sys/kernel/debug/tracing/current_tracer
}

dmsetup remove dm_mq
modprobe -r null_blk

modprobe null_blk gb=4 bs=512 hw_queue_depth=${NULL_BLK_QUEUE_DEPTH} nr_devices=1 queue_mode=2 irqmode=1 completion_nsec=1 submit_queues=${NULL_BLK_HW_QUEUES}
#run_fio /dev/nullb0 "${PERF} record -ag -e cs"
#run_fio /dev/nullb0 "${PERF} stat"

echo Y > /sys/module/dm_mod/parameters/use_blk_mq
echo ${DM_MQ_QUEUE_DEPTH} > /sys/module/dm_mod/parameters/blk_mq_queue_depth
echo ${DM_MQ_HW_QUEUES} > /sys/module/dm_mod/parameters/blk_mq_nr_hw_queues
echo "0 8388608 multipath 0 0 1 1 service-time 0 1 2 /dev/nullb0 1000 1" | dmsetup create dm_mq
#echo "0 8388608 linear /dev/nullb0 0" | dmsetup create dm_mq

run_fio_with_ftrace /dev/mapper/dm_mq

#run_fio /dev/mapper/dm_mq
#run_fio /dev/mapper/dm_mq "${PERF} record -ag -e cs"
#run_fio /dev/mapper/dm_mq "${PERF} record -ag"
#run_fio /dev/mapper/dm_mq "${PERF} stat"

#run_fio /dev/mapper/dm_mq "trace-cmd record -e all"


From sagig at dev.mellanox.co.il  Sun Feb  7 08:54:54 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 7 Feb 2016 18:54:54 +0200
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B77444.3030106@dev.mellanox.co.il>
References: <20160129233504.GA13661@redhat.com> <56AC79D0.5060104@suse.de>
 <20160130191238.GA18686@redhat.com> <56AEFF63.7050606@suse.de>
 <20160203180406.GA11591@redhat.com> <20160203182423.GA12913@redhat.com>
 <56B2F5BC.1010700@suse.de> <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com> <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com> <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com> <56B77444.3030106@dev.mellanox.co.il>
Message-ID: <56B776DE.30101@dev.mellanox.co.il>


>> If so, can you check with e.g.
>> perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
>> workload triggers perhaps lock contention ? What you need to look for in
>> the perf output is whether any functions occupy more than 10% CPU time.
>
> I will, thanks for the tip!

The perf report is very similar to the one that started this effort..

I'm afraid we'll need to resolve the per-target m->lock in order
to scale with NUMA...

-  17.33%              fio  [kernel.kallsyms]        [k] 
queued_spin_lock_slowpath
    - queued_spin_lock_slowpath
       - 52.09% _raw_spin_lock_irq
            __multipath_map
            multipath_clone_and_map
            map_request
            dm_mq_queue_rq
            __blk_mq_run_hw_queue
            blk_mq_run_hw_queue
            blk_mq_insert_requests
            blk_mq_flush_plug_list
            blk_flush_plug_list
            blk_finish_plug
            do_io_submit
            SyS_io_submit
            entry_SYSCALL_64_fastpath
          + io_submit
       - 46.87% _raw_spin_lock_irqsave
          - 99.97% multipath_busy
               dm_mq_queue_rq
               __blk_mq_run_hw_queue
               blk_mq_run_hw_queue
               blk_mq_insert_requests
               blk_mq_flush_plug_list
               blk_flush_plug_list
               blk_finish_plug
               do_io_submit
               SyS_io_submit
               entry_SYSCALL_64_fastpath
             + io_submit
+   4.99%              fio  [kernel.kallsyms]        [k] 
blk_account_io_start
+   3.93%              fio  [dm_multipath]           [k] __multipath_map
+   2.64%              fio  [dm_multipath]           [k] multipath_busy
+   2.38%              fio  [kernel.kallsyms]        [k] 
_raw_spin_lock_irqsave
+   2.31%              fio  [dm_mod]                 [k] dm_mq_queue_rq
+   2.25%              fio  [kernel.kallsyms]        [k] 
blk_mq_hctx_mark_pending
+   1.81%              fio  [kernel.kallsyms]        [k] blk_queue_enter
+   1.61%             perf  [kernel.kallsyms]        [k] 
copy_user_generic_string
+   1.40%              fio  [kernel.kallsyms]        [k] 
__blk_mq_run_hw_queue
+   1.26%              fio  [kernel.kallsyms]        [k] part_round_stats
+   1.14%              fio  [kernel.kallsyms]        [k] _raw_spin_lock_irq
+   0.96%              fio  [kernel.kallsyms]        [k] __bt_get
+   0.73%              fio  [kernel.kallsyms]        [k] enqueue_task_fair
+   0.71%              fio  [kernel.kallsyms]        [k] enqueue_entity
+   0.69%              fio  [dm_mod]                 [k] dm_start_request
+   0.60%      ksoftirqd/6  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   0.59%     ksoftirqd/10  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   0.59%              fio  [kernel.kallsyms]        [k] 
_raw_spin_unlock_irqrestore
+   0.58%     ksoftirqd/19  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   0.58%     ksoftirqd/18  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   0.58%     ksoftirqd/23  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues



From snitzer at redhat.com  Sun Feb  7 09:20:56 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Sun, 7 Feb 2016 12:20:56 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B776DE.30101@dev.mellanox.co.il>
References: <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il>
Message-ID: <20160207172055.GA6477@redhat.com>

On Sun, Feb 07 2016 at 11:54am -0500,
Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:

> 
> >>If so, can you check with e.g.
> >>perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
> >>workload triggers perhaps lock contention ? What you need to look for in
> >>the perf output is whether any functions occupy more than 10% CPU time.
> >
> >I will, thanks for the tip!
> 
> The perf report is very similar to the one that started this effort..
> 
> I'm afraid we'll need to resolve the per-target m->lock in order
> to scale with NUMA...

Could be.  Just for testing, you can try the 2 topmost commits I've put
here (once applied both __multipath_map and multipath_busy won't have
_any_ locking.. again, very much test-only):

http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel2


From axboe at kernel.dk  Sun Feb  7 14:28:55 2016
From: axboe at kernel.dk (Jens Axboe)
Date: Sun, 7 Feb 2016 15:28:55 -0700
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <1454861040.2329.4.camel@HansenPartnership.com>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com>
Message-ID: <56B7C527.6050300@kernel.dk>

On 02/07/2016 09:04 AM, James Bottomley wrote:
> On Sun, 2016-02-07 at 10:22 +0100, Christoph Hellwig wrote:
>> Keith said it should be on by default, and I promised him to change
>> it once we run into problems, which I guess this counts as.
>>
>> But just curious:  what distro are you using?  Upstream systemd
>> explicitly rejected using scsi_id for NVMe here:
>>
>> 	https://github.com/systemd/systemd/issues/1453
>>
>> and all my test systems don't do this either.
>
> This was SUSE (in my case, openSUSE Leap).  I just checked the source
> package; they patch the by-id rules back in for NVME:
>
> # PATCH-FIX-SUSE 1101-rules-persistent-device-names-for-NVMe-devices.patch (bsc#944132)
> Patch1101:      1101-rules-persistent-device-names-for-NVMe-devices.patch
>
> The bugzilla is giving access denied for bug id 944132, so it's likely
> some proprietary vendor problem.  The patch has no preamble, so it's
> hard to tell what they were thinking.

I run root-on-nvme on my laptop, and I haven't observed any problems. 
Generally I hate for options to default y unless absolutely necessary, 
it's a sure fire way to feature creep your kernel without noticing. I 
don't think getting all hot about this issue is fair, if the only known 
case is suse.

If anything, let's make the description better. It's trying to be funny, 
it'd be better if it was descriptive and covered this case as well.

-- 
Jens Axboe



From James.Bottomley at HansenPartnership.com  Sun Feb  7 15:07:21 2016
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Sun, 07 Feb 2016 15:07:21 -0800
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <56B7C527.6050300@kernel.dk>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com>
 <56B7C527.6050300@kernel.dk>
Message-ID: <1454886441.2329.27.camel@HansenPartnership.com>

On Sun, 2016-02-07 at 15:28 -0700, Jens Axboe wrote:
> On 02/07/2016 09:04 AM, James Bottomley wrote:
> > On Sun, 2016-02-07 at 10:22 +0100, Christoph Hellwig wrote:
> > > Keith said it should be on by default, and I promised him to
> > > change
> > > it once we run into problems, which I guess this counts as.
> > > 
> > > But just curious:  what distro are you using?  Upstream systemd
> > > explicitly rejected using scsi_id for NVMe here:
> > > 
> > > 	https://github.com/systemd/systemd/issues/1453
> > > 
> > > and all my test systems don't do this either.
> > 
> > This was SUSE (in my case, openSUSE Leap).  I just checked the
> > source
> > package; they patch the by-id rules back in for NVME:
> > 
> > # PATCH-FIX-SUSE 1101-rules-persistent-device-names-for-NVMe
> > -devices.patch (bsc#944132)
> > Patch1101:      1101-rules-persistent-device-names-for-NVMe
> > -devices.patch
> > 
> > The bugzilla is giving access denied for bug id 944132, so it's
> > likely
> > some proprietary vendor problem.  The patch has no preamble, so
> > it's
> > hard to tell what they were thinking.
> 
> I run root-on-nvme on my laptop, and I haven't observed any problems.

Me too apparently.  It looks like this problem may be SUSE specific
unless another distro has enabled this.  I can see why they would: you
do need persistent names for devices, even NVMe ones.

> Generally I hate for options to default y unless absolutely 
> necessary, it's a sure fire way to feature creep your kernel without 
> noticing. I don't think getting all hot about this issue is fair, if 
> the only known case is suse.

Well, OK, I'm annoyed because it was a systemd system which means
debugging boot failures are excruciatingly difficult so it took me a
week and a half to find out what the problem was.  Perhaps I was a bit
rash to label this as an easily foreseen problem.

I opened a bug against SUSE to tell them to turn it on:

https://bugzilla.opensuse.org/show_bug.cgi?id=965497

The second problem is that there's currently no way to transition to
using the serial attribute the way the udev 60-persistent-storage.rules
are written, so if distros have some by-id hack, it will have to be
maintained for a while.  I annotated the already closed bug on this in
systemd with the rules that work for me.

> If anything, let's make the description better. It's trying to be
> funny, it'd be better if it was descriptive and covered this case as
> well.

The problem with this is that when moving to new kernels, distro
maintainers don't read the new option help texts, they just take the
defaults.  However, I checked the only other distribution I use
(debian) and they don't have a nvme persistent ID hack, so if someone
checked ubuntu and Red Hat, I think all the majors are now covered and
perhaps there's no need to do anything more.

James



From hch at lst.de  Sun Feb  7 23:24:01 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 8 Feb 2016 08:24:01 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <1454886441.2329.27.camel@HansenPartnership.com>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com>
Message-ID: <20160208072401.GA30007@lst.de>

On Sun, Feb 07, 2016 at 03:07:21PM -0800, James Bottomley wrote:
> > I run root-on-nvme on my laptop, and I haven't observed any problems.
> 
> Me too apparently.  It looks like this problem may be SUSE specific
> unless another distro has enabled this.  I can see why they would: you
> do need persistent names for devices, even NVMe ones.

I don't have root on nvme, just my xfstests device, but I still didn't
see the problem, neither did my various nvme development setups.

> I opened a bug against SUSE to tell them to turn it on:
> 
> https://bugzilla.opensuse.org/show_bug.cgi?id=965497
> 
> The second problem is that there's currently no way to transition to
> using the serial attribute the way the udev 60-persistent-storage.rules
> are written, so if distros have some by-id hack, it will have to be
> maintained for a while.  I annotated the already closed bug on this in
> systemd with the rules that work for me.

We now expose the NVMe serial and NGUI, out of which the evpd page is
mangled depending on the NVMe spec version that the device supports as
sysfs attributes, distros can do the same mangling if they want to
support their old ids.


From hare at suse.de  Sun Feb  7 23:26:25 2016
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 8 Feb 2016 08:26:25 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <1454861040.2329.4.camel@HansenPartnership.com>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com>
Message-ID: <56B84321.3010308@suse.de>

On 02/07/2016 05:04 PM, James Bottomley wrote:
> On Sun, 2016-02-07 at 10:22 +0100, Christoph Hellwig wrote:
>> Keith said it should be on by default, and I promised him to change
>> it once we run into problems, which I guess this counts as.
>>
>> But just curious:  what distro are you using?  Upstream systemd
>> explicitly rejected using scsi_id for NVMe here:
>>
>> 	https://github.com/systemd/systemd/issues/1453
>>
>> and all my test systems don't do this either.
> 
> This was SUSE (in my case, openSUSE Leap).  I just checked the source
> package; they patch the by-id rules back in for NVME:
> 
> # PATCH-FIX-SUSE 1101-rules-persistent-device-names-for-NVMe-devices.patch (bsc#944132)
> Patch1101:      1101-rules-persistent-device-names-for-NVMe-devices.patch
> 
> The bugzilla is giving access denied for bug id 944132, so it's likely
> some proprietary vendor problem.  The patch has no preamble, so it's
> hard to tell what they were thinking.
> 
They didn't think at all. That abovementioned bug just states 'by-id
symlinks for NVMe drives are missing'.
And they fixed it by add the respective rules (using sg_inq) to udev.

There's no mentioning of any NVMe specific sysfs attributes whatsoever.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From hare at suse.de  Sun Feb  7 23:32:26 2016
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 8 Feb 2016 08:32:26 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <1454886441.2329.27.camel@HansenPartnership.com>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com>
Message-ID: <56B8448A.3000106@suse.de>

On 02/08/2016 12:07 AM, James Bottomley wrote:
> On Sun, 2016-02-07 at 15:28 -0700, Jens Axboe wrote:
>> On 02/07/2016 09:04 AM, James Bottomley wrote:
>>> On Sun, 2016-02-07 at 10:22 +0100, Christoph Hellwig wrote:
>>>> Keith said it should be on by default, and I promised him to
>>>> change
>>>> it once we run into problems, which I guess this counts as.
>>>>
>>>> But just curious:  what distro are you using?  Upstream systemd
>>>> explicitly rejected using scsi_id for NVMe here:
>>>>
>>>> 	https://github.com/systemd/systemd/issues/1453
>>>>
>>>> and all my test systems don't do this either.
>>>
>>> This was SUSE (in my case, openSUSE Leap).  I just checked the
>>> source
>>> package; they patch the by-id rules back in for NVME:
>>>
>>> # PATCH-FIX-SUSE 1101-rules-persistent-device-names-for-NVMe
>>> -devices.patch (bsc#944132)
>>> Patch1101:      1101-rules-persistent-device-names-for-NVMe
>>> -devices.patch
>>>
>>> The bugzilla is giving access denied for bug id 944132, so it's
>>> likely
>>> some proprietary vendor problem.  The patch has no preamble, so
>>> it's
>>> hard to tell what they were thinking.
>>
>> I run root-on-nvme on my laptop, and I haven't observed any problems.
> 
> Me too apparently.  It looks like this problem may be SUSE specific
> unless another distro has enabled this.  I can see why they would: you
> do need persistent names for devices, even NVMe ones.
> 
>> Generally I hate for options to default y unless absolutely 
>> necessary, it's a sure fire way to feature creep your kernel without 
>> noticing. I don't think getting all hot about this issue is fair, if 
>> the only known case is suse.
> 
> Well, OK, I'm annoyed because it was a systemd system which means
> debugging boot failures are excruciatingly difficult so it took me a
> week and a half to find out what the problem was.  Perhaps I was a bit
> rash to label this as an easily foreseen problem.
> 
> I opened a bug against SUSE to tell them to turn it on:
> 
> https://bugzilla.opensuse.org/show_bug.cgi?id=965497
> 
> The second problem is that there's currently no way to transition to
> using the serial attribute the way the udev 60-persistent-storage.rules
> are written, so if distros have some by-id hack, it will have to be
> maintained for a while.  I annotated the already closed bug on this in
> systemd with the rules that work for me.
> 
Why, but you can.
That's precisely what I did with the transition to sg_inq; I've
added a new set of rules (55-sg_inq.rules and 59-sg-symlinks.rules)
which will override the values from 60-persistent-storage.rules.

Do we have defined sysfs attributes for NVMe devices nowadays?
If so I'd be willing to create/send some sysfs rules for them.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From dmonakhov at openvz.org  Mon Feb  8 01:00:22 2016
From: dmonakhov at openvz.org (Dmitry Monakhov)
Date: Mon, 08 Feb 2016 12:00:22 +0300
Subject: [LSF/MM ATTEND] ext4, DAX pmem/fsync
Message-ID: <87lh6vwp95.fsf@openvz.org>



I would like to attend LSF/MM 2016 to participate in discussions around
the optimization of the block and MM for low latency NVM technologies.
Primary target of interests is to improving DAX performance for general
purpose FS.
While working on LightVM project (approach lighter that traditional VM, but more
secure that containers) we found that existing DAX infrastructure is too
far from optimum:
0) DAX msync/fsync patch-set is has a lot in TODO list
1) DAX implementation has number of significant SMP scalability issues.

I would like to attend LSF/MM this year. I believe I could contribute to the
following discussions:

*  Block & Filesystem interface from Steven Whitehouse
http://www.spinics.net/lists/linux-fsdevel/msg93552.html

* Persistent memory: pmem as storage device vs pmem as memory from Dan Williams
https://lists.01.org/pipermail/linux-nvdimm/2016-February/004331.html

* Persistent Memory Error Handling from Jeff Moyer:
http://www.spinics.net/lists/linux-mm/msg100560.html

* FS block allocator modification for SMR Zoned Block Devices

* EXT4-mini sumit
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 472 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20160208/74eaf1a4/attachment-0001.sig>

From sagig at dev.mellanox.co.il  Mon Feb  8 02:01:16 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 8 Feb 2016 12:01:16 +0200
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <56B8448A.3000106@suse.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com> <56B8448A.3000106@suse.de>
Message-ID: <56B8676C.20600@dev.mellanox.co.il>


> Do we have defined sysfs attributes for NVMe devices nowadays?

/sys/block/nvme0n1/uuid

> If so I'd be willing to create/send some sysfs rules for them.

That'd be great!


From hch at lst.de  Mon Feb  8 02:13:50 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 8 Feb 2016 11:13:50 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <56B8676C.20600@dev.mellanox.co.il>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com> <56B8448A.3000106@suse.de>
 <56B8676C.20600@dev.mellanox.co.il>
Message-ID: <20160208101350.GA32632@lst.de>

On Mon, Feb 08, 2016 at 12:01:16PM +0200, Sagi Grimberg wrote:
>
>> Do we have defined sysfs attributes for NVMe devices nowadays?
>
> /sys/block/nvme0n1/uuid

That's only supported for NVMe 1.1 and higher devices, and optional.
For older or stupid devices we need to support the algorithm based
on the serial attribute from nvme_fill_device_id_scsi_string() in
drivers/nvme/host/scsi.c.

Which btw is ?ncorrect as it doesn't identify namespaces properly,
and thus the associatation of 0 is an actively harmful lie.


From sagig at dev.mellanox.co.il  Mon Feb  8 04:21:59 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 8 Feb 2016 14:21:59 +0200
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160207172055.GA6477@redhat.com>
References: <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com> <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com> <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il> <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il> <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com>
Message-ID: <56B88867.4060308@dev.mellanox.co.il>


>> The perf report is very similar to the one that started this effort..
>>
>> I'm afraid we'll need to resolve the per-target m->lock in order
>> to scale with NUMA...
>
> Could be.  Just for testing, you can try the 2 topmost commits I've put
> here (once applied both __multipath_map and multipath_busy won't have
> _any_ locking.. again, very much test-only):
>
> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel2

Hi Mike,

So I still don't see the IOPs scale like I expected. With these two
patches applied I see ~670K IOPs while the perf output is different
and does not indicate a clear lock contention.

--
-   4.67%              fio  [kernel.kallsyms]        [k] 
blk_account_io_start
    - blk_account_io_start
       - 56.05% blk_insert_cloned_request
            map_request
            dm_mq_queue_rq
            __blk_mq_run_hw_queue
            blk_mq_run_hw_queue
            blk_mq_insert_requests
            blk_mq_flush_plug_list
            blk_flush_plug_list
            blk_finish_plug
            do_io_submit
            SyS_io_submit
            entry_SYSCALL_64_fastpath
          + io_submit
       - 43.94% blk_mq_bio_to_request
            blk_mq_make_request
            generic_make_request
            submit_bio
            do_blockdev_direct_IO
            __blockdev_direct_IO
            blkdev_direct_IO
            generic_file_read_iter
            blkdev_read_iter
            aio_run_iocb
            io_submit_one
            do_io_submit
            SyS_io_submit
            entry_SYSCALL_64_fastpath
          + io_submit
-   2.52%              fio  [dm_mod]                 [k] dm_mq_queue_rq
    - dm_mq_queue_rq
       - 99.16% __blk_mq_run_hw_queue
            blk_mq_run_hw_queue
            blk_mq_insert_requests
            blk_mq_flush_plug_list
            blk_flush_plug_list
            blk_finish_plug
            do_io_submit
            SyS_io_submit
            entry_SYSCALL_64_fastpath
          + io_submit
-   2.52%              fio  [dm_mod]                 [k] dm_mq_queue_rq
    - dm_mq_queue_rq
       - 99.16% __blk_mq_run_hw_queue
            blk_mq_run_hw_queue
            blk_mq_insert_requests
            blk_mq_flush_plug_list
            blk_flush_plug_list
            blk_finish_plug
            do_io_submit
            SyS_io_submit
            entry_SYSCALL_64_fastpath
          + io_submit
       + 0.84% blk_mq_run_hw_queue
-   2.46%              fio  [kernel.kallsyms]        [k] 
blk_mq_hctx_mark_pending
    - blk_mq_hctx_mark_pending
       - 99.79% blk_mq_insert_requests
            blk_mq_flush_plug_list
            blk_flush_plug_list
            blk_finish_plug
            do_io_submit
            SyS_io_submit
            entry_SYSCALL_64_fastpath
          + io_submit
-   2.07%      ksoftirqd/6  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
    - blk_mq_run_hw_queues
       - 99.70% rq_completed
            dm_done
            dm_softirq_done
            blk_done_softirq
          + __do_softirq
+   2.06%      ksoftirqd/0  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   2.02%      ksoftirqd/9  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   2.00%     ksoftirqd/20  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   2.00%     ksoftirqd/12  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.99%     ksoftirqd/11  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.97%     ksoftirqd/18  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.96%      ksoftirqd/1  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.95%     ksoftirqd/14  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.95%     ksoftirqd/13  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.94%      ksoftirqd/5  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.94%      ksoftirqd/8  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.93%      ksoftirqd/2  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.92%     ksoftirqd/21  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.92%     ksoftirqd/17  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.92%      ksoftirqd/7  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.91%     ksoftirqd/23  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.84%      ksoftirqd/4  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.81%     ksoftirqd/19  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.76%      ksoftirqd/3  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.76%     ksoftirqd/16  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.75%     ksoftirqd/15  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.74%     ksoftirqd/22  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.72%     ksoftirqd/10  [kernel.kallsyms]        [k] 
blk_mq_run_hw_queues
+   1.38%             perf  [kernel.kallsyms]        [k] 
copy_user_generic_string
+   1.20%              fio  [kernel.kallsyms]        [k] enqueue_task_fair
+   1.18%              fio  [kernel.kallsyms]        [k] part_round_stats
+   1.08%              fio  [kernel.kallsyms]        [k] enqueue_entity
+   1.07%              fio  [kernel.kallsyms]        [k] _raw_spin_lock
+   1.02%              fio  [kernel.kallsyms]        [k] 
__blk_mq_run_hw_queue
+   0.79%              fio  [dm_multipath]           [k] multipath_busy
+   0.57%              fio  [kernel.kallsyms]        [k] insert_work
+   0.54%              fio  [kernel.kallsyms]        [k] blk_flush_plug_list
--


From bhelgaas at google.com  Mon Feb  8 05:51:13 2016
From: bhelgaas at google.com (Bjorn Helgaas)
Date: Mon, 8 Feb 2016 07:51:13 -0600
Subject: [Bug 112121] New: Some PCIe options cause devices to be removed
 after syspend
In-Reply-To: <bug-112121-41252@https.bugzilla.kernel.org/>
References: <bug-112121-41252@https.bugzilla.kernel.org/>
Message-ID: <CAErSpo4beDk_ajsgqMUNJ3TGxDv=ysyPzhXHcx2AhUEUE9ROKA@mail.gmail.com>

[+cc linux-pci, NVMe folks,  power management folks]

On Sun, Feb 7, 2016 at 11:04 AM,  <bugzilla-daemon at bugzilla.kernel.org> wrote:
> https://bugzilla.kernel.org/show_bug.cgi?id=112121
>
>             Bug ID: 112121
>            Summary: Some PCIe options cause devices to be removed after
>                     syspend
>            Product: Drivers
>            Version: 2.5
>     Kernel Version: 4.5-rc2
>           Hardware: All
>                 OS: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: normal
>           Priority: P1
>          Component: PCI
>           Assignee: drivers_pci at kernel-bugs.osdl.org
>           Reporter: mike at fireburn.co.uk
>         Regression: No
>
> Created attachment 203091
>   --> https://bugzilla.kernel.org/attachment.cgi?id=203091&action=edit
> Dmesg showing PCIe device removals
>
> I was having issues with suspend, when the machine was being resumed iommu
> started removing devices - including my PCIe NVMe drive which contained my root
> partition
>
> The problem showed up with:
>
> [*] PCI support
> [*]   Support mmconfig PCI config space access
> [*]   PCI Express Port Bus support
> [*]     PCI Express Hotplug driver
> [*]     Root Port Advanced Error Reporting support
> [*]       PCI Express ECRC settings control
> < >       PCIe AER error injector support
> -*-     PCI Express ASPM control
> [ ]       Debug PCI Express ASPM
>           Default ASPM policy (BIOS default)  --->
> [*]   Message Signaled Interrupts (MSI and MSI-X)
> [ ]   PCI Debugging
> [*]   Enable PCI resource re-allocation detection
> < >   PCI Stub driver
> [*]   Interrupts on hypertransport devices
> [ ] PCI IOV support
> [*] PCI PRI support
> -*- PCI PASID support
>     PCI host controller drivers  ----
> < > PCCard (PCMCIA/CardBus) support  ----
> [*] Support for PCI Hotplug  --->
> < > RapidIO support
>
>
> This is what I have now:
>
> [*] PCI support
> [*]   Support mmconfig PCI config space access
> [*]   PCI Express Port Bus support
> [ ]     Root Port Advanced Error Reporting support
> -*-     PCI Express ASPM control
> [ ]       Debug PCI Express ASPM
>           Default ASPM policy (BIOS default)  --->
> [*]   Message Signaled Interrupts (MSI and MSI-X)
> [*]   PCI Debugging
> [ ]   Enable PCI resource re-allocation detection
> < >   PCI Stub driver
> [*]   Interrupts on hypertransport devices
> [ ] PCI IOV support
> [ ] PCI PRI support
> [ ] PCI PASID support
>     PCI host controller drivers  ----
> < > PCCard (PCMCIA/CardBus) support  ----
> [ ] Support for PCI Hotplug  ----
> < > RapidIO support
>
> I tried disabling the iommu driver first but it had no effect
>
> If people are interested I could play with the above options to see which one
> causes the issue

My guess is that PCI hotplug is the important one.  It would be nice
if dmesg contained enough information to connect nvme0n1 to a PCI
device.  It'd be even nicer if the PCI core noted device removals or
whatever happened here.

You don't get any more details if you boot with "ignore_loglevel", do you?

Mike, you didn't mark this as a regression, so I assume it's always
been this way, and we just haven't noticed it because most people
enable PCI hotplug (or whatever the relevant config option is).

Bjorn


From snitzer at redhat.com  Mon Feb  8 06:34:00 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Mon, 8 Feb 2016 09:34:00 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B88867.4060308@dev.mellanox.co.il>
References: <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com>
 <56B88867.4060308@dev.mellanox.co.il>
Message-ID: <20160208143359.GA12132@redhat.com>

On Mon, Feb 08 2016 at  7:21am -0500,
Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:

> 
> >>The perf report is very similar to the one that started this effort..
> >>
> >>I'm afraid we'll need to resolve the per-target m->lock in order
> >>to scale with NUMA...
> >
> >Could be.  Just for testing, you can try the 2 topmost commits I've put
> >here (once applied both __multipath_map and multipath_busy won't have
> >_any_ locking.. again, very much test-only):
> >
> >http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel2
> 
> Hi Mike,
> 
> So I still don't see the IOPs scale like I expected. With these two
> patches applied I see ~670K IOPs while the perf output is different
> and does not indicate a clear lock contention.

Right, perf (with default events) isn't the right tool to track this down.

But I'm seeing something that speaks to you not running with the first
context switching fix (which seems odd):

> -   2.07%      ksoftirqd/6  [kernel.kallsyms]        [k]
> blk_mq_run_hw_queues
>    - blk_mq_run_hw_queues
>       - 99.70% rq_completed
>            dm_done
>            dm_softirq_done
>            blk_done_softirq
>          + __do_softirq

As you can see here:
https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=for-next&id=a5b835282422ec41991c1dbdb88daa4af7d166d2

rq_completed() shouldn't be calling blk_mq_run_hw_queues() with the
latest code.

Please triple check you have the latest code, e.g.:
git diff snitzer/devel2


From keith.busch at intel.com  Mon Feb  8 07:01:33 2016
From: keith.busch at intel.com (Keith Busch)
Date: Mon, 8 Feb 2016 15:01:33 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <BJXPR01MB19916231021454BD8731EBDE0D30@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203144123.GB23910@localhost.localdomain>
 <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203163808.GC23910@localhost.localdomain>
 <BJXPR01MB19916231021454BD8731EBDE0D30@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160208150132.GA25111@localhost.localdomain>

On Sat, Feb 06, 2016 at 02:32:24PM +0000, Wenbo Wang wrote:
> Keith,
> 
> Is the following solution OK?
> synchronize_rcu guarantee that no queue_rq is running concurrently with device disable code. Together with your another patch (adding blk_sync_queue), both sync/async path shall be handled correctly.
> 
> Do you think synchronize_rcu shall be added to blk_sync_queue?

I was nearly going to suggest the same last week, but it feels wrong since
no one takes rcu_read_lock in the path we're trying to sychronoize. Is
this safe if the task is interrupted?


From keith.busch at intel.com  Mon Feb  8 07:12:58 2016
From: keith.busch at intel.com (Keith Busch)
Date: Mon, 8 Feb 2016 15:12:58 +0000
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <20160208101350.GA32632@lst.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com>
 <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com>
 <56B8448A.3000106@suse.de> <56B8676C.20600@dev.mellanox.co.il>
 <20160208101350.GA32632@lst.de>
Message-ID: <20160208151258.GB25111@localhost.localdomain>

On Mon, Feb 08, 2016 at 11:13:50AM +0100, Christoph Hellwig wrote:
> On Mon, Feb 08, 2016 at 12:01:16PM +0200, Sagi Grimberg wrote:
> >
> >> Do we have defined sysfs attributes for NVMe devices nowadays?
> >
> > /sys/block/nvme0n1/uuid
> 
> That's only supported for NVMe 1.1 and higher devices, and optional.
> For older or stupid devices we need to support the algorithm based
> on the serial attribute from nvme_fill_device_id_scsi_string() in
> drivers/nvme/host/scsi.c.

It's even worse. NGUID was defined for 1.2 devices and higher. 1.1
devices should have EUI-64 at:
 
  /sys/block/nvmeXnY/eui

1.2 devices will have either uuid or eui (or both).

The majority of devices in circulation today are 1.0, and need to concat
these three entries to make a unique identifier:

  /sys/block/nvmeXnY/device/serial
  /sys/block/nvmeXnY/device/model
  /sys/block/nvmeXnY/nsid


From hare at suse.de  Mon Feb  8 07:19:13 2016
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 8 Feb 2016 16:19:13 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <20160208151258.GB25111@localhost.localdomain>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com> <56B8448A.3000106@suse.de>
 <56B8676C.20600@dev.mellanox.co.il> <20160208101350.GA32632@lst.de>
 <20160208151258.GB25111@localhost.localdomain>
Message-ID: <56B8B1F1.4070605@suse.de>

On 02/08/2016 04:12 PM, Keith Busch wrote:
> On Mon, Feb 08, 2016 at 11:13:50AM +0100, Christoph Hellwig wrote:
>> On Mon, Feb 08, 2016 at 12:01:16PM +0200, Sagi Grimberg wrote:
>>>
>>>> Do we have defined sysfs attributes for NVMe devices nowadays?
>>>
>>> /sys/block/nvme0n1/uuid
>>
>> That's only supported for NVMe 1.1 and higher devices, and optional.
>> For older or stupid devices we need to support the algorithm based
>> on the serial attribute from nvme_fill_device_id_scsi_string() in
>> drivers/nvme/host/scsi.c.
> 
> It's even worse. NGUID was defined for 1.2 devices and higher. 1.1
> devices should have EUI-64 at:
>  
>   /sys/block/nvmeXnY/eui
> 
> 1.2 devices will have either uuid or eui (or both).
> 
> The majority of devices in circulation today are 1.0, and need to concat
> these three entries to make a unique identifier:
> 
>   /sys/block/nvmeXnY/device/serial
>   /sys/block/nvmeXnY/device/model
>   /sys/block/nvmeXnY/nsid

Ok, so what about having a 'wwid' attribute which provides combined
information (like scsi has)?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From James.Bottomley at HansenPartnership.com  Mon Feb  8 07:23:14 2016
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Mon, 08 Feb 2016 07:23:14 -0800
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <56B8448A.3000106@suse.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com>
 <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com> <56B8448A.3000106@suse.de>
Message-ID: <1454944994.2339.1.camel@HansenPartnership.com>

On Mon, 2016-02-08 at 08:32 +0100, Hannes Reinecke wrote:
> On 02/08/2016 12:07 AM, James Bottomley wrote:
> > On Sun, 2016-02-07 at 15:28 -0700, Jens Axboe wrote:
> > > On 02/07/2016 09:04 AM, James Bottomley wrote:
> > > > On Sun, 2016-02-07 at 10:22 +0100, Christoph Hellwig wrote:
> > > > > Keith said it should be on by default, and I promised him to
> > > > > change
> > > > > it once we run into problems, which I guess this counts as.
> > > > > 
> > > > > But just curious:  what distro are you using?  Upstream
> > > > > systemd
> > > > > explicitly rejected using scsi_id for NVMe here:
> > > > > 
> > > > > 	https://github.com/systemd/systemd/issues/1453
> > > > > 
> > > > > and all my test systems don't do this either.
> > > > 
> > > > This was SUSE (in my case, openSUSE Leap).  I just checked the
> > > > source
> > > > package; they patch the by-id rules back in for NVME:
> > > > 
> > > > # PATCH-FIX-SUSE 1101-rules-persistent-device-names-for-NVMe
> > > > -devices.patch (bsc#944132)
> > > > Patch1101:      1101-rules-persistent-device-names-for-NVMe
> > > > -devices.patch
> > > > 
> > > > The bugzilla is giving access denied for bug id 944132, so it's
> > > > likely
> > > > some proprietary vendor problem.  The patch has no preamble, so
> > > > it's
> > > > hard to tell what they were thinking.
> > > 
> > > I run root-on-nvme on my laptop, and I haven't observed any
> > > problems.
> > 
> > Me too apparently.  It looks like this problem may be SUSE specific
> > unless another distro has enabled this.  I can see why they would:
> > you
> > do need persistent names for devices, even NVMe ones.
> > 
> > > Generally I hate for options to default y unless absolutely 
> > > necessary, it's a sure fire way to feature creep your kernel
> > > without 
> > > noticing. I don't think getting all hot about this issue is fair,
> > > if 
> > > the only known case is suse.
> > 
> > Well, OK, I'm annoyed because it was a systemd system which means
> > debugging boot failures are excruciatingly difficult so it took me
> > a
> > week and a half to find out what the problem was.  Perhaps I was a
> > bit
> > rash to label this as an easily foreseen problem.
> > 
> > I opened a bug against SUSE to tell them to turn it on:
> > 
> > https://bugzilla.opensuse.org/show_bug.cgi?id=965497
> > 
> > The second problem is that there's currently no way to transition
> > to
> > using the serial attribute the way the udev 60-persistent
> > -storage.rules
> > are written, so if distros have some by-id hack, it will have to be
> > maintained for a while.  I annotated the already closed bug on this
> > in
> > systemd with the rules that work for me.
> > 
> Why, but you can.
> That's precisely what I did with the transition to sg_inq; I've
> added a new set of rules (55-sg_inq.rules and 59-sg-symlinks.rules)
> which will override the values from 60-persistent-storage.rules.
> 
> Do we have defined sysfs attributes for NVMe devices nowadays?
> If so I'd be willing to create/send some sysfs rules for them.

This is the one I finally settled on (for 60-persistent-storage) for
SUSE.  It keeps everything working across old and new kernels.

# nvme
SUBSYSTEMS=="nvme", ATTRS{serial}=="?*", PROGRAM="/usr/bin/echo $attr{serial}", ENV{ID_SERIAL}="%c", ENV{ID_BUS}="nvme"

I already complained to Keith that the echo is required to strip the
leading whitespace which I don't think should be in the serial
attribute.

James



From keith.busch at intel.com  Mon Feb  8 08:15:59 2016
From: keith.busch at intel.com (Keith Busch)
Date: Mon, 8 Feb 2016 16:15:59 +0000
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <56B8B1F1.4070605@suse.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com>
 <56B7C527.6050300@kernel.dk>
 <1454886441.2329.27.camel@HansenPartnership.com>
 <56B8448A.3000106@suse.de> <56B8676C.20600@dev.mellanox.co.il>
 <20160208101350.GA32632@lst.de>
 <20160208151258.GB25111@localhost.localdomain>
 <56B8B1F1.4070605@suse.de>
Message-ID: <20160208161559.GC25111@localhost.localdomain>

On Mon, Feb 08, 2016 at 04:19:13PM +0100, Hannes Reinecke wrote:
> Ok, so what about having a 'wwid' attribute which provides combined
> information (like scsi has)?

That looks like the sensible thing to do. Thanks for pointer.

Going forward, I will solicite more feedback from scsi developers
so NVMe's external attributes better align with storage that already
solved our issues. I agree with Christoph that we never should have
relied on SCSI translations for NVMe, but we don't want to reinvent
generic solutions either.


From hch at lst.de  Mon Feb  8 10:07:36 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon,  8 Feb 2016 19:07:36 +0100
Subject: [PATCH] nvme: fix drvdata setup for the nvme device
Message-ID: <1454954856-32236-1-git-send-email-hch@lst.de>

See http://www.infradead.org/rpr.html

Pass the right private data to device_create_with_groups from the
beginning, and remove the superflous call to dev_set_drvdata.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/core.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c5bf001..c326931 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1383,14 +1383,13 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 
 	ctrl->device = device_create_with_groups(nvme_class, ctrl->dev,
 				MKDEV(nvme_char_major, ctrl->instance),
-				dev, nvme_dev_attr_groups,
+				ctrl, nvme_dev_attr_groups,
 				"nvme%d", ctrl->instance);
 	if (IS_ERR(ctrl->device)) {
 		ret = PTR_ERR(ctrl->device);
 		goto out_release_instance;
 	}
 	get_device(ctrl->device);
-	dev_set_drvdata(ctrl->device, ctrl);
 
 	spin_lock(&dev_list_lock);
 	list_add_tail(&ctrl->node, &nvme_ctrl_list);
-- 
2.1.4



From hch at lst.de  Mon Feb  8 10:09:47 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon,  8 Feb 2016 19:09:47 +0100
Subject: nvme: return the whole CQE through the request passthrough interface
Message-ID: <1454954988-32306-1-git-send-email-hch@lst.de>

See http://www.infradead.org/rpr.html

For NVMe over Fabrics we need the whole completion queue available
through the request interface in some cases, and Matias mentioned could
use this for LighNVM as well.

The attached patch uses req->special to allow passing a whole CQE around
that the low level driver can fill out.



From hch at lst.de  Mon Feb  8 10:09:48 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon,  8 Feb 2016 19:09:48 +0100
Subject: [PATCH] nvme: return the whole CQE through the request passthrough
 interface
In-Reply-To: <1454954988-32306-1-git-send-email-hch@lst.de>
References: <1454954988-32306-1-git-send-email-hch@lst.de>
Message-ID: <1454954988-32306-2-git-send-email-hch@lst.de>

See http://www.infradead.org/rpr.html

Both LighNVM and NVMe over Fabrics need to look at more than just the
status and result field.

Signed-off-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Jay Freyensee <james.p.freyensee at intel.com>
Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
---
 drivers/nvme/host/core.c | 27 +++++++++++++++++++--------
 drivers/nvme/host/nvme.h |  3 ++-
 drivers/nvme/host/pci.c  | 10 +++-------
 3 files changed, 24 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c326931..6715bbf 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -107,7 +107,6 @@ struct request *nvme_alloc_request(struct request_queue *q,
 
 	req->cmd = (unsigned char *)cmd;
 	req->cmd_len = sizeof(struct nvme_command);
-	req->special = (void *)0;
 
 	return req;
 }
@@ -117,7 +116,8 @@ struct request *nvme_alloc_request(struct request_queue *q,
  * if the result is positive, it's an NVM Express status code
  */
 int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
-		void *buffer, unsigned bufflen, u32 *result, unsigned timeout)
+		struct nvme_completion *cqe, void *buffer, unsigned bufflen,
+		unsigned timeout)
 {
 	struct request *req;
 	int ret;
@@ -127,6 +127,7 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		return PTR_ERR(req);
 
 	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	req->special = cqe;
 
 	if (buffer && bufflen) {
 		ret = blk_rq_map_kern(q, req, buffer, bufflen, GFP_KERNEL);
@@ -135,8 +136,6 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 	}
 
 	blk_execute_rq(req->q, NULL, req, 0);
-	if (result)
-		*result = (u32)(uintptr_t)req->special;
 	ret = req->errors;
  out:
 	blk_mq_free_request(req);
@@ -146,7 +145,7 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void *buffer, unsigned bufflen)
 {
-	return __nvme_submit_sync_cmd(q, cmd, buffer, bufflen, NULL, 0);
+	return __nvme_submit_sync_cmd(q, cmd, NULL, buffer, bufflen, 0);
 }
 
 int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
@@ -155,6 +154,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		u32 *result, unsigned timeout)
 {
 	bool write = cmd->common.opcode & 1;
+	struct nvme_completion cqe;
 	struct nvme_ns *ns = q->queuedata;
 	struct gendisk *disk = ns ? ns->disk : NULL;
 	struct request *req;
@@ -167,6 +167,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		return PTR_ERR(req);
 
 	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	req->special = &cqe;
 
 	if (ubuffer && bufflen) {
 		ret = blk_rq_map_user(q, req, NULL, ubuffer, bufflen,
@@ -221,7 +222,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 	blk_execute_rq(req->q, disk, req, 0);
 	ret = req->errors;
 	if (result)
-		*result = (u32)(uintptr_t)req->special;
+		*result = le32_to_cpu(cqe.result);
 	if (meta && !ret && !write) {
 		if (copy_to_user(meta_buffer, meta, meta_len))
 			ret = -EFAULT;
@@ -302,6 +303,8 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
 					dma_addr_t dma_addr, u32 *result)
 {
 	struct nvme_command c;
+	struct nvme_completion cqe;
+	int ret;
 
 	memset(&c, 0, sizeof(c));
 	c.features.opcode = nvme_admin_get_features;
@@ -309,13 +312,18 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
 	c.features.prp1 = cpu_to_le64(dma_addr);
 	c.features.fid = cpu_to_le32(fid);
 
-	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
+	ret = __nvme_submit_sync_cmd(dev->admin_q, &c, &cqe, NULL, 0, 0);
+	if (ret >= 0)
+		*result = le32_to_cpu(cqe.result);
+	return ret;
 }
 
 int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 					dma_addr_t dma_addr, u32 *result)
 {
 	struct nvme_command c;
+	struct nvme_completion cqe;
+	int ret;
 
 	memset(&c, 0, sizeof(c));
 	c.features.opcode = nvme_admin_set_features;
@@ -323,7 +331,10 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 	c.features.fid = cpu_to_le32(fid);
 	c.features.dword11 = cpu_to_le32(dword11);
 
-	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
+	ret = __nvme_submit_sync_cmd(dev->admin_q, &c, &cqe, NULL, 0, 0);
+	if (ret >= 0)
+		*result = le32_to_cpu(cqe.result);
+	return ret;
 }
 
 int nvme_get_log_page(struct nvme_ctrl *dev, struct nvme_smart_log **log)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4fb5bb7..d85997f 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -247,7 +247,8 @@ void nvme_requeue_req(struct request *req);
 int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void *buf, unsigned bufflen);
 int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
-		void *buffer, unsigned bufflen,  u32 *result, unsigned timeout);
+		struct nvme_completion *cqe, void *buffer, unsigned bufflen,
+		unsigned timeout);
 int nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void __user *ubuffer, unsigned bufflen, u32 *result,
 		unsigned timeout);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 72ef832..f8ae20c 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -759,10 +759,8 @@ static void __nvme_process_cq(struct nvme_queue *nvmeq, unsigned int *tag)
 		}
 
 		req = blk_mq_tag_to_rq(*nvmeq->tags, cqe.command_id);
-		if (req->cmd_type == REQ_TYPE_DRV_PRIV) {
-			u32 result = le32_to_cpu(cqe.result);
-			req->special = (void *)(uintptr_t)result;
-		}
+		if (req->cmd_type == REQ_TYPE_DRV_PRIV && req->special)
+			memcpy(req->special, &cqe, sizeof(cqe));
 		blk_mq_complete_request(req, status >> 1);
 
 	}
@@ -905,12 +903,10 @@ static void abort_endio(struct request *req, int error)
 {
 	struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
 	struct nvme_queue *nvmeq = iod->nvmeq;
-	u32 result = (u32)(uintptr_t)req->special;
 	u16 status = req->errors;
 
-	dev_warn(nvmeq->q_dmadev, "Abort status:%x result:%x", status, result);
+	dev_warn(nvmeq->q_dmadev, "Abort status 0x%x\n", status);
 	atomic_inc(&nvmeq->dev->ctrl.abort_limit);
-
 	blk_mq_free_request(req);
 }
 
-- 
2.1.4



From hch at infradead.org  Mon Feb  8 10:10:48 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Mon, 8 Feb 2016 10:10:48 -0800
Subject: [PATCH 1/4] NVMe: Fix io incapable return values
In-Reply-To: <1454515543-21683-2-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-2-git-send-email-keith.busch@intel.com>
Message-ID: <20160208181047.GA13126@infradead.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Mon Feb  8 10:11:50 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Mon, 8 Feb 2016 10:11:50 -0800
Subject: [PATCH 2/4] NVMe: Sync stopped queues with block layer
In-Reply-To: <1454515543-21683-3-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-3-git-send-email-keith.busch@intel.com>
Message-ID: <20160208181150.GB13126@infradead.org>

I don't think a driver call to blk_sync_queue is a good idea.  It might
be useful as part of a bigger hammer, but then it should be hidden
behind that API.


From hch at infradead.org  Mon Feb  8 10:13:40 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Mon, 8 Feb 2016 10:13:40 -0800
Subject: [PATCH 4/4] blk-mq: End unstarted requests on dying queue
In-Reply-To: <1454515543-21683-5-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-5-git-send-email-keith.busch@intel.com>
Message-ID: <20160208181340.GC13126@infradead.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Mon Feb  8 10:16:40 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Mon, 8 Feb 2016 10:16:40 -0800
Subject: [PATCH 3/4] NVMe: Surprise removal fixes
In-Reply-To: <1454515543-21683-4-git-send-email-keith.busch@intel.com>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-4-git-send-email-keith.busch@intel.com>
Message-ID: <20160208181640.GD13126@infradead.org>

On Wed, Feb 03, 2016 at 09:05:42AM -0700, Keith Busch wrote:
> Temporarily halts IO queues on surprise removal prior to flushing
> pending requests to an error completion.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---
>  drivers/nvme/host/core.c | 13 +++++++------
>  drivers/nvme/host/pci.c  | 14 ++++++++++++++
>  2 files changed, 21 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 9d415489..f11cb5a 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -1186,15 +1186,16 @@ static void nvme_ns_remove(struct nvme_ns *ns)
>  	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
>  
>  	if (kill) {
> -		blk_set_queue_dying(ns->queue);
> -
>  		/*
> -		 * The controller was shutdown first if we got here through
> -		 * device removal. The shutdown may requeue outstanding
> -		 * requests. These need to be aborted immediately so
> -		 * del_gendisk doesn't block indefinitely for their completion.
> +		 * The request queue is currently stoppped if we got here.
> +		 * Prevent new requests from entering before flushing the rest.
> +		 * New requests will fail to enter a dying frozen queue, which
> +		 * allows blk_cleanup_queue to complete.
>  		 */
> +		blk_mq_freeze_queue_start(ns->queue);
> +		blk_set_queue_dying(ns->queue);
>  		blk_mq_abort_requeue_list(ns->queue);
> +		blk_mq_start_hw_queues(ns->queue);
>  	}

Do we really still need all this magic if ->queue_rq returns a failure
if the queue is dying?

> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 72ef832..4cc7398 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -640,6 +640,11 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
>  	struct nvme_command cmnd;
>  	int ret = BLK_MQ_RQ_QUEUE_OK;
>  
> +	if (unlikely(blk_queue_dying(req->q))) {
> +		blk_mq_end_request(req, -EFAULT);
> +		return BLK_MQ_RQ_QUEUE_OK;
> +	}

"Bad address" is a really odd error for a dying block device.



From jonathan.derrick at intel.com  Mon Feb  8 10:23:15 2016
From: jonathan.derrick at intel.com (Jon Derrick)
Date: Mon, 8 Feb 2016 11:23:15 -0700
Subject: [PATCH] nvme: fix drvdata setup for the nvme device
In-Reply-To: <1454954856-32236-1-git-send-email-hch@lst.de>
References: <1454954856-32236-1-git-send-email-hch@lst.de>
Message-ID: <20160208182315.GA7930@localhost.localdomain>

I noticed this one too when I was perparing the last round of CMB patches

Looks good,
Reviewed-by: Jon Derrick <jonathan.derrick at intel.com>


From keith.busch at intel.com  Mon Feb  8 10:38:23 2016
From: keith.busch at intel.com (Keith Busch)
Date: Mon, 8 Feb 2016 18:38:23 +0000
Subject: [PATCH 3/4] NVMe: Surprise removal fixes
In-Reply-To: <20160208181640.GD13126@infradead.org>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-4-git-send-email-keith.busch@intel.com>
 <20160208181640.GD13126@infradead.org>
Message-ID: <20160208183823.GA29157@localhost.localdomain>

On Mon, Feb 08, 2016 at 10:16:40AM -0800, Christoph Hellwig wrote:
> On Wed, Feb 03, 2016 at 09:05:42AM -0700, Keith Busch wrote:
> > +		blk_mq_freeze_queue_start(ns->queue);
> > +		blk_set_queue_dying(ns->queue);
> >  		blk_mq_abort_requeue_list(ns->queue);
> > +		blk_mq_start_hw_queues(ns->queue);
> >  	}
> 
> Do we really still need all this magic if ->queue_rq returns a failure
> if the queue is dying?

This is far from perfect. Let me try explaining what's happening, then
I hope to abandon this patch and do it correctly. :)

The test does buffered IO writes with 'dd'. If I yank the drive,
'dd' continues until writing the device's capacity. The device is
gone, so there is no place to write dirty pages. The capcity exceeds
available memory, so 'dd' will wait even though everything it tries to
write fails. It consumes all available memory and the system becomes
noticably slower.

Ending that process is not what this patch accomplishes. It just lets
del_gendisk and blk_cleanup_queue complete by not allowing processes
to continue entering the queue by freezing first. Driver unbind doesn't
complete without this.

But there is a possible deadlock here. If you do an orderly removal
and surprise removal immediatly after, the device will initially be
considered capable of io and let dirty data sync. The hot removal will
make that impossible, but the orderly removal is holding the namespace
mutex, so no one can cleanup the namespace's queue.

Anyway, I wish to withdraw this patch. I will send an alternate this
week using state machine driven logic that should fix the deadlock unless
there is a different proposal sooner.


From jonathan.derrick at intel.com  Mon Feb  8 11:28:48 2016
From: jonathan.derrick at intel.com (Jon Derrick)
Date: Mon, 8 Feb 2016 12:28:48 -0700
Subject: [PATCH 2/2] NVMe: Fix possible queue use after freed
In-Reply-To: <1450397295-24031-2-git-send-email-keith.busch@intel.com>
References: <1450397295-24031-1-git-send-email-keith.busch@intel.com>
 <1450397295-24031-2-git-send-email-keith.busch@intel.com>
Message-ID: <20160208192848.GA8151@localhost.localdomain>

Hi Jens,

This set was acked a while back by Christoph and reviewed by me.
I just tested it again and it still works well against 4.5-rc3.

Is there any chance we can get this in for 4.6?

Cheers


From m at bjorling.me  Mon Feb  8 11:47:42 2016
From: m at bjorling.me (=?UTF-8?Q?Matias_Bj=c3=b8rling?=)
Date: Mon, 8 Feb 2016 20:47:42 +0100
Subject: [PATCH] nvme: return the whole CQE through the request
 passthrough interface
In-Reply-To: <1454954988-32306-2-git-send-email-hch@lst.de>
References: <1454954988-32306-1-git-send-email-hch@lst.de>
 <1454954988-32306-2-git-send-email-hch@lst.de>
Message-ID: <56B8F0DE.5090808@bjorling.me>

>   		req = blk_mq_tag_to_rq(*nvmeq->tags, cqe.command_id);
> -		if (req->cmd_type == REQ_TYPE_DRV_PRIV) {
> -			u32 result = le32_to_cpu(cqe.result);
> -			req->special = (void *)(uintptr_t)result;
> -		}
> +		if (req->cmd_type == REQ_TYPE_DRV_PRIV && req->special)
> +			memcpy(req->special, &cqe, sizeof(cqe));

Thanks for sending the patch.

When LightNVM send asynchronous data commands, it would need to allocate 
extra memory for cqe. Would it make sense to pass cqe directly and 
assume that it is live through the blk_mq_complete_request call?


From incoming at interfax.net  Mon Feb  8 12:11:15 2016
From: incoming at interfax.net (Interfax)
Date: Mon, 8 Feb 2016 20:11:15 +0000
Subject: You have received fax, document 00000738555
Message-ID: <374c5992ce03b0f93a7347d762a9a567@interfax.net>

A new fax document for you.

Scanned fax document is attached to this email.

File size:           131 Kb
Scanned:             Mon, 8 Feb 2016 13:41:07 +0300
Scan time:           15 seconds
Scan quality:        300 DPI
Sender:              Adrian Shoemaker
File name:           document-00000738555.doc
Pages number:        7

Thanks for choosing Interfax!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: document-00000738555.zip
Type: application/zip
Size: 1560 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20160208/24f4db89/attachment.zip>

From mlin at kernel.org  Mon Feb  8 14:24:42 2016
From: mlin at kernel.org (Ming Lin)
Date: Mon,  8 Feb 2016 14:24:42 -0800
Subject: [PATCH 2/4] nvme: export more symbols
In-Reply-To: <1454970284-29543-1-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
Message-ID: <1454970284-29543-3-git-send-email-mlin@kernel.org>

Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 3faed20..8e4b8ac 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -102,6 +102,7 @@ void nvme_requeue_req(struct request *req)
 		blk_mq_kick_requeue_list(req->q);
 	spin_unlock_irqrestore(req->q->queue_lock, flags);
 }
+EXPORT_SYMBOL_GPL(nvme_requeue_req);
 
 struct request *nvme_alloc_request(struct request_queue *q,
 		struct nvme_command *cmd, unsigned int flags)
@@ -125,6 +126,7 @@ struct request *nvme_alloc_request(struct request_queue *q,
 
 	return req;
 }
+EXPORT_SYMBOL_GPL(nvme_alloc_request);
 
 /*
  * Returns 0 on success.  If the result is negative, it's a Linux error code;
@@ -162,6 +164,7 @@ int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 {
 	return __nvme_submit_sync_cmd(q, cmd, buffer, bufflen, NULL, 0);
 }
+EXPORT_SYMBOL_GPL(nvme_submit_sync_cmd);
 
 int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void __user *ubuffer, unsigned bufflen,
@@ -377,6 +380,7 @@ int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count)
 	*count = min(*count, nr_io_queues);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nvme_set_queue_count);
 
 static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
 {
@@ -783,6 +787,7 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 		return ret;
 	return nvme_wait_ready(ctrl, cap, false);
 }
+EXPORT_SYMBOL_GPL(nvme_disable_ctrl);
 
 int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 {
@@ -814,6 +819,7 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 		return ret;
 	return nvme_wait_ready(ctrl, cap, true);
 }
+EXPORT_SYMBOL_GPL(nvme_enable_ctrl);
 
 int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
 {
@@ -844,6 +850,7 @@ int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
 
 	return ret;
 }
+EXPORT_SYMBOL_GPL(nvme_shutdown_ctrl);
 
 /*
  * Initialize the cached copies of the Identify data and various controller
@@ -905,6 +912,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	kfree(id);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nvme_init_identify);
 
 static int nvme_dev_open(struct inode *inode, struct file *file)
 {
@@ -1310,6 +1318,7 @@ void nvme_scan_namespaces(struct nvme_ctrl *ctrl)
 	mutex_unlock(&ctrl->namespaces_mutex);
 	kfree(id);
 }
+EXPORT_SYMBOL_GPL(nvme_scan_namespaces);
 
 void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 {
@@ -1320,6 +1329,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 		nvme_ns_remove(ns);
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
 
 static DEFINE_IDA(nvme_instance_ida);
 
@@ -1351,13 +1361,14 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
 }
 
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
- {
+{
 	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 
 	spin_lock(&dev_list_lock);
 	list_del(&ctrl->node);
 	spin_unlock(&dev_list_lock);
 }
+EXPORT_SYMBOL_GPL(nvme_uninit_ctrl);
 
 static void nvme_free_ctrl(struct kref *kref)
 {
@@ -1373,6 +1384,7 @@ void nvme_put_ctrl(struct nvme_ctrl *ctrl)
 {
 	kref_put(&ctrl->kref, nvme_free_ctrl);
 }
+EXPORT_SYMBOL_GPL(nvme_put_ctrl);
 
 /*
  * Initialize a NVMe controller structures.  This needs to be called during
@@ -1416,6 +1428,7 @@ out_release_instance:
 out:
 	return ret;
 }
+EXPORT_SYMBOL_GPL(nvme_init_ctrl);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
@@ -1432,6 +1445,7 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_stop_queues);
 
 void nvme_start_queues(struct nvme_ctrl *ctrl)
 {
@@ -1445,6 +1459,7 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_start_queues);
 
 int __init nvme_core_init(void)
 {
-- 
1.9.1



From mlin at kernel.org  Mon Feb  8 14:24:41 2016
From: mlin at kernel.org (Ming Lin)
Date: Mon,  8 Feb 2016 14:24:41 -0800
Subject: [PATCH 1/4] nvme: move timeout variables to core.c and export it
In-Reply-To: <1454970284-29543-1-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
Message-ID: <1454970284-29543-2-git-send-email-mlin@kernel.org>

Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 14 ++++++++++++++
 drivers/nvme/host/pci.c  | 12 ------------
 2 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c5bf001..3faed20 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -33,6 +33,20 @@
 
 #define NVME_MINORS		(1U << MINORBITS)
 
+unsigned char admin_timeout = 60;
+module_param(admin_timeout, byte, 0644);
+MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
+EXPORT_SYMBOL_GPL(admin_timeout);
+
+unsigned char nvme_io_timeout = 30;
+module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
+MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
+EXPORT_SYMBOL_GPL(nvme_io_timeout);
+
+unsigned char shutdown_timeout = 5;
+module_param(shutdown_timeout, byte, 0644);
+MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
+
 static int nvme_major;
 module_param(nvme_major, int, 0);
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 72ef832..deba7ac 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -57,18 +57,6 @@
 #define NVME_NR_AEN_COMMANDS	1
 #define NVME_AQ_BLKMQ_DEPTH	(NVME_AQ_DEPTH - NVME_NR_AEN_COMMANDS)
 
-unsigned char admin_timeout = 60;
-module_param(admin_timeout, byte, 0644);
-MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
-
-unsigned char nvme_io_timeout = 30;
-module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
-MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
-
-unsigned char shutdown_timeout = 5;
-module_param(shutdown_timeout, byte, 0644);
-MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
-
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 
-- 
1.9.1



From mlin at kernel.org  Mon Feb  8 14:24:40 2016
From: mlin at kernel.org (Ming Lin)
Date: Mon,  8 Feb 2016 14:24:40 -0800
Subject: [PATCH 0/4] nvme: split pci module out of core module
Message-ID: <1454970284-29543-1-git-send-email-mlin@kernel.org>

We'll have nvme over fabric dirver soon.
This series split nvme pci module out of nvme core module.

nvme.ko is split into 2 modules:
nvme-core.ko: the core part
nvme.ko: the PCI driver

Ming Lin (4):
  nvme: move timeout variables to core.c and export it
  nvme: export more symbols
  nvme: split dev_list_lock
  nvme: split pci module out of core module

 drivers/nvme/host/Kconfig  | 10 +++++++---
 drivers/nvme/host/Makefile | 10 ++++++----
 drivers/nvme/host/core.c   | 40 +++++++++++++++++++++++++++++++++++++---
 drivers/nvme/host/nvme.h   |  2 --
 drivers/nvme/host/pci.c    | 26 ++------------------------
 5 files changed, 52 insertions(+), 36 deletions(-)

-- 
1.9.1



From mlin at kernel.org  Mon Feb  8 14:24:43 2016
From: mlin at kernel.org (Ming Lin)
Date: Mon,  8 Feb 2016 14:24:43 -0800
Subject: [PATCH 3/4] nvme: split dev_list_lock
In-Reply-To: <1454970284-29543-1-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
Message-ID: <1454970284-29543-4-git-send-email-mlin@kernel.org>

Split dev_list_lock into one in the core and one in the PCI driver.

Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 2 +-
 drivers/nvme/host/nvme.h | 2 --
 drivers/nvme/host/pci.c  | 1 +
 3 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 8e4b8ac..3928366 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -54,7 +54,7 @@ static int nvme_char_major;
 module_param(nvme_char_major, int, 0);
 
 static LIST_HEAD(nvme_ctrl_list);
-DEFINE_SPINLOCK(dev_list_lock);
+static DEFINE_SPINLOCK(dev_list_lock);
 
 static struct class *nvme_class;
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4fb5bb7..3cea0ed 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -265,8 +265,6 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 			dma_addr_t dma_addr, u32 *result);
 int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count);
 
-extern spinlock_t dev_list_lock;
-
 struct sg_io_hdr;
 
 int nvme_sg_io(struct nvme_ns *ns, struct sg_io_hdr __user *u_hdr);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index deba7ac..9797595 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -65,6 +65,7 @@ module_param(use_cmb_sqes, bool, 0644);
 MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
 
 static LIST_HEAD(dev_list);
+static DEFINE_SPINLOCK(dev_list_lock);
 static struct task_struct *nvme_thread;
 static struct workqueue_struct *nvme_workq;
 static wait_queue_head_t nvme_kthread_wait;
-- 
1.9.1



From mlin at kernel.org  Mon Feb  8 14:24:44 2016
From: mlin at kernel.org (Ming Lin)
Date: Mon,  8 Feb 2016 14:24:44 -0800
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <1454970284-29543-1-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
Message-ID: <1454970284-29543-5-git-send-email-mlin@kernel.org>

This splits nvme.ko into 2 modules:
nvme-core.ko: the core part
nvme.ko: the PCI driver

Also changes config name:
s/CONFIG_BLK_DEV_NVME/CONFIG_NVME_PCI
s/CONFIG_BLK_DEV_NVME_SCSI/CONFIG_NVME_SCSI

Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/Kconfig  | 10 +++++++---
 drivers/nvme/host/Makefile | 10 ++++++----
 drivers/nvme/host/core.c   |  7 ++++++-
 drivers/nvme/host/pci.c    | 13 +------------
 4 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index 5d62373..883dcd6 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -1,6 +1,10 @@
-config BLK_DEV_NVME
+config NVME_CORE
+	tristate
+
+config NVME_PCI
 	tristate "NVM Express block device"
 	depends on PCI && BLOCK
+	select NVME_CORE
 	---help---
 	  The NVM Express driver is for solid state drives directly
 	  connected to the PCI or PCI Express bus.  If you know you
@@ -9,9 +13,9 @@ config BLK_DEV_NVME
 	  To compile this driver as a module, choose M here: the
 	  module will be called nvme.
 
-config BLK_DEV_NVME_SCSI
+config NVME_SCSI
 	bool "SCSI emulation for NVMe device nodes"
-	depends on BLK_DEV_NVME
+	depends on NVME_CORE
 	---help---
 	  This adds support for the SG_IO ioctl on the NVMe character
 	  and block devices nodes, as well a a translation for a small
diff --git a/drivers/nvme/host/Makefile b/drivers/nvme/host/Makefile
index 51bf908..98b057e 100644
--- a/drivers/nvme/host/Makefile
+++ b/drivers/nvme/host/Makefile
@@ -1,6 +1,8 @@
+obj-$(CONFIG_NVME_CORE)		+= nvme-core.o
+obj-$(CONFIG_NVME_PCI)		+= nvme.o
 
-obj-$(CONFIG_BLK_DEV_NVME)     += nvme.o
+nvme-core-y				:= core.o
+nvme-core-$(CONFIG_NVME_SCSI)		+= scsi.o
+nvme-core-$(CONFIG_NVM)			+= lightnvm.o
 
-lightnvm-$(CONFIG_NVM)			:= lightnvm.o
-nvme-y					+= core.o pci.o $(lightnvm-y)
-nvme-$(CONFIG_BLK_DEV_NVME_SCSI)        += scsi.o
+nvme-y					+= pci.o
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 3928366..eaf9d34 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -485,7 +485,7 @@ static int nvme_ioctl(struct block_device *bdev, fmode_t mode,
 		return nvme_user_cmd(ns->ctrl, ns, (void __user *)arg);
 	case NVME_IOCTL_SUBMIT_IO:
 		return nvme_submit_io(ns, (void __user *)arg);
-#ifdef CONFIG_BLK_DEV_NVME_SCSI
+#ifdef CONFIG_NVME_SCSI
 	case SG_GET_VERSION_NUM:
 		return nvme_sg_get_version_num((void __user *)arg);
 	case SG_IO:
@@ -1499,3 +1499,8 @@ void nvme_core_exit(void)
 	class_destroy(nvme_class);
 	__unregister_chrdev(nvme_char_major, 0, NVME_MINORS, "nvme");
 }
+
+MODULE_LICENSE("GPL");
+MODULE_VERSION("1.0");
+module_init(nvme_core_init);
+module_exit(nvme_core_exit);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 9797595..4cc32fa 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2220,26 +2220,15 @@ static int __init nvme_init(void)
 	if (!nvme_workq)
 		return -ENOMEM;
 
-	result = nvme_core_init();
-	if (result < 0)
-		goto kill_workq;
-
 	result = pci_register_driver(&nvme_driver);
 	if (result)
-		goto core_exit;
-	return 0;
-
- core_exit:
-	nvme_core_exit();
- kill_workq:
-	destroy_workqueue(nvme_workq);
+		destroy_workqueue(nvme_workq);
 	return result;
 }
 
 static void __exit nvme_exit(void)
 {
 	pci_unregister_driver(&nvme_driver);
-	nvme_core_exit();
 	destroy_workqueue(nvme_workq);
 	BUG_ON(nvme_thread && !IS_ERR(nvme_thread));
 	_nvme_check_size();
-- 
1.9.1



From keith.busch at intel.com  Mon Feb  8 15:48:55 2016
From: keith.busch at intel.com (Keith Busch)
Date: Mon, 8 Feb 2016 23:48:55 +0000
Subject: [PATCH 3/4] NVMe: Surprise removal fixes
In-Reply-To: <20160208183823.GA29157@localhost.localdomain>
References: <1454515543-21683-1-git-send-email-keith.busch@intel.com>
 <1454515543-21683-4-git-send-email-keith.busch@intel.com>
 <20160208181640.GD13126@infradead.org>
 <20160208183823.GA29157@localhost.localdomain>
Message-ID: <20160208234855.GA3058@localhost.localdomain>

On Mon, Feb 08, 2016 at 06:38:23PM +0000, Keith Busch wrote:
> On Mon, Feb 08, 2016 at 10:16:40AM -0800, Christoph Hellwig wrote:
> > Do we really still need all this magic if ->queue_rq returns a failure
> > if the queue is dying?
> 
> This is far from perfect. Let me try explaining what's happening, then
> I hope to abandon this patch and do it correctly. :)

I've a new patch that passes all my tests, including the removal deadlock
using device states.

I have namespace capacity set to 0 and revalidate to get buffered writers
to end. The error handling is done in the nvme_workq, which currently
has WQ_MEM_RECLAIM set. That trigger a warning when revalidate_disk
attempts to sync with a non-MEM_RECLAIM work queue, so I removed the
flag from NVMe's to suppress the warning, but I don't see what having
that flag gained us in the first place. It looks to me that it just
spawns a rescuer, but do we need that?


From hare at suse.de  Mon Feb  8 23:50:33 2016
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 9 Feb 2016 08:50:33 +0100
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160207172055.GA6477@redhat.com>
References: <20160203182423.GA12913@redhat.com> <56B2F5BC.1010700@suse.de>
 <20160204135420.GA18227@redhat.com> <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com> <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il> <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il> <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com>
Message-ID: <56B99A49.5050400@suse.de>

On 02/07/2016 06:20 PM, Mike Snitzer wrote:
> On Sun, Feb 07 2016 at 11:54am -0500,
> Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:
> 
>>
>>>> If so, can you check with e.g.
>>>> perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
>>>> workload triggers perhaps lock contention ? What you need to look for in
>>>> the perf output is whether any functions occupy more than 10% CPU time.
>>>
>>> I will, thanks for the tip!
>>
>> The perf report is very similar to the one that started this effort..
>>
>> I'm afraid we'll need to resolve the per-target m->lock in order
>> to scale with NUMA...
> 
> Could be.  Just for testing, you can try the 2 topmost commits I've put
> here (once applied both __multipath_map and multipath_busy won't have
> _any_ locking.. again, very much test-only):
> 
> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel2
> 
So, I gave those patches a spin.
Sad to say, they do _not_ resolve the issue fully.

My testbed (2 paths per LUN, 40 CPUs, 4 cores) yields 505k IOPs with
those patches.
Using a single path (without those patches, but still running
multipath on top of that path) the same testbed yields 550k IOPs.
Which very much smells like a lock contention ...
We do get a slight improvement, though; without those patches I
could only get about 350k IOPs. But still, I would somehow expect 2
paths to be faster than just one ..

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From jthumshirn at suse.de  Tue Feb  9 00:56:57 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Tue, 9 Feb 2016 09:56:57 +0100
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <1454970284-29543-5-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-5-git-send-email-mlin@kernel.org>
Message-ID: <20160209085656.GJ27969@c203.arch.suse.de>

On Mon, Feb 08, 2016 at 02:24:44PM -0800, Ming Lin wrote:
> This splits nvme.ko into 2 modules:
> nvme-core.ko: the core part
> nvme.ko: the PCI driver
> 
> Also changes config name:
> s/CONFIG_BLK_DEV_NVME/CONFIG_NVME_PCI
> s/CONFIG_BLK_DEV_NVME_SCSI/CONFIG_NVME_SCSI

Is there any auto-migration for those who already have CONFIG_BLK_DEV_NVME set
in their kernel configs? If not, this could lead to a bunch of non-booting
systems.

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From hch at lst.de  Tue Feb  9 01:24:45 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 10:24:45 +0100
Subject: [PATCH 0/4] nvme: split pci module out of core module
In-Reply-To: <1454970284-29543-1-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
Message-ID: <20160209092445.GA21250@lst.de>

On Mon, Feb 08, 2016 at 02:24:40PM -0800, Ming Lin wrote:
> We'll have nvme over fabric dirver soon.
> This series split nvme pci module out of nvme core module.
> 
> nvme.ko is split into 2 modules:
> nvme-core.ko: the core part
> nvme.ko: the PCI driver

Might be worth explaining why we are doing this:  for the NVMe over
Fabrics drivers that are going to reuse the core, as well as things
like the vhost NVMe driver if we're going to merge it.

We'll also need something to take proper references to the transport
driver on open in this series.  E.g. this patch from Sagi in the Fabrics
tree:

---
>From 1fafa7768fe93bd6a7ec335738bbae9e00f4a634 Mon Sep 17 00:00:00 2001
From: Sagi Grimberg <sagig at mellanox.com>
Date: Tue, 2 Feb 2016 21:52:51 +0100
Subject: nvme/host: reference the fabric module for each bdev open callout

We don't want to be able to unload the fabric driver when we have
openened referenced to our namespaces. Thus, for each nvme_open we
take a reference on the fabric driver and put it in nvme_release.
This behavior is consistent with the scsi model.

This resolves the panic when unloading a fabric module with
mpath holders.

Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Ian Bakshan <ianb at mellanox.com>
---
 drivers/nvme/host/core.c | 19 ++++++++++++++++---
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/pci.c  |  1 +
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index a152b8c..9fe7471 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -71,11 +71,21 @@ static struct nvme_ns *nvme_get_ns_from_disk(struct gendisk *disk)
 
 	spin_lock(&dev_list_lock);
 	ns = disk->private_data;
-	if (ns && !kref_get_unless_zero(&ns->kref))
-		ns = NULL;
+	if (ns) {
+		if (!kref_get_unless_zero(&ns->kref))
+			goto fail;
+		if (!try_module_get(ns->ctrl->ops->module))
+			goto fail_put_ns;
+	}
 	spin_unlock(&dev_list_lock);
 
 	return ns;
+
+fail_put_ns:
+	kref_put(&ns->kref, nvme_free_ns);
+fail:
+	spin_unlock(&dev_list_lock);
+	return NULL;
 }
 
 void nvme_requeue_req(struct request *req)
@@ -517,7 +527,10 @@ static int nvme_open(struct block_device *bdev, fmode_t mode)
 
 static void nvme_release(struct gendisk *disk, fmode_t mode)
 {
-	nvme_put_ns(disk->private_data);
+	struct nvme_ns *ns = disk->private_data;
+
+	module_put(ns->ctrl->ops->module);
+	nvme_put_ns(ns);
 }
 
 static int nvme_getgeo(struct block_device *bdev, struct hd_geometry *geo)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 62209aa..cce40c9 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -119,6 +119,7 @@ struct nvme_ns {
 };
 
 struct nvme_ctrl_ops {
+	struct module *module;
 	int (*reg_read32)(struct nvme_ctrl *ctrl, u32 off, u32 *val);
 	int (*reg_write32)(struct nvme_ctrl *ctrl, u32 off, u32 val);
 	int (*reg_read64)(struct nvme_ctrl *ctrl, u32 off, u64 *val);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index ad24442d..526a30b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2016,6 +2016,7 @@ static int nvme_pci_reset_ctrl(struct nvme_ctrl *ctrl)
 }
 
 static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = {
+	.module			= THIS_MODULE,
 	.reg_read32		= nvme_pci_reg_read32,
 	.reg_write32		= nvme_pci_reg_write32,
 	.reg_read64		= nvme_pci_reg_read64,
-- 
2.1.4



From hch at lst.de  Tue Feb  9 01:25:01 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 10:25:01 +0100
Subject: [PATCH 1/4] nvme: move timeout variables to core.c and export it
In-Reply-To: <1454970284-29543-2-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-2-git-send-email-mlin@kernel.org>
Message-ID: <20160209092501.GB21250@lst.de>

Looks good,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at lst.de  Tue Feb  9 01:25:47 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 10:25:47 +0100
Subject: [PATCH 2/4] nvme: export more symbols
In-Reply-To: <1454970284-29543-3-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-3-git-send-email-mlin@kernel.org>
Message-ID: <20160209092547.GC21250@lst.de>

These are the first symbols exports from the nvme module :)

Also a little explanation on why might be useful, a single sentence
should be enough.

Otherwise looks good:

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at lst.de  Tue Feb  9 01:26:17 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 10:26:17 +0100
Subject: [PATCH 3/4] nvme: split dev_list_lock
In-Reply-To: <1454970284-29543-4-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-4-git-send-email-mlin@kernel.org>
Message-ID: <20160209092617.GD21250@lst.de>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at lst.de  Tue Feb  9 01:26:50 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 10:26:50 +0100
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <1454970284-29543-5-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-5-git-send-email-mlin@kernel.org>
Message-ID: <20160209092650.GE21250@lst.de>

Looks fine,

but as Johannes poined out we'll probably need some Kconfig magic
to keep existing configfs working.


From wenbo.wang at memblaze.com  Tue Feb  9 03:22:04 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Tue, 9 Feb 2016 11:22:04 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <20160208150132.GA25111@localhost.localdomain>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203144123.GB23910@localhost.localdomain>
 <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203163808.GC23910@localhost.localdomain>
 <BJXPR01MB19916231021454BD8731EBDE0D30@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160208150132.GA25111@localhost.localdomain>
Message-ID: <BJXPR01MB199E172F7B186281D1C7179E0D60@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

In most cases, rcu read lock is just a preempt_disable, which is what get_cpu does. I don't see any risk.

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Monday, February 8, 2016 11:02 PM
To: Wenbo Wang
Cc: Jens Axboe; Wenbo Wang; linux-kernel at vger.kernel.org; linux-nvme at lists.infradead.org; Wenwei.Tao
Subject: Re: [PATCH] NVMe: do not touch sq door bell if nvmeq has been suspended

On Sat, Feb 06, 2016 at 02:32:24PM +0000, Wenbo Wang wrote:
> Keith,
> 
> Is the following solution OK?
> synchronize_rcu guarantee that no queue_rq is running concurrently with device disable code. Together with your another patch (adding blk_sync_queue), both sync/async path shall be handled correctly.
> 
> Do you think synchronize_rcu shall be added to blk_sync_queue?

I was nearly going to suggest the same last week, but it feels wrong since no one takes rcu_read_lock in the path we're trying to sychronoize. Is this safe if the task is interrupted?

From jthumshirn at suse.de  Tue Feb  9 04:41:34 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Tue, 9 Feb 2016 13:41:34 +0100
Subject: [PATCH 3/4] nvme: split dev_list_lock
In-Reply-To: <1454970284-29543-4-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-4-git-send-email-mlin@kernel.org>
Message-ID: <20160209124134.GM27969@c203.arch.suse.de>

On Mon, Feb 08, 2016 at 02:24:43PM -0800, Ming Lin wrote:
> Split dev_list_lock into one in the core and one in the PCI driver.

The dev_list_lock in core.c doesn't really protect the dev_list, like it does
in pci.c. Wouldn't it be better to rename it to something more appropriate? I
just can't come up with a name, as it seems to be some kind of catch all lock.

Otherwise

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
> 
> Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
> ---
>  drivers/nvme/host/core.c | 2 +-
>  drivers/nvme/host/nvme.h | 2 --
>  drivers/nvme/host/pci.c  | 1 +
>  3 files changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 8e4b8ac..3928366 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -54,7 +54,7 @@ static int nvme_char_major;
>  module_param(nvme_char_major, int, 0);
>  
>  static LIST_HEAD(nvme_ctrl_list);
> -DEFINE_SPINLOCK(dev_list_lock);
> +static DEFINE_SPINLOCK(dev_list_lock);
>  
>  static struct class *nvme_class;
>  
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 4fb5bb7..3cea0ed 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -265,8 +265,6 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
>  			dma_addr_t dma_addr, u32 *result);
>  int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count);
>  
> -extern spinlock_t dev_list_lock;
> -
>  struct sg_io_hdr;
>  
>  int nvme_sg_io(struct nvme_ns *ns, struct sg_io_hdr __user *u_hdr);
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index deba7ac..9797595 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -65,6 +65,7 @@ module_param(use_cmb_sqes, bool, 0644);
>  MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
>  
>  static LIST_HEAD(dev_list);
> +static DEFINE_SPINLOCK(dev_list_lock);
>  static struct task_struct *nvme_thread;
>  static struct workqueue_struct *nvme_workq;
>  static wait_queue_head_t nvme_kthread_wait;
> -- 
> 1.9.1
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Tue Feb  9 04:42:49 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Tue, 9 Feb 2016 13:42:49 +0100
Subject: [PATCH 2/4] nvme: export more symbols
In-Reply-To: <1454970284-29543-3-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-3-git-send-email-mlin@kernel.org>
Message-ID: <20160209124249.GN27969@c203.arch.suse.de>

On Mon, Feb 08, 2016 at 02:24:42PM -0800, Ming Lin wrote:
> Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
> ---

Like Christoph already said, please be a bit more verbose with your commit
messages.


Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Tue Feb  9 04:43:49 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Tue, 9 Feb 2016 13:43:49 +0100
Subject: [PATCH 1/4] nvme: move timeout variables to core.c and export it
In-Reply-To: <1454970284-29543-2-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-2-git-send-email-mlin@kernel.org>
Message-ID: <20160209124349.GO27969@c203.arch.suse.de>

On Mon, Feb 08, 2016 at 02:24:41PM -0800, Ming Lin wrote:
> Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
> ---

I'd really like to have one or two sentences more in the log but anyways

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

>  drivers/nvme/host/core.c | 14 ++++++++++++++
>  drivers/nvme/host/pci.c  | 12 ------------
>  2 files changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index c5bf001..3faed20 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -33,6 +33,20 @@
>  
>  #define NVME_MINORS		(1U << MINORBITS)
>  
> +unsigned char admin_timeout = 60;
> +module_param(admin_timeout, byte, 0644);
> +MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
> +EXPORT_SYMBOL_GPL(admin_timeout);
> +
> +unsigned char nvme_io_timeout = 30;
> +module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
> +MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
> +EXPORT_SYMBOL_GPL(nvme_io_timeout);
> +
> +unsigned char shutdown_timeout = 5;
> +module_param(shutdown_timeout, byte, 0644);
> +MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
> +
>  static int nvme_major;
>  module_param(nvme_major, int, 0);
>  
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 72ef832..deba7ac 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -57,18 +57,6 @@
>  #define NVME_NR_AEN_COMMANDS	1
>  #define NVME_AQ_BLKMQ_DEPTH	(NVME_AQ_DEPTH - NVME_NR_AEN_COMMANDS)
>  
> -unsigned char admin_timeout = 60;
> -module_param(admin_timeout, byte, 0644);
> -MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
> -
> -unsigned char nvme_io_timeout = 30;
> -module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
> -MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
> -
> -unsigned char shutdown_timeout = 5;
> -module_param(shutdown_timeout, byte, 0644);
> -MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
> -
>  static int use_threaded_interrupts;
>  module_param(use_threaded_interrupts, int, 0);
>  
> -- 
> 1.9.1
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From hch at lst.de  Tue Feb  9 04:46:02 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 13:46:02 +0100
Subject: [PATCH] nvme: return the whole CQE through the request
 passthrough interface
In-Reply-To: <56B8F0DE.5090808@bjorling.me>
References: <1454954988-32306-1-git-send-email-hch@lst.de>
 <1454954988-32306-2-git-send-email-hch@lst.de> <56B8F0DE.5090808@bjorling.me>
Message-ID: <20160209124602.GA25353@lst.de>

On Mon, Feb 08, 2016 at 08:47:42PM +0100, Matias Bj?rling wrote:
> When LightNVM send asynchronous data commands, it would need to allocate 
> extra memory for cqe. Would it make sense to pass cqe directly and assume 
> that it is live through the blk_mq_complete_request call?

The CQE does not survive until ->end_io is called, both in the current
PCI driver, and in the Fabrics drivers.  So we need some caller provided
storage for it.


From m at bjorling.me  Tue Feb  9 04:46:37 2016
From: m at bjorling.me (=?UTF-8?Q?Matias_Bj=c3=b8rling?=)
Date: Tue, 9 Feb 2016 13:46:37 +0100
Subject: [PATCH 1/4] nvme: move timeout variables to core.c and export it
In-Reply-To: <1454970284-29543-2-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-2-git-send-email-mlin@kernel.org>
Message-ID: <56B9DFAD.10809@bjorling.me>

On 02/08/2016 11:24 PM, Ming Lin wrote:
> Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
> ---
>  drivers/nvme/host/core.c | 14 ++++++++++++++
>  drivers/nvme/host/pci.c  | 12 ------------
>  2 files changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index c5bf001..3faed20 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -33,6 +33,20 @@
>  
>  #define NVME_MINORS		(1U << MINORBITS)
>  
> +unsigned char admin_timeout = 60;
> +module_param(admin_timeout, byte, 0644);
> +MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
> +EXPORT_SYMBOL_GPL(admin_timeout);
> +
> +unsigned char nvme_io_timeout = 30;
> +module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
> +MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
> +EXPORT_SYMBOL_GPL(nvme_io_timeout);


Can these be exported without _GPL? There might be a couple of drivers
out of tree which would like to access these variables.



From hch at lst.de  Tue Feb  9 04:50:11 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 13:50:11 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <56B7C527.6050300@kernel.dk>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
Message-ID: <20160209125011.GC25353@lst.de>

Jens,

do you want a 'default y' patch or just a better description?  I'd be
happy to send either one.


From m at bjorling.me  Tue Feb  9 04:56:36 2016
From: m at bjorling.me (=?UTF-8?Q?Matias_Bj=c3=b8rling?=)
Date: Tue, 9 Feb 2016 13:56:36 +0100
Subject: [PATCH 2/4] nvme: export more symbols
In-Reply-To: <1454970284-29543-3-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-3-git-send-email-mlin@kernel.org>
Message-ID: <56B9E204.4080702@bjorling.me>

On 02/08/2016 11:24 PM, Ming Lin wrote:
> Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
> ---
>  drivers/nvme/host/core.c | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 3faed20..8e4b8ac 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -102,6 +102,7 @@ void nvme_requeue_req(struct request *req)
>  		blk_mq_kick_requeue_list(req->q);
>  	spin_unlock_irqrestore(req->q->queue_lock, flags);
>  }
> +EXPORT_SYMBOL_GPL(nvme_requeue_req);
>  
>  struct request *nvme_alloc_request(struct request_queue *q,
>  		struct nvme_command *cmd, unsigned int flags)
> @@ -125,6 +126,7 @@ struct request *nvme_alloc_request(struct request_queue *q,
>  
>  	return req;
>  }
> +EXPORT_SYMBOL_GPL(nvme_alloc_request);
>  
>  /*
>   * Returns 0 on success.  If the result is negative, it's a Linux error code;
> @@ -162,6 +164,7 @@ int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>  {
>  	return __nvme_submit_sync_cmd(q, cmd, buffer, bufflen, NULL, 0);
>  }
> +EXPORT_SYMBOL_GPL(nvme_submit_sync_cmd);
>  
>  int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		void __user *ubuffer, unsigned bufflen,
> @@ -377,6 +380,7 @@ int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count)
>  	*count = min(*count, nr_io_queues);
>  	return 0;
>  }
> +EXPORT_SYMBOL_GPL(nvme_set_queue_count);
>  
>  static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
>  {
> @@ -783,6 +787,7 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
>  		return ret;
>  	return nvme_wait_ready(ctrl, cap, false);
>  }
> +EXPORT_SYMBOL_GPL(nvme_disable_ctrl);
>  
>  int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
>  {
> @@ -814,6 +819,7 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
>  		return ret;
>  	return nvme_wait_ready(ctrl, cap, true);
>  }
> +EXPORT_SYMBOL_GPL(nvme_enable_ctrl);
>  
>  int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
>  {
> @@ -844,6 +850,7 @@ int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
>  
>  	return ret;
>  }
> +EXPORT_SYMBOL_GPL(nvme_shutdown_ctrl);
>  
>  /*
>   * Initialize the cached copies of the Identify data and various controller
> @@ -905,6 +912,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
>  	kfree(id);
>  	return 0;
>  }
> +EXPORT_SYMBOL_GPL(nvme_init_identify);
>  
>  static int nvme_dev_open(struct inode *inode, struct file *file)
>  {
> @@ -1310,6 +1318,7 @@ void nvme_scan_namespaces(struct nvme_ctrl *ctrl)
>  	mutex_unlock(&ctrl->namespaces_mutex);
>  	kfree(id);
>  }
> +EXPORT_SYMBOL_GPL(nvme_scan_namespaces);
>  
>  void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
>  {
> @@ -1320,6 +1329,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
>  		nvme_ns_remove(ns);
>  	mutex_unlock(&ctrl->namespaces_mutex);
>  }
> +EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
>  
>  static DEFINE_IDA(nvme_instance_ida);
>  
> @@ -1351,13 +1361,14 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
>  }
>  
>  void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
> - {
> +{
>  	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
>  
>  	spin_lock(&dev_list_lock);
>  	list_del(&ctrl->node);
>  	spin_unlock(&dev_list_lock);
>  }
> +EXPORT_SYMBOL_GPL(nvme_uninit_ctrl);
>  
>  static void nvme_free_ctrl(struct kref *kref)
>  {
> @@ -1373,6 +1384,7 @@ void nvme_put_ctrl(struct nvme_ctrl *ctrl)
>  {
>  	kref_put(&ctrl->kref, nvme_free_ctrl);
>  }
> +EXPORT_SYMBOL_GPL(nvme_put_ctrl);
>  
>  /*
>   * Initialize a NVMe controller structures.  This needs to be called during
> @@ -1416,6 +1428,7 @@ out_release_instance:
>  out:
>  	return ret;
>  }
> +EXPORT_SYMBOL_GPL(nvme_init_ctrl);
>  
>  void nvme_stop_queues(struct nvme_ctrl *ctrl)
>  {
> @@ -1432,6 +1445,7 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
>  	}
>  	mutex_unlock(&ctrl->namespaces_mutex);
>  }
> +EXPORT_SYMBOL_GPL(nvme_stop_queues);
>  
>  void nvme_start_queues(struct nvme_ctrl *ctrl)
>  {
> @@ -1445,6 +1459,7 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
>  	}
>  	mutex_unlock(&ctrl->namespaces_mutex);
>  }
> +EXPORT_SYMBOL_GPL(nvme_start_queues);
>  
>  int __init nvme_core_init(void)
>  {
> 

Hi Ming,

Can these be exported without _GPL?


From m at bjorling.me  Tue Feb  9 05:05:04 2016
From: m at bjorling.me (=?UTF-8?Q?Matias_Bj=c3=b8rling?=)
Date: Tue, 9 Feb 2016 14:05:04 +0100
Subject: [PATCH] nvme: return the whole CQE through the request
 passthrough interface
In-Reply-To: <1454954988-32306-2-git-send-email-hch@lst.de>
References: <1454954988-32306-1-git-send-email-hch@lst.de>
 <1454954988-32306-2-git-send-email-hch@lst.de>
Message-ID: <56B9E400.3010702@bjorling.me>

On 02/08/2016 07:09 PM, Christoph Hellwig wrote:
> Both LighNVM and NVMe over Fabrics need to look at more than just the
> status and result field.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
> Reviewed-by: Jay Freyensee <james.p.freyensee at intel.com>
> Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
> ---
>  drivers/nvme/host/core.c | 27 +++++++++++++++++++--------
>  drivers/nvme/host/nvme.h |  3 ++-
>  drivers/nvme/host/pci.c  | 10 +++-------
>  3 files changed, 24 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index c326931..6715bbf 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -107,7 +107,6 @@ struct request *nvme_alloc_request(struct request_queue *q,
>  
>  	req->cmd = (unsigned char *)cmd;
>  	req->cmd_len = sizeof(struct nvme_command);
> -	req->special = (void *)0;
>  
>  	return req;
>  }
> @@ -117,7 +116,8 @@ struct request *nvme_alloc_request(struct request_queue *q,
>   * if the result is positive, it's an NVM Express status code
>   */
>  int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
> -		void *buffer, unsigned bufflen, u32 *result, unsigned timeout)
> +		struct nvme_completion *cqe, void *buffer, unsigned bufflen,
> +		unsigned timeout)
>  {
>  	struct request *req;
>  	int ret;
> @@ -127,6 +127,7 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		return PTR_ERR(req);
>  
>  	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
> +	req->special = cqe;
>  
>  	if (buffer && bufflen) {
>  		ret = blk_rq_map_kern(q, req, buffer, bufflen, GFP_KERNEL);
> @@ -135,8 +136,6 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>  	}
>  
>  	blk_execute_rq(req->q, NULL, req, 0);
> -	if (result)
> -		*result = (u32)(uintptr_t)req->special;
>  	ret = req->errors;
>   out:
>  	blk_mq_free_request(req);
> @@ -146,7 +145,7 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>  int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		void *buffer, unsigned bufflen)
>  {
> -	return __nvme_submit_sync_cmd(q, cmd, buffer, bufflen, NULL, 0);
> +	return __nvme_submit_sync_cmd(q, cmd, NULL, buffer, bufflen, 0);
>  }
>  
>  int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
> @@ -155,6 +154,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		u32 *result, unsigned timeout)
>  {
>  	bool write = cmd->common.opcode & 1;
> +	struct nvme_completion cqe;
>  	struct nvme_ns *ns = q->queuedata;
>  	struct gendisk *disk = ns ? ns->disk : NULL;
>  	struct request *req;
> @@ -167,6 +167,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		return PTR_ERR(req);
>  
>  	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
> +	req->special = &cqe;
>  
>  	if (ubuffer && bufflen) {
>  		ret = blk_rq_map_user(q, req, NULL, ubuffer, bufflen,
> @@ -221,7 +222,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
>  	blk_execute_rq(req->q, disk, req, 0);
>  	ret = req->errors;
>  	if (result)
> -		*result = (u32)(uintptr_t)req->special;
> +		*result = le32_to_cpu(cqe.result);
>  	if (meta && !ret && !write) {
>  		if (copy_to_user(meta_buffer, meta, meta_len))
>  			ret = -EFAULT;
> @@ -302,6 +303,8 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
>  					dma_addr_t dma_addr, u32 *result)
>  {
>  	struct nvme_command c;
> +	struct nvme_completion cqe;
> +	int ret;
>  
>  	memset(&c, 0, sizeof(c));
>  	c.features.opcode = nvme_admin_get_features;
> @@ -309,13 +312,18 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
>  	c.features.prp1 = cpu_to_le64(dma_addr);
>  	c.features.fid = cpu_to_le32(fid);
>  
> -	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
> +	ret = __nvme_submit_sync_cmd(dev->admin_q, &c, &cqe, NULL, 0, 0);
> +	if (ret >= 0)
> +		*result = le32_to_cpu(cqe.result);
> +	return ret;
>  }
>  
>  int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
>  					dma_addr_t dma_addr, u32 *result)
>  {
>  	struct nvme_command c;
> +	struct nvme_completion cqe;
> +	int ret;
>  
>  	memset(&c, 0, sizeof(c));
>  	c.features.opcode = nvme_admin_set_features;
> @@ -323,7 +331,10 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
>  	c.features.fid = cpu_to_le32(fid);
>  	c.features.dword11 = cpu_to_le32(dword11);
>  
> -	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
> +	ret = __nvme_submit_sync_cmd(dev->admin_q, &c, &cqe, NULL, 0, 0);
> +	if (ret >= 0)
> +		*result = le32_to_cpu(cqe.result);
> +	return ret;
>  }
>  
>  int nvme_get_log_page(struct nvme_ctrl *dev, struct nvme_smart_log **log)
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 4fb5bb7..d85997f 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -247,7 +247,8 @@ void nvme_requeue_req(struct request *req);
>  int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		void *buf, unsigned bufflen);
>  int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
> -		void *buffer, unsigned bufflen,  u32 *result, unsigned timeout);
> +		struct nvme_completion *cqe, void *buffer, unsigned bufflen,
> +		unsigned timeout);
>  int nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		void __user *ubuffer, unsigned bufflen, u32 *result,
>  		unsigned timeout);
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 72ef832..f8ae20c 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -759,10 +759,8 @@ static void __nvme_process_cq(struct nvme_queue *nvmeq, unsigned int *tag)
>  		}
>  
>  		req = blk_mq_tag_to_rq(*nvmeq->tags, cqe.command_id);
> -		if (req->cmd_type == REQ_TYPE_DRV_PRIV) {
> -			u32 result = le32_to_cpu(cqe.result);
> -			req->special = (void *)(uintptr_t)result;
> -		}
> +		if (req->cmd_type == REQ_TYPE_DRV_PRIV && req->special)
> +			memcpy(req->special, &cqe, sizeof(cqe));
>  		blk_mq_complete_request(req, status >> 1);
>  
>  	}
> @@ -905,12 +903,10 @@ static void abort_endio(struct request *req, int error)
>  {
>  	struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
>  	struct nvme_queue *nvmeq = iod->nvmeq;
> -	u32 result = (u32)(uintptr_t)req->special;
>  	u16 status = req->errors;
>  
> -	dev_warn(nvmeq->q_dmadev, "Abort status:%x result:%x", status, result);
> +	dev_warn(nvmeq->q_dmadev, "Abort status 0x%x\n", status);
>  	atomic_inc(&nvmeq->dev->ctrl.abort_limit);
> -
>  	blk_mq_free_request(req);
>  }
>  
> 

Thanks.

Reviewed-by: Matias Bj?rling <m at bjorling.me>


From hch at lst.de  Tue Feb  9 05:07:21 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 14:07:21 +0100
Subject: [PATCH 2/4] nvme: export more symbols
In-Reply-To: <56B9E204.4080702@bjorling.me>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-3-git-send-email-mlin@kernel.org>
 <56B9E204.4080702@bjorling.me>
Message-ID: <20160209130721.GA25715@lst.de>

> Can these be exported without _GPL?

No way.  I consider every Linux module a derivative work, but anything
poking into the NVMe internals surely is much as it can.  We better
keep this clear warning sign.


From hch at lst.de  Tue Feb  9 05:14:08 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 14:14:08 +0100
Subject: [PATCH 3/4] nvme: split dev_list_lock
In-Reply-To: <20160209124134.GM27969@c203.arch.suse.de>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-4-git-send-email-mlin@kernel.org>
 <20160209124134.GM27969@c203.arch.suse.de>
Message-ID: <20160209131408.GA25813@lst.de>

On Tue, Feb 09, 2016 at 01:41:34PM +0100, Johannes Thumshirn wrote:
> On Mon, Feb 08, 2016 at 02:24:43PM -0800, Ming Lin wrote:
> > Split dev_list_lock into one in the core and one in the PCI driver.
> 
> The dev_list_lock in core.c doesn't really protect the dev_list, like it does
> in pci.c. Wouldn't it be better to rename it to something more appropriate? I
> just can't come up with a name, as it seems to be some kind of catch all lock.

I plans to remove the core.c dev_list_lock by rewriting the character
device interface code.  Let's keep the name for now, it'll go away.

(and the pci.c might be gone even sooner, but at least it has the right name
:))


From axboe at kernel.dk  Tue Feb  9 05:29:38 2016
From: axboe at kernel.dk (Jens Axboe)
Date: Tue, 9 Feb 2016 06:29:38 -0700
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <20160209125011.GC25353@lst.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <20160209125011.GC25353@lst.de>
Message-ID: <CB24E0BA-5A48-4D57-9B81-A81F6598F32F@kernel.dk>

On Feb 9, 2016, at 5:50 AM, Christoph Hellwig <hch at lst.de> wrote:
> 
> Jens,
> 
> do you want a 'default y' patch or just a better description?  I'd be
> happy to send either one.

A better description  

-- 
Jens Axboe


From jthumshirn at suse.de  Tue Feb  9 05:34:12 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Tue, 9 Feb 2016 14:34:12 +0100
Subject: [PATCH 3/4] nvme: split dev_list_lock
In-Reply-To: <20160209131408.GA25813@lst.de>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-4-git-send-email-mlin@kernel.org>
 <20160209124134.GM27969@c203.arch.suse.de>
 <20160209131408.GA25813@lst.de>
Message-ID: <20160209133412.GP27969@c203.arch.suse.de>

On Tue, Feb 09, 2016 at 02:14:08PM +0100, Christoph Hellwig wrote:
> On Tue, Feb 09, 2016 at 01:41:34PM +0100, Johannes Thumshirn wrote:
> > On Mon, Feb 08, 2016 at 02:24:43PM -0800, Ming Lin wrote:
> > > Split dev_list_lock into one in the core and one in the PCI driver.
> > 
> > The dev_list_lock in core.c doesn't really protect the dev_list, like it does
> > in pci.c. Wouldn't it be better to rename it to something more appropriate? I
> > just can't come up with a name, as it seems to be some kind of catch all lock.
> 
> I plans to remove the core.c dev_list_lock by rewriting the character
> device interface code.  Let's keep the name for now, it'll go away.
> 
> (and the pci.c might be gone even sooner, but at least it has the right name
> :))

OK, then I'm fine with it.

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From snitzer at redhat.com  Tue Feb  9 06:55:47 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Tue, 9 Feb 2016 09:55:47 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56B99A49.5050400@suse.de>
References: <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com>
 <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com> <56B99A49.5050400@suse.de>
Message-ID: <20160209145547.GA21623@redhat.com>

On Tue, Feb 09 2016 at  2:50am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/07/2016 06:20 PM, Mike Snitzer wrote:
> > On Sun, Feb 07 2016 at 11:54am -0500,
> > Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:
> > 
> >>
> >>>> If so, can you check with e.g.
> >>>> perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
> >>>> workload triggers perhaps lock contention ? What you need to look for in
> >>>> the perf output is whether any functions occupy more than 10% CPU time.
> >>>
> >>> I will, thanks for the tip!
> >>
> >> The perf report is very similar to the one that started this effort..
> >>
> >> I'm afraid we'll need to resolve the per-target m->lock in order
> >> to scale with NUMA...
> > 
> > Could be.  Just for testing, you can try the 2 topmost commits I've put
> > here (once applied both __multipath_map and multipath_busy won't have
> > _any_ locking.. again, very much test-only):
> > 
> > http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel2
> > 
> So, I gave those patches a spin.
> Sad to say, they do _not_ resolve the issue fully.
>
> My testbed (2 paths per LUN, 40 CPUs, 4 cores) yields 505k IOPs with
> those patches.

That isn't a surprise.  We knew the m->lock spinlock contention to be a
problem.  And NUMA makes it even worse.

> Using a single path (without those patches, but still running
> multipath on top of that path) the same testbed yields 550k IOPs.
> Which very much smells like a lock contention ...
> We do get a slight improvement, though; without those patches I
> could only get about 350k IOPs. But still, I would somehow expect 2
> paths to be faster than just one ..

https://www.redhat.com/archives/dm-devel/2016-February/msg00036.html

hint hint...


From hare at suse.de  Tue Feb  9 07:32:25 2016
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 9 Feb 2016 16:32:25 +0100
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <20160209145547.GA21623@redhat.com>
References: <20160204135420.GA18227@redhat.com>
 <20160205151334.GA82754@redhat.com> <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com> <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com> <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il> <20160207172055.GA6477@redhat.com>
 <56B99A49.5050400@suse.de> <20160209145547.GA21623@redhat.com>
Message-ID: <56BA0689.9030007@suse.de>

On 02/09/2016 03:55 PM, Mike Snitzer wrote:
> On Tue, Feb 09 2016 at  2:50am -0500,
> Hannes Reinecke <hare at suse.de> wrote:
> 
>> On 02/07/2016 06:20 PM, Mike Snitzer wrote:
>>> On Sun, Feb 07 2016 at 11:54am -0500,
>>> Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:
>>>
>>>>
>>>>>> If so, can you check with e.g.
>>>>>> perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
>>>>>> workload triggers perhaps lock contention ? What you need to look for in
>>>>>> the perf output is whether any functions occupy more than 10% CPU time.
>>>>>
>>>>> I will, thanks for the tip!
>>>>
>>>> The perf report is very similar to the one that started this effort..
>>>>
>>>> I'm afraid we'll need to resolve the per-target m->lock in order
>>>> to scale with NUMA...
>>>
>>> Could be.  Just for testing, you can try the 2 topmost commits I've put
>>> here (once applied both __multipath_map and multipath_busy won't have
>>> _any_ locking.. again, very much test-only):
>>>
>>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel2
>>>
>> So, I gave those patches a spin.
>> Sad to say, they do _not_ resolve the issue fully.
>>
>> My testbed (2 paths per LUN, 40 CPUs, 4 cores) yields 505k IOPs with
>> those patches.
> 
> That isn't a surprise.  We knew the m->lock spinlock contention to be a
> problem.  And NUMA makes it even worse.
> 
>> Using a single path (without those patches, but still running
>> multipath on top of that path) the same testbed yields 550k IOPs.
>> Which very much smells like a lock contention ...
>> We do get a slight improvement, though; without those patches I
>> could only get about 350k IOPs. But still, I would somehow expect 2
>> paths to be faster than just one ..
> 
> https://www.redhat.com/archives/dm-devel/2016-February/msg00036.html
> 
> hint hint...
> 
I hoped they wouldn't be needed with your patches.
Plus perf revealed that I first need to address a spinlock
contention in the lpfc driver before that even would make sense.

So more debugging to follow.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From James.Bottomley at HansenPartnership.com  Tue Feb  9 07:37:37 2016
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Tue, 09 Feb 2016 07:37:37 -0800
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <20160209125011.GC25353@lst.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com>
 <56B7C527.6050300@kernel.dk> <20160209125011.GC25353@lst.de>
Message-ID: <1455032257.2340.1.camel@HansenPartnership.com>

On Tue, 2016-02-09 at 13:50 +0100, Christoph Hellwig wrote:
> Jens,
> 
> do you want a 'default y' patch or just a better description?  I'd be
> happy to send either one.

Since it only appears to be SUSE and they've now been told, better
description is fine.

James




From hch at lst.de  Tue Feb  9 09:12:29 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 18:12:29 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <CB24E0BA-5A48-4D57-9B81-A81F6598F32F@kernel.dk>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <20160209125011.GC25353@lst.de>
 <CB24E0BA-5A48-4D57-9B81-A81F6598F32F@kernel.dk>
Message-ID: <20160209171229.GA30378@lst.de>

Does this looks reasonable?

---
>From 7843fae979df3fc14007735f54cc6bb2f6f66dc5 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <hch at lst.de>
Date: Tue, 9 Feb 2016 18:11:32 +0100
Subject: nvme: fix Kconfig description for BLK_DEV_NVME_SCSI

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/Kconfig | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index 59307f8..2e24156 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -17,8 +17,8 @@ config BLK_DEV_NVME_SCSI
 	  and block devices nodes, as well a a translation for a small
 	  number of selected SCSI commands to NVMe commands to the NVMe
 	  driver.  If you don't know what this means you probably want
-	  to say N here, and if you know what it means you probably
-	  want to say N as well.
+	  to say N here, unless you run a distro that abuses this for
+	  stable device names like some OpenSuSE and SLES versions.
 
 config NVME_FABRICS
 	tristate
-- 
2.1.4



From axboe at kernel.dk  Tue Feb  9 09:14:04 2016
From: axboe at kernel.dk (Jens Axboe)
Date: Tue, 9 Feb 2016 10:14:04 -0700
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <20160209171229.GA30378@lst.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <20160209125011.GC25353@lst.de>
 <CB24E0BA-5A48-4D57-9B81-A81F6598F32F@kernel.dk>
 <20160209171229.GA30378@lst.de>
Message-ID: <56BA1E5C.7040108@kernel.dk>

On 02/09/2016 10:12 AM, Christoph Hellwig wrote:
> Does this looks reasonable?
>
> ---
>  From 7843fae979df3fc14007735f54cc6bb2f6f66dc5 Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <hch at lst.de>
> Date: Tue, 9 Feb 2016 18:11:32 +0100
> Subject: nvme: fix Kconfig description for BLK_DEV_NVME_SCSI
>
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>   drivers/nvme/host/Kconfig | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
> index 59307f8..2e24156 100644
> --- a/drivers/nvme/host/Kconfig
> +++ b/drivers/nvme/host/Kconfig
> @@ -17,8 +17,8 @@ config BLK_DEV_NVME_SCSI
>   	  and block devices nodes, as well a a translation for a small
>   	  number of selected SCSI commands to NVMe commands to the NVMe
>   	  driver.  If you don't know what this means you probably want
> -	  to say N here, and if you know what it means you probably
> -	  want to say N as well.
> +	  to say N here, unless you run a distro that abuses this for
> +	  stable device names like some OpenSuSE and SLES versions.

Yep, that looks a lot more reasonable to me. Might be wort including 
that it impacts the mount-by-id on those distros.

-- 
Jens Axboe



From hch at lst.de  Tue Feb  9 09:19:11 2016
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2016 18:19:11 +0100
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <56BA1E5C.7040108@kernel.dk>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <20160209125011.GC25353@lst.de>
 <CB24E0BA-5A48-4D57-9B81-A81F6598F32F@kernel.dk>
 <20160209171229.GA30378@lst.de> <56BA1E5C.7040108@kernel.dk>
Message-ID: <20160209171911.GA30623@lst.de>

Updated version below:

---
>From d63251560cf2670badbc86c83502502f29c087e0 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <hch at lst.de>
Date: Tue, 9 Feb 2016 18:11:32 +0100
Subject: nvme: fix Kconfig description for BLK_DEV_NVME_SCSI

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/Kconfig | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index 59307f8..68fa858 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -17,8 +17,9 @@ config BLK_DEV_NVME_SCSI
 	  and block devices nodes, as well a a translation for a small
 	  number of selected SCSI commands to NVMe commands to the NVMe
 	  driver.  If you don't know what this means you probably want
-	  to say N here, and if you know what it means you probably
-	  want to say N as well.
+	  to say N here, unless you run a distro that abuses the SCSI
+	  emulation to provide stable device names for mount by id like
+	  some OpenSuSE and SLES versions.
 
 config NVME_FABRICS
 	tristate
-- 
2.1.4



From axboe at kernel.dk  Tue Feb  9 09:19:54 2016
From: axboe at kernel.dk (Jens Axboe)
Date: Tue, 9 Feb 2016 10:19:54 -0700
Subject: complete boot failure in 4.5-rc1 caused by nvme: make SG_IO
 support optional
In-Reply-To: <20160209171911.GA30623@lst.de>
References: <1454783624.2809.6.camel@HansenPartnership.com>
 <20160207092241.GA15331@lst.de>
 <1454861040.2329.4.camel@HansenPartnership.com> <56B7C527.6050300@kernel.dk>
 <20160209125011.GC25353@lst.de>
 <CB24E0BA-5A48-4D57-9B81-A81F6598F32F@kernel.dk>
 <20160209171229.GA30378@lst.de> <56BA1E5C.7040108@kernel.dk>
 <20160209171911.GA30623@lst.de>
Message-ID: <56BA1FBA.5080508@kernel.dk>

On 02/09/2016 10:19 AM, Christoph Hellwig wrote:
> Updated version below:
>
> ---
>  From d63251560cf2670badbc86c83502502f29c087e0 Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <hch at lst.de>
> Date: Tue, 9 Feb 2016 18:11:32 +0100
> Subject: nvme: fix Kconfig description for BLK_DEV_NVME_SCSI
>
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>   drivers/nvme/host/Kconfig | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
> index 59307f8..68fa858 100644
> --- a/drivers/nvme/host/Kconfig
> +++ b/drivers/nvme/host/Kconfig
> @@ -17,8 +17,9 @@ config BLK_DEV_NVME_SCSI
>   	  and block devices nodes, as well a a translation for a small
>   	  number of selected SCSI commands to NVMe commands to the NVMe
>   	  driver.  If you don't know what this means you probably want
> -	  to say N here, and if you know what it means you probably
> -	  want to say N as well.
> +	  to say N here, unless you run a distro that abuses the SCSI
> +	  emulation to provide stable device names for mount by id like
> +	  some OpenSuSE and SLES versions.

Thanks, looks good to me, I'll fold it in for the current series.

-- 
Jens Axboe



From mlin at kernel.org  Tue Feb  9 10:26:45 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue, 09 Feb 2016 10:26:45 -0800
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <20160209085656.GJ27969@c203.arch.suse.de>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-5-git-send-email-mlin@kernel.org>
 <20160209085656.GJ27969@c203.arch.suse.de>
Message-ID: <1455042405.8832.4.camel@ssi>

On Tue, 2016-02-09 at 09:56 +0100, Johannes Thumshirn wrote:
> On Mon, Feb 08, 2016 at 02:24:44PM -0800, Ming Lin wrote:
> > This splits nvme.ko into 2 modules:
> > nvme-core.ko: the core part
> > nvme.ko: the PCI driver
> > 
> > Also changes config name:
> > s/CONFIG_BLK_DEV_NVME/CONFIG_NVME_PCI
> > s/CONFIG_BLK_DEV_NVME_SCSI/CONFIG_NVME_SCSI
> 
> Is there any auto-migration for those who already have CONFIG_BLK_DEV_NVME set
> in their kernel configs? If not, this could lead to a bunch of non-booting
> systems.

Hi Michal(CCed),

As you saw above, I'm changing:
CONFIG_BLK_DEV_NVME to CONFIG_NVME_PCI
and 
CONFIG_BLK_DEV_NVME_SCSI to CONFIG_NVME_SCSI

Is there Kconfig magic to keep existing config working?
ie, if CONFIG_BLK_DEV_NVME was set in .config, then CONFIG_NVME_PCI
should be set automatically?

Thanks,
Ming





From axboe at fb.com  Tue Feb  9 11:39:52 2016
From: axboe at fb.com (Jens Axboe)
Date: Tue, 9 Feb 2016 12:39:52 -0700
Subject: [PATCH 2/2] NVMe: Fix possible queue use after freed
In-Reply-To: <20160208192848.GA8151@localhost.localdomain>
References: <1450397295-24031-1-git-send-email-keith.busch@intel.com>
 <1450397295-24031-2-git-send-email-keith.busch@intel.com>
 <20160208192848.GA8151@localhost.localdomain>
Message-ID: <56BA4088.40001@fb.com>

On 02/08/2016 12:28 PM, Jon Derrick wrote:
> Hi Jens,
>
> This set was acked a while back by Christoph and reviewed by me.
> I just tested it again and it still works well against 4.5-rc3.
>
> Is there any chance we can get this in for 4.6?

Yes, it will go into 4.6.

-- 
Jens Axboe



From axboe at kernel.dk  Tue Feb  9 11:44:28 2016
From: axboe at kernel.dk (Jens Axboe)
Date: Tue, 9 Feb 2016 12:44:28 -0700
Subject: [PATCH] nvme: fix drvdata setup for the nvme device
In-Reply-To: <1454954856-32236-1-git-send-email-hch@lst.de>
References: <1454954856-32236-1-git-send-email-hch@lst.de>
Message-ID: <56BA419C.1010806@kernel.dk>

On 02/08/2016 11:07 AM, Christoph Hellwig wrote:
> Pass the right private data to device_create_with_groups from the
> beginning, and remove the superflous call to dev_set_drvdata.

Added for 4.6.

-- 
Jens Axboe



From keith.busch at intel.com  Tue Feb  9 14:55:49 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 9 Feb 2016 22:55:49 +0000
Subject: [PATCH] NVMe: do not touch sq door bell if nvmeq has been
 suspended
In-Reply-To: <BJXPR01MB199E172F7B186281D1C7179E0D60@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1454341324-21273-1-git-send-email-mail_weber_wang@163.com>
 <56AF8DB5.70206@fb.com>
 <BJXPR01MB19940BA759DBBADC6BF6BD2E0DF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203144123.GB23910@localhost.localdomain>
 <BJXPR01MB1992FFD0706FCD368F5AD16E0D00@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160203163808.GC23910@localhost.localdomain>
 <BJXPR01MB19916231021454BD8731EBDE0D30@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160208150132.GA25111@localhost.localdomain>
 <BJXPR01MB199E172F7B186281D1C7179E0D60@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160209225548.GA32064@localhost.localdomain>

On Tue, Feb 09, 2016 at 11:22:04AM +0000, Wenbo Wang wrote:
> In most cases, rcu read lock is just a preempt_disable, which is what get_cpu does. I don't see any risk.

Yes, many rcu_read_lock cases expand similarly to get_cpu. What about
the other cases?

FWIW, I don't think we'll hit the problem with the proposed rcu sync.
Heck, we don't even have a synchronize today and we don't hit the
theoretical problem.


From mlin at kernel.org  Tue Feb  9 16:07:48 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue,  9 Feb 2016 16:07:48 -0800
Subject: [PATCH v2 0/5] split pci module out of core module
Message-ID: <1455062873-18392-1-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

We'll have nvme over fabric dirver soon.
This series split nvme pci module out of nvme core module.

nvme.ko is split into 2 modules:
nvme-core.ko: the core part
nvme.ko: the PCI driver

v2:
  - add Sagi's patch to take proper references to the transport driver on open
  - add review tags
  - don't change the CONFIG name

Ming Lin (4):
  nvme: move timeout variables to core.c and export it
  nvme: export more functions
  nvme: split dev_list_lock
  nvme: split pci module out of core module

Sagi Grimberg (1):
  nvme/host: reference the fabric module for each bdev open callout

 drivers/nvme/host/Kconfig  |  6 ++++-
 drivers/nvme/host/Makefile | 10 ++++----
 drivers/nvme/host/core.c   | 57 ++++++++++++++++++++++++++++++++++++++++++----
 drivers/nvme/host/nvme.h   |  3 +--
 drivers/nvme/host/pci.c    | 27 +++-------------------
 5 files changed, 67 insertions(+), 36 deletions(-)

-- 
1.9.1



From mlin at kernel.org  Tue Feb  9 16:07:50 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue,  9 Feb 2016 16:07:50 -0800
Subject: [PATCH v2 2/5] nvme: move timeout variables to core.c and export it
In-Reply-To: <1455062873-18392-1-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
Message-ID: <1455062873-18392-3-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

These variables are used by PCI driver and will also be used in the
forthcoming NVMe over Fabric drivers.

Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 14 ++++++++++++++
 drivers/nvme/host/pci.c  | 12 ------------
 2 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 8adc598..165ef54 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -33,6 +33,20 @@
 
 #define NVME_MINORS		(1U << MINORBITS)
 
+unsigned char admin_timeout = 60;
+module_param(admin_timeout, byte, 0644);
+MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
+EXPORT_SYMBOL_GPL(admin_timeout);
+
+unsigned char nvme_io_timeout = 30;
+module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
+MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
+EXPORT_SYMBOL_GPL(nvme_io_timeout);
+
+unsigned char shutdown_timeout = 5;
+module_param(shutdown_timeout, byte, 0644);
+MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
+
 static int nvme_major;
 module_param(nvme_major, int, 0);
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e36ca02..e3552d1 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -57,18 +57,6 @@
 #define NVME_NR_AEN_COMMANDS	1
 #define NVME_AQ_BLKMQ_DEPTH	(NVME_AQ_DEPTH - NVME_NR_AEN_COMMANDS)
 
-unsigned char admin_timeout = 60;
-module_param(admin_timeout, byte, 0644);
-MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
-
-unsigned char nvme_io_timeout = 30;
-module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
-MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
-
-unsigned char shutdown_timeout = 5;
-module_param(shutdown_timeout, byte, 0644);
-MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
-
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 
-- 
1.9.1



From mlin at kernel.org  Tue Feb  9 16:07:49 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue,  9 Feb 2016 16:07:49 -0800
Subject: [PATCH v2 1/5] nvme/host: reference the fabric module for each bdev
 open callout
In-Reply-To: <1455062873-18392-1-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
Message-ID: <1455062873-18392-2-git-send-email-mlin@kernel.org>

From: Sagi Grimberg <sagig at mellanox.com>

We don't want to be able to unload the fabric driver when we have
openened referenced to our namespaces. Thus, for each nvme_open we
take a reference on the fabric driver and put it in nvme_release.
This behavior is consistent with the scsi model.

This resolves the panic when unloading a fabric module with
mpath holders.

Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Ian Bakshan <ianb at mellanox.com>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 19 ++++++++++++++++---
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/pci.c  |  1 +
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c5bf001..8adc598 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -71,11 +71,21 @@ static struct nvme_ns *nvme_get_ns_from_disk(struct gendisk *disk)
 
 	spin_lock(&dev_list_lock);
 	ns = disk->private_data;
-	if (ns && !kref_get_unless_zero(&ns->kref))
-		ns = NULL;
+	if (ns) {
+		if (!kref_get_unless_zero(&ns->kref))
+			goto fail;
+		if (!try_module_get(ns->ctrl->ops->module))
+			goto fail_put_ns;
+	}
 	spin_unlock(&dev_list_lock);
 
 	return ns;
+
+fail_put_ns:
+	kref_put(&ns->kref, nvme_free_ns);
+fail:
+	spin_unlock(&dev_list_lock);
+	return NULL;
 }
 
 void nvme_requeue_req(struct request *req)
@@ -499,7 +509,10 @@ static int nvme_open(struct block_device *bdev, fmode_t mode)
 
 static void nvme_release(struct gendisk *disk, fmode_t mode)
 {
-	nvme_put_ns(disk->private_data);
+	struct nvme_ns *ns = disk->private_data;
+
+	module_put(ns->ctrl->ops->module);
+	nvme_put_ns(ns);
 }
 
 static int nvme_getgeo(struct block_device *bdev, struct hd_geometry *geo)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4fb5bb7..9f77386 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -117,6 +117,7 @@ struct nvme_ns {
 };
 
 struct nvme_ctrl_ops {
+	struct module *module;
 	int (*reg_read32)(struct nvme_ctrl *ctrl, u32 off, u32 *val);
 	int (*reg_write32)(struct nvme_ctrl *ctrl, u32 off, u32 val);
 	int (*reg_read64)(struct nvme_ctrl *ctrl, u32 off, u64 *val);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 72ef832..e36ca02 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2029,6 +2029,7 @@ static int nvme_pci_reset_ctrl(struct nvme_ctrl *ctrl)
 }
 
 static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = {
+	.module			= THIS_MODULE,
 	.reg_read32		= nvme_pci_reg_read32,
 	.reg_write32		= nvme_pci_reg_write32,
 	.reg_read64		= nvme_pci_reg_read64,
-- 
1.9.1



From mlin at kernel.org  Tue Feb  9 16:07:51 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue,  9 Feb 2016 16:07:51 -0800
Subject: [PATCH v2 3/5] nvme: export more functions
In-Reply-To: <1455062873-18392-1-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
Message-ID: <1455062873-18392-4-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

These functions are used by PCI driver and will also be used in the
forthcoming NVMe over Fabric drivers.

Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 165ef54..1bba21f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -112,6 +112,7 @@ void nvme_requeue_req(struct request *req)
 		blk_mq_kick_requeue_list(req->q);
 	spin_unlock_irqrestore(req->q->queue_lock, flags);
 }
+EXPORT_SYMBOL_GPL(nvme_requeue_req);
 
 struct request *nvme_alloc_request(struct request_queue *q,
 		struct nvme_command *cmd, unsigned int flags)
@@ -135,6 +136,7 @@ struct request *nvme_alloc_request(struct request_queue *q,
 
 	return req;
 }
+EXPORT_SYMBOL_GPL(nvme_alloc_request);
 
 /*
  * Returns 0 on success.  If the result is negative, it's a Linux error code;
@@ -172,6 +174,7 @@ int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 {
 	return __nvme_submit_sync_cmd(q, cmd, buffer, bufflen, NULL, 0);
 }
+EXPORT_SYMBOL_GPL(nvme_submit_sync_cmd);
 
 int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void __user *ubuffer, unsigned bufflen,
@@ -387,6 +390,7 @@ int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count)
 	*count = min(*count, nr_io_queues);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nvme_set_queue_count);
 
 static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
 {
@@ -796,6 +800,7 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 		return ret;
 	return nvme_wait_ready(ctrl, cap, false);
 }
+EXPORT_SYMBOL_GPL(nvme_disable_ctrl);
 
 int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 {
@@ -827,6 +832,7 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 		return ret;
 	return nvme_wait_ready(ctrl, cap, true);
 }
+EXPORT_SYMBOL_GPL(nvme_enable_ctrl);
 
 int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
 {
@@ -857,6 +863,7 @@ int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
 
 	return ret;
 }
+EXPORT_SYMBOL_GPL(nvme_shutdown_ctrl);
 
 /*
  * Initialize the cached copies of the Identify data and various controller
@@ -918,6 +925,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	kfree(id);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nvme_init_identify);
 
 static int nvme_dev_open(struct inode *inode, struct file *file)
 {
@@ -1323,6 +1331,7 @@ void nvme_scan_namespaces(struct nvme_ctrl *ctrl)
 	mutex_unlock(&ctrl->namespaces_mutex);
 	kfree(id);
 }
+EXPORT_SYMBOL_GPL(nvme_scan_namespaces);
 
 void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 {
@@ -1333,6 +1342,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 		nvme_ns_remove(ns);
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
 
 static DEFINE_IDA(nvme_instance_ida);
 
@@ -1364,13 +1374,14 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
 }
 
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
- {
+{
 	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 
 	spin_lock(&dev_list_lock);
 	list_del(&ctrl->node);
 	spin_unlock(&dev_list_lock);
 }
+EXPORT_SYMBOL_GPL(nvme_uninit_ctrl);
 
 static void nvme_free_ctrl(struct kref *kref)
 {
@@ -1386,6 +1397,7 @@ void nvme_put_ctrl(struct nvme_ctrl *ctrl)
 {
 	kref_put(&ctrl->kref, nvme_free_ctrl);
 }
+EXPORT_SYMBOL_GPL(nvme_put_ctrl);
 
 /*
  * Initialize a NVMe controller structures.  This needs to be called during
@@ -1429,6 +1441,7 @@ out_release_instance:
 out:
 	return ret;
 }
+EXPORT_SYMBOL_GPL(nvme_init_ctrl);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
@@ -1445,6 +1458,7 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_stop_queues);
 
 void nvme_start_queues(struct nvme_ctrl *ctrl)
 {
@@ -1458,6 +1472,7 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_start_queues);
 
 int __init nvme_core_init(void)
 {
-- 
1.9.1



From mlin at kernel.org  Tue Feb  9 16:07:52 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue,  9 Feb 2016 16:07:52 -0800
Subject: [PATCH v2 4/5] nvme: split dev_list_lock
In-Reply-To: <1455062873-18392-1-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
Message-ID: <1455062873-18392-5-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

Split dev_list_lock into one in the core and one in the PCI driver.

Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 2 +-
 drivers/nvme/host/nvme.h | 2 --
 drivers/nvme/host/pci.c  | 1 +
 3 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1bba21f..740615b 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -54,7 +54,7 @@ static int nvme_char_major;
 module_param(nvme_char_major, int, 0);
 
 static LIST_HEAD(nvme_ctrl_list);
-DEFINE_SPINLOCK(dev_list_lock);
+static DEFINE_SPINLOCK(dev_list_lock);
 
 static struct class *nvme_class;
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9f77386..63ba8a5 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -266,8 +266,6 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 			dma_addr_t dma_addr, u32 *result);
 int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count);
 
-extern spinlock_t dev_list_lock;
-
 struct sg_io_hdr;
 
 int nvme_sg_io(struct nvme_ns *ns, struct sg_io_hdr __user *u_hdr);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e3552d1..f7266cc 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -65,6 +65,7 @@ module_param(use_cmb_sqes, bool, 0644);
 MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
 
 static LIST_HEAD(dev_list);
+static DEFINE_SPINLOCK(dev_list_lock);
 static struct task_struct *nvme_thread;
 static struct workqueue_struct *nvme_workq;
 static wait_queue_head_t nvme_kthread_wait;
-- 
1.9.1



From mlin at kernel.org  Tue Feb  9 16:07:53 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue,  9 Feb 2016 16:07:53 -0800
Subject: [PATCH v2 5/5] nvme: split pci module out of core module
In-Reply-To: <1455062873-18392-1-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
Message-ID: <1455062873-18392-6-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

NVMe over Fabrics drivers are going to reuse the core,
so splits nvme.ko into 2 modules:

nvme-core.ko: the core part
nvme.ko: the PCI driver

Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/Kconfig  |  6 +++++-
 drivers/nvme/host/Makefile | 10 ++++++----
 drivers/nvme/host/core.c   |  5 +++++
 drivers/nvme/host/pci.c    | 13 +------------
 4 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index 5d62373..2ed30f0 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -1,6 +1,10 @@
+config NVME_CORE
+	tristate
+
 config BLK_DEV_NVME
 	tristate "NVM Express block device"
 	depends on PCI && BLOCK
+	select NVME_CORE
 	---help---
 	  The NVM Express driver is for solid state drives directly
 	  connected to the PCI or PCI Express bus.  If you know you
@@ -11,7 +15,7 @@ config BLK_DEV_NVME
 
 config BLK_DEV_NVME_SCSI
 	bool "SCSI emulation for NVMe device nodes"
-	depends on BLK_DEV_NVME
+	depends on NVME_CORE
 	---help---
 	  This adds support for the SG_IO ioctl on the NVMe character
 	  and block devices nodes, as well a a translation for a small
diff --git a/drivers/nvme/host/Makefile b/drivers/nvme/host/Makefile
index 51bf908..9a3ca89 100644
--- a/drivers/nvme/host/Makefile
+++ b/drivers/nvme/host/Makefile
@@ -1,6 +1,8 @@
+obj-$(CONFIG_NVME_CORE)			+= nvme-core.o
+obj-$(CONFIG_BLK_DEV_NVME)		+= nvme.o
 
-obj-$(CONFIG_BLK_DEV_NVME)     += nvme.o
+nvme-core-y				:= core.o
+nvme-core-$(CONFIG_BLK_DEV_NVME_SCSI)	+= scsi.o
+nvme-core-$(CONFIG_NVM)			+= lightnvm.o
 
-lightnvm-$(CONFIG_NVM)			:= lightnvm.o
-nvme-y					+= core.o pci.o $(lightnvm-y)
-nvme-$(CONFIG_BLK_DEV_NVME_SCSI)        += scsi.o
+nvme-y					+= pci.o
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 740615b..c7df0ab 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1512,3 +1512,8 @@ void nvme_core_exit(void)
 	class_destroy(nvme_class);
 	__unregister_chrdev(nvme_char_major, 0, NVME_MINORS, "nvme");
 }
+
+MODULE_LICENSE("GPL");
+MODULE_VERSION("1.0");
+module_init(nvme_core_init);
+module_exit(nvme_core_exit);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index f7266cc..a5a4567 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2221,26 +2221,15 @@ static int __init nvme_init(void)
 	if (!nvme_workq)
 		return -ENOMEM;
 
-	result = nvme_core_init();
-	if (result < 0)
-		goto kill_workq;
-
 	result = pci_register_driver(&nvme_driver);
 	if (result)
-		goto core_exit;
-	return 0;
-
- core_exit:
-	nvme_core_exit();
- kill_workq:
-	destroy_workqueue(nvme_workq);
+		destroy_workqueue(nvme_workq);
 	return result;
 }
 
 static void __exit nvme_exit(void)
 {
 	pci_unregister_driver(&nvme_driver);
-	nvme_core_exit();
 	destroy_workqueue(nvme_workq);
 	BUG_ON(nvme_thread && !IS_ERR(nvme_thread));
 	_nvme_check_size();
-- 
1.9.1



From mlin at kernel.org  Tue Feb  9 16:09:19 2016
From: mlin at kernel.org (Ming Lin)
Date: Tue, 9 Feb 2016 16:09:19 -0800
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <20160209092650.GE21250@lst.de>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-5-git-send-email-mlin@kernel.org>
 <20160209092650.GE21250@lst.de>
Message-ID: <CAF1ivSbi4ujnBkcmEpvW3jrJcz_g4fofHrrTmO-N5c7nkwz13A@mail.gmail.com>

On Tue, Feb 9, 2016 at 1:26 AM, Christoph Hellwig <hch at lst.de> wrote:
> Looks fine,
>
> but as Johannes poined out we'll probably need some Kconfig magic
> to keep existing configfs working.

I didn't find the magic.
So just keep the old CONFIG name in v2 patches.


From snitzer at redhat.com  Tue Feb  9 16:45:19 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Tue, 9 Feb 2016 19:45:19 -0500
Subject: [RFC PATCH] dm: fix excessive dm-mq context switching
In-Reply-To: <56BA0689.9030007@suse.de>
References: <20160205180515.GA25808@redhat.com>
 <20160205191909.GA25982@redhat.com>
 <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com> <56B99A49.5050400@suse.de>
 <20160209145547.GA21623@redhat.com> <56BA0689.9030007@suse.de>
Message-ID: <20160210004518.GA23646@redhat.com>

On Tue, Feb 09 2016 at 10:32am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/09/2016 03:55 PM, Mike Snitzer wrote:
> > On Tue, Feb 09 2016 at  2:50am -0500,
> > Hannes Reinecke <hare at suse.de> wrote:
> > 
> >> On 02/07/2016 06:20 PM, Mike Snitzer wrote:
> >>> On Sun, Feb 07 2016 at 11:54am -0500,
> >>> Sagi Grimberg <sagig at dev.mellanox.co.il> wrote:
> >>>
> >>>>
> >>>>>> If so, can you check with e.g.
> >>>>>> perf record -ags -e LLC-load-misses sleep 10 && perf report whether this
> >>>>>> workload triggers perhaps lock contention ? What you need to look for in
> >>>>>> the perf output is whether any functions occupy more than 10% CPU time.
> >>>>>
> >>>>> I will, thanks for the tip!
> >>>>
> >>>> The perf report is very similar to the one that started this effort..
> >>>>
> >>>> I'm afraid we'll need to resolve the per-target m->lock in order
> >>>> to scale with NUMA...
> >>>
> >>> Could be.  Just for testing, you can try the 2 topmost commits I've put
> >>> here (once applied both __multipath_map and multipath_busy won't have
> >>> _any_ locking.. again, very much test-only):
> >>>
> >>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel2
> >>>
> >> So, I gave those patches a spin.
> >> Sad to say, they do _not_ resolve the issue fully.
> >>
> >> My testbed (2 paths per LUN, 40 CPUs, 4 cores) yields 505k IOPs with
> >> those patches.
> > 
> > That isn't a surprise.  We knew the m->lock spinlock contention to be a
> > problem.  And NUMA makes it even worse.
> > 
> >> Using a single path (without those patches, but still running
> >> multipath on top of that path) the same testbed yields 550k IOPs.
> >> Which very much smells like a lock contention ...
> >> We do get a slight improvement, though; without those patches I
> >> could only get about 350k IOPs. But still, I would somehow expect 2
> >> paths to be faster than just one ..
> > 
> > https://www.redhat.com/archives/dm-devel/2016-February/msg00036.html
> > 
> > hint hint...
> > 
> I hoped they wouldn't be needed with your patches.
> Plus perf revealed that I first need to address a spinlock
> contention in the lpfc driver before that even would make sense.
> 
> So more debugging to follow.

OK, I took a crack at embracing RCU.  Only slightly better performance
on my single NUMA node testbed.  (But I'll have to track down a system
with multiple NUMA nodes to do any justice to the next wave of this
optimization effort)

This RCU work is very heavy-handed and way too fiddley (there could
easily be bugs).  Anyway, please see:
http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel2&id=d80a7e4f8b5be9c81e4d452137623b003fa64745

But this might give you something to build on to arrive at something
more scalable?

Mike


From hch at lst.de  Wed Feb 10 00:21:46 2016
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2016 09:21:46 +0100
Subject: [PATCH v2 0/5] split pci module out of core module
In-Reply-To: <1455062873-18392-1-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
Message-ID: <20160210082146.GA9171@lst.de>

Patch 3 still says "export more symbols" while it exports the first
symbols.  Maybe it's better to just gold patch 3 into patch 5 where
the exports are use?

Otherwise looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From jthumshirn at suse.de  Wed Feb 10 01:18:22 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Wed, 10 Feb 2016 10:18:22 +0100
Subject: [PATCH v2 1/5] nvme/host: reference the fabric module for each
 bdev open callout
In-Reply-To: <1455062873-18392-2-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <1455062873-18392-2-git-send-email-mlin@kernel.org>
Message-ID: <20160210091822.GR27969@c203.arch.suse.de>

On Tue, Feb 09, 2016 at 04:07:49PM -0800, Ming Lin wrote:
> From: Sagi Grimberg <sagig at mellanox.com>
> 
> We don't want to be able to unload the fabric driver when we have
> openened referenced to our namespaces. Thus, for each nvme_open we
> take a reference on the fabric driver and put it in nvme_release.
> This behavior is consistent with the scsi model.
> 
> This resolves the panic when unloading a fabric module with
> mpath holders.
> 
> Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
> Reviewed-by: Christoph Hellwig <hch at lst.de>
> Reviewed-by: Ian Bakshan <ianb at mellanox.com>
> Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From sagig at dev.mellanox.co.il  Wed Feb 10 02:12:19 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Wed, 10 Feb 2016 12:12:19 +0200
Subject: [PATCH RESEND v1 0/3] Move active IO termination to the core
In-Reply-To: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
References: <1454417099-19332-1-git-send-email-sagig@mellanox.com>
Message-ID: <56BB0D03.8050301@dev.mellanox.co.il>

Jens, can you pick this up for 4.6?

This set got review tags from hch and ack from
keith on the nvme core changes.

Cheers,
Sagi.


From sagig at dev.mellanox.co.il  Wed Feb 10 02:14:05 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Wed, 10 Feb 2016 12:14:05 +0200
Subject: [PATCH v1] nvme: Log the ctrl device name instead of the
 undelying pci device name
In-Reply-To: <1453126453-17976-1-git-send-email-sagig@mellanox.com>
References: <1453126453-17976-1-git-send-email-sagig@mellanox.com>
Message-ID: <56BB0D6D.8050802@dev.mellanox.co.il>

Jens, care to pick this up for 4.6?

This patch got an ack from Keith and didn't
raise any objections so far...

Cheers,
Sagi.


From sagig at dev.mellanox.co.il  Wed Feb 10 02:37:02 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Wed, 10 Feb 2016 12:37:02 +0200
Subject: [PATCH v2 2/5] nvme: move timeout variables to core.c and export
 it
In-Reply-To: <1455062873-18392-3-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <1455062873-18392-3-git-send-email-mlin@kernel.org>
Message-ID: <56BB12CE.4060303@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Wed Feb 10 02:37:30 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Wed, 10 Feb 2016 12:37:30 +0200
Subject: [PATCH v2 3/5] nvme: export more functions
In-Reply-To: <1455062873-18392-4-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <1455062873-18392-4-git-send-email-mlin@kernel.org>
Message-ID: <56BB12EA.4000403@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Wed Feb 10 02:38:13 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Wed, 10 Feb 2016 12:38:13 +0200
Subject: [PATCH v2 4/5] nvme: split dev_list_lock
In-Reply-To: <1455062873-18392-5-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <1455062873-18392-5-git-send-email-mlin@kernel.org>
Message-ID: <56BB1315.80607@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Wed Feb 10 02:38:27 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Wed, 10 Feb 2016 12:38:27 +0200
Subject: [PATCH v2 5/5] nvme: split pci module out of core module
In-Reply-To: <1455062873-18392-6-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <1455062873-18392-6-git-send-email-mlin@kernel.org>
Message-ID: <56BB1323.7040905@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Wed Feb 10 02:38:43 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Wed, 10 Feb 2016 12:38:43 +0200
Subject: [PATCH v2 5/5] nvme: split pci module out of core module
In-Reply-To: <1455062873-18392-6-git-send-email-mlin@kernel.org>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <1455062873-18392-6-git-send-email-mlin@kernel.org>
Message-ID: <56BB1333.10303@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From mmarek at suse.com  Wed Feb 10 05:35:53 2016
From: mmarek at suse.com (Michal Marek)
Date: Wed, 10 Feb 2016 14:35:53 +0100
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <1455042405.8832.4.camel@ssi>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-5-git-send-email-mlin@kernel.org>
 <20160209085656.GJ27969@c203.arch.suse.de> <1455042405.8832.4.camel@ssi>
Message-ID: <56BB3CB9.2070608@suse.com>

On 2016-02-09 19:26, Ming Lin wrote:
> On Tue, 2016-02-09 at 09:56 +0100, Johannes Thumshirn wrote:
>> On Mon, Feb 08, 2016 at 02:24:44PM -0800, Ming Lin wrote:
>>> This splits nvme.ko into 2 modules:
>>> nvme-core.ko: the core part
>>> nvme.ko: the PCI driver
>>>
>>> Also changes config name:
>>> s/CONFIG_BLK_DEV_NVME/CONFIG_NVME_PCI
>>> s/CONFIG_BLK_DEV_NVME_SCSI/CONFIG_NVME_SCSI
>>
>> Is there any auto-migration for those who already have CONFIG_BLK_DEV_NVME set
>> in their kernel configs? If not, this could lead to a bunch of non-booting
>> systems.
> 
> Hi Michal(CCed),
> 
> As you saw above, I'm changing:
> CONFIG_BLK_DEV_NVME to CONFIG_NVME_PCI
> and 
> CONFIG_BLK_DEV_NVME_SCSI to CONFIG_NVME_SCSI
> 
> Is there Kconfig magic to keep existing config working?

Not if you remove the old option from the Kconfig file. You could keep
the old option and make it select the new one. But then the benefit of
the whole operation becomes questionable.

Michal


From keith.busch at intel.com  Wed Feb 10 07:12:53 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 15:12:53 +0000
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <1454970284-29543-5-git-send-email-mlin@kernel.org>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-5-git-send-email-mlin@kernel.org>
Message-ID: <20160210151253.GB32064@localhost.localdomain>

On Mon, Feb 08, 2016 at 02:24:44PM -0800, Ming Lin wrote:
> This splits nvme.ko into 2 modules:
> nvme-core.ko: the core part
> nvme.ko: the PCI driver

Do we want to let PCI take the generic "nvme.ko" name? What will the
other nvme transport modules be called?

Otherwise, looks good to me.


From mlin at kernel.org  Wed Feb 10 07:31:05 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 07:31:05 -0800
Subject: [PATCH 4/4] nvme: split pci module out of core module
In-Reply-To: <20160210151253.GB32064@localhost.localdomain>
References: <1454970284-29543-1-git-send-email-mlin@kernel.org>
 <1454970284-29543-5-git-send-email-mlin@kernel.org>
 <20160210151253.GB32064@localhost.localdomain>
Message-ID: <CAF1ivSZ7SrjaFgvkbwNEQp_M3XfwdoC11ZMGQzjyzK-BeotXGA@mail.gmail.com>

On Wed, Feb 10, 2016 at 7:12 AM, Keith Busch <keith.busch at intel.com> wrote:
> On Mon, Feb 08, 2016 at 02:24:44PM -0800, Ming Lin wrote:
>> This splits nvme.ko into 2 modules:
>> nvme-core.ko: the core part
>> nvme.ko: the PCI driver
>
> Do we want to let PCI take the generic "nvme.ko" name? What will the
> other nvme transport modules be called?

Better name would be: nvme_pci.ko, but that may break, for example,
userspace scripts.

nvme_rdma.ko: IB/RoCE/iWARP
nvme_fc.ko: Fibre Channel
nvme_vhost.ko
....

>
> Otherwise, looks good to me.


From mlin at kernel.org  Wed Feb 10 07:37:28 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 07:37:28 -0800
Subject: [PATCH v2 0/5] split pci module out of core module
In-Reply-To: <20160210082146.GA9171@lst.de>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <20160210082146.GA9171@lst.de>
Message-ID: <CAF1ivSYhZUXZEK74CWPy4fq-vyOE=++gxgJqngnUbSqMFwEoeQ@mail.gmail.com>

On Wed, Feb 10, 2016 at 12:21 AM, Christoph Hellwig <hch at lst.de> wrote:
> Patch 3 still says "export more symbols" while it exports the first
> symbols.  Maybe it's better to just gold patch 3 into patch 5 where
> the exports are use?

Patch 2 exports the first symbols.

EXPORT_SYMBOL_GPL(admin_timeout);
EXPORT_SYMBOL_GPL(nvme_io_timeout);

>
> Otherwise looks fine,
>
> Reviewed-by: Christoph Hellwig <hch at lst.de>


From axboe at kernel.dk  Wed Feb 10 07:52:47 2016
From: axboe at kernel.dk (Jens Axboe)
Date: Wed, 10 Feb 2016 08:52:47 -0700
Subject: [PATCH v1] nvme: Log the ctrl device name instead of the
 undelying pci device name
In-Reply-To: <56BB0D6D.8050802@dev.mellanox.co.il>
References: <1453126453-17976-1-git-send-email-sagig@mellanox.com>
 <56BB0D6D.8050802@dev.mellanox.co.il>
Message-ID: <56BB5CCF.3050503@kernel.dk>

On 02/10/2016 03:14 AM, Sagi Grimberg wrote:
> Jens, care to pick this up for 4.6?
>
> This patch got an ack from Keith and didn't
> raise any objections so far...

Applied for 4.6 - I manually fixed up a reject in 
nvme_cancel_queue_ios(), jfyi.

-- 
Jens Axboe



From hch at lst.de  Wed Feb 10 08:11:08 2016
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2016 17:11:08 +0100
Subject: [PATCH v2 0/5] split pci module out of core module
In-Reply-To: <CAF1ivSYhZUXZEK74CWPy4fq-vyOE=++gxgJqngnUbSqMFwEoeQ@mail.gmail.com>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <20160210082146.GA9171@lst.de>
 <CAF1ivSYhZUXZEK74CWPy4fq-vyOE=++gxgJqngnUbSqMFwEoeQ@mail.gmail.com>
Message-ID: <20160210161108.GB17241@lst.de>

On Wed, Feb 10, 2016 at 07:37:28AM -0800, Ming Lin wrote:
> On Wed, Feb 10, 2016 at 12:21 AM, Christoph Hellwig <hch at lst.de> wrote:
> > Patch 3 still says "export more symbols" while it exports the first
> > symbols.  Maybe it's better to just gold patch 3 into patch 5 where
> > the exports are use?
> 
> Patch 2 exports the first symbols.
> 
> EXPORT_SYMBOL_GPL(admin_timeout);
> EXPORT_SYMBOL_GPL(nvme_io_timeout);

Ok, let's move all the exports into a single patch then.


From mlin at kernel.org  Wed Feb 10 08:20:19 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 08:20:19 -0800
Subject: [PATCH v2 0/5] split pci module out of core module
In-Reply-To: <20160210161108.GB17241@lst.de>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <20160210082146.GA9171@lst.de>
 <CAF1ivSYhZUXZEK74CWPy4fq-vyOE=++gxgJqngnUbSqMFwEoeQ@mail.gmail.com>
 <20160210161108.GB17241@lst.de>
Message-ID: <CAF1ivSZxXypVYQ_vpSS41ObBsL0Fi1r+6T3FYF6X9n6bh5nDaA@mail.gmail.com>

On Wed, Feb 10, 2016 at 8:11 AM, Christoph Hellwig <hch at lst.de> wrote:
> On Wed, Feb 10, 2016 at 07:37:28AM -0800, Ming Lin wrote:
>> On Wed, Feb 10, 2016 at 12:21 AM, Christoph Hellwig <hch at lst.de> wrote:
>> > Patch 3 still says "export more symbols" while it exports the first
>> > symbols.  Maybe it's better to just gold patch 3 into patch 5 where
>> > the exports are use?
>>
>> Patch 2 exports the first symbols.
>>
>> EXPORT_SYMBOL_GPL(admin_timeout);
>> EXPORT_SYMBOL_GPL(nvme_io_timeout);
>
> Ok, let's move all the exports into a single patch then.

OK. I'll merge patch 2 & 3.


From hch at lst.de  Wed Feb 10 08:22:33 2016
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2016 17:22:33 +0100
Subject: [PATCH v2 0/5] split pci module out of core module
In-Reply-To: <CAF1ivSZxXypVYQ_vpSS41ObBsL0Fi1r+6T3FYF6X9n6bh5nDaA@mail.gmail.com>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <20160210082146.GA9171@lst.de>
 <CAF1ivSYhZUXZEK74CWPy4fq-vyOE=++gxgJqngnUbSqMFwEoeQ@mail.gmail.com>
 <20160210161108.GB17241@lst.de>
 <CAF1ivSZxXypVYQ_vpSS41ObBsL0Fi1r+6T3FYF6X9n6bh5nDaA@mail.gmail.com>
Message-ID: <20160210162233.GA17738@lst.de>

On Wed, Feb 10, 2016 at 08:20:19AM -0800, Ming Lin wrote:
> > Ok, let's move all the exports into a single patch then.
> 
> OK. I'll merge patch 2 & 3.

Can I suggest the following:

 - patch 1: as-is
 - patch 2: drop the exports, just do the move
 - patch 3: fold into patch 5
 - patch 4: as-is
 - patch 5: all existing changes, plus all exports


From mlin at kernel.org  Wed Feb 10 09:29:36 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 09:29:36 -0800
Subject: [PATCH v2 0/5] split pci module out of core module
In-Reply-To: <20160210162233.GA17738@lst.de>
References: <1455062873-18392-1-git-send-email-mlin@kernel.org>
 <20160210082146.GA9171@lst.de>
 <CAF1ivSYhZUXZEK74CWPy4fq-vyOE=++gxgJqngnUbSqMFwEoeQ@mail.gmail.com>
 <20160210161108.GB17241@lst.de>
 <CAF1ivSZxXypVYQ_vpSS41ObBsL0Fi1r+6T3FYF6X9n6bh5nDaA@mail.gmail.com>
 <20160210162233.GA17738@lst.de>
Message-ID: <CAF1ivSbCZq0vYMVLhUG5LEz8M3ROf4Xxn7c461y8vQ9g1z79Zw@mail.gmail.com>

On Wed, Feb 10, 2016 at 8:22 AM, Christoph Hellwig <hch at lst.de> wrote:
> On Wed, Feb 10, 2016 at 08:20:19AM -0800, Ming Lin wrote:
>> > Ok, let's move all the exports into a single patch then.
>>
>> OK. I'll merge patch 2 & 3.
>
> Can I suggest the following:
>
>  - patch 1: as-is
>  - patch 2: drop the exports, just do the move
>  - patch 3: fold into patch 5
>  - patch 4: as-is
>  - patch 5: all existing changes, plus all exports

OK, will send v3.


From mlin at kernel.org  Wed Feb 10 10:03:28 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 10:03:28 -0800
Subject: [PATCH v3 0/4] split pci module out of core module
Message-ID: <1455127412-23940-1-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

We'll have nvme over fabrics dirver soon.
This series split nvme pci module out of nvme core module.

nvme.ko is split into 2 modules:
nvme-core.ko: the core part
nvme.ko: the PCI driver

v3:
  - restructure patches as suggested by hch
  - add more review tags

v2:
  - add Sagi's patch to take proper references to the transport driver on open
  - add review tags
  - don't change the CONFIG name

Ming Lin (3):
  nvme: move timeout variables to core.c
  nvme: split dev_list_lock
  nvme: split pci module out of core module

Sagi Grimberg (1):
  nvme/host: reference the fabric module for each bdev open callout

 drivers/nvme/host/Kconfig  |  6 ++++-
 drivers/nvme/host/Makefile | 10 ++++----
 drivers/nvme/host/core.c   | 57 ++++++++++++++++++++++++++++++++++++++++++----
 drivers/nvme/host/nvme.h   |  3 +--
 drivers/nvme/host/pci.c    | 27 +++-------------------
 5 files changed, 67 insertions(+), 36 deletions(-)

-- 
1.9.1



From mlin at kernel.org  Wed Feb 10 10:03:30 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 10:03:30 -0800
Subject: [PATCH v3 2/4] nvme: move timeout variables to core.c
In-Reply-To: <1455127412-23940-1-git-send-email-mlin@kernel.org>
References: <1455127412-23940-1-git-send-email-mlin@kernel.org>
Message-ID: <1455127412-23940-3-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

These variables are used by PCI driver and will also be used in the
forthcoming NVMe over Fabrics drivers.

Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 12 ++++++++++++
 drivers/nvme/host/pci.c  | 12 ------------
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 8adc598..8714847 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -33,6 +33,18 @@
 
 #define NVME_MINORS		(1U << MINORBITS)
 
+unsigned char admin_timeout = 60;
+module_param(admin_timeout, byte, 0644);
+MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
+
+unsigned char nvme_io_timeout = 30;
+module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
+MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
+
+unsigned char shutdown_timeout = 5;
+module_param(shutdown_timeout, byte, 0644);
+MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
+
 static int nvme_major;
 module_param(nvme_major, int, 0);
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e36ca02..e3552d1 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -57,18 +57,6 @@
 #define NVME_NR_AEN_COMMANDS	1
 #define NVME_AQ_BLKMQ_DEPTH	(NVME_AQ_DEPTH - NVME_NR_AEN_COMMANDS)
 
-unsigned char admin_timeout = 60;
-module_param(admin_timeout, byte, 0644);
-MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
-
-unsigned char nvme_io_timeout = 30;
-module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
-MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
-
-unsigned char shutdown_timeout = 5;
-module_param(shutdown_timeout, byte, 0644);
-MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
-
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 
-- 
1.9.1



From mlin at kernel.org  Wed Feb 10 10:03:29 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 10:03:29 -0800
Subject: [PATCH v3 1/4] nvme/host: reference the fabric module for each bdev
 open callout
In-Reply-To: <1455127412-23940-1-git-send-email-mlin@kernel.org>
References: <1455127412-23940-1-git-send-email-mlin@kernel.org>
Message-ID: <1455127412-23940-2-git-send-email-mlin@kernel.org>

From: Sagi Grimberg <sagig at mellanox.com>

We don't want to be able to unload the fabric driver when we have
openened referenced to our namespaces. Thus, for each nvme_open we
take a reference on the fabric driver and put it in nvme_release.
This behavior is consistent with the scsi model.

This resolves the panic when unloading a fabric module with
mpath holders.

Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Ian Bakshan <ianb at mellanox.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 19 ++++++++++++++++---
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/pci.c  |  1 +
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c5bf001..8adc598 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -71,11 +71,21 @@ static struct nvme_ns *nvme_get_ns_from_disk(struct gendisk *disk)
 
 	spin_lock(&dev_list_lock);
 	ns = disk->private_data;
-	if (ns && !kref_get_unless_zero(&ns->kref))
-		ns = NULL;
+	if (ns) {
+		if (!kref_get_unless_zero(&ns->kref))
+			goto fail;
+		if (!try_module_get(ns->ctrl->ops->module))
+			goto fail_put_ns;
+	}
 	spin_unlock(&dev_list_lock);
 
 	return ns;
+
+fail_put_ns:
+	kref_put(&ns->kref, nvme_free_ns);
+fail:
+	spin_unlock(&dev_list_lock);
+	return NULL;
 }
 
 void nvme_requeue_req(struct request *req)
@@ -499,7 +509,10 @@ static int nvme_open(struct block_device *bdev, fmode_t mode)
 
 static void nvme_release(struct gendisk *disk, fmode_t mode)
 {
-	nvme_put_ns(disk->private_data);
+	struct nvme_ns *ns = disk->private_data;
+
+	module_put(ns->ctrl->ops->module);
+	nvme_put_ns(ns);
 }
 
 static int nvme_getgeo(struct block_device *bdev, struct hd_geometry *geo)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4fb5bb7..9f77386 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -117,6 +117,7 @@ struct nvme_ns {
 };
 
 struct nvme_ctrl_ops {
+	struct module *module;
 	int (*reg_read32)(struct nvme_ctrl *ctrl, u32 off, u32 *val);
 	int (*reg_write32)(struct nvme_ctrl *ctrl, u32 off, u32 val);
 	int (*reg_read64)(struct nvme_ctrl *ctrl, u32 off, u64 *val);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 72ef832..e36ca02 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2029,6 +2029,7 @@ static int nvme_pci_reset_ctrl(struct nvme_ctrl *ctrl)
 }
 
 static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = {
+	.module			= THIS_MODULE,
 	.reg_read32		= nvme_pci_reg_read32,
 	.reg_write32		= nvme_pci_reg_write32,
 	.reg_read64		= nvme_pci_reg_read64,
-- 
1.9.1



From mlin at kernel.org  Wed Feb 10 10:03:31 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 10:03:31 -0800
Subject: [PATCH v3 3/4] nvme: split dev_list_lock
In-Reply-To: <1455127412-23940-1-git-send-email-mlin@kernel.org>
References: <1455127412-23940-1-git-send-email-mlin@kernel.org>
Message-ID: <1455127412-23940-4-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

Split dev_list_lock into one in the core and one in the PCI driver.

Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 2 +-
 drivers/nvme/host/nvme.h | 2 --
 drivers/nvme/host/pci.c  | 1 +
 3 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 8714847..38db08d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -52,7 +52,7 @@ static int nvme_char_major;
 module_param(nvme_char_major, int, 0);
 
 static LIST_HEAD(nvme_ctrl_list);
-DEFINE_SPINLOCK(dev_list_lock);
+static DEFINE_SPINLOCK(dev_list_lock);
 
 static struct class *nvme_class;
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9f77386..63ba8a5 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -266,8 +266,6 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 			dma_addr_t dma_addr, u32 *result);
 int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count);
 
-extern spinlock_t dev_list_lock;
-
 struct sg_io_hdr;
 
 int nvme_sg_io(struct nvme_ns *ns, struct sg_io_hdr __user *u_hdr);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e3552d1..f7266cc 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -65,6 +65,7 @@ module_param(use_cmb_sqes, bool, 0644);
 MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
 
 static LIST_HEAD(dev_list);
+static DEFINE_SPINLOCK(dev_list_lock);
 static struct task_struct *nvme_thread;
 static struct workqueue_struct *nvme_workq;
 static wait_queue_head_t nvme_kthread_wait;
-- 
1.9.1



From mlin at kernel.org  Wed Feb 10 10:03:32 2016
From: mlin at kernel.org (Ming Lin)
Date: Wed, 10 Feb 2016 10:03:32 -0800
Subject: [PATCH v3 4/4] nvme: split pci module out of core module
In-Reply-To: <1455127412-23940-1-git-send-email-mlin@kernel.org>
References: <1455127412-23940-1-git-send-email-mlin@kernel.org>
Message-ID: <1455127412-23940-5-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

NVMe over Fabrics drivers are going to reuse the core,
so splits nvme.ko into 2 modules:

nvme-core.ko: the core part
nvme.ko: the PCI driver

Export symbols from nvme-core.ko.

Reviewed-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/Kconfig  |  6 +++++-
 drivers/nvme/host/Makefile | 10 ++++++----
 drivers/nvme/host/core.c   | 24 +++++++++++++++++++++++-
 drivers/nvme/host/pci.c    | 13 +------------
 4 files changed, 35 insertions(+), 18 deletions(-)

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index 5d62373..2ed30f0 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -1,6 +1,10 @@
+config NVME_CORE
+	tristate
+
 config BLK_DEV_NVME
 	tristate "NVM Express block device"
 	depends on PCI && BLOCK
+	select NVME_CORE
 	---help---
 	  The NVM Express driver is for solid state drives directly
 	  connected to the PCI or PCI Express bus.  If you know you
@@ -11,7 +15,7 @@ config BLK_DEV_NVME
 
 config BLK_DEV_NVME_SCSI
 	bool "SCSI emulation for NVMe device nodes"
-	depends on BLK_DEV_NVME
+	depends on NVME_CORE
 	---help---
 	  This adds support for the SG_IO ioctl on the NVMe character
 	  and block devices nodes, as well a a translation for a small
diff --git a/drivers/nvme/host/Makefile b/drivers/nvme/host/Makefile
index 51bf908..9a3ca89 100644
--- a/drivers/nvme/host/Makefile
+++ b/drivers/nvme/host/Makefile
@@ -1,6 +1,8 @@
+obj-$(CONFIG_NVME_CORE)			+= nvme-core.o
+obj-$(CONFIG_BLK_DEV_NVME)		+= nvme.o
 
-obj-$(CONFIG_BLK_DEV_NVME)     += nvme.o
+nvme-core-y				:= core.o
+nvme-core-$(CONFIG_BLK_DEV_NVME_SCSI)	+= scsi.o
+nvme-core-$(CONFIG_NVM)			+= lightnvm.o
 
-lightnvm-$(CONFIG_NVM)			:= lightnvm.o
-nvme-y					+= core.o pci.o $(lightnvm-y)
-nvme-$(CONFIG_BLK_DEV_NVME_SCSI)        += scsi.o
+nvme-y					+= pci.o
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 38db08d..c7df0ab 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -36,10 +36,12 @@
 unsigned char admin_timeout = 60;
 module_param(admin_timeout, byte, 0644);
 MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands");
+EXPORT_SYMBOL_GPL(admin_timeout);
 
 unsigned char nvme_io_timeout = 30;
 module_param_named(io_timeout, nvme_io_timeout, byte, 0644);
 MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
+EXPORT_SYMBOL_GPL(nvme_io_timeout);
 
 unsigned char shutdown_timeout = 5;
 module_param(shutdown_timeout, byte, 0644);
@@ -110,6 +112,7 @@ void nvme_requeue_req(struct request *req)
 		blk_mq_kick_requeue_list(req->q);
 	spin_unlock_irqrestore(req->q->queue_lock, flags);
 }
+EXPORT_SYMBOL_GPL(nvme_requeue_req);
 
 struct request *nvme_alloc_request(struct request_queue *q,
 		struct nvme_command *cmd, unsigned int flags)
@@ -133,6 +136,7 @@ struct request *nvme_alloc_request(struct request_queue *q,
 
 	return req;
 }
+EXPORT_SYMBOL_GPL(nvme_alloc_request);
 
 /*
  * Returns 0 on success.  If the result is negative, it's a Linux error code;
@@ -170,6 +174,7 @@ int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 {
 	return __nvme_submit_sync_cmd(q, cmd, buffer, bufflen, NULL, 0);
 }
+EXPORT_SYMBOL_GPL(nvme_submit_sync_cmd);
 
 int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void __user *ubuffer, unsigned bufflen,
@@ -385,6 +390,7 @@ int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count)
 	*count = min(*count, nr_io_queues);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nvme_set_queue_count);
 
 static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
 {
@@ -794,6 +800,7 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 		return ret;
 	return nvme_wait_ready(ctrl, cap, false);
 }
+EXPORT_SYMBOL_GPL(nvme_disable_ctrl);
 
 int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 {
@@ -825,6 +832,7 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl, u64 cap)
 		return ret;
 	return nvme_wait_ready(ctrl, cap, true);
 }
+EXPORT_SYMBOL_GPL(nvme_enable_ctrl);
 
 int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
 {
@@ -855,6 +863,7 @@ int nvme_shutdown_ctrl(struct nvme_ctrl *ctrl)
 
 	return ret;
 }
+EXPORT_SYMBOL_GPL(nvme_shutdown_ctrl);
 
 /*
  * Initialize the cached copies of the Identify data and various controller
@@ -916,6 +925,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	kfree(id);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nvme_init_identify);
 
 static int nvme_dev_open(struct inode *inode, struct file *file)
 {
@@ -1321,6 +1331,7 @@ void nvme_scan_namespaces(struct nvme_ctrl *ctrl)
 	mutex_unlock(&ctrl->namespaces_mutex);
 	kfree(id);
 }
+EXPORT_SYMBOL_GPL(nvme_scan_namespaces);
 
 void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 {
@@ -1331,6 +1342,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 		nvme_ns_remove(ns);
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
 
 static DEFINE_IDA(nvme_instance_ida);
 
@@ -1362,13 +1374,14 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
 }
 
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
- {
+{
 	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 
 	spin_lock(&dev_list_lock);
 	list_del(&ctrl->node);
 	spin_unlock(&dev_list_lock);
 }
+EXPORT_SYMBOL_GPL(nvme_uninit_ctrl);
 
 static void nvme_free_ctrl(struct kref *kref)
 {
@@ -1384,6 +1397,7 @@ void nvme_put_ctrl(struct nvme_ctrl *ctrl)
 {
 	kref_put(&ctrl->kref, nvme_free_ctrl);
 }
+EXPORT_SYMBOL_GPL(nvme_put_ctrl);
 
 /*
  * Initialize a NVMe controller structures.  This needs to be called during
@@ -1427,6 +1441,7 @@ out_release_instance:
 out:
 	return ret;
 }
+EXPORT_SYMBOL_GPL(nvme_init_ctrl);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
@@ -1443,6 +1458,7 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl)
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_stop_queues);
 
 void nvme_start_queues(struct nvme_ctrl *ctrl)
 {
@@ -1456,6 +1472,7 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
 	}
 	mutex_unlock(&ctrl->namespaces_mutex);
 }
+EXPORT_SYMBOL_GPL(nvme_start_queues);
 
 int __init nvme_core_init(void)
 {
@@ -1495,3 +1512,8 @@ void nvme_core_exit(void)
 	class_destroy(nvme_class);
 	__unregister_chrdev(nvme_char_major, 0, NVME_MINORS, "nvme");
 }
+
+MODULE_LICENSE("GPL");
+MODULE_VERSION("1.0");
+module_init(nvme_core_init);
+module_exit(nvme_core_exit);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index f7266cc..a5a4567 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2221,26 +2221,15 @@ static int __init nvme_init(void)
 	if (!nvme_workq)
 		return -ENOMEM;
 
-	result = nvme_core_init();
-	if (result < 0)
-		goto kill_workq;
-
 	result = pci_register_driver(&nvme_driver);
 	if (result)
-		goto core_exit;
-	return 0;
-
- core_exit:
-	nvme_core_exit();
- kill_workq:
-	destroy_workqueue(nvme_workq);
+		destroy_workqueue(nvme_workq);
 	return result;
 }
 
 static void __exit nvme_exit(void)
 {
 	pci_unregister_driver(&nvme_driver);
-	nvme_core_exit();
 	destroy_workqueue(nvme_workq);
 	BUG_ON(nvme_thread && !IS_ERR(nvme_thread));
 	_nvme_check_size();
-- 
1.9.1



From keith.busch at intel.com  Wed Feb 10 10:17:21 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:21 -0700
Subject: [PATCH for-4.5 04/13] NVMe: Set queue limits max_dev_sectors
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-5-git-send-email-keith.busch@intel.com>

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 3cd921e..852b789 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1136,6 +1136,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 	blk_queue_logical_block_size(ns->queue, 1 << ns->lba_shift);
 	if (ctrl->max_hw_sectors) {
+		ns->queue->limits.max_dev_sectors = ctrl->max_hw_sectors;
 		blk_queue_max_hw_sectors(ns->queue, ctrl->max_hw_sectors);
 		blk_queue_max_segments(ns->queue,
 			(ctrl->max_hw_sectors / (ctrl->page_size >> 9)) + 1);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:18 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:18 -0700
Subject: [PATCH for-4.5 01/13] blk-mq: End unstarted requests on dying queue
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-2-git-send-email-keith.busch@intel.com>

Go directly to ending a request if it wasn't started. Previously, completing a
request may invoke a driver callback for a request it didn't initialize.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Acked-by: Christoph Hellwig <hch at lst.de>
---
 block/blk-mq.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 4c0622f..56c0a72 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -599,8 +599,10 @@ static void blk_mq_check_expired(struct blk_mq_hw_ctx *hctx,
 		 * If a request wasn't started before the queue was
 		 * marked dying, kill it here or it'll go unnoticed.
 		 */
-		if (unlikely(blk_queue_dying(rq->q)))
-			blk_mq_complete_request(rq, -EIO);
+		if (unlikely(blk_queue_dying(rq->q))) {
+			rq->errors = -EIO;
+			blk_mq_end_request(rq, rq->errors);
+		}
 		return;
 	}
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:20 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:20 -0700
Subject: [PATCH for-4.5 03/13] NVMe: Allow request merges
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-4-git-send-email-keith.busch@intel.com>

It is generally more efficient to submit larger IO.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c5bf001..3cd921e 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1121,7 +1121,6 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	ns->queue = blk_mq_init_queue(ctrl->tagset);
 	if (IS_ERR(ns->queue))
 		goto out_free_ns;
-	queue_flag_set_unlocked(QUEUE_FLAG_NOMERGES, ns->queue);
 	queue_flag_set_unlocked(QUEUE_FLAG_NONROT, ns->queue);
 	ns->queue->queuedata = ns;
 	ns->ctrl = ctrl;
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:24 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:24 -0700
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-8-git-send-email-keith.busch@intel.com>

It's possible a request may get to the driver after the nvme queue was
disabled. This has the request requeue if that happens.

Note the request is still "started" by the driver, but requeuing will
clear the start state for timeout handling.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 5bea054..f8bb781 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -678,6 +678,11 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 	blk_mq_start_request(req);
 
 	spin_lock_irq(&nvmeq->q_lock);
+	if (unlikely(nvmeq->cq_vector < 0)) {
+		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		spin_unlock_irq(&nvmeq->q_lock);
+		goto out;
+	}
 	__nvme_submit_cmd(nvmeq, &cmnd);
 	nvme_process_cq(nvmeq);
 	spin_unlock_irq(&nvmeq->q_lock);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:23 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:23 -0700
Subject: [PATCH for-4.5 06/13] NVMe: Remove WQ_MEM_RECLAIM from nvme work queue
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-7-git-send-email-keith.busch@intel.com>

This isn't used for work in the memory reclaim path, and we may need
to sync with work queues that also are not flagged memory relaim. This
fixes a kernel warning if we ever do sync with such a work queue.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 72ef832..5bea054 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2227,7 +2227,7 @@ static int __init nvme_init(void)
 
 	init_waitqueue_head(&nvme_kthread_wait);
 
-	nvme_workq = alloc_workqueue("nvme", WQ_UNBOUND | WQ_MEM_RECLAIM, 0);
+	nvme_workq = alloc_workqueue("nvme", WQ_UNBOUND, 0);
 	if (!nvme_workq)
 		return -ENOMEM;
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:17 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:17 -0700
Subject: [PATCH for-4.5 00/13] NVMe fixups for 4.5 
Message-ID: <1455128250-5984-1-git-send-email-keith.busch@intel.com>

Sorry for the patch bomb in the middle of rc, though this is fixing
legit issues.

Patches 1 and 2 are already reviewed/acked, and added those to this
reviewers (thanks!).

Patches 3 and 4 are performance related and pretty straight forward.

The remaining are specifically for surprise removal and other controller
failure scenarios. I had all these in a single patch, but there was
too much going on, so I split at what appears to be reasonable and self
contained features.

To test what was previously broken: run buffered writes to an nvme drive,
remove the drive via sysfs, then pull the drive out.

Keith Busch (13):
  blk-mq: End unstarted requests on dying queue
  NVMe: Fix io incapable return values
  NVMe: Allow request merges
  NVMe: Set queue limits max_dev_sectors
  NVMe: Fix namespace removal deadlock
  NVMe: Remove WQ_MEM_RECLAIM from nvme work queue
  NVMe: Requeue requests on suspended queues
  NVMe: Poll device while still active during remove
  NVMe: Simplify device reset failure
  NVMe: Move error handling to failed reset handler
  NVMe: Dead namespace handling
  NVMe: Mark queues as dead on degraded controller
  NVMe: Rate limit nvme IO warnings

 block/blk-mq.c           |  6 ++--
 drivers/nvme/host/core.c | 74 ++++++++++++++++++++++-----------------
 drivers/nvme/host/nvme.h | 12 +++++--
 drivers/nvme/host/pci.c  | 91 ++++++++++++++++++++++++++++++------------------
 4 files changed, 113 insertions(+), 70 deletions(-)

-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:25 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:25 -0700
Subject: [PATCH for-4.5 08/13] NVMe: Poll device while still active during
 remove
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-9-git-send-email-keith.busch@intel.com>

A device failure or link down wouldn't have been detected during namespace
removal. This patch keeps the device in the list for polling so that the
thread may see such failure and initiate a reset. The device is removed
from the list after disable, so we can safely flush the reset work as
it can't be requeued when disable completes.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index f8bb781..0303936 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2116,16 +2116,12 @@ static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
 
-	spin_lock(&dev_list_lock);
-	list_del_init(&dev->node);
-	spin_unlock(&dev_list_lock);
-
 	pci_set_drvdata(pdev, NULL);
-	flush_work(&dev->reset_work);
 	flush_work(&dev->scan_work);
 	nvme_remove_namespaces(&dev->ctrl);
 	nvme_uninit_ctrl(&dev->ctrl);
 	nvme_dev_disable(dev, true);
+	flush_work(&dev->reset_work);
 	nvme_dev_remove_admin(dev);
 	nvme_free_queues(dev, 0);
 	nvme_release_cmb(dev);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:27 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:27 -0700
Subject: [PATCH for-4.5 10/13] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-11-git-send-email-keith.busch@intel.com>

This moves the dead queue handling out of the namespace removal path
and into the reset failure path. It fixes a deadlock condition if the
controller fails or link down during del_gendisk.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 19 ++-----------------
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/pci.c  | 22 ++++++++++++++++++++++
 3 files changed, 25 insertions(+), 17 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index be27f9f..41b595c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -64,7 +64,7 @@ static void nvme_free_ns(struct kref *kref)
 	kfree(ns);
 }
 
-static void nvme_put_ns(struct nvme_ns *ns)
+void nvme_put_ns(struct nvme_ns *ns)
 {
 	kref_put(&ns->kref, nvme_free_ns);
 }
@@ -1113,28 +1113,13 @@ static struct nvme_ns *nvme_find_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 static void nvme_ns_remove_work(struct work_struct *work)
 {
 	struct nvme_ns *ns = container_of(work, struct nvme_ns, remove_work);
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
+
 	if (ns->disk->flags & GENHD_FL_UP) {
 		if (blk_get_integrity(ns->disk))
 			blk_integrity_unregister(ns->disk);
 		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
 					&nvme_ns_attr_group);
 		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index d330512..19a64b2 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -270,6 +270,7 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
 int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 			dma_addr_t dma_addr, u32 *result);
 int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count);
+void nvme_put_ns(struct nvme_ns *ns);
 
 extern spinlock_t dev_list_lock;
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 3381bac..a18e4ab 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1898,11 +1898,33 @@ static void nvme_pci_free_ctrl(struct nvme_ctrl *ctrl)
 	kfree(dev);
 }
 
+static void nvme_kill_ns_queues(struct nvme_dev *dev)
+{
+	struct nvme_ns *ns;
+	struct nvme_ctrl *ctrl = &dev->ctrl;
+
+	nvme_dev_disable(dev, false);
+
+	mutex_lock(&ctrl->namespaces_mutex);
+	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (!kref_get_unless_zero(&ns->kref))
+			continue;
+
+		blk_set_queue_dying(ns->queue);
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_mq_start_stopped_hw_queues(ns->queue, true);
+
+		nvme_put_ns(ns);
+	}
+	mutex_unlock(&ctrl->namespaces_mutex);
+}
+
 static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
 {
 	dev_warn(dev->dev, "Removing after probe failure status:%d\n", status);
 
 	kref_get(&dev->ctrl.kref);
+	nvme_kill_ns_queues(dev);
 	if (!schedule_work(&dev->remove_work))
 		nvme_put_ctrl(&dev->ctrl);
 }
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:26 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:26 -0700
Subject: [PATCH for-4.5 09/13] NVMe: Simplify device reset failure
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-10-git-send-email-keith.busch@intel.com>

A reset failure schedules the device to unbind from the driver through
the pci driver's remove. This cleans up all intialization, so there is
no need to duplicate the potentially racy cleanup.

To help understand why a reset failed, the status is logged with the
existed warning message.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 43 ++++++++++++++++---------------------------
 1 file changed, 16 insertions(+), 27 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 0303936..3381bac 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -86,7 +86,6 @@ struct nvme_queue;
 
 static int nvme_reset(struct nvme_dev *dev);
 static void nvme_process_cq(struct nvme_queue *nvmeq);
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev);
 static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown);
 
 /*
@@ -1899,10 +1898,19 @@ static void nvme_pci_free_ctrl(struct nvme_ctrl *ctrl)
 	kfree(dev);
 }
 
+static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
+{
+	dev_warn(dev->dev, "Removing after probe failure status:%d\n", status);
+
+	kref_get(&dev->ctrl.kref);
+	if (!schedule_work(&dev->remove_work))
+		nvme_put_ctrl(&dev->ctrl);
+}
+
 static void nvme_reset_work(struct work_struct *work)
 {
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, reset_work);
-	int result;
+	int result = ENODEV;
 
 	if (WARN_ON(test_bit(NVME_CTRL_RESETTING, &dev->flags)))
 		goto out;
@@ -1922,26 +1930,26 @@ static void nvme_reset_work(struct work_struct *work)
 
 	result = nvme_configure_admin_queue(dev);
 	if (result)
-		goto unmap;
+		goto out;
 
 	nvme_init_queue(dev->queues[0], 0);
 	result = nvme_alloc_admin_tags(dev);
 	if (result)
-		goto disable;
+		goto out;
 
 	result = nvme_init_identify(&dev->ctrl);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	result = nvme_setup_io_queues(dev);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
 
 	result = nvme_dev_list_add(dev);
 	if (result)
-		goto remove;
+		goto out;
 
 	/*
 	 * Keep the controller around but remove all namespaces if we don't have
@@ -1958,19 +1966,8 @@ static void nvme_reset_work(struct work_struct *work)
 	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
 	return;
 
- remove:
-	nvme_dev_list_remove(dev);
- free_tags:
-	nvme_dev_remove_admin(dev);
-	blk_put_queue(dev->ctrl.admin_q);
-	dev->ctrl.admin_q = NULL;
-	dev->queues[0]->tags = NULL;
- disable:
-	nvme_disable_admin_queue(dev, false);
- unmap:
-	nvme_dev_unmap(dev);
  out:
-	nvme_remove_dead_ctrl(dev);
+	nvme_remove_dead_ctrl(dev, result);
 }
 
 static void nvme_remove_dead_ctrl_work(struct work_struct *work)
@@ -1983,14 +1980,6 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	nvme_put_ctrl(&dev->ctrl);
 }
 
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev)
-{
-	dev_warn(dev->dev, "Removing after probe failure\n");
-	kref_get(&dev->ctrl.kref);
-	if (!schedule_work(&dev->remove_work))
-		nvme_put_ctrl(&dev->ctrl);
-}
-
 static int nvme_reset(struct nvme_dev *dev)
 {
 	if (!dev->ctrl.admin_q || blk_queue_dying(dev->ctrl.admin_q))
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:30 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:30 -0700
Subject: [PATCH for-4.5 13/13] NVMe: Rate limit nvme IO warnings
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-14-git-send-email-keith.busch@intel.com>

We don't need to spam the kernel logs with thousands of messages. We
can infer all IO's are being cancelled with fewer messages.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 18c14f6..4d3b3d0 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1005,7 +1005,7 @@ static void nvme_cancel_queue_ios(struct request *req, void *data, bool reserved
 	if (!blk_mq_request_started(req))
 		return;
 
-	dev_warn(nvmeq->q_dmadev,
+	dev_warn_ratelimited(nvmeq->q_dmadev,
 		 "Cancelling I/O %d QID %d\n", req->tag, nvmeq->qid);
 
 	status = NVME_SC_ABORT_REQ;
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:19 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:19 -0700
Subject: [PATCH for-4.5 02/13] NVMe: Fix io incapable return values
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-3-git-send-email-keith.busch@intel.com>

The function returns true when the controller can't handle IO.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Acked-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/nvme.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4fb5bb7..9664d07 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -139,9 +139,9 @@ static inline bool nvme_io_incapable(struct nvme_ctrl *ctrl)
 	u32 val = 0;
 
 	if (ctrl->ops->io_incapable(ctrl))
-		return false;
+		return true;
 	if (ctrl->ops->reg_read32(ctrl, NVME_REG_CSTS, &val))
-		return false;
+		return true;
 	return val & NVME_CSTS_CFS;
 }
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:22 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:22 -0700
Subject: [PATCH for-4.5 05/13] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-6-git-send-email-keith.busch@intel.com>

Namespace removal holds the namespace list mutex. If the controller
fails during namespace removal, the reset work will deadlock trying to
acquire the mutex, and no progress will be made.

This patch completes the potentially blocking parts of namespace
removal in a controller work queue so that reset work can cleanup a
failed controller.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 81 ++++++++++++++++++++++++++++++------------------
 drivers/nvme/host/nvme.h |  6 ++++
 2 files changed, 57 insertions(+), 30 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 852b789..be27f9f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -48,6 +48,10 @@ static void nvme_free_ns(struct kref *kref)
 {
 	struct nvme_ns *ns = container_of(kref, struct nvme_ns, kref);
 
+	mutex_lock(&ns->ctrl->namespaces_mutex);
+	list_del_init(&ns->list);
+	mutex_unlock(&ns->ctrl->namespaces_mutex);
+
 	if (ns->type == NVME_NS_LIGHTNVM)
 		nvme_nvm_unregister(ns->queue, ns->disk->disk_name);
 
@@ -1106,6 +1110,37 @@ static struct nvme_ns *nvme_find_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	return NULL;
 }
 
+static void nvme_ns_remove_work(struct work_struct *work)
+{
+	struct nvme_ns *ns = container_of(work, struct nvme_ns, remove_work);
+	bool kill = nvme_io_incapable(ns->ctrl) &&
+			!blk_queue_dying(ns->queue);
+
+	if (kill) {
+		blk_set_queue_dying(ns->queue);
+
+		/*
+		 * The controller was shutdown first if we got here through
+		 * device removal. The shutdown may requeue outstanding
+		 * requests. These need to be aborted immediately so
+		 * del_gendisk doesn't block indefinitely for their completion.
+		 */
+		blk_mq_abort_requeue_list(ns->queue);
+	}
+	if (ns->disk->flags & GENHD_FL_UP) {
+		if (blk_get_integrity(ns->disk))
+			blk_integrity_unregister(ns->disk);
+		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
+					&nvme_ns_attr_group);
+		del_gendisk(ns->disk);
+	}
+	if (kill || !blk_queue_dying(ns->queue)) {
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_cleanup_queue(ns->queue);
+	}
+	nvme_put_ns(ns);
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 {
 	struct nvme_ns *ns;
@@ -1161,6 +1196,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 	list_add_tail(&ns->list, &ctrl->namespaces);
 	kref_get(&ctrl->kref);
+	INIT_WORK(&ns->remove_work, nvme_ns_remove_work);
 	if (ns->type == NVME_NS_LIGHTNVM)
 		return;
 
@@ -1180,35 +1216,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)
 {
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-
-	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
-
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
-	if (ns->disk->flags & GENHD_FL_UP) {
-		if (blk_get_integrity(ns->disk))
-			blk_integrity_unregister(ns->disk);
-		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
-					&nvme_ns_attr_group);
-		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
-		blk_mq_abort_requeue_list(ns->queue);
-		blk_cleanup_queue(ns->queue);
-	}
-	list_del_init(&ns->list);
-	nvme_put_ns(ns);
+	if (!test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
+		queue_work(ns->ctrl->ns_workq, &ns->remove_work);
 }
 
 static void nvme_validate_ns(struct nvme_ctrl *ctrl, unsigned nsid)
@@ -1305,6 +1314,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 	list_for_each_entry_safe(ns, next, &ctrl->namespaces, list)
 		nvme_ns_remove(ns);
 	mutex_unlock(&ctrl->namespaces_mutex);
+
+	flush_workqueue(ctrl->ns_workq);
 }
 
 static DEFINE_IDA(nvme_instance_ida);
@@ -1337,12 +1348,14 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
 }
 
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
- {
+{
 	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 
 	spin_lock(&dev_list_lock);
 	list_del(&ctrl->node);
 	spin_unlock(&dev_list_lock);
+
+	destroy_workqueue(ctrl->ns_workq);
 }
 
 static void nvme_free_ctrl(struct kref *kref)
@@ -1392,11 +1405,19 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	get_device(ctrl->device);
 	dev_set_drvdata(ctrl->device, ctrl);
 
+	ctrl->ns_workq = alloc_workqueue("nvme-ns", WQ_UNBOUND, 0);
+	if (!ctrl->ns_workq) {
+		ret = -ENOMEM;
+		goto out_destroy_device;
+	}
+
 	spin_lock(&dev_list_lock);
 	list_add_tail(&ctrl->node, &nvme_ctrl_list);
 	spin_unlock(&dev_list_lock);
 
 	return 0;
+out_destroy_device:
+	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 out_release_instance:
 	nvme_release_instance(ctrl);
 out:
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9664d07..d330512 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -72,6 +72,7 @@ struct nvme_ctrl {
 	struct mutex namespaces_mutex;
 	struct device *device;	/* char device */
 	struct list_head node;
+	struct workqueue_struct *ns_workq;
 
 	char name[12];
 	char serial[20];
@@ -112,6 +113,11 @@ struct nvme_ns {
 	bool ext;
 	u8 pi_type;
 	int type;
+	struct work_struct remove_work;
+	unsigned long flags;
+
+#define NVME_NS_REMOVING 0
+
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
 };
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:29 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:29 -0700
Subject: [PATCH for-4.5 12/13] NVMe: Mark queues as dead on degraded controller
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-13-git-send-email-keith.busch@intel.com>

If a previously optimal controller goes degraded, the IO queues will
stop working, but the controller is still managable. Prior to removing
namespace, we need to fail the queues so queued and new IO will fail
and allow namespace removal to progress.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 7fd8a54..18c14f6 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1989,6 +1989,7 @@ static void nvme_reset_work(struct work_struct *work)
 	 */
 	if (dev->online_queues < 2) {
 		dev_warn(dev->dev, "IO queues not created\n");
+		nvme_kill_ns_queues(dev);
 		nvme_remove_namespaces(&dev->ctrl);
 	} else {
 		nvme_start_queues(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 10 10:17:28 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 11:17:28 -0700
Subject: [PATCH for-4.5 11/13] NVMe: Dead namespace handling
In-Reply-To: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455128250-5984-12-git-send-email-keith.busch@intel.com>

This adds a "dead" state to a namespace and revalidates such a namespace
to 0 capacity. This will force buffered writers to stop writing pages
that can't be synced, and ends requests in failure if any is submitted
to such a namespace.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c |  4 ++++
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/pci.c  | 12 +++++++++++-
 3 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 41b595c..84e9f41 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -560,6 +560,10 @@ static int nvme_revalidate_disk(struct gendisk *disk)
 	u16 old_ms;
 	unsigned short bs;
 
+	if (test_bit(NVME_NS_DEAD, &ns->flags)) {
+		set_capacity(disk, 0);
+		return -ENODEV;
+	}
 	if (nvme_identify_ns(ns->ctrl, ns->ns_id, &id)) {
 		dev_warn(ns->ctrl->dev, "%s: Identify failure nvme%dn%d\n",
 				__func__, ns->ctrl->instance, ns->ns_id);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 19a64b2..e4b4110 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -117,6 +117,7 @@ struct nvme_ns {
 	unsigned long flags;
 
 #define NVME_NS_REMOVING 0
+#define NVME_NS_DEAD     1
 
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index a18e4ab..7fd8a54 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -678,7 +678,9 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 
 	spin_lock_irq(&nvmeq->q_lock);
 	if (unlikely(nvmeq->cq_vector < 0)) {
-		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		ret = test_bit(NVME_NS_DEAD, &ns->flags) ?
+					BLK_MQ_RQ_QUEUE_ERROR :
+					BLK_MQ_RQ_QUEUE_BUSY;
 		spin_unlock_irq(&nvmeq->q_lock);
 		goto out;
 	}
@@ -1910,6 +1912,14 @@ static void nvme_kill_ns_queues(struct nvme_dev *dev)
 		if (!kref_get_unless_zero(&ns->kref))
 			continue;
 
+		/*
+		 * Revalidating a dead namespace sets capacity to 0. This will
+		 * end buffered writers dirtying more pages that can't be
+		 * synced.
+		 */
+		if (!test_and_set_bit(NVME_NS_DEAD, &ns->flags))
+			revalidate_disk(ns->disk);
+
 		blk_set_queue_dying(ns->queue);
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_mq_start_stopped_hw_queues(ns->queue, true);
-- 
2.6.2.307.g37023ba



From hch at infradead.org  Wed Feb 10 10:20:51 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:20:51 -0800
Subject: [PATCH for-4.5 04/13] NVMe: Set queue limits max_dev_sectors
In-Reply-To: <1455128250-5984-5-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-5-git-send-email-keith.busch@intel.com>
Message-ID: <20160210182051.GA28235@infradead.org>

On Wed, Feb 10, 2016 at 11:17:21AM -0700, Keith Busch wrote:
> Signed-off-by: Keith Busch <keith.busch at intel.com>

why?



From keith.busch at intel.com  Wed Feb 10 10:24:17 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 18:24:17 +0000
Subject: [PATCH for-4.5 04/13] NVMe: Set queue limits max_dev_sectors
In-Reply-To: <20160210182051.GA28235@infradead.org>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-5-git-send-email-keith.busch@intel.com>
 <20160210182051.GA28235@infradead.org>
Message-ID: <20160210182416.GC32064@localhost.localdomain>

On Wed, Feb 10, 2016 at 10:20:51AM -0800, Christoph Hellwig wrote:
> On Wed, Feb 10, 2016 at 11:17:21AM -0700, Keith Busch wrote:
> > Signed-off-by: Keith Busch <keith.busch at intel.com>
> 
> why?

Ah, missing change log here.

It is so the queue limits will be correct. The max_hw_sectors will be
wrong without this.


From hch at infradead.org  Wed Feb 10 10:36:27 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:36:27 -0800
Subject: [PATCH for-4.5 03/13] NVMe: Allow request merges
In-Reply-To: <1455128250-5984-4-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-4-git-send-email-keith.busch@intel.com>
Message-ID: <20160210183627.GA7512@infradead.org>

On Wed, Feb 10, 2016 at 11:17:20AM -0700, Keith Busch wrote:
> It is generally more efficient to submit larger IO.

In general it is, but it might be good to try to understand
why this flag was set in the first place, and why the argument
for that isn't true anymore, or never was.


From axboe at fb.com  Wed Feb 10 10:37:05 2016
From: axboe at fb.com (Jens Axboe)
Date: Wed, 10 Feb 2016 11:37:05 -0700
Subject: [PATCH for-4.5 03/13] NVMe: Allow request merges
In-Reply-To: <20160210183627.GA7512@infradead.org>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-4-git-send-email-keith.busch@intel.com>
 <20160210183627.GA7512@infradead.org>
Message-ID: <56BB8351.8040401@fb.com>

On 02/10/2016 11:36 AM, Christoph Hellwig wrote:
> On Wed, Feb 10, 2016 at 11:17:20AM -0700, Keith Busch wrote:
>> It is generally more efficient to submit larger IO.
>
> In general it is, but it might be good to try to understand
> why this flag was set in the first place, and why the argument
> for that isn't true anymore, or never was.

IIRC, it was set when the conversion to blk-mq happened, since the 
original driver could not do merges being bio based.

-- 
Jens Axboe



From hch at infradead.org  Wed Feb 10 10:40:31 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:40:31 -0800
Subject: [PATCH for-4.5 04/13] NVMe: Set queue limits max_dev_sectors
In-Reply-To: <20160210182416.GC32064@localhost.localdomain>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-5-git-send-email-keith.busch@intel.com>
 <20160210182051.GA28235@infradead.org>
 <20160210182416.GC32064@localhost.localdomain>
Message-ID: <20160210184031.GB7512@infradead.org>

On Wed, Feb 10, 2016 at 06:24:17PM +0000, Keith Busch wrote:
> On Wed, Feb 10, 2016 at 10:20:51AM -0800, Christoph Hellwig wrote:
> > On Wed, Feb 10, 2016 at 11:17:21AM -0700, Keith Busch wrote:
> > > Signed-off-by: Keith Busch <keith.busch at intel.com>
> > 
> > why?
> 
> Ah, missing change log here.
> 
> It is so the queue limits will be correct. The max_hw_sectors will be
> wrong without this.

How so?  We only ever look at max_dev_sectors in blk_queue_max_hw_sectors
and blk_stack_limits, and both do a min_not_zero.  A zero max_dev_sectors
should thus be perfectly fine, and in fact no one but the SCSI disk
driver even bothers to set one.

Ohh, I see now - blk_set_default_limits initializes max_dev_sectors to
BLK_SAFE_MAX_SECTORS instead of leaving it as zero.  I think that's the
issue which needs to be fixed.



From hch at infradead.org  Wed Feb 10 10:46:41 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:46:41 -0800
Subject: [PATCH for-4.5 06/13] NVMe: Remove WQ_MEM_RECLAIM from nvme work
 queue
In-Reply-To: <1455128250-5984-7-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-7-git-send-email-keith.busch@intel.com>
Message-ID: <20160210184641.GA26933@infradead.org>

On Wed, Feb 10, 2016 at 11:17:23AM -0700, Keith Busch wrote:
> This isn't used for work in the memory reclaim path, and we may need
> to sync with work queues that also are not flagged memory relaim. This
> fixes a kernel warning if we ever do sync with such a work queue.

We do need it during memory reclaim: memory reclaim in general
does I/O, which can be on NVMe.  We then need the workqueue to
abort a command or reset an overloaded controller to make progress.
Not having WQ_MEM_RECLAIM risks deadlocks in heavily loaded systems.


From hch at infradead.org  Wed Feb 10 10:47:09 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:47:09 -0800
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues
In-Reply-To: <1455128250-5984-8-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-8-git-send-email-keith.busch@intel.com>
Message-ID: <20160210184709.GB26933@infradead.org>

On Wed, Feb 10, 2016 at 11:17:24AM -0700, Keith Busch wrote:
> It's possible a request may get to the driver after the nvme queue was
> disabled. This has the request requeue if that happens.
> 
> Note the request is still "started" by the driver, but requeuing will
> clear the start state for timeout handling.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Wed Feb 10 10:48:33 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:48:33 -0800
Subject: [PATCH for-4.5 08/13] NVMe: Poll device while still active
 during remove
In-Reply-To: <1455128250-5984-9-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-9-git-send-email-keith.busch@intel.com>
Message-ID: <20160210184833.GC26933@infradead.org>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Wed Feb 10 10:53:16 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:53:16 -0800
Subject: [PATCH for-4.5 09/13] NVMe: Simplify device reset failure
In-Reply-To: <1455128250-5984-10-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-10-git-send-email-keith.busch@intel.com>
Message-ID: <20160210185316.GA10459@infradead.org>

> +	dev_warn(dev->dev, "Removing after probe failure status:%d\n", status);

Why not space after the colon?

>  static void nvme_reset_work(struct work_struct *work)
>  {
>  	struct nvme_dev *dev = container_of(work, struct nvme_dev, reset_work);
> -	int result;
> +	int result = ENODEV;

negative errors, please.

> - remove:
> -	nvme_dev_list_remove(dev);
> - free_tags:
> -	nvme_dev_remove_admin(dev);
> -	blk_put_queue(dev->ctrl.admin_q);
> -	dev->ctrl.admin_q = NULL;
> -	dev->queues[0]->tags = NULL;
> - disable:
> -	nvme_disable_admin_queue(dev, false);
> - unmap:
> -	nvme_dev_unmap(dev);
>   out:
> -	nvme_remove_dead_ctrl(dev);
> +	nvme_remove_dead_ctrl(dev, result);

So this relies on the cleanup path to handle all states
of partial initialization.  I suspect you've audited this in
the current version, but the remove path could use a comment
stating that any edit will have to obey this restriction.


From hch at infradead.org  Wed Feb 10 10:54:04 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2016 10:54:04 -0800
Subject: [PATCH for-4.5 13/13] NVMe: Rate limit nvme IO warnings
In-Reply-To: <1455128250-5984-14-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-14-git-send-email-keith.busch@intel.com>
Message-ID: <20160210185404.GB10459@infradead.org>

On Wed, Feb 10, 2016 at 11:17:30AM -0700, Keith Busch wrote:
> We don't need to spam the kernel logs with thousands of messages. We
> can infer all IO's are being cancelled with fewer messages.

Do we need this message at all?


From keith.busch at intel.com  Wed Feb 10 11:49:07 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 19:49:07 +0000
Subject: [PATCH for-4.5 04/13] NVMe: Set queue limits max_dev_sectors
In-Reply-To: <20160210184031.GB7512@infradead.org>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-5-git-send-email-keith.busch@intel.com>
 <20160210182051.GA28235@infradead.org>
 <20160210182416.GC32064@localhost.localdomain>
 <20160210184031.GB7512@infradead.org>
Message-ID: <20160210194907.GA23402@localhost.localdomain>

On Wed, Feb 10, 2016 at 10:40:31AM -0800, Christoph Hellwig wrote:
> Ohh, I see now - blk_set_default_limits initializes max_dev_sectors to
> BLK_SAFE_MAX_SECTORS instead of leaving it as zero.  I think that's the
> issue which needs to be fixed.

Heh, that's pretty funny. I mentioned something along those lines after
I broke the the block layer in rc1, but thought it was done that way on
purpose. :)

So instead of this nvme specific patch, we should do this?

---
diff --git a/block/blk-settings.c b/block/blk-settings.c
index dd49735..c7bb666 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -91,8 +91,8 @@ void blk_set_default_limits(struct queue_limits *lim)
        lim->seg_boundary_mask = BLK_SEG_BOUNDARY_MASK;
        lim->virt_boundary_mask = 0;
        lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
-       lim->max_sectors = lim->max_dev_sectors = lim->max_hw_sectors =
-               BLK_SAFE_MAX_SECTORS;
+       lim->max_sectors = lim->max_hw_sectors = BLK_SAFE_MAX_SECTORS;
+       lim->max_dev_sectors = 0;
        lim->chunk_sectors = 0;
        lim->max_write_same_sectors = 0;
        lim->max_discard_sectors = 0;
--


From keith.busch at intel.com  Wed Feb 10 13:01:03 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 21:01:03 +0000
Subject: [PATCH for-4.5 03/13] NVMe: Allow request merges
In-Reply-To: <56BB8351.8040401@fb.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-4-git-send-email-keith.busch@intel.com>
 <20160210183627.GA7512@infradead.org> <56BB8351.8040401@fb.com>
Message-ID: <20160210210103.GB23402@localhost.localdomain>

On Wed, Feb 10, 2016 at 11:37:05AM -0700, Jens Axboe wrote:
> On 02/10/2016 11:36 AM, Christoph Hellwig wrote:
> >In general it is, but it might be good to try to understand
> >why this flag was set in the first place, and why the argument
> >for that isn't true anymore, or never was.
> 
> IIRC, it was set when the conversion to blk-mq happened, since the
> original driver could not do merges being bio based.

This actually carried over from when it was still bio-based. The flag
was set in the driver's initial commit, but should not have been.


From keith.busch at intel.com  Wed Feb 10 13:10:26 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 21:10:26 +0000
Subject: [PATCH for-4.5 13/13] NVMe: Rate limit nvme IO warnings
In-Reply-To: <20160210185404.GB10459@infradead.org>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-14-git-send-email-keith.busch@intel.com>
 <20160210185404.GB10459@infradead.org>
Message-ID: <20160210211025.GC23402@localhost.localdomain>

On Wed, Feb 10, 2016 at 10:54:04AM -0800, Christoph Hellwig wrote:
> On Wed, Feb 10, 2016 at 11:17:30AM -0700, Keith Busch wrote:
> > We don't need to spam the kernel logs with thousands of messages. We
> > can infer all IO's are being cancelled with fewer messages.
> 
> Do we need this message at all?

I don't find it useful for anything other than confirming a test hit
that path. Delete it?


From axboe at fb.com  Wed Feb 10 13:19:23 2016
From: axboe at fb.com (Jens Axboe)
Date: Wed, 10 Feb 2016 14:19:23 -0700
Subject: [PATCH for-4.5 03/13] NVMe: Allow request merges
In-Reply-To: <20160210210103.GB23402@localhost.localdomain>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-4-git-send-email-keith.busch@intel.com>
 <20160210183627.GA7512@infradead.org> <56BB8351.8040401@fb.com>
 <20160210210103.GB23402@localhost.localdomain>
Message-ID: <56BBA95B.8070203@fb.com>

On 02/10/2016 02:01 PM, Keith Busch wrote:
> On Wed, Feb 10, 2016 at 11:37:05AM -0700, Jens Axboe wrote:
>> On 02/10/2016 11:36 AM, Christoph Hellwig wrote:
>>> In general it is, but it might be good to try to understand
>>> why this flag was set in the first place, and why the argument
>>> for that isn't true anymore, or never was.
>>
>> IIRC, it was set when the conversion to blk-mq happened, since the
>> original driver could not do merges being bio based.
>
> This actually carried over from when it was still bio-based. The flag
> was set in the driver's initial commit, but should not have been.

Yeah, but the flag was a no-op back then, since no merging is done for 
bio based drivers. So flag or no flag back then, no merging would have 
been done.

-- 
Jens Axboe



From martin.petersen at oracle.com  Wed Feb 10 14:53:10 2016
From: martin.petersen at oracle.com (Martin K. Petersen)
Date: Wed, 10 Feb 2016 17:53:10 -0500
Subject: [PATCH for-4.5 04/13] NVMe: Set queue limits max_dev_sectors
In-Reply-To: <20160210194907.GA23402@localhost.localdomain> (Keith Busch's
 message of "Wed, 10 Feb 2016 19:49:07 +0000")
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-5-git-send-email-keith.busch@intel.com>
 <20160210182051.GA28235@infradead.org>
 <20160210182416.GC32064@localhost.localdomain>
 <20160210184031.GB7512@infradead.org>
 <20160210194907.GA23402@localhost.localdomain>
Message-ID: <yq1bn7o9nzd.fsf@sermon.lab.mkp.net>

>>>>> "Keith" == Keith Busch <keith.busch at intel.com> writes:

Keith> Heh, that's pretty funny. I mentioned something along those lines
Keith> after I broke the the block layer in rc1, but thought it was done
Keith> that way on purpose. :)

Keith> So instead of this nvme specific patch, we should do this?

Yes. Thinko on my part, sorry about that.

Acked-by: Martin K. Petersen <martin.petersen at oracle.com>

-- 
Martin K. Petersen	Oracle Linux Engineering


From keith.busch at intel.com  Wed Feb 10 15:37:47 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 23:37:47 +0000
Subject: [PATCH for-4.5 06/13] NVMe: Remove WQ_MEM_RECLAIM from nvme
 work queue
In-Reply-To: <20160210184641.GA26933@infradead.org>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-7-git-send-email-keith.busch@intel.com>
 <20160210184641.GA26933@infradead.org>
Message-ID: <20160210233747.GA25988@localhost.localdomain>

On Wed, Feb 10, 2016 at 10:46:41AM -0800, Christoph Hellwig wrote:
> On Wed, Feb 10, 2016 at 11:17:23AM -0700, Keith Busch wrote:
> > This isn't used for work in the memory reclaim path, and we may need
> > to sync with work queues that also are not flagged memory relaim. This
> > fixes a kernel warning if we ever do sync with such a work queue.
> 
> We do need it during memory reclaim: memory reclaim in general
> does I/O, which can be on NVMe.  We then need the workqueue to
> abort a command or reset an overloaded controller to make progress.
> Not having WQ_MEM_RECLAIM risks deadlocks in heavily loaded systems.

Darn. Invalidating a disk drains lru, which syncs with work scheduled
on the system_wq. Syncing with that from a memory reclaim work queue
hits a kernel warning.

That lru drain work is reclaiming memory, though. Does this need
to be using a WQ_MEM_RECLAIM queue, then?

This is the alternate patch I didn't plan to submit:

---
diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h
index 0e32bc7..f7cc91e 100644
--- a/include/linux/workqueue.h
+++ b/include/linux/workqueue.h
@@ -356,6 +359,7 @@ extern struct workqueue_struct *system_unbound_wq;
 extern struct workqueue_struct *system_freezable_wq;
 extern struct workqueue_struct *system_power_efficient_wq;
 extern struct workqueue_struct *system_freezable_power_efficient_wq;
+extern struct workqueue_struct *system_mem_wq;

 extern struct workqueue_struct *
 __alloc_workqueue_key(const char *fmt, unsigned int flags, int max_active,
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 61a0264..57a50d2 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -5483,10 +5483,13 @@ static int __init init_workqueues(void)
        system_freezable_power_efficient_wq = alloc_workqueue("events_freezable_power_efficient",
                                              WQ_FREEZABLE | WQ_POWER_EFFICIENT,
                                              0);
+       system_mem_wq = alloc_workqueue("events_mem_unbound", WQ_UNBOUND | WQ_MEM_RECLAIM,
+                                           WQ_UNBOUND_MAX_ACTIVE);
        BUG_ON(!system_wq || !system_highpri_wq || !system_long_wq ||
               !system_unbound_wq || !system_freezable_wq ||
               !system_power_efficient_wq ||
-              !system_freezable_power_efficient_wq);
+              !system_freezable_power_efficient_wq ||
+              !system_mem_wq);

        wq_watchdog_init();

diff --git a/mm/swap.c b/mm/swap.c
index 09fe5e9..eecf98a 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -685,7 +685,7 @@ void lru_add_drain_all(void)
                    pagevec_count(&per_cpu(lru_deactivate_pvecs, cpu)) ||
                    need_activate_page_drain(cpu)) {
                        INIT_WORK(work, lru_add_drain_per_cpu);
-                       schedule_work_on(cpu, work);
+                       queue_work_on(cpu, system_mem_wq, work);
                        cpumask_set_cpu(cpu, &has_work);
                }
        }
--


From keith.busch at intel.com  Wed Feb 10 15:52:47 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 10 Feb 2016 16:52:47 -0700
Subject: [PATCH] block: Initialize max_dev_sectors to 0
Message-ID: <1455148367-25208-1-git-send-email-keith.busch@intel.com>

The new queue limit is not used by the majority of block drivers, and
should be initialized to 0 for the driver's requested settings to be used.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Cc: Christoph Hellwig <hch at infradead.org>
Acked-by: Martin K. Petersen <martin.petersen at oracle.com>
---
 block/blk-settings.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/blk-settings.c b/block/blk-settings.c
index dd49735..c7bb666 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -91,8 +91,8 @@ void blk_set_default_limits(struct queue_limits *lim)
 	lim->seg_boundary_mask = BLK_SEG_BOUNDARY_MASK;
 	lim->virt_boundary_mask = 0;
 	lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
-	lim->max_sectors = lim->max_dev_sectors = lim->max_hw_sectors =
-		BLK_SAFE_MAX_SECTORS;
+	lim->max_sectors = lim->max_hw_sectors = BLK_SAFE_MAX_SECTORS;
+	lim->max_dev_sectors = 0;
 	lim->chunk_sectors = 0;
 	lim->max_write_same_sectors = 0;
 	lim->max_discard_sectors = 0;
-- 
2.6.2.307.g37023ba



From snitzer at redhat.com  Wed Feb 10 19:35:41 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Wed, 10 Feb 2016 22:35:41 -0500
Subject: RCU-ified dm-mpath for testing/review
In-Reply-To: <20160211015030.GA4481@redhat.com>
References: <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com> <56B99A49.5050400@suse.de>
 <20160209145547.GA21623@redhat.com> <56BA0689.9030007@suse.de>
 <20160210004518.GA23646@redhat.com>
 <20160211015030.GA4481@redhat.com>
Message-ID: <20160211033541.GA4900@redhat.com>

On Wed, Feb 10 2016 at  8:50pm -0500,
Mike Snitzer <snitzer at redhat.com> wrote:

> On Tue, Feb 09 2016 at  7:45pm -0500,
> Mike Snitzer <snitzer at redhat.com> wrote:
> 
> > 
> > OK, I took a crack at embracing RCU.  Only slightly better performance
> > on my single NUMA node testbed.  (But I'll have to track down a system
> > with multiple NUMA nodes to do any justice to the next wave of this
> > optimization effort)
> > 
> > This RCU work is very heavy-handed and way too fiddley (there could
> > easily be bugs).  Anyway, please see:
> > http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel2&id=d80a7e4f8b5be9c81e4d452137623b003fa64745
> > 
> > But this might give you something to build on to arrive at something
> > more scalable?
> 
> I've a bit more polished version of this work (broken up into multiple
> commits, with some fixes, etc) here:
> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
> 
> Hannes and/or Sagi, if you get a chance to try this on your NUMA system
> please let me know how it goes.

FYI, with these changes my single NUMA node testbed's read IOPs went
from:

 ~1310K to ~1410K w/ nr_hw_queues dm-mq=4 and null_blk=4
 ~1330K to ~1415K w/ nr_hw_queues dm-mq=4 and null_blk=12
 ~1365K to ~1425K w/ nr_hw_queues dm-mq=12 and null_blk=12


From m at bjorling.me  Thu Feb 11 03:36:28 2016
From: m at bjorling.me (=?UTF-8?q?Matias=20Bj=C3=B8rling?=)
Date: Thu, 11 Feb 2016 12:36:28 +0100
Subject: nvme: lightnvm: return ppa completion status
Message-ID: <1455190589-20016-1-git-send-email-m@bjorling.me>

For LightNVM, the PPA completion status is communicated through DW0 and
DW1 of the completion queue entry. The patch is based on top of
Christoph's

 "nvme: return the whole CQE through the request passthrough interface"

that exposes the full CQE during completion.

Matias Bj?rling (1):
  nvme: lightnvm: return ppa completion status

 drivers/nvme/host/lightnvm.c | 17 +++++++++++++++--
 include/linux/lightnvm.h     |  1 +
 2 files changed, 16 insertions(+), 2 deletions(-)

-- 
2.1.4



From m at bjorling.me  Thu Feb 11 03:36:29 2016
From: m at bjorling.me (=?UTF-8?q?Matias=20Bj=C3=B8rling?=)
Date: Thu, 11 Feb 2016 12:36:29 +0100
Subject: [PATCH] nvme: lightnvm: return ppa completion status
In-Reply-To: <1455190589-20016-1-git-send-email-m@bjorling.me>
References: <1455190589-20016-1-git-send-email-m@bjorling.me>
Message-ID: <1455190589-20016-2-git-send-email-m@bjorling.me>

PPAs sent to device is separately acknowledge in a 64bit status
variable. The status is stored in DW0 and DW1 of the completion queue
entry. Store this status inside the nvm_rq for further processing.

This can later be used to implement retry techniques for failed writes
and reads.

Signed-off-by: Matias Bj?rling <m at bjorling.me>
---
 drivers/nvme/host/lightnvm.c | 17 +++++++++++++++--
 include/linux/lightnvm.h     |  1 +
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/lightnvm.c b/drivers/nvme/host/lightnvm.c
index 55dab93..bbb1dbf 100644
--- a/drivers/nvme/host/lightnvm.c
+++ b/drivers/nvme/host/lightnvm.c
@@ -146,6 +146,14 @@ struct nvme_nvm_command {
 	};
 };
 
+struct nvme_nvm_completion {
+	__le64	result;		/* Used by LightNVM to return ppa completions */
+	__le16	sq_head;	/* how much of this queue may be reclaimed */
+	__le16	sq_id;		/* submission queue that generated this entry */
+	__u16	command_id;	/* of the command which completed */
+	__le16	status;		/* did the command fail, and if so, why? */
+};
+
 #define NVME_NVM_LP_MLC_PAIRS 886
 struct nvme_nvm_lp_mlc {
 	__u16			num_pairs;
@@ -471,6 +479,10 @@ static inline void nvme_nvm_rqtocmd(struct request *rq, struct nvm_rq *rqd,
 static void nvme_nvm_end_io(struct request *rq, int error)
 {
 	struct nvm_rq *rqd = rq->end_io_data;
+	struct nvme_nvm_completion *cqe = rq->special;
+
+	if (cqe)
+		rqd->ppa_status = le64_to_cpu(cqe->result);
 
 	nvm_end_io(rqd, error);
 
@@ -490,7 +502,8 @@ static int nvme_nvm_submit_io(struct nvm_dev *dev, struct nvm_rq *rqd)
 	if (IS_ERR(rq))
 		return -ENOMEM;
 
-	cmd = kzalloc(sizeof(struct nvme_nvm_command), GFP_KERNEL);
+	cmd = kzalloc(sizeof(struct nvme_nvm_command) +
+				sizeof(struct nvme_nvm_completion), GFP_KERNEL);
 	if (!cmd) {
 		blk_mq_free_request(rq);
 		return -ENOMEM;
@@ -509,7 +522,7 @@ static int nvme_nvm_submit_io(struct nvm_dev *dev, struct nvm_rq *rqd)
 
 	rq->cmd = (unsigned char *)cmd;
 	rq->cmd_len = sizeof(struct nvme_nvm_command);
-	rq->special = (void *)0;
+	rq->special = cmd + 1;
 
 	rq->end_io_data = rqd;
 
diff --git a/include/linux/lightnvm.h b/include/linux/lightnvm.h
index 0fb3745..201b2a3 100644
--- a/include/linux/lightnvm.h
+++ b/include/linux/lightnvm.h
@@ -242,6 +242,7 @@ struct nvm_rq {
 	uint16_t nr_pages;
 	uint16_t flags;
 
+	u64 ppa_status; /* ppa media status */
 	int error;
 };
 
-- 
2.1.4



From jthumshirn at suse.de  Thu Feb 11 04:13:47 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:13:47 +0100
Subject: [PATCH for-4.5 01/13] blk-mq: End unstarted requests on dying
 queue
In-Reply-To: <1455128250-5984-2-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-2-git-send-email-keith.busch@intel.com>
Message-ID: <20160211121347.GC27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:18AM -0700, Keith Busch wrote:
> Go directly to ending a request if it wasn't started. Previously, completing a
> request may invoke a driver callback for a request it didn't initialize.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> Acked-by: Christoph Hellwig <hch at lst.de>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 11 04:14:30 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:14:30 +0100
Subject: [PATCH for-4.5 02/13] NVMe: Fix io incapable return values
In-Reply-To: <1455128250-5984-3-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-3-git-send-email-keith.busch@intel.com>
Message-ID: <20160211121430.GD27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:19AM -0700, Keith Busch wrote:
> The function returns true when the controller can't handle IO.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
> Acked-by: Christoph Hellwig <hch at lst.de>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 11 04:15:41 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:15:41 +0100
Subject: [PATCH for-4.5 03/13] NVMe: Allow request merges
In-Reply-To: <1455128250-5984-4-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-4-git-send-email-keith.busch@intel.com>
Message-ID: <20160211121541.GE27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:20AM -0700, Keith Busch wrote:
> It is generally more efficient to submit larger IO.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 11 04:19:44 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:19:44 +0100
Subject: [PATCH for-4.5 05/13] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455128250-5984-6-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-6-git-send-email-keith.busch@intel.com>
Message-ID: <20160211121944.GF27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:22AM -0700, Keith Busch wrote:
> Namespace removal holds the namespace list mutex. If the controller
> fails during namespace removal, the reset work will deadlock trying to
> acquire the mutex, and no progress will be made.
> 
> This patch completes the potentially blocking parts of namespace
> removal in a controller work queue so that reset work can cleanup a
> failed controller.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 11 04:22:05 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:22:05 +0100
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues
In-Reply-To: <1455128250-5984-8-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-8-git-send-email-keith.busch@intel.com>
Message-ID: <20160211122205.GG27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:24AM -0700, Keith Busch wrote:
> It's possible a request may get to the driver after the nvme queue was
> disabled. This has the request requeue if that happens.
> 
> Note the request is still "started" by the driver, but requeuing will
> clear the start state for timeout handling.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 11 04:26:03 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:26:03 +0100
Subject: [PATCH for-4.5 08/13] NVMe: Poll device while still active
 during remove
In-Reply-To: <1455128250-5984-9-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-9-git-send-email-keith.busch@intel.com>
Message-ID: <20160211122602.GH27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:25AM -0700, Keith Busch wrote:
> A device failure or link down wouldn't have been detected during namespace
> removal. This patch keeps the device in the list for polling so that the
> thread may see such failure and initiate a reset. The device is removed
> from the list after disable, so we can safely flush the reset work as
> it can't be requeued when disable completes.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 11 04:28:24 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:28:24 +0100
Subject: [PATCH for-4.5 09/13] NVMe: Simplify device reset failure
In-Reply-To: <1455128250-5984-10-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-10-git-send-email-keith.busch@intel.com>
Message-ID: <20160211122824.GI27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:26AM -0700, Keith Busch wrote:
> A reset failure schedules the device to unbind from the driver through
> the pci driver's remove. This cleans up all intialization, so there is
> no need to duplicate the potentially racy cleanup.
> 
> To help understand why a reset failed, the status is logged with the
> existed warning message.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From sagig at dev.mellanox.co.il  Thu Feb 11 04:29:59 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 14:29:59 +0200
Subject: [PATCH for-4.5 13/13] NVMe: Rate limit nvme IO warnings
In-Reply-To: <20160210211025.GC23402@localhost.localdomain>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-14-git-send-email-keith.busch@intel.com>
 <20160210185404.GB10459@infradead.org>
 <20160210211025.GC23402@localhost.localdomain>
Message-ID: <56BC7EC7.6020208@dev.mellanox.co.il>


>> Do we need this message at all?
>
> I don't find it useful for anything other than confirming a test hit
> that path. Delete it?

Perhaps we can have it dev_dbg.


From sagig at dev.mellanox.co.il  Thu Feb 11 04:30:26 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 14:30:26 +0200
Subject: [PATCH for-4.5 01/13] blk-mq: End unstarted requests on dying
 queue
In-Reply-To: <1455128250-5984-2-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-2-git-send-email-keith.busch@intel.com>
Message-ID: <56BC7EE2.3070501@dev.mellanox.co.il>

Again,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Thu Feb 11 04:33:37 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 14:33:37 +0200
Subject: [PATCH for-4.5 03/13] NVMe: Allow request merges
In-Reply-To: <1455128250-5984-4-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-4-git-send-email-keith.busch@intel.com>
Message-ID: <56BC7FA1.2020101@dev.mellanox.co.il>


> It is generally more efficient to submit larger IO.
>
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Oh yes, I wanted that for some time now, ever since Jens
and I addressed some issues with front merges and gaps which
detected by iser (I had this patch in my pipe but didn't have time to
test...)

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From jthumshirn at suse.de  Thu Feb 11 04:34:02 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:34:02 +0100
Subject: [PATCH for-4.5 10/13] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <1455128250-5984-11-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-11-git-send-email-keith.busch@intel.com>
Message-ID: <20160211123402.GJ27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:27AM -0700, Keith Busch wrote:
> This moves the dead queue handling out of the namespace removal path
> and into the reset failure path. It fixes a deadlock condition if the
> controller fails or link down during del_gendisk.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From sagig at dev.mellanox.co.il  Thu Feb 11 04:41:05 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 14:41:05 +0200
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues
In-Reply-To: <1455128250-5984-8-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-8-git-send-email-keith.busch@intel.com>
Message-ID: <56BC8161.6090400@dev.mellanox.co.il>

Looks ok,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Thu Feb 11 04:42:25 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 14:42:25 +0200
Subject: [PATCH for-4.5 08/13] NVMe: Poll device while still active during
 remove
In-Reply-To: <1455128250-5984-9-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-9-git-send-email-keith.busch@intel.com>
Message-ID: <56BC81B1.8080009@dev.mellanox.co.il>

Lookd good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From jthumshirn at suse.de  Thu Feb 11 04:43:25 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:43:25 +0100
Subject: [PATCH for-4.5 11/13] NVMe: Dead namespace handling
In-Reply-To: <1455128250-5984-12-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-12-git-send-email-keith.busch@intel.com>
Message-ID: <20160211124325.GK27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:28AM -0700, Keith Busch wrote:
> This adds a "dead" state to a namespace and revalidates such a namespace
> to 0 capacity. This will force buffered writers to stop writing pages
> that can't be synced, and ends requests in failure if any is submitted
> to such a namespace.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 11 04:44:04 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 11 Feb 2016 13:44:04 +0100
Subject: [PATCH for-4.5 12/13] NVMe: Mark queues as dead on degraded
 controller
In-Reply-To: <1455128250-5984-13-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-13-git-send-email-keith.busch@intel.com>
Message-ID: <20160211124404.GL27969@c203.arch.suse.de>

On Wed, Feb 10, 2016 at 11:17:29AM -0700, Keith Busch wrote:
> If a previously optimal controller goes degraded, the IO queues will
> stop working, but the controller is still managable. Prior to removing
> namespace, we need to fail the queues so queued and new IO will fail
> and allow namespace removal to progress.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From sagig at dev.mellanox.co.il  Thu Feb 11 04:50:54 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 14:50:54 +0200
Subject: [PATCH for-4.5 10/13] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <1455128250-5984-11-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-11-git-send-email-keith.busch@intel.com>
Message-ID: <56BC83AE.8080702@dev.mellanox.co.il>



On 10/02/2016 20:17, Keith Busch wrote:
> This moves the dead queue handling out of the namespace removal path
> and into the reset failure path. It fixes a deadlock condition if the
> controller fails or link down during del_gendisk.

How does it fix the deadlock?

>
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---
>   drivers/nvme/host/core.c | 19 ++-----------------
>   drivers/nvme/host/nvme.h |  1 +
>   drivers/nvme/host/pci.c  | 22 ++++++++++++++++++++++
>   3 files changed, 25 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index be27f9f..41b595c 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -64,7 +64,7 @@ static void nvme_free_ns(struct kref *kref)
>   	kfree(ns);
>   }
>
> -static void nvme_put_ns(struct nvme_ns *ns)
> +void nvme_put_ns(struct nvme_ns *ns)
>   {
>   	kref_put(&ns->kref, nvme_free_ns);
>   }
> @@ -1113,28 +1113,13 @@ static struct nvme_ns *nvme_find_ns(struct nvme_ctrl *ctrl, unsigned nsid)
>   static void nvme_ns_remove_work(struct work_struct *work)
>   {
>   	struct nvme_ns *ns = container_of(work, struct nvme_ns, remove_work);
> -	bool kill = nvme_io_incapable(ns->ctrl) &&
> -			!blk_queue_dying(ns->queue);
> -
> -	if (kill) {
> -		blk_set_queue_dying(ns->queue);
> -
> -		/*
> -		 * The controller was shutdown first if we got here through
> -		 * device removal. The shutdown may requeue outstanding
> -		 * requests. These need to be aborted immediately so
> -		 * del_gendisk doesn't block indefinitely for their completion.
> -		 */
> -		blk_mq_abort_requeue_list(ns->queue);
> -	}
> +
>   	if (ns->disk->flags & GENHD_FL_UP) {
>   		if (blk_get_integrity(ns->disk))
>   			blk_integrity_unregister(ns->disk);
>   		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
>   					&nvme_ns_attr_group);
>   		del_gendisk(ns->disk);
> -	}
> -	if (kill || !blk_queue_dying(ns->queue)) {
>   		blk_mq_abort_requeue_list(ns->queue);
>   		blk_cleanup_queue(ns->queue);
>   	}
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index d330512..19a64b2 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -270,6 +270,7 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
>   int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
>   			dma_addr_t dma_addr, u32 *result);
>   int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count);
> +void nvme_put_ns(struct nvme_ns *ns);
>
>   extern spinlock_t dev_list_lock;
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 3381bac..a18e4ab 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1898,11 +1898,33 @@ static void nvme_pci_free_ctrl(struct nvme_ctrl *ctrl)
>   	kfree(dev);
>   }
>
> +static void nvme_kill_ns_queues(struct nvme_dev *dev)
> +{
> +	struct nvme_ns *ns;
> +	struct nvme_ctrl *ctrl = &dev->ctrl;
> +
> +	nvme_dev_disable(dev, false);
> +
> +	mutex_lock(&ctrl->namespaces_mutex);
> +	list_for_each_entry(ns, &ctrl->namespaces, list) {
> +		if (!kref_get_unless_zero(&ns->kref))
> +			continue;
> +
> +		blk_set_queue_dying(ns->queue);
> +		blk_mq_abort_requeue_list(ns->queue);
> +		blk_mq_start_stopped_hw_queues(ns->queue, true);
> +
> +		nvme_put_ns(ns);
> +	}
> +	mutex_unlock(&ctrl->namespaces_mutex);
> +}
> +

Why on earth is this pci specific? This should be in the
core. Aside from that, I'd really prefer if the core can handle this
without having the pci (or other) triggering it explicitly, but if
this must move out of the ns remove then we need documentation on
what are the rules of when the driver needs to call it.


From sagig at dev.mellanox.co.il  Thu Feb 11 04:59:08 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 14:59:08 +0200
Subject: [PATCH for-4.5 11/13] NVMe: Dead namespace handling
In-Reply-To: <1455128250-5984-12-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-12-git-send-email-keith.busch@intel.com>
Message-ID: <56BC859C.6030206@dev.mellanox.co.il>


> This adds a "dead" state to a namespace and revalidates such a namespace
> to 0 capacity. This will force buffered writers to stop writing pages
> that can't be synced, and ends requests in failure if any is submitted
> to such a namespace.

This sorta going towards a namespace state machine (like scsi). Maybe
we need to centralize it correctly instead of adding states that are
relevant is sporadic areas?

>
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---
>   drivers/nvme/host/core.c |  4 ++++
>   drivers/nvme/host/nvme.h |  1 +
>   drivers/nvme/host/pci.c  | 12 +++++++++++-
>   3 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 41b595c..84e9f41 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -560,6 +560,10 @@ static int nvme_revalidate_disk(struct gendisk *disk)
>   	u16 old_ms;
>   	unsigned short bs;
>
> +	if (test_bit(NVME_NS_DEAD, &ns->flags)) {
> +		set_capacity(disk, 0);
> +		return -ENODEV;
> +	}
>   	if (nvme_identify_ns(ns->ctrl, ns->ns_id, &id)) {
>   		dev_warn(ns->ctrl->dev, "%s: Identify failure nvme%dn%d\n",
>   				__func__, ns->ctrl->instance, ns->ns_id);
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 19a64b2..e4b4110 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -117,6 +117,7 @@ struct nvme_ns {
>   	unsigned long flags;
>
>   #define NVME_NS_REMOVING 0
> +#define NVME_NS_DEAD     1
>
>   	u64 mode_select_num_blocks;
>   	u32 mode_select_block_len;
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index a18e4ab..7fd8a54 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -678,7 +678,9 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
>
>   	spin_lock_irq(&nvmeq->q_lock);
>   	if (unlikely(nvmeq->cq_vector < 0)) {
> -		ret = BLK_MQ_RQ_QUEUE_BUSY;
> +		ret = test_bit(NVME_NS_DEAD, &ns->flags) ?
> +					BLK_MQ_RQ_QUEUE_ERROR :
> +					BLK_MQ_RQ_QUEUE_BUSY;
>   		spin_unlock_irq(&nvmeq->q_lock);
>   		goto out;
>   	}

I can't say I'm a fan of doing all this in queue_rq...

besides why is this state check under the cq_vector < 0 condition?
This is really confusing...


From sagig at dev.mellanox.co.il  Thu Feb 11 05:00:48 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 15:00:48 +0200
Subject: [PATCH for-4.5 12/13] NVMe: Mark queues as dead on degraded
 controller
In-Reply-To: <1455128250-5984-13-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-13-git-send-email-keith.busch@intel.com>
Message-ID: <56BC8600.1010902@dev.mellanox.co.il>


> If a previously optimal controller goes degraded, the IO queues will
> stop working, but the controller is still managable. Prior to removing
> namespace, we need to fail the queues so queued and new IO will fail
> and allow namespace removal to progress.
>
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---
>   drivers/nvme/host/pci.c | 1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 7fd8a54..18c14f6 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1989,6 +1989,7 @@ static void nvme_reset_work(struct work_struct *work)
>   	 */
>   	if (dev->online_queues < 2) {
>   		dev_warn(dev->dev, "IO queues not created\n");
> +		nvme_kill_ns_queues(dev);

Another example why we need to document when should a driver
call this....


From sagig at dev.mellanox.co.il  Thu Feb 11 05:01:45 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 11 Feb 2016 15:01:45 +0200
Subject: [PATCH] block: Initialize max_dev_sectors to 0
In-Reply-To: <1455148367-25208-1-git-send-email-keith.busch@intel.com>
References: <1455148367-25208-1-git-send-email-keith.busch@intel.com>
Message-ID: <56BC8639.6050006@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From keith.busch at intel.com  Thu Feb 11 06:52:14 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 14:52:14 +0000
Subject: [PATCH for-4.5 06/13] NVMe: Remove WQ_MEM_RECLAIM from nvme
 work queue
In-Reply-To: <20160210184641.GA26933@infradead.org>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-7-git-send-email-keith.busch@intel.com>
 <20160210184641.GA26933@infradead.org>
Message-ID: <20160211145214.GA11908@localhost.localdomain>

On Wed, Feb 10, 2016 at 10:46:41AM -0800, Christoph Hellwig wrote:
> On Wed, Feb 10, 2016 at 11:17:23AM -0700, Keith Busch wrote:
> > This isn't used for work in the memory reclaim path, and we may need
> > to sync with work queues that also are not flagged memory relaim. This
> > fixes a kernel warning if we ever do sync with such a work queue.
> 
> We do need it during memory reclaim: memory reclaim in general
> does I/O, which can be on NVMe.  We then need the workqueue to
> abort a command or reset an overloaded controller to make progress.
> Not having WQ_MEM_RECLAIM risks deadlocks in heavily loaded systems.

There was a more obvious way to do this that took me another day to
see. We schedule remove_work on the system work queue, and all this
cleanup can happen there instead of in the reset_work on the nvme
work queue.

I will drop the unneeded patches from this series, and resubmit the rest
with the reviews, fixes, and recommended code changes and documentation.


From axboe at fb.com  Thu Feb 11 07:07:54 2016
From: axboe at fb.com (Jens Axboe)
Date: Thu, 11 Feb 2016 08:07:54 -0700
Subject: [PATCH v3 0/4] split pci module out of core module
In-Reply-To: <1455127412-23940-1-git-send-email-mlin@kernel.org>
References: <1455127412-23940-1-git-send-email-mlin@kernel.org>
Message-ID: <56BCA3CA.4020808@fb.com>

On 02/10/2016 11:03 AM, Ming Lin wrote:
> From: Ming Lin <ming.l at ssi.samsung.com>
>
> We'll have nvme over fabrics dirver soon.
> This series split nvme pci module out of nvme core module.
>
> nvme.ko is split into 2 modules:
> nvme-core.ko: the core part
> nvme.ko: the PCI driver

Applied for 4.6.

-- 
Jens Axboe



From keith.busch at intel.com  Thu Feb 11 07:07:25 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 15:07:25 +0000
Subject: [PATCH for-4.5 11/13] NVMe: Dead namespace handling
In-Reply-To: <56BC859C.6030206@dev.mellanox.co.il>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-12-git-send-email-keith.busch@intel.com>
 <56BC859C.6030206@dev.mellanox.co.il>
Message-ID: <20160211150724.GB11908@localhost.localdomain>

On Thu, Feb 11, 2016 at 02:59:08PM +0200, Sagi Grimberg wrote:
> >This adds a "dead" state to a namespace and revalidates such a namespace
> >to 0 capacity. This will force buffered writers to stop writing pages
> >that can't be synced, and ends requests in failure if any is submitted
> >to such a namespace.
> 
> This sorta going towards a namespace state machine (like scsi). Maybe
> we need to centralize it correctly instead of adding states that are
> relevant is sporadic areas?

Yeah, we need to borrow the better ideas from scsi. Slowly moving to a
state machine.
 
> >  	if (unlikely(nvmeq->cq_vector < 0)) {
> >-		ret = BLK_MQ_RQ_QUEUE_BUSY;
> >+		ret = test_bit(NVME_NS_DEAD, &ns->flags) ?
> >+					BLK_MQ_RQ_QUEUE_ERROR :
> >+					BLK_MQ_RQ_QUEUE_BUSY;
> >  		spin_unlock_irq(&nvmeq->q_lock);
> >  		goto out;
> >  	}
> 
> I can't say I'm a fan of doing all this in queue_rq...
> 
> besides why is this state check under the cq_vector < 0 condition?
> This is really confusing...

Agreed, that's not very clear. We will potentially have an option to
disable interrupt vectors entirely in the future, so this won't be
appropriate criteria if/when that happens.

I've a follow up series that uses queue states instead of irq and
namespace states, but planned to target 4.6 unless an alternate proposal
surfaces.


From keith.busch at intel.com  Thu Feb 11 07:11:47 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 15:11:47 +0000
Subject: [PATCH for-4.5 10/13] NVMe: Move error handling to failed
 reset handler
In-Reply-To: <56BC83AE.8080702@dev.mellanox.co.il>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-11-git-send-email-keith.busch@intel.com>
 <56BC83AE.8080702@dev.mellanox.co.il>
Message-ID: <20160211151147.GC11908@localhost.localdomain>

On Thu, Feb 11, 2016 at 02:50:54PM +0200, Sagi Grimberg wrote:
> On 10/02/2016 20:17, Keith Busch wrote:
> >This moves the dead queue handling out of the namespace removal path
> >and into the reset failure path. It fixes a deadlock condition if the
> >controller fails or link down during del_gendisk.
> 
> How does it fix the deadlock?

Previously the queues were setup for failure prior to calling del_gendisk
only if the controller was broken. If the controller happened to be
optimal, this process would have been skipped. If the controller then
failed, the queues wouldn't be killed.
 
> >+	nvme_dev_disable(dev, false);
> >+
> >+	mutex_lock(&ctrl->namespaces_mutex);
> >+	list_for_each_entry(ns, &ctrl->namespaces, list) {
> >+		if (!kref_get_unless_zero(&ns->kref))
> >+			continue;
> >+
> >+		blk_set_queue_dying(ns->queue);
> >+		blk_mq_abort_requeue_list(ns->queue);
> >+		blk_mq_start_stopped_hw_queues(ns->queue, true);
> >+
> >+		nvme_put_ns(ns);
> >+	}
> >+	mutex_unlock(&ctrl->namespaces_mutex);
> >+}
> >+
> 
> Why on earth is this pci specific? This should be in the
> core. Aside from that, I'd really prefer if the core can handle this
> without having the pci (or other) triggering it explicitly, but if
> this must move out of the ns remove then we need documentation on
> what are the rules of when the driver needs to call it.

It's PCI specific only because of the potential need to disable the
controller first (nvme_dev_disable), which is currently PCI specific.


From keith.busch at intel.com  Thu Feb 11 07:12:45 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 15:12:45 +0000
Subject: [PATCH for-4.5 13/13] NVMe: Rate limit nvme IO warnings
In-Reply-To: <56BC7EC7.6020208@dev.mellanox.co.il>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-14-git-send-email-keith.busch@intel.com>
 <20160210185404.GB10459@infradead.org>
 <20160210211025.GC23402@localhost.localdomain>
 <56BC7EC7.6020208@dev.mellanox.co.il>
Message-ID: <20160211151245.GD11908@localhost.localdomain>

On Thu, Feb 11, 2016 at 02:29:59PM +0200, Sagi Grimberg wrote:
> 
> >>Do we need this message at all?
> >
> >I don't find it useful for anything other than confirming a test hit
> >that path. Delete it?
> 
> Perhaps we can have it dev_dbg.

Thanks, sounds like the right compromise to me.


From hch at infradead.org  Thu Feb 11 07:18:59 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 11 Feb 2016 07:18:59 -0800
Subject: [PATCH for-4.5 13/13] NVMe: Rate limit nvme IO warnings
In-Reply-To: <20160211151245.GD11908@localhost.localdomain>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-14-git-send-email-keith.busch@intel.com>
 <20160210185404.GB10459@infradead.org>
 <20160210211025.GC23402@localhost.localdomain>
 <56BC7EC7.6020208@dev.mellanox.co.il>
 <20160211151245.GD11908@localhost.localdomain>
Message-ID: <20160211151859.GA3072@infradead.org>

On Thu, Feb 11, 2016 at 03:12:45PM +0000, Keith Busch wrote:
> On Thu, Feb 11, 2016 at 02:29:59PM +0200, Sagi Grimberg wrote:
> > 
> > >>Do we need this message at all?
> > >
> > >I don't find it useful for anything other than confirming a test hit
> > >that path. Delete it?
> > 
> > Perhaps we can have it dev_dbg.
> 
> Thanks, sounds like the right compromise to me.

Fine with me.


From hch at infradead.org  Thu Feb 11 07:20:02 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 11 Feb 2016 07:20:02 -0800
Subject: [PATCH] block: Initialize max_dev_sectors to 0
In-Reply-To: <1455148367-25208-1-git-send-email-keith.busch@intel.com>
References: <1455148367-25208-1-git-send-email-keith.busch@intel.com>
Message-ID: <20160211152002.GB3072@infradead.org>

On Wed, Feb 10, 2016 at 04:52:47PM -0700, Keith Busch wrote:
> The new queue limit is not used by the majority of block drivers, and
> should be initialized to 0 for the driver's requested settings to be used.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> Cc: Christoph Hellwig <hch at infradead.org>
> Acked-by: Martin K. Petersen <martin.petersen at oracle.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From axboe at fb.com  Thu Feb 11 07:21:24 2016
From: axboe at fb.com (Jens Axboe)
Date: Thu, 11 Feb 2016 08:21:24 -0700
Subject: [PATCH] block: Initialize max_dev_sectors to 0
In-Reply-To: <1455148367-25208-1-git-send-email-keith.busch@intel.com>
References: <1455148367-25208-1-git-send-email-keith.busch@intel.com>
Message-ID: <56BCA6F4.6030807@fb.com>

On 02/10/2016 04:52 PM, Keith Busch wrote:
> The new queue limit is not used by the majority of block drivers, and
> should be initialized to 0 for the driver's requested settings to be used.

That's much better, thanks.


-- 
Jens Axboe



From snitzer at redhat.com  Thu Feb 11 07:34:53 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Thu, 11 Feb 2016 10:34:53 -0500
Subject: RCU-ified dm-mpath for testing/review
In-Reply-To: <20160211015030.GA4481@redhat.com>
References: <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com>
 <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com> <56B99A49.5050400@suse.de>
 <20160209145547.GA21623@redhat.com> <56BA0689.9030007@suse.de>
 <20160210004518.GA23646@redhat.com>
 <20160211015030.GA4481@redhat.com>
Message-ID: <20160211153452.GA7827@redhat.com>

On Wed, Feb 10 2016 at  8:50pm -0500,
Mike Snitzer <snitzer at redhat.com> wrote:

> On Tue, Feb 09 2016 at  7:45pm -0500,
> Mike Snitzer <snitzer at redhat.com> wrote:
> 
> > 
> > OK, I took a crack at embracing RCU.  Only slightly better performance
> > on my single NUMA node testbed.  (But I'll have to track down a system
> > with multiple NUMA nodes to do any justice to the next wave of this
> > optimization effort)
> > 
> > This RCU work is very heavy-handed and way too fiddley (there could
> > easily be bugs).  Anyway, please see:
> > http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel2&id=d80a7e4f8b5be9c81e4d452137623b003fa64745
> > 
> > But this might give you something to build on to arrive at something
> > more scalable?
> 
> I've a bit more polished version of this work (broken up into multiple
> commits, with some fixes, etc) here:
> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
> 
> Hannes and/or Sagi, if you get a chance to try this on your NUMA system
> please let me know how it goes.

Initial review has uncovered some locking problems with the current code
(nothing that caused crashes or hangs in my testing but...) so please
hold off on testing until you hear from me (hopefully tomorrow).

Thanks,
Mike


From wenbo.wang at memblaze.com  Thu Feb 11 08:38:45 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Thu, 11 Feb 2016 16:38:45 +0000
Subject: [PATCH for-4.5 05/13] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455128250-5984-6-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-6-git-send-email-keith.busch@intel.com>
Message-ID: <BJXPR01MB1994CF464CC02616EAC036BE0A80@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

Not sure if async namespace removal has any potential risk.

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Thursday, February 11, 2016 2:17 AM
To: linux-nvme at lists.infradead.org; Jens Axboe
Cc: Christoph Hellwig; Sagi Grimberg; Wenbo Wang; Keith Busch
Subject: [PATCH for-4.5 05/13] NVMe: Fix namespace removal deadlock

Namespace removal holds the namespace list mutex. If the controller fails during namespace removal, the reset work will deadlock trying to acquire the mutex, and no progress will be made.

This patch completes the potentially blocking parts of namespace removal in a controller work queue so that reset work can cleanup a failed controller.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 81 ++++++++++++++++++++++++++++++------------------
 drivers/nvme/host/nvme.h |  6 ++++
 2 files changed, 57 insertions(+), 30 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 852b789..be27f9f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -48,6 +48,10 @@ static void nvme_free_ns(struct kref *kref)  {
 	struct nvme_ns *ns = container_of(kref, struct nvme_ns, kref);
 
+	mutex_lock(&ns->ctrl->namespaces_mutex);
+	list_del_init(&ns->list);
+	mutex_unlock(&ns->ctrl->namespaces_mutex);
+
 	if (ns->type == NVME_NS_LIGHTNVM)
 		nvme_nvm_unregister(ns->queue, ns->disk->disk_name);
 
@@ -1106,6 +1110,37 @@ static struct nvme_ns *nvme_find_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	return NULL;
 }
 
+static void nvme_ns_remove_work(struct work_struct *work) {
+	struct nvme_ns *ns = container_of(work, struct nvme_ns, remove_work);
+	bool kill = nvme_io_incapable(ns->ctrl) &&
+			!blk_queue_dying(ns->queue);
+
+	if (kill) {
+		blk_set_queue_dying(ns->queue);
+
+		/*
+		 * The controller was shutdown first if we got here through
+		 * device removal. The shutdown may requeue outstanding
+		 * requests. These need to be aborted immediately so
+		 * del_gendisk doesn't block indefinitely for their completion.
+		 */
+		blk_mq_abort_requeue_list(ns->queue);
+	}
+	if (ns->disk->flags & GENHD_FL_UP) {
+		if (blk_get_integrity(ns->disk))
+			blk_integrity_unregister(ns->disk);
+		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
+					&nvme_ns_attr_group);
+		del_gendisk(ns->disk);
+	}
+	if (kill || !blk_queue_dying(ns->queue)) {
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_cleanup_queue(ns->queue);
+	}
+	nvme_put_ns(ns);
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)  {
 	struct nvme_ns *ns;
@@ -1161,6 +1196,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 	list_add_tail(&ns->list, &ctrl->namespaces);
 	kref_get(&ctrl->kref);
+	INIT_WORK(&ns->remove_work, nvme_ns_remove_work);
 	if (ns->type == NVME_NS_LIGHTNVM)
 		return;
 
@@ -1180,35 +1216,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)  {
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-
-	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
-
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
-	if (ns->disk->flags & GENHD_FL_UP) {
-		if (blk_get_integrity(ns->disk))
-			blk_integrity_unregister(ns->disk);
-		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
-					&nvme_ns_attr_group);
-		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
-		blk_mq_abort_requeue_list(ns->queue);
-		blk_cleanup_queue(ns->queue);
-	}
-	list_del_init(&ns->list);
-	nvme_put_ns(ns);
+	if (!test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
+		queue_work(ns->ctrl->ns_workq, &ns->remove_work);
 }
 
 static void nvme_validate_ns(struct nvme_ctrl *ctrl, unsigned nsid) @@ -1305,6 +1314,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 	list_for_each_entry_safe(ns, next, &ctrl->namespaces, list)
 		nvme_ns_remove(ns);
 	mutex_unlock(&ctrl->namespaces_mutex);
+
+	flush_workqueue(ctrl->ns_workq);
 }
 
 static DEFINE_IDA(nvme_instance_ida);
@@ -1337,12 +1348,14 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)  }
 
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
- {
+{
 	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 
 	spin_lock(&dev_list_lock);
 	list_del(&ctrl->node);
 	spin_unlock(&dev_list_lock);
+
+	destroy_workqueue(ctrl->ns_workq);
 }
 
 static void nvme_free_ctrl(struct kref *kref) @@ -1392,11 +1405,19 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	get_device(ctrl->device);
 	dev_set_drvdata(ctrl->device, ctrl);
 
+	ctrl->ns_workq = alloc_workqueue("nvme-ns", WQ_UNBOUND, 0);
+	if (!ctrl->ns_workq) {
+		ret = -ENOMEM;
+		goto out_destroy_device;
+	}
+
 	spin_lock(&dev_list_lock);
 	list_add_tail(&ctrl->node, &nvme_ctrl_list);
 	spin_unlock(&dev_list_lock);
 
 	return 0;
+out_destroy_device:
+	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 out_release_instance:
 	nvme_release_instance(ctrl);
 out:
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 9664d07..d330512 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -72,6 +72,7 @@ struct nvme_ctrl {
 	struct mutex namespaces_mutex;
 	struct device *device;	/* char device */
 	struct list_head node;
+	struct workqueue_struct *ns_workq;
 
 	char name[12];
 	char serial[20];
@@ -112,6 +113,11 @@ struct nvme_ns {
 	bool ext;
 	u8 pi_type;
 	int type;
+	struct work_struct remove_work;
+	unsigned long flags;
+
+#define NVME_NS_REMOVING 0
+
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
 };
--
2.6.2.307.g37023ba


From wenbo.wang at memblaze.com  Thu Feb 11 08:47:31 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Thu, 11 Feb 2016 16:47:31 +0000
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues
In-Reply-To: <1455128250-5984-8-git-send-email-keith.busch@intel.com>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-8-git-send-email-keith.busch@intel.com>
Message-ID: <BJXPR01MB19918E58636267AE304ACC1E0A80@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

If adding synchronize_rcu and blk_sync_queue, I guess this is not needed.

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Thursday, February 11, 2016 2:17 AM
To: linux-nvme at lists.infradead.org; Jens Axboe
Cc: Christoph Hellwig; Sagi Grimberg; Wenbo Wang; Keith Busch
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues

It's possible a request may get to the driver after the nvme queue was disabled. This has the request requeue if that happens.

Note the request is still "started" by the driver, but requeuing will clear the start state for timeout handling.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 5bea054..f8bb781 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -678,6 +678,11 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 	blk_mq_start_request(req);
 
 	spin_lock_irq(&nvmeq->q_lock);
+	if (unlikely(nvmeq->cq_vector < 0)) {
+		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		spin_unlock_irq(&nvmeq->q_lock);
+		goto out;
+	}
 	__nvme_submit_cmd(nvmeq, &cmnd);
 	nvme_process_cq(nvmeq);
 	spin_unlock_irq(&nvmeq->q_lock);
--
2.6.2.307.g37023ba


From keith.busch at intel.com  Thu Feb 11 09:00:12 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 17:00:12 +0000
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues
In-Reply-To: <BJXPR01MB19918E58636267AE304ACC1E0A80@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-8-git-send-email-keith.busch@intel.com>
 <BJXPR01MB19918E58636267AE304ACC1E0A80@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160211170012.GE11908@localhost.localdomain>

On Thu, Feb 11, 2016 at 04:47:31PM +0000, Wenbo Wang wrote:
> If adding synchronize_rcu and blk_sync_queue, I guess this is not needed.

That was to sync IO tasks to make sure they saw the h/w cxt STOPPED
flag. We still need to restart those queues at some point to flush queued
requests to completion, and this patch is just prepping for that.

I used "blk_queue_dying" in a previous patch series. I went going back
and forth on using that criteria. SCSI has device states, so trying to
apply those ideas to namespaces.


From wenbo.wang at memblaze.com  Thu Feb 11 09:21:45 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Thu, 11 Feb 2016 17:21:45 +0000
Subject: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues
In-Reply-To: <20160211170012.GE11908@localhost.localdomain>
References: <1455128250-5984-1-git-send-email-keith.busch@intel.com>
 <1455128250-5984-8-git-send-email-keith.busch@intel.com>
 <BJXPR01MB19918E58636267AE304ACC1E0A80@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160211170012.GE11908@localhost.localdomain>
Message-ID: <BJXPR01MB199232746608AC5BE884F52E0A80@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>

If device is removed, I think nvme_clear_queue cleans all those pending requests.
If devices is restarted, should not cq_vector be setup before enabling the h/w ctx?

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Friday, February 12, 2016 1:00 AM
To: Wenbo Wang
Cc: linux-nvme at lists.infradead.org; Jens Axboe; Christoph Hellwig; Sagi Grimberg
Subject: Re: [PATCH for-4.5 07/13] NVMe: Requeue requests on suspended queues

On Thu, Feb 11, 2016 at 04:47:31PM +0000, Wenbo Wang wrote:
> If adding synchronize_rcu and blk_sync_queue, I guess this is not needed.

That was to sync IO tasks to make sure they saw the h/w cxt STOPPED flag. We still need to restart those queues at some point to flush queued requests to completion, and this patch is just prepping for that.

I used "blk_queue_dying" in a previous patch series. I went going back and forth on using that criteria. SCSI has device states, so trying to apply those ideas to namespaces.

From keith.busch at intel.com  Thu Feb 11 12:05:37 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:37 -0700
Subject: [PATCHv2-4.5 00/10] NVMe fixes
Message-ID: <1455221147-24228-1-git-send-email-keith.busch@intel.com>

Thank you everyone for the earlier review comments.

The entire series is here, though most of the code is not changing from
the previous.

I'll gladly handle merging this with the 4.6 branch if/when this goes
through.

v1 -> v2:

  Added all recieved "Reviewed-by".

  Dropped patch for NVMe's max_dev_sectors. This is fixed in the block
  layer with a different and generic patch external to this series.

  Dropped patch removing nvme_workq's WQ_MEM_RECLAIM. The flush sync
  warning that patch was trying to address is fixed by moving the
  namespace cleanup from the nvme_workq to an existing nvme callback
  scheduled on the system workqueue.

  Implemented Sagi's suggestion to move the "nvme_kill_ns_queues" to from
  pci to core. There was no real dependency on PCI after a very minor
  modification. Also added code comments to explain the function usage.

  Combined the "NS_DEAD" state patch with the namespace queue cleanup
  handling in the reset_work. The first really counted on the second to
  actually be useful, so making it a single commit.

  Fixed an error code to be negative, pointed out in review.

  Commented the requirement for nvme_remove()'s necessity to handle a
  partially initialized.

  Made nvme's "Cancelling I/O ..." message debug level, in addition to
  rate limted.

Keith Busch (10):
  blk-mq: End unstarted requests on dying queue
  NVMe: Fix io incapable return values
  NVMe: Allow request merges
  NVMe: Fix namespace removal deadlock
  NVMe: Requeue requests on suspended queues
  NVMe: Poll device while still active during remove
  NVMe: Simplify device reset failure
  NVMe: Move error handling to failed reset handler
  NVMe: Mark queues as dead on degraded controller
  NVMe: Rate limit nvme IO warnings

 block/blk-mq.c           |   6 ++-
 drivers/nvme/host/core.c | 105 ++++++++++++++++++++++++++++++++---------------
 drivers/nvme/host/nvme.h |  13 +++++-
 drivers/nvme/host/pci.c  |  66 ++++++++++++++---------------
 4 files changed, 121 insertions(+), 69 deletions(-)

-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:40 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:40 -0700
Subject: [PATCHv2-4.5 03/10] NVMe: Allow request merges
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-4-git-send-email-keith.busch@intel.com>

It is generally more efficient to submit larger IO.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
---
 drivers/nvme/host/core.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c5bf001..3cd921e 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1121,7 +1121,6 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	ns->queue = blk_mq_init_queue(ctrl->tagset);
 	if (IS_ERR(ns->queue))
 		goto out_free_ns;
-	queue_flag_set_unlocked(QUEUE_FLAG_NOMERGES, ns->queue);
 	queue_flag_set_unlocked(QUEUE_FLAG_NONROT, ns->queue);
 	ns->queue->queuedata = ns;
 	ns->ctrl = ctrl;
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:38 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:38 -0700
Subject: [PATCHv2-4.5 01/10] blk-mq: End unstarted requests on dying queue
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-2-git-send-email-keith.busch@intel.com>

Go directly to ending a request if it wasn't started. Previously, completing a
request may invoke a driver callback for a request it didn't initialize.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Acked-by: Christoph Hellwig <hch at lst.de>
---
 block/blk-mq.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 4c0622f..56c0a72 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -599,8 +599,10 @@ static void blk_mq_check_expired(struct blk_mq_hw_ctx *hctx,
 		 * If a request wasn't started before the queue was
 		 * marked dying, kill it here or it'll go unnoticed.
 		 */
-		if (unlikely(blk_queue_dying(rq->q)))
-			blk_mq_complete_request(rq, -EIO);
+		if (unlikely(blk_queue_dying(rq->q))) {
+			rq->errors = -EIO;
+			blk_mq_end_request(rq, rq->errors);
+		}
 		return;
 	}
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:44 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:44 -0700
Subject: [PATCHv2-4.5 07/10] NVMe: Simplify device reset failure
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-8-git-send-email-keith.busch@intel.com>

A reset failure schedules the device to unbind from the driver through
the pci driver's 'remove'. This already cleans up all intialization
from a partial or completed state, so there is no need to duplicate the
cleanup that could potentially race with a pci hot removal.

To help understand why a reset failed, the status is logged with the
existing warning message.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
 drivers/nvme/host/pci.c | 48 +++++++++++++++++++++---------------------------
 1 file changed, 21 insertions(+), 27 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 09cc4da..9e1be57 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -86,7 +86,6 @@ struct nvme_queue;
 
 static int nvme_reset(struct nvme_dev *dev);
 static void nvme_process_cq(struct nvme_queue *nvmeq);
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev);
 static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown);
 
 /*
@@ -1899,10 +1898,19 @@ static void nvme_pci_free_ctrl(struct nvme_ctrl *ctrl)
 	kfree(dev);
 }
 
+static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
+{
+	dev_warn(dev->dev, "Removing after probe failure status: %d\n", status);
+
+	kref_get(&dev->ctrl.kref);
+	if (!schedule_work(&dev->remove_work))
+		nvme_put_ctrl(&dev->ctrl);
+}
+
 static void nvme_reset_work(struct work_struct *work)
 {
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, reset_work);
-	int result;
+	int result = -ENODEV;
 
 	if (WARN_ON(test_bit(NVME_CTRL_RESETTING, &dev->flags)))
 		goto out;
@@ -1922,26 +1930,26 @@ static void nvme_reset_work(struct work_struct *work)
 
 	result = nvme_configure_admin_queue(dev);
 	if (result)
-		goto unmap;
+		goto out;
 
 	nvme_init_queue(dev->queues[0], 0);
 	result = nvme_alloc_admin_tags(dev);
 	if (result)
-		goto disable;
+		goto out;
 
 	result = nvme_init_identify(&dev->ctrl);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	result = nvme_setup_io_queues(dev);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
 
 	result = nvme_dev_list_add(dev);
 	if (result)
-		goto remove;
+		goto out;
 
 	/*
 	 * Keep the controller around but remove all namespaces if we don't have
@@ -1958,19 +1966,8 @@ static void nvme_reset_work(struct work_struct *work)
 	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
 	return;
 
- remove:
-	nvme_dev_list_remove(dev);
- free_tags:
-	nvme_dev_remove_admin(dev);
-	blk_put_queue(dev->ctrl.admin_q);
-	dev->ctrl.admin_q = NULL;
-	dev->queues[0]->tags = NULL;
- disable:
-	nvme_disable_admin_queue(dev, false);
- unmap:
-	nvme_dev_unmap(dev);
  out:
-	nvme_remove_dead_ctrl(dev);
+	nvme_remove_dead_ctrl(dev, result);
 }
 
 static void nvme_remove_dead_ctrl_work(struct work_struct *work)
@@ -1983,14 +1980,6 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	nvme_put_ctrl(&dev->ctrl);
 }
 
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev)
-{
-	dev_warn(dev->dev, "Removing after probe failure\n");
-	kref_get(&dev->ctrl.kref);
-	if (!schedule_work(&dev->remove_work))
-		nvme_put_ctrl(&dev->ctrl);
-}
-
 static int nvme_reset(struct nvme_dev *dev)
 {
 	if (!dev->ctrl.admin_q || blk_queue_dying(dev->ctrl.admin_q))
@@ -2112,6 +2101,11 @@ static void nvme_shutdown(struct pci_dev *pdev)
 	nvme_dev_disable(dev, true);
 }
 
+/*
+ * The driver's remove may be called on a device in a partially initialized
+ * state. This function must not have any dependencies on the init state in
+ * order to proceed, and may even execute while initiazation is in progress.
+ */
 static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:47 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:47 -0700
Subject: [PATCHv2-4.5 10/10] NVMe: Rate limit nvme IO warnings
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-11-git-send-email-keith.busch@intel.com>

We don't need to spam the kernel logs with thousands of IO cancelling
messages. We can infer all IO's are being cancelled with fewer, or
even none at all. This patch rate limits the message and uses the debug
log level as it is mainly used for testing purposes.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 4de17ae..1b0e447 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1005,7 +1005,7 @@ static void nvme_cancel_queue_ios(struct request *req, void *data, bool reserved
 	if (!blk_mq_request_started(req))
 		return;
 
-	dev_warn(nvmeq->q_dmadev,
+	dev_dbg_ratelimited(nvmeq->q_dmadev,
 		 "Cancelling I/O %d QID %d\n", req->tag, nvmeq->qid);
 
 	status = NVME_SC_ABORT_REQ;
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:41 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:41 -0700
Subject: [PATCHv2-4.5 04/10] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-5-git-send-email-keith.busch@intel.com>

Namespace removal holds the namespace list mutex. If the controller
fails during namespace removal, the reset work would deadlock trying to
acquire the mutex.

This patch completes the potentially blocking parts of namespace
removal in a controller work queue so that reset work can cleanup a
failed controller.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
 drivers/nvme/host/core.c | 81 ++++++++++++++++++++++++++++++------------------
 drivers/nvme/host/nvme.h |  6 ++++
 2 files changed, 57 insertions(+), 30 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 3cd921e..ceea7f0 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -48,6 +48,10 @@ static void nvme_free_ns(struct kref *kref)
 {
 	struct nvme_ns *ns = container_of(kref, struct nvme_ns, kref);
 
+	mutex_lock(&ns->ctrl->namespaces_mutex);
+	list_del_init(&ns->list);
+	mutex_unlock(&ns->ctrl->namespaces_mutex);
+
 	if (ns->type == NVME_NS_LIGHTNVM)
 		nvme_nvm_unregister(ns->queue, ns->disk->disk_name);
 
@@ -1106,6 +1110,37 @@ static struct nvme_ns *nvme_find_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	return NULL;
 }
 
+static void nvme_ns_remove_work(struct work_struct *work)
+{
+	struct nvme_ns *ns = container_of(work, struct nvme_ns, remove_work);
+	bool kill = nvme_io_incapable(ns->ctrl) &&
+			!blk_queue_dying(ns->queue);
+
+	if (kill) {
+		blk_set_queue_dying(ns->queue);
+
+		/*
+		 * The controller was shutdown first if we got here through
+		 * device removal. The shutdown may requeue outstanding
+		 * requests. These need to be aborted immediately so
+		 * del_gendisk doesn't block indefinitely for their completion.
+		 */
+		blk_mq_abort_requeue_list(ns->queue);
+	}
+	if (ns->disk->flags & GENHD_FL_UP) {
+		if (blk_get_integrity(ns->disk))
+			blk_integrity_unregister(ns->disk);
+		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
+					&nvme_ns_attr_group);
+		del_gendisk(ns->disk);
+	}
+	if (kill || !blk_queue_dying(ns->queue)) {
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_cleanup_queue(ns->queue);
+	}
+	nvme_put_ns(ns);
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 {
 	struct nvme_ns *ns;
@@ -1160,6 +1195,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 	list_add_tail(&ns->list, &ctrl->namespaces);
 	kref_get(&ctrl->kref);
+	INIT_WORK(&ns->remove_work, nvme_ns_remove_work);
 	if (ns->type == NVME_NS_LIGHTNVM)
 		return;
 
@@ -1179,35 +1215,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)
 {
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-
-	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
-
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
-	if (ns->disk->flags & GENHD_FL_UP) {
-		if (blk_get_integrity(ns->disk))
-			blk_integrity_unregister(ns->disk);
-		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
-					&nvme_ns_attr_group);
-		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
-		blk_mq_abort_requeue_list(ns->queue);
-		blk_cleanup_queue(ns->queue);
-	}
-	list_del_init(&ns->list);
-	nvme_put_ns(ns);
+	if (!test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
+		queue_work(ns->ctrl->ns_workq, &ns->remove_work);
 }
 
 static void nvme_validate_ns(struct nvme_ctrl *ctrl, unsigned nsid)
@@ -1304,6 +1313,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 	list_for_each_entry_safe(ns, next, &ctrl->namespaces, list)
 		nvme_ns_remove(ns);
 	mutex_unlock(&ctrl->namespaces_mutex);
+
+	flush_workqueue(ctrl->ns_workq);
 }
 
 static DEFINE_IDA(nvme_instance_ida);
@@ -1336,12 +1347,14 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
 }
 
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
- {
+{
 	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 
 	spin_lock(&dev_list_lock);
 	list_del(&ctrl->node);
 	spin_unlock(&dev_list_lock);
+
+	destroy_workqueue(ctrl->ns_workq);
 }
 
 static void nvme_free_ctrl(struct kref *kref)
@@ -1391,11 +1404,19 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	get_device(ctrl->device);
 	dev_set_drvdata(ctrl->device, ctrl);
 
+	ctrl->ns_workq = alloc_workqueue("nvme-ns", WQ_UNBOUND, 0);
+	if (!ctrl->ns_workq) {
+		ret = -ENOMEM;
+		goto out_destroy_device;
+	}
+
 	spin_lock(&dev_list_lock);
 	list_add_tail(&ctrl->node, &nvme_ctrl_list);
 	spin_unlock(&dev_list_lock);
 
 	return 0;
+out_destroy_device:
+	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 out_release_instance:
 	nvme_release_instance(ctrl);
 out:
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9664d07..d330512 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -72,6 +72,7 @@ struct nvme_ctrl {
 	struct mutex namespaces_mutex;
 	struct device *device;	/* char device */
 	struct list_head node;
+	struct workqueue_struct *ns_workq;
 
 	char name[12];
 	char serial[20];
@@ -112,6 +113,11 @@ struct nvme_ns {
 	bool ext;
 	u8 pi_type;
 	int type;
+	struct work_struct remove_work;
+	unsigned long flags;
+
+#define NVME_NS_REMOVING 0
+
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
 };
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:39 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:39 -0700
Subject: [PATCHv2-4.5 02/10] NVMe: Fix io incapable return values
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-3-git-send-email-keith.busch@intel.com>

The function returns true when the controller can't handle IO.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Acked-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/nvme.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4fb5bb7..9664d07 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -139,9 +139,9 @@ static inline bool nvme_io_incapable(struct nvme_ctrl *ctrl)
 	u32 val = 0;
 
 	if (ctrl->ops->io_incapable(ctrl))
-		return false;
+		return true;
 	if (ctrl->ops->reg_read32(ctrl, NVME_REG_CSTS, &val))
-		return false;
+		return true;
 	return val & NVME_CSTS_CFS;
 }
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:46 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:46 -0700
Subject: [PATCHv2-4.5 09/10] NVMe: Mark queues as dead on degraded controller
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-10-git-send-email-keith.busch@intel.com>

If a previously optimal controller goes degraded, the IO queues will
stop working, but the controller is still managable. Prior to removing
namespace, we need to fail the queues so queued and new IO will fail
and allow namespace removal to progress.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
 drivers/nvme/host/pci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 32286e9..4de17ae 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1960,6 +1960,7 @@ static void nvme_reset_work(struct work_struct *work)
 	 */
 	if (dev->online_queues < 2) {
 		dev_warn(dev->dev, "IO queues not created\n");
+		nvme_kill_ns_queues(&dev->ctrl);
 		nvme_remove_namespaces(&dev->ctrl);
 	} else {
 		nvme_start_queues(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:45 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:45 -0700
Subject: [PATCHv2-4.5 08/10] NVMe: Move error handling to failed reset handler
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-9-git-send-email-keith.busch@intel.com>

This moves failed queue handling out of the namespace removal path and into
the reset failure path, fixing a deadlock condition if the controller
fails or link down during del_gendisk. Previously the driver had to see
the controller as degraded prior to calling del_gendisk to setup the
queues to fail. If the controller happened to fail after this though,
there was no task to end the request_queue.

On failure, all namespace states are set to 'dead'. This has capacity
revalidate to 0, and ends all new requests with error status.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
 drivers/nvme/host/core.c | 51 ++++++++++++++++++++++++++++++++++--------------
 drivers/nvme/host/nvme.h |  2 ++
 drivers/nvme/host/pci.c  |  6 +++++-
 3 files changed, 43 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ceea7f0..5ff1e77 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -560,6 +560,10 @@ static int nvme_revalidate_disk(struct gendisk *disk)
 	u16 old_ms;
 	unsigned short bs;
 
+	if (test_bit(NVME_NS_DEAD, &ns->flags)) {
+		set_capacity(disk, 0);
+		return -ENODEV;
+	}
 	if (nvme_identify_ns(ns->ctrl, ns->ns_id, &id)) {
 		dev_warn(ns->ctrl->dev, "%s: Identify failure nvme%dn%d\n",
 				__func__, ns->ctrl->instance, ns->ns_id);
@@ -1113,34 +1117,51 @@ static struct nvme_ns *nvme_find_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 static void nvme_ns_remove_work(struct work_struct *work)
 {
 	struct nvme_ns *ns = container_of(work, struct nvme_ns, remove_work);
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
 
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
 	if (ns->disk->flags & GENHD_FL_UP) {
 		if (blk_get_integrity(ns->disk))
 			blk_integrity_unregister(ns->disk);
 		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
 					&nvme_ns_attr_group);
 		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
 	nvme_put_ns(ns);
 }
 
+/**
+ * nvme_kill_ns_queues(): Ends all namespace queues
+ * @ctrl: the dead controller that needs to end
+ *
+ * Call this function when the driver determines it is unable to get the
+ * controller in a state capable of servicing IO.
+ */
+void nvme_kill_ns_queues(struct nvme_ctrl *ctrl)
+{
+	struct nvme_ns *ns;
+
+	mutex_lock(&ctrl->namespaces_mutex);
+	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (!kref_get_unless_zero(&ns->kref))
+			continue;
+
+		/*
+		 * Revalidating a dead namespace sets capacity to 0. This will
+		 * end buffered writers dirtying pages that can't be synced.
+		 */
+		if (!test_and_set_bit(NVME_NS_DEAD, &ns->flags))
+			revalidate_disk(ns->disk);
+
+		blk_set_queue_dying(ns->queue);
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_mq_start_stopped_hw_queues(ns->queue, true);
+
+		nvme_put_ns(ns);
+	}
+	mutex_unlock(&ctrl->namespaces_mutex);
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 {
 	struct nvme_ns *ns;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index d330512..8fa5ceb 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -117,6 +117,7 @@ struct nvme_ns {
 	unsigned long flags;
 
 #define NVME_NS_REMOVING 0
+#define NVME_NS_DEAD     1
 
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
@@ -270,6 +271,7 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
 int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 			dma_addr_t dma_addr, u32 *result);
 int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count);
+void nvme_kill_ns_queues(struct nvme_ctrl *ctrl);
 
 extern spinlock_t dev_list_lock;
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 9e1be57..32286e9 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -678,7 +678,9 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 
 	spin_lock_irq(&nvmeq->q_lock);
 	if (unlikely(nvmeq->cq_vector < 0)) {
-		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		ret = test_bit(NVME_NS_DEAD, &ns->flags) ?
+					BLK_MQ_RQ_QUEUE_ERROR :
+					BLK_MQ_RQ_QUEUE_BUSY;
 		spin_unlock_irq(&nvmeq->q_lock);
 		goto out;
 	}
@@ -1903,6 +1905,7 @@ static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
 	dev_warn(dev->dev, "Removing after probe failure status: %d\n", status);
 
 	kref_get(&dev->ctrl.kref);
+	nvme_dev_disable(dev, false);
 	if (!schedule_work(&dev->remove_work))
 		nvme_put_ctrl(&dev->ctrl);
 }
@@ -1975,6 +1978,7 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, remove_work);
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
+	nvme_kill_ns_queues(&dev->ctrl);
 	if (pci_get_drvdata(pdev))
 		pci_stop_and_remove_bus_device_locked(pdev);
 	nvme_put_ctrl(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:43 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:43 -0700
Subject: [PATCHv2-4.5 06/10] NVMe: Poll device while still active during remove
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-7-git-send-email-keith.busch@intel.com>

A device failure or link down wouldn't have been detected during namespace
removal. This patch keeps the device in the list for polling so that the
thread may see such failure and initiate a reset. The device is removed
from the list after disable, so we can safely flush the reset work as
it can't be requeued when disable completes.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
---
 drivers/nvme/host/pci.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e5c2bea..09cc4da 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2116,16 +2116,12 @@ static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
 
-	spin_lock(&dev_list_lock);
-	list_del_init(&dev->node);
-	spin_unlock(&dev_list_lock);
-
 	pci_set_drvdata(pdev, NULL);
-	flush_work(&dev->reset_work);
 	flush_work(&dev->scan_work);
 	nvme_remove_namespaces(&dev->ctrl);
 	nvme_uninit_ctrl(&dev->ctrl);
 	nvme_dev_disable(dev, true);
+	flush_work(&dev->reset_work);
 	nvme_dev_remove_admin(dev);
 	nvme_free_queues(dev, 0);
 	nvme_release_cmb(dev);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 12:05:42 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 13:05:42 -0700
Subject: [PATCHv2-4.5 05/10] NVMe: Requeue requests on suspended queues
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455221147-24228-6-git-send-email-keith.busch@intel.com>

It's possible a request may get to the driver after the nvme queue was
disabled. This has the request requeue if that happens.

Note the request is still "started" by the driver, but requeuing will
clear the start state for timeout handling.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
---
 drivers/nvme/host/pci.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 72ef832..e5c2bea 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -678,6 +678,11 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 	blk_mq_start_request(req);
 
 	spin_lock_irq(&nvmeq->q_lock);
+	if (unlikely(nvmeq->cq_vector < 0)) {
+		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		spin_unlock_irq(&nvmeq->q_lock);
+		goto out;
+	}
 	__nvme_submit_cmd(nvmeq, &cmnd);
 	nvme_process_cq(nvmeq);
 	spin_unlock_irq(&nvmeq->q_lock);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 14:28:49 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 22:28:49 +0000
Subject: [PATCHv2-4.5 00/10] NVMe fixes
In-Reply-To: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
Message-ID: <20160211222848.GA19595@localhost.localdomain>

Just want to point out a couple issues. Both are easy fixes and will
send in a few minutes.

First, admin queues don't have namespaces, so the new namespace state
check isn't always correct. I haven't been able to synthesize a test that
gets an admin command down this path in the disabled queue condition,
but it's clearly not correct.

The second issue potentially existed all along, but it's a littlie easier
to hit now: we need to lock out shutdown from manipulating registers
while they're being setup. This was hit by forcing controller reset from
the shelll in a loop, then "rmmod nvme" at the same time.


From axboe at fb.com  Thu Feb 11 14:38:02 2016
From: axboe at fb.com (Jens Axboe)
Date: Thu, 11 Feb 2016 15:38:02 -0700
Subject: [PATCHv2-4.5 00/10] NVMe fixes
In-Reply-To: <20160211222848.GA19595@localhost.localdomain>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <20160211222848.GA19595@localhost.localdomain>
Message-ID: <56BD0D4A.60003@fb.com>

On 02/11/2016 03:28 PM, Keith Busch wrote:
> Just want to point out a couple issues. Both are easy fixes and will
> send in a few minutes.
>
> First, admin queues don't have namespaces, so the new namespace state
> check isn't always correct. I haven't been able to synthesize a test that
> gets an admin command down this path in the disabled queue condition,
> but it's clearly not correct.
>
> The second issue potentially existed all along, but it's a littlie easier
> to hit now: we need to lock out shutdown from manipulating registers
> while they're being setup. This was hit by forcing controller reset from
> the shelll in a loop, then "rmmod nvme" at the same time.

I already applied these for 4.5, just didn't send the ack before I had 
done some testing. JFYI.

-- 
Jens Axboe



From keith.busch at intel.com  Thu Feb 11 15:00:26 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 16:00:26 -0700
Subject: [PATCH 2/2] NVMe: NULL pointer check for admin queues
In-Reply-To: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
References: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455231626-7166-2-git-send-email-keith.busch@intel.com>

Admin commands don't have a namespace, so don't check the state when
there isn't one.

An admin command that does happen to make it to a disabled queue can
failed immediatly: Initialization commands should not be requeued
across resets as they would mess up the controller if/when it is
reinitialized. If a user command made hits this condition, it should
be able to handle this error as that's the existing result for commands
when the controller is reset with outstanding admin commands.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 24a7972..44ccd23 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -678,7 +678,12 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 
 	spin_lock_irq(&nvmeq->q_lock);
 	if (unlikely(nvmeq->cq_vector < 0)) {
-		ret = test_bit(NVME_NS_DEAD, &ns->flags) ?
+		/*
+		 * Admin commands need to fail immediatly when device is
+		 * disabled: initialization commands should not requeue
+		 * across resets.
+		 */
+		ret = !ns || test_bit(NVME_NS_DEAD, &ns->flags) ?
 					BLK_MQ_RQ_QUEUE_ERROR :
 					BLK_MQ_RQ_QUEUE_BUSY;
 		spin_unlock_irq(&nvmeq->q_lock);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 11 15:00:25 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 11 Feb 2016 16:00:25 -0700
Subject: [PATCH 1/2] NVMe: Lock out shutdown during pci init
Message-ID: <1455231626-7166-1-git-send-email-keith.busch@intel.com>

An unexpected driver unbind needs to be prevented from unmapping the
pci nvme registers while they're being initialized.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 1b0e447..24a7972 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1927,13 +1927,19 @@ static void nvme_reset_work(struct work_struct *work)
 
 	set_bit(NVME_CTRL_RESETTING, &dev->flags);
 
+	/*
+	 * Lock out shutdown so an unexpected driver removal won't unmap
+	 * registers while reset work is setting them up.
+	 */
+	mutex_lock(&dev->shutdown_lock);
 	result = nvme_dev_map(dev);
 	if (result)
-		goto out;
+		goto out_unlock;
 
 	result = nvme_configure_admin_queue(dev);
 	if (result)
-		goto out;
+		goto out_unlock;
+	mutex_unlock(&dev->shutdown_lock);
 
 	nvme_init_queue(dev->queues[0], 0);
 	result = nvme_alloc_admin_tags(dev);
@@ -1970,6 +1976,8 @@ static void nvme_reset_work(struct work_struct *work)
 	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
 	return;
 
+ out_unlock:
+	mutex_unlock(&dev->shutdown_lock);
  out:
 	nvme_remove_dead_ctrl(dev, result);
 }
-- 
2.6.2.307.g37023ba



From jthumshirn at suse.de  Fri Feb 12 00:16:19 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Fri, 12 Feb 2016 09:16:19 +0100
Subject: [PATCHv2-4.5 10/10] NVMe: Rate limit nvme IO warnings
In-Reply-To: <1455221147-24228-11-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <1455221147-24228-11-git-send-email-keith.busch@intel.com>
Message-ID: <20160212081619.GN27969@c203.arch.suse.de>

On Thu, Feb 11, 2016 at 01:05:47PM -0700, Keith Busch wrote:
> We don't need to spam the kernel logs with thousands of IO cancelling
> messages. We can infer all IO's are being cancelled with fewer, or
> even none at all. This patch rate limits the message and uses the debug
> log level as it is mainly used for testing purposes.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From hch at infradead.org  Fri Feb 12 00:35:08 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Fri, 12 Feb 2016 00:35:08 -0800
Subject: [PATCHv2-4.5 00/10] NVMe fixes
In-Reply-To: <56BD0D4A.60003@fb.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <20160211222848.GA19595@localhost.localdomain>
 <56BD0D4A.60003@fb.com>
Message-ID: <20160212083508.GA18432@infradead.org>

On Thu, Feb 11, 2016 at 03:38:02PM -0700, Jens Axboe wrote:
> I already applied these for 4.5, just didn't send the ack before I had done
> some testing. JFYI.

Argg - some of these including the namespace removal changes absolutely
move into the wrong direction.  I've just been busy reviewing the easier
parts first before coming up with better ways for those.


From axboe at fb.com  Fri Feb 12 07:09:05 2016
From: axboe at fb.com (Jens Axboe)
Date: Fri, 12 Feb 2016 08:09:05 -0700
Subject: [PATCHv2-4.5 00/10] NVMe fixes
In-Reply-To: <20160212083508.GA18432@infradead.org>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <20160211222848.GA19595@localhost.localdomain> <56BD0D4A.60003@fb.com>
 <20160212083508.GA18432@infradead.org>
Message-ID: <56BDF591.5030804@fb.com>

On 02/12/2016 01:35 AM, Christoph Hellwig wrote:
> On Thu, Feb 11, 2016 at 03:38:02PM -0700, Jens Axboe wrote:
>> I already applied these for 4.5, just didn't send the ack before I had done
>> some testing. JFYI.
>
> Argg - some of these including the namespace removal changes absolutely
> move into the wrong direction.  I've just been busy reviewing the easier
> parts first before coming up with better ways for those.

Let's just kill the series, it's not in the 4.6 branch yet either. I'll 
rebase for-linus.

-- 
Jens Axboe



From hare at suse.de  Fri Feb 12 07:18:47 2016
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 12 Feb 2016 16:18:47 +0100
Subject: RCU-ified dm-mpath for testing/review
In-Reply-To: <20160211153452.GA7827@redhat.com>
References: <56B7659C.8040601@dev.mellanox.co.il>
 <56B772D6.2090403@sandisk.com> <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il> <20160207172055.GA6477@redhat.com>
 <56B99A49.5050400@suse.de> <20160209145547.GA21623@redhat.com>
 <56BA0689.9030007@suse.de> <20160210004518.GA23646@redhat.com>
 <20160211015030.GA4481@redhat.com> <20160211153452.GA7827@redhat.com>
Message-ID: <56BDF7D7.8040106@suse.de>

On 02/11/2016 04:34 PM, Mike Snitzer wrote:
> On Wed, Feb 10 2016 at  8:50pm -0500,
> Mike Snitzer <snitzer at redhat.com> wrote:
> 
>> On Tue, Feb 09 2016 at  7:45pm -0500,
>> Mike Snitzer <snitzer at redhat.com> wrote:
>>
>>>
>>> OK, I took a crack at embracing RCU.  Only slightly better performance
>>> on my single NUMA node testbed.  (But I'll have to track down a system
>>> with multiple NUMA nodes to do any justice to the next wave of this
>>> optimization effort)
>>>
>>> This RCU work is very heavy-handed and way too fiddley (there could
>>> easily be bugs).  Anyway, please see:
>>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel2&id=d80a7e4f8b5be9c81e4d452137623b003fa64745
>>>
>>> But this might give you something to build on to arrive at something
>>> more scalable?
>>
>> I've a bit more polished version of this work (broken up into multiple
>> commits, with some fixes, etc) here:
>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
>>
>> Hannes and/or Sagi, if you get a chance to try this on your NUMA system
>> please let me know how it goes.
> 
> Initial review has uncovered some locking problems with the current code
> (nothing that caused crashes or hangs in my testing but...) so please
> hold off on testing until you hear from me (hopefully tomorrow).
> 
Good news is that I've managed to hit the roof for my array with the
devel2 version of those patches. (And a _heavily_ patched-up lpfc
driver :-)
So from that perspective everything's fine now; we've reached the
hardware limit for my setup.
Which in itself is quite impressive; beating Intel P3700 with 16FC
is not bad methinks :-)

So thanks for all your work here.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From keith.busch at intel.com  Fri Feb 12 07:24:48 2016
From: keith.busch at intel.com (Keith Busch)
Date: Fri, 12 Feb 2016 15:24:48 +0000
Subject: [PATCHv2-4.5 00/10] NVMe fixes
In-Reply-To: <20160212083508.GA18432@infradead.org>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <20160211222848.GA19595@localhost.localdomain>
 <56BD0D4A.60003@fb.com> <20160212083508.GA18432@infradead.org>
Message-ID: <20160212152447.GA24020@localhost.localdomain>

On Fri, Feb 12, 2016 at 12:35:08AM -0800, Christoph Hellwig wrote:
> Argg - some of these including the namespace removal changes absolutely
> move into the wrong direction.  I've just been busy reviewing the easier
> parts first before coming up with better ways for those.

Fair enough. I know you're busy in many other areas as well, so I
appreciate the time you've provided.

To the task at hand, this started out to fix a hot-plug regression (is
there any other kind?). More poking revealed other cracks, some of which
are pretty obscure that probably never worked, but still scrambling to
fix before the release anyway.

I'm not trying to significantly alter the design, but I think you're
saying these bolted on fixes are not helping long term. More focus is
moving to Fabrics, so I really want to get this one PCI feature working
before that happens. If you have something in the right direction that
will also help the current cycle, I can wait for when you're ready to
share that.


From snitzer at redhat.com  Fri Feb 12 07:26:51 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Fri, 12 Feb 2016 10:26:51 -0500
Subject: RCU-ified dm-mpath for testing/review
In-Reply-To: <56BDF7D7.8040106@suse.de>
References: <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il>
 <20160207172055.GA6477@redhat.com> <56B99A49.5050400@suse.de>
 <20160209145547.GA21623@redhat.com> <56BA0689.9030007@suse.de>
 <20160210004518.GA23646@redhat.com>
 <20160211015030.GA4481@redhat.com>
 <20160211153452.GA7827@redhat.com> <56BDF7D7.8040106@suse.de>
Message-ID: <20160212152651.GB11104@redhat.com>

On Fri, Feb 12 2016 at 10:18am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/11/2016 04:34 PM, Mike Snitzer wrote:
> > On Wed, Feb 10 2016 at  8:50pm -0500,
> > Mike Snitzer <snitzer at redhat.com> wrote:
> > 
> >> On Tue, Feb 09 2016 at  7:45pm -0500,
> >> Mike Snitzer <snitzer at redhat.com> wrote:
> >>
> >>>
> >>> OK, I took a crack at embracing RCU.  Only slightly better performance
> >>> on my single NUMA node testbed.  (But I'll have to track down a system
> >>> with multiple NUMA nodes to do any justice to the next wave of this
> >>> optimization effort)
> >>>
> >>> This RCU work is very heavy-handed and way too fiddley (there could
> >>> easily be bugs).  Anyway, please see:
> >>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel2&id=d80a7e4f8b5be9c81e4d452137623b003fa64745
> >>>
> >>> But this might give you something to build on to arrive at something
> >>> more scalable?
> >>
> >> I've a bit more polished version of this work (broken up into multiple
> >> commits, with some fixes, etc) here:
> >> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
> >>
> >> Hannes and/or Sagi, if you get a chance to try this on your NUMA system
> >> please let me know how it goes.
> > 
> > Initial review has uncovered some locking problems with the current code
> > (nothing that caused crashes or hangs in my testing but...) so please
> > hold off on testing until you hear from me (hopefully tomorrow).
> > 
> Good news is that I've managed to hit the roof for my array with the
> devel2 version of those patches. (And a _heavily_ patched-up lpfc
> driver :-)
> So from that perspective everything's fine now; we've reached the
> hardware limit for my setup.
> Which in itself is quite impressive; beating Intel P3700 with 16FC
> is not bad methinks :-)
> 
> So thanks for all your work here.

Ah, that's really good news!  But devel2 is definitely _not_ destined
for upstream.  'devel3' is much closer to "ready".  But your testing and
review would really push it forward.

Please see/test:
http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3

Also, please read this header:
http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel3&id=65a01b76502dd68e8ca298ee6614c0151b677f4a

Even with devel2 I hacked it such that repeat_count > 1 is effectively
broken.  I'm now _seriously_ considering deprecating repeat_count
completely (adding a DMWARN that will inform the user. e.g.:
"repeat_count > 1 is no longer supported").  I see no point going to
great lengths to maintain a dm-mpath feature that was only a hack for
when dm-mpath was bio-based.  What do you think?

Thanks,
Mike


From hare at suse.de  Fri Feb 12 08:04:37 2016
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 12 Feb 2016 17:04:37 +0100
Subject: RCU-ified dm-mpath for testing/review
In-Reply-To: <20160212152651.GB11104@redhat.com>
References: <56B77444.3030106@dev.mellanox.co.il>
 <56B776DE.30101@dev.mellanox.co.il> <20160207172055.GA6477@redhat.com>
 <56B99A49.5050400@suse.de> <20160209145547.GA21623@redhat.com>
 <56BA0689.9030007@suse.de> <20160210004518.GA23646@redhat.com>
 <20160211015030.GA4481@redhat.com> <20160211153452.GA7827@redhat.com>
 <56BDF7D7.8040106@suse.de> <20160212152651.GB11104@redhat.com>
Message-ID: <56BE0295.4090809@suse.de>

On 02/12/2016 04:26 PM, Mike Snitzer wrote:
> On Fri, Feb 12 2016 at 10:18am -0500,
> Hannes Reinecke <hare at suse.de> wrote:
>
>> On 02/11/2016 04:34 PM, Mike Snitzer wrote:
>>> On Wed, Feb 10 2016 at  8:50pm -0500,
>>> Mike Snitzer <snitzer at redhat.com> wrote:
>>>
>>>> On Tue, Feb 09 2016 at  7:45pm -0500,
>>>> Mike Snitzer <snitzer at redhat.com> wrote:
>>>>
>>>>>
>>>>> OK, I took a crack at embracing RCU.  Only slightly better performance
>>>>> on my single NUMA node testbed.  (But I'll have to track down a system
>>>>> with multiple NUMA nodes to do any justice to the next wave of this
>>>>> optimization effort)
>>>>>
>>>>> This RCU work is very heavy-handed and way too fiddley (there could
>>>>> easily be bugs).  Anyway, please see:
>>>>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel2&id=d80a7e4f8b5be9c81e4d452137623b003fa64745
>>>>>
>>>>> But this might give you something to build on to arrive at something
>>>>> more scalable?
>>>>
>>>> I've a bit more polished version of this work (broken up into multiple
>>>> commits, with some fixes, etc) here:
>>>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
>>>>
>>>> Hannes and/or Sagi, if you get a chance to try this on your NUMA system
>>>> please let me know how it goes.
>>>
>>> Initial review has uncovered some locking problems with the current code
>>> (nothing that caused crashes or hangs in my testing but...) so please
>>> hold off on testing until you hear from me (hopefully tomorrow).
>>>
>> Good news is that I've managed to hit the roof for my array with the
>> devel2 version of those patches. (And a _heavily_ patched-up lpfc
>> driver :-)
>> So from that perspective everything's fine now; we've reached the
>> hardware limit for my setup.
>> Which in itself is quite impressive; beating Intel P3700 with 16FC
>> is not bad methinks :-)
>>
>> So thanks for all your work here.
>
> Ah, that's really good news!  But devel2 is definitely _not_ destined
> for upstream.  'devel3' is much closer to "ready".  But your testing and
> review would really push it forward.
>
> Please see/test:
> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
>
> Also, please read this header:
> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel3&id=65a01b76502dd68e8ca298ee6614c0151b677f4a
>
> Even with devel2 I hacked it such that repeat_count > 1 is effectively
> broken.  I'm now _seriously_ considering deprecating repeat_count
> completely (adding a DMWARN that will inform the user. e.g.:
> "repeat_count > 1 is no longer supported").  I see no point going to
> great lengths to maintain a dm-mpath feature that was only a hack for
> when dm-mpath was bio-based.  What do you think?
>
Drop it, and make setting of which a no-op.
Never liked it anyway, and these decisions should really be delegated to 
the path selector.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare at suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: J. Hawn, J. Guild, F. Imend?rffer, HRB 16746 (AG N?rnberg)


From snitzer at redhat.com  Fri Feb 12 10:00:32 2016
From: snitzer at redhat.com (Mike Snitzer)
Date: Fri, 12 Feb 2016 13:00:32 -0500
Subject: RCU-ified dm-mpath for testing/review
In-Reply-To: <56BE0295.4090809@suse.de>
References: <20160207172055.GA6477@redhat.com> <56B99A49.5050400@suse.de>
 <20160209145547.GA21623@redhat.com> <56BA0689.9030007@suse.de>
 <20160210004518.GA23646@redhat.com>
 <20160211015030.GA4481@redhat.com>
 <20160211153452.GA7827@redhat.com> <56BDF7D7.8040106@suse.de>
 <20160212152651.GB11104@redhat.com> <56BE0295.4090809@suse.de>
Message-ID: <20160212180032.GA14013@redhat.com>

On Fri, Feb 12 2016 at 11:04am -0500,
Hannes Reinecke <hare at suse.de> wrote:

> On 02/12/2016 04:26 PM, Mike Snitzer wrote:
> >On Fri, Feb 12 2016 at 10:18am -0500,
> >Hannes Reinecke <hare at suse.de> wrote:
> >>>
> >>Good news is that I've managed to hit the roof for my array with the
> >>devel2 version of those patches. (And a _heavily_ patched-up lpfc
> >>driver :-)
> >>So from that perspective everything's fine now; we've reached the
> >>hardware limit for my setup.
> >>Which in itself is quite impressive; beating Intel P3700 with 16FC
> >>is not bad methinks :-)
> >>
> >>So thanks for all your work here.
> >
> >Ah, that's really good news!  But devel2 is definitely _not_ destined
> >for upstream.  'devel3' is much closer to "ready".  But your testing and
> >review would really push it forward.
> >
> >Please see/test:
> >http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
> >
> >Also, please read this header:
> >http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel3&id=65a01b76502dd68e8ca298ee6614c0151b677f4a
> >
> >Even with devel2 I hacked it such that repeat_count > 1 is effectively
> >broken.  I'm now _seriously_ considering deprecating repeat_count
> >completely (adding a DMWARN that will inform the user. e.g.:
> >"repeat_count > 1 is no longer supported").  I see no point going to
> >great lengths to maintain a dm-mpath feature that was only a hack for
> >when dm-mpath was bio-based.  What do you think?
>
> Drop it, and make setting of which a no-op.
> Never liked it anyway, and these decisions should really be
> delegated to the path selector.

Sure, but my point is DM-mpath will no longer be able to provide the
ability to properly handle repeat_count > 1 (because updating the
->current_pgpath crushes the write-side of the RCU).

As such I've rebased 'devel3' to impose repeat_count = 1 (both in
dm-mpath.c and the defaults in each path selector).


From hch at infradead.org  Sat Feb 13 01:21:02 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 01:21:02 -0800
Subject: [PATCHv2-4.5 07/10] NVMe: Simplify device reset failure
In-Reply-To: <1455221147-24228-8-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <1455221147-24228-8-git-send-email-keith.busch@intel.com>
Message-ID: <20160213092102.GA31848@infradead.org>

On Thu, Feb 11, 2016 at 01:05:44PM -0700, Keith Busch wrote:
> A reset failure schedules the device to unbind from the driver through
> the pci driver's 'remove'. This already cleans up all intialization
> from a partial or completed state, so there is no need to duplicate the
> cleanup that could potentially race with a pci hot removal.
> 
> To help understand why a reset failed, the status is logged with the
> existing warning message.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

Looks good,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Sat Feb 13 01:44:01 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 01:44:01 -0800
Subject: [PATCHv2-4.5 04/10] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455221147-24228-5-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <1455221147-24228-5-git-send-email-keith.busch@intel.com>
Message-ID: <20160213094401.GA15318@infradead.org>

So, this one is the thing I don't like.  The real issue here is the
broad scope of namespaces_mutex.  I think we should have a small
scope lock protecting just the list manipulations, which is not
held over I/O, and the issue would sort itself out.

Note that we already have a high level serialization for scanning
by always executing it in the scan_work work_item, so there is no
need for the high level serialization anyway.


From hch at infradead.org  Sat Feb 13 01:46:34 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 01:46:34 -0800
Subject: [PATCHv2-4.5 08/10] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <1455221147-24228-9-git-send-email-keith.busch@intel.com>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <1455221147-24228-9-git-send-email-keith.busch@intel.com>
Message-ID: <20160213094634.GB15318@infradead.org>

On Thu, Feb 11, 2016 at 01:05:45PM -0700, Keith Busch wrote:
> This moves failed queue handling out of the namespace removal path and into
> the reset failure path, fixing a deadlock condition if the controller
> fails or link down during del_gendisk. Previously the driver had to see
> the controller as degraded prior to calling del_gendisk to setup the
> queues to fail. If the controller happened to fail after this though,
> there was no task to end the request_queue.
> 
> On failure, all namespace states are set to 'dead'. This has capacity
> revalidate to 0, and ends all new requests with error status.

I like this a lot, but a few comments below:

> +/**
> + * nvme_kill_ns_queues(): Ends all namespace queues
> + * @ctrl: the dead controller that needs to end
> + *
> + * Call this function when the driver determines it is unable to get the
> + * controller in a state capable of servicing IO.
> + */
> +void nvme_kill_ns_queues(struct nvme_ctrl *ctrl)

Why do we have a separate function for this instead of driving this
from nvme_remove_namespaces?

> @@ -678,7 +678,9 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
>  
>  	spin_lock_irq(&nvmeq->q_lock);
>  	if (unlikely(nvmeq->cq_vector < 0)) {
> -		ret = BLK_MQ_RQ_QUEUE_BUSY;
> +		ret = test_bit(NVME_NS_DEAD, &ns->flags) ?
> +					BLK_MQ_RQ_QUEUE_ERROR :
> +					BLK_MQ_RQ_QUEUE_BUSY;

This would be easier to read with a good old 'if'.


From hch at infradead.org  Sat Feb 13 01:49:35 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 01:49:35 -0800
Subject: [PATCHv2-4.5 00/10] NVMe fixes
In-Reply-To: <20160212152447.GA24020@localhost.localdomain>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <20160211222848.GA19595@localhost.localdomain>
 <56BD0D4A.60003@fb.com> <20160212083508.GA18432@infradead.org>
 <20160212152447.GA24020@localhost.localdomain>
Message-ID: <20160213094935.GC15318@infradead.org>

On Fri, Feb 12, 2016 at 03:24:48PM +0000, Keith Busch wrote:
> Fair enough. I know you're busy in many other areas as well, so I
> appreciate the time you've provided.

Sorry for the delay - I planned to get to it thursday afternoon, but
the common cold nocked me out unexpectedly just around that time and
it took a day to recover me into a state where I could carefully
review complicated changes.

I'm actually much happier now with the remaining patches than I initially
thought - my main issue is the namespace removal work item which should
not be nessecary if we apply locking good practices by reducing the
namespace_mutex scope.


From hch at infradead.org  Sat Feb 13 01:58:51 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 01:58:51 -0800
Subject: [PATCH 1/2] NVMe: Lock out shutdown during pci init
In-Reply-To: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
References: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
Message-ID: <20160213095851.GA15206@infradead.org>

On Thu, Feb 11, 2016 at 04:00:25PM -0700, Keith Busch wrote:
> An unexpected driver unbind needs to be prevented from unmapping the
> pci nvme registers while they're being initialized.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---
>  drivers/nvme/host/pci.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 1b0e447..24a7972 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1927,13 +1927,19 @@ static void nvme_reset_work(struct work_struct *work)
>  
>  	set_bit(NVME_CTRL_RESETTING, &dev->flags);
>  
> +	/*
> +	 * Lock out shutdown so an unexpected driver removal won't unmap
> +	 * registers while reset work is setting them up.
> +	 */
> +	mutex_lock(&dev->shutdown_lock);
>  	result = nvme_dev_map(dev);
>  	if (result)
> -		goto out;
> +		goto out_unlock;
>  
>  	result = nvme_configure_admin_queue(dev);
>  	if (result)
> -		goto out;
> +		goto out_unlock;
> +	mutex_unlock(&dev->shutdown_lock);

Seems like we should be holding the lock over the call
to nvme_dev_disable to get protection for the whole execution,
e.g. by adding a __nvme_dev_disable variant that expects the lock
to be held.

Maybe it's also worth renaming shutdown_lock to register_mutex
or similar to better document what it protects.

>  
>  	nvme_init_queue(dev->queues[0], 0);
>  	result = nvme_alloc_admin_tags(dev);
> @@ -1970,6 +1976,8 @@ static void nvme_reset_work(struct work_struct *work)
>  	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
>  	return;
>  
> + out_unlock:
> +	mutex_unlock(&dev->shutdown_lock);
>   out:
>  	nvme_remove_dead_ctrl(dev, result);
>  }
> -- 
> 2.6.2.307.g37023ba
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
---end quoted text---


From hch at infradead.org  Sat Feb 13 01:59:24 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 01:59:24 -0800
Subject: [PATCH 2/2] NVMe: NULL pointer check for admin queues
In-Reply-To: <1455231626-7166-2-git-send-email-keith.busch@intel.com>
References: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
 <1455231626-7166-2-git-send-email-keith.busch@intel.com>
Message-ID: <20160213095924.GB15206@infradead.org>

On Thu, Feb 11, 2016 at 04:00:26PM -0700, Keith Busch wrote:
> Admin commands don't have a namespace, so don't check the state when
> there isn't one.
> 
> An admin command that does happen to make it to a disabled queue can
> failed immediatly: Initialization commands should not be requeued
> across resets as they would mess up the controller if/when it is
> reinitialized. If a user command made hits this condition, it should
> be able to handle this error as that's the existing result for commands
> when the controller is reset with outstanding admin commands.

Looks fine, but I'd say fold this directly into the respin of the patch
introducing the check.


From hch at infradead.org  Sat Feb 13 02:01:01 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 02:01:01 -0800
Subject: [PATCH] nvme: lightnvm: return ppa completion status
In-Reply-To: <1455190589-20016-2-git-send-email-m@bjorling.me>
References: <1455190589-20016-1-git-send-email-m@bjorling.me>
 <1455190589-20016-2-git-send-email-m@bjorling.me>
Message-ID: <20160213100101.GA28353@infradead.org>

On Thu, Feb 11, 2016 at 12:36:29PM +0100, Matias Bj??rling wrote:
> PPAs sent to device is separately acknowledge in a 64bit status
> variable. The status is stored in DW0 and DW1 of the completion queue
> entry. Store this status inside the nvm_rq for further processing.
> 
> This can later be used to implement retry techniques for failed writes
> and reads.
> 
> Signed-off-by: Matias Bj??rling <m at bjorling.me>

Looks reasonable:

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Sat Feb 13 02:01:48 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sat, 13 Feb 2016 02:01:48 -0800
Subject: [PATCH] nvme: return the whole CQE through the request
 passthrough interface
In-Reply-To: <1454954988-32306-2-git-send-email-hch@lst.de>
References: <1454954988-32306-1-git-send-email-hch@lst.de>
 <1454954988-32306-2-git-send-email-hch@lst.de>
Message-ID: <20160213100148.GB28353@infradead.org>

Jens, Keith:

does this looks fine to you?


From mike at fireburn.co.uk  Sat Feb 13 15:39:52 2016
From: mike at fireburn.co.uk (Mike Lothian)
Date: Sat, 13 Feb 2016 23:39:52 +0000
Subject: [Bug 112121] New: Some PCIe options cause devices to be removed
 after syspend
In-Reply-To: <CAErSpo4beDk_ajsgqMUNJ3TGxDv=ysyPzhXHcx2AhUEUE9ROKA@mail.gmail.com>
References: <bug-112121-41252@https.bugzilla.kernel.org/>
 <CAErSpo4beDk_ajsgqMUNJ3TGxDv=ysyPzhXHcx2AhUEUE9ROKA@mail.gmail.com>
Message-ID: <CAHbf0-GBiMVr4=n3ZT1NSH8Xgdh3TcdDd=Di+pok6Ep6h42ZOQ@mail.gmail.com>

Hi

I've just tested this again, I enabled PCI Hotplug & PCIe Hotplug and
nothing - then I noticed I hadn't enabled the ACPI Hotplug driver -
once I did the issue re-appeared

I then had to use testdisk to restore my partition table :'(

I've attached the updated dmesg & my .config

Cheers

Mike

On 8 February 2016 at 13:51, Bjorn Helgaas <bhelgaas at google.com> wrote:
> [+cc linux-pci, NVMe folks,  power management folks]
>
> On Sun, Feb 7, 2016 at 11:04 AM,  <bugzilla-daemon at bugzilla.kernel.org> wrote:
>> https://bugzilla.kernel.org/show_bug.cgi?id=112121
>>
>>             Bug ID: 112121
>>            Summary: Some PCIe options cause devices to be removed after
>>                     syspend
>>            Product: Drivers
>>            Version: 2.5
>>     Kernel Version: 4.5-rc2
>>           Hardware: All
>>                 OS: Linux
>>               Tree: Mainline
>>             Status: NEW
>>           Severity: normal
>>           Priority: P1
>>          Component: PCI
>>           Assignee: drivers_pci at kernel-bugs.osdl.org
>>           Reporter: mike at fireburn.co.uk
>>         Regression: No
>>
>> Created attachment 203091
>>   --> https://bugzilla.kernel.org/attachment.cgi?id=203091&action=edit
>> Dmesg showing PCIe device removals
>>
>> I was having issues with suspend, when the machine was being resumed iommu
>> started removing devices - including my PCIe NVMe drive which contained my root
>> partition
>>
>> The problem showed up with:
>>
>> [*] PCI support
>> [*]   Support mmconfig PCI config space access
>> [*]   PCI Express Port Bus support
>> [*]     PCI Express Hotplug driver
>> [*]     Root Port Advanced Error Reporting support
>> [*]       PCI Express ECRC settings control
>> < >       PCIe AER error injector support
>> -*-     PCI Express ASPM control
>> [ ]       Debug PCI Express ASPM
>>           Default ASPM policy (BIOS default)  --->
>> [*]   Message Signaled Interrupts (MSI and MSI-X)
>> [ ]   PCI Debugging
>> [*]   Enable PCI resource re-allocation detection
>> < >   PCI Stub driver
>> [*]   Interrupts on hypertransport devices
>> [ ] PCI IOV support
>> [*] PCI PRI support
>> -*- PCI PASID support
>>     PCI host controller drivers  ----
>> < > PCCard (PCMCIA/CardBus) support  ----
>> [*] Support for PCI Hotplug  --->
>> < > RapidIO support
>>
>>
>> This is what I have now:
>>
>> [*] PCI support
>> [*]   Support mmconfig PCI config space access
>> [*]   PCI Express Port Bus support
>> [ ]     Root Port Advanced Error Reporting support
>> -*-     PCI Express ASPM control
>> [ ]       Debug PCI Express ASPM
>>           Default ASPM policy (BIOS default)  --->
>> [*]   Message Signaled Interrupts (MSI and MSI-X)
>> [*]   PCI Debugging
>> [ ]   Enable PCI resource re-allocation detection
>> < >   PCI Stub driver
>> [*]   Interrupts on hypertransport devices
>> [ ] PCI IOV support
>> [ ] PCI PRI support
>> [ ] PCI PASID support
>>     PCI host controller drivers  ----
>> < > PCCard (PCMCIA/CardBus) support  ----
>> [ ] Support for PCI Hotplug  ----
>> < > RapidIO support
>>
>> I tried disabling the iommu driver first but it had no effect
>>
>> If people are interested I could play with the above options to see which one
>> causes the issue
>
> My guess is that PCI hotplug is the important one.  It would be nice
> if dmesg contained enough information to connect nvme0n1 to a PCI
> device.  It'd be even nicer if the PCI core noted device removals or
> whatever happened here.
>
> You don't get any more details if you boot with "ignore_loglevel", do you?
>
> Mike, you didn't mark this as a regression, so I assume it's always
> been this way, and we just haven't noticed it because most people
> enable PCI hotplug (or whatever the relevant config option is).
>
> Bjorn


From willy at linux.intel.com  Sun Feb 14 01:32:58 2016
From: willy at linux.intel.com (Matthew Wilcox)
Date: Sun, 14 Feb 2016 04:32:58 -0500
Subject: M2/NVM vs M2/Sata vs SATA/6GB benchmarks
In-Reply-To: <20160214031333.GA20879@merlins.org>
References: <20160214031333.GA20879@merlins.org>
Message-ID: <20160214093258.GC2590@linux.intel.com>


Hi Marc,

I shan't have time to look at this for another week, but I've added
the NVMe mailing list so somebody else can comment in the meantime. 

On Sat, Feb 13, 2016 at 07:13:33PM -0800, Marc MERLIN wrote:
> Hi Matthew,
> 
> I didn't find a list for NVMe, I'll Cc linux-ide in case this is useful
> to others.
> 
> I just got a Thinkpad P70 skylake laptop and thought I'd try 3 SSDs:
> NVMe/M2 vs Sata/M2 vs Sata/6GB.
> I'm not a benchmarking filesystem expert, so my apologies if I made some
> stupid choices, but hopefully the benchmarks are still useful.
> 
> I'll give the conclusions first. The buffer cache issue with NVMe is weird,
> maybe there is a non default setting I need to tweak?
> 
> a) NVMe is faster for big contiguous I/O, even through dmcrypt
> b) M2 NVMe and M2 sata do not seem to go through buffer cache for writes, but SATA/6GB does
> c) When using buffer cache, SATA/6GB beats NVMe/M2 and SATA/M2 by a lot on block reads. Why?
> d) for random I/O, NVMe is up to 50% faster, and faster with ext4 than btrfs
> e) Encrypted NVMe is a bit faster than non encrypted Sata/M2
> f) At $328 for 512GB M2/NVMe vs $168 for 500GB M2/Sata, unless you're doing big contiguous I/O, the 2x price difference is hard to justify, cached I/O and random I/O are not significantly faster on NVMe.
> 
> 
> 
> 3 SSDs:
> 1) M2/SATA Samsung 850 EVO 500 GB M.2 3.5-Inch SSD (MZ-N5E500BW) 33c/GB
> 
> 2) M2/NVMe: Samsung 950 PRO -Series 512GB PCIe NVMe - M.2 Internal SSD 2-Inch MZ-V5P512BW 64c/GB
> 70:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd Device a802 (rev 01)
> 
> 3) SATA6G: Samsung 850 EVO 2 TB 2.5-Inch SATA III Internal SSD (MZ-75E2T0B/AM) 30c/GB
> 
> For price comparison
> Seagate 2TB Laptop HDD SATA 6Gb/s 32MB Cache 2.5-Inch Internal Bare Drive (ST2000LM003) 4.6c/GB
> 
> Laptop: Skylake Lenovo Thinkpad P70
> Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz x 8 (4 cores HT)
> 
> Filesystem: btrfs on top of 
> cryptsetup luksFormat --align-payload=8192 -s 256 -c aes-xts-plain64 /dev/device
> ddd test: sync; dd if=/mnt/ram/file of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=100M
> dd0 test: sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> 
> Summary: 
> 0) NVMe only really looks faster on big block sequential read/write (indeed 4X read, 2X write).
>    On Random I/O, it's no faster. Other overheads kill its performance advantage.
> 1) For Cached I/O NVMe seems slowed down by dmcrypt in hdparm/ioz ranread/write whereas M2/Sata is not
> 2) 2TB Sata 6GB SSD is faster for read/write encrypted than NVMe is non encrypted (except hdparm)
> 3) iozone through block cache is faster on M2/SATA than M2/NVMe, weird
> 4) iozone through block cache is much faster for everything on Sata6G, also weird
> 5) iozone direct I/O shows NVMe is fastest (by 50% only) and external SATA just a bit faster than M2/Sata
> 6) when using dmcrypt, NVMe is only marginally faster in iozone than Sata/M2 or Sata/6G
> 7) kernel build speed is same on SATA/M2 and NVMe/M2, encrypted or not, but SATA6G is 10% faster (probably the same effect that block cache works better on SATA/6G than SATA/M2 or NVMe/M2)
> 8) ext4 does better on iozone -I (directIO) than btrfs, 2x as fast on random read for NVMe
> 
> 
> 
> 		SATA/M2 dmcrypt	SATA/M2		NVMe dmcrypt	NVMe		SATA6GB/dmcrypt
> hdparm -t	 534MB/s	 517MB/s	1307MB/s	2140MB/s	 534MB/s
> ddd 10GB read	 550MB/s	 560MB/s	2000MB/s	2400MB/s	 550MB/s
> ddd 10GB write	 500MB/s	 503MB/s	1300MB/s	1200MB/s	 506MB/s
> 
> dd with /dev/zero hits optimizations that actually penalize NVMe:
> dd0 10GB read	   7GB/s	 6.7GB/s	 2.3GB/s	 4.4GB/s	 6.6GB/s
> dd0 10GB write	 495MB/s	 503MB/s	 1.3GB/s	 1.3GB/s	 2.0GB/s
> 
> iozone -e -a -s 4096 -i 0 -i 1 -i 2 (cache makes Sata6G faster than M2, weird):
> ioz read 4KB	1760MB/s	1790MB/s	 675MB/s	 766MB/s	7863MB/s
> ioz write 4KB	 268MB/s	 298MB/s	 295MB/s	 291MB/s	 777MB/s
> ioz ranread4KB	5535MB/s	7392MB/s	3185MB/s	7261MB/s	7892MB/s
> ioz ranwrite4KB	 288MB/s	 329MB/s	 205MB/s	 381MB/s	 796MB/s
> 
> iozone -I -e -a -s 4096 -i 0 -i 1 -i 2 (direct IO):
> 		SATA/M2 dmcrypt	 SATA/M2	 NVMe dmcrypt	 NVMe		 SATA6GB
> 										 dmcrypt notcrypted
> 		 btrfs		 btrfs   ext4	 btrfs		 btrfs   ext4	 btrfs     ext4
> ioz read 4KB	 255MB/s	 303MB/s 408MB/s 303MB/s	 396MB/s 562MB/s 310MB/s 379MB/s
> ioz write 4KB	 195MB/s	 238MB/s 344MB/s 257MB/s	 358MB/s 361MB/s 233MB/s 365MB/s
> ioz ranread4KB	 305MB/s	 351MB/s 482MB/s 476MB/s	 699MB/s1434MB/s 309MB/s 475MB/s
> ioz ranwrite4KB	 265MB/s	 223MB/s 392MB/s 260MB/s	 217MB/s 315MB/s 254MB/s 366MB/s
> 
> 
> Details of each test, per type of drive;
> ================================================================================
> saruman M2 SATA 512GB encrypted btrfs
> 
> saruman:/tmp# hdparm -t /dev/mapper/cryptroot
>  Timing buffered disk reads: 1604 MB in  3.00 seconds = 534.03 MB/sec
> 
> iozone -e -a -s 4096 -i 0 -i 1 -i 2:
>                                                               random    random
>               kB  reclen    write  rewrite    read    reread    read     write
>             4096       4   257076   281089  1761711  6094733  6066754   290373
>             4096     512   277108   295167  1588844  8426437 11128258   318731
>             4096    4096   268940   308248  1760087  8062616  8377131   320802
> 
> iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
>             4096       4    38626    45227   102236   111141    30936    64317
>             4096     512   138467   128878   134674   137832   140067   126248
>             4096    4096   195439   273100   255379   307300   305605   265093
> 
> 
> saruman:/mnt/mnt# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> 10485760000 bytes (10 GB) copied, 21.1761 s, 495 MB/s
> 10485760000 bytes (10 GB) copied, 1.49373 s, 7.0 GB/s
> 
> kernel 4.4.1 make -j8:
> real    17m54.712s
> user    126m26.620s
> sys     6m21.948s
> 
> btrfs send/receive encrypted partition to non encrypted, 87GB: 10mn30
> 
> saruman M2 SATA non encrypted btrfs
> -----------------------------------
> saruman:/mnt/mnt4# hdparm -t /dev/sdc4
> /dev/sdc4:
>  Timing buffered disk reads: 1554 MB in  3.00 seconds = 517.93 MB/sec
> 
> iozone -e -a -s 4096 -i 0 -i 1 -i 2:
>                                                               random    random
>               kB  reclen    write  rewrite    read    reread    read     write
>             4096       4   285020   297565  1739944  7434579  6605595   305785
>             4096     512   310937   336617  1298278  3703511  5483860   299416
>             4096    4096   298433   317101  1790911  7392988  7434579   337476
> iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
>             4096       4    47219    93497   118900   126065    39160    86462
>             4096     512   214381   213388   194307   209215   206951   209289
>             4096    4096   238792   221839   303768   352951   351342   223618
> ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
>             4096       4   119524   146035   150173   159105    53090   140697
>             4096     512   358954   354810   366466   423088   419760   375542
>             4096    4096   344087   398828   408259   478337   482569   392487
> 
> saruman:/mnt/mnt3#  sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> 10485760000 bytes (10 GB) copied, 20.8537 s, 503 MB/s
> 10485760000 bytes (10 GB) copied, 1.55651 s, 6.7 GB/s
> 
> kernel 4.4.1 make -j8:
> real    17m55.612s
> user    126m31.952s
> sys     6m27.452s
> 
> 
> ********************************************************************************
> saruman M2 NVMe 512GB encrypted btrfs
> 
> saruman:/tmp# hdparm -t /dev/mapper/cryptroot2
> /dev/mapper/cryptroot2:
>  Timing buffered disk reads: 3924 MB in  3.00 seconds = 1307.56 MB/sec
> 
> iozone -e -a -s 4096 -i 0 -i 1 -i 2:
>                                                               random    random
>               kB  reclen    write  rewrite    read    reread    read     write
>             4096       4   284205   341651   742181  6195843  6178018   341101
>             4096     512   314351   319573   864883  8865630  8902382   384993
>             4096    4096   295741   201042   675234  3281854  3185110   205974
> iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
>             4096       4    80042    83160   122616   127129    41421    53963
>             4096     512   168713   153333   167710   209923   198123   168026
>             4096    4096   257968   258566   303023   490828   476440   260014
> 
> 
> saruman:/mnt/mnt2# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> 10485760000 bytes (10 GB) copied, 8.01494 s, 1.3 GB/s
> 10485760000 bytes (10 GB) copied, 4.63397 s, 2.3 GB/s
> 
> kernel 4.4.1 make -j8:
> real    17m57.513s
> user    126m58.360s
> sys     6m25.164s
> 
> btrfs send/receive encrypted partition to non encrypted, 87GB: 7mn
> (33% faster than Sata M2)
> 
> saruman M2 NVMe non encrypted btrfs
> -----------------------------------
> saruman:/mnt/mnt4# hdparm -t /dev/nvme0n1p4
> /dev/nvme0n1p4:
>  Timing buffered disk reads: 6422 MB in  3.00 seconds = 2140.38 MB/sec
> 
> 
> iozone -e -a -s 4096 -i 0 -i 1 -i 2:
>                                                               random    random
>               kB  reclen    write  rewrite    read    reread    read     write
>             4096       4   274808   271851   684271  2634219  2280673   177630
>             4096     512   290191   336755   668196  3362795  4039655   184779
>             4096    4096   291427   340345   766588  7355007  7261741   381768
> iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
>             4096       4    94814   117116   147645   159695    44075   108039
>             4096     512   271130   267800   239969   325933   301841   271062
>             4096    4096   358602   275933   396591   719099   699939   217665
> ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
>             4096       4   107281   200735   205900   263035    54832   124219
>             4096     512   537478   374641   525060  1173664  1139338   558807
>             4096    4096   361545   526137   562632  1483477  1434069   315883
> 
> saruman:/mnt/mnt4# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> 10485760000 bytes (10 GB) copied, 7.90653 s, 1.3 GB/s
> 10485760000 bytes (10 GB) copied, 2.35863 s, 4.4 GB/s
> 
> kernel 4.4.1 make -j8:
> real    17m54.221s
> user    126m46.264s
> sys     6m10.592s
> 
> 
> ********************************************************************************
> saruman Samsung Evo 850 2TB SSD encrypted btrfs
> 
> hdparm -t
>  Timing buffered disk reads: 1606 MB in  3.00 seconds = 534.87 MB/sec
> 
> iozone -e -a -s 4096 -i 0 -i 1 -i 2:
>                                                               random    random
>               kB  reclen    write  rewrite    read    reread    read     write
>             4096       4   513924   738957  8586475  9144037  8017464   697665
>             4096     512   772100   820362  9287391 10136778 10586522   840551
>             4096    4096   719099   777517  7863339  8047509  7892238   796108
> 
> iozone -I -e -a -s 4096 -i 0 -i 1 -i 2
>             4096       4    76626    47961    92886    80947    36063    63788
>             4096     512   135087   138998   143127   150981   144393   132219
>             4096    4096   233245   234058   310797   311552   309855   254080
> 
> ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2 (unencrypted)
>             4096       4   118013   141212   130575   111534    42343   138654
>             4096     512   336347   312192   332847   414404   424542   342707
>             4096    4096   365227   364638   379298   484160   475122   366536
> 
> 
> saruman:/tmp# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> 10485760000 bytes (10 GB) copied, 5.23919 s, 2.0 GB/s
> 10485760000 bytes (10 GB) copied, 1.58287 s, 6.6 GB/s
> 
> 
> kernel 4.4.1 make -j8:
> real    16m4.305s
> user    104m0.816s
> sys     7m2.832s
> 
> 
> -- 
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems ....
>                                       .... what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901


From sagig at dev.mellanox.co.il  Sun Feb 14 01:59:28 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 14 Feb 2016 11:59:28 +0200
Subject: 4.5-rc iser issues
In-Reply-To: <56C04294.3090701@dev.mellanox.co.il>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il>
Message-ID: <56C05000.1040001@dev.mellanox.co.il>


>> The only other kernel version I had available quickly is 3.16 from Debian
>> Jessie, and that works fine.
>
> Thanks for reporting, I'll have a look.
>
> I suspect this is coming from Keith+Ming changes in
> blk_bio_segment_split()...

OK,

I can clearly see that the block layer commitment to respect the
driver virtual boundary was broken in 4.5.

 From the log:
iser: sg[0] dma_addr:0x85FC06000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[1] dma_addr:0x860334000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
iser: sg[2] dma_addr:0x860335000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
iser: sg[3] dma_addr:0x8621EA000 off:0x0 sz:0x200 dma_len:0x200 ...
iser: sg[4] dma_addr:0x8621EB000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[5] dma_addr:0x860384000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[6] dma_addr:0x860385000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[7] dma_addr:0x860316000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[8] dma_addr:0x860317000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[9] dma_addr:0x860294000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[10] dma_addr:0x860295000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[11] dma_addr:0x8609F8000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[12] dma_addr:0x8609F9000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[13] dma_addr:0x8607DA000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[14] dma_addr:0x8607DB000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[15] dma_addr:0x8607D4000 off:0x0 sz:0x200 dma_len:0x200
--

While iser sets the virtual boundary to be 4096, we can
clearly see that each of the SG elements contain a gap
and should not ever see those...

I'm bisecting now, there are a couple of patches from Ming in
the area of the bio splitting code...

CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
wrt the virtual boundary so I think nvme will break as well.

Attaching a small test program I used to force gappy I/O.

$ ./scatter_data -l 64k -n 128 -d <dev>
-------------- next part --------------
/**
 * Scattered IO test
 *
 * Author: Adir Lev
 **/
#define _GNU_SOURCE

#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <malloc.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <errno.h>
#include <string.h>
#include <getopt.h>
#include <sys/uio.h>
#include <sys/time.h>
#include <assert.h>


#define MAX_SGE 128

int do_write = 0;
int count = 1;
int num_sge = 0;
int bs = 0;
char *dev;
size_t page_size;
void *ibuf;
void *obuf;
long disk_sz = 0;


double time_diff(struct timeval x , struct timeval y) {
	double x_ms , y_ms , diff;
	x_ms = (double)x.tv_sec*1000000 + (double)x.tv_usec;
	y_ms = (double)y.tv_sec*1000000 + (double)y.tv_usec;
	diff = (double)y_ms - (double)x_ms;
	return diff;
}

void print_usage(char* cmd) {
	printf("USAGE: %s -l 1024 -n 2 -d /dev/sdb [-C 1000]\n", cmd);
	printf("\t-l bs in KBytes\n");
	printf("\t-n num of sges to use\n");
	printf("\t-d block device\n");
	printf("\t[-C] num of iterations\n");
}

int open_block_dev() {
	FILE *fp;
	int fd, rc;
	long sz;

	printf("Device: %s\n", dev);
	fd = open(dev, O_RDWR|O_DIRECT|O_SYNC, 777);
	if (fd < 0) {
		perror("Unable to open block device");
		return fd;
	}

	fp = fdopen(fd, "w+");
	if (!fp) {
		printf("failed to fdopen, errno=%d\n", errno);
		return -1;
	}

	rc = fseek(fp, 0, SEEK_END);
	if (rc < 0) {
		printf("failed to fseek, errno=%d\n", errno);
		return -1;
	}

	disk_sz = ftell(fp);
	if (disk_sz < 0) {
		printf("failed to ftell, errno=%d\n", errno);
		return -1;
	}

	rewind(fp);

	return fd;
}

int my_rewind(fd) {
	FILE *fp;

	fp = fdopen(fd, "w+");
	if (!fp) {
		printf("failed to fdopen, errno=%d\n", errno);
		return -1;
	}
	rewind(fp);
	return 0;
}

int parse_args(int argc, char **argv) {
	int option = 0;

	while ((option = getopt(argc, argv,"wC:l:n:d:")) != -1) {
		switch (option) {
			case 'w':
				do_write = 1;
				break;
			case 'C':
				count = atoi(optarg);
				break;
			case 'd':
				dev = optarg;
				break;
			case 'l':
				bs = atoi(optarg);
				break;
			case 'n':
				num_sge = atoi(optarg);
				break;
			default:
				print_usage(argv[0]);
				return -1;
		}
	}

	/* sanity check args */
	if (optind < 4) {
		printf("Mandatory argument(s) missing\n");
		print_usage(argv[0]);
		return -1;
	}

	if (bs == 512) {
		printf("ERROR: Block size must exceed 512Bytes \n");
		return -1;
	}
	bs = bs * 1024;
	if (num_sge > MAX_SGE) {
		printf("ERROR: num_sge (-n) cannot exceed 128\n");
		return -1;
	}

	if (bs % 512 != 0) {
		printf("ERROR: Block size must be multiple of 512\n");
		return -1;
	}

	if ((bs / num_sge) % 512 != 0) {
		printf("ERROR: Block size/num_sge must be multiple of 512\n");
		return -1;
	}

	if (bs > (page_size * 128)) {
		printf("ERROR: Block size cannot exceed 524288 Bytes (4096B * 128)\n");
		return -1;
	}

	if (count < 1) {
		printf("ERROR: count needs to be higher than 0\n");
		return -1;
	}

	return 0;
}

void* alloc_sges()
{
	void *buf;
	int sge_size = bs / num_sge;

	if (sge_size > page_size) {
		printf("ERROR: sge size cannot exceed page size\n");
		return NULL;
	}

	buf = memalign(page_size, num_sge * page_size);
	if (!buf)
		perror( "ERROR: cannot allocate memory");

	memset(buf, 0, num_sge * page_size);

	return buf;
}

int sample_counter() {
	FILE *fp;
	int val;

	system("iscsiadm -m session -s | grep fmr_un | awk '{print $2}'"
	       " | awk '{ sum+=$1} END {print sum}' >> /tmp/indir_counter");

	fp = fopen("/tmp/indir_counter", "rw");
	if (!fp) {
		perror("Unable to open counter file");
		return -1;
	}

	fscanf(fp, "%d", &val);

	if (val < 0) {
		printf("Failed to get fmr_unaligned counter\n");
		return -1;
	}

	fclose(fp);
	unlink("/tmp/indir_counter");
	return val;
}

void get_stats(struct timeval t_before, struct timeval t_after) {
	double t_diff;
	float iops;
	long bw;

	t_diff = time_diff(t_before, t_after);
	iops = (float)count / t_diff * 1000;
	bw = iops * bs;

	printf("time elapsed in sec %f\n", t_diff/1000000);
	printf("iops: %.2fkiops\n", iops);
	printf("BW: %ldKB\n", bw);
}

int calc_counter(int before, int after) {
	int total = 0;

	total = after - before;
	if (total != count * 2) {
		printf("count: %d, fmr_unaligned_cntr: %d\n", count, total);
		return -1;
	} else {
		return 0;
	}
}

static void dump_bufs(void *s1, void *s2, int len)
{
	int i;

	for (i = 0; i < len; i += 8) {
		uint64_t idword = *(uint64_t *)&(((char *)s2)[i]);
		uint64_t odword = *(uint64_t *)&(((char *)s1)[i]);

		printf("obuf[%x]: %x,		ibuf[%x]: %x\n",
			i, odword, i, idword);
	}
}

static int run_rw(int is_write, int fd, void *buf)
{
	struct iovec iov[num_sge];
	int sge_size = bs / num_sge;
	int max = page_size - sge_size;
	int i = 0, j = 0, offset = 0, rc = 0;
	ssize_t bytes_read;
	long bytes_left = disk_sz;

	/* for every iteration */
	for (i = 0; i < count; i++) {
		if (max > 0)
			offset = (512 * i) % max;
		if (bytes_left < bs) {
			rc = my_rewind(fd);
			if (rc < 0)
				return rc;
			printf("count: %d, no space left on block "
			       "device, rewinding\n", i);
			bytes_left = disk_sz;
		}
		/* for every sge */
		for (j = 0; j < num_sge; j++) {
			/* change offset in page */
			iov[j].iov_base = buf + (page_size * j) + offset;
			iov[j].iov_len = sge_size;
			if (is_write)
				memset(iov[j].iov_base, i+j, iov[j].iov_len);
		}

		if (is_write) {
			bytes_read = writev(fd, iov, num_sge);
			if (bytes_read < bs) {
				if (bytes_read < 0) {
					printf("failed to writev, bytes=%d, "
						"errno=%d\n", bytes_read, errno);
					perror("failed to writev");
				} else
					printf("writev less than expected. "
						"Bytes=%d, expected %d\n", bytes_read, bs);
				return -1;
			}
		} else {
			bytes_read = readv(fd, iov, num_sge);
			if (bytes_read < bs) {
				if (bytes_read < 0) {
					printf("failed to readv, bytes=%d, "
						"errno=%d\n", bytes_read, errno);
					perror("failed to readv");
				} else
					printf("readv less than expected. "
						"Bytes=%d, expected %d\n", bytes_read, bs);
				return -1;
			}
		}
		bytes_left -= bs;
	}

	return 0;
}

int run_iovec_traffic(int fd)
{
	int rc;

	rc = my_rewind(fd);
	if (rc) {
		printf("rewind failed\n");
		return -1;
	}

	rc = run_rw(1, fd, obuf);
	if (rc) {
		printf("write failed\n");
		return -1;
	}

	rc = my_rewind(fd);
	if (rc) {
		printf("rewind failed\n");
		return -1;
	}

	rc = run_rw(0, fd, ibuf);
	if (rc) {
		printf("read failed\n");
		return -1;
	}

	rc = memcmp(ibuf, obuf, bs);
	if (rc) {
		printf("memcmp failed\n");
		dump_bufs(obuf, ibuf, bs);
		return -1;
	}

	return rc;
}

int main(int argc, char **argv) {
	struct timeval t_before, t_after;
	void **page_list = NULL;
	int fd, before_counter = 0, after_counter = 0, rc = 0;

	page_size = sysconf(_SC_PAGESIZE);
	rc = parse_args(argc, argv);
	if (rc)
		return -1;

	fd = open_block_dev();
	if (fd < 0)
		return -1;

	ibuf = alloc_sges();
	if (!ibuf) {
		rc = -ENOMEM;
		goto out;
	}

	obuf = alloc_sges();
	if (!obuf) {
		rc = -ENOMEM;
		goto out;
	}

	before_counter = sample_counter();
	if (before_counter < 0) {
		rc = -1;
		goto out;
	}

	gettimeofday(&t_before, NULL);
	rc = run_iovec_traffic(fd);
	gettimeofday(&t_after, NULL);

	if (rc) {
		printf("Exiting with rc=%d\n", rc);
		goto out;
	}

	get_stats(t_before, t_after);

	after_counter = sample_counter();
	if (after_counter < 0) {
		rc = -1;
		goto out;
	}

	rc = calc_counter(before_counter, after_counter);
	if (rc) {
		printf("Test Failed unaligned count\n");
		goto out;
	}

	printf("Test Passes\n");
out:
	close(fd);
	free (ibuf);
	free (obuf);

	return rc;
}

From sagig at dev.mellanox.co.il  Sun Feb 14 03:36:15 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 14 Feb 2016 13:36:15 +0200
Subject: 4.5-rc iser issues
In-Reply-To: <56C05000.1040001@dev.mellanox.co.il>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il> <56C05000.1040001@dev.mellanox.co.il>
Message-ID: <56C066AF.6050901@dev.mellanox.co.il>


> I'm bisecting now, there are a couple of patches from Ming in
> the area of the bio splitting code...
>
> CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
> wrt the virtual boundary so I think nvme will break as well.

Bisection reveals that this one is the culprit:

commit 52cc6eead9095e2faf2ec7afc013aa3af1f01ac5
Author: Ming Lei <ming.lei at canonical.com>
Date:   Thu Sep 17 09:58:38 2015 -0600

     block: blk-merge: fast-clone bio when splitting rw bios

     biovecs has become immutable since v3.13, so it isn't necessary
     to allocate biovecs for the new cloned bios, then we can save
     one extra biovecs allocation/copy, and the allocation is often
     not fixed-length and a bit more expensive.

     For example, if the 'max_sectors_kb' of null blk's queue is set
     as 16(32 sectors) via sysfs just for making more splits, this patch
     can increase throught about ~70% in the sequential read test over
     null_blk(direct io, bs: 1M).

     Cc: Christoph Hellwig <hch at infradead.org>
     Cc: Kent Overstreet <kent.overstreet at gmail.com>
     Cc: Ming Lin <ming.l at ssi.samsung.com>
     Cc: Dongsu Park <dpark at posteo.net>
     Signed-off-by: Ming Lei <ming.lei at canonical.com>

     This fixes a performance regression introduced by commit 54efd50bfd,
     and allows us to take full advantage of the fact that we have immutable
     bio_vecs. Hand applied, as it rejected violently with commit
     5014c311baa2.

     Signed-off-by: Jens Axboe <axboe at fb.com>
--


From sagig at dev.mellanox.co.il  Sun Feb 14 06:02:18 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 14 Feb 2016 16:02:18 +0200
Subject: 4.5-rc iser issues
In-Reply-To: <56C066AF.6050901@dev.mellanox.co.il>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il> <56C05000.1040001@dev.mellanox.co.il>
 <56C066AF.6050901@dev.mellanox.co.il>
Message-ID: <56C088EA.1050901@dev.mellanox.co.il>


>> I'm bisecting now, there are a couple of patches from Ming in
>> the area of the bio splitting code...
>>
>> CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
>> wrt the virtual boundary so I think nvme will break as well.
>
> Bisection reveals that this one is the culprit:
>
> commit 52cc6eead9095e2faf2ec7afc013aa3af1f01ac5
> Author: Ming Lei <ming.lei at canonical.com>
> Date:   Thu Sep 17 09:58:38 2015 -0600
>
>      block: blk-merge: fast-clone bio when splitting rw bios
>
>      biovecs has become immutable since v3.13, so it isn't necessary
>      to allocate biovecs for the new cloned bios, then we can save
>      one extra biovecs allocation/copy, and the allocation is often
>      not fixed-length and a bit more expensive.
>
>      For example, if the 'max_sectors_kb' of null blk's queue is set
>      as 16(32 sectors) via sysfs just for making more splits, this patch
>      can increase throught about ~70% in the sequential read test over
>      null_blk(direct io, bs: 1M).
>
>      Cc: Christoph Hellwig <hch at infradead.org>
>      Cc: Kent Overstreet <kent.overstreet at gmail.com>
>      Cc: Ming Lin <ming.l at ssi.samsung.com>
>      Cc: Dongsu Park <dpark at posteo.net>
>      Signed-off-by: Ming Lei <ming.lei at canonical.com>
>
>      This fixes a performance regression introduced by commit 54efd50bfd,
>      and allows us to take full advantage of the fact that we have
> immutable
>      bio_vecs. Hand applied, as it rejected violently with commit
>      5014c311baa2.
>
>      Signed-off-by: Jens Axboe <axboe at fb.com>
> --

Looks like there is a problem with bio_clone_fast()

This change makes the problem go away:
--
diff --git a/block/bio.c b/block/bio.c
index dbabd48..5e93733 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1791,7 +1791,7 @@ struct bio *bio_split(struct bio *bio, int sectors,
          * Discards need a mutable bio_vec to accommodate the payload
          * required by the DSM TRIM and UNMAP commands.
          */
-       if (bio->bi_rw & REQ_DISCARD)
+       if (1 || bio->bi_rw & REQ_DISCARD)
                 split = bio_clone_bioset(bio, gfp, bs);
         else
                 split = bio_clone_fast(bio, gfp, bs);
--

Any thoughts?


From hch at infradead.org  Sun Feb 14 07:22:23 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sun, 14 Feb 2016 07:22:23 -0800
Subject: 4.5-rc iser issues
In-Reply-To: <56C088EA.1050901@dev.mellanox.co.il>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il>
 <56C05000.1040001@dev.mellanox.co.il>
 <56C066AF.6050901@dev.mellanox.co.il>
 <56C088EA.1050901@dev.mellanox.co.il>
Message-ID: <20160214152223.GA5323@infradead.org>

Adding Ming to Cc.

But I don't think simply not cloning the biovecs is the right thing
to do in the end.  This must be something with the bvec iterators.

Full quote for Ming:

On Sun, Feb 14, 2016 at 04:02:18PM +0200, Sagi Grimberg wrote:
> 
> >>I'm bisecting now, there are a couple of patches from Ming in
> >>the area of the bio splitting code...
> >>
> >>CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
> >>wrt the virtual boundary so I think nvme will break as well.
> >
> >Bisection reveals that this one is the culprit:
> >
> >commit 52cc6eead9095e2faf2ec7afc013aa3af1f01ac5
> >Author: Ming Lei <ming.lei at canonical.com>
> >Date:   Thu Sep 17 09:58:38 2015 -0600
> >
> >     block: blk-merge: fast-clone bio when splitting rw bios
> >
> >     biovecs has become immutable since v3.13, so it isn't necessary
> >     to allocate biovecs for the new cloned bios, then we can save
> >     one extra biovecs allocation/copy, and the allocation is often
> >     not fixed-length and a bit more expensive.
> >
> >     For example, if the 'max_sectors_kb' of null blk's queue is set
> >     as 16(32 sectors) via sysfs just for making more splits, this patch
> >     can increase throught about ~70% in the sequential read test over
> >     null_blk(direct io, bs: 1M).
> >
> >     Cc: Christoph Hellwig <hch at infradead.org>
> >     Cc: Kent Overstreet <kent.overstreet at gmail.com>
> >     Cc: Ming Lin <ming.l at ssi.samsung.com>
> >     Cc: Dongsu Park <dpark at posteo.net>
> >     Signed-off-by: Ming Lei <ming.lei at canonical.com>
> >
> >     This fixes a performance regression introduced by commit 54efd50bfd,
> >     and allows us to take full advantage of the fact that we have
> >immutable
> >     bio_vecs. Hand applied, as it rejected violently with commit
> >     5014c311baa2.
> >
> >     Signed-off-by: Jens Axboe <axboe at fb.com>
> >--
> 
> Looks like there is a problem with bio_clone_fast()
> 
> This change makes the problem go away:
> --
> diff --git a/block/bio.c b/block/bio.c
> index dbabd48..5e93733 100644
> --- a/block/bio.c
> +++ b/block/bio.c
> @@ -1791,7 +1791,7 @@ struct bio *bio_split(struct bio *bio, int sectors,
>          * Discards need a mutable bio_vec to accommodate the payload
>          * required by the DSM TRIM and UNMAP commands.
>          */
> -       if (bio->bi_rw & REQ_DISCARD)
> +       if (1 || bio->bi_rw & REQ_DISCARD)
>                 split = bio_clone_bioset(bio, gfp, bs);
>         else
>                 split = bio_clone_fast(bio, gfp, bs);
> --
> 
> Any thoughts?
---end quoted text---


From ming.lei at canonical.com  Sun Feb 14 08:20:58 2016
From: ming.lei at canonical.com (Ming Lei)
Date: Mon, 15 Feb 2016 00:20:58 +0800
Subject: 4.5-rc iser issues
In-Reply-To: <20160214152223.GA5323@infradead.org>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il>
 <56C05000.1040001@dev.mellanox.co.il>
 <56C066AF.6050901@dev.mellanox.co.il>
 <56C088EA.1050901@dev.mellanox.co.il>
 <20160214152223.GA5323@infradead.org>
Message-ID: <CACVXFVNAkf5+XFyf2JkcfeEPO2sv9bqHMR+N+=kKFzrtXpFEnA@mail.gmail.com>

Hi Sagi,

On Sun, Feb 14, 2016 at 11:22 PM, Christoph Hellwig <hch at infradead.org> wrote:
> Adding Ming to Cc.
>
> But I don't think simply not cloning the biovecs is the right thing
> to do in the end.  This must be something with the bvec iterators.

I agree with Christoph, and there might be issues somewhere.

> From the log:
> iser: sg[0] dma_addr:0x85FC06000 off:0x0 sz:0x200 dma_len:0x200
> iser: sg[1] dma_addr:0x860334000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
> iser: sg[2] dma_addr:0x860335000 off:0x0 sz:0x200 dma_len:0x200 <-- gap

The above gap shouldn't have come since blk_bio_segment_split() splits
out one new bio if gap is detected.

Sort of the following code can be added in driver or prep_fn to check if
bvec of  the rq is correct:

rq_for_each_segment(bvec, sc->request, iter) {
     //check if there is gap between bvec
}

I don't know how to use iser, and looks everything works fine after
I setup virt boundary as 4095 for null_blk by the attachment
patch.

> Full quote for Ming:
>
> On Sun, Feb 14, 2016 at 04:02:18PM +0200, Sagi Grimberg wrote:
>>
>> >>I'm bisecting now, there are a couple of patches from Ming in
>> >>the area of the bio splitting code...
>> >>
>> >>CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
>> >>wrt the virtual boundary so I think nvme will break as well.

The bisected commit is merged to v4.3, and looks no such kind of
report from nvme.

>> >
>> >Bisection reveals that this one is the culprit:
>> >
>> >commit 52cc6eead9095e2faf2ec7afc013aa3af1f01ac5
>> >Author: Ming Lei <ming.lei at canonical.com>
>> >Date:   Thu Sep 17 09:58:38 2015 -0600
>> >
>> >     block: blk-merge: fast-clone bio when splitting rw bios
>> >
>> >     biovecs has become immutable since v3.13, so it isn't necessary
>> >     to allocate biovecs for the new cloned bios, then we can save
>> >     one extra biovecs allocation/copy, and the allocation is often
>> >     not fixed-length and a bit more expensive.
>> >
>> >     For example, if the 'max_sectors_kb' of null blk's queue is set
>> >     as 16(32 sectors) via sysfs just for making more splits, this patch
>> >     can increase throught about ~70% in the sequential read test over
>> >     null_blk(direct io, bs: 1M).
>> >
>> >     Cc: Christoph Hellwig <hch at infradead.org>
>> >     Cc: Kent Overstreet <kent.overstreet at gmail.com>
>> >     Cc: Ming Lin <ming.l at ssi.samsung.com>
>> >     Cc: Dongsu Park <dpark at posteo.net>
>> >     Signed-off-by: Ming Lei <ming.lei at canonical.com>
>> >
>> >     This fixes a performance regression introduced by commit 54efd50bfd,
>> >     and allows us to take full advantage of the fact that we have
>> >immutable
>> >     bio_vecs. Hand applied, as it rejected violently with commit
>> >     5014c311baa2.
>> >
>> >     Signed-off-by: Jens Axboe <axboe at fb.com>
>> >--
>>
>> Looks like there is a problem with bio_clone_fast()
>>
>> This change makes the problem go away:
>> --
>> diff --git a/block/bio.c b/block/bio.c
>> index dbabd48..5e93733 100644
>> --- a/block/bio.c
>> +++ b/block/bio.c
>> @@ -1791,7 +1791,7 @@ struct bio *bio_split(struct bio *bio, int sectors,
>>          * Discards need a mutable bio_vec to accommodate the payload
>>          * required by the DSM TRIM and UNMAP commands.
>>          */
>> -       if (bio->bi_rw & REQ_DISCARD)
>> +       if (1 || bio->bi_rw & REQ_DISCARD)
>>                 split = bio_clone_bioset(bio, gfp, bs);
>>         else
>>                 split = bio_clone_fast(bio, gfp, bs);
>> --
>>
>> Any thoughts?

I don't think bio_clone_fast() is wrong, which use bvec table from
the original bio, and drivers are not allowed to change the table, and
should just call the standard iterator helpers to access bvec.

Also bio_for_each_segment_all() can't be used to iterate over one
cloned bio.

Thanks,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: null_blk_sg_virt_boundary.patch
Type: text/x-patch
Size: 1621 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20160215/26a7157b/attachment.bin>

From sagig at dev.mellanox.co.il  Sun Feb 14 08:29:56 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 14 Feb 2016 18:29:56 +0200
Subject: 4.5-rc iser issues
In-Reply-To: <CACVXFVNAkf5+XFyf2JkcfeEPO2sv9bqHMR+N+=kKFzrtXpFEnA@mail.gmail.com>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il> <56C05000.1040001@dev.mellanox.co.il>
 <56C066AF.6050901@dev.mellanox.co.il> <56C088EA.1050901@dev.mellanox.co.il>
 <20160214152223.GA5323@infradead.org>
 <CACVXFVNAkf5+XFyf2JkcfeEPO2sv9bqHMR+N+=kKFzrtXpFEnA@mail.gmail.com>
Message-ID: <56C0AB84.9040004@dev.mellanox.co.il>

> Hi Sagi,

Hey,

>> But I don't think simply not cloning the biovecs is the right thing
>> to do in the end.  This must be something with the bvec iterators.
>
> I agree with Christoph, and there might be issues somewhere.
>

Me too, it was just an isolation step...

>>  From the log:
>> iser: sg[0] dma_addr:0x85FC06000 off:0x0 sz:0x200 dma_len:0x200
>> iser: sg[1] dma_addr:0x860334000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
>> iser: sg[2] dma_addr:0x860335000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
>
> The above gap shouldn't have come since blk_bio_segment_split() splits
> out one new bio if gap is detected.
>
> Sort of the following code can be added in driver or prep_fn to check if
> bvec of  the rq is correct:
>
> rq_for_each_segment(bvec, sc->request, iter) {
>       //check if there is gap between bvec
> }

I added this indication and the gap detection does trigger a bio
split.

>
> I don't know how to use iser, and looks everything works fine after
> I setup virt boundary as 4095 for null_blk by the attachment
> patch.

That's probably because it's artificial and there is no HW with a real
limitation...

>
>> Full quote for Ming:
>>
>> On Sun, Feb 14, 2016 at 04:02:18PM +0200, Sagi Grimberg wrote:
>>>
>>>>> I'm bisecting now, there are a couple of patches from Ming in
>>>>> the area of the bio splitting code...
>>>>>
>>>>> CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
>>>>> wrt the virtual boundary so I think nvme will break as well.
>
> The bisected commit is merged to v4.3, and looks no such kind of
> report from nvme.

I'm wandering how can that be... because clearly iser is seeing gaps
which like nvme, it can't handle those. Maybe this is scsi specific?


From ming.lei at canonical.com  Sun Feb 14 08:39:04 2016
From: ming.lei at canonical.com (Ming Lei)
Date: Mon, 15 Feb 2016 00:39:04 +0800
Subject: 4.5-rc iser issues
In-Reply-To: <56C0AB84.9040004@dev.mellanox.co.il>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il>
 <56C05000.1040001@dev.mellanox.co.il>
 <56C066AF.6050901@dev.mellanox.co.il>
 <56C088EA.1050901@dev.mellanox.co.il>
 <20160214152223.GA5323@infradead.org>
 <CACVXFVNAkf5+XFyf2JkcfeEPO2sv9bqHMR+N+=kKFzrtXpFEnA@mail.gmail.com>
 <56C0AB84.9040004@dev.mellanox.co.il>
Message-ID: <CACVXFVOYu0j12ixWokg-LfQusKb8TMUCKAiAo3N80z3xMpdTYQ@mail.gmail.com>

On Mon, Feb 15, 2016 at 12:29 AM, Sagi Grimberg
<sagig at dev.mellanox.co.il> wrote:
>> Hi Sagi,
>
>
> Hey,
>
>>> But I don't think simply not cloning the biovecs is the right thing
>>> to do in the end.  This must be something with the bvec iterators.
>>
>>
>> I agree with Christoph, and there might be issues somewhere.
>>
>
> Me too, it was just an isolation step...
>
>>>  From the log:
>>> iser: sg[0] dma_addr:0x85FC06000 off:0x0 sz:0x200 dma_len:0x200
>>> iser: sg[1] dma_addr:0x860334000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
>>> iser: sg[2] dma_addr:0x860335000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
>>
>>
>> The above gap shouldn't have come since blk_bio_segment_split() splits
>> out one new bio if gap is detected.
>>
>> Sort of the following code can be added in driver or prep_fn to check if
>> bvec of  the rq is correct:
>>
>> rq_for_each_segment(bvec, sc->request, iter) {
>>       //check if there is gap between bvec
>> }
>
>
> I added this indication and the gap detection does trigger a bio
> split.

Then I suggest to add the similar check inside blk_queue_split()
further to see if the result bio('res' bio in the function) has gap.

>
>>
>> I don't know how to use iser, and looks everything works fine after
>> I setup virt boundary as 4095 for null_blk by the attachment
>> patch.
>
>
> That's probably because it's artificial and there is no HW with a real
> limitation...

I do check the trace_printk log, and basically one rq only includes one
segment if I set bs as 512 in fio test.

Thanks,

>
>>
>>> Full quote for Ming:
>>>
>>> On Sun, Feb 14, 2016 at 04:02:18PM +0200, Sagi Grimberg wrote:
>>>>
>>>>
>>>>>> I'm bisecting now, there are a couple of patches from Ming in
>>>>>> the area of the bio splitting code...
>>>>>>
>>>>>> CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
>>>>>> wrt the virtual boundary so I think nvme will break as well.
>>
>>
>> The bisected commit is merged to v4.3, and looks no such kind of
>> report from nvme.
>
>
> I'm wandering how can that be... because clearly iser is seeing gaps
> which like nvme, it can't handle those. Maybe this is scsi specific?


From hare at suse.de  Sun Feb 14 22:47:35 2016
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 15 Feb 2016 07:47:35 +0100
Subject: RCU-ified dm-mpath for testing/review
In-Reply-To: <20160212180032.GA14013@redhat.com>
References: <20160207172055.GA6477@redhat.com> <56B99A49.5050400@suse.de>
 <20160209145547.GA21623@redhat.com> <56BA0689.9030007@suse.de>
 <20160210004518.GA23646@redhat.com> <20160211015030.GA4481@redhat.com>
 <20160211153452.GA7827@redhat.com> <56BDF7D7.8040106@suse.de>
 <20160212152651.GB11104@redhat.com> <56BE0295.4090809@suse.de>
 <20160212180032.GA14013@redhat.com>
Message-ID: <56C17487.7050301@suse.de>

On 02/12/2016 07:00 PM, Mike Snitzer wrote:
> On Fri, Feb 12 2016 at 11:04am -0500,
> Hannes Reinecke <hare at suse.de> wrote:
> 
>> On 02/12/2016 04:26 PM, Mike Snitzer wrote:
>>> On Fri, Feb 12 2016 at 10:18am -0500,
>>> Hannes Reinecke <hare at suse.de> wrote:
>>>>>
>>>> Good news is that I've managed to hit the roof for my array with the
>>>> devel2 version of those patches. (And a _heavily_ patched-up lpfc
>>>> driver :-)
>>>> So from that perspective everything's fine now; we've reached the
>>>> hardware limit for my setup.
>>>> Which in itself is quite impressive; beating Intel P3700 with 16FC
>>>> is not bad methinks :-)
>>>>
>>>> So thanks for all your work here.
>>>
>>> Ah, that's really good news!  But devel2 is definitely _not_ destined
>>> for upstream.  'devel3' is much closer to "ready".  But your testing and
>>> review would really push it forward.
>>>
>>> Please see/test:
>>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/log/?h=devel3
>>>
>>> Also, please read this header:
>>> http://git.kernel.org/cgit/linux/kernel/git/snitzer/linux.git/commit/?h=devel3&id=65a01b76502dd68e8ca298ee6614c0151b677f4a
>>>
>>> Even with devel2 I hacked it such that repeat_count > 1 is effectively
>>> broken.  I'm now _seriously_ considering deprecating repeat_count
>>> completely (adding a DMWARN that will inform the user. e.g.:
>>> "repeat_count > 1 is no longer supported").  I see no point going to
>>> great lengths to maintain a dm-mpath feature that was only a hack for
>>> when dm-mpath was bio-based.  What do you think?
>>
>> Drop it, and make setting of which a no-op.
>> Never liked it anyway, and these decisions should really be
>> delegated to the path selector.
> 
> Sure, but my point is DM-mpath will no longer be able to provide the
> ability to properly handle repeat_count > 1 (because updating the
> ->current_pgpath crushes the write-side of the RCU).
> 
Fully understood. But as I said, the _need_ for repeat_count has
essentially vanished with the move to request-based multipath.

> As such I've rebased 'devel3' to impose repeat_count = 1 (both in
> dm-mpath.c and the defaults in each path selector).

Kewl.
I'll give it a go.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From ming.lei at canonical.com  Sun Feb 14 23:11:11 2016
From: ming.lei at canonical.com (Ming Lei)
Date: Mon, 15 Feb 2016 15:11:11 +0800
Subject: 4.5-rc iser issues
In-Reply-To: <56C0AB84.9040004@dev.mellanox.co.il>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il>
 <56C05000.1040001@dev.mellanox.co.il>
 <56C066AF.6050901@dev.mellanox.co.il>
 <56C088EA.1050901@dev.mellanox.co.il>
 <20160214152223.GA5323@infradead.org>
 <CACVXFVNAkf5+XFyf2JkcfeEPO2sv9bqHMR+N+=kKFzrtXpFEnA@mail.gmail.com>
 <56C0AB84.9040004@dev.mellanox.co.il>
Message-ID: <CACVXFVOBjT43CkzfG1gRz1A27BQRSJ2qKOZ91oU16ZbzUh7ejQ@mail.gmail.com>

On Mon, Feb 15, 2016 at 12:29 AM, Sagi Grimberg
<sagig at dev.mellanox.co.il> wrote:
>> Hi Sagi,
>
>
> Hey,
>
>>> But I don't think simply not cloning the biovecs is the right thing
>>> to do in the end.  This must be something with the bvec iterators.
>>
>>
>> I agree with Christoph, and there might be issues somewhere.
>>
>
> Me too, it was just an isolation step...
>
>>>  From the log:
>>> iser: sg[0] dma_addr:0x85FC06000 off:0x0 sz:0x200 dma_len:0x200
>>> iser: sg[1] dma_addr:0x860334000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
>>> iser: sg[2] dma_addr:0x860335000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
>>
>>
>> The above gap shouldn't have come since blk_bio_segment_split() splits
>> out one new bio if gap is detected.
>>
>> Sort of the following code can be added in driver or prep_fn to check if
>> bvec of  the rq is correct:
>>
>> rq_for_each_segment(bvec, sc->request, iter) {
>>       //check if there is gap between bvec
>> }
>
>
> I added this indication and the gap detection does trigger a bio
> split.
>
>>
>> I don't know how to use iser, and looks everything works fine after
>> I setup virt boundary as 4095 for null_blk by the attachment
>> patch.
>
>
> That's probably because it's artificial and there is no HW with a real
> limitation...
>
>>
>>> Full quote for Ming:
>>>
>>> On Sun, Feb 14, 2016 at 04:02:18PM +0200, Sagi Grimberg wrote:
>>>>
>>>>
>>>>>> I'm bisecting now, there are a couple of patches from Ming in
>>>>>> the area of the bio splitting code...
>>>>>>
>>>>>> CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
>>>>>> wrt the virtual boundary so I think nvme will break as well.
>>
>>
>> The bisected commit is merged to v4.3, and looks no such kind of
>> report from nvme.
>
>
> I'm wandering how can that be... because clearly iser is seeing gaps
> which like nvme, it can't handle those. Maybe this is scsi specific?

I can reproduce the issue now, and it is easy to trigger it via your test
code on scsi device, but a bit difficult to get it on null_blk.

Turns out it is a block core issue, and it is in bio_will_gap() which gets
the last bvec via 'bi_io_vec[prev->bi_vcnt - 1]' directly.  I have posted
out one patchset for fixing the issue:

http://marc.info/?l=linux-kernel&m=145551975429092&w=2

Thanks,
Ming


From sagig at dev.mellanox.co.il  Sun Feb 14 23:45:13 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 15 Feb 2016 09:45:13 +0200
Subject: 4.5-rc iser issues
In-Reply-To: <CACVXFVOBjT43CkzfG1gRz1A27BQRSJ2qKOZ91oU16ZbzUh7ejQ@mail.gmail.com>
References: <20160214074119.GA24558@infradead.org>
 <56C04294.3090701@dev.mellanox.co.il> <56C05000.1040001@dev.mellanox.co.il>
 <56C066AF.6050901@dev.mellanox.co.il> <56C088EA.1050901@dev.mellanox.co.il>
 <20160214152223.GA5323@infradead.org>
 <CACVXFVNAkf5+XFyf2JkcfeEPO2sv9bqHMR+N+=kKFzrtXpFEnA@mail.gmail.com>
 <56C0AB84.9040004@dev.mellanox.co.il>
 <CACVXFVOBjT43CkzfG1gRz1A27BQRSJ2qKOZ91oU16ZbzUh7ejQ@mail.gmail.com>
Message-ID: <56C18209.7090903@dev.mellanox.co.il>


> I can reproduce the issue now, and it is easy to trigger it via your test
> code on scsi device, but a bit difficult to get it on null_blk.
>
> Turns out it is a block core issue, and it is in bio_will_gap() which gets
> the last bvec via 'bi_io_vec[prev->bi_vcnt - 1]' directly.  I have posted
> out one patchset for fixing the issue:
>
> http://marc.info/?l=linux-kernel&m=145551975429092&w=2

Thanks Ming!

I'll give it a shot...


From keith.busch at intel.com  Tue Feb 16 10:40:47 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 16 Feb 2016 18:40:47 +0000
Subject: [PATCH 1/2] NVMe: Lock out shutdown during pci init
In-Reply-To: <20160213095851.GA15206@infradead.org>
References: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
 <20160213095851.GA15206@infradead.org>
Message-ID: <20160216184046.GA8415@localhost.localdomain>

On Sat, Feb 13, 2016 at 01:58:51AM -0800, Christoph Hellwig wrote:
> Seems like we should be holding the lock over the call
> to nvme_dev_disable to get protection for the whole execution,
> e.g. by adding a __nvme_dev_disable variant that expects the lock
> to be held.
> 
> Maybe it's also worth renaming shutdown_lock to register_mutex
> or similar to better document what it protects.

Perhaps we shouldn't unmap the registers on a reset. That'd fix several
problems, including this one and writing queue doorbells.


From keith.busch at intel.com  Tue Feb 16 13:57:47 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 16 Feb 2016 21:57:47 +0000
Subject: [PATCHv2-4.5 08/10] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <20160213094634.GB15318@infradead.org>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <1455221147-24228-9-git-send-email-keith.busch@intel.com>
 <20160213094634.GB15318@infradead.org>
Message-ID: <20160216215746.GB8415@localhost.localdomain>

On Sat, Feb 13, 2016 at 01:46:34AM -0800, Christoph Hellwig wrote:
> On Thu, Feb 11, 2016 at 01:05:45PM -0700, Keith Busch wrote:
> > +/**
> > + * nvme_kill_ns_queues(): Ends all namespace queues
> > + * @ctrl: the dead controller that needs to end
> > + *
> > + * Call this function when the driver determines it is unable to get the
> > + * controller in a state capable of servicing IO.
> > + */
> > +void nvme_kill_ns_queues(struct nvme_ctrl *ctrl)
> 
> Why do we have a separate function for this instead of driving this
> from nvme_remove_namespaces?

nvme_remove_namespaces attempts to remove namespaces assuming the
controller is still functional. For example, user pushed the "Attention"
removal button and waits for the acknowledgement.

This new API assumes the controller is not functional and never will be.

If your question is why nvme_remove_namespaces doesn't handle both cases,
it's because the controller could break/link-down only after calling
del_gendisk, at which point its too late for the driver to kill the
queue in that path.


From hch at infradead.org  Tue Feb 16 23:59:13 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Tue, 16 Feb 2016 23:59:13 -0800
Subject: [PATCH 1/2] NVMe: Lock out shutdown during pci init
In-Reply-To: <20160216184046.GA8415@localhost.localdomain>
References: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
 <20160213095851.GA15206@infradead.org>
 <20160216184046.GA8415@localhost.localdomain>
Message-ID: <20160217075913.GA14303@infradead.org>

On Tue, Feb 16, 2016 at 06:40:47PM +0000, Keith Busch wrote:
> On Sat, Feb 13, 2016 at 01:58:51AM -0800, Christoph Hellwig wrote:
> > Seems like we should be holding the lock over the call
> > to nvme_dev_disable to get protection for the whole execution,
> > e.g. by adding a __nvme_dev_disable variant that expects the lock
> > to be held.
> > 
> > Maybe it's also worth renaming shutdown_lock to register_mutex
> > or similar to better document what it protects.
> 
> Perhaps we shouldn't unmap the registers on a reset. That'd fix several
> problems, including this one and writing queue doorbells.

I always thought there was a deep reason we'd need to unmap it.  But
if there isn't getting rid of the unmap should make the driver a lot
more robust.


From hch at infradead.org  Wed Feb 17 00:06:01 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 17 Feb 2016 00:06:01 -0800
Subject: [PATCHv2-4.5 08/10] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <20160216215746.GB8415@localhost.localdomain>
References: <1455221147-24228-1-git-send-email-keith.busch@intel.com>
 <1455221147-24228-9-git-send-email-keith.busch@intel.com>
 <20160213094634.GB15318@infradead.org>
 <20160216215746.GB8415@localhost.localdomain>
Message-ID: <20160217080601.GB14303@infradead.org>

On Tue, Feb 16, 2016 at 09:57:47PM +0000, Keith Busch wrote:
> > 
> > Why do we have a separate function for this instead of driving this
> > from nvme_remove_namespaces?
> 
> nvme_remove_namespaces attempts to remove namespaces assuming the
> controller is still functional. For example, user pushed the "Attention"
> removal button and waits for the acknowledgement.

But we'll always call it before removing a controller anyway.

> This new API assumes the controller is not functional and never will be.
> 
> If your question is why nvme_remove_namespaces doesn't handle both cases,
> it's because the controller could break/link-down only after calling
> del_gendisk, at which point its too late for the driver to kill the
> queue in that path.

My idea was more to have a kill flag for nvme_remove_namespaces given
that we basically always follow up with a nvme_remove_namespaces when
calling nvme_kill_ns_queues.  But looking at the code this might
actually end up more convoluted, so I'll take my comment back.


From keith.busch at intel.com  Wed Feb 17 05:40:58 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 13:40:58 +0000
Subject: [PATCH 1/2] NVMe: Lock out shutdown during pci init
In-Reply-To: <20160217075913.GA14303@infradead.org>
References: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
 <20160213095851.GA15206@infradead.org>
 <20160216184046.GA8415@localhost.localdomain>
 <20160217075913.GA14303@infradead.org>
Message-ID: <20160217134057.GA30059@localhost.localdomain>

On Tue, Feb 16, 2016 at 11:59:13PM -0800, Christoph Hellwig wrote:
> I always thought there was a deep reason we'd need to unmap it.  But
> if there isn't getting rid of the unmap should make the driver a lot
> more robust.

Remapping after each reset was prepping for a non-existent feature.
We'd like to create domains safe for the PCI core to reassign bus numbers
and memory windows on a live system for various hot plug scenarios,
but I don't see that happening soon.


From hch at lst.de  Wed Feb 17 10:03:12 2016
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 17 Feb 2016 19:03:12 +0100
Subject: [PATCH 1/3] nvme: use a work item to submit async event requests
In-Reply-To: <1455732194-30790-1-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
Message-ID: <1455732194-30790-2-git-send-email-hch@lst.de>

Use a dedicated work item to submit async event requests instead of the
global kthread.  This simplifies the code and reduces the latencies to
resubmit a request once an even notification happened.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index fec7479..21b0be4 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -100,6 +100,7 @@ struct nvme_dev {
 	struct work_struct reset_work;
 	struct work_struct scan_work;
 	struct work_struct remove_work;
+	struct work_struct async_work;
 	struct mutex shutdown_lock;
 	bool subsystem;
 	void __iomem *cmb;
@@ -281,8 +282,11 @@ static void nvme_complete_async_event(struct nvme_dev *dev,
 	u16 status = le16_to_cpu(cqe->status) >> 1;
 	u32 result = le32_to_cpu(cqe->result);
 
-	if (status == NVME_SC_SUCCESS || status == NVME_SC_ABORT_REQ)
+	if (status == NVME_SC_SUCCESS || status == NVME_SC_ABORT_REQ) {
 		++dev->ctrl.event_limit;
+		queue_work(nvme_workq, &dev->async_work);
+	}
+
 	if (status != NVME_SC_SUCCESS)
 		return;
 
@@ -816,15 +820,22 @@ static int nvme_poll(struct blk_mq_hw_ctx *hctx, unsigned int tag)
 	return 0;
 }
 
-static void nvme_submit_async_event(struct nvme_dev *dev)
+static void nvme_async_event_work(struct work_struct *work)
 {
+	struct nvme_dev *dev = container_of(work, struct nvme_dev, async_work);
+	struct nvme_queue *nvmeq = dev->queues[0];
 	struct nvme_command c;
 
 	memset(&c, 0, sizeof(c));
 	c.common.opcode = nvme_admin_async_event;
-	c.common.command_id = NVME_AQ_BLKMQ_DEPTH + --dev->ctrl.event_limit;
 
-	__nvme_submit_cmd(dev->queues[0], &c);
+	spin_lock_irq(&nvmeq->q_lock);
+	while (dev->ctrl.event_limit > 0) {
+		c.common.command_id = NVME_AQ_BLKMQ_DEPTH +
+			--dev->ctrl.event_limit;
+		__nvme_submit_cmd(nvmeq, &c);
+	}
+	spin_unlock_irq(&nvmeq->q_lock);
 }
 
 static int adapter_delete_queue(struct nvme_dev *dev, u8 opcode, u16 id)
@@ -1358,9 +1369,6 @@ static int nvme_kthread(void *data)
 					continue;
 				spin_lock_irq(&nvmeq->q_lock);
 				nvme_process_cq(nvmeq);
-
-				while (i == 0 && dev->ctrl.event_limit > 0)
-					nvme_submit_async_event(dev);
 				spin_unlock_irq(&nvmeq->q_lock);
 			}
 		}
@@ -1929,6 +1937,7 @@ static void nvme_reset_work(struct work_struct *work)
 		goto free_tags;
 
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
+	queue_work(nvme_workq, &dev->async_work);
 
 	result = nvme_dev_list_add(dev);
 	if (result)
@@ -2062,6 +2071,7 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	INIT_WORK(&dev->scan_work, nvme_dev_scan);
 	INIT_WORK(&dev->reset_work, nvme_reset_work);
 	INIT_WORK(&dev->remove_work, nvme_remove_dead_ctrl_work);
+	INIT_WORK(&dev->async_work, nvme_async_event_work);
 	mutex_init(&dev->shutdown_lock);
 	init_completion(&dev->ioq_wait);
 
@@ -2115,6 +2125,7 @@ static void nvme_remove(struct pci_dev *pdev)
 	spin_unlock(&dev_list_lock);
 
 	pci_set_drvdata(pdev, NULL);
+	flush_work(&dev->async_work);
 	flush_work(&dev->reset_work);
 	flush_work(&dev->scan_work);
 	nvme_remove_namespaces(&dev->ctrl);
-- 
2.1.4



From hch at lst.de  Wed Feb 17 10:03:13 2016
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 17 Feb 2016 19:03:13 +0100
Subject: [PATCH 2/3] nvme: don't poll the CQ from the kthread
In-Reply-To: <1455732194-30790-1-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
Message-ID: <1455732194-30790-3-git-send-email-hch@lst.de>

There is no reason to do unconditional polling of CQs per the NVMe
spec.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 12 ------------
 1 file changed, 12 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 21b0be4..10839f7 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1156,9 +1156,6 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, int qid,
 	nvmeq->qid = qid;
 	nvmeq->cq_vector = -1;
 	dev->queues[qid] = nvmeq;
-
-	/* make sure queue descriptor is set before queue count, for kthread */
-	mb();
 	dev->queue_count++;
 
 	return nvmeq;
@@ -1345,7 +1342,6 @@ static int nvme_kthread(void *data)
 		set_current_state(TASK_INTERRUPTIBLE);
 		spin_lock(&dev_list_lock);
 		list_for_each_entry_safe(dev, next, &dev_list, node) {
-			int i;
 			u32 csts = readl(dev->bar + NVME_REG_CSTS);
 
 			/*
@@ -1363,14 +1359,6 @@ static int nvme_kthread(void *data)
 				}
 				continue;
 			}
-			for (i = 0; i < dev->queue_count; i++) {
-				struct nvme_queue *nvmeq = dev->queues[i];
-				if (!nvmeq)
-					continue;
-				spin_lock_irq(&nvmeq->q_lock);
-				nvme_process_cq(nvmeq);
-				spin_unlock_irq(&nvmeq->q_lock);
-			}
 		}
 		spin_unlock(&dev_list_lock);
 		schedule_timeout(round_jiffies_relative(HZ));
-- 
2.1.4



From hch at lst.de  Wed Feb 17 10:03:11 2016
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 17 Feb 2016 19:03:11 +0100
Subject: nvme: remove the global device list and kthread
Message-ID: <1455732194-30790-1-git-send-email-hch@lst.de>

This series drops the global nvme thread, and the device list only
needed for it.  Note that I haven't been able to trigger an AER with
my test controller, but the code is based on work for the NVMe over
Fabrics driver that has been properly tested.



From hch at lst.de  Wed Feb 17 10:03:14 2016
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 17 Feb 2016 19:03:14 +0100
Subject: [PATCH 3/3] nvme: replace the kthread with a per-device watchdog timer
In-Reply-To: <1455732194-30790-1-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
Message-ID: <1455732194-30790-4-git-send-email-hch@lst.de>

The only work left in the kthread is the periodic health check for each
controller.  There is no need to run this from process context or keep
a thread context around for it, so replace it with a simpler timer.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 112 ++++++++++--------------------------------------
 1 file changed, 23 insertions(+), 89 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 10839f7..a623360 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -27,7 +27,6 @@
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/kdev_t.h>
-#include <linux/kthread.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/module.h>
@@ -39,6 +38,7 @@
 #include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/t10-pi.h>
+#include <linux/timer.h>
 #include <linux/types.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <asm/unaligned.h>
@@ -64,11 +64,7 @@ static bool use_cmb_sqes = true;
 module_param(use_cmb_sqes, bool, 0644);
 MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
 
-static LIST_HEAD(dev_list);
-static DEFINE_SPINLOCK(dev_list_lock);
-static struct task_struct *nvme_thread;
 static struct workqueue_struct *nvme_workq;
-static wait_queue_head_t nvme_kthread_wait;
 
 struct nvme_dev;
 struct nvme_queue;
@@ -82,7 +78,6 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown);
  * Represents an NVM Express device.  Each nvme_dev is a PCI function.
  */
 struct nvme_dev {
-	struct list_head node;
 	struct nvme_queue **queues;
 	struct blk_mq_tag_set tagset;
 	struct blk_mq_tag_set admin_tagset;
@@ -101,6 +96,7 @@ struct nvme_dev {
 	struct work_struct scan_work;
 	struct work_struct remove_work;
 	struct work_struct async_work;
+	struct timer_list watchdog_timer;
 	struct mutex shutdown_lock;
 	bool subsystem;
 	void __iomem *cmb;
@@ -1334,36 +1330,26 @@ static int nvme_configure_admin_queue(struct nvme_dev *dev)
 	return result;
 }
 
-static int nvme_kthread(void *data)
+static void nvme_watchdog_timer(unsigned long data)
 {
-	struct nvme_dev *dev, *next;
+	struct nvme_dev *dev = (struct nvme_dev *)data;
+	u32 csts = readl(dev->bar + NVME_REG_CSTS);
 
-	while (!kthread_should_stop()) {
-		set_current_state(TASK_INTERRUPTIBLE);
-		spin_lock(&dev_list_lock);
-		list_for_each_entry_safe(dev, next, &dev_list, node) {
-			u32 csts = readl(dev->bar + NVME_REG_CSTS);
-
-			/*
-			 * Skip controllers currently under reset.
-			 */
-			if (work_pending(&dev->reset_work) || work_busy(&dev->reset_work))
-				continue;
-
-			if ((dev->subsystem && (csts & NVME_CSTS_NSSRO)) ||
-							csts & NVME_CSTS_CFS) {
-				if (queue_work(nvme_workq, &dev->reset_work)) {
-					dev_warn(dev->ctrl.device,
-						"Failed status: %x, reset controller\n",
-						readl(dev->bar + NVME_REG_CSTS));
-				}
-				continue;
-			}
+	/*
+	 * Skip controllers currently under reset.
+	 */
+	if (!work_pending(&dev->reset_work) && !work_busy(&dev->reset_work) &&
+	    ((csts & NVME_CSTS_CFS) ||
+	     (dev->subsystem && (csts & NVME_CSTS_NSSRO)))) {
+		if (queue_work(nvme_workq, &dev->reset_work)) {
+			dev_warn(dev->dev,
+				"Failed status: 0x%x, reset controller.\n",
+				csts);
 		}
-		spin_unlock(&dev_list_lock);
-		schedule_timeout(round_jiffies_relative(HZ));
+		return;
 	}
-	return 0;
+
+	mod_timer(&dev->watchdog_timer, round_jiffies(jiffies + HZ));
 }
 
 static int nvme_create_io_queues(struct nvme_dev *dev)
@@ -1777,56 +1763,12 @@ static void nvme_dev_unmap(struct nvme_dev *dev)
 	}
 }
 
-static int nvme_dev_list_add(struct nvme_dev *dev)
-{
-	bool start_thread = false;
-
-	spin_lock(&dev_list_lock);
-	if (list_empty(&dev_list) && IS_ERR_OR_NULL(nvme_thread)) {
-		start_thread = true;
-		nvme_thread = NULL;
-	}
-	list_add(&dev->node, &dev_list);
-	spin_unlock(&dev_list_lock);
-
-	if (start_thread) {
-		nvme_thread = kthread_run(nvme_kthread, NULL, "nvme");
-		wake_up_all(&nvme_kthread_wait);
-	} else
-		wait_event_killable(nvme_kthread_wait, nvme_thread);
-
-	if (IS_ERR_OR_NULL(nvme_thread))
-		return nvme_thread ? PTR_ERR(nvme_thread) : -EINTR;
-
-	return 0;
-}
-
-/*
-* Remove the node from the device list and check
-* for whether or not we need to stop the nvme_thread.
-*/
-static void nvme_dev_list_remove(struct nvme_dev *dev)
-{
-	struct task_struct *tmp = NULL;
-
-	spin_lock(&dev_list_lock);
-	list_del_init(&dev->node);
-	if (list_empty(&dev_list) && !IS_ERR_OR_NULL(nvme_thread)) {
-		tmp = nvme_thread;
-		nvme_thread = NULL;
-	}
-	spin_unlock(&dev_list_lock);
-
-	if (tmp)
-		kthread_stop(tmp);
-}
-
 static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 {
 	int i;
 	u32 csts = -1;
 
-	nvme_dev_list_remove(dev);
+	del_timer_sync(&dev->watchdog_timer);
 
 	mutex_lock(&dev->shutdown_lock);
 	if (dev->bar) {
@@ -1927,9 +1869,7 @@ static void nvme_reset_work(struct work_struct *work)
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
 	queue_work(nvme_workq, &dev->async_work);
 
-	result = nvme_dev_list_add(dev);
-	if (result)
-		goto remove;
+	mod_timer(&dev->watchdog_timer, round_jiffies(jiffies + HZ));
 
 	/*
 	 * Keep the controller around but remove all namespaces if we don't have
@@ -1946,8 +1886,6 @@ static void nvme_reset_work(struct work_struct *work)
 	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
 	return;
 
- remove:
-	nvme_dev_list_remove(dev);
  free_tags:
 	nvme_dev_remove_admin(dev);
 	blk_put_queue(dev->ctrl.admin_q);
@@ -2055,11 +1993,12 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	dev->dev = get_device(&pdev->dev);
 	pci_set_drvdata(pdev, dev);
 
-	INIT_LIST_HEAD(&dev->node);
 	INIT_WORK(&dev->scan_work, nvme_dev_scan);
 	INIT_WORK(&dev->reset_work, nvme_reset_work);
 	INIT_WORK(&dev->remove_work, nvme_remove_dead_ctrl_work);
 	INIT_WORK(&dev->async_work, nvme_async_event_work);
+	setup_timer(&dev->watchdog_timer, nvme_watchdog_timer,
+		(unsigned long)dev);
 	mutex_init(&dev->shutdown_lock);
 	init_completion(&dev->ioq_wait);
 
@@ -2108,9 +2047,7 @@ static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
 
-	spin_lock(&dev_list_lock);
-	list_del_init(&dev->node);
-	spin_unlock(&dev_list_lock);
+	del_timer_sync(&dev->watchdog_timer);
 
 	pci_set_drvdata(pdev, NULL);
 	flush_work(&dev->async_work);
@@ -2223,8 +2160,6 @@ static int __init nvme_init(void)
 {
 	int result;
 
-	init_waitqueue_head(&nvme_kthread_wait);
-
 	nvme_workq = alloc_workqueue("nvme", WQ_UNBOUND | WQ_MEM_RECLAIM, 0);
 	if (!nvme_workq)
 		return -ENOMEM;
@@ -2239,7 +2174,6 @@ static void __exit nvme_exit(void)
 {
 	pci_unregister_driver(&nvme_driver);
 	destroy_workqueue(nvme_workq);
-	BUG_ON(nvme_thread && !IS_ERR(nvme_thread));
 	_nvme_check_size();
 }
 
-- 
2.1.4



From keith.busch at intel.com  Wed Feb 17 10:36:52 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 11:36:52 -0700
Subject: [PATCH-4.5-v3 2/5] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455734215-20396-3-git-send-email-keith.busch@intel.com>

This patch makes nvme namespace removal lockless. It is up to the caller
to ensure no active namespace scanning is occuring.

The lockless removal allows the driver to cleanup a namespace
request_queue if the controller fails during removal. Previously this
could deadlock trying to acquire the namespace mutex in order to handle
such events.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 15 +++++++++------
 drivers/nvme/host/nvme.h |  4 ++++
 2 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 3cd921e..1b7387d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -48,6 +48,10 @@ static void nvme_free_ns(struct kref *kref)
 {
 	struct nvme_ns *ns = container_of(kref, struct nvme_ns, kref);
 
+	mutex_lock(&ns->ctrl->namespaces_mutex);
+	list_del_init(&ns->list);
+	mutex_unlock(&ns->ctrl->namespaces_mutex);
+
 	if (ns->type == NVME_NS_LIGHTNVM)
 		nvme_nvm_unregister(ns->queue, ns->disk->disk_name);
 
@@ -1179,11 +1183,13 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)
 {
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
+	bool kill;
 
-	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
+	if (test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
+		return;
 
+	kill = nvme_io_incapable(ns->ctrl) &&
+			!blk_queue_dying(ns->queue);
 	if (kill) {
 		blk_set_queue_dying(ns->queue);
 
@@ -1206,7 +1212,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
-	list_del_init(&ns->list);
 	nvme_put_ns(ns);
 }
 
@@ -1300,10 +1305,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 {
 	struct nvme_ns *ns, *next;
 
-	mutex_lock(&ctrl->namespaces_mutex);
 	list_for_each_entry_safe(ns, next, &ctrl->namespaces, list)
 		nvme_ns_remove(ns);
-	mutex_unlock(&ctrl->namespaces_mutex);
 }
 
 static DEFINE_IDA(nvme_instance_ida);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9664d07..db413a7 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -112,6 +112,10 @@ struct nvme_ns {
 	bool ext;
 	u8 pi_type;
 	int type;
+	unsigned long flags;
+
+#define NVME_NS_REMOVING 0
+
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
 };
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 17 10:36:53 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 11:36:53 -0700
Subject: [PATCH-4.5-v3 3/5] NVMe: Simplify device reset failure
In-Reply-To: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455734215-20396-4-git-send-email-keith.busch@intel.com>

A reset failure schedules the device to unbind from the driver through
the pci driver's remove. This cleans up all intialization, so there is
no need to duplicate the potentially racy cleanup.

To help understand why a reset failed, the status is logged with the
existing warning message.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 48 +++++++++++++++++++++---------------------------
 1 file changed, 21 insertions(+), 27 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index f0e97a3..9e050dd 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -86,7 +86,6 @@ struct nvme_queue;
 
 static int nvme_reset(struct nvme_dev *dev);
 static void nvme_process_cq(struct nvme_queue *nvmeq);
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev);
 static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown);
 
 /*
@@ -1902,10 +1901,19 @@ static void nvme_pci_free_ctrl(struct nvme_ctrl *ctrl)
 	kfree(dev);
 }
 
+static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
+{
+	dev_warn(dev->dev, "Removing after probe failure status: %d\n", status);
+
+	kref_get(&dev->ctrl.kref);
+	if (!schedule_work(&dev->remove_work))
+		nvme_put_ctrl(&dev->ctrl);
+}
+
 static void nvme_reset_work(struct work_struct *work)
 {
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, reset_work);
-	int result;
+	int result = -ENODEV;
 
 	if (WARN_ON(test_bit(NVME_CTRL_RESETTING, &dev->flags)))
 		goto out;
@@ -1925,26 +1933,26 @@ static void nvme_reset_work(struct work_struct *work)
 
 	result = nvme_configure_admin_queue(dev);
 	if (result)
-		goto unmap;
+		goto out;
 
 	nvme_init_queue(dev->queues[0], 0);
 	result = nvme_alloc_admin_tags(dev);
 	if (result)
-		goto disable;
+		goto out;
 
 	result = nvme_init_identify(&dev->ctrl);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	result = nvme_setup_io_queues(dev);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
 
 	result = nvme_dev_list_add(dev);
 	if (result)
-		goto remove;
+		goto out;
 
 	/*
 	 * Keep the controller around but remove all namespaces if we don't have
@@ -1961,19 +1969,8 @@ static void nvme_reset_work(struct work_struct *work)
 	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
 	return;
 
- remove:
-	nvme_dev_list_remove(dev);
- free_tags:
-	nvme_dev_remove_admin(dev);
-	blk_put_queue(dev->ctrl.admin_q);
-	dev->ctrl.admin_q = NULL;
-	dev->queues[0]->tags = NULL;
- disable:
-	nvme_disable_admin_queue(dev, false);
- unmap:
-	nvme_dev_unmap(dev);
  out:
-	nvme_remove_dead_ctrl(dev);
+	nvme_remove_dead_ctrl(dev, result);
 }
 
 static void nvme_remove_dead_ctrl_work(struct work_struct *work)
@@ -1986,14 +1983,6 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	nvme_put_ctrl(&dev->ctrl);
 }
 
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev)
-{
-	dev_warn(dev->dev, "Removing after probe failure\n");
-	kref_get(&dev->ctrl.kref);
-	if (!schedule_work(&dev->remove_work))
-		nvme_put_ctrl(&dev->ctrl);
-}
-
 static int nvme_reset(struct nvme_dev *dev)
 {
 	if (!dev->ctrl.admin_q || blk_queue_dying(dev->ctrl.admin_q))
@@ -2115,6 +2104,11 @@ static void nvme_shutdown(struct pci_dev *pdev)
 	nvme_dev_disable(dev, true);
 }
 
+/*
+ * The driver's remove may be called on a device in a partially initialized
+ * state. This function must not have any dependencies on the device state in
+ * order to proceed.
+ */
 static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 17 10:36:55 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 11:36:55 -0700
Subject: [PATCH-4.5-v3 5/5] NVMe: Don't unmap controller registers on reset
In-Reply-To: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455734215-20396-6-git-send-email-keith.busch@intel.com>

Unmapping the registers on reset or shutdown is not necessary. Keeping
the mapping simplifies reset handling.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 71 +++++++++++++++++++++++++++++--------------------
 1 file changed, 42 insertions(+), 29 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 27042c0..f11daba 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1699,10 +1699,10 @@ static int nvme_dev_add(struct nvme_dev *dev)
 	return 0;
 }
 
-static int nvme_dev_map(struct nvme_dev *dev)
+static int nvme_pci_enable(struct nvme_dev *dev)
 {
 	u64 cap;
-	int bars, result = -ENOMEM;
+	int result = -ENOMEM;
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
 	if (pci_enable_device_mem(pdev))
@@ -1710,24 +1710,14 @@ static int nvme_dev_map(struct nvme_dev *dev)
 
 	dev->entry[0].vector = pdev->irq;
 	pci_set_master(pdev);
-	bars = pci_select_bars(pdev, IORESOURCE_MEM);
-	if (!bars)
-		goto disable_pci;
-
-	if (pci_request_selected_regions(pdev, bars, "nvme"))
-		goto disable_pci;
 
 	if (dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(64)) &&
 	    dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(32)))
 		goto disable;
 
-	dev->bar = ioremap(pci_resource_start(pdev, 0), 8192);
-	if (!dev->bar)
-		goto disable;
-
 	if (readl(dev->bar + NVME_REG_CSTS) == -1) {
 		result = -ENODEV;
-		goto unmap;
+		goto disable;
 	}
 
 	/*
@@ -1737,7 +1727,7 @@ static int nvme_dev_map(struct nvme_dev *dev)
 	if (!pdev->irq) {
 		result = pci_enable_msix(pdev, dev->entry, 1);
 		if (result < 0)
-			goto unmap;
+			goto disable;
 	}
 
 	cap = lo_hi_readq(dev->bar + NVME_REG_CAP);
@@ -1764,18 +1754,20 @@ static int nvme_dev_map(struct nvme_dev *dev)
 	pci_save_state(pdev);
 	return 0;
 
- unmap:
-	iounmap(dev->bar);
-	dev->bar = NULL;
  disable:
-	pci_release_regions(pdev);
- disable_pci:
 	pci_disable_device(pdev);
 	return result;
 }
 
 static void nvme_dev_unmap(struct nvme_dev *dev)
 {
+	if (dev->bar)
+		iounmap(dev->bar);
+	pci_release_regions(to_pci_dev(dev->dev));
+}
+
+static void nvme_pci_disable(struct nvme_dev *dev)
+{
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
 	if (pdev->msi_enabled)
@@ -1783,12 +1775,6 @@ static void nvme_dev_unmap(struct nvme_dev *dev)
 	else if (pdev->msix_enabled)
 		pci_disable_msix(pdev);
 
-	if (dev->bar) {
-		iounmap(dev->bar);
-		dev->bar = NULL;
-		pci_release_regions(pdev);
-	}
-
 	if (pci_is_enabled(pdev)) {
 		pci_disable_pcie_error_reporting(pdev);
 		pci_disable_device(pdev);
@@ -1847,7 +1833,7 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 	nvme_dev_list_remove(dev);
 
 	mutex_lock(&dev->shutdown_lock);
-	if (dev->bar) {
+	if (pci_is_enabled(to_pci_dev(dev->dev))) {
 		nvme_stop_queues(&dev->ctrl);
 		csts = readl(dev->bar + NVME_REG_CSTS);
 	}
@@ -1860,7 +1846,7 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 		nvme_disable_io_queues(dev);
 		nvme_disable_admin_queue(dev, shutdown);
 	}
-	nvme_dev_unmap(dev);
+	nvme_pci_disable(dev);
 
 	for (i = dev->queue_count - 1; i >= 0; i--)
 		nvme_clear_queue(dev->queues[i]);
@@ -1926,12 +1912,12 @@ static void nvme_reset_work(struct work_struct *work)
 	 * If we're called to reset a live controller first shut it down before
 	 * moving on.
 	 */
-	if (dev->bar)
+	if (dev->ctrl.ctrl_config & NVME_CC_ENABLE)
 		nvme_dev_disable(dev, false);
 
 	set_bit(NVME_CTRL_RESETTING, &dev->flags);
 
-	result = nvme_dev_map(dev);
+	result = nvme_pci_enable(dev);
 	if (result)
 		goto out;
 
@@ -2039,6 +2025,27 @@ static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = {
 	.free_ctrl		= nvme_pci_free_ctrl,
 };
 
+static int nvme_dev_map(struct nvme_dev *dev)
+{
+	int bars;
+	struct pci_dev *pdev = to_pci_dev(dev->dev);
+
+	bars = pci_select_bars(pdev, IORESOURCE_MEM);
+	if (!bars)
+		return -ENODEV;
+	if (pci_request_selected_regions(pdev, bars, "nvme"))
+		return -ENODEV;
+
+	dev->bar = ioremap(pci_resource_start(pdev, 0), 8192);
+	if (!dev->bar)
+		goto release;
+
+       return 0;
+  release:
+       pci_release_regions(pdev);
+       return -ENODEV;
+}
+
 static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
 	int node, result = -ENOMEM;
@@ -2063,6 +2070,10 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	dev->dev = get_device(&pdev->dev);
 	pci_set_drvdata(pdev, dev);
 
+	result = nvme_dev_map(dev);
+	if (result)
+		goto free;
+
 	INIT_LIST_HEAD(&dev->node);
 	INIT_WORK(&dev->scan_work, nvme_dev_scan);
 	INIT_WORK(&dev->reset_work, nvme_reset_work);
@@ -2086,6 +2097,7 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	nvme_release_prp_pools(dev);
  put_pci:
 	put_device(dev->dev);
+	nvme_dev_unmap(dev);
  free:
 	kfree(dev->queues);
 	kfree(dev->entry);
@@ -2129,6 +2141,7 @@ static void nvme_remove(struct pci_dev *pdev)
 	nvme_free_queues(dev, 0);
 	nvme_release_cmb(dev);
 	nvme_release_prp_pools(dev);
+	nvme_dev_unmap(dev);
 	nvme_put_ctrl(&dev->ctrl);
 }
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 17 10:36:54 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 11:36:54 -0700
Subject: [PATCH-4.5-v3 4/5] NVMe: Move error handling to failed reset handler
In-Reply-To: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455734215-20396-5-git-send-email-keith.busch@intel.com>

This moves failed queue handling out of the namespace removal path and
into the reset failure path, fixing a hanging condition if the controller
fails or link down during del_gendisk. Previously the driver had to see
the controller as degraded prior to calling del_gendisk to setup the
queues to fail. If the controller happened to fail after this though,
there was no task to end the request_queue.

On failure, all namespace states are set to dead. This has capacity
revalidate to 0, and ends all new requests with error status.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 53 ++++++++++++++++++++++++++++++++----------------
 drivers/nvme/host/nvme.h |  2 ++
 drivers/nvme/host/pci.c  |  7 ++++++-
 3 files changed, 44 insertions(+), 18 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1b7387d..face31f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -560,6 +560,10 @@ static int nvme_revalidate_disk(struct gendisk *disk)
 	u16 old_ms;
 	unsigned short bs;
 
+	if (test_bit(NVME_NS_DEAD, &ns->flags)) {
+		set_capacity(disk, 0);
+		return -ENODEV;
+	}
 	if (nvme_identify_ns(ns->ctrl, ns->ns_id, &id)) {
 		dev_warn(ns->ctrl->dev, "%s: Identify failure nvme%dn%d\n",
 				__func__, ns->ctrl->instance, ns->ns_id);
@@ -1183,32 +1187,15 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)
 {
-	bool kill;
-
 	if (test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
 		return;
 
-	kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
 	if (ns->disk->flags & GENHD_FL_UP) {
 		if (blk_get_integrity(ns->disk))
 			blk_integrity_unregister(ns->disk);
 		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
 					&nvme_ns_attr_group);
 		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
@@ -1405,6 +1392,38 @@ out:
 	return ret;
 }
 
+/**
+ * nvme_kill_queues(): Ends all namespace queues
+ * @ctrl: the dead controller that needs to end
+ *
+ * Call this function when the driver determines it is unable to get the
+ * controller in a state capable of servicing IO.
+ */
+void nvme_kill_queues(struct nvme_ctrl *ctrl)
+{
+	struct nvme_ns *ns;
+
+	mutex_lock(&ctrl->namespaces_mutex);
+	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (!kref_get_unless_zero(&ns->kref))
+			continue;
+
+		/*
+		 * Revalidating a dead namespace sets capacity to 0. This will
+		 * end buffered writers dirtying pages that can't be synced.
+		 */
+		if (!test_and_set_bit(NVME_NS_DEAD, &ns->flags))
+			revalidate_disk(ns->disk);
+
+		blk_set_queue_dying(ns->queue);
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_mq_start_stopped_hw_queues(ns->queue, true);
+
+		nvme_put_ns(ns);
+	}
+	mutex_unlock(&ctrl->namespaces_mutex);
+}
+
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
 	struct nvme_ns *ns;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index db413a7..dcdc2d7 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -115,6 +115,7 @@ struct nvme_ns {
 	unsigned long flags;
 
 #define NVME_NS_REMOVING 0
+#define NVME_NS_DEAD     1
 
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
@@ -244,6 +245,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl);
 void nvme_start_queues(struct nvme_ctrl *ctrl);
+void nvme_kill_queues(struct nvme_ctrl *ctrl);
 
 struct request *nvme_alloc_request(struct request_queue *q,
 		struct nvme_command *cmd, unsigned int flags);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 9e050dd..27042c0 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -679,7 +679,10 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 
 	spin_lock_irq(&nvmeq->q_lock);
 	if (unlikely(nvmeq->cq_vector < 0)) {
-		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		if (ns && test_bit(NVME_NS_DEAD, &ns->flags))
+			ret = BLK_MQ_RQ_QUEUE_ERROR;
+		else
+			ret = BLK_MQ_RQ_QUEUE_BUSY;
 		spin_unlock_irq(&nvmeq->q_lock);
 		goto out;
 	}
@@ -1906,6 +1909,7 @@ static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
 	dev_warn(dev->dev, "Removing after probe failure status: %d\n", status);
 
 	kref_get(&dev->ctrl.kref);
+	nvme_dev_disable(dev, false);
 	if (!schedule_work(&dev->remove_work))
 		nvme_put_ctrl(&dev->ctrl);
 }
@@ -1978,6 +1982,7 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, remove_work);
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
+	nvme_kill_queues(&dev->ctrl);
 	if (pci_get_drvdata(pdev))
 		pci_stop_and_remove_bus_device_locked(pdev);
 	nvme_put_ctrl(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 17 10:36:51 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 11:36:51 -0700
Subject: [PATCH-4.5-v3 1/5] NVMe: Add controller removing state
In-Reply-To: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455734215-20396-2-git-send-email-keith.busch@intel.com>

The patch is preparing for reducing the scope of the namespace mutex. It
adds a removing state the namespace scan work uses to know if it is safe
to proceed. The scan work is flushed after setting the device state so
the removal path can safely walk the namespace list without locking it.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/pci.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index a128672..f0e97a3 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -120,6 +120,7 @@ struct nvme_dev {
 	unsigned long flags;
 
 #define NVME_CTRL_RESETTING    0
+#define NVME_CTRL_REMOVING     1
 
 	struct nvme_ctrl ctrl;
 	struct completion ioq_wait;
@@ -1583,6 +1584,8 @@ static void nvme_dev_scan(struct work_struct *work)
 {
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, scan_work);
 
+	if (test_bit(NVME_CTRL_REMOVING, &dev->flags))
+		return;
 	if (!dev->tagset.tags)
 		return;
 	nvme_scan_namespaces(&dev->ctrl);
@@ -2116,6 +2119,7 @@ static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
 
+	set_bit(NVME_CTRL_REMOVING, &dev->flags);
 	pci_set_drvdata(pdev, NULL);
 	flush_work(&dev->scan_work);
 	nvme_remove_namespaces(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 17 10:36:50 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 11:36:50 -0700
Subject: [PATCH-4.5-v3 0/5] NVMe hot plug stuff
Message-ID: <1455734215-20396-1-git-send-email-keith.busch@intel.com>

This is based of the current linux-block/for-linus branch.

Only 4/5 is the same as from v2, and have retained the reviews. The rest
are new or different.

v2->v3:

  Used a different approach to the namespace removal deadlock by reducing
  the scope of the namespace mutex so removal doesn't hold it during
  IO. This has a dependency on the first patch to ensure a scan doesn't
  somehow trigger (however unlikely) during removal.

  Squashed v2's follow up fix for checking namespace state in nvme's queue_rq.

  The final patch doesn't unmap controller registers until removal. This
  makes it safe to run startup and shutdown concurrently. We could
  possibly later micro optimize the IO path since the queue doorbell is
  always mapped. The patch was tested with suspend/resume using rtcwake.

Keith Busch (5):
  NVMe: Add controller removing state
  NVMe: Fix namespace removal deadlock
  NVMe: Simplify device reset failure
  NVMe: Move error handling to failed reset handler
  NVMe: Don't unmap controller registers on reset

 drivers/nvme/host/core.c |  62 +++++++++++++++--------
 drivers/nvme/host/nvme.h |   6 +++
 drivers/nvme/host/pci.c  | 125 ++++++++++++++++++++++++++---------------------
 3 files changed, 118 insertions(+), 75 deletions(-)

-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 17 10:37:54 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 18:37:54 +0000
Subject: [PATCH 1/3] nvme: use a work item to submit async event requests
In-Reply-To: <1455732194-30790-2-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-2-git-send-email-hch@lst.de>
Message-ID: <20160217183754.GC30059@localhost.localdomain>

On Wed, Feb 17, 2016 at 07:03:12PM +0100, Christoph Hellwig wrote:
> Use a dedicated work item to submit async event requests instead of the
> global kthread.  This simplifies the code and reduces the latencies to
> resubmit a request once an even notification happened.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>

Looks good.

Reviewed-by: Keith Busch <keith.busch at intel.com>


From keith.busch at intel.com  Wed Feb 17 10:44:01 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 18:44:01 +0000
Subject: [PATCH 2/3] nvme: don't poll the CQ from the kthread
In-Reply-To: <1455732194-30790-3-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-3-git-send-email-hch@lst.de>
Message-ID: <20160217184400.GD30059@localhost.localdomain>

On Wed, Feb 17, 2016 at 07:03:13PM +0100, Christoph Hellwig wrote:
> There is no reason to do unconditional polling of CQs per the NVMe
> spec.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>

Looks good. Now we'll find out if anyone was unknowingly relying on
this. :)

The legacy IRQ used for the very first NVMe command after reset tripped
up some devices in the past, but the polling hid the breakage.

Reviewed-by: Keith Busch <keith.busch at intel.com>


From keith.busch at intel.com  Wed Feb 17 10:44:19 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 17 Feb 2016 18:44:19 +0000
Subject: [PATCH 3/3] nvme: replace the kthread with a per-device
 watchdog timer
In-Reply-To: <1455732194-30790-4-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-4-git-send-email-hch@lst.de>
Message-ID: <20160217184418.GE30059@localhost.localdomain>

On Wed, Feb 17, 2016 at 07:03:14PM +0100, Christoph Hellwig wrote:
> The only work left in the kthread is the periodic health check for each
> controller.  There is no need to run this from process context or keep
> a thread context around for it, so replace it with a simpler timer.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>

Looks good.

Reviewed-by: Keith Busch <keith.busch at intel.com>


From wenbo.wang at memblaze.com  Wed Feb 17 18:00:47 2016
From: wenbo.wang at memblaze.com (Wenbo Wang)
Date: Thu, 18 Feb 2016 02:00:47 +0000
Subject: [PATCH-4.5-v3 2/5] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455734215-20396-3-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-3-git-send-email-keith.busch@intel.com>
Message-ID: <BJXPR01MB1997CEFCF124760350CFEEDE0AF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>


> @@ -1300,10 +1305,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)  {
> 	struct nvme_ns *ns, *next;
> 
> -	mutex_lock(&ctrl->namespaces_mutex);
> 	list_for_each_entry_safe(ns, next, &ctrl->namespaces, list)
> 		nvme_ns_remove(ns);
>-	mutex_unlock(&ctrl->namespaces_mutex);
>}

Without this mutex, is it safe to run concurrently with nvme_free_ns which manipulates the list?

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com] 
Sent: Thursday, February 18, 2016 2:37 AM
To: linux-nvme at lists.infradead.org; Jens Axboe; Christoph Hellwig
Cc: Sagi Grimberg; Wenbo Wang; Johannes Thumshirn; Keith Busch
Subject: [PATCH-4.5-v3 2/5] NVMe: Fix namespace removal deadlock

This patch makes nvme namespace removal lockless. It is up to the caller to ensure no active namespace scanning is occuring.

The lockless removal allows the driver to cleanup a namespace request_queue if the controller fails during removal. Previously this could deadlock trying to acquire the namespace mutex in order to handle such events.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 15 +++++++++------  drivers/nvme/host/nvme.h |  4 ++++
 2 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 3cd921e..1b7387d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -48,6 +48,10 @@ static void nvme_free_ns(struct kref *kref)  {
 	struct nvme_ns *ns = container_of(kref, struct nvme_ns, kref);
 
+	mutex_lock(&ns->ctrl->namespaces_mutex);
+	list_del_init(&ns->list);
+	mutex_unlock(&ns->ctrl->namespaces_mutex);
+
 	if (ns->type == NVME_NS_LIGHTNVM)
 		nvme_nvm_unregister(ns->queue, ns->disk->disk_name);
 
@@ -1179,11 +1183,13 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)  {
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
+	bool kill;
 
-	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
+	if (test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
+		return;
 
+	kill = nvme_io_incapable(ns->ctrl) &&
+			!blk_queue_dying(ns->queue);
 	if (kill) {
 		blk_set_queue_dying(ns->queue);
 
@@ -1206,7 +1212,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
-	list_del_init(&ns->list);
 	nvme_put_ns(ns);
 }
 
@@ -1300,10 +1305,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)  {
 	struct nvme_ns *ns, *next;
 
-	mutex_lock(&ctrl->namespaces_mutex);
 	list_for_each_entry_safe(ns, next, &ctrl->namespaces, list)
 		nvme_ns_remove(ns);
-	mutex_unlock(&ctrl->namespaces_mutex);
 }
 
 static DEFINE_IDA(nvme_instance_ida);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 9664d07..db413a7 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -112,6 +112,10 @@ struct nvme_ns {
 	bool ext;
 	u8 pi_type;
 	int type;
+	unsigned long flags;
+
+#define NVME_NS_REMOVING 0
+
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
 };
--
2.6.2.307.g37023ba


From hch at infradead.org  Wed Feb 17 20:09:18 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 17 Feb 2016 20:09:18 -0800
Subject: [PATCH-4.5-v3 1/5] NVMe: Add controller removing state
In-Reply-To: <1455734215-20396-2-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-2-git-send-email-keith.busch@intel.com>
Message-ID: <20160218040918.GA12379@infradead.org>

On Wed, Feb 17, 2016 at 11:36:51AM -0700, Keith Busch wrote:
> The patch is preparing for reducing the scope of the namespace mutex. It
> adds a removing state the namespace scan work uses to know if it is safe
> to proceed. The scan work is flushed after setting the device state so
> the removal path can safely walk the namespace list without locking it.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Looks fine, although the flushing / list access magic could relaly
use a comment describing it in the source code.


From hch at infradead.org  Wed Feb 17 20:09:56 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 17 Feb 2016 20:09:56 -0800
Subject: [PATCH-4.5-v3 2/5] NVMe: Fix namespace removal deadlock
In-Reply-To: <BJXPR01MB1997CEFCF124760350CFEEDE0AF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-3-git-send-email-keith.busch@intel.com>
 <BJXPR01MB1997CEFCF124760350CFEEDE0AF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
Message-ID: <20160218040956.GB12379@infradead.org>

On Thu, Feb 18, 2016 at 02:00:47AM +0000, Wenbo Wang wrote:
> Without this mutex, is it safe to run concurrently with nvme_free_ns which manipulates the list?

see the first patch for an explanation on why this should be safe.


From hch at infradead.org  Wed Feb 17 20:11:22 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 17 Feb 2016 20:11:22 -0800
Subject: [PATCH-4.5-v3 4/5] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <1455734215-20396-5-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-5-git-send-email-keith.busch@intel.com>
Message-ID: <20160218041122.GC12379@infradead.org>

> +/**
> + * nvme_kill_queues(): Ends all namespace queues
> + * @ctrl: the dead controller that needs to end
> + *
> + * Call this function when the driver determines it is unable to get the
> + * controller in a state capable of servicing IO.
> + */
> +void nvme_kill_queues(struct nvme_ctrl *ctrl)

Is there a good reason to skyp the admin queue here entirely?

Otherwise looks fine to me.


From jthumshirn at suse.de  Thu Feb 18 00:32:29 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 18 Feb 2016 09:32:29 +0100
Subject: [PATCH 1/3] nvme: use a work item to submit async event requests
In-Reply-To: <1455732194-30790-2-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-2-git-send-email-hch@lst.de>
Message-ID: <20160218083229.GN18134@c203.arch.suse.de>

On Wed, Feb 17, 2016 at 07:03:12PM +0100, Christoph Hellwig wrote:
> Use a dedicated work item to submit async event requests instead of the
> global kthread.  This simplifies the code and reduces the latencies to
> resubmit a request once an even notification happened.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 18 00:34:37 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 18 Feb 2016 09:34:37 +0100
Subject: [PATCH 2/3] nvme: don't poll the CQ from the kthread
In-Reply-To: <1455732194-30790-3-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-3-git-send-email-hch@lst.de>
Message-ID: <20160218083437.GO18134@c203.arch.suse.de>

On Wed, Feb 17, 2016 at 07:03:13PM +0100, Christoph Hellwig wrote:
> There is no reason to do unconditional polling of CQs per the NVMe
> spec.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>  drivers/nvme/host/pci.c | 12 ------------
>  1 file changed, 12 deletions(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 21b0be4..10839f7 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1156,9 +1156,6 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, int qid,
>  	nvmeq->qid = qid;
>  	nvmeq->cq_vector = -1;
>  	dev->queues[qid] = nvmeq;
> -
> -	/* make sure queue descriptor is set before queue count, for kthread */
> -	mb();
>  	dev->queue_count++;
>  
>  	return nvmeq;
> @@ -1345,7 +1342,6 @@ static int nvme_kthread(void *data)
>  		set_current_state(TASK_INTERRUPTIBLE);
>  		spin_lock(&dev_list_lock);
>  		list_for_each_entry_safe(dev, next, &dev_list, node) {
> -			int i;
>  			u32 csts = readl(dev->bar + NVME_REG_CSTS);
>  
>  			/*
> @@ -1363,14 +1359,6 @@ static int nvme_kthread(void *data)
>  				}
>  				continue;
>  			}
> -			for (i = 0; i < dev->queue_count; i++) {
> -				struct nvme_queue *nvmeq = dev->queues[i];
> -				if (!nvmeq)
> -					continue;
> -				spin_lock_irq(&nvmeq->q_lock);
> -				nvme_process_cq(nvmeq);
> -				spin_unlock_irq(&nvmeq->q_lock);
> -			}
>  		}
>  		spin_unlock(&dev_list_lock);
>  		schedule_timeout(round_jiffies_relative(HZ));
> -- 
> 2.1.4
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Oh yeah, I was hunting down a lockup in that loop ("fixable" with
touch_nmi_watchdog()). Highly appreciate it's gone.


Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 18 00:36:52 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 18 Feb 2016 09:36:52 +0100
Subject: [PATCH 3/3] nvme: replace the kthread with a per-device watchdog
 timer
In-Reply-To: <1455732194-30790-4-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-4-git-send-email-hch@lst.de>
Message-ID: <20160218083651.GP18134@c203.arch.suse.de>

On Wed, Feb 17, 2016 at 07:03:14PM +0100, Christoph Hellwig wrote:
> The only work left in the kthread is the periodic health check for each
> controller.  There is no need to run this from process context or keep
> a thread context around for it, so replace it with a simpler timer.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 18 01:37:09 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 18 Feb 2016 10:37:09 +0100
Subject: [PATCH-4.5-v3 1/5] NVMe: Add controller removing state
In-Reply-To: <1455734215-20396-2-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-2-git-send-email-keith.busch@intel.com>
Message-ID: <20160218093709.GR18134@c203.arch.suse.de>

On Wed, Feb 17, 2016 at 11:36:51AM -0700, Keith Busch wrote:
> The patch is preparing for reducing the scope of the namespace mutex. It
> adds a removing state the namespace scan work uses to know if it is safe
> to proceed. The scan work is flushed after setting the device state so
> the removal path can safely walk the namespace list without locking it.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 18 01:39:27 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 18 Feb 2016 10:39:27 +0100
Subject: [PATCH-4.5-v3 2/5] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455734215-20396-3-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-3-git-send-email-keith.busch@intel.com>
Message-ID: <20160218093927.GS18134@c203.arch.suse.de>

On Wed, Feb 17, 2016 at 11:36:52AM -0700, Keith Busch wrote:
> This patch makes nvme namespace removal lockless. It is up to the caller
> to ensure no active namespace scanning is occuring.
> 
> The lockless removal allows the driver to cleanup a namespace
> request_queue if the controller fails during removal. Previously this
> could deadlock trying to acquire the namespace mutex in order to handle
> such events.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 18 01:42:36 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 18 Feb 2016 10:42:36 +0100
Subject: [PATCH-4.5-v3 4/5] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <1455734215-20396-5-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-5-git-send-email-keith.busch@intel.com>
Message-ID: <20160218094236.GT18134@c203.arch.suse.de>

On Wed, Feb 17, 2016 at 11:36:54AM -0700, Keith Busch wrote:
> This moves failed queue handling out of the namespace removal path and
> into the reset failure path, fixing a hanging condition if the controller
> fails or link down during del_gendisk. Previously the driver had to see
> the controller as degraded prior to calling del_gendisk to setup the
> queues to fail. If the controller happened to fail after this though,
> there was no task to end the request_queue.
> 
> On failure, all namespace states are set to dead. This has capacity
> revalidate to 0, and ends all new requests with error status.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 18 01:45:08 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 18 Feb 2016 10:45:08 +0100
Subject: [PATCH-4.5-v3 5/5] NVMe: Don't unmap controller registers on reset
In-Reply-To: <1455734215-20396-6-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-6-git-send-email-keith.busch@intel.com>
Message-ID: <20160218094508.GU18134@c203.arch.suse.de>

On Wed, Feb 17, 2016 at 11:36:55AM -0700, Keith Busch wrote:
> Unmapping the registers on reset or shutdown is not necessary. Keeping
> the mapping simplifies reset handling.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From hare at suse.de  Thu Feb 18 05:04:28 2016
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 18 Feb 2016 14:04:28 +0100
Subject: NVMe wwid revisited
Message-ID: <56C5C15C.8030903@suse.de>

Hi Keith,

I've had a look at the current NVMe implementation for SCSI VPD page
0x83, in order to implement a generic 'wwid' field.

And I've even looked at the NVM-Express-SCSI-Translation Reference
version 1.1. What a mess :-(

First of all, there's an error in the spec:

Section 6.1.4.3 T10 Vendor ID Based Descriptor has:

DESIGNATOR TYPE
Shall be set to 8h indicating SCSI name string format and assignment
authority.

T10 VENDOR IDENTIFICATION
Refer to 3.12

VENDOR SPECIFIC IDENTIFIER
Shall be set the concatenation of the translation of PRODUCT
IDENTIFICATION field from standard INQUIRY data as specified in
3.9, and the IEEE Extended Unique Identifier (EUI64) field of the
Identify Namespace Data Structure

This does _not_ match with T-10 usage; T-10 Vendor ID based
identification is _NOT_ a scsi name string.
And the scsi name string has to start with either 'eui.', 'naa.' or
'iqn.'. Plus the SCSI name string has to be in UTF-8, not ASCII.

So I guess they actually meant for the designator type to be of T-10
Vendor Identification (ie designator type 1).

Can you address this with the NVMe committee?
Or do you have someone which I can poke?

Which, of course, raises the question: What _is_ the correct
identification?
Implement it according to the translation spec, leaving out the 'T10
Vendor Identification' field?
Or implement it with the correct designator type?

(Which is interesting in itself; one simply _cannot_ implement a
valid translation according to the spec. How curious ...)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: F. Imend?rffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG N?rnberg)


From hch at lst.de  Thu Feb 18 05:28:56 2016
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 18 Feb 2016 14:28:56 +0100
Subject: NVMe wwid revisited
In-Reply-To: <56C5C15C.8030903@suse.de>
References: <56C5C15C.8030903@suse.de>
Message-ID: <20160218132856.GA5150@lst.de>

On Thu, Feb 18, 2016 at 02:04:28PM +0100, Hannes Reinecke wrote:
> Hi Keith,
> 
> I've had a look at the current NVMe implementation for SCSI VPD page
> 0x83, in order to implement a generic 'wwid' field.
> 
> And I've even looked at the NVM-Express-SCSI-Translation Reference
> version 1.1. What a mess :-(

There is a version 1.5 by now..

> So I guess they actually meant for the designator type to be of T-10
> Vendor Identification (ie designator type 1).

The 1.5 spec has actually fixed this up:

6.1.4.3 T10 Vendor ID Based Descriptor

...

DESIGNATOR TYPE  |  Shall be set to 1h indicating T10 vendor ID based format.

> Which, of course, raises the question: What _is_ the correct
> identification?
> Implement it according to the translation spec, leaving out the 'T10
> Vendor Identification' field?
> Or implement it with the correct designator type?

Leave it as-is, given a certain Linux distributor started using it for
their udev rules and we'd otherwise break their setup.


From keith.busch at intel.com  Thu Feb 18 07:10:11 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 15:10:11 +0000
Subject: NVMe wwid revisited
In-Reply-To: <20160218132856.GA5150@lst.de>
References: <56C5C15C.8030903@suse.de>
 <20160218132856.GA5150@lst.de>
Message-ID: <20160218151010.GA4901@localhost.localdomain>

On Thu, Feb 18, 2016 at 02:28:56PM +0100, Christoph Hellwig wrote:
> On Thu, Feb 18, 2016 at 02:04:28PM +0100, Hannes Reinecke wrote:
> > Which, of course, raises the question: What _is_ the correct
> > identification?
> > Implement it according to the translation spec, leaving out the 'T10
> > Vendor Identification' field?
> > Or implement it with the correct designator type?
> 
> Leave it as-is, given a certain Linux distributor started using it for
> their udev rules and we'd otherwise break their setup.

We mentioned exposing a single sysfs handle that provides the unique so
the user doesn't have to check/concat multiple files or use the deprecated
NVMe SG_IO interface.

Does the below patch look alright? It uses either 64 or 128 bit EUI if
provided, or concats other parts to make a unique (and long!) identifier
if the namespace doesn't support its own unique id.

Here's a couple examples with a 1.0 controller and a 1.2 controller
(I don't have a 1.1 controller; no EUI-64 example):

  1.0:
    # cat /sys/block/nvme0n1/wwid
    nvme.8086-43564654343032333030324b38303043474e-494e54454c205353445045444d443830304734-00000001

  1.2:
    # cat /sys/block/nvme1n2/wwid
    eui.5cd2e400000000000000000000000100000002

---
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index face31f..93ec2e0 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -872,6 +876,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		return -EIO;
 	}
 
+	ctrl->vid = le16_to_cpu(id->vid);
 	ctrl->oncs = le16_to_cpup(&id->oncs);
 	atomic_set(&ctrl->abort_limit, id->acl + 1);
 	ctrl->vwc = id->vwc;
@@ -1008,6 +1013,30 @@ static ssize_t nvme_sysfs_reset(struct device *dev,
 }
 static DEVICE_ATTR(reset_controller, S_IWUSR, NULL, nvme_sysfs_reset);
 
+static ssize_t wwid_show(struct device *dev, struct device_attribute *attr,
+								char *buf)
+{
+	struct nvme_ns *ns = dev_to_disk(dev)->private_data;
+	struct nvme_ctrl *ctrl = ns->ctrl;
+	int serial_len = sizeof(ctrl->serial);
+	int model_len = sizeof(ctrl->model);
+
+	if (memchr_inv(ns->uuid, 0, sizeof(ns->uuid)))
+		return sprintf(buf, "eui.%16phN\n", ns->uuid);
+
+	if (memchr_inv(ns->eui, 0, sizeof(ns->eui)))
+		return sprintf(buf, "eui.%8phN\n", ns->eui);
+
+	while (ctrl->serial[serial_len - 1] == ' ')
+		serial_len--;
+	while (ctrl->model[model_len - 1] == ' ')
+		model_len--;
+
+	return sprintf(buf, "nvme.%04x-%*phN-%*phN-%08x\n", ctrl->vid,
+		serial_len, ctrl->serial, model_len, ctrl->model, ns->ns_id);
+}
+static DEVICE_ATTR(wwid, S_IRUGO, wwid_show, NULL);
+
 static ssize_t uuid_show(struct device *dev, struct device_attribute *attr,
 								char *buf)
 {
@@ -1033,6 +1062,7 @@ static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
 static DEVICE_ATTR(nsid, S_IRUGO, nsid_show, NULL);
 
 static struct attribute *nvme_ns_attrs[] = {
+	&dev_attr_wwid.attr,
 	&dev_attr_uuid.attr,
 	&dev_attr_eui.attr,
 	&dev_attr_nsid.attr,
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index dcdc2d7..b50e591 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -84,6 +84,7 @@ struct nvme_ctrl {
 	u32 max_hw_sectors;
 	u32 stripe_size;
 	u16 oncs;
+	u16 vid;
 	atomic_t abort_limit;
 	u8 event_limit;
 	u8 vwc;
---


From hare at suse.de  Thu Feb 18 07:39:45 2016
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 18 Feb 2016 16:39:45 +0100
Subject: NVMe wwid revisited
In-Reply-To: <20160218151010.GA4901@localhost.localdomain>
References: <56C5C15C.8030903@suse.de> <20160218132856.GA5150@lst.de>
 <20160218151010.GA4901@localhost.localdomain>
Message-ID: <56C5E5C1.1070707@suse.de>

On 02/18/2016 04:10 PM, Keith Busch wrote:
> On Thu, Feb 18, 2016 at 02:28:56PM +0100, Christoph Hellwig wrote:
>> On Thu, Feb 18, 2016 at 02:04:28PM +0100, Hannes Reinecke wrote:
>>> Which, of course, raises the question: What _is_ the correct
>>> identification?
>>> Implement it according to the translation spec, leaving out the 'T10
>>> Vendor Identification' field?
>>> Or implement it with the correct designator type?
>>
>> Leave it as-is, given a certain Linux distributor started using it for
>> their udev rules and we'd otherwise break their setup.
>
> We mentioned exposing a single sysfs handle that provides the unique so
> the user doesn't have to check/concat multiple files or use the deprecated
> NVMe SG_IO interface.
>
> Does the below patch look alright? It uses either 64 or 128 bit EUI if
> provided, or concats other parts to make a unique (and long!) identifier
> if the namespace doesn't support its own unique id.
>
> Here's a couple examples with a 1.0 controller and a 1.2 controller
> (I don't have a 1.1 controller; no EUI-64 example):
>
>    1.0:
>      # cat /sys/block/nvme0n1/wwid
>      nvme.8086-43564654343032333030324b38303043474e-494e54454c205353445045444d443830304734-00000001
>
>    1.2:
>      # cat /sys/block/nvme1n2/wwid
>      eui.5cd2e400000000000000000000000100000002
>
> ---
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index face31f..93ec2e0 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -872,6 +876,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
>   		return -EIO;
>   	}
>
> +	ctrl->vid = le16_to_cpu(id->vid);
>   	ctrl->oncs = le16_to_cpup(&id->oncs);
>   	atomic_set(&ctrl->abort_limit, id->acl + 1);
>   	ctrl->vwc = id->vwc;
> @@ -1008,6 +1013,30 @@ static ssize_t nvme_sysfs_reset(struct device *dev,
>   }
>   static DEVICE_ATTR(reset_controller, S_IWUSR, NULL, nvme_sysfs_reset);
>
> +static ssize_t wwid_show(struct device *dev, struct device_attribute *attr,
> +								char *buf)
> +{
> +	struct nvme_ns *ns = dev_to_disk(dev)->private_data;
> +	struct nvme_ctrl *ctrl = ns->ctrl;
> +	int serial_len = sizeof(ctrl->serial);
> +	int model_len = sizeof(ctrl->model);
> +
> +	if (memchr_inv(ns->uuid, 0, sizeof(ns->uuid)))
> +		return sprintf(buf, "eui.%16phN\n", ns->uuid);
> +
> +	if (memchr_inv(ns->eui, 0, sizeof(ns->eui)))
> +		return sprintf(buf, "eui.%8phN\n", ns->eui);
> +
> +	while (ctrl->serial[serial_len - 1] == ' ')
> +		serial_len--;
> +	while (ctrl->model[model_len - 1] == ' ')
> +		model_len--;
> +
> +	return sprintf(buf, "nvme.%04x-%*phN-%*phN-%08x\n", ctrl->vid,
> +		serial_len, ctrl->serial, model_len, ctrl->model, ns->ns_id);
> +}
> +static DEVICE_ATTR(wwid, S_IRUGO, wwid_show, NULL);
> +
>   static ssize_t uuid_show(struct device *dev, struct device_attribute *attr,
>   								char *buf)
>   {
> @@ -1033,6 +1062,7 @@ static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
>   static DEVICE_ATTR(nsid, S_IRUGO, nsid_show, NULL);
>
>   static struct attribute *nvme_ns_attrs[] = {
> +	&dev_attr_wwid.attr,
>   	&dev_attr_uuid.attr,
>   	&dev_attr_eui.attr,
>   	&dev_attr_nsid.attr,
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index dcdc2d7..b50e591 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -84,6 +84,7 @@ struct nvme_ctrl {
>   	u32 max_hw_sectors;
>   	u32 stripe_size;
>   	u16 oncs;
> +	u16 vid;
>   	atomic_t abort_limit;
>   	u8 event_limit;
>   	u8 vwc;
> ---
>
Yeah, that looks good.

Reviewed-by: Hannes Reinecke <hare at suse.com>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare at suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: J. Hawn, J. Guild, F. Imend?rffer, HRB 16746 (AG N?rnberg)


From keith.busch at intel.com  Thu Feb 18 08:26:14 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 16:26:14 +0000
Subject: [PATCH-4.5-v3 2/5] NVMe: Fix namespace removal deadlock
In-Reply-To: <20160218040956.GB12379@infradead.org>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-3-git-send-email-keith.busch@intel.com>
 <BJXPR01MB1997CEFCF124760350CFEEDE0AF0@BJXPR01MB199.CHNPR01.prod.partner.outlook.cn>
 <20160218040956.GB12379@infradead.org>
Message-ID: <20160218162613.GB4901@localhost.localdomain>

On Wed, Feb 17, 2016 at 08:09:56PM -0800, Christoph Hellwig wrote:
> On Thu, Feb 18, 2016 at 02:00:47AM +0000, Wenbo Wang wrote:
> > Without this mutex, is it safe to run concurrently with nvme_free_ns which manipulates the list?
> 
> see the first patch for an explanation on why this should be safe.

I think the potential problem Wenbo is talking about is if someone is
holding an open reference (mounted, for example) after detaching the
namespace.

Removing it from the list when all references are released keeps the list
consistent with the block layer so that attaching a different namespace
with the same NSID won't create a naming conflict. The newly attached
namespace isn't be usable in this case, though.

Instead of using NSID, I think the disk name should use an IDA like the
controller's instance. Then we can unlink the namespace from the list
when detached instead of with the last reference.

The downside is you can't infer the NSID from the name, but we have
NVME_IOCTL_ID and /sys/block/nvmeXnY/nsid for that.


From keith.busch at intel.com  Thu Feb 18 08:57:48 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 09:57:48 -0700
Subject: [PATCH] NVMe: Expose ns wwid through single sysfs entry
Message-ID: <1455814668-784-1-git-send-email-keith.busch@intel.com>

The method to uniquely identify a namespace depends on the controller's
specification revision level and implemented capabilities. This patch
has the driver figure this out and exports the unique string through a
single 'wwid' attribute so the user doesn't have this burden.

The longest namespace unique identifier is used if available. If not
available, the driver will concat the controller's vendor, serial,
and model with the namespace ID. The specification provides this as a
unique indentifier.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Hannes Reinecke <hare at suse.com>
---
 drivers/nvme/host/core.c | 26 ++++++++++++++++++++++++++
 drivers/nvme/host/nvme.h |  1 +
 2 files changed, 27 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index face31f..b29fae6 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -872,6 +872,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		return -EIO;
 	}
 
+	ctrl->vid = le16_to_cpu(id->vid);
 	ctrl->oncs = le16_to_cpup(&id->oncs);
 	atomic_set(&ctrl->abort_limit, id->acl + 1);
 	ctrl->vwc = id->vwc;
@@ -1008,6 +1009,30 @@ static ssize_t nvme_sysfs_reset(struct device *dev,
 }
 static DEVICE_ATTR(reset_controller, S_IWUSR, NULL, nvme_sysfs_reset);
 
+static ssize_t wwid_show(struct device *dev, struct device_attribute *attr,
+								char *buf)
+{
+	struct nvme_ns *ns = dev_to_disk(dev)->private_data;
+	struct nvme_ctrl *ctrl = ns->ctrl;
+	int serial_len = sizeof(ctrl->serial);
+	int model_len = sizeof(ctrl->model);
+
+	if (memchr_inv(ns->uuid, 0, sizeof(ns->uuid)))
+		return sprintf(buf, "eui.%16phN\n", ns->uuid);
+
+	if (memchr_inv(ns->eui, 0, sizeof(ns->eui)))
+		return sprintf(buf, "eui.%8phN\n", ns->eui);
+
+	while (ctrl->serial[serial_len - 1] == ' ')
+		serial_len--;
+	while (ctrl->model[model_len - 1] == ' ')
+		model_len--;
+
+	return sprintf(buf, "nvme.%04x-%*phN-%*phN-%08x\n", ctrl->vid,
+		serial_len, ctrl->serial, model_len, ctrl->model, ns->ns_id);
+}
+static DEVICE_ATTR(wwid, S_IRUGO, wwid_show, NULL);
+
 static ssize_t uuid_show(struct device *dev, struct device_attribute *attr,
 								char *buf)
 {
@@ -1033,6 +1058,7 @@ static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
 static DEVICE_ATTR(nsid, S_IRUGO, nsid_show, NULL);
 
 static struct attribute *nvme_ns_attrs[] = {
+	&dev_attr_wwid.attr,
 	&dev_attr_uuid.attr,
 	&dev_attr_eui.attr,
 	&dev_attr_nsid.attr,
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index dcdc2d7..b50e591 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -84,6 +84,7 @@ struct nvme_ctrl {
 	u32 max_hw_sectors;
 	u32 stripe_size;
 	u16 oncs;
+	u16 vid;
 	atomic_t abort_limit;
 	u8 event_limit;
 	u8 vwc;
-- 
2.6.2.307.g37023ba



From hch at infradead.org  Thu Feb 18 09:01:19 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 18 Feb 2016 09:01:19 -0800
Subject: [PATCH] NVMe: Expose ns wwid through single sysfs entry
In-Reply-To: <1455814668-784-1-git-send-email-keith.busch@intel.com>
References: <1455814668-784-1-git-send-email-keith.busch@intel.com>
Message-ID: <20160218170119.GA13776@infradead.org>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From keith.busch at intel.com  Thu Feb 18 10:18:05 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 11:18:05 -0700
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
Message-ID: <1455819485-3730-1-git-send-email-keith.busch@intel.com>

A namespace may be detached from a controller, but a user may be holding
a reference to it. Attaching a new namespace with the same NSID will create
duplicate names when using the NSID to name the disk.

This patch uses an IDA that is released only when the last reference is
released instead of using the namespace ID.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 25 ++++++++++++++++++-------
 drivers/nvme/host/nvme.h |  2 ++
 2 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index b29fae6..9b1253f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -48,10 +48,6 @@ static void nvme_free_ns(struct kref *kref)
 {
 	struct nvme_ns *ns = container_of(kref, struct nvme_ns, kref);
 
-	mutex_lock(&ns->ctrl->namespaces_mutex);
-	list_del_init(&ns->list);
-	mutex_unlock(&ns->ctrl->namespaces_mutex);
-
 	if (ns->type == NVME_NS_LIGHTNVM)
 		nvme_nvm_unregister(ns->queue, ns->disk->disk_name);
 
@@ -59,8 +55,9 @@ static void nvme_free_ns(struct kref *kref)
 	ns->disk->private_data = NULL;
 	spin_unlock(&dev_list_lock);
 
-	nvme_put_ctrl(ns->ctrl);
 	put_disk(ns->disk);
+	ida_simple_remove(&ns->ctrl->ns_ida, ns->instance);
+	nvme_put_ctrl(ns->ctrl);
 	kfree(ns);
 }
 
@@ -1152,9 +1149,13 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	if (!ns)
 		return;
 
+	ns->instance = ida_simple_get(&ctrl->ns_ida, 1, 0, GFP_KERNEL);
+	if (ns->instance < 0)
+		goto out_free_ns;
+
 	ns->queue = blk_mq_init_queue(ctrl->tagset);
 	if (IS_ERR(ns->queue))
-		goto out_free_ns;
+		goto out_release_instance;
 	queue_flag_set_unlocked(QUEUE_FLAG_NONROT, ns->queue);
 	ns->queue->queuedata = ns;
 	ns->ctrl = ctrl;
@@ -1187,7 +1188,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	disk->queue = ns->queue;
 	disk->driverfs_dev = ctrl->device;
 	disk->flags = GENHD_FL_EXT_DEVT;
-	sprintf(disk->disk_name, "nvme%dn%d", ctrl->instance, nsid);
+	sprintf(disk->disk_name, "nvme%dn%d", ctrl->instance, ns->instance);
 
 	if (nvme_revalidate_disk(ns->disk))
 		goto out_free_disk;
@@ -1207,6 +1208,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	kfree(disk);
  out_free_queue:
 	blk_cleanup_queue(ns->queue);
+ out_release_instance:
+	ida_simple_remove(&ctrl->ns_ida, ns->instance);
  out_free_ns:
 	kfree(ns);
 }
@@ -1225,6 +1228,11 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
+
+	mutex_lock(&ns->ctrl->namespaces_mutex);
+	list_del_init(&ns->list);
+	mutex_unlock(&ns->ctrl->namespaces_mutex);
+
 	nvme_put_ns(ns);
 }
 
@@ -1358,6 +1366,8 @@ void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
 	spin_lock(&dev_list_lock);
 	list_del(&ctrl->node);
 	spin_unlock(&dev_list_lock);
+
+	ida_destroy(&ctrl->ns_ida);
 }
 
 static void nvme_free_ctrl(struct kref *kref)
@@ -1406,6 +1416,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	}
 	get_device(ctrl->device);
 	dev_set_drvdata(ctrl->device, ctrl);
+	ida_init(&ctrl->ns_ida);
 
 	spin_lock(&dev_list_lock);
 	list_add_tail(&ctrl->node, &nvme_ctrl_list);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index b50e591..3b0f84a 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -72,6 +72,7 @@ struct nvme_ctrl {
 	struct mutex namespaces_mutex;
 	struct device *device;	/* char device */
 	struct list_head node;
+	struct ida ns_ida;
 
 	char name[12];
 	char serial[20];
@@ -103,6 +104,7 @@ struct nvme_ns {
 	struct request_queue *queue;
 	struct gendisk *disk;
 	struct kref kref;
+	int instance;
 
 	u8 eui[8];
 	u8 uuid[16];
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 18 11:38:51 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 19:38:51 +0000
Subject: [PATCH-4.5-v3 4/5] NVMe: Move error handling to failed reset
 handler
In-Reply-To: <20160218041122.GC12379@infradead.org>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-5-git-send-email-keith.busch@intel.com>
 <20160218041122.GC12379@infradead.org>
Message-ID: <20160218193850.GA9620@localhost.localdomain>

On Wed, Feb 17, 2016 at 08:11:22PM -0800, Christoph Hellwig wrote:
> Is there a good reason to skyp the admin queue here entirely?

It is not necessary to progress namespace removal, and we don't want to
step on the controller removal path in case it actually is making progress
(we only think it's dead in this path).

A catch all place to restart a failed admin queue to allow controller
removal to complete is nvme_dev_remove_admin().

---
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index f11daba..4953008 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1253,6 +1253,12 @@ static struct blk_mq_ops nvme_mq_ops = {
 static void nvme_dev_remove_admin(struct nvme_dev *dev)
 {
 	if (dev->ctrl.admin_q && !blk_queue_dying(dev->ctrl.admin_q)) {
+		/*
+		 * If the controller was reset during removal, it's possible
+		 * user requests may be waiting on a stopped queue. Start the
+		 * queue to flush these to completion.
+		 */
+		blk_mq_start_stopped_hw_queues(dev->ctrl.admin_q, true);
 		blk_cleanup_queue(dev->ctrl.admin_q);
 		blk_mq_free_tag_set(&dev->admin_tagset);
 	}
--

BTW, from testing this I notice the logic in nvme_queue_rq was screwed
up in this patch when I switched from the ternary '?:' to 'if-else'.

Jens hasn't pushed anything this series yet so I'll resend and fold the
above in.


From keith.busch at intel.com  Thu Feb 18 12:00:19 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 13:00:19 -0700
Subject: [PATCHv4] NVMe: Move error handling to failed reset handler
Message-ID: <1455825619-5750-1-git-send-email-keith.busch@intel.com>

This moves failed queue handling out of the namespace removal path and into
the reset failure path, fixing a deadlock condition if the controller
fails or link down during del_gendisk. Previously the driver had to see
the controller as degraded prior to calling del_gendisk to setup the
queues to fail. If the controller happened to fail after this though,
there was no task to end the request_queue.

On failure, all namespace states are set to dead. This has capacity
revalidate to 0, and ends all new requests with error status.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
Replaces this one:

  http://lists.infradead.org/pipermail/linux-nvme/2016-February/004040.html

Fixed the logic to end a request in error, and kick the admin queue so
requests can end, similar to the namespace queues.

 drivers/nvme/host/core.c | 53 ++++++++++++++++++++++++++++++++----------------
 drivers/nvme/host/nvme.h |  2 ++
 drivers/nvme/host/pci.c  | 13 +++++++++++-
 3 files changed, 50 insertions(+), 18 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1b7387d..face31f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -560,6 +560,10 @@ static int nvme_revalidate_disk(struct gendisk *disk)
 	u16 old_ms;
 	unsigned short bs;
 
+	if (test_bit(NVME_NS_DEAD, &ns->flags)) {
+		set_capacity(disk, 0);
+		return -ENODEV;
+	}
 	if (nvme_identify_ns(ns->ctrl, ns->ns_id, &id)) {
 		dev_warn(ns->ctrl->dev, "%s: Identify failure nvme%dn%d\n",
 				__func__, ns->ctrl->instance, ns->ns_id);
@@ -1183,32 +1187,15 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)
 {
-	bool kill;
-
 	if (test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
 		return;
 
-	kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
 	if (ns->disk->flags & GENHD_FL_UP) {
 		if (blk_get_integrity(ns->disk))
 			blk_integrity_unregister(ns->disk);
 		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
 					&nvme_ns_attr_group);
 		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
@@ -1405,6 +1392,38 @@ out:
 	return ret;
 }
 
+/**
+ * nvme_kill_queues(): Ends all namespace queues
+ * @ctrl: the dead controller that needs to end
+ *
+ * Call this function when the driver determines it is unable to get the
+ * controller in a state capable of servicing IO.
+ */
+void nvme_kill_queues(struct nvme_ctrl *ctrl)
+{
+	struct nvme_ns *ns;
+
+	mutex_lock(&ctrl->namespaces_mutex);
+	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (!kref_get_unless_zero(&ns->kref))
+			continue;
+
+		/*
+		 * Revalidating a dead namespace sets capacity to 0. This will
+		 * end buffered writers dirtying pages that can't be synced.
+		 */
+		if (!test_and_set_bit(NVME_NS_DEAD, &ns->flags))
+			revalidate_disk(ns->disk);
+
+		blk_set_queue_dying(ns->queue);
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_mq_start_stopped_hw_queues(ns->queue, true);
+
+		nvme_put_ns(ns);
+	}
+	mutex_unlock(&ctrl->namespaces_mutex);
+}
+
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
 	struct nvme_ns *ns;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index db413a7..dcdc2d7 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -115,6 +115,7 @@ struct nvme_ns {
 	unsigned long flags;
 
 #define NVME_NS_REMOVING 0
+#define NVME_NS_DEAD     1
 
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
@@ -244,6 +245,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl);
 void nvme_start_queues(struct nvme_ctrl *ctrl);
+void nvme_kill_queues(struct nvme_ctrl *ctrl);
 
 struct request *nvme_alloc_request(struct request_queue *q,
 		struct nvme_command *cmd, unsigned int flags);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 9e050dd..f64eb7a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -679,7 +679,10 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 
 	spin_lock_irq(&nvmeq->q_lock);
 	if (unlikely(nvmeq->cq_vector < 0)) {
-		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		if (ns && !test_bit(NVME_NS_DEAD, &ns->flags))
+			ret = BLK_MQ_RQ_QUEUE_BUSY;
+		else
+			ret = BLK_MQ_RQ_QUEUE_ERROR;
 		spin_unlock_irq(&nvmeq->q_lock);
 		goto out;
 	}
@@ -1250,6 +1253,12 @@ static struct blk_mq_ops nvme_mq_ops = {
 static void nvme_dev_remove_admin(struct nvme_dev *dev)
 {
 	if (dev->ctrl.admin_q && !blk_queue_dying(dev->ctrl.admin_q)) {
+		/*
+		 * If the controller was reset during removal, it's possible
+		 * user requests may be waiting on a stopped queue. Start the
+		 * queue to flush requests to their end.
+		 */
+		blk_mq_start_stopped_hw_queues(dev->ctrl.admin_q, true);
 		blk_cleanup_queue(dev->ctrl.admin_q);
 		blk_mq_free_tag_set(&dev->admin_tagset);
 	}
@@ -1906,6 +1915,7 @@ static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
 	dev_warn(dev->dev, "Removing after probe failure status: %d\n", status);
 
 	kref_get(&dev->ctrl.kref);
+	nvme_dev_disable(dev, false);
 	if (!schedule_work(&dev->remove_work))
 		nvme_put_ctrl(&dev->ctrl);
 }
@@ -1978,6 +1988,7 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, remove_work);
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
+	nvme_kill_queues(&dev->ctrl);
 	if (pci_get_drvdata(pdev))
 		pci_stop_and_remove_bus_device_locked(pdev);
 	nvme_put_ctrl(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 18 15:21:16 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 16:21:16 -0700
Subject: [PATCH-4.6 2/3] NVMe: Code naming cleanup
In-Reply-To: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455837677-23248-2-git-send-email-keith.busch@intel.com>

Consistency help readability. This renames all instances of struct
nvme_ctrl to "ctrl" that weren't already named that.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
 drivers/nvme/host/core.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 04443b2..33ad10d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -248,7 +248,7 @@ int nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 			result, timeout);
 }
 
-int nvme_identify_ctrl(struct nvme_ctrl *dev, struct nvme_id_ctrl **id)
+int nvme_identify_ctrl(struct nvme_ctrl *ctrl, struct nvme_id_ctrl **id)
 {
 	struct nvme_command c = { };
 	int error;
@@ -261,24 +261,24 @@ int nvme_identify_ctrl(struct nvme_ctrl *dev, struct nvme_id_ctrl **id)
 	if (!*id)
 		return -ENOMEM;
 
-	error = nvme_submit_sync_cmd(dev->admin_q, &c, *id,
+	error = nvme_submit_sync_cmd(ctrl->admin_q, &c, *id,
 			sizeof(struct nvme_id_ctrl));
 	if (error)
 		kfree(*id);
 	return error;
 }
 
-static int nvme_identify_ns_list(struct nvme_ctrl *dev, unsigned nsid, __le32 *ns_list)
+static int nvme_identify_ns_list(struct nvme_ctrl *ctrl, unsigned nsid, __le32 *ns_list)
 {
 	struct nvme_command c = { };
 
 	c.identify.opcode = nvme_admin_identify;
 	c.identify.cns = cpu_to_le32(2);
 	c.identify.nsid = cpu_to_le32(nsid);
-	return nvme_submit_sync_cmd(dev->admin_q, &c, ns_list, 0x1000);
+	return nvme_submit_sync_cmd(ctrl->admin_q, &c, ns_list, 0x1000);
 }
 
-int nvme_identify_ns(struct nvme_ctrl *dev, unsigned nsid,
+int nvme_identify_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 		struct nvme_id_ns **id)
 {
 	struct nvme_command c = { };
@@ -292,14 +292,14 @@ int nvme_identify_ns(struct nvme_ctrl *dev, unsigned nsid,
 	if (!*id)
 		return -ENOMEM;
 
-	error = nvme_submit_sync_cmd(dev->admin_q, &c, *id,
+	error = nvme_submit_sync_cmd(ctrl->admin_q, &c, *id,
 			sizeof(struct nvme_id_ns));
 	if (error)
 		kfree(*id);
 	return error;
 }
 
-int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
+int nvme_get_features(struct nvme_ctrl *ctrl, unsigned fid, unsigned nsid,
 					dma_addr_t dma_addr, u32 *result)
 {
 	struct nvme_command c;
@@ -310,10 +310,10 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
 	c.features.prp1 = cpu_to_le64(dma_addr);
 	c.features.fid = cpu_to_le32(fid);
 
-	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
+	return __nvme_submit_sync_cmd(ctrl->admin_q, &c, NULL, 0, result, 0);
 }
 
-int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
+int nvme_set_features(struct nvme_ctrl *ctrl, unsigned fid, unsigned dword11,
 					dma_addr_t dma_addr, u32 *result)
 {
 	struct nvme_command c;
@@ -324,10 +324,10 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 	c.features.fid = cpu_to_le32(fid);
 	c.features.dword11 = cpu_to_le32(dword11);
 
-	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
+	return __nvme_submit_sync_cmd(ctrl->admin_q, &c, NULL, 0, result, 0);
 }
 
-int nvme_get_log_page(struct nvme_ctrl *dev, struct nvme_smart_log **log)
+int nvme_get_log_page(struct nvme_ctrl *ctrl, struct nvme_smart_log **log)
 {
 	struct nvme_command c = { };
 	int error;
@@ -342,7 +342,7 @@ int nvme_get_log_page(struct nvme_ctrl *dev, struct nvme_smart_log **log)
 	if (!*log)
 		return -ENOMEM;
 
-	error = nvme_submit_sync_cmd(dev->admin_q, &c, *log,
+	error = nvme_submit_sync_cmd(ctrl->admin_q, &c, *log,
 			sizeof(struct nvme_smart_log));
 	if (error)
 		kfree(*log);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 18 15:21:15 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 16:21:15 -0700
Subject: [PATCH-4.6 1/3] NVMe: Use simple ida interface
Message-ID: <1455837677-23248-1-git-send-email-keith.busch@intel.com>

This is an easier interface to use and removes one of the overloaded
dev_list_lock uses.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
Feature was originally posted by Lee Duncan last October, but this missed
being applied.

 drivers/nvme/host/core.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 9b1253f..04443b2 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1334,29 +1334,17 @@ static DEFINE_IDA(nvme_instance_ida);
 
 static int nvme_set_instance(struct nvme_ctrl *ctrl)
 {
-	int instance, error;
-
-	do {
-		if (!ida_pre_get(&nvme_instance_ida, GFP_KERNEL))
-			return -ENODEV;
-
-		spin_lock(&dev_list_lock);
-		error = ida_get_new(&nvme_instance_ida, &instance);
-		spin_unlock(&dev_list_lock);
-	} while (error == -EAGAIN);
-
-	if (error)
-		return -ENODEV;
+	int instance = ida_simple_get(&nvme_instance_ida, 0, 0, GFP_KERNEL);
 
+	if (instance < 0)
+		return instance;
 	ctrl->instance = instance;
 	return 0;
 }
 
 static void nvme_release_instance(struct nvme_ctrl *ctrl)
 {
-	spin_lock(&dev_list_lock);
-	ida_remove(&nvme_instance_ida, ctrl->instance);
-	spin_unlock(&dev_list_lock);
+	ida_simple_remove(&nvme_instance_ida, ctrl->instance);
 }
 
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Thu Feb 18 15:21:17 2016
From: keith.busch at intel.com (Keith Busch)
Date: Thu, 18 Feb 2016 16:21:17 -0700
Subject: [PATCH-4.6 3/3] NVMe: Don't allow unsupported flags
In-Reply-To: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
Message-ID: <1455837677-23248-3-git-send-email-keith.busch@intel.com>

The command flags can change the meaning of other fields in the command
that the driver is not prepared to handle. Specifically, the user could
passthrough an SGL flag, causing the controller to misinterpret the PRP
list the driver created, potentially corrupting memory or data.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
Alternatively I have a different patch that builds SGL's if the flags
has it set and the device supports SGL. Any interest?

I didn't post it since the fast path only gets PRP-able scatter lists,
and the additional logic to handle SGL's complicates handling the NVMe
IO descriptor.

 drivers/nvme/host/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 33ad10d..d8c3a55 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -398,7 +398,7 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
 
 	memset(&c, 0, sizeof(c));
 	c.rw.opcode = io.opcode;
-	c.rw.flags = io.flags;
+	c.rw.flags = 0;
 	c.rw.nsid = cpu_to_le32(ns->ns_id);
 	c.rw.slba = cpu_to_le64(io.slba);
 	c.rw.length = cpu_to_le16(io.nblocks);
@@ -428,7 +428,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 
 	memset(&c, 0, sizeof(c));
 	c.common.opcode = cmd.opcode;
-	c.common.flags = cmd.flags;
+	c.common.flags = 0;
 	c.common.nsid = cpu_to_le32(cmd.nsid);
 	c.common.cdw2[0] = cpu_to_le32(cmd.cdw2);
 	c.common.cdw2[1] = cpu_to_le32(cmd.cdw3);
-- 
2.6.2.307.g37023ba



From jonathan.derrick at intel.com  Fri Feb 19 11:14:10 2016
From: jonathan.derrick at intel.com (Derrick, Jonathan)
Date: Fri, 19 Feb 2016 19:14:10 +0000
Subject: [PATCH-4.6 3/3] NVMe: Don't allow unsupported flags
In-Reply-To: <1455837677-23248-3-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
 <1455837677-23248-3-git-send-email-keith.busch@intel.com>
Message-ID: <8D943B239464E04C9184454D8B3F5CFF0AC93EE1@ORSMSX101.amr.corp.intel.com>

I haven't seen a great argument for SGL other than that the devices support it. If anyone can prove it faster, I'd like to see it.

Otherwise the set looks good

-----Original Message-----
From: Linux-nvme [mailto:linux-nvme-bounces at lists.infradead.org] On Behalf Of Keith Busch
Sent: Thursday, February 18, 2016 4:21 PM
To: linux-nvme at lists.infradead.org; Jens Axboe <axboe at fb.com>; Christoph Hellwig <hch at infradead.org>
Cc: Busch, Keith <keith.busch at intel.com>
Subject: [PATCH-4.6 3/3] NVMe: Don't allow unsupported flags

The command flags can change the meaning of other fields in the command that the driver is not prepared to handle. Specifically, the user could passthrough an SGL flag, causing the controller to misinterpret the PRP list the driver created, potentially corrupting memory or data.

Signed-off-by: Keith Busch <keith.busch at intel.com>
---
Alternatively I have a different patch that builds SGL's if the flags has it set and the device supports SGL. Any interest?

I didn't post it since the fast path only gets PRP-able scatter lists, and the additional logic to handle SGL's complicates handling the NVMe IO descriptor.

 drivers/nvme/host/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 33ad10d..d8c3a55 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -398,7 +398,7 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
 
 	memset(&c, 0, sizeof(c));
 	c.rw.opcode = io.opcode;
-	c.rw.flags = io.flags;
+	c.rw.flags = 0;
 	c.rw.nsid = cpu_to_le32(ns->ns_id);
 	c.rw.slba = cpu_to_le64(io.slba);
 	c.rw.length = cpu_to_le16(io.nblocks); @@ -428,7 +428,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 
 	memset(&c, 0, sizeof(c));
 	c.common.opcode = cmd.opcode;
-	c.common.flags = cmd.flags;
+	c.common.flags = 0;
 	c.common.nsid = cpu_to_le32(cmd.nsid);
 	c.common.cdw2[0] = cpu_to_le32(cmd.cdw2);
 	c.common.cdw2[1] = cpu_to_le32(cmd.cdw3);
--
2.6.2.307.g37023ba


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme


From hch at infradead.org  Sun Feb 21 08:12:49 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sun, 21 Feb 2016 08:12:49 -0800
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
In-Reply-To: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
References: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
Message-ID: <20160221161249.GA21099@infradead.org>

On Thu, Feb 18, 2016 at 11:18:05AM -0700, Keith Busch wrote:
> A namespace may be detached from a controller, but a user may be holding
> a reference to it. Attaching a new namespace with the same NSID will create
> duplicate names when using the NSID to name the disk.

This looks sensible, but I fear we might be running into the same
issue with the backing_device embedded in the request_queue.


From hch at infradead.org  Sun Feb 21 08:15:02 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sun, 21 Feb 2016 08:15:02 -0800
Subject: [PATCH-4.5-v3 5/5] NVMe: Don't unmap controller registers on reset
In-Reply-To: <1455734215-20396-6-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-6-git-send-email-keith.busch@intel.com>
Message-ID: <20160221161502.GA21896@infradead.org>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Sun Feb 21 08:17:14 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sun, 21 Feb 2016 08:17:14 -0800
Subject: [PATCH-4.5-v3 2/5] NVMe: Fix namespace removal deadlock
In-Reply-To: <1455734215-20396-3-git-send-email-keith.busch@intel.com>
References: <1455734215-20396-1-git-send-email-keith.busch@intel.com>
 <1455734215-20396-3-git-send-email-keith.busch@intel.com>
Message-ID: <20160221161714.GB21896@infradead.org>

On Wed, Feb 17, 2016 at 11:36:52AM -0700, Keith Busch wrote:
> This patch makes nvme namespace removal lockless. It is up to the caller
> to ensure no active namespace scanning is occuring.

I wonder if it wouldn't be safer in the long run if we just switch
to RCU locking - that way we don't need any locks in the readers, they
can just grab a reference under rcu_read_lock(), and we don't need to
rely in implicit serialization for the updaters.


From hch at infradead.org  Sun Feb 21 08:18:20 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sun, 21 Feb 2016 08:18:20 -0800
Subject: [PATCH-4.6 1/3] NVMe: Use simple ida interface
In-Reply-To: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
Message-ID: <20160221161820.GC21896@infradead.org>

On Thu, Feb 18, 2016 at 04:21:15PM -0700, Keith Busch wrote:
> This is an easier interface to use and removes one of the overloaded
> dev_list_lock uses.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> ---
> Feature was originally posted by Lee Duncan last October, but this missed
> being applied.

So is Lee's patch or yours?

Either way it looks fine to me:

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Sun Feb 21 08:18:35 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sun, 21 Feb 2016 08:18:35 -0800
Subject: [PATCH-4.6 2/3] NVMe: Code naming cleanup
In-Reply-To: <1455837677-23248-2-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
 <1455837677-23248-2-git-send-email-keith.busch@intel.com>
Message-ID: <20160221161835.GD21896@infradead.org>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Sun Feb 21 08:20:48 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Sun, 21 Feb 2016 08:20:48 -0800
Subject: [PATCH-4.6 3/3] NVMe: Don't allow unsupported flags
In-Reply-To: <1455837677-23248-3-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
 <1455837677-23248-3-git-send-email-keith.busch@intel.com>
Message-ID: <20160221162048.GE21896@infradead.org>

On Thu, Feb 18, 2016 at 04:21:17PM -0700, Keith Busch wrote:
> Alternatively I have a different patch that builds SGL's if the flags
> has it set and the device supports SGL. Any interest?
> 
> I didn't post it since the fast path only gets PRP-able scatter lists,
> and the additional logic to handle SGL's complicates handling the NVMe
> IO descriptor.

I don't think we should allow applications to control the SGL use.
There are a couple reasons for that in upcoming specs that I can't talk
about, but I think it should be common sense to not expose these sorts
of low level details to userspace.

>  drivers/nvme/host/core.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 33ad10d..d8c3a55 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -398,7 +398,7 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
>  
>  	memset(&c, 0, sizeof(c));
>  	c.rw.opcode = io.opcode;
> -	c.rw.flags = io.flags;
> +	c.rw.flags = 0;

Shouldn't we check that flags is zero in the user structure and reject
the command if not?  Otherwise we can't safely add back support for
flags in the future if useful ones appear that we might want to support.


From sagig at dev.mellanox.co.il  Mon Feb 22 00:37:20 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:37:20 +0200
Subject: [PATCH 1/3] nvme: use a work item to submit async event requests
In-Reply-To: <1455732194-30790-2-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-2-git-send-email-hch@lst.de>
Message-ID: <56CAC8C0.8060405@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Mon Feb 22 00:38:46 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:38:46 +0200
Subject: [PATCH 2/3] nvme: don't poll the CQ from the kthread
In-Reply-To: <1455732194-30790-3-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-3-git-send-email-hch@lst.de>
Message-ID: <56CAC916.5020805@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Mon Feb 22 00:43:34 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:43:34 +0200
Subject: [PATCH 3/3] nvme: replace the kthread with a per-device watchdog
 timer
In-Reply-To: <1455732194-30790-4-git-send-email-hch@lst.de>
References: <1455732194-30790-1-git-send-email-hch@lst.de>
 <1455732194-30790-4-git-send-email-hch@lst.de>
Message-ID: <56CACA36.6030408@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Mon Feb 22 00:44:09 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:44:09 +0200
Subject: [PATCH 2/2] NVMe: NULL pointer check for admin queues
In-Reply-To: <1455231626-7166-2-git-send-email-keith.busch@intel.com>
References: <1455231626-7166-1-git-send-email-keith.busch@intel.com>
 <1455231626-7166-2-git-send-email-keith.busch@intel.com>
Message-ID: <56CACA59.1020201@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Mon Feb 22 00:45:01 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:45:01 +0200
Subject: [PATCH] NVMe: Expose ns wwid through single sysfs entry
In-Reply-To: <1455814668-784-1-git-send-email-keith.busch@intel.com>
References: <1455814668-784-1-git-send-email-keith.busch@intel.com>
Message-ID: <56CACA8D.2070803@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Mon Feb 22 00:54:05 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:54:05 +0200
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
In-Reply-To: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
References: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
Message-ID: <56CACCAD.7030402@dev.mellanox.co.il>

This looks good to me,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Mon Feb 22 00:56:56 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:56:56 +0200
Subject: [PATCH-4.6 1/3] NVMe: Use simple ida interface
In-Reply-To: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
Message-ID: <56CACD58.6090108@dev.mellanox.co.il>

Looks good to me,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From sagig at dev.mellanox.co.il  Mon Feb 22 00:57:24 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Mon, 22 Feb 2016 10:57:24 +0200
Subject: [PATCH-4.6 2/3] NVMe: Code naming cleanup
In-Reply-To: <1455837677-23248-2-git-send-email-keith.busch@intel.com>
References: <1455837677-23248-1-git-send-email-keith.busch@intel.com>
 <1455837677-23248-2-git-send-email-keith.busch@intel.com>
Message-ID: <56CACD74.6040906@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From keith.busch at intel.com  Tue Feb 23 08:36:13 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 23 Feb 2016 09:36:13 -0700
Subject: [PATCH] NVMe: Fix 0-length integrity payload
Message-ID: <1456245373-17398-1-git-send-email-keith.busch@intel.com>

A cheeky user could send a passthrough IO command with a metadata pointer,
but on a namespace without metadata. With metadata length of 0, kmalloc
returns ZERO_SIZE_PTR. Since that is not NULL, the driver would have
set this as the bio's integrity payload, which causes an access fault
on completion.

This patch ignores the users metadata buffer if the namespace format
does not support separate metadata. This is preferred over returning an
invalid error to work with existing user space applications.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reported-by: Stephen Bates <stephen.bates at microsemi.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d8c3a55..5340352 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -184,7 +184,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 			goto out_unmap;
 		}
 
-		if (meta_buffer) {
+		if (meta_buffer && meta_len) {
 			struct bio_integrity_payload *bip;
 
 			meta = kmalloc(meta_len, GFP_KERNEL);
-- 
2.6.2.307.g37023ba



From sagig at dev.mellanox.co.il  Tue Feb 23 09:11:52 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Tue, 23 Feb 2016 19:11:52 +0200
Subject: [PATCH] NVMe: Fix 0-length integrity payload
In-Reply-To: <1456245373-17398-1-git-send-email-keith.busch@intel.com>
References: <1456245373-17398-1-git-send-email-keith.busch@intel.com>
Message-ID: <56CC92D8.4060204@dev.mellanox.co.il>


> A cheeky user could send a passthrough IO command with a metadata pointer,
> but on a namespace without metadata. With metadata length of 0, kmalloc
> returns ZERO_SIZE_PTR. Since that is not NULL, the driver would have
> set this as the bio's integrity payload, which causes an access fault
> on completion.
>
> This patch ignores the users metadata buffer if the namespace format
> does not support separate metadata. This is preferred over returning an
> invalid error to work with existing user space applications.

So having user-space get it wrong forever is preferable? Although I
assume that this is the equivalent of a strip operation which is
perfectly valid. The one difference is that strip validates the meta
data by definition...

If you want to be extra pedantic, you can verify that the meta-data is
still valid, but given that the data didn't travel the pci I don't know
if it makes real sense...

So all-in-all this looks fine,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>

CC'ing mkp


From keith.busch at intel.com  Tue Feb 23 09:32:31 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 23 Feb 2016 17:32:31 +0000
Subject: [PATCH] NVMe: Fix 0-length integrity payload
In-Reply-To: <56CC92D8.4060204@dev.mellanox.co.il>
References: <1456245373-17398-1-git-send-email-keith.busch@intel.com>
 <56CC92D8.4060204@dev.mellanox.co.il>
Message-ID: <20160223173231.GC21865@localhost.localdomain>

On Tue, Feb 23, 2016 at 07:11:52PM +0200, Sagi Grimberg wrote:
> 
> >A cheeky user could send a passthrough IO command with a metadata pointer,
> >but on a namespace without metadata. With metadata length of 0, kmalloc
> >returns ZERO_SIZE_PTR. Since that is not NULL, the driver would have
> >set this as the bio's integrity payload, which causes an access fault
> >on completion.
> >
> >This patch ignores the users metadata buffer if the namespace format
> >does not support separate metadata. This is preferred over returning an
> >invalid error to work with existing user space applications.
> 
> So having user-space get it wrong forever is preferable? Although I
> assume that this is the equivalent of a strip operation which is
> perfectly valid. The one difference is that strip validates the meta
> data by definition...

Yeah, some have had it wrong for years, but it worked from their
perspective. Now it crashes the machine, so this just reinstates the
previously established behavior. That seemed better than forcing users to
fix their apps. They don't like "fixing" in reaction to kernel changes.
Though I'd argue their command was nonsense in the first place, and
probably doesn't do what they think they're doing.


From keith.busch at intel.com  Tue Feb 23 14:56:36 2016
From: keith.busch at intel.com (Keith Busch)
Date: Tue, 23 Feb 2016 22:56:36 +0000
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
In-Reply-To: <20160221161249.GA21099@infradead.org>
References: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
 <20160221161249.GA21099@infradead.org>
Message-ID: <20160223225635.GB20250@localhost.localdomain>

On Sun, Feb 21, 2016 at 08:12:49AM -0800, Christoph Hellwig wrote:
> On Thu, Feb 18, 2016 at 11:18:05AM -0700, Keith Busch wrote:
> > A namespace may be detached from a controller, but a user may be holding
> > a reference to it. Attaching a new namespace with the same NSID will create
> > duplicate names when using the NSID to name the disk.
> 
> This looks sensible, but I fear we might be running into the same
> issue with the backing_device embedded in the request_queue.

I don't think I follow. On rescanning, a new namespace with the same
NSID as a previously deleted one will allocate a new request_queue.


From hch at infradead.org  Wed Feb 24 00:35:46 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2016 00:35:46 -0800
Subject: [PATCH] NVMe: Fix 0-length integrity payload
In-Reply-To: <1456245373-17398-1-git-send-email-keith.busch@intel.com>
References: <1456245373-17398-1-git-send-email-keith.busch@intel.com>
Message-ID: <20160224083546.GA12620@infradead.org>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Wed Feb 24 05:05:20 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2016 05:05:20 -0800
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
In-Reply-To: <20160223225635.GB20250@localhost.localdomain>
References: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
 <20160221161249.GA21099@infradead.org>
 <20160223225635.GB20250@localhost.localdomain>
Message-ID: <20160224130520.GA17955@infradead.org>

On Tue, Feb 23, 2016 at 10:56:36PM +0000, Keith Busch wrote:
> > This looks sensible, but I fear we might be running into the same
> > issue with the backing_device embedded in the request_queue.
> 
> I don't think I follow. On rescanning, a new namespace with the same
> NSID as a previously deleted one will allocate a new request_queue.

But the bdi name embedds the major/minor of the device.  So if we
haven't fully torn down the device we might run into sysfs name
conflicts.


From keith.busch at intel.com  Wed Feb 24 06:58:36 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 14:58:36 +0000
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
In-Reply-To: <20160224130520.GA17955@infradead.org>
References: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
 <20160221161249.GA21099@infradead.org>
 <20160223225635.GB20250@localhost.localdomain>
 <20160224130520.GA17955@infradead.org>
Message-ID: <20160224145835.GA7494@localhost.localdomain>

On Wed, Feb 24, 2016 at 05:05:20AM -0800, Christoph Hellwig wrote:
> But the bdi name embedds the major/minor of the device.  So if we
> haven't fully torn down the device we might run into sysfs name
> conflicts.

We don't tear down the minor until the final put_disk, fixing the
major/minor lifetime issue. That was commit:

  https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2da78092dda13f1efd26edbbf99a567776913750

The namespace's disk_name using the IDA is released in the same path as
the final put_disk, so I think we're safe.


From keith.busch at intel.com  Wed Feb 24 08:15:56 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:56 -0700
Subject: [PATCHv4-4.5 5/7] NVMe: Move error handling to failed reset handler
In-Reply-To: <1456330558-819-1-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
Message-ID: <1456330558-819-6-git-send-email-keith.busch@intel.com>

This moves failed queue handling out of the namespace removal path and
into the reset failure path, fixing a hanging condition if the controller
fails or link down during del_gendisk. Previously the driver had to see
the controller as degraded prior to calling del_gendisk to setup the
queues to fail. But, if the controller happened to fail after this,
there was no task to end outstanding requests.

On failure, all namespace states are set to dead. This has capacity
revalidate to 0, and ends all new requests with error status.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
v3->v4:

  Set the BLK_QUEUE_RQ_ERROR when no namespace is available, as was the
  original intention.

  Restart the admin h/w queue to flush potentially pending requests to
  their completion.

 drivers/nvme/host/core.c | 53 ++++++++++++++++++++++++++++++++----------------
 drivers/nvme/host/nvme.h |  2 ++
 drivers/nvme/host/pci.c  | 13 +++++++++++-
 3 files changed, 50 insertions(+), 18 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 08b748e..d08b7e4 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -557,6 +557,10 @@ static int nvme_revalidate_disk(struct gendisk *disk)
 	u16 old_ms;
 	unsigned short bs;
 
+	if (test_bit(NVME_NS_DEAD, &ns->flags)) {
+		set_capacity(disk, 0);
+		return -ENODEV;
+	}
 	if (nvme_identify_ns(ns->ctrl, ns->ns_id, &id)) {
 		dev_warn(ns->ctrl->dev, "%s: Identify failure nvme%dn%d\n",
 				__func__, ns->ctrl->instance, ns->ns_id);
@@ -1186,32 +1190,15 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)
 {
-	bool kill;
-
 	if (test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
 		return;
 
-	kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
-	if (kill) {
-		blk_set_queue_dying(ns->queue);
-
-		/*
-		 * The controller was shutdown first if we got here through
-		 * device removal. The shutdown may requeue outstanding
-		 * requests. These need to be aborted immediately so
-		 * del_gendisk doesn't block indefinitely for their completion.
-		 */
-		blk_mq_abort_requeue_list(ns->queue);
-	}
 	if (ns->disk->flags & GENHD_FL_UP) {
 		if (blk_get_integrity(ns->disk))
 			blk_integrity_unregister(ns->disk);
 		sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
 					&nvme_ns_attr_group);
 		del_gendisk(ns->disk);
-	}
-	if (kill || !blk_queue_dying(ns->queue)) {
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
@@ -1414,6 +1401,38 @@ out:
 	return ret;
 }
 
+/**
+ * nvme_kill_queues(): Ends all namespace queues
+ * @ctrl: the dead controller that needs to end
+ *
+ * Call this function when the driver determines it is unable to get the
+ * controller in a state capable of servicing IO.
+ */
+void nvme_kill_queues(struct nvme_ctrl *ctrl)
+{
+	struct nvme_ns *ns;
+
+	mutex_lock(&ctrl->namespaces_mutex);
+	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (!kref_get_unless_zero(&ns->kref))
+			continue;
+
+		/*
+		 * Revalidating a dead namespace sets capacity to 0. This will
+		 * end buffered writers dirtying pages that can't be synced.
+		 */
+		if (!test_and_set_bit(NVME_NS_DEAD, &ns->flags))
+			revalidate_disk(ns->disk);
+
+		blk_set_queue_dying(ns->queue);
+		blk_mq_abort_requeue_list(ns->queue);
+		blk_mq_start_stopped_hw_queues(ns->queue, true);
+
+		nvme_put_ns(ns);
+	}
+	mutex_unlock(&ctrl->namespaces_mutex);
+}
+
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
 	struct nvme_ns *ns;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 4075fa9..fb15ba5 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -117,6 +117,7 @@ struct nvme_ns {
 	unsigned long flags;
 
 #define NVME_NS_REMOVING 0
+#define NVME_NS_DEAD     1
 
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
@@ -246,6 +247,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl);
 void nvme_start_queues(struct nvme_ctrl *ctrl);
+void nvme_kill_queues(struct nvme_ctrl *ctrl);
 
 struct request *nvme_alloc_request(struct request_queue *q,
 		struct nvme_command *cmd, unsigned int flags);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 6d2e425..680f578 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -690,7 +690,10 @@ static int nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
 
 	spin_lock_irq(&nvmeq->q_lock);
 	if (unlikely(nvmeq->cq_vector < 0)) {
-		ret = BLK_MQ_RQ_QUEUE_BUSY;
+		if (ns && !test_bit(NVME_NS_DEAD, &ns->flags))
+			ret = BLK_MQ_RQ_QUEUE_BUSY;
+		else
+			ret = BLK_MQ_RQ_QUEUE_ERROR;
 		spin_unlock_irq(&nvmeq->q_lock);
 		goto out;
 	}
@@ -1261,6 +1264,12 @@ static struct blk_mq_ops nvme_mq_ops = {
 static void nvme_dev_remove_admin(struct nvme_dev *dev)
 {
 	if (dev->ctrl.admin_q && !blk_queue_dying(dev->ctrl.admin_q)) {
+		/*
+		 * If the controller was reset during removal, it's possible
+		 * user requests may be waiting on a stopped queue. Start the
+		 * queue to flush these to completion.
+		 */
+		blk_mq_start_stopped_hw_queues(dev->ctrl.admin_q, true);
 		blk_cleanup_queue(dev->ctrl.admin_q);
 		blk_mq_free_tag_set(&dev->admin_tagset);
 	}
@@ -1901,6 +1910,7 @@ static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
 	dev_warn(dev->dev, "Removing after probe failure status: %d\n", status);
 
 	kref_get(&dev->ctrl.kref);
+	nvme_dev_disable(dev, false);
 	if (!schedule_work(&dev->remove_work))
 		nvme_put_ctrl(&dev->ctrl);
 }
@@ -1973,6 +1983,7 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, remove_work);
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
+	nvme_kill_queues(&dev->ctrl);
 	if (pci_get_drvdata(pdev))
 		pci_stop_and_remove_bus_device_locked(pdev);
 	nvme_put_ctrl(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 24 08:15:57 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:57 -0700
Subject: [PATCHv4-4.5 6/7] NVMe: Don't allow unsupported flags
In-Reply-To: <1456330558-819-1-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
Message-ID: <1456330558-819-7-git-send-email-keith.busch@intel.com>

The command flags can change the meaning of other fields in the command
that the driver is not prepared to handle. Specifically, the user could
passthrough an SGL flag, causing the controller to misinterpret the PRP
list the driver created, potentially corrupting memory or data.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Jon Derrick <jonathan.derrick at intel.com>
---
>From previous version:

  Return an error if the user requested an unsupported flag.

 drivers/nvme/host/core.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d08b7e4..9011685 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -374,6 +374,8 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
 
 	if (copy_from_user(&io, uio, sizeof(io)))
 		return -EFAULT;
+	if (io.flags)
+		return -EINVAL;
 
 	switch (io.opcode) {
 	case nvme_cmd_write:
@@ -425,6 +427,8 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 		return -EACCES;
 	if (copy_from_user(&cmd, ucmd, sizeof(cmd)))
 		return -EFAULT;
+	if (cmd.flags)
+		return -EINVAL;
 
 	memset(&c, 0, sizeof(c));
 	c.common.opcode = cmd.opcode;
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 24 08:15:52 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:52 -0700
Subject: [PATCHv4-4.5 1/7] NVMe: Don't unmap controller registers on reset
In-Reply-To: <1456330558-819-1-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
Message-ID: <1456330558-819-2-git-send-email-keith.busch@intel.com>

Unmapping the registers on reset or shutdown is not necessary. Keeping
the mapping simplifies reset handling.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 71 +++++++++++++++++++++++++++++--------------------
 1 file changed, 42 insertions(+), 29 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index a128672..2ea3e39 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1694,10 +1694,10 @@ static int nvme_dev_add(struct nvme_dev *dev)
 	return 0;
 }
 
-static int nvme_dev_map(struct nvme_dev *dev)
+static int nvme_pci_enable(struct nvme_dev *dev)
 {
 	u64 cap;
-	int bars, result = -ENOMEM;
+	int result = -ENOMEM;
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
 	if (pci_enable_device_mem(pdev))
@@ -1705,24 +1705,14 @@ static int nvme_dev_map(struct nvme_dev *dev)
 
 	dev->entry[0].vector = pdev->irq;
 	pci_set_master(pdev);
-	bars = pci_select_bars(pdev, IORESOURCE_MEM);
-	if (!bars)
-		goto disable_pci;
-
-	if (pci_request_selected_regions(pdev, bars, "nvme"))
-		goto disable_pci;
 
 	if (dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(64)) &&
 	    dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(32)))
 		goto disable;
 
-	dev->bar = ioremap(pci_resource_start(pdev, 0), 8192);
-	if (!dev->bar)
-		goto disable;
-
 	if (readl(dev->bar + NVME_REG_CSTS) == -1) {
 		result = -ENODEV;
-		goto unmap;
+		goto disable;
 	}
 
 	/*
@@ -1732,7 +1722,7 @@ static int nvme_dev_map(struct nvme_dev *dev)
 	if (!pdev->irq) {
 		result = pci_enable_msix(pdev, dev->entry, 1);
 		if (result < 0)
-			goto unmap;
+			goto disable;
 	}
 
 	cap = lo_hi_readq(dev->bar + NVME_REG_CAP);
@@ -1759,18 +1749,20 @@ static int nvme_dev_map(struct nvme_dev *dev)
 	pci_save_state(pdev);
 	return 0;
 
- unmap:
-	iounmap(dev->bar);
-	dev->bar = NULL;
  disable:
-	pci_release_regions(pdev);
- disable_pci:
 	pci_disable_device(pdev);
 	return result;
 }
 
 static void nvme_dev_unmap(struct nvme_dev *dev)
 {
+	if (dev->bar)
+		iounmap(dev->bar);
+	pci_release_regions(to_pci_dev(dev->dev));
+}
+
+static void nvme_pci_disable(struct nvme_dev *dev)
+{
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
 
 	if (pdev->msi_enabled)
@@ -1778,12 +1770,6 @@ static void nvme_dev_unmap(struct nvme_dev *dev)
 	else if (pdev->msix_enabled)
 		pci_disable_msix(pdev);
 
-	if (dev->bar) {
-		iounmap(dev->bar);
-		dev->bar = NULL;
-		pci_release_regions(pdev);
-	}
-
 	if (pci_is_enabled(pdev)) {
 		pci_disable_pcie_error_reporting(pdev);
 		pci_disable_device(pdev);
@@ -1842,7 +1828,7 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 	nvme_dev_list_remove(dev);
 
 	mutex_lock(&dev->shutdown_lock);
-	if (dev->bar) {
+	if (pci_is_enabled(to_pci_dev(dev->dev))) {
 		nvme_stop_queues(&dev->ctrl);
 		csts = readl(dev->bar + NVME_REG_CSTS);
 	}
@@ -1855,7 +1841,7 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 		nvme_disable_io_queues(dev);
 		nvme_disable_admin_queue(dev, shutdown);
 	}
-	nvme_dev_unmap(dev);
+	nvme_pci_disable(dev);
 
 	for (i = dev->queue_count - 1; i >= 0; i--)
 		nvme_clear_queue(dev->queues[i]);
@@ -1911,12 +1897,12 @@ static void nvme_reset_work(struct work_struct *work)
 	 * If we're called to reset a live controller first shut it down before
 	 * moving on.
 	 */
-	if (dev->bar)
+	if (dev->ctrl.ctrl_config & NVME_CC_ENABLE)
 		nvme_dev_disable(dev, false);
 
 	set_bit(NVME_CTRL_RESETTING, &dev->flags);
 
-	result = nvme_dev_map(dev);
+	result = nvme_pci_enable(dev);
 	if (result)
 		goto out;
 
@@ -2042,6 +2028,27 @@ static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = {
 	.free_ctrl		= nvme_pci_free_ctrl,
 };
 
+static int nvme_dev_map(struct nvme_dev *dev)
+{
+	int bars;
+	struct pci_dev *pdev = to_pci_dev(dev->dev);
+
+	bars = pci_select_bars(pdev, IORESOURCE_MEM);
+	if (!bars)
+		return -ENODEV;
+	if (pci_request_selected_regions(pdev, bars, "nvme"))
+		return -ENODEV;
+
+	dev->bar = ioremap(pci_resource_start(pdev, 0), 8192);
+	if (!dev->bar)
+		goto release;
+
+       return 0;
+  release:
+       pci_release_regions(pdev);
+       return -ENODEV;
+}
+
 static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
 	int node, result = -ENOMEM;
@@ -2066,6 +2073,10 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	dev->dev = get_device(&pdev->dev);
 	pci_set_drvdata(pdev, dev);
 
+	result = nvme_dev_map(dev);
+	if (result)
+		goto free;
+
 	INIT_LIST_HEAD(&dev->node);
 	INIT_WORK(&dev->scan_work, nvme_dev_scan);
 	INIT_WORK(&dev->reset_work, nvme_reset_work);
@@ -2089,6 +2100,7 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	nvme_release_prp_pools(dev);
  put_pci:
 	put_device(dev->dev);
+	nvme_dev_unmap(dev);
  free:
 	kfree(dev->queues);
 	kfree(dev->entry);
@@ -2126,6 +2138,7 @@ static void nvme_remove(struct pci_dev *pdev)
 	nvme_free_queues(dev, 0);
 	nvme_release_cmb(dev);
 	nvme_release_prp_pools(dev);
+	nvme_dev_unmap(dev);
 	nvme_put_ctrl(&dev->ctrl);
 }
 
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 24 08:15:54 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:54 -0700
Subject: [PATCHv4-4.5 3/7] NVMe: Fix namespace removal deadlock
In-Reply-To: <1456330558-819-1-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
Message-ID: <1456330558-819-4-git-send-email-keith.busch@intel.com>

This patch makes nvme namespace removal lockless. It is up to the caller
to ensure no active namespace scanning is occuring. To ensure no scan
work occurs, the nvme pci driver adds a removing state to the controller
device to avoid queueing scan work during removal. The work is flushed
after setting the state, so no new scan work can be queued.

The lockless removal allows the driver to cleanup a namespace
request_queue if the controller fails during removal. Previously this
could deadlock trying to acquire the namespace mutex in order to handle
such events.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
v3->v4:

  This combines two patches from the previous series into a single commit
  as it these two make a single feature.

  The test for controller removing is done prior to queueing scan work
  instead of in the scan work itself, and adds a code comment explaining
  its purpose.

 drivers/nvme/host/core.c | 12 +++++++-----
 drivers/nvme/host/nvme.h |  4 ++++
 drivers/nvme/host/pci.c  | 17 +++++++++++++++--
 3 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4b54081..08b748e 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1186,11 +1186,13 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 static void nvme_ns_remove(struct nvme_ns *ns)
 {
-	bool kill = nvme_io_incapable(ns->ctrl) &&
-			!blk_queue_dying(ns->queue);
+	bool kill;
 
-	lockdep_assert_held(&ns->ctrl->namespaces_mutex);
+	if (test_and_set_bit(NVME_NS_REMOVING, &ns->flags))
+		return;
 
+	kill = nvme_io_incapable(ns->ctrl) &&
+			!blk_queue_dying(ns->queue);
 	if (kill) {
 		blk_set_queue_dying(ns->queue);
 
@@ -1213,7 +1215,9 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 		blk_mq_abort_requeue_list(ns->queue);
 		blk_cleanup_queue(ns->queue);
 	}
+	mutex_lock(&ns->ctrl->namespaces_mutex);
 	list_del_init(&ns->list);
+	mutex_unlock(&ns->ctrl->namespaces_mutex);
 	nvme_put_ns(ns);
 }
 
@@ -1308,10 +1312,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 {
 	struct nvme_ns *ns, *next;
 
-	mutex_lock(&ctrl->namespaces_mutex);
 	list_for_each_entry_safe(ns, next, &ctrl->namespaces, list)
 		nvme_ns_remove(ns);
-	mutex_unlock(&ctrl->namespaces_mutex);
 }
 
 static DEFINE_IDA(nvme_instance_ida);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9407f2f..4075fa9 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -114,6 +114,10 @@ struct nvme_ns {
 	bool ext;
 	u8 pi_type;
 	int type;
+	unsigned long flags;
+
+#define NVME_NS_REMOVING 0
+
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
 };
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 2ea3e39..122f803 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -120,6 +120,7 @@ struct nvme_dev {
 	unsigned long flags;
 
 #define NVME_CTRL_RESETTING    0
+#define NVME_CTRL_REMOVING     1
 
 	struct nvme_ctrl ctrl;
 	struct completion ioq_wait;
@@ -286,6 +287,17 @@ static int nvme_init_request(void *data, struct request *req,
 	return 0;
 }
 
+static void nvme_queue_scan(struct nvme_dev *dev)
+{
+	/*
+	 * Do not queue new scan work when a controller is reset during
+	 * removal.
+	 */
+	if (test_bit(NVME_CTRL_REMOVING, &dev->flags))
+		return;
+	queue_work(nvme_workq, &dev->scan_work);
+}
+
 static void nvme_complete_async_event(struct nvme_dev *dev,
 		struct nvme_completion *cqe)
 {
@@ -300,7 +312,7 @@ static void nvme_complete_async_event(struct nvme_dev *dev,
 	switch (result & 0xff07) {
 	case NVME_AER_NOTICE_NS_CHANGED:
 		dev_info(dev->dev, "rescanning\n");
-		queue_work(nvme_workq, &dev->scan_work);
+		nvme_queue_scan(dev);
 	default:
 		dev_warn(dev->dev, "async event result %08x\n", result);
 	}
@@ -1690,7 +1702,7 @@ static int nvme_dev_add(struct nvme_dev *dev)
 			return 0;
 		dev->ctrl.tagset = &dev->tagset;
 	}
-	queue_work(nvme_workq, &dev->scan_work);
+	nvme_queue_scan(dev);
 	return 0;
 }
 
@@ -2128,6 +2140,7 @@ static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
 
+	set_bit(NVME_CTRL_REMOVING, &dev->flags);
 	pci_set_drvdata(pdev, NULL);
 	flush_work(&dev->scan_work);
 	nvme_remove_namespaces(&dev->ctrl);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 24 08:15:55 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:55 -0700
Subject: [PATCHv4-4.5 4/7] NVMe: Simplify device reset failure
In-Reply-To: <1456330558-819-1-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
Message-ID: <1456330558-819-5-git-send-email-keith.busch@intel.com>

A reset failure schedules the device to unbind from the driver through
the pci driver's remove. This cleans up all intialization, so there is
no need to duplicate the potentially racy cleanup.

To help understand why a reset failed, the status is logged with the
existing warning message.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
Reviewed-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 48 +++++++++++++++++++++---------------------------
 1 file changed, 21 insertions(+), 27 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 122f803..6d2e425 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -86,7 +86,6 @@ struct nvme_queue;
 
 static int nvme_reset(struct nvme_dev *dev);
 static void nvme_process_cq(struct nvme_queue *nvmeq);
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev);
 static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown);
 
 /*
@@ -1897,10 +1896,19 @@ static void nvme_pci_free_ctrl(struct nvme_ctrl *ctrl)
 	kfree(dev);
 }
 
+static void nvme_remove_dead_ctrl(struct nvme_dev *dev, int status)
+{
+	dev_warn(dev->dev, "Removing after probe failure status: %d\n", status);
+
+	kref_get(&dev->ctrl.kref);
+	if (!schedule_work(&dev->remove_work))
+		nvme_put_ctrl(&dev->ctrl);
+}
+
 static void nvme_reset_work(struct work_struct *work)
 {
 	struct nvme_dev *dev = container_of(work, struct nvme_dev, reset_work);
-	int result;
+	int result = -ENODEV;
 
 	if (WARN_ON(test_bit(NVME_CTRL_RESETTING, &dev->flags)))
 		goto out;
@@ -1920,26 +1928,26 @@ static void nvme_reset_work(struct work_struct *work)
 
 	result = nvme_configure_admin_queue(dev);
 	if (result)
-		goto unmap;
+		goto out;
 
 	nvme_init_queue(dev->queues[0], 0);
 	result = nvme_alloc_admin_tags(dev);
 	if (result)
-		goto disable;
+		goto out;
 
 	result = nvme_init_identify(&dev->ctrl);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	result = nvme_setup_io_queues(dev);
 	if (result)
-		goto free_tags;
+		goto out;
 
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
 
 	result = nvme_dev_list_add(dev);
 	if (result)
-		goto remove;
+		goto out;
 
 	/*
 	 * Keep the controller around but remove all namespaces if we don't have
@@ -1956,19 +1964,8 @@ static void nvme_reset_work(struct work_struct *work)
 	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
 	return;
 
- remove:
-	nvme_dev_list_remove(dev);
- free_tags:
-	nvme_dev_remove_admin(dev);
-	blk_put_queue(dev->ctrl.admin_q);
-	dev->ctrl.admin_q = NULL;
-	dev->queues[0]->tags = NULL;
- disable:
-	nvme_disable_admin_queue(dev, false);
- unmap:
-	nvme_dev_unmap(dev);
  out:
-	nvme_remove_dead_ctrl(dev);
+	nvme_remove_dead_ctrl(dev, result);
 }
 
 static void nvme_remove_dead_ctrl_work(struct work_struct *work)
@@ -1981,14 +1978,6 @@ static void nvme_remove_dead_ctrl_work(struct work_struct *work)
 	nvme_put_ctrl(&dev->ctrl);
 }
 
-static void nvme_remove_dead_ctrl(struct nvme_dev *dev)
-{
-	dev_warn(dev->dev, "Removing after probe failure\n");
-	kref_get(&dev->ctrl.kref);
-	if (!schedule_work(&dev->remove_work))
-		nvme_put_ctrl(&dev->ctrl);
-}
-
 static int nvme_reset(struct nvme_dev *dev)
 {
 	if (!dev->ctrl.admin_q || blk_queue_dying(dev->ctrl.admin_q))
@@ -2136,6 +2125,11 @@ static void nvme_shutdown(struct pci_dev *pdev)
 	nvme_dev_disable(dev, true);
 }
 
+/*
+ * The driver's remove may be called on a device in a partially initialized
+ * state. This function must not have any dependencies on the device state in
+ * order to proceed.
+ */
 static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 24 08:15:51 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:51 -0700
Subject: [PATCHv4-4.5 0/7] Rebased fixes
Message-ID: <1456330558-819-1-git-send-email-keith.busch@intel.com>

This rebases the hot plug fixes to the current linux-block/for-linus,
folds in the review comments, a couple minor fixes (described in the
individual patches), reorders the series to minimize churn, and squashes
two previous patches for "namespace removal deadlock"; the two separate
patches weren't really useful on their own.

The last two patches are not hot-plug related, but fix kernel crashes
from user misuse/abuse of the passthru interface. These were already
posted separatedly, but they are included in this series to complete
everything I believe includes all important fixes I know about for 4.5.

Keith Busch (7):
  NVMe: Don't unmap controller registers on reset
  NVMe: Use IDA for namespace disk naming
  NVMe: Fix namespace removal deadlock
  NVMe: Simplify device reset failure
  NVMe: Move error handling to failed reset handler
  NVMe: Don't allow unsupported flags
  NVMe: Fix 0-length integrity payload

 drivers/nvme/host/core.c |  81 ++++++++++++++++++--------
 drivers/nvme/host/nvme.h |   8 +++
 drivers/nvme/host/pci.c  | 149 ++++++++++++++++++++++++++++-------------------
 3 files changed, 156 insertions(+), 82 deletions(-)

-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 24 08:15:58 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:58 -0700
Subject: [PATCHv4-4.5 7/7] NVMe: Fix 0-length integrity payload
In-Reply-To: <1456330558-819-1-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
Message-ID: <1456330558-819-8-git-send-email-keith.busch@intel.com>

A user could send a passthrough IO command with a metadata pointer to a
namespace without metadata. With metadata length of 0, kmalloc returns
ZERO_SIZE_PTR. Since that is not NULL, the driver would have set this as
the bio's integrity payload, which causes an access fault on completion.

This patch ignores the users metadata buffer if the namespace format
does not support separate metadata.

Reported-by: Stephen Bates <stephen.bates at microsemi.com>
Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 9011685..5455ca0 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -184,7 +184,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 			goto out_unmap;
 		}
 
-		if (meta_buffer) {
+		if (meta_buffer && meta_len) {
 			struct bio_integrity_payload *bip;
 
 			meta = kmalloc(meta_len, GFP_KERNEL);
-- 
2.6.2.307.g37023ba



From keith.busch at intel.com  Wed Feb 24 08:15:53 2016
From: keith.busch at intel.com (Keith Busch)
Date: Wed, 24 Feb 2016 09:15:53 -0700
Subject: [PATCHv4-4.5 2/7] NVMe: Use IDA for namespace disk naming
In-Reply-To: <1456330558-819-1-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
Message-ID: <1456330558-819-3-git-send-email-keith.busch@intel.com>

A namespace may be detached from a controller, but a user may be holding
a reference to it. Attaching a new namespace with the same NSID will create
duplicate names when using the NSID to name the disk.

This patch uses an IDA that is released only when the last reference is
released instead of using the namespace ID.

Signed-off-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
---
Changes from previous:

  Delete the controller's IDA when the last controller reference is
  released rather than device is removed. This keeps it available for
  all namespace lifetime.

 drivers/nvme/host/core.c | 16 +++++++++++++---
 drivers/nvme/host/nvme.h |  2 ++
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 3cd921e..4b54081 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -55,8 +55,9 @@ static void nvme_free_ns(struct kref *kref)
 	ns->disk->private_data = NULL;
 	spin_unlock(&dev_list_lock);
 
-	nvme_put_ctrl(ns->ctrl);
 	put_disk(ns->disk);
+	ida_simple_remove(&ns->ctrl->ns_ida, ns->instance);
+	nvme_put_ctrl(ns->ctrl);
 	kfree(ns);
 }
 
@@ -1118,9 +1119,13 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	if (!ns)
 		return;
 
+	ns->instance = ida_simple_get(&ctrl->ns_ida, 1, 0, GFP_KERNEL);
+	if (ns->instance < 0)
+		goto out_free_ns;
+
 	ns->queue = blk_mq_init_queue(ctrl->tagset);
 	if (IS_ERR(ns->queue))
-		goto out_free_ns;
+		goto out_release_instance;
 	queue_flag_set_unlocked(QUEUE_FLAG_NONROT, ns->queue);
 	ns->queue->queuedata = ns;
 	ns->ctrl = ctrl;
@@ -1153,7 +1158,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	disk->queue = ns->queue;
 	disk->driverfs_dev = ctrl->device;
 	disk->flags = GENHD_FL_EXT_DEVT;
-	sprintf(disk->disk_name, "nvme%dn%d", ctrl->instance, nsid);
+	sprintf(disk->disk_name, "nvme%dn%d", ctrl->instance, ns->instance);
 
 	if (nvme_revalidate_disk(ns->disk))
 		goto out_free_disk;
@@ -1173,6 +1178,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	kfree(disk);
  out_free_queue:
 	blk_cleanup_queue(ns->queue);
+ out_release_instance:
+	ida_simple_remove(&ctrl->ns_ida, ns->instance);
  out_free_ns:
 	kfree(ns);
 }
@@ -1350,6 +1358,7 @@ static void nvme_free_ctrl(struct kref *kref)
 
 	put_device(ctrl->device);
 	nvme_release_instance(ctrl);
+	ida_destroy(&ctrl->ns_ida);
 
 	ctrl->ops->free_ctrl(ctrl);
 }
@@ -1390,6 +1399,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	}
 	get_device(ctrl->device);
 	dev_set_drvdata(ctrl->device, ctrl);
+	ida_init(&ctrl->ns_ida);
 
 	spin_lock(&dev_list_lock);
 	list_add_tail(&ctrl->node, &nvme_ctrl_list);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9664d07..9407f2f 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -72,6 +72,7 @@ struct nvme_ctrl {
 	struct mutex namespaces_mutex;
 	struct device *device;	/* char device */
 	struct list_head node;
+	struct ida ns_ida;
 
 	char name[12];
 	char serial[20];
@@ -102,6 +103,7 @@ struct nvme_ns {
 	struct request_queue *queue;
 	struct gendisk *disk;
 	struct kref kref;
+	int instance;
 
 	u8 eui[8];
 	u8 uuid[16];
-- 
2.6.2.307.g37023ba



From hch at infradead.org  Wed Feb 24 11:33:29 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2016 11:33:29 -0800
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
In-Reply-To: <20160224145835.GA7494@localhost.localdomain>
References: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
 <20160221161249.GA21099@infradead.org>
 <20160223225635.GB20250@localhost.localdomain>
 <20160224130520.GA17955@infradead.org>
 <20160224145835.GA7494@localhost.localdomain>
Message-ID: <20160224193329.GA25509@infradead.org>

On Wed, Feb 24, 2016 at 02:58:36PM +0000, Keith Busch wrote:
> On Wed, Feb 24, 2016 at 05:05:20AM -0800, Christoph Hellwig wrote:
> > But the bdi name embedds the major/minor of the device.  So if we
> > haven't fully torn down the device we might run into sysfs name
> > conflicts.
> 
> We don't tear down the minor until the final put_disk, fixing the
> major/minor lifetime issue. That was commit:
> 
>   https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2da78092dda13f1efd26edbbf99a567776913750
> 
> The namespace's disk_name using the IDA is released in the same path as
> the final put_disk, so I think we're safe.

Ok, we should be fine then:

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Wed Feb 24 11:33:58 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2016 11:33:58 -0800
Subject: [PATCHv4-4.5 2/7] NVMe: Use IDA for namespace disk naming
In-Reply-To: <1456330558-819-3-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
 <1456330558-819-3-git-send-email-keith.busch@intel.com>
Message-ID: <20160224193358.GB25509@infradead.org>

On Wed, Feb 24, 2016 at 09:15:53AM -0700, Keith Busch wrote:
> A namespace may be detached from a controller, but a user may be holding
> a reference to it. Attaching a new namespace with the same NSID will create
> duplicate names when using the NSID to name the disk.
> 
> This patch uses an IDA that is released only when the last reference is
> released instead of using the namespace ID.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Wed Feb 24 11:34:36 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2016 11:34:36 -0800
Subject: [PATCHv4-4.5 3/7] NVMe: Fix namespace removal deadlock
In-Reply-To: <1456330558-819-4-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
 <1456330558-819-4-git-send-email-keith.busch@intel.com>
Message-ID: <20160224193436.GC25509@infradead.org>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From hch at infradead.org  Wed Feb 24 11:35:00 2016
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2016 11:35:00 -0800
Subject: [PATCHv4-4.5 6/7] NVMe: Don't allow unsupported flags
In-Reply-To: <1456330558-819-7-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
 <1456330558-819-7-git-send-email-keith.busch@intel.com>
Message-ID: <20160224193500.GD25509@infradead.org>

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From jthumshirn at suse.de  Thu Feb 25 00:07:54 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 25 Feb 2016 09:07:54 +0100
Subject: [PATCHv4-4.5 2/7] NVMe: Use IDA for namespace disk naming
In-Reply-To: <1456330558-819-3-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
 <1456330558-819-3-git-send-email-keith.busch@intel.com>
Message-ID: <20160225080754.GB29716@c203.arch.suse.de>

On Wed, Feb 24, 2016 at 09:15:53AM -0700, Keith Busch wrote:
> A namespace may be detached from a controller, but a user may be holding
> a reference to it. Attaching a new namespace with the same NSID will create
> duplicate names when using the NSID to name the disk.
> 
> This patch uses an IDA that is released only when the last reference is
> released instead of using the namespace ID.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> Reviewed-by: Sagi Grimberg <sagig at mellanox.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 25 00:12:34 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 25 Feb 2016 09:12:34 +0100
Subject: [PATCHv4-4.5 6/7] NVMe: Don't allow unsupported flags
In-Reply-To: <1456330558-819-7-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
 <1456330558-819-7-git-send-email-keith.busch@intel.com>
Message-ID: <20160225081233.GC29716@c203.arch.suse.de>

On Wed, Feb 24, 2016 at 09:15:57AM -0700, Keith Busch wrote:
> The command flags can change the meaning of other fields in the command
> that the driver is not prepared to handle. Specifically, the user could
> passthrough an SGL flag, causing the controller to misinterpret the PRP
> list the driver created, potentially corrupting memory or data.
> 
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> Reviewed-by: Jon Derrick <jonathan.derrick at intel.com>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From jthumshirn at suse.de  Thu Feb 25 00:13:21 2016
From: jthumshirn at suse.de (Johannes Thumshirn)
Date: Thu, 25 Feb 2016 09:13:21 +0100
Subject: [PATCHv4-4.5 7/7] NVMe: Fix 0-length integrity payload
In-Reply-To: <1456330558-819-8-git-send-email-keith.busch@intel.com>
References: <1456330558-819-1-git-send-email-keith.busch@intel.com>
 <1456330558-819-8-git-send-email-keith.busch@intel.com>
Message-ID: <20160225081321.GD29716@c203.arch.suse.de>

On Wed, Feb 24, 2016 at 09:15:58AM -0700, Keith Busch wrote:
> A user could send a passthrough IO command with a metadata pointer to a
> namespace without metadata. With metadata length of 0, kmalloc returns
> ZERO_SIZE_PTR. Since that is not NULL, the driver would have set this as
> the bio's integrity payload, which causes an access fault on completion.
> 
> This patch ignores the users metadata buffer if the namespace format
> does not support separate metadata.
> 
> Reported-by: Stephen Bates <stephen.bates at microsemi.com>
> Signed-off-by: Keith Busch <keith.busch at intel.com>
> Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
> Reviewed-by: Christoph Hellwig <hch at lst.de>

Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>

-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850


From sagig at dev.mellanox.co.il  Thu Feb 25 00:55:08 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Thu, 25 Feb 2016 10:55:08 +0200
Subject: [PATCH] NVMe: Use IDA for namespace disk naming
In-Reply-To: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
References: <1455819485-3730-1-git-send-email-keith.busch@intel.com>
Message-ID: <56CEC16C.1050806@dev.mellanox.co.il>

Looks fine to me too,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From marc at merlins.org  Thu Feb 25 10:20:09 2016
From: marc at merlins.org (Marc MERLIN)
Date: Thu, 25 Feb 2016 10:20:09 -0800
Subject: M2/NVM vs M2/Sata vs SATA/6GB benchmarks
In-Reply-To: <20160214093258.GC2590@linux.intel.com>
References: <20160214031333.GA20879@merlins.org>
 <20160214093258.GC2590@linux.intel.com>
Message-ID: <20160225182009.GD2290@merlins.org>

On Sun, Feb 14, 2016 at 04:32:58AM -0500, Matthew Wilcox wrote:
> 
> Hi Marc,
> 
> I shan't have time to look at this for another week, but I've added
> the NVMe mailing list so somebody else can comment in the meantime. 
 
Hi Matthew :)

> On Sat, Feb 13, 2016 at 07:13:33PM -0800, Marc MERLIN wrote:
> > Hi Matthew,
> > 
> > I didn't find a list for NVMe, I'll Cc linux-ide in case this is useful
> > to others.
> > 
> > I just got a Thinkpad P70 skylake laptop and thought I'd try 3 SSDs:
> > NVMe/M2 vs Sata/M2 vs Sata/6GB.
> > I'm not a benchmarking filesystem expert, so my apologies if I made some
> > stupid choices, but hopefully the benchmarks are still useful.
> > 
> > I'll give the conclusions first. The buffer cache issue with NVMe is weird,
> > maybe there is a non default setting I need to tweak?
> > 
> > a) NVMe is faster for big contiguous I/O, even through dmcrypt
> > b) M2 NVMe and M2 sata do not seem to go through buffer cache for writes, but SATA/6GB does
> > c) When using buffer cache, SATA/6GB beats NVMe/M2 and SATA/M2 by a lot on block reads. Why?
> > d) for random I/O, NVMe is up to 50% faster, and faster with ext4 than btrfs
> > e) Encrypted NVMe is a bit faster than non encrypted Sata/M2
> > f) At $328 for 512GB M2/NVMe vs $168 for 500GB M2/Sata, unless you're doing big contiguous I/O, the 2x price difference is hard to justify, cached I/O and random I/O are not significantly faster on NVMe.
> > 
> > 
> > 
> > 3 SSDs:
> > 1) M2/SATA Samsung 850 EVO 500 GB M.2 3.5-Inch SSD (MZ-N5E500BW) 33c/GB
> > 
> > 2) M2/NVMe: Samsung 950 PRO -Series 512GB PCIe NVMe - M.2 Internal SSD 2-Inch MZ-V5P512BW 64c/GB
> > 70:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd Device a802 (rev 01)
> > 
> > 3) SATA6G: Samsung 850 EVO 2 TB 2.5-Inch SATA III Internal SSD (MZ-75E2T0B/AM) 30c/GB
> > 
> > For price comparison
> > Seagate 2TB Laptop HDD SATA 6Gb/s 32MB Cache 2.5-Inch Internal Bare Drive (ST2000LM003) 4.6c/GB
> > 
> > Laptop: Skylake Lenovo Thinkpad P70
> > Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz x 8 (4 cores HT)
> > 
> > Filesystem: btrfs on top of 
> > cryptsetup luksFormat --align-payload=8192 -s 256 -c aes-xts-plain64 /dev/device
> > ddd test: sync; dd if=/mnt/ram/file of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=100M
> > dd0 test: sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> > 
> > Summary: 
> > 0) NVMe only really looks faster on big block sequential read/write (indeed 4X read, 2X write).
> >    On Random I/O, it's no faster. Other overheads kill its performance advantage.
> > 1) For Cached I/O NVMe seems slowed down by dmcrypt in hdparm/ioz ranread/write whereas M2/Sata is not
> > 2) 2TB Sata 6GB SSD is faster for read/write encrypted than NVMe is non encrypted (except hdparm)
> > 3) iozone through block cache is faster on M2/SATA than M2/NVMe, weird
> > 4) iozone through block cache is much faster for everything on Sata6G, also weird
> > 5) iozone direct I/O shows NVMe is fastest (by 50% only) and external SATA just a bit faster than M2/Sata
> > 6) when using dmcrypt, NVMe is only marginally faster in iozone than Sata/M2 or Sata/6G
> > 7) kernel build speed is same on SATA/M2 and NVMe/M2, encrypted or not, but SATA6G is 10% faster (probably the same effect that block cache works better on SATA/6G than SATA/M2 or NVMe/M2)
> > 8) ext4 does better on iozone -I (directIO) than btrfs, 2x as fast on random read for NVMe
> > 
> > 
> > 
> > 		SATA/M2 dmcrypt	SATA/M2		NVMe dmcrypt	NVMe		SATA6GB/dmcrypt
> > hdparm -t	 534MB/s	 517MB/s	1307MB/s	2140MB/s	 534MB/s
> > ddd 10GB read	 550MB/s	 560MB/s	2000MB/s	2400MB/s	 550MB/s
> > ddd 10GB write	 500MB/s	 503MB/s	1300MB/s	1200MB/s	 506MB/s
> > 
> > dd with /dev/zero hits optimizations that actually penalize NVMe:
> > dd0 10GB read	   7GB/s	 6.7GB/s	 2.3GB/s	 4.4GB/s	 6.6GB/s
> > dd0 10GB write	 495MB/s	 503MB/s	 1.3GB/s	 1.3GB/s	 2.0GB/s
> > 
> > iozone -e -a -s 4096 -i 0 -i 1 -i 2 (cache makes Sata6G faster than M2, weird):
> > ioz read 4KB	1760MB/s	1790MB/s	 675MB/s	 766MB/s	7863MB/s
> > ioz write 4KB	 268MB/s	 298MB/s	 295MB/s	 291MB/s	 777MB/s
> > ioz ranread4KB	5535MB/s	7392MB/s	3185MB/s	7261MB/s	7892MB/s
> > ioz ranwrite4KB	 288MB/s	 329MB/s	 205MB/s	 381MB/s	 796MB/s
> > 
> > iozone -I -e -a -s 4096 -i 0 -i 1 -i 2 (direct IO):
> > 		SATA/M2 dmcrypt	 SATA/M2	 NVMe dmcrypt	 NVMe		 SATA6GB
> > 										 dmcrypt notcrypted
> > 		 btrfs		 btrfs   ext4	 btrfs		 btrfs   ext4	 btrfs     ext4
> > ioz read 4KB	 255MB/s	 303MB/s 408MB/s 303MB/s	 396MB/s 562MB/s 310MB/s 379MB/s
> > ioz write 4KB	 195MB/s	 238MB/s 344MB/s 257MB/s	 358MB/s 361MB/s 233MB/s 365MB/s
> > ioz ranread4KB	 305MB/s	 351MB/s 482MB/s 476MB/s	 699MB/s1434MB/s 309MB/s 475MB/s
> > ioz ranwrite4KB	 265MB/s	 223MB/s 392MB/s 260MB/s	 217MB/s 315MB/s 254MB/s 366MB/s
> > 
> > 
> > Details of each test, per type of drive;
> > ================================================================================
> > saruman M2 SATA 512GB encrypted btrfs
> > 
> > saruman:/tmp# hdparm -t /dev/mapper/cryptroot
> >  Timing buffered disk reads: 1604 MB in  3.00 seconds = 534.03 MB/sec
> > 
> > iozone -e -a -s 4096 -i 0 -i 1 -i 2:
> >                                                               random    random
> >               kB  reclen    write  rewrite    read    reread    read     write
> >             4096       4   257076   281089  1761711  6094733  6066754   290373
> >             4096     512   277108   295167  1588844  8426437 11128258   318731
> >             4096    4096   268940   308248  1760087  8062616  8377131   320802
> > 
> > iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
> >             4096       4    38626    45227   102236   111141    30936    64317
> >             4096     512   138467   128878   134674   137832   140067   126248
> >             4096    4096   195439   273100   255379   307300   305605   265093
> > 
> > 
> > saruman:/mnt/mnt# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> > 10485760000 bytes (10 GB) copied, 21.1761 s, 495 MB/s
> > 10485760000 bytes (10 GB) copied, 1.49373 s, 7.0 GB/s
> > 
> > kernel 4.4.1 make -j8:
> > real    17m54.712s
> > user    126m26.620s
> > sys     6m21.948s
> > 
> > btrfs send/receive encrypted partition to non encrypted, 87GB: 10mn30
> > 
> > saruman M2 SATA non encrypted btrfs
> > -----------------------------------
> > saruman:/mnt/mnt4# hdparm -t /dev/sdc4
> > /dev/sdc4:
> >  Timing buffered disk reads: 1554 MB in  3.00 seconds = 517.93 MB/sec
> > 
> > iozone -e -a -s 4096 -i 0 -i 1 -i 2:
> >                                                               random    random
> >               kB  reclen    write  rewrite    read    reread    read     write
> >             4096       4   285020   297565  1739944  7434579  6605595   305785
> >             4096     512   310937   336617  1298278  3703511  5483860   299416
> >             4096    4096   298433   317101  1790911  7392988  7434579   337476
> > iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
> >             4096       4    47219    93497   118900   126065    39160    86462
> >             4096     512   214381   213388   194307   209215   206951   209289
> >             4096    4096   238792   221839   303768   352951   351342   223618
> > ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
> >             4096       4   119524   146035   150173   159105    53090   140697
> >             4096     512   358954   354810   366466   423088   419760   375542
> >             4096    4096   344087   398828   408259   478337   482569   392487
> > 
> > saruman:/mnt/mnt3#  sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> > 10485760000 bytes (10 GB) copied, 20.8537 s, 503 MB/s
> > 10485760000 bytes (10 GB) copied, 1.55651 s, 6.7 GB/s
> > 
> > kernel 4.4.1 make -j8:
> > real    17m55.612s
> > user    126m31.952s
> > sys     6m27.452s
> > 
> > 
> > ********************************************************************************
> > saruman M2 NVMe 512GB encrypted btrfs
> > 
> > saruman:/tmp# hdparm -t /dev/mapper/cryptroot2
> > /dev/mapper/cryptroot2:
> >  Timing buffered disk reads: 3924 MB in  3.00 seconds = 1307.56 MB/sec
> > 
> > iozone -e -a -s 4096 -i 0 -i 1 -i 2:
> >                                                               random    random
> >               kB  reclen    write  rewrite    read    reread    read     write
> >             4096       4   284205   341651   742181  6195843  6178018   341101
> >             4096     512   314351   319573   864883  8865630  8902382   384993
> >             4096    4096   295741   201042   675234  3281854  3185110   205974
> > iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
> >             4096       4    80042    83160   122616   127129    41421    53963
> >             4096     512   168713   153333   167710   209923   198123   168026
> >             4096    4096   257968   258566   303023   490828   476440   260014
> > 
> > 
> > saruman:/mnt/mnt2# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> > 10485760000 bytes (10 GB) copied, 8.01494 s, 1.3 GB/s
> > 10485760000 bytes (10 GB) copied, 4.63397 s, 2.3 GB/s
> > 
> > kernel 4.4.1 make -j8:
> > real    17m57.513s
> > user    126m58.360s
> > sys     6m25.164s
> > 
> > btrfs send/receive encrypted partition to non encrypted, 87GB: 7mn
> > (33% faster than Sata M2)
> > 
> > saruman M2 NVMe non encrypted btrfs
> > -----------------------------------
> > saruman:/mnt/mnt4# hdparm -t /dev/nvme0n1p4
> > /dev/nvme0n1p4:
> >  Timing buffered disk reads: 6422 MB in  3.00 seconds = 2140.38 MB/sec
> > 
> > 
> > iozone -e -a -s 4096 -i 0 -i 1 -i 2:
> >                                                               random    random
> >               kB  reclen    write  rewrite    read    reread    read     write
> >             4096       4   274808   271851   684271  2634219  2280673   177630
> >             4096     512   290191   336755   668196  3362795  4039655   184779
> >             4096    4096   291427   340345   766588  7355007  7261741   381768
> > iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
> >             4096       4    94814   117116   147645   159695    44075   108039
> >             4096     512   271130   267800   239969   325933   301841   271062
> >             4096    4096   358602   275933   396591   719099   699939   217665
> > ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2:
> >             4096       4   107281   200735   205900   263035    54832   124219
> >             4096     512   537478   374641   525060  1173664  1139338   558807
> >             4096    4096   361545   526137   562632  1483477  1434069   315883
> > 
> > saruman:/mnt/mnt4# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> > 10485760000 bytes (10 GB) copied, 7.90653 s, 1.3 GB/s
> > 10485760000 bytes (10 GB) copied, 2.35863 s, 4.4 GB/s
> > 
> > kernel 4.4.1 make -j8:
> > real    17m54.221s
> > user    126m46.264s
> > sys     6m10.592s
> > 
> > 
> > ********************************************************************************
> > saruman Samsung Evo 850 2TB SSD encrypted btrfs
> > 
> > hdparm -t
> >  Timing buffered disk reads: 1606 MB in  3.00 seconds = 534.87 MB/sec
> > 
> > iozone -e -a -s 4096 -i 0 -i 1 -i 2:
> >                                                               random    random
> >               kB  reclen    write  rewrite    read    reread    read     write
> >             4096       4   513924   738957  8586475  9144037  8017464   697665
> >             4096     512   772100   820362  9287391 10136778 10586522   840551
> >             4096    4096   719099   777517  7863339  8047509  7892238   796108
> > 
> > iozone -I -e -a -s 4096 -i 0 -i 1 -i 2
> >             4096       4    76626    47961    92886    80947    36063    63788
> >             4096     512   135087   138998   143127   150981   144393   132219
> >             4096    4096   233245   234058   310797   311552   309855   254080
> > 
> > ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2 (unencrypted)
> >             4096       4   118013   141212   130575   111534    42343   138654
> >             4096     512   336347   312192   332847   414404   424542   342707
> >             4096    4096   365227   364638   379298   484160   475122   366536
> > 
> > 
> > saruman:/tmp# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M
> > 10485760000 bytes (10 GB) copied, 5.23919 s, 2.0 GB/s
> > 10485760000 bytes (10 GB) copied, 1.58287 s, 6.6 GB/s
> > 
> > 
> > kernel 4.4.1 make -j8:
> > real    16m4.305s
> > user    104m0.816s
> > sys     7m2.832s
> > 
> > 
> > -- 
> > "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> > Microsoft is to operating systems ....
> >                                       .... what McDonalds is to gourmet cooking
> > Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901
> 

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901


From jonathan.derrick at intel.com  Thu Feb 25 18:36:35 2016
From: jonathan.derrick at intel.com (Jon Derrick)
Date: Thu, 25 Feb 2016 19:36:35 -0700
Subject: [Patchv2 0/3] NVMe: Introduce CMB allocation scheme
Message-ID: <1456454198-26683-1-git-send-email-jonathan.derrick@intel.com>

This set is a continuation of v1 introduced in December [1].
v1 was missing the controls to reduce the queue depth, and so would
allow tags to be issued above the depth of the CMB allocated queues.

v2 drops patch 3 of v1 which created a cmb_resources file with a layout
similar to the pci_dev resource file.

This set was tested with QEMU with NVMe device emulation [2]

Applies to Jen's for-4.6/drivers [3]

[1] http://lists.infradead.org/pipermail/linux-nvme/2015-December/003510.html
[2] git://git.infradead.org/users/kbusch/qemu-nvme.git
[3] https://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git/log/?h=for-4.6/drivers

Jon Derrick (3):
  blk-mq: helper to reduce queue depth on all hardware contexts
  NVMe: Introduce sysfs entries for submission queues in CMB
  NVMe: Generate resource tree for CMB

 block/blk-mq.c           |  20 ++++
 drivers/nvme/host/core.c | 136 ++++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h |  22 +++++
 drivers/nvme/host/pci.c  | 238 ++++++++++++++++++++++++++++++++---------------
 include/linux/blk-mq.h   |   1 +
 5 files changed, 339 insertions(+), 78 deletions(-)

-- 
2.1.4



From jonathan.derrick at intel.com  Thu Feb 25 18:36:36 2016
From: jonathan.derrick at intel.com (Jon Derrick)
Date: Thu, 25 Feb 2016 19:36:36 -0700
Subject: [Patchv2 1/3] blk-mq: helper to reduce queue depth on all hardware
 contexts
In-Reply-To: <1456454198-26683-1-git-send-email-jonathan.derrick@intel.com>
References: <1456454198-26683-1-git-send-email-jonathan.derrick@intel.com>
Message-ID: <1456454198-26683-2-git-send-email-jonathan.derrick@intel.com>

Creates helper that limits the tags on all request_queues attached to
the tagset. This functionality is used to reassign queues in host memory
to controller memory buffers with lower queue depths.

Signed-off-by: Jon Derrick <jonathan.derrick at intel.com>
---
 block/blk-mq.c         | 20 ++++++++++++++++++++
 include/linux/blk-mq.h |  1 +
 2 files changed, 21 insertions(+)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 645eb9e..c8bbb62 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2345,6 +2345,26 @@ int blk_mq_update_nr_requests(struct request_queue *q, unsigned int nr)
 	return ret;
 }
 
+void blk_mq_update_nr_hw_requests(struct blk_mq_tag_set *set,
+				  int nr_requests)
+{
+	struct request_queue *q;
+
+	if (nr_requests < set->reserved_tags + 1 ||
+	    nr_requests == set->queue_depth)
+		return;
+
+	list_for_each_entry(q, &set->tag_list, tag_set_list)
+		blk_mq_freeze_queue(q);
+
+	list_for_each_entry(q, &set->tag_list, tag_set_list)
+		blk_mq_update_nr_requests(q, nr_requests);
+
+	list_for_each_entry(q, &set->tag_list, tag_set_list)
+		blk_mq_unfreeze_queue(q);
+}
+EXPORT_SYMBOL_GPL(blk_mq_update_nr_hw_requests);
+
 void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues)
 {
 	struct request_queue *q;
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 15a73d4..2596be6 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -244,6 +244,7 @@ void blk_mq_freeze_queue(struct request_queue *q);
 void blk_mq_unfreeze_queue(struct request_queue *q);
 void blk_mq_freeze_queue_start(struct request_queue *q);
 
+void blk_mq_update_nr_hw_requests(struct blk_mq_tag_set *set, int nr_requests);
 void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues);
 
 /*
-- 
2.1.4



From jonathan.derrick at intel.com  Thu Feb 25 18:36:38 2016
From: jonathan.derrick at intel.com (Jon Derrick)
Date: Thu, 25 Feb 2016 19:36:38 -0700
Subject: [Patchv2 3/3] NVMe: Generate resource tree for CMB
In-Reply-To: <1456454198-26683-1-git-send-email-jonathan.derrick@intel.com>
References: <1456454198-26683-1-git-send-email-jonathan.derrick@intel.com>
Message-ID: <1456454198-26683-4-git-send-email-jonathan.derrick@intel.com>

Maintains a resource tree for CMB resources. This new scheme uses
resource/region based allocation to manage CMB memory for in-driver
usage. For the NVMe PCIe driver, this information is exposed through
/proc/iomem.

Signed-off-by: Jon Derrick <jonathan.derrick at intel.com>
---
 drivers/nvme/host/core.c |  2 +-
 drivers/nvme/host/nvme.h |  3 +-
 drivers/nvme/host/pci.c  | 76 +++++++++++++++++++++++++++++-------------------
 3 files changed, 48 insertions(+), 33 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index af58e3b..34f7b66 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1083,7 +1083,7 @@ static ssize_t nvme_cmb_sq_offset_store(struct device *dev,
 	u64 sq_offset;
 
 	sscanf(buf, "%llu", &sq_offset);
-	if (sq_offset >= cmb->size)
+	if (sq_offset >= resource_size(cmb->res))
 		return -EINVAL;
 
 	cmb->sq_offset = sq_offset;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index b5d0814..1c2a6a9 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -118,9 +118,8 @@ struct nvme_ns {
 };
 
 struct nvme_cmb {
+	struct resource *res;
 	void __iomem *cmb;
-	dma_addr_t dma_addr;
-	u64 size;
 	u64 sq_offset;
 	u16 sq_depth;
 	unsigned long flags;
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 97a46f4..a006fb2 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -118,12 +118,13 @@ static inline struct nvme_dev *to_nvme_dev(struct nvme_ctrl *ctrl)
 struct nvme_queue {
 	struct device *q_dmadev;
 	struct nvme_dev *dev;
-	char irqname[24];	/* nvme4294967295-65535\0 */
+	char name[24];	/* nvme4294967295-65535\0 */
 	spinlock_t q_lock;
 	struct nvme_command *sq_cmds;
 	struct nvme_command __iomem *sq_cmds_io;
 	volatile struct nvme_completion *cqes;
 	struct blk_mq_tags **tags;
+	struct resource *res;
 	dma_addr_t sq_dma_addr;
 	dma_addr_t cq_dma_addr;
 	u32 __iomem *q_db;
@@ -1087,15 +1088,8 @@ static void nvme_disable_admin_queue(struct nvme_dev *dev, bool shutdown)
 static int nvme_cmb_sq_depth(struct nvme_dev *dev, int nr_io_queues)
 {
 	struct nvme_cmb *cmb = dev->ctrl.cmb;
-	u32 sq_size;
-	u64 sqes_size;
-
-	if (!cmb->sq_depth)
-		return -EINVAL;
-
-	sq_size = cmb->sq_depth * sizeof(struct nvme_command);
-	sqes_size = sq_size * nr_io_queues;
-	if (cmb->sq_offset + sqes_size > cmb->size)
+	u64 sqes_size =	(u64) SQ_SIZE(cmb->sq_depth) * nr_io_queues;
+	if (cmb->sq_offset + sqes_size > resource_size(cmb->res))
 		return -ENOMEM;
 
 	return cmb->sq_depth;
@@ -1107,7 +1101,15 @@ static int nvme_alloc_sq_cmds(struct nvme_dev *dev, struct nvme_queue *nvmeq,
 	struct nvme_cmb *cmb = dev->ctrl.cmb;
 	if (qid && cmb->cmb && cmb->sq_depth) {
 		u32 offset = (qid - 1) * SQ_SIZE(depth);
-		nvmeq->sq_dma_addr = cmb->dma_addr + offset;
+		struct resource *res = __request_region(cmb->res,
+					cmb->res->start + offset,
+					SQ_SIZE(depth), nvmeq->name,
+					IORESOURCE_EXCLUSIVE);
+		if (!res)
+			return -ENOMEM;
+		nvmeq->res = res;
+
+		nvmeq->sq_dma_addr = res->start;
 		nvmeq->sq_cmds_io = cmb->cmb + offset;
 	} else {
 		nvmeq->sq_cmds = dma_alloc_coherent(dev->dev, SQ_SIZE(depth),
@@ -1157,7 +1159,7 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, int qid,
 
 	nvmeq->q_dmadev = dev->dev;
 	nvmeq->dev = dev;
-	snprintf(nvmeq->irqname, sizeof(nvmeq->irqname), "nvme%dq%d",
+	snprintf(nvmeq->name, sizeof(nvmeq->name), "nvme%dq%d",
 			dev->ctrl.instance, qid);
 	spin_lock_init(&nvmeq->q_lock);
 	nvmeq->cq_head = 0;
@@ -1221,7 +1223,7 @@ static int nvme_create_queue(struct nvme_queue *nvmeq, int qid)
 	if (result < 0)
 		goto release_cq;
 
-	result = queue_request_irq(dev, nvmeq, nvmeq->irqname);
+	result = queue_request_irq(dev, nvmeq, nvmeq->name);
 	if (result < 0)
 		goto release_sq;
 
@@ -1335,7 +1337,7 @@ static int nvme_configure_admin_queue(struct nvme_dev *dev)
 		goto free_nvmeq;
 
 	nvmeq->cq_vector = 0;
-	result = queue_request_irq(dev, nvmeq, nvmeq->irqname);
+	result = queue_request_irq(dev, nvmeq, nvmeq->name);
 	if (result) {
 		nvmeq->cq_vector = -1;
 		goto free_nvmeq;
@@ -1456,25 +1458,26 @@ static int nvme_create_io_queues(struct nvme_dev *dev)
 
 static int nvme_pci_map_cmb(struct nvme_ctrl *ctrl)
 {
+	struct pci_dev *pdev = to_pci_dev(ctrl->dev);
+	struct nvme_dev *dev = to_nvme_dev(ctrl);
+	struct nvme_cmb *cmb = ctrl->cmb;
+	struct resource *res, *parent;
 	u64 szu, size, offset;
 	u32 cmbsz, cmbloc;
 	resource_size_t bar_size;
-	struct nvme_cmb *cmb = ctrl->cmb;
-	struct pci_dev *pdev = to_pci_dev(ctrl->dev);
-	struct nvme_dev *dev = to_nvme_dev(ctrl);
-	dma_addr_t dma_addr;
-	void __iomem *cmb_ioaddr;
+	int bir;
 
 	cmbsz = readl(dev->bar + NVME_REG_CMBSZ);
 	if (!(NVME_CMB_SZ(cmbsz)))
 		return -EINVAL;
 
 	cmbloc = readl(dev->bar + NVME_REG_CMBLOC);
+	bir = NVME_CMB_BIR(cmbloc);
 
 	szu = (u64)1 << (12 + 4 * NVME_CMB_SZU(cmbsz));
 	size = szu * NVME_CMB_SZ(cmbsz);
 	offset = szu * NVME_CMB_OFST(cmbloc);
-	bar_size = pci_resource_len(pdev, NVME_CMB_BIR(cmbloc));
+	bar_size = pci_resource_len(pdev, bir);
 
 	if (offset > bar_size) {
 		dev_warn(dev->dev, "CMB supported but offset does not fit "
@@ -1490,14 +1493,18 @@ static int nvme_pci_map_cmb(struct nvme_ctrl *ctrl)
 	if (size > bar_size - offset)
 		size = bar_size - offset;
 
-	dma_addr = pci_resource_start(pdev, NVME_CMB_BIR(cmbloc)) + offset;
-	cmb_ioaddr = ioremap_wc(dma_addr, size);
-	if (!cmb_ioaddr)
+	parent = pdev->resource[bir].child;
+	res = __request_region(parent, parent->start + offset, size, "cmb", 0);
+	if (!res)
 		return -ENOMEM;
 
-	cmb->cmb = cmb_ioaddr;
-	cmb->dma_addr = dma_addr;
-	cmb->size = size;
+	cmb->cmb = ioremap_wc(res->start, size);
+	if (!cmb->cmb) {
+		__release_region(parent, res->start, resource_size(res));
+		return -ENOMEM;
+	}
+
+	cmb->res = res;
 	cmb->flags |= NVME_CMB_SQS(cmbsz) ? NVME_CMB_SQ_SUPPORTED : 0;
 	cmb->flags |= NVME_CMB_CQS(cmbsz) ? NVME_CMB_CQ_SUPPORTED : 0;
 	cmb->flags |= NVME_CMB_WDS(cmbsz) ? NVME_CMB_WD_SUPPORTED : 0;
@@ -1510,9 +1517,15 @@ static void nvme_pci_unmap_cmb(struct nvme_ctrl *ctrl)
 {
 	struct nvme_cmb *cmb = ctrl->cmb;
 	if (cmb->cmb) {
+		struct resource *res;
+		for (res = cmb->res->child; res; res = res->sibling) {
+			__release_region(res->parent, res->start,
+						resource_size(res));
+		}
+		__release_region(cmb->res->parent, cmb->res->start,
+					resource_size(cmb->res));
 		iounmap(cmb->cmb);
 		cmb->cmb = NULL;
-		cmb->dma_addr = 0;
 	}
 }
 
@@ -1545,12 +1558,15 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)
 		result = 0;
 	}
 
-	if (cmb->flags & NVME_CMB_SQ_SUPPORTED) {
+	if (cmb->flags & NVME_CMB_SQ_SUPPORTED && cmb->sq_depth) {
 		result = nvme_cmb_sq_depth(dev, nr_io_queues);
 		if (result > 0)
 			dev->q_depth = result;
-		else
+		else {
+			dev_warn(dev->dev, "Could not allocate %d-deep queues "
+					"in CMB\n", cmb->sq_depth);
 			cmb->sq_depth = 0;
+		}
 	}
 
 	size = db_bar_size(dev, nr_io_queues);
@@ -1600,7 +1616,7 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)
 	nr_io_queues = vecs;
 	dev->max_qid = nr_io_queues;
 
-	result = queue_request_irq(dev, adminq, adminq->irqname);
+	result = queue_request_irq(dev, adminq, adminq->name);
 	if (result) {
 		adminq->cq_vector = -1;
 		goto free_queues;
-- 
2.1.4



From jonathan.derrick at intel.com  Thu Feb 25 18:36:37 2016
From: jonathan.derrick at intel.com (Jon Derrick)
Date: Thu, 25 Feb 2016 19:36:37 -0700
Subject: [Patchv2 2/3] NVMe: Introduce sysfs entries for submission queues in
 CMB
In-Reply-To: <1456454198-26683-1-git-send-email-jonathan.derrick@intel.com>
References: <1456454198-26683-1-git-send-email-jonathan.derrick@intel.com>
Message-ID: <1456454198-26683-3-git-send-email-jonathan.derrick@intel.com>

Currently submission queues are always mapped to the CMB if possible and
allowed by a module parameter. To allow userspace more control over the
CMB, this patch introduces a sysfs/cmb framework into the core nvme code
and refactors the pci portion.

If the controller supports SQes in the CMB, sysfs files cmb_sq_depth and
cmb_sq_offset are visible. To apply changes to the queues, users must
write the sysfs reset_controller entry after changing cmb parameters.

Signed-off-by: Jon Derrick <jonathan.derrick at intel.com>
---
 drivers/nvme/host/core.c | 136 +++++++++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h |  23 ++++++
 drivers/nvme/host/pci.c  | 200 ++++++++++++++++++++++++++++++++---------------
 3 files changed, 292 insertions(+), 67 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 07b7ec69..af58e3b 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1035,6 +1035,89 @@ static ssize_t nvme_sysfs_reset(struct device *dev,
 }
 static DEVICE_ATTR(reset_controller, S_IWUSR, NULL, nvme_sysfs_reset);
 
+static ssize_t nvme_cmb_sq_depth_show(struct device *dev,
+					struct device_attribute *attr,
+					char *buf)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvme_cmb *cmb = ctrl->cmb;
+	return sprintf(buf, "%u\n", cmb->sq_depth);
+}
+
+static ssize_t nvme_cmb_sq_depth_store(struct device *dev,
+					struct device_attribute *attr,
+					const char *buf, size_t count)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvme_cmb *cmb = ctrl->cmb;
+	u32 sq_depth;
+
+	sscanf(buf, "%u", &sq_depth);
+	if (sq_depth > 0xffff)
+		return -EINVAL;
+
+	if (sq_depth > 0 && sq_depth < ctrl->tagset->reserved_tags + 1)
+		return -EINVAL;
+
+	cmb->sq_depth = sq_depth;
+	return count;
+}
+static DEVICE_ATTR(cmb_sq_depth, S_IWUSR | S_IRUGO, nvme_cmb_sq_depth_show,
+						nvme_cmb_sq_depth_store);
+
+static ssize_t nvme_cmb_sq_offset_show(struct device *dev,
+					struct device_attribute *attr,
+					char *buf)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvme_cmb *cmb = ctrl->cmb;
+	return sprintf(buf, "%llu\n", cmb->sq_offset);
+}
+
+static ssize_t nvme_cmb_sq_offset_store(struct device *dev,
+					struct device_attribute *attr,
+					const char *buf, size_t count)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvme_cmb *cmb = ctrl->cmb;
+	u64 sq_offset;
+
+	sscanf(buf, "%llu", &sq_offset);
+	if (sq_offset >= cmb->size)
+		return -EINVAL;
+
+	cmb->sq_offset = sq_offset;
+	return count;
+}
+static DEVICE_ATTR(cmb_sq_offset, S_IWUSR | S_IRUGO, nvme_cmb_sq_offset_show,
+						nvme_cmb_sq_offset_store);
+
+static struct attribute *nvme_cmb_attrs[] = {
+	&dev_attr_cmb_sq_depth.attr,
+	&dev_attr_cmb_sq_offset.attr,
+	NULL
+};
+
+static umode_t nvme_cmb_attrs_are_visible(struct kobject *kobj,
+		struct attribute *a, int n)
+{
+	struct device *dev = kobj_to_dev(kobj);
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvme_cmb *cmb = ctrl->cmb;
+
+	if ((a == &dev_attr_cmb_sq_depth.attr) ||
+	    (a == &dev_attr_cmb_sq_offset.attr)) {
+		if (!(cmb->flags & NVME_CMB_SQ_SUPPORTED))
+			return 0;
+	}
+	return a->mode;
+}
+
+static struct attribute_group nvme_cmb_attr_group = {
+	.attrs 		= nvme_cmb_attrs,
+	.is_visible	= nvme_cmb_attrs_are_visible,
+};
+
 static ssize_t uuid_show(struct device *dev, struct device_attribute *attr,
 								char *buf)
 {
@@ -1066,7 +1149,7 @@ static struct attribute *nvme_ns_attrs[] = {
 	NULL,
 };
 
-static umode_t nvme_attrs_are_visible(struct kobject *kobj,
+static umode_t nvme_ns_attrs_are_visible(struct kobject *kobj,
 		struct attribute *a, int n)
 {
 	struct device *dev = container_of(kobj, struct device, kobj);
@@ -1085,7 +1168,7 @@ static umode_t nvme_attrs_are_visible(struct kobject *kobj,
 
 static const struct attribute_group nvme_ns_attr_group = {
 	.attrs		= nvme_ns_attrs,
-	.is_visible	= nvme_attrs_are_visible,
+	.is_visible	= nvme_ns_attrs_are_visible,
 };
 
 #define nvme_show_function(field)						\
@@ -1344,6 +1427,47 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 }
 EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
 
+static int nvme_init_cmb(struct nvme_ctrl *ctrl)
+{
+	/* Preserve across device resets */
+	if (ctrl->cmb)
+		return 0;
+
+	ctrl->cmb = kzalloc(sizeof(*ctrl->cmb), GFP_KERNEL);
+	if (!ctrl->cmb)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static void nvme_release_cmb(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->cmb) {
+		kfree(ctrl->cmb);
+		ctrl->cmb = NULL;
+	}
+}
+
+void nvme_map_cmb(struct nvme_ctrl *ctrl)
+{
+	struct device *dev = ctrl->device;
+
+	if (ctrl->ops->map_cmb(ctrl))
+		return;
+
+	if (sysfs_create_group(&dev->kobj, &nvme_cmb_attr_group))
+		dev_warn(dev, "failed to create sysfs group for CMB\n");
+}
+EXPORT_SYMBOL_GPL(nvme_map_cmb);
+
+void nvme_unmap_cmb(struct nvme_ctrl *ctrl)
+{
+	struct device *dev = ctrl->device;
+	ctrl->ops->unmap_cmb(ctrl);
+	sysfs_remove_group(&dev->kobj, &nvme_cmb_attr_group);
+}
+EXPORT_SYMBOL_GPL(nvme_unmap_cmb);
+
 static DEFINE_IDA(nvme_instance_ida);
 
 static int nvme_set_instance(struct nvme_ctrl *ctrl)
@@ -1388,6 +1512,7 @@ static void nvme_free_ctrl(struct kref *kref)
 	struct nvme_ctrl *ctrl = container_of(kref, struct nvme_ctrl, kref);
 
 	put_device(ctrl->device);
+	nvme_release_cmb(ctrl);
 	nvme_release_instance(ctrl);
 
 	ctrl->ops->free_ctrl(ctrl);
@@ -1430,11 +1555,18 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	}
 	get_device(ctrl->device);
 
+	ret = nvme_init_cmb(ctrl);
+	if (ret)
+		goto out_destroy_device;
+
 	spin_lock(&dev_list_lock);
 	list_add_tail(&ctrl->node, &nvme_ctrl_list);
 	spin_unlock(&dev_list_lock);
 
 	return 0;
+out_destroy_device:
+	put_device(ctrl->device);
+	device_destroy(nvme_class, MKDEV(nvme_char_major, ctrl->instance));
 out_release_instance:
 	nvme_release_instance(ctrl);
 out:
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 63ba8a5..b5d0814 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -72,6 +72,7 @@ struct nvme_ctrl {
 	struct mutex namespaces_mutex;
 	struct device *device;	/* char device */
 	struct list_head node;
+	struct nvme_cmb *cmb;
 
 	char name[12];
 	char serial[20];
@@ -116,6 +117,23 @@ struct nvme_ns {
 	u32 mode_select_block_len;
 };
 
+struct nvme_cmb {
+	void __iomem *cmb;
+	dma_addr_t dma_addr;
+	u64 size;
+	u64 sq_offset;
+	u16 sq_depth;
+	unsigned long flags;
+};
+
+enum nvme_cmb_flags {
+	NVME_CMB_SQ_SUPPORTED	= (1 << 0),
+	NVME_CMB_CQ_SUPPORTED	= (1 << 1),
+	NVME_CMB_WD_SUPPORTED	= (1 << 2),
+	NVME_CMB_RD_SUPPORTED	= (1 << 3),
+	NVME_CMB_PRP_SUPPORTED	= (1 << 4),
+};
+
 struct nvme_ctrl_ops {
 	struct module *module;
 	int (*reg_read32)(struct nvme_ctrl *ctrl, u32 off, u32 *val);
@@ -124,6 +142,8 @@ struct nvme_ctrl_ops {
 	bool (*io_incapable)(struct nvme_ctrl *ctrl);
 	int (*reset_ctrl)(struct nvme_ctrl *ctrl);
 	void (*free_ctrl)(struct nvme_ctrl *ctrl);
+	int (*map_cmb)(struct nvme_ctrl *ctrl);
+	void (*unmap_cmb)(struct nvme_ctrl *ctrl);
 };
 
 static inline bool nvme_ctrl_ready(struct nvme_ctrl *ctrl)
@@ -239,6 +259,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl);
 void nvme_scan_namespaces(struct nvme_ctrl *ctrl);
 void nvme_remove_namespaces(struct nvme_ctrl *ctrl);
 
+void nvme_map_cmb(struct nvme_ctrl *ctrl);
+void nvme_unmap_cmb(struct nvme_ctrl *ctrl);
+
 void nvme_stop_queues(struct nvme_ctrl *ctrl);
 void nvme_start_queues(struct nvme_ctrl *ctrl);
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index fec7479..97a46f4 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -60,10 +60,6 @@
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0);
 
-static bool use_cmb_sqes = true;
-module_param(use_cmb_sqes, bool, 0644);
-MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
-
 static LIST_HEAD(dev_list);
 static DEFINE_SPINLOCK(dev_list_lock);
 static struct task_struct *nvme_thread;
@@ -102,10 +98,6 @@ struct nvme_dev {
 	struct work_struct remove_work;
 	struct mutex shutdown_lock;
 	bool subsystem;
-	void __iomem *cmb;
-	dma_addr_t cmb_dma_addr;
-	u64 cmb_size;
-	u32 cmbsz;
 	unsigned long flags;
 
 #define NVME_CTRL_RESETTING    0
@@ -999,13 +991,29 @@ static void nvme_cancel_queue_ios(struct request *req, void *data, bool reserved
 	blk_mq_complete_request(req, status);
 }
 
-static void nvme_free_queue(struct nvme_queue *nvmeq)
+static void nvme_release_sq(struct nvme_queue *nvmeq)
 {
-	dma_free_coherent(nvmeq->q_dmadev, CQ_SIZE(nvmeq->q_depth),
-				(void *)nvmeq->cqes, nvmeq->cq_dma_addr);
 	if (nvmeq->sq_cmds)
 		dma_free_coherent(nvmeq->q_dmadev, SQ_SIZE(nvmeq->q_depth),
 					nvmeq->sq_cmds, nvmeq->sq_dma_addr);
+
+	nvmeq->sq_cmds = NULL;
+	nvmeq->sq_cmds_io = NULL;
+}
+
+static void nvme_release_cq(struct nvme_queue *nvmeq)
+{
+	if (nvmeq->cqes)
+		dma_free_coherent(nvmeq->q_dmadev, CQ_SIZE(nvmeq->q_depth),
+				(void *)nvmeq->cqes, nvmeq->cq_dma_addr);
+
+	nvmeq->cqes = NULL;
+}
+
+static void nvme_free_queue(struct nvme_queue *nvmeq)
+{
+	nvme_release_sq(nvmeq);
+	nvme_release_cq(nvmeq);
 	kfree(nvmeq);
 }
 
@@ -1076,38 +1084,31 @@ static void nvme_disable_admin_queue(struct nvme_dev *dev, bool shutdown)
 	spin_unlock_irq(&nvmeq->q_lock);
 }
 
-static int nvme_cmb_qdepth(struct nvme_dev *dev, int nr_io_queues,
-				int entry_size)
+static int nvme_cmb_sq_depth(struct nvme_dev *dev, int nr_io_queues)
 {
-	int q_depth = dev->q_depth;
-	unsigned q_size_aligned = roundup(q_depth * entry_size,
-					  dev->ctrl.page_size);
+	struct nvme_cmb *cmb = dev->ctrl.cmb;
+	u32 sq_size;
+	u64 sqes_size;
 
-	if (q_size_aligned * nr_io_queues > dev->cmb_size) {
-		u64 mem_per_q = div_u64(dev->cmb_size, nr_io_queues);
-		mem_per_q = round_down(mem_per_q, dev->ctrl.page_size);
-		q_depth = div_u64(mem_per_q, entry_size);
+	if (!cmb->sq_depth)
+		return -EINVAL;
 
-		/*
-		 * Ensure the reduced q_depth is above some threshold where it
-		 * would be better to map queues in system memory with the
-		 * original depth
-		 */
-		if (q_depth < 64)
-			return -ENOMEM;
-	}
+	sq_size = cmb->sq_depth * sizeof(struct nvme_command);
+	sqes_size = sq_size * nr_io_queues;
+	if (cmb->sq_offset + sqes_size > cmb->size)
+		return -ENOMEM;
 
-	return q_depth;
+	return cmb->sq_depth;
 }
 
 static int nvme_alloc_sq_cmds(struct nvme_dev *dev, struct nvme_queue *nvmeq,
 				int qid, int depth)
 {
-	if (qid && dev->cmb && use_cmb_sqes && NVME_CMB_SQS(dev->cmbsz)) {
-		unsigned offset = (qid - 1) * roundup(SQ_SIZE(depth),
-						      dev->ctrl.page_size);
-		nvmeq->sq_dma_addr = dev->cmb_dma_addr + offset;
-		nvmeq->sq_cmds_io = dev->cmb + offset;
+	struct nvme_cmb *cmb = dev->ctrl.cmb;
+	if (qid && cmb->cmb && cmb->sq_depth) {
+		u32 offset = (qid - 1) * SQ_SIZE(depth);
+		nvmeq->sq_dma_addr = cmb->dma_addr + offset;
+		nvmeq->sq_cmds_io = cmb->cmb + offset;
 	} else {
 		nvmeq->sq_cmds = dma_alloc_coherent(dev->dev, SQ_SIZE(depth),
 					&nvmeq->sq_dma_addr, GFP_KERNEL);
@@ -1118,6 +1119,27 @@ static int nvme_alloc_sq_cmds(struct nvme_dev *dev, struct nvme_queue *nvmeq,
 	return 0;
 }
 
+static bool nvme_queue_needs_remap(struct nvme_dev *dev, struct nvme_queue *nvmeq)
+{
+	if (dev->queue_count > 1) {
+		struct nvme_cmb *cmb = dev->ctrl.cmb;
+		/*
+		 * This condition occurs if SQes were previously mapped
+		 * in Memory or CMB and need to be switched over to the
+		 * other. This also occurs if SQes are currently mapped
+		 * in the CMB and CMB parameters change.
+		 *
+		 * However it doesn't hurt to remap CMB SQes if the
+		 * parameters don't change, so to simplify we can check
+		 * if they are currently in the CMB or will be in the
+		 * CMB after queue creation.
+		 */
+		return (nvmeq->sq_cmds_io || cmb->sq_depth);
+	}
+
+	return false;
+}
+
 static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, int qid,
 							int depth)
 {
@@ -1370,6 +1392,30 @@ static int nvme_kthread(void *data)
 	return 0;
 }
 
+static int nvme_remap_queue(struct nvme_dev *dev, struct nvme_queue *nvmeq)
+{
+	struct nvme_cmb *cmb = dev->ctrl.cmb;
+
+	nvme_release_sq(nvmeq);
+	nvme_release_cq(nvmeq);
+
+	if (!cmb->sq_depth)
+		dev->q_depth = dev->tagset.queue_depth;
+	nvmeq->q_depth = dev->q_depth;
+
+	nvmeq->cqes = dma_zalloc_coherent(dev->dev, CQ_SIZE(nvmeq->q_depth),
+					  &nvmeq->cq_dma_addr, GFP_KERNEL);
+	if (!nvmeq->cqes)
+		return -ENOMEM;
+
+	if (nvme_alloc_sq_cmds(dev, nvmeq, nvmeq->qid, nvmeq->q_depth)) {
+		dma_free_coherent(dev->dev, CQ_SIZE(nvmeq->q_depth),
+				(void *)nvmeq->cqes, nvmeq->cq_dma_addr);
+		return -ENOMEM;
+	}
+	return 0;
+}
+
 static int nvme_create_io_queues(struct nvme_dev *dev)
 {
 	unsigned i, max;
@@ -1384,6 +1430,14 @@ static int nvme_create_io_queues(struct nvme_dev *dev)
 
 	max = min(dev->max_qid, dev->queue_count - 1);
 	for (i = dev->online_queues; i <= max; i++) {
+		if (nvme_queue_needs_remap(dev, dev->queues[i])) {
+			ret = nvme_remap_queue(dev, dev->queues[i]);
+			if (ret) {
+				nvme_free_queues(dev, i);
+				break;
+			}
+		}
+
 		ret = nvme_create_queue(dev->queues[i], i);
 		if (ret) {
 			nvme_free_queues(dev, i);
@@ -1400,31 +1454,33 @@ static int nvme_create_io_queues(struct nvme_dev *dev)
 	return ret >= 0 ? 0 : ret;
 }
 
-static void __iomem *nvme_map_cmb(struct nvme_dev *dev)
+static int nvme_pci_map_cmb(struct nvme_ctrl *ctrl)
 {
 	u64 szu, size, offset;
-	u32 cmbloc;
+	u32 cmbsz, cmbloc;
 	resource_size_t bar_size;
-	struct pci_dev *pdev = to_pci_dev(dev->dev);
-	void __iomem *cmb;
+	struct nvme_cmb *cmb = ctrl->cmb;
+	struct pci_dev *pdev = to_pci_dev(ctrl->dev);
+	struct nvme_dev *dev = to_nvme_dev(ctrl);
 	dma_addr_t dma_addr;
+	void __iomem *cmb_ioaddr;
 
-	if (!use_cmb_sqes)
-		return NULL;
-
-	dev->cmbsz = readl(dev->bar + NVME_REG_CMBSZ);
-	if (!(NVME_CMB_SZ(dev->cmbsz)))
-		return NULL;
+	cmbsz = readl(dev->bar + NVME_REG_CMBSZ);
+	if (!(NVME_CMB_SZ(cmbsz)))
+		return -EINVAL;
 
 	cmbloc = readl(dev->bar + NVME_REG_CMBLOC);
 
-	szu = (u64)1 << (12 + 4 * NVME_CMB_SZU(dev->cmbsz));
-	size = szu * NVME_CMB_SZ(dev->cmbsz);
+	szu = (u64)1 << (12 + 4 * NVME_CMB_SZU(cmbsz));
+	size = szu * NVME_CMB_SZ(cmbsz);
 	offset = szu * NVME_CMB_OFST(cmbloc);
 	bar_size = pci_resource_len(pdev, NVME_CMB_BIR(cmbloc));
 
-	if (offset > bar_size)
-		return NULL;
+	if (offset > bar_size) {
+		dev_warn(dev->dev, "CMB supported but offset does not fit "
+			"within bar (%#llx/%#llx)\n", offset, bar_size);
+		return -ENOMEM;
+	}
 
 	/*
 	 * Controllers may support a CMB size larger than their BAR,
@@ -1435,20 +1491,28 @@ static void __iomem *nvme_map_cmb(struct nvme_dev *dev)
 		size = bar_size - offset;
 
 	dma_addr = pci_resource_start(pdev, NVME_CMB_BIR(cmbloc)) + offset;
-	cmb = ioremap_wc(dma_addr, size);
-	if (!cmb)
-		return NULL;
+	cmb_ioaddr = ioremap_wc(dma_addr, size);
+	if (!cmb_ioaddr)
+		return -ENOMEM;
 
-	dev->cmb_dma_addr = dma_addr;
-	dev->cmb_size = size;
-	return cmb;
+	cmb->cmb = cmb_ioaddr;
+	cmb->dma_addr = dma_addr;
+	cmb->size = size;
+	cmb->flags |= NVME_CMB_SQS(cmbsz) ? NVME_CMB_SQ_SUPPORTED : 0;
+	cmb->flags |= NVME_CMB_CQS(cmbsz) ? NVME_CMB_CQ_SUPPORTED : 0;
+	cmb->flags |= NVME_CMB_WDS(cmbsz) ? NVME_CMB_WD_SUPPORTED : 0;
+	cmb->flags |= NVME_CMB_RDS(cmbsz) ? NVME_CMB_RD_SUPPORTED : 0;
+	cmb->flags |= NVME_CMB_LISTS(cmbsz) ? NVME_CMB_PRP_SUPPORTED : 0;
+	return 0;
 }
 
-static inline void nvme_release_cmb(struct nvme_dev *dev)
+static void nvme_pci_unmap_cmb(struct nvme_ctrl *ctrl)
 {
-	if (dev->cmb) {
-		iounmap(dev->cmb);
-		dev->cmb = NULL;
+	struct nvme_cmb *cmb = ctrl->cmb;
+	if (cmb->cmb) {
+		iounmap(cmb->cmb);
+		cmb->cmb = NULL;
+		cmb->dma_addr = 0;
 	}
 }
 
@@ -1461,6 +1525,7 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)
 {
 	struct nvme_queue *adminq = dev->queues[0];
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
+	struct nvme_cmb *cmb = dev->ctrl.cmb;
 	int result, i, vecs, nr_io_queues, size;
 
 	nr_io_queues = num_possible_cpus();
@@ -1480,13 +1545,12 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)
 		result = 0;
 	}
 
-	if (dev->cmb && NVME_CMB_SQS(dev->cmbsz)) {
-		result = nvme_cmb_qdepth(dev, nr_io_queues,
-				sizeof(struct nvme_command));
+	if (cmb->flags & NVME_CMB_SQ_SUPPORTED) {
+		result = nvme_cmb_sq_depth(dev, nr_io_queues);
 		if (result > 0)
 			dev->q_depth = result;
 		else
-			nvme_release_cmb(dev);
+			cmb->sq_depth = 0;
 	}
 
 	size = db_bar_size(dev, nr_io_queues);
@@ -1675,6 +1739,7 @@ static int nvme_dev_add(struct nvme_dev *dev)
 			return 0;
 		dev->ctrl.tagset = &dev->tagset;
 	} else {
+		blk_mq_update_nr_hw_requests(&dev->tagset, dev->q_depth);
 		blk_mq_update_nr_hw_queues(&dev->tagset, dev->online_queues - 1);
 
 		/* Free previously allocated queues that are no longer usable */
@@ -1744,7 +1809,7 @@ static int nvme_dev_map(struct nvme_dev *dev)
 	}
 
 	if (readl(dev->bar + NVME_REG_VS) >= NVME_VS(1, 2))
-		dev->cmb = nvme_map_cmb(dev);
+		nvme_map_cmb(&dev->ctrl);
 
 	pci_enable_pcie_error_reporting(pdev);
 	pci_save_state(pdev);
@@ -1827,6 +1892,7 @@ static void nvme_dev_list_remove(struct nvme_dev *dev)
 
 static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 {
+	struct nvme_cmb *cmb = dev->ctrl.cmb;
 	int i;
 	u32 csts = -1;
 
@@ -1846,6 +1912,9 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 		nvme_disable_io_queues(dev);
 		nvme_disable_admin_queue(dev, shutdown);
 	}
+
+	if (cmb->cmb)
+		nvme_unmap_cmb(&dev->ctrl);
 	nvme_dev_unmap(dev);
 
 	for (i = dev->queue_count - 1; i >= 0; i--)
@@ -2032,6 +2101,8 @@ static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = {
 	.io_incapable		= nvme_pci_io_incapable,
 	.reset_ctrl		= nvme_pci_reset_ctrl,
 	.free_ctrl		= nvme_pci_free_ctrl,
+	.map_cmb		= nvme_pci_map_cmb,
+	.unmap_cmb		= nvme_pci_unmap_cmb,
 };
 
 static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
@@ -2118,11 +2189,10 @@ static void nvme_remove(struct pci_dev *pdev)
 	flush_work(&dev->reset_work);
 	flush_work(&dev->scan_work);
 	nvme_remove_namespaces(&dev->ctrl);
-	nvme_uninit_ctrl(&dev->ctrl);
 	nvme_dev_disable(dev, true);
+	nvme_uninit_ctrl(&dev->ctrl);
 	nvme_dev_remove_admin(dev);
 	nvme_free_queues(dev, 0);
-	nvme_release_cmb(dev);
 	nvme_release_prp_pools(dev);
 	nvme_put_ctrl(&dev->ctrl);
 }
-- 
2.1.4



From mlin at kernel.org  Fri Feb 26 13:24:19 2016
From: mlin at kernel.org (Ming Lin)
Date: Fri, 26 Feb 2016 13:24:19 -0800
Subject: [PATCH] nvme: expose cntlid in sysfs
Message-ID: <1456521859-26357-1-git-send-email-mlin@kernel.org>

From: Ming Lin <ming.l at ssi.samsung.com>

For NVMe over Fabrics, the cntlid will be used by systemd/udev to
create link to the device, for example,

/dev/disk/by-path/<fabrics-info>-<cntlid>-<namespace> -> /dev/nvme0n1

Signed-off-by: Ming Lin <ming.l at ssi.samsung.com>
---
 drivers/nvme/host/core.c | 20 ++++++++++++++++----
 drivers/nvme/host/nvme.h |  1 +
 2 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c7df0ab..0b04119 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -901,6 +901,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	ctrl->oncs = le16_to_cpup(&id->oncs);
 	atomic_set(&ctrl->abort_limit, id->acl + 1);
 	ctrl->vwc = id->vwc;
+	ctrl->cntlid = le16_to_cpup(&id->cntlid);
 	memcpy(ctrl->serial, id->sn, sizeof(id->sn));
 	memcpy(ctrl->model, id->mn, sizeof(id->mn));
 	memcpy(ctrl->firmware_rev, id->fr, sizeof(id->fr));
@@ -1088,7 +1089,7 @@ static const struct attribute_group nvme_ns_attr_group = {
 	.is_visible	= nvme_attrs_are_visible,
 };
 
-#define nvme_show_function(field)						\
+#define nvme_show_str_function(field)						\
 static ssize_t  field##_show(struct device *dev,				\
 			    struct device_attribute *attr, char *buf)		\
 {										\
@@ -1097,15 +1098,26 @@ static ssize_t  field##_show(struct device *dev,				\
 }										\
 static DEVICE_ATTR(field, S_IRUGO, field##_show, NULL);
 
-nvme_show_function(model);
-nvme_show_function(serial);
-nvme_show_function(firmware_rev);
+#define nvme_show_int_function(field)						\
+static ssize_t  field##_show(struct device *dev,				\
+			    struct device_attribute *attr, char *buf)		\
+{										\
+        struct nvme_ctrl *ctrl = dev_get_drvdata(dev);				\
+        return sprintf(buf, "%d\n", ctrl->field);	\
+}										\
+static DEVICE_ATTR(field, S_IRUGO, field##_show, NULL);
+
+nvme_show_str_function(model);
+nvme_show_str_function(serial);
+nvme_show_str_function(firmware_rev);
+nvme_show_int_function(cntlid);
 
 static struct attribute *nvme_dev_attrs[] = {
 	&dev_attr_reset_controller.attr,
 	&dev_attr_model.attr,
 	&dev_attr_serial.attr,
 	&dev_attr_firmware_rev.attr,
+	&dev_attr_cntlid.attr,
 	NULL
 };
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 63ba8a5..2e041e7 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -77,6 +77,7 @@ struct nvme_ctrl {
 	char serial[20];
 	char model[40];
 	char firmware_rev[8];
+	int cntlid;
 
 	u32 ctrl_config;
 
-- 
1.9.1



From keith.busch at intel.com  Fri Feb 26 15:09:45 2016
From: keith.busch at intel.com (Keith Busch)
Date: Fri, 26 Feb 2016 23:09:45 +0000
Subject: [PATCH] nvme: expose cntlid in sysfs
In-Reply-To: <1456521859-26357-1-git-send-email-mlin@kernel.org>
References: <1456521859-26357-1-git-send-email-mlin@kernel.org>
Message-ID: <20160226230944.GA4512@localhost.localdomain>

On Fri, Feb 26, 2016 at 01:24:19PM -0800, Ming Lin wrote:
> From: Ming Lin <ming.l at ssi.samsung.com>
> 
> For NVMe over Fabrics, the cntlid will be used by systemd/udev to
> create link to the device, for example,
> 
> /dev/disk/by-path/<fabrics-info>-<cntlid>-<namespace> -> /dev/nvme0n1

Looks good. Sounds horrific if I try to pronounce it, but the spec
abbreviated "controller" that way!

Reviewed-by: Keith Busch <keith.busch at intel.com>


From hch at lst.de  Sun Feb 28 01:10:46 2016
From: hch at lst.de (Christoph Hellwig)
Date: Sun, 28 Feb 2016 10:10:46 +0100
Subject: [PATCH] nvme: expose cntlid in sysfs
In-Reply-To: <1456521859-26357-1-git-send-email-mlin@kernel.org>
References: <1456521859-26357-1-git-send-email-mlin@kernel.org>
Message-ID: <20160228091046.GB18953@lst.de>

On Fri, Feb 26, 2016 at 01:24:19PM -0800, Ming Lin wrote:
> From: Ming Lin <ming.l at ssi.samsung.com>
> 
> For NVMe over Fabrics, the cntlid will be used by systemd/udev to
> create link to the device, for example,

Looks fine,

Reviewed-by: Christoph Hellwig <hch at lst.de>


From sagig at dev.mellanox.co.il  Sun Feb 28 03:18:41 2016
From: sagig at dev.mellanox.co.il (Sagi Grimberg)
Date: Sun, 28 Feb 2016 13:18:41 +0200
Subject: [PATCH] nvme: expose cntlid in sysfs
In-Reply-To: <1456521859-26357-1-git-send-email-mlin@kernel.org>
References: <1456521859-26357-1-git-send-email-mlin@kernel.org>
Message-ID: <56D2D791.9060801@dev.mellanox.co.il>

Looks good,

Reviewed-by: Sagi Grimberg <sagig at mellanox.com>


From hch at lst.de  Mon Feb 29 06:59:46 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 29 Feb 2016 15:59:46 +0100
Subject: [PATCH 3/4] nvme: replace the kthread with a per-device watchdog timer
In-Reply-To: <1456757987-7244-1-git-send-email-hch@lst.de>
References: <1456757987-7244-1-git-send-email-hch@lst.de>
Message-ID: <1456757987-7244-4-git-send-email-hch@lst.de>

The only work left in the kthread is the periodic health check for each
controller.  There is no need to run this from process context or keep
a thread context around for it, so replace it with a simpler timer.

Signed-off-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
 drivers/nvme/host/pci.c | 112 ++++++++++--------------------------------------
 1 file changed, 23 insertions(+), 89 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 10839f7..a623360 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -27,7 +27,6 @@
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/kdev_t.h>
-#include <linux/kthread.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/module.h>
@@ -39,6 +38,7 @@
 #include <linux/sched.h>
 #include <linux/slab.h>
 #include <linux/t10-pi.h>
+#include <linux/timer.h>
 #include <linux/types.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <asm/unaligned.h>
@@ -64,11 +64,7 @@ static bool use_cmb_sqes = true;
 module_param(use_cmb_sqes, bool, 0644);
 MODULE_PARM_DESC(use_cmb_sqes, "use controller's memory buffer for I/O SQes");
 
-static LIST_HEAD(dev_list);
-static DEFINE_SPINLOCK(dev_list_lock);
-static struct task_struct *nvme_thread;
 static struct workqueue_struct *nvme_workq;
-static wait_queue_head_t nvme_kthread_wait;
 
 struct nvme_dev;
 struct nvme_queue;
@@ -82,7 +78,6 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown);
  * Represents an NVM Express device.  Each nvme_dev is a PCI function.
  */
 struct nvme_dev {
-	struct list_head node;
 	struct nvme_queue **queues;
 	struct blk_mq_tag_set tagset;
 	struct blk_mq_tag_set admin_tagset;
@@ -101,6 +96,7 @@ struct nvme_dev {
 	struct work_struct scan_work;
 	struct work_struct remove_work;
 	struct work_struct async_work;
+	struct timer_list watchdog_timer;
 	struct mutex shutdown_lock;
 	bool subsystem;
 	void __iomem *cmb;
@@ -1334,36 +1330,26 @@ static int nvme_configure_admin_queue(struct nvme_dev *dev)
 	return result;
 }
 
-static int nvme_kthread(void *data)
+static void nvme_watchdog_timer(unsigned long data)
 {
-	struct nvme_dev *dev, *next;
+	struct nvme_dev *dev = (struct nvme_dev *)data;
+	u32 csts = readl(dev->bar + NVME_REG_CSTS);
 
-	while (!kthread_should_stop()) {
-		set_current_state(TASK_INTERRUPTIBLE);
-		spin_lock(&dev_list_lock);
-		list_for_each_entry_safe(dev, next, &dev_list, node) {
-			u32 csts = readl(dev->bar + NVME_REG_CSTS);
-
-			/*
-			 * Skip controllers currently under reset.
-			 */
-			if (work_pending(&dev->reset_work) || work_busy(&dev->reset_work))
-				continue;
-
-			if ((dev->subsystem && (csts & NVME_CSTS_NSSRO)) ||
-							csts & NVME_CSTS_CFS) {
-				if (queue_work(nvme_workq, &dev->reset_work)) {
-					dev_warn(dev->ctrl.device,
-						"Failed status: %x, reset controller\n",
-						readl(dev->bar + NVME_REG_CSTS));
-				}
-				continue;
-			}
+	/*
+	 * Skip controllers currently under reset.
+	 */
+	if (!work_pending(&dev->reset_work) && !work_busy(&dev->reset_work) &&
+	    ((csts & NVME_CSTS_CFS) ||
+	     (dev->subsystem && (csts & NVME_CSTS_NSSRO)))) {
+		if (queue_work(nvme_workq, &dev->reset_work)) {
+			dev_warn(dev->dev,
+				"Failed status: 0x%x, reset controller.\n",
+				csts);
 		}
-		spin_unlock(&dev_list_lock);
-		schedule_timeout(round_jiffies_relative(HZ));
+		return;
 	}
-	return 0;
+
+	mod_timer(&dev->watchdog_timer, round_jiffies(jiffies + HZ));
 }
 
 static int nvme_create_io_queues(struct nvme_dev *dev)
@@ -1777,56 +1763,12 @@ static void nvme_dev_unmap(struct nvme_dev *dev)
 	}
 }
 
-static int nvme_dev_list_add(struct nvme_dev *dev)
-{
-	bool start_thread = false;
-
-	spin_lock(&dev_list_lock);
-	if (list_empty(&dev_list) && IS_ERR_OR_NULL(nvme_thread)) {
-		start_thread = true;
-		nvme_thread = NULL;
-	}
-	list_add(&dev->node, &dev_list);
-	spin_unlock(&dev_list_lock);
-
-	if (start_thread) {
-		nvme_thread = kthread_run(nvme_kthread, NULL, "nvme");
-		wake_up_all(&nvme_kthread_wait);
-	} else
-		wait_event_killable(nvme_kthread_wait, nvme_thread);
-
-	if (IS_ERR_OR_NULL(nvme_thread))
-		return nvme_thread ? PTR_ERR(nvme_thread) : -EINTR;
-
-	return 0;
-}
-
-/*
-* Remove the node from the device list and check
-* for whether or not we need to stop the nvme_thread.
-*/
-static void nvme_dev_list_remove(struct nvme_dev *dev)
-{
-	struct task_struct *tmp = NULL;
-
-	spin_lock(&dev_list_lock);
-	list_del_init(&dev->node);
-	if (list_empty(&dev_list) && !IS_ERR_OR_NULL(nvme_thread)) {
-		tmp = nvme_thread;
-		nvme_thread = NULL;
-	}
-	spin_unlock(&dev_list_lock);
-
-	if (tmp)
-		kthread_stop(tmp);
-}
-
 static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown)
 {
 	int i;
 	u32 csts = -1;
 
-	nvme_dev_list_remove(dev);
+	del_timer_sync(&dev->watchdog_timer);
 
 	mutex_lock(&dev->shutdown_lock);
 	if (dev->bar) {
@@ -1927,9 +1869,7 @@ static void nvme_reset_work(struct work_struct *work)
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
 	queue_work(nvme_workq, &dev->async_work);
 
-	result = nvme_dev_list_add(dev);
-	if (result)
-		goto remove;
+	mod_timer(&dev->watchdog_timer, round_jiffies(jiffies + HZ));
 
 	/*
 	 * Keep the controller around but remove all namespaces if we don't have
@@ -1946,8 +1886,6 @@ static void nvme_reset_work(struct work_struct *work)
 	clear_bit(NVME_CTRL_RESETTING, &dev->flags);
 	return;
 
- remove:
-	nvme_dev_list_remove(dev);
  free_tags:
 	nvme_dev_remove_admin(dev);
 	blk_put_queue(dev->ctrl.admin_q);
@@ -2055,11 +1993,12 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	dev->dev = get_device(&pdev->dev);
 	pci_set_drvdata(pdev, dev);
 
-	INIT_LIST_HEAD(&dev->node);
 	INIT_WORK(&dev->scan_work, nvme_dev_scan);
 	INIT_WORK(&dev->reset_work, nvme_reset_work);
 	INIT_WORK(&dev->remove_work, nvme_remove_dead_ctrl_work);
 	INIT_WORK(&dev->async_work, nvme_async_event_work);
+	setup_timer(&dev->watchdog_timer, nvme_watchdog_timer,
+		(unsigned long)dev);
 	mutex_init(&dev->shutdown_lock);
 	init_completion(&dev->ioq_wait);
 
@@ -2108,9 +2047,7 @@ static void nvme_remove(struct pci_dev *pdev)
 {
 	struct nvme_dev *dev = pci_get_drvdata(pdev);
 
-	spin_lock(&dev_list_lock);
-	list_del_init(&dev->node);
-	spin_unlock(&dev_list_lock);
+	del_timer_sync(&dev->watchdog_timer);
 
 	pci_set_drvdata(pdev, NULL);
 	flush_work(&dev->async_work);
@@ -2223,8 +2160,6 @@ static int __init nvme_init(void)
 {
 	int result;
 
-	init_waitqueue_head(&nvme_kthread_wait);
-
 	nvme_workq = alloc_workqueue("nvme", WQ_UNBOUND | WQ_MEM_RECLAIM, 0);
 	if (!nvme_workq)
 		return -ENOMEM;
@@ -2239,7 +2174,6 @@ static void __exit nvme_exit(void)
 {
 	pci_unregister_driver(&nvme_driver);
 	destroy_workqueue(nvme_workq);
-	BUG_ON(nvme_thread && !IS_ERR(nvme_thread));
 	_nvme_check_size();
 }
 
-- 
2.1.4



From hch at lst.de  Mon Feb 29 06:59:47 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 29 Feb 2016 15:59:47 +0100
Subject: [PATCH 4/4] nvme: return the whole CQE through the request
 passthrough interface
In-Reply-To: <1456757987-7244-1-git-send-email-hch@lst.de>
References: <1456757987-7244-1-git-send-email-hch@lst.de>
Message-ID: <1456757987-7244-5-git-send-email-hch@lst.de>

Both LighNVM and NVMe over Fabrics need to look at more than just the
status and result field.

Signed-off-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Matias Bj?rling <m at bjorling.me>
Reviewed-by: Jay Freyensee <james.p.freyensee at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Signed-off-by: Sagi Grimberg <sagig at mellanox.com>
---
 drivers/nvme/host/core.c | 27 +++++++++++++++++++--------
 drivers/nvme/host/nvme.h |  3 ++-
 drivers/nvme/host/pci.c  | 11 +++--------
 3 files changed, 24 insertions(+), 17 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 07b7ec69..66fd3d9 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -132,7 +132,6 @@ struct request *nvme_alloc_request(struct request_queue *q,
 
 	req->cmd = (unsigned char *)cmd;
 	req->cmd_len = sizeof(struct nvme_command);
-	req->special = (void *)0;
 
 	return req;
 }
@@ -143,7 +142,8 @@ EXPORT_SYMBOL_GPL(nvme_alloc_request);
  * if the result is positive, it's an NVM Express status code
  */
 int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
-		void *buffer, unsigned bufflen, u32 *result, unsigned timeout)
+		struct nvme_completion *cqe, void *buffer, unsigned bufflen,
+		unsigned timeout)
 {
 	struct request *req;
 	int ret;
@@ -153,6 +153,7 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		return PTR_ERR(req);
 
 	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	req->special = cqe;
 
 	if (buffer && bufflen) {
 		ret = blk_rq_map_kern(q, req, buffer, bufflen, GFP_KERNEL);
@@ -161,8 +162,6 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 	}
 
 	blk_execute_rq(req->q, NULL, req, 0);
-	if (result)
-		*result = (u32)(uintptr_t)req->special;
 	ret = req->errors;
  out:
 	blk_mq_free_request(req);
@@ -172,7 +171,7 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void *buffer, unsigned bufflen)
 {
-	return __nvme_submit_sync_cmd(q, cmd, buffer, bufflen, NULL, 0);
+	return __nvme_submit_sync_cmd(q, cmd, NULL, buffer, bufflen, 0);
 }
 EXPORT_SYMBOL_GPL(nvme_submit_sync_cmd);
 
@@ -182,6 +181,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		u32 *result, unsigned timeout)
 {
 	bool write = cmd->common.opcode & 1;
+	struct nvme_completion cqe;
 	struct nvme_ns *ns = q->queuedata;
 	struct gendisk *disk = ns ? ns->disk : NULL;
 	struct request *req;
@@ -194,6 +194,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		return PTR_ERR(req);
 
 	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	req->special = &cqe;
 
 	if (ubuffer && bufflen) {
 		ret = blk_rq_map_user(q, req, NULL, ubuffer, bufflen,
@@ -248,7 +249,7 @@ int __nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 	blk_execute_rq(req->q, disk, req, 0);
 	ret = req->errors;
 	if (result)
-		*result = (u32)(uintptr_t)req->special;
+		*result = le32_to_cpu(cqe.result);
 	if (meta && !ret && !write) {
 		if (copy_to_user(meta_buffer, meta, meta_len))
 			ret = -EFAULT;
@@ -329,6 +330,8 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
 					dma_addr_t dma_addr, u32 *result)
 {
 	struct nvme_command c;
+	struct nvme_completion cqe;
+	int ret;
 
 	memset(&c, 0, sizeof(c));
 	c.features.opcode = nvme_admin_get_features;
@@ -336,13 +339,18 @@ int nvme_get_features(struct nvme_ctrl *dev, unsigned fid, unsigned nsid,
 	c.features.prp1 = cpu_to_le64(dma_addr);
 	c.features.fid = cpu_to_le32(fid);
 
-	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
+	ret = __nvme_submit_sync_cmd(dev->admin_q, &c, &cqe, NULL, 0, 0);
+	if (ret >= 0)
+		*result = le32_to_cpu(cqe.result);
+	return ret;
 }
 
 int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 					dma_addr_t dma_addr, u32 *result)
 {
 	struct nvme_command c;
+	struct nvme_completion cqe;
+	int ret;
 
 	memset(&c, 0, sizeof(c));
 	c.features.opcode = nvme_admin_set_features;
@@ -350,7 +358,10 @@ int nvme_set_features(struct nvme_ctrl *dev, unsigned fid, unsigned dword11,
 	c.features.fid = cpu_to_le32(fid);
 	c.features.dword11 = cpu_to_le32(dword11);
 
-	return __nvme_submit_sync_cmd(dev->admin_q, &c, NULL, 0, result, 0);
+	ret = __nvme_submit_sync_cmd(dev->admin_q, &c, &cqe, NULL, 0, 0);
+	if (ret >= 0)
+		*result = le32_to_cpu(cqe.result);
+	return ret;
 }
 
 int nvme_get_log_page(struct nvme_ctrl *dev, struct nvme_smart_log **log)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 63ba8a5..2ac7539 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -248,7 +248,8 @@ void nvme_requeue_req(struct request *req);
 int nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void *buf, unsigned bufflen);
 int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
-		void *buffer, unsigned bufflen,  u32 *result, unsigned timeout);
+		struct nvme_completion *cqe, void *buffer, unsigned bufflen,
+		unsigned timeout);
 int nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd,
 		void __user *ubuffer, unsigned bufflen, u32 *result,
 		unsigned timeout);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index a623360..d47b087 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -748,10 +748,8 @@ static void __nvme_process_cq(struct nvme_queue *nvmeq, unsigned int *tag)
 		}
 
 		req = blk_mq_tag_to_rq(*nvmeq->tags, cqe.command_id);
-		if (req->cmd_type == REQ_TYPE_DRV_PRIV) {
-			u32 result = le32_to_cpu(cqe.result);
-			req->special = (void *)(uintptr_t)result;
-		}
+		if (req->cmd_type == REQ_TYPE_DRV_PRIV && req->special)
+			memcpy(req->special, &cqe, sizeof(cqe));
 		blk_mq_complete_request(req, status >> 1);
 
 	}
@@ -901,13 +899,10 @@ static void abort_endio(struct request *req, int error)
 {
 	struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
 	struct nvme_queue *nvmeq = iod->nvmeq;
-	u32 result = (u32)(uintptr_t)req->special;
 	u16 status = req->errors;
 
-	dev_warn(nvmeq->dev->ctrl.device,
-		"Abort status:%x result:%x", status, result);
+	dev_warn(nvmeq->dev->ctrl.device, "Abort status: 0x%x", status);
 	atomic_inc(&nvmeq->dev->ctrl.abort_limit);
-
 	blk_mq_free_request(req);
 }
 
-- 
2.1.4



From hch at lst.de  Mon Feb 29 06:59:45 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 29 Feb 2016 15:59:45 +0100
Subject: [PATCH 2/4] nvme: don't poll the CQ from the kthread
In-Reply-To: <1456757987-7244-1-git-send-email-hch@lst.de>
References: <1456757987-7244-1-git-send-email-hch@lst.de>
Message-ID: <1456757987-7244-3-git-send-email-hch@lst.de>

There is no reason to do unconditional polling of CQs per the NVMe
spec.

Signed-off-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
 drivers/nvme/host/pci.c | 12 ------------
 1 file changed, 12 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 21b0be4..10839f7 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1156,9 +1156,6 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, int qid,
 	nvmeq->qid = qid;
 	nvmeq->cq_vector = -1;
 	dev->queues[qid] = nvmeq;
-
-	/* make sure queue descriptor is set before queue count, for kthread */
-	mb();
 	dev->queue_count++;
 
 	return nvmeq;
@@ -1345,7 +1342,6 @@ static int nvme_kthread(void *data)
 		set_current_state(TASK_INTERRUPTIBLE);
 		spin_lock(&dev_list_lock);
 		list_for_each_entry_safe(dev, next, &dev_list, node) {
-			int i;
 			u32 csts = readl(dev->bar + NVME_REG_CSTS);
 
 			/*
@@ -1363,14 +1359,6 @@ static int nvme_kthread(void *data)
 				}
 				continue;
 			}
-			for (i = 0; i < dev->queue_count; i++) {
-				struct nvme_queue *nvmeq = dev->queues[i];
-				if (!nvmeq)
-					continue;
-				spin_lock_irq(&nvmeq->q_lock);
-				nvme_process_cq(nvmeq);
-				spin_unlock_irq(&nvmeq->q_lock);
-			}
 		}
 		spin_unlock(&dev_list_lock);
 		schedule_timeout(round_jiffies_relative(HZ));
-- 
2.1.4



From hch at lst.de  Mon Feb 29 06:59:43 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 29 Feb 2016 15:59:43 +0100
Subject: roolup of my pending NVMe patches for 4.6
Message-ID: <1456757987-7244-1-git-send-email-hch@lst.de>

All of them had a fair amount of review, but for the last one an ACK
from Keith would be nice.



From hch at lst.de  Mon Feb 29 06:59:44 2016
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 29 Feb 2016 15:59:44 +0100
Subject: [PATCH 1/4] nvme: use a work item to submit async event requests
In-Reply-To: <1456757987-7244-1-git-send-email-hch@lst.de>
References: <1456757987-7244-1-git-send-email-hch@lst.de>
Message-ID: <1456757987-7244-2-git-send-email-hch@lst.de>

Use a dedicated work item to submit async event requests instead of the
global kthread.  This simplifies the code and reduces the latencies to
resubmit a request once an even notification happened.

Signed-off-by: Christoph Hellwig <hch at lst.de>
Reviewed-by: Keith Busch <keith.busch at intel.com>
Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
---
 drivers/nvme/host/pci.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index fec7479..21b0be4 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -100,6 +100,7 @@ struct nvme_dev {
 	struct work_struct reset_work;
 	struct work_struct scan_work;
 	struct work_struct remove_work;
+	struct work_struct async_work;
 	struct mutex shutdown_lock;
 	bool subsystem;
 	void __iomem *cmb;
@@ -281,8 +282,11 @@ static void nvme_complete_async_event(struct nvme_dev *dev,
 	u16 status = le16_to_cpu(cqe->status) >> 1;
 	u32 result = le32_to_cpu(cqe->result);
 
-	if (status == NVME_SC_SUCCESS || status == NVME_SC_ABORT_REQ)
+	if (status == NVME_SC_SUCCESS || status == NVME_SC_ABORT_REQ) {
 		++dev->ctrl.event_limit;
+		queue_work(nvme_workq, &dev->async_work);
+	}
+
 	if (status != NVME_SC_SUCCESS)
 		return;
 
@@ -816,15 +820,22 @@ static int nvme_poll(struct blk_mq_hw_ctx *hctx, unsigned int tag)
 	return 0;
 }
 
-static void nvme_submit_async_event(struct nvme_dev *dev)
+static void nvme_async_event_work(struct work_struct *work)
 {
+	struct nvme_dev *dev = container_of(work, struct nvme_dev, async_work);
+	struct nvme_queue *nvmeq = dev->queues[0];
 	struct nvme_command c;
 
 	memset(&c, 0, sizeof(c));
 	c.common.opcode = nvme_admin_async_event;
-	c.common.command_id = NVME_AQ_BLKMQ_DEPTH + --dev->ctrl.event_limit;
 
-	__nvme_submit_cmd(dev->queues[0], &c);
+	spin_lock_irq(&nvmeq->q_lock);
+	while (dev->ctrl.event_limit > 0) {
+		c.common.command_id = NVME_AQ_BLKMQ_DEPTH +
+			--dev->ctrl.event_limit;
+		__nvme_submit_cmd(nvmeq, &c);
+	}
+	spin_unlock_irq(&nvmeq->q_lock);
 }
 
 static int adapter_delete_queue(struct nvme_dev *dev, u8 opcode, u16 id)
@@ -1358,9 +1369,6 @@ static int nvme_kthread(void *data)
 					continue;
 				spin_lock_irq(&nvmeq->q_lock);
 				nvme_process_cq(nvmeq);
-
-				while (i == 0 && dev->ctrl.event_limit > 0)
-					nvme_submit_async_event(dev);
 				spin_unlock_irq(&nvmeq->q_lock);
 			}
 		}
@@ -1929,6 +1937,7 @@ static void nvme_reset_work(struct work_struct *work)
 		goto free_tags;
 
 	dev->ctrl.event_limit = NVME_NR_AEN_COMMANDS;
+	queue_work(nvme_workq, &dev->async_work);
 
 	result = nvme_dev_list_add(dev);
 	if (result)
@@ -2062,6 +2071,7 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	INIT_WORK(&dev->scan_work, nvme_dev_scan);
 	INIT_WORK(&dev->reset_work, nvme_reset_work);
 	INIT_WORK(&dev->remove_work, nvme_remove_dead_ctrl_work);
+	INIT_WORK(&dev->async_work, nvme_async_event_work);
 	mutex_init(&dev->shutdown_lock);
 	init_completion(&dev->ioq_wait);
 
@@ -2115,6 +2125,7 @@ static void nvme_remove(struct pci_dev *pdev)
 	spin_unlock(&dev_list_lock);
 
 	pci_set_drvdata(pdev, NULL);
+	flush_work(&dev->async_work);
 	flush_work(&dev->reset_work);
 	flush_work(&dev->scan_work);
 	nvme_remove_namespaces(&dev->ctrl);
-- 
2.1.4



From keith.busch at intel.com  Mon Feb 29 07:42:52 2016
From: keith.busch at intel.com (Keith Busch)
Date: Mon, 29 Feb 2016 15:42:52 +0000
Subject: [PATCH 4/4] nvme: return the whole CQE through the request
 passthrough interface
In-Reply-To: <1456757987-7244-5-git-send-email-hch@lst.de>
References: <1456757987-7244-1-git-send-email-hch@lst.de>
 <1456757987-7244-5-git-send-email-hch@lst.de>
Message-ID: <20160229154251.GA15406@localhost.localdomain>

On Mon, Feb 29, 2016 at 03:59:47PM +0100, Christoph Hellwig wrote:
> Both LighNVM and NVMe over Fabrics need to look at more than just the
> status and result field.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> Reviewed-by: Matias Bj?rling <m at bjorling.me>
> Reviewed-by: Jay Freyensee <james.p.freyensee at intel.com>
> Reviewed-by: Sagi Grimberg <sagig at mellanox.com>
> Signed-off-by: Sagi Grimberg <sagig at mellanox.com>

Looks good,

Reviewed-by: Keith Busch <keith.busch at intel.com>


From axboe at fb.com  Mon Feb 29 07:47:48 2016
From: axboe at fb.com (Jens Axboe)
Date: Mon, 29 Feb 2016 08:47:48 -0700
Subject: roolup of my pending NVMe patches for 4.6
In-Reply-To: <1456757987-7244-1-git-send-email-hch@lst.de>
References: <1456757987-7244-1-git-send-email-hch@lst.de>
Message-ID: <56D46824.9000206@fb.com>

On 02/29/2016 07:59 AM, Christoph Hellwig wrote:
> All of them had a fair amount of review, but for the last one an ACK
> from Keith would be nice.

Queued them up for 4.6, with Keith's ack. Thanks!

-- 
Jens Axboe



From axboe at fb.com  Mon Feb 29 11:31:55 2016
From: axboe at fb.com (Jens Axboe)
Date: Mon, 29 Feb 2016 12:31:55 -0700
Subject: [PATCH] nvme: expose cntlid in sysfs
In-Reply-To: <1456521859-26357-1-git-send-email-mlin@kernel.org>
References: <1456521859-26357-1-git-send-email-mlin@kernel.org>
Message-ID: <56D49CAB.1070208@fb.com>

On 02/26/2016 02:24 PM, Ming Lin wrote:
> From: Ming Lin <ming.l at ssi.samsung.com>
>
> For NVMe over Fabrics, the cntlid will be used by systemd/udev to
> create link to the device, for example,
>
> /dev/disk/by-path/<fabrics-info>-<cntlid>-<namespace> -> /dev/nvme0n1

Added for 4.6 with the acks, thanks.

-- 
Jens Axboe



From willy at linux.intel.com  Mon Feb 29 12:56:07 2016
From: willy at linux.intel.com (Matthew Wilcox)
Date: Mon, 29 Feb 2016 15:56:07 -0500
Subject: [PATCH 2/4] nvme: don't poll the CQ from the kthread
In-Reply-To: <1456757987-7244-3-git-send-email-hch@lst.de>
References: <1456757987-7244-1-git-send-email-hch@lst.de>
 <1456757987-7244-3-git-send-email-hch@lst.de>
Message-ID: <20160229205607.GC3730@linux.intel.com>

On Mon, Feb 29, 2016 at 03:59:45PM +0100, Christoph Hellwig wrote:
> There is no reason to do unconditional polling of CQs per the NVMe
> spec.

Sure, per the spec there isn't.  But in reality, devices ship with
broken interrupt support.  What's your plan for dealing with the bug
reports from users who have misconfigured MSI support or IRQ lines that
are routed to the wrong place?  From my time maintaining SCSI drivers,
this wasn't an uncommon bug report.


