From leonro at nvidia.com  Thu Oct  1 01:01:01 2020
From: leonro at nvidia.com (Leon Romanovsky)
Date: Thu, 1 Oct 2020 08:01:01 +0300
Subject: [PATCH blk-next 0/2] Delete the get_vector_affinity leftovers
In-Reply-To: <20200929091358.421086-1-leon@kernel.org>
References: <20200929091358.421086-1-leon@kernel.org>
Message-ID: <20201001050101.GT3094@unreal>

On Tue, Sep 29, 2020 at 12:13:56PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro at nvidia.com>
>
> There are no drivers that implement .get_vector_affinity(), so delete
> the RDMA function and simplify block code.
>
> Thanks
>
> P.S. Probably it should go through block tree.
>
> Leon Romanovsky (2):
>   blk-mq-rdma: Delete not-used multi-queue RDMA map queue code
>   RDMA/core: Delete not-implemented get_vector_affinity

Jens, Keith

How can we progress here?

Thanks

>
>  block/Kconfig                    |  5 ----
>  block/Makefile                   |  1 -
>  block/blk-mq-rdma.c              | 44 --------------------------------
>  drivers/infiniband/core/device.c |  1 -
>  drivers/nvme/host/rdma.c         |  7 ++---
>  include/linux/blk-mq-rdma.h      | 11 --------
>  include/rdma/ib_verbs.h          | 23 -----------------
>  7 files changed, 2 insertions(+), 90 deletions(-)
>  delete mode 100644 block/blk-mq-rdma.c
>  delete mode 100644 include/linux/blk-mq-rdma.h
>
> --
> 2.26.2
>


From colyli at suse.de  Thu Oct  1 03:54:01 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:01 +0800
Subject: [PATCH v9 0/7] Introduce sendpage_ok() to detect misused sendpage in
 network related drivers
Message-ID: <20201001075408.25508-1-colyli@suse.de>

This series was original by a bug fix in nvme-over-tcp driver which only
checked whether a page was allocated from slab allcoator, but forgot to
check its page_count: The page handled by sendpage should be neither a
Slab page nor 0 page_count page.

As Sagi Grimberg suggested, the original fix is refind to a more common
inline routine:
    static inline bool sendpage_ok(struct page *page)
    {
        return  (!PageSlab(page) && page_count(page) >= 1);
    }
If sendpage_ok() returns true, the checking page can be handled by the
concrete zero-copy sendpage method in network layer.

The v9 series has 7 patches, no change from v8 series,
- The 1st patch in this series introduces sendpage_ok() in header file
  include/linux/net.h.
- The 2nd patch adds WARN_ONCE() for improper zero-copy send in
  kernel_sendpage().
- The 3rd patch fixes the page checking issue in nvme-over-tcp driver.
- The 4th patch adds page_count check by using sendpage_ok() in
  do_tcp_sendpages() as Eric Dumazet suggested.
- The 5th and 6th patches just replace existing open coded checks with
  the inline sendpage_ok() routine.

Coly Li

Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Cc: Chris Leech <cleech at redhat.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Cong Wang <amwang at redhat.com>
Cc: David S. Miller <davem at davemloft.net>
Cc: Eric Dumazet <eric.dumazet at gmail.com>
Cc: Hannes Reinecke <hare at suse.de>
Cc: Ilya Dryomov <idryomov at gmail.com>
Cc: Jan Kara <jack at suse.com>
Cc: Jeff Layton <jlayton at kernel.org>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Lee Duncan <lduncan at suse.com>
Cc: Mike Christie <michaelc at cs.wisc.edu>
Cc: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Vasily Averin <vvs at virtuozzo.com>
Cc: Vlastimil Babka <vbabka at suse.com>
---
Changelog:
v9, fix a typo pointed out by Greg KH.
    add Acked-by tags from Martin K. Petersen and Ilya Dryomov.
v8: add WARN_ONCE() in kernel_sendpage() as Christoph suggested.
v7: remove outer brackets from the return line of sendpage_ok() as
    Eric Dumazet suggested.
v6: fix page check in do_tcp_sendpages(), as Eric Dumazet suggested.
    replace other open coded checks with sendpage_ok() in libceph,
    iscsi drivers.
v5, include linux/mm.h in include/linux/net.h
v4, change sendpage_ok() as an inline helper, and post it as
    separate patch, as Christoph Hellwig suggested.
v3, introduce a more common sendpage_ok() as Sagi Grimberg suggested.
v2, fix typo in patch subject
v1, the initial version.


Coly Li (7):
  net: introduce helper sendpage_ok() in include/linux/net.h
  net: add WARN_ONCE in kernel_sendpage() for improper zero-copy send
  nvme-tcp: check page by sendpage_ok() before calling kernel_sendpage()
  tcp: use sendpage_ok() to detect misused .sendpage
  drbd: code cleanup by using sendpage_ok() to check page for
    kernel_sendpage()
  scsi: libiscsi: use sendpage_ok() in iscsi_tcp_segment_map()
  libceph: use sendpage_ok() in ceph_tcp_sendpage()

 drivers/block/drbd/drbd_main.c |  2 +-
 drivers/nvme/host/tcp.c        |  7 +++----
 drivers/scsi/libiscsi_tcp.c    |  2 +-
 include/linux/net.h            | 16 ++++++++++++++++
 net/ceph/messenger.c           |  2 +-
 net/ipv4/tcp.c                 |  3 ++-
 net/socket.c                   |  6 ++++--
 7 files changed, 28 insertions(+), 10 deletions(-)

-- 
2.26.2



From colyli at suse.de  Thu Oct  1 03:54:02 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:02 +0800
Subject: [PATCH v9 1/7] net: introduce helper sendpage_ok() in
 include/linux/net.h
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001075408.25508-2-colyli@suse.de>

The original problem was from nvme-over-tcp code, who mistakenly uses
kernel_sendpage() to send pages allocated by __get_free_pages() without
__GFP_COMP flag. Such pages don't have refcount (page_count is 0) on
tail pages, sending them by kernel_sendpage() may trigger a kernel panic
from a corrupted kernel heap, because these pages are incorrectly freed
in network stack as page_count 0 pages.

This patch introduces a helper sendpage_ok(), it returns true if the
checking page,
- is not slab page: PageSlab(page) is false.
- has page refcount: page_count(page) is not zero

All drivers who want to send page to remote end by kernel_sendpage()
may use this helper to check whether the page is OK. If the helper does
not return true, the driver should try other non sendpage method (e.g.
sock_no_sendpage()) to handle the page.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Hannes Reinecke <hare at suse.de>
Cc: Jan Kara <jack at suse.com>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Vlastimil Babka <vbabka at suse.com>
Cc: stable at vger.kernel.org
---
 include/linux/net.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/linux/net.h b/include/linux/net.h
index d48ff1180879..ae713c851342 100644
--- a/include/linux/net.h
+++ b/include/linux/net.h
@@ -21,6 +21,7 @@
 #include <linux/rcupdate.h>
 #include <linux/once.h>
 #include <linux/fs.h>
+#include <linux/mm.h>
 #include <linux/sockptr.h>
 
 #include <uapi/linux/net.h>
@@ -286,6 +287,21 @@ do {									\
 #define net_get_random_once_wait(buf, nbytes)			\
 	get_random_once_wait((buf), (nbytes))
 
+/*
+ * E.g. XFS meta- & log-data is in slab pages, or bcache meta
+ * data pages, or other high order pages allocated by
+ * __get_free_pages() without __GFP_COMP, which have a page_count
+ * of 0 and/or have PageSlab() set. We cannot use send_page for
+ * those, as that does get_page(); put_page(); and would cause
+ * either a VM_BUG directly, or __page_cache_release a page that
+ * would actually still be referenced by someone, leading to some
+ * obscure delayed Oops somewhere else.
+ */
+static inline bool sendpage_ok(struct page *page)
+{
+	return !PageSlab(page) && page_count(page) >= 1;
+}
+
 int kernel_sendmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec,
 		   size_t num, size_t len);
 int kernel_sendmsg_locked(struct sock *sk, struct msghdr *msg,
-- 
2.26.2



From colyli at suse.de  Thu Oct  1 03:54:03 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:03 +0800
Subject: [PATCH v9 2/7] net: add WARN_ONCE in kernel_sendpage() for improper
 zero-copy send
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001075408.25508-3-colyli@suse.de>

If a page sent into kernel_sendpage() is a slab page or it doesn't have
ref_count, this page is improper to send by the zero copy sendpage()
method. Otherwise such page might be unexpected released in network code
path and causes impredictable panic due to kernel memory management data
structure corruption.

This path adds a WARN_ON() on the sending page before sends it into the
concrete zero-copy sendpage() method, if the page is improper for the
zero-copy sendpage() method, a warning message can be observed before
the consequential unpredictable kernel panic.

This patch does not change existing kernel_sendpage() behavior for the
improper page zero-copy send, it just provides hint warning message for
following potential panic due the kernel memory heap corruption.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Cong Wang <amwang at redhat.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: David S. Miller <davem at davemloft.net>
Cc: Sridhar Samudrala <sri at us.ibm.com>
---
 net/socket.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/net/socket.c b/net/socket.c
index 0c0144604f81..771456a1d947 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -3638,9 +3638,11 @@ EXPORT_SYMBOL(kernel_getpeername);
 int kernel_sendpage(struct socket *sock, struct page *page, int offset,
 		    size_t size, int flags)
 {
-	if (sock->ops->sendpage)
+	if (sock->ops->sendpage) {
+		/* Warn in case the improper page to zero-copy send */
+		WARN_ONCE(!sendpage_ok(page));
 		return sock->ops->sendpage(sock, page, offset, size, flags);
-
+	}
 	return sock_no_sendpage(sock, page, offset, size, flags);
 }
 EXPORT_SYMBOL(kernel_sendpage);
-- 
2.26.2



From colyli at suse.de  Thu Oct  1 03:54:04 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:04 +0800
Subject: [PATCH v9 3/7] nvme-tcp: check page by sendpage_ok() before calling
 kernel_sendpage()
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001075408.25508-4-colyli@suse.de>

Currently nvme_tcp_try_send_data() doesn't use kernel_sendpage() to
send slab pages. But for pages allocated by __get_free_pages() without
__GFP_COMP, which also have refcount as 0, they are still sent by
kernel_sendpage() to remote end, this is problematic.

The new introduced helper sendpage_ok() checks both PageSlab tag and
page_count counter, and returns true if the checking page is OK to be
sent by kernel_sendpage().

This patch fixes the page checking issue of nvme_tcp_try_send_data()
with sendpage_ok(). If sendpage_ok() returns true, send this page by
kernel_sendpage(), otherwise use sock_no_sendpage to handle this page.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Hannes Reinecke <hare at suse.de>
Cc: Jan Kara <jack at suse.com>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Vlastimil Babka <vbabka at suse.com>
Cc: stable at vger.kernel.org
---
 drivers/nvme/host/tcp.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8f4f29f18b8c..d6a3e1487354 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -913,12 +913,11 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_request *req)
 		else
 			flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST;
 
-		/* can't zcopy slab pages */
-		if (unlikely(PageSlab(page))) {
-			ret = sock_no_sendpage(queue->sock, page, offset, len,
+		if (sendpage_ok(page)) {
+			ret = kernel_sendpage(queue->sock, page, offset, len,
 					flags);
 		} else {
-			ret = kernel_sendpage(queue->sock, page, offset, len,
+			ret = sock_no_sendpage(queue->sock, page, offset, len,
 					flags);
 		}
 		if (ret <= 0)
-- 
2.26.2



From colyli at suse.de  Thu Oct  1 03:54:05 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:05 +0800
Subject: [PATCH v9 4/7] tcp: use sendpage_ok() to detect misused .sendpage
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001075408.25508-5-colyli@suse.de>

commit a10674bf2406 ("tcp: detecting the misuse of .sendpage for Slab
objects") adds the checks for Slab pages, but the pages don't have
page_count are still missing from the check.

Network layer's sendpage method is not designed to send page_count 0
pages neither, therefore both PageSlab() and page_count() should be
both checked for the sending page. This is exactly what sendpage_ok()
does.

This patch uses sendpage_ok() in do_tcp_sendpages() to detect misused
.sendpage, to make the code more robust.

Fixes: a10674bf2406 ("tcp: detecting the misuse of .sendpage for Slab objects")
Suggested-by: Eric Dumazet <eric.dumazet at gmail.com>
Signed-off-by: Coly Li <colyli at suse.de>
Cc: Vasily Averin <vvs at virtuozzo.com>
Cc: David S. Miller <davem at davemloft.net>
Cc: stable at vger.kernel.org
---
 net/ipv4/tcp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 31f3b858db81..2135ee7c806d 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -970,7 +970,8 @@ ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset,
 	long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT);
 
 	if (IS_ENABLED(CONFIG_DEBUG_VM) &&
-	    WARN_ONCE(PageSlab(page), "page must not be a Slab one"))
+	    WARN_ONCE(!sendpage_ok(page),
+		      "page must not be a Slab one and have page_count > 0"))
 		return -EINVAL;
 
 	/* Wait for a connection to finish. One exception is TCP Fast Open
-- 
2.26.2



From colyli at suse.de  Thu Oct  1 03:54:06 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:06 +0800
Subject: [PATCH v9 5/7] drbd: code cleanup by using sendpage_ok() to check
 page for kernel_sendpage()
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001075408.25508-6-colyli@suse.de>

In _drbd_send_page() a page is checked by following code before sending
it by kernel_sendpage(),
        (page_count(page) < 1) || PageSlab(page)
If the check is true, this page won't be send by kernel_sendpage() and
handled by sock_no_sendpage().

This kind of check is exactly what macro sendpage_ok() does, which is
introduced into include/linux/net.h to solve a similar send page issue
in nvme-tcp code.

This patch uses macro sendpage_ok() to replace the open coded checks to
page type and refcount in _drbd_send_page(), as a code cleanup.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/block/drbd/drbd_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 04b6bde9419d..573dbf6f0c31 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -1553,7 +1553,7 @@ static int _drbd_send_page(struct drbd_peer_device *peer_device, struct page *pa
 	 * put_page(); and would cause either a VM_BUG directly, or
 	 * __page_cache_release a page that would actually still be referenced
 	 * by someone, leading to some obscure delayed Oops somewhere else. */
-	if (drbd_disable_sendpage || (page_count(page) < 1) || PageSlab(page))
+	if (drbd_disable_sendpage || !sendpage_ok(page))
 		return _drbd_no_send_page(peer_device, page, offset, size, msg_flags);
 
 	msg_flags |= MSG_NOSIGNAL;
-- 
2.26.2



From colyli at suse.de  Thu Oct  1 03:54:07 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:07 +0800
Subject: [PATCH v9 6/7] scsi: libiscsi: use sendpage_ok() in
 iscsi_tcp_segment_map()
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001075408.25508-7-colyli@suse.de>

In iscsci driver, iscsi_tcp_segment_map() uses the following code to
check whether the page should or not be handled by sendpage:
    if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))

The "page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)" part is to
make sure the page can be sent to network layer's zero copy path. This
part is exactly what sendpage_ok() does.

This patch uses  use sendpage_ok() in iscsi_tcp_segment_map() to replace
the original open coded checks.

Signed-off-by: Coly Li <colyli at suse.de>
Acked-by: Martin K. Petersen <martin.petersen at oracle.com>
Cc: Vasily Averin <vvs at virtuozzo.com>
Cc: Cong Wang <amwang at redhat.com>
Cc: Mike Christie <michaelc at cs.wisc.edu>
Cc: Lee Duncan <lduncan at suse.com>
Cc: Chris Leech <cleech at redhat.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Hannes Reinecke <hare at suse.de>
---
 drivers/scsi/libiscsi_tcp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
index 37e5d4e48c2f..83f14b2c8804 100644
--- a/drivers/scsi/libiscsi_tcp.c
+++ b/drivers/scsi/libiscsi_tcp.c
@@ -128,7 +128,7 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
 	 * coalescing neighboring slab objects into a single frag which
 	 * triggers one of hardened usercopy checks.
 	 */
-	if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
+	if (!recv && sendpage_ok(sg_page(sg)))
 		return;
 
 	if (recv) {
-- 
2.26.2



From colyli at suse.de  Thu Oct  1 03:54:08 2020
From: colyli at suse.de (Coly Li)
Date: Thu,  1 Oct 2020 15:54:08 +0800
Subject: [PATCH v9 7/7] libceph: use sendpage_ok() in ceph_tcp_sendpage()
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001075408.25508-8-colyli@suse.de>

In libceph, ceph_tcp_sendpage() does the following checks before handle
the page by network layer's zero copy sendpage method,
	if (page_count(page) >= 1 && !PageSlab(page))

This check is exactly what sendpage_ok() does. This patch replace the
open coded checks by sendpage_ok() as a code cleanup.

Signed-off-by: Coly Li <colyli at suse.de>
Acked-by: Jeff Layton <jlayton at kernel.org>
Cc: Ilya Dryomov <idryomov at gmail.com>
---
 net/ceph/messenger.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c
index bdfd66ba3843..d4d7a0e52491 100644
--- a/net/ceph/messenger.c
+++ b/net/ceph/messenger.c
@@ -575,7 +575,7 @@ static int ceph_tcp_sendpage(struct socket *sock, struct page *page,
 	 * coalescing neighboring slab objects into a single frag which
 	 * triggers one of hardened usercopy checks.
 	 */
-	if (page_count(page) >= 1 && !PageSlab(page))
+	if (sendpage_ok(page))
 		sendpage = sock->ops->sendpage;
 	else
 		sendpage = sock_no_sendpage;
-- 
2.26.2



From hare at suse.de  Thu Oct  1 04:55:00 2020
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 1 Oct 2020 10:55:00 +0200
Subject: [PATCH v9] nvme-fabrics: reject I/O to offline device
In-Reply-To: <319b8b1869f34a48b26fbd902883ed71@kioxia.com>
References: <319b8b1869f34a48b26fbd902883ed71@kioxia.com>
Message-ID: <b6f09a8e-e633-e303-1f39-cd380fd92f2c@suse.de>

On 9/29/20 5:27 PM, Victor Gladkov wrote:
> Commands get stuck while Host NVMe-oF controller is in reconnect state.
> NVMe controller enters into reconnect state when it loses connection with the
> target. It tries to reconnect every 10 seconds
> (default) until successful reconnection or until reconnect time-out is reached.
> The default reconnect time out is 10 minutes.
> 
> Applications are expecting commands to complete with success or error within
> a certain timeout (30 seconds by default).  The NVMe host is enforcing that
> timeout while it is connected, never the less, during reconnection, the timeout
> is not enforced and commands may get stuck for a long period or even forever.
> 
> To fix this long delay due to the default timeout we introduce new session
> parameter "fast_io_fail_tmo". The timeout is measured in seconds from the
> controller reconnect, any command beyond that timeout is rejected. The new
> parameter value may be passed during 'connect'.
> The default value of -1 means no timeout (similar to current behavior).
> 
> We add a new controller NVME_CTRL_FAILFAST_EXPIRED and respective
> delayed work that updates the NVME_CTRL_FAILFAST_EXPIRED flag.
> 
> When the controller is entering the CONNECTING state, we schedule the
> delayed_work based on failfast timeout value. If the transition is out of
> CONNECTING, terminate delayed work item and ensure failfast_expired is
> false. If delayed work item expires then set "NVME_CTRL_FAILFAST_EXPIRED"
> flag to true.
> 
> We also update nvmf_fail_nonready_command() and
> nvme_available_path() functions with check the
> "NVME_CTRL_FAILFAST_EXPIRED" controller flag.
> 
> For multipath (function nvme_available_path()):
> The path will not be considered available if
> "NVME_CTRL_FAILFAST_EXPIRED" controller flag  is set
> and the controller in NVME_CTRL_CONNECTING state.
> This prevents commands from getting stuck
> when available paths have tried to reconnect for too long.
> 
> Signed-off-by: Victor Gladkov <victor.gladkov at kioxia.com>
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> Reviewed-by: Hannes Reinecke <hare at suse.de>
> 
> ---
> Changes from V8:
> 
> 1. Added multipath behavior description as requested by Sagi Grimberg
>      (Fri Sep 18 16:38:58 EDT 2020)
> 
> Changes from V7:
> 
> 1. Expanded the patch description as requested by James Smart
>      (Thu Aug 13 11:00:25 EDT 2020)
> 
> Changes from V6:
> 
> 1. Changed according to Hannes Reinecke review:
>      in nvme_start_failfast_work() and nvme_stop_failfast_work() procedures.
> 
> Changes from V5:
> 
> 1. Drop the "off" string option for fast_io_fail_tmo.
> 
> Changes from V4:
> 
> 1. Remove subsysnqn from dev_info just keep "failfast expired"
>      in nvme_failfast_work().
> 2. Remove excess lock in nvme_failfast_work().
> 3. Change "timeout disabled" value to -1, '0' is to fail I/O right away now.
> 4. Add "off" string for fast_io_fail_tmo option as a -1 equivalent.
> 
> Changes from V3:
> 
> 1. BUG_FIX: Fix a bug in nvme_start_failfast_work() amd
>      nvme_stop_failfast_work() when accessing ctrl->opts as it will fail
>      for PCIe transport when is called nvme_change_ctrl_state()
>      from nvme_reset_work(), since we don't set the ctrl->opts for
>      PCIe transport.
> 2. Line wrap in nvme_start_failfast_work(), nvme_parse_option() and
>      for macro NVMF_ALLOWED_OPTS definition.
> 3. Just like all the state change code add a switch for newly
>      added state handling outside of the state machine in
>      nvme_state_change().
> 4. nvme_available_path() add /* fallthru */ after if..break inside
>      switch which is under list_for_each_entry_rcu().
> 5. Align newly added member nvmf_ctrl_options fast_io_fail_tmp.
> 6. Fix the tabs before if in nvme_available_path() and line wrap
>      for the same.
> 7. In nvme_failfast_work() use state != NVME_CTRL_CONNECTING
>      instead of == to get rid of the parentheses and avoid char > 80.
> 8. Get rid of the ";" at the end of the comment for @fast_io_fail_tmp.
> 9. Change the commit log style to match the one we have in the NVMe
>      repo.
> 
> Changes from V2:
> 
> 1. Several coding style and small fixes.
> 2. Fix the comment for NVMF_DEF_FAIL_FAST_TMO.
> 3. Don't call functionality from the state machine.
> 4. Rename fast_fail_tmo -> fast_io_fail_tmo to match SCSI
>      semantics.
> 
> Changes from V1:
> 
> 1. Add a new session parameter called "fast_fail_tmo". The timeout
>      is measured in seconds from the controller reconnect, any command
>      beyond that timeout is rejected. The new parameter value may be
>      passed during 'connect', and its default value is 30 seconds.
>      A value of 0 means no timeout (similar to current behavior).
> 2. Add a controller flag of "failfast_expired".
> 3. Add dedicated delayed_work that update the "failfast_expired"
>      controller flag.
> 4. When entering CONNECTING, schedule the delayed_work based on
>      failfast timeout value. If transition out of CONNECTING, terminate
>      delayed work item and ensure failfast_expired is false.
>      If delayed work item expires: set "failfast_expired" flag to true.
> 5. Update nvmf_fail_nonready_command() with check the
>      "failfast_expired" controller flag.
> 
> ---
>   drivers/nvme/host/core.c      | 49
> ++++++++++++++++++++++++++++++++++++++++++-
>   drivers/nvme/host/fabrics.c   | 25 +++++++++++++++++++---
>   drivers/nvme/host/fabrics.h   |  5 +++++
>   drivers/nvme/host/multipath.c |  5 ++++-
>   drivers/nvme/host/nvme.h      |  3 +++
>   5 files changed, 82 insertions(+), 5 deletions(-)
> 
I did some more experiments with this, and found that there are some 
issues with ANA handling.
If reconnect works, but the ANA state indicates that we still can't sent 
I/O (eg by still being in transitioning), we hit the 'requeueing I/O' 
state despite fast_io_fail_tmo being set. Not sure if that's the 
expected outcome.
For that it might be better to move the FAILFAST_EXPIRED bit into the 
namespace, as then we could selectively clear the bit in 
nvme_failfast_work():

@@ -151,12 +151,16 @@ EXPORT_SYMBOL_GPL(nvme_try_sched_reset);
  static void nvme_failfast_work(struct work_struct *work) {
         struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
                         struct nvme_ctrl, failfast_work);
+       struct nvme_ns *ns;

-       if (ctrl->state != NVME_CTRL_CONNECTING)
-               return;
-
-
-       set_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags);
+       down_read(&ctrl->namespaces_rwsem);
+       list_for_each_entry(ns, &ctrl->namespaces, list) {
+               if (ctrl->state != NVME_CTRL_LIVE ||
+                   (ns->ana_state != NVME_ANA_OPTIMIZED &&
+                    ns->ana_state != NVME_ANA_NONOPTIMIZED))
+                       set_bit(NVME_NS_FAILFAST_EXPIRED, &ns->flags);
+       }
+       up_read(&ctrl->namespaces_rwsem);
         dev_info(ctrl->device, "failfast expired\n");

...and we could leave the failfast worker running even after the 
controller transitioned to LIVE.
Cf the attached patch for details.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		           Kernel Storage Architect
hare at suse.de			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), GF: Felix Imend?rffer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-nvme-move-FAILFAST_EXPIRED-bit-into-namespace.patch
Type: text/x-patch
Size: 5553 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201001/fe47591b/attachment-0001.bin>

From lduncan at suse.com  Thu Oct  1 12:10:05 2020
From: lduncan at suse.com (Lee Duncan)
Date: Thu, 1 Oct 2020 09:10:05 -0700
Subject: [PATCH v9 6/7] scsi: libiscsi: use sendpage_ok() in
 iscsi_tcp_segment_map()
In-Reply-To: <20201001075408.25508-7-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
 <20201001075408.25508-7-colyli@suse.de>
Message-ID: <0df9cb0e-7fd1-5ba0-564a-378aa0b8955f@suse.com>

On 10/1/20 12:54 AM, Coly Li wrote:
> In iscsci driver, iscsi_tcp_segment_map() uses the following code to
> check whether the page should or not be handled by sendpage:
>     if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
> 
> The "page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)" part is to
> make sure the page can be sent to network layer's zero copy path. This
> part is exactly what sendpage_ok() does.
> 
> This patch uses  use sendpage_ok() in iscsi_tcp_segment_map() to replace
> the original open coded checks.
> 
> Signed-off-by: Coly Li <colyli at suse.de>
> Acked-by: Martin K. Petersen <martin.petersen at oracle.com>
> Cc: Vasily Averin <vvs at virtuozzo.com>
> Cc: Cong Wang <amwang at redhat.com>
> Cc: Mike Christie <michaelc at cs.wisc.edu>
> Cc: Lee Duncan <lduncan at suse.com>
> Cc: Chris Leech <cleech at redhat.com>
> Cc: Christoph Hellwig <hch at lst.de>
> Cc: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/scsi/libiscsi_tcp.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
> index 37e5d4e48c2f..83f14b2c8804 100644
> --- a/drivers/scsi/libiscsi_tcp.c
> +++ b/drivers/scsi/libiscsi_tcp.c
> @@ -128,7 +128,7 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
>  	 * coalescing neighboring slab objects into a single frag which
>  	 * triggers one of hardened usercopy checks.
>  	 */
> -	if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
> +	if (!recv && sendpage_ok(sg_page(sg)))
>  		return;
>  
>  	if (recv) {
> 

Reviewed-by: Lee Duncan <lduncan at suse.com>



From hch at lst.de  Thu Oct  1 13:14:57 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 1 Oct 2020 19:14:57 +0200
Subject: [PATCH 17/20] nvme: query namespae identifiers before adding
 the namespace
In-Reply-To: <20200930100453.GA191465@localhost.localdomain>
References: <20200928123502.435373-1-hch@lst.de>
 <20200928123502.435373-18-hch@lst.de>
 <20200930094411.GA189371@localhost.localdomain>
 <20200930100453.GA191465@localhost.localdomain>
Message-ID: <20201001171457.GA29475@lst.de>

On Wed, Sep 30, 2020 at 10:04:54AM +0000, Niklas Cassel wrote:
> > For the subject of this patch:
> > s/namespae/namespace

Fixed and force pushed.

> > nvme: remove the namespace identifier verification in __nvme_validate_ns
> > 
> > None of the identifiers (including the new CSI) can cange over the life
> > time of a namespace, so don't bother with the extra query here.
> > "
> > 
> > which now seems to have been squashed with this patch.
> > 
> > Squashing it is fine, but perhaps you could add that information in this
> > commit message?
> 
> Hm..
> 
> I now see that nvme_validate_or_alloc_ns() in v2 of this series does a
> nvme_identify_ns_descs(), while nvme_validate_or_alloc_ns() in v1 doesnt.
> 
> I don't see a change log, but considering this, my suggestion that you
> the above sentence from the squashed v1 commit no longer makes sense.
> 
> What was the reason for this change?

Where does this this still show up in a change log?

The isn't much point in checking the changing identifiers, but we pretty
much get it for free, so..


From hch at lst.de  Thu Oct  1 13:24:29 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 1 Oct 2020 19:24:29 +0200
Subject: [PATCH 1/3] nvmet: allow user to set the ns readonly
In-Reply-To: <20201001024427.40753-2-chaitanya.kulkarni@wdc.com>
References: <20201001024427.40753-1-chaitanya.kulkarni@wdc.com>
 <20201001024427.40753-2-chaitanya.kulkarni@wdc.com>
Message-ID: <20201001172429.GA29997@lst.de>

On Wed, Sep 30, 2020 at 07:44:25PM -0700, Chaitanya Kulkarni wrote:
> From: "Mark Ruijter" <mruijter at primelogic.nl>
> 
> User may want to export readonly ns to host in order to export
> read-only snapshots. Right now we don't have configfs attribute to
> allow user to mark readonly & export ns.
> 
> Add a configfs attribute such that user can set the ns and export ns
> readonly when it is not enabled.

I thin kwe also need to expose this as permantently write protected
throught the write protection feature on the get side, and ensure
a set features can't change the setting.


From Niklas.Cassel at wdc.com  Thu Oct  1 13:43:23 2020
From: Niklas.Cassel at wdc.com (Niklas Cassel)
Date: Thu, 1 Oct 2020 17:43:23 +0000
Subject: [PATCH 17/20] nvme: query namespae identifiers before adding the
 namespace
In-Reply-To: <20201001171457.GA29475@lst.de>
References: <20200928123502.435373-1-hch@lst.de>
 <20200928123502.435373-18-hch@lst.de>
 <20200930094411.GA189371@localhost.localdomain>
 <20200930100453.GA191465@localhost.localdomain>
 <20201001171457.GA29475@lst.de>
Message-ID: <20201001174321.GA343057@localhost.localdomain>

On Thu, Oct 01, 2020 at 07:14:57PM +0200, Christoph Hellwig wrote:
> On Wed, Sep 30, 2020 at 10:04:54AM +0000, Niklas Cassel wrote:
> > > For the subject of this patch:
> > > s/namespae/namespace
> 
> Fixed and force pushed.
> 
> > > nvme: remove the namespace identifier verification in __nvme_validate_ns
> > > 
> > > None of the identifiers (including the new CSI) can cange over the life
> > > time of a namespace, so don't bother with the extra query here.
> > > "
> > > 
> > > which now seems to have been squashed with this patch.
> > > 
> > > Squashing it is fine, but perhaps you could add that information in this
> > > commit message?
> > 
> > Hm..
> > 
> > I now see that nvme_validate_or_alloc_ns() in v2 of this series does a
> > nvme_identify_ns_descs(), while nvme_validate_or_alloc_ns() in v1 doesnt.
> > 
> > I don't see a change log, but considering this, my suggestion that you
> > the above sentence from the squashed v1 commit no longer makes sense.
> > 
> > What was the reason for this change?
> 
> Where does this this still show up in a change log?

The sentence only exists in the V1 patch series,
not in the V2 patch series. So that is all good.


My questions is rather:

In V1 nvme_validate_or_alloc_ns():
http://git.infradead.org/users/hch/misc.git/blob/refs/heads/nvme-scanning-cleanup:/drivers/nvme/host/core.c#l3975

You will only call nvme_identify_ns_descs() once, and once only per NSID,
in the nvme_alloc_ns() path.
Subsequent calls that goes through the nvme_validate_ns() path will not
call nvme_identify_ns_descs().


In V2 nvme_validate_or_alloc_ns():
http://git.infradead.org/users/hch/misc.git/blob/refs/heads/nvme-scanning-cleanup.2:/drivers/nvme/host/core.c#l3971

You will call nvme_identify_ns_descs() both for the initial nvme_alloc_ns()
path, but then also in each subsequent call, which goes through the
nvme_validate_ns() path.


V1 seems more optimized.
Since we know that none of the identifiers can change during the lifetime
of a namespace, why not do like V1, and only call it in the nvme_alloc_ns()
path?

Something like this:

static void nvme_validate_or_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
{
	...

	ns = nvme_find_get_ns(ctrl, nsid);
	if (ns) {
		nvme_validate_ns(ns, &ids);
		nvme_put_ns(ns);
		return;
	}

	/* only fetch the ns id desc list if we didn't find a ns */
	if (nvme_identify_ns_descs(ctrl, nsid, &ids))
		return;

	switch (ids.csi) {
	...
	}
	...
}


Is there something I'm missing, or why did you change this between V1 and V2?


Kind regards,
Niklas

From kbusch at kernel.org  Thu Oct  1 14:06:01 2020
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 1 Oct 2020 11:06:01 -0700
Subject: [PATCH 2/2] nvme-core: remove extra condition for vwc
In-Reply-To: <20200930232447.20677-3-chaitanya.kulkarni@wdc.com>
References: <20200930232447.20677-1-chaitanya.kulkarni@wdc.com>
 <20200930232447.20677-3-chaitanya.kulkarni@wdc.com>
Message-ID: <20201001180601.GB564827@dhcp-10-100-145-180.wdl.wdc.com>

On Wed, Sep 30, 2020 at 04:24:47PM -0700, Chaitanya Kulkarni wrote:
> @@ -1970,7 +1970,7 @@ static int nvme_configure_metadata(struct nvme_ns *ns, struct nvme_id_ns *id)
>  static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
>  		struct request_queue *q)
>  {
> -	bool vwc = false;
> +	bool vwc = ctrl->vwc & NVME_CTRL_VWC_PRESENT ? true : false;

No need for a ternary here. It can just be:

	bool vwc = ctrl->vwc & NVME_CTRL_VWC_PRESENT;


From chaitanya.kulkarni at wdc.com  Thu Oct  1 14:54:30 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 11:54:30 -0700
Subject: [PATCH V2 0/2] nvme-core: increamental cleanup
Message-ID: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>

Hi,

This is an increamental cleanup which was discussed during review of
"refactor the nvme scanning and validation code " [1]. This fixes minor
nits in nvme_validate_ns() and nvme_set_queue_limits().

Regards,
Chaitanya

* Changes from V1 :-

1. Remove the use of ternery operator in the second patch.

[1] http://lists.infradead.org/pipermail/linux-nvme/2020-September/019886.html

Chaitanya Kulkarni (2):
  nvme-core: remove extra variable
  nvme-core: remove extra condition for vwc

 drivers/nvme/host/core.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct  1 14:54:31 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 11:54:31 -0700
Subject: [PATCH V2 1/2] nvme-core: remove extra variable
In-Reply-To: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>
References: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201001185432.5464-2-chaitanya.kulkarni@wdc.com>

In nvme_validate_ns() the exra variable ctrl is used only twice.
Using ns->ctrl directly still maintains the redability and original
length of the lines in the code. Get rid of the extra variable ctrl &
use ns->ctrl directly.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 96dc87ea07b3..9a565096e072 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3931,20 +3931,19 @@ static void nvme_ns_remove_by_nsid(struct nvme_ctrl *ctrl, u32 nsid)
 
 static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
 {
-	struct nvme_ctrl *ctrl = ns->ctrl;
 	struct nvme_id_ns *id;
 	int ret = -ENODEV;
 
 	if (test_bit(NVME_NS_DEAD, &ns->flags))
 		goto out;
 
-	ret = nvme_identify_ns(ctrl, ns->head->ns_id, ids, &id);
+	ret = nvme_identify_ns(ns->ctrl, ns->head->ns_id, ids, &id);
 	if (ret)
 		goto out;
 
 	ret = -ENODEV;
 	if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
-		dev_err(ctrl->device,
+		dev_err(ns->ctrl->device,
 			"identifiers changed for nsid %d\n", ns->head->ns_id);
 		goto out_free_id;
 	}
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct  1 14:54:32 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 11:54:32 -0700
Subject: [PATCH V2 2/2] nvme-core: remove extra condition for vwc
In-Reply-To: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>
References: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201001185432.5464-3-chaitanya.kulkarni@wdc.com>

In nvme_set_queue_limits() we initialize vwc to false and later add
a condition to set vwc true. The value of the vwc can be declare
initialized which makes all the blk_queue_XXX() calls uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 9a565096e072..c20ce8eabe82 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1970,7 +1970,7 @@ static int nvme_configure_metadata(struct nvme_ns *ns, struct nvme_id_ns *id)
 static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
 		struct request_queue *q)
 {
-	bool vwc = false;
+	bool vwc = ctrl->vwc & NVME_CTRL_VWC_PRESENT;
 
 	if (ctrl->max_hw_sectors) {
 		u32 max_segments =
@@ -1982,8 +1982,6 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
 	}
 	blk_queue_virt_boundary(q, NVME_CTRL_PAGE_SIZE - 1);
 	blk_queue_dma_alignment(q, 7);
-	if (ctrl->vwc & NVME_CTRL_VWC_PRESENT)
-		vwc = true;
 	blk_queue_write_cache(q, vwc, vwc);
 }
 
-- 
2.22.1



From davem at davemloft.net  Thu Oct  1 15:43:45 2020
From: davem at davemloft.net (David Miller)
Date: Thu, 01 Oct 2020 12:43:45 -0700 (PDT)
Subject: [PATCH v9 0/7] Introduce sendpage_ok() to detect misused
 sendpage in network related drivers
In-Reply-To: <20201001075408.25508-1-colyli@suse.de>
References: <20201001075408.25508-1-colyli@suse.de>
Message-ID: <20201001.124345.2303686561459641833.davem@davemloft.net>

From: Coly Li <colyli at suse.de>
Date: Thu,  1 Oct 2020 15:54:01 +0800

> This series was original by a bug fix in nvme-over-tcp driver which only
> checked whether a page was allocated from slab allcoator, but forgot to
> check its page_count: The page handled by sendpage should be neither a
> Slab page nor 0 page_count page.
> 
> As Sagi Grimberg suggested, the original fix is refind to a more common
> inline routine:
>     static inline bool sendpage_ok(struct page *page)
>     {
>         return  (!PageSlab(page) && page_count(page) >= 1);
>     }
> If sendpage_ok() returns true, the checking page can be handled by the
> concrete zero-copy sendpage method in network layer.
> 
> The v9 series has 7 patches, no change from v8 series,
> - The 1st patch in this series introduces sendpage_ok() in header file
>   include/linux/net.h.
> - The 2nd patch adds WARN_ONCE() for improper zero-copy send in
>   kernel_sendpage().
> - The 3rd patch fixes the page checking issue in nvme-over-tcp driver.
> - The 4th patch adds page_count check by using sendpage_ok() in
>   do_tcp_sendpages() as Eric Dumazet suggested.
> - The 5th and 6th patches just replace existing open coded checks with
 ...

Series applied and queued up for -stable, thank you.


From davem at davemloft.net  Thu Oct  1 15:48:15 2020
From: davem at davemloft.net (David Miller)
Date: Thu, 01 Oct 2020 12:48:15 -0700 (PDT)
Subject: [PATCH v9 0/7] Introduce sendpage_ok() to detect misused
 sendpage in network related drivers
In-Reply-To: <20201001.124345.2303686561459641833.davem@davemloft.net>
References: <20201001075408.25508-1-colyli@suse.de>
 <20201001.124345.2303686561459641833.davem@davemloft.net>
Message-ID: <20201001.124815.793423380665613978.davem@davemloft.net>

From: David Miller <davem at davemloft.net>
Date: Thu, 01 Oct 2020 12:43:45 -0700 (PDT)

> Series applied and queued up for -stable, thank you.

Actually, this doesn't even build:

In file included from ./arch/x86/include/asm/bug.h:93,
                 from ./include/linux/bug.h:5,
                 from ./include/linux/mmdebug.h:5,
                 from ./include/linux/mm.h:9,
                 from net/socket.c:55:
net/socket.c: In function ?kernel_sendpage?:
./include/asm-generic/bug.h:97:3: error: too few arguments to function ?__warn_printk?
   97 |   __warn_printk(arg);     \
      |   ^~~~~~~~~~~~~

Was this even build tested?


From kbusch at kernel.org  Thu Oct  1 16:51:40 2020
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 1 Oct 2020 13:51:40 -0700
Subject: [PATCH V2 2/2] nvme-core: remove extra condition for vwc
In-Reply-To: <20201001185432.5464-3-chaitanya.kulkarni@wdc.com>
References: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>
 <20201001185432.5464-3-chaitanya.kulkarni@wdc.com>
Message-ID: <20201001205140.GA565143@dhcp-10-100-145-180.wdl.wdc.com>

On Thu, Oct 01, 2020 at 11:54:32AM -0700, Chaitanya Kulkarni wrote:
> In nvme_set_queue_limits() we initialize vwc to false and later add
> a condition to set vwc true. The value of the vwc can be declare
> initialized which makes all the blk_queue_XXX() calls uniform.

Looks fine.

Reviewed-by: Keith Busch <kbusch at kernel.org>


From kbusch at kernel.org  Thu Oct  1 17:35:11 2020
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 1 Oct 2020 14:35:11 -0700
Subject: [PATCHv4 1/3] block: add zone specific block statuses
In-Reply-To: <20200924205330.4043232-2-kbusch@kernel.org>
References: <20200924205330.4043232-1-kbusch@kernel.org>
 <20200924205330.4043232-2-kbusch@kernel.org>
Message-ID: <20201001213511.GB565143@dhcp-10-100-145-180.wdl.wdc.com>

On Thu, Sep 24, 2020 at 01:53:28PM -0700, Keith Busch wrote:
> A zoned device with limited resources to open or activate zones may
> return an error when the host exceeds those limits. The same command may
> be successful if retried later, but the host needs to wait for specific
> zone states before it should expect a retry to succeed. Have the block
> layer provide an appropriate status for these conditions so applications
> can distinuguish this error for special handling.

Hi Jens,

Just checking in -- do you have any thoughts on this, or should I just
wait a bit more?

thx,
Keith


From axboe at kernel.dk  Thu Oct  1 21:28:05 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 1 Oct 2020 19:28:05 -0600
Subject: [PATCH blk-next 0/2] Delete the get_vector_affinity leftovers
In-Reply-To: <20201001050101.GT3094@unreal>
References: <20200929091358.421086-1-leon@kernel.org>
 <20201001050101.GT3094@unreal>
Message-ID: <063a6648-a8bc-cefd-ced8-573d78a9cb8d@kernel.dk>

On 9/30/20 11:01 PM, Leon Romanovsky wrote:
> On Tue, Sep 29, 2020 at 12:13:56PM +0300, Leon Romanovsky wrote:
>> From: Leon Romanovsky <leonro at nvidia.com>
>>
>> There are no drivers that implement .get_vector_affinity(), so delete
>> the RDMA function and simplify block code.
>>
>> Thanks
>>
>> P.S. Probably it should go through block tree.
>>
>> Leon Romanovsky (2):
>>   blk-mq-rdma: Delete not-used multi-queue RDMA map queue code
>>   RDMA/core: Delete not-implemented get_vector_affinity
> 
> Jens, Keith
> 
> How can we progress here?

I'd really like for the nvme side to sign off on this first.

-- 
Jens Axboe



From chaitanya.kulkarni at wdc.com  Thu Oct  1 22:56:45 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 19:56:45 -0700
Subject: [PATCH V2 0/4] nvmet: allow target to export readonly ns
Message-ID: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>

Hi,

This patch series allows the target to export the read-only ns. The first
patch changes the type of the ns bool read-only attr to nvme_ns_wp_state
write-protect so that we can store the state.

The second patch adds the new ns write_protect configfs attr which user
can set to make ns read-only, we don't allow user to change the 
write-protect value when it is set from configfs.

The last two patches allow the?user to open the ns in the read-only mode since
by default we open the ns in read-write mode which will fail if ns
backend is set to read-only.

Regards,
Chaitanya

* Changes from V1:-

1. Add a prep patch to store the ns write protect state.
2. Don't allow the host to change the ns-write-protect state when the state
? ?is set from configfs to the NVME_NS_WRITE_PROTECT_PERMANENT.
3. Change the ns bool type readonly to nvme_ns_wp_state type
? ?write_protect.
4. Update the id-ctrl field nwpc to inidicate support for write protection
   permanent state and don't set the bit 1 of the replay protection block
   Namesapce Write Protection Control so indicate any attempts of
   set-feature cmd with permanent write protection ns value will fail.

Chaitanya Kulkarni (4):
  nvmet: change the type of the ns->readonly
  nvmet: allow user to set the ns readonly
  nvmet: allow readonly bdev-ns to be configured
  nvmet: allow readonly file-ns to be configured

 drivers/nvme/target/admin-cmd.c   | 36 ++++++++++++++++++++-----------
 drivers/nvme/target/configfs.c    | 33 ++++++++++++++++++++++++++++
 drivers/nvme/target/core.c        |  3 ++-
 drivers/nvme/target/io-cmd-bdev.c |  4 ++--
 drivers/nvme/target/io-cmd-file.c |  2 +-
 drivers/nvme/target/nvmet.h       |  2 +-
 include/linux/nvme.h              |  2 +-
 7 files changed, 64 insertions(+), 18 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct  1 22:56:46 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 19:56:46 -0700
Subject: [PATCH V2 1/4] nvmet: change the type of the ns->readonly
In-Reply-To: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
References: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201002025649.30724-2-chaitanya.kulkarni@wdc.com>

In this patch, we change the type of the ns->readonly attribute from
bool to the?nvme_ns_wp_state and rename it to write_protect so that we
can actually store the state of the ns write protection. This is needed
to store the ns write protection permanent state since in the next patch
we allow user to set the write protection from the configfs ns attr to
export the read-only snapshots such that host should not change the write
protect value when it is set to permanent from configfs.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 27 ++++++++++++++++-----------
 drivers/nvme/target/core.c      |  3 ++-
 drivers/nvme/target/nvmet.h     |  2 +-
 include/linux/nvme.h            |  2 +-
 4 files changed, 20 insertions(+), 14 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index dca34489a1dc..263cfff6ddd4 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -537,8 +537,17 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 		id->lbaf[0].ms = cpu_to_le16(ns->metadata_size);
 	}
 
-	if (ns->readonly)
+	switch (ns->write_protect) {
+	case NVME_NS_NO_WRITE_PROTECT:
+		id->nsattr |= 0;
+		break;
+	case NVME_NS_WRITE_PROTECT:
 		id->nsattr |= (1 << 0);
+		break;
+	default:
+		break;
+	}
+
 	nvmet_put_namespace(ns);
 done:
 	status = nvmet_copy_to_sgl(req, 0, id, sizeof(*id));
@@ -704,13 +713,14 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 	mutex_lock(&subsys->lock);
 	switch (write_protect) {
 	case NVME_NS_WRITE_PROTECT:
-		req->ns->readonly = true;
 		status = nvmet_write_protect_flush_sync(req);
-		if (status)
-			req->ns->readonly = false;
+		if (status == NVME_SC_SUCCESS)
+			req->ns->write_protect = NVME_NS_WRITE_PROTECT;
+		else
+			req->ns->write_protect = NVME_NS_NO_WRITE_PROTECT;
 		break;
 	case NVME_NS_NO_WRITE_PROTECT:
-		req->ns->readonly = false;
+		req->ns->write_protect = NVME_NS_NO_WRITE_PROTECT;
 		status = 0;
 		break;
 	default:
@@ -798,7 +808,6 @@ void nvmet_execute_set_features(struct nvmet_req *req)
 static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 {
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
-	u32 result;
 
 	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->common.nsid);
 	if (!req->ns)  {
@@ -806,11 +815,7 @@ static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 		return NVME_SC_INVALID_NS | NVME_SC_DNR;
 	}
 	mutex_lock(&subsys->lock);
-	if (req->ns->readonly == true)
-		result = NVME_NS_WRITE_PROTECT;
-	else
-		result = NVME_NS_NO_WRITE_PROTECT;
-	nvmet_set_result(req, result);
+	nvmet_set_result(req, req->ns->write_protect);
 	mutex_unlock(&subsys->lock);
 
 	return 0;
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 25d62d867563..036ce39cf838 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -681,6 +681,7 @@ struct nvmet_ns *nvmet_ns_alloc(struct nvmet_subsys *subsys, u32 nsid)
 
 	uuid_gen(&ns->uuid);
 	ns->buffered_io = false;
+	ns->write_protect = NVME_NS_NO_WRITE_PROTECT;
 
 	return ns;
 }
@@ -839,7 +840,7 @@ static inline u16 nvmet_check_ana_state(struct nvmet_port *port,
 
 static inline u16 nvmet_io_cmd_check_access(struct nvmet_req *req)
 {
-	if (unlikely(req->ns->readonly)) {
+	if (unlikely(req->ns->write_protect)) {
 		switch (req->cmd->common.opcode) {
 		case nvme_cmd_read:
 		case nvme_cmd_flush:
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 559a15ccc322..7b9d64c097eb 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -57,7 +57,7 @@ struct nvmet_ns {
 	struct percpu_ref	ref;
 	struct block_device	*bdev;
 	struct file		*file;
-	bool			readonly;
+	enum nvme_ns_wp_state	write_protect;
 	u32			nsid;
 	u32			blksize_shift;
 	loff_t			size;
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index d92535997687..ae54132cfab0 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -1067,7 +1067,7 @@ enum {
 };
 
 /* NVMe Namespace Write Protect State */
-enum {
+enum nvme_ns_wp_state {
 	NVME_NS_NO_WRITE_PROTECT = 0,
 	NVME_NS_WRITE_PROTECT,
 	NVME_NS_WRITE_PROTECT_POWER_CYCLE,
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct  1 22:56:47 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 19:56:47 -0700
Subject: [PATCH V2 2/4] nvmet: allow user to set the ns readonly
In-Reply-To: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
References: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201002025649.30724-3-chaitanya.kulkarni@wdc.com>

User may want to export readonly ns to host in order to export
read-only snapshots. Right now we don't have configfs attribute to
allow user to mark readonly & export ns.

Add a configfs attribute "write_protect" such that user can set the ns
and export ns readonly when it is not enabled.

Also, when user is setting the ns write protected from configfs we set
ns write protect state to NVME_NS_WRITE_PROTECT_PERMANENT and don't
allow user to change the value for the ns-write-protect feature.

We also update the id->nwpc to indicate the support for namespace
write protect permanent state. For RPMB clear the bit 1 of the field
Namesapce Write Protection Authentication Control to indicate Set
Features command which attenpts to set the namespace write protection
state to permanent write protect will fail.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c |  9 ++++++++-
 drivers/nvme/target/configfs.c  | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 263cfff6ddd4..44ff847db83e 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -457,7 +457,8 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 	id->psd[0].entry_lat = cpu_to_le32(0x10);
 	id->psd[0].exit_lat = cpu_to_le32(0x4);
 
-	id->nwpc = 1 << 0; /* write protect and no write protect */
+	/* write protect permanent, write protect and no write protect */
+	id->nwpc = 1 << 2 | 1 << 0;
 
 	status = nvmet_copy_to_sgl(req, 0, id, sizeof(*id));
 
@@ -711,6 +712,11 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 	}
 
 	mutex_lock(&subsys->lock);
+	if (req->ns->write_protect == NVME_NS_WRITE_PROTECT_PERMANENT) {
+		req->error_loc = offsetof(struct nvme_common_command, nsid);
+		goto out;
+	}
+
 	switch (write_protect) {
 	case NVME_NS_WRITE_PROTECT:
 		status = nvmet_write_protect_flush_sync(req);
@@ -729,6 +735,7 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 
 	if (!status)
 		nvmet_ns_changed(subsys, req->ns->nsid);
+out:
 	mutex_unlock(&subsys->lock);
 	return status;
 }
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 37e1d7784e17..5d8ee00b8fd0 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -595,6 +595,38 @@ static ssize_t nvmet_ns_revalidate_size_store(struct config_item *item,
 
 CONFIGFS_ATTR_WO(nvmet_ns_, revalidate_size);
 
+static ssize_t nvmet_ns_write_protect_show(struct config_item *item, char *page)
+{
+	return sprintf(page, "%d\n", to_nvmet_ns(item)->write_protect);
+}
+
+static ssize_t nvmet_ns_write_protect_store(struct config_item *item,
+		const char *page, size_t count)
+{
+	struct nvmet_ns *ns = to_nvmet_ns(item);
+	bool val;
+
+	if (strtobool(page, &val))
+		return -EINVAL;
+
+	mutex_lock(&ns->subsys->lock);
+	if (ns->enabled) {
+		pr_err("disable ns before setting write_protect value.\n");
+		mutex_unlock(&ns->subsys->lock);
+		return -EINVAL;
+	}
+
+	if (val)
+		ns->write_protect = NVME_NS_WRITE_PROTECT_PERMANENT;
+	else
+		ns->write_protect = NVME_NS_NO_WRITE_PROTECT;
+
+	mutex_unlock(&ns->subsys->lock);
+	return count;
+}
+
+CONFIGFS_ATTR(nvmet_ns_, write_protect);
+
 static struct configfs_attribute *nvmet_ns_attrs[] = {
 	&nvmet_ns_attr_device_path,
 	&nvmet_ns_attr_device_nguid,
@@ -603,6 +635,7 @@ static struct configfs_attribute *nvmet_ns_attrs[] = {
 	&nvmet_ns_attr_enable,
 	&nvmet_ns_attr_buffered_io,
 	&nvmet_ns_attr_revalidate_size,
+	&nvmet_ns_attr_write_protect,
 #ifdef CONFIG_PCI_P2PDMA
 	&nvmet_ns_attr_p2pmem,
 #endif
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct  1 22:56:48 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 19:56:48 -0700
Subject: [PATCH V2 3/4] nvmet: allow readonly bdev-ns to be configured
In-Reply-To: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
References: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201002025649.30724-4-chaitanya.kulkarni@wdc.com>

Right now we open bdev-ns in read-write mode. This doesn't allow us to
export the bdev which is marked read-only using blockdev --setro.

Allow user to export the readonly bdev by considering the
ns->write_protect configfs attribute when opening the bdev. Use
FMODE_WRITE when ns->write_protect is not set.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-bdev.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c
index 125dde3f410e..0e6b5312ad1e 100644
--- a/drivers/nvme/target/io-cmd-bdev.c
+++ b/drivers/nvme/target/io-cmd-bdev.c
@@ -65,10 +65,10 @@ static void nvmet_bdev_ns_enable_integrity(struct nvmet_ns *ns)
 
 int nvmet_bdev_ns_enable(struct nvmet_ns *ns)
 {
+	fmode_t mode = (ns->write_protect ? 0 : FMODE_WRITE) | FMODE_READ;
 	int ret;
 
-	ns->bdev = blkdev_get_by_path(ns->device_path,
-			FMODE_READ | FMODE_WRITE, NULL);
+	ns->bdev = blkdev_get_by_path(ns->device_path, mode, NULL);
 	if (IS_ERR(ns->bdev)) {
 		ret = PTR_ERR(ns->bdev);
 		if (ret != -ENOTBLK) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct  1 22:56:49 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu,  1 Oct 2020 19:56:49 -0700
Subject: [PATCH V2 4/4] nvmet: allow readonly file-ns to be configured
In-Reply-To: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
References: <20201002025649.30724-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201002025649.30724-5-chaitanya.kulkarni@wdc.com>

Right now we open file-ns in read-write mode. This doesn't allow us to
export the file which is marked read-only.

Allow user to export the readonly file by considering the
ns->write_protect configfs attribute when opening the file. Use O_RDWR
when ns->write_protect is not set.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-file.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c
index 0abbefd9925e..9270626f0758 100644
--- a/drivers/nvme/target/io-cmd-file.c
+++ b/drivers/nvme/target/io-cmd-file.c
@@ -41,7 +41,7 @@ void nvmet_file_ns_disable(struct nvmet_ns *ns)
 
 int nvmet_file_ns_enable(struct nvmet_ns *ns)
 {
-	int flags = O_RDWR | O_LARGEFILE;
+	int flags = (ns->write_protect ? O_RDONLY : O_RDWR) | O_LARGEFILE;
 	int ret;
 
 	if (!ns->buffered_io)
-- 
2.22.1



From lkp at intel.com  Fri Oct  2 01:18:03 2020
From: lkp at intel.com (kernel test robot)
Date: Fri, 02 Oct 2020 13:18:03 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 5ad9675443f6e73deeb2245bf2affb73c92ae363
Message-ID: <5f76b80b.C3K5gHZ7AN6TLyse%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: 5ad9675443f6e73deeb2245bf2affb73c92ae363  nvme: remove nvme_identify_ns_list

elapsed time: 720m

configs tested: 158
configs skipped: 3

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
sh                          landisk_defconfig
sh                             sh03_defconfig
arm64                            alldefconfig
powerpc                 mpc832x_mds_defconfig
powerpc                     rainier_defconfig
m68k                        mvme147_defconfig
arm                          gemini_defconfig
mips                malta_qemu_32r6_defconfig
mips                          malta_defconfig
riscv                             allnoconfig
arm                              alldefconfig
m68k                          hp300_defconfig
arc                      axs103_smp_defconfig
csky                             alldefconfig
sh                            shmin_defconfig
powerpc                        cell_defconfig
mips                malta_kvm_guest_defconfig
um                           x86_64_defconfig
sh                           se7724_defconfig
xtensa                    smp_lx200_defconfig
arm                     am200epdkit_defconfig
powerpc                 mpc834x_mds_defconfig
powerpc                      pcm030_defconfig
ia64                        generic_defconfig
arc                            hsdk_defconfig
arm                             ezx_defconfig
sh                             shx3_defconfig
mips                     cu1830-neo_defconfig
arm                            mmp2_defconfig
powerpc                    gamecube_defconfig
mips                          ath79_defconfig
arm                          simpad_defconfig
s390                       zfcpdump_defconfig
powerpc                       ppc64_defconfig
powerpc                      ppc6xx_defconfig
powerpc                     taishan_defconfig
arm                           spitz_defconfig
h8300                               defconfig
arc                          axs101_defconfig
riscv                    nommu_k210_defconfig
sh                ecovec24-romimage_defconfig
alpha                               defconfig
powerpc                      ppc40x_defconfig
mips                          ath25_defconfig
arm                           efm32_defconfig
arm                   milbeaut_m10v_defconfig
h8300                            alldefconfig
arm                           viper_defconfig
i386                             alldefconfig
parisc                              defconfig
powerpc                    klondike_defconfig
xtensa                              defconfig
sh                   rts7751r2dplus_defconfig
powerpc                 mpc85xx_cds_defconfig
h8300                     edosk2674_defconfig
sh                                  defconfig
arm                           omap1_defconfig
mips                   sb1250_swarm_defconfig
arm                         axm55xx_defconfig
m68k                        m5307c3_defconfig
xtensa                  audio_kc705_defconfig
microblaze                      mmu_defconfig
powerpc                          allmodconfig
nios2                               defconfig
riscv                            alldefconfig
arm                         palmz72_defconfig
powerpc                     ksi8560_defconfig
sh                        sh7785lcr_defconfig
powerpc                 mpc8313_rdb_defconfig
powerpc                    mvme5100_defconfig
s390                             alldefconfig
arm                          pxa910_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                           allnoconfig
i386                 randconfig-a003-20200930
i386                 randconfig-a002-20200930
i386                 randconfig-a006-20200930
i386                 randconfig-a005-20200930
i386                 randconfig-a004-20200930
i386                 randconfig-a001-20200930
x86_64               randconfig-a015-20200930
x86_64               randconfig-a013-20200930
x86_64               randconfig-a012-20200930
x86_64               randconfig-a016-20200930
x86_64               randconfig-a014-20200930
x86_64               randconfig-a011-20200930
x86_64               randconfig-a012-20201001
x86_64               randconfig-a015-20201001
x86_64               randconfig-a014-20201001
x86_64               randconfig-a013-20201001
x86_64               randconfig-a011-20201001
x86_64               randconfig-a016-20201001
i386                 randconfig-a011-20200930
i386                 randconfig-a015-20200930
i386                 randconfig-a012-20200930
i386                 randconfig-a014-20200930
i386                 randconfig-a016-20200930
i386                 randconfig-a013-20200930
i386                 randconfig-a014-20201001
i386                 randconfig-a015-20201001
i386                 randconfig-a013-20201001
i386                 randconfig-a016-20201001
i386                 randconfig-a011-20201001
i386                 randconfig-a012-20201001
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a001-20200930
x86_64               randconfig-a005-20200930
x86_64               randconfig-a003-20200930
x86_64               randconfig-a004-20200930
x86_64               randconfig-a002-20200930
x86_64               randconfig-a006-20200930
x86_64               randconfig-a004-20201001
x86_64               randconfig-a001-20201001
x86_64               randconfig-a002-20201001
x86_64               randconfig-a003-20201001
x86_64               randconfig-a005-20201001
x86_64               randconfig-a006-20201001

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From hch at lst.de  Fri Oct  2 02:41:10 2020
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 2 Oct 2020 08:41:10 +0200
Subject: [PATCH 17/20] nvme: query namespae identifiers before adding
 the namespace
In-Reply-To: <20201001174321.GA343057@localhost.localdomain>
References: <20200928123502.435373-1-hch@lst.de>
 <20200928123502.435373-18-hch@lst.de>
 <20200930094411.GA189371@localhost.localdomain>
 <20200930100453.GA191465@localhost.localdomain>
 <20201001171457.GA29475@lst.de>
 <20201001174321.GA343057@localhost.localdomain>
Message-ID: <20201002064110.GA9571@lst.de>

On Thu, Oct 01, 2020 at 05:43:23PM +0000, Niklas Cassel wrote:
> Is there something I'm missing, or why did you change this between V1 and V2?

rescan is not exactly a common fast path.


From hch at lst.de  Fri Oct  2 02:45:05 2020
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 2 Oct 2020 08:45:05 +0200
Subject: [PATCH blk-next 1/2] blk-mq-rdma: Delete not-used multi-queue
 RDMA map queue code
In-Reply-To: <879916e4-b572-16b9-7b92-94dba7e918a3@grimberg.me>
References: <20200929091358.421086-1-leon@kernel.org>
 <20200929091358.421086-2-leon@kernel.org> <20200929102046.GA14445@lst.de>
 <20200929103549.GE3094@unreal>
 <879916e4-b572-16b9-7b92-94dba7e918a3@grimberg.me>
Message-ID: <20201002064505.GA9593@lst.de>

On Tue, Sep 29, 2020 at 11:24:49AM -0700, Sagi Grimberg wrote:
> Yes, basically usage of managed affinity caused people to report
> regressions not being able to change irq affinity from procfs.

Well, why would they change it?  The whole point of the infrastructure
is that there is a single sane affinity setting for a given setup. Now
that setting needed some refinement from the original series (e.g. the
current series about only using housekeeping cpus if cpu isolation is
in use).  But allowing random users to modify affinity is just a receipe
for a trainwreck.

So I think we need to bring this back ASAP, as doing affinity right
out of the box is an absolute requirement for sane performance without
all the benchmarketing deep magic.


From colyli at suse.de  Fri Oct  2 04:27:27 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:27 +0800
Subject: [PATCH v10 0/7] Introduce sendpage_ok() to detect misused sendpage in
 network related drivers
Message-ID: <20201002082734.13925-1-colyli@suse.de>

As Sagi Grimberg suggested, the original fix is refind to a more common
inline routine:
    static inline bool sendpage_ok(struct page *page)
    {
        return  (!PageSlab(page) && page_count(page) >= 1);
    }
If sendpage_ok() returns true, the checking page can be handled by the
concrete zero-copy sendpage method in network layer.

The v10 series has 7 patches, fixes a WARN_ONCE() usage from v9 series,
- The 1st patch in this series introduces sendpage_ok() in header file
  include/linux/net.h.
- The 2nd patch adds WARN_ONCE() for improper zero-copy send in
  kernel_sendpage().
- The 3rd patch fixes the page checking issue in nvme-over-tcp driver.
- The 4th patch adds page_count check by using sendpage_ok() in
  do_tcp_sendpages() as Eric Dumazet suggested.
- The 5th and 6th patches just replace existing open coded checks with
  the inline sendpage_ok() routine.

Coly Li

Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Cc: Chris Leech <cleech at redhat.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Cong Wang <amwang at redhat.com>
Cc: David S. Miller <davem at davemloft.net>
Cc: Eric Dumazet <eric.dumazet at gmail.com>
Cc: Hannes Reinecke <hare at suse.de>
Cc: Ilya Dryomov <idryomov at gmail.com>
Cc: Jan Kara <jack at suse.com>
Cc: Jeff Layton <jlayton at kernel.org>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Lee Duncan <lduncan at suse.com>
Cc: Mike Christie <michaelc at cs.wisc.edu>
Cc: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Vasily Averin <vvs at virtuozzo.com>
Cc: Vlastimil Babka <vbabka at suse.com>
---
Changelog:
v10, fix WARN_ONCE() usage, and add Reivewed-by tag from Lee Duncan.
v9, fix a typo pointed out by Greg KH.
    add Acked-by tags from Martin K. Petersen and Ilya Dryomov.
v8: add WARN_ONCE() in kernel_sendpage() as Christoph suggested.
v7: remove outer brackets from the return line of sendpage_ok() as
    Eric Dumazet suggested.
v6: fix page check in do_tcp_sendpages(), as Eric Dumazet suggested.
    replace other open coded checks with sendpage_ok() in libceph,
    iscsi drivers.
v5, include linux/mm.h in include/linux/net.h
v4, change sendpage_ok() as an inline helper, and post it as
    separate patch, as Christoph Hellwig suggested.
v3, introduce a more common sendpage_ok() as Sagi Grimberg suggested.
v2, fix typo in patch subject
v1, the initial version. 

Coly Li (7):
  net: introduce helper sendpage_ok() in include/linux/net.h
  net: add WARN_ONCE in kernel_sendpage() for improper zero-copy send
  nvme-tcp: check page by sendpage_ok() before calling kernel_sendpage()
  tcp: use sendpage_ok() to detect misused .sendpage
  drbd: code cleanup by using sendpage_ok() to check page for
    kernel_sendpage()
  scsi: libiscsi: use sendpage_ok() in iscsi_tcp_segment_map()
  libceph: use sendpage_ok() in ceph_tcp_sendpage()

 drivers/block/drbd/drbd_main.c |  2 +-
 drivers/nvme/host/tcp.c        |  7 +++----
 drivers/scsi/libiscsi_tcp.c    |  2 +-
 include/linux/net.h            | 16 ++++++++++++++++
 net/ceph/messenger.c           |  2 +-
 net/ipv4/tcp.c                 |  3 ++-
 net/socket.c                   |  6 ++++--
 7 files changed, 28 insertions(+), 10 deletions(-)

-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:27:28 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:28 +0800
Subject: [PATCH v10 1/7] net: introduce helper sendpage_ok() in
 include/linux/net.h
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002082734.13925-2-colyli@suse.de>

The original problem was from nvme-over-tcp code, who mistakenly uses
kernel_sendpage() to send pages allocated by __get_free_pages() without
__GFP_COMP flag. Such pages don't have refcount (page_count is 0) on
tail pages, sending them by kernel_sendpage() may trigger a kernel panic
from a corrupted kernel heap, because these pages are incorrectly freed
in network stack as page_count 0 pages.

This patch introduces a helper sendpage_ok(), it returns true if the
checking page,
- is not slab page: PageSlab(page) is false.
- has page refcount: page_count(page) is not zero

All drivers who want to send page to remote end by kernel_sendpage()
may use this helper to check whether the page is OK. If the helper does
not return true, the driver should try other non sendpage method (e.g.
sock_no_sendpage()) to handle the page.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Hannes Reinecke <hare at suse.de>
Cc: Jan Kara <jack at suse.com>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Vlastimil Babka <vbabka at suse.com>
Cc: stable at vger.kernel.org
---
 include/linux/net.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/linux/net.h b/include/linux/net.h
index d48ff1180879..ae713c851342 100644
--- a/include/linux/net.h
+++ b/include/linux/net.h
@@ -21,6 +21,7 @@
 #include <linux/rcupdate.h>
 #include <linux/once.h>
 #include <linux/fs.h>
+#include <linux/mm.h>
 #include <linux/sockptr.h>
 
 #include <uapi/linux/net.h>
@@ -286,6 +287,21 @@ do {									\
 #define net_get_random_once_wait(buf, nbytes)			\
 	get_random_once_wait((buf), (nbytes))
 
+/*
+ * E.g. XFS meta- & log-data is in slab pages, or bcache meta
+ * data pages, or other high order pages allocated by
+ * __get_free_pages() without __GFP_COMP, which have a page_count
+ * of 0 and/or have PageSlab() set. We cannot use send_page for
+ * those, as that does get_page(); put_page(); and would cause
+ * either a VM_BUG directly, or __page_cache_release a page that
+ * would actually still be referenced by someone, leading to some
+ * obscure delayed Oops somewhere else.
+ */
+static inline bool sendpage_ok(struct page *page)
+{
+	return !PageSlab(page) && page_count(page) >= 1;
+}
+
 int kernel_sendmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec,
 		   size_t num, size_t len);
 int kernel_sendmsg_locked(struct sock *sk, struct msghdr *msg,
-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:27:29 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:29 +0800
Subject: [PATCH v10 2/7] net: add WARN_ONCE in kernel_sendpage() for improper
 zero-copy send
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002082734.13925-3-colyli@suse.de>

If a page sent into kernel_sendpage() is a slab page or it doesn't have
ref_count, this page is improper to send by the zero copy sendpage()
method. Otherwise such page might be unexpected released in network code
path and causes impredictable panic due to kernel memory management data
structure corruption.

This path adds a WARN_ON() on the sending page before sends it into the
concrete zero-copy sendpage() method, if the page is improper for the
zero-copy sendpage() method, a warning message can be observed before
the consequential unpredictable kernel panic.

This patch does not change existing kernel_sendpage() behavior for the
improper page zero-copy send, it just provides hint warning message for
following potential panic due the kernel memory heap corruption.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Cong Wang <amwang at redhat.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: David S. Miller <davem at davemloft.net>
Cc: Sridhar Samudrala <sri at us.ibm.com>
---
 net/socket.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/net/socket.c b/net/socket.c
index 0c0144604f81..58cac2da5f66 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -3638,9 +3638,11 @@ EXPORT_SYMBOL(kernel_getpeername);
 int kernel_sendpage(struct socket *sock, struct page *page, int offset,
 		    size_t size, int flags)
 {
-	if (sock->ops->sendpage)
+	if (sock->ops->sendpage) {
+		/* Warn in case the improper page to zero-copy send */
+		WARN_ONCE(!sendpage_ok(page), "improper page for zero-copy send");
 		return sock->ops->sendpage(sock, page, offset, size, flags);
-
+	}
 	return sock_no_sendpage(sock, page, offset, size, flags);
 }
 EXPORT_SYMBOL(kernel_sendpage);
-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:27:30 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:30 +0800
Subject: [PATCH v10 3/7] nvme-tcp: check page by sendpage_ok() before calling
 kernel_sendpage()
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002082734.13925-4-colyli@suse.de>

Currently nvme_tcp_try_send_data() doesn't use kernel_sendpage() to
send slab pages. But for pages allocated by __get_free_pages() without
__GFP_COMP, which also have refcount as 0, they are still sent by
kernel_sendpage() to remote end, this is problematic.

The new introduced helper sendpage_ok() checks both PageSlab tag and
page_count counter, and returns true if the checking page is OK to be
sent by kernel_sendpage().

This patch fixes the page checking issue of nvme_tcp_try_send_data()
with sendpage_ok(). If sendpage_ok() returns true, send this page by
kernel_sendpage(), otherwise use sock_no_sendpage to handle this page.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Hannes Reinecke <hare at suse.de>
Cc: Jan Kara <jack at suse.com>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Vlastimil Babka <vbabka at suse.com>
Cc: stable at vger.kernel.org
---
 drivers/nvme/host/tcp.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8f4f29f18b8c..d6a3e1487354 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -913,12 +913,11 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_request *req)
 		else
 			flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST;
 
-		/* can't zcopy slab pages */
-		if (unlikely(PageSlab(page))) {
-			ret = sock_no_sendpage(queue->sock, page, offset, len,
+		if (sendpage_ok(page)) {
+			ret = kernel_sendpage(queue->sock, page, offset, len,
 					flags);
 		} else {
-			ret = kernel_sendpage(queue->sock, page, offset, len,
+			ret = sock_no_sendpage(queue->sock, page, offset, len,
 					flags);
 		}
 		if (ret <= 0)
-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:27:31 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:31 +0800
Subject: [PATCH v10 4/7] tcp: use sendpage_ok() to detect misused .sendpage
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002082734.13925-5-colyli@suse.de>

commit a10674bf2406 ("tcp: detecting the misuse of .sendpage for Slab
objects") adds the checks for Slab pages, but the pages don't have
page_count are still missing from the check.

Network layer's sendpage method is not designed to send page_count 0
pages neither, therefore both PageSlab() and page_count() should be
both checked for the sending page. This is exactly what sendpage_ok()
does.

This patch uses sendpage_ok() in do_tcp_sendpages() to detect misused
.sendpage, to make the code more robust.

Fixes: a10674bf2406 ("tcp: detecting the misuse of .sendpage for Slab objects")
Suggested-by: Eric Dumazet <eric.dumazet at gmail.com>
Signed-off-by: Coly Li <colyli at suse.de>
Cc: Vasily Averin <vvs at virtuozzo.com>
Cc: David S. Miller <davem at davemloft.net>
Cc: stable at vger.kernel.org
---
 net/ipv4/tcp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 31f3b858db81..2135ee7c806d 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -970,7 +970,8 @@ ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset,
 	long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT);
 
 	if (IS_ENABLED(CONFIG_DEBUG_VM) &&
-	    WARN_ONCE(PageSlab(page), "page must not be a Slab one"))
+	    WARN_ONCE(!sendpage_ok(page),
+		      "page must not be a Slab one and have page_count > 0"))
 		return -EINVAL;
 
 	/* Wait for a connection to finish. One exception is TCP Fast Open
-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:27:32 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:32 +0800
Subject: [PATCH v10 5/7] drbd: code cleanup by using sendpage_ok() to check
 page for kernel_sendpage()
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002082734.13925-6-colyli@suse.de>

In _drbd_send_page() a page is checked by following code before sending
it by kernel_sendpage(),
        (page_count(page) < 1) || PageSlab(page)
If the check is true, this page won't be send by kernel_sendpage() and
handled by sock_no_sendpage().

This kind of check is exactly what macro sendpage_ok() does, which is
introduced into include/linux/net.h to solve a similar send page issue
in nvme-tcp code.

This patch uses macro sendpage_ok() to replace the open coded checks to
page type and refcount in _drbd_send_page(), as a code cleanup.

Signed-off-by: Coly Li <colyli at suse.de>
Cc: Philipp Reisner <philipp.reisner at linbit.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/block/drbd/drbd_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 04b6bde9419d..573dbf6f0c31 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -1553,7 +1553,7 @@ static int _drbd_send_page(struct drbd_peer_device *peer_device, struct page *pa
 	 * put_page(); and would cause either a VM_BUG directly, or
 	 * __page_cache_release a page that would actually still be referenced
 	 * by someone, leading to some obscure delayed Oops somewhere else. */
-	if (drbd_disable_sendpage || (page_count(page) < 1) || PageSlab(page))
+	if (drbd_disable_sendpage || !sendpage_ok(page))
 		return _drbd_no_send_page(peer_device, page, offset, size, msg_flags);
 
 	msg_flags |= MSG_NOSIGNAL;
-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:27:33 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:33 +0800
Subject: [PATCH v10 6/7] scsi: libiscsi: use sendpage_ok() in
 iscsi_tcp_segment_map()
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002082734.13925-7-colyli@suse.de>

In iscsci driver, iscsi_tcp_segment_map() uses the following code to
check whether the page should or not be handled by sendpage:
    if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))

The "page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)" part is to
make sure the page can be sent to network layer's zero copy path. This
part is exactly what sendpage_ok() does.

This patch uses  use sendpage_ok() in iscsi_tcp_segment_map() to replace
the original open coded checks.

Signed-off-by: Coly Li <colyli at suse.de>
Reviewed-by: Lee Duncan <lduncan at suse.com>
Acked-by: Martin K. Petersen <martin.petersen at oracle.com>
Cc: Vasily Averin <vvs at virtuozzo.com>
Cc: Cong Wang <amwang at redhat.com>
Cc: Mike Christie <michaelc at cs.wisc.edu>
Cc: Chris Leech <cleech at redhat.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Hannes Reinecke <hare at suse.de>
---
 drivers/scsi/libiscsi_tcp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
index 37e5d4e48c2f..83f14b2c8804 100644
--- a/drivers/scsi/libiscsi_tcp.c
+++ b/drivers/scsi/libiscsi_tcp.c
@@ -128,7 +128,7 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
 	 * coalescing neighboring slab objects into a single frag which
 	 * triggers one of hardened usercopy checks.
 	 */
-	if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
+	if (!recv && sendpage_ok(sg_page(sg)))
 		return;
 
 	if (recv) {
-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:27:34 2020
From: colyli at suse.de (Coly Li)
Date: Fri,  2 Oct 2020 16:27:34 +0800
Subject: [PATCH v10 7/7] libceph: use sendpage_ok() in ceph_tcp_sendpage()
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002082734.13925-8-colyli@suse.de>

In libceph, ceph_tcp_sendpage() does the following checks before handle
the page by network layer's zero copy sendpage method,
	if (page_count(page) >= 1 && !PageSlab(page))

This check is exactly what sendpage_ok() does. This patch replace the
open coded checks by sendpage_ok() as a code cleanup.

Signed-off-by: Coly Li <colyli at suse.de>
Acked-by: Jeff Layton <jlayton at kernel.org>
Cc: Ilya Dryomov <idryomov at gmail.com>
---
 net/ceph/messenger.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c
index bdfd66ba3843..d4d7a0e52491 100644
--- a/net/ceph/messenger.c
+++ b/net/ceph/messenger.c
@@ -575,7 +575,7 @@ static int ceph_tcp_sendpage(struct socket *sock, struct page *page,
 	 * coalescing neighboring slab objects into a single frag which
 	 * triggers one of hardened usercopy checks.
 	 */
-	if (page_count(page) >= 1 && !PageSlab(page))
+	if (sendpage_ok(page))
 		sendpage = sock->ops->sendpage;
 	else
 		sendpage = sock_no_sendpage;
-- 
2.26.2



From colyli at suse.de  Fri Oct  2 04:30:12 2020
From: colyli at suse.de (Coly Li)
Date: Fri, 2 Oct 2020 16:30:12 +0800
Subject: [PATCH v9 0/7] Introduce sendpage_ok() to detect misused sendpage
 in network related drivers
In-Reply-To: <20201001.124815.793423380665613978.davem@davemloft.net>
References: <20201001075408.25508-1-colyli@suse.de>
 <20201001.124345.2303686561459641833.davem@davemloft.net>
 <20201001.124815.793423380665613978.davem@davemloft.net>
Message-ID: <3a46f056-8314-4467-4a11-40d11ddad99e@suse.de>

On 2020/10/2 03:48, David Miller wrote:
> From: David Miller <davem at davemloft.net>
> Date: Thu, 01 Oct 2020 12:43:45 -0700 (PDT)
> 
>> Series applied and queued up for -stable, thank you.
> 
> Actually, this doesn't even build:
> 
> In file included from ./arch/x86/include/asm/bug.h:93,
>                  from ./include/linux/bug.h:5,
>                  from ./include/linux/mmdebug.h:5,
>                  from ./include/linux/mm.h:9,
>                  from net/socket.c:55:
> net/socket.c: In function ?kernel_sendpage?:
> ./include/asm-generic/bug.h:97:3: error: too few arguments to function ?__warn_printk?
>    97 |   __warn_printk(arg);     \
>       |   ^~~~~~~~~~~~~
> 
> Was this even build tested?
> 

Hi David,

Obviously my fault and no excuse for leaking this uncompleted version to
you. I just re-post a v10 version which I make sure all patches are the
latest version.

Sorry for the inconvenience and thank you in advance for taking this set.

Coly Li


From sagi at grimberg.me  Fri Oct  2 12:03:26 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 2 Oct 2020 09:03:26 -0700
Subject: [PATCH 10/20] nvme: freeze the queue over ->lba_shift updates
In-Reply-To: <e6c5bbb7-ea1a-b449-541d-2c8317ed462d@grimberg.me>
References: <20200928123502.435373-1-hch@lst.de>
 <20200928123502.435373-11-hch@lst.de>
 <39b98188-2803-2f49-e225-8d3fbe3bfb1f@grimberg.me>
 <20200929183214.GD12948@lst.de>
 <e6c5bbb7-ea1a-b449-541d-2c8317ed462d@grimberg.me>
Message-ID: <73932a4a-8618-5058-3d1d-434cf934aa18@grimberg.me>


>>>> Ensure that there can't be any I/O in flight went we change the disk
>>>> geometry in nvme_update_ns_info, most notable the LBA size by lifting
>>>> the queue free from nvme_update_disk_info into the caller
>>>
>>> The queue is frozen on the queue logical block size, why should
>>> we care about I/O while ns->lba_shift?
>>
>> Because we use it for all kinds of calculations in the I/O path.? By
>> moving all assignments into the frozen queue critical sections we
>> avoid all possibly inconsistencies.
> 
> I'd think that it would be better to never use ns->lba_shift but rather
> the request queue block size and keep the queue freeze span only
> that mutation.

?


From hare at suse.de  Fri Oct  2 12:06:34 2020
From: hare at suse.de (Hannes Reinecke)
Date: Fri,  2 Oct 2020 18:06:34 +0200
Subject: [RFC PATCH] nvme: add 'queue_if_no_path' semantics
Message-ID: <20201002160634.11821-1-hare@suse.de>

Currently namespaces behave differently depending on the 'CMIC'
setting. If CMIC is zero, the device is removed once the last path
goes away. If CMIC has the multipath bit set, the device is retained
even if the last path is removed.
This is okay for fabrics, where one can do an explicit disconnect
to remove the device, but for nvme-pci this induces a regression
with PCI hotplug.
When the NVMe device is opened (eg by MD), the NVMe device is not
removed after a PCI hot-remove. Hence MD will not be notified about
the event, and will continue to consider this device as operational.
Consequently, upon PCI hot-add the device shows up as a new NVMe
device, and MD will fail to reattach the device.
So this patch adds NVME_NSHEAD_QUEUE_IF_NO_PATH flag to the nshead
to restore the original behaviour for non-fabrics NVMe devices.

Suggested-by: Keith Busch <keith.busch at wdc.com>
Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      | 42 ++++++++++++++++++++++++++++++++++-
 drivers/nvme/host/multipath.c |  5 ++++-
 drivers/nvme/host/nvme.h      |  3 ++-
 3 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 385b10317873..87589f0adea3 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -474,7 +474,11 @@ static void nvme_free_ns_head(struct kref *ref)
 	struct nvme_ns_head *head =
 		container_of(ref, struct nvme_ns_head, ref);
 
-	nvme_mpath_remove_disk(head);
+	if (head->disk) {
+		if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags))
+			nvme_mpath_remove_disk(head);
+		put_disk(head->disk);
+	}
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
 	nvme_put_subsystem(head->subsys);
@@ -3345,12 +3349,42 @@ static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
 }
 static DEVICE_ATTR_RO(nsid);
 
+static ssize_t queue_if_no_path_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct nvme_ns_head *head = dev_to_ns_head(dev);
+
+	return sprintf(buf, "%s\n",
+		       test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags) ?
+		       "on" : "off");
+}
+
+static ssize_t queue_if_no_path_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ns_head *head = dev_to_ns_head(dev);
+	int queue_if_no_path, err;
+
+	err = kstrtoint(buf, 10, &queue_if_no_path);
+	if (err)
+		return -EINVAL;
+
+	else if (queue_if_no_path <= 0)
+		clear_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
+	else
+		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
+	return count;
+}
+static DEVICE_ATTR(queue_if_no_path, S_IRUGO | S_IWUSR,
+	queue_if_no_path_show, queue_if_no_path_store);
+
 static struct attribute *nvme_ns_id_attrs[] = {
 	&dev_attr_wwid.attr,
 	&dev_attr_uuid.attr,
 	&dev_attr_nguid.attr,
 	&dev_attr_eui.attr,
 	&dev_attr_nsid.attr,
+	&dev_attr_queue_if_no_path.attr,
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
@@ -3377,6 +3411,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!memchr_inv(ids->eui64, 0, sizeof(ids->eui64)))
 			return 0;
 	}
+	if (a == &dev_attr_queue_if_no_path.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_fops)
+			return 0;
+	}
 #ifdef CONFIG_NVME_MULTIPATH
 	if (a == &dev_attr_ana_grpid.attr || a == &dev_attr_ana_state.attr) {
 		if (dev_to_disk(dev)->fops != &nvme_fops) /* per-path attr */
@@ -3685,6 +3723,8 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
 	head->subsys = ctrl->subsys;
 	head->ns_id = nsid;
 	head->ids = *ids;
+	if (ctrl->ops->flags & NVME_F_FABRICS)
+		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
 	kref_init(&head->ref);
 
 	ret = __nvme_check_ids(ctrl->subsys, head);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 74896be40c17..05c4fbc35bb2 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -682,6 +682,10 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 {
 	if (!head->disk)
 		return;
+	if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags)) {
+		kblockd_schedule_work(&head->requeue_work);
+		return;
+	}
 	if (head->disk->flags & GENHD_FL_UP)
 		del_gendisk(head->disk);
 	blk_set_queue_dying(head->disk->queue);
@@ -697,7 +701,6 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 		 */
 		head->disk->queue = NULL;
 	}
-	put_disk(head->disk);
 }
 
 int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 566776100126..baf4a84918d8 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -411,6 +411,7 @@ struct nvme_ns_head {
 	struct mutex		lock;
 	unsigned long		flags;
 #define NVME_NSHEAD_DISK_LIVE	0
+#define NVME_NSHEAD_QUEUE_IF_NO_PATH 1
 	struct nvme_ns __rcu	*current_path[];
 #endif
 };
@@ -670,7 +671,7 @@ static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
 	struct nvme_ns_head *head = ns->head;
 
 	if (head->disk && list_empty(&head->list))
-		kblockd_schedule_work(&head->requeue_work);
+		nvme_mpath_remove_disk(head);
 }
 
 static inline void nvme_trace_bio_complete(struct request *req,
-- 
2.26.2



From sagi at grimberg.me  Fri Oct  2 16:20:35 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 2 Oct 2020 13:20:35 -0700
Subject: [PATCH blk-next 1/2] blk-mq-rdma: Delete not-used multi-queue
 RDMA map queue code
In-Reply-To: <20201002064505.GA9593@lst.de>
References: <20200929091358.421086-1-leon@kernel.org>
 <20200929091358.421086-2-leon@kernel.org> <20200929102046.GA14445@lst.de>
 <20200929103549.GE3094@unreal>
 <879916e4-b572-16b9-7b92-94dba7e918a3@grimberg.me>
 <20201002064505.GA9593@lst.de>
Message-ID: <14fab6a7-f7b5-2f9d-e01f-923b1c36816d@grimberg.me>


>> Yes, basically usage of managed affinity caused people to report
>> regressions not being able to change irq affinity from procfs.
> 
> Well, why would they change it?  The whole point of the infrastructure
> is that there is a single sane affinity setting for a given setup. Now
> that setting needed some refinement from the original series (e.g. the
> current series about only using housekeeping cpus if cpu isolation is
> in use).  But allowing random users to modify affinity is just a receipe
> for a trainwreck.

Well allowing people to mangle irq affinity settings seem to be a hard
requirement from the discussions in the past.

> So I think we need to bring this back ASAP, as doing affinity right
> out of the box is an absolute requirement for sane performance without
> all the benchmarketing deep magic.

Well, it's hard to say that setting custom irq affinity settings is
deemed non-useful to anyone and hence should be prevented. I'd expect
that irq settings have a sane default that works and if someone wants to
change it, it can but there should be no guarantees on optimal
performance. But IIRC this had some dependencies on drivers and some
more infrastructure to handle dynamic changes...


From davem at davemloft.net  Fri Oct  2 18:28:29 2020
From: davem at davemloft.net (David Miller)
Date: Fri, 02 Oct 2020 15:28:29 -0700 (PDT)
Subject: [PATCH v10 0/7] Introduce sendpage_ok() to detect misused
 sendpage in network related drivers
In-Reply-To: <20201002082734.13925-1-colyli@suse.de>
References: <20201002082734.13925-1-colyli@suse.de>
Message-ID: <20201002.152829.1002796270145913943.davem@davemloft.net>

From: Coly Li <colyli at suse.de>
Date: Fri,  2 Oct 2020 16:27:27 +0800

> As Sagi Grimberg suggested, the original fix is refind to a more common
> inline routine:
>     static inline bool sendpage_ok(struct page *page)
>     {
>         return  (!PageSlab(page) && page_count(page) >= 1);
>     }
> If sendpage_ok() returns true, the checking page can be handled by the
> concrete zero-copy sendpage method in network layer.

Series applied.

> The v10 series has 7 patches, fixes a WARN_ONCE() usage from v9 series,
 ...

I still haven't heard from you how such a fundamental build failure
was even possible.

If the v9 patch series did not even compile, how in the world did you
perform functional testing of these changes?

Please explain this to me, instead of just quietly fixing it and
posting an updated series.

Thank you.


From davem at davemloft.net  Fri Oct  2 18:29:25 2020
From: davem at davemloft.net (David Miller)
Date: Fri, 02 Oct 2020 15:29:25 -0700 (PDT)
Subject: [PATCH v9 0/7] Introduce sendpage_ok() to detect misused
 sendpage in network related drivers
In-Reply-To: <3a46f056-8314-4467-4a11-40d11ddad99e@suse.de>
References: <20201001.124345.2303686561459641833.davem@davemloft.net>
 <20201001.124815.793423380665613978.davem@davemloft.net>
 <3a46f056-8314-4467-4a11-40d11ddad99e@suse.de>
Message-ID: <20201002.152925.826224771231840847.davem@davemloft.net>

From: Coly Li <colyli at suse.de>
Date: Fri, 2 Oct 2020 16:30:12 +0800

> Obviously my fault and no excuse for leaking this uncompleted version to
> you. I just re-post a v10 version which I make sure all patches are the
> latest version.
> 
> Sorry for the inconvenience and thank you in advance for taking this set.

How did this happen?

How did you functionally test the patch set if it didn't even compile?

I want you to explain why you sent a completely untested patch set.


From colyli at suse.de  Sat Oct  3 23:38:49 2020
From: colyli at suse.de (Coly Li)
Date: Sun, 4 Oct 2020 11:38:49 +0800
Subject: [PATCH v10 0/7] Introduce sendpage_ok() to detect misused
 sendpage in network related drivers
In-Reply-To: <20201002.152829.1002796270145913943.davem@davemloft.net>
References: <20201002082734.13925-1-colyli@suse.de>
 <20201002.152829.1002796270145913943.davem@davemloft.net>
Message-ID: <e4482d6a-ee44-04e4-42d0-bb9ab6fc23c7@suse.de>

On 2020/10/3 06:28, David Miller wrote:
> From: Coly Li <colyli at suse.de>
> Date: Fri,  2 Oct 2020 16:27:27 +0800
> 
>> As Sagi Grimberg suggested, the original fix is refind to a more common
>> inline routine:
>>     static inline bool sendpage_ok(struct page *page)
>>     {
>>         return  (!PageSlab(page) && page_count(page) >= 1);
>>     }
>> If sendpage_ok() returns true, the checking page can be handled by the
>> concrete zero-copy sendpage method in network layer.
> 
> Series applied.
> 
>> The v10 series has 7 patches, fixes a WARN_ONCE() usage from v9 series,
>  ...
> 
> I still haven't heard from you how such a fundamental build failure
> was even possible.
> 

Hi David,

Here is the detail steps how I leaked this uncompleted patch to you,
1) Add WARN_ONCE() as WARN_ON() to kernel_sendpage(). Maybe I was still
hesitating when I typed WARN_ONCE() on keyboard.
2) Generate the patches, prepare to post
3) Hmm, compiling failed, oh it is WARN_ONCE(). Yeah, WARN_ONCE() might
be more informative and better.
4) Modify to use WARN_ONCE() and compile and try, looks fine.
5) Re-generate the patches to overwrite the previous ones.
6) Post the patches.

The missing part was, before I post the patches, I should do rebase and
commit the change, but (interrupted by other stuffs) it skipped in my
mind. Although I regenerated the series but the change was not included.
The result was, uncompleted patch posted and the second-half change
still stayed in my local file.


> If the v9 patch series did not even compile, how in the world did you
> perform functional testing of these changes?
> 

Only 0002-net-add-WARN_ONCE-in-kernel_sendpage-for-improper-ze.patch was
tested in v9 series, other tests were done in previous versions.

> Please explain this to me, instead of just quietly fixing it and
> posting an updated series.


And not all the patches in the series were tested. Here is the testing
coverage of the series:

The following ones were tested and verified to break nothing and avoid
the mm corruption and panic,
0001-net-introduce-helper-sendpage_ok-in-include-linux-ne.patch
0002-net-add-WARN_ONCE-in-kernel_sendpage-for-improper-ze.patch
0003-nvme-tcp-check-page-by-sendpage_ok-before-calling-ke.patch
0006-scsi-libiscsi-use-sendpage_ok-in-iscsi_tcp_segment_m.patch

The following ones were not tested, due to complicated environment setup,
0005-drbd-code-cleanup-by-using-sendpage_ok-to-check-page.patch
0007-libceph-use-sendpage_ok-in-ceph_tcp_sendpage.patch

This patch I didn't explicitly test, due to lack of knowledge to modify
network code to trigger a buggy condition. It just went with other
tested patches,
0004-tcp-use-sendpage_ok-to-detect-misused-.sendpage.patch


Back to the built failure, I don't have excuse for leaking this
uncompleted version to you. Of cause I will try to avoid to
inefficiently occupy maintainer's time by such silly mess up.

Thanks for your review and the thorough maintenance.

Coly Li


From hch at lst.de  Mon Oct  5 04:32:20 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 10:32:20 +0200
Subject: [PATCH 10/20] nvme: freeze the queue over ->lba_shift updates
In-Reply-To: <73932a4a-8618-5058-3d1d-434cf934aa18@grimberg.me>
References: <20200928123502.435373-1-hch@lst.de>
 <20200928123502.435373-11-hch@lst.de>
 <39b98188-2803-2f49-e225-8d3fbe3bfb1f@grimberg.me>
 <20200929183214.GD12948@lst.de>
 <e6c5bbb7-ea1a-b449-541d-2c8317ed462d@grimberg.me>
 <73932a4a-8618-5058-3d1d-434cf934aa18@grimberg.me>
Message-ID: <20201005083220.GA14548@lst.de>

On Fri, Oct 02, 2020 at 09:03:26AM -0700, Sagi Grimberg wrote:
>
>>>>> Ensure that there can't be any I/O in flight went we change the disk
>>>>> geometry in nvme_update_ns_info, most notable the LBA size by lifting
>>>>> the queue free from nvme_update_disk_info into the caller
>>>>
>>>> The queue is frozen on the queue logical block size, why should
>>>> we care about I/O while ns->lba_shift?
>>>
>>> Because we use it for all kinds of calculations in the I/O path.? By
>>> moving all assignments into the frozen queue critical sections we
>>> avoid all possibly inconsistencies.
>>
>> I'd think that it would be better to never use ns->lba_shift but rather
>> the request queue block size and keep the queue freeze span only
>> that mutation.
>
> ?

So that we'll need to multiply instead of shift in the I/O path?
Maybe we could add a shift in the queue, but that's a much bigger
change..


From hch at lst.de  Mon Oct  5 04:38:17 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 10:38:17 +0200
Subject: [PATCH blk-next 1/2] blk-mq-rdma: Delete not-used multi-queue
 RDMA map queue code
In-Reply-To: <14fab6a7-f7b5-2f9d-e01f-923b1c36816d@grimberg.me>
References: <20200929091358.421086-1-leon@kernel.org>
 <20200929091358.421086-2-leon@kernel.org> <20200929102046.GA14445@lst.de>
 <20200929103549.GE3094@unreal>
 <879916e4-b572-16b9-7b92-94dba7e918a3@grimberg.me>
 <20201002064505.GA9593@lst.de>
 <14fab6a7-f7b5-2f9d-e01f-923b1c36816d@grimberg.me>
Message-ID: <20201005083817.GA14908@lst.de>

On Fri, Oct 02, 2020 at 01:20:35PM -0700, Sagi Grimberg wrote:
>> Well, why would they change it?  The whole point of the infrastructure
>> is that there is a single sane affinity setting for a given setup. Now
>> that setting needed some refinement from the original series (e.g. the
>> current series about only using housekeeping cpus if cpu isolation is
>> in use).  But allowing random users to modify affinity is just a receipe
>> for a trainwreck.
>
> Well allowing people to mangle irq affinity settings seem to be a hard
> requirement from the discussions in the past.
>
>> So I think we need to bring this back ASAP, as doing affinity right
>> out of the box is an absolute requirement for sane performance without
>> all the benchmarketing deep magic.
>
> Well, it's hard to say that setting custom irq affinity settings is
> deemed non-useful to anyone and hence should be prevented. I'd expect
> that irq settings have a sane default that works and if someone wants to
> change it, it can but there should be no guarantees on optimal
> performance. But IIRC this had some dependencies on drivers and some
> more infrastructure to handle dynamic changes...

The problem is that people change random settings.  We need to generalize
it into a sane API (e.g. the housekeeping CPUs thing which totally makes
sense).


From hch at lst.de  Mon Oct  5 05:24:05 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 11:24:05 +0200
Subject: [RFC PATCH] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201002160634.11821-1-hare@suse.de>
References: <20201002160634.11821-1-hare@suse.de>
Message-ID: <20201005092405.GA18193@lst.de>

On Fri, Oct 02, 2020 at 06:06:34PM +0200, Hannes Reinecke wrote:
> Currently namespaces behave differently depending on the 'CMIC'
> setting. If CMIC is zero, the device is removed once the last path
> goes away. If CMIC has the multipath bit set, the device is retained
> even if the last path is removed.
> This is okay for fabrics, where one can do an explicit disconnect
> to remove the device, but for nvme-pci this induces a regression
> with PCI hotplug.
> When the NVMe device is opened (eg by MD), the NVMe device is not
> removed after a PCI hot-remove. Hence MD will not be notified about
> the event, and will continue to consider this device as operational.
> Consequently, upon PCI hot-add the device shows up as a new NVMe
> device, and MD will fail to reattach the device.
> So this patch adds NVME_NSHEAD_QUEUE_IF_NO_PATH flag to the nshead
> to restore the original behaviour for non-fabrics NVMe devices.
> 
> Suggested-by: Keith Busch <keith.busch at wdc.com>
> Signed-off-by: Hannes Reinecke <hare at suse.de>

This seems to include the patch from Keith to tear down the disk
earlier. I think that needs to stay a separate prep patch, otherwise
it will be very hard to review and reason about.


From hare at suse.de  Mon Oct  5 05:30:50 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 5 Oct 2020 11:30:50 +0200
Subject: [RFC PATCH] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201005092405.GA18193@lst.de>
References: <20201002160634.11821-1-hare@suse.de>
 <20201005092405.GA18193@lst.de>
Message-ID: <d6710652-0733-15fe-518f-9ec4ab10a57d@suse.de>

On 10/5/20 11:24 AM, Christoph Hellwig wrote:
> On Fri, Oct 02, 2020 at 06:06:34PM +0200, Hannes Reinecke wrote:
>> Currently namespaces behave differently depending on the 'CMIC'
>> setting. If CMIC is zero, the device is removed once the last path
>> goes away. If CMIC has the multipath bit set, the device is retained
>> even if the last path is removed.
>> This is okay for fabrics, where one can do an explicit disconnect
>> to remove the device, but for nvme-pci this induces a regression
>> with PCI hotplug.
>> When the NVMe device is opened (eg by MD), the NVMe device is not
>> removed after a PCI hot-remove. Hence MD will not be notified about
>> the event, and will continue to consider this device as operational.
>> Consequently, upon PCI hot-add the device shows up as a new NVMe
>> device, and MD will fail to reattach the device.
>> So this patch adds NVME_NSHEAD_QUEUE_IF_NO_PATH flag to the nshead
>> to restore the original behaviour for non-fabrics NVMe devices.
>>
>> Suggested-by: Keith Busch <keith.busch at wdc.com>
>> Signed-off-by: Hannes Reinecke <hare at suse.de>
> 
> This seems to include the patch from Keith to tear down the disk
> earlier. I think that needs to stay a separate prep patch, otherwise
> it will be very hard to review and reason about.
> 
Wasn't sure how to approach this best, but yes, sure, I can split it off 
into separate patches, with that one from Keith being the first one.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Mon Oct  5 05:42:53 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon,  5 Oct 2020 11:42:53 +0200
Subject: [RFC PATCHv2 0/3] nvme: queue_if_no_path functionality
Message-ID: <20201005094256.49358-1-hare@suse.de>

Hi all,

this is a patchset based on Keiths original patch for restoring pre-fabrics
behaviour when the last path to a multipath device is removed.
Originally, the nvme device had been removed once the underlying hardware
had been removed. With the introduction of multipath support things changed;
it's now the 'CMIC' bit in the controller identification which controls the
behaviour. If it's set to non-zero, the device is retained even if the hardware
is removed. While this is okay for fabrics (as we can manually connect and
disconnect the devices), for nvme-pci this means that PCI hotplug ceases
to work as the device is never removed, and when reinserting the hardware
a new nvme device is created.
This patchset introduces a 'queue_if_no_path' flag to control the handling of
the last path; it's set for fabrics to retain the current functionality,
but unset for PCI to revert to the original, pre-fabrics behaviour.

Hannes Reinecke (2):
  nvme: add 'queue_if_no_path' semantics
  nvme: add 'queue_if_no_path' sysfs attribute

Keith Busch (1):
  nvme-mpath: delete disk after last connection

 drivers/nvme/host/core.c      | 42 +++++++++++++++++++++++++++++++++++++++++-
 drivers/nvme/host/multipath.c |  5 ++++-
 drivers/nvme/host/nvme.h      |  3 ++-
 3 files changed, 47 insertions(+), 3 deletions(-)

-- 
2.16.4



From hare at suse.de  Mon Oct  5 05:42:55 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon,  5 Oct 2020 11:42:55 +0200
Subject: [PATCH 2/3] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201005094256.49358-1-hare@suse.de>
References: <20201005094256.49358-1-hare@suse.de>
Message-ID: <20201005094256.49358-3-hare@suse.de>

Instead of reverting the handling of the 'last' path for all
setups we should be restricting it to the non-fabrics case.
So add a 'queue_if_no_path' flag which will switch to the
current behaviour; disabling this flag will revert to the
original (pre-fabrics) behaviour.
And set this flag per default for fabrics.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      | 7 ++++++-
 drivers/nvme/host/multipath.c | 4 ++++
 drivers/nvme/host/nvme.h      | 1 +
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index af950d58c562..79b88b4c448f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -474,8 +474,11 @@ static void nvme_free_ns_head(struct kref *ref)
 	struct nvme_ns_head *head =
 		container_of(ref, struct nvme_ns_head, ref);
 
-	if (head->disk)
+	if (head->disk) {
+		if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags))
+			nvme_mpath_remove_disk(head);
 		put_disk(head->disk);
+	}
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
 	nvme_put_subsystem(head->subsys);
@@ -3686,6 +3689,8 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
 	head->subsys = ctrl->subsys;
 	head->ns_id = nsid;
 	head->ids = *ids;
+	if (ctrl->ops->flags & NVME_F_FABRICS)
+		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
 	kref_init(&head->ref);
 
 	ret = __nvme_check_ids(ctrl->subsys, head);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 55045291b4de..05c4fbc35bb2 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -682,6 +682,10 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 {
 	if (!head->disk)
 		return;
+	if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags)) {
+		kblockd_schedule_work(&head->requeue_work);
+		return;
+	}
 	if (head->disk->flags & GENHD_FL_UP)
 		del_gendisk(head->disk);
 	blk_set_queue_dying(head->disk->queue);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index b6180bb3361d..baf4a84918d8 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -411,6 +411,7 @@ struct nvme_ns_head {
 	struct mutex		lock;
 	unsigned long		flags;
 #define NVME_NSHEAD_DISK_LIVE	0
+#define NVME_NSHEAD_QUEUE_IF_NO_PATH 1
 	struct nvme_ns __rcu	*current_path[];
 #endif
 };
-- 
2.16.4



From hare at suse.de  Mon Oct  5 05:42:54 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon,  5 Oct 2020 11:42:54 +0200
Subject: [PATCH 1/3] nvme-mpath: delete disk after last connection
In-Reply-To: <20201005094256.49358-1-hare@suse.de>
References: <20201005094256.49358-1-hare@suse.de>
Message-ID: <20201005094256.49358-2-hare@suse.de>

From: Keith Busch <kbusch at kernel.org>

The multipath code currently deletes the disk only after all references
to it are dropped rather than when the last path to that disk is lost.
This has been reported to cause problems with some usage, like MD RAID.

Delete the disk when the last path is gone. This is the same behavior we
currently have with non-multipathed nvme devices.

The following is just a simple example that demonstrates what is currently
observed using a simple nvme loop back (loop setup file not shown):

 # nvmetcli restore loop.json
 [   31.156452] nvmet: adding nsid 1 to subsystem testnqn1
 [   31.159140] nvmet: adding nsid 1 to subsystem testnqn2

 # nvme connect -t loop -n testnqn1 -q hostnqn
 [   36.866302] nvmet: creating controller 1 for subsystem testnqn1 for NQN hostnqn.
 [   36.872926] nvme nvme3: new ctrl: "testnqn1"

 # nvme connect -t loop -n testnqn1 -q hostnqn
 [   38.227186] nvmet: creating controller 2 for subsystem testnqn1 for NQN hostnqn.
 [   38.234450] nvme nvme4: new ctrl: "testnqn1"

 # nvme connect -t loop -n testnqn2 -q hostnqn
 [   43.902761] nvmet: creating controller 3 for subsystem testnqn2 for NQN hostnqn.
 [   43.907401] nvme nvme5: new ctrl: "testnqn2"

 # nvme connect -t loop -n testnqn2 -q hostnqn
 [   44.627689] nvmet: creating controller 4 for subsystem testnqn2 for NQN hostnqn.
 [   44.641773] nvme nvme6: new ctrl: "testnqn2"

 # mdadm --create /dev/md0 --level=mirror --raid-devices=2 /dev/nvme3n1 /dev/nvme5n1
 [   53.497038] md/raid1:md0: active with 2 out of 2 mirrors
 [   53.501717] md0: detected capacity change from 0 to 66060288

 # cat /proc/mdstat
 Personalities : [raid1]
 md0 : active raid1 nvme5n1[1] nvme3n1[0]
       64512 blocks super 1.2 [2/2] [UU]

Now delete all paths to one of the namespaces:

 # echo 1 > /sys/class/nvme/nvme3/delete_controller
 # echo 1 > /sys/class/nvme/nvme4/delete_controller

We have no path, but mdstat says:

 # cat /proc/mdstat
 Personalities : [raid1]
 md0 : active (auto-read-only) raid1 nvme5n1[1]
       64512 blocks super 1.2 [2/1] [_U]

And this is reported to cause a problem.

With the proposed patch, the following messages appear:

 [  227.516807] md/raid1:md0: Disk failure on nvme3n1, disabling device.
 [  227.516807] md/raid1:md0: Operation continuing on 1 devices.

And mdstat shows only the viable members:

 # cat /proc/mdstat
 Personalities : [raid1]
 md0 : active (auto-read-only) raid1 nvme5n1[1]
       64512 blocks super 1.2 [2/1] [_U]

Reported-by: Hannes Reinecke <hare at suse.de>
Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/core.c      | 3 ++-
 drivers/nvme/host/multipath.c | 1 -
 drivers/nvme/host/nvme.h      | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 385b10317873..af950d58c562 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -474,7 +474,8 @@ static void nvme_free_ns_head(struct kref *ref)
 	struct nvme_ns_head *head =
 		container_of(ref, struct nvme_ns_head, ref);
 
-	nvme_mpath_remove_disk(head);
+	if (head->disk)
+		put_disk(head->disk);
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
 	nvme_put_subsystem(head->subsys);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 74896be40c17..55045291b4de 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -697,7 +697,6 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 		 */
 		head->disk->queue = NULL;
 	}
-	put_disk(head->disk);
 }
 
 int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 566776100126..b6180bb3361d 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -670,7 +670,7 @@ static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
 	struct nvme_ns_head *head = ns->head;
 
 	if (head->disk && list_empty(&head->list))
-		kblockd_schedule_work(&head->requeue_work);
+		nvme_mpath_remove_disk(head);
 }
 
 static inline void nvme_trace_bio_complete(struct request *req,
-- 
2.16.4



From hare at suse.de  Mon Oct  5 05:42:56 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon,  5 Oct 2020 11:42:56 +0200
Subject: [PATCH 3/3] nvme: add 'queue_if_no_path' sysfs attribute
In-Reply-To: <20201005094256.49358-1-hare@suse.de>
References: <20201005094256.49358-1-hare@suse.de>
Message-ID: <20201005094256.49358-4-hare@suse.de>

Add a sysfs attribute 'queue_if_no_path' to allow the admin to
view and modify the 'queue_if_no_path' flag.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 79b88b4c448f..87589f0adea3 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3349,12 +3349,42 @@ static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
 }
 static DEVICE_ATTR_RO(nsid);
 
+static ssize_t queue_if_no_path_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct nvme_ns_head *head = dev_to_ns_head(dev);
+
+	return sprintf(buf, "%s\n",
+		       test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags) ?
+		       "on" : "off");
+}
+
+static ssize_t queue_if_no_path_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ns_head *head = dev_to_ns_head(dev);
+	int queue_if_no_path, err;
+
+	err = kstrtoint(buf, 10, &queue_if_no_path);
+	if (err)
+		return -EINVAL;
+
+	else if (queue_if_no_path <= 0)
+		clear_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
+	else
+		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
+	return count;
+}
+static DEVICE_ATTR(queue_if_no_path, S_IRUGO | S_IWUSR,
+	queue_if_no_path_show, queue_if_no_path_store);
+
 static struct attribute *nvme_ns_id_attrs[] = {
 	&dev_attr_wwid.attr,
 	&dev_attr_uuid.attr,
 	&dev_attr_nguid.attr,
 	&dev_attr_eui.attr,
 	&dev_attr_nsid.attr,
+	&dev_attr_queue_if_no_path.attr,
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
@@ -3381,6 +3411,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!memchr_inv(ids->eui64, 0, sizeof(ids->eui64)))
 			return 0;
 	}
+	if (a == &dev_attr_queue_if_no_path.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_fops)
+			return 0;
+	}
 #ifdef CONFIG_NVME_MULTIPATH
 	if (a == &dev_attr_ana_grpid.attr || a == &dev_attr_ana_state.attr) {
 		if (dev_to_disk(dev)->fops != &nvme_fops) /* per-path attr */
-- 
2.16.4



From hch at lst.de  Mon Oct  5 07:27:53 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 13:27:53 +0200
Subject: [PATCH 1/3] nvme-mpath: delete disk after last connection
In-Reply-To: <20201005094256.49358-2-hare@suse.de>
References: <20201005094256.49358-1-hare@suse.de>
 <20201005094256.49358-2-hare@suse.de>
Message-ID: <20201005112753.GA26872@lst.de>

On Mon, Oct 05, 2020 at 11:42:54AM +0200, Hannes Reinecke wrote:
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 385b10317873..af950d58c562 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -474,7 +474,8 @@ static void nvme_free_ns_head(struct kref *ref)
>  	struct nvme_ns_head *head =
>  		container_of(ref, struct nvme_ns_head, ref);
>  
> -	nvme_mpath_remove_disk(head);
> +	if (head->disk)
> +		put_disk(head->disk);

This wasn't compile for the !CONFIG_NVME_MULTIPATH case.


From hch at lst.de  Mon Oct  5 07:29:16 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 13:29:16 +0200
Subject: [PATCH 2/3] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201005094256.49358-3-hare@suse.de>
References: <20201005094256.49358-1-hare@suse.de>
 <20201005094256.49358-3-hare@suse.de>
Message-ID: <20201005112916.GB26872@lst.de>

On Mon, Oct 05, 2020 at 11:42:55AM +0200, Hannes Reinecke wrote:
> Instead of reverting the handling of the 'last' path for all
> setups we should be restricting it to the non-fabrics case.
> So add a 'queue_if_no_path' flag which will switch to the
> current behaviour; disabling this flag will revert to the
> original (pre-fabrics) behaviour.
> And set this flag per default for fabrics.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c      | 7 ++++++-
>  drivers/nvme/host/multipath.c | 4 ++++
>  drivers/nvme/host/nvme.h      | 1 +
>  3 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index af950d58c562..79b88b4c448f 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -474,8 +474,11 @@ static void nvme_free_ns_head(struct kref *ref)
>  	struct nvme_ns_head *head =
>  		container_of(ref, struct nvme_ns_head, ref);
>  
> -	if (head->disk)
> +	if (head->disk) {
> +		if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags))
> +			nvme_mpath_remove_disk(head);
>  		put_disk(head->disk);
> +	}

This needs to be in a helper in multipath.c so that it an compile fine
for the non-multipath case.

> @@ -3686,6 +3689,8 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
>  	head->subsys = ctrl->subsys;
>  	head->ns_id = nsid;
>  	head->ids = *ids;
> +	if (ctrl->ops->flags & NVME_F_FABRICS)
> +		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);

This needs a comment at very least.  In fact I'm not sure we should
mess with defaults here. 

How will this bit get set for the non-fabrics case?


From hch at lst.de  Mon Oct  5 07:31:21 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 13:31:21 +0200
Subject: [PATCH 3/3] nvme: add 'queue_if_no_path' sysfs attribute
In-Reply-To: <20201005094256.49358-4-hare@suse.de>
References: <20201005094256.49358-1-hare@suse.de>
 <20201005094256.49358-4-hare@suse.de>
Message-ID: <20201005113121.GC26872@lst.de>

On Mon, Oct 05, 2020 at 11:42:56AM +0200, Hannes Reinecke wrote:
> Add a sysfs attribute 'queue_if_no_path' to allow the admin to
> view and modify the 'queue_if_no_path' flag.

ok, here it gets added.  I think this needs to be folded into the
previous patch.

> +{
> +	struct nvme_ns_head *head = dev_to_ns_head(dev);
> +	int queue_if_no_path, err;
> +
> +	err = kstrtoint(buf, 10, &queue_if_no_path);
> +	if (err)
> +		return -EINVAL;
> +
> +	else if (queue_if_no_path <= 0)

I think this needs to use kstrtobool..

>  #ifdef CONFIG_NVME_MULTIPATH
>  	&dev_attr_ana_grpid.attr,
>  	&dev_attr_ana_state.attr,
> @@ -3381,6 +3411,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
>  		if (!memchr_inv(ids->eui64, 0, sizeof(ids->eui64)))
>  			return 0;
>  	}
> +	if (a == &dev_attr_queue_if_no_path.attr) {
> +		if (dev_to_disk(dev)->fops == &nvme_fops)
> +			return 0;
> +	}
>  #ifdef CONFIG_NVME_MULTIPATH

The attribute only makes sense for the multipathing code, so it should
be under CONFIG_NVME_MULTIPATH.


From hch at lst.de  Mon Oct  5 07:38:18 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 13:38:18 +0200
Subject: [PATCH 3/3] nvme: add 'queue_if_no_path' sysfs attribute
In-Reply-To: <20201005094256.49358-4-hare@suse.de>
References: <20201005094256.49358-1-hare@suse.de>
 <20201005094256.49358-4-hare@suse.de>
Message-ID: <20201005113818.GA27690@lst.de>

Oh, and shouldn't the attribute be per-subsystem, similar to the
iopolicy one?


From hare at suse.de  Mon Oct  5 07:56:11 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 5 Oct 2020 13:56:11 +0200
Subject: [PATCH 3/3] nvme: add 'queue_if_no_path' sysfs attribute
In-Reply-To: <20201005113818.GA27690@lst.de>
References: <20201005094256.49358-1-hare@suse.de>
 <20201005094256.49358-4-hare@suse.de> <20201005113818.GA27690@lst.de>
Message-ID: <68f08802-e9f4-09cb-f293-bd7abfae7218@suse.de>

On 10/5/20 1:38 PM, Christoph Hellwig wrote:
> Oh, and shouldn't the attribute be per-subsystem, similar to the
> iopolicy one?
> 
Well, I've thought about that, too; but then figured that it's pretty 
much an admin choice.
He _might_ want to choose either behaviour, depending on the use-case 
(read: MD raid or cluster scenarios). But these use-cases are pretty 
much per namespace, so I'm not sure if per-subsystem is a good fit here.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Mon Oct  5 08:44:59 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon,  5 Oct 2020 14:44:59 +0200
Subject: [PATCH 1/2] nvme-mpath: delete disk after last connection
In-Reply-To: <20201005124500.6015-1-hare@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
Message-ID: <20201005124500.6015-2-hare@suse.de>

From: Keith Busch <kbusch at kernel.org>

The multipath code currently deletes the disk only after all references
to it are dropped rather than when the last path to that disk is lost.
This has been reported to cause problems with some usage, like MD RAID.

Delete the disk when the last path is gone. This is the same behavior we
currently have with non-multipathed nvme devices.

The following is just a simple example that demonstrates what is currently
observed using a simple nvme loop back (loop setup file not shown):

 # nvmetcli restore loop.json
 [   31.156452] nvmet: adding nsid 1 to subsystem testnqn1
 [   31.159140] nvmet: adding nsid 1 to subsystem testnqn2

 # nvme connect -t loop -n testnqn1 -q hostnqn
 [   36.866302] nvmet: creating controller 1 for subsystem testnqn1 for NQN hostnqn.
 [   36.872926] nvme nvme3: new ctrl: "testnqn1"

 # nvme connect -t loop -n testnqn1 -q hostnqn
 [   38.227186] nvmet: creating controller 2 for subsystem testnqn1 for NQN hostnqn.
 [   38.234450] nvme nvme4: new ctrl: "testnqn1"

 # nvme connect -t loop -n testnqn2 -q hostnqn
 [   43.902761] nvmet: creating controller 3 for subsystem testnqn2 for NQN hostnqn.
 [   43.907401] nvme nvme5: new ctrl: "testnqn2"

 # nvme connect -t loop -n testnqn2 -q hostnqn
 [   44.627689] nvmet: creating controller 4 for subsystem testnqn2 for NQN hostnqn.
 [   44.641773] nvme nvme6: new ctrl: "testnqn2"

 # mdadm --create /dev/md0 --level=mirror --raid-devices=2 /dev/nvme3n1 /dev/nvme5n1
 [   53.497038] md/raid1:md0: active with 2 out of 2 mirrors
 [   53.501717] md0: detected capacity change from 0 to 66060288

 # cat /proc/mdstat
 Personalities : [raid1]
 md0 : active raid1 nvme5n1[1] nvme3n1[0]
       64512 blocks super 1.2 [2/2] [UU]

Now delete all paths to one of the namespaces:

 # echo 1 > /sys/class/nvme/nvme3/delete_controller
 # echo 1 > /sys/class/nvme/nvme4/delete_controller

We have no path, but mdstat says:

 # cat /proc/mdstat
 Personalities : [raid1]
 md0 : active (auto-read-only) raid1 nvme5n1[1]
       64512 blocks super 1.2 [2/1] [_U]

And this is reported to cause a problem.

With the proposed patch, the following messages appear:

 [  227.516807] md/raid1:md0: Disk failure on nvme3n1, disabling device.
 [  227.516807] md/raid1:md0: Operation continuing on 1 devices.

And mdstat shows only the viable members:

 # cat /proc/mdstat
 Personalities : [raid1]
 md0 : active (auto-read-only) raid1 nvme5n1[1]
       64512 blocks super 1.2 [2/1] [_U]

Reported-by: Hannes Reinecke <hare at suse.de>
Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/core.c      | 5 ++++-
 drivers/nvme/host/multipath.c | 1 -
 drivers/nvme/host/nvme.h      | 2 +-
 3 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 385b10317873..4459a40b057c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -474,7 +474,10 @@ static void nvme_free_ns_head(struct kref *ref)
 	struct nvme_ns_head *head =
 		container_of(ref, struct nvme_ns_head, ref);
 
-	nvme_mpath_remove_disk(head);
+#ifdef CONFIG_NVME_MULTIPATH
+	if (head->disk)
+		put_disk(head->disk);
+#endif
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
 	nvme_put_subsystem(head->subsys);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 74896be40c17..55045291b4de 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -697,7 +697,6 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 		 */
 		head->disk->queue = NULL;
 	}
-	put_disk(head->disk);
 }
 
 int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 566776100126..b6180bb3361d 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -670,7 +670,7 @@ static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
 	struct nvme_ns_head *head = ns->head;
 
 	if (head->disk && list_empty(&head->list))
-		kblockd_schedule_work(&head->requeue_work);
+		nvme_mpath_remove_disk(head);
 }
 
 static inline void nvme_trace_bio_complete(struct request *req,
-- 
2.16.4



From hare at suse.de  Mon Oct  5 08:45:00 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon,  5 Oct 2020 14:45:00 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201005124500.6015-1-hare@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
Message-ID: <20201005124500.6015-3-hare@suse.de>

Currently namespaces behave differently depending on the 'CMIC'
setting. If CMIC is zero, the device is removed once the last path
goes away. If CMIC has the multipath bit set, the device is retained
even if the last path is removed.
This is okay for fabrics, where one can do an explicit disconnect
to remove the device, but for nvme-pci this induces a regression
with PCI hotplug.
When the NVMe device is opened (eg by MD), the NVMe device is not
removed after a PCI hot-remove. Hence MD will not be notified about
the event, and will continue to consider this device as operational.
Consequently, upon PCI hot-add the device shows up as a new NVMe
device, and MD will fail to reattach the device.
So this patch adds NVME_NSHEAD_QUEUE_IF_NO_PATH flag to the nshead
to restore the original behaviour for non-fabrics NVMe devices.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      | 10 +++++++++-
 drivers/nvme/host/multipath.c | 38 ++++++++++++++++++++++++++++++++++++++
 drivers/nvme/host/nvme.h      |  2 ++
 3 files changed, 49 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4459a40b057c..e21c32ea4b51 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -475,8 +475,11 @@ static void nvme_free_ns_head(struct kref *ref)
 		container_of(ref, struct nvme_ns_head, ref);
 
 #ifdef CONFIG_NVME_MULTIPATH
-	if (head->disk)
+	if (head->disk) {
+		if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags))
+			nvme_mpath_remove_disk(head);
 		put_disk(head->disk);
+	}
 #endif
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
@@ -3357,6 +3360,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
+	&dev_attr_queue_if_no_path.attr,
 #endif
 	NULL,
 };
@@ -3387,6 +3391,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
 			return 0;
 	}
+	if (a == &dev_attr_queue_if_no_path.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_fops)
+			return 0;
+	}
 #endif
 	return a->mode;
 }
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 55045291b4de..bbdad5917112 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -381,6 +381,9 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head)
 	/* set to a default value for 512 until disk is validated */
 	blk_queue_logical_block_size(q, 512);
 	blk_set_stacking_limits(&q->limits);
+	/* Enable queue_if_no_path semantics for fabrics */
+	if (ctrl->ops->flags & NVME_F_FABRICS)
+		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
 
 	/* we need to propagate up the VMC settings */
 	if (ctrl->vwc & NVME_CTRL_VWC_PRESENT)
@@ -640,6 +643,37 @@ static ssize_t ana_state_show(struct device *dev, struct device_attribute *attr,
 }
 DEVICE_ATTR_RO(ana_state);
 
+static ssize_t queue_if_no_path_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct nvme_ns_head *head = nvme_get_ns_from_dev(dev)->head;
+
+	return sprintf(buf, "%s\n",
+		       test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags) ?
+		       "on" : "off");
+}
+
+static ssize_t queue_if_no_path_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ns_head *head = nvme_get_ns_from_dev(dev)->head;
+	int err;
+	bool queue_if_no_path;
+
+	err = kstrtobool(buf, &queue_if_no_path);
+	if (err)
+		return -EINVAL;
+
+	if (queue_if_no_path)
+		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
+	else
+		clear_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
+
+	return count;
+}
+DEVICE_ATTR(queue_if_no_path, S_IRUGO | S_IWUSR,
+	queue_if_no_path_show, queue_if_no_path_store);
+
 static int nvme_lookup_ana_group_desc(struct nvme_ctrl *ctrl,
 		struct nvme_ana_group_desc *desc, void *data)
 {
@@ -682,6 +716,10 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 {
 	if (!head->disk)
 		return;
+	if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags)) {
+		kblockd_schedule_work(&head->requeue_work);
+		return;
+	}
 	if (head->disk->flags & GENHD_FL_UP)
 		del_gendisk(head->disk);
 	blk_set_queue_dying(head->disk->queue);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index b6180bb3361d..94ae06cdc934 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -411,6 +411,7 @@ struct nvme_ns_head {
 	struct mutex		lock;
 	unsigned long		flags;
 #define NVME_NSHEAD_DISK_LIVE	0
+#define NVME_NSHEAD_QUEUE_IF_NO_PATH 1
 	struct nvme_ns __rcu	*current_path[];
 #endif
 };
@@ -684,6 +685,7 @@ static inline void nvme_trace_bio_complete(struct request *req,
 
 extern struct device_attribute dev_attr_ana_grpid;
 extern struct device_attribute dev_attr_ana_state;
+extern struct device_attribute dev_attr_queue_if_no_path;
 extern struct device_attribute subsys_attr_iopolicy;
 
 #else
-- 
2.16.4



From hare at suse.de  Mon Oct  5 08:44:58 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon,  5 Oct 2020 14:44:58 +0200
Subject: [RFC PATCHv3 0/2] nvme: queue_if_no_path functionality
Message-ID: <20201005124500.6015-1-hare@suse.de>

Hi all,

this is a patchset based on Keiths original patch for restoring pre-fabrics
behaviour when the last path to a multipath device is removed.
Originally, the nvme device had been removed once the underlying hardware
had been removed. With the introduction of multipath support things changed;
it's now the 'CMIC' bit in the controller identification which controls the
behaviour. If it's set to non-zero, the device is retained even if the hardware
is removed. While this is okay for fabrics (as we can manually connect and
disconnect the devices), for nvme-pci this means that PCI hotplug ceases
to work as the device is never removed, and when reinserting the hardware
a new nvme device is created.
This patchset introduces a 'queue_if_no_path' flag to control the handling of
the last path; it's set for fabrics to retain the current functionality,
but unset for PCI to revert to the original, pre-fabrics behaviour.

Changes to v2:
- Merge two patches
- Fix to make it compile without CONFIG_NVME_MULTIPATH
- Move sysfs attributes to be compiled in only for CONFIG_NVME_MULTIPATH

Hannes Reinecke (1):
  nvme: add 'queue_if_no_path' semantics

Keith Busch (1):
  nvme-mpath: delete disk after last connection

 drivers/nvme/host/core.c      | 13 ++++++++++++-
 drivers/nvme/host/multipath.c | 39 ++++++++++++++++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h      |  4 +++-
 3 files changed, 53 insertions(+), 3 deletions(-)

-- 
2.16.4



From hch at lst.de  Mon Oct  5 08:46:05 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 14:46:05 +0200
Subject: [PATCH V2 1/2] nvme-loop: use xarray for loop ctrl tracking
In-Reply-To: <20200930045557.53669-2-chaitanya.kulkarni@wdc.com>
References: <20200930045557.53669-1-chaitanya.kulkarni@wdc.com>
 <20200930045557.53669-2-chaitanya.kulkarni@wdc.com>
Message-ID: <20201005124605.GA518@lst.de>

On Tue, Sep 29, 2020 at 09:55:56PM -0700, Chaitanya Kulkarni wrote:
> @@ -262,12 +260,10 @@ static void nvme_loop_free_ctrl(struct nvme_ctrl *nctrl)
>  {
>  	struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
>  
> -	if (list_empty(&ctrl->list))
> +	if (!xa_load(&nvme_loop_ctrls, nctrl->cntlid))
>  		goto free_ctrl;
>  
> -	mutex_lock(&nvme_loop_ctrl_mutex);
> -	list_del(&ctrl->list);
> -	mutex_unlock(&nvme_loop_ctrl_mutex);
> +	xa_erase(&nvme_loop_ctrls, nctrl->cntlid);

xa_erase is fine with an already deleted object, so the above xa_load
check doesn't make any sense.

> @@ -599,6 +590,12 @@ static struct nvme_ctrl *nvme_loop_create_ctrl(struct device *dev,
>  	if (ret)
>  		goto out_free_queues;
>  
> +	/* unusual place to update xarray, makes unwind code simple */
> +	ret = xa_insert(&nvme_loop_ctrls, ctrl->ctrl.cntlid, &ctrl,
> +			GFP_KERNEL);
> +	if (ret)
> +		goto out_remove_ctrl;
> +

>  out_remove_admin_queue:
>  	nvme_loop_destroy_admin_queue(ctrl);
> +out_remove_ctrl:
> +	xa_erase(&nvme_loop_ctrls, ctrl->ctrl.cntlid);

This looks weird.  Why would you remove the controller when inserting
it fails?

Also did you do an audit that none of the lookups will do something
funny with the insered but not fully initialized controller?

> @@ -678,7 +673,7 @@ static struct nvmf_transport_ops nvme_loop_transport = {
>  	.name		= "loop",
>  	.module		= THIS_MODULE,
>  	.create_ctrl	= nvme_loop_create_ctrl,
> -	.allowed_opts	= NVMF_OPT_TRADDR,
> +	.allowed_opts	= NVMF_OPT_TRADDR | NVMF_OPT_CTRL_LOSS_TMO,

This looks unrelated?

> +	xa_init(&nvme_loop_ctrls);

DEFINE_XARRAY seems like the btter choice for nvme_loop_ctrls.


>  	return ret;
>  }
>  
>  static void __exit nvme_loop_cleanup_module(void)
>  {
> -	struct nvme_loop_ctrl *ctrl, *next;
> +	struct nvme_loop_ctrl *ctrl;
> +	unsigned long idx;
>  
>  	nvmf_unregister_transport(&nvme_loop_transport);
>  	nvmet_unregister_transport(&nvme_loop_ops);
>  
> -	mutex_lock(&nvme_loop_ctrl_mutex);
> -	list_for_each_entry_safe(ctrl, next, &nvme_loop_ctrl_list, list)
> +	xa_for_each(&nvme_loop_ctrls, idx, ctrl)
>  		nvme_delete_ctrl(&ctrl->ctrl);
> -	mutex_unlock(&nvme_loop_ctrl_mutex);
> +
> +	xa_destroy(&nvme_loop_ctrls);

What replace the protection previously provided by nvme_loop_ctrl_mutex?


From hch at lst.de  Mon Oct  5 08:48:35 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 14:48:35 +0200
Subject: [PATCH V2 2/2] nvme-loop: use xarray for loop port tracking
In-Reply-To: <20200930045557.53669-3-chaitanya.kulkarni@wdc.com>
References: <20200930045557.53669-1-chaitanya.kulkarni@wdc.com>
 <20200930045557.53669-3-chaitanya.kulkarni@wdc.com>
Message-ID: <20201005124835.GB518@lst.de>

> +	unsigned long idx;
>  
> -	mutex_lock(&nvme_loop_ports_mutex);
> -	list_for_each_entry(p, &nvme_loop_ports, entry) {
> +	rcu_read_lock();
> +	xa_for_each(&nvme_loop_ports, idx, p) {

Why is the rcu_read_lock()?  We'd need a synchronize_rcu on the port
disable side.  But the manual mutex_locking seems more sensible here
anyway.

> +	xa_init(&nvme_loop_ports);

I don't think this is needed.


From hch at lst.de  Mon Oct  5 08:48:51 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 14:48:51 +0200
Subject: [PATCH V2 0/2] nvme-core: increamental cleanup
In-Reply-To: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>
References: <20201001185432.5464-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201005124851.GC518@lst.de>

Thanks,

applied to nvme-5.10.


From hch at lst.de  Mon Oct  5 08:50:54 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 14:50:54 +0200
Subject: [PATCH 1/2] nvme-mpath: delete disk after last connection
In-Reply-To: <20201005124500.6015-2-hare@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-2-hare@suse.de>
Message-ID: <20201005125054.GA1125@lst.de>

> -	nvme_mpath_remove_disk(head);
> +#ifdef CONFIG_NVME_MULTIPATH
> +	if (head->disk)
> +		put_disk(head->disk);
> +#endif

Please move this into a helper in multipath.c with a stub for the
non-multipath case.


From hch at lst.de  Mon Oct  5 08:52:01 2020
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 5 Oct 2020 14:52:01 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201005124500.6015-3-hare@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de>
Message-ID: <20201005125201.GB1125@lst.de>

On Mon, Oct 05, 2020 at 02:45:00PM +0200, Hannes Reinecke wrote:
> Currently namespaces behave differently depending on the 'CMIC'
> setting. If CMIC is zero, the device is removed once the last path
> goes away. If CMIC has the multipath bit set, the device is retained
> even if the last path is removed.
> This is okay for fabrics, where one can do an explicit disconnect
> to remove the device, but for nvme-pci this induces a regression
> with PCI hotplug.
> When the NVMe device is opened (eg by MD), the NVMe device is not
> removed after a PCI hot-remove. Hence MD will not be notified about
> the event, and will continue to consider this device as operational.
> Consequently, upon PCI hot-add the device shows up as a new NVMe
> device, and MD will fail to reattach the device.
> So this patch adds NVME_NSHEAD_QUEUE_IF_NO_PATH flag to the nshead
> to restore the original behaviour for non-fabrics NVMe devices.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c      | 10 +++++++++-
>  drivers/nvme/host/multipath.c | 38 ++++++++++++++++++++++++++++++++++++++
>  drivers/nvme/host/nvme.h      |  2 ++
>  3 files changed, 49 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 4459a40b057c..e21c32ea4b51 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -475,8 +475,11 @@ static void nvme_free_ns_head(struct kref *ref)
>  		container_of(ref, struct nvme_ns_head, ref);
>  
>  #ifdef CONFIG_NVME_MULTIPATH
> -	if (head->disk)
> +	if (head->disk) {
> +		if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags))
> +			nvme_mpath_remove_disk(head);
>  		put_disk(head->disk);
> +	}
>  #endif
>  	ida_simple_remove(&head->subsys->ns_ida, head->instance);
>  	cleanup_srcu_struct(&head->srcu);
> @@ -3357,6 +3360,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
>  #ifdef CONFIG_NVME_MULTIPATH
>  	&dev_attr_ana_grpid.attr,
>  	&dev_attr_ana_state.attr,
> +	&dev_attr_queue_if_no_path.attr,
>  #endif
>  	NULL,
>  };
> @@ -3387,6 +3391,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
>  		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
>  			return 0;
>  	}
> +	if (a == &dev_attr_queue_if_no_path.attr) {
> +		if (dev_to_disk(dev)->fops == &nvme_fops)
> +			return 0;
> +	}
>  #endif
>  	return a->mode;
>  }
> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> index 55045291b4de..bbdad5917112 100644
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -381,6 +381,9 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head)
>  	/* set to a default value for 512 until disk is validated */
>  	blk_queue_logical_block_size(q, 512);
>  	blk_set_stacking_limits(&q->limits);
> +	/* Enable queue_if_no_path semantics for fabrics */
> +	if (ctrl->ops->flags & NVME_F_FABRICS)
> +		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);

Well, that is blindly obvious from the code.  But why would we treat
fabrics special? 


From james.smart at broadcom.com  Mon Oct  5 13:14:29 2020
From: james.smart at broadcom.com (James Smart)
Date: Mon, 5 Oct 2020 10:14:29 -0700
Subject: [PATCH 1/7] nvme-fcloop: flush workqueue before calling
 nvme_fc_unregister_remoteport()
In-Reply-To: <20200922121501.32851-2-hare@suse.de>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-2-hare@suse.de>
Message-ID: <7d114c5a-59a8-a92e-5003-ce4d9fd14fca@broadcom.com>



On 9/22/2020 5:14 AM, Hannes Reinecke wrote:
> nvme_fc_unregister_remoteport() will be sending LS requests, which then
> would end up on a workqueue for processing. This will deadlock with
> fcloop_remoteport_delete() which would try to flush the very same queue.
>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/target/fcloop.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
> index c97e60b71bbc..082aa4dee406 100644
> --- a/drivers/nvme/target/fcloop.c
> +++ b/drivers/nvme/target/fcloop.c
> @@ -979,7 +979,6 @@ fcloop_remoteport_delete(struct nvme_fc_remote_port *remoteport)
>   {
>   	struct fcloop_rport *rport = remoteport->private;
>   
> -	flush_work(&rport->ls_work);
>   	fcloop_nport_put(rport->nport);
>   }
>   
> @@ -1313,6 +1312,7 @@ __remoteport_unreg(struct fcloop_nport *nport, struct fcloop_rport *rport)
>   	if (!rport)
>   		return -EALREADY;
>   
> +	flush_work(&rport->ls_work);
>   	return nvme_fc_unregister_remoteport(rport->remoteport);
>   }
>   

Reviewed-by: James Smart <james.smart at broadcom.com>

Looks Good.

-- james

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201005/30313341/attachment.p7s>

From james.smart at broadcom.com  Mon Oct  5 13:18:09 2020
From: james.smart at broadcom.com (James Smart)
Date: Mon, 5 Oct 2020 10:18:09 -0700
Subject: [PATCH 2/7] nvmet-fc: use per-target workqueue when removing
 associations
In-Reply-To: <20200922121501.32851-3-hare@suse.de>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-3-hare@suse.de>
Message-ID: <41ce327c-6b29-8835-9541-6ff41a0fb9c8@broadcom.com>

On 9/22/2020 5:14 AM, Hannes Reinecke wrote:
> When removing target ports all outstanding associations need to be
> terminated / cleaned up. As this involves several exchanges on the
> wire a synchronization point is required to establish when these
> exchanges have ran their course and it's safe to delete the association.
> So add a per-target workqueue and flush this workqueue to ensure
> the association can really be deleted.
>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/target/fc.c | 29 +++++++++++++++++++++++------
>   1 file changed, 23 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/nvme/target/fc.c b/drivers/nvme/target/fc.c
> index 04ec0076ae59..63f5deb3b68a 100644
> --- a/drivers/nvme/target/fc.c
> +++ b/drivers/nvme/target/fc.c
> @@ -99,6 +99,7 @@ struct nvmet_fc_tgtport {
>   	struct list_head		tgt_list; /* nvmet_fc_target_list */
>   	struct device			*dev;	/* dev for dma mapping */
>   	struct nvmet_fc_target_template	*ops;
> +	struct workqueue_struct		*work_q;
>   
>   	struct nvmet_fc_ls_iod		*iod;
>   	spinlock_t			lock;
> @@ -1403,10 +1404,17 @@ nvmet_fc_register_targetport(struct nvmet_fc_port_info *pinfo,
>   	ida_init(&newrec->assoc_cnt);
>   	newrec->max_sg_cnt = template->max_sgl_segments;
>   
> +	newrec->work_q = alloc_workqueue("ntfc%d", 0, 0,
> +					 newrec->fc_target_port.port_num);
> +	if (!newrec->work_q) {
> +		ret = -ENOMEM;
> +		goto out_free_newrec;
> +	}
> +
>   	ret = nvmet_fc_alloc_ls_iodlist(newrec);
>   	if (ret) {
>   		ret = -ENOMEM;
> -		goto out_free_newrec;
> +		goto out_free_workq;
>   	}
>   
>   	nvmet_fc_portentry_rebind_tgt(newrec);
> @@ -1418,6 +1426,8 @@ nvmet_fc_register_targetport(struct nvmet_fc_port_info *pinfo,
>   	*portptr = &newrec->fc_target_port;
>   	return 0;
>   
> +out_free_workq:
> +	destroy_workqueue(newrec->work_q);
>   out_free_newrec:
>   	put_device(dev);
>   out_ida_put:
> @@ -1443,6 +1453,8 @@ nvmet_fc_free_tgtport(struct kref *ref)
>   	list_del(&tgtport->tgt_list);
>   	spin_unlock_irqrestore(&nvmet_fc_tgtlock, flags);
>   
> +	destroy_workqueue(tgtport->work_q);
> +
>   	nvmet_fc_free_ls_iodlist(tgtport);
>   
>   	/* let the LLDD know we've finished tearing it down */
> @@ -1481,11 +1493,13 @@ __nvmet_fc_free_assocs(struct nvmet_fc_tgtport *tgtport)
>   				&tgtport->assoc_list, a_list) {
>   		if (!nvmet_fc_tgt_a_get(assoc))
>   			continue;
> -		if (!schedule_work(&assoc->del_work))
> +		if (!queue_work(tgtport->work_q, &assoc->del_work))
>   			/* already deleting - release local reference */
>   			nvmet_fc_tgt_a_put(assoc);
>   	}
>   	spin_unlock_irqrestore(&tgtport->lock, flags);
> +
> +	flush_workqueue(tgtport->work_q);
>   }
>   
>   /**
> @@ -1536,12 +1550,14 @@ nvmet_fc_invalidate_host(struct nvmet_fc_target_port *target_port,
>   			continue;
>   		assoc->hostport->invalid = 1;
>   		noassoc = false;
> -		if (!schedule_work(&assoc->del_work))
> +		if (!queue_work(tgtport->work_q, &assoc->del_work))
>   			/* already deleting - release local reference */
>   			nvmet_fc_tgt_a_put(assoc);
>   	}
>   	spin_unlock_irqrestore(&tgtport->lock, flags);
>   
> +	flush_workqueue(tgtport->work_q);
> +
>   	/* if there's nothing to wait for - call the callback */
>   	if (noassoc && tgtport->ops->host_release)
>   		tgtport->ops->host_release(hosthandle);
> @@ -1579,14 +1595,15 @@ nvmet_fc_delete_ctrl(struct nvmet_ctrl *ctrl)
>   		}
>   		spin_unlock_irqrestore(&tgtport->lock, flags);
>   
> -		nvmet_fc_tgtport_put(tgtport);
> -
>   		if (found_ctrl) {
> -			if (!schedule_work(&assoc->del_work))
> +			if (!queue_work(tgtport->work_q, &assoc->del_work))
>   				/* already deleting - release local reference */
>   				nvmet_fc_tgt_a_put(assoc);
> +			flush_workqueue(tgtport->work_q);
> +			nvmet_fc_tgtport_put(tgtport);
>   			return;
>   		}
> +		nvmet_fc_tgtport_put(tgtport);
>   
>   		spin_lock_irqsave(&nvmet_fc_tgtlock, flags);
>   	}

Looks reasonable.

Reviewed-by: James Smart <james.smart at broadcom.com>

-- james
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201005/0e86827f/attachment.p7s>

From james.smart at broadcom.com  Mon Oct  5 13:33:23 2020
From: james.smart at broadcom.com (James Smart)
Date: Mon, 5 Oct 2020 10:33:23 -0700
Subject: [PATCH 3/7] nvme-fcloop: use IDA for port ids
In-Reply-To: <20200922121501.32851-4-hare@suse.de>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-4-hare@suse.de>
Message-ID: <d0a614ff-c8b5-22fd-07f9-712a0259d760@broadcom.com>

On 9/22/2020 5:14 AM, Hannes Reinecke wrote:
> Use an IDA to register port IDs to avoid duplicate port IDs. It
> also will generate a unique port ID if none is specified.

I'd prefer a little more description as I didn't pick up what this 
actually did from the comment.? Meaning: Add auto-address assignment if 
one isn't specified. Use an ida for allocation for uniqueness. Use 
user-supplied addresses as long as not in use and within range supported 
for ida.

> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/target/fcloop.c | 51 +++++++++++++++++++++++++++++++++++++-------
>   1 file changed, 43 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
> index 082aa4dee406..e56f323fa7d4 100644
> --- a/drivers/nvme/target/fcloop.c
> +++ b/drivers/nvme/target/fcloop.c
> @@ -12,6 +12,7 @@
>   #include <linux/nvme-fc-driver.h>
>   #include <linux/nvme-fc.h>
>   
> +#define FCLOOP_MAX_AL_PA 0xEF

This is odd - no reason to bring in ALPA's and arb loop.?? Please rename 
it to state it's the max address value allowed.? I'd prefer if this was 
a 24 bit value, but that won't matter much and 255 is fine.

>   
>   enum {
>   	NVMF_OPT_ERR		= 0,
> @@ -63,6 +64,9 @@ fcloop_parse_options(struct fcloop_ctrl_options *opts,
>   	int token, ret = 0;
>   	u64 token64;
>   
> +	/* Default setting */
> +	opts->fcaddr = 1;
> +
>   	options = o = kstrdup(buf, GFP_KERNEL);
>   	if (!options)
>   		return -ENOMEM;
> @@ -102,6 +106,10 @@ fcloop_parse_options(struct fcloop_ctrl_options *opts,
>   				ret = -EINVAL;
>   				goto out_free_options;
>   			}
> +			if (!token || token > FCLOOP_MAX_AL_PA) {
> +				ret = -EINVAL;
> +				goto out_free_options;
> +			}
>   			opts->fcaddr = token;
>   			break;
>   		case NVMF_OPT_LPWWNN:
> @@ -203,11 +211,13 @@ fcloop_parse_nm_options(struct device *dev, u64 *nname, u64 *pname,
>   static DEFINE_SPINLOCK(fcloop_lock);
>   static LIST_HEAD(fcloop_lports);
>   static LIST_HEAD(fcloop_nports);
> +static DEFINE_IDA(fcloop_portid_ida);
>   
>   struct fcloop_lport {
>   	struct nvme_fc_local_port *localport;
>   	struct list_head lport_list;
>   	struct completion unreg_done;
> +	u32 port_id;
>   };
>   
>   struct fcloop_lport_priv {
> @@ -949,6 +959,8 @@ fcloop_nport_free(struct kref *ref)
>   	list_del(&nport->nport_list);
>   	spin_unlock_irqrestore(&fcloop_lock, flags);
>   
> +	ida_free(&fcloop_portid_ida, nport->port_id);
> +
>   	kfree(nport);
>   }
>   
> @@ -1047,7 +1059,7 @@ fcloop_create_local_port(struct device *dev, struct device_attribute *attr,
>   	struct fcloop_lport *lport;
>   	struct fcloop_lport_priv *lport_priv;
>   	unsigned long flags;
> -	int ret = -ENOMEM;
> +	int ret = -ENOMEM, port_id;
>   
>   	lport = kzalloc(sizeof(*lport), GFP_KERNEL);
>   	if (!lport)
> @@ -1067,11 +1079,22 @@ fcloop_create_local_port(struct device *dev, struct device_attribute *attr,
>   		goto out_free_opts;
>   	}
>   
> +	port_id = ida_alloc_range(&fcloop_portid_ida,
> +				  opts->fcaddr, FCLOOP_MAX_AL_PA, GFP_KERNEL);
> +	if (port_id < 0) {
> +		ret = port_id;
> +		goto out_free_opts;
> +	}
> +	if ((opts->mask & NVMF_OPT_FCADDR) && opts->fcaddr != port_id) {
> +		ret = -EINVAL;
> +		goto out_free_ida;
> +	}
> +
>   	memset(&pinfo, 0, sizeof(pinfo));
>   	pinfo.node_name = opts->wwnn;
>   	pinfo.port_name = opts->wwpn;
>   	pinfo.port_role = opts->roles;
> -	pinfo.port_id = opts->fcaddr;
> +	pinfo.port_id = port_id;
>   
>   	ret = nvme_fc_register_localport(&pinfo, &fctemplate, NULL, &localport);
>   	if (!ret) {
> @@ -1086,7 +1109,9 @@ fcloop_create_local_port(struct device *dev, struct device_attribute *attr,
>   		list_add_tail(&lport->lport_list, &fcloop_lports);
>   		spin_unlock_irqrestore(&fcloop_lock, flags);
>   	}
> -
> +out_free_ida:
> +	if (ret)
> +		ida_free(&fcloop_portid_ida, port_id);
>   out_free_opts:
>   	kfree(opts);
>   out_free_lport:
> @@ -1115,6 +1140,8 @@ __wait_localport_unreg(struct fcloop_lport *lport)
>   
>   	wait_for_completion(&lport->unreg_done);
>   
> +	ida_free(&fcloop_portid_ida, lport->port_id);
> +
>   	kfree(lport);
>   
>   	return ret;
> @@ -1162,7 +1189,7 @@ fcloop_alloc_nport(const char *buf, size_t count, bool remoteport)
>   	struct fcloop_ctrl_options *opts;
>   	unsigned long flags;
>   	u32 opts_mask = (remoteport) ? RPORT_OPTS : TGTPORT_OPTS;
> -	int ret;
> +	int ret, port_id;
>   
>   	opts = kzalloc(sizeof(*opts), GFP_KERNEL);
>   	if (!opts)
> @@ -1182,13 +1209,21 @@ fcloop_alloc_nport(const char *buf, size_t count, bool remoteport)
>   	if (!newnport)
>   		goto out_free_opts;
>   
> +	port_id = ida_alloc_range(&fcloop_portid_ida,
> +				  opts->fcaddr, FCLOOP_MAX_AL_PA, GFP_KERNEL);
> +	if (port_id < 0)
> +		goto out_free_newnport;
> +
> +	if ((opts->mask & NVMF_OPT_FCADDR) && opts->fcaddr != port_id)
> +		goto out_free_ida;
> +
>   	INIT_LIST_HEAD(&newnport->nport_list);
>   	newnport->node_name = opts->wwnn;
>   	newnport->port_name = opts->wwpn;
> +	newnport->port_id = port_id;
>   	if (opts->mask & NVMF_OPT_ROLES)
>   		newnport->port_role = opts->roles;
> -	if (opts->mask & NVMF_OPT_FCADDR)
> -		newnport->port_id = opts->fcaddr;
> +
>   	kref_init(&newnport->ref);
>   
>   	spin_lock_irqsave(&fcloop_lock, flags);
> @@ -1226,8 +1261,6 @@ fcloop_alloc_nport(const char *buf, size_t count, bool remoteport)
>   				nport->lport = lport;
>   			if (opts->mask & NVMF_OPT_ROLES)
>   				nport->port_role = opts->roles;
> -			if (opts->mask & NVMF_OPT_FCADDR)
> -				nport->port_id = opts->fcaddr;
>   			goto out_free_newnport;
>   		}
>   	}
> @@ -1241,6 +1274,8 @@ fcloop_alloc_nport(const char *buf, size_t count, bool remoteport)
>   
>   out_invalid_opts:
>   	spin_unlock_irqrestore(&fcloop_lock, flags);
> +out_free_ida:
> +	ida_free(&fcloop_portid_ida, port_id);
>   out_free_newnport:
>   	kfree(newnport);
>   out_free_opts:

These are minor nits - I'm good with the patch

Reviewed-by: James Smart <james.smart at broadcom.com>

-- james
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201005/e8d691f7/attachment-0001.p7s>

From james.smart at broadcom.com  Mon Oct  5 13:37:11 2020
From: james.smart at broadcom.com (James Smart)
Date: Mon, 5 Oct 2020 10:37:11 -0700
Subject: [PATCH 4/7] nvmet-fc: use feature flag for virtual LLDD
In-Reply-To: <20200922121501.32851-5-hare@suse.de>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-5-hare@suse.de>
Message-ID: <990c02ea-0b84-5301-0d0d-e75556df02b7@broadcom.com>

On 9/22/2020 5:14 AM, Hannes Reinecke wrote:
> Virtual LLDDs like fcloop don't need to do DMA, but still
> might want to expose a device. So add a new feature flag
> to mark these LLDDs instead of relying on a non-existing
> struct device.
>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/target/fc.c       | 93 +++++++++++++++++++++++-------------------
>   drivers/nvme/target/fcloop.c   |  2 +-
>   include/linux/nvme-fc-driver.h |  2 +
>   3 files changed, 55 insertions(+), 42 deletions(-)
>
> diff --git a/drivers/nvme/target/fc.c b/drivers/nvme/target/fc.c
> index 63f5deb3b68a..6f5784767d35 100644
> --- a/drivers/nvme/target/fc.c
> +++ b/drivers/nvme/target/fc.c
> @@ -273,41 +273,50 @@ static void nvmet_fc_xmt_ls_rsp(struct nvmet_fc_tgtport *tgtport,
>    * in the scatter list, setting all dma addresses to 0.
>    */
>   
> +static bool fc_lldd_is_virtual(struct nvmet_fc_tgtport *tgtport)
> +{
> +	return !!(tgtport->ops->target_features & NVMET_FCTGTFEAT_VIRTUAL_DMA);
> +}
> +
>   static inline dma_addr_t
> -fc_dma_map_single(struct device *dev, void *ptr, size_t size,
> +fc_dma_map_single(struct nvmet_fc_tgtport *tgtport, void *ptr, size_t size,
>   		enum dma_data_direction dir)
>   {
> -	return dev ? dma_map_single(dev, ptr, size, dir) : (dma_addr_t)0L;
> +	if (fc_lldd_is_virtual(tgtport))
> +		return (dma_addr_t)0L;
> +	return dma_map_single(tgtport->dev, ptr, size, dir);
>   }
>   
>   static inline int
> -fc_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
> +fc_dma_mapping_error(struct nvmet_fc_tgtport *tgtport, dma_addr_t dma_addr)
>   {
> -	return dev ? dma_mapping_error(dev, dma_addr) : 0;
> +	if (fc_lldd_is_virtual(tgtport))
> +		return 0;
> +	return dma_mapping_error(tgtport->dev, dma_addr);
>   }
>   
>   static inline void
> -fc_dma_unmap_single(struct device *dev, dma_addr_t addr, size_t size,
> -	enum dma_data_direction dir)
> +fc_dma_unmap_single(struct nvmet_fc_tgtport *tgtport, dma_addr_t addr,
> +	size_t size, enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_unmap_single(dev, addr, size, dir);
> +	if (!fc_lldd_is_virtual(tgtport))
> +		dma_unmap_single(tgtport->dev, addr, size, dir);
>   }
>   
>   static inline void
> -fc_dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr, size_t size,
> -		enum dma_data_direction dir)
> +fc_dma_sync_single_for_cpu(struct nvmet_fc_tgtport *tgtport, dma_addr_t addr,
> +	size_t size, enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_sync_single_for_cpu(dev, addr, size, dir);
> +	if (!fc_lldd_is_virtual(tgtport))
> +		dma_sync_single_for_cpu(tgtport->dev, addr, size, dir);
>   }
>   
>   static inline void
> -fc_dma_sync_single_for_device(struct device *dev, dma_addr_t addr, size_t size,
> -		enum dma_data_direction dir)
> +fc_dma_sync_single_for_device(struct nvmet_fc_tgtport *tgtport, dma_addr_t addr,
> +	size_t size, enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_sync_single_for_device(dev, addr, size, dir);
> +	if (!fc_lldd_is_virtual(tgtport))
> +		dma_sync_single_for_device(tgtport->dev, addr, size, dir);
>   }
>   
>   /* pseudo dma_map_sg call */
> @@ -329,18 +338,20 @@ fc_map_sg(struct scatterlist *sg, int nents)
>   }
>   
>   static inline int
> -fc_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
> -		enum dma_data_direction dir)
> +fc_dma_map_sg(struct nvmet_fc_tgtport *tgtport, struct scatterlist *sg,
> +	int nents, enum dma_data_direction dir)
>   {
> -	return dev ? dma_map_sg(dev, sg, nents, dir) : fc_map_sg(sg, nents);
> +	if (fc_lldd_is_virtual(tgtport))
> +		return fc_map_sg(sg, nents);
> +	return dma_map_sg(tgtport->dev, sg, nents, dir);
>   }
>   
>   static inline void
> -fc_dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
> -		enum dma_data_direction dir)
> +fc_dma_unmap_sg(struct nvmet_fc_tgtport *tgtport, struct scatterlist *sg,
> +	int nents, enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_unmap_sg(dev, sg, nents, dir);
> +	if (!fc_lldd_is_virtual(tgtport))
> +		dma_unmap_sg(tgtport->dev, sg, nents, dir);
>   }
>   
>   
> @@ -368,7 +379,7 @@ __nvmet_fc_finish_ls_req(struct nvmet_fc_ls_req_op *lsop)
>   
>   	spin_unlock_irqrestore(&tgtport->lock, flags);
>   
> -	fc_dma_unmap_single(tgtport->dev, lsreq->rqstdma,
> +	fc_dma_unmap_single(tgtport, lsreq->rqstdma,
>   				  (lsreq->rqstlen + lsreq->rsplen),
>   				  DMA_BIDIRECTIONAL);
>   
> @@ -391,10 +402,10 @@ __nvmet_fc_send_ls_req(struct nvmet_fc_tgtport *tgtport,
>   	lsop->req_queued = false;
>   	INIT_LIST_HEAD(&lsop->lsreq_list);
>   
> -	lsreq->rqstdma = fc_dma_map_single(tgtport->dev, lsreq->rqstaddr,
> +	lsreq->rqstdma = fc_dma_map_single(tgtport, lsreq->rqstaddr,
>   				  lsreq->rqstlen + lsreq->rsplen,
>   				  DMA_BIDIRECTIONAL);
> -	if (fc_dma_mapping_error(tgtport->dev, lsreq->rqstdma))
> +	if (fc_dma_mapping_error(tgtport, lsreq->rqstdma))
>   		return -EFAULT;
>   
>   	lsreq->rspdma = lsreq->rqstdma + lsreq->rqstlen;
> @@ -420,7 +431,7 @@ __nvmet_fc_send_ls_req(struct nvmet_fc_tgtport *tgtport,
>   	lsop->req_queued = false;
>   	list_del(&lsop->lsreq_list);
>   	spin_unlock_irqrestore(&tgtport->lock, flags);
> -	fc_dma_unmap_single(tgtport->dev, lsreq->rqstdma,
> +	fc_dma_unmap_single(tgtport, lsreq->rqstdma,
>   				  (lsreq->rqstlen + lsreq->rsplen),
>   				  DMA_BIDIRECTIONAL);
>   	return ret;
> @@ -555,10 +566,10 @@ nvmet_fc_alloc_ls_iodlist(struct nvmet_fc_tgtport *tgtport)
>   
>   		iod->rspbuf = (union nvmefc_ls_responses *)&iod->rqstbuf[1];
>   
> -		iod->rspdma = fc_dma_map_single(tgtport->dev, iod->rspbuf,
> +		iod->rspdma = fc_dma_map_single(tgtport, iod->rspbuf,
>   						sizeof(*iod->rspbuf),
>   						DMA_TO_DEVICE);
> -		if (fc_dma_mapping_error(tgtport->dev, iod->rspdma))
> +		if (fc_dma_mapping_error(tgtport, iod->rspdma))
>   			goto out_fail;
>   	}
>   
> @@ -568,7 +579,7 @@ nvmet_fc_alloc_ls_iodlist(struct nvmet_fc_tgtport *tgtport)
>   	kfree(iod->rqstbuf);
>   	list_del(&iod->ls_rcv_list);
>   	for (iod--, i--; i >= 0; iod--, i--) {
> -		fc_dma_unmap_single(tgtport->dev, iod->rspdma,
> +		fc_dma_unmap_single(tgtport, iod->rspdma,
>   				sizeof(*iod->rspbuf), DMA_TO_DEVICE);
>   		kfree(iod->rqstbuf);
>   		list_del(&iod->ls_rcv_list);
> @@ -586,7 +597,7 @@ nvmet_fc_free_ls_iodlist(struct nvmet_fc_tgtport *tgtport)
>   	int i;
>   
>   	for (i = 0; i < NVMET_LS_CTX_COUNT; iod++, i++) {
> -		fc_dma_unmap_single(tgtport->dev,
> +		fc_dma_unmap_single(tgtport,
>   				iod->rspdma, sizeof(*iod->rspbuf),
>   				DMA_TO_DEVICE);
>   		kfree(iod->rqstbuf);
> @@ -640,12 +651,12 @@ nvmet_fc_prep_fcp_iodlist(struct nvmet_fc_tgtport *tgtport,
>   		list_add_tail(&fod->fcp_list, &queue->fod_list);
>   		spin_lock_init(&fod->flock);
>   
> -		fod->rspdma = fc_dma_map_single(tgtport->dev, &fod->rspiubuf,
> +		fod->rspdma = fc_dma_map_single(tgtport, &fod->rspiubuf,
>   					sizeof(fod->rspiubuf), DMA_TO_DEVICE);
> -		if (fc_dma_mapping_error(tgtport->dev, fod->rspdma)) {
> +		if (fc_dma_mapping_error(tgtport, fod->rspdma)) {
>   			list_del(&fod->fcp_list);
>   			for (fod--, i--; i >= 0; fod--, i--) {
> -				fc_dma_unmap_single(tgtport->dev, fod->rspdma,
> +				fc_dma_unmap_single(tgtport, fod->rspdma,
>   						sizeof(fod->rspiubuf),
>   						DMA_TO_DEVICE);
>   				fod->rspdma = 0L;
> @@ -666,7 +677,7 @@ nvmet_fc_destroy_fcp_iodlist(struct nvmet_fc_tgtport *tgtport,
>   
>   	for (i = 0; i < queue->sqsize; fod++, i++) {
>   		if (fod->rspdma)
> -			fc_dma_unmap_single(tgtport->dev, fod->rspdma,
> +			fc_dma_unmap_single(tgtport, fod->rspdma,
>   				sizeof(fod->rspiubuf), DMA_TO_DEVICE);
>   	}
>   }
> @@ -730,7 +741,7 @@ nvmet_fc_free_fcp_iod(struct nvmet_fc_tgt_queue *queue,
>   	struct nvmet_fc_defer_fcp_req *deferfcp;
>   	unsigned long flags;
>   
> -	fc_dma_sync_single_for_cpu(tgtport->dev, fod->rspdma,
> +	fc_dma_sync_single_for_cpu(tgtport, fod->rspdma,
>   				sizeof(fod->rspiubuf), DMA_TO_DEVICE);
>   
>   	fcpreq->nvmet_fc_private = NULL;
> @@ -1925,7 +1936,7 @@ nvmet_fc_xmt_ls_rsp_done(struct nvmefc_ls_rsp *lsrsp)
>   	struct nvmet_fc_ls_iod *iod = lsrsp->nvme_fc_private;
>   	struct nvmet_fc_tgtport *tgtport = iod->tgtport;
>   
> -	fc_dma_sync_single_for_cpu(tgtport->dev, iod->rspdma,
> +	fc_dma_sync_single_for_cpu(tgtport, iod->rspdma,
>   				sizeof(*iod->rspbuf), DMA_TO_DEVICE);
>   	nvmet_fc_free_ls_iod(tgtport, iod);
>   	nvmet_fc_tgtport_put(tgtport);
> @@ -1937,7 +1948,7 @@ nvmet_fc_xmt_ls_rsp(struct nvmet_fc_tgtport *tgtport,
>   {
>   	int ret;
>   
> -	fc_dma_sync_single_for_device(tgtport->dev, iod->rspdma,
> +	fc_dma_sync_single_for_device(tgtport, iod->rspdma,
>   				  sizeof(*iod->rspbuf), DMA_TO_DEVICE);
>   
>   	ret = tgtport->ops->xmt_ls_rsp(&tgtport->fc_target_port, iod->lsrsp);
> @@ -2091,7 +2102,7 @@ nvmet_fc_alloc_tgt_pgs(struct nvmet_fc_fcp_iod *fod)
>   
>   	fod->data_sg = sg;
>   	fod->data_sg_cnt = nent;
> -	fod->data_sg_cnt = fc_dma_map_sg(fod->tgtport->dev, sg, nent,
> +	fod->data_sg_cnt = fc_dma_map_sg(fod->tgtport, sg, nent,
>   				((fod->io_dir == NVMET_FCP_WRITE) ?
>   					DMA_FROM_DEVICE : DMA_TO_DEVICE));
>   				/* note: write from initiator perspective */
> @@ -2109,7 +2120,7 @@ nvmet_fc_free_tgt_pgs(struct nvmet_fc_fcp_iod *fod)
>   	if (!fod->data_sg || !fod->data_sg_cnt)
>   		return;
>   
> -	fc_dma_unmap_sg(fod->tgtport->dev, fod->data_sg, fod->data_sg_cnt,
> +	fc_dma_unmap_sg(fod->tgtport, fod->data_sg, fod->data_sg_cnt,
>   				((fod->io_dir == NVMET_FCP_WRITE) ?
>   					DMA_FROM_DEVICE : DMA_TO_DEVICE));
>   	sgl_free(fod->data_sg);
> @@ -2193,7 +2204,7 @@ nvmet_fc_prep_fcp_rsp(struct nvmet_fc_tgtport *tgtport,
>   		fod->fcpreq->rsplen = sizeof(*ersp);
>   	}
>   
> -	fc_dma_sync_single_for_device(tgtport->dev, fod->rspdma,
> +	fc_dma_sync_single_for_device(tgtport, fod->rspdma,
>   				  sizeof(fod->rspiubuf), DMA_TO_DEVICE);
>   }
>   
> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
> index e56f323fa7d4..2ccb941efb21 100644
> --- a/drivers/nvme/target/fcloop.c
> +++ b/drivers/nvme/target/fcloop.c
> @@ -1043,7 +1043,7 @@ static struct nvmet_fc_target_template tgttemplate = {
>   	.max_dif_sgl_segments	= FCLOOP_SGL_SEGS,
>   	.dma_boundary		= FCLOOP_DMABOUND_4G,
>   	/* optional features */
> -	.target_features	= 0,
> +	.target_features	= NVMET_FCTGTFEAT_VIRTUAL_DMA,
>   	/* sizes of additional private data for data structures */
>   	.target_priv_sz		= sizeof(struct fcloop_tport),
>   	.lsrqst_priv_sz		= sizeof(struct fcloop_lsreq),
> diff --git a/include/linux/nvme-fc-driver.h b/include/linux/nvme-fc-driver.h
> index 2a38f2b477a5..675c7ef6df17 100644
> --- a/include/linux/nvme-fc-driver.h
> +++ b/include/linux/nvme-fc-driver.h
> @@ -707,6 +707,8 @@ enum {
>   		 * sequence in one LLDD operation. Errors during Data
>   		 * sequence transmit must not allow RSP sequence to be sent.
>   		 */
> +	NVMET_FCTGTFEAT_VIRTUAL_DMA = (1 << 1),
> +		/* Bit 1: Virtual LLDD with no DMA support */
>   };
>   
>   

Ok.

Reviewed-by: James Smart <james.smart at broadcom.com>


-- james
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201005/448c4ed3/attachment.p7s>

From james.smart at broadcom.com  Mon Oct  5 13:38:41 2020
From: james.smart at broadcom.com (James Smart)
Date: Mon, 5 Oct 2020 10:38:41 -0700
Subject: [PATCH 5/7] nvme-fc: use feature flag for virtual LLDD
In-Reply-To: <20200922121501.32851-6-hare@suse.de>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-6-hare@suse.de>
Message-ID: <0dc3926c-fd72-8c6d-68c4-c9f231503943@broadcom.com>

On 9/22/2020 5:14 AM, Hannes Reinecke wrote:
> Virtual LLDDs like fcloop don't need to do DMA, but still
> might want to expose a device. So add a new feature flag
> to mark these LLDDs instead of relying on a non-existing
> struct device.
>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/host/fc.c         | 93 +++++++++++++++++++++++-------------------
>   drivers/nvme/target/fcloop.c   |  2 +
>   include/linux/nvme-fc-driver.h | 10 +++++
>   3 files changed, 64 insertions(+), 41 deletions(-)
>
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index e8ef42b9d50c..d30ea7a39183 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -237,6 +237,11 @@ static void __nvme_fc_delete_hw_queue(struct nvme_fc_ctrl *,
>   static void nvme_fc_handle_ls_rqst_work(struct work_struct *work);
>   
>   
> +static bool nvme_fc_lport_is_physical(struct nvme_fc_lport *lport)
> +{
> +	return !(lport->ops->port_features & NVME_FCPORTFEAT_VIRTUAL_DMA);
> +}
> +
>   static void
>   nvme_fc_free_lport(struct kref *ref)
>   {
> @@ -427,7 +432,7 @@ nvme_fc_register_localport(struct nvme_fc_port_info *pinfo,
>   	list_add_tail(&newrec->port_list, &nvme_fc_lport_list);
>   	spin_unlock_irqrestore(&nvme_fc_lock, flags);
>   
> -	if (dev)
> +	if (nvme_fc_lport_is_physical(newrec))
>   		dma_set_seg_boundary(dev, template->dma_boundary);
>   
>   	*portptr = &newrec->localport;
> @@ -953,40 +958,44 @@ EXPORT_SYMBOL_GPL(nvme_fc_set_remoteport_devloss);
>    */
>   
>   static inline dma_addr_t
> -fc_dma_map_single(struct device *dev, void *ptr, size_t size,
> +fc_dma_map_single(struct nvme_fc_lport *lport, void *ptr, size_t size,
>   		enum dma_data_direction dir)
>   {
> -	return dev ? dma_map_single(dev, ptr, size, dir) : (dma_addr_t)0L;
> +	if (nvme_fc_lport_is_physical(lport))
> +		return dma_map_single(lport->dev, ptr, size, dir);
> +	return (dma_addr_t)0L;
>   }
>   
>   static inline int
> -fc_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
> +fc_dma_mapping_error(struct nvme_fc_lport *lport, dma_addr_t dma_addr)
>   {
> -	return dev ? dma_mapping_error(dev, dma_addr) : 0;
> +	if (nvme_fc_lport_is_physical(lport))
> +		return dma_mapping_error(lport->dev, dma_addr);
> +	return 0;
>   }
>   
>   static inline void
> -fc_dma_unmap_single(struct device *dev, dma_addr_t addr, size_t size,
> +fc_dma_unmap_single(struct nvme_fc_lport *lport, dma_addr_t addr, size_t size,
>   	enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_unmap_single(dev, addr, size, dir);
> +	if (nvme_fc_lport_is_physical(lport))
> +		dma_unmap_single(lport->dev, addr, size, dir);
>   }
>   
>   static inline void
> -fc_dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr, size_t size,
> -		enum dma_data_direction dir)
> +fc_dma_sync_single_for_cpu(struct nvme_fc_lport *lport, dma_addr_t addr,
> +	size_t size, enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_sync_single_for_cpu(dev, addr, size, dir);
> +	if (nvme_fc_lport_is_physical(lport))
> +		dma_sync_single_for_cpu(lport->dev, addr, size, dir);
>   }
>   
>   static inline void
> -fc_dma_sync_single_for_device(struct device *dev, dma_addr_t addr, size_t size,
> -		enum dma_data_direction dir)
> +fc_dma_sync_single_for_device(struct nvme_fc_lport *lport, dma_addr_t addr,
> +	size_t size, enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_sync_single_for_device(dev, addr, size, dir);
> +	if (nvme_fc_lport_is_physical(lport))
> +		dma_sync_single_for_device(lport->dev, addr, size, dir);
>   }
>   
>   /* pseudo dma_map_sg call */
> @@ -1008,18 +1017,20 @@ fc_map_sg(struct scatterlist *sg, int nents)
>   }
>   
>   static inline int
> -fc_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
> +fc_dma_map_sg(struct nvme_fc_lport *lport, struct scatterlist *sg, int nents,
>   		enum dma_data_direction dir)
>   {
> -	return dev ? dma_map_sg(dev, sg, nents, dir) : fc_map_sg(sg, nents);
> +	if (nvme_fc_lport_is_physical(lport))
> +		return dma_map_sg(lport->dev, sg, nents, dir);
> +	return fc_map_sg(sg, nents);
>   }
>   
>   static inline void
> -fc_dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
> +fc_dma_unmap_sg(struct nvme_fc_lport *lport, struct scatterlist *sg, int nents,
>   		enum dma_data_direction dir)
>   {
> -	if (dev)
> -		dma_unmap_sg(dev, sg, nents, dir);
> +	if (nvme_fc_lport_is_physical(lport))
> +		dma_unmap_sg(lport->dev, sg, nents, dir);
>   }
>   
>   /* *********************** FC-NVME LS Handling **************************** */
> @@ -1049,7 +1060,7 @@ __nvme_fc_finish_ls_req(struct nvmefc_ls_req_op *lsop)
>   
>   	spin_unlock_irqrestore(&rport->lock, flags);
>   
> -	fc_dma_unmap_single(rport->dev, lsreq->rqstdma,
> +	fc_dma_unmap_single(rport->lport, lsreq->rqstdma,
>   				  (lsreq->rqstlen + lsreq->rsplen),
>   				  DMA_BIDIRECTIONAL);
>   
> @@ -1077,10 +1088,10 @@ __nvme_fc_send_ls_req(struct nvme_fc_rport *rport,
>   	INIT_LIST_HEAD(&lsop->lsreq_list);
>   	init_completion(&lsop->ls_done);
>   
> -	lsreq->rqstdma = fc_dma_map_single(rport->dev, lsreq->rqstaddr,
> +	lsreq->rqstdma = fc_dma_map_single(rport->lport, lsreq->rqstaddr,
>   				  lsreq->rqstlen + lsreq->rsplen,
>   				  DMA_BIDIRECTIONAL);
> -	if (fc_dma_mapping_error(rport->dev, lsreq->rqstdma)) {
> +	if (fc_dma_mapping_error(rport->lport, lsreq->rqstdma)) {
>   		ret = -EFAULT;
>   		goto out_putrport;
>   	}
> @@ -1107,7 +1118,7 @@ __nvme_fc_send_ls_req(struct nvme_fc_rport *rport,
>   	lsop->req_queued = false;
>   	list_del(&lsop->lsreq_list);
>   	spin_unlock_irqrestore(&rport->lock, flags);
> -	fc_dma_unmap_single(rport->dev, lsreq->rqstdma,
> +	fc_dma_unmap_single(rport->lport, lsreq->rqstdma,
>   				  (lsreq->rqstlen + lsreq->rsplen),
>   				  DMA_BIDIRECTIONAL);
>   out_putrport:
> @@ -1465,9 +1476,9 @@ nvme_fc_xmt_ls_rsp_done(struct nvmefc_ls_rsp *lsrsp)
>   	list_del(&lsop->lsrcv_list);
>   	spin_unlock_irqrestore(&rport->lock, flags);
>   
> -	fc_dma_sync_single_for_cpu(lport->dev, lsop->rspdma,
> +	fc_dma_sync_single_for_cpu(lport, lsop->rspdma,
>   				sizeof(*lsop->rspbuf), DMA_TO_DEVICE);
> -	fc_dma_unmap_single(lport->dev, lsop->rspdma,
> +	fc_dma_unmap_single(lport, lsop->rspdma,
>   			sizeof(*lsop->rspbuf), DMA_TO_DEVICE);
>   
>   	kfree(lsop);
> @@ -1483,7 +1494,7 @@ nvme_fc_xmt_ls_rsp(struct nvmefc_ls_rcv_op *lsop)
>   	struct fcnvme_ls_rqst_w0 *w0 = &lsop->rqstbuf->w0;
>   	int ret;
>   
> -	fc_dma_sync_single_for_device(lport->dev, lsop->rspdma,
> +	fc_dma_sync_single_for_device(lport, lsop->rspdma,
>   				  sizeof(*lsop->rspbuf), DMA_TO_DEVICE);
>   
>   	ret = lport->ops->xmt_ls_rsp(&lport->localport, &rport->remoteport,
> @@ -1761,10 +1772,10 @@ nvme_fc_rcv_ls_req(struct nvme_fc_remote_port *portptr,
>   	lsop->rqstbuf = (union nvmefc_ls_requests *)&lsop[1];
>   	lsop->rspbuf = (union nvmefc_ls_responses *)&lsop->rqstbuf[1];
>   
> -	lsop->rspdma = fc_dma_map_single(lport->dev, lsop->rspbuf,
> +	lsop->rspdma = fc_dma_map_single(lport, lsop->rspbuf,
>   					sizeof(*lsop->rspbuf),
>   					DMA_TO_DEVICE);
> -	if (fc_dma_mapping_error(lport->dev, lsop->rspdma)) {
> +	if (fc_dma_mapping_error(lport, lsop->rspdma)) {
>   		dev_info(lport->dev,
>   			"RCV %s LS failed: DMA mapping failure\n",
>   			(w0->ls_cmd <= NVME_FC_LAST_LS_CMD_VALUE) ?
> @@ -1793,7 +1804,7 @@ nvme_fc_rcv_ls_req(struct nvme_fc_remote_port *portptr,
>   	return 0;
>   
>   out_unmap:
> -	fc_dma_unmap_single(lport->dev, lsop->rspdma,
> +	fc_dma_unmap_single(lport, lsop->rspdma,
>   			sizeof(*lsop->rspbuf), DMA_TO_DEVICE);
>   out_free:
>   	kfree(lsop);
> @@ -1810,9 +1821,9 @@ static void
>   __nvme_fc_exit_request(struct nvme_fc_ctrl *ctrl,
>   		struct nvme_fc_fcp_op *op)
>   {
> -	fc_dma_unmap_single(ctrl->lport->dev, op->fcp_req.rspdma,
> +	fc_dma_unmap_single(ctrl->lport, op->fcp_req.rspdma,
>   				sizeof(op->rsp_iu), DMA_FROM_DEVICE);
> -	fc_dma_unmap_single(ctrl->lport->dev, op->fcp_req.cmddma,
> +	fc_dma_unmap_single(ctrl->lport, op->fcp_req.cmddma,
>   				sizeof(op->cmd_iu), DMA_TO_DEVICE);
>   
>   	atomic_set(&op->state, FCPOP_STATE_UNINIT);
> @@ -1936,7 +1947,7 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
>   
>   	opstate = atomic_xchg(&op->state, FCPOP_STATE_COMPLETE);
>   
> -	fc_dma_sync_single_for_cpu(ctrl->lport->dev, op->fcp_req.rspdma,
> +	fc_dma_sync_single_for_cpu(ctrl->lport, op->fcp_req.rspdma,
>   				sizeof(op->rsp_iu), DMA_FROM_DEVICE);
>   
>   	if (opstate == FCPOP_STATE_ABORTED)
> @@ -2073,19 +2084,19 @@ __nvme_fc_init_request(struct nvme_fc_ctrl *ctrl,
>   	else
>   		cmdiu->rsv_cat = fccmnd_set_cat_admin(0);
>   
> -	op->fcp_req.cmddma = fc_dma_map_single(ctrl->lport->dev,
> +	op->fcp_req.cmddma = fc_dma_map_single(ctrl->lport,
>   				&op->cmd_iu, sizeof(op->cmd_iu), DMA_TO_DEVICE);
> -	if (fc_dma_mapping_error(ctrl->lport->dev, op->fcp_req.cmddma)) {
> +	if (fc_dma_mapping_error(ctrl->lport, op->fcp_req.cmddma)) {
>   		dev_err(ctrl->dev,
>   			"FCP Op failed - cmdiu dma mapping failed.\n");
>   		ret = -EFAULT;
>   		goto out_on_error;
>   	}
>   
> -	op->fcp_req.rspdma = fc_dma_map_single(ctrl->lport->dev,
> +	op->fcp_req.rspdma = fc_dma_map_single(ctrl->lport,
>   				&op->rsp_iu, sizeof(op->rsp_iu),
>   				DMA_FROM_DEVICE);
> -	if (fc_dma_mapping_error(ctrl->lport->dev, op->fcp_req.rspdma)) {
> +	if (fc_dma_mapping_error(ctrl->lport, op->fcp_req.rspdma)) {
>   		dev_err(ctrl->dev,
>   			"FCP Op failed - rspiu dma mapping failed.\n");
>   		ret = -EFAULT;
> @@ -2485,7 +2496,7 @@ nvme_fc_map_data(struct nvme_fc_ctrl *ctrl, struct request *rq,
>   
>   	op->nents = blk_rq_map_sg(rq->q, rq, freq->sg_table.sgl);
>   	WARN_ON(op->nents > blk_rq_nr_phys_segments(rq));
> -	freq->sg_cnt = fc_dma_map_sg(ctrl->lport->dev, freq->sg_table.sgl,
> +	freq->sg_cnt = fc_dma_map_sg(ctrl->lport, freq->sg_table.sgl,
>   				op->nents, rq_dma_dir(rq));
>   	if (unlikely(freq->sg_cnt <= 0)) {
>   		sg_free_table_chained(&freq->sg_table, NVME_INLINE_SG_CNT);
> @@ -2508,7 +2519,7 @@ nvme_fc_unmap_data(struct nvme_fc_ctrl *ctrl, struct request *rq,
>   	if (!freq->sg_cnt)
>   		return;
>   
> -	fc_dma_unmap_sg(ctrl->lport->dev, freq->sg_table.sgl, op->nents,
> +	fc_dma_unmap_sg(ctrl->lport, freq->sg_table.sgl, op->nents,
>   			rq_dma_dir(rq));
>   
>   	sg_free_table_chained(&freq->sg_table, NVME_INLINE_SG_CNT);
> @@ -2609,7 +2620,7 @@ nvme_fc_start_fcp_op(struct nvme_fc_ctrl *ctrl, struct nvme_fc_queue *queue,
>   		}
>   	}
>   
> -	fc_dma_sync_single_for_device(ctrl->lport->dev, op->fcp_req.cmddma,
> +	fc_dma_sync_single_for_device(ctrl->lport, op->fcp_req.cmddma,
>   				  sizeof(op->cmd_iu), DMA_TO_DEVICE);
>   
>   	atomic_set(&op->state, FCPOP_STATE_ACTIVE);
> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
> index 2ccb941efb21..7dab98545979 100644
> --- a/drivers/nvme/target/fcloop.c
> +++ b/drivers/nvme/target/fcloop.c
> @@ -1026,6 +1026,8 @@ static struct nvme_fc_port_template fctemplate = {
>   	.remote_priv_sz		= sizeof(struct fcloop_rport),
>   	.lsrqst_priv_sz		= sizeof(struct fcloop_lsreq),
>   	.fcprqst_priv_sz	= sizeof(struct fcloop_ini_fcpreq),
> +	/* optional features */
> +	.port_features		= NVME_FCPORTFEAT_VIRTUAL_DMA,
>   };
>   
>   static struct nvmet_fc_target_template tgttemplate = {
> diff --git a/include/linux/nvme-fc-driver.h b/include/linux/nvme-fc-driver.h
> index 675c7ef6df17..17cf967484b2 100644
> --- a/include/linux/nvme-fc-driver.h
> +++ b/include/linux/nvme-fc-driver.h
> @@ -334,6 +334,11 @@ struct nvme_fc_remote_port {
>   	enum nvme_fc_obj_state port_state;
>   } __aligned(sizeof(u64));	/* alignment for other things alloc'd with */
>   
> +/* Port Features (Bit fields) LLDD supports */
> +enum {
> +	NVME_FCPORTFEAT_VIRTUAL_DMA = (1 << 0),
> +		/* Bit 0: Virtual LLDD with no DMA support */
> +};
>   
>   /**
>    * struct nvme_fc_port_template - structure containing static entrypoints and
> @@ -470,6 +475,8 @@ struct nvme_fc_remote_port {
>    *       memory area solely for the of the LLDD and its location is
>    *       specified by the fcp_request->private pointer.
>    *       Value is Mandatory. Allowed to be zero.
> + *
> + * @port_features: Optional feature for the LLDD
>    */
>   struct nvme_fc_port_template {
>   	/* initiator-based functions */
> @@ -508,6 +515,9 @@ struct nvme_fc_port_template {
>   	u32	remote_priv_sz;
>   	u32	lsrqst_priv_sz;
>   	u32	fcprqst_priv_sz;
> +
> +	/* Optional port features */
> +	u32	port_features;
>   };
>   
>   

OK.

Reviewed-by: James Smart <james.smart at broadcom.com>


-- james
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201005/85a92df0/attachment-0001.p7s>

From james.smart at broadcom.com  Mon Oct  5 13:41:15 2020
From: james.smart at broadcom.com (James Smart)
Date: Mon, 5 Oct 2020 10:41:15 -0700
Subject: [PATCH 6/7] nvme-fcloop: use a device for nport
In-Reply-To: <20200922121501.32851-7-hare@suse.de>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-7-hare@suse.de>
Message-ID: <4ca1d58e-0831-6596-020f-311932b3c439@broadcom.com>

On 9/22/2020 5:15 AM, Hannes Reinecke wrote:
> Add a 'struct device' to the nport such that the nport will show
> up in sysfs and we get a valid name in the logging output.
>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/target/fcloop.c | 31 +++++++++++++++++++++----------
>   1 file changed, 21 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
> index 7dab98545979..b84f8754b3a3 100644
> --- a/drivers/nvme/target/fcloop.c
> +++ b/drivers/nvme/target/fcloop.c
> @@ -245,11 +245,11 @@ struct fcloop_tport {
>   };
>   
>   struct fcloop_nport {
> +	struct device dev;
>   	struct fcloop_rport *rport;
>   	struct fcloop_tport *tport;
>   	struct fcloop_lport *lport;
>   	struct list_head nport_list;
> -	struct kref ref;
>   	u64 node_name;
>   	u64 port_name;
>   	u32 port_role;
> @@ -949,10 +949,10 @@ fcloop_fcp_abort(struct nvme_fc_local_port *localport,
>   }
>   
>   static void
> -fcloop_nport_free(struct kref *ref)
> +fcloop_release_nport(struct device *dev)
>   {
>   	struct fcloop_nport *nport =
> -		container_of(ref, struct fcloop_nport, ref);
> +		container_of(dev, struct fcloop_nport, dev);
>   	unsigned long flags;
>   
>   	spin_lock_irqsave(&fcloop_lock, flags);
> @@ -967,13 +967,13 @@ fcloop_nport_free(struct kref *ref)
>   static void
>   fcloop_nport_put(struct fcloop_nport *nport)
>   {
> -	kref_put(&nport->ref, fcloop_nport_free);
> +	put_device(&nport->dev);
>   }
>   
> -static int
> +static void
>   fcloop_nport_get(struct fcloop_nport *nport)
>   {
> -	return kref_get_unless_zero(&nport->ref);
> +	get_device(&nport->dev);
>   }
>   
>   static void
> @@ -1007,6 +1007,8 @@ fcloop_targetport_delete(struct nvmet_fc_target_port *targetport)
>   #define	FCLOOP_SGL_SEGS			256
>   #define FCLOOP_DMABOUND_4G		0xFFFFFFFF
>   
> +static struct class *fcloop_class;
> +
>   static struct nvme_fc_port_template fctemplate = {
>   	.localport_delete	= fcloop_localport_delete,
>   	.remoteport_delete	= fcloop_remoteport_delete,
> @@ -1225,8 +1227,10 @@ fcloop_alloc_nport(const char *buf, size_t count, bool remoteport)
>   	newnport->port_id = port_id;
>   	if (opts->mask & NVMF_OPT_ROLES)
>   		newnport->port_role = opts->roles;
> -
> -	kref_init(&newnport->ref);
> +	newnport->dev.class = fcloop_class;
> +	newnport->dev.release = fcloop_release_nport;
> +	dev_set_name(&newnport->dev, "nport%d", newnport->port_id);
> +	device_initialize(&newnport->dev);
>   
>   	spin_lock_irqsave(&fcloop_lock, flags);
>   
> @@ -1267,6 +1271,10 @@ fcloop_alloc_nport(const char *buf, size_t count, bool remoteport)
>   		}
>   	}
>   
> +	ret = device_add(&newnport->dev);
> +	if (ret)
> +		goto out_invalid_opts;
> +
>   	list_add_tail(&newnport->nport_list, &fcloop_nports);
>   
>   	spin_unlock_irqrestore(&fcloop_lock, flags);
> @@ -1339,6 +1347,8 @@ __unlink_remote_port(struct fcloop_nport *nport)
>   	if (rport && nport->tport)
>   		nport->tport->remoteport = NULL;
>   	nport->rport = NULL;
> +	if (!nport->tport)
> +		device_del(&nport->dev);
>   
>   	return rport;
>   }
> @@ -1406,7 +1416,7 @@ fcloop_create_target_port(struct device *dev, struct device_attribute *attr,
>   	tinfo.port_name = nport->port_name;
>   	tinfo.port_id = nport->port_id;
>   
> -	ret = nvmet_fc_register_targetport(&tinfo, &tgttemplate, NULL,
> +	ret = nvmet_fc_register_targetport(&tinfo, &tgttemplate, &nport->dev,
>   						&targetport);
>   	if (ret) {
>   		fcloop_nport_put(nport);
> @@ -1438,6 +1448,8 @@ __unlink_target_port(struct fcloop_nport *nport)
>   	if (tport && nport->rport)
>   		nport->rport->targetport = NULL;
>   	nport->tport = NULL;
> +	if (!nport->rport)
> +		device_del(&nport->dev);
>   
>   	return tport;
>   }
> @@ -1513,7 +1525,6 @@ static const struct attribute_group *fcloop_dev_attr_groups[] = {
>   	NULL,
>   };
>   
> -static struct class *fcloop_class;
>   static struct device *fcloop_device;
>   
>   

ok.

Reviewed-by: James Smart <james.smart at broadcom.com>

-- james
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201005/949abec5/attachment.p7s>

From james.smart at broadcom.com  Mon Oct  5 13:45:17 2020
From: james.smart at broadcom.com (James Smart)
Date: Mon, 5 Oct 2020 10:45:17 -0700
Subject: [PATCH 7/7] nvme-fcloop: use a device for lport
In-Reply-To: <20200922121501.32851-8-hare@suse.de>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-8-hare@suse.de>
Message-ID: <18cf0738-2643-d59a-d634-aebc85eb6ba5@broadcom.com>

On 9/22/2020 5:15 AM, Hannes Reinecke wrote:
> Add a 'struct device' to the lport such that the lport will show
> up in sysfs and we get a valid name in the logging output.
>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/target/fcloop.c | 53 ++++++++++++++++++++++++++++++++------------
>   1 file changed, 39 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
> index b84f8754b3a3..1af1917ddf4c 100644
> --- a/drivers/nvme/target/fcloop.c
> +++ b/drivers/nvme/target/fcloop.c
> @@ -214,6 +214,7 @@ static LIST_HEAD(fcloop_nports);
>   static DEFINE_IDA(fcloop_portid_ida);
>   
>   struct fcloop_lport {
> +	struct device dev;
>   	struct nvme_fc_local_port *localport;
>   	struct list_head lport_list;
>   	struct completion unreg_done;
> @@ -1053,6 +1054,16 @@ static struct nvmet_fc_target_template tgttemplate = {
>   	.lsrqst_priv_sz		= sizeof(struct fcloop_lsreq),
>   };
>   
> +static void fcloop_release_lport(struct device *dev)
> +{
> +	struct fcloop_lport *lport =
> +		container_of(dev, struct fcloop_lport, dev);
> +
> +	ida_free(&fcloop_portid_ida, lport->port_id);
> +
> +	kfree(lport);
> +}
> +
>   static ssize_t
>   fcloop_create_local_port(struct device *dev, struct device_attribute *attr,
>   		const char *buf, size_t count)
> @@ -1063,7 +1074,7 @@ fcloop_create_local_port(struct device *dev, struct device_attribute *attr,
>   	struct fcloop_lport *lport;
>   	struct fcloop_lport_priv *lport_priv;
>   	unsigned long flags;
> -	int ret = -ENOMEM, port_id;
> +	int ret = -ENOMEM, port_id, port_num;
>   
>   	lport = kzalloc(sizeof(*lport), GFP_KERNEL);
>   	if (!lport)
> @@ -1094,25 +1105,39 @@ fcloop_create_local_port(struct device *dev, struct device_attribute *attr,
>   		goto out_free_ida;
>   	}
>   
> +	lport->port_id = port_id;
> +	lport->dev.class = fcloop_class;
> +	lport->dev.release = fcloop_release_lport;
> +	dev_set_name(&lport->dev, "lport%d", lport->port_id);
> +	device_initialize(&lport->dev);
> +
>   	memset(&pinfo, 0, sizeof(pinfo));
>   	pinfo.node_name = opts->wwnn;
>   	pinfo.port_name = opts->wwpn;
>   	pinfo.port_role = opts->roles;
>   	pinfo.port_id = port_id;
>   
> -	ret = nvme_fc_register_localport(&pinfo, &fctemplate, NULL, &localport);
> -	if (!ret) {
> -		/* success */
> -		lport_priv = localport->private;
> -		lport_priv->lport = lport;
> +	ret = nvme_fc_register_localport(&pinfo, &fctemplate,
> +					 &lport->dev, &localport);
> +	if (ret)
> +		goto out_free_ida;
>   
> -		lport->localport = localport;
> -		INIT_LIST_HEAD(&lport->lport_list);
> +	/* success */
> +	lport_priv = localport->private;
> +	lport_priv->lport = lport;
>   
> -		spin_lock_irqsave(&fcloop_lock, flags);
> -		list_add_tail(&lport->lport_list, &fcloop_lports);
> -		spin_unlock_irqrestore(&fcloop_lock, flags);
> +	lport->localport = localport;
> +	INIT_LIST_HEAD(&lport->lport_list);
> +
> +	ret = device_add(&lport->dev);
> +	if (ret) {
> +		nvme_fc_unregister_localport(lport->localport);
> +		goto out_free_ida;
>   	}
> +	spin_lock_irqsave(&fcloop_lock, flags);
> +	list_add_tail(&lport->lport_list, &fcloop_lports);
> +	spin_unlock_irqrestore(&fcloop_lock, flags);
> +
>   out_free_ida:
>   	if (ret)
>   		ida_free(&fcloop_portid_ida, port_id);
> @@ -1138,15 +1163,15 @@ __wait_localport_unreg(struct fcloop_lport *lport)
>   {
>   	int ret;
>   
> +	device_del(&lport->dev);
> +
>   	init_completion(&lport->unreg_done);
>   
>   	ret = nvme_fc_unregister_localport(lport->localport);
>   
>   	wait_for_completion(&lport->unreg_done);
>   
> -	ida_free(&fcloop_portid_ida, lport->port_id);
> -
> -	kfree(lport);
> +	put_device(&lport->dev);
>   
>   	return ret;
>   }
ok.

Reviewed-by: James Smart <james.smart at broadcom.com>

-- james
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201005/4e2da7e8/attachment.p7s>

From lkp at intel.com  Mon Oct  5 21:03:06 2020
From: lkp at intel.com (kernel test robot)
Date: Tue, 06 Oct 2020 09:03:06 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 9ac37305a2e29615e695496feb73b4d7f935bcfc
Message-ID: <5f7bc24a.ls1qYLd6Ovik3ShA%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: 9ac37305a2e29615e695496feb73b4d7f935bcfc  nvme-core: remove extra condition for vwc

elapsed time: 725m

configs tested: 187
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
mips                  maltasmvp_eva_defconfig
m68k                        mvme16x_defconfig
sh                ecovec24-romimage_defconfig
ia64                        generic_defconfig
powerpc                      acadia_defconfig
mips                       bmips_be_defconfig
mips                malta_qemu_32r6_defconfig
powerpc                      katmai_defconfig
mips                          ath79_defconfig
powerpc                     skiroot_defconfig
powerpc                     ksi8560_defconfig
mips                           ip28_defconfig
sh                   sh7724_generic_defconfig
m68k                          atari_defconfig
arc                            hsdk_defconfig
mips                        qi_lb60_defconfig
mips                      maltaaprp_defconfig
arm                        shmobile_defconfig
powerpc                    socrates_defconfig
mips                        jmr3927_defconfig
arm                         vf610m4_defconfig
arm                        neponset_defconfig
mips                           ip22_defconfig
arm                         cm_x300_defconfig
powerpc                      pasemi_defconfig
mips                      bmips_stb_defconfig
powerpc                      cm5200_defconfig
powerpc                     stx_gp3_defconfig
sh                           se7721_defconfig
powerpc                     tqm8541_defconfig
arm                         axm55xx_defconfig
arm                          moxart_defconfig
mips                           xway_defconfig
sh                              ul2_defconfig
riscv                          rv32_defconfig
mips                         bigsur_defconfig
arm                           stm32_defconfig
arm                          iop32x_defconfig
arm                          tango4_defconfig
powerpc                mpc7448_hpc2_defconfig
mips                        nlm_xlp_defconfig
m68k                       m5475evb_defconfig
sh                         apsh4a3a_defconfig
mips                            ar7_defconfig
powerpc                        warp_defconfig
arm                        multi_v7_defconfig
sh                        edosk7705_defconfig
arm                        clps711x_defconfig
arm                            hisi_defconfig
i386                             alldefconfig
microblaze                          defconfig
powerpc                        icon_defconfig
m68k                          amiga_defconfig
arm                       imx_v6_v7_defconfig
powerpc                     tqm8555_defconfig
ia64                         bigsur_defconfig
nds32                               defconfig
ia64                             allyesconfig
m68k                         apollo_defconfig
arm                        spear6xx_defconfig
sparc                               defconfig
mips                          malta_defconfig
arm                          ep93xx_defconfig
sh                            titan_defconfig
arc                             nps_defconfig
sh                             shx3_defconfig
powerpc                      ppc44x_defconfig
arm                            zeus_defconfig
mips                         tb0219_defconfig
arm                        multi_v5_defconfig
xtensa                           alldefconfig
powerpc                         ps3_defconfig
powerpc                  mpc885_ads_defconfig
sh                          rsk7269_defconfig
m68k                            mac_defconfig
powerpc                     tqm8540_defconfig
h8300                            alldefconfig
arm                           corgi_defconfig
powerpc                      ep88xc_defconfig
arm                            pleb_defconfig
c6x                              alldefconfig
powerpc                      chrp32_defconfig
arm                         palmz72_defconfig
arm                          ixp4xx_defconfig
sh                           se7722_defconfig
sh                   secureedge5410_defconfig
m68k                        m5272c3_defconfig
powerpc                 mpc85xx_cds_defconfig
powerpc                      pcm030_defconfig
arm                        keystone_defconfig
arm                          exynos_defconfig
mips                      pistachio_defconfig
mips                       lemote2f_defconfig
arm                       mainstone_defconfig
arm                         bcm2835_defconfig
sh                          kfr2r09_defconfig
powerpc                 mpc8313_rdb_defconfig
sh                        sh7763rdp_defconfig
sh                   rts7751r2dplus_defconfig
sh                          urquell_defconfig
riscv                    nommu_virt_defconfig
sh                        apsh4ad0a_defconfig
mips                         mpc30x_defconfig
arm                        spear3xx_defconfig
mips                          ath25_defconfig
powerpc                 canyonlands_defconfig
arm                             pxa_defconfig
sh                         ap325rxa_defconfig
powerpc                     powernv_defconfig
parisc                              defconfig
sh                               allmodconfig
mips                     loongson1c_defconfig
sh                          rsk7264_defconfig
powerpc                          allmodconfig
arm                           viper_defconfig
ia64                             allmodconfig
ia64                                defconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                           allnoconfig
i386                 randconfig-a006-20201005
i386                 randconfig-a005-20201005
i386                 randconfig-a001-20201005
i386                 randconfig-a004-20201005
i386                 randconfig-a003-20201005
i386                 randconfig-a002-20201005
x86_64               randconfig-a012-20201005
x86_64               randconfig-a015-20201005
x86_64               randconfig-a014-20201005
x86_64               randconfig-a013-20201005
x86_64               randconfig-a011-20201005
x86_64               randconfig-a016-20201005
i386                 randconfig-a014-20201005
i386                 randconfig-a015-20201005
i386                 randconfig-a013-20201005
i386                 randconfig-a016-20201005
i386                 randconfig-a011-20201005
i386                 randconfig-a012-20201005
i386                 randconfig-a014-20201004
i386                 randconfig-a015-20201004
i386                 randconfig-a013-20201004
i386                 randconfig-a016-20201004
i386                 randconfig-a011-20201004
i386                 randconfig-a012-20201004
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a004-20201005
x86_64               randconfig-a002-20201005
x86_64               randconfig-a001-20201005
x86_64               randconfig-a003-20201005
x86_64               randconfig-a005-20201005
x86_64               randconfig-a006-20201005

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From Chaitanya.Kulkarni at wdc.com  Mon Oct  5 22:56:37 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 6 Oct 2020 02:56:37 +0000
Subject: [PATCH V2 1/2] nvme-loop: use xarray for loop ctrl tracking
References: <20200930045557.53669-1-chaitanya.kulkarni@wdc.com>
 <20200930045557.53669-2-chaitanya.kulkarni@wdc.com>
 <20201005124605.GA518@lst.de>
Message-ID: <BYAPR04MB4965356BA38313308C7A5D75860D0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/5/20 05:46, Christoph Hellwig wrote:
> On Tue, Sep 29, 2020 at 09:55:56PM -0700, Chaitanya Kulkarni wrote:
>> @@ -262,12 +260,10 @@ static void nvme_loop_free_ctrl(struct nvme_ctrl *nctrl)
>>  {
>>  	struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
>>  
>> -	if (list_empty(&ctrl->list))
>> +	if (!xa_load(&nvme_loop_ctrls, nctrl->cntlid))
>>  		goto free_ctrl;
>>  
>> -	mutex_lock(&nvme_loop_ctrl_mutex);
>> -	list_del(&ctrl->list);
>> -	mutex_unlock(&nvme_loop_ctrl_mutex);
>> +	xa_erase(&nvme_loop_ctrls, nctrl->cntlid);
> xa_erase is fine with an already deleted object, so the above xa_load
> check doesn't make any sense.
Okay, we can certainly get rid of the xa_load().
>> @@ -599,6 +590,12 @@ static struct nvme_ctrl *nvme_loop_create_ctrl(struct device *dev,
>>  	if (ret)
>>  		goto out_free_queues;
>>  
>> +	/* unusual place to update xarray, makes unwind code simple */
>> +	ret = xa_insert(&nvme_loop_ctrls, ctrl->ctrl.cntlid, &ctrl,
>> +			GFP_KERNEL);
>> +	if (ret)
>> +		goto out_remove_ctrl;
>> +
>>  out_remove_admin_queue:
>>  	nvme_loop_destroy_admin_queue(ctrl);
>> +out_remove_ctrl:
>> +	xa_erase(&nvme_loop_ctrls, ctrl->ctrl.cntlid);
> This looks weird.  Why would you remove the controller when inserting
> it fails?
This is wrong it should be "goto out_free_queues:".
> Also did you do an audit that none of the lookups will do something
> funny with the insered but not fully initialized controller?

Now looking at the code xa_insert() should just replace original

list_add_tail() and do the right error handling for destructing the io

queues if any. Will adjust this patch in the next version.

>> @@ -678,7 +673,7 @@ static struct nvmf_transport_ops nvme_loop_transport = {
>>  	.name		= "loop",
>>  	.module		= THIS_MODULE,
>>  	.create_ctrl	= nvme_loop_create_ctrl,
>> -	.allowed_opts	= NVMF_OPT_TRADDR,
>> +	.allowed_opts	= NVMF_OPT_TRADDR | NVMF_OPT_CTRL_LOSS_TMO,
> This looks unrelated?
Yes, will remove.
>> +	xa_init(&nvme_loop_ctrls);
> DEFINE_XARRAY seems like the btter choice for nvme_loop_ctrls.
>
True.
>>  	return ret;
>>  }
>>  
>>  static void __exit nvme_loop_cleanup_module(void)
>>  {
>> -	struct nvme_loop_ctrl *ctrl, *next;
>> +	struct nvme_loop_ctrl *ctrl;
>> +	unsigned long idx;
>>  
>>  	nvmf_unregister_transport(&nvme_loop_transport);
>>  	nvmet_unregister_transport(&nvme_loop_ops);
>>  
>> -	mutex_lock(&nvme_loop_ctrl_mutex);
>> -	list_for_each_entry_safe(ctrl, next, &nvme_loop_ctrl_list, list)
>> +	xa_for_each(&nvme_loop_ctrls, idx, ctrl)
>>  		nvme_delete_ctrl(&ctrl->ctrl);
>> -	mutex_unlock(&nvme_loop_ctrl_mutex);
>> +
>> +	xa_destroy(&nvme_loop_ctrls);

Hmm, let me see if we can fix this in the next version without adding more

complexity.




From leon at kernel.org  Tue Oct  6 00:58:00 2020
From: leon at kernel.org (Leon Romanovsky)
Date: Tue, 6 Oct 2020 07:58:00 +0300
Subject: [PATCH blk-next 1/2] blk-mq-rdma: Delete not-used multi-queue
 RDMA map queue code
In-Reply-To: <20201005083817.GA14908@lst.de>
References: <20200929091358.421086-1-leon@kernel.org>
 <20200929091358.421086-2-leon@kernel.org>
 <20200929102046.GA14445@lst.de> <20200929103549.GE3094@unreal>
 <879916e4-b572-16b9-7b92-94dba7e918a3@grimberg.me>
 <20201002064505.GA9593@lst.de>
 <14fab6a7-f7b5-2f9d-e01f-923b1c36816d@grimberg.me>
 <20201005083817.GA14908@lst.de>
Message-ID: <20201006045800.GE1874917@unreal>

On Mon, Oct 05, 2020 at 10:38:17AM +0200, Christoph Hellwig wrote:
> On Fri, Oct 02, 2020 at 01:20:35PM -0700, Sagi Grimberg wrote:
> >> Well, why would they change it?  The whole point of the infrastructure
> >> is that there is a single sane affinity setting for a given setup. Now
> >> that setting needed some refinement from the original series (e.g. the
> >> current series about only using housekeeping cpus if cpu isolation is
> >> in use).  But allowing random users to modify affinity is just a receipe
> >> for a trainwreck.
> >
> > Well allowing people to mangle irq affinity settings seem to be a hard
> > requirement from the discussions in the past.
> >
> >> So I think we need to bring this back ASAP, as doing affinity right
> >> out of the box is an absolute requirement for sane performance without
> >> all the benchmarketing deep magic.
> >
> > Well, it's hard to say that setting custom irq affinity settings is
> > deemed non-useful to anyone and hence should be prevented. I'd expect
> > that irq settings have a sane default that works and if someone wants to
> > change it, it can but there should be no guarantees on optimal
> > performance. But IIRC this had some dependencies on drivers and some
> > more infrastructure to handle dynamic changes...
>
> The problem is that people change random settings.  We need to generalize
> it into a sane API (e.g. the housekeeping CPUs thing which totally makes
> sense).

I don't see many people jump on the bandwagon, someone should do it, but
who will? I personally have no knowledge in that area to do anything
meaningful.

Thanks


From hare at suse.de  Tue Oct  6 01:48:40 2020
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 6 Oct 2020 07:48:40 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201005125201.GB1125@lst.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
Message-ID: <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>

On 10/5/20 2:52 PM, Christoph Hellwig wrote:
> On Mon, Oct 05, 2020 at 02:45:00PM +0200, Hannes Reinecke wrote:
>> Currently namespaces behave differently depending on the 'CMIC'
>> setting. If CMIC is zero, the device is removed once the last path
>> goes away. If CMIC has the multipath bit set, the device is retained
>> even if the last path is removed.
>> This is okay for fabrics, where one can do an explicit disconnect
>> to remove the device, but for nvme-pci this induces a regression
>> with PCI hotplug.
>> When the NVMe device is opened (eg by MD), the NVMe device is not
>> removed after a PCI hot-remove. Hence MD will not be notified about
>> the event, and will continue to consider this device as operational.
>> Consequently, upon PCI hot-add the device shows up as a new NVMe
>> device, and MD will fail to reattach the device.
>> So this patch adds NVME_NSHEAD_QUEUE_IF_NO_PATH flag to the nshead
>> to restore the original behaviour for non-fabrics NVMe devices.
>>
>> Signed-off-by: Hannes Reinecke <hare at suse.de>
>> ---
>>   drivers/nvme/host/core.c      | 10 +++++++++-
>>   drivers/nvme/host/multipath.c | 38 ++++++++++++++++++++++++++++++++++++++
>>   drivers/nvme/host/nvme.h      |  2 ++
>>   3 files changed, 49 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index 4459a40b057c..e21c32ea4b51 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -475,8 +475,11 @@ static void nvme_free_ns_head(struct kref *ref)
>>   		container_of(ref, struct nvme_ns_head, ref);
>>   
>>   #ifdef CONFIG_NVME_MULTIPATH
>> -	if (head->disk)
>> +	if (head->disk) {
>> +		if (test_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags))
>> +			nvme_mpath_remove_disk(head);
>>   		put_disk(head->disk);
>> +	}
>>   #endif
>>   	ida_simple_remove(&head->subsys->ns_ida, head->instance);
>>   	cleanup_srcu_struct(&head->srcu);
>> @@ -3357,6 +3360,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
>>   #ifdef CONFIG_NVME_MULTIPATH
>>   	&dev_attr_ana_grpid.attr,
>>   	&dev_attr_ana_state.attr,
>> +	&dev_attr_queue_if_no_path.attr,
>>   #endif
>>   	NULL,
>>   };
>> @@ -3387,6 +3391,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
>>   		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
>>   			return 0;
>>   	}
>> +	if (a == &dev_attr_queue_if_no_path.attr) {
>> +		if (dev_to_disk(dev)->fops == &nvme_fops)
>> +			return 0;
>> +	}
>>   #endif
>>   	return a->mode;
>>   }
>> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
>> index 55045291b4de..bbdad5917112 100644
>> --- a/drivers/nvme/host/multipath.c
>> +++ b/drivers/nvme/host/multipath.c
>> @@ -381,6 +381,9 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head)
>>   	/* set to a default value for 512 until disk is validated */
>>   	blk_queue_logical_block_size(q, 512);
>>   	blk_set_stacking_limits(&q->limits);
>> +	/* Enable queue_if_no_path semantics for fabrics */
>> +	if (ctrl->ops->flags & NVME_F_FABRICS)
>> +		set_bit(NVME_NSHEAD_QUEUE_IF_NO_PATH, &head->flags);
> 
> Well, that is blindly obvious from the code.  But why would we treat
> fabrics special?
> 
Well, because it's established behaviour of the current code.
Changing it now has the potential to break existing scenarios.

For PCI (ie non-fabrics) the current behaviour is arguably a corner case 
(as one needs to have a PCI-NVMe with CMIC bit set), but once you have 
it PCI hotplug is done for. So there we really want to change the 
behaviour to get the same user experience for all NVMe drives.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hch at lst.de  Tue Oct  6 03:51:53 2020
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 6 Oct 2020 09:51:53 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
Message-ID: <20201006075153.GA10041@lst.de>

On Tue, Oct 06, 2020 at 07:48:40AM +0200, Hannes Reinecke wrote:
> Well, because it's established behaviour of the current code.
> Changing it now has the potential to break existing scenarios.
>
> For PCI (ie non-fabrics) the current behaviour is arguably a corner case 
> (as one needs to have a PCI-NVMe with CMIC bit set), but once you have it 
> PCI hotplug is done for. So there we really want to change the behaviour to 
> get the same user experience for all NVMe drives.

Dual port PCI NVMe devices are pretty common in the enterprise space.
I really don't see any good reason to differ here.


From hare at suse.de  Tue Oct  6 04:07:49 2020
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 6 Oct 2020 10:07:49 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201006075153.GA10041@lst.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
 <20201006075153.GA10041@lst.de>
Message-ID: <b720433b-5849-2e7f-d359-53247976bad5@suse.de>

On 10/6/20 9:51 AM, Christoph Hellwig wrote:
> On Tue, Oct 06, 2020 at 07:48:40AM +0200, Hannes Reinecke wrote:
>> Well, because it's established behaviour of the current code.
>> Changing it now has the potential to break existing scenarios.
>>
>> For PCI (ie non-fabrics) the current behaviour is arguably a corner case
>> (as one needs to have a PCI-NVMe with CMIC bit set), but once you have it
>> PCI hotplug is done for. So there we really want to change the behaviour to
>> get the same user experience for all NVMe drives.
> 
> Dual port PCI NVMe devices are pretty common in the enterprise space.
> I really don't see any good reason to differ here.
> 
?
Differ from what?

Normal (non-dual ported) PCI-NVMe?
NVMe-oF?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		           Kernel Storage Architect
hare at suse.de			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), GF: Felix Imend?rffer


From hch at lst.de  Tue Oct  6 04:27:27 2020
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 6 Oct 2020 10:27:27 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <b720433b-5849-2e7f-d359-53247976bad5@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
 <20201006075153.GA10041@lst.de>
 <b720433b-5849-2e7f-d359-53247976bad5@suse.de>
Message-ID: <20201006082727.GC10243@lst.de>

On Tue, Oct 06, 2020 at 10:07:49AM +0200, Hannes Reinecke wrote:
> On 10/6/20 9:51 AM, Christoph Hellwig wrote:
>> On Tue, Oct 06, 2020 at 07:48:40AM +0200, Hannes Reinecke wrote:
>>> Well, because it's established behaviour of the current code.
>>> Changing it now has the potential to break existing scenarios.
>>>
>>> For PCI (ie non-fabrics) the current behaviour is arguably a corner case
>>> (as one needs to have a PCI-NVMe with CMIC bit set), but once you have it
>>> PCI hotplug is done for. So there we really want to change the behaviour to
>>> get the same user experience for all NVMe drives.
>>
>> Dual port PCI NVMe devices are pretty common in the enterprise space.
>> I really don't see any good reason to differ here.
>>
> ?
> Differ from what?
>
> Normal (non-dual ported) PCI-NVMe?
> NVMe-oF?

All multipath devices should behave the same.  No special casing for
PCIe, please.


From hare at suse.de  Tue Oct  6 04:29:49 2020
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 6 Oct 2020 10:29:49 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201006082727.GC10243@lst.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
 <20201006075153.GA10041@lst.de>
 <b720433b-5849-2e7f-d359-53247976bad5@suse.de>
 <20201006082727.GC10243@lst.de>
Message-ID: <a848297e-3243-6366-86ee-0800d51198e1@suse.de>

On 10/6/20 10:27 AM, Christoph Hellwig wrote:
> On Tue, Oct 06, 2020 at 10:07:49AM +0200, Hannes Reinecke wrote:
>> On 10/6/20 9:51 AM, Christoph Hellwig wrote:
>>> On Tue, Oct 06, 2020 at 07:48:40AM +0200, Hannes Reinecke wrote:
>>>> Well, because it's established behaviour of the current code.
>>>> Changing it now has the potential to break existing scenarios.
>>>>
>>>> For PCI (ie non-fabrics) the current behaviour is arguably a corner case
>>>> (as one needs to have a PCI-NVMe with CMIC bit set), but once you have it
>>>> PCI hotplug is done for. So there we really want to change the behaviour to
>>>> get the same user experience for all NVMe drives.
>>>
>>> Dual port PCI NVMe devices are pretty common in the enterprise space.
>>> I really don't see any good reason to differ here.
>>>
>> ?
>> Differ from what?
>>
>> Normal (non-dual ported) PCI-NVMe?
>> NVMe-oF?
> 
> All multipath devices should behave the same.  No special casing for
> PCIe, please.
> 
Even if the default behaviour breaks PCI hotplug?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		           Kernel Storage Architect
hare at suse.de			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), GF: Felix Imend?rffer


From hch at lst.de  Tue Oct  6 04:39:26 2020
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 6 Oct 2020 10:39:26 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <a848297e-3243-6366-86ee-0800d51198e1@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
 <20201006075153.GA10041@lst.de>
 <b720433b-5849-2e7f-d359-53247976bad5@suse.de>
 <20201006082727.GC10243@lst.de>
 <a848297e-3243-6366-86ee-0800d51198e1@suse.de>
Message-ID: <20201006083926.GA13568@lst.de>

On Tue, Oct 06, 2020 at 10:29:49AM +0200, Hannes Reinecke wrote:
>> All multipath devices should behave the same.  No special casing for
>> PCIe, please.
>>
> Even if the default behaviour breaks PCI hotplug?

Why would it "break" PCI hotplug?


From hare at suse.de  Tue Oct  6 09:30:14 2020
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 6 Oct 2020 15:30:14 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <20201006083926.GA13568@lst.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
 <20201006075153.GA10041@lst.de>
 <b720433b-5849-2e7f-d359-53247976bad5@suse.de>
 <20201006082727.GC10243@lst.de>
 <a848297e-3243-6366-86ee-0800d51198e1@suse.de>
 <20201006083926.GA13568@lst.de>
Message-ID: <00e75643-d422-ca12-1648-02ca89044182@suse.de>

On 10/6/20 10:39 AM, Christoph Hellwig wrote:
> On Tue, Oct 06, 2020 at 10:29:49AM +0200, Hannes Reinecke wrote:
>>> All multipath devices should behave the same.  No special casing for
>>> PCIe, please.
>>>
>> Even if the default behaviour breaks PCI hotplug?
> 
> Why would it "break" PCI hotplug?
> 
When running under MD RAID:
Before hotplug:
# nvme list
Node             SN                   Model 
       Namespace Usage                      Format           FW Rev
---------------- -------------------- 
---------------------------------------- --------- 
-------------------------- ---------------- --------
/dev/nvme0n1     SLESNVME1            QEMU NVMe Ctrl 
       1          17.18  GB /  17.18  GB    512   B +  0 B   1.0
/dev/nvme1n1     SLESNVME2            QEMU NVMe Ctrl 
       1           4.29  GB /   4.29  GB    512   B +  0 B   1.0
/dev/nvme2n1     SLESNVME3            QEMU NVMe Ctrl 
       1           4.29  GB /   4.29  GB    512   B +  0 B   1.0
After hotplug:

# nvme list
Node             SN                   Model 
       Namespace Usage                      Format           FW Rev
---------------- -------------------- 
---------------------------------------- --------- 
-------------------------- ---------------- --------
/dev/nvme0n1     SLESNVME1            QEMU NVMe Ctrl 
       1          17.18  GB /  17.18  GB    512   B +  0 B   1.0
/dev/nvme1n1     SLESNVME2            QEMU NVMe Ctrl 
       -1          0.00   B /   0.00   B      1   B +  0 B   1.0
/dev/nvme1n2     SLESNVME2            QEMU NVMe Ctrl 
       1           4.29  GB /   4.29  GB    512   B +  0 B   1.0
/dev/nvme2n1     SLESNVME3            QEMU NVMe Ctrl 
       1           4.29  GB /   4.29  GB    512   B +  0 B   1.0

And MD hasn't been notified that the device is gone:
# cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 nvme2n1[1] nvme1n1[0]
       4189184 blocks super 1.2 2 near-copies [2/2] [UU]
       bitmap: 0/1 pages [0KB], 65536KB chunk

unused devices: <none>

Once I do some I/O to it MD recognized a faulty device:

# cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 nvme2n1[1] nvme1n1[0](F)
       4189184 blocks super 1.2 2 near-copies [2/1] [_U]
       bitmap: 0/1 pages [0KB], 65536KB chunk

unused devices: <none>

but the re-added device isn't added to the MD RAID.
In fact, it has been assigned a _different_ namespace ID:

[  904.299065] pcieport 0000:00:08.0: pciehp: Slot(0-1): Card present
[  904.299067] pcieport 0000:00:08.0: pciehp: Slot(0-1): Link Up
[  904.435314] pci 0000:02:00.0: [8086:5845] type 00 class 0x010802
[  904.435523] pci 0000:02:00.0: reg 0x10: [mem 0x00000000-0x00001fff 64bit]
[  904.435676] pci 0000:02:00.0: reg 0x20: [mem 0x00000000-0x00000fff]
[  904.436982] pci 0000:02:00.0: BAR 0: assigned [mem 
0xc1200000-0xc1201fff 64bit]
[  904.437086] pci 0000:02:00.0: BAR 4: assigned [mem 0xc1202000-0xc1202fff]
[  904.437118] pcieport 0000:00:08.0: PCI bridge to [bus 02]
[  904.437137] pcieport 0000:00:08.0:   bridge window [io  0x7000-0x7fff]
[  904.439024] pcieport 0000:00:08.0:   bridge window [mem 
0xc1200000-0xc13fffff]
[  904.440229] pcieport 0000:00:08.0:   bridge window [mem 
0x802000000-0x803ffffff 64bit pref]
[  904.447150] nvme nvme3: pci function 0000:02:00.0
[  904.447487] nvme 0000:02:00.0: enabling device (0000 -> 0002)
[  904.458880] nvme nvme3: 1/0/0 default/read/poll queues
[  904.461296] nvme1n2: detected capacity change from 0 to 4294967296

and the 'old', pre-hotplug device still lingers on in the 'nvme list' 
output.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Tue Oct  6 09:45:01 2020
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 6 Oct 2020 15:45:01 +0200
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <00e75643-d422-ca12-1648-02ca89044182@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
 <20201006075153.GA10041@lst.de>
 <b720433b-5849-2e7f-d359-53247976bad5@suse.de>
 <20201006082727.GC10243@lst.de>
 <a848297e-3243-6366-86ee-0800d51198e1@suse.de>
 <20201006083926.GA13568@lst.de>
 <00e75643-d422-ca12-1648-02ca89044182@suse.de>
Message-ID: <ce2c93e1-ba38-cebb-33b3-d506116a61aa@suse.de>

On 10/6/20 3:30 PM, Hannes Reinecke wrote:
> On 10/6/20 10:39 AM, Christoph Hellwig wrote:
>> On Tue, Oct 06, 2020 at 10:29:49AM +0200, Hannes Reinecke wrote:
>>>> All multipath devices should behave the same.? No special casing for
>>>> PCIe, please.
>>>>
>>> Even if the default behaviour breaks PCI hotplug?
>>
>> Why would it "break" PCI hotplug?
>>
> When running under MD RAID:
> Before hotplug:
> # nvme list
> Node???????????? SN?????????????????? Model ????? Namespace 
> Usage????????????????????? Format?????????? FW Rev
> ---------------- -------------------- 
> ---------------------------------------- --------- 
> -------------------------- ---------------- --------
> /dev/nvme0n1???? SLESNVME1??????????? QEMU NVMe Ctrl ????? 1          
> 17.18? GB /? 17.18? GB??? 512?? B +? 0 B?? 1.0
> /dev/nvme1n1???? SLESNVME2??????????? QEMU NVMe Ctrl ????? 1           
> 4.29? GB /?? 4.29? GB??? 512?? B +? 0 B?? 1.0
> /dev/nvme2n1???? SLESNVME3??????????? QEMU NVMe Ctrl ????? 1           
> 4.29? GB /?? 4.29? GB??? 512?? B +? 0 B?? 1.0
> After hotplug:
> 
> # nvme list
> Node???????????? SN?????????????????? Model ????? Namespace 
> Usage????????????????????? Format?????????? FW Rev
> ---------------- -------------------- 
> ---------------------------------------- --------- 
> -------------------------- ---------------- --------
> /dev/nvme0n1???? SLESNVME1??????????? QEMU NVMe Ctrl ????? 1          
> 17.18? GB /? 17.18? GB??? 512?? B +? 0 B?? 1.0
> /dev/nvme1n1???? SLESNVME2??????????? QEMU NVMe Ctrl ????? -1          
> 0.00?? B /?? 0.00?? B????? 1?? B +? 0 B?? 1.0
> /dev/nvme1n2???? SLESNVME2??????????? QEMU NVMe Ctrl ????? 1           
> 4.29? GB /?? 4.29? GB??? 512?? B +? 0 B?? 1.0
> /dev/nvme2n1???? SLESNVME3??????????? QEMU NVMe Ctrl ????? 1           
> 4.29? GB /?? 4.29? GB??? 512?? B +? 0 B?? 1.0
> 
> And MD hasn't been notified that the device is gone:
> # cat /proc/mdstat
> Personalities : [raid10]
> md0 : active raid10 nvme2n1[1] nvme1n1[0]
>  ????? 4189184 blocks super 1.2 2 near-copies [2/2] [UU]
>  ????? bitmap: 0/1 pages [0KB], 65536KB chunk
> 
> unused devices: <none>
> 
> Once I do some I/O to it MD recognized a faulty device:
> 
> # cat /proc/mdstat
> Personalities : [raid10]
> md0 : active raid10 nvme2n1[1] nvme1n1[0](F)
>  ????? 4189184 blocks super 1.2 2 near-copies [2/1] [_U]
>  ????? bitmap: 0/1 pages [0KB], 65536KB chunk
> 
> unused devices: <none>
> 
> but the re-added device isn't added to the MD RAID.
> In fact, it has been assigned a _different_ namespace ID:
> 
> [? 904.299065] pcieport 0000:00:08.0: pciehp: Slot(0-1): Card present
> [? 904.299067] pcieport 0000:00:08.0: pciehp: Slot(0-1): Link Up
> [? 904.435314] pci 0000:02:00.0: [8086:5845] type 00 class 0x010802
> [? 904.435523] pci 0000:02:00.0: reg 0x10: [mem 0x00000000-0x00001fff 
> 64bit]
> [? 904.435676] pci 0000:02:00.0: reg 0x20: [mem 0x00000000-0x00000fff]
> [? 904.436982] pci 0000:02:00.0: BAR 0: assigned [mem 
> 0xc1200000-0xc1201fff 64bit]
> [? 904.437086] pci 0000:02:00.0: BAR 4: assigned [mem 
> 0xc1202000-0xc1202fff]
> [? 904.437118] pcieport 0000:00:08.0: PCI bridge to [bus 02]
> [? 904.437137] pcieport 0000:00:08.0:?? bridge window [io? 0x7000-0x7fff]
> [? 904.439024] pcieport 0000:00:08.0:?? bridge window [mem 
> 0xc1200000-0xc13fffff]
> [? 904.440229] pcieport 0000:00:08.0:?? bridge window [mem 
> 0x802000000-0x803ffffff 64bit pref]
> [? 904.447150] nvme nvme3: pci function 0000:02:00.0
> [? 904.447487] nvme 0000:02:00.0: enabling device (0000 -> 0002)
> [? 904.458880] nvme nvme3: 1/0/0 default/read/poll queues
> [? 904.461296] nvme1n2: detected capacity change from 0 to 4294967296
> 
> and the 'old', pre-hotplug device still lingers on in the 'nvme list' 
> output.
> 
Compare that to the 'standard', non-CMIC nvme, where with the same setup 
MD would detach the nvme on its own:

# cat /proc/mdstat
Personalities : [raid10]
md127 : active (auto-read-only) raid10 nvme2n1[1]
       4189184 blocks super 1.2 2 near-copies [2/1] [_U]
       bitmap: 0/1 pages [0KB], 65536KB chunk

unused devices: <none>
# nvme list
Node             SN                   Model 
       Namespace Usage                      Format           FW Rev
---------------- -------------------- 
---------------------------------------- --------- 
-------------------------- ---------------- --------
/dev/nvme0n1     SLESNVME1            QEMU NVMe Ctrl 
       1          17.18  GB /  17.18  GB    512   B +  0 B   1.0
/dev/nvme2n1     SLESNVME3            QEMU NVMe Ctrl 
       1           4.29  GB /   4.29  GB    512   B +  0 B   1.0

And yes, this is exactly the same setup, the only difference being the 
CMIC setting for the NVMe device.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From kbusch at kernel.org  Tue Oct  6 13:41:00 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 6 Oct 2020 10:41:00 -0700
Subject: [PATCH 2/2] nvme: add 'queue_if_no_path' semantics
In-Reply-To: <00e75643-d422-ca12-1648-02ca89044182@suse.de>
References: <20201005124500.6015-1-hare@suse.de>
 <20201005124500.6015-3-hare@suse.de> <20201005125201.GB1125@lst.de>
 <8d7d4803-5808-0839-ee4f-e36a12756497@suse.de>
 <20201006075153.GA10041@lst.de>
 <b720433b-5849-2e7f-d359-53247976bad5@suse.de>
 <20201006082727.GC10243@lst.de>
 <a848297e-3243-6366-86ee-0800d51198e1@suse.de>
 <20201006083926.GA13568@lst.de>
 <00e75643-d422-ca12-1648-02ca89044182@suse.de>
Message-ID: <20201006174100.GA959043@dhcp-10-100-145-180.wdl.wdc.com>

On Tue, Oct 06, 2020 at 03:30:14PM +0200, Hannes Reinecke wrote:
> but the re-added device isn't added to the MD RAID.
> In fact, it has been assigned a _different_ namespace ID:

That's actually just a driver assigned instance rather than a Namespace
ID. 
 
> [  904.299065] pcieport 0000:00:08.0: pciehp: Slot(0-1): Card present
> [  904.299067] pcieport 0000:00:08.0: pciehp: Slot(0-1): Link Up
> [  904.435314] pci 0000:02:00.0: [8086:5845] type 00 class 0x010802
> [  904.435523] pci 0000:02:00.0: reg 0x10: [mem 0x00000000-0x00001fff 64bit]
> [  904.435676] pci 0000:02:00.0: reg 0x20: [mem 0x00000000-0x00000fff]
> [  904.436982] pci 0000:02:00.0: BAR 0: assigned [mem 0xc1200000-0xc1201fff 64bit]
> [  904.437086] pci 0000:02:00.0: BAR 4: assigned [mem 0xc1202000-0xc1202fff]
> [  904.437118] pcieport 0000:00:08.0: PCI bridge to [bus 02]
> [  904.437137] pcieport 0000:00:08.0:   bridge window [io  0x7000-0x7fff]
> [  904.439024] pcieport 0000:00:08.0:   bridge window [mem 0xc1200000-0xc13fffff]
> [  904.440229] pcieport 0000:00:08.0:   bridge window [mem 0x802000000-0x803ffffff 64bit pref]
> [  904.447150] nvme nvme3: pci function 0000:02:00.0
> [  904.447487] nvme 0000:02:00.0: enabling device (0000 -> 0002)
> [  904.458880] nvme nvme3: 1/0/0 default/read/poll queues
> [  904.461296] nvme1n2: detected capacity change from 0 to 4294967296
> 
> and the 'old', pre-hotplug device still lingers on in the 'nvme list'
> output.

I think what you're showing is a problem for everyone, including
fabrics. The namespace was attached to the previously existing
subsystem, nvme1, but we didn't find the previously existing "head" to
reattach the namespace.

I just threw the below together very hastily, so I'm sure there's
something wrong with it, but just to capture what's wrong with the
current code:

---
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 3833584b5cee..0b3bbbe6d1e8 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3897,8 +3897,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 
 	mutex_lock(&ns->ctrl->subsys->lock);
 	list_del_rcu(&ns->siblings);
-	if (list_empty(&ns->head->list))
-		list_del_init(&ns->head->entry);
 	mutex_unlock(&ns->ctrl->subsys->lock);
 
 	synchronize_rcu(); /* guarantee not available in head->list */
--


From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:42 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:42 -0700
Subject: [PATCH V4 0/7] nvme-core: timeout related fixes and cleanup
Message-ID: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>

Hi,

This patch series uses NVME_IO_TIMEOUT for the sync request submission
and user request submission when the timeout is not specified by the 
caller and request queue data is set to NULL which is true for admin 
queue.

Also in this version, I've added timeout values setting for the NVMeOF
passthru controller given that passthru VUCs (both admin and I/O VUCs)
can differ in execution time than general-purpose admin and I/O command
set following is the detailed scenario :-

Consider NVMeOF target is setup in the passthru mode and both target and 
host modules are loaded with the default timeout values present
in the nvme-core.

1. User issues nvme cmd from the host to the target of nvme-cli where
?? ??timeout is taken from the controller log page ??e.g. :-
?? ??# nvme io-passthru --timeout=XXX ... 
?? ??OR ??
?? ??# nvme admin-passthru --timeout=XXX ... 
?? ??The timeout value in the above example is greater than the default
?? ??timeout value present in host-core for both on host machine and the 
?? ??target machine.
2. The host-core on the host side will set the timeout for the host
?? ??request and issues NVMe cmd over transport.
3. In the absence of [1] target will set the timeout value which is
?? ??default timeout and that is smaller than the timeout set on the host
?? ??side.
4. Due to lack of timeout value not passed over the transport leading
?? ??to smaller timeout value for the request on the target than the host
?? ??side, target side command will timeout.
?? ??
?? ??This breaks the scenario where the same command with timeout value
   from nvme-cli is able to execute with success when issued to NVMe
   PCIe ctrl but will fail when it is executed on the NVMeOF Passthru
   ctrl.

Regards,
Chaitanya

[1] nvmet: add passthru admin timeout value attr
    nvmet: add passthru io timeout value attr

* Changes from V3:-

1. Rebase and retest on nvme-5.10.
2. Update the cover-letter with detailed scenario why NVMeOF target
   passthru is needed.
3. Wrap the line under 80 char for the last patch.
4. Change the nvme_default_timeout() -> nvme_set_req_default_timeout()
   such that it will not take timeout argument.

* Changes from V2:-

1. Introduce nvme_defalt_timeout() helper and use it in host/core.c.
2. Use nvme_default_timeout() in the lightnvme.c

* Changes from V1:-

1. Instead of using qid to decide IO or ADMIN timeout use request
   queue's queuedata whch we only set for non admin queue
   __nvme_submit_sync_cmd().
2. Add second patch to set IO timeout for nvme_submit_user_cmd().
3. Set the NVMeOF passthru ctrl timeout values with default values from
   nvme-core module.
4. Add admin and I/O timeout configfs attributes for NVMeOF passthru
   controller.

Chaitanya Kulkarni (7):
  nvme-core: use I/O timeout in submit sync cmd
  nvme-core: use I/O timeout in nvme_submit_user_cmd
  lightnvm: use I/O timeout in nvm submit user cmd
  nvmet: set default timeout for passthru requests
  nvmet: add passthru admin timeout value attr
  nvmet: add passthru io timeout value attr
  nvme: use consistent macro name for timeout

 drivers/nvme/host/core.c       | 10 ++++--
 drivers/nvme/host/fc.c         |  2 +-
 drivers/nvme/host/lightnvm.c   |  3 +-
 drivers/nvme/host/nvme.h       | 10 +++++-
 drivers/nvme/host/pci.c        |  8 ++---
 drivers/nvme/host/rdma.c       |  2 +-
 drivers/nvme/host/tcp.c        |  2 +-
 drivers/nvme/target/configfs.c | 64 ++++++++++++++++++++++++++++++++++
 drivers/nvme/target/loop.c     |  2 +-
 drivers/nvme/target/nvmet.h    |  2 ++
 drivers/nvme/target/passthru.c | 11 ++++++
 11 files changed, 103 insertions(+), 13 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:43 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:43 -0700
Subject: [PATCH V4 1/7] nvme-core: use I/O timeout in submit sync cmd
In-Reply-To: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
References: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201006233049.55458-2-chaitanya.kulkarni@wdc.com>

In the function __nvme_submit_sync_cmd() it uses ADMIN_TIMEOUT when
caller doesn't specify value for the timeout variable. This function is
also called from the NVMe commands contexts (nvme_pr_command()/
nvme_ns_report_zones()) where NVME_IO_TIMEOUT can be used instead of
ADMIN_TIMEOUT.

For now we don't set the request queue's queuedata for admin command.

Introduce a helper nvme_default_timeout() and when timeout is not set
for the block layer request  based on the request queue's queuedata,
set Admin timeout else I/O timeout.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 3 ++-
 drivers/nvme/host/nvme.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c20ce8eabe82..49662e2190d5 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -897,7 +897,8 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
-	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	if (!req->timeout)
+		nvme_req_set_default_timeout(req);
 
 	if (buffer && bufflen) {
 		ret = blk_rq_map_kern(q, req, buffer, bufflen, GFP_KERNEL);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index e7c88b40f5bb..3106d6133c8c 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -638,6 +638,14 @@ struct nvme_ns *nvme_get_ns_from_disk(struct gendisk *disk,
 		struct nvme_ns_head **head, int *srcu_idx);
 void nvme_put_ns_from_disk(struct nvme_ns_head *head, int idx);
 
+static inline void nvme_req_set_default_timeout(struct request *req)
+{
+	if (req->q->queuedata)
+		req->timeout = NVME_IO_TIMEOUT;
+	else /* no queuedata implies admin queue */
+		req->timeout = ADMIN_TIMEOUT;
+}
+
 extern const struct attribute_group *nvme_ns_id_attr_groups[];
 extern const struct block_device_operations nvme_ns_head_ops;
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:44 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:44 -0700
Subject: [PATCH V4 2/7] nvme-core: use I/O timeout in nvme_submit_user_cmd
In-Reply-To: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
References: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201006233049.55458-3-chaitanya.kulkarni@wdc.com>

In the function nvme_submit_user_cmd() it uses ADMIN_TIMEOUT when
caller doesn't specify value for the timeout variable. This function is
also called from the user I/O command contexts (nvme_submit_io,
nvme_user_cmd_XXX) where NVME_IO_TIMEOUT can be used instead of
ADMIN_TIMEOUT.

For now we don't set the request queue's queuedata for admin command.

Use newly introduced helper function nvme_default_timeout() to set the
block layer request timeout value.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 49662e2190d5..f97003e54db8 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1068,7 +1068,9 @@ static int nvme_submit_user_cmd(struct request_queue *q,
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
-	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	if (!req->timeout)
+		nvme_req_set_default_timeout(req);
+
 	nvme_req(req)->flags |= NVME_REQ_USERCMD;
 
 	if (ubuffer && bufflen) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:45 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:45 -0700
Subject: [PATCH V4 3/7] lightnvm: use I/O timeout in nvm submit user cmd
In-Reply-To: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
References: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201006233049.55458-4-chaitanya.kulkarni@wdc.com>

In the function nvme_nvm_submit_user_cmd() it uses ADMIN_TIMEOUT when
caller doesn't specify value for the timeout variable.

For now we don't set the request queue's queuedata for admin command.

Use newly introduced helper function nvme_default_timeout() to set the
block layer request timeout value which sets the appropriate I/O or
Admin timeout values when caller doesn't specify the timeout value.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/lightnvm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/lightnvm.c b/drivers/nvme/host/lightnvm.c
index 8e562d0f2c30..784be80ee2f9 100644
--- a/drivers/nvme/host/lightnvm.c
+++ b/drivers/nvme/host/lightnvm.c
@@ -774,7 +774,8 @@ static int nvme_nvm_submit_user_cmd(struct request_queue *q,
 		goto err_cmd;
 	}
 
-	rq->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	if (!rq->timeout)
+		nvme_req_set_default_timeout(rq);
 
 	if (ppa_buf && ppa_len) {
 		ppa_list = dma_pool_alloc(dev->dma_pool, GFP_KERNEL, &ppa_dma);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:46 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:46 -0700
Subject: [PATCH V4 4/7] nvmet: set default timeout for passthru requests
In-Reply-To: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
References: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201006233049.55458-5-chaitanya.kulkarni@wdc.com>

In nvmet_passthru_execute_cmd() we don't set the default timeout values
for passthru requests. Use the default values ro set the passthru
request timeout.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 56c571052216..9f9ab5665dd2 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -242,6 +242,8 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		goto out_put_ns;
 	}
 
+	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : ADMIN_TIMEOUT;
+
 	if (req->sg_cnt) {
 		ret = nvmet_passthru_map_sg(req, rq);
 		if (unlikely(ret)) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:47 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:47 -0700
Subject: [PATCH V4 5/7] nvmet: add passthru admin timeout value attr
In-Reply-To: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
References: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201006233049.55458-6-chaitanya.kulkarni@wdc.com>

NVMeOF controller in the passsthru mode is capable of handling wide set
of admin commands including Vender unique passhtru admin comands (VUAC).

The VUACs are used to read the large driver logs and can take longer
than default NVMe commands, that is for passthru requests the timeout
value may differ from the passthru controller's default timeout values
(nvme-core:admin_timeout).

Add a configfs attribute so that user can set the admin timeout values.
In case if this configfs value is not set default to ADMIN_TIMEOUT
value.

This attribute setting is only allowed when ctrl is disable to avoid
rcu calls in the fast path, in future when needed we can always make it
fast path friendly using RCU.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/configfs.c | 32 ++++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c |  7 ++++++-
 3 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 37e1d7784e17..7625f8892fb5 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -736,9 +736,41 @@ static ssize_t nvmet_passthru_enable_store(struct config_item *item,
 }
 CONFIGFS_ATTR(nvmet_passthru_, enable);
 
+static ssize_t nvmet_passthru_admin_timeout_show(struct config_item *item,
+		char *page)
+{
+	return sprintf(page, "%u\n", to_subsys(item->ci_parent)->admin_timeout);
+}
+
+static ssize_t nvmet_passthru_admin_timeout_store(struct config_item *item,
+		const char *page, size_t count)
+{
+	struct nvmet_subsys *subsys = to_subsys(item->ci_parent);
+	unsigned int admin_timeout;
+	int ret = 0;
+
+	mutex_lock(&subsys->lock);
+	if (subsys->passthru_ctrl) {
+		pr_err("disable passthru ctrl before setting admin_timeout\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (kstrtouint(page, 0, &admin_timeout)) {
+		ret = -EINVAL;
+		goto out;
+	}
+	subsys->admin_timeout = admin_timeout;
+out:
+	mutex_unlock(&subsys->lock);
+	return ret ? ret : count;
+}
+CONFIGFS_ATTR(nvmet_passthru_, admin_timeout);
+
 static struct configfs_attribute *nvmet_passthru_attrs[] = {
 	&nvmet_passthru_attr_device_path,
 	&nvmet_passthru_attr_enable,
+	&nvmet_passthru_attr_admin_timeout,
 	NULL,
 };
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 559a15ccc322..a0c80e5179a2 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -249,6 +249,7 @@ struct nvmet_subsys {
 	struct nvme_ctrl	*passthru_ctrl;
 	char			*passthru_ctrl_path;
 	struct config_group	passthru_group;
+	unsigned int		admin_timeout;
 #endif /* CONFIG_NVME_TARGET_PASSTHRU */
 };
 
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 9f9ab5665dd2..253ca2b6ed10 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -215,6 +215,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 
 static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 {
+	unsigned int admin_timeout = req->sq->ctrl->subsys->admin_timeout;
 	struct nvme_ctrl *ctrl = nvmet_req_passthru_ctrl(req);
 	struct request_queue *q = ctrl->admin_q;
 	struct nvme_ns *ns = NULL;
@@ -242,7 +243,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		goto out_put_ns;
 	}
 
-	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : ADMIN_TIMEOUT;
+	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : admin_timeout;
 
 	if (req->sg_cnt) {
 		ret = nvmet_passthru_map_sg(req, rq);
@@ -540,6 +541,10 @@ int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys)
 			NVME_TERTIARY(subsys->ver));
 		subsys->ver = NVME_VS(1, 2, 1);
 	}
+
+	if (!subsys->admin_timeout)
+		subsys->admin_timeout = ADMIN_TIMEOUT;
+
 	nvme_get_ctrl(ctrl);
 	__module_get(subsys->passthru_ctrl->ops->module);
 	ret = 0;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:48 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:48 -0700
Subject: [PATCH V4 6/7] nvmet: add passthru io timeout value attr
In-Reply-To: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
References: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201006233049.55458-7-chaitanya.kulkarni@wdc.com>

NVMe controller in the passsthru mode is capable of handling wide set
of admin commands including Vender unique passhtru I/O comands
(VUICs).

The VUICs can take longer than default NVMe commands, that is for
passthru requests the timeout value may differ from the passthru
controller's default timeout values (nvme-core:io_timeout).

Add configfs attribute so that user can set the I/O timeout values.
In case if this configfs value is not set default to NVME_IO_TIMEOUT
value.

This attribute setting is only allowed when ctrl is disable to avoid
rcu calls in the fast path, in future when needed we can always make it
fast path friendly using RCU.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/configfs.c | 32 ++++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c |  6 +++++-
 3 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 7625f8892fb5..5beae49ba6ad 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -767,10 +767,42 @@ static ssize_t nvmet_passthru_admin_timeout_store(struct config_item *item,
 }
 CONFIGFS_ATTR(nvmet_passthru_, admin_timeout);
 
+static ssize_t nvmet_passthru_io_timeout_show(struct config_item *item,
+		char *page)
+{
+	return sprintf(page, "%u\n", to_subsys(item->ci_parent)->io_timeout);
+}
+
+static ssize_t nvmet_passthru_io_timeout_store(struct config_item *item,
+		const char *page, size_t count)
+{
+	struct nvmet_subsys *subsys = to_subsys(item->ci_parent);
+	unsigned int io_timeout;
+	int ret = 0;
+
+	mutex_lock(&subsys->lock);
+	if (subsys->passthru_ctrl) {
+		pr_err("disable passthru ctrl before setting io_timeout\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (kstrtouint(page, 0, &io_timeout)) {
+		ret = -EINVAL;
+		goto out;
+	}
+	subsys->io_timeout = io_timeout;
+out:
+	mutex_unlock(&subsys->lock);
+	return ret ? ret : count;
+}
+CONFIGFS_ATTR(nvmet_passthru_, io_timeout);
+
 static struct configfs_attribute *nvmet_passthru_attrs[] = {
 	&nvmet_passthru_attr_device_path,
 	&nvmet_passthru_attr_enable,
 	&nvmet_passthru_attr_admin_timeout,
+	&nvmet_passthru_attr_io_timeout,
 	NULL,
 };
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index a0c80e5179a2..2f9635273629 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -250,6 +250,7 @@ struct nvmet_subsys {
 	char			*passthru_ctrl_path;
 	struct config_group	passthru_group;
 	unsigned int		admin_timeout;
+	unsigned int		io_timeout;
 #endif /* CONFIG_NVME_TARGET_PASSTHRU */
 };
 
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 253ca2b6ed10..2039ad7cbbab 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -216,6 +216,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 {
 	unsigned int admin_timeout = req->sq->ctrl->subsys->admin_timeout;
+	unsigned int io_timeout = req->sq->ctrl->subsys->io_timeout;
 	struct nvme_ctrl *ctrl = nvmet_req_passthru_ctrl(req);
 	struct request_queue *q = ctrl->admin_q;
 	struct nvme_ns *ns = NULL;
@@ -243,7 +244,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		goto out_put_ns;
 	}
 
-	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : admin_timeout;
+	rq->timeout = q->queuedata ? io_timeout : admin_timeout;
 
 	if (req->sg_cnt) {
 		ret = nvmet_passthru_map_sg(req, rq);
@@ -545,6 +546,9 @@ int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys)
 	if (!subsys->admin_timeout)
 		subsys->admin_timeout = ADMIN_TIMEOUT;
 
+	if (!subsys->io_timeout)
+		subsys->io_timeout = NVME_IO_TIMEOUT;
+
 	nvme_get_ctrl(ctrl);
 	__module_get(subsys->passthru_ctrl->ops->module);
 	ret = 0;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:30:49 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:30:49 -0700
Subject: [PATCH V4 7/7] nvme: use consistent macro name for timeout
In-Reply-To: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
References: <20201006233049.55458-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201006233049.55458-8-chaitanya.kulkarni@wdc.com>

This is purely a clenaup patch, add prefix NVME to the ADMIN_TIMEOUT
to make consistent with NVME_IO_TIMEOUT.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c       | 3 ++-
 drivers/nvme/host/fc.c         | 2 +-
 drivers/nvme/host/nvme.h       | 4 ++--
 drivers/nvme/host/pci.c        | 8 ++++----
 drivers/nvme/host/rdma.c       | 2 +-
 drivers/nvme/host/tcp.c        | 2 +-
 drivers/nvme/target/loop.c     | 2 +-
 drivers/nvme/target/passthru.c | 2 +-
 8 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f97003e54db8..e0629861aa31 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2263,7 +2263,8 @@ int nvme_sec_submit(void *data, u16 spsp, u8 secp, void *buffer, size_t len,
 	cmd.common.cdw11 = cpu_to_le32(len);
 
 	return __nvme_submit_sync_cmd(ctrl->admin_q, &cmd, NULL, buffer, len,
-				      ADMIN_TIMEOUT, NVME_QID_ANY, 1, 0, false);
+				      NVME_ADMIN_TIMEOUT, NVME_QID_ANY, 1, 0,
+				      false);
 }
 EXPORT_SYMBOL_GPL(nvme_sec_submit);
 #endif /* CONFIG_BLK_SED_OPAL */
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index e2e09e25c056..8198b062e1eb 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3480,7 +3480,7 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 			    ctrl->lport->ops->fcprqst_priv_sz);
 	ctrl->admin_tag_set.driver_data = ctrl;
 	ctrl->admin_tag_set.nr_hw_queues = 1;
-	ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
+	ctrl->admin_tag_set.timeout = NVME_ADMIN_TIMEOUT;
 	ctrl->admin_tag_set.flags = BLK_MQ_F_NO_SCHED;
 
 	ret = blk_mq_alloc_tag_set(&ctrl->admin_tag_set);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 3106d6133c8c..28efca2d7171 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -24,7 +24,7 @@ extern unsigned int nvme_io_timeout;
 #define NVME_IO_TIMEOUT	(nvme_io_timeout * HZ)
 
 extern unsigned int admin_timeout;
-#define ADMIN_TIMEOUT	(admin_timeout * HZ)
+#define NVME_ADMIN_TIMEOUT	(admin_timeout * HZ)
 
 #define NVME_DEFAULT_KATO	5
 #define NVME_KATO_GRACE		10
@@ -643,7 +643,7 @@ static inline void nvme_req_set_default_timeout(struct request *req)
 	if (req->q->queuedata)
 		req->timeout = NVME_IO_TIMEOUT;
 	else /* no queuedata implies admin queue */
-		req->timeout = ADMIN_TIMEOUT;
+		req->timeout = NVME_ADMIN_TIMEOUT;
 }
 
 extern const struct attribute_group *nvme_ns_id_attr_groups[];
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e5b02242f3ca..5bb93b687641 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1296,7 +1296,7 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved)
 		return BLK_EH_RESET_TIMER;
 	}
 
-	abort_req->timeout = ADMIN_TIMEOUT;
+	abort_req->timeout = NVME_ADMIN_TIMEOUT;
 	abort_req->end_io_data = NULL;
 	blk_execute_rq_nowait(abort_req->q, NULL, abort_req, 0, abort_endio);
 
@@ -1592,7 +1592,7 @@ static int nvme_alloc_admin_tags(struct nvme_dev *dev)
 		dev->admin_tagset.nr_hw_queues = 1;
 
 		dev->admin_tagset.queue_depth = NVME_AQ_MQ_TAG_DEPTH;
-		dev->admin_tagset.timeout = ADMIN_TIMEOUT;
+		dev->admin_tagset.timeout = NVME_ADMIN_TIMEOUT;
 		dev->admin_tagset.numa_node = dev->ctrl.numa_node;
 		dev->admin_tagset.cmd_size = sizeof(struct nvme_iod);
 		dev->admin_tagset.flags = BLK_MQ_F_NO_SCHED;
@@ -2208,7 +2208,7 @@ static int nvme_delete_queue(struct nvme_queue *nvmeq, u8 opcode)
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
-	req->timeout = ADMIN_TIMEOUT;
+	req->timeout = NVME_ADMIN_TIMEOUT;
 	req->end_io_data = nvmeq;
 
 	init_completion(&nvmeq->delete_done);
@@ -2224,7 +2224,7 @@ static bool __nvme_disable_io_queues(struct nvme_dev *dev, u8 opcode)
 	unsigned long timeout;
 
  retry:
-	timeout = ADMIN_TIMEOUT;
+	timeout = NVME_ADMIN_TIMEOUT;
 	while (nr_queues > 0) {
 		if (nvme_delete_queue(&dev->queues[nr_queues], opcode))
 			break;
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..de36ec783527 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -798,7 +798,7 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
 				NVME_RDMA_DATA_SGL_SIZE;
 		set->driver_data = ctrl;
 		set->nr_hw_queues = 1;
-		set->timeout = ADMIN_TIMEOUT;
+		set->timeout = NVME_ADMIN_TIMEOUT;
 		set->flags = BLK_MQ_F_NO_SCHED;
 	} else {
 		set = &ctrl->tag_set;
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8f4f29f18b8c..9aae5af37df6 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1570,7 +1570,7 @@ static struct blk_mq_tag_set *nvme_tcp_alloc_tagset(struct nvme_ctrl *nctrl,
 		set->cmd_size = sizeof(struct nvme_tcp_request);
 		set->driver_data = ctrl;
 		set->nr_hw_queues = 1;
-		set->timeout = ADMIN_TIMEOUT;
+		set->timeout = NVME_ADMIN_TIMEOUT;
 	} else {
 		set = &ctrl->tag_set;
 		memset(set, 0, sizeof(*set));
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index f6d81239be21..76d8c0a9a87d 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -345,7 +345,7 @@ static int nvme_loop_configure_admin_queue(struct nvme_loop_ctrl *ctrl)
 		NVME_INLINE_SG_CNT * sizeof(struct scatterlist);
 	ctrl->admin_tag_set.driver_data = ctrl;
 	ctrl->admin_tag_set.nr_hw_queues = 1;
-	ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
+	ctrl->admin_tag_set.timeout = NVME_ADMIN_TIMEOUT;
 	ctrl->admin_tag_set.flags = BLK_MQ_F_NO_SCHED;
 
 	ctrl->queues[0].ctrl = ctrl;
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 2039ad7cbbab..728d79e5dabc 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -544,7 +544,7 @@ int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys)
 	}
 
 	if (!subsys->admin_timeout)
-		subsys->admin_timeout = ADMIN_TIMEOUT;
+		subsys->admin_timeout = NVME_ADMIN_TIMEOUT;
 
 	if (!subsys->io_timeout)
 		subsys->io_timeout = NVME_IO_TIMEOUT;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct  6 19:36:47 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  6 Oct 2020 16:36:47 -0700
Subject: [PATCH] nvme-core: put ctrl ref when module ref get fail
Message-ID: <20201006233647.57760-1-chaitanya.kulkarni@wdc.com>

When try_module_get() fails in the nvme_dev_open() it returns without
releasing the ctrl reference which was taken earlier.

Put the ctrl reference which is taken before calling the
try_module_get() in the error return code path.

Fixes: 52a3974feb1a "nvme-core: get/put ctrl and transport module in nvme_dev_open/release()"
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c20ce8eabe82..56e2a22e8a02 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3158,8 +3158,10 @@ static int nvme_dev_open(struct inode *inode, struct file *file)
 	}
 
 	nvme_get_ctrl(ctrl);
-	if (!try_module_get(ctrl->ops->module))
+	if (!try_module_get(ctrl->ops->module)) {
+		nvme_put_ctrl(ctrl);
 		return -EINVAL;
+	}
 
 	file->private_data = ctrl;
 	return 0;
-- 
2.22.1



From Chaitanya.Kulkarni at wdc.com  Tue Oct  6 19:44:20 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 6 Oct 2020 23:44:20 +0000
Subject: [PATCH blktests v2 02/11] common/xfs: Create common helper to
 check for XFS support
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-3-logang@deltatee.com>
Message-ID: <BYAPR04MB4965388F1D0E70EDCEDD57E2860D0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 9/30/20 11:54, Logan Gunthorpe wrote:
>  
>  requires() {
>  	_nvme_requires
> -	_have_program mkfs.xfs && _have_fio
> +	_have_xfs
> +	_have_fio
Can you make _have_xfs return true false ? so it can be used with && ?


From Chaitanya.Kulkarni at wdc.com  Tue Oct  6 19:50:37 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 6 Oct 2020 23:50:37 +0000
Subject: [PATCH blktests v2 03/11] common/xfs: Create common helper to
 verify block device with xfs
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-4-logang@deltatee.com>
Message-ID: <BYAPR04MB4965C2DDD4A5FBD86A128FE9860D0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 9/30/20 11:54, Logan Gunthorpe wrote:
> Make a common helper from the code in tests nvme/012 and nvme/013
> to run an fio verify on a XFS file system backed by the
> specified block device.
>
> While we are at it, all the output is redirected to $FULL instead of
> /dev/null.
>
> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
> ---
>  common/xfs     | 22 ++++++++++++++++++++++
>  tests/nvme/012 | 14 +-------------
>  tests/nvme/013 | 14 +-------------
>  3 files changed, 24 insertions(+), 26 deletions(-)

The common namespace is getting cluttered. Can you please create

a subdirectory common/fs/xfs ?

>
> diff --git a/common/xfs b/common/xfs
> index d1a603b8c7b5..210c924cdd41 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -9,3 +9,25 @@
>  _have_xfs() {
>  	_have_fs xfs && _have_program mkfs.xfs
>  }
> +
> +_xfs_mkfs_and_mount() {
> +	local bdev=$1
> +	local mount_dir=$2
> +
> +	mkdir -p "${mount_dir}"
> +	umount "${mount_dir}"
> +	mkfs.xfs -l size=32m -f "${bdev}"
> +	mount "${bdev}" "${mount_dir}"
> +}
> +
> +_xfs_run_fio_verify_io() {
> +	local mount_dir="/mnt/blktests"

The mount dir should be a parameter and not the hardcode value

to make it reusable.

> +	local bdev=$1
> +
> +	_xfs_mkfs_and_mount "${bdev}" "${mount_dir}" >> "${FULL}" 2>&1
> +
> +	_run_fio_verify_io --size=950m --directory="${mount_dir}/"
> +
> +	umount "${mount_dir}" >> "${FULL}" 2>&1
> +	rm -fr "${mount_dir}"
> +}
> diff --git a/tests/nvme/012 b/tests/nvme/012
> index 1d8d8e3cc271..a13cd08ce6bf 100755
> --- a/tests/nvme/012
> +++ b/tests/nvme/012
> @@ -26,12 +26,9 @@ test() {
>  	local port
>  	local nvmedev
>  	local loop_dev
> -	local mount_dir="/mnt/blktests"
>  	local file_path="${TMPDIR}/img"
>  	local subsys_name="blktests-subsystem-1"
>  
> -	mkdir -p "${mount_dir}" > /dev/null 2>&1
> -
>  	truncate -s 1G "${file_path}"
>  
>  	loop_dev="$(losetup -f --show "${file_path}")"
> @@ -47,15 +44,7 @@ test() {
>  	cat "/sys/block/${nvmedev}n1/uuid"
>  	cat "/sys/block/${nvmedev}n1/wwid"
>  
> -	umount ${mount_dir} > /dev/null 2>&1
> -
> -	mkfs.xfs -l size=32m -f /dev/"${nvmedev}n1" > /dev/null 2>&1
> -
> -	mount /dev/"${nvmedev}n1" "${mount_dir}"
> -
> -	_run_fio_verify_io --size=950m --directory="${mount_dir}/"
> -
> -	umount "${mount_dir}" > /dev/null 2>&1
> +	_xfs_run_fio_verify_io "/dev/${nvmedev}n1"
>  
>  	_nvme_disconnect_subsys "${subsys_name}"
>  
> @@ -66,7 +55,6 @@ test() {
>  	losetup -d "${loop_dev}"
>  
>  	rm "${file_path}"
> -	rm -fr "${mount_dir}"
>  
>  	echo "Test complete"
>  }
> diff --git a/tests/nvme/013 b/tests/nvme/013
> index 3819a2730d9b..1ac725ea83f2 100755
> --- a/tests/nvme/013
> +++ b/tests/nvme/013
> @@ -24,13 +24,10 @@ test() {
>  
>  	local port
>  	local nvmedev
> -	local mount_dir="/mnt/blktests/"
>  	local file_path="${TMPDIR}/img"
>  
>  	local subsys_name="blktests-subsystem-1"
>  
> -	mkdir -p "${mount_dir}" > /dev/null 2>&1
> -
>  	truncate -s 1G "${file_path}"
>  
>  	_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
> @@ -44,15 +41,7 @@ test() {
>  	cat "/sys/block/${nvmedev}n1/uuid"
>  	cat "/sys/block/${nvmedev}n1/wwid"
>  
> -	umount ${mount_dir} > /dev/null 2>&1
> -
> -	mkfs.xfs -l size=32m -f /dev/"${nvmedev}n1" > /dev/null 2>&1
> -
> -	mount /dev/"${nvmedev}n1" "${mount_dir}"
> -
> -	_run_fio_verify_io --size=800m --directory="${mount_dir}/"
> -
> -	umount "${mount_dir}" > /dev/null 2>&1
> +	_xfs_run_fio_verify_io "/dev/${nvmedev}n1"
>  
>  	_nvme_disconnect_subsys "${subsys_name}"
>  
> @@ -61,7 +50,6 @@ test() {
>  	_remove_nvmet_port "${port}"
>  
>  	rm "${file_path}"
> -	rm -fr "${mount_dir}"
>  
>  	echo "Test complete"
>  }

rest looks good



From logang at deltatee.com  Tue Oct  6 19:51:18 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 6 Oct 2020 17:51:18 -0600
Subject: [PATCH blktests v2 02/11] common/xfs: Create common helper to
 check for XFS support
In-Reply-To: <BYAPR04MB4965388F1D0E70EDCEDD57E2860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-3-logang@deltatee.com>
 <BYAPR04MB4965388F1D0E70EDCEDD57E2860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <b809f785-fc8b-4fa5-5de2-26dc9a4aed94@deltatee.com>



On 2020-10-06 5:44 p.m., Chaitanya Kulkarni wrote:
> On 9/30/20 11:54, Logan Gunthorpe wrote:
>>  
>>  requires() {
>>  	_nvme_requires
>> -	_have_program mkfs.xfs && _have_fio
>> +	_have_xfs
>> +	_have_fio
> Can you make _have_xfs return true false ? so it can be used with && ?

_have_xfs() does return true/false and can be used with && or in a
conditional.

Per [1], my opinion is that using && in the requires() function where
the return value is ignored is confusing so I prefer not to do it in new
code.

If we want to reconsider this we, should add a check to ensure the
return value of requires() matches the expectation of the global
variable it uses.

Logan

[1]
https://lore.kernel.org/linux-block/92478e6f-622a-a1ae-6189-4009f9a307bc at deltatee.com/


From Chaitanya.Kulkarni at wdc.com  Tue Oct  6 19:55:17 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 6 Oct 2020 23:55:17 +0000
Subject: [PATCH blktests v2 04/11] nvme: Search for specific subsysnqn in
 _find_nvme_loop_dev
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-5-logang@deltatee.com>
Message-ID: <BYAPR04MB49657A481271ADC1F576377B860D0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 9/30/20 11:54, Logan Gunthorpe wrote:
> This ensures we find the correct nvme loop device if others exist on a
> given system (which is generally not expected on test systems).
>
> Additionally, this will be required in the upcomming test nvme/037 which
> will have controllers racing with ones being destroyed.
>
> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>

If I create a passthru testcase with :-

1. Create nvme-loop based nvme ctrl backed by null_blk say /dev/nvme1

2. Create a nvme-loop based passthru ctrl backed by /dev/nvme1 say nvme2.


With this patch or this series will I be able to write the testcase ?



From logang at deltatee.com  Tue Oct  6 19:59:35 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 6 Oct 2020 17:59:35 -0600
Subject: [PATCH blktests v2 03/11] common/xfs: Create common helper to
 verify block device with xfs
In-Reply-To: <BYAPR04MB4965C2DDD4A5FBD86A128FE9860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-4-logang@deltatee.com>
 <BYAPR04MB4965C2DDD4A5FBD86A128FE9860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <868f8fb6-6024-d60d-a9aa-6513b9d0986f@deltatee.com>



On 2020-10-06 5:50 p.m., Chaitanya Kulkarni wrote:
> On 9/30/20 11:54, Logan Gunthorpe wrote:
>> Make a common helper from the code in tests nvme/012 and nvme/013
>> to run an fio verify on a XFS file system backed by the
>> specified block device.
>>
>> While we are at it, all the output is redirected to $FULL instead of
>> /dev/null.
>>
>> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
>> ---
>>  common/xfs     | 22 ++++++++++++++++++++++
>>  tests/nvme/012 | 14 +-------------
>>  tests/nvme/013 | 14 +-------------
>>  3 files changed, 24 insertions(+), 26 deletions(-)
> 
> The common namespace is getting cluttered. Can you please create
> 
> a subdirectory common/fs/xfs ?

I disagree. The common directory only has 9 files. And given there
appear to be no other files to add to the fs directory I don't think now
is the time to create a directory. We can do so if and when a number of
other fs helpers show up and there's a reason to group them together.

>>
>> diff --git a/common/xfs b/common/xfs
>> index d1a603b8c7b5..210c924cdd41 100644
>> --- a/common/xfs
>> +++ b/common/xfs
>> @@ -9,3 +9,25 @@
>>  _have_xfs() {
>>  	_have_fs xfs && _have_program mkfs.xfs
>>  }
>> +
>> +_xfs_mkfs_and_mount() {
>> +	local bdev=$1
>> +	local mount_dir=$2
>> +
>> +	mkdir -p "${mount_dir}"
>> +	umount "${mount_dir}"
>> +	mkfs.xfs -l size=32m -f "${bdev}"
>> +	mount "${bdev}" "${mount_dir}"
>> +}
>> +
>> +_xfs_run_fio_verify_io() {
>> +	local mount_dir="/mnt/blktests"
> 
> The mount dir should be a parameter and not the hardcode value
> to make it reusable.

I also disagree here. It is already reusable and is used in a number of
places; none of those places require changing the mount directory. If
and when a use comes up that requires a different directory (not sure
what that would be), a parameter can be added. It is typically standard
practice in the Linux community to not add features that have no users
as it's confusing to people reading the code.

Logan


From Chaitanya.Kulkarni at wdc.com  Tue Oct  6 20:02:09 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 7 Oct 2020 00:02:09 +0000
Subject: [PATCH blktests v2 05/11] nvme: Add common helpers for passthru
 tests
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-6-logang@deltatee.com>
Message-ID: <BYAPR04MB49654DCBAD729B05759EB6FF860A0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 9/30/20 12:01, Logan Gunthorpe wrote:
> Add some simple helpers to setup a passthru target that passes through
> to a nvme test device.
>
> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
> ---
>  tests/nvme/rc | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 76 insertions(+)
>
> diff --git a/tests/nvme/rc b/tests/nvme/rc
> index dfa57a299625..1ea23308a3f7 100644
> --- a/tests/nvme/rc
> +++ b/tests/nvme/rc
> @@ -73,6 +73,17 @@ _require_nvme_trtype_is_fabrics() {
>  	return 0
>  }
>  
> +_test_dev_nvme_ctrl() {
> +	local dev
> +
> +	dev=$(cat "${TEST_DEV_SYSFS}/device/dev")
 can you initialize dev this at the time of declaration ?
> +	echo "/dev/char/${dev}"
> +}
> +
> +_test_dev_nvme_nsid() {
> +	cat "${TEST_DEV_SYSFS}/nsid"
> +}
> +
>  _cleanup_nvmet() {
>  	local dev
>  	local port
> @@ -257,6 +268,27 @@ _remove_nvmet_subsystem() {
>  	rmdir "${subsys_path}"
>  }
>  
> +_create_nvmet_passthru() {
> +	local nvmet_subsystem="$1"
> +	local subsys_path="${NVMET_CFS}/subsystems/${nvmet_subsystem}"
> +	local passthru_path="${subsys_path}/passthru"
> +
> +	mkdir -p "${subsys_path}"
> +	printf 1 > "${subsys_path}/attr_allow_any_host"
> +
> +	printf "%s" "$(_test_dev_nvme_ctrl)" > "${passthru_path}/device_path"
> +	printf 1 > "${passthru_path}/enable"

can you please echo in general and printf only when it is needed ?

I know existing code is a bit inconsistent I'll send a clenup to make it
uniform.

> +}
> +
> +_remove_nvmet_passhtru() {
> +	local nvmet_subsystem="$1"
> +	local subsys_path="${NVMET_CFS}/subsystems/${nvmet_subsystem}"
> +	local passthru_path="${subsys_path}/passthru"
> +
> +	printf 0 > "${passthru_path}/enable"
> +	rmdir "${subsys_path}"
> +}
> +
>  _add_nvmet_subsys_to_port() {
>  	local port="$1"
>  	local nvmet_subsystem="$2"
> @@ -292,6 +324,50 @@ _find_nvme_dev() {
>  	done
>  }
>  
> +_find_nvme_passthru_loop_dev() {
> +	local subsys=$1
> +	local nsid
> +	local dev
> +
> +	dev=$(_find_nvme_dev "${subsys}")
> +	nsid=$(_test_dev_nvme_nsid)
> +	echo "/dev/${dev}n${nsid}"
> +}
> +
> +_nvmet_passthru_target_setup() {
> +	local subsys_name=$1
> +
> +	_create_nvmet_passthru "${subsys_name}"
> +	port="$(_create_nvmet_port "loop")"
> +	_add_nvmet_subsys_to_port "${port}" "${subsys_name}"
> +
> +	echo "$port"
> +}
> +
> +_nvmet_passthru_target_connect() {
> +	local trtype=$1
> +	local subsys_name=$2
> +
> +	_nvme_connect_subsys "${trtype}" "${subsys_name}"
> +	nsdev=$(_find_nvme_passthru_loop_dev "${subsys_name}")
> +
> +	# The following tests can race with the creation
> +	# of the device so ensure the block device exists
> +	# before continuing
> +	while [ ! -b "${nsdev}" ]; do sleep 1; done
> +
> +	echo "${nsdev}"
> +}
> +
> +_nvmet_passthru_target_cleanup() {
> +	local port=$1
> +	local subsys_name=$2
> +
> +	_remove_nvmet_subsystem_from_port "${port}" "${subsys_name}"
> +	_remove_nvmet_port "${port}"
> +	_remove_nvmet_passhtru "${subsys_name}"
> +}
> +
>  _filter_discovery() {
>  	sed -n -r -e "s/Generation counter [0-9]+/Generation counter X/" \
>  		  -e '/Discovery Log Number|Log Entry|trtype|subnqn/p'




From logang at deltatee.com  Tue Oct  6 20:10:16 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 6 Oct 2020 18:10:16 -0600
Subject: [PATCH blktests v2 04/11] nvme: Search for specific subsysnqn in
 _find_nvme_loop_dev
In-Reply-To: <BYAPR04MB49657A481271ADC1F576377B860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-5-logang@deltatee.com>
 <BYAPR04MB49657A481271ADC1F576377B860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <517a5fd9-50eb-eb5e-5911-093dc5d1a759@deltatee.com>



On 2020-10-06 5:55 p.m., Chaitanya Kulkarni wrote:
> On 9/30/20 11:54, Logan Gunthorpe wrote:
>> This ensures we find the correct nvme loop device if others exist on a
>> given system (which is generally not expected on test systems).
>>
>> Additionally, this will be required in the upcomming test nvme/037 which
>> will have controllers racing with ones being destroyed.
>>
>> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
> 
> If I create a passthru testcase with :-
> 
> 1. Create nvme-loop based nvme ctrl backed by null_blk say /dev/nvme1
> 
> 2. Create a nvme-loop based passthru ctrl backed by /dev/nvme1 say nvme2.
> 
> 
> With this patch or this series will I be able to write the testcase ?

This patch helps with that but other helpers introduced in this series
would require minor changes.

As far as I can see, you'd only have to adjust _create_nvmet_passthru()
to take an optional argument because, presently, it always uses
$_test_dev_nvme_ctrl for the backing device.

This can easily be done if and when someone writes such a test.

However, I'm not even sure right now if that test would pass in the
kernel as is -- it seems like an odd thing to do.

Logan


From logang at deltatee.com  Tue Oct  6 20:13:33 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 6 Oct 2020 18:13:33 -0600
Subject: [PATCH blktests v2 05/11] nvme: Add common helpers for passthru
 tests
In-Reply-To: <BYAPR04MB49654DCBAD729B05759EB6FF860A0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-6-logang@deltatee.com>
 <BYAPR04MB49654DCBAD729B05759EB6FF860A0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <e7bdf462-c86d-f248-58f4-d9d41b2bd343@deltatee.com>



On 2020-10-06 6:02 p.m., Chaitanya Kulkarni wrote:
> On 9/30/20 12:01, Logan Gunthorpe wrote:
>> Add some simple helpers to setup a passthru target that passes through
>> to a nvme test device.
>>
>> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
>> ---
>>  tests/nvme/rc | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 76 insertions(+)
>>
>> diff --git a/tests/nvme/rc b/tests/nvme/rc
>> index dfa57a299625..1ea23308a3f7 100644
>> --- a/tests/nvme/rc
>> +++ b/tests/nvme/rc
>> @@ -73,6 +73,17 @@ _require_nvme_trtype_is_fabrics() {
>>  	return 0
>>  }
>>  
>> +_test_dev_nvme_ctrl() {
>> +	local dev
>> +
>> +	dev=$(cat "${TEST_DEV_SYSFS}/device/dev")
>  can you initialize dev this at the time of declaration ?

Yup, will fix.

>> +	echo "/dev/char/${dev}"
>> +}
>> +
>> +_test_dev_nvme_nsid() {
>> +	cat "${TEST_DEV_SYSFS}/nsid"
>> +}
>> +
>>  _cleanup_nvmet() {
>>  	local dev
>>  	local port
>> @@ -257,6 +268,27 @@ _remove_nvmet_subsystem() {
>>  	rmdir "${subsys_path}"
>>  }
>>  
>> +_create_nvmet_passthru() {
>> +	local nvmet_subsystem="$1"
>> +	local subsys_path="${NVMET_CFS}/subsystems/${nvmet_subsystem}"
>> +	local passthru_path="${subsys_path}/passthru"
>> +
>> +	mkdir -p "${subsys_path}"
>> +	printf 1 > "${subsys_path}/attr_allow_any_host"
>> +
>> +	printf "%s" "$(_test_dev_nvme_ctrl)" > "${passthru_path}/device_path"
>> +	printf 1 > "${passthru_path}/enable"
> 
> can you please echo in general and printf only when it is needed ?>
> I know existing code is a bit inconsistent I'll send a clenup to make it
> uniform.

Yes, I agree. I will it fix in v3.
Thanks,

Logan


From logang at deltatee.com  Tue Oct  6 20:20:00 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 6 Oct 2020 18:20:00 -0600
Subject: [PATCH] nvme-core: put ctrl ref when module ref get fail
In-Reply-To: <20201006233647.57760-1-chaitanya.kulkarni@wdc.com>
References: <20201006233647.57760-1-chaitanya.kulkarni@wdc.com>
Message-ID: <e2c46d78-972b-dd7c-6662-edbf91a095fa@deltatee.com>



On 2020-10-06 5:36 p.m., Chaitanya Kulkarni wrote:
> When try_module_get() fails in the nvme_dev_open() it returns without
> releasing the ctrl reference which was taken earlier.
> 
> Put the ctrl reference which is taken before calling the
> try_module_get() in the error return code path.
> 
> Fixes: 52a3974feb1a "nvme-core: get/put ctrl and transport module in nvme_dev_open/release()"
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>

I guess I missed this too in my review of that patch. Nice catch.

Reviewed-by: Logan Gunthorpe <logang at deltatee.com>

> ---
>  drivers/nvme/host/core.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index c20ce8eabe82..56e2a22e8a02 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -3158,8 +3158,10 @@ static int nvme_dev_open(struct inode *inode, struct file *file)
>  	}
>  
>  	nvme_get_ctrl(ctrl);
> -	if (!try_module_get(ctrl->ops->module))
> +	if (!try_module_get(ctrl->ops->module)) {
> +		nvme_put_ctrl(ctrl);
>  		return -EINVAL;
> +	}
>  
>  	file->private_data = ctrl;
>  	return 0;
> 


From Chaitanya.Kulkarni at wdc.com  Tue Oct  6 20:58:23 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 7 Oct 2020 00:58:23 +0000
Subject: [PATCH blktests v2 02/11] common/xfs: Create common helper to
 check for XFS support
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-3-logang@deltatee.com>
 <BYAPR04MB4965388F1D0E70EDCEDD57E2860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b809f785-fc8b-4fa5-5de2-26dc9a4aed94@deltatee.com>
Message-ID: <BYAPR04MB4965E6FFC184E347DDB78A29860A0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/6/20 16:51, Logan Gunthorpe wrote:
> _have_xfs() does return true/false and can be used with && or in a
> conditional.
>
> Per [1], my opinion is that using && in the requires() function where
> the return value is ignored is confusing so I prefer not to do it in new
> code.
>
> If we want to reconsider this we, should add a check to ensure the
> return value of requires() matches the expectation of the global
> variable it uses.
>
> Logan
>
> [1]
> https://lore.kernel.org/linux-block/92478e6f-622a-a1ae-6189-4009f9a307bc at deltatee.com/

Make sense to me, lets not change this, thanks for pointing that out.



From ming.lei at redhat.com  Tue Oct  6 21:04:40 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed,  7 Oct 2020 09:04:40 +0800
Subject: [PATCH V7 1/4] block: use test_and_{clear|test}_bit to set/clear
 QUEUE_FLAG_QUIESCED
In-Reply-To: <20201007010443.71456-1-ming.lei@redhat.com>
References: <20201007010443.71456-1-ming.lei@redhat.com>
Message-ID: <20201007010443.71456-2-ming.lei@redhat.com>

Prepare for replacing srcu with percpu-refcount for implementing queue
quiesce.

The following patch needs to avoid duplicated quiesce action for
BLK_MQ_F_BLOCKING, so use test_and_{clear|test}_bit to set/clear
QUEUE_FLAG_QUIESCED.

Signed-off-by: Ming Lei <ming.lei at redhat.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 block/blk-core.c | 13 +++++++++++++
 block/blk-mq.c   | 11 ++++++++---
 block/blk.h      |  2 ++
 3 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index 3b01e7f3c7a3..a5afe67559fe 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -107,6 +107,19 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_queue_flag_test_and_set);
 
+/**
+ * blk_queue_flag_test_and_clear - atomically test and clear a queue flag
+ * @flag: flag to be clear
+ * @q: request queue
+ *
+ * Returns the previous value of @flag - 0 if the flag was not set and 1 if
+ * the flag was set.
+ */
+bool blk_queue_flag_test_and_clear(unsigned int flag, struct request_queue *q)
+{
+	return test_and_clear_bit(flag, &q->queue_flags);
+}
+
 void blk_rq_init(struct request_queue *q, struct request *rq)
 {
 	memset(rq, 0, sizeof(*rq));
diff --git a/block/blk-mq.c b/block/blk-mq.c
index c5fefd39d0c0..3fbcf431fcf4 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -199,13 +199,18 @@ void blk_mq_unfreeze_queue(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_mq_unfreeze_queue);
 
+static bool __blk_mq_quiesce_queue_nowait(struct request_queue *q)
+{
+	return blk_queue_flag_test_and_set(QUEUE_FLAG_QUIESCED, q);
+}
+
 /*
  * FIXME: replace the scsi_internal_device_*block_nowait() calls in the
  * mpt3sas driver such that this function can be removed.
  */
 void blk_mq_quiesce_queue_nowait(struct request_queue *q)
 {
-	blk_queue_flag_set(QUEUE_FLAG_QUIESCED, q);
+	__blk_mq_quiesce_queue_nowait(q);
 }
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_nowait);
 
@@ -224,7 +229,7 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	unsigned int i;
 	bool rcu = false;
 
-	blk_mq_quiesce_queue_nowait(q);
+	__blk_mq_quiesce_queue_nowait(q);
 
 	queue_for_each_hw_ctx(q, hctx, i) {
 		if (hctx->flags & BLK_MQ_F_BLOCKING)
@@ -246,7 +251,7 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
  */
 void blk_mq_unquiesce_queue(struct request_queue *q)
 {
-	blk_queue_flag_clear(QUEUE_FLAG_QUIESCED, q);
+	blk_queue_flag_test_and_clear(QUEUE_FLAG_QUIESCED, q);
 
 	/* dispatch requests which are inserted during quiescing */
 	blk_mq_run_hw_queues(q, true);
diff --git a/block/blk.h b/block/blk.h
index dfab98465db9..f925268ce7fd 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -444,4 +444,6 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio,
 		struct page *page, unsigned int len, unsigned int offset,
 		unsigned int max_sectors, bool *same_page);
 
+bool blk_queue_flag_test_and_clear(unsigned int flag, struct request_queue *q);
+
 #endif /* BLK_INTERNAL_H */
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct  6 21:04:41 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed,  7 Oct 2020 09:04:41 +0800
Subject: [PATCH V7 2/4] blk-mq: implement queue quiesce via percpu_ref for
 BLK_MQ_F_BLOCKING
In-Reply-To: <20201007010443.71456-1-ming.lei@redhat.com>
References: <20201007010443.71456-1-ming.lei@redhat.com>
Message-ID: <20201007010443.71456-3-ming.lei@redhat.com>

In case of BLK_MQ_F_BLOCKING, blk-mq uses SRCU to mark read critical
section during dispatching request, then request queue quiesce is based on
SRCU. What we want to get is low cost added in fast path.

With percpu-ref, it is cleaner and simpler & enough for implementing queue
quiesce. The main requirement is to make sure all read sections to observe
QUEUE_FLAG_QUIESCED once blk_mq_quiesce_queue() returns.

Also it becomes much easier to add interface of async queue quiesce.

Meantime memory footprint can be reduced with per-request-queue percpu-ref.

>From implementation viewpoint, in fast path, not see percpu_ref is
slower than SRCU, and srcu tree(default option in most distributions)
could be slower since memory barrier is required in both lock & unlock,
and rcu_read_lock()/rcu_read_unlock() should be much cheap than
smp_mb().

1) percpu_ref just hold the rcu_read_lock, then run a check &
   increase/decrease on the percpu variable:

   rcu_read_lock()
   if (__ref_is_percpu(ref, &percpu_count))
	this_cpu_inc(*percpu_count);
   rcu_read_unlock()

2) srcu tree:
        idx = READ_ONCE(ssp->srcu_idx) & 0x1;
        this_cpu_inc(ssp->sda->srcu_lock_count[idx]);
        smp_mb(); /* B */  /* Avoid leaking the critical section. */

Also from my test on null_blk(blocking), not observe percpu-ref performs
worse than srcu, see the following test:

1) test steps:

rmmod null_blk > /dev/null 2>&1
modprobe null_blk nr_devices=1 submit_queues=1 blocking=1
fio --bs=4k --size=512G  --rw=randread --norandommap --direct=1 --ioengine=libaio \
	--iodepth=64 --runtime=60 --group_reporting=1  --name=nullb0 \
	--filename=/dev/nullb0 --numjobs=32

test machine: HP DL380, 16 cpu cores, 2 threads per core, dual
sockets/numa, Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz

2) test result:
- srcu quiesce: 6063K IOPS
- percpu-ref quiesce: 6113K IOPS

Signed-off-by: Ming Lei <ming.lei at redhat.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 128 ++++++++++++++++++++++-------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   8 ---
 include/linux/blkdev.h |   4 ++
 5 files changed, 79 insertions(+), 69 deletions(-)

diff --git a/block/blk-mq-sysfs.c b/block/blk-mq-sysfs.c
index 062229395a50..799db7937105 100644
--- a/block/blk-mq-sysfs.c
+++ b/block/blk-mq-sysfs.c
@@ -38,8 +38,6 @@ static void blk_mq_hw_sysfs_release(struct kobject *kobj)
 
 	cancel_delayed_work_sync(&hctx->run_work);
 
-	if (hctx->flags & BLK_MQ_F_BLOCKING)
-		cleanup_srcu_struct(hctx->srcu);
 	blk_free_flush_queue(hctx->fq);
 	sbitmap_free(&hctx->ctx_map);
 	free_cpumask_var(hctx->cpumask);
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 3fbcf431fcf4..738753bc0caa 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -225,19 +225,23 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_nowait);
  */
 void blk_mq_quiesce_queue(struct request_queue *q)
 {
-	struct blk_mq_hw_ctx *hctx;
-	unsigned int i;
-	bool rcu = false;
+	bool blocking = !!(q->tag_set->flags & BLK_MQ_F_BLOCKING);
+	bool was_quiesced =__blk_mq_quiesce_queue_nowait(q);
 
-	__blk_mq_quiesce_queue_nowait(q);
+	if (!was_quiesced && blocking)
+		percpu_ref_kill(&q->dispatch_counter);
 
-	queue_for_each_hw_ctx(q, hctx, i) {
-		if (hctx->flags & BLK_MQ_F_BLOCKING)
-			synchronize_srcu(hctx->srcu);
-		else
-			rcu = true;
-	}
-	if (rcu)
+	/*
+	 * In case of F_BLOCKING, if driver unquiesces its queue being
+	 * quiesced, it can cause bigger trouble, and we simply return &
+	 * warn once for avoiding hang here.
+	 */
+	if (blocking)
+		wait_event(q->mq_quiesce_wq,
+				percpu_ref_is_zero(&q->dispatch_counter) ||
+				WARN_ON_ONCE(!percpu_ref_is_dying(
+						&q->dispatch_counter)));
+	else
 		synchronize_rcu();
 }
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
@@ -251,7 +255,10 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
  */
 void blk_mq_unquiesce_queue(struct request_queue *q)
 {
-	blk_queue_flag_test_and_clear(QUEUE_FLAG_QUIESCED, q);
+	if (blk_queue_flag_test_and_clear(QUEUE_FLAG_QUIESCED, q)) {
+		if (q->tag_set->flags & BLK_MQ_F_BLOCKING)
+			percpu_ref_resurrect(&q->dispatch_counter);
+	}
 
 	/* dispatch requests which are inserted during quiescing */
 	blk_mq_run_hw_queues(q, true);
@@ -704,24 +711,21 @@ void blk_mq_complete_request(struct request *rq)
 }
 EXPORT_SYMBOL(blk_mq_complete_request);
 
-static void hctx_unlock(struct blk_mq_hw_ctx *hctx, int srcu_idx)
-	__releases(hctx->srcu)
+static void hctx_unlock(struct blk_mq_hw_ctx *hctx)
 {
-	if (!(hctx->flags & BLK_MQ_F_BLOCKING))
-		rcu_read_unlock();
+	if (hctx->flags & BLK_MQ_F_BLOCKING)
+		percpu_ref_put(&hctx->queue->dispatch_counter);
 	else
-		srcu_read_unlock(hctx->srcu, srcu_idx);
+		rcu_read_unlock();
 }
 
-static void hctx_lock(struct blk_mq_hw_ctx *hctx, int *srcu_idx)
-	__acquires(hctx->srcu)
+/* Returning false means that queue is being quiesced */
+static inline bool hctx_lock(struct blk_mq_hw_ctx *hctx)
 {
-	if (!(hctx->flags & BLK_MQ_F_BLOCKING)) {
-		/* shut up gcc false positive */
-		*srcu_idx = 0;
-		rcu_read_lock();
-	} else
-		*srcu_idx = srcu_read_lock(hctx->srcu);
+	if (hctx->flags & BLK_MQ_F_BLOCKING)
+		return percpu_ref_tryget_live(&hctx->queue->dispatch_counter);
+	rcu_read_lock();
+	return true;
 }
 
 /**
@@ -1501,8 +1505,6 @@ bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx *hctx, struct list_head *list,
  */
 static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
 {
-	int srcu_idx;
-
 	/*
 	 * We should be running this queue from one of the CPUs that
 	 * are mapped to it.
@@ -1536,9 +1538,10 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
 
 	might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING);
 
-	hctx_lock(hctx, &srcu_idx);
-	blk_mq_sched_dispatch_requests(hctx);
-	hctx_unlock(hctx, srcu_idx);
+	if (hctx_lock(hctx)) {
+		blk_mq_sched_dispatch_requests(hctx);
+		hctx_unlock(hctx);
+	}
 }
 
 static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
@@ -1650,7 +1653,6 @@ EXPORT_SYMBOL(blk_mq_delay_run_hw_queue);
  */
 void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async)
 {
-	int srcu_idx;
 	bool need_run;
 
 	/*
@@ -1661,10 +1663,12 @@ void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async)
 	 * And queue will be rerun in blk_mq_unquiesce_queue() if it is
 	 * quiesced.
 	 */
-	hctx_lock(hctx, &srcu_idx);
+	if (!hctx_lock(hctx))
+		return;
+
 	need_run = !blk_queue_quiesced(hctx->queue) &&
 		blk_mq_hctx_has_pending(hctx);
-	hctx_unlock(hctx, srcu_idx);
+	hctx_unlock(hctx);
 
 	if (need_run)
 		__blk_mq_delay_run_hw_queue(hctx, async, 0);
@@ -2009,7 +2013,7 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx,
 	bool run_queue = true;
 
 	/*
-	 * RCU or SRCU read lock is needed before checking quiesced flag.
+	 * hctx_lock() is needed before checking quiesced flag.
 	 *
 	 * When queue is stopped or quiesced, ignore 'bypass_insert' from
 	 * blk_mq_request_issue_directly(), and return BLK_STS_OK to caller,
@@ -2057,11 +2061,14 @@ static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx,
 		struct request *rq, blk_qc_t *cookie)
 {
 	blk_status_t ret;
-	int srcu_idx;
 
 	might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING);
 
-	hctx_lock(hctx, &srcu_idx);
+	/* Insert request to queue in case of being quiesced */
+	if (!hctx_lock(hctx)) {
+		blk_mq_sched_insert_request(rq, false, false, false);
+		return;
+	}
 
 	ret = __blk_mq_try_issue_directly(hctx, rq, cookie, false, true);
 	if (ret == BLK_STS_RESOURCE || ret == BLK_STS_DEV_RESOURCE)
@@ -2069,19 +2076,22 @@ static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx,
 	else if (ret != BLK_STS_OK)
 		blk_mq_end_request(rq, ret);
 
-	hctx_unlock(hctx, srcu_idx);
+	hctx_unlock(hctx);
 }
 
 blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last)
 {
 	blk_status_t ret;
-	int srcu_idx;
 	blk_qc_t unused_cookie;
 	struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
 
-	hctx_lock(hctx, &srcu_idx);
+	/* Insert request to queue in case of being quiesced */
+	if (!hctx_lock(hctx)) {
+		blk_mq_sched_insert_request(rq, false, false, false);
+		return BLK_STS_OK;
+	}
 	ret = __blk_mq_try_issue_directly(hctx, rq, &unused_cookie, true, last);
-	hctx_unlock(hctx, srcu_idx);
+	hctx_unlock(hctx);
 
 	return ret;
 }
@@ -2612,20 +2622,6 @@ static void blk_mq_exit_hw_queues(struct request_queue *q,
 	}
 }
 
-static int blk_mq_hw_ctx_size(struct blk_mq_tag_set *tag_set)
-{
-	int hw_ctx_size = sizeof(struct blk_mq_hw_ctx);
-
-	BUILD_BUG_ON(ALIGN(offsetof(struct blk_mq_hw_ctx, srcu),
-			   __alignof__(struct blk_mq_hw_ctx)) !=
-		     sizeof(struct blk_mq_hw_ctx));
-
-	if (tag_set->flags & BLK_MQ_F_BLOCKING)
-		hw_ctx_size += sizeof(struct srcu_struct);
-
-	return hw_ctx_size;
-}
-
 static int blk_mq_init_hctx(struct request_queue *q,
 		struct blk_mq_tag_set *set,
 		struct blk_mq_hw_ctx *hctx, unsigned hctx_idx)
@@ -2663,7 +2659,7 @@ blk_mq_alloc_hctx(struct request_queue *q, struct blk_mq_tag_set *set,
 	struct blk_mq_hw_ctx *hctx;
 	gfp_t gfp = GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY;
 
-	hctx = kzalloc_node(blk_mq_hw_ctx_size(set), gfp, node);
+	hctx = kzalloc_node(sizeof(struct blk_mq_hw_ctx), gfp, node);
 	if (!hctx)
 		goto fail_alloc_hctx;
 
@@ -2706,8 +2702,6 @@ blk_mq_alloc_hctx(struct request_queue *q, struct blk_mq_tag_set *set,
 	if (!hctx->fq)
 		goto free_bitmap;
 
-	if (hctx->flags & BLK_MQ_F_BLOCKING)
-		init_srcu_struct(hctx->srcu);
 	blk_mq_hctx_kobj_init(hctx);
 
 	return hctx;
@@ -3187,6 +3181,13 @@ static void blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set,
 	mutex_unlock(&q->sysfs_lock);
 }
 
+static void blk_mq_dispatch_counter_release(struct percpu_ref *ref)
+{
+	struct request_queue *q = container_of(ref, struct request_queue,
+				dispatch_counter);
+	wake_up_all(&q->mq_quiesce_wq);
+}
+
 struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 						  struct request_queue *q,
 						  bool elevator_init)
@@ -3203,6 +3204,14 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 	if (blk_mq_alloc_ctxs(q))
 		goto err_poll;
 
+	if (set->flags & BLK_MQ_F_BLOCKING) {
+		init_waitqueue_head(&q->mq_quiesce_wq);
+		if (percpu_ref_init(&q->dispatch_counter,
+					blk_mq_dispatch_counter_release,
+					PERCPU_REF_ALLOW_REINIT, GFP_KERNEL))
+			goto err_hctxs;
+	}
+
 	/* init q->mq_kobj and sw queues' kobjects */
 	blk_mq_sysfs_init(q);
 
@@ -3211,7 +3220,7 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 
 	blk_mq_realloc_hw_ctxs(set, q);
 	if (!q->nr_hw_queues)
-		goto err_hctxs;
+		goto err_dispatch_counter;
 
 	INIT_WORK(&q->timeout_work, blk_mq_timeout_work);
 	blk_queue_rq_timeout(q, set->timeout ? set->timeout : 30 * HZ);
@@ -3245,6 +3254,9 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 
 	return q;
 
+err_dispatch_counter:
+	if (set->flags & BLK_MQ_F_BLOCKING)
+		percpu_ref_exit(&q->dispatch_counter);
 err_hctxs:
 	kfree(q->queue_hw_ctx);
 	q->nr_hw_queues = 0;
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index 76b54c7750b0..5d31b34abb8f 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -798,9 +798,13 @@ static void blk_release_queue(struct kobject *kobj)
 
 	blk_queue_free_zone_bitmaps(q);
 
-	if (queue_is_mq(q))
+	if (queue_is_mq(q)) {
 		blk_mq_release(q);
 
+		if (q->tag_set->flags & BLK_MQ_F_BLOCKING)
+			percpu_ref_exit(&q->dispatch_counter);
+	}
+
 	blk_trace_shutdown(q);
 	mutex_lock(&q->debugfs_mutex);
 	debugfs_remove_recursive(q->debugfs_dir);
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index b23eeca4d677..df642055f02c 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -4,7 +4,6 @@
 
 #include <linux/blkdev.h>
 #include <linux/sbitmap.h>
-#include <linux/srcu.h>
 
 struct blk_mq_tags;
 struct blk_flush_queue;
@@ -173,13 +172,6 @@ struct blk_mq_hw_ctx {
 	 * q->unused_hctx_list.
 	 */
 	struct list_head	hctx_list;
-
-	/**
-	 * @srcu: Sleepable RCU. Use as lock when type of the hardware queue is
-	 * blocking (BLK_MQ_F_BLOCKING). Must be the last member - see also
-	 * blk_mq_hw_ctx_size().
-	 */
-	struct srcu_struct	srcu[];
 };
 
 /**
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index cda786fa4341..7490d9aa670c 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -572,6 +572,10 @@ struct request_queue {
 	 */
 	struct mutex		mq_freeze_lock;
 
+	/* only used for BLK_MQ_F_BLOCKING */
+	struct percpu_ref	dispatch_counter;
+	wait_queue_head_t	mq_quiesce_wq;
+
 	struct blk_mq_tag_set	*tag_set;
 	struct list_head	tag_set_list;
 	struct bio_set		bio_split;
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct  6 21:04:42 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed,  7 Oct 2020 09:04:42 +0800
Subject: [PATCH V7 3/4] blk-mq: add tagset quiesce interface
In-Reply-To: <20201007010443.71456-1-ming.lei@redhat.com>
References: <20201007010443.71456-1-ming.lei@redhat.com>
Message-ID: <20201007010443.71456-4-ming.lei@redhat.com>

drivers that have shared tagsets may need to quiesce potentially a lot
of request queues that all share a single tagset (e.g. nvme). Add an interface
to quiesce all the queues on a given tagset. This interface is useful because
it can speedup the quiesce by doing it in parallel.

For tagsets that have BLK_MQ_F_BLOCKING set, we kill request queue's dispatch
percpu-refcount such that all of them wait for the counter becoming zero. For
tagsets that don't have BLK_MQ_F_BLOCKING set, we simply call a single
synchronize_rcu as this is sufficient.

This patch is against Sagi's original post.
Signed-off-by: Ming Lei <ming.lei at redhat.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 block/blk-mq.c         | 59 +++++++++++++++++++++++++++++++++++-------
 include/linux/blk-mq.h |  2 ++
 2 files changed, 51 insertions(+), 10 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 738753bc0caa..429b73ddd64c 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -214,16 +214,7 @@ void blk_mq_quiesce_queue_nowait(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_nowait);
 
-/**
- * blk_mq_quiesce_queue() - wait until all ongoing dispatches have finished
- * @q: request queue.
- *
- * Note: this function does not prevent that the struct request end_io()
- * callback function is invoked. Once this function is returned, we make
- * sure no dispatch can happen until the queue is unquiesced via
- * blk_mq_unquiesce_queue().
- */
-void blk_mq_quiesce_queue(struct request_queue *q)
+static void __blk_mq_quiesce_queue(struct request_queue *q, bool wait)
 {
 	bool blocking = !!(q->tag_set->flags & BLK_MQ_F_BLOCKING);
 	bool was_quiesced =__blk_mq_quiesce_queue_nowait(q);
@@ -231,6 +222,9 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	if (!was_quiesced && blocking)
 		percpu_ref_kill(&q->dispatch_counter);
 
+	if (!wait)
+		return;
+
 	/*
 	 * In case of F_BLOCKING, if driver unquiesces its queue being
 	 * quiesced, it can cause bigger trouble, and we simply return &
@@ -244,6 +238,20 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	else
 		synchronize_rcu();
 }
+
+/*
+ * blk_mq_quiesce_queue() - wait until all ongoing dispatches have finished
+ * @q: request queue.
+ *
+ * Note: this function does not prevent that the struct request end_io()
+ * callback function is invoked. Once this function is returned, we make
+ * sure no dispatch can happen until the queue is unquiesced via
+ * blk_mq_unquiesce_queue().
+ */
+void blk_mq_quiesce_queue(struct request_queue *q)
+{
+	__blk_mq_quiesce_queue(q, true);
+}
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
 
 /*
@@ -265,6 +273,37 @@ void blk_mq_unquiesce_queue(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_mq_unquiesce_queue);
 
+void blk_mq_quiesce_tagset(struct blk_mq_tag_set *set)
+{
+	struct request_queue *q;
+
+	mutex_lock(&set->tag_list_lock);
+	list_for_each_entry(q, &set->tag_list, tag_set_list)
+		__blk_mq_quiesce_queue(q, false);
+
+	/* wait until all queues' quiesce is done */
+	if (set->flags & BLK_MQ_F_BLOCKING) {
+		list_for_each_entry(q, &set->tag_list, tag_set_list)
+			wait_event(q->mq_quiesce_wq,
+				   percpu_ref_is_zero(&q->dispatch_counter));
+	} else {
+		synchronize_rcu();
+	}
+	mutex_unlock(&set->tag_list_lock);
+}
+EXPORT_SYMBOL_GPL(blk_mq_quiesce_tagset);
+
+void blk_mq_unquiesce_tagset(struct blk_mq_tag_set *set)
+{
+	struct request_queue *q;
+
+	mutex_lock(&set->tag_list_lock);
+	list_for_each_entry(q, &set->tag_list, tag_set_list)
+		blk_mq_unquiesce_queue(q);
+	mutex_unlock(&set->tag_list_lock);
+}
+EXPORT_SYMBOL_GPL(blk_mq_unquiesce_tagset);
+
 void blk_mq_wake_waiters(struct request_queue *q)
 {
 	struct blk_mq_hw_ctx *hctx;
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index df642055f02c..90da3582b91d 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -519,6 +519,8 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap);
 void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues);
 
 void blk_mq_quiesce_queue_nowait(struct request_queue *q);
+void blk_mq_quiesce_tagset(struct blk_mq_tag_set *set);
+void blk_mq_unquiesce_tagset(struct blk_mq_tag_set *set);
 
 unsigned int blk_mq_rq_cpu(struct request *rq);
 
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct  6 21:04:43 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed,  7 Oct 2020 09:04:43 +0800
Subject: [PATCH V7 4/4] nvme: use blk_mq_[un]quiesce_tagset
In-Reply-To: <20201007010443.71456-1-ming.lei@redhat.com>
References: <20201007010443.71456-1-ming.lei@redhat.com>
Message-ID: <20201007010443.71456-5-ming.lei@redhat.com>

From: Sagi Grimberg <sagi at grimberg.me>

All controller namespaces share the same tagset, so we
can use this interface which does the optimal operation
for parallel quiesce based on the tagset type (e.g.
blocking tagsets and non-blocking tagsets).

Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>

Add code to unquiesce ctrl->connect_q in nvme_stop_queues(), meantime
avoid to call blk_mq_quiesce_tagset()/blk_mq_unquiesce_tagset() if
this tagset isn't initialized.
Signed-off-by: Ming Lei <ming.lei at redhat.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/core.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c190c56bf702..2dd2d6ff8e7a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4623,23 +4623,22 @@ EXPORT_SYMBOL_GPL(nvme_start_freeze);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
-	struct nvme_ns *ns;
+	if (list_empty_careful(&ctrl->namespaces))
+		return;
 
-	down_read(&ctrl->namespaces_rwsem);
-	list_for_each_entry(ns, &ctrl->namespaces, list)
-		blk_mq_quiesce_queue(ns->queue);
-	up_read(&ctrl->namespaces_rwsem);
+	blk_mq_quiesce_tagset(ctrl->tagset);
+
+	if (ctrl->connect_q)
+		blk_mq_unquiesce_queue(ctrl->connect_q);
 }
 EXPORT_SYMBOL_GPL(nvme_stop_queues);
 
 void nvme_start_queues(struct nvme_ctrl *ctrl)
 {
-	struct nvme_ns *ns;
+	if (list_empty_careful(&ctrl->namespaces))
+		return;
 
-	down_read(&ctrl->namespaces_rwsem);
-	list_for_each_entry(ns, &ctrl->namespaces, list)
-		blk_mq_unquiesce_queue(ns->queue);
-	up_read(&ctrl->namespaces_rwsem);
+	blk_mq_unquiesce_tagset(ctrl->tagset);
 }
 EXPORT_SYMBOL_GPL(nvme_start_queues);
 
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct  6 21:04:39 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed,  7 Oct 2020 09:04:39 +0800
Subject: [PATCH V7 0/4] blk-mq: implement queue quiesce via percpu_ref for
 BLK_MQ_F_BLOCKING
Message-ID: <20201007010443.71456-1-ming.lei@redhat.com>

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce,
and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

The 3rd patch adds tagset quiesce interface.

The 4th patch applies tagset quiesce interface for NVMe subsystem.

V7:
	- base on latest for-5.10/block, only there is small change on 2/4

V6:
	- base on for-5.10/block directly, instead of being against on patchset of
	'percpu_ref & block: reduce memory footprint of percpu_ref in fast path',
	because these patches don't depend on that patchset. 

V5:
	- warn once in case that driver unquiesces its queue being
	  quiesce and not done, only patch 2 is modified

V4:
	- remove .mq_quiesce_mutex, and switch to test_and_[set|clear] for
	avoiding duplicated quiesce action
	- pass blktests(block, nvme)

V3:
	- add tagset quiesce interface
	- apply tagset quiesce interface for NVMe
	- pass blktests(block, nvme)

V2:
	- add .mq_quiesce_lock
	- add comment on patch 2 wrt. handling hctx_lock() failure
	- trivial patch style change


Ming Lei (3):
  block: use test_and_{clear|test}_bit to set/clear QUEUE_FLAG_QUIESCED
  blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING
  blk-mq: add tagset quiesce interface

Sagi Grimberg (1):
  nvme: use blk_mq_[un]quiesce_tagset

 block/blk-core.c         |  13 +++
 block/blk-mq-sysfs.c     |   2 -
 block/blk-mq.c           | 182 +++++++++++++++++++++++++--------------
 block/blk-sysfs.c        |   6 +-
 block/blk.h              |   2 +
 drivers/nvme/host/core.c |  19 ++--
 include/linux/blk-mq.h   |  10 +--
 include/linux/blkdev.h   |   4 +
 8 files changed, 154 insertions(+), 84 deletions(-)

Cc: Hannes Reinecke <hare at suse.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
-- 
2.25.2



From hch at lst.de  Wed Oct  7 01:55:55 2020
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 7 Oct 2020 07:55:55 +0200
Subject: [PATCH] nvme-core: put ctrl ref when module ref get fail
In-Reply-To: <e2c46d78-972b-dd7c-6662-edbf91a095fa@deltatee.com>
 <20201006233647.57760-1-chaitanya.kulkarni@wdc.com>
References: <20201006233647.57760-1-chaitanya.kulkarni@wdc.com>
 <e2c46d78-972b-dd7c-6662-edbf91a095fa@deltatee.com>
 <20201006233647.57760-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201007055555.GA10017@lst.de>

Applied to nvme-5.9.


From hch at infradead.org  Wed Oct  7 10:23:24 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 7 Oct 2020 16:23:24 +0200
Subject: [GIT PULL] nvme fix for 5.9
Message-ID: <20201007142324.GA1458015@infradead.org>

The following changes since commit 6d53a9fe5a1983490bc14b3a64d49fabb4ccc651:

  block/scsi-ioctl: Fix kernel-infoleak in scsi_put_cdrom_generic_arg() (2020-10-02 12:01:47 -0600)

are available in the Git repository at:

  git://git.infradead.org/nvme.git tags/nvme-5.9-2020-10-07

for you to fetch changes up to 4bab69093044ca81f394bd0780be1b71c5a4d308:

  nvme-core: put ctrl ref when module ref get fail (2020-10-07 07:55:40 +0200)

----------------------------------------------------------------
nvme fix for 5.9:

 - fix a recently introduced controller leak (Logan Gunthorpe)

----------------------------------------------------------------
Chaitanya Kulkarni (1):
      nvme-core: put ctrl ref when module ref get fail

 drivers/nvme/host/core.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)


From axboe at kernel.dk  Wed Oct  7 10:25:02 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Wed, 7 Oct 2020 08:25:02 -0600
Subject: [GIT PULL] nvme fix for 5.9
In-Reply-To: <20201007142324.GA1458015@infradead.org>
References: <20201007142324.GA1458015@infradead.org>
Message-ID: <d2e10d2e-66a7-5785-881a-2c1daf310d60@kernel.dk>

On 10/7/20 8:23 AM, Christoph Hellwig wrote:
> The following changes since commit 6d53a9fe5a1983490bc14b3a64d49fabb4ccc651:
> 
>   block/scsi-ioctl: Fix kernel-infoleak in scsi_put_cdrom_generic_arg() (2020-10-02 12:01:47 -0600)
> 
> are available in the Git repository at:
> 
>   git://git.infradead.org/nvme.git tags/nvme-5.9-2020-10-07

Pulled, thanks.

-- 
Jens Axboe



From logang at deltatee.com  Wed Oct  7 11:53:44 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Wed, 7 Oct 2020 09:53:44 -0600
Subject: [PATCH blktests v2 03/11] common/xfs: Create common helper to
 verify block device with xfs
In-Reply-To: <BYAPR04MB49652338A4FE3805F9394A88860A0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-4-logang@deltatee.com>
 <BYAPR04MB4965C2DDD4A5FBD86A128FE9860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
 <868f8fb6-6024-d60d-a9aa-6513b9d0986f@deltatee.com>
 <BYAPR04MB49652338A4FE3805F9394A88860A0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <2af98b64-1f3b-6570-d5be-a2e2bc4950f9@deltatee.com>



On 2020-10-06 6:20 p.m., Chaitanya Kulkarni wrote:
> On 10/6/20 16:59, Logan Gunthorpe wrote:
>>> The mount dir should be a parameter and not the hardcode value
>>> to make it reusable.
>> I also disagree here. It is already reusable and is used in a number of
>> places; none of those places require changing the mount directory. If
>> and when a use comes up that requires a different directory (not sure
>> what that would be), a parameter can be added. It is typically standard
>> practice in the Linux community to not add features that have no users
>> as it's confusing to people reading the code.
>>
>> Logan
>>
> Well if you are making a helper it should be generic if you have a usecase,

"Generic" isn't a binary yes/no quality. Why add the mount option (that nobody is using) 
and not a size option as well that nobody uses? For that matter, fio has a ton of options
we could expose. (think io-method, read/write pattern, etc, etc). The criteria we
decide upon which options get exposed as arguments is what the code that's actually
using it needs -- not what's available or what you imagine future use cases might be.
If there are no users in the code it should not be exposed. If a use case comes along,
an argument can easily be added when the new test is added to the code base.

> mounted on different mount points not just one which is important testcase,
> 
> that will require a prep patch.

So? That's normal.
 
> Why can't we do that right now when we have a clear usecase ?

We don't have a clear use case that's being added to the code though... We 
have an imagined use case that hasn't been written. Add the feature when you
add this use case.

Logan


From logang at deltatee.com  Wed Oct  7 11:55:17 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Wed, 7 Oct 2020 09:55:17 -0600
Subject: [PATCH blktests v2 04/11] nvme: Search for specific subsysnqn in
 _find_nvme_loop_dev
In-Reply-To: <BYAPR04MB49650C6419A84705D04FFE63860A0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20200930185422.11494-1-logang@deltatee.com>
 <20200930185422.11494-5-logang@deltatee.com>
 <BYAPR04MB49657A481271ADC1F576377B860D0@BYAPR04MB4965.namprd04.prod.outlook.com>
 <517a5fd9-50eb-eb5e-5911-093dc5d1a759@deltatee.com>
 <BYAPR04MB49650C6419A84705D04FFE63860A0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <39ab50dd-b117-2e53-a7ca-b390ade7dd1e@deltatee.com>



On 2020-10-06 6:24 p.m., Chaitanya Kulkarni wrote:
> On 10/6/20 17:10, Logan Gunthorpe wrote:
>>> With this patch or this series will I be able to write the testcase ?
>> This patch helps with that but other helpers introduced in this series
>> would require minor changes.
>>
>> As far as I can see, you'd only have to adjust _create_nvmet_passthru()
>> to take an optional argument because, presently, it always uses
>> $_test_dev_nvme_ctrl for the backing device.
>>
>> This can easily be done if and when someone writes such a test.
>>
>> However, I'm not even sure right now if that test would pass in the
>> kernel as is -- it seems like an odd thing to do.
>>
>> Logan
>>
> This test should pass if I remember the code correctly where we don't
> 
> have any PCIe specific checks for the passthru controller and it is an

Yes, there's no explicit restrictions, but that doesn't mean there are no bugs
with that particular stack.

> important to support this scenario in order to write device independent
> 
> testcases as rest of the testcases are.

Ok, feel free to write a test for this. It's not important to me.

Logan



From lkp at intel.com  Wed Oct  7 14:03:31 2020
From: lkp at intel.com (kernel test robot)
Date: Thu, 08 Oct 2020 02:03:31 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 c4485252cf36ae62c8bf12c4aede72345cad0d2b
Message-ID: <5f7e02f3.NRSBvOBpGHg5ERvO%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: c4485252cf36ae62c8bf12c4aede72345cad0d2b  nvme-core: remove extra condition for vwc

elapsed time: 723m

configs tested: 231
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
arm                  colibri_pxa300_defconfig
powerpc                    mvme5100_defconfig
sh                             espt_defconfig
powerpc                     mpc512x_defconfig
powerpc                     rainier_defconfig
sh                        apsh4ad0a_defconfig
arm                          ixp4xx_defconfig
mips                        bcm63xx_defconfig
mips                           ip28_defconfig
mips                        omega2p_defconfig
arc                             nps_defconfig
mips                      pistachio_defconfig
sh                           se7721_defconfig
arm                           stm32_defconfig
csky                             alldefconfig
arm                       versatile_defconfig
m68k                        mvme147_defconfig
arm                       aspeed_g4_defconfig
xtensa                    xip_kc705_defconfig
powerpc                      arches_defconfig
arm                         palmz72_defconfig
powerpc                      ep88xc_defconfig
powerpc                  mpc885_ads_defconfig
openrisc                         alldefconfig
arm                         at91_dt_defconfig
arm                         s5pv210_defconfig
powerpc                     mpc5200_defconfig
arm                            zeus_defconfig
powerpc                 mpc834x_itx_defconfig
powerpc                      obs600_defconfig
powerpc                 mpc8313_rdb_defconfig
sh                     sh7710voipgw_defconfig
sh                   sh7724_generic_defconfig
arm64                            alldefconfig
powerpc                          g5_defconfig
arm                           sunxi_defconfig
arm                       imx_v4_v5_defconfig
powerpc                    adder875_defconfig
m68k                       bvme6000_defconfig
powerpc                        warp_defconfig
microblaze                      mmu_defconfig
arm                          pxa168_defconfig
arm                         hackkit_defconfig
mips                 pnx8335_stb225_defconfig
arm                         s3c6400_defconfig
sh                           se7619_defconfig
mips                 decstation_r4k_defconfig
arm                          moxart_defconfig
sh                          urquell_defconfig
arm                            mmp2_defconfig
m68k                         amcore_defconfig
sparc64                          alldefconfig
powerpc                 mpc8560_ads_defconfig
arc                           tb10x_defconfig
powerpc                         wii_defconfig
riscv                             allnoconfig
h8300                            alldefconfig
mips                        workpad_defconfig
powerpc               mpc834x_itxgp_defconfig
arm                           h3600_defconfig
arm                              zx_defconfig
powerpc                 canyonlands_defconfig
arm                       imx_v6_v7_defconfig
arm                        mvebu_v5_defconfig
powerpc                     tqm8555_defconfig
mips                            e55_defconfig
parisc                           alldefconfig
arm                            dove_defconfig
powerpc                 mpc837x_mds_defconfig
sh                            migor_defconfig
arc                          axs101_defconfig
arm                         lpc18xx_defconfig
alpha                            allyesconfig
arm                          exynos_defconfig
powerpc                     ppa8548_defconfig
h8300                     edosk2674_defconfig
parisc                generic-64bit_defconfig
sh                   secureedge5410_defconfig
mips                           ip32_defconfig
h8300                       h8s-sim_defconfig
powerpc                    ge_imp3a_defconfig
mips                      malta_kvm_defconfig
parisc                generic-32bit_defconfig
sh                           se7705_defconfig
sh                               alldefconfig
arm                      footbridge_defconfig
c6x                        evmc6472_defconfig
sh                         ecovec24_defconfig
arm                            qcom_defconfig
arm                             rpc_defconfig
c6x                              alldefconfig
arm                      integrator_defconfig
sh                        sh7757lcr_defconfig
powerpc                     redwood_defconfig
sh                          rsk7269_defconfig
arm                              alldefconfig
powerpc                 xes_mpc85xx_defconfig
sh                          sdk7786_defconfig
mips                         mpc30x_defconfig
powerpc                     mpc83xx_defconfig
arm                           corgi_defconfig
m68k                          amiga_defconfig
mips                     cu1000-neo_defconfig
arm                    vt8500_v6_v7_defconfig
s390                             allyesconfig
s390                       zfcpdump_defconfig
powerpc                       ebony_defconfig
mips                         bigsur_defconfig
arm                           h5000_defconfig
powerpc                 mpc836x_mds_defconfig
powerpc                 mpc837x_rdb_defconfig
powerpc                 mpc832x_rdb_defconfig
arm                       aspeed_g5_defconfig
arm                         nhk8815_defconfig
sh                           se7722_defconfig
mips                    maltaup_xpa_defconfig
mips                      loongson3_defconfig
arm                        clps711x_defconfig
arm                          ep93xx_defconfig
h8300                               defconfig
sh                              ul2_defconfig
sh                          r7780mp_defconfig
microblaze                          defconfig
mips                           ip22_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                                defconfig
parisc                           allyesconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
x86_64               randconfig-a004-20201006
x86_64               randconfig-a002-20201006
x86_64               randconfig-a001-20201006
x86_64               randconfig-a005-20201006
x86_64               randconfig-a003-20201006
x86_64               randconfig-a006-20201006
i386                 randconfig-a006-20201005
i386                 randconfig-a005-20201005
i386                 randconfig-a001-20201005
i386                 randconfig-a004-20201005
i386                 randconfig-a003-20201005
i386                 randconfig-a002-20201005
i386                 randconfig-a006-20201007
i386                 randconfig-a005-20201007
i386                 randconfig-a001-20201007
i386                 randconfig-a004-20201007
i386                 randconfig-a003-20201007
i386                 randconfig-a002-20201007
x86_64               randconfig-a012-20201005
x86_64               randconfig-a015-20201005
x86_64               randconfig-a014-20201005
x86_64               randconfig-a013-20201005
x86_64               randconfig-a011-20201005
x86_64               randconfig-a016-20201005
x86_64               randconfig-a012-20201007
x86_64               randconfig-a015-20201007
x86_64               randconfig-a014-20201007
x86_64               randconfig-a013-20201007
x86_64               randconfig-a011-20201007
x86_64               randconfig-a016-20201007
i386                 randconfig-a015-20201008
i386                 randconfig-a013-20201008
i386                 randconfig-a014-20201008
i386                 randconfig-a016-20201008
i386                 randconfig-a011-20201008
i386                 randconfig-a012-20201008
i386                 randconfig-a014-20201005
i386                 randconfig-a015-20201005
i386                 randconfig-a013-20201005
i386                 randconfig-a016-20201005
i386                 randconfig-a011-20201005
i386                 randconfig-a012-20201005
i386                 randconfig-a014-20201007
i386                 randconfig-a013-20201007
i386                 randconfig-a015-20201007
i386                 randconfig-a016-20201007
i386                 randconfig-a011-20201007
i386                 randconfig-a012-20201007
riscv                    nommu_k210_defconfig
riscv                    nommu_virt_defconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allyesconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a004-20201007
x86_64               randconfig-a002-20201007
x86_64               randconfig-a001-20201007
x86_64               randconfig-a005-20201007
x86_64               randconfig-a003-20201007
x86_64               randconfig-a006-20201007
x86_64               randconfig-a004-20201005
x86_64               randconfig-a002-20201005
x86_64               randconfig-a001-20201005
x86_64               randconfig-a003-20201005
x86_64               randconfig-a005-20201005
x86_64               randconfig-a006-20201005

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From lkp at intel.com  Wed Oct  7 14:03:33 2020
From: lkp at intel.com (kernel test robot)
Date: Thu, 08 Oct 2020 02:03:33 +0800
Subject: [linux-nvme:nvme-5.9] BUILD SUCCESS
 4bab69093044ca81f394bd0780be1b71c5a4d308
Message-ID: <5f7e02f5.OzK0yC0EyuIb1dEF%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.9
branch HEAD: 4bab69093044ca81f394bd0780be1b71c5a4d308  nvme-core: put ctrl ref when module ref get fail

elapsed time: 723m

configs tested: 167
configs skipped: 3

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
mips                           ip28_defconfig
mips                        omega2p_defconfig
arc                             nps_defconfig
mips                      pistachio_defconfig
sh                           se7721_defconfig
arm                           stm32_defconfig
csky                             alldefconfig
arm                       versatile_defconfig
m68k                        mvme147_defconfig
arm                       aspeed_g4_defconfig
powerpc                      ep88xc_defconfig
xtensa                    xip_kc705_defconfig
powerpc                      arches_defconfig
arm                         palmz72_defconfig
powerpc                     mpc5200_defconfig
arm                            zeus_defconfig
powerpc                 mpc834x_itx_defconfig
powerpc                      obs600_defconfig
powerpc                 mpc8313_rdb_defconfig
sh                     sh7710voipgw_defconfig
sh                   sh7724_generic_defconfig
arm64                            alldefconfig
powerpc                          g5_defconfig
arm                           sunxi_defconfig
arm                       imx_v4_v5_defconfig
powerpc                    adder875_defconfig
m68k                       bvme6000_defconfig
powerpc                        warp_defconfig
microblaze                      mmu_defconfig
arm                          pxa168_defconfig
arm                         hackkit_defconfig
mips                 pnx8335_stb225_defconfig
arm                         s3c6400_defconfig
sh                           se7619_defconfig
mips                 decstation_r4k_defconfig
arm                          moxart_defconfig
sh                          urquell_defconfig
arm                            mmp2_defconfig
m68k                         amcore_defconfig
sparc64                          alldefconfig
powerpc                 mpc8560_ads_defconfig
arc                           tb10x_defconfig
powerpc                         wii_defconfig
riscv                             allnoconfig
h8300                            alldefconfig
parisc                           alldefconfig
arm                            dove_defconfig
powerpc                 mpc837x_mds_defconfig
sh                            migor_defconfig
h8300                       h8s-sim_defconfig
powerpc                    ge_imp3a_defconfig
mips                      malta_kvm_defconfig
parisc                generic-32bit_defconfig
sh                           se7705_defconfig
sh                               alldefconfig
arm                      footbridge_defconfig
c6x                        evmc6472_defconfig
sh                         ecovec24_defconfig
nios2                            alldefconfig
m68k                       m5208evb_defconfig
powerpc                       eiger_defconfig
mips                         tb0226_defconfig
mips                        nlm_xlr_defconfig
powerpc                     redwood_defconfig
sh                          rsk7269_defconfig
arm                              alldefconfig
arm                           corgi_defconfig
m68k                          amiga_defconfig
mips                     cu1000-neo_defconfig
alpha                            allyesconfig
arm                    vt8500_v6_v7_defconfig
s390                             allyesconfig
sh                             espt_defconfig
powerpc                 mpc836x_mds_defconfig
powerpc                 mpc837x_rdb_defconfig
sh                           se7722_defconfig
mips                    maltaup_xpa_defconfig
mips                      loongson3_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a006-20201005
i386                 randconfig-a005-20201005
i386                 randconfig-a001-20201005
i386                 randconfig-a004-20201005
i386                 randconfig-a003-20201005
i386                 randconfig-a002-20201005
i386                 randconfig-a006-20201007
i386                 randconfig-a005-20201007
i386                 randconfig-a001-20201007
i386                 randconfig-a004-20201007
i386                 randconfig-a003-20201007
i386                 randconfig-a002-20201007
x86_64               randconfig-a012-20201005
x86_64               randconfig-a015-20201005
x86_64               randconfig-a014-20201005
x86_64               randconfig-a013-20201005
x86_64               randconfig-a011-20201005
x86_64               randconfig-a016-20201005
x86_64               randconfig-a012-20201007
x86_64               randconfig-a015-20201007
x86_64               randconfig-a014-20201007
x86_64               randconfig-a013-20201007
x86_64               randconfig-a011-20201007
x86_64               randconfig-a016-20201007
i386                 randconfig-a014-20201005
i386                 randconfig-a015-20201005
i386                 randconfig-a013-20201005
i386                 randconfig-a016-20201005
i386                 randconfig-a011-20201005
i386                 randconfig-a012-20201005
i386                 randconfig-a014-20201007
i386                 randconfig-a013-20201007
i386                 randconfig-a015-20201007
i386                 randconfig-a016-20201007
i386                 randconfig-a011-20201007
i386                 randconfig-a012-20201007
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a004-20201007
x86_64               randconfig-a002-20201007
x86_64               randconfig-a001-20201007
x86_64               randconfig-a005-20201007
x86_64               randconfig-a003-20201007
x86_64               randconfig-a006-20201007

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From yi.zhang at redhat.com  Thu Oct  8 06:16:42 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Thu, 8 Oct 2020 06:16:42 -0400 (EDT)
Subject: [bug report] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28
 refcount_warn_saturate+0xa6/0xf0
In-Reply-To: <1992220618.3314096.1602151485520.JavaMail.zimbra@redhat.com>
Message-ID: <1934331639.3314730.1602152202454.JavaMail.zimbra@redhat.com>

Hello

I saw this failure[2] on both stable and block/for-next recently, seems it was introduced from[1], pls help check it, thanks.


[1]
commit 236187c4ed195161dfa4237c7beffbba0c5ae45b
Author: Sagi Grimberg <sagi at grimberg.me>
Date:   Tue Jul 28 13:16:36 2020 -0700

    nvme-tcp: fix timeout handler


[2]
#nvme_trtype=tcp ./check nvme/012

[ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
[ 2152.799640] loop: module loaded
[ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[ 2152.937889] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[ 2152.952085] nvme nvme0: creating 12 I/O queues.
[ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
[ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
[ 2154.942432] XFS (nvme0n1): Ending clean mount
[ 2154.948183] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
[ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
[ 2215.199331] nvme nvme0: starting error recovery
[ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
[ 2215.204483] block nvme0n1: no usable path - requeuing I/O
[ 2215.214976] block nvme0n1: no usable path - requeuing I/O
[ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
[ 2215.215502] ------------[ cut here ]------------
[ 2215.215505] refcount_t: underflow; use-after-free.
[ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
[ 2215.215619] Modules linked in: loop nvme_tcp nvme_fabrics nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_hda_codec snd_hda_core snd_hwdep hp_wmi amd64_edac_mod edac_mce_amd kvm_amd ccp kvm irqbypass k10temp sparse_keymap rfkill pcspkr nv_tco wmi_bmof snd_pcm snd_timer snd soundcore i2c_nforce2 acpi_cpufreq ip_tables xfs nouveau video mxm_wmi i2c_algo_bit ata_generic drm_kms_helper cec mptsas ttm scsi_transport_sas firewire_ohci mptscsih pata_acpi drm serio_raw mptbase firewire_core forcedeth crc_itu_t sata_nv pata_amd wmi floppy
[ 2215.215710] CPU: 6 PID: 45 Comm: kworker/6:0H Not tainted 5.9.0-rc8+ #1
[ 2215.215713] Hardware name: Hewlett-Packard HP xw9400 Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
[ 2215.215721] Workqueue: kblockd blk_mq_timeout_work
[ 2215.215736] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
[ 2215.215742] Code: 05 ca 98 22 01 01 e8 bf cf ad ff 0f 0b c3 80 3d b8 98 22 01 00 75 95 48 c7 c7 d8 53 3e a7 c6 05 a8 98 22 01 01 e8 a0 cf ad ff <0f> 0b c3 80 3d 97 98 22 01 00 0f 85 72 ff ff ff 48 c7 c7 30 54 3e
[ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
[ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff93f37dcd8d08
[ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff93f37dcd8d00
[ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: ffffffffa7fffbc4
[ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: ffff93f37a8124e8
[ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: ffff93f37a2b0000
[ 2215.215765] FS:  0000000000000000(0000) GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
[ 2215.215768] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 00000000000006e0
[ 2215.215773] Call Trace:
[ 2215.215784]  blk_mq_check_expired+0x109/0x1b0
[ 2215.215797]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
[ 2215.215801]  ? blk_poll+0x300/0x300
[ 2215.215806]  blk_mq_timeout_work+0x44/0xe0
[ 2215.215814]  process_one_work+0x1b4/0x370
[ 2215.215820]  worker_thread+0x53/0x3e0
[ 2215.215825]  ? process_one_work+0x370/0x370
[ 2215.215829]  kthread+0x11b/0x140
[ 2215.215834]  ? __kthread_bind_mask+0x60/0x60
[ 2215.215841]  ret_from_fork+0x22/0x30
[ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
[ 2215.220371] block nvme0n1: no usable path - requeuing I/O
[ 2215.220377] block nvme0n1: no usable path - requeuing I/O
[ 2215.466666] block nvme0n1: no usable path - requeuing I/O
[ 2215.472056] block nvme0n1: no usable path - requeuing I/O
[ 2215.477444] block nvme0n1: no usable path - requeuing I/O
[ 2215.482828] block nvme0n1: no usable path - requeuing I/O
[ 2215.488219] block nvme0n1: no usable path - requeuing I/O
[ 2215.493625] block nvme0n1: no usable path - requeuing I/O
[ 2240.794172] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
[ 2240.800189] nvmet: ctrl 2 fatal error occurred!
[ 2245.402268] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[ 2245.408286] nvmet: ctrl 1 fatal error occurred!
[ 2262.862528] nvmet: creating controller 2 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[ 2288.411172] nvme nvme0: queue 0: timeout request 0x0 type 4
[ 2288.416794] nvme nvme0: Connect command failed, error wo/DNR bit: 881
[ 2288.423254] nvme nvme0: failed to connect queue: 0 ret=881
[ 2288.428773] nvme nvme0: Failed reconnect attempt 1
[ 2288.433568] nvme nvme0: Reconnecting in 10 seconds...
[ 2298.651953] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[ 2298.665918] nvme_ns_head_submit_bio: 30 callbacks suppressed
[ 2298.665926] block nvme0n1: no usable path - requeuing I/O
[ 2298.676993] block nvme0n1: no usable path - requeuing I/O
[ 2298.682393] block nvme0n1: no usable path - requeuing I/O
[ 2298.687791] block nvme0n1: no usable path - requeuing I/O
[ 2298.693202] block nvme0n1: no usable path - requeuing I/O
[ 2298.698593] block nvme0n1: no usable path - requeuing I/O
[ 2298.703988] block nvme0n1: no usable path - requeuing I/O
[ 2298.709375] block nvme0n1: no usable path - requeuing I/O
[ 2298.714758] block nvme0n1: no usable path - requeuing I/O
[ 2298.720152] block nvme0n1: no usable path - requeuing I/O
[ 2298.725691] nvme nvme0: creating 12 I/O queues.
[ 2298.737066] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[ 2298.745612] nvme nvme0: Successfully reconnected (2 attempt)



Best Regards,
  Yi Zhang




From hch at infradead.org  Thu Oct  8 10:10:41 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 8 Oct 2020 16:10:41 +0200
Subject: [GIT PULL] nvme updates for 5.10
Message-ID: <20201008141041.GA1493538@infradead.org>

The following changes since commit 103fbf8e4020845e4fcf63819288cedb092a3c91:

  scsi: megaraid_sas: Added support for shared host tagset for cpuhotplug (2020-10-06 08:33:44 -0600)

are available in the Git repository at:

  git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-08

for you to fetch changes up to c4485252cf36ae62c8bf12c4aede72345cad0d2b:

  nvme-core: remove extra condition for vwc (2020-10-07 07:56:20 +0200)

----------------------------------------------------------------
nvme update for 5.8:

 - fix a controller refcount leak on init failure (Chaitanya Kulkarni)
 - misc cleanups (Chaitanya Kulkarni)
 - major refactoring of the scanning code (me)

----------------------------------------------------------------
Chaitanya Kulkarni (3):
      nvme-loop: don't put ctrl on nvme_init_ctrl error
      nvme-core: remove extra variable
      nvme-core: remove extra condition for vwc

Christoph Hellwig (20):
      block: optimize blk_queue_zoned_model for !CONFIG_BLK_DEV_ZONED
      nvme: fix initialization of the zone bitmaps
      nvme: remove the disk argument to nvme_update_zone_info
      nvme: rename nvme_validate_ns to nvme_validate_or_alloc_ns
      nvme: rename _nvme_revalidate_disk
      nvme: rename __nvme_revalidate_disk
      nvme: lift the check for an unallocated namespace into nvme_identify_ns
      nvme: call nvme_identify_ns as the first thing in nvme_alloc_ns_block
      nvme: factor out a nvme_configure_metadata helper
      nvme: freeze the queue over ->lba_shift updates
      nvme: clean up the check for too large logic block sizes
      nvme: remove the 0 lba_shift check in nvme_update_ns_info
      nvme: set the queue limits in nvme_update_ns_info
      nvme: update the known admin effects
      nvme: remove nvme_update_formats
      nvme: revalidate zone bitmaps in nvme_update_ns_info
      nvme: query namespace identifiers before adding the namespace
      nvme: move nvme_validate_ns
      nvme: refactor nvme_validate_ns
      nvme: remove nvme_identify_ns_list

 drivers/nvme/host/core.c   | 455 ++++++++++++++++++++-------------------------
 drivers/nvme/host/nvme.h   |   9 +-
 drivers/nvme/host/zns.c    |  16 +-
 drivers/nvme/target/loop.c |   4 +-
 include/linux/blkdev.h     |   4 +-
 5 files changed, 220 insertions(+), 268 deletions(-)


From axboe at kernel.dk  Thu Oct  8 10:52:50 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 8 Oct 2020 08:52:50 -0600
Subject: [GIT PULL] nvme updates for 5.10
In-Reply-To: <20201008141041.GA1493538@infradead.org>
References: <20201008141041.GA1493538@infradead.org>
Message-ID: <2b4ad1d6-3d77-d14e-2275-0f9326b19514@kernel.dk>

On 10/8/20 8:10 AM, Christoph Hellwig wrote:
> The following changes since commit 103fbf8e4020845e4fcf63819288cedb092a3c91:
> 
>   scsi: megaraid_sas: Added support for shared host tagset for cpuhotplug (2020-10-06 08:33:44 -0600)
> 
> are available in the Git repository at:
> 
>   git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-08
> 
> for you to fetch changes up to c4485252cf36ae62c8bf12c4aede72345cad0d2b:
> 
>   nvme-core: remove extra condition for vwc (2020-10-07 07:56:20 +0200)
> 
> ----------------------------------------------------------------
> nvme update for 5.8:
> 
>  - fix a controller refcount leak on init failure (Chaitanya Kulkarni)
>  - misc cleanups (Chaitanya Kulkarni)
>  - major refactoring of the scanning code (me)

Seems to be a typo here, 5.8?

-- 
Jens Axboe



From hch at infradead.org  Thu Oct  8 10:56:14 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 8 Oct 2020 15:56:14 +0100
Subject: [GIT PULL] nvme updates for 5.10
In-Reply-To: <2b4ad1d6-3d77-d14e-2275-0f9326b19514@kernel.dk>
References: <20201008141041.GA1493538@infradead.org>
 <2b4ad1d6-3d77-d14e-2275-0f9326b19514@kernel.dk>
Message-ID: <20201008145614.GA23622@infradead.org>

On Thu, Oct 08, 2020 at 08:52:50AM -0600, Jens Axboe wrote:
> On 10/8/20 8:10 AM, Christoph Hellwig wrote:
> > The following changes since commit 103fbf8e4020845e4fcf63819288cedb092a3c91:
> > 
> >   scsi: megaraid_sas: Added support for shared host tagset for cpuhotplug (2020-10-06 08:33:44 -0600)
> > 
> > are available in the Git repository at:
> > 
> >   git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-08
> > 
> > for you to fetch changes up to c4485252cf36ae62c8bf12c4aede72345cad0d2b:
> > 
> >   nvme-core: remove extra condition for vwc (2020-10-07 07:56:20 +0200)
> > 
> > ----------------------------------------------------------------
> > nvme update for 5.8:
> > 
> >  - fix a controller refcount leak on init failure (Chaitanya Kulkarni)
> >  - misc cleanups (Chaitanya Kulkarni)
> >  - major refactoring of the scanning code (me)
> 
> Seems to be a typo here, 5.8?

should be 5.10 of course.

Do you want a resend as this is also in the tag?


From axboe at kernel.dk  Thu Oct  8 10:57:26 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 8 Oct 2020 08:57:26 -0600
Subject: [GIT PULL] nvme updates for 5.10
In-Reply-To: <20201008145614.GA23622@infradead.org>
References: <20201008141041.GA1493538@infradead.org>
 <2b4ad1d6-3d77-d14e-2275-0f9326b19514@kernel.dk>
 <20201008145614.GA23622@infradead.org>
Message-ID: <6c77e5f5-0316-9893-1c8a-8fe25bbbb506@kernel.dk>

On 10/8/20 8:56 AM, Christoph Hellwig wrote:
> On Thu, Oct 08, 2020 at 08:52:50AM -0600, Jens Axboe wrote:
>> On 10/8/20 8:10 AM, Christoph Hellwig wrote:
>>> The following changes since commit 103fbf8e4020845e4fcf63819288cedb092a3c91:
>>>
>>>   scsi: megaraid_sas: Added support for shared host tagset for cpuhotplug (2020-10-06 08:33:44 -0600)
>>>
>>> are available in the Git repository at:
>>>
>>>   git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-08
>>>
>>> for you to fetch changes up to c4485252cf36ae62c8bf12c4aede72345cad0d2b:
>>>
>>>   nvme-core: remove extra condition for vwc (2020-10-07 07:56:20 +0200)
>>>
>>> ----------------------------------------------------------------
>>> nvme update for 5.8:
>>>
>>>  - fix a controller refcount leak on init failure (Chaitanya Kulkarni)
>>>  - misc cleanups (Chaitanya Kulkarni)
>>>  - major refactoring of the scanning code (me)
>>
>> Seems to be a typo here, 5.8?
> 
> should be 5.10 of course.
> 
> Do you want a resend as this is also in the tag?

Yeah I think that'd be best, thanks.

-- 
Jens Axboe



From hch at infradead.org  Thu Oct  8 11:01:25 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 8 Oct 2020 17:01:25 +0200
Subject: [GIT PULL] nvme updates for 5.10
Message-ID: <20201008150125.GA1495381@infradead.org>

The following changes since commit 103fbf8e4020845e4fcf63819288cedb092a3c91:

  scsi: megaraid_sas: Added support for shared host tagset for cpuhotplug (2020-10-06 08:33:44 -0600)

are available in the Git repository at:

  git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-08

for you to fetch changes up to c4485252cf36ae62c8bf12c4aede72345cad0d2b:

  nvme-core: remove extra condition for vwc (2020-10-07 07:56:20 +0200)

----------------------------------------------------------------
nvme update for 5.10:

 - fix a controller refcount leak on init failure (Chaitanya Kulkarni)
 - misc cleanups (Chaitanya Kulkarni)
 - major refactoring of the scanning code (me)

----------------------------------------------------------------
Chaitanya Kulkarni (3):
      nvme-loop: don't put ctrl on nvme_init_ctrl error
      nvme-core: remove extra variable
      nvme-core: remove extra condition for vwc

Christoph Hellwig (20):
      block: optimize blk_queue_zoned_model for !CONFIG_BLK_DEV_ZONED
      nvme: fix initialization of the zone bitmaps
      nvme: remove the disk argument to nvme_update_zone_info
      nvme: rename nvme_validate_ns to nvme_validate_or_alloc_ns
      nvme: rename _nvme_revalidate_disk
      nvme: rename __nvme_revalidate_disk
      nvme: lift the check for an unallocated namespace into nvme_identify_ns
      nvme: call nvme_identify_ns as the first thing in nvme_alloc_ns_block
      nvme: factor out a nvme_configure_metadata helper
      nvme: freeze the queue over ->lba_shift updates
      nvme: clean up the check for too large logic block sizes
      nvme: remove the 0 lba_shift check in nvme_update_ns_info
      nvme: set the queue limits in nvme_update_ns_info
      nvme: update the known admin effects
      nvme: remove nvme_update_formats
      nvme: revalidate zone bitmaps in nvme_update_ns_info
      nvme: query namespace identifiers before adding the namespace
      nvme: move nvme_validate_ns
      nvme: refactor nvme_validate_ns
      nvme: remove nvme_identify_ns_list

 drivers/nvme/host/core.c   | 455 ++++++++++++++++++++-------------------------
 drivers/nvme/host/nvme.h   |   9 +-
 drivers/nvme/host/zns.c    |  16 +-
 drivers/nvme/target/loop.c |   4 +-
 include/linux/blkdev.h     |   4 +-
 5 files changed, 220 insertions(+), 268 deletions(-)


From axboe at kernel.dk  Thu Oct  8 11:48:09 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 8 Oct 2020 09:48:09 -0600
Subject: [GIT PULL] nvme updates for 5.10
In-Reply-To: <20201008150125.GA1495381@infradead.org>
References: <20201008150125.GA1495381@infradead.org>
Message-ID: <78afe105-d7c6-997c-e4e6-13a6e3f84e7f@kernel.dk>

On 10/8/20 9:01 AM, Christoph Hellwig wrote:
> The following changes since commit 103fbf8e4020845e4fcf63819288cedb092a3c91:
> 
>   scsi: megaraid_sas: Added support for shared host tagset for cpuhotplug (2020-10-06 08:33:44 -0600)
> 
> are available in the Git repository at:
> 
>   git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-08

Pulled, thanks.

-- 
Jens Axboe



From logang at deltatee.com  Thu Oct  8 12:40:13 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:13 -0600
Subject: [PATCH blktests v3 00/11] NVMe Target Passthru Block Tests
Message-ID: <20201008164024.12546-1-logang@deltatee.com>

Hi,

This series adds blktests for the nvmet passthru feature that was merged
for 5.9. It's been reconciled with Sagi's blktest series that Omar
recently merged.

This series is based off of the current blktests master and a git repo is
available for this here:

https://github.com/Eideticom/blktests nvmet_passthru_v3

Thanks,

Logan

--

Changes in v3:
- Fixed a nit with variable initialization in patch 5 (per Chaitanya)
- Replaced use of printf with echo (per Chaitanya)

Changes in v2:

- Rebased on latest blktests master and changed to use the common
  helpers Sagi introduced in his series
- Collected Chaitanya's reviewed-by tag

--


Logan Gunthorpe (11):
  common/fio: Remove state file in common helper
  common/xfs: Create common helper to check for XFS support
  common/xfs: Create common helper to verify block device with xfs
  nvme: Search for specific subsysnqn in _find_nvme_loop_dev
  nvme: Add common helpers for passthru tests
  nvme/033: Simple test to create and connect to a passthru target
  nvme/034: Add test for passthru data verification
  nvme/035: Add test to verify passthru controller with a filesystem
  nvme/036: Add test for testing reset command on nvme-passthru
  nvme/037: Add test which loops passthru connect and disconnect
  nvme/038: Test removal of un-enabled subsystem and ports

 common/fio         |  1 +
 common/rc          |  8 +++++
 common/xfs         | 33 +++++++++++++++++++
 tests/nvme/004     |  2 +-
 tests/nvme/005     |  2 +-
 tests/nvme/008     |  2 +-
 tests/nvme/009     |  2 +-
 tests/nvme/010     |  3 +-
 tests/nvme/011     |  3 +-
 tests/nvme/012     | 23 ++++---------
 tests/nvme/013     | 21 +++---------
 tests/nvme/014     |  2 +-
 tests/nvme/015     |  2 +-
 tests/nvme/018     |  2 +-
 tests/nvme/019     |  2 +-
 tests/nvme/020     |  2 +-
 tests/nvme/021     |  2 +-
 tests/nvme/022     |  2 +-
 tests/nvme/023     |  2 +-
 tests/nvme/024     |  2 +-
 tests/nvme/025     |  2 +-
 tests/nvme/026     |  2 +-
 tests/nvme/027     |  2 +-
 tests/nvme/028     |  2 +-
 tests/nvme/029     |  2 +-
 tests/nvme/033     | 67 +++++++++++++++++++++++++++++++++++++
 tests/nvme/033.out |  7 ++++
 tests/nvme/034     | 35 ++++++++++++++++++++
 tests/nvme/034.out |  3 ++
 tests/nvme/035     | 37 +++++++++++++++++++++
 tests/nvme/035.out |  3 ++
 tests/nvme/036     | 37 +++++++++++++++++++++
 tests/nvme/036.out |  3 ++
 tests/nvme/037     | 35 ++++++++++++++++++++
 tests/nvme/037.out |  2 ++
 tests/nvme/038     | 36 ++++++++++++++++++++
 tests/nvme/038.out |  2 ++
 tests/nvme/rc      | 82 ++++++++++++++++++++++++++++++++++++++++++++--
 38 files changed, 419 insertions(+), 58 deletions(-)
 create mode 100644 common/xfs
 create mode 100755 tests/nvme/033
 create mode 100644 tests/nvme/033.out
 create mode 100755 tests/nvme/034
 create mode 100644 tests/nvme/034.out
 create mode 100755 tests/nvme/035
 create mode 100644 tests/nvme/035.out
 create mode 100755 tests/nvme/036
 create mode 100644 tests/nvme/036.out
 create mode 100755 tests/nvme/037
 create mode 100644 tests/nvme/037.out
 create mode 100755 tests/nvme/038
 create mode 100644 tests/nvme/038.out


base-commit: 20445c5eb6456addca9131ec6917d2a2d7414e04
--
2.20.1


From logang at deltatee.com  Thu Oct  8 12:40:15 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:15 -0600
Subject: [PATCH blktests v3 02/11] common/xfs: Create common helper to check
 for XFS support
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-3-logang@deltatee.com>

Two nvme tests create and mount XFS filesystems and check for mkfs.xfs.

They should also check for XFS support in the kernel so create a common
helper for this.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 common/rc      |  8 ++++++++
 common/xfs     | 11 +++++++++++
 tests/nvme/012 |  6 ++++--
 tests/nvme/013 |  4 +++-
 4 files changed, 26 insertions(+), 3 deletions(-)
 create mode 100644 common/xfs

diff --git a/common/rc b/common/rc
index cdc0150ea5ea..44cb218c2fac 100644
--- a/common/rc
+++ b/common/rc
@@ -181,6 +181,14 @@ _have_tracepoint() {
 	return 0
 }
 
+_have_fs() {
+	modprobe "$1" >/dev/null 2>&1
+	if [[ ! -d "/sys/fs/$1" ]]; then
+		SKIP_REASON="kernel does not support filesystem $1"
+		return 1
+	fi
+}
+
 _test_dev_is_rotational() {
 	[[ $(cat "${TEST_DEV_SYSFS}/queue/rotational") -ne 0 ]]
 }
diff --git a/common/xfs b/common/xfs
new file mode 100644
index 000000000000..d1a603b8c7b5
--- /dev/null
+++ b/common/xfs
@@ -0,0 +1,11 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2017 Omar Sandoval
+#
+# fio helper functions.
+
+. common/shellcheck
+
+_have_xfs() {
+	_have_fs xfs && _have_program mkfs.xfs
+}
diff --git a/tests/nvme/012 b/tests/nvme/012
index 1dbd59804ed7..1d8d8e3cc271 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -5,14 +5,16 @@
 # Test mkfs with data verification for block device backed ns.
 
 . tests/nvme/rc
+. common/xfs
 
 DESCRIPTION="run mkfs and data verification fio job on NVMeOF block device-backed ns"
 TIMED=1
 
 requires() {
 	_nvme_requires
-	_have_program mkfs.xfs && _have_program fio && \
-		_have_modules loop
+	_have_xfs
+	_have_program fio
+	_have_modules loop
 	_require_nvme_trtype_is_fabrics
 }
 
diff --git a/tests/nvme/013 b/tests/nvme/013
index df7f23e69252..3819a2730d9b 100755
--- a/tests/nvme/013
+++ b/tests/nvme/013
@@ -5,13 +5,15 @@
 # Test mkfs with data verification for file backed ns.
 
 . tests/nvme/rc
+. common/xfs
 
 DESCRIPTION="run mkfs and data verification fio job on NVMeOF file-backed ns"
 TIMED=1
 
 requires() {
 	_nvme_requires
-	_have_program mkfs.xfs && _have_fio
+	_have_xfs
+	_have_fio
 	_require_nvme_trtype_is_fabrics
 }
 
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:17 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:17 -0600
Subject: [PATCH blktests v3 04/11] nvme: Search for specific subsysnqn in
 _find_nvme_loop_dev
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-5-logang@deltatee.com>

This ensures we find the correct nvme loop device if others exist on a
given system (which is generally not expected on test systems).

Additionally, this will be required in the upcomming test nvme/037 which
will have controllers racing with ones being destroyed.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/004 | 2 +-
 tests/nvme/005 | 2 +-
 tests/nvme/008 | 2 +-
 tests/nvme/009 | 2 +-
 tests/nvme/010 | 2 +-
 tests/nvme/011 | 2 +-
 tests/nvme/012 | 2 +-
 tests/nvme/013 | 2 +-
 tests/nvme/014 | 2 +-
 tests/nvme/015 | 2 +-
 tests/nvme/018 | 2 +-
 tests/nvme/019 | 2 +-
 tests/nvme/020 | 2 +-
 tests/nvme/021 | 2 +-
 tests/nvme/022 | 2 +-
 tests/nvme/023 | 2 +-
 tests/nvme/024 | 2 +-
 tests/nvme/025 | 2 +-
 tests/nvme/026 | 2 +-
 tests/nvme/027 | 2 +-
 tests/nvme/028 | 2 +-
 tests/nvme/029 | 2 +-
 tests/nvme/rc  | 7 ++++---
 23 files changed, 26 insertions(+), 25 deletions(-)

diff --git a/tests/nvme/004 b/tests/nvme/004
index dfca79aab20c..4b0b7ae50a5e 100755
--- a/tests/nvme/004
+++ b/tests/nvme/004
@@ -37,7 +37,7 @@ test() {
 	_nvme_connect_subsys "${nvme_trtype}" blktests-subsystem-1
 
 	local nvmedev
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "blktests-subsystem-1")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/005 b/tests/nvme/005
index 0d5801868bc0..9f3e388dc695 100755
--- a/tests/nvme/005
+++ b/tests/nvme/005
@@ -37,7 +37,7 @@ test() {
 	_nvme_connect_subsys "${nvme_trtype}" blktests-subsystem-1
 
 	local nvmedev
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "blktests-subsystem-1")
 
 	udevadm settle
 
diff --git a/tests/nvme/008 b/tests/nvme/008
index 8616617ad398..219fe9b0ca6a 100755
--- a/tests/nvme/008
+++ b/tests/nvme/008
@@ -37,7 +37,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/009 b/tests/nvme/009
index e91d79065cb1..2814c79164ee 100755
--- a/tests/nvme/009
+++ b/tests/nvme/009
@@ -33,7 +33,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/010 b/tests/nvme/010
index 0188e842213e..150a4e540f3e 100755
--- a/tests/nvme/010
+++ b/tests/nvme/010
@@ -37,7 +37,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/011 b/tests/nvme/011
index 543dbe840874..4bfe9af084e4 100755
--- a/tests/nvme/011
+++ b/tests/nvme/011
@@ -35,7 +35,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/012 b/tests/nvme/012
index a13cd08ce6bf..c4e75b09796a 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -40,7 +40,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/013 b/tests/nvme/013
index 1ac725ea83f2..265b6968fd34 100755
--- a/tests/nvme/013
+++ b/tests/nvme/013
@@ -37,7 +37,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/014 b/tests/nvme/014
index e3c70364e332..48f8caaec0b3 100755
--- a/tests/nvme/014
+++ b/tests/nvme/014
@@ -37,7 +37,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/015 b/tests/nvme/015
index 46fa4f605749..e33cfde5d72e 100755
--- a/tests/nvme/015
+++ b/tests/nvme/015
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/018 b/tests/nvme/018
index 6d7934d09d99..7f407da2ce19 100755
--- a/tests/nvme/018
+++ b/tests/nvme/018
@@ -35,7 +35,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/019 b/tests/nvme/019
index 486b5acff713..8259e2e0c157 100755
--- a/tests/nvme/019
+++ b/tests/nvme/019
@@ -39,7 +39,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/020 b/tests/nvme/020
index c8053f440e2e..16fdfcc94918 100755
--- a/tests/nvme/020
+++ b/tests/nvme/020
@@ -35,7 +35,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/021 b/tests/nvme/021
index f543a1d8fd92..fb77f9cbd99f 100755
--- a/tests/nvme/021
+++ b/tests/nvme/021
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/022 b/tests/nvme/022
index e824ed31f6f0..62c4690e35fe 100755
--- a/tests/nvme/022
+++ b/tests/nvme/022
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/023 b/tests/nvme/023
index bdef3dc8abca..bce21b56c9f1 100755
--- a/tests/nvme/023
+++ b/tests/nvme/023
@@ -37,7 +37,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/024 b/tests/nvme/024
index 78f779e8a08a..ffec36cf3333 100755
--- a/tests/nvme/024
+++ b/tests/nvme/024
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/025 b/tests/nvme/025
index 223430965d7e..3d3f01bc45fd 100755
--- a/tests/nvme/025
+++ b/tests/nvme/025
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/026 b/tests/nvme/026
index 7f82284d9c57..2f5607793cd3 100755
--- a/tests/nvme/026
+++ b/tests/nvme/026
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/027 b/tests/nvme/027
index da96e6c5008d..53f06646a3d0 100755
--- a/tests/nvme/027
+++ b/tests/nvme/027
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/028 b/tests/nvme/028
index f826b67623f1..3d9084f18636 100755
--- a/tests/nvme/028
+++ b/tests/nvme/028
@@ -34,7 +34,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/029 b/tests/nvme/029
index 5bed9b8e70ae..960e5f5a63bf 100755
--- a/tests/nvme/029
+++ b/tests/nvme/029
@@ -70,7 +70,7 @@ test() {
 
 	_nvme_connect_subsys "${nvme_trtype}" "${subsys_name}"
 
-	nvmedev="$(_find_nvme_dev)"
+	nvmedev=$(_find_nvme_dev "${subsys_name}")
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
diff --git a/tests/nvme/rc b/tests/nvme/rc
index 4c5b2e8edf0d..dfa57a299625 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -273,12 +273,13 @@ _remove_nvmet_subsystem_from_port() {
 }
 
 _find_nvme_dev() {
+	local subsys=$1
+	local subsysnqn
 	local dev
-	local transport
 	for dev in /sys/class/nvme/nvme*; do
 		dev="$(basename "$dev")"
-		transport="$(cat "/sys/class/nvme/${dev}/transport")"
-		if [[ "$transport" == "${nvme_trtype}" ]]; then
+		subsysnqn="$(cat "/sys/class/nvme/${dev}/subsysnqn")"
+		if [[ "$subsysnqn" == "$subsys" ]]; then
 			echo "$dev"
 			for ((i = 0; i < 10; i++)); do
 				if [[ -e /sys/block/$dev/uuid &&
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:14 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:14 -0600
Subject: [PATCH blktests v3 01/11] common/fio: Remove state file in common
 helper
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-2-logang@deltatee.com>

Instead of each individual test removing this file, just do it
in the common helper.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 common/fio     | 1 +
 tests/nvme/010 | 1 -
 tests/nvme/011 | 1 -
 tests/nvme/012 | 1 -
 tests/nvme/013 | 1 -
 5 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/common/fio b/common/fio
index 8bfad4238dda..94c65c107a14 100644
--- a/common/fio
+++ b/common/fio
@@ -181,6 +181,7 @@ _run_fio_rand_io() {
 _run_fio_verify_io() {
 	_run_fio --name=verify --rw=randwrite --direct=1 --ioengine=libaio --bs=4k \
 		--iodepth=16 --verify=crc32c "$@"
+	rm -f local*verify*state
 }
 
 _fio_perf_report() {
diff --git a/tests/nvme/010 b/tests/nvme/010
index 9d96d7803be3..0188e842213e 100755
--- a/tests/nvme/010
+++ b/tests/nvme/010
@@ -52,7 +52,6 @@ test() {
 	losetup -d "${loop_dev}"
 
 	rm "${file_path}"
-	rm -f local*verify*state
 
 	echo "Test complete"
 }
diff --git a/tests/nvme/011 b/tests/nvme/011
index 06dc568fb6ea..543dbe840874 100755
--- a/tests/nvme/011
+++ b/tests/nvme/011
@@ -48,7 +48,6 @@ test() {
 	_remove_nvmet_port "${port}"
 
 	rm "${file_path}"
-	rm -f local-write-and-verify*state
 
 	echo "Test complete"
 }
diff --git a/tests/nvme/012 b/tests/nvme/012
index 8110430e49d4..1dbd59804ed7 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -63,7 +63,6 @@ test() {
 
 	losetup -d "${loop_dev}"
 
-	rm -f local*verify*state
 	rm "${file_path}"
 	rm -fr "${mount_dir}"
 
diff --git a/tests/nvme/013 b/tests/nvme/013
index 176b11b9ccb5..df7f23e69252 100755
--- a/tests/nvme/013
+++ b/tests/nvme/013
@@ -58,7 +58,6 @@ test() {
 	_remove_nvmet_subsystem "${subsys_name}"
 	_remove_nvmet_port "${port}"
 
-	rm -f local*verify*state
 	rm "${file_path}"
 	rm -fr "${mount_dir}"
 
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:16 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:16 -0600
Subject: [PATCH blktests v3 03/11] common/xfs: Create common helper to verify
 block device with xfs
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-4-logang@deltatee.com>

Make a common helper from the code in tests nvme/012 and nvme/013
to run an fio verify on a XFS file system backed by the
specified block device.

While we are at it, all the output is redirected to $FULL instead of
/dev/null.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 common/xfs     | 22 ++++++++++++++++++++++
 tests/nvme/012 | 14 +-------------
 tests/nvme/013 | 14 +-------------
 3 files changed, 24 insertions(+), 26 deletions(-)

diff --git a/common/xfs b/common/xfs
index d1a603b8c7b5..210c924cdd41 100644
--- a/common/xfs
+++ b/common/xfs
@@ -9,3 +9,25 @@
 _have_xfs() {
 	_have_fs xfs && _have_program mkfs.xfs
 }
+
+_xfs_mkfs_and_mount() {
+	local bdev=$1
+	local mount_dir=$2
+
+	mkdir -p "${mount_dir}"
+	umount "${mount_dir}"
+	mkfs.xfs -l size=32m -f "${bdev}"
+	mount "${bdev}" "${mount_dir}"
+}
+
+_xfs_run_fio_verify_io() {
+	local mount_dir="/mnt/blktests"
+	local bdev=$1
+
+	_xfs_mkfs_and_mount "${bdev}" "${mount_dir}" >> "${FULL}" 2>&1
+
+	_run_fio_verify_io --size=950m --directory="${mount_dir}/"
+
+	umount "${mount_dir}" >> "${FULL}" 2>&1
+	rm -fr "${mount_dir}"
+}
diff --git a/tests/nvme/012 b/tests/nvme/012
index 1d8d8e3cc271..a13cd08ce6bf 100755
--- a/tests/nvme/012
+++ b/tests/nvme/012
@@ -26,12 +26,9 @@ test() {
 	local port
 	local nvmedev
 	local loop_dev
-	local mount_dir="/mnt/blktests"
 	local file_path="${TMPDIR}/img"
 	local subsys_name="blktests-subsystem-1"
 
-	mkdir -p "${mount_dir}" > /dev/null 2>&1
-
 	truncate -s 1G "${file_path}"
 
 	loop_dev="$(losetup -f --show "${file_path}")"
@@ -47,15 +44,7 @@ test() {
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
-	umount ${mount_dir} > /dev/null 2>&1
-
-	mkfs.xfs -l size=32m -f /dev/"${nvmedev}n1" > /dev/null 2>&1
-
-	mount /dev/"${nvmedev}n1" "${mount_dir}"
-
-	_run_fio_verify_io --size=950m --directory="${mount_dir}/"
-
-	umount "${mount_dir}" > /dev/null 2>&1
+	_xfs_run_fio_verify_io "/dev/${nvmedev}n1"
 
 	_nvme_disconnect_subsys "${subsys_name}"
 
@@ -66,7 +55,6 @@ test() {
 	losetup -d "${loop_dev}"
 
 	rm "${file_path}"
-	rm -fr "${mount_dir}"
 
 	echo "Test complete"
 }
diff --git a/tests/nvme/013 b/tests/nvme/013
index 3819a2730d9b..1ac725ea83f2 100755
--- a/tests/nvme/013
+++ b/tests/nvme/013
@@ -24,13 +24,10 @@ test() {
 
 	local port
 	local nvmedev
-	local mount_dir="/mnt/blktests/"
 	local file_path="${TMPDIR}/img"
 
 	local subsys_name="blktests-subsystem-1"
 
-	mkdir -p "${mount_dir}" > /dev/null 2>&1
-
 	truncate -s 1G "${file_path}"
 
 	_create_nvmet_subsystem "${subsys_name}" "${file_path}" \
@@ -44,15 +41,7 @@ test() {
 	cat "/sys/block/${nvmedev}n1/uuid"
 	cat "/sys/block/${nvmedev}n1/wwid"
 
-	umount ${mount_dir} > /dev/null 2>&1
-
-	mkfs.xfs -l size=32m -f /dev/"${nvmedev}n1" > /dev/null 2>&1
-
-	mount /dev/"${nvmedev}n1" "${mount_dir}"
-
-	_run_fio_verify_io --size=800m --directory="${mount_dir}/"
-
-	umount "${mount_dir}" > /dev/null 2>&1
+	_xfs_run_fio_verify_io "/dev/${nvmedev}n1"
 
 	_nvme_disconnect_subsys "${subsys_name}"
 
@@ -61,7 +50,6 @@ test() {
 	_remove_nvmet_port "${port}"
 
 	rm "${file_path}"
-	rm -fr "${mount_dir}"
 
 	echo "Test complete"
 }
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:24 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:24 -0600
Subject: [PATCH blktests v3 11/11] nvme/038: Test removal of un-enabled
 subsystem and ports
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-12-logang@deltatee.com>

Test that we can remove a subsystem that has not been enabled by
passthru or any ns. Do the same for ports while we are at it.

This was an issue in the original passthru patches and is
not commonly tested. So this test will ensure we don't regress this.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/038     | 36 ++++++++++++++++++++++++++++++++++++
 tests/nvme/038.out |  2 ++
 2 files changed, 38 insertions(+)
 create mode 100755 tests/nvme/038
 create mode 100644 tests/nvme/038.out

diff --git a/tests/nvme/038 b/tests/nvme/038
new file mode 100755
index 000000000000..24f02d4ad4d1
--- /dev/null
+++ b/tests/nvme/038
@@ -0,0 +1,36 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Logan Gunthorpe
+# Copyright (C) 2019 Eideticom Communications Inc.
+#
+# Test that we can remove a subsystem that has not been enabled by
+# passthru or any ns. Do the same for ports while we are at it.
+#
+# This was an issue in the original passthru patches and is
+# not commonly tested. So this test will ensure we don't regress this.
+#
+. tests/nvme/rc
+
+DESCRIPTION="test deletion of NVMeOF subsystem without enabling"
+QUICK=1
+
+requires() {
+	_nvme_requires
+}
+
+test() {
+	local subsys_path="${NVMET_CFS}/subsystems/blktests-subsystem-1"
+	local port
+
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+
+	mkdir -p "${subsys_path}"
+	rmdir "${subsys_path}"
+
+	port=$(_create_nvmet_port loop)
+	_remove_nvmet_port "${port}"
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/038.out b/tests/nvme/038.out
new file mode 100644
index 000000000000..06bc98022c33
--- /dev/null
+++ b/tests/nvme/038.out
@@ -0,0 +1,2 @@
+Running nvme/038
+Test complete
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:20 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:20 -0600
Subject: [PATCH blktests v3 07/11] nvme/034: Add test for passthru data
 verification
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-8-logang@deltatee.com>

Similar to test nvme/010 and nvme/011 but for a passthru controller

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/034     | 35 +++++++++++++++++++++++++++++++++++
 tests/nvme/034.out |  3 +++
 2 files changed, 38 insertions(+)
 create mode 100755 tests/nvme/034
 create mode 100644 tests/nvme/034.out

diff --git a/tests/nvme/034 b/tests/nvme/034
new file mode 100755
index 000000000000..f92e5e20865b
--- /dev/null
+++ b/tests/nvme/034
@@ -0,0 +1,35 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Logan Gunthorpe
+# Copyright (C) 2019 Eideticom Communications Inc.
+
+. tests/nvme/rc
+
+DESCRIPTION="run data verification fio job on an NVMeOF passthru controller"
+TIMED=1
+
+requires() {
+	_nvme_requires
+	_have_kernel_option NVME_TARGET_PASSTHRU
+	_have_fio
+}
+
+test_device() {
+	local subsys="blktests-subsystem-1"
+	local ctrldev
+	local nsdev
+	local port
+
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+	port=$(_nvmet_passthru_target_setup "${subsys}")
+	nsdev=$(_nvmet_passthru_target_connect "${nvme_trtype}" "${subsys}")
+
+	_run_fio_verify_io --size=950m --filename="${nsdev}"
+
+	_nvme_disconnect_subsys "${subsys}"
+	_nvmet_passthru_target_cleanup "${port}" "${subsys}"
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/034.out b/tests/nvme/034.out
new file mode 100644
index 000000000000..0a7bd2f90dae
--- /dev/null
+++ b/tests/nvme/034.out
@@ -0,0 +1,3 @@
+Running nvme/034
+NQN:blktests-subsystem-1 disconnected 1 controller(s)
+Test complete
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:22 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:22 -0600
Subject: [PATCH blktests v3 09/11] nvme/036: Add test for testing reset
 command on nvme-passthru
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-10-logang@deltatee.com>

Similar to test 022 but for passthru controllers.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/036     | 37 +++++++++++++++++++++++++++++++++++++
 tests/nvme/036.out |  3 +++
 2 files changed, 40 insertions(+)
 create mode 100755 tests/nvme/036
 create mode 100644 tests/nvme/036.out

diff --git a/tests/nvme/036 b/tests/nvme/036
new file mode 100755
index 000000000000..8218c6538dfd
--- /dev/null
+++ b/tests/nvme/036
@@ -0,0 +1,37 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Logan Gunthorpe
+# Copyright (C) 2019 Eideticom Communications Inc.
+
+. tests/nvme/rc
+
+DESCRIPTION="test NVMe reset command on an NVMeOF target with a passthru controller"
+QUICK=1
+
+requires() {
+	_nvme_requires
+	_have_kernel_option NVME_TARGET_PASSTHRU
+}
+
+test_device() {
+	local subsys="blktests-subsystem-1"
+	local ctrldev
+	local port
+
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+	port=$(_nvmet_passthru_target_setup "${subsys}")
+	nsdev=$(_nvmet_passthru_target_connect "${nvme_trtype}" "${subsys}")
+
+	ctrldev=$(_find_nvme_dev "${subsys}")
+
+	if ! nvme reset "/dev/${ctrldev}" >> "$FULL" 2>&1; then
+		echo "ERROR: reset failed"
+	fi
+
+	_nvme_disconnect_subsys "${subsys}"
+	_nvmet_passthru_target_cleanup "${port}" "${subsys}"
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/036.out b/tests/nvme/036.out
new file mode 100644
index 000000000000..d294f8646b20
--- /dev/null
+++ b/tests/nvme/036.out
@@ -0,0 +1,3 @@
+Running nvme/036
+NQN:blktests-subsystem-1 disconnected 1 controller(s)
+Test complete
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:23 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:23 -0600
Subject: [PATCH blktests v3 10/11] nvme/037: Add test which loops passthru
 connect and disconnect
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-11-logang@deltatee.com>

Similar to test nvme/031 except for passthru controllers.

Note: it's normal to get I/O errors in this test as when the controller
disconnects it races with the partition table read.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/037     | 35 +++++++++++++++++++++++++++++++++++
 tests/nvme/037.out |  2 ++
 2 files changed, 37 insertions(+)
 create mode 100755 tests/nvme/037
 create mode 100644 tests/nvme/037.out

diff --git a/tests/nvme/037 b/tests/nvme/037
new file mode 100755
index 000000000000..fc6c21343652
--- /dev/null
+++ b/tests/nvme/037
@@ -0,0 +1,35 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Logan Gunthorpe
+# Copyright (C) 2019 Eideticom Communications Inc.
+
+. tests/nvme/rc
+
+DESCRIPTION="test deletion of NVMeOF passthru controllers immediately after setup"
+
+requires() {
+	_nvme_requires
+	_have_kernel_option NVME_TARGET_PASSTHRU
+}
+
+test_device() {
+	local subsys="blktests-subsystem-"
+	local iterations=10
+	local ctrldev
+	local port
+
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+
+	for ((i = 0; i < iterations; i++)); do
+		port=$(_nvmet_passthru_target_setup "${subsys}${i}")
+		nsdev=$(_nvmet_passthru_target_connect "${nvme_trtype}" \
+				"${subsys}${i}")
+
+		_nvme_disconnect_subsys "${subsys}${i}" >>"${FULL}" 2>&1
+		_nvmet_passthru_target_cleanup "${port}" "${subsys}${i}"
+	done
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/037.out b/tests/nvme/037.out
new file mode 100644
index 000000000000..eaf903d0520e
--- /dev/null
+++ b/tests/nvme/037.out
@@ -0,0 +1,2 @@
+Running nvme/037
+Test complete
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:21 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:21 -0600
Subject: [PATCH blktests v3 08/11] nvme/035: Add test to verify passthru
 controller with a filesystem
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-9-logang@deltatee.com>

This is a similar test as nvme/012 and nvme/013, except with a
passthru controller.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/035     | 37 +++++++++++++++++++++++++++++++++++++
 tests/nvme/035.out |  3 +++
 2 files changed, 40 insertions(+)
 create mode 100755 tests/nvme/035
 create mode 100644 tests/nvme/035.out

diff --git a/tests/nvme/035 b/tests/nvme/035
new file mode 100755
index 000000000000..ee78a7586f35
--- /dev/null
+++ b/tests/nvme/035
@@ -0,0 +1,37 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Logan Gunthorpe
+# Copyright (C) 2019 Eideticom Communications Inc.
+
+. tests/nvme/rc
+. common/xfs
+
+DESCRIPTION="run mkfs and data verification fio job on an NVMeOF passthru controller"
+TIMED=1
+
+requires() {
+	_nvme_requires
+	_have_kernel_option NVME_TARGET_PASSTHRU
+	_have_xfs
+	_have_fio
+}
+
+test_device() {
+	local subsys="blktests-subsystem-1"
+	local ctrldev
+	local nsdev
+	local port
+
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+	port=$(_nvmet_passthru_target_setup "${subsys}")
+	nsdev=$(_nvmet_passthru_target_connect "${nvme_trtype}" "${subsys}")
+
+	_xfs_run_fio_verify_io "${nsdev}"
+
+	_nvme_disconnect_subsys "${subsys}"
+	_nvmet_passthru_target_cleanup "${port}" "${subsys}"
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/035.out b/tests/nvme/035.out
new file mode 100644
index 000000000000..a6027138fbe4
--- /dev/null
+++ b/tests/nvme/035.out
@@ -0,0 +1,3 @@
+Running nvme/035
+NQN:blktests-subsystem-1 disconnected 1 controller(s)
+Test complete
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:19 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:19 -0600
Subject: [PATCH blktests v3 06/11] nvme/033: Simple test to create and connect
 to a passthru target
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-7-logang@deltatee.com>

This tests creates and connects to a passthru controller backed
by a test NVMe namespace. It then verifies that some common fields
in id-ctrl and id-ns are the same in the target and the orginial
device.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/033     | 67 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/033.out |  7 +++++
 2 files changed, 74 insertions(+)
 create mode 100755 tests/nvme/033
 create mode 100644 tests/nvme/033.out

diff --git a/tests/nvme/033 b/tests/nvme/033
new file mode 100755
index 000000000000..c6a3f7feb50e
--- /dev/null
+++ b/tests/nvme/033
@@ -0,0 +1,67 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Logan Gunthorpe
+# Copyright (C) 2019 Eideticom Communications Inc.
+
+. tests/nvme/rc
+
+DESCRIPTION="create and connect to an NVMeOF target with a passthru controller"
+QUICK=1
+
+requires() {
+	_nvme_requires
+	_have_kernel_option NVME_TARGET_PASSTHRU
+}
+
+nvme_info() {
+	local ns=$1
+
+	nvme id-ctrl "${ns}" | grep -E '^(vid|sn|mn|fr) '
+	nvme id-ns "${ns}" | grep -E '^(nsze|ncap) '
+}
+
+compare_dev_info() {
+	local passthru_dev=$1
+	local testdev_info
+	local passthru_info
+
+	testdev_info=$(nvme_info "${TEST_DEV}")
+	passthru_info=$(nvme_info "${passthru_dev}")
+
+	cat >> "${FULL}" <<- EOF
+
+	Test Device ${TEST_DEV} Info:
+	$testdev_info
+
+	Passthru Device ${passthru_dev} Info:
+	$passthru_info
+
+	EOF
+
+	diff -u <(echo "${testdev_info}") <(echo "${passthru_info}")
+	if [[ "${testdev_info}" != "${passthru_info}" ]]; then
+		echo "ERROR: Device information does not match! (See ${FULL})"
+	fi
+}
+
+test_device() {
+	local subsys="blktests-subsystem-1"
+	local nsdev
+	local port
+
+	echo "Running ${TEST_NAME}"
+
+	_setup_nvmet
+	port=$(_nvmet_passthru_target_setup "${subsys}")
+
+	_nvme_discover "${nvme_trtype}" | _filter_discovery
+
+	nsdev=$(_nvmet_passthru_target_connect "${nvme_trtype}" "${subsys}")
+
+	compare_dev_info "${nsdev}"
+
+	_nvme_disconnect_subsys "${subsys}"
+	_nvmet_passthru_target_cleanup "${port}" "${subsys}"
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/033.out b/tests/nvme/033.out
new file mode 100644
index 000000000000..6f45a1d5ec1d
--- /dev/null
+++ b/tests/nvme/033.out
@@ -0,0 +1,7 @@
+Running nvme/033
+Discovery Log Number of Records 1, Generation counter X
+=====Discovery Log Entry 0======
+trtype:  loop
+subnqn:  blktests-subsystem-1
+NQN:blktests-subsystem-1 disconnected 1 controller(s)
+Test complete
-- 
2.20.1



From logang at deltatee.com  Thu Oct  8 12:40:18 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu,  8 Oct 2020 10:40:18 -0600
Subject: [PATCH blktests v3 05/11] nvme: Add common helpers for passthru tests
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <20201008164024.12546-6-logang@deltatee.com>

Add some simple helpers to setup a passthru target that passes through
to a nvme test device.

Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
---
 tests/nvme/rc | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/tests/nvme/rc b/tests/nvme/rc
index dfa57a299625..f0b6f360da8c 100644
--- a/tests/nvme/rc
+++ b/tests/nvme/rc
@@ -73,6 +73,16 @@ _require_nvme_trtype_is_fabrics() {
 	return 0
 }
 
+_test_dev_nvme_ctrl() {
+	local dev=$(cat "${TEST_DEV_SYSFS}/device/dev")
+
+	echo "/dev/char/${dev}"
+}
+
+_test_dev_nvme_nsid() {
+	cat "${TEST_DEV_SYSFS}/nsid"
+}
+
 _cleanup_nvmet() {
 	local dev
 	local port
@@ -257,6 +267,27 @@ _remove_nvmet_subsystem() {
 	rmdir "${subsys_path}"
 }
 
+_create_nvmet_passthru() {
+	local nvmet_subsystem="$1"
+	local subsys_path="${NVMET_CFS}/subsystems/${nvmet_subsystem}"
+	local passthru_path="${subsys_path}/passthru"
+
+	mkdir -p "${subsys_path}"
+	echo 1 > "${subsys_path}/attr_allow_any_host"
+
+	echo "$(_test_dev_nvme_ctrl)" > "${passthru_path}/device_path"
+	echo 1 > "${passthru_path}/enable"
+}
+
+_remove_nvmet_passhtru() {
+	local nvmet_subsystem="$1"
+	local subsys_path="${NVMET_CFS}/subsystems/${nvmet_subsystem}"
+	local passthru_path="${subsys_path}/passthru"
+
+	echo 0 > "${passthru_path}/enable"
+	rmdir "${subsys_path}"
+}
+
 _add_nvmet_subsys_to_port() {
 	local port="$1"
 	local nvmet_subsystem="$2"
@@ -292,6 +323,50 @@ _find_nvme_dev() {
 	done
 }
 
+_find_nvme_passthru_loop_dev() {
+	local subsys=$1
+	local nsid
+	local dev
+
+	dev=$(_find_nvme_dev "${subsys}")
+	nsid=$(_test_dev_nvme_nsid)
+	echo "/dev/${dev}n${nsid}"
+}
+
+_nvmet_passthru_target_setup() {
+	local subsys_name=$1
+
+	_create_nvmet_passthru "${subsys_name}"
+	port="$(_create_nvmet_port "loop")"
+	_add_nvmet_subsys_to_port "${port}" "${subsys_name}"
+
+	echo "$port"
+}
+
+_nvmet_passthru_target_connect() {
+	local trtype=$1
+	local subsys_name=$2
+
+	_nvme_connect_subsys "${trtype}" "${subsys_name}"
+	nsdev=$(_find_nvme_passthru_loop_dev "${subsys_name}")
+
+	# The following tests can race with the creation
+	# of the device so ensure the block device exists
+	# before continuing
+	while [ ! -b "${nsdev}" ]; do sleep 1; done
+
+	echo "${nsdev}"
+}
+
+_nvmet_passthru_target_cleanup() {
+	local port=$1
+	local subsys_name=$2
+
+	_remove_nvmet_subsystem_from_port "${port}" "${subsys_name}"
+	_remove_nvmet_port "${port}"
+	_remove_nvmet_passhtru "${subsys_name}"
+}
+
 _filter_discovery() {
 	sed -n -r -e "s/Generation counter [0-9]+/Generation counter X/" \
 		  -e '/Discovery Log Number|Log Entry|trtype|subnqn/p'
-- 
2.20.1



From sagi at grimberg.me  Thu Oct  8 15:25:37 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 12:25:37 -0700
Subject: [bug report] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28
 refcount_warn_saturate+0xa6/0xf0
In-Reply-To: <1934331639.3314730.1602152202454.JavaMail.zimbra@redhat.com>
References: <1934331639.3314730.1602152202454.JavaMail.zimbra@redhat.com>
Message-ID: <595779fc-0ae8-7c53-9bf3-619c14d74c74@grimberg.me>

> Hello
> 
> I saw this failure[2] on both stable and block/for-next recently, seems it was introduced from[1], pls help check it, thanks.
> 
> 
> [1]
> commit 236187c4ed195161dfa4237c7beffbba0c5ae45b
> Author: Sagi Grimberg <sagi at grimberg.me>
> Date:   Tue Jul 28 13:16:36 2020 -0700
> 
>      nvme-tcp: fix timeout handler
> 

Thanks for reporting Yi,

I'm not able to reproduce it, but it appears that there is
a possible race with blk_mq_complete_request_remote.

Can you try this and report if this reproduces?
--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8f4f29f18b8c..ee996f32cb26 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2178,6 +2178,8 @@ static void nvme_tcp_complete_timed_out(struct 
request *rq)
         if (!blk_mq_request_completed(rq)) {
                 nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
                 blk_mq_complete_request(rq);
+               while (blk_mq_request_started(rq))
+                       msleep(10);
         }
         mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
  }
--

> 
> [2]
> #nvme_trtype=tcp ./check nvme/012
> 
> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
> [ 2152.799640] loop: module loaded
> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
> [ 2152.937889] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
> [ 2154.942432] XFS (nvme0n1): Ending clean mount
> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
> [ 2215.199331] nvme nvme0: starting error recovery
> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
> [ 2215.215502] ------------[ cut here ]------------
> [ 2215.215505] refcount_t: underflow; use-after-free.
> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
> [ 2215.215619] Modules linked in: loop nvme_tcp nvme_fabrics nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_hda_codec snd_hda_core snd_hwdep hp_wmi amd64_edac_mod edac_mce_amd kvm_amd ccp kvm irqbypass k10temp sparse_keymap rfkill pcspkr nv_tco wmi_bmof snd_pcm snd_timer snd soundcore i2c_nforce2 acpi_cpufreq ip_tables xfs nouveau video mxm_wmi i2c_algo_bit ata_generic drm_kms_helper cec mptsas ttm scsi_transport_sas firewire_ohci mptscsih pata_acpi drm serio_raw mptbase firewire_core forcedeth crc_itu_t sata_nv pata_amd wmi floppy
> [ 2215.215710] CPU: 6 PID: 45 Comm: kworker/6:0H Not tainted 5.9.0-rc8+ #1
> [ 2215.215713] Hardware name: Hewlett-Packard HP xw9400 Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
> [ 2215.215721] Workqueue: kblockd blk_mq_timeout_work
> [ 2215.215736] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
> [ 2215.215742] Code: 05 ca 98 22 01 01 e8 bf cf ad ff 0f 0b c3 80 3d b8 98 22 01 00 75 95 48 c7 c7 d8 53 3e a7 c6 05 a8 98 22 01 01 e8 a0 cf ad ff <0f> 0b c3 80 3d 97 98 22 01 00 0f 85 72 ff ff ff 48 c7 c7 30 54 3e
> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff93f37dcd8d08
> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff93f37dcd8d00
> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: ffffffffa7fffbc4
> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: ffff93f37a8124e8
> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: ffff93f37a2b0000
> [ 2215.215765] FS:  0000000000000000(0000) GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
> [ 2215.215768] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 00000000000006e0
> [ 2215.215773] Call Trace:
> [ 2215.215784]  blk_mq_check_expired+0x109/0x1b0
> [ 2215.215797]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
> [ 2215.215801]  ? blk_poll+0x300/0x300
> [ 2215.215806]  blk_mq_timeout_work+0x44/0xe0
> [ 2215.215814]  process_one_work+0x1b4/0x370
> [ 2215.215820]  worker_thread+0x53/0x3e0
> [ 2215.215825]  ? process_one_work+0x370/0x370
> [ 2215.215829]  kthread+0x11b/0x140
> [ 2215.215834]  ? __kthread_bind_mask+0x60/0x60
> [ 2215.215841]  ret_from_fork+0x22/0x30
> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
> [ 2215.220371] block nvme0n1: no usable path - requeuing I/O
> [ 2215.220377] block nvme0n1: no usable path - requeuing I/O
> [ 2215.466666] block nvme0n1: no usable path - requeuing I/O
> [ 2215.472056] block nvme0n1: no usable path - requeuing I/O
> [ 2215.477444] block nvme0n1: no usable path - requeuing I/O
> [ 2215.482828] block nvme0n1: no usable path - requeuing I/O
> [ 2215.488219] block nvme0n1: no usable path - requeuing I/O
> [ 2215.493625] block nvme0n1: no usable path - requeuing I/O
> [ 2240.794172] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
> [ 2240.800189] nvmet: ctrl 2 fatal error occurred!
> [ 2245.402268] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
> [ 2245.408286] nvmet: ctrl 1 fatal error occurred!
> [ 2262.862528] nvmet: creating controller 2 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 2288.411172] nvme nvme0: queue 0: timeout request 0x0 type 4
> [ 2288.416794] nvme nvme0: Connect command failed, error wo/DNR bit: 881
> [ 2288.423254] nvme nvme0: failed to connect queue: 0 ret=881
> [ 2288.428773] nvme nvme0: Failed reconnect attempt 1
> [ 2288.433568] nvme nvme0: Reconnecting in 10 seconds...
> [ 2298.651953] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 2298.665918] nvme_ns_head_submit_bio: 30 callbacks suppressed
> [ 2298.665926] block nvme0n1: no usable path - requeuing I/O
> [ 2298.676993] block nvme0n1: no usable path - requeuing I/O
> [ 2298.682393] block nvme0n1: no usable path - requeuing I/O
> [ 2298.687791] block nvme0n1: no usable path - requeuing I/O
> [ 2298.693202] block nvme0n1: no usable path - requeuing I/O
> [ 2298.698593] block nvme0n1: no usable path - requeuing I/O
> [ 2298.703988] block nvme0n1: no usable path - requeuing I/O
> [ 2298.709375] block nvme0n1: no usable path - requeuing I/O
> [ 2298.714758] block nvme0n1: no usable path - requeuing I/O
> [ 2298.720152] block nvme0n1: no usable path - requeuing I/O
> [ 2298.725691] nvme nvme0: creating 12 I/O queues.
> [ 2298.737066] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [ 2298.745612] nvme nvme0: Successfully reconnected (2 attempt)
> 
> 
> 
> Best Regards,
>    Yi Zhang
> 
> 


From sagi at grimberg.me  Thu Oct  8 17:37:50 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu,  8 Oct 2020 14:37:50 -0700
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
Message-ID: <20201008213750.899462-1-sagi@grimberg.me>

In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
needs to complete an aborted request. The timeout handler serializes
against an error recovery sequence that iterates over the inflight
requests and cancels them.

However, the fact that blk_mq_complete_request may defer to a different
core that serialization breaks.

Hence re-introduce blk_mq_complete_request_sync and use that in the
timeout handler to make sure that we don't get a use-after-free
condition.

This was uncovered by the blktests:
--
$ nvme_trtype=tcp ./check nvme/012

[ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
[ 2152.799640] loop: module loaded
[ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[ 2152.937889] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[ 2152.952085] nvme nvme0: creating 12 I/O queues.
[ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
[ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
[ 2154.942432] XFS (nvme0n1): Ending clean mount
[ 2154.948183] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
[ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
[ 2215.199331] nvme nvme0: starting error recovery
[ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
[ 2215.204483] block nvme0n1: no usable path - requeuing I/O
[ 2215.214976] block nvme0n1: no usable path - requeuing I/O
[ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
[ 2215.215502] ------------[ cut here ]------------
[ 2215.215505] refcount_t: underflow; use-after-free.
[ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
[ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
[ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff93f37dcd8d08
[ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff93f37dcd8d00
[ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: ffffffffa7fffbc4
[ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: ffff93f37a8124e8
[ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: ffff93f37a2b0000
[ 2215.215765] FS:  0000000000000000(0000) GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
[ 2215.215768] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 00000000000006e0
[ 2215.215773] Call Trace:
[ 2215.215784]  blk_mq_check_expired+0x109/0x1b0
[ 2215.215797]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
[ 2215.215801]  ? blk_poll+0x300/0x300
[ 2215.215806]  blk_mq_timeout_work+0x44/0xe0
[ 2215.215814]  process_one_work+0x1b4/0x370
[ 2215.215820]  worker_thread+0x53/0x3e0
[ 2215.215825]  ? process_one_work+0x370/0x370
[ 2215.215829]  kthread+0x11b/0x140
[ 2215.215834]  ? __kthread_bind_mask+0x60/0x60
[ 2215.215841]  ret_from_fork+0x22/0x30
[ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
--

Reported-by: Yi Zhang <yi.zhang at redhat.com>
Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
Yi, would appreciate your Tested-by tag

 block/blk-mq.c           | 7 +++++++
 drivers/nvme/host/rdma.c | 2 +-
 drivers/nvme/host/tcp.c  | 2 +-
 include/linux/blk-mq.h   | 1 +
 4 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index f8e1e759c905..2d154722ef39 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct request *rq)
 }
 EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
 
+void blk_mq_complete_request_sync(struct request *rq)
+{
+	WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
+	rq->q->mq_ops->complete(rq);
+}
+EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
+
 /**
  * blk_mq_complete_request - end I/O on a request
  * @rq:		the request being processed
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..45fc605349da 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1975,7 +1975,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
 	nvme_rdma_stop_queue(queue);
 	if (!blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
-		blk_mq_complete_request(rq);
+		blk_mq_complete_request_sync(rq);
 	}
 	mutex_unlock(&ctrl->teardown_lock);
 }
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8f4f29f18b8c..629b025685d1 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
 	if (!blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
-		blk_mq_complete_request(rq);
+		blk_mq_complete_request_sync(rq);
 	}
 	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 90da3582b91d..f90c258b5075 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -491,6 +491,7 @@ void blk_mq_requeue_request(struct request *rq, bool kick_requeue_list);
 void blk_mq_kick_requeue_list(struct request_queue *q);
 void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs);
 void blk_mq_complete_request(struct request *rq);
+void blk_mq_complete_request_sync(struct request *rq);
 bool blk_mq_complete_request_remote(struct request *rq);
 bool blk_mq_queue_stopped(struct request_queue *q);
 void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx);
-- 
2.25.1



From sagi at grimberg.me  Thu Oct  8 17:47:08 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 14:47:08 -0700
Subject: [PATCH net-next RFC v1 02/10] net: Introduce direct data
 placement tcp offload
In-Reply-To: <20200930162010.21610-3-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-3-borisp@mellanox.com>
Message-ID: <ab791897-6398-f38e-c2a6-f54c5ca5e1c5@grimberg.me>


> + * tcp_ddp.h
> + *	Author:	Boris Pismenny <borisp at mellanox.com>
> + *	Copyright (C) 2020 Mellanox Technologies.
> + */
> +#ifndef _TCP_DDP_H
> +#define _TCP_DDP_H
> +
> +#include <linux/blkdev.h>

Why is blkdev.h needed?

> +#include <linux/netdevice.h>
> +#include <net/inet_connection_sock.h>
> +#include <net/sock.h>
> +
> +/* limits returned by the offload driver, zero means don't care */
> +struct tcp_ddp_limits {
> +	int	 max_ddp_sgl_len;
> +};
> +
> +enum tcp_ddp_type {
> +	TCP_DDP_NVME = 1,
> +};
> +
> +struct tcp_ddp_config {
> +	enum tcp_ddp_type    type;
> +	unsigned char        buf[];

A little kdoc may help here as its not exactly clear what is
buf used for (at this point at least)...

> +};
> +
> +struct nvme_tcp_config {

struct nvme_tcp_ddp_config

> +	struct tcp_ddp_config   cfg;
> +
> +	u16			pfv;
> +	u8			cpda;
> +	u8			dgst;
> +	int			queue_size;
> +	int			queue_id;
> +	int			io_cpu;
> +};
> +

Other than that this looks good to me.


From sagi at grimberg.me  Thu Oct  8 17:51:50 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 14:51:50 -0700
Subject: [PATCH net-next RFC v1 03/10] net: Introduce crc offload for tcp
 ddp ulp
In-Reply-To: <20200930162010.21610-4-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-4-borisp@mellanox.com>
Message-ID: <904a5edf-3f50-5f6f-6bc6-3d1a1a664aa5@grimberg.me>


> This commit itroduces support for CRC offload to direct data placement
               introduces
> ULP on the receive side. Both DDP and CRC share a common API to
> initialize the offload for a TCP socket. But otherwise, both can
> be executed independently.

 From the API it not clear that the offload engine does crc32c, do you
see this extended to other crc types in the future?

Other than this the patch looks good to me,
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From sagi at grimberg.me  Thu Oct  8 17:56:34 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 14:56:34 -0700
Subject: [PATCH net-next RFC v1 04/10] net/tls: expose get_netdev_for_sock
In-Reply-To: <20200930162010.21610-5-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-5-borisp@mellanox.com>
Message-ID: <c7f5fa38-7629-1bd6-63eb-c463daae0ef9@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From sagi at grimberg.me  Thu Oct  8 18:19:34 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 15:19:34 -0700
Subject: [PATCH net-next RFC v1 05/10] nvme-tcp: Add DDP offload control
 path
In-Reply-To: <20200930162010.21610-6-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-6-borisp@mellanox.com>
Message-ID: <c6bb16cc-fdda-3c4e-41f6-9155911aa2c8@grimberg.me>



On 9/30/20 9:20 AM, Boris Pismenny wrote:
> This commit introduces direct data placement offload to NVME
> TCP. There is a context per queue, which is established after the
> handshake
> using the tcp_ddp_sk_add/del NDOs.
> 
> Additionally, a resynchronization routine is used to assist
> hardware recovery from TCP OOO, and continue the offload.
> Resynchronization operates as follows:
> 1. TCP OOO causes the NIC HW to stop the offload
> 2. NIC HW identifies a PDU header at some TCP sequence number,
> and asks NVMe-TCP to confirm it.
> This request is delivered from the NIC driver to NVMe-TCP by first
> finding the socket for the packet that triggered the request, and
> then fiding the nvme_tcp_queue that is used by this routine.
> Finally, the request is recorded in the nvme_tcp_queue.
> 3. When NVMe-TCP observes the requested TCP sequence, it will compare
> it with the PDU header TCP sequence, and report the result to the
> NIC driver (tcp_ddp_resync), which will update the HW,
> and resume offload when all is successful.
> 
> Furthermore, we let the offloading driver advertise what is the max hw
> sectors/segments via tcp_ddp_limits.
> 
> A follow-up patch introduces the data-path changes required for this
> offload.
> 
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>   drivers/nvme/host/tcp.c  | 188 +++++++++++++++++++++++++++++++++++++++
>   include/linux/nvme-tcp.h |   2 +
>   2 files changed, 190 insertions(+)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 8f4f29f18b8c..06711ac095f2 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -62,6 +62,7 @@ enum nvme_tcp_queue_flags {
>   	NVME_TCP_Q_ALLOCATED	= 0,
>   	NVME_TCP_Q_LIVE		= 1,
>   	NVME_TCP_Q_POLLING	= 2,
> +	NVME_TCP_Q_OFFLOADS     = 3,
>   };
>   
>   enum nvme_tcp_recv_state {
> @@ -110,6 +111,8 @@ struct nvme_tcp_queue {
>   	void (*state_change)(struct sock *);
>   	void (*data_ready)(struct sock *);
>   	void (*write_space)(struct sock *);
> +
> +	atomic64_t  resync_req;
>   };
>   
>   struct nvme_tcp_ctrl {
> @@ -129,6 +132,8 @@ struct nvme_tcp_ctrl {
>   	struct delayed_work	connect_work;
>   	struct nvme_tcp_request async_req;
>   	u32			io_queues[HCTX_MAX_TYPES];
> +
> +	struct net_device       *offloading_netdev;
>   };
>   
>   static LIST_HEAD(nvme_tcp_ctrl_list);
> @@ -223,6 +228,159 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>   	return nvme_tcp_pdu_data_left(req) <= len;
>   }
>   
> +#ifdef CONFIG_TCP_DDP
> +
> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
> +const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops __read_mostly = {
> +	.resync_request		= nvme_tcp_resync_request,
> +};
> +
> +static
> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,
> +			    struct nvme_tcp_config *config)
> +{
> +	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
> +	struct tcp_ddp_config *ddp_config = (struct tcp_ddp_config *)config;
> +	int ret;
> +
> +	if (unlikely(!netdev)) {

Let's remove unlikely from non datapath routines, its slightly
confusing.

> +		pr_info_ratelimited("%s: netdev not found\n", __func__);

dev_info_ratelimited with queue->ctrl->ctrl.device ?
Also, lets remove __func__. This usually is not very helpful.

> +		return -EINVAL;
> +	}
> +
> +	if (!(netdev->features & NETIF_F_HW_TCP_DDP)) {
> +		dev_put(netdev);
> +		return -EINVAL;

EINVAL or ENODEV?

> +	}
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_sk_add(netdev,
> +						 queue->sock->sk,
> +						 ddp_config);
> +	if (!ret)
> +		inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = &nvme_tcp_ddp_ulp_ops;
> +	else
> +		dev_put(netdev);
> +	return ret;
> +}
> +
> +static
> +void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
> +{
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);

Same here.

> +		return;
> +	}
> +
> +	netdev->tcp_ddp_ops->tcp_ddp_sk_del(netdev, queue->sock->sk);
> +
> +	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = NULL;

Just a general question, why is this needed?

> +	dev_put(netdev); /* put the queue_init get_netdev_for_sock() */
> +}
> +
> +static
> +int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue,
> +			    struct tcp_ddp_limits *limits)
> +{
> +	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
> +	int ret = 0;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
> +		return -EINVAL;

Same here

> +	}
> +
> +	if (netdev->features & NETIF_F_HW_TCP_DDP &&
> +	    netdev->tcp_ddp_ops &&
> +	    netdev->tcp_ddp_ops->tcp_ddp_limits)
> +			ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, limits);
> +	else
> +			ret = -EOPNOTSUPP;
> +
> +	if (!ret) {
> +		queue->ctrl->offloading_netdev = netdev;
> +		pr_info("%s netdev %s offload limits: max_ddp_sgl_len %d\n",
> +			__func__, netdev->name, limits->max_ddp_sgl_len);

dev_info, and given that it per-queue, please make it dev_dbg.

> +		queue->ctrl->ctrl.max_segments = limits->max_ddp_sgl_len;
> +		queue->ctrl->ctrl.max_hw_sectors =
> +			limits->max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
> +	} else {
> +		queue->ctrl->offloading_netdev = NULL;

Maybe nullify in the controller setup intead?

> +	}
> +
> +	dev_put(netdev);
> +
> +	return ret;
> +}
> +
> +static
> +void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
> +			      unsigned int pdu_seq)
> +{
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	u64 resync_val;
> +	u32 resync_seq;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
> +		return;

What happens now, fallback to SW? Maybe dev_warn then..
will the SW keep seeing these responses after one failed?

> +	}
> +
> +	resync_val = atomic64_read(&queue->resync_req);
> +	if ((resync_val & TCP_DDP_RESYNC_REQ) == 0)
> +		return;
> +
> +	resync_seq = resync_val >> 32;
> +	if (before(pdu_seq, resync_seq))
> +		return;

I think it will be better to pass the skb to this func and keep the
pdu_seq contained locally.

> +
> +	if (atomic64_cmpxchg(&queue->resync_req, resync_val, (resync_val - 1)))
> +		netdev->tcp_ddp_ops->tcp_ddp_resync(netdev, queue->sock->sk, pdu_seq);

A small comment on this manipulation may help the reader.

> +}
> +
> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
> +{
> +	struct nvme_tcp_queue *queue = sk->sk_user_data;
> +
> +	atomic64_set(&queue->resync_req,
> +		     (((uint64_t)seq << 32) | flags));
> +
> +	return true;
> +}
> +
> +#else
> +
> +static
> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,
> +			    struct nvme_tcp_config *config)
> +{
> +	return -EINVAL;
> +}
> +
> +static
> +void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
> +{}
> +
> +static
> +int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue,
> +			    struct tcp_ddp_limits *limits)
> +{
> +	return -EINVAL;
> +}
> +
> +static
> +void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
> +			      unsigned int pdu_seq)
> +{}
> +
> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
> +{
> +	return false;
> +}
> +
> +#endif
> +
>   static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>   		unsigned int dir)
>   {
> @@ -628,6 +786,11 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   	size_t rcv_len = min_t(size_t, *len, queue->pdu_remaining);
>   	int ret;
>   
> +	u64 pdu_seq = TCP_SKB_CB(skb)->seq + *offset - queue->pdu_offset;
> +
> +	if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags))
> +		nvme_tcp_resync_response(queue, pdu_seq);

Here, just pass in (queue, skb)

> +
>   	ret = skb_copy_bits(skb, *offset,
>   		&pdu[queue->pdu_offset], rcv_len);
>   	if (unlikely(ret))
> @@ -1370,6 +1533,8 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl,
>   {
>   	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
>   	struct nvme_tcp_queue *queue = &ctrl->queues[qid];
> +	struct nvme_tcp_config config;
> +	struct tcp_ddp_limits limits;
>   	int ret, rcv_pdu_size;
>   
>   	queue->ctrl = ctrl;
> @@ -1487,6 +1652,26 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl,
>   #endif
>   	write_unlock_bh(&queue->sock->sk->sk_callback_lock);
>   
> +	if (nvme_tcp_queue_id(queue) != 0) {

	if (!nvme_tcp_admin_queue(queue)) {

> +		config.cfg.type		= TCP_DDP_NVME;
> +		config.pfv		= NVME_TCP_PFV_1_0;
> +		config.cpda		= 0;
> +		config.dgst		= queue->hdr_digest ?
> +						NVME_TCP_HDR_DIGEST_ENABLE : 0;
> +		config.dgst		|= queue->data_digest ?
> +						NVME_TCP_DATA_DIGEST_ENABLE : 0;
> +		config.queue_size	= queue->queue_size;
> +		config.queue_id		= nvme_tcp_queue_id(queue);
> +		config.io_cpu		= queue->io_cpu;

Can the config initialization move to nvme_tcp_offload_socket?

> +
> +		ret = nvme_tcp_offload_socket(queue, &config);
> +		if (!ret)
> +			set_bit(NVME_TCP_Q_OFFLOADS, &queue->flags);
> +	} else {
> +		ret = nvme_tcp_offload_limits(queue, &limits);
> +	}

I'm thinking that instead of this conditional, we want to place
nvme nvme_tcp_alloc_admin_queue in nvme_tcp_alloc_admin_queue, and
also move nvme_tcp_alloc_admin_queue to __nvme_tcp_alloc_io_queues
loop.

> +	/* offload is opportunistic - failure is non-critical */

Than make it void...

> +
>   	return 0;
>   
>   err_init_connect:
> @@ -1519,6 +1704,9 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
>   	kernel_sock_shutdown(queue->sock, SHUT_RDWR);
>   	nvme_tcp_restore_sock_calls(queue);
>   	cancel_work_sync(&queue->io_work);
> +
> +	if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags))
> +		nvme_tcp_unoffload_socket(queue);

Why not in nvme_tcp_free_queue, symmetric to the alloc?

>   }
>   
>   static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
> diff --git a/include/linux/nvme-tcp.h b/include/linux/nvme-tcp.h
> index 959e0bd9a913..65df64c34ecd 100644
> --- a/include/linux/nvme-tcp.h
> +++ b/include/linux/nvme-tcp.h
> @@ -8,6 +8,8 @@
>   #define _LINUX_NVME_TCP_H
>   
>   #include <linux/nvme.h>
> +#include <net/sock.h>
> +#include <net/tcp_ddp.h>

Why is this needed? I think we want to place this in tcp.c no?


From sagi at grimberg.me  Thu Oct  8 18:29:08 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 15:29:08 -0700
Subject: [PATCH net-next RFC v1 06/10] nvme-tcp: Add DDP data-path
In-Reply-To: <20200930162010.21610-7-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-7-borisp@mellanox.com>
Message-ID: <5a23d221-fd3e-5802-ce68-7edec55068bb@grimberg.me>


>   bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
> +void nvme_tcp_ddp_teardown_done(void *ddp_ctx);
>   const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops __read_mostly = {
> +
>   	.resync_request		= nvme_tcp_resync_request,
> +	.ddp_teardown_done	= nvme_tcp_ddp_teardown_done,
>   };
>   
> +static
> +int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> +			  uint16_t command_id,
> +			  struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	int ret;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);

dev_info_ratelimited

> +		return -EINVAL;
> +	}
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_teardown(netdev, queue->sock->sk,
> +						    &req->ddp, rq);
> +	sg_free_table_chained(&req->ddp.sg_table, SG_CHUNK_SIZE);
> +	req->offloaded = false;
> +	return ret;
> +}
> +
> +void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> +{
> +	struct request *rq = ddp_ctx;
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +
> +	if (!nvme_try_complete_req(rq, cpu_to_le16(req->status << 1), req->result))
> +		nvme_complete_rq(rq);
> +}
> +
> +static
> +int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> +		       uint16_t command_id,
> +		       struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	int ret;
> +
> +	req->offloaded = false;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
> +		return -EINVAL;
> +	}
> +
> +	req->ddp.command_id = command_id;
> +	req->ddp.sg_table.sgl = req->ddp.first_sgl;
> +	ret = sg_alloc_table_chained(&req->ddp.sg_table,
> +		blk_rq_nr_phys_segments(rq), req->ddp.sg_table.sgl,
> +		SG_CHUNK_SIZE);
> +	if (ret)
> +		return -ENOMEM;

newline here

> +	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
> +						 queue->sock->sk,
> +						 &req->ddp);
> +	if (!ret)
> +		req->offloaded = true;
> +	return ret;
> +}
> +
>   static
>   int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,
>   			    struct nvme_tcp_config *config)
> @@ -351,6 +422,25 @@ bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
>   
>   #else
>   
> +static
> +int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> +		       uint16_t command_id,
> +		       struct request *rq)
> +{
> +	return -EINVAL;
> +}
> +
> +static
> +int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> +			  uint16_t command_id,
> +			  struct request *rq)
> +{
> +	return -EINVAL;
> +}
> +
> +void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> +{}
> +
>   static
>   int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,
>   			    struct nvme_tcp_config *config)
> @@ -630,6 +720,7 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
>   static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		struct nvme_completion *cqe)
>   {
> +	struct nvme_tcp_request *req;
>   	struct request *rq;
>   
>   	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), cqe->command_id);
> @@ -641,8 +732,15 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		return -EINVAL;
>   	}
>   
> -	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> -		nvme_complete_rq(rq);
> +	req = blk_mq_rq_to_pdu(rq);
> +	if (req->offloaded) {
> +		req->status = cqe->status;
> +		req->result = cqe->result;
> +		nvme_tcp_teardown_ddp(queue, cqe->command_id, rq);
> +	} else {
> +		if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> +			nvme_complete_rq(rq);
> +	}
>   	queue->nr_cqe++;
>   
>   	return 0;
> @@ -836,9 +934,18 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   static inline void nvme_tcp_end_request(struct request *rq, u16 status)
>   {
>   	union nvme_result res = {};
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct nvme_tcp_queue *queue = req->queue;
> +	struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
>   
> -	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> -		nvme_complete_rq(rq);
> +	if (req->offloaded) {
> +		req->status = cpu_to_le16(status << 1);
> +		req->result = res;
> +		nvme_tcp_teardown_ddp(queue, pdu->command_id, rq);
> +	} else {
> +		if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> +			nvme_complete_rq(rq);
> +	}
>   }
>   
>   static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> @@ -1115,6 +1222,7 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>   	bool inline_data = nvme_tcp_has_inline_data(req);
>   	u8 hdgst = nvme_tcp_hdgst_len(queue);
>   	int len = sizeof(*pdu) + hdgst - req->offset;
> +	struct request *rq = blk_mq_rq_from_pdu(req);
>   	int flags = MSG_DONTWAIT;
>   	int ret;
>   
> @@ -1123,6 +1231,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>   	else
>   		flags |= MSG_EOR;
>   
> +	if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags) &&
> +	    blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
> +		nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);

I'd assume that this is something we want to setup in
nvme_tcp_setup_cmd_pdu. Why do it here?


From sagi at grimberg.me  Thu Oct  8 18:44:07 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 15:44:07 -0700
Subject: [PATCH net-next RFC v1 07/10] nvme-tcp : Recalculate crc in the
 end of the capsule
In-Reply-To: <20200930162010.21610-8-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-8-borisp@mellanox.com>
Message-ID: <a17cf1ca-4183-8f6c-8470-9d45febb755b@grimberg.me>


> crc offload of the nvme capsule. Check if all the skb bits
> are on, and if not recalculate the crc in SW and check it.

Can you clarify in the patch description that this is only
for pdu data digest and not header digest?

> 
> This patch reworks the receive-side crc calculation to always
> run at the end, so as to keep a single flow for both offload
> and non-offload. This change simplifies the code, but it may degrade
> performance for non-offload crc calculation.

??

 From my scan it doeesn't look like you do that.. Am I missing something?
Can you explain?

> 
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>   drivers/nvme/host/tcp.c | 66 ++++++++++++++++++++++++++++++++++++-----
>   1 file changed, 58 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 7bd97f856677..9a620d1dacb4 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -94,6 +94,7 @@ struct nvme_tcp_queue {
>   	size_t			data_remaining;
>   	size_t			ddgst_remaining;
>   	unsigned int		nr_cqe;
> +	bool			crc_valid;
>   
>   	/* send state */
>   	struct nvme_tcp_request *request;
> @@ -233,6 +234,41 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>   	return nvme_tcp_pdu_data_left(req) <= len;
>   }
>   
> +static inline bool nvme_tcp_device_ddgst_ok(struct nvme_tcp_queue *queue)

Maybe call it nvme_tcp_ddp_ddgst_ok?

> +{
> +	return queue->crc_valid;
> +}
> +
> +static inline void nvme_tcp_device_ddgst_update(struct nvme_tcp_queue *queue,
> +						struct sk_buff *skb)

Maybe call it nvme_tcp_ddp_ddgst_update?

> +{
> +	if (queue->crc_valid)
> +#ifdef CONFIG_TCP_DDP_CRC
> +		queue->crc_valid = skb->ddp_crc;
> +#else
> +		queue->crc_valid = false;
> +#endif
> +}
> +
> +static void nvme_tcp_crc_recalculate(struct nvme_tcp_queue *queue,
> +				     struct nvme_tcp_data_pdu *pdu)

Maybe call it nvme_tcp_ddp_ddgst_recalc?

> +{
> +	struct nvme_tcp_request *req;
> +	struct request *rq;
> +
> +	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
> +	if (!rq)
> +		return;
> +	req = blk_mq_rq_to_pdu(rq);
> +	crypto_ahash_init(queue->rcv_hash);
> +	req->ddp.sg_table.sgl = req->ddp.first_sgl;
> +	/* req->ddp.sg_table is allocated and filled in nvme_tcp_setup_ddp */
> +	ahash_request_set_crypt(queue->rcv_hash, req->ddp.sg_table.sgl, NULL,
> +				le32_to_cpu(pdu->data_length));
> +	crypto_ahash_update(queue->rcv_hash);
> +}
> +
> +
>   #ifdef CONFIG_TCP_DDP
>   
>   bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
> @@ -706,6 +742,7 @@ static void nvme_tcp_init_recv_ctx(struct nvme_tcp_queue *queue)
>   	queue->pdu_offset = 0;
>   	queue->data_remaining = -1;
>   	queue->ddgst_remaining = 0;
> +	queue->crc_valid = true;
>   }
>   
>   static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
> @@ -955,6 +992,8 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   	struct nvme_tcp_request *req;
>   	struct request *rq;
>   
> +	if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags))
> +		nvme_tcp_device_ddgst_update(queue, skb);

Is the queue->data_digest condition missing here?

>   	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
>   	if (!rq) {
>   		dev_err(queue->ctrl->ctrl.device,
> @@ -992,7 +1031,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   		recv_len = min_t(size_t, recv_len,
>   				iov_iter_count(&req->iter));
>   
> -		if (queue->data_digest)
> +		if (queue->data_digest && !test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags))
>   			ret = skb_copy_and_hash_datagram_iter(skb, *offset,
>   				&req->iter, recv_len, queue->rcv_hash);

This is the skb copy and hash, not clear why you say that you move this
to the end...

>   		else
> @@ -1012,7 +1051,6 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   
>   	if (!queue->data_remaining) {
>   		if (queue->data_digest) {
> -			nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);

If I instead do:
			if (!test_bit(NVME_TCP_Q_OFFLOADS,
                                       &queue->flags))
				nvme_tcp_ddgst_final(queue->rcv_hash,
						     &queue->exp_ddgst);

Does that help the mess in nvme_tcp_recv_ddgst?

>   			queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH;
>   		} else {
>   			if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
> @@ -1033,8 +1071,11 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
>   	char *ddgst = (char *)&queue->recv_ddgst;
>   	size_t recv_len = min_t(size_t, *len, queue->ddgst_remaining);
>   	off_t off = NVME_TCP_DIGEST_LENGTH - queue->ddgst_remaining;
> +	bool ddgst_offload_fail;
>   	int ret;
>   
> +	if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags))
> +		nvme_tcp_device_ddgst_update(queue, skb);
>   	ret = skb_copy_bits(skb, *offset, &ddgst[off], recv_len);
>   	if (unlikely(ret))
>   		return ret;
> @@ -1045,12 +1086,21 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
>   	if (queue->ddgst_remaining)
>   		return 0;
>   
> -	if (queue->recv_ddgst != queue->exp_ddgst) {
> -		dev_err(queue->ctrl->ctrl.device,
> -			"data digest error: recv %#x expected %#x\n",
> -			le32_to_cpu(queue->recv_ddgst),
> -			le32_to_cpu(queue->exp_ddgst));
> -		return -EIO;
> +	ddgst_offload_fail = !nvme_tcp_device_ddgst_ok(queue);
> +	if (!test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags) ||
> +	    ddgst_offload_fail) {
> +		if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags) &&
> +		    ddgst_offload_fail)
> +			nvme_tcp_crc_recalculate(queue, pdu);
> +
> +		nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
> +		if (queue->recv_ddgst != queue->exp_ddgst) {
> +			dev_err(queue->ctrl->ctrl.device,
> +				"data digest error: recv %#x expected %#x\n",
> +				le32_to_cpu(queue->recv_ddgst),
> +				le32_to_cpu(queue->exp_ddgst));
> +			return -EIO;

This gets convoluted here...

> +		}
>   	}
>   
>   	if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
> 


From sagi at grimberg.me  Thu Oct  8 18:47:37 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 15:47:37 -0700
Subject: [PATCH net-next RFC v1 08/10] nvme-tcp: Deal with netdevice DOWN
 events
In-Reply-To: <20200930162010.21610-9-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-9-borisp@mellanox.com>
Message-ID: <67e29f83-5bab-4abd-44c0-9c5ae29d5784@grimberg.me>



On 9/30/20 9:20 AM, Boris Pismenny wrote:
> From: Or Gerlitz <ogerlitz at mellanox.com>
> 
> For ddp setup/teardown and resync, the offloading logic
> uses HW resources at the NIC driver such as SQ and CQ.
> 
> These resources are destroyed when the netdevice does down
> and hence we must stop using them before the NIC driver
> destroyes them.
> 
> Use netdevice notifier for that matter -- offloaded connections
> are stopped before the stack continues to call the NIC driver
> close ndo.
> 
> We use the existing recovery flow which has the advantage
> of resuming the offload once the connection is re-set.
> 
> Since the recovery flow runs in a separate/dedicated WQ
> we need to wait in the notifier code for an ACK that all
> offloaded queues were stopped which means that the teardown
> queue offload ndo was called and the NIC doesn't have any
> resources related to that connection any more.
> 
> This also buys us proper handling for the UNREGISTER event
> b/c our offloading starts in the UP state, and down is always
> there between up to unregister.
> 
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>   drivers/nvme/host/tcp.c | 39 +++++++++++++++++++++++++++++++++++++--
>   1 file changed, 37 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 9a620d1dacb4..7569b47f0414 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -144,6 +144,7 @@ struct nvme_tcp_ctrl {
>   
>   static LIST_HEAD(nvme_tcp_ctrl_list);
>   static DEFINE_MUTEX(nvme_tcp_ctrl_mutex);
> +static struct notifier_block nvme_tcp_netdevice_nb;
>   static struct workqueue_struct *nvme_tcp_wq;
>   static const struct blk_mq_ops nvme_tcp_mq_ops;
>   static const struct blk_mq_ops nvme_tcp_admin_mq_ops;
> @@ -412,8 +413,6 @@ int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue,
>   		queue->ctrl->ctrl.max_segments = limits->max_ddp_sgl_len;
>   		queue->ctrl->ctrl.max_hw_sectors =
>   			limits->max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
> -	} else {
> -		queue->ctrl->offloading_netdev = NULL;

Squash this change to the patch that introduced it.

>   	}
>   
>   	dev_put(netdev);
> @@ -1992,6 +1991,8 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
>   {
>   	int ret;
>   
> +	to_tcp_ctrl(ctrl)->offloading_netdev = NULL;
> +
>   	ret = nvme_tcp_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
>   	if (ret)
>   		return ret;
> @@ -2885,6 +2886,26 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
>   	return ERR_PTR(ret);
>   }
>   
> +static int nvme_tcp_netdev_event(struct notifier_block *this,
> +				 unsigned long event, void *ptr)
> +{
> +	struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
> +	struct nvme_tcp_ctrl *ctrl;
> +
> +	switch (event) {
> +	case NETDEV_GOING_DOWN:
> +		mutex_lock(&nvme_tcp_ctrl_mutex);
> +		list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list) {
> +			if (ndev != ctrl->offloading_netdev)
> +				continue;
> +			nvme_tcp_error_recovery(&ctrl->ctrl);
> +		}
> +		mutex_unlock(&nvme_tcp_ctrl_mutex);
> +		flush_workqueue(nvme_reset_wq);

Worth a small comment that this we want the err_work to complete
here. So if someone changes workqueues he may see this.

> +	}
> +	return NOTIFY_DONE;
> +}
> +
>   static struct nvmf_transport_ops nvme_tcp_transport = {
>   	.name		= "tcp",
>   	.module		= THIS_MODULE,


From sagi at grimberg.me  Thu Oct  8 19:00:30 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 16:00:30 -0700
Subject: [PATCH net-next RFC v1 06/10] nvme-tcp: Add DDP data-path
In-Reply-To: <5a23d221-fd3e-5802-ce68-7edec55068bb@grimberg.me>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-7-borisp@mellanox.com>
 <5a23d221-fd3e-5802-ce68-7edec55068bb@grimberg.me>
Message-ID: <24ea956e-40a2-8b7b-cf8a-b604e7cd5644@grimberg.me>


>> ? static
>> ? int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,
>> ????????????????? struct nvme_tcp_config *config)
>> @@ -630,6 +720,7 @@ static void nvme_tcp_error_recovery(struct 
>> nvme_ctrl *ctrl)
>> ? static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>> ????????? struct nvme_completion *cqe)
>> ? {
>> +??? struct nvme_tcp_request *req;
>> ????? struct request *rq;
>> ????? rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), cqe->command_id);
>> @@ -641,8 +732,15 @@ static int nvme_tcp_process_nvme_cqe(struct 
>> nvme_tcp_queue *queue,
>> ????????? return -EINVAL;
>> ????? }
>> -??? if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
>> -??????? nvme_complete_rq(rq);
>> +??? req = blk_mq_rq_to_pdu(rq);
>> +??? if (req->offloaded) {
>> +??????? req->status = cqe->status;
>> +??????? req->result = cqe->result;
>> +??????? nvme_tcp_teardown_ddp(queue, cqe->command_id, rq);
>> +??? } else {
>> +??????? if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
>> +??????????? nvme_complete_rq(rq);
>> +??? }

Oh forgot to ask,

We have places in the driver that we may complete (cancel) one
or more requests from the error recovery or timeout flow. We
first prevent future incoming RX on the socket such that we
can safely cancel requests. This may break with the deferred
completion in ddp_teardown_done.

If I have a request that is waiting for ddp_teardown_done do
I have a way to tell the HW to never call ddp_teardown_done
on a specific socket?

If so the place to is in nvme_tcp_stop_queue.


From sagi at grimberg.me  Thu Oct  8 19:05:43 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 16:05:43 -0700
Subject: [PATCH net-next RFC v1 01/10] iov_iter: Skip copy in
 memcpy_to_page if src==dst
In-Reply-To: <20200930162010.21610-2-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-2-borisp@mellanox.com>
Message-ID: <3820c19e-0fea-9345-0e0e-692119f7aa84@grimberg.me>

You probably want Al to have a look at this..


From sagi at grimberg.me  Thu Oct  8 20:08:06 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 17:08:06 -0700
Subject: [PATCH net-next RFC v1 00/10] nvme-tcp receive offloads
In-Reply-To: <20200930162010.21610-1-borisp@mellanox.com>
References: <20200930162010.21610-1-borisp@mellanox.com>
Message-ID: <42560022-6b2e-327f-2a77-0700132ab730@grimberg.me>



On 9/30/20 9:20 AM, Boris Pismenny wrote:
> This series adds support for nvme-tcp receive offloads
> which do not mandate the offload of the network stack to the device.
> Instead, these work together with TCP to offload:
> 1. copy from SKB to the block layer buffers
> 2. CRC verification for received PDU
> 
> The series implements these as a generic offload infrastructure for storage
> protocols, which we call TCP Direct Data Placement (TCP_DDP) and TCP DDP CRC,
> respectively. We use this infrastructure to implement NVMe-TCP offload for copy
> and CRC. Future implementations can reuse the same infrastructure for other
> protcols such as iSCSI.
> 
> Note:
> These offloads are similar in nature to the packet-based NIC TLS offloads,
> which are already upstream (see net/tls/tls_device.c).
> You can read more about TLS offload here:
> https://www.kernel.org/doc/html/latest/networking/tls-offload.html
> 
> Initialization and teardown:
> =========================================
> The offload for IO queues is initialized after the handshake of the
> NVMe-TCP protocol is finished by calling `nvme_tcp_offload_socket`
> with the tcp socket of the nvme_tcp_queue:
> This operation sets all relevant hardware contexts in
> hardware. If it fails, then the IO queue proceeds as usually with no offload.
> If it succeeds then `nvme_tcp_setup_ddp` and `nvme_tcp_teardown_ddp` may be
> called to perform copy offload, and crc offload will be used.
> This initialization does not change the normal operation of nvme-tcp in any
> way besides adding the option to call the above mentioned NDO operations.
> 
> For the admin queue, nvme-tcp does not initialize the offload.
> Instead, nvme-tcp calls the driver to configure limits for the controller,
> such as max_hw_sectors and max_segments; these must be limited to accomodate
> potential HW resource limits, and to improve performance.
> 
> If some error occured, and the IO queue must be closed or reconnected, then
> offload is teardown and initialized again. Additionally, we handle netdev
> down events via the existing error recovery flow.
> 
> Copy offload works as follows:
> =========================================
> The nvme-tcp layer calls the NIC drive to map block layer buffers to ccid using
> `nvme_tcp_setup_ddp` before sending the read request. When the repsonse is
> received, then the NIC HW will write the PDU payload directly into the
> designated buffer, and build an SKB such that it points into the destination
> buffer; this SKB represents the entire packet received on the wire, but it
> points to the block layer buffers. Once nvme-tcp attempts to copy data from
> this SKB to the block layer buffer it can skip the copy by checking in the
> copying function (memcpy_to_page):
> if (src == dst) -> skip copy
> Finally, when the PDU has been processed to completion, the nvme-tcp layer
> releases the NIC HW context be calling `nvme_tcp_teardown_ddp` which
> asynchronously unmaps the buffers from NIC HW.
> 
> As the last change is to a sensative function, we are careful to place it under
> static_key which is only enabled when this functionality is actually used for
> nvme-tcp copy offload.
> 
> Asynchronous completion:
> =========================================
> The NIC must release its mapping between command IDs and the target buffers.
> This mapping is released when NVMe-TCP calls the NIC
> driver (`nvme_tcp_offload_socket`).
> As completing IOs is performance criticial, we introduce asynchronous
> completions for NVMe-TCP, i.e. NVMe-TCP calls the NIC, which will later
> call NVMe-TCP to complete the IO (`nvme_tcp_ddp_teardown_done`).
> 
> An alternative approach is to move all the functions related to coping from
> SKBs to the block layer buffers inside the nvme-tcp code - about 200 LOC.
> 
> CRC offload works as follows:
> =========================================
> After offload is initialized, we use the SKB's ddp_crc bit to indicate that:
> "there was no problem with the verification of all CRC fields in this packet's
> payload". The bit is set to zero if there was an error, or if HW skipped
> offload for some reason. If *any* SKB in a PDU has (ddp_crc != 1), then software
> must compute the CRC, and check it. We perform this check, and
> accompanying software fallback at the end of the processing of a received PDU.
> 
> SKB changes:
> =========================================
> The CRC offload requires an additional bit in the SKB, which is useful for
> preventing the coalescing of SKB with different crc offload values. This bit
> is similar in concept to the "decrypted" bit.
> 
> Performance:
> =========================================
> The expected performance gain from this offload varies with the block size.
> We perform a CPU cycles breakdown of the copy/CRC operations in nvme-tcp
> fio random read workloads:
> For 4K blocks we see up to 11% improvement for a 100% read fio workload,
> while for 128K blocks we see upto 52%. If we run nvme-tcp, and skip these
> operations, then we observe a gain of about 1.1x and 2x respectively.

Nice!

> 
> Resynchronization:
> =========================================
> The resynchronization flow is performed to reset the hardware tracking of
> NVMe-TCP PDUs within the TCP stream. The flow consists of a request from
> the driver, regarding a possible location of a PDU header. Followed by
> a response from the nvme-tcp driver.
> 
> This flow is rare, and it should happen only after packet loss or
> reordering events that involve nvme-tcp PDU headers.
> 
> The patches are organized as follows:
> =========================================
> Patch 1         the iov_iter change to skip copy if (src == dst)
> Patches 2-3     the infrastructure for all TCP DDP
>                  and TCP DDP CRC offloads, respectively.
> Patch 4         exposes the get_netdev_for_sock function from TLS
> Patch 5         NVMe-TCP changes to call NIC driver on queue init/teardown
> Patches 6       NVMe-TCP changes to call NIC driver on IO operation
>                  setup/teardown, and support async completions.
> Patches 7       NVMe-TCP changes to support CRC offload on receive.
>                  Also, this patch moves CRC calculation to the end of PDU
>                  in case offload requires software fallback.
> Patches 8       NVMe-TCP handling of netdev events: stop the offload if
>                  netdev is going down
> Patches 9-10    implement support for NVMe-TCP copy and CRC offload in
>                  the mlx5 NIC driver
> 
> Testing:
> =========================================
> This series was tested using fio with various configurations of IO sizes,
> depths, MTUs, and with both the SPDK and kernel NVMe-TCP targets.
> 
> Future work:
> =========================================
> A follow-up series will introduce support for transmit side CRC. Then,
> we will work on adding support for TLS in NVMe-TCP and combining the
> two offloads.

Boris, Or and Yoray

Thanks for submitting this work. Overall this looks good to me.
The model here is not messy at all which is not trivial when it comes to 
tcp offloads.

Gave you comments in the patches themselves but overall this looks
good!

Would love to see TLS work from you moving forward.


From sagi at grimberg.me  Thu Oct  8 20:24:12 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 8 Oct 2020 17:24:12 -0700
Subject: Exporting NVMe-oF devices read-only enforced on export from the
 NVMe-oF target.
In-Reply-To: <BYAPR04MB4965DEBF8F5F06FC026B502786330@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <BBC87AAC-A523-41F0-8CDE-38757C407C7F@primelogic.nl>
 <BYAPR04MB4965DEBF8F5F06FC026B502786330@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <f4445536-ed3b-203c-95ee-bac737e5574c@grimberg.me>


>> Unless I am mistaken the NVMe-oF target driver does currently not support to export a device read-only from the target system.
>> For exporting readonly snapshot devices I am currently using the attached patch.
> 
> It does, have you tried using nvme cli with NVME_NS_WRITE_PROTECT feature ?
> 
> we have both host and core patched for read-only mode, if you find any bug
> 
> please let me know.

Chaitnaya,

btw, I just noticed that 1293477f4f32 ("nvme: set gendisk read only 
based on nsattr") breaks BLKROSET ioctls.

If I now do:
blockdev --setro /dev/nvme0n1
and then I will write to it, it will override the disk ro as
invalidate kicks in when the write fails.

I think we need to start by reverting this patch, and then
add it back in a way that doesn't break BLKROSET...


From ming.lei at redhat.com  Fri Oct  9 00:39:38 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Fri, 9 Oct 2020 12:39:38 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201008213750.899462-1-sagi@grimberg.me>
References: <20201008213750.899462-1-sagi@grimberg.me>
Message-ID: <20201009043938.GC27356@T590>

On Thu, Oct 08, 2020 at 02:37:50PM -0700, Sagi Grimberg wrote:
> In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
> needs to complete an aborted request. The timeout handler serializes
> against an error recovery sequence that iterates over the inflight
> requests and cancels them.
> 
> However, the fact that blk_mq_complete_request may defer to a different
> core that serialization breaks.
> 
> Hence re-introduce blk_mq_complete_request_sync and use that in the
> timeout handler to make sure that we don't get a use-after-free
> condition.
> 
> This was uncovered by the blktests:
> --
> $ nvme_trtype=tcp ./check nvme/012
> 
> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
> [ 2152.799640] loop: module loaded
> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
> [ 2152.937889] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
> [ 2154.942432] XFS (nvme0n1): Ending clean mount
> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
> [ 2215.199331] nvme nvme0: starting error recovery
> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
> [ 2215.215502] ------------[ cut here ]------------
> [ 2215.215505] refcount_t: underflow; use-after-free.
> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff93f37dcd8d08
> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff93f37dcd8d00
> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: ffffffffa7fffbc4
> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: ffff93f37a8124e8
> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: ffff93f37a2b0000
> [ 2215.215765] FS:  0000000000000000(0000) GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
> [ 2215.215768] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 00000000000006e0
> [ 2215.215773] Call Trace:
> [ 2215.215784]  blk_mq_check_expired+0x109/0x1b0
> [ 2215.215797]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
> [ 2215.215801]  ? blk_poll+0x300/0x300
> [ 2215.215806]  blk_mq_timeout_work+0x44/0xe0
> [ 2215.215814]  process_one_work+0x1b4/0x370
> [ 2215.215820]  worker_thread+0x53/0x3e0
> [ 2215.215825]  ? process_one_work+0x370/0x370
> [ 2215.215829]  kthread+0x11b/0x140
> [ 2215.215834]  ? __kthread_bind_mask+0x60/0x60
> [ 2215.215841]  ret_from_fork+0x22/0x30
> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
> --
> 
> Reported-by: Yi Zhang <yi.zhang at redhat.com>
> Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
> Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
> ---
> Yi, would appreciate your Tested-by tag
> 
>  block/blk-mq.c           | 7 +++++++
>  drivers/nvme/host/rdma.c | 2 +-
>  drivers/nvme/host/tcp.c  | 2 +-
>  include/linux/blk-mq.h   | 1 +
>  4 files changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index f8e1e759c905..2d154722ef39 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct request *rq)
>  }
>  EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
>  
> +void blk_mq_complete_request_sync(struct request *rq)
> +{
> +	WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
> +	rq->q->mq_ops->complete(rq);
> +}
> +EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
> +
>  /**
>   * blk_mq_complete_request - end I/O on a request
>   * @rq:		the request being processed
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 9e378d0a0c01..45fc605349da 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1975,7 +1975,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
>  	nvme_rdma_stop_queue(queue);
>  	if (!blk_mq_request_completed(rq)) {
>  		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> -		blk_mq_complete_request(rq);
> +		blk_mq_complete_request_sync(rq);
>  	}
>  	mutex_unlock(&ctrl->teardown_lock);
>  }
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 8f4f29f18b8c..629b025685d1 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>  	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>  	if (!blk_mq_request_completed(rq)) {
>  		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> -		blk_mq_complete_request(rq);
> +		blk_mq_complete_request_sync(rq);

Or complete the request in the following way? Then one block layer API
can be saved:

	blk_mq_complete_request_remote(rq);
	nvme_complete_rq(rq);


thanks,
Ming



From yi.zhang at redhat.com  Fri Oct  9 01:03:50 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Fri, 9 Oct 2020 01:03:50 -0400 (EDT)
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201009043938.GC27356@T590>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
Message-ID: <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>

Hi Sagi

I applied this patch on block origin/for-next and still can reproduce it.

[  724.267865] run blktests nvme/012 at 2020-10-09 00:48:52
[  724.518498] loop: module loaded
[  724.552880] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[  724.586609] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[  724.648898] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[  724.663300] nvme nvme0: creating 12 I/O queues.
[  724.669826] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[  724.681350] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
[  726.641605] XFS (nvme0n1): Mounting V5 Filesystem
[  726.654637] XFS (nvme0n1): Ending clean mount
[  726.660302] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
[  787.042066] nvme nvme0: queue 2: timeout request 0xe type 4
[  787.047649] nvme nvme0: starting error recovery
[  787.052211] nvme nvme0: queue 4: timeout request 0x61 type 4
[  787.053407] block nvme0n1: no usable path - requeuing I/O
[  787.054543] nvme nvme0: Reconnecting in 10 seconds...
[  787.057877] ------------[ cut here ]------------
[  787.063274] block nvme0n1: no usable path - requeuing I/O
[  787.068313] refcount_t: underflow; use-after-free.
[  787.068370] WARNING: CPU: 3 PID: 30 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
[  787.072941] block nvme0n1: no usable path - requeuing I/O
[  787.072945] block nvme0n1: no usable path - requeuing I/O
[  787.078333] Modules linked in: loop nvme_tcp nvme_fabrics nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer amd64_edac_mod snd edac_mce_amd soundcore kvm_amd ccp kvm pcspkr irqbypass hp_wmi sparse_keymap rfkill k10temp wmi_bmof nv_tco i2c_nforce2 acpi_cpufreq ip_tables xfs nouveau video mxm_wmi i2c_algo_bit drm_kms_helper mptsas cec scsi_transport_sas ttm mptscsih ata_generic firewire_ohci pata_acpi firewire_core drm serio_raw mptbase forcedeth crc_itu_t sata_nv pata_amd wmi floppy
[  787.083142] block nvme0n1: no usable path - requeuing I/O
[  787.091304] CPU: 3 PID: 30 Comm: kworker/3:0H Not tainted 5.9.0-rc8.update+ #3
[  787.096673] block nvme0n1: no usable path - requeuing I/O
[  787.096674] block nvme0n1: no usable path - requeuing I/O
[  787.096680] block nvme0n1: no usable path - requeuing I/O
[  787.096681] block nvme0n1: no usable path - requeuing I/O
[  787.096683] block nvme0n1: no usable path - requeuing I/O
[  787.201242] Hardware name: Hewlett-Packard HP xw9400 Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
[  787.210366] Workqueue: kblockd blk_mq_timeout_work
[  787.215161] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
[  787.220295] Code: 05 aa 98 22 01 01 e8 9f cf ad ff 0f 0b c3 80 3d 98 98 22 01 00 75 95 48 c7 c7 48 54 3e a2 c6 05 88 98 22 01 01 e8 80 cf ad ff <0f> 0b c3 80 3d 77 98 22 01 00 0f 85 72 ff ff ff 48 c7 c7 a0 54 3e
[  787.239021] RSP: 0018:ffffb860407b3dc8 EFLAGS: 00010292
[  787.244240] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff9d93ddc58d08
[  787.251361] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff9d93ddc58d00
[  787.258480] RBP: ffff9d93dbd6c200 R08: 000000b740ed60e8 R09: ffffffffa2fffbc4
[  787.265612] R10: 0000000000000477 R11: 0000000000028320 R12: ffff9d93dbd6c2e8
[  787.272741] R13: ffff9d93dbea0000 R14: ffffb860407b3e70 R15: ffff9d93dbea0000
[  787.279864] FS:  0000000000000000(0000) GS:ffff9d93ddc40000(0000) knlGS:0000000000000000
[  787.287948] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  787.293695] CR2: 00007f45fd8fa024 CR3: 00000000792dc000 CR4: 00000000000006e0
[  787.300820] Call Trace:
[  787.303277]  blk_mq_check_expired+0x109/0x1b0
[  787.307638]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
[  787.312524]  ? blk_poll+0x300/0x300
[  787.316022]  blk_mq_timeout_work+0x44/0xe0
[  787.320128]  process_one_work+0x1b4/0x370
[  787.324137]  worker_thread+0x53/0x3e0
[  787.327809]  ? process_one_work+0x370/0x370
[  787.332015]  kthread+0x11b/0x140
[  787.335245]  ? __kthread_bind_mask+0x60/0x60
[  787.339515]  ret_from_fork+0x22/0x30
[  787.343094] ---[ end trace 606cf9189379fcfc ]---


Best Regards,
  Yi Zhang


----- Original Message -----
From: "Ming Lei" <ming.lei at redhat.com>
To: "Sagi Grimberg" <sagi at grimberg.me>
Cc: "Jens Axboe" <axboe at kernel.dk>, "Yi Zhang" <yi.zhang at redhat.com>, linux-nvme at lists.infradead.org, linux-block at vger.kernel.org, "Keith Busch" <kbusch at kernel.org>, "Christoph Hellwig" <hch at lst.de>
Sent: Friday, October 9, 2020 12:39:38 PM
Subject: Re: [PATCH] block: re-introduce blk_mq_complete_request_sync

On Thu, Oct 08, 2020 at 02:37:50PM -0700, Sagi Grimberg wrote:
> In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
> needs to complete an aborted request. The timeout handler serializes
> against an error recovery sequence that iterates over the inflight
> requests and cancels them.
> 
> However, the fact that blk_mq_complete_request may defer to a different
> core that serialization breaks.
> 
> Hence re-introduce blk_mq_complete_request_sync and use that in the
> timeout handler to make sure that we don't get a use-after-free
> condition.
> 
> This was uncovered by the blktests:
> --
> $ nvme_trtype=tcp ./check nvme/012
> 
> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
> [ 2152.799640] loop: module loaded
> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
> [ 2152.937889] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
> [ 2154.942432] XFS (nvme0n1): Ending clean mount
> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
> [ 2215.199331] nvme nvme0: starting error recovery
> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
> [ 2215.215502] ------------[ cut here ]------------
> [ 2215.215505] refcount_t: underflow; use-after-free.
> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff93f37dcd8d08
> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff93f37dcd8d00
> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: ffffffffa7fffbc4
> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: ffff93f37a8124e8
> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: ffff93f37a2b0000
> [ 2215.215765] FS:  0000000000000000(0000) GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
> [ 2215.215768] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 00000000000006e0
> [ 2215.215773] Call Trace:
> [ 2215.215784]  blk_mq_check_expired+0x109/0x1b0
> [ 2215.215797]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
> [ 2215.215801]  ? blk_poll+0x300/0x300
> [ 2215.215806]  blk_mq_timeout_work+0x44/0xe0
> [ 2215.215814]  process_one_work+0x1b4/0x370
> [ 2215.215820]  worker_thread+0x53/0x3e0
> [ 2215.215825]  ? process_one_work+0x370/0x370
> [ 2215.215829]  kthread+0x11b/0x140
> [ 2215.215834]  ? __kthread_bind_mask+0x60/0x60
> [ 2215.215841]  ret_from_fork+0x22/0x30
> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
> --
> 
> Reported-by: Yi Zhang <yi.zhang at redhat.com>
> Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
> Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
> ---
> Yi, would appreciate your Tested-by tag
> 
>  block/blk-mq.c           | 7 +++++++
>  drivers/nvme/host/rdma.c | 2 +-
>  drivers/nvme/host/tcp.c  | 2 +-
>  include/linux/blk-mq.h   | 1 +
>  4 files changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index f8e1e759c905..2d154722ef39 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct request *rq)
>  }
>  EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
>  
> +void blk_mq_complete_request_sync(struct request *rq)
> +{
> +	WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
> +	rq->q->mq_ops->complete(rq);
> +}
> +EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
> +
>  /**
>   * blk_mq_complete_request - end I/O on a request
>   * @rq:		the request being processed
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 9e378d0a0c01..45fc605349da 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1975,7 +1975,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
>  	nvme_rdma_stop_queue(queue);
>  	if (!blk_mq_request_completed(rq)) {
>  		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> -		blk_mq_complete_request(rq);
> +		blk_mq_complete_request_sync(rq);
>  	}
>  	mutex_unlock(&ctrl->teardown_lock);
>  }
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 8f4f29f18b8c..629b025685d1 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>  	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>  	if (!blk_mq_request_completed(rq)) {
>  		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> -		blk_mq_complete_request(rq);
> +		blk_mq_complete_request_sync(rq);

Or complete the request in the following way? Then one block layer API
can be saved:

	blk_mq_complete_request_remote(rq);
	nvme_complete_rq(rq);


thanks,
Ming


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme



From sagi at grimberg.me  Fri Oct  9 03:57:59 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 9 Oct 2020 00:57:59 -0700
Subject: Exporting NVMe-oF devices read-only enforced on export from the
 NVMe-oF target.
In-Reply-To: <BYAPR04MB496559D7BD43F180E366D91A86080@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <BBC87AAC-A523-41F0-8CDE-38757C407C7F@primelogic.nl>
 <BYAPR04MB4965DEBF8F5F06FC026B502786330@BYAPR04MB4965.namprd04.prod.outlook.com>
 <f4445536-ed3b-203c-95ee-bac737e5574c@grimberg.me>
 <BYAPR04MB496559D7BD43F180E366D91A86080@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <759e98a7-5609-bc5b-7d90-8e64ffe085d0@grimberg.me>


>> btw, I just noticed that 1293477f4f32 ("nvme: set gendisk read only
>> based on nsattr") breaks BLKROSET ioctls.
>>
>> If I now do:
>> blockdev --setro /dev/nvme0n1
>> and then I will write to it, it will override the disk ro as
>> invalidate kicks in when the write fails.
>>
>> I think we need to start by reverting this patch, and then
>> add it back in a way that doesn't break BLKROSET...
> 
> Thanks for reporting this, let me add this fix in the V3
> 
> [PATCH V2 0/4] nvmet: allow target to export readonly ns.

To be clear,

I'm talking about commit 1293477f4f32.


From sagi at grimberg.me  Fri Oct  9 04:09:16 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 9 Oct 2020 01:09:16 -0700
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
Message-ID: <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>

> Hi Sagi
> 
> I applied this patch on block origin/for-next and still can reproduce it.

That's unexpected, can you try this patch?
--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 629b025685d1..46428ff0b0fc 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct 
request *rq)
         /* fence other contexts that may complete the command */
         mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
         nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
-       if (!blk_mq_request_completed(rq)) {
+       if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
                 nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
                 blk_mq_complete_request_sync(rq);
         }
--

> 
> [  724.267865] run blktests nvme/012 at 2020-10-09 00:48:52
> [  724.518498] loop: module loaded
> [  724.552880] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
> [  724.586609] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
> [  724.648898] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [  724.663300] nvme nvme0: creating 12 I/O queues.
> [  724.669826] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [  724.681350] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
> [  726.641605] XFS (nvme0n1): Mounting V5 Filesystem
> [  726.654637] XFS (nvme0n1): Ending clean mount
> [  726.660302] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
> [  787.042066] nvme nvme0: queue 2: timeout request 0xe type 4
> [  787.047649] nvme nvme0: starting error recovery
> [  787.052211] nvme nvme0: queue 4: timeout request 0x61 type 4
> [  787.053407] block nvme0n1: no usable path - requeuing I/O
> [  787.054543] nvme nvme0: Reconnecting in 10 seconds...
> [  787.057877] ------------[ cut here ]------------
> [  787.063274] block nvme0n1: no usable path - requeuing I/O
> [  787.068313] refcount_t: underflow; use-after-free.
> [  787.068370] WARNING: CPU: 3 PID: 30 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
> [  787.072941] block nvme0n1: no usable path - requeuing I/O
> [  787.072945] block nvme0n1: no usable path - requeuing I/O
> [  787.078333] Modules linked in: loop nvme_tcp nvme_fabrics nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer amd64_edac_mod snd edac_mce_amd soundcore kvm_amd ccp kvm pcspkr irqbypass hp_wmi sparse_keymap rfkill k10temp wmi_bmof nv_tco i2c_nforce2 acpi_cpufreq ip_tables xfs nouveau video mxm_wmi i2c_algo_bit drm_kms_helper mptsas cec scsi_transport_sas ttm mptscsih ata_generic firewire_ohci pata_acpi firewire_core drm serio_raw mptbase forcedeth crc_itu_t sata_nv pata_amd wmi floppy
> [  787.083142] block nvme0n1: no usable path - requeuing I/O
> [  787.091304] CPU: 3 PID: 30 Comm: kworker/3:0H Not tainted 5.9.0-rc8.update+ #3
> [  787.096673] block nvme0n1: no usable path - requeuing I/O
> [  787.096674] block nvme0n1: no usable path - requeuing I/O
> [  787.096680] block nvme0n1: no usable path - requeuing I/O
> [  787.096681] block nvme0n1: no usable path - requeuing I/O
> [  787.096683] block nvme0n1: no usable path - requeuing I/O
> [  787.201242] Hardware name: Hewlett-Packard HP xw9400 Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
> [  787.210366] Workqueue: kblockd blk_mq_timeout_work
> [  787.215161] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
> [  787.220295] Code: 05 aa 98 22 01 01 e8 9f cf ad ff 0f 0b c3 80 3d 98 98 22 01 00 75 95 48 c7 c7 48 54 3e a2 c6 05 88 98 22 01 01 e8 80 cf ad ff <0f> 0b c3 80 3d 77 98 22 01 00 0f 85 72 ff ff ff 48 c7 c7 a0 54 3e
> [  787.239021] RSP: 0018:ffffb860407b3dc8 EFLAGS: 00010292
> [  787.244240] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff9d93ddc58d08
> [  787.251361] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff9d93ddc58d00
> [  787.258480] RBP: ffff9d93dbd6c200 R08: 000000b740ed60e8 R09: ffffffffa2fffbc4
> [  787.265612] R10: 0000000000000477 R11: 0000000000028320 R12: ffff9d93dbd6c2e8
> [  787.272741] R13: ffff9d93dbea0000 R14: ffffb860407b3e70 R15: ffff9d93dbea0000
> [  787.279864] FS:  0000000000000000(0000) GS:ffff9d93ddc40000(0000) knlGS:0000000000000000
> [  787.287948] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  787.293695] CR2: 00007f45fd8fa024 CR3: 00000000792dc000 CR4: 00000000000006e0
> [  787.300820] Call Trace:
> [  787.303277]  blk_mq_check_expired+0x109/0x1b0
> [  787.307638]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
> [  787.312524]  ? blk_poll+0x300/0x300
> [  787.316022]  blk_mq_timeout_work+0x44/0xe0
> [  787.320128]  process_one_work+0x1b4/0x370
> [  787.324137]  worker_thread+0x53/0x3e0
> [  787.327809]  ? process_one_work+0x370/0x370
> [  787.332015]  kthread+0x11b/0x140
> [  787.335245]  ? __kthread_bind_mask+0x60/0x60
> [  787.339515]  ret_from_fork+0x22/0x30
> [  787.343094] ---[ end trace 606cf9189379fcfc ]---
> 
> 
> Best Regards,
>    Yi Zhang
> 
> 
> ----- Original Message -----
> From: "Ming Lei" <ming.lei at redhat.com>
> To: "Sagi Grimberg" <sagi at grimberg.me>
> Cc: "Jens Axboe" <axboe at kernel.dk>, "Yi Zhang" <yi.zhang at redhat.com>, linux-nvme at lists.infradead.org, linux-block at vger.kernel.org, "Keith Busch" <kbusch at kernel.org>, "Christoph Hellwig" <hch at lst.de>
> Sent: Friday, October 9, 2020 12:39:38 PM
> Subject: Re: [PATCH] block: re-introduce blk_mq_complete_request_sync
> 
> On Thu, Oct 08, 2020 at 02:37:50PM -0700, Sagi Grimberg wrote:
>> In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
>> needs to complete an aborted request. The timeout handler serializes
>> against an error recovery sequence that iterates over the inflight
>> requests and cancels them.
>>
>> However, the fact that blk_mq_complete_request may defer to a different
>> core that serialization breaks.
>>
>> Hence re-introduce blk_mq_complete_request_sync and use that in the
>> timeout handler to make sure that we don't get a use-after-free
>> condition.
>>
>> This was uncovered by the blktests:
>> --
>> $ nvme_trtype=tcp ./check nvme/012
>>
>> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
>> [ 2152.799640] loop: module loaded
>> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>> [ 2152.937889] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
>> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
>> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
>> [ 2154.942432] XFS (nvme0n1): Ending clean mount
>> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
>> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
>> [ 2215.199331] nvme nvme0: starting error recovery
>> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
>> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
>> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
>> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
>> [ 2215.215502] ------------[ cut here ]------------
>> [ 2215.215505] refcount_t: underflow; use-after-free.
>> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
>> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
>> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff93f37dcd8d08
>> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff93f37dcd8d00
>> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: ffffffffa7fffbc4
>> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: ffff93f37a8124e8
>> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: ffff93f37a2b0000
>> [ 2215.215765] FS:  0000000000000000(0000) GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
>> [ 2215.215768] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 00000000000006e0
>> [ 2215.215773] Call Trace:
>> [ 2215.215784]  blk_mq_check_expired+0x109/0x1b0
>> [ 2215.215797]  blk_mq_queue_tag_busy_iter+0x1b8/0x330
>> [ 2215.215801]  ? blk_poll+0x300/0x300
>> [ 2215.215806]  blk_mq_timeout_work+0x44/0xe0
>> [ 2215.215814]  process_one_work+0x1b4/0x370
>> [ 2215.215820]  worker_thread+0x53/0x3e0
>> [ 2215.215825]  ? process_one_work+0x370/0x370
>> [ 2215.215829]  kthread+0x11b/0x140
>> [ 2215.215834]  ? __kthread_bind_mask+0x60/0x60
>> [ 2215.215841]  ret_from_fork+0x22/0x30
>> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
>> --
>>
>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>> Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
>> Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
>> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
>> ---
>> Yi, would appreciate your Tested-by tag
>>
>>   block/blk-mq.c           | 7 +++++++
>>   drivers/nvme/host/rdma.c | 2 +-
>>   drivers/nvme/host/tcp.c  | 2 +-
>>   include/linux/blk-mq.h   | 1 +
>>   4 files changed, 10 insertions(+), 2 deletions(-)
>>
>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>> index f8e1e759c905..2d154722ef39 100644
>> --- a/block/blk-mq.c
>> +++ b/block/blk-mq.c
>> @@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct request *rq)
>>   }
>>   EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
>>   
>> +void blk_mq_complete_request_sync(struct request *rq)
>> +{
>> +	WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
>> +	rq->q->mq_ops->complete(rq);
>> +}
>> +EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
>> +
>>   /**
>>    * blk_mq_complete_request - end I/O on a request
>>    * @rq:		the request being processed
>> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
>> index 9e378d0a0c01..45fc605349da 100644
>> --- a/drivers/nvme/host/rdma.c
>> +++ b/drivers/nvme/host/rdma.c
>> @@ -1975,7 +1975,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
>>   	nvme_rdma_stop_queue(queue);
>>   	if (!blk_mq_request_completed(rq)) {
>>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>> -		blk_mq_complete_request(rq);
>> +		blk_mq_complete_request_sync(rq);
>>   	}
>>   	mutex_unlock(&ctrl->teardown_lock);
>>   }
>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>> index 8f4f29f18b8c..629b025685d1 100644
>> --- a/drivers/nvme/host/tcp.c
>> +++ b/drivers/nvme/host/tcp.c
>> @@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>   	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>   	if (!blk_mq_request_completed(rq)) {
>>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>> -		blk_mq_complete_request(rq);
>> +		blk_mq_complete_request_sync(rq);
> 
> Or complete the request in the following way? Then one block layer API
> can be saved:
> 
> 	blk_mq_complete_request_remote(rq);
> 	nvme_complete_rq(rq);
> 
> 
> thanks,
> Ming
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
> 


From sagi at grimberg.me  Fri Oct  9 04:11:46 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 9 Oct 2020 01:11:46 -0700
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201009043938.GC27356@T590>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
Message-ID: <189a7d80-ff0f-e6b0-f8f0-b50dfdd028f8@grimberg.me>


>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>> index 8f4f29f18b8c..629b025685d1 100644
>> --- a/drivers/nvme/host/tcp.c
>> +++ b/drivers/nvme/host/tcp.c
>> @@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>   	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>   	if (!blk_mq_request_completed(rq)) {
>>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>> -		blk_mq_complete_request(rq);
>> +		blk_mq_complete_request_sync(rq);
> 
> Or complete the request in the following way? Then one block layer API
> can be saved:
> 
> 	blk_mq_complete_request_remote(rq);
> 	nvme_complete_rq(rq);

Not sure I follow, how does this work?

Anyways, I think that blk_mq_complete_request_sync is a clean
and useful interface. We can do it in nvme but I don't think
it would be better.


From yi.zhang at redhat.com  Fri Oct  9 09:55:29 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Fri, 9 Oct 2020 21:55:29 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
Message-ID: <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>

Hi Sagi

On 10/9/20 4:09 PM, Sagi Grimberg wrote:
>> Hi Sagi
>>
>> I applied this patch on block origin/for-next and still can reproduce 
>> it.
>
> That's unexpected, can you try this patch?
> -- 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 629b025685d1..46428ff0b0fc 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct 
> request *rq)
> ??????? /* fence other contexts that may complete the command */
> ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
> ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
> -?????? if (!blk_mq_request_completed(rq)) {
> +?????? if (blk_mq_request_started(rq) && 
> !blk_mq_request_completed(rq)) {
> ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> ??????????????? blk_mq_complete_request_sync(rq);
> ??????? }
No WARNING with this patch, but the test still will be hang and never 
finished, here is the kernel log:

[ 3327.011067] run blktests nvme/012 at 2020-10-09 06:21:59
[ 3327.278169] loop: module loaded
[ 3327.314287] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[ 3327.346276] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[ 3327.396746] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[ 3327.411000] nvme nvme0: creating 12 I/O queues.
[ 3327.416963] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[ 3327.428836] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
127.0.0.1:4420
[ 3329.403658] XFS (nvme0n1): Mounting V5 Filesystem
[ 3329.417213] XFS (nvme0n1): Ending clean mount
[ 3329.423044] xfs filesystem being mounted at /mnt/blktests supports 
timestamps until 2038 (0x7fffffff)
[ 3390.069126] nvme nvme0: queue 6: timeout request 0x21 type 4
[ 3390.074797] nvme nvme0: starting error recovery
[ 3390.079362] nvme nvme0: queue 6: timeout request 0x22 type 4
[ 3390.079932] block nvme0n1: no usable path - requeuing I/O
[ 3390.081999] nvme nvme0: Reconnecting in 10 seconds...
[ 3390.095467] block nvme0n1: no usable path - requeuing I/O
[ 3390.100879] block nvme0n1: no usable path - requeuing I/O
[ 3390.106279] block nvme0n1: no usable path - requeuing I/O
[ 3390.111691] block nvme0n1: no usable path - requeuing I/O
[ 3390.117098] block nvme0n1: no usable path - requeuing I/O
[ 3390.122507] block nvme0n1: no usable path - requeuing I/O
[ 3390.127920] block nvme0n1: no usable path - requeuing I/O
[ 3390.133333] block nvme0n1: no usable path - requeuing I/O
[ 3390.138734] block nvme0n1: no usable path - requeuing I/O
[ 3415.669659] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
[ 3415.675682] nvmet: ctrl 2 fatal error occurred!
[ 3419.765738] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[ 3419.771762] nvmet: ctrl 1 fatal error occurred!
[ 3440.000352] nvmet: creating controller 2 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[ 3463.286700] nvme nvme0: queue 0: timeout request 0x0 type 4
[ 3463.292357] nvme nvme0: Connect command failed, error wo/DNR bit: 881
[ 3463.298830] nvme nvme0: failed to connect queue: 0 ret=881
[ 3463.304352] nvme nvme0: Failed reconnect attempt 1
[ 3463.309156] nvme nvme0: Reconnecting in 10 seconds...
[ 3473.527574] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
[ 3473.541438] nvme_ns_head_submit_bio: 47 callbacks suppressed
[ 3473.541445] block nvme0n1: no usable path - requeuing I/O
[ 3473.552521] block nvme0n1: no usable path - requeuing I/O
[ 3473.557923] block nvme0n1: no usable path - requeuing I/O
[ 3473.563319] block nvme0n1: no usable path - requeuing I/O
[ 3473.568739] block nvme0n1: no usable path - requeuing I/O
[ 3473.574678] block nvme0n1: no usable path - requeuing I/O
[ 3473.580068] block nvme0n1: no usable path - requeuing I/O
[ 3473.585461] block nvme0n1: no usable path - requeuing I/O
[ 3473.590861] block nvme0n1: no usable path - requeuing I/O
[ 3473.596249] block nvme0n1: no usable path - requeuing I/O
[ 3473.601880] nvme nvme0: creating 12 I/O queues.
[ 3473.613955] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[ 3473.622106] nvme nvme0: Successfully reconnected (2 attempt)
[ 3531.384170] nvme nvme0: queue 1: timeout request 0x62 type 4
[ 3531.389837] nvme nvme0: starting error recovery
[ 3531.394396] nvme nvme0: queue 7: timeout request 0x61 type 4
[ 3531.394954] nvme_ns_head_submit_bio: 14 callbacks suppressed
[ 3531.394964] block nvme0n1: no usable path - requeuing I/O
[ 3531.411147] block nvme0n1: no usable path - requeuing I/O
[ 3531.416548] block nvme0n1: no usable path - requeuing I/O
[ 3531.421954] block nvme0n1: no usable path - requeuing I/O
[ 3531.427361] block nvme0n1: no usable path - requeuing I/O
[ 3531.432763] block nvme0n1: no usable path - requeuing I/O
[ 3531.438287] block nvme0n1: no usable path - requeuing I/O
[ 3531.443701] block nvme0n1: no usable path - requeuing I/O
[ 3531.449103] block nvme0n1: no usable path - requeuing I/O
[ 3531.454497] block nvme0n1: no usable path - requeuing I/O
[ 3560.568832] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[ 3560.574839] nvmet: ctrl 1 fatal error occurred!


> -- 
>
>>
>> [? 724.267865] run blktests nvme/012 at 2020-10-09 00:48:52
>> [? 724.518498] loop: module loaded
>> [? 724.552880] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>> [? 724.586609] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>> [? 724.648898] nvmet: creating controller 1 for subsystem 
>> blktests-subsystem-1 for NQN 
>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>> [? 724.663300] nvme nvme0: creating 12 I/O queues.
>> [? 724.669826] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>> [? 724.681350] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
>> 127.0.0.1:4420
>> [? 726.641605] XFS (nvme0n1): Mounting V5 Filesystem
>> [? 726.654637] XFS (nvme0n1): Ending clean mount
>> [? 726.660302] xfs filesystem being mounted at /mnt/blktests supports 
>> timestamps until 2038 (0x7fffffff)
>> [? 787.042066] nvme nvme0: queue 2: timeout request 0xe type 4
>> [? 787.047649] nvme nvme0: starting error recovery
>> [? 787.052211] nvme nvme0: queue 4: timeout request 0x61 type 4
>> [? 787.053407] block nvme0n1: no usable path - requeuing I/O
>> [? 787.054543] nvme nvme0: Reconnecting in 10 seconds...
>> [? 787.057877] ------------[ cut here ]------------
>> [? 787.063274] block nvme0n1: no usable path - requeuing I/O
>> [? 787.068313] refcount_t: underflow; use-after-free.
>> [? 787.068370] WARNING: CPU: 3 PID: 30 at lib/refcount.c:28 
>> refcount_warn_saturate+0xa6/0xf0
>> [? 787.072941] block nvme0n1: no usable path - requeuing I/O
>> [? 787.072945] block nvme0n1: no usable path - requeuing I/O
>> [? 787.078333] Modules linked in: loop nvme_tcp nvme_fabrics 
>> nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 
>> dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek 
>> snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg 
>> snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer amd64_edac_mod 
>> snd edac_mce_amd soundcore kvm_amd ccp kvm pcspkr irqbypass hp_wmi 
>> sparse_keymap rfkill k10temp wmi_bmof nv_tco i2c_nforce2 acpi_cpufreq 
>> ip_tables xfs nouveau video mxm_wmi i2c_algo_bit drm_kms_helper 
>> mptsas cec scsi_transport_sas ttm mptscsih ata_generic firewire_ohci 
>> pata_acpi firewire_core drm serio_raw mptbase forcedeth crc_itu_t 
>> sata_nv pata_amd wmi floppy
>> [? 787.083142] block nvme0n1: no usable path - requeuing I/O
>> [? 787.091304] CPU: 3 PID: 30 Comm: kworker/3:0H Not tainted 
>> 5.9.0-rc8.update+ #3
>> [? 787.096673] block nvme0n1: no usable path - requeuing I/O
>> [? 787.096674] block nvme0n1: no usable path - requeuing I/O
>> [? 787.096680] block nvme0n1: no usable path - requeuing I/O
>> [? 787.096681] block nvme0n1: no usable path - requeuing I/O
>> [? 787.096683] block nvme0n1: no usable path - requeuing I/O
>> [? 787.201242] Hardware name: Hewlett-Packard HP xw9400 
>> Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
>> [? 787.210366] Workqueue: kblockd blk_mq_timeout_work
>> [? 787.215161] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
>> [? 787.220295] Code: 05 aa 98 22 01 01 e8 9f cf ad ff 0f 0b c3 80 3d 
>> 98 98 22 01 00 75 95 48 c7 c7 48 54 3e a2 c6 05 88 98 22 01 01 e8 80 
>> cf ad ff <0f> 0b c3 80 3d 77 98 22 01 00 0f 85 72 ff ff ff 48 c7 c7 
>> a0 54 3e
>> [? 787.239021] RSP: 0018:ffffb860407b3dc8 EFLAGS: 00010292
>> [? 787.244240] RAX: 0000000000000026 RBX: 0000000000000000 RCX: 
>> ffff9d93ddc58d08
>> [? 787.251361] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: 
>> ffff9d93ddc58d00
>> [? 787.258480] RBP: ffff9d93dbd6c200 R08: 000000b740ed60e8 R09: 
>> ffffffffa2fffbc4
>> [? 787.265612] R10: 0000000000000477 R11: 0000000000028320 R12: 
>> ffff9d93dbd6c2e8
>> [? 787.272741] R13: ffff9d93dbea0000 R14: ffffb860407b3e70 R15: 
>> ffff9d93dbea0000
>> [? 787.279864] FS:? 0000000000000000(0000) GS:ffff9d93ddc40000(0000) 
>> knlGS:0000000000000000
>> [? 787.287948] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [? 787.293695] CR2: 00007f45fd8fa024 CR3: 00000000792dc000 CR4: 
>> 00000000000006e0
>> [? 787.300820] Call Trace:
>> [? 787.303277]? blk_mq_check_expired+0x109/0x1b0
>> [? 787.307638]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>> [? 787.312524]? ? blk_poll+0x300/0x300
>> [? 787.316022]? blk_mq_timeout_work+0x44/0xe0
>> [? 787.320128]? process_one_work+0x1b4/0x370
>> [? 787.324137]? worker_thread+0x53/0x3e0
>> [? 787.327809]? ? process_one_work+0x370/0x370
>> [? 787.332015]? kthread+0x11b/0x140
>> [? 787.335245]? ? __kthread_bind_mask+0x60/0x60
>> [? 787.339515]? ret_from_fork+0x22/0x30
>> [? 787.343094] ---[ end trace 606cf9189379fcfc ]---
>>
>>
>> Best Regards,
>> ?? Yi Zhang
>>
>>
>> ----- Original Message -----
>> From: "Ming Lei" <ming.lei at redhat.com>
>> To: "Sagi Grimberg" <sagi at grimberg.me>
>> Cc: "Jens Axboe" <axboe at kernel.dk>, "Yi Zhang" <yi.zhang at redhat.com>, 
>> linux-nvme at lists.infradead.org, linux-block at vger.kernel.org, "Keith 
>> Busch" <kbusch at kernel.org>, "Christoph Hellwig" <hch at lst.de>
>> Sent: Friday, October 9, 2020 12:39:38 PM
>> Subject: Re: [PATCH] block: re-introduce blk_mq_complete_request_sync
>>
>> On Thu, Oct 08, 2020 at 02:37:50PM -0700, Sagi Grimberg wrote:
>>> In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
>>> needs to complete an aborted request. The timeout handler serializes
>>> against an error recovery sequence that iterates over the inflight
>>> requests and cancels them.
>>>
>>> However, the fact that blk_mq_complete_request may defer to a different
>>> core that serialization breaks.
>>>
>>> Hence re-introduce blk_mq_complete_request_sync and use that in the
>>> timeout handler to make sure that we don't get a use-after-free
>>> condition.
>>>
>>> This was uncovered by the blktests:
>>> -- 
>>> $ nvme_trtype=tcp ./check nvme/012
>>>
>>> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
>>> [ 2152.799640] loop: module loaded
>>> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>> [ 2152.937889] nvmet: creating controller 1 for subsystem 
>>> blktests-subsystem-1 for NQN 
>>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
>>> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", 
>>> addr 127.0.0.1:4420
>>> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
>>> [ 2154.942432] XFS (nvme0n1): Ending clean mount
>>> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests 
>>> supports timestamps until 2038 (0x7fffffff)
>>> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
>>> [ 2215.199331] nvme nvme0: starting error recovery
>>> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
>>> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
>>> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
>>> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
>>> [ 2215.215502] ------------[ cut here ]------------
>>> [ 2215.215505] refcount_t: underflow; use-after-free.
>>> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 
>>> refcount_warn_saturate+0xa6/0xf0
>>> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
>>> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: 
>>> ffff93f37dcd8d08
>>> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: 
>>> ffff93f37dcd8d00
>>> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: 
>>> ffffffffa7fffbc4
>>> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: 
>>> ffff93f37a8124e8
>>> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: 
>>> ffff93f37a2b0000
>>> [ 2215.215765] FS:? 0000000000000000(0000) GS:ffff93f37dcc0000(0000) 
>>> knlGS:0000000000000000
>>> [ 2215.215768] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 
>>> 00000000000006e0
>>> [ 2215.215773] Call Trace:
>>> [ 2215.215784]? blk_mq_check_expired+0x109/0x1b0
>>> [ 2215.215797]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>>> [ 2215.215801]? ? blk_poll+0x300/0x300
>>> [ 2215.215806]? blk_mq_timeout_work+0x44/0xe0
>>> [ 2215.215814]? process_one_work+0x1b4/0x370
>>> [ 2215.215820]? worker_thread+0x53/0x3e0
>>> [ 2215.215825]? ? process_one_work+0x370/0x370
>>> [ 2215.215829]? kthread+0x11b/0x140
>>> [ 2215.215834]? ? __kthread_bind_mask+0x60/0x60
>>> [ 2215.215841]? ret_from_fork+0x22/0x30
>>> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
>>> -- 
>>>
>>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>>> Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
>>> Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
>>> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
>>> ---
>>> Yi, would appreciate your Tested-by tag
>>>
>>> ? block/blk-mq.c?????????? | 7 +++++++
>>> ? drivers/nvme/host/rdma.c | 2 +-
>>> ? drivers/nvme/host/tcp.c? | 2 +-
>>> ? include/linux/blk-mq.h?? | 1 +
>>> ? 4 files changed, 10 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>>> index f8e1e759c905..2d154722ef39 100644
>>> --- a/block/blk-mq.c
>>> +++ b/block/blk-mq.c
>>> @@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct 
>>> request *rq)
>>> ? }
>>> ? EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
>>> ? +void blk_mq_complete_request_sync(struct request *rq)
>>> +{
>>> +??? WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
>>> +??? rq->q->mq_ops->complete(rq);
>>> +}
>>> +EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
>>> +
>>> ? /**
>>> ?? * blk_mq_complete_request - end I/O on a request
>>> ?? * @rq:??????? the request being processed
>>> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
>>> index 9e378d0a0c01..45fc605349da 100644
>>> --- a/drivers/nvme/host/rdma.c
>>> +++ b/drivers/nvme/host/rdma.c
>>> @@ -1975,7 +1975,7 @@ static void 
>>> nvme_rdma_complete_timed_out(struct request *rq)
>>> ????? nvme_rdma_stop_queue(queue);
>>> ????? if (!blk_mq_request_completed(rq)) {
>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>> -??????? blk_mq_complete_request(rq);
>>> +??????? blk_mq_complete_request_sync(rq);
>>> ????? }
>>> ????? mutex_unlock(&ctrl->teardown_lock);
>>> ? }
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index 8f4f29f18b8c..629b025685d1 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct 
>>> request *rq)
>>> ????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>> ????? if (!blk_mq_request_completed(rq)) {
>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>> -??????? blk_mq_complete_request(rq);
>>> +??????? blk_mq_complete_request_sync(rq);
>>
>> Or complete the request in the following way? Then one block layer API
>> can be saved:
>>
>> ????blk_mq_complete_request_remote(rq);
>> ????nvme_complete_rq(rq);
>>
>>
>> thanks,
>> Ming
>>
>>
>> _______________________________________________
>> Linux-nvme mailing list
>> Linux-nvme at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>
>>
>> _______________________________________________
>> Linux-nvme mailing list
>> Linux-nvme at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>
>



From hare at suse.de  Fri Oct  9 09:57:09 2020
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 9 Oct 2020 15:57:09 +0200
Subject: [PATCH 3/7] nvme-fcloop: use IDA for port ids
In-Reply-To: <d0a614ff-c8b5-22fd-07f9-712a0259d760@broadcom.com>
References: <20200922121501.32851-1-hare@suse.de>
 <20200922121501.32851-4-hare@suse.de>
 <d0a614ff-c8b5-22fd-07f9-712a0259d760@broadcom.com>
Message-ID: <38dcf65b-d05d-24f5-8fce-cc4739af3a8e@suse.de>

On 10/5/20 7:33 PM, James Smart wrote:
> On 9/22/2020 5:14 AM, Hannes Reinecke wrote:
>> Use an IDA to register port IDs to avoid duplicate port IDs. It
>> also will generate a unique port ID if none is specified.
> 
> I'd prefer a little more description as I didn't pick up what this 
> actually did from the comment.? Meaning: Add auto-address assignment if 
> one isn't specified. Use an ida for allocation for uniqueness. Use 
> user-supplied addresses as long as not in use and within range supported 
> for ida.
> 

Ok.

>> Signed-off-by: Hannes Reinecke <hare at suse.de>
>> ---
>> ? drivers/nvme/target/fcloop.c | 51 
>> +++++++++++++++++++++++++++++++++++++-------
>> ? 1 file changed, 43 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
>> index 082aa4dee406..e56f323fa7d4 100644
>> --- a/drivers/nvme/target/fcloop.c
>> +++ b/drivers/nvme/target/fcloop.c
>> @@ -12,6 +12,7 @@
>> ? #include <linux/nvme-fc-driver.h>
>> ? #include <linux/nvme-fc.h>
>> +#define FCLOOP_MAX_AL_PA 0xEF
> 
> This is odd - no reason to bring in ALPA's and arb loop.?? Please rename 
> it to state it's the max address value allowed.? I'd prefer if this was 
> a 24 bit value, but that won't matter much and 255 is fine.
> 
Hehe; yeah, pulling in ALPA etc is probably not necessary.
But I wanted to restrict things to less than 255 to get alignment with 
real PtP Port IDs.

I'll be resending the patchset.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From kbusch at kernel.org  Fri Oct  9 11:29:27 2020
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 9 Oct 2020 08:29:27 -0700
Subject: [PATCHv4 1/3] block: add zone specific block statuses
In-Reply-To: <20200924205330.4043232-2-kbusch@kernel.org>
References: <20200924205330.4043232-1-kbusch@kernel.org>
 <20200924205330.4043232-2-kbusch@kernel.org>
Message-ID: <20201009152927.GA1023921@dhcp-10-100-145-180.wdl.wdc.com>

On Thu, Sep 24, 2020 at 01:53:28PM -0700, Keith Busch wrote:
> A zoned device with limited resources to open or activate zones may
> return an error when the host exceeds those limits. The same command may
> be successful if retried later, but the host needs to wait for specific
> zone states before it should expect a retry to succeed. Have the block
> layer provide an appropriate status for these conditions so applications
> can distinuguish this error for special handling.

Hi Jens,

Can be considered for inclusion? The series still applies cleanly to
block for-next.


From sagi at grimberg.me  Fri Oct  9 14:29:06 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 9 Oct 2020 11:29:06 -0700
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
Message-ID: <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>



On 10/9/20 6:55 AM, Yi Zhang wrote:
> Hi Sagi
> 
> On 10/9/20 4:09 PM, Sagi Grimberg wrote:
>>> Hi Sagi
>>>
>>> I applied this patch on block origin/for-next and still can reproduce 
>>> it.
>>
>> That's unexpected, can you try this patch?
>> -- 
>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>> index 629b025685d1..46428ff0b0fc 100644
>> --- a/drivers/nvme/host/tcp.c
>> +++ b/drivers/nvme/host/tcp.c
>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct 
>> request *rq)
>> ??????? /* fence other contexts that may complete the command */
>> ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>> ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>> -?????? if (!blk_mq_request_completed(rq)) {
>> +?????? if (blk_mq_request_started(rq) && 
>> !blk_mq_request_completed(rq)) {
>> ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>> ??????????????? blk_mq_complete_request_sync(rq);
>> ??????? }
> No WARNING with this patch, but the test still will be hang and never 
> finished, here is the kernel log:

Thanks Yi,

Good to know no warning, that confirms the issue analysis.

As for the test, I'm able to successfuly pass this on my VM, and
this test is not designed to generate errors, just fio to
a file backed namespace. I'm wandering what is causing these
delays in I/O that cause timeouts...

> 
> [ 3327.011067] run blktests nvme/012 at 2020-10-09 06:21:59
> [ 3327.278169] loop: module loaded
> [ 3327.314287] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
> [ 3327.346276] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
> [ 3327.396746] nvmet: creating controller 1 for subsystem 
> blktests-subsystem-1 for NQN 
> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 3327.411000] nvme nvme0: creating 12 I/O queues.
> [ 3327.416963] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [ 3327.428836] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
> 127.0.0.1:4420
> [ 3329.403658] XFS (nvme0n1): Mounting V5 Filesystem
> [ 3329.417213] XFS (nvme0n1): Ending clean mount
> [ 3329.423044] xfs filesystem being mounted at /mnt/blktests supports 
> timestamps until 2038 (0x7fffffff)
> [ 3390.069126] nvme nvme0: queue 6: timeout request 0x21 type 4
> [ 3390.074797] nvme nvme0: starting error recovery
> [ 3390.079362] nvme nvme0: queue 6: timeout request 0x22 type 4
> [ 3390.079932] block nvme0n1: no usable path - requeuing I/O
> [ 3390.081999] nvme nvme0: Reconnecting in 10 seconds...
> [ 3390.095467] block nvme0n1: no usable path - requeuing I/O
> [ 3390.100879] block nvme0n1: no usable path - requeuing I/O
> [ 3390.106279] block nvme0n1: no usable path - requeuing I/O
> [ 3390.111691] block nvme0n1: no usable path - requeuing I/O
> [ 3390.117098] block nvme0n1: no usable path - requeuing I/O
> [ 3390.122507] block nvme0n1: no usable path - requeuing I/O
> [ 3390.127920] block nvme0n1: no usable path - requeuing I/O
> [ 3390.133333] block nvme0n1: no usable path - requeuing I/O
> [ 3390.138734] block nvme0n1: no usable path - requeuing I/O
> [ 3415.669659] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
> [ 3415.675682] nvmet: ctrl 2 fatal error occurred!
> [ 3419.765738] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
> [ 3419.771762] nvmet: ctrl 1 fatal error occurred!
> [ 3440.000352] nvmet: creating controller 2 for subsystem 
> blktests-subsystem-1 for NQN 
> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 3463.286700] nvme nvme0: queue 0: timeout request 0x0 type 4
> [ 3463.292357] nvme nvme0: Connect command failed, error wo/DNR bit: 881
> [ 3463.298830] nvme nvme0: failed to connect queue: 0 ret=881
> [ 3463.304352] nvme nvme0: Failed reconnect attempt 1
> [ 3463.309156] nvme nvme0: Reconnecting in 10 seconds...
> [ 3473.527574] nvmet: creating controller 1 for subsystem 
> blktests-subsystem-1 for NQN 
> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
> [ 3473.541438] nvme_ns_head_submit_bio: 47 callbacks suppressed
> [ 3473.541445] block nvme0n1: no usable path - requeuing I/O
> [ 3473.552521] block nvme0n1: no usable path - requeuing I/O
> [ 3473.557923] block nvme0n1: no usable path - requeuing I/O
> [ 3473.563319] block nvme0n1: no usable path - requeuing I/O
> [ 3473.568739] block nvme0n1: no usable path - requeuing I/O
> [ 3473.574678] block nvme0n1: no usable path - requeuing I/O
> [ 3473.580068] block nvme0n1: no usable path - requeuing I/O
> [ 3473.585461] block nvme0n1: no usable path - requeuing I/O
> [ 3473.590861] block nvme0n1: no usable path - requeuing I/O
> [ 3473.596249] block nvme0n1: no usable path - requeuing I/O
> [ 3473.601880] nvme nvme0: creating 12 I/O queues.
> [ 3473.613955] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [ 3473.622106] nvme nvme0: Successfully reconnected (2 attempt)
> [ 3531.384170] nvme nvme0: queue 1: timeout request 0x62 type 4
> [ 3531.389837] nvme nvme0: starting error recovery
> [ 3531.394396] nvme nvme0: queue 7: timeout request 0x61 type 4
> [ 3531.394954] nvme_ns_head_submit_bio: 14 callbacks suppressed
> [ 3531.394964] block nvme0n1: no usable path - requeuing I/O
> [ 3531.411147] block nvme0n1: no usable path - requeuing I/O
> [ 3531.416548] block nvme0n1: no usable path - requeuing I/O
> [ 3531.421954] block nvme0n1: no usable path - requeuing I/O
> [ 3531.427361] block nvme0n1: no usable path - requeuing I/O
> [ 3531.432763] block nvme0n1: no usable path - requeuing I/O
> [ 3531.438287] block nvme0n1: no usable path - requeuing I/O
> [ 3531.443701] block nvme0n1: no usable path - requeuing I/O
> [ 3531.449103] block nvme0n1: no usable path - requeuing I/O
> [ 3531.454497] block nvme0n1: no usable path - requeuing I/O
> [ 3560.568832] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
> [ 3560.574839] nvmet: ctrl 1 fatal error occurred!
> 
> 
>> -- 
>>
>>>
>>> [? 724.267865] run blktests nvme/012 at 2020-10-09 00:48:52
>>> [? 724.518498] loop: module loaded
>>> [? 724.552880] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>> [? 724.586609] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>> [? 724.648898] nvmet: creating controller 1 for subsystem 
>>> blktests-subsystem-1 for NQN 
>>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>> [? 724.663300] nvme nvme0: creating 12 I/O queues.
>>> [? 724.669826] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>> [? 724.681350] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
>>> 127.0.0.1:4420
>>> [? 726.641605] XFS (nvme0n1): Mounting V5 Filesystem
>>> [? 726.654637] XFS (nvme0n1): Ending clean mount
>>> [? 726.660302] xfs filesystem being mounted at /mnt/blktests supports 
>>> timestamps until 2038 (0x7fffffff)
>>> [? 787.042066] nvme nvme0: queue 2: timeout request 0xe type 4
>>> [? 787.047649] nvme nvme0: starting error recovery
>>> [? 787.052211] nvme nvme0: queue 4: timeout request 0x61 type 4
>>> [? 787.053407] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.054543] nvme nvme0: Reconnecting in 10 seconds...
>>> [? 787.057877] ------------[ cut here ]------------
>>> [? 787.063274] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.068313] refcount_t: underflow; use-after-free.
>>> [? 787.068370] WARNING: CPU: 3 PID: 30 at lib/refcount.c:28 
>>> refcount_warn_saturate+0xa6/0xf0
>>> [? 787.072941] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.072945] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.078333] Modules linked in: loop nvme_tcp nvme_fabrics 
>>> nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 
>>> dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek 
>>> snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg 
>>> snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer amd64_edac_mod 
>>> snd edac_mce_amd soundcore kvm_amd ccp kvm pcspkr irqbypass hp_wmi 
>>> sparse_keymap rfkill k10temp wmi_bmof nv_tco i2c_nforce2 acpi_cpufreq 
>>> ip_tables xfs nouveau video mxm_wmi i2c_algo_bit drm_kms_helper 
>>> mptsas cec scsi_transport_sas ttm mptscsih ata_generic firewire_ohci 
>>> pata_acpi firewire_core drm serio_raw mptbase forcedeth crc_itu_t 
>>> sata_nv pata_amd wmi floppy
>>> [? 787.083142] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.091304] CPU: 3 PID: 30 Comm: kworker/3:0H Not tainted 
>>> 5.9.0-rc8.update+ #3
>>> [? 787.096673] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.096674] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.096680] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.096681] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.096683] block nvme0n1: no usable path - requeuing I/O
>>> [? 787.201242] Hardware name: Hewlett-Packard HP xw9400 
>>> Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
>>> [? 787.210366] Workqueue: kblockd blk_mq_timeout_work
>>> [? 787.215161] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
>>> [? 787.220295] Code: 05 aa 98 22 01 01 e8 9f cf ad ff 0f 0b c3 80 3d 
>>> 98 98 22 01 00 75 95 48 c7 c7 48 54 3e a2 c6 05 88 98 22 01 01 e8 80 
>>> cf ad ff <0f> 0b c3 80 3d 77 98 22 01 00 0f 85 72 ff ff ff 48 c7 c7 
>>> a0 54 3e
>>> [? 787.239021] RSP: 0018:ffffb860407b3dc8 EFLAGS: 00010292
>>> [? 787.244240] RAX: 0000000000000026 RBX: 0000000000000000 RCX: 
>>> ffff9d93ddc58d08
>>> [? 787.251361] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: 
>>> ffff9d93ddc58d00
>>> [? 787.258480] RBP: ffff9d93dbd6c200 R08: 000000b740ed60e8 R09: 
>>> ffffffffa2fffbc4
>>> [? 787.265612] R10: 0000000000000477 R11: 0000000000028320 R12: 
>>> ffff9d93dbd6c2e8
>>> [? 787.272741] R13: ffff9d93dbea0000 R14: ffffb860407b3e70 R15: 
>>> ffff9d93dbea0000
>>> [? 787.279864] FS:? 0000000000000000(0000) GS:ffff9d93ddc40000(0000) 
>>> knlGS:0000000000000000
>>> [? 787.287948] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> [? 787.293695] CR2: 00007f45fd8fa024 CR3: 00000000792dc000 CR4: 
>>> 00000000000006e0
>>> [? 787.300820] Call Trace:
>>> [? 787.303277]? blk_mq_check_expired+0x109/0x1b0
>>> [? 787.307638]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>>> [? 787.312524]? ? blk_poll+0x300/0x300
>>> [? 787.316022]? blk_mq_timeout_work+0x44/0xe0
>>> [? 787.320128]? process_one_work+0x1b4/0x370
>>> [? 787.324137]? worker_thread+0x53/0x3e0
>>> [? 787.327809]? ? process_one_work+0x370/0x370
>>> [? 787.332015]? kthread+0x11b/0x140
>>> [? 787.335245]? ? __kthread_bind_mask+0x60/0x60
>>> [? 787.339515]? ret_from_fork+0x22/0x30
>>> [? 787.343094] ---[ end trace 606cf9189379fcfc ]---
>>>
>>>
>>> Best Regards,
>>> ?? Yi Zhang
>>>
>>>
>>> ----- Original Message -----
>>> From: "Ming Lei" <ming.lei at redhat.com>
>>> To: "Sagi Grimberg" <sagi at grimberg.me>
>>> Cc: "Jens Axboe" <axboe at kernel.dk>, "Yi Zhang" <yi.zhang at redhat.com>, 
>>> linux-nvme at lists.infradead.org, linux-block at vger.kernel.org, "Keith 
>>> Busch" <kbusch at kernel.org>, "Christoph Hellwig" <hch at lst.de>
>>> Sent: Friday, October 9, 2020 12:39:38 PM
>>> Subject: Re: [PATCH] block: re-introduce blk_mq_complete_request_sync
>>>
>>> On Thu, Oct 08, 2020 at 02:37:50PM -0700, Sagi Grimberg wrote:
>>>> In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
>>>> needs to complete an aborted request. The timeout handler serializes
>>>> against an error recovery sequence that iterates over the inflight
>>>> requests and cancels them.
>>>>
>>>> However, the fact that blk_mq_complete_request may defer to a different
>>>> core that serialization breaks.
>>>>
>>>> Hence re-introduce blk_mq_complete_request_sync and use that in the
>>>> timeout handler to make sure that we don't get a use-after-free
>>>> condition.
>>>>
>>>> This was uncovered by the blktests:
>>>> -- 
>>>> $ nvme_trtype=tcp ./check nvme/012
>>>>
>>>> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
>>>> [ 2152.799640] loop: module loaded
>>>> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>>> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>>> [ 2152.937889] nvmet: creating controller 1 for subsystem 
>>>> blktests-subsystem-1 for NQN 
>>>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>>> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
>>>> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>>> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", 
>>>> addr 127.0.0.1:4420
>>>> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
>>>> [ 2154.942432] XFS (nvme0n1): Ending clean mount
>>>> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests 
>>>> supports timestamps until 2038 (0x7fffffff)
>>>> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
>>>> [ 2215.199331] nvme nvme0: starting error recovery
>>>> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
>>>> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
>>>> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
>>>> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
>>>> [ 2215.215502] ------------[ cut here ]------------
>>>> [ 2215.215505] refcount_t: underflow; use-after-free.
>>>> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 
>>>> refcount_warn_saturate+0xa6/0xf0
>>>> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
>>>> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: 
>>>> ffff93f37dcd8d08
>>>> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: 
>>>> ffff93f37dcd8d00
>>>> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: 
>>>> ffffffffa7fffbc4
>>>> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: 
>>>> ffff93f37a8124e8
>>>> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: 
>>>> ffff93f37a2b0000
>>>> [ 2215.215765] FS:? 0000000000000000(0000) GS:ffff93f37dcc0000(0000) 
>>>> knlGS:0000000000000000
>>>> [ 2215.215768] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 
>>>> 00000000000006e0
>>>> [ 2215.215773] Call Trace:
>>>> [ 2215.215784]? blk_mq_check_expired+0x109/0x1b0
>>>> [ 2215.215797]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>>>> [ 2215.215801]? ? blk_poll+0x300/0x300
>>>> [ 2215.215806]? blk_mq_timeout_work+0x44/0xe0
>>>> [ 2215.215814]? process_one_work+0x1b4/0x370
>>>> [ 2215.215820]? worker_thread+0x53/0x3e0
>>>> [ 2215.215825]? ? process_one_work+0x370/0x370
>>>> [ 2215.215829]? kthread+0x11b/0x140
>>>> [ 2215.215834]? ? __kthread_bind_mask+0x60/0x60
>>>> [ 2215.215841]? ret_from_fork+0x22/0x30
>>>> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
>>>> -- 
>>>>
>>>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>>>> Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
>>>> Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
>>>> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
>>>> ---
>>>> Yi, would appreciate your Tested-by tag
>>>>
>>>> ? block/blk-mq.c?????????? | 7 +++++++
>>>> ? drivers/nvme/host/rdma.c | 2 +-
>>>> ? drivers/nvme/host/tcp.c? | 2 +-
>>>> ? include/linux/blk-mq.h?? | 1 +
>>>> ? 4 files changed, 10 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>>>> index f8e1e759c905..2d154722ef39 100644
>>>> --- a/block/blk-mq.c
>>>> +++ b/block/blk-mq.c
>>>> @@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct 
>>>> request *rq)
>>>> ? }
>>>> ? EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
>>>> ? +void blk_mq_complete_request_sync(struct request *rq)
>>>> +{
>>>> +??? WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
>>>> +??? rq->q->mq_ops->complete(rq);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
>>>> +
>>>> ? /**
>>>> ?? * blk_mq_complete_request - end I/O on a request
>>>> ?? * @rq:??????? the request being processed
>>>> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
>>>> index 9e378d0a0c01..45fc605349da 100644
>>>> --- a/drivers/nvme/host/rdma.c
>>>> +++ b/drivers/nvme/host/rdma.c
>>>> @@ -1975,7 +1975,7 @@ static void 
>>>> nvme_rdma_complete_timed_out(struct request *rq)
>>>> ????? nvme_rdma_stop_queue(queue);
>>>> ????? if (!blk_mq_request_completed(rq)) {
>>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>> -??????? blk_mq_complete_request(rq);
>>>> +??????? blk_mq_complete_request_sync(rq);
>>>> ????? }
>>>> ????? mutex_unlock(&ctrl->teardown_lock);
>>>> ? }
>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>> index 8f4f29f18b8c..629b025685d1 100644
>>>> --- a/drivers/nvme/host/tcp.c
>>>> +++ b/drivers/nvme/host/tcp.c
>>>> @@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct 
>>>> request *rq)
>>>> ????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>> ????? if (!blk_mq_request_completed(rq)) {
>>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>> -??????? blk_mq_complete_request(rq);
>>>> +??????? blk_mq_complete_request_sync(rq);
>>>
>>> Or complete the request in the following way? Then one block layer API
>>> can be saved:
>>>
>>> ????blk_mq_complete_request_remote(rq);
>>> ????nvme_complete_rq(rq);
>>>
>>>
>>> thanks,
>>> Ming
>>>
>>>
>>> _______________________________________________
>>> Linux-nvme mailing list
>>> Linux-nvme at lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>>
>>>
>>> _______________________________________________
>>> Linux-nvme mailing list
>>> Linux-nvme at lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>>
>>
> 


From logang at deltatee.com  Fri Oct  9 19:18:16 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Fri,  9 Oct 2020 17:18:16 -0600
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
Message-ID: <20201009231816.1524-1-logang@deltatee.com>

Clean up some confusing elements of nvmet_passthru_map_sg() by returning
early if the request is greater than the maximum bio size. This allows
us to drop the sg_cnt variable.

This should not result in any functional change but makes the code
clearer and more understandable. The original code allocated a truncated
bio then would return EINVAL when bio_add_pc_page() filled that bio. The
new code just returns EINVAL early if this would happen.

Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index dacfa7435d0b..076e829490a1 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -180,18 +180,20 @@ static void nvmet_passthru_req_done(struct request *rq,
 
 static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 {
-	int sg_cnt = req->sg_cnt;
 	struct scatterlist *sg;
 	int op_flags = 0;
 	struct bio *bio;
 	int i, ret;
 
+	if (req->sg_cnt > BIO_MAX_PAGES)
+		return -EINVAL;
+
 	if (req->cmd->common.opcode == nvme_cmd_flush)
 		op_flags = REQ_FUA;
 	else if (nvme_is_write(req->cmd))
 		op_flags = REQ_SYNC | REQ_IDLE;
 
-	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
+	bio = bio_alloc(GFP_KERNEL, req->sg_cnt);
 	bio->bi_end_io = bio_put;
 	bio->bi_opf = req_op(rq) | op_flags;
 
@@ -201,7 +203,6 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 			bio_put(bio);
 			return -EINVAL;
 		}
-		sg_cnt--;
 	}
 
 	ret = blk_rq_append_bio(rq, &bio);

base-commit: 549738f15da0e5a00275977623be199fbbf7df50
-- 
2.20.1



From yi.zhang at redhat.com  Sat Oct 10 02:08:32 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Sat, 10 Oct 2020 14:08:32 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
Message-ID: <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>



On 10/10/20 2:29 AM, Sagi Grimberg wrote:
>
>
> On 10/9/20 6:55 AM, Yi Zhang wrote:
>> Hi Sagi
>>
>> On 10/9/20 4:09 PM, Sagi Grimberg wrote:
>>>> Hi Sagi
>>>>
>>>> I applied this patch on block origin/for-next and still can 
>>>> reproduce it.
>>>
>>> That's unexpected, can you try this patch?
>>> -- 
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index 629b025685d1..46428ff0b0fc 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct 
>>> request *rq)
>>> ??????? /* fence other contexts that may complete the command */
>>> ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>> ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>> -?????? if (!blk_mq_request_completed(rq)) {
>>> +?????? if (blk_mq_request_started(rq) && 
>>> !blk_mq_request_completed(rq)) {
>>> ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>> ??????????????? blk_mq_complete_request_sync(rq);
>>> ??????? }
>> No WARNING with this patch, but the test still will be hang and never 
>> finished, here is the kernel log:
>
> Thanks Yi,
>
> Good to know no warning, that confirms the issue analysis.
>
Cool, feel fee to add Tested-by: Yi Zhang <yi.zhang at redhat.com>

> As for the test, I'm able to successfuly pass this on my VM, and
> this test is not designed to generate errors, just fio to
> a file backed namespace. I'm wandering what is causing these
> delays in I/O that cause timeouts...
>
I just found all the failure occurred on AMD with 4G memory, not sure 
whether it's related to the low memory.
Let me know if you need any test for further investigation . :)
>>
>> [ 3327.011067] run blktests nvme/012 at 2020-10-09 06:21:59
>> [ 3327.278169] loop: module loaded
>> [ 3327.314287] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>> [ 3327.346276] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>> [ 3327.396746] nvmet: creating controller 1 for subsystem 
>> blktests-subsystem-1 for NQN 
>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>> [ 3327.411000] nvme nvme0: creating 12 I/O queues.
>> [ 3327.416963] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>> [ 3327.428836] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
>> 127.0.0.1:4420
>> [ 3329.403658] XFS (nvme0n1): Mounting V5 Filesystem
>> [ 3329.417213] XFS (nvme0n1): Ending clean mount
>> [ 3329.423044] xfs filesystem being mounted at /mnt/blktests supports 
>> timestamps until 2038 (0x7fffffff)
>> [ 3390.069126] nvme nvme0: queue 6: timeout request 0x21 type 4
>> [ 3390.074797] nvme nvme0: starting error recovery
>> [ 3390.079362] nvme nvme0: queue 6: timeout request 0x22 type 4
>> [ 3390.079932] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.081999] nvme nvme0: Reconnecting in 10 seconds...
>> [ 3390.095467] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.100879] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.106279] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.111691] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.117098] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.122507] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.127920] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.133333] block nvme0n1: no usable path - requeuing I/O
>> [ 3390.138734] block nvme0n1: no usable path - requeuing I/O
>> [ 3415.669659] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
>> [ 3415.675682] nvmet: ctrl 2 fatal error occurred!
>> [ 3419.765738] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
>> [ 3419.771762] nvmet: ctrl 1 fatal error occurred!
>> [ 3440.000352] nvmet: creating controller 2 for subsystem 
>> blktests-subsystem-1 for NQN 
>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>> [ 3463.286700] nvme nvme0: queue 0: timeout request 0x0 type 4
>> [ 3463.292357] nvme nvme0: Connect command failed, error wo/DNR bit: 881
>> [ 3463.298830] nvme nvme0: failed to connect queue: 0 ret=881
>> [ 3463.304352] nvme nvme0: Failed reconnect attempt 1
>> [ 3463.309156] nvme nvme0: Reconnecting in 10 seconds...
>> [ 3473.527574] nvmet: creating controller 1 for subsystem 
>> blktests-subsystem-1 for NQN 
>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>> [ 3473.541438] nvme_ns_head_submit_bio: 47 callbacks suppressed
>> [ 3473.541445] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.552521] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.557923] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.563319] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.568739] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.574678] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.580068] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.585461] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.590861] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.596249] block nvme0n1: no usable path - requeuing I/O
>> [ 3473.601880] nvme nvme0: creating 12 I/O queues.
>> [ 3473.613955] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>> [ 3473.622106] nvme nvme0: Successfully reconnected (2 attempt)
>> [ 3531.384170] nvme nvme0: queue 1: timeout request 0x62 type 4
>> [ 3531.389837] nvme nvme0: starting error recovery
>> [ 3531.394396] nvme nvme0: queue 7: timeout request 0x61 type 4
>> [ 3531.394954] nvme_ns_head_submit_bio: 14 callbacks suppressed
>> [ 3531.394964] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.411147] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.416548] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.421954] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.427361] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.432763] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.438287] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.443701] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.449103] block nvme0n1: no usable path - requeuing I/O
>> [ 3531.454497] block nvme0n1: no usable path - requeuing I/O
>> [ 3560.568832] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
>> [ 3560.574839] nvmet: ctrl 1 fatal error occurred!
>>
>>
>>> -- 
>>>
>>>>
>>>> [? 724.267865] run blktests nvme/012 at 2020-10-09 00:48:52
>>>> [? 724.518498] loop: module loaded
>>>> [? 724.552880] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>>> [? 724.586609] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>>> [? 724.648898] nvmet: creating controller 1 for subsystem 
>>>> blktests-subsystem-1 for NQN 
>>>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>>> [? 724.663300] nvme nvme0: creating 12 I/O queues.
>>>> [? 724.669826] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>>> [? 724.681350] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", 
>>>> addr 127.0.0.1:4420
>>>> [? 726.641605] XFS (nvme0n1): Mounting V5 Filesystem
>>>> [? 726.654637] XFS (nvme0n1): Ending clean mount
>>>> [? 726.660302] xfs filesystem being mounted at /mnt/blktests 
>>>> supports timestamps until 2038 (0x7fffffff)
>>>> [? 787.042066] nvme nvme0: queue 2: timeout request 0xe type 4
>>>> [? 787.047649] nvme nvme0: starting error recovery
>>>> [? 787.052211] nvme nvme0: queue 4: timeout request 0x61 type 4
>>>> [? 787.053407] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.054543] nvme nvme0: Reconnecting in 10 seconds...
>>>> [? 787.057877] ------------[ cut here ]------------
>>>> [? 787.063274] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.068313] refcount_t: underflow; use-after-free.
>>>> [? 787.068370] WARNING: CPU: 3 PID: 30 at lib/refcount.c:28 
>>>> refcount_warn_saturate+0xa6/0xf0
>>>> [? 787.072941] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.072945] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.078333] Modules linked in: loop nvme_tcp nvme_fabrics 
>>>> nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 
>>>> dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek 
>>>> snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg 
>>>> snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer 
>>>> amd64_edac_mod snd edac_mce_amd soundcore kvm_amd ccp kvm pcspkr 
>>>> irqbypass hp_wmi sparse_keymap rfkill k10temp wmi_bmof nv_tco 
>>>> i2c_nforce2 acpi_cpufreq ip_tables xfs nouveau video mxm_wmi 
>>>> i2c_algo_bit drm_kms_helper mptsas cec scsi_transport_sas ttm 
>>>> mptscsih ata_generic firewire_ohci pata_acpi firewire_core drm 
>>>> serio_raw mptbase forcedeth crc_itu_t sata_nv pata_amd wmi floppy
>>>> [? 787.083142] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.091304] CPU: 3 PID: 30 Comm: kworker/3:0H Not tainted 
>>>> 5.9.0-rc8.update+ #3
>>>> [? 787.096673] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.096674] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.096680] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.096681] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.096683] block nvme0n1: no usable path - requeuing I/O
>>>> [? 787.201242] Hardware name: Hewlett-Packard HP xw9400 
>>>> Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
>>>> [? 787.210366] Workqueue: kblockd blk_mq_timeout_work
>>>> [? 787.215161] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
>>>> [? 787.220295] Code: 05 aa 98 22 01 01 e8 9f cf ad ff 0f 0b c3 80 
>>>> 3d 98 98 22 01 00 75 95 48 c7 c7 48 54 3e a2 c6 05 88 98 22 01 01 
>>>> e8 80 cf ad ff <0f> 0b c3 80 3d 77 98 22 01 00 0f 85 72 ff ff ff 48 
>>>> c7 c7 a0 54 3e
>>>> [? 787.239021] RSP: 0018:ffffb860407b3dc8 EFLAGS: 00010292
>>>> [? 787.244240] RAX: 0000000000000026 RBX: 0000000000000000 RCX: 
>>>> ffff9d93ddc58d08
>>>> [? 787.251361] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: 
>>>> ffff9d93ddc58d00
>>>> [? 787.258480] RBP: ffff9d93dbd6c200 R08: 000000b740ed60e8 R09: 
>>>> ffffffffa2fffbc4
>>>> [? 787.265612] R10: 0000000000000477 R11: 0000000000028320 R12: 
>>>> ffff9d93dbd6c2e8
>>>> [? 787.272741] R13: ffff9d93dbea0000 R14: ffffb860407b3e70 R15: 
>>>> ffff9d93dbea0000
>>>> [? 787.279864] FS:? 0000000000000000(0000) 
>>>> GS:ffff9d93ddc40000(0000) knlGS:0000000000000000
>>>> [? 787.287948] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>> [? 787.293695] CR2: 00007f45fd8fa024 CR3: 00000000792dc000 CR4: 
>>>> 00000000000006e0
>>>> [? 787.300820] Call Trace:
>>>> [? 787.303277]? blk_mq_check_expired+0x109/0x1b0
>>>> [? 787.307638]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>>>> [? 787.312524]? ? blk_poll+0x300/0x300
>>>> [? 787.316022]? blk_mq_timeout_work+0x44/0xe0
>>>> [? 787.320128]? process_one_work+0x1b4/0x370
>>>> [? 787.324137]? worker_thread+0x53/0x3e0
>>>> [? 787.327809]? ? process_one_work+0x370/0x370
>>>> [? 787.332015]? kthread+0x11b/0x140
>>>> [? 787.335245]? ? __kthread_bind_mask+0x60/0x60
>>>> [? 787.339515]? ret_from_fork+0x22/0x30
>>>> [? 787.343094] ---[ end trace 606cf9189379fcfc ]---
>>>>
>>>>
>>>> Best Regards,
>>>> ?? Yi Zhang
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Ming Lei" <ming.lei at redhat.com>
>>>> To: "Sagi Grimberg" <sagi at grimberg.me>
>>>> Cc: "Jens Axboe" <axboe at kernel.dk>, "Yi Zhang" 
>>>> <yi.zhang at redhat.com>, linux-nvme at lists.infradead.org, 
>>>> linux-block at vger.kernel.org, "Keith Busch" <kbusch at kernel.org>, 
>>>> "Christoph Hellwig" <hch at lst.de>
>>>> Sent: Friday, October 9, 2020 12:39:38 PM
>>>> Subject: Re: [PATCH] block: re-introduce blk_mq_complete_request_sync
>>>>
>>>> On Thu, Oct 08, 2020 at 02:37:50PM -0700, Sagi Grimberg wrote:
>>>>> In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
>>>>> needs to complete an aborted request. The timeout handler serializes
>>>>> against an error recovery sequence that iterates over the inflight
>>>>> requests and cancels them.
>>>>>
>>>>> However, the fact that blk_mq_complete_request may defer to a 
>>>>> different
>>>>> core that serialization breaks.
>>>>>
>>>>> Hence re-introduce blk_mq_complete_request_sync and use that in the
>>>>> timeout handler to make sure that we don't get a use-after-free
>>>>> condition.
>>>>>
>>>>> This was uncovered by the blktests:
>>>>> -- 
>>>>> $ nvme_trtype=tcp ./check nvme/012
>>>>>
>>>>> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
>>>>> [ 2152.799640] loop: module loaded
>>>>> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>>>> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>>>> [ 2152.937889] nvmet: creating controller 1 for subsystem 
>>>>> blktests-subsystem-1 for NQN 
>>>>> nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>>>> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
>>>>> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>>>> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", 
>>>>> addr 127.0.0.1:4420
>>>>> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
>>>>> [ 2154.942432] XFS (nvme0n1): Ending clean mount
>>>>> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests 
>>>>> supports timestamps until 2038 (0x7fffffff)
>>>>> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
>>>>> [ 2215.199331] nvme nvme0: starting error recovery
>>>>> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
>>>>> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
>>>>> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
>>>>> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
>>>>> [ 2215.215502] ------------[ cut here ]------------
>>>>> [ 2215.215505] refcount_t: underflow; use-after-free.
>>>>> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 
>>>>> refcount_warn_saturate+0xa6/0xf0
>>>>> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
>>>>> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: 
>>>>> ffff93f37dcd8d08
>>>>> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: 
>>>>> ffff93f37dcd8d00
>>>>> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: 
>>>>> ffffffffa7fffbc4
>>>>> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: 
>>>>> ffff93f37a8124e8
>>>>> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: 
>>>>> ffff93f37a2b0000
>>>>> [ 2215.215765] FS:? 0000000000000000(0000) 
>>>>> GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
>>>>> [ 2215.215768] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 
>>>>> 00000000000006e0
>>>>> [ 2215.215773] Call Trace:
>>>>> [ 2215.215784]? blk_mq_check_expired+0x109/0x1b0
>>>>> [ 2215.215797]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>>>>> [ 2215.215801]? ? blk_poll+0x300/0x300
>>>>> [ 2215.215806]? blk_mq_timeout_work+0x44/0xe0
>>>>> [ 2215.215814]? process_one_work+0x1b4/0x370
>>>>> [ 2215.215820]? worker_thread+0x53/0x3e0
>>>>> [ 2215.215825]? ? process_one_work+0x370/0x370
>>>>> [ 2215.215829]? kthread+0x11b/0x140
>>>>> [ 2215.215834]? ? __kthread_bind_mask+0x60/0x60
>>>>> [ 2215.215841]? ret_from_fork+0x22/0x30
>>>>> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
>>>>> -- 
>>>>>
>>>>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>>>>> Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
>>>>> Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
>>>>> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
>>>>> ---
>>>>> Yi, would appreciate your Tested-by tag
>>>>>
>>>>> ? block/blk-mq.c?????????? | 7 +++++++
>>>>> ? drivers/nvme/host/rdma.c | 2 +-
>>>>> ? drivers/nvme/host/tcp.c? | 2 +-
>>>>> ? include/linux/blk-mq.h?? | 1 +
>>>>> ? 4 files changed, 10 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>>>>> index f8e1e759c905..2d154722ef39 100644
>>>>> --- a/block/blk-mq.c
>>>>> +++ b/block/blk-mq.c
>>>>> @@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct 
>>>>> request *rq)
>>>>> ? }
>>>>> ? EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
>>>>> ? +void blk_mq_complete_request_sync(struct request *rq)
>>>>> +{
>>>>> +??? WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
>>>>> +??? rq->q->mq_ops->complete(rq);
>>>>> +}
>>>>> +EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
>>>>> +
>>>>> ? /**
>>>>> ?? * blk_mq_complete_request - end I/O on a request
>>>>> ?? * @rq:??????? the request being processed
>>>>> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
>>>>> index 9e378d0a0c01..45fc605349da 100644
>>>>> --- a/drivers/nvme/host/rdma.c
>>>>> +++ b/drivers/nvme/host/rdma.c
>>>>> @@ -1975,7 +1975,7 @@ static void 
>>>>> nvme_rdma_complete_timed_out(struct request *rq)
>>>>> ????? nvme_rdma_stop_queue(queue);
>>>>> ????? if (!blk_mq_request_completed(rq)) {
>>>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>>> -??????? blk_mq_complete_request(rq);
>>>>> +??????? blk_mq_complete_request_sync(rq);
>>>>> ????? }
>>>>> ????? mutex_unlock(&ctrl->teardown_lock);
>>>>> ? }
>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>> index 8f4f29f18b8c..629b025685d1 100644
>>>>> --- a/drivers/nvme/host/tcp.c
>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>> @@ -2177,7 +2177,7 @@ static void 
>>>>> nvme_tcp_complete_timed_out(struct request *rq)
>>>>> ????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>>> ????? if (!blk_mq_request_completed(rq)) {
>>>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>>> -??????? blk_mq_complete_request(rq);
>>>>> +??????? blk_mq_complete_request_sync(rq);
>>>>
>>>> Or complete the request in the following way? Then one block layer API
>>>> can be saved:
>>>>
>>>> ????blk_mq_complete_request_remote(rq);
>>>> ????nvme_complete_rq(rq);
>>>>
>>>>
>>>> thanks,
>>>> Ming
>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-nvme mailing list
>>>> Linux-nvme at lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-nvme mailing list
>>>> Linux-nvme at lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>>>
>>>
>>
>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme



From gerlitz.or at gmail.com  Sun Oct 11 02:54:37 2020
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Sun, 11 Oct 2020 09:54:37 +0300
Subject: [PATCH net-next RFC v1 08/10] nvme-tcp: Deal with netdevice DOWN
 events
In-Reply-To: <67e29f83-5bab-4abd-44c0-9c5ae29d5784@grimberg.me>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-9-borisp@mellanox.com>
 <67e29f83-5bab-4abd-44c0-9c5ae29d5784@grimberg.me>
Message-ID: <CAJ3xEMh4FBQ583bhxbXwHZKmZLX28Obf6gArP0qpd5OebnuuSw@mail.gmail.com>

On Fri, Oct 9, 2020 at 1:50 AM Sagi Grimberg <sagi at grimberg.me> wrote:

> > @@ -412,8 +413,6 @@ int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue,
> >               queue->ctrl->ctrl.max_segments = limits->max_ddp_sgl_len;
> >               queue->ctrl->ctrl.max_hw_sectors =
> >                       limits->max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
> > -     } else {
> > -             queue->ctrl->offloading_netdev = NULL;
>
> Squash this change to the patch that introduced it.

OK, will look on that and I guess it should be fine to make this as
you suggested


> > +     case NETDEV_GOING_DOWN:
> > +             mutex_lock(&nvme_tcp_ctrl_mutex);
> > +             list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list) {
> > +                     if (ndev != ctrl->offloading_netdev)
> > +                             continue;
> > +                     nvme_tcp_error_recovery(&ctrl->ctrl);
> > +             }
> > +             mutex_unlock(&nvme_tcp_ctrl_mutex);
> > +             flush_workqueue(nvme_reset_wq);
>
> Worth a small comment that this we want the err_work to complete
> here. So if someone changes workqueues he may see this.


ack


From lengchao at huawei.com  Sun Oct 11 23:59:21 2020
From: lengchao at huawei.com (Chao Leng)
Date: Mon, 12 Oct 2020 11:59:21 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
Message-ID: <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>



On 2020/10/10 14:08, Yi Zhang wrote:
> 
> 
> On 10/10/20 2:29 AM, Sagi Grimberg wrote:
>>
>>
>> On 10/9/20 6:55 AM, Yi Zhang wrote:
>>> Hi Sagi
>>>
>>> On 10/9/20 4:09 PM, Sagi Grimberg wrote:
>>>>> Hi Sagi
>>>>>
>>>>> I applied this patch on block origin/for-next and still can reproduce it.
>>>>
>>>> That's unexpected, can you try this patch?
>>>> -- 
>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>> index 629b025685d1..46428ff0b0fc 100644
>>>> --- a/drivers/nvme/host/tcp.c
>>>> +++ b/drivers/nvme/host/tcp.c
>>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>>> ??????? /* fence other contexts that may complete the command */
>>>> ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>> ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>> -?????? if (!blk_mq_request_completed(rq)) {
>>>> +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>>>> ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>> ??????????????? blk_mq_complete_request_sync(rq);
>>>> ??????? }
This may just reduce the probability. The concurrency of timeout and teardown will cause the same request
be treated repeatly, this is not we expected.
In the teardown process, after quiesced queues delete the timer and cancel the timeout work maybe a better option.
Teardown process will cancel all doing IO, timer and timeout work should be canceled.
After unquiesce queue, the timer of queue will be add when start request.
>>> No WARNING with this patch, but the test still will be hang and never finished, here is the kernel log:
>>
>> Thanks Yi,
>>
>> Good to know no warning, that confirms the issue analysis.
>>
> Cool, feel fee to add Tested-by: Yi Zhang <yi.zhang at redhat.com>
> 
>> As for the test, I'm able to successfuly pass this on my VM, and
>> this test is not designed to generate errors, just fio to
>> a file backed namespace. I'm wandering what is causing these
>> delays in I/O that cause timeouts...
>>
> I just found all the failure occurred on AMD with 4G memory, not sure whether it's related to the low memory.
> Let me know if you need any test for further investigation . :)
>>>
>>> [ 3327.011067] run blktests nvme/012 at 2020-10-09 06:21:59
>>> [ 3327.278169] loop: module loaded
>>> [ 3327.314287] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>> [ 3327.346276] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>> [ 3327.396746] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>> [ 3327.411000] nvme nvme0: creating 12 I/O queues.
>>> [ 3327.416963] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>> [ 3327.428836] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
>>> [ 3329.403658] XFS (nvme0n1): Mounting V5 Filesystem
>>> [ 3329.417213] XFS (nvme0n1): Ending clean mount
>>> [ 3329.423044] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
>>> [ 3390.069126] nvme nvme0: queue 6: timeout request 0x21 type 4
>>> [ 3390.074797] nvme nvme0: starting error recovery
>>> [ 3390.079362] nvme nvme0: queue 6: timeout request 0x22 type 4
>>> [ 3390.079932] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.081999] nvme nvme0: Reconnecting in 10 seconds...
>>> [ 3390.095467] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.100879] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.106279] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.111691] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.117098] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.122507] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.127920] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.133333] block nvme0n1: no usable path - requeuing I/O
>>> [ 3390.138734] block nvme0n1: no usable path - requeuing I/O
>>> [ 3415.669659] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
>>> [ 3415.675682] nvmet: ctrl 2 fatal error occurred!
>>> [ 3419.765738] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
>>> [ 3419.771762] nvmet: ctrl 1 fatal error occurred!
>>> [ 3440.000352] nvmet: creating controller 2 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>> [ 3463.286700] nvme nvme0: queue 0: timeout request 0x0 type 4
>>> [ 3463.292357] nvme nvme0: Connect command failed, error wo/DNR bit: 881
>>> [ 3463.298830] nvme nvme0: failed to connect queue: 0 ret=881
>>> [ 3463.304352] nvme nvme0: Failed reconnect attempt 1
>>> [ 3463.309156] nvme nvme0: Reconnecting in 10 seconds...
>>> [ 3473.527574] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>> [ 3473.541438] nvme_ns_head_submit_bio: 47 callbacks suppressed
>>> [ 3473.541445] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.552521] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.557923] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.563319] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.568739] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.574678] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.580068] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.585461] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.590861] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.596249] block nvme0n1: no usable path - requeuing I/O
>>> [ 3473.601880] nvme nvme0: creating 12 I/O queues.
>>> [ 3473.613955] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>> [ 3473.622106] nvme nvme0: Successfully reconnected (2 attempt)
>>> [ 3531.384170] nvme nvme0: queue 1: timeout request 0x62 type 4
>>> [ 3531.389837] nvme nvme0: starting error recovery
>>> [ 3531.394396] nvme nvme0: queue 7: timeout request 0x61 type 4
>>> [ 3531.394954] nvme_ns_head_submit_bio: 14 callbacks suppressed
>>> [ 3531.394964] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.411147] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.416548] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.421954] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.427361] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.432763] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.438287] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.443701] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.449103] block nvme0n1: no usable path - requeuing I/O
>>> [ 3531.454497] block nvme0n1: no usable path - requeuing I/O
>>> [ 3560.568832] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
>>> [ 3560.574839] nvmet: ctrl 1 fatal error occurred!
>>>
>>>
>>>> -- 
>>>>
>>>>>
>>>>> [? 724.267865] run blktests nvme/012 at 2020-10-09 00:48:52
>>>>> [? 724.518498] loop: module loaded
>>>>> [? 724.552880] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>>>> [? 724.586609] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>>>> [? 724.648898] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>>>> [? 724.663300] nvme nvme0: creating 12 I/O queues.
>>>>> [? 724.669826] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>>>> [? 724.681350] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
>>>>> [? 726.641605] XFS (nvme0n1): Mounting V5 Filesystem
>>>>> [? 726.654637] XFS (nvme0n1): Ending clean mount
>>>>> [? 726.660302] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
>>>>> [? 787.042066] nvme nvme0: queue 2: timeout request 0xe type 4
>>>>> [? 787.047649] nvme nvme0: starting error recovery
>>>>> [? 787.052211] nvme nvme0: queue 4: timeout request 0x61 type 4
>>>>> [? 787.053407] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.054543] nvme nvme0: Reconnecting in 10 seconds...
>>>>> [? 787.057877] ------------[ cut here ]------------
>>>>> [? 787.063274] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.068313] refcount_t: underflow; use-after-free.
>>>>> [? 787.068370] WARNING: CPU: 3 PID: 30 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
>>>>> [? 787.072941] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.072945] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.078333] Modules linked in: loop nvme_tcp nvme_fabrics nvme_core nvmet_tcp nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer amd64_edac_mod snd edac_mce_amd soundcore kvm_amd ccp kvm pcspkr irqbypass hp_wmi sparse_keymap rfkill k10temp wmi_bmof nv_tco i2c_nforce2 acpi_cpufreq ip_tables xfs nouveau video mxm_wmi i2c_algo_bit drm_kms_helper mptsas cec scsi_transport_sas ttm mptscsih ata_generic firewire_ohci pata_acpi firewire_core drm serio_raw mptbase forcedeth crc_itu_t sata_nv pata_amd wmi floppy
>>>>> [? 787.083142] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.091304] CPU: 3 PID: 30 Comm: kworker/3:0H Not tainted 5.9.0-rc8.update+ #3
>>>>> [? 787.096673] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.096674] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.096680] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.096681] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.096683] block nvme0n1: no usable path - requeuing I/O
>>>>> [? 787.201242] Hardware name: Hewlett-Packard HP xw9400 Workstation/0A1Ch, BIOS 786D6 v04.02 04/21/2009
>>>>> [? 787.210366] Workqueue: kblockd blk_mq_timeout_work
>>>>> [? 787.215161] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
>>>>> [? 787.220295] Code: 05 aa 98 22 01 01 e8 9f cf ad ff 0f 0b c3 80 3d 98 98 22 01 00 75 95 48 c7 c7 48 54 3e a2 c6 05 88 98 22 01 01 e8 80 cf ad ff <0f> 0b c3 80 3d 77 98 22 01 00 0f 85 72 ff ff ff 48 c7 c7 a0 54 3e
>>>>> [? 787.239021] RSP: 0018:ffffb860407b3dc8 EFLAGS: 00010292
>>>>> [? 787.244240] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff9d93ddc58d08
>>>>> [? 787.251361] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff9d93ddc58d00
>>>>> [? 787.258480] RBP: ffff9d93dbd6c200 R08: 000000b740ed60e8 R09: ffffffffa2fffbc4
>>>>> [? 787.265612] R10: 0000000000000477 R11: 0000000000028320 R12: ffff9d93dbd6c2e8
>>>>> [? 787.272741] R13: ffff9d93dbea0000 R14: ffffb860407b3e70 R15: ffff9d93dbea0000
>>>>> [? 787.279864] FS:? 0000000000000000(0000) GS:ffff9d93ddc40000(0000) knlGS:0000000000000000
>>>>> [? 787.287948] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>> [? 787.293695] CR2: 00007f45fd8fa024 CR3: 00000000792dc000 CR4: 00000000000006e0
>>>>> [? 787.300820] Call Trace:
>>>>> [? 787.303277]? blk_mq_check_expired+0x109/0x1b0
>>>>> [? 787.307638]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>>>>> [? 787.312524]? ? blk_poll+0x300/0x300
>>>>> [? 787.316022]? blk_mq_timeout_work+0x44/0xe0
>>>>> [? 787.320128]? process_one_work+0x1b4/0x370
>>>>> [? 787.324137]? worker_thread+0x53/0x3e0
>>>>> [? 787.327809]? ? process_one_work+0x370/0x370
>>>>> [? 787.332015]? kthread+0x11b/0x140
>>>>> [? 787.335245]? ? __kthread_bind_mask+0x60/0x60
>>>>> [? 787.339515]? ret_from_fork+0x22/0x30
>>>>> [? 787.343094] ---[ end trace 606cf9189379fcfc ]---
>>>>>
>>>>>
>>>>> Best Regards,
>>>>> ?? Yi Zhang
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> From: "Ming Lei" <ming.lei at redhat.com>
>>>>> To: "Sagi Grimberg" <sagi at grimberg.me>
>>>>> Cc: "Jens Axboe" <axboe at kernel.dk>, "Yi Zhang" <yi.zhang at redhat.com>, linux-nvme at lists.infradead.org, linux-block at vger.kernel.org, "Keith Busch" <kbusch at kernel.org>, "Christoph Hellwig" <hch at lst.de>
>>>>> Sent: Friday, October 9, 2020 12:39:38 PM
>>>>> Subject: Re: [PATCH] block: re-introduce blk_mq_complete_request_sync
>>>>>
>>>>> On Thu, Oct 08, 2020 at 02:37:50PM -0700, Sagi Grimberg wrote:
>>>>>> In nvme-tcp and nvme-rdma, the timeout handler in certain conditions
>>>>>> needs to complete an aborted request. The timeout handler serializes
>>>>>> against an error recovery sequence that iterates over the inflight
>>>>>> requests and cancels them.
>>>>>>
>>>>>> However, the fact that blk_mq_complete_request may defer to a different
>>>>>> core that serialization breaks.
>>>>>>
>>>>>> Hence re-introduce blk_mq_complete_request_sync and use that in the
>>>>>> timeout handler to make sure that we don't get a use-after-free
>>>>>> condition.
>>>>>>
>>>>>> This was uncovered by the blktests:
>>>>>> -- 
>>>>>> $ nvme_trtype=tcp ./check nvme/012
>>>>>>
>>>>>> [ 2152.546343] run blktests nvme/012 at 2020-10-08 05:59:03
>>>>>> [ 2152.799640] loop: module loaded
>>>>>> [ 2152.835222] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>>>>> [ 2152.860697] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>>>>> [ 2152.937889] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:e544149126a24145abeeb6129b0ec52c.
>>>>>> [ 2152.952085] nvme nvme0: creating 12 I/O queues.
>>>>>> [ 2152.958042] nvme nvme0: mapped 12/0/0 default/read/poll queues.
>>>>>> [ 2152.969948] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
>>>>>> [ 2154.927953] XFS (nvme0n1): Mounting V5 Filesystem
>>>>>> [ 2154.942432] XFS (nvme0n1): Ending clean mount
>>>>>> [ 2154.948183] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
>>>>>> [ 2215.193645] nvme nvme0: queue 7: timeout request 0x11 type 4
>>>>>> [ 2215.199331] nvme nvme0: starting error recovery
>>>>>> [ 2215.203890] nvme nvme0: queue 7: timeout request 0x12 type 4
>>>>>> [ 2215.204483] block nvme0n1: no usable path - requeuing I/O
>>>>>> [ 2215.214976] block nvme0n1: no usable path - requeuing I/O
>>>>>> [ 2215.215495] nvme nvme0: Reconnecting in 10 seconds...
>>>>>> [ 2215.215502] ------------[ cut here ]------------
>>>>>> [ 2215.215505] refcount_t: underflow; use-after-free.
>>>>>> [ 2215.215617] WARNING: CPU: 6 PID: 45 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
>>>>>> [ 2215.215745] RSP: 0018:ffffb71b80837dc8 EFLAGS: 00010292
>>>>>> [ 2215.215750] RAX: 0000000000000026 RBX: 0000000000000000 RCX: ffff93f37dcd8d08
>>>>>> [ 2215.215753] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff93f37dcd8d00
>>>>>> [ 2215.215755] RBP: ffff93f37a812400 R08: 00000203c5221fce R09: ffffffffa7fffbc4
>>>>>> [ 2215.215758] R10: 0000000000000477 R11: 000000000002835c R12: ffff93f37a8124e8
>>>>>> [ 2215.215761] R13: ffff93f37a2b0000 R14: ffffb71b80837e70 R15: ffff93f37a2b0000
>>>>>> [ 2215.215765] FS:? 0000000000000000(0000) GS:ffff93f37dcc0000(0000) knlGS:0000000000000000
>>>>>> [ 2215.215768] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>>> [ 2215.215771] CR2: 00005637b4137028 CR3: 000000007c1be000 CR4: 00000000000006e0
>>>>>> [ 2215.215773] Call Trace:
>>>>>> [ 2215.215784]? blk_mq_check_expired+0x109/0x1b0
>>>>>> [ 2215.215797]? blk_mq_queue_tag_busy_iter+0x1b8/0x330
>>>>>> [ 2215.215801]? ? blk_poll+0x300/0x300
>>>>>> [ 2215.215806]? blk_mq_timeout_work+0x44/0xe0
>>>>>> [ 2215.215814]? process_one_work+0x1b4/0x370
>>>>>> [ 2215.215820]? worker_thread+0x53/0x3e0
>>>>>> [ 2215.215825]? ? process_one_work+0x370/0x370
>>>>>> [ 2215.215829]? kthread+0x11b/0x140
>>>>>> [ 2215.215834]? ? __kthread_bind_mask+0x60/0x60
>>>>>> [ 2215.215841]? ret_from_fork+0x22/0x30
>>>>>> [ 2215.215847] ---[ end trace 7d137e36e23c0d19 ]---
>>>>>> -- 
>>>>>>
>>>>>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>>>>>> Fixes: 236187c4ed19 ("nvme-tcp: fix timeout handler")
>>>>>> Fixes: 0475a8dcbcee ("nvme-rdma: fix timeout handler")
>>>>>> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
>>>>>> ---
>>>>>> Yi, would appreciate your Tested-by tag
>>>>>>
>>>>>> ? block/blk-mq.c?????????? | 7 +++++++
>>>>>> ? drivers/nvme/host/rdma.c | 2 +-
>>>>>> ? drivers/nvme/host/tcp.c? | 2 +-
>>>>>> ? include/linux/blk-mq.h?? | 1 +
>>>>>> ? 4 files changed, 10 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>>>>>> index f8e1e759c905..2d154722ef39 100644
>>>>>> --- a/block/blk-mq.c
>>>>>> +++ b/block/blk-mq.c
>>>>>> @@ -729,6 +729,13 @@ bool blk_mq_complete_request_remote(struct request *rq)
>>>>>> ? }
>>>>>> ? EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote);
>>>>>> ? +void blk_mq_complete_request_sync(struct request *rq)
>>>>>> +{
>>>>>> +??? WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
>>>>>> +??? rq->q->mq_ops->complete(rq);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(blk_mq_complete_request_sync);
>>>>>> +
>>>>>> ? /**
>>>>>> ?? * blk_mq_complete_request - end I/O on a request
>>>>>> ?? * @rq:??????? the request being processed
>>>>>> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
>>>>>> index 9e378d0a0c01..45fc605349da 100644
>>>>>> --- a/drivers/nvme/host/rdma.c
>>>>>> +++ b/drivers/nvme/host/rdma.c
>>>>>> @@ -1975,7 +1975,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
>>>>>> ????? nvme_rdma_stop_queue(queue);
>>>>>> ????? if (!blk_mq_request_completed(rq)) {
>>>>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>>>> -??????? blk_mq_complete_request(rq);
>>>>>> +??????? blk_mq_complete_request_sync(rq);
>>>>>> ????? }
>>>>>> ????? mutex_unlock(&ctrl->teardown_lock);
>>>>>> ? }
>>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>>> index 8f4f29f18b8c..629b025685d1 100644
>>>>>> --- a/drivers/nvme/host/tcp.c
>>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>>> @@ -2177,7 +2177,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>>>>> ????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>>>> ????? if (!blk_mq_request_completed(rq)) {
>>>>>> ????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>>>> -??????? blk_mq_complete_request(rq);
>>>>>> +??????? blk_mq_complete_request_sync(rq);
>>>>>
>>>>> Or complete the request in the following way? Then one block layer API
>>>>> can be saved:
>>>>>
>>>>> ????blk_mq_complete_request_remote(rq);
>>>>> ????nvme_complete_rq(rq);
>>>>>
>>>>>
>>>>> thanks,
>>>>> Ming
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Linux-nvme mailing list
>>>>> Linux-nvme at lists.infradead.org
>>>>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Linux-nvme mailing list
>>>>> Linux-nvme at lists.infradead.org
>>>>> http://lists.infradead.org/mailman/listinfo/linux-nvme
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Linux-nvme mailing list
>> Linux-nvme at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-nvme
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme


From lengchao at huawei.com  Mon Oct 12 04:10:40 2020
From: lengchao at huawei.com (Chao Leng)
Date: Mon, 12 Oct 2020 16:10:40 +0800
Subject: [PATCH] nvme-rdma: fix crash when connect rejected
Message-ID: <20201012081040.31741-1-lengchao@huawei.com>

A crash happened when connect rejected. host establish the connect after
received ConnectReply, and then continue to send FabricsConnect.
If target do not receive ReadyToUse, host may receive ConnectReject.
Call nvme_rdma_destroy_queue_ib after host received the event of
RDMA_CM_EVENT_REJECTED. Then the command of FabricsConnect timeout,
nvme_rdma_timeout call nvme_rdma_complete_rq to fail the request.
The crash happened due to use after free in nvme_rdma_complete_rq.

nvme_rdma_destroy_queue_ib is redundant when handle the event of
RDMA_CM_EVENT_REJECTED. nvme_rdma_destroy_queue_ib is already
called in the error handling process of connection failure.

Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/rdma.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..116902b1b2c3 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1926,7 +1926,6 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
 		complete(&queue->cm_done);
 		return 0;
 	case RDMA_CM_EVENT_REJECTED:
-		nvme_rdma_destroy_queue_ib(queue);
 		cm_error = nvme_rdma_conn_rejected(queue, ev);
 		break;
 	case RDMA_CM_EVENT_ROUTE_ERROR:
-- 
2.16.4



From ming.lei at redhat.com  Mon Oct 12 04:13:06 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Mon, 12 Oct 2020 16:13:06 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
Message-ID: <20201012081306.GB556731@T590>

On Mon, Oct 12, 2020 at 11:59:21AM +0800, Chao Leng wrote:
> 
> 
> On 2020/10/10 14:08, Yi Zhang wrote:
> > 
> > 
> > On 10/10/20 2:29 AM, Sagi Grimberg wrote:
> > > 
> > > 
> > > On 10/9/20 6:55 AM, Yi Zhang wrote:
> > > > Hi Sagi
> > > > 
> > > > On 10/9/20 4:09 PM, Sagi Grimberg wrote:
> > > > > > Hi Sagi
> > > > > > 
> > > > > > I applied this patch on block origin/for-next and still can reproduce it.
> > > > > 
> > > > > That's unexpected, can you try this patch?
> > > > > -- 
> > > > > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > > > > index 629b025685d1..46428ff0b0fc 100644
> > > > > --- a/drivers/nvme/host/tcp.c
> > > > > +++ b/drivers/nvme/host/tcp.c
> > > > > @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
> > > > > ??????? /* fence other contexts that may complete the command */
> > > > > ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
> > > > > ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
> > > > > -?????? if (!blk_mq_request_completed(rq)) {
> > > > > +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
> > > > > ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> > > > > ??????????????? blk_mq_complete_request_sync(rq);
> > > > > ??????? }
> This may just reduce the probability. The concurrency of timeout and teardown will cause the same request
> be treated repeatly, this is not we expected.

That is right, not like SCSI, NVME doesn't apply atomic request completion, so
request may be completed/freed from both timeout & nvme_cancel_request().

.teardown_lock still may cover the race with Sagi's patch because teardown
actually cancels requests in sync style.

> In the teardown process, after quiesced queues delete the timer and cancel the timeout work maybe a better option.

Seems better solution, given it is aligned with NVME PCI's reset
handling. nvme_sync_queues() may be called in nvme_tcp_teardown_io_queues() to
avoid this race.


Thanks, 
Ming



From lengchao at huawei.com  Mon Oct 12 04:55:37 2020
From: lengchao at huawei.com (Chao Leng)
Date: Mon, 12 Oct 2020 16:55:37 +0800
Subject: [PATCH] nvme-rdma: fix crash due to incorrect cqe
Message-ID: <20201012085537.2599-1-lengchao@huawei.com>

A crash happened due to injecting error test. The cqe has incorrect
command id, host may find a request which already be freed.
req->mr->rkey cause a crash in nvme_rdma_process_nvme_rsp.
Because the mr is already freed.

Add a check for the mr to fix it.

Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/rdma.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..70dbc289d22e 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1730,10 +1730,11 @@ static void nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
 	req->result = cqe->result;
 
 	if (wc->wc_flags & IB_WC_WITH_INVALIDATE) {
-		if (unlikely(wc->ex.invalidate_rkey != req->mr->rkey)) {
+		if (unlikely(!req->mr ||
+			     wc->ex.invalidate_rkey != req->mr->rkey)) {
 			dev_err(queue->ctrl->ctrl.device,
 				"Bogus remote invalidation for rkey %#x\n",
-				req->mr->rkey);
+				req->mr ? req->mr->rkey : 0);
 			nvme_rdma_error_recovery(queue->ctrl);
 		}
 	} else if (req->mr) {
-- 
2.16.4



From lengchao at huawei.com  Mon Oct 12 05:06:23 2020
From: lengchao at huawei.com (Chao Leng)
Date: Mon, 12 Oct 2020 17:06:23 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201012081306.GB556731@T590>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
Message-ID: <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>



On 2020/10/12 16:13, Ming Lei wrote:
> On Mon, Oct 12, 2020 at 11:59:21AM +0800, Chao Leng wrote:
>>
>>
>> On 2020/10/10 14:08, Yi Zhang wrote:
>>>
>>>
>>> On 10/10/20 2:29 AM, Sagi Grimberg wrote:
>>>>
>>>>
>>>> On 10/9/20 6:55 AM, Yi Zhang wrote:
>>>>> Hi Sagi
>>>>>
>>>>> On 10/9/20 4:09 PM, Sagi Grimberg wrote:
>>>>>>> Hi Sagi
>>>>>>>
>>>>>>> I applied this patch on block origin/for-next and still can reproduce it.
>>>>>>
>>>>>> That's unexpected, can you try this patch?
>>>>>> -- 
>>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>>> index 629b025685d1..46428ff0b0fc 100644
>>>>>> --- a/drivers/nvme/host/tcp.c
>>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>>>>>  ??????? /* fence other contexts that may complete the command */
>>>>>>  ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>>>  ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>>>> -?????? if (!blk_mq_request_completed(rq)) {
>>>>>> +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>>>>>>  ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>>>>  ??????????????? blk_mq_complete_request_sync(rq);
>>>>>>  ??????? }
>> This may just reduce the probability. The concurrency of timeout and teardown will cause the same request
>> be treated repeatly, this is not we expected.
> 
> That is right, not like SCSI, NVME doesn't apply atomic request completion, so
> request may be completed/freed from both timeout & nvme_cancel_request().
> 
> .teardown_lock still may cover the race with Sagi's patch because teardown
> actually cancels requests in sync style.
In extreme scenarios, the request may be already retry success(rq state change to inflight).
Timeout processing may wrongly stop the queue and abort the request.
teardown_lock serialize the process of timeout and teardown, but do not avoid the race.
It might not be safe.
> 
>> In the teardown process, after quiesced queues delete the timer and cancel the timeout work maybe a better option.
> 
> Seems better solution, given it is aligned with NVME PCI's reset
> handling. nvme_sync_queues() may be called in nvme_tcp_teardown_io_queues() to
> avoid this race.
> 
> 
> Thanks,
> Ming
> 
> .
> 


From chris.friesen at windriver.com  Mon Oct 12 11:49:38 2020
From: chris.friesen at windriver.com (Chris Friesen)
Date: Mon, 12 Oct 2020 09:49:38 -0600
Subject: NVME, isolcpus, and irq affinity
Message-ID: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>

Hi,

I'm not subscribed to the list so please CC me on replies.

I've got a linux system running the RT kernel with threaded irqs.? On 
startup we affine the various irq threads to the housekeeping CPUs, but 
I recently hit a scenario where after some days of uptime we ended up 
with a number of NVME irq threads affined to application cores instead 
(not good when we're trying to run low-latency applications).

Looking at the code, it appears that the NVME driver can in some 
scenarios call nvme_setup_io_queues() after the initial setup and thus 
allocate new IRQ threads at runtime.? It appears that this will then 
call pci_alloc_irq_vectors_affinity(), which seems to determine affinity 
without any regard for things like "isolcpus" or "cset shield".

There seem to be other reports of similar issues:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1831566

Am I worried about nothing, or is there a risk that those irq threads 
would actually need to do real work (which would cause unacceptable 
jitter in my application)?

Assuming I'm reading the code correctly, how does it make sense for the 
NVME driver to affine interrupts to CPUs which have explicitly been 
designated as "isolated"?

Thanks,

Chris




From nmeeramohide at micron.com  Mon Oct 12 12:27:14 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:14 -0500
Subject: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
Message-ID: <20201012162736.65241-1-nmeeramohide@micron.com>

This patch series introduces the mpool object storage media pool driver.
Mpool implements a simple transactional object store on top of block
storage devices.

Mpool was developed for the Heterogeneous-Memory Storage Engine (HSE)
project, which is a high-performance key-value storage engine designed
for SSDs. HSE stores its data exclusively in mpool.

Mpool is readily applicable to other storage systems built on immutable
objects. For example, the many databases that store records in
immutable SSTables organized as an LSM-tree or similar data structure.

We developed mpool for HSE storage, versus using a file system or raw
block device, for several reasons.

A primary motivator was the need for a storage model that maps naturally
to conventional block storage devices, as well as to emerging device
interfaces we plan to support in the future, such as
* NVMe Zoned Namespaces (ZNS)
* NVMe Streams
* Persistent memory accessed via CXL or similar technologies

Another motivator was the need for a storage model that readily supports
multiple classes of storage devices or media in a single storage pool,
such as
* QLC SSDs for storing the bulk of objects, and
* 3DXP SSDs or persistent memory for storing objects requiring
  low-latency access

The mpool object storage model meets these needs. It also provides
other features that benefit storage systems built on immutable objects,
including
* Facilities to memory-map a specified collection of objects into a
  linear address space
* Concurrent access to object data directly and memory-mapped to greatly
  reduce page cache pollution from background operations such as
  LSM-tree compaction
* Proactive eviction of object data from the page cache, based on
  object-level metrics, to avoid excessive memory pressure and its
  associated performance impacts
* High concurrency and short code paths for efficient access to
  low-latency storage devices

HSE takes advantage of all these mpool features to achieve high
throughput with low tail-latencies.

Mpool is implemented as a character device driver where
* /dev/mpoolctl is the control file (minor number 0) supporting mpool
  management ioctls
* /dev/mpool/<mpool-name> are mpool files (minor numbers >0), one per
  mpool, supporting object management ioctls

CLI/UAPI access to /dev/mpoolctl and /dev/mpool/<mpool-name> are
controlled by their UID, GID, and mode bits. To provide a familiar look
and feel, the mpool management model and CLI are intentionally aligned
to those of LVM to the degree practical.

An mpool is created with a block storage device specified for its
required capacity media class, and optionally a second block storage
device specified for its staging media class. We recommend virtual
block devices (such as LVM logical volumes) to aggregate the performance
and capacity of multiple physical block devices, to enable sharing of
physical block devices between mpools (or for other uses), and to
support extending the size of a block device used for an mpool media
class. The libblkid library recognizes mpool formatted block devices as
of util-linux v2.32.

Mpool implements a transactional object store with two simple object
abstractions: mblocks and mlogs.

Mblock objects are containers comprising a linear sequence of bytes that
can be written exactly once, are immutable after writing, and can be
read in whole or in part as needed until deleted. Mblocks in a media
class are currently fixed size, which is configured when an mpool is
created, though the amount of data written to mblocks will differ.

Mlog objects are containers for record logging. Records of arbitrary
size can be appended to an mlog until it is full. Once full, an mlog
must be erased before additional records can be appended. Mlog records
can be read sequentially from the beginning at any time. Mlogs in a
media class are always a multiple of the mblock size for that media
class.

Mblock and mlog writes avoid the page cache. Mblocks are written,
committed, and made immutable before they can be read either directly
(avoiding the page cache) or mmaped. Mlogs are always read and updated
directly (avoiding the page cache) and cannot be mmaped.

Mpool also provides the metadata container (MDC) APIs that clients can
use to simplify storing and maintaining metadata. These MDC APIs are
helper functions built on a pair of mlogs per MDC.

The mpool Wiki contains full details on the
* Management model in the "Configure mpools" section
* Object model in the "Develop mpool Applications" section
* Kernel module architecture in the "Explore mpool Internals" section,
  which provides context for reviewing this patch series

See https://github.com/hse-project/mpool/wiki

The mpool UAPI and kernel module (not the patchset) are available on
GitHub at:

https://github.com/hse-project/mpool

https://github.com/hse-project/mpool-kmod

The HSE key-value storage engine is available on GitHub at:

https://github.com/hse-project/hse

Changes in v2:

- Fixes build errors/warnings reported by bot on ARCH=m68k
Reported-by: kernel test robot <lkp at intel.com>

- Addresses review comments from Randy and Hillf:
  * Updates ioctl-number.rst file with mpool driver's ioctl code
  * Fixes issues in the usage of printk_timed_ratelimit()

- Fixes a readahead issue found by internal testing

Nabeel M Mohamed (22):
  mpool: add utility routines and ioctl definitions
  mpool: add in-memory struct definitions
  mpool: add on-media struct definitions
  mpool: add pool drive component which handles mpool IO using the block
    layer API
  mpool: add space map component which manages free space on mpool
    devices
  mpool: add on-media pack, unpack and upgrade routines
  mpool: add superblock management routines
  mpool: add pool metadata routines to manage object lifecycle and IO
  mpool: add mblock lifecycle management and IO routines
  mpool: add mlog IO utility routines
  mpool: add mlog lifecycle management and IO routines
  mpool: add metadata container or mlog-pair framework
  mpool: add utility routines for mpool lifecycle management
  mpool: add pool metadata routines to create persistent mpools
  mpool: add mpool lifecycle management routines
  mpool: add mpool control plane utility routines
  mpool: add mpool lifecycle management ioctls
  mpool: add object lifecycle management ioctls
  mpool: add support to mmap arbitrary collection of mblocks
  mpool: add support to proactively evict cached mblock data from the
    page-cache
  mpool: add documentation
  mpool: add Kconfig and Makefile

 .../userspace-api/ioctl/ioctl-number.rst      |    3 +-
 drivers/Kconfig                               |    2 +
 drivers/Makefile                              |    1 +
 drivers/mpool/Kconfig                         |   28 +
 drivers/mpool/Makefile                        |   11 +
 drivers/mpool/assert.h                        |   25 +
 drivers/mpool/init.c                          |  126 +
 drivers/mpool/init.h                          |   17 +
 drivers/mpool/mblock.c                        |  432 +++
 drivers/mpool/mblock.h                        |  161 +
 drivers/mpool/mcache.c                        | 1072 +++++++
 drivers/mpool/mcache.h                        |  104 +
 drivers/mpool/mclass.c                        |  103 +
 drivers/mpool/mclass.h                        |  137 +
 drivers/mpool/mdc.c                           |  486 +++
 drivers/mpool/mdc.h                           |  106 +
 drivers/mpool/mlog.c                          | 1667 ++++++++++
 drivers/mpool/mlog.h                          |  212 ++
 drivers/mpool/mlog_utils.c                    | 1352 ++++++++
 drivers/mpool/mlog_utils.h                    |   63 +
 drivers/mpool/mp.c                            | 1086 +++++++
 drivers/mpool/mp.h                            |  231 ++
 drivers/mpool/mpcore.c                        |  987 ++++++
 drivers/mpool/mpcore.h                        |  354 +++
 drivers/mpool/mpctl.c                         | 2747 +++++++++++++++++
 drivers/mpool/mpctl.h                         |   58 +
 drivers/mpool/mpool-locking.rst               |   90 +
 drivers/mpool/mpool_ioctl.h                   |  636 ++++
 drivers/mpool/mpool_printk.h                  |   43 +
 drivers/mpool/omf.c                           | 1316 ++++++++
 drivers/mpool/omf.h                           |  593 ++++
 drivers/mpool/omf_if.h                        |  381 +++
 drivers/mpool/params.h                        |  116 +
 drivers/mpool/pd.c                            |  424 +++
 drivers/mpool/pd.h                            |  202 ++
 drivers/mpool/pmd.c                           | 2046 ++++++++++++
 drivers/mpool/pmd.h                           |  379 +++
 drivers/mpool/pmd_obj.c                       | 1569 ++++++++++
 drivers/mpool/pmd_obj.h                       |  499 +++
 drivers/mpool/reaper.c                        |  686 ++++
 drivers/mpool/reaper.h                        |   71 +
 drivers/mpool/sb.c                            |  625 ++++
 drivers/mpool/sb.h                            |  162 +
 drivers/mpool/smap.c                          | 1031 +++++++
 drivers/mpool/smap.h                          |  334 ++
 drivers/mpool/sysfs.c                         |   48 +
 drivers/mpool/sysfs.h                         |   48 +
 drivers/mpool/upgrade.c                       |  138 +
 drivers/mpool/upgrade.h                       |  128 +
 drivers/mpool/uuid.h                          |   59 +
 50 files changed, 23194 insertions(+), 1 deletion(-)
 create mode 100644 drivers/mpool/Kconfig
 create mode 100644 drivers/mpool/Makefile
 create mode 100644 drivers/mpool/assert.h
 create mode 100644 drivers/mpool/init.c
 create mode 100644 drivers/mpool/init.h
 create mode 100644 drivers/mpool/mblock.c
 create mode 100644 drivers/mpool/mblock.h
 create mode 100644 drivers/mpool/mcache.c
 create mode 100644 drivers/mpool/mcache.h
 create mode 100644 drivers/mpool/mclass.c
 create mode 100644 drivers/mpool/mclass.h
 create mode 100644 drivers/mpool/mdc.c
 create mode 100644 drivers/mpool/mdc.h
 create mode 100644 drivers/mpool/mlog.c
 create mode 100644 drivers/mpool/mlog.h
 create mode 100644 drivers/mpool/mlog_utils.c
 create mode 100644 drivers/mpool/mlog_utils.h
 create mode 100644 drivers/mpool/mp.c
 create mode 100644 drivers/mpool/mp.h
 create mode 100644 drivers/mpool/mpcore.c
 create mode 100644 drivers/mpool/mpcore.h
 create mode 100644 drivers/mpool/mpctl.c
 create mode 100644 drivers/mpool/mpctl.h
 create mode 100644 drivers/mpool/mpool-locking.rst
 create mode 100644 drivers/mpool/mpool_ioctl.h
 create mode 100644 drivers/mpool/mpool_printk.h
 create mode 100644 drivers/mpool/omf.c
 create mode 100644 drivers/mpool/omf.h
 create mode 100644 drivers/mpool/omf_if.h
 create mode 100644 drivers/mpool/params.h
 create mode 100644 drivers/mpool/pd.c
 create mode 100644 drivers/mpool/pd.h
 create mode 100644 drivers/mpool/pmd.c
 create mode 100644 drivers/mpool/pmd.h
 create mode 100644 drivers/mpool/pmd_obj.c
 create mode 100644 drivers/mpool/pmd_obj.h
 create mode 100644 drivers/mpool/reaper.c
 create mode 100644 drivers/mpool/reaper.h
 create mode 100644 drivers/mpool/sb.c
 create mode 100644 drivers/mpool/sb.h
 create mode 100644 drivers/mpool/smap.c
 create mode 100644 drivers/mpool/smap.h
 create mode 100644 drivers/mpool/sysfs.c
 create mode 100644 drivers/mpool/sysfs.h
 create mode 100644 drivers/mpool/upgrade.c
 create mode 100644 drivers/mpool/upgrade.h
 create mode 100644 drivers/mpool/uuid.h

-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:15 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:15 -0500
Subject: [PATCH v2 01/22] mpool: add utility routines and ioctl definitions
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-2-nmeeramohide@micron.com>

This adds structures used by mpool ioctls and utility routines
for logging, UUID management etc.

The mpool ioctls can be categorized as follows:

1. IOCTLs issued to the mpool control device (/dev/mpoolctl)
   - Mpool life cycle management (MPIOC_MP_*)

2. IOCTLs issued to the mpool device (/dev/mpool/<mpool-name>)
   - Mpool parameters (MPIOC_PARAMS_*)
   - Mpool properties (MPIOC_PROP_*)
   - Mpool media class management (MPIOC_MP_MCLASS_*)
   - Device management (MPIOC_DRV_*)
   - Mblock object life cycle management and IO (MPIOC_MB_*)
   - Mlog object life cycle management and IO (MPIOC_MLOG_*)
   - Mblock cache management (MPIOC_VMA_*)

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/assert.h       |  25 ++
 drivers/mpool/init.c         |  22 ++
 drivers/mpool/mpool_ioctl.h  | 636 +++++++++++++++++++++++++++++++++++
 drivers/mpool/mpool_printk.h |  43 +++
 drivers/mpool/uuid.h         |  59 ++++
 5 files changed, 785 insertions(+)
 create mode 100644 drivers/mpool/assert.h
 create mode 100644 drivers/mpool/init.c
 create mode 100644 drivers/mpool/mpool_ioctl.h
 create mode 100644 drivers/mpool/mpool_printk.h
 create mode 100644 drivers/mpool/uuid.h

diff --git a/drivers/mpool/assert.h b/drivers/mpool/assert.h
new file mode 100644
index 000000000000..a2081e71ec93
--- /dev/null
+++ b/drivers/mpool/assert.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_ASSERT_H
+#define MPOOL_ASSERT_H
+
+#include <linux/bug.h>
+
+#ifdef CONFIG_MPOOL_ASSERT
+__cold __noreturn
+static inline void assertfail(const char *expr, const char *file, int line)
+{
+	pr_err("mpool assertion failed: %s in %s:%d\n", expr, file, line);
+	BUG();
+}
+
+#define ASSERT(_expr)   (likely(_expr) ? (void)0 : assertfail(#_expr, __FILE__, __LINE__))
+
+#else
+#define ASSERT(_expr)   (void)(_expr)
+#endif
+
+#endif /* MPOOL_ASSERT_H */
diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
new file mode 100644
index 000000000000..0493fb5b1157
--- /dev/null
+++ b/drivers/mpool/init.c
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#include <linux/module.h>
+
+static __init int mpool_init(void)
+{
+	return 0;
+}
+
+static __exit void mpool_exit(void)
+{
+}
+
+module_init(mpool_init);
+module_exit(mpool_exit);
+
+MODULE_DESCRIPTION("Object Storage Media Pool (mpool)");
+MODULE_AUTHOR("Micron Technology, Inc.");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/mpool/mpool_ioctl.h b/drivers/mpool/mpool_ioctl.h
new file mode 100644
index 000000000000..599da0618a09
--- /dev/null
+++ b/drivers/mpool/mpool_ioctl.h
@@ -0,0 +1,636 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_IOCTL_H
+#define MPOOL_IOCTL_H
+
+#include <linux/uuid.h>
+#include <linux/uio.h>
+
+#ifndef __user
+#define __user
+#endif
+
+/*
+ * Maximum name lengths including NUL terminator.  Note that the maximum
+ * mpool name length is baked into libblkid and may not be changed here.
+ */
+#define MPOOL_NAMESZ_MAX            32
+#define MPOOL_LABELSZ_MAX           64
+#define PD_NAMESZ_MAX               32
+
+#define MPC_DEV_SUBDIR              "mpool"
+#define MPC_DEV_CTLNAME             MPC_DEV_SUBDIR "ctl"
+#define MPC_DEV_CTLPATH             "/dev/" MPC_DEV_CTLNAME
+
+#define MPOOL_LABEL_INVALID         ""
+#define MPOOL_LABEL_DEFAULT         "raw"
+
+#define MPOOL_RA_PAGES_INVALID      U32_MAX
+#define MPOOL_RA_PAGES_MAX          ((128 * 1024) / PAGE_SIZE)
+
+#define MPOOL_MCLASS_INVALID        MP_MED_INVALID
+#define MPOOL_MCLASS_DEFAULT        MP_MED_CAPACITY
+
+#define MPOOL_SPARES_INVALID        U8_MAX
+#define MPOOL_SPARES_DEFAULT        5
+
+#define MPOOL_ROOT_LOG_CAP          (8 * 1024 * 1024)
+
+#define MPOOL_MBSIZE_MB_DEFAULT     32
+
+#define MPOOL_MDCNUM_DEFAULT        16
+
+
+/**
+ * mp_mgmt_flags - Mpool Management Flags
+ * @MP_FLAGS_FORCE:
+ * @MP_PERMIT_META_CONV: permit mpool metadata conversion. That is, allow the
+ *	mpool activate to write back the mpool metadata to the latest version
+ *	used by the binary activating the mpool.
+ * @MP_FLAGS_RESIZE: Resize mpool
+ */
+enum mp_mgmt_flags {
+	MP_FLAGS_FORCE,
+	MP_FLAGS_PERMIT_META_CONV,
+	MP_FLAGS_RESIZE,
+};
+
+/**
+ * mp_media_classp = Media classes
+ *
+ * @MP_MED_STAGING:  Initial data ingest, hot data storage, or similar.
+ * @MP_MED_CAPACITY: Primary data storage, cold data, or similar.
+ */
+enum mp_media_classp {
+	MP_MED_STAGING   = 0,
+	MP_MED_CAPACITY  = 1,
+};
+
+#define MP_MED_BASE        MP_MED_STAGING
+#define MP_MED_NUMBER      (MP_MED_CAPACITY + 1)
+#define MP_MED_INVALID     U8_MAX
+
+/**
+ * struct mpool_devprops -
+ * @pdp_devid:   UUID for drive
+ * @pdp_mclassp: enum mp_media_classp
+ * @pdp_status:  enum pd_status
+ * @pdp_total:   raw capacity of drive
+ * @pdp_avail:   available capacity (total - bad zones) of drive
+ * @pdp_spare:   spare capacity of drive
+ * @pdp_fspare:  free spare capacity of drive
+ * @pdp_usable:  usable capacity of drive
+ * @pdp_fusable: free usable capacity of drive
+ * @pdp_used:    used capacity of drive:
+ */
+struct mpool_devprops {
+	uuid_le    pdp_devid;
+	uint8_t    pdp_mclassp;
+	uint8_t    pdp_status;
+	uint8_t    pdp_rsvd1[6];
+	uint64_t   pdp_total;
+	uint64_t   pdp_avail;
+	uint64_t   pdp_spare;
+	uint64_t   pdp_fspare;
+	uint64_t   pdp_usable;
+	uint64_t   pdp_fusable;
+	uint64_t   pdp_used;
+	uint64_t   pdp_rsvd2;
+};
+
+/**
+ * struct mpool_params -
+ * @mp_poolid:          UUID of mpool
+ * @mp_type:            user-specified identifier
+ * @mp_uid:
+ * @mp_gid:
+ * @mp_mode:
+ * @mp_stat:            overall mpool status (enum mpool_status)
+ * @mp_mdc_captgt:      user MDC capacity
+ * @mp_oidv:            user MDC OIDs
+ * @mp_ra_pages_max:    max VMA map readahead pages
+ * @mp_vma_size_max:    max VMA map size (log2)
+ * @mp_mblocksz:        mblock size by media class (MiB)
+ * @mp_utype:           user-defined type
+ * @mp_label:           user specified label
+ * @mp_name:            mpool name (2x for planned expansion)
+ */
+struct mpool_params {
+	uuid_le     mp_poolid;
+	uid_t       mp_uid;
+	gid_t       mp_gid;
+	mode_t      mp_mode;
+	uint8_t     mp_stat;
+	uint8_t     mp_spare_cap;
+	uint8_t     mp_spare_stg;
+	uint8_t     mp_rsvd0;
+	uint64_t    mp_mdc_captgt;
+	uint64_t    mp_oidv[2];
+	uint32_t    mp_ra_pages_max;
+	uint32_t    mp_vma_size_max;
+	uint32_t    mp_mblocksz[MP_MED_NUMBER];
+	uint16_t    mp_mdc0cap;
+	uint16_t    mp_mdcncap;
+	uint16_t    mp_mdcnum;
+	uint16_t    mp_rsvd1;
+	uint32_t    mp_rsvd2;
+	uint64_t    mp_rsvd3;
+	uint64_t    mp_rsvd4;
+	uuid_le     mp_utype;
+	char        mp_label[MPOOL_LABELSZ_MAX];
+	char        mp_name[MPOOL_NAMESZ_MAX * 2];
+};
+
+/**
+ * struct mpool_usage - in bytes
+ * @mpu_total:   raw capacity for all drives
+ * @mpu_usable:  usable capacity for all drives
+ * @mpu_fusable: free usable capacity for all drives
+ * @mpu_used:    used capacity for all drives; possible for
+ *               used > usable when fusable=0; see smap
+ *               module for details
+ * @mpu_spare:   total spare space
+ * @mpu_fspare:  free spare space
+ *
+ * @mpu_mblock_alen: mblock allocated length
+ * @mpu_mblock_wlen: mblock written length
+ * @mpu_mlog_alen:   mlog allocated length
+ * @mpu_mblock_cnt:  number of active mblocks
+ * @mpu_mlog_cnt:    number of active mlogs
+ */
+struct mpool_usage {
+	uint64_t   mpu_total;
+	uint64_t   mpu_usable;
+	uint64_t   mpu_fusable;
+	uint64_t   mpu_used;
+	uint64_t   mpu_spare;
+	uint64_t   mpu_fspare;
+
+	uint64_t   mpu_alen;
+	uint64_t   mpu_wlen;
+	uint64_t   mpu_mblock_alen;
+	uint64_t   mpu_mblock_wlen;
+	uint64_t   mpu_mlog_alen;
+	uint32_t   mpu_mblock_cnt;
+	uint32_t   mpu_mlog_cnt;
+};
+
+/**
+ * mpool_mclass_xprops -
+ * @mc_devtype: type of devices in the media class
+ *                  (enum pd_devtype)
+ * @mc_mclass: media class (enum mp_media_classp)
+ * @mc_sectorsz: media class (enum mp_media_classp)
+ * @mc_spare: percent spare zones for drives
+ * @mc_uacnt: UNAVAIL status drive count
+ * @mc_zonepg: pages per zone
+ * @mc_features: feature bitmask
+ * @mc_usage: feature bitmask
+ */
+struct mpool_mclass_xprops {
+	uint8_t                    mc_devtype;
+	uint8_t                    mc_mclass;
+	uint8_t                    mc_sectorsz;
+	uint8_t                    mc_rsvd1;
+	uint32_t                   mc_spare;
+	uint16_t                   mc_uacnt;
+	uint16_t                   mc_rsvd2;
+	uint32_t                   mc_zonepg;
+	uint64_t                   mc_features;
+	uint64_t                   mc_rsvd3;
+	struct mpool_usage         mc_usage;
+};
+
+/**
+ * mpool_mclass_props -
+ *
+ * @mc_mblocksz:   mblock size in MiB
+ * @mc_rsvd:       reserved struct field (for future use)
+ * @mc_total:      total space in the media class (mc_usable + mc_spare)
+ * @mc_usable:     usable space in bytes
+ * @mc_used:       bytes allocated from usable space
+ * @mc_spare:      spare space in bytes
+ * @mc_spare_used: bytes allocated from spare space
+ */
+struct mpool_mclass_props {
+	uint32_t   mc_mblocksz;
+	uint32_t   mc_rsvd;
+	uint64_t   mc_total;
+	uint64_t   mc_usable;
+	uint64_t   mc_used;
+	uint64_t   mc_spare;
+	uint64_t   mc_spare_used;
+};
+
+/**
+ * struct mpool_xprops - Extended mpool properties
+ * @ppx_params: mpool configuration parameters
+ * @ppx_drive_spares: percent spare zones for drives in each media class
+ * @ppx_uacnt:  UNAVAIL status drive count in each media class
+ */
+struct mpool_xprops {
+	struct mpool_params     ppx_params;
+	uint8_t                 ppx_rsvd[MP_MED_NUMBER];
+	uint8_t                 ppx_drive_spares[MP_MED_NUMBER];
+	uint16_t                ppx_uacnt[MP_MED_NUMBER];
+	uint32_t                ppx_pd_mclassv[MP_MED_NUMBER];
+	char                    ppx_pd_namev[MP_MED_NUMBER][PD_NAMESZ_MAX];
+};
+
+
+/*
+ * struct mblock_props -
+ *
+ * @mpr_objid:        mblock identifier
+ * @mpr_alloc_cap:    allocated capacity in bytes
+ * @mpr_write_len:    written user-data in bytes
+ * @mpr_optimal_wrsz: optimal write size(in bytes) for all but the last incremental mblock write
+ * @mpr_mclassp:      media class
+ * @mpr_iscommitted:  Is this mblock committed?
+ */
+struct mblock_props {
+	uint64_t                mpr_objid;
+	uint32_t                mpr_alloc_cap;
+	uint32_t                mpr_write_len;
+	uint32_t                mpr_optimal_wrsz;
+	uint32_t                mpr_mclassp; /* enum mp_media_classp */
+	uint8_t                 mpr_iscommitted;
+	uint8_t                 mpr_rsvd1[7];
+	uint64_t                mpr_rsvd2;
+};
+
+struct mblock_props_ex {
+	struct mblock_props     mbx_props;
+	uint8_t                 mbx_zonecnt;      /* zone count per strip */
+	uint8_t                 mbx_rsvd1[7];
+	uint64_t                mbx_rsvd2;
+};
+
+
+/*
+ * enum mlog_open_flags -
+ * @MLOG_OF_COMPACT_SEM: Enforce compaction semantics
+ * @MLOG_OF_SKIP_SER:    Appends and reads are guaranteed to be serialized
+ *                       outside of the mlog API
+ */
+enum mlog_open_flags {
+	MLOG_OF_COMPACT_SEM = 0x1,
+	MLOG_OF_SKIP_SER    = 0x2,
+};
+
+/*
+ * NOTE:
+ * + a value of 0 for targets (*tgt) means no specific target and the
+ *   allocator is free to choose based on media class configuration
+ */
+struct mlog_capacity {
+	uint64_t    lcp_captgt;       /* capacity target for mlog in bytes */
+	uint8_t     lcp_spare;        /* true if alloc mlog from spare space */
+	uint8_t     lcp_rsvd1[7];
+};
+
+/*
+ * struct mlog_props -
+ *
+ * @lpr_uuid:        UUID or mlog magic
+ * @lpr_objid:       mlog identifier
+ * @lpr_alloc_cap:   maximum capacity in bytes
+ * @lpr_gen:         generation no. (user mlogs)
+ * @lpr_mclassp:     media class
+ * @lpr_iscommitted: Is this mlog committed?
+ */
+struct mlog_props {
+	uuid_le     lpr_uuid;
+	uint64_t    lpr_objid;
+	uint64_t    lpr_alloc_cap;
+	uint64_t    lpr_gen;
+	uint8_t     lpr_mclassp;
+	uint8_t     lpr_iscommitted;
+	uint8_t     lpr_rsvd1[6];
+	uint64_t    lpr_rsvd2;
+};
+
+/*
+ * struct mlog_props_ex -
+ *
+ * @lpx_props:
+ * @lpx_totsec:   total number of sectors
+ * @lpx_zonecnt:   zone count per strip
+ * @lpx_state:    mlog layout state
+ * @lpx_secshift: sector shift
+ */
+struct mlog_props_ex {
+	struct mlog_props   lpx_props;
+	uint32_t            lpx_totsec;
+	uint32_t            lpx_zonecnt;
+	uint8_t             lpx_state;
+	uint8_t             lpx_secshift;
+	uint8_t             lpx_rsvd1[6];
+	uint64_t            lpx_rsvd2;
+};
+
+
+/**
+ * enum mdc_open_flags -
+ * @MDC_OF_SKIP_SER: appends and reads are guaranteed to be serialized
+ *                   outside of the MDC API
+ */
+enum mdc_open_flags {
+	MDC_OF_SKIP_SER  = 0x1,
+};
+
+/**
+ * struct mdc_capacity -
+ * @mdt_captgt: capacity target for mlog in bytes
+ * @mpt_spare:  true if alloc MDC from spare space
+ */
+struct mdc_capacity {
+	uint64_t   mdt_captgt;
+	bool       mdt_spare;
+};
+
+/**
+ * struct mdc_props -
+ * @mdc_objid1:
+ * @mdc_objid2:
+ * @mdc_alloc_cap:
+ * @mdc_mclassp:
+ */
+struct mdc_props {
+	uint64_t               mdc_objid1;
+	uint64_t               mdc_objid2;
+	uint64_t               mdc_alloc_cap;
+	enum mp_media_classp   mdc_mclassp;
+};
+
+
+/**
+ * enum mpc_vma_advice -
+ * @MPC_VMA_COLD:
+ * @MPC_VMA_WARM:
+ * @MPC_VMA_HOT:
+ * @MPC_VMA_PINNED:
+ */
+enum mpc_vma_advice {
+	MPC_VMA_COLD = 0,
+	MPC_VMA_WARM,
+	MPC_VMA_HOT,
+	MPC_VMA_PINNED
+};
+
+
+/**
+ * struct pd_znparam - zone parameter arg used in compute/set API functions
+ * @dvb_zonepg:     zone size in PAGE_SIZE units.
+ * @dvb_zonetot:    total number of zones
+ */
+struct pd_znparam {
+	uint32_t   dvb_zonepg;
+	uint32_t   dvb_zonetot;
+	uint64_t   dvb_rsvd1;
+};
+
+#define PD_DEV_ID_LEN              64
+
+/**
+ * struct pd_prop - PD properties
+ * @pdp_didstr:         drive id string (model)
+ * @pdp_devtype:	device type (enum pd_devtype)
+ * @pdp_phys_if:	physical interface of the drive
+ *			Determined by the device discovery.
+ *			(device_phys_if)
+ * @pdp_mclassp:        performance characteristic of the media class
+ *			Determined by the user, not by the device discovery.
+ *			(enum mp_media_classp)
+ * @pdp_cmdopt:         enum pd_cmd_opt. Features of the PD.
+ * @pdp_zparam:	zone parameters
+ * @pdp_discard_granularity: specified by
+ *	/sys/block/<disk>/queue/discard_granularity
+ * @pdp_sectorsz:	Sector size, exponent base 2
+ * @pdp_optiosz:        Optimal IO size
+ * @pdp_devsz:		device size in bytes
+ *
+ * Note: in order to avoid passing enums across user-kernel boundary
+ * declare the following as uint8_t
+ * pdp_devtype: enum pd_devtype
+ * pdp_devstate: enum pd_state
+ * pdp_phys_if: enum device_phys_if
+ * pdp_mclassp: enum mp_media_classp
+ */
+struct pd_prop {
+	char		        pdp_didstr[PD_DEV_ID_LEN];
+	uint8_t                 pdp_devtype;
+	uint8_t                 pdp_devstate;
+	uint8_t                 pdp_phys_if;
+	uint8_t                 pdp_mclassp;
+	bool                    pdp_fua;
+	uint64_t                pdp_cmdopt;
+
+	struct pd_znparam       pdp_zparam;
+	uint32_t                pdp_discard_granularity;
+	uint32_t                pdp_sectorsz;
+	uint32_t                pdp_optiosz;
+	uint32_t                pdp_rsvd2;
+	uint64_t	        pdp_devsz;
+	uint64_t	        pdp_rsvd3;
+};
+
+
+struct mpioc_mpool {
+	struct mpool_params     mp_params;
+	uint32_t                mp_flags;       /* mp_mgmt_flags */
+	uint32_t                mp_dpathc;      /* Count of device paths */
+	uint32_t                mp_dpathssz;    /* Length of mp_dpaths */
+	uint32_t                mp_rsvd1;
+	uint64_t                mp_rsvd2;
+	char __user            *mp_dpaths;      /* Newline separated paths */
+	struct pd_prop __user  *mp_pd_prop;     /* mp_dpathc elements */
+};
+
+/**
+ * struct mpioc_params -
+ * @mps_params;
+ */
+struct mpioc_params {
+	struct mpool_params     mps_params;
+};
+
+struct mpioc_mclass {
+	struct mpool_mclass_xprops __user  *mcl_xprops;
+	uint32_t                            mcl_cnt;
+	uint32_t                            mcl_rsvd1;
+};
+
+struct mpioc_drive {
+	uint32_t	        drv_flags;   /* mp_mgmt_flags */
+	uint32_t	        drv_rsvd1;
+	uint32_t                drv_dpathc;  /* Count of device paths */
+	uint32_t                drv_dpathssz;/* Length of mp_dpaths */
+	struct pd_prop __user  *drv_pd_prop; /* mp_dpathc elements */
+	char __user            *drv_dpaths;  /* Newline separated device paths*/
+};
+
+enum mpioc_list_cmd {
+	MPIOC_LIST_CMD_INVALID     = 0,
+	MPIOC_LIST_CMD_PROP_GET    = 1,       /* Used by mpool get command */
+	MPIOC_LIST_CMD_PROP_LIST   = 2,       /* Used by mpool list command */
+	MPIOC_LIST_CMD_LAST = MPIOC_LIST_CMD_PROP_LIST,
+};
+
+struct mpioc_list {
+	uint32_t        ls_cmd;     /* enum mpioc_list_cmd */
+	uint32_t        ls_listc;
+	void __user    *ls_listv;
+};
+
+struct mpioc_prop {
+	struct mpool_xprops         pr_xprops;
+	struct mpool_usage          pr_usage;
+	struct mpool_mclass_xprops  pr_mcxv[MP_MED_NUMBER];
+	uint32_t                    pr_mcxc;
+	uint32_t                    pr_rsvd1;
+	uint64_t                    pr_rsvd2;
+};
+
+struct mpioc_devprops {
+	char                    dpr_pdname[PD_NAMESZ_MAX];
+	struct mpool_devprops   dpr_devprops;
+};
+
+/**
+ * struct mpioc_mblock:
+ * @mb_objid:   mblock unique ID (permanent)
+ * @mb_offset:  mblock read offset (ephemeral)
+ * @mb_props:
+ * @mb_layout
+ * @mb_spare:
+ * @mb_mclassp: enum mp_media_classp, declared as uint8_t
+ */
+struct mpioc_mblock {
+	uint64_t                mb_objid;
+	int64_t                 mb_offset;
+	struct mblock_props_ex  mb_props;
+	uint8_t                 mb_spare;
+	uint8_t                 mb_mclassp;
+	uint16_t                mb_rsvd1;
+	uint32_t                mb_rsvd2;
+	uint64_t                mb_rsvd3;
+};
+
+struct mpioc_mblock_id {
+	uint64_t    mi_objid;
+};
+
+#define MPIOC_KIOV_MAX          (1024)
+
+struct mpioc_mblock_rw {
+	uint64_t                    mb_objid;
+	int64_t                     mb_offset;
+	uint32_t                    mb_rsvd2;
+	uint16_t                    mb_rsvd3;
+	uint16_t                    mb_iov_cnt;
+	const struct iovec __user  *mb_iov;
+};
+
+struct mpioc_mlog {
+	uint64_t                ml_objid;
+	uint64_t                ml_rsvd;
+	struct mlog_props_ex    ml_props;
+	struct mlog_capacity    ml_cap;
+	uint8_t                 ml_mclassp; /* enum mp_media_classp */
+	uint8_t                 ml_rsvd1[7];
+	uint64_t                ml_rsvd2;
+};
+
+struct mpioc_mlog_id {
+	uint64_t    mi_objid;
+	uint64_t    mi_gen;
+	uint8_t     mi_state;
+	uint8_t     mi_rsvd1[7];
+};
+
+struct mpioc_mlog_io {
+	uint64_t                mi_objid;
+	int64_t                 mi_off;
+	uint8_t                 mi_op;
+	uint8_t                 mi_rsvd1[5];
+	uint16_t                mi_iovc;
+	struct iovec __user    *mi_iov;
+	uint64_t                mi_rsvd2;
+};
+
+struct mpioc_vma {
+	uint32_t            im_advice;
+	uint32_t            im_mbidc;
+	uint64_t __user    *im_mbidv;
+	uint64_t            im_bktsz;
+	int64_t             im_offset;
+	uint64_t            im_len;
+	uint64_t            im_vssp;
+	uint64_t            im_rssp;
+	uint64_t            im_rsvd;
+};
+
+union mpioc_union {
+	struct mpioc_mpool      mpu_mpool;
+	struct mpioc_drive      mpu_drive;
+	struct mpioc_params     mpu_params;
+	struct mpioc_mclass     mpu_mclass;
+	struct mpioc_list       mpu_list;
+	struct mpioc_prop       mpu_prop;
+	struct mpioc_devprops   mpu_devprops;
+	struct mpioc_mlog       mpu_mlog;
+	struct mpioc_mlog_id    mpu_mlog_id;
+	struct mpioc_mlog_io    mpu_mlog_io;
+	struct mpioc_mblock     mpu_mblock;
+	struct mpioc_mblock_id  mpu_mblock_id;
+	struct mpioc_mblock_rw  mpu_mblock_rw;
+	struct mpioc_vma        mpu_vma;
+};
+
+#define MPIOC_MAGIC             ('2')
+
+#define MPIOC_MP_CREATE         _IOWR(MPIOC_MAGIC, 1, struct mpioc_mpool)
+#define MPIOC_MP_DESTROY        _IOW(MPIOC_MAGIC, 2, struct mpioc_mpool)
+#define MPIOC_MP_ACTIVATE       _IOWR(MPIOC_MAGIC, 5, struct mpioc_mpool)
+#define MPIOC_MP_DEACTIVATE     _IOW(MPIOC_MAGIC, 6, struct mpioc_mpool)
+#define MPIOC_MP_RENAME         _IOWR(MPIOC_MAGIC, 7, struct mpioc_mpool)
+
+#define MPIOC_PARAMS_GET        _IOWR(MPIOC_MAGIC, 10, struct mpioc_params)
+#define MPIOC_PARAMS_SET        _IOWR(MPIOC_MAGIC, 11, struct mpioc_params)
+#define MPIOC_MP_MCLASS_GET     _IOWR(MPIOC_MAGIC, 12, struct mpioc_mclass)
+
+#define MPIOC_DRV_ADD           _IOWR(MPIOC_MAGIC, 15, struct mpioc_drive)
+#define MPIOC_DRV_SPARES        _IOWR(MPIOC_MAGIC, 16, struct mpioc_drive)
+
+#define MPIOC_PROP_GET          _IOWR(MPIOC_MAGIC, 20, struct mpioc_list)
+#define MPIOC_PROP_SET          _IOWR(MPIOC_MAGIC, 21, struct mpioc_list)
+#define MPIOC_DEVPROPS_GET      _IOWR(MPIOC_MAGIC, 22, struct mpioc_devprops)
+
+#define MPIOC_MLOG_ALLOC        _IOWR(MPIOC_MAGIC, 30, struct mpioc_mlog)
+#define MPIOC_MLOG_COMMIT       _IOWR(MPIOC_MAGIC, 32, struct mpioc_mlog_id)
+#define MPIOC_MLOG_ABORT        _IOW(MPIOC_MAGIC, 33, struct mpioc_mlog_id)
+#define MPIOC_MLOG_DELETE       _IOW(MPIOC_MAGIC, 34, struct mpioc_mlog_id)
+#define MPIOC_MLOG_FIND         _IOWR(MPIOC_MAGIC, 37, struct mpioc_mlog)
+#define MPIOC_MLOG_READ         _IOW(MPIOC_MAGIC, 40, struct mpioc_mlog_io)
+#define MPIOC_MLOG_WRITE        _IOW(MPIOC_MAGIC, 41, struct mpioc_mlog_io)
+#define MPIOC_MLOG_PROPS        _IOWR(MPIOC_MAGIC, 42, struct mpioc_mlog)
+#define MPIOC_MLOG_ERASE        _IOWR(MPIOC_MAGIC, 43, struct mpioc_mlog_id)
+
+#define MPIOC_MB_ALLOC          _IOWR(MPIOC_MAGIC, 50, struct mpioc_mblock)
+#define MPIOC_MB_ABORT          _IOW(MPIOC_MAGIC, 52, struct mpioc_mblock_id)
+#define MPIOC_MB_COMMIT         _IOW(MPIOC_MAGIC, 53, struct mpioc_mblock_id)
+#define MPIOC_MB_DELETE         _IOW(MPIOC_MAGIC, 54, struct mpioc_mblock_id)
+#define MPIOC_MB_FIND           _IOWR(MPIOC_MAGIC, 56, struct mpioc_mblock)
+#define MPIOC_MB_READ           _IOW(MPIOC_MAGIC, 60, struct mpioc_mblock_rw)
+#define MPIOC_MB_WRITE          _IOW(MPIOC_MAGIC, 61, struct mpioc_mblock_rw)
+
+#define MPIOC_VMA_CREATE        _IOWR(MPIOC_MAGIC, 70, struct mpioc_vma)
+#define MPIOC_VMA_DESTROY       _IOW(MPIOC_MAGIC, 71, struct mpioc_vma)
+#define MPIOC_VMA_PURGE         _IOW(MPIOC_MAGIC, 72, struct mpioc_vma)
+#define MPIOC_VMA_VRSS          _IOWR(MPIOC_MAGIC, 73, struct mpioc_vma)
+
+#endif
diff --git a/drivers/mpool/mpool_printk.h b/drivers/mpool/mpool_printk.h
new file mode 100644
index 000000000000..280a8e064115
--- /dev/null
+++ b/drivers/mpool/mpool_printk.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_PRINTK_H
+#define MPOOL_PRINTK_H
+
+#include <linux/printk.h>
+
+static unsigned long mp_pr_rl_state __maybe_unused;
+
+/* TODO: Use dev_crit(), dev_err(), ... */
+
+#define mp_pr_crit(_fmt, _err, ...)				\
+	pr_crit("%s: " _fmt ": errno %d", __func__, ## __VA_ARGS__, (_err))
+
+#define mp_pr_err(_fmt, _err, ...)				\
+	pr_err("%s: " _fmt ": errno %d", __func__, ## __VA_ARGS__, (_err))
+
+#define mp_pr_warn(_fmt, ...)					\
+	pr_warn("%s: " _fmt, __func__, ## __VA_ARGS__)
+
+#define mp_pr_notice(_fmt, ...)					\
+	pr_notice("%s: " _fmt, __func__, ## __VA_ARGS__)
+
+#define mp_pr_info(_fmt, ...)					\
+	pr_info("%s: " _fmt, __func__, ## __VA_ARGS__)
+
+#define mp_pr_debug(_fmt, _err, ...)				\
+	pr_debug("%s: " _fmt ": errno %d", __func__, ## __VA_ARGS__,  (_err))
+
+
+/* Rate limited version of mp_pr_err(). */
+#define mp_pr_rl(_fmt, _err, ...)				\
+do {								\
+	if (printk_timed_ratelimit(&mp_pr_rl_state, 333)) {	\
+		pr_err("%s: " _fmt ": errno %d",		\
+		       __func__, ## __VA_ARGS__, (_err));	\
+	}							\
+} while (0)
+
+#endif /* MPOOL_PRINTK_H */
diff --git a/drivers/mpool/uuid.h b/drivers/mpool/uuid.h
new file mode 100644
index 000000000000..28e53be68662
--- /dev/null
+++ b/drivers/mpool/uuid.h
@@ -0,0 +1,59 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_UUID_H
+#define MPOOL_UUID_H
+
+#define MPOOL_UUID_SIZE        16
+#define MPOOL_UUID_STRING_LEN  36
+
+#include <linux/kernel.h>
+#include <linux/uuid.h>
+
+struct mpool_uuid {
+	unsigned char uuid[MPOOL_UUID_SIZE];
+};
+
+/* mpool_uuid uses the LE version in the kernel */
+static inline void mpool_generate_uuid(struct mpool_uuid *uuid)
+{
+	generate_random_guid(uuid->uuid);
+}
+
+static inline void mpool_uuid_copy(struct mpool_uuid *u_dst, const struct mpool_uuid *u_src)
+{
+	memcpy(u_dst->uuid, u_src->uuid, MPOOL_UUID_SIZE);
+}
+
+static inline int mpool_uuid_compare(const struct mpool_uuid *uuid1, const struct mpool_uuid *uuid2)
+{
+	return memcmp(uuid1, uuid2, MPOOL_UUID_SIZE);
+}
+
+static inline void mpool_uuid_clear(struct mpool_uuid *uuid)
+{
+	memset(uuid->uuid, 0, MPOOL_UUID_SIZE);
+}
+
+static inline int mpool_uuid_is_null(const struct mpool_uuid *uuid)
+{
+	const struct mpool_uuid zero = { };
+
+	return !memcmp(&zero, uuid, sizeof(zero));
+}
+
+static inline void mpool_unparse_uuid(const struct mpool_uuid *uuid, char *dst)
+{
+	const unsigned char *u = uuid->uuid;
+
+	snprintf(dst, MPOOL_UUID_STRING_LEN + 1,
+		 "%02x%02x%02x%02x-%02x%02x-%02x%02x-%02x%02x-%02x%02x%02x%02x%02x%02x",
+		 u[0], u[1], u[2], u[3],
+		 u[4], u[5], u[6], u[7],
+		 u[8], u[9], u[10], u[11],
+		 u[12], u[13], u[14], u[15]);
+}
+
+#endif /* MPOOL_UUID_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:18 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:18 -0500
Subject: [PATCH v2 04/22] mpool: add pool drive component which handles mpool
 IO using the block layer API
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-5-nmeeramohide@micron.com>

The pool drive (pd) component interfaces with the block layer to
read, write, flush, and discard mpool objects.

The underlying block device(s) are opened during mpool activation
and remains open until deactivated.

Read/write IO to an mpool device is chunked by the PD layer.
Chunking interleaves IO from different streams providing better
QoS.  The size of each chunk (or BIO size) is determined by the
module parameter 'chunk_size_kb'.  All the chunks from a single
r/w request is issued asynchronously to the block layer using
BIO chaining.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/init.c |  31 +++-
 drivers/mpool/init.h |  12 ++
 drivers/mpool/pd.c   | 424 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 466 insertions(+), 1 deletion(-)
 create mode 100644 drivers/mpool/init.h
 create mode 100644 drivers/mpool/pd.c

diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
index 0493fb5b1157..294cf3cbbaa7 100644
--- a/drivers/mpool/init.c
+++ b/drivers/mpool/init.c
@@ -5,13 +5,42 @@
 
 #include <linux/module.h>
 
+#include "mpool_printk.h"
+
+#include "pd.h"
+
+/*
+ * Module params...
+ */
+unsigned int rsvd_bios_max __read_mostly = 16;
+module_param(rsvd_bios_max, uint, 0444);
+MODULE_PARM_DESC(rsvd_bios_max, "max reserved bios in mpool bioset");
+
+int chunk_size_kb __read_mostly = 128;
+module_param(chunk_size_kb, uint, 0644);
+MODULE_PARM_DESC(chunk_size_kb, "Chunk size (in KiB) for device I/O");
+
+static void mpool_exit_impl(void)
+{
+	pd_exit();
+}
+
 static __init int mpool_init(void)
 {
-	return 0;
+	int rc;
+
+	rc = pd_init();
+	if (rc) {
+		mp_pr_err("pd init failed", rc);
+		mpool_exit_impl();
+	}
+
+	return rc;
 }
 
 static __exit void mpool_exit(void)
 {
+	mpool_exit_impl();
 }
 
 module_init(mpool_init);
diff --git a/drivers/mpool/init.h b/drivers/mpool/init.h
new file mode 100644
index 000000000000..e02a9672e727
--- /dev/null
+++ b/drivers/mpool/init.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_INIT_H
+#define MPOOL_INIT_H
+
+extern unsigned int rsvd_bios_max;
+extern int chunk_size_kb;
+
+#endif /* MPOOL_INIT_H */
diff --git a/drivers/mpool/pd.c b/drivers/mpool/pd.c
new file mode 100644
index 000000000000..f13c7704efad
--- /dev/null
+++ b/drivers/mpool/pd.c
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * Pool drive module with backing block devices.
+ *
+ * Defines functions for probing, reading, and writing drives in an mpool.
+ * IO is done using kerel BIO facilities.
+ */
+
+#define _LARGEFILE64_SOURCE
+
+#include <linux/fs.h>
+#include <linux/blkdev.h>
+#include <linux/blk_types.h>
+
+#include "mpool_printk.h"
+#include "assert.h"
+
+#include "init.h"
+#include "omf_if.h"
+#include "pd.h"
+
+#ifndef SECTOR_SHIFT
+#define SECTOR_SHIFT   9
+#endif
+
+static struct bio_set mpool_bioset;
+
+static const fmode_t    pd_bio_fmode = FMODE_READ | FMODE_WRITE | FMODE_EXCL;
+static char            *pd_bio_holder = "mpool";
+
+int pd_dev_open(const char *path, struct pd_dev_parm *dparm, struct pd_prop *pd_prop)
+{
+	struct block_device *bdev;
+
+	bdev = blkdev_get_by_path(path, pd_bio_fmode, pd_bio_holder);
+	if (IS_ERR(bdev))
+		return PTR_ERR(bdev);
+
+	dparm->dpr_dev_private = bdev;
+	dparm->dpr_prop = *pd_prop;
+
+	if ((pd_prop->pdp_devtype != PD_DEV_TYPE_BLOCK_STD) &&
+	    (pd_prop->pdp_devtype != PD_DEV_TYPE_BLOCK_NVDIMM)) {
+		int rc = -EINVAL;
+
+		mp_pr_err("unsupported PD type %d", rc, pd_prop->pdp_devtype);
+		return rc;
+	}
+
+	return 0;
+}
+
+int pd_dev_close(struct pd_dev_parm *dparm)
+{
+	struct block_device *bdev = dparm->dpr_dev_private;
+
+	if (bdev) {
+		dparm->dpr_dev_private = NULL;
+		sync_blockdev(bdev);
+		invalidate_bdev(bdev);
+		blkdev_put(bdev, pd_bio_fmode);
+	}
+
+	return bdev ? 0 : -EINVAL;
+}
+
+int pd_dev_flush(struct pd_dev_parm *dparm)
+{
+	struct block_device *bdev;
+	int rc;
+
+	bdev = dparm->dpr_dev_private;
+	if (!bdev) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s not registered", rc, dparm->dpr_name);
+		return rc;
+	}
+
+	rc = blkdev_issue_flush(bdev, GFP_NOIO);
+	if (rc)
+		mp_pr_err("bdev %s, flush failed", rc, dparm->dpr_name);
+
+	return rc;
+}
+
+/**
+ * pd_bio_discard() - issue discard command to erase a byte-aligned region
+ * @dparm:
+ * @off:
+ * @len:
+ */
+static int pd_bio_discard(struct pd_dev_parm *dparm, loff_t off, size_t len)
+{
+	struct block_device *bdev;
+	int rc;
+
+	bdev = dparm->dpr_dev_private;
+	if (!bdev) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s not registered", rc, dparm->dpr_name);
+		return rc;
+	}
+
+	/* Validate I/O offset is sector-aligned */
+	if (off & PD_SECTORMASK(&dparm->dpr_prop)) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s, offset 0x%lx not multiple of sec size %u",
+			  rc, dparm->dpr_name, (ulong)off, (1 << PD_SECTORSZ(&dparm->dpr_prop)));
+		return rc;
+	}
+
+	if (off > PD_LEN(&dparm->dpr_prop)) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s, offset 0x%lx past end 0x%lx",
+			  rc, dparm->dpr_name, (ulong)off, (ulong)PD_LEN(&dparm->dpr_prop));
+		return rc;
+	}
+
+	rc = blkdev_issue_discard(bdev, off >> SECTOR_SHIFT, len >> SECTOR_SHIFT, GFP_NOIO, 0);
+	if (rc)
+		mp_pr_err("bdev %s, offset 0x%lx len 0x%lx, discard faiure",
+			  rc, dparm->dpr_name, (ulong)off, (ulong)len);
+
+	return rc;
+}
+
+/**
+ * pd_zone_erase() - issue write-zeros or discard commands to erase PD
+ * @dparm:
+ * @zaddr:
+ * @zonecnt:
+ * @flag:
+ * @afp:
+ */
+int pd_zone_erase(struct pd_dev_parm *dparm, u64 zaddr, u32 zonecnt, bool reads_erased)
+{
+	int rc = 0;
+	u64 cmdopt;
+
+	/* Validate args against zone param */
+	if (zaddr >= dparm->dpr_zonetot)
+		return -EINVAL;
+
+	if (zonecnt == 0)
+		zonecnt = dparm->dpr_zonetot - zaddr;
+
+	if (zonecnt > (dparm->dpr_zonetot - zaddr))
+		return -EINVAL;
+
+	if (zonecnt == 0)
+		return 0;
+
+	/*
+	 * When both DIF and SED are enabled, read from a discared block
+	 * would fail, so we can't discard blocks if both DIF and SED are
+	 * enabled AND we need to read blocks after erase.
+	 */
+	cmdopt = dparm->dpr_cmdopt;
+	if ((cmdopt & PD_CMD_DISCARD) &&
+	    !(reads_erased && (cmdopt & PD_CMD_DIF_ENABLED) && (cmdopt & PD_CMD_SED_ENABLED))) {
+		size_t zlen;
+
+		zlen = dparm->dpr_zonepg << PAGE_SHIFT;
+		rc = pd_bio_discard(dparm, zaddr * zlen, zonecnt * zlen);
+	}
+
+	return rc;
+}
+
+static void pd_bio_init(struct bio *bio, struct block_device *bdev, int rw, loff_t off, int flags)
+{
+	bio_set_op_attrs(bio, rw, flags);
+	bio->bi_iter.bi_sector = off >> SECTOR_SHIFT;
+	bio_set_dev(bio, bdev);
+}
+
+static struct bio *pd_bio_chain(struct bio *target, unsigned int nr_pages, gfp_t gfp)
+{
+	struct bio *new;
+
+	new = bio_alloc_bioset(gfp, nr_pages, &mpool_bioset);
+
+	if (!target)
+		return new;
+
+	if (new) {
+		bio_chain(target, new);
+		submit_bio(target);
+	} else {
+		submit_bio_wait(target);
+		bio_put(target);
+	}
+
+	return new;
+}
+
+/**
+ * pd_bio_rw() -
+ * @dparm:
+ * @iov:
+ * @iovcnt:
+ * @off: offset in bytes on disk
+ * @rw:
+ * @opflags:
+ *
+ * pd_bio_rw() expects a list of kvecs wherein each base ptr is sector
+ * aligned and each length is multiple of sectors.
+ *
+ * If the IO is bigger than 1MiB (BIO_MAX_PAGES pages) or chunk_size_kb,
+ * it is split in several IOs.
+ */
+static int pd_bio_rw(struct pd_dev_parm *dparm, const struct kvec *iov,
+		     int iovcnt, loff_t off, int rw, int opflags)
+{
+	struct block_device *bdev;
+	struct page *page;
+	struct bio *bio;
+	uintptr_t iov_base;
+	u64 sector_mask;
+	u32 tot_pages, tot_len, len, iov_len, left, iolimit;
+	int i, cc, rc = 0;
+
+	if (iovcnt < 1)
+		return 0;
+
+	bdev = dparm->dpr_dev_private;
+	if (!bdev) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s not registered", rc, dparm->dpr_name);
+		return rc;
+	}
+
+	sector_mask = PD_SECTORMASK(&dparm->dpr_prop);
+	if (off & sector_mask) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s, %s offset 0x%lx not multiple of sector size %u",
+			  rc, dparm->dpr_name, (rw == REQ_OP_READ) ? "read" : "write",
+			  (ulong)off, (1 << PD_SECTORSZ(&dparm->dpr_prop)));
+		return rc;
+	}
+
+	if (off > PD_LEN(&dparm->dpr_prop)) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s, %s offset 0x%lx past device end 0x%lx",
+			  rc, dparm->dpr_name, (rw == REQ_OP_READ) ? "read" : "write",
+			  (ulong)off, (ulong)PD_LEN(&dparm->dpr_prop));
+		return rc;
+	}
+
+	tot_pages = 0;
+	tot_len = 0;
+	for (i = 0; i < iovcnt; i++) {
+		if (!PAGE_ALIGNED((uintptr_t)iov[i].iov_base) || (iov[i].iov_len & sector_mask)) {
+			rc = -EINVAL;
+			mp_pr_err("bdev %s, %s off 0x%lx, misaligned kvec, base 0x%lx, len 0x%lx",
+				  rc, dparm->dpr_name, (rw == REQ_OP_READ) ? "read" : "write",
+				  (ulong)off, (ulong)iov[i].iov_base, (ulong)iov[i].iov_len);
+			return rc;
+		}
+
+		iov_len = iov[i].iov_len;
+		tot_len += iov_len;
+		while (iov_len > 0) {
+			len = min_t(size_t, PAGE_SIZE, iov_len);
+			iov_len -= len;
+			tot_pages++;
+		}
+	}
+
+	if (off + tot_len > PD_LEN(&dparm->dpr_prop)) {
+		rc = -EINVAL;
+		mp_pr_err("bdev %s, %s I/O end past device end 0x%lx, 0x%lx:0x%x",
+			  rc, dparm->dpr_name, (rw == REQ_OP_READ) ? "read" : "write",
+			  (ulong)PD_LEN(&dparm->dpr_prop), (ulong)off, tot_len);
+		return rc;
+	}
+
+	if (tot_len == 0)
+		return 0;
+
+	/* IO size for each bio is determined by the chunk size. */
+	iolimit = chunk_size_kb >> (PAGE_SHIFT - 10);
+	iolimit = clamp_t(u32, iolimit, 32, BIO_MAX_PAGES);
+
+	left = 0;
+	bio = NULL;
+
+	for (i = 0; i < iovcnt; i++) {
+		iov_base = (uintptr_t)iov[i].iov_base;
+		iov_len = iov[i].iov_len;
+
+		while (iov_len > 0) {
+			if (left == 0) {
+				left = min_t(size_t, tot_pages, iolimit);
+
+				bio = pd_bio_chain(bio, left, GFP_NOIO);
+				if (!bio)
+					return -ENOMEM;
+
+				pd_bio_init(bio, bdev, rw, off, opflags);
+			}
+
+			len = min_t(size_t, PAGE_SIZE, iov_len);
+			page = virt_to_page(iov_base);
+			cc = -1;
+
+			if (page)
+				cc = bio_add_page(bio, page, len, 0);
+
+			if (cc != len) {
+				if (cc == 0 && bio->bi_vcnt > 0) {
+					left = 0;
+					continue;
+				}
+
+				bio_io_error(bio);
+				bio_put(bio);
+				return -ENOTRECOVERABLE;
+			}
+
+			iov_len -= len;
+			iov_base += len;
+			off += len;
+			left--;
+			tot_pages--;
+		}
+	}
+
+	ASSERT(bio);
+	ASSERT(tot_pages == 0);
+
+	rc = submit_bio_wait(bio);
+	bio_put(bio);
+
+	return rc;
+}
+
+int pd_zone_pwritev(struct pd_dev_parm *dparm, const struct kvec *iov,
+		    int iovcnt, u64 zaddr, loff_t boff, int opflags)
+{
+	loff_t woff;
+
+	woff = ((u64)dparm->dpr_zonepg << PAGE_SHIFT) * zaddr + boff;
+
+	return pd_bio_rw(dparm, iov, iovcnt, woff, REQ_OP_WRITE, opflags);
+}
+
+int pd_zone_pwritev_sync(struct pd_dev_parm *dparm, const struct kvec *iov,
+			 int iovcnt, u64 zaddr, loff_t boff)
+{
+	struct block_device *bdev;
+	int rc;
+
+	rc = pd_zone_pwritev(dparm, iov, iovcnt, zaddr, boff, REQ_FUA);
+	if (rc)
+		return rc;
+
+	/*
+	 * This sync & invalidate bdev ensures that the data written from the
+	 * kernel is immediately visible to the user-space.
+	 */
+	bdev = dparm->dpr_dev_private;
+	if (bdev) {
+		sync_blockdev(bdev);
+		invalidate_bdev(bdev);
+	}
+
+	return 0;
+}
+
+int pd_zone_preadv(struct pd_dev_parm *dparm, const struct kvec *iov,
+		   int iovcnt, u64 zaddr, loff_t boff)
+{
+	loff_t roff;
+
+	roff = ((u64)dparm->dpr_zonepg << PAGE_SHIFT) * zaddr + boff;
+
+	return pd_bio_rw(dparm, iov, iovcnt, roff, REQ_OP_READ, 0);
+}
+
+void pd_dev_set_unavail(struct pd_dev_parm *dparm, struct omf_devparm_descriptor *omf_devparm)
+{
+	struct pd_prop *pd_prop = &(dparm->dpr_prop);
+
+	/*
+	 * Fill in dparm for unavailable drive; sets zone parm and other
+	 * PD properties we keep in metadata; no ops vector because we need
+	 * the device to be available to know it (the discovery gets it).
+	 */
+	strncpy(dparm->dpr_prop.pdp_didstr, PD_DEV_ID_PDUNAVAILABLE, PD_DEV_ID_LEN);
+	pd_prop->pdp_devstate = PD_DEV_STATE_UNAVAIL;
+	pd_prop->pdp_cmdopt = PD_CMD_NONE;
+
+	pd_prop->pdp_zparam.dvb_zonepg  = omf_devparm->odp_zonepg;
+	pd_prop->pdp_zparam.dvb_zonetot = omf_devparm->odp_zonetot;
+	pd_prop->pdp_mclassp  = omf_devparm->odp_mclassp;
+	pd_prop->pdp_phys_if  = 0;
+	pd_prop->pdp_sectorsz = omf_devparm->odp_sectorsz;
+	pd_prop->pdp_devsz    = omf_devparm->odp_devsz;
+}
+
+
+int pd_init(void)
+{
+	int rc;
+
+	chunk_size_kb = clamp_t(uint, chunk_size_kb, 128, 1024);
+
+	rsvd_bios_max = clamp_t(uint, rsvd_bios_max, 1, 1024);
+
+	rc = bioset_init(&mpool_bioset, rsvd_bios_max, 0, BIOSET_NEED_BVECS);
+	if (rc)
+		mp_pr_err("mpool bioset init failed", rc);
+
+	return rc;
+}
+
+void pd_exit(void)
+{
+	bioset_exit(&mpool_bioset);
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:17 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:17 -0500
Subject: [PATCH v2 03/22] mpool: add on-media struct definitions
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-4-nmeeramohide@micron.com>

This adds headers containing the following on-media formats:
- Mpool superblock
- Object management records: create, update, delete, and erase
- Mpool configuration record
- Media class config and spare record
- OID checkpoint and version record
- Mlog page header and framing records

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/omf.h     | 593 ++++++++++++++++++++++++++++++++++++++++
 drivers/mpool/omf_if.h  | 381 ++++++++++++++++++++++++++
 drivers/mpool/upgrade.h | 128 +++++++++
 3 files changed, 1102 insertions(+)
 create mode 100644 drivers/mpool/omf.h
 create mode 100644 drivers/mpool/omf_if.h
 create mode 100644 drivers/mpool/upgrade.h

diff --git a/drivers/mpool/omf.h b/drivers/mpool/omf.h
new file mode 100644
index 000000000000..c750573720dd
--- /dev/null
+++ b/drivers/mpool/omf.h
@@ -0,0 +1,593 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * Pool on-drive format (omf) module.
+ *
+ * Defines:
+ * + on-drive format for mpool superblocks
+ * + on-drive formats for mlogs, mblocks, and metadata containers (mdc)
+ * + utility functions for working with these on-drive formats
+ * That includes structures and enums used by the on-drive format.
+ *
+ * All mpool metadata is versioned and stored on media in little-endian format.
+ *
+ * Naming conventions:
+ * -------------------
+ * The name of the structures ends with _omf
+ * The name of the structure members start with a "p" that means "packed".
+ */
+
+#ifndef MPOOL_OMF_H
+#define MPOOL_OMF_H
+
+#include <linux/bug.h>
+#include <asm/byteorder.h>
+
+/*
+ * The following two macros exist solely to enable the OMF_SETGET macros to
+ * work on 8 bit members as well as 16, 32 and 64 bit members.
+ */
+#define le8_to_cpu(x)  (x)
+#define cpu_to_le8(x)  (x)
+
+
+/* Helper macro to define set/get methods for 8, 16, 32 or 64 bit scalar OMF struct members. */
+#define OMF_SETGET(type, member, bits) \
+	OMF_SETGET2(type, member, bits, member)
+
+#define OMF_SETGET2(type, member, bits, name)				\
+	static __always_inline u##bits omf_##name(const type * s)	\
+	{								\
+		BUILD_BUG_ON(sizeof(((type *)0)->member)*8 != (bits));	\
+		return le##bits##_to_cpu(s->member);			\
+	}								\
+	static __always_inline void omf_set_##name(type *s, u##bits val)\
+	{								\
+		s->member = cpu_to_le##bits(val);			\
+	}
+
+/* Helper macro to define set/get methods for character strings embedded in OMF structures. */
+#define OMF_SETGET_CHBUF(type, member) \
+	OMF_SETGET_CHBUF2(type, member, member)
+
+#define OMF_SETGET_CHBUF2(type, member, name)				\
+	static inline void omf_set_##name(type *s, const void *p, size_t plen) \
+	{								\
+		size_t len = sizeof(((type *)0)->member);		\
+		memcpy(s->member, p, len < plen ? len : plen);		\
+	}								\
+	static inline void omf_##name(const type *s, void *p, size_t plen)\
+	{								\
+		size_t len = sizeof(((type *)0)->member);		\
+		memcpy(p, s->member, len < plen ? len : plen);		\
+	}
+
+
+/* MPOOL_NAMESZ_MAX should match OMF_MPOOL_NAME_LEN */
+#define OMF_MPOOL_NAME_LEN 32
+
+/* MPOOL_UUID_SIZE should match OMF_UUID_PACKLEN */
+#define OMF_UUID_PACKLEN 16
+
+/**
+ * enum mc_features_omf - Drive features that participate in media classes
+ *	                  definition. These values are ored in a 64 bits field.
+ */
+enum mc_features_omf {
+	OMF_MC_FEAT_MLOG_TGT   = 0x1,
+	OMF_MC_FEAT_MBLOCK_TGT = 0x2,
+	OMF_MC_FEAT_CHECKSUM   = 0x4,
+};
+
+
+/**
+ * enum devtype_omf -
+ * @OMF_PD_DEV_TYPE_BLOCK_STREAM: Block device implementing streams.
+ * @OMF_PD_DEV_TYPE_BLOCK_STD:    Standard (non-streams) device (SSD, HDD).
+ * @OMF_PD_DEV_TYPE_FILE:	  File in user space for UT.
+ * @OMF_PD_DEV_TYPE_MEM:	  Memory semantic device. Such as NVDIMM
+ *                                direct access (raw or dax mode).
+ * @OMF_PD_DEV_TYPE_ZONE:	  zone-like device, such as open channel SSD
+ *				  (OC-SSD) and SMR HDD (using ZBC/ZAC).
+ * @OMF_PD_DEV_TYPE_BLOCK_NVDIMM: Standard (non-streams) NVDIMM in sector mode.
+ */
+enum devtype_omf {
+	OMF_PD_DEV_TYPE_BLOCK_STREAM	= 1,
+	OMF_PD_DEV_TYPE_BLOCK_STD	= 2,
+	OMF_PD_DEV_TYPE_FILE		= 3,
+	OMF_PD_DEV_TYPE_MEM		= 4,
+	OMF_PD_DEV_TYPE_ZONE		= 5,
+	OMF_PD_DEV_TYPE_BLOCK_NVDIMM    = 6,
+};
+
+
+/**
+ * struct layout_descriptor_omf - Layout descriptor version 1.
+ * @pol_zcnt: number of zones
+ * @pol_zaddr: zone start addr
+ *
+ * Introduced with binary version 1.0.0.0.
+ * "pol_" = packed omf layout
+ */
+struct layout_descriptor_omf {
+	__le32 pol_zcnt;
+	__le64 pol_zaddr;
+} __packed;
+
+/* Define set/get methods for layout_descriptor_omf */
+OMF_SETGET(struct layout_descriptor_omf, pol_zcnt, 32)
+OMF_SETGET(struct layout_descriptor_omf, pol_zaddr, 64)
+#define OMF_LAYOUT_DESC_PACKLEN (sizeof(struct layout_descriptor_omf))
+
+
+/**
+ * struct devparm descriptor_omf - packed omf devparm descriptor
+ * @podp_devid:    UUID for drive
+ * @podp_zonetot:  total number of zones
+ * @podp_devsz:    size of partition in bytes
+ * @podp_features: Features, ored bits of enum mc_features_omf
+ * @podp_mclassp:   enum mp_media_classp
+ * @podp_devtype:   PD type (enum devtype_omf)
+ * @podp_sectorsz:  2^podp_sectorsz = sector size
+ * @podp_zonepg:    zone size in number of zone pages
+ *
+ * The fields mclassp, devtype, sectosz, and zonepg uniquely identify the media class of the PD.
+ * All drives in a media class must have the same values in these fields.
+ */
+struct devparm_descriptor_omf {
+	u8     podp_mclassp;
+	u8     podp_devtype;
+	u8     podp_sectorsz;
+	u8     podp_devid[OMF_UUID_PACKLEN];
+	u8     podp_pad[5];
+	__le32 podp_zonepg;
+	__le32 podp_zonetot;
+	__le64 podp_devsz;
+	__le64 podp_features;
+} __packed;
+
+/* Define set/get methods for devparm_descriptor_omf */
+OMF_SETGET(struct devparm_descriptor_omf, podp_mclassp, 8)
+OMF_SETGET(struct devparm_descriptor_omf, podp_devtype, 8)
+OMF_SETGET(struct devparm_descriptor_omf, podp_sectorsz, 8)
+OMF_SETGET_CHBUF(struct devparm_descriptor_omf, podp_devid)
+OMF_SETGET(struct devparm_descriptor_omf, podp_zonepg, 32)
+OMF_SETGET(struct devparm_descriptor_omf, podp_zonetot, 32)
+OMF_SETGET(struct devparm_descriptor_omf, podp_devsz, 64)
+OMF_SETGET(struct devparm_descriptor_omf, podp_features, 64)
+#define OMF_DEVPARM_DESC_PACKLEN (sizeof(struct devparm_descriptor_omf))
+
+
+/*
+ * mlog structure:
+ * + An mlog comprises a consecutive sequence of log blocks,
+ *   where each log block is a single page within a zone
+ * + A log block comprises a header and a consecutive sequence of records
+ * + A record is a typed blob
+ *
+ * Log block headers must be versioned. Log block records do not
+ * require version numbers because they are typed and new types can
+ * always be added.
+ */
+
+/*
+ * Log block format -- version 1
+ *
+ * log block := header record+ eolb? trailer?
+ *
+ * header := struct omf_logblock_header where vers=2
+ *
+ * record := lrd byte*
+ *
+ * lrd := struct omf_logrec_descriptor with value
+ *   (<record length>, <chunk length>, enum logrec_type_omf value)
+ *
+ * eolb (end of log block marker) := struct omf_logrec_descriptor with value
+ *   (0, 0, enum logrec_type_omf.EOLB/0)
+ *
+ * trailer := zero bytes from end of last log block record to end of log block
+ *
+ * OMF_LOGREC_CEND must be the max. value for this enum.
+ */
+
+/**
+ * enum logrec_type_omf -
+ * @OMF_LOGREC_EOLB:      end of log block marker (start of trailer)
+ * @OMF_LOGREC_DATAFULL:  data record; contains all specified data
+ * @OMF_LOGREC_DATAFIRST: data record; contains first part of specified data
+ * @OMF_LOGREC_DATAMID:   data record; contains interior part of data
+ * @OMF_LOGREC_DATALAST:  data record; contains final part of specified data
+ * @OMF_LOGREC_CSTART:    compaction start marker
+ * @OMF_LOGREC_CEND:      compaction end marker
+ *
+ * A log record type of 0 signifies EOLB. This is really the start of the
+ * trailer but this simplifies parsing for partially filled log blocks.
+ * DATAFIRST, -MID, -LAST types are used for chunking logical data records.
+ */
+enum logrec_type_omf {
+	OMF_LOGREC_EOLB      = 0,
+	OMF_LOGREC_DATAFULL  = 1,
+	OMF_LOGREC_DATAFIRST = 2,
+	OMF_LOGREC_DATAMID   = 3,
+	OMF_LOGREC_DATALAST  = 4,
+	OMF_LOGREC_CSTART    = 5,
+	OMF_LOGREC_CEND      = 6,
+};
+
+
+/**
+ * struct logrec_descriptor_omf -packed omf logrec descriptor
+ * @polr_tlen:  logical length of data record (all chunks)
+ * @polr_rlen:  length of data chunk in this log record
+ * @polr_rtype: enum logrec_type_omf value
+ */
+struct logrec_descriptor_omf {
+	__le32 polr_tlen;
+	__le16 polr_rlen;
+	u8     polr_rtype;
+	u8     polr_pad;
+} __packed;
+
+/* Define set/get methods for logrec_descriptor_omf */
+OMF_SETGET(struct logrec_descriptor_omf, polr_tlen, 32)
+OMF_SETGET(struct logrec_descriptor_omf, polr_rlen, 16)
+OMF_SETGET(struct logrec_descriptor_omf, polr_rtype, 8)
+#define OMF_LOGREC_DESC_PACKLEN (sizeof(struct logrec_descriptor_omf))
+#define OMF_LOGREC_DESC_RLENMAX 65535
+
+
+#define OMF_LOGBLOCK_VERS    1
+
+/**
+ * struct logblock_header_omf - packed omf logblock header for all versions
+ * @polh_vers:    log block hdr version, offset 0 in all vers
+ * @polh_magic:   unique magic per mlog
+ * @polh_pfsetid: flush set ID of the previous log block
+ * @polh_cfsetid: flush set ID this log block belongs to
+ * @polh_gen:     generation number
+ */
+struct logblock_header_omf {
+	__le16 polh_vers;
+	u8     polh_magic[OMF_UUID_PACKLEN];
+	u8     polh_pad[6];
+	__le32 polh_pfsetid;
+	__le32 polh_cfsetid;
+	__le64 polh_gen;
+} __packed;
+
+/* Define set/get methods for logblock_header_omf */
+OMF_SETGET(struct logblock_header_omf, polh_vers, 16)
+OMF_SETGET_CHBUF(struct logblock_header_omf, polh_magic)
+OMF_SETGET(struct logblock_header_omf, polh_pfsetid, 32)
+OMF_SETGET(struct logblock_header_omf, polh_cfsetid, 32)
+OMF_SETGET(struct logblock_header_omf, polh_gen, 64)
+/* On-media log block header length */
+#define OMF_LOGBLOCK_HDR_PACKLEN (sizeof(struct logblock_header_omf))
+
+
+/*
+ * Metadata container (mdc) mlog data record formats.
+ *
+ * NOTE: mdc records are typed and as such do not need a version number as new
+ * types can always be added as required.
+ */
+/**
+ * enum mdcrec_type_omf -
+ * @OMF_MDR_UNDEF:   undefined; should never occur
+ * @OMF_MDR_OCREATE:  object create
+ * @OMF_MDR_OUPDATE:  object update
+ * @OMF_MDR_ODELETE:  object delete
+ * @OMF_MDR_OIDCKPT:  object id checkpoint
+ * @OMF_MDR_OERASE:   object erase, also log mlog gen number
+ * @OMF_MDR_MCCONFIG: media class config
+ * @OMF_MDR_MCSPARE:  media class spare zones set
+ * @OMF_MDR_VERSION:  MDC content version.
+ * @OMF_MDR_MPCONFIG:  mpool config record
+ */
+enum mdcrec_type_omf {
+	OMF_MDR_UNDEF       = 0,
+	OMF_MDR_OCREATE     = 1,
+	OMF_MDR_OUPDATE     = 2,
+	OMF_MDR_ODELETE     = 3,
+	OMF_MDR_OIDCKPT     = 4,
+	OMF_MDR_OERASE      = 5,
+	OMF_MDR_MCCONFIG    = 6,
+	OMF_MDR_MCSPARE     = 7,
+	OMF_MDR_VERSION     = 8,
+	OMF_MDR_MPCONFIG    = 9,
+	OMF_MDR_MAX         = 10,
+};
+
+/**
+ * struct mdcver_omf - packed mdc version, version of an mpool MDC content.
+ * @pv_rtype:      OMF_MDR_VERSION
+ * @pv_mdcv_major: to compare with MAJOR in binary version.
+ * @pv_mdcv_minor: to compare with MINOR in binary version.
+ * @pv_mdcv_patch: to compare with PATCH in binary version.
+ * @pv_mdcv_dev:   used during development cycle when the above
+ *                 numbers don't change.
+ *
+ * This is not the version of the message framing used for the MDC. This is
+ * version of the binary that introduced that version of the MDC content.
+ */
+struct mdcver_omf {
+	u8     pv_rtype;
+	u8     pv_pad;
+	__le16 pv_mdcv_major;
+	__le16 pv_mdcv_minor;
+	__le16 pv_mdcv_patch;
+	__le16 pv_mdcv_dev;
+} __packed;
+
+/* Define set/get methods for mdcrec_version_omf */
+OMF_SETGET(struct mdcver_omf, pv_rtype, 8)
+OMF_SETGET(struct mdcver_omf, pv_mdcv_major, 16)
+OMF_SETGET(struct mdcver_omf, pv_mdcv_minor, 16)
+OMF_SETGET(struct mdcver_omf, pv_mdcv_patch, 16)
+OMF_SETGET(struct mdcver_omf, pv_mdcv_dev,   16)
+
+
+/**
+ * struct mdcrec_data_odelete_omf - packed data record odelete
+ * @pdro_rtype: mdrec_type_omf:OMF_MDR_ODELETE, OMF_MDR_OIDCKPT
+ * @pdro_objid: object identifier
+ */
+struct mdcrec_data_odelete_omf {
+	u8     pdro_rtype;
+	u8     pdro_pad[7];
+	__le64 pdro_objid;
+} __packed;
+
+/* Define set/get methods for  mdcrec_data_odelete_omf */
+OMF_SETGET(struct  mdcrec_data_odelete_omf, pdro_rtype, 8)
+OMF_SETGET(struct  mdcrec_data_odelete_omf, pdro_objid, 64)
+
+
+/**
+ * struct mdcrec_data_oerase_omf - packed data record oerase
+ * @pdrt_rtype: mdrec_type_omf: OMF_MDR_OERASE
+ * @pdrt_objid: object identifier
+ * @pdrt_gen:   object generation number
+ */
+struct mdcrec_data_oerase_omf {
+	u8     pdrt_rtype;
+	u8     pdrt_pad[7];
+	__le64 pdrt_objid;
+	__le64 pdrt_gen;
+} __packed;
+
+/* Define set/get methods for mdcrec_data_oerase_omf */
+OMF_SETGET(struct mdcrec_data_oerase_omf, pdrt_rtype, 8)
+OMF_SETGET(struct mdcrec_data_oerase_omf, pdrt_objid, 64)
+OMF_SETGET(struct mdcrec_data_oerase_omf, pdrt_gen, 64)
+#define OMF_MDCREC_OERASE_PACKLEN (sizeof(struct mdcrec_data_oerase_omf))
+
+
+/**
+ * struct mdcrec_data_mcconfig_omf - packed data record mclass config
+ * @pdrs_rtype: mdrec_type_omf: OMF_MDR_MCCONFIG
+ * @pdrs_parm:
+ */
+struct mdcrec_data_mcconfig_omf {
+	u8                             pdrs_rtype;
+	u8                             pdrs_pad[7];
+	struct devparm_descriptor_omf  pdrs_parm;
+} __packed;
+
+
+OMF_SETGET(struct mdcrec_data_mcconfig_omf, pdrs_rtype, 8)
+#define OMF_MDCREC_MCCONFIG_PACKLEN (sizeof(struct mdcrec_data_mcconfig_omf))
+
+
+/**
+ * struct mdcrec_data_mcspare_omf - packed data record mcspare
+ * @pdra_rtype:   mdrec_type_omf: OMF_MDR_MCSPARE
+ * @pdra_mclassp: enum mp_media_classp
+ * @pdra_spzone:   percent spare zones for drives in media class
+ */
+struct mdcrec_data_mcspare_omf {
+	u8     pdra_rtype;
+	u8     pdra_mclassp;
+	u8     pdra_spzone;
+} __packed;
+
+/* Define set/get methods for mdcrec_data_mcspare_omf */
+OMF_SETGET(struct mdcrec_data_mcspare_omf, pdra_rtype, 8)
+OMF_SETGET(struct mdcrec_data_mcspare_omf, pdra_mclassp, 8)
+OMF_SETGET(struct mdcrec_data_mcspare_omf, pdra_spzone, 8)
+#define OMF_MDCREC_CLS_SPARE_PACKLEN (sizeof(struct mdcrec_data_mcspare_omf))
+
+
+/**
+ * struct mdcrec_data_ocreate_omf - packed data record ocreate
+ * @pdrc_rtype:     mdrec_type_omf: OMF_MDR_OCREATE or OMF_MDR_OUPDATE
+ * @pdrc_mclass:
+ * @pdrc_uuid:
+ * @pdrc_ld:
+ * @pdrc_objid:     object identifier
+ * @pdrc_gen:       object generation number
+ * @pdrc_mblen:     amount of data written in the mblock, for mlog this is 0
+ * @pdrc_uuid:      Used only for mlogs. Must be at the end of this struct.
+ */
+struct mdcrec_data_ocreate_omf {
+	u8                             pdrc_rtype;
+	u8                             pdrc_mclass;
+	u8                             pdrc_pad[2];
+	struct layout_descriptor_omf   pdrc_ld;
+	__le64                         pdrc_objid;
+	__le64                         pdrc_gen;
+	__le64                         pdrc_mblen;
+	u8                             pdrc_uuid[];
+} __packed;
+
+/* Define set/get methods for mdcrec_data_ocreate_omf */
+OMF_SETGET(struct mdcrec_data_ocreate_omf, pdrc_rtype, 8)
+OMF_SETGET(struct mdcrec_data_ocreate_omf, pdrc_mclass, 8)
+OMF_SETGET(struct mdcrec_data_ocreate_omf, pdrc_objid, 64)
+OMF_SETGET(struct mdcrec_data_ocreate_omf, pdrc_gen, 64)
+OMF_SETGET(struct mdcrec_data_ocreate_omf, pdrc_mblen, 64)
+#define OMF_MDCREC_OBJCMN_PACKLEN (sizeof(struct mdcrec_data_ocreate_omf) + \
+				   OMF_UUID_PACKLEN)
+
+
+/**
+ * struct mdcrec_data_mpconfig_omf - packed data mpool config
+ * @pdmc_rtype:
+ * @pdmc_oid1:
+ * @pdmc_oid2:
+ * @pdmc_uid:
+ * @pdmc_gid:
+ * @pdmc_mode:
+ * @pdmc_mclassp:
+ * @pdmc_captgt:
+ * @pdmc_ra_pages_max:
+ * @pdmc_vma_size_max:
+ * @pdmc_utype:         user-defined type (uuid)
+ * @pdmc_label:         user-defined label (ascii)
+ */
+struct mdcrec_data_mpconfig_omf {
+	u8      pdmc_rtype;
+	u8      pdmc_pad[7];
+	__le64  pdmc_oid1;
+	__le64  pdmc_oid2;
+	__le32  pdmc_uid;
+	__le32  pdmc_gid;
+	__le32  pdmc_mode;
+	__le32  pdmc_rsvd0;
+	__le64  pdmc_captgt;
+	__le32  pdmc_ra_pages_max;
+	__le32  pdmc_vma_size_max;
+	__le32  pdmc_rsvd1;
+	__le32  pdmc_rsvd2;
+	__le64  pdmc_rsvd3;
+	__le64  pdmc_rsvd4;
+	u8      pdmc_utype[16];
+	u8      pdmc_label[MPOOL_LABELSZ_MAX];
+} __packed;
+
+/* Define set/get methods for mdcrec_data_mpconfig_omf */
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_rtype, 8)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_oid1, 64)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_oid2, 64)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_uid, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_gid, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_mode, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_rsvd0, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_captgt, 64)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_ra_pages_max, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_vma_size_max, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_rsvd1, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_rsvd2, 32)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_rsvd3, 64)
+OMF_SETGET(struct mdcrec_data_mpconfig_omf, pdmc_rsvd4, 64)
+OMF_SETGET_CHBUF(struct mdcrec_data_mpconfig_omf, pdmc_utype)
+OMF_SETGET_CHBUF(struct mdcrec_data_mpconfig_omf, pdmc_label)
+#define OMF_MDCREC_MPCONFIG_PACKLEN (sizeof(struct mdcrec_data_mpconfig_omf))
+
+
+/*
+ * Object types embedded in opaque uint64 object ids by the pmd module.
+ * This encoding is also present in the object ids stored in the
+ * data records on media.
+ *
+ * The obj_type field is 4 bits. There are two valid obj types.
+ */
+enum obj_type_omf {
+	OMF_OBJ_UNDEF       = 0,
+	OMF_OBJ_MBLOCK      = 1,
+	OMF_OBJ_MLOG        = 2,
+};
+
+/**
+ * sb_descriptor_ver_omf - Mpool super block version
+ * @OMF_SB_DESC_UNDEF: value not on media
+ */
+enum sb_descriptor_ver_omf {
+	OMF_SB_DESC_UNDEF        = 0,
+	OMF_SB_DESC_V1           = 1,
+
+};
+#define OMF_SB_DESC_VER_LAST   OMF_SB_DESC_V1
+
+
+/**
+ * struct sb_descriptor_omf - packed super block, super block descriptor format version 1.
+ * @psb_magic:  mpool magic value; offset 0 in all vers
+ * @psb_name:   mpool name
+ * @psb_poolid: UUID of pool this drive belongs to
+ * @psb_vers:   sb format version; offset 56
+ * @psb_gen:    sb generation number on this drive
+ * @psb_cksum1: checksum of all fields above
+ * @psb_parm:   parameters for this drive
+ * @psb_cksum2: checksum of psb_parm
+ * @psb_mdc01gen:   mdc0 log1 generation number
+ * @psb_mdc01uuid:
+ * @psb_mdc01devid: mdc0 log1 device UUID
+ * @psb_mdc01strip: mdc0 log1 strip desc.
+ * @psb_mdc01desc:  mdc0 log1 layout
+ * @psb_mdc02gen:   mdc0 log2 generation number
+ * @psb_mdc02uuid:
+ * @psb_mdc02devid: mdc0 log2 device UUID
+ * @psb_mdc02strip: mdc0 log2 strip desc.
+ * @psb_mdc02desc:  mdc0 log2 layout
+ * @psb_mdc0dev:    drive param for mdc0 strip
+ *
+ * Note: these fields, up to and including psb_cksum1, are known to libblkid.
+ * cannot change them without havoc. Fields from psb_magic to psb_cksum1
+ * included are at same offset in all versions.
+ */
+struct sb_descriptor_omf {
+	__le64                         psb_magic;
+	u8                             psb_name[OMF_MPOOL_NAME_LEN];
+	u8                             psb_poolid[OMF_UUID_PACKLEN];
+	__le16                         psb_vers;
+	__le32                         psb_gen;
+	u8                             psb_cksum1[4];
+
+	u8                             psb_pad1[6];
+	struct devparm_descriptor_omf  psb_parm;
+	u8                             psb_cksum2[4];
+
+	u8                             psb_pad2[4];
+	__le64                         psb_mdc01gen;
+	u8                             psb_mdc01uuid[OMF_UUID_PACKLEN];
+	u8                             psb_mdc01devid[OMF_UUID_PACKLEN];
+	struct layout_descriptor_omf   psb_mdc01desc;
+
+	u8                             psb_pad3[4];
+	__le64                         psb_mdc02gen;
+	u8                             psb_mdc02uuid[OMF_UUID_PACKLEN];
+	u8                             psb_mdc02devid[OMF_UUID_PACKLEN];
+	struct layout_descriptor_omf   psb_mdc02desc;
+
+	u8                             psb_pad4[4];
+	struct devparm_descriptor_omf  psb_mdc0dev;
+} __packed;
+
+OMF_SETGET(struct sb_descriptor_omf, psb_magic, 64)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_name)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_poolid)
+OMF_SETGET(struct sb_descriptor_omf, psb_vers, 16)
+OMF_SETGET(struct sb_descriptor_omf, psb_gen, 32)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_cksum1)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_cksum2)
+OMF_SETGET(struct sb_descriptor_omf, psb_mdc01gen, 64)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_mdc01uuid)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_mdc01devid)
+OMF_SETGET(struct sb_descriptor_omf, psb_mdc02gen, 64)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_mdc02uuid)
+OMF_SETGET_CHBUF(struct sb_descriptor_omf, psb_mdc02devid)
+#define OMF_SB_DESC_PACKLEN (sizeof(struct sb_descriptor_omf))
+
+/*
+ * For object-related records OCREATE/OUPDATE is max so compute that here as:
+ * rtype + objid + gen + layout desc
+ */
+#define OMF_MDCREC_PACKLEN_MAX max(OMF_MDCREC_OBJCMN_PACKLEN,            \
+				   max(OMF_MDCREC_MCCONFIG_PACKLEN,      \
+				       max(OMF_MDCREC_CLS_SPARE_PACKLEN, \
+					   OMF_MDCREC_MPCONFIG_PACKLEN)))
+
+#endif /* MPOOL_OMF_H */
diff --git a/drivers/mpool/omf_if.h b/drivers/mpool/omf_if.h
new file mode 100644
index 000000000000..5f11a03ef500
--- /dev/null
+++ b/drivers/mpool/omf_if.h
@@ -0,0 +1,381 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_OMF_IF_H
+#define MPOOL_OMF_IF_H
+
+#include "uuid.h"
+#include "mpool_ioctl.h"
+
+#include "mp.h"
+#include "omf.h"
+
+struct mpool_descriptor;
+struct pmd_layout;
+
+/*
+ * Common defs: versioned via version number field of enclosing structs
+ */
+
+/**
+ * struct omf_layout_descriptor - version 1 layout descriptor
+ * @ol_zaddr:
+ * @ol_zcnt: number of zones
+ * @ol_pdh:
+ */
+struct omf_layout_descriptor {
+	u64    ol_zaddr;
+	u32    ol_zcnt;
+	u16    ol_pdh;
+};
+
+/**
+ * struct omf_devparm_descriptor - version 1 devparm descriptor
+ * @odp_devid:    UUID for drive
+ * @odp_devsz:    size, in bytes, of the volume/device
+ * @odp_zonetot:  total number of zones
+ * @odp_zonepg:   zone size in number of zone pages
+ * @odp_mclassp:  enum mp_media_classp
+ * @odp_devtype:  PD type. Enum pd_devtype
+ * @odp_sectorsz: 2^podp_sectorsz = sector size
+ * @odp_features: Features, ored bits of enum mp_mc_features
+ *
+ * The fields zonepg, mclassp, devtype, sectosz, and features uniquely identify
+ * the media class of the PD.
+ * All drives in a media class must have the same values in the below fields.
+ */
+struct omf_devparm_descriptor {
+	struct mpool_uuid  odp_devid;
+	u64                odp_devsz;
+	u32                odp_zonetot;
+
+	u32                odp_zonepg;
+	u8                 odp_mclassp;
+	u8                 odp_devtype;
+	u8                 odp_sectorsz;
+	u64                odp_features;
+};
+
+/*
+ * Superblock (sb) -- version 1
+ *
+ * Note this is 8-byte-wide reversed to get correct ascii order
+ */
+#define OMF_SB_MAGIC  0x7665446c6f6f706dULL  /* ASCII mpoolDev - no null */
+
+/**
+ * struct omf_sb_descriptor - version 1 superblock descriptor
+ * @osb_magic:  mpool magic value
+ * @osb_name:   mpool name, contains a terminating 0 byte
+ * @osb_cktype: enum mp_cksum_type value
+ * @osb_vers:   sb format version
+ * @osb_poolid: UUID of pool this drive belongs to
+ * @osb_gen:    sb generation number on this drive
+ * @osb_parm:   parameters for this drive
+ * @osb_mdc01gen:   mdc0 log1 generation number
+ * @osb_mdc01uuid:
+ * @osb_mdc01devid:
+ * @osb_mdc01desc:  mdc0 log1 layout
+ * @osb_mdc02gen:   mdc0 log2 generation number
+ * @osb_mdc02uuid:
+ * @osb_mdc02devid:
+ * @osb_mdc02desc:  mdc0 log2 layout
+ * @osb_mdc0dev:   drive param for mdc0
+ */
+struct omf_sb_descriptor {
+	u64                            osb_magic;
+	u8                             osb_name[MPOOL_NAMESZ_MAX];
+	u8                             osb_cktype;
+	u16                            osb_vers;
+	struct mpool_uuid              osb_poolid;
+	u32                            osb_gen;
+	struct omf_devparm_descriptor  osb_parm;
+
+	u64                            osb_mdc01gen;
+	struct mpool_uuid              osb_mdc01uuid;
+	struct mpool_uuid              osb_mdc01devid;
+	struct omf_layout_descriptor   osb_mdc01desc;
+
+	u64                            osb_mdc02gen;
+	struct mpool_uuid              osb_mdc02uuid;
+	struct mpool_uuid              osb_mdc02devid;
+	struct omf_layout_descriptor   osb_mdc02desc;
+
+	struct omf_devparm_descriptor  osb_mdc0dev;
+};
+
+/**
+ * struct omf_logrec_descriptor -
+ * @olr_tlen:  logical length of data record (all chunks)
+ * @olr_rlen:  length of data chunk in this log record
+ * @olr_rtype: enum logrec_type_omf value
+ *
+ */
+struct omf_logrec_descriptor {
+	u32    olr_tlen;
+	u16    olr_rlen;
+	u8     olr_rtype;
+};
+
+/**
+ * struct omf_logblock_header -
+ * @olh_magic:   unique ID per mlog
+ * @olh_pfsetid: flush set ID of the previous log block
+ * @olh_cfsetid: flush set ID this log block
+ * @olh_gen:     generation number
+ * @olh_vers:    log block format version
+ */
+struct omf_logblock_header {
+	struct mpool_uuid    olh_magic;
+	u32                olh_pfsetid;
+	u32                olh_cfsetid;
+	u64                olh_gen;
+	u16                olh_vers;
+};
+
+/**
+ * struct omf_mdcver - version of an mpool MDC content.
+ * @mdcver:
+ *
+ * mdcver[0]: major version number
+ * mdcver[1]: minor version number
+ * mdcver[2]: patch version number
+ * mdcver[3]: development version number. Used during development cycle when
+ *            the above numbers don't change.
+ *
+ * This is not the version of the message framing used for the MDC.
+ * This the version of the binary that introduced that version of the MDC
+ * content.
+ */
+struct omf_mdcver {
+	u16    mdcver[4];
+};
+
+#define mdcv_major    mdcver[0]
+#define mdcv_minor    mdcver[1]
+#define mdcv_patch    mdcver[2]
+#define mdcv_dev      mdcver[3]
+
+/**
+ * struct omf_mdcrec_data -
+ * @omd_version:  OMF_MDR_VERSION record
+ * @omd_objid:  object identifier
+ * @omd_gen:    object generation number
+ * @omd_layout:
+ * @omd_mblen:  Length of written data in object
+ * @omd_old:
+ * @omd_uuid:
+ * @omd_parm:
+ * @omd_mclassp: mp_media_classp
+ * @omd_spzone:   percent spare zones for drives in media class
+ * @omd_cfg:
+ * @omd_rtype: enum mdcrec_type_omf value
+ *
+ * object-related rtypes:
+ * ODELETE, OIDCKPT: objid field only; others ignored
+ * OERASE: objid and gen fields only; others ignored
+ * OCREATE, OUPDATE: layout field only; others ignored
+ */
+struct omf_mdcrec_data {
+	union ustruct {
+		struct omf_mdcver omd_version;
+
+		struct object {
+			u64                             omd_objid;
+			u64                             omd_gen;
+			struct pmd_layout              *omd_layout;
+			u64                             omd_mblen;
+			struct omf_layout_descriptor    omd_old;
+			struct mpool_uuid               omd_uuid;
+			u8                              omd_mclass;
+		} obj;
+
+		struct drive_state {
+			struct omf_devparm_descriptor  omd_parm;
+		} dev;
+
+		struct media_cls_spare {
+			u8 omd_mclassp;
+			u8 omd_spzone;
+		} mcs;
+
+		struct mpool_config    omd_cfg;
+	} u;
+
+	u8             omd_rtype;
+};
+
+/**
+ * objid_type() - Return the type field from an objid
+ * @objid:
+ */
+static inline int objid_type(u64 objid)
+{
+	return ((objid & 0xF00) >> 8);
+}
+
+static inline bool objtype_valid(enum obj_type_omf otype)
+{
+	return otype && (otype <= 2);
+};
+
+/*
+ * omf API functions -- exported functions for working with omf structures
+ */
+
+/**
+ * omf_sb_pack_htole() - pack superblock
+ * @sb: struct omf_sb_descriptor *
+ * @outbuf: char *
+ *
+ * Pack superblock into outbuf little-endian computing specified checksum.
+ *
+ * Return: 0 if successful, -EINVAL otherwise
+ */
+int omf_sb_pack_htole(struct omf_sb_descriptor *sb, char *outbuf);
+
+/**
+ * omf_sb_unpack_letoh() - unpack superblock
+ * @sb: struct omf_sb_descriptor *
+ * @inbuf: char *
+ * @omf_ver: on-media-format superblock version
+ *
+ * Unpack little-endian superblock from inbuf into sb verifying checksum.
+ *
+ * Return: 0 if successful, -errno otherwise
+ */
+int omf_sb_unpack_letoh(struct omf_sb_descriptor *sb, const char *inbuf, u16 *omf_ver);
+
+/**
+ * omf_sb_has_magic_le() - Determine if buffer has superblock magic value
+ * @inbuf: char *
+ *
+ * Determine if little-endian buffer inbuf has superblock magic value
+ * where expected; does NOT imply inbuf is a valid superblock.
+ *
+ * Return: 1 if true; 0 otherwise
+ */
+bool omf_sb_has_magic_le(const char *inbuf);
+
+/**
+ * omf_logblock_header_pack_htole() - pack log block header
+ * @lbh: struct omf_logblock_header *
+ * @outbuf: char *
+ *
+ * Pack header into little-endian log block buffer lbuf, ex-checksum.
+ *
+ * Return: 0 if successful, -errno otherwise
+ */
+int omf_logblock_header_pack_htole(struct omf_logblock_header *lbh, char *lbuf);
+
+/**
+ * omf_logblock_header_len_le() - Determine header length of log block
+ * @lbuf: char *
+ *
+ * Check little-endian log block in lbuf to determine header length.
+ *
+ * Return: bytes in packed header; -EINVAL if invalid header vers
+ */
+int omf_logblock_header_len_le(char *lbuf);
+
+/**
+ * omf_logblock_header_unpack_letoh() - unpack log block header
+ * @lbh: struct omf_logblock_header *
+ * @inbuf: char *
+ *
+ * Unpack little-endian log block header from lbuf into lbh; does not
+ * verify checksum.
+ *
+ * Return: 0 if successful, -EINVAL if invalid log block header vers
+ */
+int omf_logblock_header_unpack_letoh(struct omf_logblock_header *lbh, const char *inbuf);
+
+/**
+ * omf_logrec_desc_pack_htole() - pack log record descriptor
+ * @lrd: struct omf_logrec_descriptor *
+ * @outbuf: char *
+ *
+ * Pack log record descriptor into outbuf little-endian.
+ *
+ * Return: 0 if successful, -EINVAL if invalid log rec type
+ */
+int omf_logrec_desc_pack_htole(struct omf_logrec_descriptor *lrd, char *outbuf);
+
+/**
+ * omf_logrec_desc_unpack_letoh() - unpack log record descriptor
+ * @lrd: struct omf_logrec_descriptor *
+ * @inbuf: char *
+ *
+ * Unpack little-endian log record descriptor from inbuf into lrd.
+ */
+void omf_logrec_desc_unpack_letoh(struct omf_logrec_descriptor *lrd, const char *inbuf);
+
+/**
+ * omf_mdcrec_pack_htole() - pack mdc record
+ * @mp: struct mpool_descriptor *
+ * @cdr: struct omf_mdcrec_data *
+ * @outbuf: char *
+ *
+ * Pack mdc record into outbuf little-endian.
+ * NOTE: Assumes outbuf has enough space for the layout structure.
+ *
+ * Return: bytes packed if successful, -EINVAL otherwise
+ */
+int omf_mdcrec_pack_htole(struct mpool_descriptor *mp, struct omf_mdcrec_data *cdr, char *outbuf);
+
+/**
+ * omf_mdcrec_unpack_letoh() - unpack mdc record
+ * @mdcver: mdc content version of the mdc from which this data comes.
+ *          NULL means latest MDC content version known by this binary.
+ * @mp:     struct mpool_descriptor *
+ * @cdr:    struct omf_mdcrec_data *
+ * @inbuf:  char *
+ *
+ * Unpack little-endian mdc record from inbuf into cdr.
+ *
+ * Return: 0 if successful, -errno on error
+ */
+int omf_mdcrec_unpack_letoh(struct omf_mdcver *mdcver, struct mpool_descriptor *mp,
+			    struct omf_mdcrec_data *cdr, const char *inbuf);
+
+/**
+ * omf_mdcrec_isobj_le() - determine if mdc recordis object-related
+ * @inbuf: char *
+ *
+ * Return true if little-endian mdc record in inbuf is object-related.
+ */
+int omf_mdcrec_isobj_le(const char *inbuf);
+
+/**
+ * omf_mdcver_unpack_letoh() - Unpack le mdc version record from inbuf.
+ * @cdr:
+ * @inbuf:
+ */
+void omf_mdcver_unpack_letoh(struct omf_mdcrec_data *cdr, const char *inbuf);
+
+/**
+ * omf_mdcrec_unpack_type_letoh() - extract the record type from a packed MDC record.
+ * @inbuf: packed MDC record.
+ */
+u8 omf_mdcrec_unpack_type_letoh(const char *inbuf);
+
+/**
+ * logrec_type_datarec() - data record or not
+ * @rtype:
+ *
+ * Return: true if the log record type is related to a data record.
+ */
+bool logrec_type_datarec(enum logrec_type_omf rtype);
+
+/**
+ * omf_sbver_to_mdcver() - Returns the matching mdc version for a given superblock version
+ * @sbver: superblock version
+ */
+struct omf_mdcver *omf_sbver_to_mdcver(enum sb_descriptor_ver_omf sbver);
+
+int omf_init(void) __cold;
+void omf_exit(void) __cold;
+
+#endif /* MPOOL_OMF_IF_H */
diff --git a/drivers/mpool/upgrade.h b/drivers/mpool/upgrade.h
new file mode 100644
index 000000000000..3b3748c47a3e
--- /dev/null
+++ b/drivers/mpool/upgrade.h
@@ -0,0 +1,128 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * Defines structures for upgrading MPOOL meta data
+ */
+
+#ifndef MPOOL_UPGRADE_H
+#define MPOOL_UPGRADE_H
+
+#include "omf_if.h"
+
+/*
+ * Size of version converted to string.
+ * 4 * (5 bytes for a u16) + 3 * (1 byte for the '.') + 1 byte for \0
+ */
+#define MAX_MDCVERSTR          24
+
+/*
+ * Naming conventions:
+ *
+ * omf structures:
+ * ---------------
+ * The old structure names end with _omf_v<version number>.
+ * For example: layout_descriptor_omf_v1
+ * The current/latest structure name end simply with _omf.
+ * For example: layout_descriptor_omf
+ *
+ * Conversion functions:
+ * ---------------------
+ * They are named like:
+ * omf_convert_<blabla>_<maj>_<min>_<patch>_<dev>to<maj>_<min>_<patch>_<dev>()
+ *
+ * For example: omf_convert_sb_1_0_0_0to1_0_0_1()
+ *
+ * They are not named like omf_convert_<blabla>_v1tov2() because sometimes the
+ * input and output structures are exactly the same and the conversion is
+ * related to some subtle interpretation of structure filed[s] content.
+ *
+ * Unpack functions:
+ * -----------------
+ * They are named like:
+ * omf_<blabla>_unpack_letoh_v<version number>()
+ * <version number> being the version of the structure.
+ *
+ * For example: omf_layout_unpack_letoh_v1()
+ * Note that for the latest/current version of the structure we cannot
+ * name the unpack function omf_<blabla>_unpack_letoh() because that would
+ * introduce a name conflict with the top unpack function that calls
+ * omf_unpack_letoh_and_convert()
+ *
+ * For example for layout we have:
+ * omf_layout_unpack_letoh_v1() unpacks layout_descriptor_omf_v1
+ * omf_layout_unpack_letoh_v2() unpacks layout_descriptor_omf
+ * omf_layout_unpack_letoh() calls one of the two above.
+ */
+
+/**
+ * struct upgrade_history -
+ * @uh_size:    size of the current version in-memory structure
+ * @uh_unpack:  unpacking function from on-media format to in-memory format
+ * @uh_conv:    conversion function from previous version to current version,
+ *              set to NULL for the first version
+ * @uh_sbver:   corresponding superblock version since which the change has
+ *              been introduced. If this structure is not used by superblock
+ *              set uh_sbver =  OMF_SB_DESC_UNDEF.
+ * @uh_mdcver: corresponding mdc ver since which the change has been
+ *              introduced
+ *
+ * Every time we update a nested structure in superblock or MDC, we need to
+ * save the following information about this update, such that we can keep the
+ * update history of this structure
+ */
+struct upgrade_history {
+	size_t                      uh_size;
+	int (*uh_unpack)(void *out, const char *inbuf);
+	int (*uh_conv)(const void *pre, void *cur);
+	enum sb_descriptor_ver_omf  uh_sbver;
+	struct omf_mdcver          uh_mdcver;
+};
+
+/**
+ * omfu_mdcver_cur() - Return the latest mpool MDC content version understood by this binary
+ */
+struct omf_mdcver *omfu_mdcver_cur(void);
+
+/**
+ * omfu_mdcver_comment() - Return mpool MDC content version comment passed in via "mdcver".
+ * @mdcver:
+ */
+const char *omfu_mdcver_comment(struct omf_mdcver *mdcver);
+
+/**
+ * omfu_mdcver_to_str() - convert a version into a string.
+ * @mdcver: version to convert
+ * @buf:    buffer in which to place the conversion.
+ * @sz:     size of "buf" in bytes.
+ *
+ * Returns "buf"
+ */
+char *omfu_mdcver_to_str(struct omf_mdcver *mdcver, char *buf, size_t sz);
+
+/**
+ * omfu_mdcver_cmp() - compare two versions a and b
+ * @a:  first version
+ * @op: compare operator (C syntax), can be "<", "<=", ">", ">=", "==".
+ * @b:  second version
+ *
+ * Return (a op b)
+ */
+bool omfu_mdcver_cmp(struct omf_mdcver *a, char *op, struct omf_mdcver *b);
+
+/**
+ * omfu_mdcver_cmp2() - compare two versions
+ * @a:     first version
+ * @op:    compare operator (C syntax), can be "<", "<=", ">", ">=", "==".
+ * @major: major, minor, patch and dev which composes the second version
+ * @minor:
+ * @patch:
+ * @dev:
+ *
+ * Return true (a op b)
+ */
+bool omfu_mdcver_cmp2(struct omf_mdcver *a, char *op, u16 major, u16 minor, u16 patch, u16 dev);
+
+#endif /* MPOOL_UPGRADE_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:19 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:19 -0500
Subject: [PATCH v2 05/22] mpool: add space map component which manages free
 space on mpool devices
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-6-nmeeramohide@micron.com>

The smap layer implements a free space map for each media class
volume in an active mpool.

Free space maps are maintained in memory only. When an mpool is
activated, the free space map is reconstructed from the object
metadata read from media. This approach has the following
advantages:
- Objects can be allocated or freed without any space map device IO
- No overhead of tracking both the allocated and free space,
  and keeping them synchronized

The LBA space of a media class volume is subdivided into regions.
Allocation requests for a volume are distributed across these
regions. There's a separate space map per region which is
protected by a region mutex 'pdi_rmlock'.

Each region is further subdivided into zones. The zone size is
determined at mpool create time, and it defaults to 32MiB.
The free space in each region is represented by a rbtree,
where the key is a zone number and the value is the length of
the free space specified as a zone count.

A configurable percentage of the total zones on a given volume
are marked as spare zones, and the rest are marked as usable
zones. The smap supports different allocation policies which
determine which zone type is used to satisfy an allocation
request - usable or spare or usable then spare or
spare then usable.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/init.c   |   17 +-
 drivers/mpool/mclass.c |  103 ++++
 drivers/mpool/smap.c   | 1031 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 1150 insertions(+), 1 deletion(-)
 create mode 100644 drivers/mpool/mclass.c
 create mode 100644 drivers/mpool/smap.c

diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
index 294cf3cbbaa7..031408815b48 100644
--- a/drivers/mpool/init.c
+++ b/drivers/mpool/init.c
@@ -8,6 +8,7 @@
 #include "mpool_printk.h"
 
 #include "pd.h"
+#include "smap.h"
 
 /*
  * Module params...
@@ -22,16 +23,30 @@ MODULE_PARM_DESC(chunk_size_kb, "Chunk size (in KiB) for device I/O");
 
 static void mpool_exit_impl(void)
 {
+	smap_exit();
 	pd_exit();
 }
 
 static __init int mpool_init(void)
 {
+	const char *errmsg = NULL;
 	int rc;
 
 	rc = pd_init();
 	if (rc) {
-		mp_pr_err("pd init failed", rc);
+		errmsg = "pd init failed";
+		goto errout;
+	}
+
+	rc = smap_init();
+	if (rc) {
+		errmsg = "smap init failed";
+		goto errout;
+	}
+
+errout:
+	if (rc) {
+		mp_pr_err("%s", rc, errmsg);
 		mpool_exit_impl();
 	}
 
diff --git a/drivers/mpool/mclass.c b/drivers/mpool/mclass.c
new file mode 100644
index 000000000000..a81ee5ee9468
--- /dev/null
+++ b/drivers/mpool/mclass.c
@@ -0,0 +1,103 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * This file contains the media class accessor functions.
+ */
+
+#include <linux/errno.h>
+
+#include "omf_if.h"
+#include "pd.h"
+#include "params.h"
+#include "mclass.h"
+
+void mc_init_class(struct media_class *mc, struct mc_parms *mc_parms, struct mc_smap_parms *mcsp)
+{
+	memcpy(&(mc->mc_parms), mc_parms, sizeof(*mc_parms));
+	mc->mc_uacnt = 0;
+	mc->mc_sparms = *mcsp;
+}
+
+void mc_omf_devparm2mc_parms(struct omf_devparm_descriptor *omf_devparm, struct mc_parms *mc_parms)
+{
+	/* Zeroes mc_ parms because memcmp() may be used on it later. */
+	memset(mc_parms, 0, sizeof(*mc_parms));
+	mc_parms->mcp_classp   = omf_devparm->odp_mclassp;
+	mc_parms->mcp_zonepg   = omf_devparm->odp_zonepg;
+	mc_parms->mcp_sectorsz = omf_devparm->odp_sectorsz;
+	mc_parms->mcp_devtype  = omf_devparm->odp_devtype;
+	mc_parms->mcp_features = omf_devparm->odp_features;
+}
+
+void mc_parms2omf_devparm(struct mc_parms *mc_parms, struct omf_devparm_descriptor *omf_devparm)
+{
+	omf_devparm->odp_mclassp  = mc_parms->mcp_classp;
+	omf_devparm->odp_zonepg   = mc_parms->mcp_zonepg;
+	omf_devparm->odp_sectorsz = mc_parms->mcp_sectorsz;
+	omf_devparm->odp_devtype  = mc_parms->mcp_devtype;
+	omf_devparm->odp_features = mc_parms->mcp_features;
+}
+
+int mc_cmp_omf_devparm(struct omf_devparm_descriptor *omfd1, struct omf_devparm_descriptor *omfd2)
+{
+	struct mc_parms mc_parms1;
+	struct mc_parms mc_parms2;
+
+	mc_omf_devparm2mc_parms(omfd1, &mc_parms1);
+	mc_omf_devparm2mc_parms(omfd2, &mc_parms2);
+
+	return memcmp(&mc_parms1, &mc_parms2, sizeof(mc_parms1));
+}
+
+void mc_pd_prop2mc_parms(struct pd_prop *pd_prop, struct mc_parms *mc_parms)
+{
+	/* Zeroes mc_ parms because memcmp() may be used on it later. */
+	memset(mc_parms, 0, sizeof(*mc_parms));
+	mc_parms->mcp_classp	= pd_prop->pdp_mclassp;
+	mc_parms->mcp_zonepg	= pd_prop->pdp_zparam.dvb_zonepg;
+	mc_parms->mcp_sectorsz	= PD_SECTORSZ(pd_prop);
+	mc_parms->mcp_devtype	= pd_prop->pdp_devtype;
+	mc_parms->mcp_features	= OMF_MC_FEAT_MBLOCK_TGT;
+
+	if (pd_prop->pdp_cmdopt & PD_CMD_SECTOR_UPDATABLE)
+		mc_parms->mcp_features |= OMF_MC_FEAT_MLOG_TGT;
+	if (pd_prop->pdp_cmdopt & PD_CMD_DIF_ENABLED)
+		mc_parms->mcp_features |= OMF_MC_FEAT_CHECKSUM;
+}
+
+int mc_set_spzone(struct media_class *mc, u8 spzone)
+{
+	if (!mc)
+		return -EINVAL;
+
+	if (mc->mc_pdmc < 0)
+		return -ENOENT;
+
+	mc->mc_sparms.mcsp_spzone = spzone;
+
+	return 0;
+}
+
+static void mc_smap_parms_get_internal(struct mpcore_params *params, struct mc_smap_parms *mcsp)
+{
+	mcsp->mcsp_spzone = params->mp_spare;
+	mcsp->mcsp_rgnc = params->mp_smaprgnc;
+	mcsp->mcsp_align = params->mp_smapalign;
+}
+
+int mc_smap_parms_get(struct media_class *mc, struct mpcore_params *params,
+		      struct mc_smap_parms *mcsp)
+{
+	if (!mc || !mcsp)
+		return -EINVAL;
+
+	if (mc->mc_pdmc >= 0)
+		*mcsp = mc->mc_sparms;
+	else
+		mc_smap_parms_get_internal(params, mcsp);
+
+	return 0;
+}
diff --git a/drivers/mpool/smap.c b/drivers/mpool/smap.c
new file mode 100644
index 000000000000..a62aaa2f0113
--- /dev/null
+++ b/drivers/mpool/smap.c
@@ -0,0 +1,1031 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * Space map module.
+ *
+ * Implements space maps for managing free space on drives.
+ */
+
+#include <linux/log2.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+
+#include "assert.h"
+#include "mpool_printk.h"
+
+#include "pd.h"
+#include "sb.h"
+#include "mclass.h"
+#include "smap.h"
+#include "mpcore.h"
+
+static struct kmem_cache  *smap_zone_cache __read_mostly;
+
+static int smap_drive_alloc(struct mpool_descriptor *mp, struct mc_smap_parms *mcsp, u16 pdh);
+static int smap_drive_sballoc(struct mpool_descriptor *mp, u16 pdh);
+
+/*
+ * smap API functions
+ */
+
+static struct smap_zone *smap_zone_find(struct rb_root *root, u64 key)
+{
+	struct rb_node *node = root->rb_node;
+	struct smap_zone *elem;
+
+	while (node) {
+		elem = rb_entry(node, typeof(*elem), smz_node);
+
+		if (key < elem->smz_key)
+			node = node->rb_left;
+		else if (key > elem->smz_key)
+			node = node->rb_right;
+		else
+			return elem;
+	}
+
+	return NULL;
+}
+
+static int smap_zone_insert(struct rb_root *root, struct smap_zone *item)
+{
+	struct rb_node **pos = &root->rb_node, *parent = NULL;
+	struct smap_zone *this;
+
+	/* Figure out where to put new node */
+	while (*pos) {
+		this = rb_entry(*pos, typeof(*this), smz_node);
+		parent = *pos;
+
+		if (item->smz_key < this->smz_key)
+			pos = &(*pos)->rb_left;
+		else if (item->smz_key > this->smz_key)
+			pos = &(*pos)->rb_right;
+		else
+			return false;
+	}
+
+	/* Add new node and rebalance tree. */
+	rb_link_node(&item->smz_node, parent, pos);
+	rb_insert_color(&item->smz_node, root);
+
+	return true;
+}
+
+int smap_mpool_init(struct mpool_descriptor *mp)
+{
+	struct mpool_dev_info *pd = NULL;
+	struct media_class *mc;
+	u64 pdh = 0;
+	int rc = 0;
+
+	for (pdh = 0; pdh < mp->pds_pdvcnt; pdh++) {
+		struct mc_smap_parms   mcsp;
+
+		pd = &mp->pds_pdv[pdh];
+		mc = &mp->pds_mc[pd->pdi_mclass];
+		rc = mc_smap_parms_get(&mp->pds_mc[mc->mc_parms.mcp_classp],
+				       &mp->pds_params, &mcsp);
+		if (rc)
+			break;
+
+		rc = smap_drive_init(mp, &mcsp, pdh);
+		if (rc) {
+			mp_pr_err("smap(%s, %s): drive init failed",
+				  rc, mp->pds_name, pd->pdi_name);
+			break;
+		}
+	}
+
+	if (rc)
+		smap_mpool_free(mp);
+
+	return rc;
+}
+
+void smap_mpool_free(struct mpool_descriptor *mp)
+{
+	u64 pdh = 0;
+
+	for (pdh = 0; pdh < mp->pds_pdvcnt; pdh++)
+		smap_drive_free(mp, pdh);
+}
+
+void smap_mpool_usage(struct mpool_descriptor *mp, u8 mclass, struct mpool_usage *usage)
+{
+	if (mclass == MP_MED_ALL) {
+		u32 i;
+
+		for (i = 0; i < MP_MED_NUMBER; i++)
+			smap_mclass_usage(mp, i, usage);
+	} else {
+		smap_mclass_usage(mp, mclass, usage);
+	}
+}
+
+int smap_drive_spares(struct mpool_descriptor *mp, enum mp_media_classp mclassp, u8 spzone)
+{
+	struct mpool_dev_info *pd = NULL;
+	struct media_class *mc;
+	int rc;
+	u8 i;
+
+	if (!mclass_isvalid(mclassp) || spzone > 100) {
+		rc = -EINVAL;
+		mp_pr_err("smap mpool %s: smap drive spares failed mclassp %d spzone %u",
+			  rc, mp->pds_name, mclassp, spzone);
+		return rc;
+	}
+
+	/* Loop on all classes matching mclassp. */
+	for (i = 0; i < MP_MED_NUMBER; i++) {
+		mc = &mp->pds_mc[i];
+		if (mc->mc_parms.mcp_classp != mclassp || mc->mc_pdmc < 0)
+			continue;
+
+		pd = &mp->pds_pdv[mc->mc_pdmc];
+
+		spin_lock(&pd->pdi_ds.sda_dalock);
+		/* Adjust utgt but not uact; possible for uact > utgt due to spzone change. */
+		pd->pdi_ds.sda_utgt = (pd->pdi_ds.sda_zoneeff * (100 - spzone)) / 100;
+		/* Adjust stgt and sact maintaining invariant that sact <= stgt */
+		pd->pdi_ds.sda_stgt = pd->pdi_ds.sda_zoneeff - pd->pdi_ds.sda_utgt;
+		if (pd->pdi_ds.sda_sact > pd->pdi_ds.sda_stgt) {
+			pd->pdi_ds.sda_uact += (pd->pdi_ds.sda_sact - pd->pdi_ds.sda_stgt);
+			pd->pdi_ds.sda_sact = pd->pdi_ds.sda_stgt;
+		}
+		spin_unlock(&pd->pdi_ds.sda_dalock);
+
+	}
+	return 0;
+}
+
+/*
+ * Compute zone stats for drive pd per comments in smap_dev_alloc.
+ */
+static void smap_calc_znstats(struct mpool_dev_info *pd, struct smap_dev_znstats *zones)
+{
+	zones->sdv_total = pd->pdi_parm.dpr_zonetot;
+	zones->sdv_avail = pd->pdi_ds.sda_zoneeff;
+	zones->sdv_usable = pd->pdi_ds.sda_utgt;
+
+	if (pd->pdi_ds.sda_utgt > pd->pdi_ds.sda_uact)
+		zones->sdv_fusable = pd->pdi_ds.sda_utgt - pd->pdi_ds.sda_uact;
+	else
+		zones->sdv_fusable = 0;
+
+	zones->sdv_spare = pd->pdi_ds.sda_stgt;
+	zones->sdv_fspare = pd->pdi_ds.sda_stgt - pd->pdi_ds.sda_sact;
+	zones->sdv_used = pd->pdi_ds.sda_uact;
+}
+
+int smap_drive_usage(struct mpool_descriptor *mp, u16 pdh, struct mpool_devprops *dprop)
+{
+	struct mpool_dev_info *pd = &mp->pds_pdv[pdh];
+	struct smap_dev_znstats zones;
+	u32 zonepg = 0;
+
+	zonepg = pd->pdi_parm.dpr_zonepg;
+
+	spin_lock(&pd->pdi_ds.sda_dalock);
+	smap_calc_znstats(pd, &zones);
+	spin_unlock(&pd->pdi_ds.sda_dalock);
+
+	dprop->pdp_total = (zones.sdv_total * zonepg) << PAGE_SHIFT;
+	dprop->pdp_avail = (zones.sdv_avail * zonepg) << PAGE_SHIFT;
+	dprop->pdp_spare = (zones.sdv_spare * zonepg) << PAGE_SHIFT;
+	dprop->pdp_fspare = (zones.sdv_fspare * zonepg) << PAGE_SHIFT;
+	dprop->pdp_usable = (zones.sdv_usable * zonepg) << PAGE_SHIFT;
+	dprop->pdp_fusable = (zones.sdv_fusable * zonepg) << PAGE_SHIFT;
+	dprop->pdp_used = (zones.sdv_used * zonepg) << PAGE_SHIFT;
+
+	return 0;
+}
+
+int smap_drive_init(struct mpool_descriptor *mp, struct mc_smap_parms *mcsp, u16 pdh)
+{
+	struct mpool_dev_info *pd __maybe_unused;
+	int rc;
+
+	pd = &mp->pds_pdv[pdh];
+
+	if ((mcsp->mcsp_spzone > 100) || !(mcsp->mcsp_rgnc > 0)) {
+		rc = -EINVAL;
+		mp_pr_err("smap(%s, %s): drive init failed, spzone %u rcnt %lu", rc, mp->pds_name,
+			  pd->pdi_name, mcsp->mcsp_spzone, (ulong)mcsp->mcsp_rgnc);
+		return rc;
+	}
+
+	rc = smap_drive_alloc(mp, mcsp, pdh);
+	if (!rc) {
+		rc = smap_drive_sballoc(mp, pdh);
+		if (rc)
+			mp_pr_err("smap(%s, %s): sb alloc failed", rc, mp->pds_name, pd->pdi_name);
+	} else {
+		mp_pr_err("smap(%s, %s): drive alloc failed", rc, mp->pds_name, pd->pdi_name);
+	}
+
+	if (rc)
+		smap_drive_free(mp, pdh);
+
+	return rc;
+}
+
+void smap_drive_free(struct mpool_descriptor *mp, u16 pdh)
+{
+	struct mpool_dev_info *pd = &mp->pds_pdv[pdh];
+	u8 rgn = 0;
+
+	if (pd->pdi_rmbktv) {
+		struct media_class     *mc;
+		struct mc_smap_parms    mcsp;
+
+		mc = &mp->pds_mc[pd->pdi_mclass];
+		(void)mc_smap_parms_get(&mp->pds_mc[mc->mc_parms.mcp_classp],
+					&mp->pds_params, &mcsp);
+
+		for (rgn = 0; rgn < mcsp.mcsp_rgnc; rgn++) {
+			struct smap_zone   *zone, *tmp;
+			struct rb_root     *root;
+
+			root = &pd->pdi_rmbktv[rgn].pdi_rmroot;
+
+			rbtree_postorder_for_each_entry_safe(zone, tmp, root, smz_node) {
+				kmem_cache_free(smap_zone_cache, zone);
+			}
+		}
+
+		kfree(pd->pdi_rmbktv);
+		pd->pdi_rmbktv = NULL;
+	}
+
+	pd->pdi_ds.sda_rgnsz = 0;
+	pd->pdi_ds.sda_rgnladdr = 0;
+	pd->pdi_ds.sda_rgnalloc = 0;
+	pd->pdi_ds.sda_zoneeff = 0;
+	pd->pdi_ds.sda_utgt = 0;
+	pd->pdi_ds.sda_uact = 0;
+}
+
+static bool smap_alloccheck(struct mpool_dev_info *pd, u64 zonecnt, enum smap_space_type sapolicy)
+{
+	struct smap_dev_alloc *ds;
+	bool alloced = false;
+	u64 zoneextra;
+
+	ds = &pd->pdi_ds;
+
+	spin_lock(&ds->sda_dalock);
+
+	switch (sapolicy) {
+
+	case SMAP_SPC_USABLE_ONLY:
+		if ((ds->sda_uact + zonecnt) > ds->sda_utgt)
+			break;
+
+		ds->sda_uact = ds->sda_uact + zonecnt;
+		alloced = true;
+		break;
+
+	case SMAP_SPC_SPARE_ONLY:
+		if ((ds->sda_sact + zonecnt) > ds->sda_stgt)
+			break;
+
+		ds->sda_sact = ds->sda_sact + zonecnt;
+		alloced = true;
+		break;
+
+	case SMAP_SPC_USABLE_2_SPARE:
+		if ((ds->sda_uact + ds->sda_sact + zonecnt) > ds->sda_zoneeff)
+			break;
+
+		if ((ds->sda_uact + zonecnt) <= ds->sda_utgt) {
+			ds->sda_uact = ds->sda_uact + zonecnt;
+		} else {
+			zoneextra = (ds->sda_uact + zonecnt) - ds->sda_utgt;
+			ds->sda_uact = ds->sda_utgt;
+			ds->sda_sact = ds->sda_sact + zoneextra;
+		}
+		alloced = true;
+		break;
+
+	case SMAP_SPC_SPARE_2_USABLE:
+		if ((ds->sda_sact + ds->sda_uact + zonecnt) > ds->sda_zoneeff)
+			break;
+
+		if ((ds->sda_sact + zonecnt) <= ds->sda_stgt) {
+			ds->sda_sact = ds->sda_sact + zonecnt;
+		} else {
+			zoneextra = (ds->sda_sact + zonecnt) - ds->sda_stgt;
+			ds->sda_sact = ds->sda_stgt;
+			ds->sda_uact = ds->sda_uact + zoneextra;
+		}
+		alloced = true;
+		break;
+
+	default:
+		break;
+	}
+
+	spin_unlock(&ds->sda_dalock);
+
+	return alloced;
+}
+
+int smap_alloc(struct mpool_descriptor *mp, u16 pdh, u64 zonecnt,
+	       enum smap_space_type sapolicy, u64 *zoneaddr, u64 align)
+{
+	struct mc_smap_parms mcsp;
+	struct mpool_dev_info *pd;
+	struct smap_dev_alloc *ds;
+	struct smap_zone *elem = NULL;
+	struct rb_root *rmap = NULL;
+	struct mutex *rmlock = NULL;
+	struct media_class *mc;
+	u64 fsoff = 0, fslen = 0, ualen = 0;
+	u8 rgn = 0, rgnc;
+	s8 rgnleft;
+	bool res;
+	int rc;
+
+	*zoneaddr = 0;
+	pd = &mp->pds_pdv[pdh];
+
+	if (!zonecnt || !saptype_valid(sapolicy))
+		return -EINVAL;
+
+	ASSERT(is_power_of_2(align));
+
+	ds = &pd->pdi_ds;
+	mc = &mp->pds_mc[pd->pdi_mclass];
+	rc = mc_smap_parms_get(&mp->pds_mc[mc->mc_parms.mcp_classp], &mp->pds_params, &mcsp);
+	if (rc)
+		return rc;
+
+	rgnc = mcsp.mcsp_rgnc;
+
+	/*
+	 * We do not update the last rgn alloced beyond this point as it
+	 * would incur search penalty if all the regions except one are highly
+	 * fragmented, i.e., the last alloc rgn would never change in this case.
+	 */
+	spin_lock(&ds->sda_dalock);
+	ds->sda_rgnalloc = (ds->sda_rgnalloc + 1) % rgnc;
+	rgn = ds->sda_rgnalloc;
+	spin_unlock(&ds->sda_dalock);
+
+	rgnleft = rgnc;
+
+	/* Search per-rgn space maps for contiguous region. */
+	while (rgnleft--) {
+		struct rb_node *node;
+
+		rmlock = &pd->pdi_rmbktv[rgn].pdi_rmlock;
+		rmap = &pd->pdi_rmbktv[rgn].pdi_rmroot;
+
+		mutex_lock(rmlock);
+
+		for (node = rb_first(rmap); node; node = rb_next(node)) {
+			elem  = rb_entry(node, struct smap_zone, smz_node);
+			fsoff = elem->smz_key;
+			fslen = elem->smz_value;
+
+			if (zonecnt > fslen)
+				continue;
+
+			if (IS_ALIGNED(fsoff, align)) {
+				ualen = 0;
+				break;
+			}
+
+			ualen = ALIGN(fsoff, align) - fsoff;
+			if (ualen + zonecnt > fslen)
+				continue;
+
+			break;
+		}
+
+		if (node)
+			break;
+
+		mutex_unlock(rmlock);
+
+		rgn = (rgn + 1) % rgnc;
+	}
+
+	if (rgnleft < 0)
+		return -ENOSPC;
+
+	/* Alloc from this free space if permitted. First fit. */
+	res = smap_alloccheck(pd, zonecnt, sapolicy);
+	if (!res) {
+		mutex_unlock(rmlock);
+		return -ENOSPC;
+	}
+
+	fsoff = fsoff + ualen;
+	fslen = fslen - ualen;
+
+	*zoneaddr = fsoff;
+	rb_erase(&elem->smz_node, rmap);
+
+	if (zonecnt < fslen) {
+		/* Re-use elem */
+		elem->smz_key   = fsoff + zonecnt;
+		elem->smz_value = fslen - zonecnt;
+		smap_zone_insert(rmap, elem);
+		elem = NULL;
+	}
+
+	if (ualen) {
+		if (!elem) {
+			elem = kmem_cache_alloc(smap_zone_cache, GFP_ATOMIC);
+			if (!elem) {
+				mutex_unlock(rmlock);
+				return -ENOMEM;
+			}
+		}
+
+		elem->smz_key   = fsoff - ualen;
+		elem->smz_value = ualen;
+		smap_zone_insert(rmap, elem);
+		elem = NULL;
+	}
+
+	mutex_unlock(rmlock);
+
+	if (elem)
+		kmem_cache_free(smap_zone_cache, elem);
+
+	return 0;
+}
+
+/*
+ * smap internal functions
+ */
+
+/*
+ * Init empty space map for drive pdh with a % spare zones of spzone.
+ * Returns: 0 if successful, -errno otherwise...
+ */
+static int smap_drive_alloc(struct mpool_descriptor *mp, struct mc_smap_parms *mcsp, u16 pdh)
+{
+	struct mpool_dev_info *pd = &mp->pds_pdv[pdh];
+	struct smap_zone *urb_elem = NULL;
+	struct smap_zone *found_ue = NULL;
+	u32 rgnsz = 0;
+	u8 rgn = 0;
+	u8 rgn2 = 0;
+	u8 rgnc;
+	int rc;
+
+	rgnc  = mcsp->mcsp_rgnc;
+	rgnsz = pd->pdi_parm.dpr_zonetot / rgnc;
+	if (!rgnsz) {
+		rc = -EINVAL;
+		mp_pr_err("smap(%s, %s): drive alloc failed, invalid rgn size",
+			  rc, mp->pds_name, pd->pdi_name);
+		return rc;
+	}
+
+	/* Allocate and init per channel space maps and associated locks */
+	pd->pdi_rmbktv = kcalloc(rgnc, sizeof(*pd->pdi_rmbktv), GFP_KERNEL);
+	if (!pd->pdi_rmbktv) {
+		rc = -ENOMEM;
+		mp_pr_err("smap(%s, %s): rmbktv alloc failed", rc, mp->pds_name, pd->pdi_name);
+		return rc;
+	}
+
+	/* Define all space on all channels as being free (drive empty) */
+	for (rgn = 0; rgn < rgnc; rgn++) {
+		mutex_init(&pd->pdi_rmbktv[rgn].pdi_rmlock);
+
+		urb_elem = kmem_cache_alloc(smap_zone_cache, GFP_KERNEL);
+		if (!urb_elem) {
+			struct rb_root *rmroot;
+
+			for (rgn2 = 0; rgn2 < rgn; rgn2++) {
+				rmroot = &pd->pdi_rmbktv[rgn2].pdi_rmroot;
+
+				found_ue = smap_zone_find(rmroot, 0);
+				if (found_ue) {
+					rb_erase(&found_ue->smz_node, rmroot);
+					kmem_cache_free(smap_zone_cache, found_ue);
+				}
+			}
+
+			kfree(pd->pdi_rmbktv);
+			pd->pdi_rmbktv = NULL;
+
+			rc = -ENOMEM;
+			mp_pr_err("smap(%s, %s): rb node alloc failed, rgn %u",
+				  rc, mp->pds_name, pd->pdi_name, rgn);
+			return rc;
+		}
+
+		urb_elem->smz_key = rgn * rgnsz;
+		if (rgn < rgnc - 1)
+			urb_elem->smz_value = rgnsz;
+		else
+			urb_elem->smz_value = pd->pdi_parm.dpr_zonetot - (rgn * rgnsz);
+		smap_zone_insert(&pd->pdi_rmbktv[rgn].pdi_rmroot, urb_elem);
+	}
+
+	spin_lock_init(&pd->pdi_ds.sda_dalock);
+	pd->pdi_ds.sda_rgnalloc = 0;
+	pd->pdi_ds.sda_rgnsz = rgnsz;
+	pd->pdi_ds.sda_rgnladdr = (rgnc - 1) * rgnsz;
+	pd->pdi_ds.sda_zoneeff = pd->pdi_parm.dpr_zonetot;
+	pd->pdi_ds.sda_utgt = (pd->pdi_ds.sda_zoneeff * (100 - mcsp->mcsp_spzone)) / 100;
+	pd->pdi_ds.sda_uact = 0;
+	pd->pdi_ds.sda_stgt = pd->pdi_ds.sda_zoneeff - pd->pdi_ds.sda_utgt;
+	pd->pdi_ds.sda_sact = 0;
+
+	return 0;
+}
+
+/*
+ * Add entry to space map covering superblocks on drive pdh.
+ * Returns: 0 if successful, -errno otherwise...
+ */
+static int smap_drive_sballoc(struct mpool_descriptor *mp, u16 pdh)
+{
+	struct mpool_dev_info *pd = &mp->pds_pdv[pdh];
+	int rc;
+	u32 cnt;
+
+	cnt = sb_zones_for_sbs(&(pd->pdi_prop));
+	if (cnt < 1) {
+		rc = -ESPIPE;
+		mp_pr_err("smap(%s, %s): identifying sb failed", rc, mp->pds_name, pd->pdi_name);
+		return rc;
+	}
+
+	rc = smap_insert(mp, pdh, 0, cnt);
+	if (rc)
+		mp_pr_err("smap(%s, %s): insert failed, cnt %u",
+			  rc, mp->pds_name, pd->pdi_name, cnt);
+
+	return rc;
+}
+
+void smap_mclass_usage(struct mpool_descriptor *mp, u8 mclass, struct mpool_usage *usage)
+{
+	struct smap_dev_znstats zones;
+	struct mpool_dev_info *pd;
+	struct media_class *mc;
+	u32 zonepg = 0;
+
+	mc = &mp->pds_mc[mclass];
+	if (mc->mc_pdmc < 0)
+		return;
+
+	pd = &mp->pds_pdv[mc->mc_pdmc];
+	zonepg = pd->pdi_zonepg;
+
+	spin_lock(&pd->pdi_ds.sda_dalock);
+	smap_calc_znstats(pd, &zones);
+	spin_unlock(&pd->pdi_ds.sda_dalock);
+
+	usage->mpu_total  += ((zones.sdv_total * zonepg) << PAGE_SHIFT);
+	usage->mpu_usable += ((zones.sdv_usable * zonepg) << PAGE_SHIFT);
+	usage->mpu_used   += ((zones.sdv_used * zonepg) << PAGE_SHIFT);
+	usage->mpu_spare  += ((zones.sdv_spare * zonepg) << PAGE_SHIFT);
+	usage->mpu_fspare += ((zones.sdv_fspare * zonepg) << PAGE_SHIFT);
+	usage->mpu_fusable += ((zones.sdv_fusable * zonepg) << PAGE_SHIFT);
+}
+
+static u32 smap_addr2rgn(struct mpool_descriptor *mp, struct mpool_dev_info *pd, u64 zoneaddr)
+{
+	struct mc_smap_parms   mcsp;
+
+	mc_smap_parms_get(&mp->pds_mc[pd->pdi_mclass], &mp->pds_params, &mcsp);
+
+	if (zoneaddr >= pd->pdi_ds.sda_rgnladdr)
+		return mcsp.mcsp_rgnc - 1;
+
+	return zoneaddr / pd->pdi_ds.sda_rgnsz;
+}
+
+/*
+ * Add entry to space map in rgn starting at zoneaddr
+ * and continuing for zonecnt blocks.
+ *
+ *   Returns: 0 if successful, -errno otherwise...
+ */
+static int smap_insert_byrgn(struct mpool_dev_info *pd, u32 rgn, u64 zoneaddr, u16 zonecnt)
+{
+	const char *msg __maybe_unused;
+	struct smap_zone *elem = NULL;
+	struct rb_root *rmap;
+	struct rb_node *node;
+	u64 fsoff, fslen;
+	int rc;
+
+	fsoff = fslen = 0;
+	rc = 0;
+	msg = NULL;
+
+	mutex_lock(&pd->pdi_rmbktv[rgn].pdi_rmlock);
+	rmap = &pd->pdi_rmbktv[rgn].pdi_rmroot;
+
+	node = rmap->rb_node;
+	if (!node) {
+		msg = "invalid rgn map";
+		rc = -EINVAL;
+		goto errout;
+	}
+
+	/* Use binary search to find the insertion point in the tree.
+	 */
+	while (node) {
+		elem = rb_entry(node, struct smap_zone, smz_node);
+
+		if (zoneaddr < elem->smz_key)
+			node = node->rb_left;
+		else if (zoneaddr > elem->smz_key + elem->smz_value)
+			node = node->rb_right;
+		else
+			break;
+	}
+
+	fsoff = elem->smz_key;
+	fslen = elem->smz_value;
+
+	/* Bail out if we're past zoneaddr in space map w/o finding the required chunk. */
+	if (zoneaddr < fsoff) {
+		elem = NULL;
+		msg = "requested range not free";
+		rc = -EINVAL;
+		goto errout;
+	}
+
+	/* The allocation must fit entirely within this chunk or it fails. */
+	if (zoneaddr + zonecnt > fsoff + fslen) {
+		elem = NULL;
+		msg = "requested range does not fit";
+		rc = -EINVAL;
+		goto errout;
+	}
+
+	rb_erase(&elem->smz_node, rmap);
+
+	if (zoneaddr > fsoff) {
+		elem->smz_key = fsoff;
+		elem->smz_value = zoneaddr - fsoff;
+		smap_zone_insert(rmap, elem);
+		elem = NULL;
+	}
+	if (zoneaddr + zonecnt < fsoff + fslen) {
+		if (!elem)
+			elem = kmem_cache_alloc(smap_zone_cache, GFP_KERNEL);
+		if (!elem) {
+			msg = "chunk alloc failed";
+			rc = -ENOMEM;
+			goto errout;
+		}
+
+		elem->smz_key = zoneaddr + zonecnt;
+		elem->smz_value = (fsoff + fslen) - (zoneaddr + zonecnt);
+		smap_zone_insert(rmap, elem);
+		elem = NULL;
+	}
+
+	/* Insert consumes usable only; possible for uact > utgt.*/
+	spin_lock(&pd->pdi_ds.sda_dalock);
+	pd->pdi_ds.sda_uact = pd->pdi_ds.sda_uact + zonecnt;
+	spin_unlock(&pd->pdi_ds.sda_dalock);
+
+errout:
+	mutex_unlock(&pd->pdi_rmbktv[rgn].pdi_rmlock);
+
+	if (elem != NULL) {
+		/* Was an exact match */
+		ASSERT((zoneaddr == fsoff) && (zonecnt == fslen));
+		kmem_cache_free(smap_zone_cache, elem);
+	}
+
+	if (rc)
+		mp_pr_err("smap pd %s: %s, zoneaddr %lu zonecnt %u fsoff %lu fslen %lu",
+			  rc, pd->pdi_name, msg ? msg : "(no detail)",
+			  (ulong)zoneaddr, zonecnt, (ulong)fsoff, (ulong)fslen);
+
+	return rc;
+}
+
+int smap_insert(struct mpool_descriptor *mp, u16 pdh, u64 zoneaddr, u32 zonecnt)
+{
+	struct mpool_dev_info *pd = &mp->pds_pdv[pdh];
+	u32 rstart = 0, rend = 0;
+	u64 raddr = 0, rcnt = 0;
+	u64 zoneadded = 0;
+	int rgn = 0;
+	int rc = 0;
+
+	if (zoneaddr >= pd->pdi_parm.dpr_zonetot ||
+	    (zoneaddr + zonecnt) > pd->pdi_parm.dpr_zonetot) {
+		rc = -EINVAL;
+		mp_pr_err("smap(%s, %s): insert failed, zoneaddr %lu zonecnt %u zonetot %u",
+			  rc, mp->pds_name, pd->pdi_name, (ulong)zoneaddr,
+			  zonecnt, pd->pdi_parm.dpr_zonetot);
+		return rc;
+	}
+
+	/*
+	 * smap_alloc() never crosses regions. however a previous instantiation
+	 * of this mpool might have used a different value of rgn count
+	 * so must handle inserts that cross regions.
+	 */
+	rstart = smap_addr2rgn(mp, pd, zoneaddr);
+	rend = smap_addr2rgn(mp, pd, zoneaddr + zonecnt - 1);
+	zoneadded = 0;
+
+	for (rgn = rstart; rgn < rend + 1; rgn++) {
+		/* Compute zone address and count for this rgn */
+		if (rgn == rstart)
+			raddr = zoneaddr;
+		else
+			raddr = (u64)rgn * pd->pdi_ds.sda_rgnsz;
+
+		if (rgn < rend)
+			rcnt = ((rgn + 1) * pd->pdi_ds.sda_rgnsz) - raddr;
+		else
+			rcnt = zonecnt - zoneadded;
+
+		rc = smap_insert_byrgn(pd, rgn, raddr, rcnt);
+		if (rc) {
+			mp_pr_err("smap(%s, %s): insert byrgn failed, rgn %d raddr %lu rcnt %lu",
+				  rc, mp->pds_name, pd->pdi_name, rgn, (ulong)raddr, (ulong)rcnt);
+			break;
+		}
+		zoneadded = zoneadded + rcnt;
+	}
+
+	return rc;
+}
+
+/**
+ * smap_free_byrgn() - free the specified range of zones
+ * @pd:         physical device object
+ * @rgn:       allocation rgn specifier
+ * @zoneaddr:    offset into the space map
+ * @zonecnt:     length of range to be freed
+ *
+ * Free the given range of zone (i.e., [%zoneaddr, %zoneaddr + %zonecnt])
+ * back to the indicated space map.  Always coalesces ranges in the space
+ * map that abut the range to be freed so as to minimize fragmentation.
+ *
+ * Return: 0 if successful, -errno otherwise...
+ */
+static int smap_free_byrgn(struct mpool_dev_info *pd, u32 rgn, u64 zoneaddr, u32 zonecnt)
+{
+	const char *msg __maybe_unused;
+	struct smap_zone *left, *right;
+	struct smap_zone *new, *old;
+	struct rb_root *rmap;
+	struct rb_node *node;
+	u32 orig_zonecnt = zonecnt;
+	int rc = 0;
+
+	new = old = left = right = NULL;
+	msg = NULL;
+
+	mutex_lock(&pd->pdi_rmbktv[rgn].pdi_rmlock);
+	rmap = &pd->pdi_rmbktv[rgn].pdi_rmroot;
+
+	node = rmap->rb_node;
+
+	/* Use binary search to find chunks to the left and/or right of the range being freed. */
+	while (node) {
+		struct smap_zone *this;
+
+		this = rb_entry(node, struct smap_zone, smz_node);
+
+		if (zoneaddr + zonecnt <= this->smz_key) {
+			right = this;
+			node = node->rb_left;
+		} else if (zoneaddr >= this->smz_key + this->smz_value) {
+			left = this;
+			node = node->rb_right;
+		} else {
+			msg = "chunk overlapping";
+			rc = -EINVAL;
+			goto unlock;
+		}
+	}
+
+	/* If the request abuts the chunk to the right then coalesce them. */
+	if (right) {
+		if (zoneaddr + zonecnt == right->smz_key) {
+			zonecnt += right->smz_value;
+			rb_erase(&right->smz_node, rmap);
+
+			new = right;  /* re-use right node */
+		}
+	}
+
+	/* If the request abuts the chunk to the left then coalesce them. */
+	if (left) {
+		if (left->smz_key + left->smz_value == zoneaddr) {
+			zoneaddr = left->smz_key;
+			zonecnt += left->smz_value;
+			rb_erase(&left->smz_node, rmap);
+
+			old = new;  /* free new/left outside the critsec */
+			new = left; /* re-use left node */
+		}
+	}
+
+	/*
+	 * If the request did not abut either the current or the previous
+	 * chunk (i.e., new == NULL) then we must create a new chunk node
+	 * and insert it into the smap.  Otherwise, we'll re-use one of
+	 * the abutting chunk nodes (i.e., left or right).
+	 *
+	 * Note: If we have to call kmalloc and it fails (unlikely) then
+	 * this chunk will be lost only for the current session.  It will
+	 * be recovered once the mpool is closed and re-opened.
+	 */
+	if (!new) {
+		new = kmem_cache_alloc(smap_zone_cache, GFP_ATOMIC);
+		if (!new) {
+			msg = "chunk alloc failed";
+			rc = -ENOMEM;
+			goto unlock;
+		}
+	}
+
+	new->smz_key = zoneaddr;
+	new->smz_value = zonecnt;
+
+	if (!smap_zone_insert(rmap, new)) {
+		kmem_cache_free(smap_zone_cache, new);
+		msg = "chunk insert failed";
+		rc = -ENOTRECOVERABLE;
+		goto unlock;
+	}
+
+	/* Freed space goes to spare first then usable. */
+	zonecnt = orig_zonecnt;
+
+	spin_lock(&pd->pdi_ds.sda_dalock);
+	if (pd->pdi_ds.sda_sact > 0) {
+		if (pd->pdi_ds.sda_sact > zonecnt) {
+			pd->pdi_ds.sda_sact -= zonecnt;
+			zonecnt = 0;
+		} else {
+			zonecnt -= pd->pdi_ds.sda_sact;
+			pd->pdi_ds.sda_sact = 0;
+		}
+	}
+
+	pd->pdi_ds.sda_uact -= zonecnt;
+	spin_unlock(&pd->pdi_ds.sda_dalock);
+
+unlock:
+	mutex_unlock(&pd->pdi_rmbktv[rgn].pdi_rmlock);
+
+	if (old)
+		kmem_cache_free(smap_zone_cache, old);
+
+	if (rc)
+		mp_pr_err("smap pd %s: %s, free byrgn failed, rgn %u zoneaddr %lu zonecnt %u",
+			  rc, pd->pdi_name, msg ? msg : "(no detail)",
+			  rgn, (ulong)zoneaddr, zonecnt);
+
+	return rc;
+}
+
+int smap_free(struct mpool_descriptor *mp, u16 pdh, u64 zoneaddr, u16 zonecnt)
+{
+	struct mpool_dev_info *pd = NULL;
+	u32 rstart = 0, rend = 0;
+	u32 raddr = 0, rcnt = 0;
+	u64 zonefreed = 0;
+	u32 rgn = 0;
+	int rc = 0;
+
+	pd = &mp->pds_pdv[pdh];
+
+	if (zoneaddr >= pd->pdi_parm.dpr_zonetot || zoneaddr + zonecnt > pd->pdi_parm.dpr_zonetot) {
+		rc = -EINVAL;
+		mp_pr_err("smap(%s, %s): free failed, zoneaddr %lu zonecnt %u zonetot: %u",
+			  rc, mp->pds_name, pd->pdi_name, (ulong)zoneaddr,
+			  zonecnt, pd->pdi_parm.dpr_zonetot);
+		return rc;
+	}
+
+	if (!zonecnt)
+		return 0; /* Nothing to be returned */
+
+	/*
+	 * smap_alloc() never crosses regions. however a previous instantiation
+	 * of this mpool might have used a different value of rgn count
+	 * so must handle frees that cross regions.
+	 */
+
+	rstart = smap_addr2rgn(mp, pd, zoneaddr);
+	rend = smap_addr2rgn(mp, pd, zoneaddr + zonecnt - 1);
+
+	for (rgn = rstart; rgn < rend + 1; rgn++) {
+		/* Compute zone address and count for this rgn */
+		if (rgn == rstart)
+			raddr = zoneaddr;
+		else
+			raddr = rgn * pd->pdi_ds.sda_rgnsz;
+
+		if (rgn < rend)
+			rcnt = ((u64)(rgn + 1) * pd->pdi_ds.sda_rgnsz) - raddr;
+		else
+			rcnt = zonecnt - zonefreed;
+
+		rc = smap_free_byrgn(pd, rgn, raddr, rcnt);
+		if (rc) {
+			mp_pr_err("smap(%s, %s): free byrgn failed, rgn %d raddr %lu, rcnt %lu",
+				  rc, mp->pds_name, pd->pdi_name, rgn, (ulong)raddr, (ulong)rcnt);
+			break;
+		}
+		zonefreed = zonefreed + rcnt;
+	}
+
+	return rc;
+}
+
+void smap_wait_usage_done(struct mpool_descriptor *mp)
+{
+	struct smap_usage_work *usagew = &mp->pds_smap_usage_work;
+
+	cancel_delayed_work_sync(&usagew->smapu_wstruct);
+}
+
+#define SMAP_FREEPCT_DELTA 5
+#define SMAP_FREEPCT_LOG_THLD   50
+
+void smap_log_mpool_usage(struct work_struct *ws)
+{
+	struct smap_usage_work *smapu;
+	struct mpool_descriptor *mp;
+	struct mpool_usage usage;
+	int last, cur, delta;
+
+	smapu = container_of(ws, struct smap_usage_work, smapu_wstruct.work);
+	mp = smapu->smapu_mp;
+
+	/* Get the current mpool space usage stats */
+	smap_mpool_usage(mp, MP_MED_ALL, &usage);
+
+	if (usage.mpu_usable == 0) {
+		mp_pr_err("smap mpool %s: zero usable space", -EINVAL, mp->pds_name);
+		return;
+	}
+	/*
+	 * Calculate the delta of free usable space/total usable space,
+	 * since last time a message was logged
+	 */
+	last = smapu->smapu_freepct;
+	cur = usage.mpu_fusable * 100 / usage.mpu_usable;
+	delta = cur - last;
+
+	/*
+	 * Log a message if delta >= 5% && free usable space % < 50%
+	 */
+	if ((abs(delta) >= SMAP_FREEPCT_DELTA) && (cur < SMAP_FREEPCT_LOG_THLD)) {
+
+		smapu->smapu_freepct = cur;
+		if (last == 0)
+			mp_pr_info("smap mpool %s, free space %d%%",
+				   mp->pds_name, smapu->smapu_freepct);
+
+		else
+			mp_pr_info("smap mpool %s, free space %s from %d%% to %d%%",
+				   mp->pds_name, (delta > 0) ? "increases" : "decreases",
+				   last, smapu->smapu_freepct);
+	}
+
+	/* Schedule the next run of smap_log_mpool_usage() */
+	queue_delayed_work(mp->pds_workq, &smapu->smapu_wstruct,
+			   msecs_to_jiffies(mp->pds_params.mp_mpusageperiod));
+}
+
+int smap_init(void)
+{
+	int rc = 0;
+
+	smap_zone_cache = kmem_cache_create("mpool_smap_zone", sizeof(struct smap_zone),
+					    0, SLAB_HWCACHE_ALIGN | SLAB_POISON, NULL);
+	if (!smap_zone_cache) {
+		rc = -ENOMEM;
+		mp_pr_err("kmem_cache_create(smap_zone, %zu) failed",
+			  rc, sizeof(struct smap_zone));
+	}
+
+	return rc;
+}
+
+void smap_exit(void)
+{
+	kmem_cache_destroy(smap_zone_cache);
+	smap_zone_cache = NULL;
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:16 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:16 -0500
Subject: [PATCH v2 02/22] mpool: add in-memory struct definitions
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-3-nmeeramohide@micron.com>

Add headers containing the basic in-memory structures used by mpool.

- mclass.h: media classes
- mlog.h: mlog objects
- mp.h, mpcore.h: mpool objects
- params.h: mpool parameters
- pd.h: pool drive interface
- pmd.h, pmd_obj.h: Metadata manager
- sb.h: superblock interface
- smap.h: space map interface

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mclass.h  | 137 +++++++++++
 drivers/mpool/mlog.h    | 212 +++++++++++++++++
 drivers/mpool/mp.h      | 231 +++++++++++++++++++
 drivers/mpool/mpcore.h  | 354 ++++++++++++++++++++++++++++
 drivers/mpool/params.h  | 116 ++++++++++
 drivers/mpool/pd.h      | 202 ++++++++++++++++
 drivers/mpool/pmd.h     | 379 ++++++++++++++++++++++++++++++
 drivers/mpool/pmd_obj.h | 499 ++++++++++++++++++++++++++++++++++++++++
 drivers/mpool/sb.h      | 162 +++++++++++++
 drivers/mpool/smap.h    | 334 +++++++++++++++++++++++++++
 10 files changed, 2626 insertions(+)
 create mode 100644 drivers/mpool/mclass.h
 create mode 100644 drivers/mpool/mlog.h
 create mode 100644 drivers/mpool/mp.h
 create mode 100644 drivers/mpool/mpcore.h
 create mode 100644 drivers/mpool/params.h
 create mode 100644 drivers/mpool/pd.h
 create mode 100644 drivers/mpool/pmd.h
 create mode 100644 drivers/mpool/pmd_obj.h
 create mode 100644 drivers/mpool/sb.h
 create mode 100644 drivers/mpool/smap.h

diff --git a/drivers/mpool/mclass.h b/drivers/mpool/mclass.h
new file mode 100644
index 000000000000..2ecdcd08de9f
--- /dev/null
+++ b/drivers/mpool/mclass.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_MCLASS_H
+#define MPOOL_MCLASS_H
+
+#include "mpool_ioctl.h"
+
+struct omf_devparm_descriptor;
+struct mpool_descriptor;
+struct mpcore_params;
+
+/*
+ * This file contains the media class structures definitions and prototypes
+ * private to mpool core.
+ */
+
+/**
+ * struct mc_parms - media class parameters
+ * @mcp_classp:    class performance characteristics, enum mp_media_classp
+ * @mcp_zonepg:    zone size in number of zone pages
+ * @mcp_sectorsz:  2^sectorsz is the logical sector size
+ * @mcp_devtype:   device type. Enum pd_devtype.
+ * @mcp_features:  ored bits from mp_mc_features
+ *
+ * Two PDs can't be placed in the same media class if they have different
+ * mc_parms.
+ */
+struct mc_parms {
+	u8  mcp_classp;
+	u32 mcp_zonepg;
+	u8  mcp_sectorsz;
+	u8  mcp_devtype;
+	u64 mcp_features;
+};
+
+/**
+ * struct mc_smap_parms - media class space map parameters
+ * @mcsp_spzone: percent spare zones for drives.
+ * @mcsp_rgnc: no. of space map zones for drives in each media class
+ * @mcsp_align: space map zone alignment for drives in each media class
+ */
+struct mc_smap_parms {
+	u8		mcsp_spzone;
+	u8		mcsp_rgnc;
+	u8		mcsp_align;
+};
+
+/**
+ * struct media_class - define a media class
+ * @mc_parms:  define a media class, content differ for each media class
+ * @mc_sparms: space map params for this media class
+ * @mc_pdmc:   active pdv entries grouped by media class array
+ * @mc_uacnt:  UNAVAIL status drive count in each media class
+ *
+ * Locking:
+ *    Protected by mp.pds_pdvlock.
+ */
+struct media_class {
+	struct mc_parms        mc_parms;
+	struct mc_smap_parms   mc_sparms;
+	s8                     mc_pdmc;
+	u8                     mc_uacnt;
+};
+
+/**
+ * mc_pd_prop2mc_parms() -  Convert PD properties into media class parameters.
+ * @pd_prop: input, pd properties.
+ * @mc_parms: output, media class parameters.
+ *
+ * Typically used before a lookup (mc_lookup_from_mc_parms()) to know in
+ * which media class a PD belongs to.
+ */
+void mc_pd_prop2mc_parms(struct pd_prop *pd_prop, struct mc_parms *mc_parms);
+
+/**
+ * mc_omf_devparm2mc_parms() - convert a omf_devparm_descriptor into an mc_parms.
+ * @omf_devparm: input
+ * @mc_parms: output
+ */
+void mc_omf_devparm2mc_parms(struct omf_devparm_descriptor *omf_devparm, struct mc_parms *mc_parms);
+
+/**
+ * mc_parms2omf_devparm() - convert a mc_parms in a omf_devparm_descriptor
+ * @mc_parms: input
+ * @omf_devparm: output
+ */
+void mc_parms2omf_devparm(struct mc_parms *mc_parms, struct omf_devparm_descriptor *omf_devparm);
+
+/**
+ * mc_cmp_omf_devparm() - check if two omf_devparm_descriptor corresponds
+ *	to the same media class.
+ * @omf_devparm1:
+ * @omf_devparm2:
+ *
+ * Returns 0 if in same media class.
+ */
+int mc_cmp_omf_devparm(struct omf_devparm_descriptor *omfd1, struct omf_devparm_descriptor *omfd2);
+
+/**
+ * mc_init_class() - initialize a media class
+ * @mc:
+ * @mc_parms: parameters of the media class
+ * @mcsp:     smap parameters for mc
+ */
+void mc_init_class(struct media_class *mc, struct mc_parms *mc_parms, struct mc_smap_parms *mcsp);
+
+/**
+ * mc_set_spzone() - set the percent spare on the media class mclass.
+ * @mc:
+ * @spzone:
+ *
+ * Return: 0, or -ENOENT if the specified mclass doesn't exist.
+ */
+int mc_set_spzone(struct media_class *mc, u8 spzone);
+
+/**
+ * mclass_isvalid() - Return true if the media class is valid.
+ * @mclass:
+ */
+static inline bool mclass_isvalid(enum mp_media_classp mclass)
+{
+	return (mclass >= 0 && mclass < MP_MED_NUMBER);
+}
+
+/**
+ * mc_smap_parms_get() - get space map params for the specified mclass.
+ * @mp:
+ * @mclass:
+ * @mcsp: (output)
+ */
+int mc_smap_parms_get(struct media_class *mc, struct mpcore_params *params,
+		      struct mc_smap_parms *mcsp);
+
+#endif /* MPOOL_MCLASS_H */
diff --git a/drivers/mpool/mlog.h b/drivers/mpool/mlog.h
new file mode 100644
index 000000000000..0de816335d55
--- /dev/null
+++ b/drivers/mpool/mlog.h
@@ -0,0 +1,212 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * Defines functions for writing, reading, and managing the lifecycle of mlogs.
+ */
+
+#ifndef MPOOL_MLOG_H
+#define MPOOL_MLOG_H
+
+#include <linux/uio.h>
+
+#include "mpool_ioctl.h"
+
+#define MB       (1024 * 1024)
+struct pmd_layout;
+struct mpool_descriptor;
+struct mlog_descriptor;
+
+
+/**
+ * struct mlog_read_iter -
+ * @lri_layout: Layout of log being read
+ * @lri_soff:   Sector offset of next log block to read from
+ * @lri_gen:    Log generation number at iterator initialization
+ * @lri_roff:   Next offset in log block soff to read from
+ * @lri_rbidx:  Read buffer page index currently reading from
+ * @lri_sidx:   Log block index in lri_rbidx
+ * @lri_valid:  1 if iterator is valid; 0 otherwise
+ */
+struct mlog_read_iter {
+	struct pmd_layout  *lri_layout;
+	off_t               lri_soff;
+	u64                 lri_gen;
+	u16                 lri_roff;
+	u16                 lri_rbidx;
+	u16                 lri_sidx;
+	u8                  lri_valid;
+};
+
+/**
+ * struct mlog_fsetparms -
+ *
+ * @mfp_totsec: Total number of log blocks in mlog
+ * @mfp_secpga: Is sector size page-aligned?
+ * @mfp_lpgsz:  Size of each page in read/append buffer
+ * @mfp_npgmb:  No. of pages in 1 MiB buffer
+ * @mfp_sectsz: Sector size obtained from PD prop
+ * @mfp_nsecmb: No. of sectors/log blocks in 1 MiB buffer
+ * @mfp_nsecpg: No. of sectors/log blocks per page
+ */
+struct mlog_fsetparms {
+	u32    mfp_totsec;
+	bool   mfp_secpga;
+	u32    mfp_lpgsz;
+	u16    mfp_nlpgmb;
+	u16    mfp_sectsz;
+	u16    mfp_nsecmb;
+	u16    mfp_nseclpg;
+};
+
+/**
+ * struct mlog_stat - mlog open status (referenced by associated struct pmd_layout)
+ * @lst_citr:    Current mlog read iterator
+ * @lst_mfp:     Mlog flush set parameters
+ * @lst_abuf:    Append buffer, max 1 MiB size
+ * @lst_rbuf:    Read buffer, max 1 MiB size - immutable
+ * @lst_rsoff:   LB offset of the 1st log block in lst_rbuf
+ * @lst_rseoff:  LB offset of the last log block in lst_rbuf
+ * @lst_asoff:   LB offset of the 1st log block in CFS
+ * @lst_wsoff:   Offset of the accumulating log block
+ * @lst_abdirty: true, if append buffer is dirty
+ * @lst_pfsetid: Prev. fSetID of the first log block in CFS
+ * @lst_cfsetid: Current fSetID of the CFS
+ * @lst_cfssoff: Offset within the 1st log block from where CFS starts
+ * @lst_aoff:    Next byte offset[0, sectsz) to fill in the current log block
+ * @lst_abidx:   Index of current filling page in lst_abuf
+ * @lst_csem:    enforce compaction semantics if true
+ * @lst_cstart:  valid compaction start marker in log?
+ * @lst_cend:    valid compaction end marker in log?
+ */
+struct mlog_stat {
+	struct mlog_read_iter  lst_citr;
+	struct mlog_fsetparms  lst_mfp;
+	char  **lst_abuf;
+	char  **lst_rbuf;
+	off_t   lst_rsoff;
+	off_t   lst_rseoff;
+	off_t   lst_asoff;
+	off_t   lst_wsoff;
+	bool    lst_abdirty;
+	u32     lst_pfsetid;
+	u32     lst_cfsetid;
+	u16     lst_cfssoff;
+	u16     lst_aoff;
+	u16     lst_abidx;
+	u8      lst_csem;
+	u8      lst_cstart;
+	u8      lst_cend;
+};
+
+#define MLOG_TOTSEC(lstat)  ((lstat)->lst_mfp.mfp_totsec)
+#define MLOG_LPGSZ(lstat)   ((lstat)->lst_mfp.mfp_lpgsz)
+#define MLOG_NLPGMB(lstat)  ((lstat)->lst_mfp.mfp_nlpgmb)
+#define MLOG_SECSZ(lstat)   ((lstat)->lst_mfp.mfp_sectsz)
+#define MLOG_NSECMB(lstat)  ((lstat)->lst_mfp.mfp_nsecmb)
+#define MLOG_NSECLPG(lstat) ((lstat)->lst_mfp.mfp_nseclpg)
+
+#define IS_SECPGA(lstat)    ((lstat)->lst_mfp.mfp_secpga)
+
+/*
+ * mlog API functions
+ */
+
+/*
+ * Error codes: all mlog fns can return one or more of:
+ * -EINVAL = invalid fn args
+ * -ENOENT = log not open or logid not found
+ * -EFBIG = log full
+ * -EMSGSIZE = cstart w/o cend indicating a crash during compaction
+ * -ENODATA = malformed or corrupted log
+ * -EIO = unable to read/write log on media
+ * -ENOMEM = insufficient room in copy-out buffer
+ * -EBUSY = log is in erasing state; wait or retry erase
+ */
+
+int mlog_alloc(struct mpool_descriptor *mp, struct mlog_capacity *capreq,
+	       enum mp_media_classp mclassp, struct mlog_props *prop,
+	       struct mlog_descriptor **mlh);
+
+int mlog_realloc(struct mpool_descriptor *mp, u64 objid, struct mlog_capacity *capreq,
+		 enum mp_media_classp mclassp, struct mlog_props *prop,
+		 struct mlog_descriptor **mlh);
+
+int mlog_find_get(struct mpool_descriptor *mp, u64 objid, int which,
+		  struct mlog_props *prop, struct mlog_descriptor **mlh);
+
+void mlog_put(struct mlog_descriptor *layout);
+
+void mlog_lookup_rootids(u64 *id1, u64 *id2);
+
+int mlog_commit(struct mpool_descriptor *mp, struct mlog_descriptor *mlh);
+
+int mlog_abort(struct mpool_descriptor *mp, struct mlog_descriptor *mlh);
+
+int mlog_delete(struct mpool_descriptor *mp, struct mlog_descriptor *mlh);
+
+/**
+ * mlog_open() - Open committed log, validate contents, and return its generation number
+ * @mp:
+ * @mlh:
+ * @flags:
+ * @gen: output
+ *
+ * If log is already open just returns gen; if csem is true enforces compaction
+ * semantics so that open fails if valid cstart/cend markers are not present.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+int mlog_open(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, u8 flags, u64 *gen);
+
+int mlog_close(struct mpool_descriptor *mp, struct mlog_descriptor *mlh);
+
+int mlog_gen(struct mlog_descriptor *mlh, u64 *gen);
+
+int mlog_empty(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, bool *empty);
+
+int mlog_erase(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, u64 mingen);
+
+int mlog_append_cstart(struct mpool_descriptor *mp, struct mlog_descriptor *mlh);
+
+int mlog_append_cend(struct mpool_descriptor *mp, struct mlog_descriptor *mlh);
+
+int mlog_append_data(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+		     char *buf, u64 buflen, int sync);
+
+int mlog_read_data_init(struct mlog_descriptor *mlh);
+
+/**
+ * mlog_read_data_next() -
+ * @mp:
+ * @mlh:
+ * @buf:
+ * @buflen:
+ * @rdlen:
+ *
+ * Returns:
+ *   If -EOVERFLOW is returned, then "buf" is too small to
+ *   hold the read data. Can be retried with a bigger receive buffer whose
+ *   size is returned in rdlen.
+ */
+int mlog_read_data_next(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+			char *buf, u64 buflen, u64 *rdlen);
+
+int mlog_get_props_ex(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+		      struct mlog_props_ex *prop);
+
+void mlog_precompact_alsz(struct mpool_descriptor *mp, struct mlog_descriptor *mlh);
+
+int mlog_rw_raw(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+		const struct kvec *iov, int iovcnt, u64 boff, u8 rw);
+
+void mlogutil_closeall(struct mpool_descriptor *mp);
+
+bool mlog_objid(u64 objid);
+
+struct pmd_layout *mlog2layout(struct mlog_descriptor *mlh);
+
+struct mlog_descriptor *layout2mlog(struct pmd_layout *layout);
+
+#endif /* MPOOL_MLOG_H */
diff --git a/drivers/mpool/mp.h b/drivers/mpool/mp.h
new file mode 100644
index 000000000000..e1570f8c8d0c
--- /dev/null
+++ b/drivers/mpool/mp.h
@@ -0,0 +1,231 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_MP_H
+#define MPOOL_MP_H
+
+#include "mpool_ioctl.h"
+#include "uuid.h"
+#include "params.h"
+
+struct mpool_descriptor;
+
+#define MPOOL_OP_READ  0
+#define MPOOL_OP_WRITE 1
+#define PD_DEV_ID_PDUNAVAILABLE "DID_PDUNAVAILABLE"
+
+#define MPOOL_DRIVES_MAX       MP_MED_NUMBER
+#define MP_MED_ALL             MP_MED_NUMBER
+
+/* Object types */
+enum mp_obj_type {
+	MP_OBJ_UNDEF  = 0,
+	MP_OBJ_MBLOCK = 1,
+	MP_OBJ_MLOG   = 2,
+};
+
+/**
+ * struct mpool_config -
+ * @mc_oid1:
+ * @mc_oid2:
+ * @mc_uid:
+ * @mc_gid:
+ * @mc_mode:
+ * @mc_mclassp:
+ * @mc_captgt:
+ * @mc_ra_pages_max:
+ * @mc_vma_sz_max:
+ * @mc_utype:           user-defined type
+ * @mc_label:           user-defined label
+
+ */
+struct mpool_config {
+	u64                     mc_oid1;
+	u64                     mc_oid2;
+	uid_t                   mc_uid;
+	gid_t                   mc_gid;
+	mode_t                  mc_mode;
+	u32                     mc_rsvd0;
+	u64                     mc_captgt;
+	u32                     mc_ra_pages_max;
+	u32                     mc_vma_size_max;
+	u32                     mc_rsvd1;
+	u32                     mc_rsvd2;
+	u64                     mc_rsvd3;
+	u64                     mc_rsvd4;
+	uuid_le                 mc_utype;
+	char                    mc_label[MPOOL_LABELSZ_MAX];
+};
+
+/*
+ * mpool API functions
+ */
+
+/**
+ * mpool_create() - Create an mpool
+ * @mpname:
+ * @flags: enum mp_mgmt_flags
+ * @dpaths:
+ * @pd_prop: PDs properties obtained by mpool_create() caller.
+ * @params:  mpcore parameters
+ * @mlog_cap:
+ *
+ * Create an mpool from dcnt drive paths dpaths; store mpool metadata as
+ * specified by mdparm;
+ *
+ * Return:
+ * %0 if successful, -errno otherwise..
+ * ENODEV if insufficient number of drives meeting mdparm,
+ */
+int mpool_create(const char *name, u32 flags, char **dpaths, struct pd_prop *pd_prop,
+		 struct mpcore_params *params, u64 mlog_cap);
+
+/**
+ * mpool_activate() - Activate an mpool
+ * @dcnt:
+ * @dpaths:
+ * @pd_prop: properties of the PDs. dcnt elements.
+ * @mlog_cap:
+ * @params:   mpcore parameters
+ * @flags:
+ * @mpp: *mpp is set to NULL if error
+ *
+ * Activate mpool on dcnt drive paths dpaths; if force flag is set tolerate
+ * unavailable drives up to redundancy limit; if successful *mpp is a handle
+ * for the mpool.
+ *
+ * Return:
+ * %0 if successful, -errno otherwise
+ * ENODEV if too many drives unavailable or failed,
+ * ENXIO if device previously removed from mpool and is no longer a member
+ */
+int mpool_activate(u64 dcnt, char **dpaths, struct pd_prop *pd_prop, u64 mlog_cap,
+		   struct mpcore_params *params, u32 flags, struct mpool_descriptor **mpp);
+
+
+/**
+ * mpool_deactivate() - Deactivate an mpool.
+ * @mp: mpool descriptor
+ *
+ * Deactivate mpool; caller must ensure no other thread can access mp; mp is
+ * invalid after call.
+ */
+int mpool_deactivate(struct mpool_descriptor *mp);
+
+/**
+ * mpool_destroy() - Destroy an mpool
+ * @dcnt:
+ * @dpaths:
+ * @pd_prop: PD properties.
+ * @flags:
+ *
+ * Destroy mpool on dcnt drive paths dpaths;
+ *
+ * Return:
+ * %0 if successful, -errno otherwise
+ */
+int mpool_destroy(u64 dcnt, char **dpaths, struct pd_prop *pd_prop, u32 flags);
+
+/**
+ * mpool_rename() - Rename mpool to mp_newname
+ * @dcnt:
+ * @dpaths:
+ * @pd_prop: PD properties.
+ * @flags:
+ * @mp_newname:
+ *
+ * Return:
+ * %0 if successful, -errno otherwise
+ */
+int mpool_rename(u64 dcnt, char **dpaths, struct pd_prop *pd_prop, u32 flags,
+		 const char *mp_newname);
+
+/**
+ * mpool_drive_add() - Add new drive dpath to mpool.
+ * @mp:
+ * @dpath:
+ * @pd_prop: PD properties.
+ *
+ * Return: %0 if successful; -enno otherwise...
+ */
+int mpool_drive_add(struct mpool_descriptor *mp, char *dpath, struct pd_prop *pd_prop);
+
+/**
+ * mpool_drive_spares() - Set percent spare zones to spzone for drives in media class mclassp.
+ * @mp:
+ * @mclassp:
+ * @spzone:
+ *
+ * Return: 0 if successful, -errno otherwise...
+ */
+int mpool_drive_spares(struct mpool_descriptor *mp, enum mp_media_classp mclassp, u8 spzone);
+
+/**
+ * mpool_mclass_get_cnt() - Get a count of media classes with drives in this mpool
+ * @mp:
+ * @info:
+ */
+void mpool_mclass_get_cnt(struct mpool_descriptor *mp, u32 *cnt);
+
+/**
+ * mpool_mclass_get() - Get a information on mcl_cnt media classes
+ * @mp:
+ * @mcic:
+ * @mciv:
+ *
+ * Return: 0 if successful, -errno otherwise...
+ */
+int mpool_mclass_get(struct mpool_descriptor *mp, u32 *mcxc, struct mpool_mclass_xprops *mcxv);
+
+/**
+ * mpool_get_xprops() - Retrieve extended mpool properties
+ * @mp:
+ * @prop:
+ */
+void mpool_get_xprops(struct mpool_descriptor *mp, struct mpool_xprops *xprops);
+
+/**
+ * mpool_get_devprops_by_name() - Fill in dprop for active drive with name pdname
+ * @mp:
+ * @pdname:
+ * @dprop:
+ *
+ * Return: %0 if success, -errno otherwise...
+ * -ENOENT if device with specified name cannot be found
+ */
+int mpool_get_devprops_by_name(struct mpool_descriptor *mp, char *pdname,
+			       struct mpool_devprops *dprop);
+
+/**
+ * mpool_get_usage() - Fill in stats with mpool space usage for the media class mclassp
+ * @mp:
+ * @mclassp:
+ * @usage:
+ *
+ * If mclassp is MCLASS_ALL, report on entire pool (all media classes).
+ *
+ * Return: %0 if successful; err_t otherwise...
+ */
+void
+mpool_get_usage(
+	struct mpool_descriptor    *mp,
+	enum mp_media_classp        mclassp,
+	struct mpool_usage         *usage);
+
+/**
+ * mpool_config_store() - store a config record in MDC0
+ * @mp:
+ * @cfg:
+ */
+int mpool_config_store(struct mpool_descriptor *mp, const struct mpool_config *cfg);
+
+/**
+ * mpool_config_fetch() - fetch the current mpool config
+ * @mp:
+ * @cfg:
+ */
+int mpool_config_fetch(struct mpool_descriptor *mp, struct mpool_config *cfg);
+
+#endif /* MPOOL_MP_H */
diff --git a/drivers/mpool/mpcore.h b/drivers/mpool/mpcore.h
new file mode 100644
index 000000000000..904763d49814
--- /dev/null
+++ b/drivers/mpool/mpcore.h
@@ -0,0 +1,354 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_MPCORE_H
+#define MPOOL_MPCORE_H
+
+#include <linux/rbtree.h>
+#include <linux/workqueue.h>
+#include <linux/mutex.h>
+
+#include "uuid.h"
+
+#include "mp.h"
+#include "pd.h"
+#include "smap.h"
+#include "mclass.h"
+#include "pmd.h"
+#include "params.h"
+
+extern struct rb_root mpool_pools;
+
+struct pmd_layout;
+
+/**
+ * enum mpool_status -
+ * @MPOOL_STAT_UNDEF:
+ * @MPOOL_STAT_OPTIMAL:
+ * @MPOOL_STAT_FAULTED:
+ */
+enum mpool_status {
+	MPOOL_STAT_UNDEF    = 0,
+	MPOOL_STAT_OPTIMAL  = 1,
+	MPOOL_STAT_FAULTED  = 2,
+	MPOOL_STAT_LAST = MPOOL_STAT_FAULTED,
+};
+
+_Static_assert((MPOOL_STAT_LAST < 256), "enum mpool_status must fit in u8");
+
+/**
+ * struct mpool_dev_info - Pool drive state, status, and params
+ * @pdi_devid:    UUID for this drive
+ * @pdi_parm:     drive parms
+ * @pdi_status:   enum pd_status value: drive status
+ * @pdi_ds:       drive space allocation info
+ * @pdi_rmap:     per allocation zone space maps rbtree array, node:
+ *                struct u64_to_u64_rb
+ * @pdi_rmlock:   lock protects per zone space maps
+ * @pdi_name:     device name (only the last path name component)
+ *
+ * Pool drive state, status, and params
+ *
+ * LOCKING:
+ *    devid, mclass : constant; no locking required
+ *    parm: constant EXCEPT in rare change of status from UNAVAIL; see below
+ *    status: usage does not require locking, but MUST get/set via accessors
+ *    state: protected by pdvlock in enclosing mpool_descriptor
+ *    ds: protected by ds.dalock defined in smap module
+ *    zmap[x]: protected by zmlock[x]
+ *
+ * parm fields are constant except in a rare change of status from UNAVAIL,
+ * during which a subset of the fields are modified.  see the pd module for
+ * details on how this is handled w/o requiring locking.
+ */
+struct mpool_dev_info {
+	atomic_t                pdi_status; /* Barriers or acq/rel required */
+	struct pd_dev_parm      pdi_parm;
+	struct smap_dev_alloc   pdi_ds;
+	struct rmbkt           *pdi_rmbktv;
+	struct mpool_uuid       pdi_devid;
+};
+
+/* Shortcuts */
+#define pdi_didstr    pdi_parm.dpr_prop.pdp_didstr
+#define pdi_zonepg    pdi_parm.dpr_prop.pdp_zparam.dvb_zonepg
+#define pdi_zonetot   pdi_parm.dpr_prop.pdp_zparam.dvb_zonetot
+#define pdi_devtype   pdi_parm.dpr_prop.pdp_devtype
+#define pdi_cmdopt    pdi_parm.dpr_prop.pdp_cmdopt
+#define pdi_mclass    pdi_parm.dpr_prop.pdp_mclassp
+#define pdi_devsz     pdi_parm.dpr_prop.pdp_devsz
+#define pdi_sectorsz  pdi_parm.dpr_prop.pdp_sectorsz
+#define pdi_optiosz   pdi_parm.dpr_prop.pdp_optiosz
+#define pdi_fua       pdi_parm.dpr_prop.pdp_fua
+#define pdi_prop      pdi_parm.dpr_prop
+#define pdi_name      pdi_parm.dpr_name
+
+/**
+ * struct uuid_to_mpdesc_rb -
+ * @utm_node:
+ * @utm_uuid_le:
+ * @utm_md:
+ */
+struct uuid_to_mpdesc_rb {
+	struct rb_node              utm_node;
+	struct mpool_uuid           utm_uuid_le;
+	struct mpool_descriptor    *utm_md;
+};
+
+/**
+ * struct mpdesc_mdparm - parameters used for the MDCs of the mpool.
+ * @md_mclass:  media class used for the mpool metadata
+ */
+struct mpdesc_mdparm {
+	u8     md_mclass;
+};
+
+/**
+ * struct pre_compact_ctrl - used to start/stop/control precompaction
+ * @pco_dwork:
+ * @pco_mp:
+ * @pco_nmtoc: next MDC to compact
+
+ * Each time pmd_precompact_cb() runs it will consider the next MDC
+ * for compaction.
+ */
+struct pre_compact_ctrl {
+	struct delayed_work	 pco_dwork;
+	struct mpool_descriptor *pco_mp;
+	atomic_t		 pco_nmtoc;
+};
+
+/**
+ * struct mpool_descriptor - Media pool descriptor
+ * @pds_pdvlock:  drive membership/state lock
+ * @pds_pdv:      per drive info array
+ * @pds_omlock:   open mlog index lock
+ * @pds_oml:      rbtree of open mlog layouts. indexed by objid
+ *                node type: objid_to_layout_rb
+ * @pds_poolid:   UUID of pool
+ * @pds_mdparm:   mclass id of mclass used for mdc layouts
+ * @pds_cfg:      mpool config
+ * @pds_pdvcnt:   cnt of valid pdv entries
+ * @pds_mc        table of media classes
+ * @pds_uctxt     used by user-space mlogs to indicate the context
+ * @pds_node:     for linking this object into an rbtree
+ * @pds_params:   Per mpool parameters
+ * @pds_workq:    Workqueue per mpool.
+ * @pds_sbmdc0:   Used to store in RAM the MDC0 metadata. Loaded at activate
+ *                time, changed when MDC0 is compacted.
+ * @pds_mda:      metadata container array (this thing is huge!)
+ *
+ * LOCKING:
+ *    poolid, ospagesz, mdparm: constant; no locking required
+ *    mda: protected by internal locks as documented in pmd module
+ *    oml: protected by omlock
+ *    pdv: see note
+ *    pds_mc: protected by pds_pdvlock
+ *	Update of pds_mc[].mc_sparams.mc_spzone must also be enclosed
+ *	with mpool_s_lock to serialize the spzone updates, because they include
+ *	an append of an MDC0 record on top of updating mc_spzone.
+ *    all other fields: protected by pds_pdvlock (as is pds_pdv[x].state)
+ *    pds_sbmdc0: Used to store in RAM the MDC0 metadata. Loaded when mpool
+ *	activated, no lock needed at that time (single) threaded.
+ *	Then changed during MDC0 compaction. At that time it is protected by
+ *	MDC0 compact lock.
+ *
+ * NOTE:
+ *    pds_pdvcnt only ever increases so that pds_pdv[x], x < pdvcnt, can be
+ *    accessed without locking, other than as required by the struct
+ *    mpool_dev_info.
+ *    mc_spzone is written and read only by mpool functions that are serialized
+ *    via mpool_s_lock.
+ */
+struct mpool_descriptor {
+	struct rw_semaphore         pds_pdvlock;
+
+	____cacheline_aligned
+	struct mpool_dev_info       pds_pdv[MPOOL_DRIVES_MAX];
+
+	____cacheline_aligned
+	struct mutex                pds_oml_lock;
+	struct rb_root              pds_oml_root;
+
+	/* Read-mostly fields... */
+	____cacheline_aligned
+	u16                         pds_pdvcnt;
+	struct mpdesc_mdparm        pds_mdparm;
+	struct workqueue_struct    *pds_workq;
+	struct workqueue_struct    *pds_erase_wq;
+	struct workqueue_struct    *pds_precompact_wq;
+
+	struct media_class          pds_mc[MP_MED_NUMBER];
+	struct mpcore_params        pds_params;
+	struct omf_sb_descriptor    pds_sbmdc0;
+	struct pre_compact_ctrl     pds_pco;
+	struct smap_usage_work      pds_smap_usage_work;
+
+	/* Rarey used fields... */
+	struct mpool_config         pds_cfg;
+	struct rb_node              pds_node;
+	struct mpool_uuid           pds_poolid;
+	char                        pds_name[MPOOL_NAMESZ_MAX];
+
+	/* pds_mda is enormous (91K) */
+	struct pmd_mda_info         pds_mda;
+};
+
+/**
+ * mpool_desc_unavail_add() - Add unavailable drive to mpool descriptor.
+ * @mp:
+ * @omf_devparm:
+ *
+ * Add unavailable drive to mpool descriptor; caller must guarantee that
+ * devparm.devid is not already there.
+ * As part of adding the drive to the mpool descriptor, the drive is added
+ * in its media class.
+ *
+ * Return: 0 if successful, -errno (-EINVAL or -ENOMEM) otherwise
+ */
+int mpool_desc_unavail_add(struct mpool_descriptor *mp, struct omf_devparm_descriptor *devparm);
+
+/**
+ * mpool_desc_pdmc_add() - Add a device in its media class.
+ * @mp:
+ * @pdh:
+ * @omf_devparm:
+ * @check_only: if true, the call doesn't change any state, it only check
+ *	if the PD could be added in a media class.
+ *
+ * If the media class doesn't exist yet, it is created here.
+ *
+ * This function has two inputs related to the PD it is acting on:
+ *  "phd"
+ *  and "omf_devparm"
+ *
+ * If omf_devparm is NULL, it means that the media class in which the PD must
+ * be placed is derived from mp->pds_pdv[pdh].pdi_parm.dpr_prop
+ * In that case the PD properties (.dpr_prop) must be updated and
+ * correct when entering this function.
+ * devparm is NULL when the device is available, that means the discovery
+ * was able to update .dpr_prop.
+ *
+ * If omf_devparm is not NULL, it means that the media class in which the PD
+ * must be placed is derived from omf_devparm.
+ * This is used when unavailable PDs are placed in their media class. In this
+ * situation (because the PD is unavailable) the discovery couldn't discover
+ * the PD properties and mp->pds_pdv[pdh].pdi_parm.dpr_prop has not been
+ * updated because of that.
+ * So we can't use .dpr_prop to place the PD in its class, instead we use what
+ * is coming from the persitent metadata (PD state record in MDC0). Aka
+ * omf_devparm.
+ * mp->pds_pdv[pdh].pdi_parm.dpr_prop will be update if/when the PD is available
+ * again.
+ *
+ * Restrictions in placing PDs in media classes
+ * --------------------------------------------
+ * This function enforces these restrictions.
+ * These restrictions are:
+ * a) in a mpool, for a given mclassp (enum mp_media_classp), there is
+ *    at maximum one media class.
+ * b) All drives of a media class must checksummed or none, no mix allowed.
+ * c) The STAGING and CAPACITY classes must be both checksummed or both not
+ *    checksummed.
+ *
+ * Locking:
+ * -------
+ *	Should be called with mp.pds_pdvlock held in write.
+ *	Except if mpool is single threaded (during activate for example).
+ */
+int
+mpool_desc_pdmc_add(
+	struct mpool_descriptor		*mp,
+	u16				 pdh,
+	struct omf_devparm_descriptor	*omf_devparm,
+	bool				 check_only);
+
+int uuid_to_mpdesc_insert(struct rb_root *root, struct mpool_descriptor *data);
+
+int
+mpool_dev_sbwrite(
+	struct mpool_descriptor    *mp,
+	struct mpool_dev_info      *pd,
+	struct omf_sb_descriptor   *sbmdc0);
+
+int
+mpool_mdc0_sb2obj(
+	struct mpool_descriptor    *mp,
+	struct omf_sb_descriptor   *sb,
+	struct pmd_layout         **l1,
+	struct pmd_layout         **l2);
+
+int mpool_desc_init_newpool(struct mpool_descriptor *mp, u32 flags);
+
+int
+mpool_dev_init_all(
+	struct mpool_dev_info  *pdv,
+	u64                     dcnt,
+	char                  **dpaths,
+	struct pd_prop	       *pd_prop);
+
+void mpool_mdc_cap_init(struct mpool_descriptor *mp, struct mpool_dev_info *pd);
+
+int
+mpool_desc_init_sb(
+	struct mpool_descriptor    *mp,
+	struct omf_sb_descriptor   *sbmdc0,
+	u32                         flags,
+	bool                       *mc_resize);
+
+int mpool_dev_sbwrite_newpool(struct mpool_descriptor *mp, struct omf_sb_descriptor *sbmdc0);
+
+int check_for_dups(char **listv, int cnt, int *dup, int *offset);
+
+void fill_in_devprops(struct mpool_descriptor *mp, u64 pdh, struct mpool_devprops *dprop);
+
+int mpool_create_rmlogs(struct mpool_descriptor *mp, u64 mlog_cap);
+
+struct mpool_descriptor *mpool_desc_alloc(void);
+
+void mpool_desc_free(struct mpool_descriptor *mp);
+
+int mpool_dev_check_new(struct mpool_descriptor *mp, struct mpool_dev_info *pd);
+
+static inline enum pd_status mpool_pd_status_get(struct mpool_dev_info *pd)
+{
+	enum pd_status  val;
+
+	/* Acquire semantics used so that no reads will be re-ordered from
+	 * before to after this read.
+	 */
+	val = atomic_read_acquire(&pd->pdi_status);
+
+	return val;
+}
+
+static inline void mpool_pd_status_set(struct mpool_dev_info *pd, enum pd_status status)
+{
+	/* All prior writes must be visible prior to the status change */
+	smp_wmb();
+	atomic_set(&pd->pdi_status, status);
+}
+
+/**
+ * mpool_get_mpname() - Get the mpool name
+ * @mp:     mpool descriptor of the mpool
+ * @mpname: buffer to copy the mpool name into
+ * @mplen:  buffer length
+ *
+ * Return:
+ * %0 if successful, -EINVAL otherwise
+ */
+static inline int mpool_get_mpname(struct mpool_descriptor *mp, char *mpname, size_t mplen)
+{
+	if (!mp || !mpname)
+		return -EINVAL;
+
+	strlcpy(mpname, mp->pds_name, mplen);
+
+	return 0;
+}
+
+
+#endif /* MPOOL_MPCORE_H */
diff --git a/drivers/mpool/params.h b/drivers/mpool/params.h
new file mode 100644
index 000000000000..5d1f40857a2a
--- /dev/null
+++ b/drivers/mpool/params.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_PARAMS_H
+#define MPOOL_PARAMS_H
+
+#define MPOOL_MDC_SET_SZ                16
+
+/* Mpool metadata container compaction retries; keep relatively small */
+#define MPOOL_MDC_COMPACT_RETRY_DEFAULT 5
+
+/*
+ * Space map allocation zones per drive; bounds number of concurrent obj
+ * allocs
+ */
+#define MPOOL_SMAP_RGNCNT_DEFAULT       4
+
+/*
+ * Space map alignment in number of zones.
+ */
+#define MPOOL_SMAP_ZONEALIGN_DEFAULT    1
+
+/*
+ * Number of concurent jobs for loading user MDC 1~N
+ */
+#define MPOOL_OBJ_LOAD_JOBS_DEFAULT     8
+
+/*
+ * Defaults for MDC1/255 pre-compaction.
+ */
+#define MPOOL_PCO_PCTFULL               70
+#define MPOOL_PCO_PCTGARBAGE            20
+#define MPOOL_PCO_NBNOALLOC              2
+#define MPOOL_PCO_PERIOD                 5
+#define MPOOL_PCO_FILLBIAS	      1000
+#define MPOOL_PD_USAGE_PERIOD        60000
+#define MPOOL_CREATE_MDC_PCTFULL  (MPOOL_PCO_PCTFULL - MPOOL_PCO_PCTGARBAGE)
+#define MPOOL_CREATE_MDC_PCTGRBG   MPOOL_PCO_PCTGARBAGE
+
+
+/**
+ * struct mpcore_params - mpool core parameters. Not exported to public API.
+ * @mp_mdc0cap: MDC0 capacity,  *ONLY* for testing purpose
+ * @mp_mdcncap: MDCN capacity,  *ONLY* for testing purpose
+ * @mp_mdcnnum: Number of MDCs, *ONLY* for testing purpose
+ * @mp_smaprgnc:
+ * @mp_smapalign:
+ * @mp_spare:
+ * @mp_objloadjobs: number of concurrent MDC loading jobs
+ *
+ * The below parameters starting with "pco" are used for the pre-compaction
+ * of MDC1/255
+ * @mp_pcopctfull:  % (0-100) of fill of MDCi active mlog that must be reached
+ *	before a pre-compaction is attempted.
+ * @mp_pcopctgarbage:  % (0-100) of garbage in MDCi active mlog that must be
+ *	reached	before a pre-compaction is attempted.
+ * @mp_pconbnoalloc: Number of MDCs from which no object is allocated from.
+ *	If 0, that disable the background pre compaction.
+ * @mp_pcoperiod: In seconds. Period at which a background thread check if
+ *	a MDC needs compaction.
+ * @mp_pcofillbias: If the next mpool MDC has less objects than
+ *	(current MDC objects - pcofillbias), then allocate an object
+ *	from the next MDC instead of from the current one.
+ *	This bias favors object allocation from less filled MDCs (in term
+ *	of number of committed objects).
+ *	The bigger the number, the less bias.
+ * @mp_crtmdcpctfull: percent full threshold across all MDCs in combination
+ *      with crtmdcpctgrbg percent is used as a trigger to create new MDCs
+ * @mp_crtmdcpctgrbg: percent garbage threshold in combination with
+ *      @crtmdcpctfull percent is used as a trigger to create new MDCs
+ * @mp_mpusageperiod: period at which a background thread check mpool space
+ * usage, in milliseconds
+ */
+struct mpcore_params {
+	u64    mp_mdcnum;
+	u64    mp_mdc0cap;
+	u64    mp_mdcncap;
+	u64    mp_smaprgnc;
+	u64    mp_smapalign;
+	u64    mp_spare;
+	u64    mp_objloadjobs;
+	u64    mp_pcopctfull;
+	u64    mp_pcopctgarbage;
+	u64    mp_pconbnoalloc;
+	u64    mp_pcoperiod;
+	u64    mp_pcofillbias;
+	u64    mp_crtmdcpctfull;
+	u64    mp_crtmdcpctgrbg;
+	u64    mp_mpusageperiod;
+};
+
+/**
+ * mpcore_params_defaults() -
+ */
+static inline void mpcore_params_defaults(struct mpcore_params *params)
+{
+	params->mp_mdcnum          = MPOOL_MDCNUM_DEFAULT;
+	params->mp_mdc0cap         = 0;
+	params->mp_mdcncap         = 0;
+	params->mp_smaprgnc        = MPOOL_SMAP_RGNCNT_DEFAULT;
+	params->mp_smapalign       = MPOOL_SMAP_ZONEALIGN_DEFAULT;
+	params->mp_spare           = MPOOL_SPARES_DEFAULT;
+	params->mp_pcopctfull	   = MPOOL_PCO_PCTFULL;
+	params->mp_pcopctgarbage   = MPOOL_PCO_PCTGARBAGE;
+	params->mp_pconbnoalloc    = MPOOL_PCO_NBNOALLOC;
+	params->mp_pcoperiod       = MPOOL_PCO_PERIOD;
+	params->mp_pcofillbias     = MPOOL_PCO_FILLBIAS;
+	params->mp_crtmdcpctfull   = MPOOL_CREATE_MDC_PCTFULL;
+	params->mp_crtmdcpctgrbg   = MPOOL_CREATE_MDC_PCTGRBG;
+	params->mp_mpusageperiod   = MPOOL_PD_USAGE_PERIOD;
+	params->mp_objloadjobs     = MPOOL_OBJ_LOAD_JOBS_DEFAULT;
+}
+
+#endif /* MPOOL_PARAMS_H */
diff --git a/drivers/mpool/pd.h b/drivers/mpool/pd.h
new file mode 100644
index 000000000000..c8faefc7cf11
--- /dev/null
+++ b/drivers/mpool/pd.h
@@ -0,0 +1,202 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_PD_H
+#define MPOOL_PD_H
+
+#include <linux/uio.h>
+
+#include "uuid.h"
+#include "mpool_ioctl.h"
+
+/* Returns PD length in bytes. */
+#define PD_LEN(_pd_prop) ((_pd_prop)->pdp_devsz)
+
+/* Returns PD sector size (exponent, power of 2) */
+#define PD_SECTORSZ(_pd_prop) ((_pd_prop)->pdp_sectorsz)
+
+/* Return PD sector size mask */
+#define PD_SECTORMASK(_pd_prop) ((uint64_t)(1 << PD_SECTORSZ(_pd_prop)) - 1)
+
+struct omf_devparm_descriptor;
+
+/**
+ * struct pd_dev_parm -
+ * @dpr_prop:		drive properties including zone parameters
+ * @dpr_dev_private:    private info for implementation
+ * @dpr_name:           device name
+ */
+struct pd_dev_parm {
+	struct pd_prop	         dpr_prop;
+	void		        *dpr_dev_private;
+	char                     dpr_name[PD_NAMESZ_MAX];
+};
+
+/* Shortcuts */
+#define dpr_zonepg        dpr_prop.pdp_zparam.dvb_zonepg
+#define dpr_zonetot       dpr_prop.pdp_zparam.dvb_zonetot
+#define dpr_devsz         dpr_prop.pdp_devsz
+#define dpr_didstr        dpr_prop.pdp_didstr
+#define dpr_mediachar     dpr_prop.pdp_mediachar
+#define dpr_cmdopt        dpr_prop.pdp_cmdopt
+#define dpr_optiosz       dpr_prop.pdp_optiosz
+
+/**
+ * enum pd_status - Transient drive status.
+ * @PD_STAT_UNDEF:       undefined; should never occur
+ * @PD_STAT_ONLINE:      drive is responding to I/O requests
+ * @PD_STAT_SUSPECT:     drive is failing some I/O requests
+ * @PD_STAT_OFFLINE:     drive declared non-responsive to I/O requests
+ * @PD_STAT_UNAVAIL:     drive path not provided or open failed when mpool was opened
+ *
+ * Transient drive status, these are stored as atomic_t variable
+ * values
+ */
+enum pd_status {
+	PD_STAT_UNDEF      = 0,
+	PD_STAT_ONLINE     = 1,
+	PD_STAT_SUSPECT    = 2,
+	PD_STAT_OFFLINE    = 3,
+	PD_STAT_UNAVAIL    = 4
+};
+
+_Static_assert((PD_STAT_UNAVAIL < 256), "enum pd_status must fit in uint8_t");
+
+/**
+ * enum pd_cmd_opt - drive command options
+ * @PD_CMD_DISCARD:	     the device has TRIM/UNMAP command.
+ * @PD_CMD_SECTOR_UPDATABLE: the device can be read/written with sector granularity.
+ * @PD_CMD_DIF_ENABLED:      T10 DIF is used on this device.
+ * @PD_CMD_SED_ENABLED:      Self encrypting enabled
+ * @PD_CMD_DISCARD_ZERO:     the device supports discard_zero
+ * @PD_CMD_RDONLY:           activate mpool with PDs in RDONLY mode,
+ *                           write/discard commands are No-OPs.
+ * Defined as a bit vector so can combine.
+ * Fields holding such a vector should uint64_t.
+ *
+ * TODO: we need to find a way to detect if SED is enabled on a device
+ */
+enum pd_cmd_opt {
+	PD_CMD_NONE             = 0,
+	PD_CMD_DISCARD          = 0x1,
+	PD_CMD_SECTOR_UPDATABLE = 0x2,
+	PD_CMD_DIF_ENABLED      = 0x4,
+	PD_CMD_SED_ENABLED      = 0x8,
+	PD_CMD_DISCARD_ZERO     = 0x10,
+	PD_CMD_RDONLY           = 0x20,
+};
+
+/**
+ * enum pd_devtype - Device types
+ * @PD_DEV_TYPE_BLOCK_STREAM: Block device implementing streams.
+ * @PD_DEV_TYPE_BLOCK_STD:    Standard (non-streams) device (SSD, HDD).
+ * @PD_DEV_TYPE_FILE:	      File in user space for UT.
+ * @PD_DEV_TYPE_MEM:	      Memory semantic device, e.g. NVDIMM direct access (raw or dax mode)
+ * @PD_DEV_TYPE_ZONE:	      zone-like device, e.g., open channel SSD and SMR HDD (using ZBC/ZAC)
+ * @PD_DEV_TYPE_BLOCK_NVDIMM: Standard (non-streams) NVDIMM in sector mode.
+ */
+enum pd_devtype {
+	PD_DEV_TYPE_BLOCK_STREAM = 1,
+	PD_DEV_TYPE_BLOCK_STD,
+	PD_DEV_TYPE_FILE,
+	PD_DEV_TYPE_MEM,
+	PD_DEV_TYPE_ZONE,
+	PD_DEV_TYPE_BLOCK_NVDIMM,
+	PD_DEV_TYPE_LAST = PD_DEV_TYPE_BLOCK_NVDIMM,
+};
+
+_Static_assert((PD_DEV_TYPE_LAST < 256), "enum pd_devtype must fit in uint8_t");
+
+/**
+ * enum pd_state - Device states
+ * @PD_DEV_STATE_AVAIL:       Device is available
+ * @PD_DEV_STATE_UNAVAIL:     Device is unavailable
+ */
+enum pd_state {
+	PD_DEV_STATE_UNDEFINED = 0,
+	PD_DEV_STATE_AVAIL = 1,
+	PD_DEV_STATE_UNAVAIL = 2,
+	PD_DEV_STATE_LAST = PD_DEV_STATE_UNAVAIL,
+};
+
+_Static_assert((PD_DEV_STATE_LAST < 256), "enum pd_state must fit in uint8_t");
+
+/*
+ * pd API functions -- device-type independent dparm ops
+ */
+
+/*
+ * Error codes: All pd functions can return one or more of:
+ *
+ * -EINVAL    invalid fn args
+ * -EBADSLT   attempt to read or write a bad zone on a zone device
+ * -EIO       all other errors
+ */
+
+int pd_dev_open(const char *path, struct pd_dev_parm *dparm, struct pd_prop *pd_prop);
+int pd_dev_close(struct pd_dev_parm *dparm);
+int pd_dev_flush(struct pd_dev_parm *dparm);
+
+/**
+ * pd_bio_erase() -
+ * @pd:
+ * @zaddr:
+ * @zonecnt:
+ * @reads_erased: whether the data can be read post DISCARD
+ *
+ * Return:
+ */
+int pd_zone_erase(struct pd_dev_parm *dparm, u64 zaddr, u32 zonecnt, bool reads_erased);
+
+/*
+ * pd API functions - device dependent operations
+ */
+
+/**
+ * pd_zone_pwritev() -
+ * @pd:
+ * @iov:
+ * @iovcnt:
+ * @zaddr:
+ * @boff: offset in bytes from the start of "zaddr".
+ * @opflags:
+ *
+ * Return:
+ */
+int pd_zone_pwritev(struct pd_dev_parm *dparm, const struct kvec *iov,
+		    int iovcnt, u64 zaddr, loff_t boff, int opflags);
+
+/**
+ * pd_zone_pwritev_sync() -
+ * @pd:
+ * @iov:
+ * @iovcnt:
+ * @zaddr:
+ * @boff: Offset in bytes from the start of zaddr.
+ *
+ * Return:
+ */
+int pd_zone_pwritev_sync(struct pd_dev_parm *dparm, const struct kvec *iov,
+			 int iovcnt, u64 zaddr, loff_t boff);
+
+/**
+ * pd_zone_preadv() -
+ * @pd:
+ * @iov:
+ * @iovcnt:
+ * @zaddr: target zone for this I/O
+ * @boff:    byte offset into the target zone
+ *
+ * Return:
+ */
+int pd_zone_preadv(struct pd_dev_parm *dparm, const struct kvec *iov,
+		   int iovcnt, u64 zaddr, loff_t boff);
+
+void pd_dev_set_unavail(struct pd_dev_parm *dparm, struct omf_devparm_descriptor *omf_devparm);
+
+int pd_init(void) __cold;
+void pd_exit(void) __cold;
+
+#endif /* MPOOL_PD_H */
diff --git a/drivers/mpool/pmd.h b/drivers/mpool/pmd.h
new file mode 100644
index 000000000000..5fd6ca020fd1
--- /dev/null
+++ b/drivers/mpool/pmd.h
@@ -0,0 +1,379 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_PMD_H
+#define MPOOL_PMD_H
+
+#include <linux/atomic.h>
+#include <linux/rbtree.h>
+#include <linux/mutex.h>
+#include <linux/workqueue.h>
+#include <linux/spinlock.h>
+
+#include "mpool_ioctl.h"
+#include "omf_if.h"
+#include "pmd_obj.h"
+
+/**
+ * DOC: Module info.
+ *
+ * Pool metadata (pmd) module.
+ *
+ * Implements functions for mpool metadata management.
+ *
+ */
+
+struct mpool_descriptor;
+struct mpool_dev_info;
+struct mp_mdc;
+struct pmd_layout;
+struct mpool_config;
+
+/**
+ * DOC: Object lifecycle
+ *
+ * +) all mblock/mlog objects are owned by mpool layer users, excepting
+ *     mdc mlogs
+ * +) users are responsible for object lifecycle mgmt and must not violate it;
+ *    e.g. by using an object handle (layout pointer) after deleting that
+ *    object
+ * +) the mpool layer never independently aborts or deletes user objects
+ */
+
+/**
+ * DOC: Object ids
+ * Object ids for mblocks and mlogs are a unit64 of the form:
+ * <uniquifier (52-bits), type (4-bits), slot # (8 bits)>
+ *
+ */
+
+/**
+ * DOC: NOTES
+ * + metadata for a given object is stored in the mdc specified by slot #
+ * + uniquifiers are only guaranteed unique for a given slot #
+ * + metadata for all mdc (except mdc 0) are stored in mdc 0
+ * + mdc 0 is a distinguished container whose metadata is stored in superblocks
+ * + mdc 0 only stores object metadata for mdc 1-255
+ * + mdc N is implemented via mlogs with objids (2N, MLOG, 0) & (2N+1, MLOG, 0)
+ * + mdc 0 mlog objids are (0, MLOG, 0) and (1, MLOG, 0) where a slot # of 0
+ *   indicates the mlog metadata is stored in mdc 0 whereas it is actually in
+ *   superblocks; see comments in pmd_mdc0_init() for how we exploit this.
+ */
+
+/**
+ * struct pre_compact_ctrs - objects records counters, used for pre compaction of MDC1/255.
+ * @pcc_cr:   count of object create records
+ * @pcc_up:   count of object update records
+ * @pcc_del:  count of object delete records. If the object is shceduled for
+ *	deletion in the background, the counter is incremented (while the
+ *	delete record has not been written yet).
+ * @pcc_er:   count of object erase records
+ * @pcc_cobj: count of committed objects (and not deleted).
+ * @pcc_cap: In bytes, size of each mlog of the MDC
+ * @pcc_len: In bytes, how much is filled the active mlog.
+ *
+ * One such structure per mpool MDC.
+ *
+ * Locking:
+ *	Updates are serialized by the MDC compact lock.
+ *	The reads by the pre-compaction thread are done without holding any
+ *	lock. This is why atomic variables are used.
+ *	However because the variables are integers, the atomic read translates
+ *	into a simple load and the set translate in a simple store.
+ *
+ * The counters pcc_up, pcc_del, pcc_er are cleared at each compaction.
+ *
+ * Relaxed access is appropriate for all of these atomics
+ */
+struct pre_compact_ctrs {
+	atomic_t   pcc_cr;
+	atomic_t   pcc_up;
+	atomic_t   pcc_del;
+	atomic_t   pcc_er;
+	atomic_t   pcc_cobj;
+	atomic64_t pcc_cap;
+	atomic64_t pcc_len;
+};
+
+/**
+ * struct credit_info - mdc selector info
+ * @ci_credit:      available credit
+ * @ci_free:        available free space
+ * @ci_slot:        MDC slot number
+ *
+ * Contains information about available credit and a balance. Available
+ * credit is based on an rate at which records can be written to
+ * mdc such that all MDC will fill at the same time.
+ */
+struct credit_info  {
+	u64                 ci_credit;
+	u64                 ci_free;
+	u8                  ci_slot;
+};
+
+/**
+ * struct pmd_mdc_stats - per MDC space usage stats
+ * @pms_mblock_alen: mblock alloc len
+ * @pms_mblock_wlen: mblock write len
+ * @pms_mlog_alen: mlog alloc len
+ * @pms_mblock_cnt: mblock count
+ * @pms_mlog_cnt: mlog count
+ */
+struct pmd_mdc_stats {
+	u64    pms_mblock_alen;
+	u64    pms_mblock_wlen;
+	u64    pms_mlog_alen;
+	u32    pms_mblock_cnt;
+	u32    pms_mlog_cnt;
+};
+
+/**
+ * struct pmd_mdc_info - Metadata container (mdc) info.
+ * @mmi_compactlock: compaction lock
+ * @mmi_uc_lock:     uncommitted objects tree lock
+ * @mmi_uc_root:     uncommitted objects tree root
+ * @mmi_co_lock:     committed objects tree lock
+ * @mmi_co_root:     committed objects tree root
+ * @mmi_uqlock:      uniquifier lock
+ * @mmi_luniq:       uniquifier of last object assigned to container
+ * @mmi_mdc:         MDC implementing container
+ * @mmi_recbuf:      buffer for (un)packing log records
+ * @mmi_lckpt:       last objid checkpointed
+ * @mmi_stats:       per-MDC usage stats
+ * @mmi_stats_lock:  lock for protecting mmi_stats
+ * @mmi_pco_cnt:     counters used by the pre compaction of MDC1/255.
+ * @mmi_mdcver:      version of the mdc content on media when the mpool was
+ *                   activated. That may not be the current version on media
+ *                   if a MDC metadata conversion took place during activate.
+ * @mmi_credit       MDC credit info
+ *
+ * LOCKING:
+ * + mmi_luniq: protected by uqlock
+ * + mmi_mdc, recbuf, lckpt: protected by compactlock
+ * + mmi_co_root: protected by co_lock
+ * + mmi_uc_root: protected by uc_lock
+ * + mmi_stats: protected by mmi_stats_lock
+ * + mmi_pco_counters: updates serialized by mmi_compactlock
+ *
+ * NOTE:
+ *  + for mdc0 mmi_luniq is the slot # of the last mdc created
+ *  + logging to a mdc cannot execute concurrent with compacting
+ *    that mdc;
+ *    mmi_compactlock is used to enforce this
+ *  + compacting a mdc requires freezing both the list of committed
+ *    objects in that mdc and the metadata for those objects;
+ *    compactlock facilitates this in a way that avoids locking each
+ *    object during compaction; as a result object metadata updates
+ *    are serialized, but even without mdc compaction this would be
+ *    the case because all such metadata updates must be logged to
+ *    the object's mdc and mdc logging is inherently serial
+ *  + see struct pmd_layout comments for specifics on how
+ *    compactlock is used to freeze metadata for committed objects
+ */
+struct pmd_mdc_info {
+	struct mutex            mmi_compactlock;
+	char                   *mmi_recbuf;
+	u64                     mmi_lckpt;
+	struct mp_mdc          *mmi_mdc;
+
+	____cacheline_aligned
+	struct mutex            mmi_uc_lock;
+	struct rb_root          mmi_uc_root;
+
+	____cacheline_aligned
+	struct rw_semaphore     mmi_co_lock;
+	struct rb_root          mmi_co_root;
+
+	____cacheline_aligned
+	struct mutex            mmi_uqlock;
+	u64                     mmi_luniq;
+
+	____cacheline_aligned
+	struct credit_info      mmi_credit;
+	struct omf_mdcver       mmi_mdcver;
+
+	____cacheline_aligned
+	struct mutex            mmi_stats_lock;
+	struct pmd_mdc_stats    mmi_stats;
+
+	struct pre_compact_ctrs mmi_pco_cnt;
+};
+
+/**
+ * struct pmd_mdc_selector - Object containing MDC slots for allocation
+ * @mds_tbl_idx:      idx of the MDC slot selector in the mds_tbl
+ * @mds_tbl:          slot table used for MDC selection
+ * @mds_mdc:          scratch pad for sorting mdc by free size
+ *
+ * LOCKING:
+ *  + mdi_slotvlock lock will be taken to protect this object.
+ *
+ */
+struct pmd_mdc_selector {
+	atomic_t    mds_tbl_idx;
+	u8          mds_tbl[MDC_TBL_SZ];
+	void       *mds_smdc[MDC_SLOTS];
+};
+
+/**
+ * struct pmd_mda_info - Metadata container array (mda).
+ * @mdi_slotvlock:   it is assumed that this spinlock is NOT taken from interrupt context
+ * @mdi_slotvcnt:    number of active slotv entries
+ * @mdi_slotv:       per mdc info
+ * @mdi_sel:         MDC allocation selector
+ *
+ * LOCKING:
+ *  + mdi_slotvcnt: protected by mdi_slotvlock
+ *
+ * NOTE:
+ *  + mdi_slotvcnt only ever increases so mdi_slotv[x], x < mdi_slotvcnt, is
+ *    always active
+ *  + all mdi_slotv[] entries are initialized whether or not active so they
+ *    can all be accessed w/o locking except as required by pmd_mdc_info struct
+ */
+struct pmd_mda_info {
+	spinlock_t              mdi_slotvlock;
+	u16                     mdi_slotvcnt;
+
+	struct pmd_mdc_info     mdi_slotv[MDC_SLOTS];
+	struct pmd_mdc_selector mdi_sel;
+};
+
+/**
+ * struct pmd_obj_load_work - work struct for loading MDC 1~N
+ * @olw_work:     work struct
+ * @olw_mp:
+ * @olw_progress: Progress index. It is an (atomic_t *) so that multiple
+ *                pmd_obj_load_work structs can point to a single atomic_t
+ *                for grabbing the next MDC number to be processed.
+ * @olw_err:
+ */
+struct pmd_obj_load_work {
+	struct work_struct          olw_work;
+	struct mpool_descriptor    *olw_mp;
+	atomic_t                   *olw_progress; /* relaxed is correct */
+	atomic_t                   *olw_err;
+};
+
+/**
+ * pmd_mpool_activate() - Load all metadata for mpool mp.
+ * @mp:
+ * @mdc01:
+ * @mdc02:
+ * @create:
+ *
+ * Load all metadata for mpool mp; create flag indicates if is a new pool;
+ * caller must ensure no other thread accesses mp until activation is complete.
+ * note: pmd module owns mdc01/2 memory mgmt whether succeeds or fails
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_mpool_activate(struct mpool_descriptor *mp, struct pmd_layout *mdc01,
+		       struct pmd_layout *mdc02, int create);
+
+/**
+ * pmd_mpool_deactivate() - Deactivate mpool mp.
+ * @mp:
+ *
+ * Free all metadata for mpool mp excepting mp itself; caller must ensure
+ * no other thread can access mp during deactivation.
+ */
+void pmd_mpool_deactivate(struct mpool_descriptor *mp);
+
+/**
+ * pmd_mdc_alloc() - Add a metadata container to mpool.
+ * @mp:
+ * @mincap:
+ * @iter: the role of this parameter is to get the active mlogs of the mpool
+ *	MDCs uniformely spread on the mpool devices.
+ *	When pmd_mdc_alloc() is called in a loop to allocate several mpool MDCs,
+ *	iter should be incremented at each subsequent call.
+ *
+ * Add a metadata container (mdc) to mpool with a minimum capacity of mincap
+ * bytes.  Once added an mdc can never be deleted.
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_mdc_alloc(struct mpool_descriptor *mp, u64 mincap, u32 iter);
+
+/**
+ * pmd_mdc_cap() - Get metadata container (mdc) capacity stats.
+ * @mp:
+ * @mdcmax:
+ * @mdccap:
+ * @mdc0cap:
+ *
+ * Get metadata container (mdc) stats: count, aggregate capacity ex-mdc0 and
+ * mdc0 cap
+ */
+void pmd_mdc_cap(struct mpool_descriptor *mp, u64 *mdcmax, u64 *mdccap, u64 *mdc0cap);
+
+/**
+ * pmd_prop_mcconfig() -
+ * @mp:
+ * @pd:
+ * @compacting: if true, called by a compaction.
+ *
+ * Persist state (new or update) for drive pd; caller must hold mp.pdvlock
+ * if pd is an in-use member of mp.pdv.
+ *
+ * Locking: caller must hold MDC0 compact lock.
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_prop_mcconfig(struct mpool_descriptor *mp, struct mpool_dev_info *pd, bool compacting);
+
+/**
+ * pmd_prop_mcspare() -
+ * @mp:
+ * @mclassp:
+ * @spzone:
+ * @compacting: if true, called by a compaction.
+ *
+ * Persist spare zone info for drives in media class (new or update).
+ *
+ * Locking: caller must hold MDC0 compact lock.
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_prop_mcspare(struct mpool_descriptor *mp, enum mp_media_classp mclassp,
+		     u8 spzone, bool compacting);
+
+int pmd_prop_mpconfig(struct mpool_descriptor *mp, const struct mpool_config *cfg, bool compacting);
+
+/**
+ * pmd_precompact_start() - start MDC1/255 precompaction
+ * @mp:
+ */
+void pmd_precompact_start(struct mpool_descriptor *mp);
+
+/**
+ * pmd_precompact_stop() - stop MDC1/255 precompaction
+ * @mp:
+ */
+void pmd_precompact_stop(struct mpool_descriptor *mp);
+
+/**
+ * pmd_mdc_addrec_version() -add a version record in a mpool MDC.
+ * @mp:
+ * @cslot:
+ */
+int pmd_mdc_addrec_version(struct mpool_descriptor *mp, u8 cslot);
+
+int pmd_log_delete(struct mpool_descriptor *mp, u64 objid);
+
+int pmd_log_create(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+int pmd_log_erase(struct mpool_descriptor *mp, u64 objid, u64 gen);
+
+int pmd_log_idckpt(struct mpool_descriptor *mp, u64 objid);
+
+#define PMD_MDC0_COMPACTLOCK(_mp) \
+	pmd_mdc_lock(&((_mp)->pds_mda.mdi_slotv[0].mmi_compactlock), 0)
+
+#define PMD_MDC0_COMPACTUNLOCK(_mp) \
+	pmd_mdc_unlock(&((_mp)->pds_mda.mdi_slotv[0].mmi_compactlock))
+
+#endif /* MPOOL_PMD_H */
diff --git a/drivers/mpool/pmd_obj.h b/drivers/mpool/pmd_obj.h
new file mode 100644
index 000000000000..7cf5dea80f9d
--- /dev/null
+++ b/drivers/mpool/pmd_obj.h
@@ -0,0 +1,499 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_PMD_OBJ_H
+#define MPOOL_PMD_OBJ_H
+
+#include <linux/sort.h>
+#include <linux/rbtree.h>
+#include <linux/kref.h>
+#include <linux/rwsem.h>
+#include <linux/workqueue.h>
+
+#include "uuid.h"
+#include "mpool_ioctl.h"
+#include "omf_if.h"
+#include "mlog.h"
+
+struct mpool_descriptor;
+struct pmd_mdc_info;
+
+/*
+ * objid uniquifier checkpoint interval; used to avoid reissuing an outstanding
+ * objid after a crash; supports pmd_{mblock|mlog}_realloc()
+ */
+#define OBJID_UNIQ_POW2 8
+#define OBJID_UNIQ_DELTA (1 << OBJID_UNIQ_POW2)
+
+/* MDC_SLOTS is 256 [0,255] to fit in 8-bit slot field in objid.
+ */
+#define MDC_SLOTS           256
+#define MDC_TBL_SZ          (MDC_SLOTS * 4)
+
+#define UROOT_OBJID_LOG1 logid_make(0, 1)
+#define UROOT_OBJID_LOG2 logid_make(1, 1)
+#define UROOT_OBJID_MAX  1
+
+#define MDC0_OBJID_LOG1 logid_make(0, 0)
+#define MDC0_OBJID_LOG2 logid_make(1, 0)
+
+/**
+ * enum pmd_lock_class -
+ * @PMD_NONE:
+ * @PMD_OBJ_CLIENT:
+ *      For layout rwlock,
+ *              - Object id contains a non-zero slot number
+ * @PMD_MDC_NORMAL:
+ *      For layout rwlock,
+ *              - Object id contains a zero slot number AND
+ *              - Object id is neither of the well-known MDC-0 objids
+ *      For pmd_mdc_info.* locks,
+ *              - Array index of pmd_mda_info.slov[] is > 0.
+ * @PMD_MDC_ZERO:
+ *      For layout rwlock,
+ *              - Object id contains a zero slot number AND
+ *              - Object id is either of the well-known MDC-0 objids
+ *      For pmd_mdc_info.* locks,
+ *              - Array index of pmd_mda_info.slov[] is == 0.
+ *
+ * NOTE:
+ * - Object layout rw locks must be acquired before any MDC locks.
+ * - MDC-0 locks of a given class are below MDC-1/255 locks of those same
+ *   classes.
+ */
+enum pmd_lock_class {
+	PMD_NONE       = 0,
+	PMD_OBJ_CLIENT = 1,
+	PMD_MDC_NORMAL = 2,
+	PMD_MDC_ZERO   = 3,
+};
+
+/**
+ * enum pmd_obj_op -
+ * @PMD_OBJ_LOAD:
+ * @PMD_OBJ_ALLOC:
+ * @PMD_OBJ_COMMIT:
+ * @PMD_OBJ_ABORT:
+ * @PMD_OBJ_DELETE:
+ */
+enum pmd_obj_op {
+	PMD_OBJ_LOAD     = 1,
+	PMD_OBJ_ALLOC    = 2,
+	PMD_OBJ_COMMIT   = 3,
+	PMD_OBJ_ABORT    = 4,
+	PMD_OBJ_DELETE   = 5,
+};
+
+/**
+ * enum pmd_layout_state - object state flags
+ * @PMD_LYT_COMMITTED: object is committed to media
+ * @PMD_LYT_REMOVED:   object logically removed (aborted or deleted)
+ */
+enum pmd_layout_state {
+	PMD_LYT_COMMITTED  = 0x01,
+	PMD_LYT_REMOVED    = 0x02,
+};
+
+/**
+ * struct pmd_layout_mlpriv - mlog private data for pmd_layout
+ * @mlp_uuid:       unique ID per mlog
+ * @mlp_lstat:      mlog status
+ * @mlp_nodeoml:    "open mlog" rbtree linkage
+ */
+struct pmd_layout_mlpriv {
+	struct mpool_uuid   mlp_uuid;
+	struct rb_node      mlp_nodeoml;
+	struct mlog_stat    mlp_lstat;
+};
+
+/**
+ * union pmd_layout_priv - pmd_layout object type specific private data
+ * @mlpriv: mlog private data
+ */
+union pmd_layout_priv {
+	struct pmd_layout_mlpriv    mlpriv;
+};
+
+/**
+ * struct pmd_layout - object layout (in-memory version)
+ * @eld_nodemdc: rbtree node for uncommitted and committed objects
+ * @eld_objid:   object ID associated with layout
+ * @eld_mblen:   Amount of data written in the mblock in bytes (0 for mlogs)
+ * @eld_state:   enum pmd_layout_state
+ * @eld_flags:   enum mlog_open_flags for mlogs
+ * @eld_gen:     object generation
+ * @eld_ld:
+ * @eld_ref:     user ref count from alloc/get/put
+ * @eld_rwlock:  implements pmd_obj_*lock() for this layout
+ * @dle_mlpriv:  mlog private data
+ *
+ * LOCKING:
+ * + objid: constant; no locking required
+ * + lstat: lstat and *lstat are protected by pmd_obj_*lock()
+ * + all other fields: see notes
+ *
+ * NOTE:
+ * + committed object fields (other): to update hold pmd_obj_wrlock()
+ *   AND
+ *   compactlock for object's mdc; to read hold pmd_obj_*lock()
+ *   See the comments associated with struct pmd_mdc_info for
+ *   further details.
+ *
+ * eld_priv[] contains exactly one element if the object type
+ * is and mlog, otherwise it contains exactly zero element.
+ */
+struct pmd_layout {
+	struct rb_node                  eld_nodemdc;
+	u64                             eld_objid;
+	u32                             eld_mblen;
+	u8                              eld_state;
+	u8                              eld_flags;
+	u64                             eld_gen;
+	struct omf_layout_descriptor    eld_ld;
+
+	/* The above fields are read-mostly, while the
+	 * following two fields mutate frequently.
+	 */
+	struct kref                     eld_ref;
+	struct rw_semaphore             eld_rwlock;
+
+	union pmd_layout_priv           eld_priv[];
+};
+
+/* Shortcuts for mlog private data...
+ */
+#define eld_mlpriv      eld_priv->mlpriv
+#define eld_uuid        eld_mlpriv.mlp_uuid
+#define eld_lstat       eld_mlpriv.mlp_lstat
+#define eld_nodeoml     eld_mlpriv.mlp_nodeoml
+
+/**
+ * struct pmd_obj_capacity -
+ * @moc_captgt:  capacity target for object in bytes
+ * @moc_spare:   true, if alloc obj from spare space
+ */
+struct pmd_obj_capacity {
+	u64    moc_captgt;
+	bool   moc_spare;
+};
+
+/**
+ * struct pmd_obj_erase_work - workqueue job struct for object erase and free
+ * @oef_mp:             mpool
+ * @oef_layout:         object layout
+ * @oef_cache:          kmem cache to free work (or NULL)
+ * @oef_wqstruct:	workq struct
+ */
+struct pmd_obj_erase_work {
+	struct mpool_descriptor    *oef_mp;
+	struct pmd_layout          *oef_layout;
+	struct kmem_cache          *oef_cache;
+	struct work_struct          oef_wqstruct;
+};
+
+/**
+ * struct mdc_csm_info - mdc credit set member info
+ * @m_slot:      mdc slot number
+ * @ci_credit:   available credit
+ */
+struct mdc_csm_info {
+	u8   m_slot;
+	u16  m_credit;
+};
+
+/**
+ * struct mdc_credit_set - mdc credit set
+ * @cs_idx:      index of current credit set member
+ * @cs_num_csm:  number of credit set members in this credit set
+ * @cs_csm:      array of credit set members
+ */
+struct mdc_credit_set {
+	u8                    cs_idx;
+	u8                    cs_num_csm;
+	struct mdc_csm_info   csm[MPOOL_MDC_SET_SZ];
+};
+
+/**
+ * pmd_obj_alloc() - Allocate an object.
+ * @mp:
+ * @otype:
+ * @ocap:
+ * @mclassp: media class
+ * @layoutp:
+ *
+ * Allocate object of type otype with parameters and capacity as specified
+ * by ocap on drives in media class mclassp providing a minimum capacity of
+ * mincap bytes; if successful returns object layout.
+ *
+ * Note:
+ * Object is not persistent until committed; allocation can be aborted.
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_obj_alloc(struct mpool_descriptor *mp, enum obj_type_omf otype,
+		  struct pmd_obj_capacity *ocap, enum mp_media_classp mclassp,
+		  struct pmd_layout **layoutp);
+
+
+/**
+ * pmd_obj_realloc() - Re-allocate an object.
+ * @mp:
+ * @objid:
+ * @ocap:
+ * @mclassp: media class
+ * @layoutp:
+ *
+ * Allocate object with specified objid to support crash recovery; otherwise
+ * is equivalent to pmd_obj_alloc(); if successful returns object layout.
+ *
+ * Note:
+ * Object is not persistent until committed; allocation can be aborted.
+ *
+ * Return: %0 if successful; -errno otherwise
+ */
+int pmd_obj_realloc(struct mpool_descriptor *mp, u64 objid, struct pmd_obj_capacity *ocap,
+		    enum mp_media_classp mclassp, struct pmd_layout **layoutp);
+
+
+/**
+ * pmd_obj_commit() - Commit an object.
+ * @mp:
+ * @layout:
+ *
+ * Make allocated object persistent; if fails object remains uncommitted so
+ * can retry commit or abort; object cannot be committed while in erasing or
+ * aborting state; caller MUST NOT hold pmd_obj_*lock() on layout.
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_obj_commit(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+/**
+ * pmd_obj_abort() - Discard un-committed object.
+ * @mp:
+ * @layout:
+ *
+ * Discard uncommitted object; caller MUST NOT hold pmd_obj_*lock() on
+ * layout; if successful layout is invalid after call.
+ *
+ * Return: %0 if successful; -errno otherwise
+ */
+int pmd_obj_abort(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+/**
+ * pmd_obj_delete() - Delete committed object.
+ * @mp:
+ * @layout:
+ *
+ * Delete committed object; caller MUST NOT hold pmd_obj_*lock() on layout;
+ * if successful layout is invalid.
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_obj_delete(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+/**
+ * pmd_obj_erase() - Log erase for object and set state flag and generation number
+ * @mp:
+ * @layout:
+ * @gen:
+ *
+ * Object must be in committed state; caller MUST hold pmd_obj_wrlock() on layout.
+ *
+ * Return: %0 if successful, -errno otherwise
+ */
+int pmd_obj_erase(struct mpool_descriptor *mp, struct pmd_layout *layout, u64 gen);
+
+/**
+ * pmd_obj_find_get() - Get a reference for a layout for objid.
+ * @mp:
+ * @objid:
+ * @which:
+ *
+ * Get layout for object with specified objid; return NULL either if not found
+ *
+ * Return: pointer to layout if successful, NULL otherwise
+ */
+struct pmd_layout *pmd_obj_find_get(struct mpool_descriptor *mp, u64 objid, int which);
+
+/**
+ * pmd_obj_rdlock() - Read-lock object layout with appropriate nesting level.
+ * @layout:
+ */
+void pmd_obj_rdlock(struct pmd_layout *layout);
+
+/**
+ * pmd_obj_rdunlock() - Release read lock on object layout.
+ * @layout:
+ */
+void pmd_obj_rdunlock(struct pmd_layout *layout);
+
+/**
+ * pmd_obj_wrlock() - Write-lock object layout with appropriate nesting level.
+ * @layout:
+ */
+void pmd_obj_wrlock(struct pmd_layout *layout);
+
+/**
+ * pmd_obj_wrunlock() - Release write lock on object layout.
+ * @layout:
+ */
+void pmd_obj_wrunlock(struct pmd_layout *layout);
+
+/**
+ * pmd_init_credit() - udpates available credit and setup mdc selector table
+ * @mp: mpool object
+ *
+ * Lock: No Lock required
+ *
+ * Used to initialize credit when new MDCs are added and add the mds to
+ * available
+ * credit list.
+ */
+void pmd_update_credit(struct mpool_descriptor *mp);
+
+/**
+ * pmd_mpool_usage() - calculate per-mpool space usage
+ * @mp:
+ * @usage:
+ */
+void pmd_mpool_usage(struct mpool_descriptor *mp, struct mpool_usage *usage);
+
+/**
+ * pmd_precompact_alsz() - Inform MDC1/255 pre-compacting about the active
+ *	mlog of an mpool MDCi 0<i<=255.
+ *	The size and how much is used are passed in.
+ *	"alsz" stands for active mlog size.
+ * @mp:
+ * @objid: objid of the active mlog of the mpool MDCi
+ * @len: In bytes, how much of the active mlog is used.
+ * @cap: In bytes, size of the active mlog.
+ */
+void pmd_precompact_alsz(struct mpool_descriptor *mp, u64 objid, u64 len, u64 cap);
+
+/**
+ * pmd_layout_alloc() - create and initialize an pmd_layout
+ * @objid:  mblock/mlog object ID
+ * @gen:    generation number
+ * @mblen:  mblock written length
+ * @zcnt:   number of zones in a strip
+ *
+ * Alloc and init object layout; non-arg fields and all strip descriptor
+ * fields are set to 0/UNDEF/NONE; no auxiliary object info is allocated.
+ *
+ * Return: NULL if allocation fails.
+ */
+struct pmd_layout *pmd_layout_alloc(struct mpool_uuid *uuid, u64 objid,
+				    u64 gen, u64 mblen, u32 zcnt);
+
+/**
+ * pmd_layout_release() - free pmd_layout and internal elements
+ * @layout:
+ *
+ * Deallocate all memory associated with object layout.
+ *
+ * Return: void
+ */
+void pmd_layout_release(struct kref *refp);
+
+int pmd_layout_rw(struct mpool_descriptor *mp, struct pmd_layout *layout,
+		  const struct kvec *iov, int iovcnt, u64 boff, int flags, u8 rw);
+
+struct mpool_dev_info *pmd_layout_pd_get(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+u64 pmd_layout_cap_get(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+int pmd_layout_erase(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+int pmd_obj_alloc_cmn(struct mpool_descriptor *mp, u64 objid, enum obj_type_omf otype,
+		      struct pmd_obj_capacity *ocap, enum mp_media_classp mclass,
+		      int realloc, bool needref, struct pmd_layout **layoutp);
+
+void pmd_update_obj_stats(struct mpool_descriptor *mp, struct pmd_layout *layout,
+			  struct pmd_mdc_info *cinfo, enum pmd_obj_op op);
+
+void pmd_obj_rdlock(struct pmd_layout *layout);
+void pmd_obj_rdunlock(struct pmd_layout *layout);
+
+void pmd_obj_wrlock(struct pmd_layout *layout);
+void pmd_obj_wrunlock(struct pmd_layout *layout);
+
+void pmd_co_rlock(struct pmd_mdc_info *cinfo, u8 slot);
+void pmd_co_runlock(struct pmd_mdc_info *cinfo);
+
+struct pmd_layout *pmd_co_find(struct pmd_mdc_info *cinfo, u64 objid);
+struct pmd_layout *pmd_co_insert(struct pmd_mdc_info *cinfo, struct pmd_layout *layout);
+struct pmd_layout *pmd_co_remove(struct pmd_mdc_info *cinfo, struct pmd_layout *layout);
+
+int pmd_smap_insert(struct mpool_descriptor *mp, struct pmd_layout *layout);
+
+int pmd_init(void) __cold;
+void pmd_exit(void) __cold;
+
+static inline bool objtype_user(enum obj_type_omf otype)
+{
+	return (otype == OMF_OBJ_MBLOCK || otype == OMF_OBJ_MLOG);
+}
+
+static inline u64 objid_make(u64 uniq, enum obj_type_omf otype, u8 cslot)
+{
+	return ((uniq << 12) | ((otype & 0xF) << 8) | (cslot & 0xFF));
+}
+
+static inline u64 objid_uniq(u64 objid)
+{
+	return (objid >> 12);
+}
+
+static inline u8 objid_slot(u64 objid)
+{
+	return (objid & 0xFF);
+}
+
+static inline bool objid_ckpt(u64 objid)
+{
+	return !(objid_uniq(objid) & (OBJID_UNIQ_DELTA - 1));
+}
+
+static inline u64 logid_make(u64 uniq, u8 cslot)
+{
+	return objid_make(uniq, OMF_OBJ_MLOG, cslot);
+}
+
+static inline bool objid_mdc0log(u64 objid)
+{
+	return ((objid == MDC0_OBJID_LOG1) || (objid == MDC0_OBJID_LOG2));
+}
+
+static inline enum obj_type_omf pmd_objid_type(u64 objid)
+{
+	enum obj_type_omf otype = objid_type(objid);
+
+	return objtype_valid(otype) ? otype : OMF_OBJ_UNDEF;
+}
+
+/* True if objid is an mpool user object (versus mpool metadata object). */
+static inline bool pmd_objid_isuser(u64 objid)
+{
+	return objtype_user(objid_type(objid)) && objid_slot(objid);
+}
+
+static inline void pmd_obj_put(struct pmd_layout *layout)
+{
+	kref_put(&layout->eld_ref, pmd_layout_release);
+}
+
+/* General mdc locking (has external callers...) */
+static inline void pmd_mdc_lock(struct mutex *lock, u8 slot)
+{
+	mutex_lock_nested(lock, slot > 0 ? PMD_MDC_NORMAL : PMD_MDC_ZERO);
+}
+
+static inline void pmd_mdc_unlock(struct mutex *lock)
+{
+	mutex_unlock(lock);
+}
+
+#endif /* MPOOL_PMD_OBJ_H */
diff --git a/drivers/mpool/sb.h b/drivers/mpool/sb.h
new file mode 100644
index 000000000000..673a5f742f7c
--- /dev/null
+++ b/drivers/mpool/sb.h
@@ -0,0 +1,162 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_SB_PRIV_H
+#define MPOOL_SB_PRIV_H
+
+#include "mpool_ioctl.h"
+
+struct pd_dev_parm;
+struct omf_sb_descriptor;
+struct pd_prop;
+
+/*
+ * Drives have 2 superblocks.
+ * + sb0 at byte offset 0
+ * + sb1 at byte offset SB_AREA_SZ
+ *
+ * Read: sb0 is the authoritative copy, other copies are not used.
+ * Updates: sb0 is updated first; if successful sb1 is updated
+ */
+/* Number of superblock per Physical Device.  */
+#define SB_SB_COUNT        2
+
+/*
+ * Size in byte of the area occupied by a superblock. The superblock itself
+ * may be smaller, but always starts at the beginning of its area.
+ */
+#define SB_AREA_SZ        (4096ULL)
+
+/*
+ * Size in byte of an area located just after the superblock areas.
+ * Not used in 1.0. Later can be used for MDC0 metadata and/or voting sets.
+ */
+#define MDC0MD_AREA_SZ    (4096ULL)
+
+/*
+ * sb API functions
+ */
+
+/**
+ * sb_magic_check() - check for sb magic value
+ * @dparm: struct pd_dev_parm *
+ *
+ * Determine if the mpool magic value exists in at least one place where
+ * expected on drive pd.  Does NOT imply drive has a valid superblock.
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Return: 1 if found, 0 if not found, -(errno) if error reading
+ */
+int sb_magic_check(struct pd_dev_parm *dparm);
+
+/**
+ * sb_write_new() - write superblock to new drive
+ * @dparm: struct pd_dev_parm *
+ * @sb: struct omf_sb_descriptor *
+ *
+ * Write superblock sb to new (non-pool) drive
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Return: 0 if successful; -errno otherwise
+ */
+int sb_write_new(struct pd_dev_parm *dparm, struct omf_sb_descriptor *sb);
+
+/**
+ * sb_write_update() - update superblock
+ * @dparm: "dparm" info is not used to fill up the super block, only "sb" content is used.
+ * @sb: "sb" content is written in the super block.
+ *
+ * Update superblock on pool drive
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Return: 0 if successful; -errno otherwise
+ */
+int sb_write_update(struct pd_dev_parm *dparm, struct omf_sb_descriptor *sb);
+
+/**
+ * sb_erase() - erase superblock
+ * @dparm: struct pd_dev_parm *
+ *
+ * Erase superblock on drive pd.
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Return: 0 if successful; -errno otherwise
+ */
+int sb_erase(struct pd_dev_parm *dparm);
+
+/**
+ * sb_read() - read superblock
+ * @dparm: struct pd_dev_parm *
+ * @sb: struct omf_sb_descriptor *
+ * @omf_ver: omf sb version
+ * @force:
+ *
+ * Read superblock from drive pd; make repairs as necessary.
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Return: 0 if successful; -errno otherwise
+ */
+int sb_read(struct pd_dev_parm *dparm, struct omf_sb_descriptor *sb, u16 *omf_ver, bool force);
+
+/**
+ * sbutil_mdc0_clear() - clear mdc0 of superblock
+ * @sb: struct omf_sb_descriptor *)
+ *
+ * Clear (set to zeros) mdc0 portion of sb.
+ *
+ * Return: void
+ */
+void sbutil_mdc0_clear(struct omf_sb_descriptor *sb);
+
+/**
+ * sbutil_mdc0_isclear() - Test if mdc0 is clear
+ * @sb: struct omf_sb_descriptor *
+ *
+ * Return: 1 if mdc0 portion of sb is clear.
+ */
+int sbutil_mdc0_isclear(struct omf_sb_descriptor *sb);
+
+/**
+ * sbutil_mdc0_copy() - copy mdc0 from one superblock to another
+ * @tgtsb: struct omf_sb_descriptor *
+ * @srcsb: struct omf_sb_descriptor *
+ *
+ * Copy mdc0 portion of srcsb to tgtsb.
+ *
+ * Return void
+ */
+void sbutil_mdc0_copy(struct omf_sb_descriptor *tgtsb, struct omf_sb_descriptor *srcsb);
+
+/**
+ * sbutil_mdc0_isvalid() - validate mdc0 of a superblock
+ * @sb: struct omf_sb_descriptor *
+ *
+ * Validate mdc0 portion of sb and extract mdparm.
+ * Return: 1 if valid and mdparm set; 0 otherwise.
+ */
+int sbutil_mdc0_isvalid(struct omf_sb_descriptor *sb);
+
+/**
+ * sb_zones_for_sbs() - compute how many zones are needed to contain the superblocks.
+ * @pd_prop:
+ */
+static inline u32 sb_zones_for_sbs(struct pd_prop *pd_prop)
+{
+	u32 zonebyte;
+
+	zonebyte = pd_prop->pdp_zparam.dvb_zonepg << PAGE_SHIFT;
+
+	return (2 * (SB_AREA_SZ + MDC0MD_AREA_SZ) + (zonebyte - 1)) / zonebyte;
+}
+
+int sb_init(void) __cold;
+void sb_exit(void) __cold;
+
+#endif /* MPOOL_SB_PRIV_H */
diff --git a/drivers/mpool/smap.h b/drivers/mpool/smap.h
new file mode 100644
index 000000000000..b9b72d3182c6
--- /dev/null
+++ b/drivers/mpool/smap.h
@@ -0,0 +1,334 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_SMAP_H
+#define MPOOL_SMAP_H
+
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+#include <linux/rbtree.h>
+#include <linux/workqueue.h>
+
+#include "mpool_ioctl.h"
+
+/* Forward Decls */
+struct mpool_usage;
+struct mpool_devprops;
+struct mc_smap_parms;
+struct mpool_descriptor;
+
+/*
+ * Common defs
+ */
+
+/**
+ * struct rmbkt - region map bucket
+ */
+struct rmbkt {
+	struct mutex    pdi_rmlock;
+	struct rb_root  pdi_rmroot;
+} ____cacheline_aligned;
+
+/**
+ * struct smap_zone -
+ * @smz_node:
+ * @smz_key:
+ * @smz_value:
+ */
+struct smap_zone {
+	struct rb_node  smz_node;
+	u64             smz_key;
+	u64             smz_value;
+};
+
+/**
+ * enum smap_space_type - space allocation policy flag
+ * @SMAP_SPC_UNDEF:
+ * @SMAP_SPC_USABLE_ONLY:    allocate from usable space only
+ * @SMAP_SPC_USABLE_2_SPARE: allocate from usable space first then spare
+ *                          if needed
+ * @SMAP_SPC_SPARE_ONLY:     allocate from spare space only
+ * @SMAP_SPC_SPARE_2_USABLE: allocate from spare space first then usable
+ *                          if needed
+ */
+enum smap_space_type {
+	SMAP_SPC_UNDEF           = 0,
+	SMAP_SPC_USABLE_ONLY     = 1,
+	SMAP_SPC_USABLE_2_SPARE  = 2,
+	SMAP_SPC_SPARE_ONLY      = 3,
+	SMAP_SPC_SPARE_2_USABLE  = 4
+};
+
+static inline int saptype_valid(enum smap_space_type saptype)
+{
+	return (saptype && saptype <= 4);
+}
+
+/*
+ * drive allocation info
+ *
+ * LOCKING:
+ * + rgnsz, rgnladdr: constants; no locking required
+ * + all other fields: protected by dalock
+ */
+
+/**
+ * struct smap_dev_alloc -
+ * @sda_dalock:
+ * @sda_rgnsz:    number of zones per rgn, excepting last
+ * @sda_rgnladdr: address of first zone in last rgn
+ * @sda_rgnalloc: rgn last alloced from
+ * @sda_zoneeff:    total zones (zonetot) minus bad zones
+ * @sda_utgt:      target max usable zones to allocate
+ * @sda_uact:      actual usable zones allocated
+ * @sda_stgt:      target max spare zones to allocate
+ * @sda_sact       actual spare zones allocated
+ *
+ * NOTE:
+ * + must maintain invariant that sact <= stgt
+ * + however it is possible for uact > utgt due to changing % spare
+ *   zones or zone failures.  this condition corrects when
+ *   sufficient space is freed or if % spare zones is changed
+ *   (again).
+ *
+ * Capacity pools and calcs:
+ * + total zones = zonetot
+ * + avail zones = zoneeff
+ * + usable zones = utgt which is (zoneeff * (1 - spzone/100))
+ * + free usable zones = max(0, utgt - uact); max handles uact > utgt
+ * + used zones = uact; possible for used > usable (uact > utgt)
+ * + spare zones = stgt which is (zoneeff - utgt)
+ * + free spare zones = (stgt - sact); guaranteed that sact <= stgt
+ */
+struct smap_dev_alloc {
+	spinlock_t sda_dalock;
+	u32        sda_rgnsz;
+	u32        sda_rgnladdr;
+	u32        sda_rgnalloc;
+	u32        sda_zoneeff;
+	u32        sda_utgt;
+	u32        sda_uact;
+	u32        sda_stgt;
+	u32        sda_sact;
+};
+
+struct smap_dev_znstats {
+	u64    sdv_total;
+	u64    sdv_avail;
+	u64    sdv_usable;
+	u64    sdv_fusable;
+	u64    sdv_spare;
+	u64    sdv_fspare;
+	u64    sdv_used;
+};
+
+/**
+ * smap_usage_work - delayed work struct for checking mpool free usable space usage
+ * @smapu_wstruct:
+ * @smapu_mp:
+ * @smapu_freepct: free space %
+ */
+struct smap_usage_work {
+	struct delayed_work             smapu_wstruct;
+	struct mpool_descriptor        *smapu_mp;
+	int                             smapu_freepct;
+};
+
+/*
+ * smap API functions
+ */
+
+/*
+ * Return: all smap fns can return -errno with the following errno values
+ * on failure:
+ * + -EINVAL = invalid fn args
+ * + -ENOSPC = unable to allocate requested space
+ * + -ENOMEM = insufficient memory to complete operation
+ */
+
+/*
+ * smap API usage notes:
+ * + During mpool activation call smap_insert() for all existing objects
+ *   before calling smap_alloc() or smap_free().
+ */
+
+/**
+ * smap_mpool_init() - initialize the smaps for an initialized mpool_descriptor
+ * @mp: struct mpool_descriptor *
+ *
+ * smap_mpool_init must be called once per mpool as it is being activated.
+ *
+ * Init space maps for all drives in mpool that are empty except for
+ * superblocks; caller must ensure no other thread can access mp.
+ *
+ * TODO: Traversing smap rbtrees may need fix, since there may be unsafe
+ * erases within loops.
+ *
+ * Return:
+ * 0 if successful, -errno with the following errno values on failure:
+ * -EINVAL if spare zone percentage is > 100%,
+ * -EINVAL if rgn count is 0, or
+ * -EINVAL if zonecnt on one of the drives is < rgn count
+ * -ENOMEM if there is no memory available
+ */
+int smap_mpool_init(struct mpool_descriptor *mp);
+
+/**
+ * smap_mpool_free() - free smap structures in a mpool_descriptor
+ * @mp: struct mpool_descriptor *
+ *
+ * Free space maps for all drives in mpool; caller must ensure no other
+ * thread can access mp.
+ *
+ * Return: void
+ */
+void smap_mpool_free(struct mpool_descriptor *mp);
+
+/**
+ * smap_mpool_usage() - present stats of smap usage
+ * @mp: struct mpool_descriptor *
+ * @mclass: media class or MP_MED_ALL for all classes
+ * @usage: struct mpool_usage *
+ *
+ * Fill in stats with space usage for media class; if MP_MED_ALL
+ * report on all media classes; caller must hold mp.pdvlock.
+ *
+ * Locking: the caller should hold the pds_pdvlock at least in read to
+ *	    be protected against media classes updates.
+ */
+void smap_mpool_usage(struct mpool_descriptor *mp, u8 mclass, struct mpool_usage *usage);
+
+/**
+ * smap_drive_spares() - Set percentage of zones to set aside as spares
+ * @mp: struct mpool_descriptor *
+ * @mclassp: media class
+ * @spzone: percentage of zones to use as spares
+ *
+ * Set percent spare zones to spzone for drives in media class mclass;
+ * caller must hold mp.pdvlock.
+ *
+ * Locking: the caller should hold the pds_pdvlock at least in read to
+ *	    be protected against media classes updates.
+ *
+ * Return: 0 if successful; -errno otherwise
+ */
+int smap_drive_spares(struct mpool_descriptor *mp, enum mp_media_classp mclassp, u8 spzone);
+
+/**
+ * smap_drive_usage() - Fill in a given drive's portion of dprop struct.
+ * @mp:    struct mpool_descriptor *
+ * @pdh:   drive number within the mpool_descriptor
+ * @dprop: struct mpool_devprops *, structure to fill in
+ *
+ * Fill in usage portion of dprop for drive pdh; caller must hold mp.pdvlock
+ *
+ * Return: 0 if successful, -errno otherwise
+ */
+int smap_drive_usage(struct mpool_descriptor *mp, u16 pdh, struct mpool_devprops *dprop);
+
+/**
+ * smap_drive_init() - Initialize a specific drive within a mpool_descriptor
+ * @mp:    struct mpool_descriptor *
+ * @mcsp:  smap parameters
+ * @pdh:   u16, drive number within the mpool_descriptor
+ *
+ * Init space map for pool drive pdh that is empty except for superblocks
+ * with a percent spare zones of spzone; caller must ensure pdh is not in use.
+ *
+ * Return: 0 if successful, -errno otherwise
+ */
+int smap_drive_init(struct mpool_descriptor *mp, struct mc_smap_parms *mcsp, u16 pdh);
+
+/**
+ * smap_drive_free() - Release resources for a specific drive
+ * @mp:  struct mpool_descriptor *
+ * @pdh: u16, drive number within the mpool_descriptor
+ *
+ * Free space map for pool drive pdh including partial (failed) inits;
+ * caller must ensure pdh is not in use.
+ *
+ * Return: void
+ */
+void smap_drive_free(struct mpool_descriptor *mp, u16 pdh);
+
+/**
+ * smap_insert() - Inject an entry to an smap for existing object
+ * @mp: struct mpool_descriptor *
+ * @pdh: drive number within the mpool_descriptor
+ * @zoneaddr: starting zone for entry
+ * @zonecnt: number of zones in entry
+ *
+ * Add entry to space map for an existing object with a strip on drive pdh
+ * starting at zoneaddr and continuing for zonecnt blocks.
+ *
+ * Used, in part for superblocks.
+ *
+ * Return: 0 if successful, -errno otherwise
+ */
+int smap_insert(struct mpool_descriptor *mp, u16 pdh, u64 zoneaddr, u32 zonecnt);
+
+/**
+ * smap_alloc() - Allocate a new contiguous zone range on a specific drive
+ * @mp: struct mpool_descriptor
+ * @pdh: u16, drive number within the mpool_descriptor
+ * @zonecnt: u64, the number of zones requested
+ * @sapolicy: enum smap_space_type, usable only, spare only, etc.
+ * @zoneaddr: u64 *, the starting zone for the allocated range
+ * @align: no. of zones (must be a power-of-2)
+ *
+ * Attempt to allocate zonecnt contiguous zones on drive pdh
+ * in accordance with space allocation policy sapolicy.
+ *
+ * Return: 0 if succcessful; -errno otherwise
+ */
+int smap_alloc(struct mpool_descriptor *mp, u16 pdh, u64 zonecnt,
+	       enum smap_space_type sapolicy, u64 *zoneaddr, u64 align);
+
+/**
+ * smap_free() - Free a previously allocated range of zones in the smap
+ * @mp: struct mpool_descriptor *
+ * @pdh: u16, number of the disk within the mpool_descriptor
+ * @zoneaddr: u64, starting zone for the range to free
+ * @zonecnt: u16, the number of zones in the range
+ *
+ * Free currently allocated space starting at zoneaddr
+ * and continuing for zonecnt blocks.
+ *
+ * Return: 0 if successful, -errno otherwise
+ */
+int smap_free(struct mpool_descriptor *mp, u16 pdh, u64 zoneaddr, u16 zonecnt);
+
+/*
+ * smap internal functions
+ */
+
+/**
+ * smap_mpool_usage() - Get the media class usage for a given mclass.
+ * @mp:
+ * @mclass: if MP_MED_ALL, return the sum of the stats for all media class,
+ *	else the stats only for one media class.
+ * @usage: output
+ *
+ * Locking: the caller should hold the pds_pdvlock at least in read to
+ *	    be protected against media classes updates.
+ */
+void smap_mclass_usage(struct mpool_descriptor *mp, u8 mclass, struct mpool_usage *usage);
+
+/**
+ * smap_log_mpool_usage() - check drive mpool free usable space %, and log a message if needed
+ * @ws:
+ */
+void smap_log_mpool_usage(struct work_struct *ws);
+
+/**
+ * smap_wait_usage_done() - wait for periodical job for logging pd free usable space % to complete
+ * @mp:
+ */
+void smap_wait_usage_done(struct mpool_descriptor *mp);
+
+int smap_init(void) __cold;
+void smap_exit(void) __cold;
+
+#endif /* MPOOL_SMAP_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:21 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:21 -0500
Subject: [PATCH v2 07/22] mpool: add superblock management routines
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-8-nmeeramohide@micron.com>

Provides utilities to initialize, read, update, and erase mpool
superblocks.

Mpool stores two copies of superblocks, one at offset 0 and the
other at offset 8K in zone 0 of each media class volume. SB0 is
the authoritative copy and SB1 is used for recovery in the event
of corruption.

The superblock comprises the metadata required to uniquely identify
a media class volume, the name and UUID of the mpool to which this
volume belongs to, the superblock version, checksum and device
properties. The superblock on the capacity media class volume also
includes metadata for accessing the metadata container 0 (MDC-0).
MDC-0 is introduced in a future patch.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/init.c |   8 +
 drivers/mpool/sb.c   | 625 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 633 insertions(+)
 create mode 100644 drivers/mpool/sb.c

diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
index 70f907ccc28a..261ce67e94dd 100644
--- a/drivers/mpool/init.c
+++ b/drivers/mpool/init.c
@@ -10,6 +10,7 @@
 #include "omf_if.h"
 #include "pd.h"
 #include "smap.h"
+#include "sb.h"
 
 /*
  * Module params...
@@ -25,6 +26,7 @@ MODULE_PARM_DESC(chunk_size_kb, "Chunk size (in KiB) for device I/O");
 static void mpool_exit_impl(void)
 {
 	smap_exit();
+	sb_exit();
 	omf_exit();
 	pd_exit();
 }
@@ -46,6 +48,12 @@ static __init int mpool_init(void)
 		goto errout;
 	}
 
+	rc = sb_init();
+	if (rc) {
+		errmsg = "sb init failed";
+		goto errout;
+	}
+
 	rc = smap_init();
 	if (rc) {
 		errmsg = "smap init failed";
diff --git a/drivers/mpool/sb.c b/drivers/mpool/sb.c
new file mode 100644
index 000000000000..c161eff2bc0d
--- /dev/null
+++ b/drivers/mpool/sb.c
@@ -0,0 +1,625 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * Superblock module.
+ *
+ * Defines functions for managing per drive superblocks.
+ *
+ */
+
+#include <linux/slab.h>
+#include <linux/uio.h>
+
+#include "mpool_printk.h"
+#include "assert.h"
+
+#include "mpool_ioctl.h"
+#include "pd.h"
+#include "omf_if.h"
+#include "sb.h"
+#include "mclass.h"
+
+/* Cleared out sb */
+static struct omf_sb_descriptor SBCLEAR;
+
+/*
+ * Drives have 2 superblocks.
+ * + sb0 at byte offset 0
+ * + sb1 at byte offset SB_AREA_SZ + MDC0MD_AREA_SZ
+ *
+ * Read: sb0 is the authoritative copy, other copies are not used.
+ * Updates: sb0 is updated first; if successful sb1 is updated
+ */
+
+/*
+ * sb internal functions
+ */
+
+/**
+ * sb_prop_valid() - Validate the PD properties needed to read the erase superblocks.
+ *
+ * When the superblocks are read, the zone parameters may not been known
+ * yet. They may be obtained from the superblocks.
+ *
+ * Returns: true if we have enough to read the superblocks.
+ */
+static bool sb_prop_valid(struct pd_dev_parm *dparm)
+{
+	struct pd_prop *pd_prop = &dparm->dpr_prop;
+
+	if (SB_AREA_SZ < OMF_SB_DESC_PACKLEN) {
+
+		/* Guarantee that the SB area is large enough to hold an SB */
+		mp_pr_err("sb(%s): structure too big %lu %lu",
+			  -EINVAL, dparm->dpr_name, (ulong)SB_AREA_SZ, OMF_SB_DESC_PACKLEN);
+		return false;
+	}
+
+	if ((pd_prop->pdp_devtype != PD_DEV_TYPE_BLOCK_STD) &&
+	    (pd_prop->pdp_devtype != PD_DEV_TYPE_BLOCK_NVDIMM) &&
+	    (pd_prop->pdp_devtype != PD_DEV_TYPE_FILE)) {
+		mp_pr_err("sb(%s): unknown device type %d",
+			  -EINVAL, dparm->dpr_name, pd_prop->pdp_devtype);
+		return false;
+	}
+
+	if (PD_LEN(pd_prop) == 0) {
+		mp_pr_err("sb(%s): unknown device size", -EINVAL, dparm->dpr_name);
+		return false;
+	}
+
+	return true;
+};
+
+static u64 sb_idx2woff(u32 idx)
+{
+	return (u64)idx * (SB_AREA_SZ + MDC0MD_AREA_SZ);
+}
+
+/**
+ * sb_parm_valid() - Validate parameters passed to an SB API function
+ * @dparm: struct pd_dev_parm
+ *
+ * When this function is called it is assumed that the zone parameters of the
+ * PD are already known.
+ *
+ * Part of the validation is enforcing the rule from the comment above that
+ * there needs to be at least one more zone than those consumed by the
+ * (SB_SB_COUNT) superblocks.
+ *
+ * Returns: true if drive pd meets criteria for sb, false otherwise.
+ */
+static bool sb_parm_valid(struct pd_dev_parm *dparm)
+{
+	struct pd_prop *pd_prop = &dparm->dpr_prop;
+	u32 cnt;
+
+	if (SB_AREA_SZ < OMF_SB_DESC_PACKLEN) {
+		/* Guarantee that the SB area is large enough to hold an SB */
+		return false;
+	}
+
+	if (pd_prop->pdp_zparam.dvb_zonepg == 0) {
+		/* Zone size can't be 0. */
+		return false;
+	}
+
+	cnt = sb_zones_for_sbs(pd_prop);
+	if (cnt < 1) {
+		/* At least one zone is needed to hold SB0 and SB1. */
+		return false;
+	}
+
+	if (dparm->dpr_zonetot < (cnt + 1)) {
+		/* Guarantee that there is at least one zone not consumed by SBs. */
+		return false;
+	}
+
+	return true;
+};
+
+/*
+ * Write packed superblock in outbuf to sb copy number idx on drive pd.
+ * Returns: 0 if successful; -errno otherwise...
+ */
+static int sb_write_sbx(struct pd_dev_parm *dparm, char *outbuf, u32 idx)
+{
+	const struct kvec iov = { outbuf, SB_AREA_SZ };
+	u64 woff;
+	int rc;
+
+	woff = sb_idx2woff(idx);
+
+	rc = pd_zone_pwritev_sync(dparm, &iov, 1, 0, woff);
+	if (rc) {
+		mp_pr_err("sb(%s, %d): write failed, woff %lu",
+			  rc, dparm->dpr_name, idx, (ulong)woff);
+		return rc;
+	}
+
+	return 0;
+}
+
+/*
+ * Read packed superblock into inbuf from sb copy number idx.
+ * Returns: 0 if successful; -errno otherwise...
+ *
+ */
+static int sb_read_sbx(struct pd_dev_parm *dparm, char *inbuf, u32 idx)
+{
+	const struct kvec  iov = { inbuf, SB_AREA_SZ };
+	u64 woff;
+	int rc;
+
+	woff = sb_idx2woff(idx);
+
+	rc = pd_zone_preadv(dparm, &iov, 1, 0, woff);
+	if (rc) {
+		mp_pr_err("sb(%s, %d): read failed, woff %lu",
+			  rc, dparm->dpr_name, idx, (ulong)woff);
+		return rc;
+	}
+
+	return 0;
+}
+
+/*
+ * sb API functions
+ */
+
+_Static_assert(SB_AREA_SZ >= OMF_SB_DESC_PACKLEN, "sb_area_sz < omf_sb_desc_packlen");
+
+/*
+ * Determine if the mpool magic value exists in at least one place where
+ * expected on drive pd.  Does NOT imply drive has a valid superblock.
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Returns: 1 if found, 0 if not found, -(errno) if error reading
+ *
+ */
+int sb_magic_check(struct pd_dev_parm *dparm)
+{
+	int rval = 0, i;
+	char *inbuf;
+	int rc;
+
+	if (!sb_prop_valid(dparm)) {
+		rc = -EINVAL;
+		mp_pr_err("sb(%s): invalid param, zonepg %u zonetot %u",
+			  rc, dparm->dpr_name, dparm->dpr_zonepg, dparm->dpr_zonetot);
+		return rc;
+	}
+
+	inbuf = kmalloc_large(SB_AREA_SZ + 1, GFP_KERNEL);
+	if (!inbuf) {
+		rc = -ENOMEM;
+		mp_pr_err("sb(%s) magic check: buffer alloc failed", rc, dparm->dpr_name);
+		return rc;
+	}
+
+	for (i = 0; i < SB_SB_COUNT; i++) {
+		const struct kvec iov = { inbuf, SB_AREA_SZ };
+		u64 woff = sb_idx2woff(i);
+
+		memset(inbuf, 0, SB_AREA_SZ);
+
+		rc = pd_zone_preadv(dparm, &iov, 1, 0, woff);
+		if (rc) {
+			rval = rc;
+			mp_pr_err("sb(%s, %d) magic: read failed, woff %lu",
+				  rc, dparm->dpr_name, i, (ulong)woff);
+		} else if (omf_sb_has_magic_le(inbuf)) {
+			kfree(inbuf);
+			return 1;
+		}
+	}
+
+	kfree(inbuf);
+	return rval;
+}
+
+/*
+ * Write superblock sb to new (non-pool) drive
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Returns: 0 if successful; -errno otherwise...
+ *
+ */
+int sb_write_new(struct pd_dev_parm *dparm, struct omf_sb_descriptor *sb)
+{
+	char *outbuf;
+	int rc, i;
+
+	if (!sb_parm_valid(dparm)) {
+		rc = -EINVAL;
+		mp_pr_err("sb(%s) invalid param, zonepg %u zonetot %u",
+			  rc, dparm->dpr_name, dparm->dpr_zonepg, dparm->dpr_zonetot);
+		return rc;
+	}
+
+	outbuf = kmalloc_large(SB_AREA_SZ + 1, GFP_KERNEL);
+	if (!outbuf)
+		return -ENOMEM;
+
+	memset(outbuf, 0, SB_AREA_SZ);
+
+	rc = omf_sb_pack_htole(sb, outbuf);
+	if (rc) {
+		mp_pr_err("sb(%s) packing failed", rc, dparm->dpr_name);
+		kfree(outbuf);
+		return rc;
+	}
+
+	/*
+	 * since pd is not yet a pool member only succeed if write all sb
+	 * copies.
+	 */
+	for (i = 0; i < SB_SB_COUNT; i++) {
+		rc = sb_write_sbx(dparm, outbuf, i);
+		if (rc) {
+			mp_pr_err("sb(%s, %d): write sbx failed", rc, dparm->dpr_name, i);
+			break;
+		}
+	}
+
+	kfree(outbuf);
+	return rc;
+}
+
+/*
+ * Update superblock on pool drive
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Returns: 0 if successful; -errno otherwise..
+ *
+ */
+int sb_write_update(struct pd_dev_parm *dparm, struct omf_sb_descriptor *sb)
+{
+	char *outbuf;
+	int rc, i;
+
+	if (!sb_parm_valid(dparm)) {
+		rc = -EINVAL;
+		mp_pr_err("sb(%s) invalid param, zonepg %u zonetot %u partlen %lu",
+			  rc, dparm->dpr_name, dparm->dpr_zonepg, dparm->dpr_zonetot,
+			  (ulong)PD_LEN(&dparm->dpr_prop));
+		return rc;
+	}
+
+	outbuf = kmalloc_large(SB_AREA_SZ + 1, GFP_KERNEL);
+	if (!outbuf)
+		return -ENOMEM;
+
+	memset(outbuf, 0, SB_AREA_SZ);
+
+	rc = omf_sb_pack_htole(sb, outbuf);
+	if (rc) {
+		mp_pr_err("sb(%s) packing failed", rc, dparm->dpr_name);
+		kfree(outbuf);
+		return rc;
+	}
+
+	/* Update sb0 first and then sb1 if that is successful */
+	for (i = 0; i < SB_SB_COUNT; i++) {
+		rc = sb_write_sbx(dparm, outbuf, i);
+		if (rc) {
+			mp_pr_err("sb(%s, %d) sbx write failed", rc, dparm->dpr_name, i);
+			if (i == 0)
+				break;
+			rc = 0;
+		}
+	}
+
+	kfree(outbuf);
+
+	return rc;
+}
+
+/*
+ * Erase superblock on drive pd.
+ *
+ * Note: only pd properties must be set.
+ *
+ * Returns: 0 if successful; -errno otherwise...
+ *
+ */
+int sb_erase(struct pd_dev_parm *dparm)
+{
+	int rc = 0, i;
+	char *buf;
+
+	if (!sb_prop_valid(dparm)) {
+		rc = -EINVAL;
+		mp_pr_err("sb(%s) invalid param, zonepg %u zonetot %u", rc, dparm->dpr_name,
+			  dparm->dpr_zonepg, dparm->dpr_zonetot);
+		return rc;
+	}
+
+	buf = kmalloc_large(SB_AREA_SZ + 1, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	memset(buf, 0, SB_AREA_SZ);
+
+	for (i = 0; i < SB_SB_COUNT; i++) {
+		const struct kvec iov = { buf, SB_AREA_SZ };
+		u64 woff = sb_idx2woff(i);
+
+		rc = pd_zone_pwritev_sync(dparm, &iov, 1, 0, woff);
+		if (rc)
+			mp_pr_err("sb(%s, %d): erase failed", rc, dparm->dpr_name, i);
+	}
+
+	kfree(buf);
+
+	return rc;
+}
+
+static int sb_reconcile(struct omf_sb_descriptor *sb, struct pd_dev_parm *dparm, bool force)
+{
+	struct omf_devparm_descriptor *sb_parm = &sb->osb_parm;
+	struct pd_prop *pd_prop = &dparm->dpr_prop;
+	struct mc_parms mc_parms;
+	int rc;
+
+	pd_prop->pdp_mclassp = sb_parm->odp_mclassp;
+	pd_prop->pdp_zparam.dvb_zonepg = sb_parm->odp_zonepg;
+	pd_prop->pdp_zparam.dvb_zonetot = sb_parm->odp_zonetot;
+
+	if (force)
+		return 0;
+
+	if (pd_prop->pdp_devsz < sb_parm->odp_devsz) {
+		rc = -EINVAL;
+
+		mp_pr_err("sb(%s): devsz(%lu) > discovered (%lu)",
+			  rc, dparm->dpr_name, (ulong)sb_parm->odp_devsz,
+			  (ulong)pd_prop->pdp_devsz);
+		return rc;
+	}
+
+	if (PD_SECTORSZ(pd_prop) != sb_parm->odp_sectorsz) {
+		rc = -EINVAL;
+
+		mp_pr_err("sb(%s) sector size(%u) mismatches discovered(%u)",
+			  rc, dparm->dpr_name, sb_parm->odp_sectorsz, PD_SECTORSZ(pd_prop));
+		return rc;
+	}
+
+	if (pd_prop->pdp_devtype != sb_parm->odp_devtype) {
+		rc = -EINVAL;
+
+		mp_pr_err("sb(%s), pd type(%u) mismatches discovered(%u)",
+			  rc, dparm->dpr_name, sb_parm->odp_devtype, pd_prop->pdp_devtype);
+		return rc;
+	}
+
+	mc_pd_prop2mc_parms(pd_prop, &mc_parms);
+	if (mc_parms.mcp_features != sb_parm->odp_features) {
+		rc = -EINVAL;
+
+		mp_pr_err("sb(%s), pd features(%lu) mismatches discovered(%lu)",
+			  rc, dparm->dpr_name, (ulong)sb_parm->odp_features,
+			  (ulong)mc_parms.mcp_features);
+		return rc;
+	}
+
+	return 0;
+}
+
+/*
+ * Read superblock from drive pd.
+ *
+ * Note: only pd.status and pd.parm must be set; no other pd fields accessed.
+ *
+ * Returns: 0 if successful; -errno otherwise...
+ *
+ */
+int sb_read(struct pd_dev_parm *dparm, struct omf_sb_descriptor *sb, u16 *omf_ver, bool force)
+{
+	struct omf_sb_descriptor *sbtmp;
+	int rc = 0, i;
+	char *buf;
+
+	if (!sb_prop_valid(dparm)) {
+		rc = -EINVAL;
+		mp_pr_err("sb(%s) invalid parameter, zonepg %u zonetot %u",
+			  rc, dparm->dpr_name, dparm->dpr_zonepg, dparm->dpr_zonetot);
+		return rc;
+	}
+
+	sbtmp = kzalloc(sizeof(*sbtmp), GFP_KERNEL);
+	if (!sbtmp)
+		return -ENOMEM;
+
+	buf = kmalloc_large(SB_AREA_SZ + 1, GFP_KERNEL);
+	if (!buf) {
+		kfree(sbtmp);
+		return -ENOMEM;
+	}
+
+	/*
+	 * In 1.0, voting + SB gen numbers across the drive SBs is not used.
+	 * There is one authoritave replica that is SB0.
+	 * SB1 only used for debugging.
+	 */
+	for (i = 0; i < SB_SB_COUNT; i++) {
+		memset(buf, 0, SB_AREA_SZ);
+
+		rc = sb_read_sbx(dparm, buf, i);
+		if (rc)
+			mp_pr_err("sb(%s, %d) read sbx failed", rc, dparm->dpr_name, i);
+		else {
+			rc = omf_sb_unpack_letoh(sbtmp, buf, omf_ver);
+			if (rc)
+				mp_pr_err("sb(%s, %d) bad magic/version/cksum",
+					  rc, dparm->dpr_name, i);
+			else if (i == 0)
+				/* Deep copy to main struct  */
+				*sb = *sbtmp;
+		}
+		if (rc && (i == 0)) {
+			/*
+			 * SB0 has the authoritative replica of
+			 * MDC0 metadata. We need it.
+			 */
+			goto exit;
+		}
+	}
+
+	/*
+	 * Check that superblock SB0 is consistent and
+	 * update the PD properties from it.
+	 */
+	rc = sb_reconcile(sb, dparm, force);
+
+exit:
+	kfree(sbtmp);
+	kfree(buf);
+	return rc;
+}
+
+/*
+ * Clear (set to zeros) mdc0 portion of sb.
+ */
+void sbutil_mdc0_clear(struct omf_sb_descriptor *sb)
+{
+	sb->osb_mdc01gen = 0;
+	sb->osb_mdc01desc.ol_zcnt = 0;
+	mpool_uuid_clear(&sb->osb_mdc01uuid);
+
+	mpool_uuid_clear(&sb->osb_mdc01devid);
+	sb->osb_mdc01desc.ol_zaddr = 0;
+
+	sb->osb_mdc02gen = 0;
+	sb->osb_mdc02desc.ol_zcnt = 0;
+	mpool_uuid_clear(&sb->osb_mdc02uuid);
+
+	mpool_uuid_clear(&sb->osb_mdc02devid);
+	sb->osb_mdc02desc.ol_zaddr = 0;
+
+	mpool_uuid_clear(&sb->osb_mdc0dev.odp_devid);
+	sb->osb_mdc0dev.odp_zonetot = 0;
+	sb->osb_mdc0dev.odp_zonepg = 0;
+	sb->osb_mdc0dev.odp_mclassp = 0;
+	sb->osb_mdc0dev.odp_devtype = 0;
+	sb->osb_mdc0dev.odp_sectorsz = 0;
+	sb->osb_mdc0dev.odp_features = 0;
+}
+
+/*
+ * Copy mdc0 portion of srcsb to tgtsb.
+ */
+void sbutil_mdc0_copy(struct omf_sb_descriptor *tgtsb, struct omf_sb_descriptor *srcsb)
+{
+	tgtsb->osb_mdc01gen = srcsb->osb_mdc01gen;
+	mpool_uuid_copy(&tgtsb->osb_mdc01uuid, &srcsb->osb_mdc01uuid);
+	mpool_uuid_copy(&tgtsb->osb_mdc01devid, &srcsb->osb_mdc01devid);
+	tgtsb->osb_mdc01desc.ol_zcnt = srcsb->osb_mdc01desc.ol_zcnt;
+	tgtsb->osb_mdc01desc.ol_zaddr = srcsb->osb_mdc01desc.ol_zaddr;
+
+	tgtsb->osb_mdc02gen = srcsb->osb_mdc02gen;
+	mpool_uuid_copy(&tgtsb->osb_mdc02uuid, &srcsb->osb_mdc02uuid);
+	mpool_uuid_copy(&tgtsb->osb_mdc02devid, &srcsb->osb_mdc02devid);
+	tgtsb->osb_mdc02desc.ol_zcnt = srcsb->osb_mdc02desc.ol_zcnt;
+	tgtsb->osb_mdc02desc.ol_zaddr = srcsb->osb_mdc02desc.ol_zaddr;
+
+	mpool_uuid_copy(&tgtsb->osb_mdc0dev.odp_devid, &srcsb->osb_mdc0dev.odp_devid);
+	tgtsb->osb_mdc0dev.odp_devsz    = srcsb->osb_mdc0dev.odp_devsz;
+	tgtsb->osb_mdc0dev.odp_zonetot  = srcsb->osb_mdc0dev.odp_zonetot;
+	tgtsb->osb_mdc0dev.odp_zonepg   = srcsb->osb_mdc0dev.odp_zonepg;
+	tgtsb->osb_mdc0dev.odp_mclassp  = srcsb->osb_mdc0dev.odp_mclassp;
+	tgtsb->osb_mdc0dev.odp_devtype  = srcsb->osb_mdc0dev.odp_devtype;
+	tgtsb->osb_mdc0dev.odp_sectorsz = srcsb->osb_mdc0dev.odp_sectorsz;
+	tgtsb->osb_mdc0dev.odp_features = srcsb->osb_mdc0dev.odp_features;
+}
+
+/*
+ * Compare mdc0 portions of sb1 and sb2.
+ */
+static int sbutil_mdc0_eq(struct omf_sb_descriptor *sb1, struct omf_sb_descriptor *sb2)
+{
+	if (sb1->osb_mdc01gen != sb2->osb_mdc01gen ||
+	    sb1->osb_mdc01desc.ol_zcnt != sb2->osb_mdc01desc.ol_zcnt)
+		return 0;
+
+	if (mpool_uuid_compare(&sb1->osb_mdc01devid, &sb2->osb_mdc01devid) ||
+	    sb1->osb_mdc01desc.ol_zaddr != sb2->osb_mdc01desc.ol_zaddr)
+		return 0;
+
+	if (sb1->osb_mdc02gen != sb2->osb_mdc02gen ||
+	    sb1->osb_mdc02desc.ol_zcnt != sb2->osb_mdc02desc.ol_zcnt)
+		return 0;
+
+	if (mpool_uuid_compare(&sb1->osb_mdc02devid, &sb2->osb_mdc02devid) ||
+	    sb1->osb_mdc02desc.ol_zaddr != sb2->osb_mdc02desc.ol_zaddr)
+		return 0;
+
+	if (mpool_uuid_compare(&sb1->osb_mdc0dev.odp_devid, &sb2->osb_mdc0dev.odp_devid) ||
+	    sb1->osb_mdc0dev.odp_zonetot != sb2->osb_mdc0dev.odp_zonetot ||
+	    mc_cmp_omf_devparm(&sb1->osb_mdc0dev, &sb2->osb_mdc0dev))
+		return 0;
+
+	return 1;
+}
+
+/**
+ * sbutil_mdc0_isclear() - returns 1 if there is no MDC0 metadata in the
+ *	                   mdc0 portion of the super block.
+ * @sb:
+ *
+ * Some fields in the MDC0 portion of "sb" may not be 0 even if there is no
+ * MDC0 metadata present. It is due to metadata upgrade.
+ * Metadata upgrade may have to place a specific (non zero) value in a field
+ * that was not existing in a previous metadata version to indicate that
+ * the value is invalid.
+ */
+int sbutil_mdc0_isclear(struct omf_sb_descriptor *sb)
+{
+	return sbutil_mdc0_eq(&SBCLEAR, sb);
+}
+
+/*
+ * Validate mdc0 portion of sb
+ * Returns: 1 if valid; 0 otherwise.
+ */
+int sbutil_mdc0_isvalid(struct omf_sb_descriptor *sb)
+{
+	/* Basic consistency validation; can make more extensive as needed */
+
+	if (mpool_uuid_compare(&sb->osb_mdc01devid, &sb->osb_mdc02devid) ||
+	    mpool_uuid_compare(&sb->osb_mdc01devid, &sb->osb_mdc0dev.odp_devid))
+		return 0;
+
+	if (mpool_uuid_is_null(&sb->osb_mdc01devid))
+		return 0;
+
+	if (mpool_uuid_is_null(&sb->osb_parm.odp_devid))
+		return 0;
+
+	/* Confirm this drive is supposed to contain this mdc0 info */
+	if (mpool_uuid_compare(&sb->osb_mdc01devid, &sb->osb_parm.odp_devid))
+		return 0;
+
+	/* Found this drive in mdc0 strip list; confirm param and ownership */
+	if (mc_cmp_omf_devparm(&sb->osb_parm, &sb->osb_mdc0dev))
+		return 0;
+
+	return (sb->osb_mdc01desc.ol_zcnt == sb->osb_mdc02desc.ol_zcnt);
+}
+
+int sb_init(void)
+{
+	sbutil_mdc0_clear(&SBCLEAR);
+
+	return 0;
+}
+
+void sb_exit(void)
+{
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:23 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:23 -0500
Subject: [PATCH v2 09/22] mpool: add mblock lifecycle management and IO
 routines
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-10-nmeeramohide@micron.com>

Implements the mblock lifecycle management functions: allocate,
commit, abort, read, write, destroy etc.

Mblocks are containers comprising a linear sequence of bytes that
can be written exactly once, are immutable after writing, and can
be read in whole or in part as needed until deleted.

Mpool currently supports only fixed size mblocks whose size is the
same as the zone size and is established at mpool create.

Mblock API uses the metadata manager interface to reserve storage
space at allocation time, store metadata for the mblock in its
associated MDC-K when committing it, record end-of-life for the
mblock in its associated MDC-K when deleting it and read/write
mblock data.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mblock.c | 432 +++++++++++++++++++++++++++++++++++++++++
 drivers/mpool/mblock.h | 161 +++++++++++++++
 2 files changed, 593 insertions(+)
 create mode 100644 drivers/mpool/mblock.c
 create mode 100644 drivers/mpool/mblock.h

diff --git a/drivers/mpool/mblock.c b/drivers/mpool/mblock.c
new file mode 100644
index 000000000000..10c47ec74ff6
--- /dev/null
+++ b/drivers/mpool/mblock.c
@@ -0,0 +1,432 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * DOC: Module info.
+ *
+ * Mblock module.
+ *
+ * Defines functions for writing, reading, and managing the lifecycle
+ * of mblocks.
+ *
+ */
+
+#include <linux/vmalloc.h>
+#include <linux/blk_types.h>
+#include <linux/mm.h>
+
+#include "mpool_printk.h"
+#include "assert.h"
+
+#include "pd.h"
+#include "pmd_obj.h"
+#include "mpcore.h"
+#include "mblock.h"
+
+/**
+ * mblock2layout() - convert opaque mblock handle to pmd_layout
+ *
+ * This function converts the opaque handle (mblock_descriptor) used by
+ * clients to the internal representation (pmd_layout).  The
+ * conversion is a simple cast, followed by a sanity check to verify the
+ * layout object is an mblock object.  If the validation fails, a NULL
+ * pointer is returned.
+ */
+static struct pmd_layout *mblock2layout(struct mblock_descriptor *mbh)
+{
+	struct pmd_layout *layout = (void *)mbh;
+
+	if (!layout)
+		return NULL;
+
+	ASSERT(layout->eld_objid > 0);
+	ASSERT(kref_read(&layout->eld_ref) >= 2);
+
+	return mblock_objid(layout->eld_objid) ? layout : NULL;
+}
+
+static u32 mblock_optimal_iosz_get(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct mpool_dev_info *pd = pmd_layout_pd_get(mp, layout);
+
+	return pd->pdi_optiosz;
+}
+
+/**
+ * layout2mblock() - convert pmd_layout to opaque mblock_descriptor
+ *
+ * This function converts the internally used pmd_layout to
+ * the externally used opaque mblock_descriptor.
+ */
+static struct mblock_descriptor *layout2mblock(struct pmd_layout *layout)
+{
+	return (struct mblock_descriptor *)layout;
+}
+
+static void mblock_getprops_cmn(struct mpool_descriptor *mp, struct pmd_layout *layout,
+				struct mblock_props *prop)
+{
+	struct mpool_dev_info *pd;
+
+	pd = pmd_layout_pd_get(mp, layout);
+
+	prop->mpr_objid = layout->eld_objid;
+	prop->mpr_alloc_cap = pmd_layout_cap_get(mp, layout);
+	prop->mpr_write_len = layout->eld_mblen;
+	prop->mpr_optimal_wrsz = mblock_optimal_iosz_get(mp, layout);
+	prop->mpr_mclassp = pd->pdi_mclass;
+	prop->mpr_iscommitted = layout->eld_state & PMD_LYT_COMMITTED;
+}
+
+static int mblock_alloc_cmn(struct mpool_descriptor *mp, u64 objid,
+			    enum mp_media_classp mclassp, bool spare,
+			    struct mblock_props *prop, struct mblock_descriptor **mbh)
+{
+	struct pmd_obj_capacity ocap = { .moc_spare = spare };
+	struct pmd_layout *layout = NULL;
+	int rc;
+
+	if (!mp)
+		return -EINVAL;
+
+	*mbh = NULL;
+
+	if (!objid) {
+		rc = pmd_obj_alloc(mp, OMF_OBJ_MBLOCK, &ocap, mclassp, &layout);
+		if (rc)
+			return rc;
+	} else {
+		rc = pmd_obj_realloc(mp, objid, &ocap, mclassp, &layout);
+		if (rc) {
+			if (rc != -ENOENT)
+				mp_pr_err("mpool %s, re-allocating mblock 0x%lx failed",
+					  rc, mp->pds_name, (ulong)objid);
+			return rc;
+		}
+	}
+
+	if (!layout)
+		return -ENOTRECOVERABLE;
+
+	if (prop) {
+		pmd_obj_rdlock(layout);
+		mblock_getprops_cmn(mp, layout, prop);
+		pmd_obj_rdunlock(layout);
+	}
+
+	*mbh = layout2mblock(layout);
+
+	return 0;
+}
+
+int mblock_alloc(struct mpool_descriptor *mp, enum mp_media_classp mclassp, bool spare,
+		 struct mblock_descriptor **mbh, struct mblock_props *prop)
+{
+	return mblock_alloc_cmn(mp, 0, mclassp, spare, prop, mbh);
+}
+
+int mblock_find_get(struct mpool_descriptor *mp, u64 objid, int which,
+		    struct mblock_props *prop, struct mblock_descriptor **mbh)
+{
+	struct pmd_layout *layout;
+
+	*mbh = NULL;
+
+	if (!mblock_objid(objid))
+		return -EINVAL;
+
+	layout = pmd_obj_find_get(mp, objid, which);
+	if (!layout)
+		return -ENOENT;
+
+	if (prop) {
+		pmd_obj_rdlock(layout);
+		mblock_getprops_cmn(mp, layout, prop);
+		pmd_obj_rdunlock(layout);
+	}
+
+	*mbh = layout2mblock(layout);
+
+	return 0;
+}
+
+void mblock_put(struct mblock_descriptor *mbh)
+{
+	struct pmd_layout *layout;
+
+	layout = mblock2layout(mbh);
+	if (layout)
+		pmd_obj_put(layout);
+}
+
+/*
+ * Helper function to log a message that many functions need to log:
+ */
+#define mp_pr_layout_not_found(_mp, _mbh)				\
+do {									\
+	static unsigned long state;					\
+	uint dly = msecs_to_jiffies(1000);				\
+									\
+	if (printk_timed_ratelimit(&state, dly)) {			\
+		mp_pr_warn("mpool %s, layout not found: mbh %p",	\
+			   (_mp)->pds_name, (_mbh));			\
+		dump_stack();						\
+	}								\
+} while (0)
+
+int mblock_commit(struct mpool_descriptor *mp, struct mblock_descriptor *mbh)
+{
+	struct mpool_dev_info *pd;
+	struct pmd_layout *layout;
+	int rc;
+
+	layout = mblock2layout(mbh);
+	if (!layout) {
+		mp_pr_layout_not_found(mp, mbh);
+		return -EINVAL;
+	}
+
+	pd = pmd_layout_pd_get(mp, layout);
+	if (!pd->pdi_fua) {
+		rc = pd_dev_flush(&pd->pdi_parm);
+		if (rc)
+			return rc;
+	}
+
+	/* Commit will fail with EBUSY if aborting flag set. */
+	rc = pmd_obj_commit(mp, layout);
+	if (rc) {
+		mp_pr_rl("mpool %s, committing mblock 0x%lx failed",
+			 rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	return 0;
+}
+
+int mblock_abort(struct mpool_descriptor *mp, struct mblock_descriptor *mbh)
+{
+	struct pmd_layout *layout;
+	int rc;
+
+	layout = mblock2layout(mbh);
+	if (!layout) {
+		mp_pr_layout_not_found(mp, mbh);
+		return -EINVAL;
+	}
+
+	rc = pmd_obj_abort(mp, layout);
+	if (rc) {
+		mp_pr_err("mpool %s, aborting mblock 0x%lx failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	return 0;
+}
+
+int mblock_delete(struct mpool_descriptor *mp, struct mblock_descriptor *mbh)
+{
+	struct pmd_layout *layout;
+
+	layout = mblock2layout(mbh);
+	if (!layout) {
+		mp_pr_layout_not_found(mp, mbh);
+		return -EINVAL;
+	}
+
+	return pmd_obj_delete(mp, layout);
+}
+
+/**
+ * mblock_rw_argcheck() - Validate mblock_write() and mblock_read().
+ * @mp:      Mpool descriptor
+ * @layout:  Layout of the mblock
+ * @iov:     iovec array
+ * @iovcnt:  iovec count
+ * @boff:    Byte offset into the layout.  Must be equal to layout->eld_mblen for write
+ * @rw:      MPOOL_OP_READ or MPOOL_OP_WRITE
+ * @len:     number of bytes in iov list
+ *
+ * Returns: 0 if successful, -errno otherwise
+ *
+ * Note: be aware that there are checks in this function that prevent illegal
+ * arguments in lower level functions (lower level functions should assert the
+ * requirements but not otherwise check them)
+ */
+static int mblock_rw_argcheck(struct mpool_descriptor *mp, struct pmd_layout *layout,
+			      loff_t boff, int rw, size_t len)
+{
+	u64 opt_iosz;
+	u32 mblock_cap;
+	int rc;
+
+	mblock_cap = pmd_layout_cap_get(mp, layout);
+	opt_iosz = mblock_optimal_iosz_get(mp, layout);
+
+	if (rw == MPOOL_OP_READ) {
+		/* boff must be a multiple of the OS page size */
+		if (!PAGE_ALIGNED(boff)) {
+			rc = -EINVAL;
+			mp_pr_err("mpool %s, read offset 0x%lx is not multiple of OS page size",
+				  rc, mp->pds_name, (ulong) boff);
+			return rc;
+		}
+
+		/* Check that boff is not past end of mblock capacity. */
+		if (mblock_cap <= boff) {
+			rc = -EINVAL;
+			mp_pr_err("mpool %s, read offset 0x%lx >= mblock capacity 0x%x",
+				  rc, mp->pds_name, (ulong)boff, mblock_cap);
+			return rc;
+		}
+
+		/*
+		 * Check that the request does not extend past the data
+		 * written.  Don't record an error if this appears to
+		 * be an mcache readahead request.
+		 *
+		 * TODO: Use (len != MCACHE_RA_PAGES_MAX)
+		 */
+		if (boff + len > layout->eld_mblen)
+			return -EINVAL;
+	} else {
+		/* Write boff required to match eld_mblen */
+		if (boff != layout->eld_mblen) {
+			rc = -EINVAL;
+			mp_pr_err("mpool %s write boff (%ld) != eld_mblen (%d)",
+				  rc, mp->pds_name, (ulong)boff, layout->eld_mblen);
+			return rc;
+		}
+
+		/* Writes must be optimal iosz aligned */
+		if (boff % opt_iosz) {
+			rc = -EINVAL;
+			mp_pr_err("mpool %s, write not optimal iosz aligned, offset 0x%lx",
+				  rc, mp->pds_name, (ulong)boff);
+			return rc;
+		}
+
+		/* Check for write past end of allocated space (!) */
+		if ((len + boff) > mblock_cap) {
+			rc = -EINVAL;
+			mp_pr_err("(write): len %lu + boff %lu > mblock_cap %lu",
+				  rc, (ulong)len, (ulong)boff, (ulong)mblock_cap);
+			return rc;
+		}
+	}
+
+	return 0;
+}
+
+int mblock_write(struct mpool_descriptor *mp, struct mblock_descriptor *mbh,
+		 const struct kvec *iov, int iovcnt, size_t len)
+{
+	struct pmd_layout *layout;
+	loff_t boff;
+	u8 state;
+	int rc;
+
+	layout = mblock2layout(mbh);
+	if (!layout) {
+		mp_pr_layout_not_found(mp, mbh);
+		return -EINVAL;
+	}
+
+	rc = mblock_rw_argcheck(mp, layout, layout->eld_mblen, MPOOL_OP_WRITE, len);
+	if (rc) {
+		mp_pr_debug("mblock write argcheck failed ", rc);
+		return rc;
+	}
+
+	if (len == 0)
+		return 0;
+
+	boff = layout->eld_mblen;
+
+	ASSERT(PAGE_ALIGNED(len));
+	ASSERT(PAGE_ALIGNED(boff));
+	ASSERT(iovcnt == (len >> PAGE_SHIFT));
+
+	pmd_obj_wrlock(layout);
+	state = layout->eld_state;
+	if (!(state & PMD_LYT_COMMITTED)) {
+		struct mpool_dev_info *pd = pmd_layout_pd_get(mp, layout);
+		int flags = 0;
+
+		if (pd->pdi_fua)
+			flags = REQ_FUA;
+
+		rc = pmd_layout_rw(mp, layout, iov, iovcnt, boff, flags, MPOOL_OP_WRITE);
+		if (!rc)
+			layout->eld_mblen += len;
+	}
+	pmd_obj_wrunlock(layout);
+
+	return !(state & PMD_LYT_COMMITTED) ? rc : -EALREADY;
+}
+
+int mblock_read(struct mpool_descriptor *mp, struct mblock_descriptor *mbh,
+		const struct kvec *iov, int iovcnt, loff_t boff, size_t len)
+{
+	struct pmd_layout *layout;
+	u8 state;
+	int rc;
+
+	layout = mblock2layout(mbh);
+	if (!layout) {
+		mp_pr_layout_not_found(mp, mbh);
+		return -EINVAL;
+	}
+
+	rc = mblock_rw_argcheck(mp, layout, boff, MPOOL_OP_READ, len);
+	if (rc) {
+		mp_pr_debug("mblock read argcheck failed ", rc);
+		return rc;
+	}
+
+	if (len == 0)
+		return 0;
+
+	ASSERT(PAGE_ALIGNED(len));
+	ASSERT(PAGE_ALIGNED(boff));
+	ASSERT(iovcnt == (len >> PAGE_SHIFT));
+
+	/*
+	 * Read lock the mblock layout; mblock reads can proceed concurrently;
+	 * Mblock writes are serialized but concurrent with reads
+	 */
+	pmd_obj_rdlock(layout);
+	state = layout->eld_state;
+	if (state & PMD_LYT_COMMITTED)
+		rc = pmd_layout_rw(mp, layout, iov, iovcnt, boff, 0, MPOOL_OP_READ);
+	pmd_obj_rdunlock(layout);
+
+	return (state & PMD_LYT_COMMITTED) ? rc : -EAGAIN;
+}
+
+int mblock_get_props_ex(struct mpool_descriptor *mp, struct mblock_descriptor *mbh,
+			struct mblock_props_ex *prop)
+{
+	struct pmd_layout *layout;
+
+	layout = mblock2layout(mbh);
+	if (!layout) {
+		mp_pr_layout_not_found(mp, mbh);
+		return -EINVAL;
+	}
+
+	pmd_obj_rdlock(layout);
+	prop->mbx_zonecnt = layout->eld_ld.ol_zcnt;
+	mblock_getprops_cmn(mp, layout, &prop->mbx_props);
+	pmd_obj_rdunlock(layout);
+
+	return 0;
+}
+
+bool mblock_objid(u64 objid)
+{
+	return objid && (pmd_objid_type(objid) == OMF_OBJ_MBLOCK);
+}
diff --git a/drivers/mpool/mblock.h b/drivers/mpool/mblock.h
new file mode 100644
index 000000000000..2a5435875a92
--- /dev/null
+++ b/drivers/mpool/mblock.h
@@ -0,0 +1,161 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * DOC: Module info.
+ *
+ * Defines functions for writing, reading, and managing the lifecycle of mlogs.
+ *
+ */
+
+#ifndef MPOOL_MBLOCK_H
+#define MPOOL_MBLOCK_H
+
+#include <linux/uio.h>
+
+#include "mpool_ioctl.h"
+/*
+ * Opaque handles for clients
+ */
+struct mpool_descriptor;
+struct mblock_descriptor;
+struct mpool_obj_layout;
+
+/*
+ * mblock API functions
+ */
+
+/**
+ * mblock_alloc() - Allocate an mblock.
+ * @mp:         mpool descriptor
+ * @capreq:     mblock capacity requested
+ * @mclassp:    media class
+ * @mbh:        mblock handle returned
+ * @prop:       mblock properties returned
+ *
+ * Allocate mblock with erasure code and capacity params as specified in
+ * ecparm and capreq on drives in a media class mclassp;
+ * if successful mbh is a handle for the mblock and prop contains its
+ * properties.
+ * Note: mblock is not persistent until committed; allocation can be aborted.
+ *
+ * Return: %0 if successful, -errno otherwise...
+ */
+int mblock_alloc(struct mpool_descriptor *mp, enum mp_media_classp mclassp, bool spare,
+		 struct mblock_descriptor **mbh, struct mblock_props *prop);
+
+/**
+ * mblock_find_get() - Get handle and properties for existing mblock with specified objid.
+ * @mp:
+ * @objid:
+ * @which:
+ * @prop:
+ * @mbh:
+ *
+ * If successful, the caller holds a ref on the mblock (which must be put eventually).
+ *
+ * Return: %0 if successful, -errno otherwise...
+ */
+int mblock_find_get(struct mpool_descriptor *mp, u64 objid, int which,
+		    struct mblock_props *prop, struct mblock_descriptor **mbh);
+
+/**
+ * mblock_put() - Put (release) a ref on an mblock
+ * @mbh:
+ *
+ * Put a ref on a known mblock.
+ *
+ * Return: %0 if successful, -errno otherwise...
+ */
+void mblock_put(struct mblock_descriptor *mbh);
+
+/**
+ * mblock_commit() - Make allocated mblock persistent
+ * @mp:
+ * @mbh:
+ *
+ * if fails mblock still exists in an
+ * uncommitted state so can retry commit or abort except as noted.
+ *
+ * Return: %0 if successful, -errno otherwise...
+ * EBUSY if must abort
+ */
+int mblock_commit(struct mpool_descriptor *mp, struct mblock_descriptor *mbh);
+
+/**
+ * mblock_abort() - Discard uncommitted mblock
+ * @mp:
+ * @mbh:
+ *
+ * If successful mbh is invalid after call.
+ *
+ * Return: %0 if successful, -errno otherwise...
+ *
+ */
+int mblock_abort(struct mpool_descriptor *mp, struct mblock_descriptor *mbh);
+
+/**
+ * mblock_delete() - Delete committed mblock
+ * @mp:
+ * @mbh:
+ *
+ * If successful mbh is invalid after call.
+ *
+ * Return: %0 if successful, -errno otherwise...
+ */
+int mblock_delete(struct mpool_descriptor *mp, struct mblock_descriptor *mbh);
+
+/**
+ * mblock_write() - Write iov to mblock
+ * @mp:
+ * @mbh:
+ * @iov:
+ * @iovcnt:
+ * @len:
+ *
+ * Mblocks can be written until they are committed, or
+ * until they are full.  If a caller needs to issue more than one write call
+ * to the same mblock, all but the last write call must be optimal write size aligned.
+ * The mpr_optimal_wrsz field in struct mblock_props gives the optimal write size.
+ *
+ * Return: %0 if success, -errno otherwise...
+ */
+int mblock_write(struct mpool_descriptor *mp, struct mblock_descriptor *mbh,
+		 const struct kvec *iov, int iovcnt, size_t len);
+
+/**
+ * mblock_read() - Read data from mblock mbnum in committed mblock into iov
+ * @mp:
+ * @mbh:
+ * @iov:
+ * @iovcnt:
+ * @boff:
+ * @len:
+ *
+ * Read data from mblock mbnum in committed mblock into iov starting at
+ * byte offset boff; boff and iov buffers must be a multiple of OS page
+ * size for the mblock.
+ *
+ * If fails can call mblock_get_props() to confirm mblock was written.
+ *
+ * Return: 0 if successful, -errno otherwise...
+ */
+int mblock_read(struct mpool_descriptor *mp, struct mblock_descriptor *mbh,
+		const struct kvec *iov, int iovcnt, loff_t boff, size_t len);
+
+/**
+ * mblock_get_props_ex() - Return extended mblock properties in prop
+ * @mp:
+ * @mbh:
+ * @prop:
+ *
+ * Return: %0 if successful, -errno otherwise...
+ */
+int mblock_get_props_ex(struct mpool_descriptor *mp, struct mblock_descriptor *mbh,
+			struct mblock_props_ex *prop);
+
+bool mblock_objid(u64 objid);
+
+#endif /* MPOOL_MBLOCK_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:20 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:20 -0500
Subject: [PATCH v2 06/22] mpool: add on-media pack, unpack and upgrade routines
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-7-nmeeramohide@micron.com>

This adds utilities to translate structures to and from their
on-media format. All mpool metadata is stored on media in
little-endian format.

The metadata records are both versioned and contains a record
type. This allows the record format to change over time, new
record types to be added and old record types to be deprecated.

All structs, enums, constants etc., representing on-media format
ends with a postfix "_omf". The functions for serializing
in-memory structures to on-media format are named with prefix
"omf_" and with postfix "_pack_htole". The corresponding
deserializing functions are named with postfix "_unpack_letoh".

The mpool metadata records are upgraded at mpool activation time.
The newer module code reads the metadata records created by an
older mpool version, converts them into the current in-memory
format and then serializes the in-memory format to the current
on-media format.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/init.c    |    8 +
 drivers/mpool/omf.c     | 1318 +++++++++++++++++++++++++++++++++++++++
 drivers/mpool/upgrade.c |  138 ++++
 3 files changed, 1464 insertions(+)
 create mode 100644 drivers/mpool/omf.c
 create mode 100644 drivers/mpool/upgrade.c

diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
index 031408815b48..70f907ccc28a 100644
--- a/drivers/mpool/init.c
+++ b/drivers/mpool/init.c
@@ -7,6 +7,7 @@
 
 #include "mpool_printk.h"
 
+#include "omf_if.h"
 #include "pd.h"
 #include "smap.h"
 
@@ -24,6 +25,7 @@ MODULE_PARM_DESC(chunk_size_kb, "Chunk size (in KiB) for device I/O");
 static void mpool_exit_impl(void)
 {
 	smap_exit();
+	omf_exit();
 	pd_exit();
 }
 
@@ -38,6 +40,12 @@ static __init int mpool_init(void)
 		goto errout;
 	}
 
+	rc = omf_init();
+	if (rc) {
+		errmsg = "omf init failed";
+		goto errout;
+	}
+
 	rc = smap_init();
 	if (rc) {
 		errmsg = "smap init failed";
diff --git a/drivers/mpool/omf.c b/drivers/mpool/omf.c
new file mode 100644
index 000000000000..8e612d35b19a
--- /dev/null
+++ b/drivers/mpool/omf.c
@@ -0,0 +1,1318 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * Pool on-drive format (omf) module.
+ *
+ * Defines:
+ * + on-drive format for mpool superblocks
+ * + on-drive formats for mlogs, mblocks, and metadata containers (mdc)
+ * + utility functions for working with these on-drive formats
+ *
+ * All mpool metadata is versioned and stored on media in little-endian format.
+ */
+
+#include <linux/slab.h>
+#include <crypto/hash.h>
+
+#include "assert.h"
+#include "mpool_printk.h"
+
+#include "upgrade.h"
+#include "pmd_obj.h"
+#include "mpcore.h"
+
+static struct crypto_shash *mpool_tfm;
+
+enum unpack_only {
+	UNPACKONLY    = 0,
+	UNPACKCONVERT = 1,
+};
+
+/*
+ * Forward declarations.
+ */
+static int omf_layout_unpack_letoh_v1(void *out, const char *inbuf);
+static int omf_dparm_unpack_letoh_v1(void *out, const char *inbuf);
+static int omf_mdcrec_mcspare_unpack_letoh_v1(void *out, const char *inbuf);
+static int omf_sb_unpack_letoh_v1(void *out, const char *inbuf);
+static int omf_pmd_layout_unpack_letoh_v1(void *out, const char *inbuf);
+
+/*
+ * layout_descriptor_table: track changes in OMF and in-memory layout descriptor
+ */
+static struct upgrade_history layout_descriptor_table[] = {
+	{
+		sizeof(struct omf_layout_descriptor),
+		omf_layout_unpack_letoh_v1,
+		NULL,
+		OMF_SB_DESC_V1,
+		{ {1, 0, 0, 0} }
+	},
+};
+
+/*
+ * devparm_descriptor_table: track changes in dev parm descriptor
+ */
+static struct upgrade_history devparm_descriptor_table[] = {
+	{
+		sizeof(struct omf_devparm_descriptor),
+		omf_dparm_unpack_letoh_v1,
+		NULL,
+		OMF_SB_DESC_V1,
+		{ {1, 0, 0, 0} }
+	},
+};
+
+/*
+ * mdcrec_data_mcspare_table: track changes in spare % record.
+ */
+static struct upgrade_history mdcrec_data_mcspare_table[]
+	= {
+	{
+		sizeof(struct omf_mdcrec_data),
+		omf_mdcrec_mcspare_unpack_letoh_v1,
+		NULL,
+		OMF_SB_DESC_UNDEF,
+		{ {1, 0, 0, 0} },
+	},
+};
+
+/*
+ * sb_descriptor_table: track changes in mpool superblock descriptor
+ */
+static struct upgrade_history sb_descriptor_table[] = {
+	{
+		sizeof(struct omf_sb_descriptor),
+		omf_sb_unpack_letoh_v1,
+		NULL,
+		OMF_SB_DESC_V1,
+		{ {1, 0, 0, 0} }
+	},
+};
+
+/*
+ * mdcrec_data_ocreate_table: track changes in OCREATE mdc record.
+ */
+static struct upgrade_history mdcrec_data_ocreate_table[]
+	= {
+	{
+		sizeof(struct omf_mdcrec_data),
+		omf_pmd_layout_unpack_letoh_v1,
+		NULL,
+		OMF_SB_DESC_UNDEF,
+		{ {1, 0, 0, 0} }
+	},
+};
+
+
+/*
+ * Generic routines
+ */
+
+/**
+ * omf_find_upgrade_hist() - Find the upgrade history entry for the given sb or mdc version
+ * @uhtab:
+ * @tabsz:   NELEM of upgrade_table
+ * @sbver:   superblock version
+ * @mdcver:  mdc version
+ *
+ * Given a superblock version or a mpool MDC content version, find the
+ * corresponding upgrade history entry which matches the given sb or mdc
+ * version.  That is the entry with the highest version such as
+ * entry version <= the version passed in.
+ *
+ * Note that caller of this routine can pass in either a valid superblock
+ * version or a valid mdc version. If a valid superblock version is passed in,
+ * mdcver need to be set to NULL. If a mdc version is passed in, sbver
+ * need to set to 0.
+ *
+ * For example,
+ * We update a structure "struct abc" three times, which is part of mpool
+ * superblock or MDC. when superblock version is  1, 3 and 5 respectively.
+ * Each time we add an entry in the upgrade table for this structure.
+ * The upgrade history table looks like:
+ *
+ * struct upgrade_history abc_hist[] =
+ * {{sizeof(struct abc_v1), abc_unpack_v1, NULL, OMF_SB_DESC_V1, NULL},
+ *  {sizeof(struct abc_v2), abc_unpack_v2, NULL, OMF_SB_DESC_V3, NULL},
+ *  {sizeof(struct abc_v3), abc_unpack_v3, NULL, OMF_SB_DESC_V5, NULL}}
+ *
+ * if caller needs to find the upgrade history entry matches
+ * sb version 3(OMF_SB_DESC_V3), this routine finds the exact match and
+ * returns &abc_hist[1].
+ *
+ * if caller needs to find the upgrade history entry which matches
+ * sb version 4 (OMF_SB_DESC_V4), since we don't update this structure
+ * in sb version 4, this routine finds the prior entry which matches
+ * the sb version 3, return &abc_hist[1]
+ *
+ */
+static struct upgrade_history *
+omf_find_upgrade_hist(struct upgrade_history *uhtab, size_t tabsz,
+		      enum sb_descriptor_ver_omf sbver, struct omf_mdcver *mdcver)
+{
+	struct upgrade_history *cur = NULL;
+	int beg = 0, end = tabsz, mid;
+
+	while (beg < end) {
+		mid = (beg + end) / 2;
+		cur = &uhtab[mid];
+		if (mdcver) {
+			ASSERT(sbver == 0);
+
+			if (omfu_mdcver_cmp(mdcver, "==", &cur->uh_mdcver))
+				return cur;
+			else if (omfu_mdcver_cmp(mdcver, ">", &cur->uh_mdcver))
+				beg = mid + 1;
+			else
+				end = mid;
+		} else {
+			ASSERT(sbver <= OMF_SB_DESC_VER_LAST);
+
+			if (sbver == cur->uh_sbver)
+				return cur;
+			else if (sbver > cur->uh_sbver)
+				beg = mid + 1;
+			else
+				end = mid;
+		}
+	}
+
+	if (end == 0)
+		return NULL; /* not found */
+
+	return &uhtab[end - 1];
+}
+
+/**
+ * omf_upgrade_convert_only() - Convert sb/mdc from v1 to v2 (v1 <= v2)
+ * @out:   v2 in-memory metadata structure
+ * @in:    v1 in-memory metadata structure
+ * @uhtab: upgrade history table for this structure
+ * @tabsz: NELEM(uhtab)
+ * @sbv1:  superblock version converting from
+ * @sbv2:  superblock version converting to
+ * @mdcv1: mdc version converting from
+ * @mdcv2: mdc version converting to
+ *
+ * Convert a nested metadata structure in mpool superblock or
+ * MDC from v1 to v2 (v1 <= v2)
+ *
+ * Note that callers can pass in either mdc beg/end ver (mdcv1/mdcv2),
+ * or superblock beg/end versions (sbv1/sbv2). Set both mdcv1 and
+ * mdcv2 to NULL, if caller wants to use superblock versions
+ */
+static __maybe_unused int
+omf_upgrade_convert_only(void *out, const void *in, struct upgrade_history *uhtab,
+			 size_t tabsz, enum sb_descriptor_ver_omf sbv1,
+			 enum sb_descriptor_ver_omf sbv2,
+			 struct omf_mdcver *mdcv1, struct omf_mdcver *mdcv2)
+{
+	struct upgrade_history *v1, *v2, *cur;
+	void *new, *old;
+	size_t newsz;
+
+	v1 = omf_find_upgrade_hist(uhtab, tabsz, sbv1, mdcv1);
+	ASSERT(v1);
+
+	v2 = omf_find_upgrade_hist(uhtab, tabsz, sbv2, mdcv2);
+	ASSERT(v2);
+	ASSERT(v1 <= v2);
+
+	if (v1 == v2)
+		/* No need to do conversion */
+		return 0;
+
+	if (v2 == v1 + 1) {
+		/*
+		 * Single step conversion, Don't need to allocate/free
+		 * buffers for intermediate conversion states
+		 */
+		if (v2->uh_conv != NULL)
+			v2->uh_conv(in, out);
+		return 0;
+	}
+
+	/*
+	 * Make a local copy of input buffer, we won't free it
+	 * in the for loop below
+	 */
+	old = kmalloc(v1->uh_size, GFP_KERNEL);
+	if (!old)
+		return -ENOMEM;
+	memcpy(old, in, v1->uh_size);
+
+	new = old;
+	newsz = v1->uh_size;
+
+	for (cur = v1 + 1; cur <= v2; cur++) {
+		if (!cur->uh_conv)
+			continue;
+		new = kzalloc(cur->uh_size, GFP_KERNEL);
+		if (!new) {
+			kfree(old);
+			return -ENOMEM;
+		}
+		newsz = cur->uh_size;
+		cur->uh_conv(old, new);
+		kfree(old);
+		old = new;
+	}
+
+	memcpy(out, new, newsz);
+	kfree(new);
+
+	return 0;
+}
+
+/**
+ * omf_upgrade_unpack_only() - unpack OMF meta data
+ * @out:     output buffer for in-memory structure
+ * @inbuf:   OMF structure
+ * @uhtab:   upgrade history table
+ * @tabsz:   NELEM of uhtab
+ * @sbver:   superblock version
+ * @mdcver: mpool MDC content version
+ */
+static int omf_upgrade_unpack_only(void *out, const char *inbuf, struct upgrade_history *uhtab,
+				   size_t tabsz, enum sb_descriptor_ver_omf sbver,
+				   struct omf_mdcver *mdcver)
+{
+	struct upgrade_history *res;
+
+	res = omf_find_upgrade_hist(uhtab, tabsz, sbver, mdcver);
+
+	return res->uh_unpack(out, inbuf);
+}
+
+/**
+ * omf_unpack_letoh_and_convert() - Unpack OMF meta data and convert it to the latest version.
+ * @out:    in-memory structure
+ * @outsz:  size of in-memory structure
+ * @inbuf:  OMF structure
+ * @uhtab:  upgrade history table
+ * @tabsz:  number of elements in uhtab
+ * @sbver:  superblock version
+ * @mdcver: mdc version. if set to NULL, use sbver to find the corresponding
+ *          nested structure upgrade table
+ */
+static int omf_unpack_letoh_and_convert(void *out, size_t outsz, const char *inbuf,
+					struct upgrade_history *uhtab,
+					size_t tabsz, enum sb_descriptor_ver_omf sbver,
+					struct omf_mdcver *mdcver)
+{
+	struct upgrade_history *cur, *omf;
+	void *old, *new;
+	size_t newsz;
+	int rc;
+
+	omf = omf_find_upgrade_hist(uhtab, tabsz, sbver, mdcver);
+	ASSERT(omf);
+
+	if (omf == &uhtab[tabsz - 1]) {
+		/*
+		 * Current version is the latest version.
+		 * Don't need to do any conversion
+		 */
+		return omf->uh_unpack(out, inbuf);
+	}
+
+	old = kzalloc(omf->uh_size, GFP_KERNEL);
+	if (!old)
+		return -ENOMEM;
+
+	rc = omf->uh_unpack(old, inbuf);
+	if (rc) {
+		kfree(old);
+		return rc;
+	}
+
+	new = old;
+	newsz = omf->uh_size;
+
+	for (cur = omf + 1; cur <= &uhtab[tabsz - 1]; cur++) {
+		if (!cur->uh_conv)
+			continue;
+		new = kzalloc(cur->uh_size, GFP_KERNEL);
+		if (!new) {
+			kfree(old);
+			return -ENOMEM;
+		}
+		newsz = cur->uh_size;
+		cur->uh_conv(old, new);
+		kfree(old);
+		old = new;
+	}
+
+	ASSERT(newsz == outsz);
+
+	memcpy(out, new, newsz);
+	kfree(new);
+
+	return 0;
+}
+
+_Static_assert(MPOOL_UUID_SIZE == OMF_UUID_PACKLEN, "mpool uuid sz != omf uuid packlen");
+_Static_assert(OMF_MPOOL_NAME_LEN == MPOOL_NAMESZ_MAX, "omf mpool name len != mpool namesz max");
+
+/*
+ * devparm_descriptor
+ */
+static void omf_dparm_pack_htole(struct omf_devparm_descriptor *dp, char *outbuf)
+{
+	struct devparm_descriptor_omf *dp_omf;
+
+	dp_omf = (struct devparm_descriptor_omf *)outbuf;
+	omf_set_podp_devid(dp_omf, dp->odp_devid.uuid, MPOOL_UUID_SIZE);
+	omf_set_podp_devsz(dp_omf, dp->odp_devsz);
+	omf_set_podp_zonetot(dp_omf, dp->odp_zonetot);
+	omf_set_podp_zonepg(dp_omf, dp->odp_zonepg);
+	omf_set_podp_mclassp(dp_omf, dp->odp_mclassp);
+	/* Translate pd_devtype into devtype_omf */
+	omf_set_podp_devtype(dp_omf, dp->odp_devtype);
+	omf_set_podp_sectorsz(dp_omf, dp->odp_sectorsz);
+	omf_set_podp_features(dp_omf, dp->odp_features);
+}
+
+/**
+ * omf_dparm_unpack_letoh()- unpack version 1 omf devparm descriptor into
+ *                            in-memory format
+ * @out: in-memory format
+ * @inbuf: omf format
+ */
+static int omf_dparm_unpack_letoh_v1(void *out, const char *inbuf)
+{
+	struct devparm_descriptor_omf *dp_omf;
+	struct omf_devparm_descriptor *dp;
+
+	dp_omf = (struct devparm_descriptor_omf *)inbuf;
+	dp = (struct omf_devparm_descriptor *)out;
+
+	omf_podp_devid(dp_omf, dp->odp_devid.uuid, MPOOL_UUID_SIZE);
+	dp->odp_devsz     = omf_podp_devsz(dp_omf);
+	dp->odp_zonetot    = omf_podp_zonetot(dp_omf);
+	dp->odp_zonepg = omf_podp_zonepg(dp_omf);
+	dp->odp_mclassp   = omf_podp_mclassp(dp_omf);
+	/* Translate devtype_omf into mp_devtype */
+	dp->odp_devtype	  = omf_podp_devtype(dp_omf);
+	dp->odp_sectorsz  = omf_podp_sectorsz(dp_omf);
+	dp->odp_features  = omf_podp_features(dp_omf);
+
+	return 0;
+}
+
+static int omf_dparm_unpack_letoh(struct omf_devparm_descriptor *dp, const char *inbuf,
+				  enum sb_descriptor_ver_omf sbver,
+				  struct omf_mdcver *mdcver,
+				  enum unpack_only unpackonly)
+{
+	const size_t sz = ARRAY_SIZE(layout_descriptor_table);
+	int rc;
+
+	if (unpackonly == UNPACKONLY)
+		rc = omf_upgrade_unpack_only(dp, inbuf, devparm_descriptor_table,
+					     sz, sbver, mdcver);
+	else
+		rc = omf_unpack_letoh_and_convert(dp, sizeof(*dp), inbuf, devparm_descriptor_table,
+						  sz, sbver, mdcver);
+
+	return rc;
+}
+
+
+/*
+ * layout_descriptor
+ */
+static void omf_layout_pack_htole(const struct omf_layout_descriptor *ld, char *outbuf)
+{
+	struct layout_descriptor_omf *ld_omf;
+
+	ld_omf = (struct layout_descriptor_omf *)outbuf;
+	omf_set_pol_zcnt(ld_omf, ld->ol_zcnt);
+	omf_set_pol_zaddr(ld_omf, ld->ol_zaddr);
+}
+
+/**
+ * omf_layout_unpack_letoh_v1() - unpack omf layout descriptor version 1
+ * @out: in-memory layout descriptor
+ * @in: on-media layout descriptor
+ */
+int omf_layout_unpack_letoh_v1(void *out, const char *inbuf)
+{
+	struct omf_layout_descriptor *ld;
+	struct layout_descriptor_omf *ld_omf;
+
+	ld = (struct omf_layout_descriptor *)out;
+	ld_omf = (struct layout_descriptor_omf *)inbuf;
+
+	ld->ol_zcnt = omf_pol_zcnt(ld_omf);
+	ld->ol_zaddr = omf_pol_zaddr(ld_omf);
+
+	return 0;
+}
+
+static int omf_layout_unpack_letoh(struct omf_layout_descriptor *ld, const char *inbuf,
+				   enum sb_descriptor_ver_omf sbver,
+				   struct omf_mdcver *mdcver,
+				   enum unpack_only unpackonly)
+{
+	const size_t sz = ARRAY_SIZE(layout_descriptor_table);
+	int rc;
+
+	if (unpackonly == UNPACKONLY)
+		rc = omf_upgrade_unpack_only(ld, inbuf, layout_descriptor_table,
+					     sz, sbver, mdcver);
+	else
+		rc = omf_unpack_letoh_and_convert(ld, sizeof(*ld), inbuf, layout_descriptor_table,
+						  sz, sbver, mdcver);
+
+	return rc;
+}
+
+/*
+ * pmd_layout
+ */
+static int omf_pmd_layout_pack_htole(const struct mpool_descriptor *mp, u8 rtype,
+				     struct pmd_layout *ecl, char *outbuf)
+{
+	struct mdcrec_data_ocreate_omf *ocre_omf;
+	int data_rec_sz;
+
+	if (rtype != OMF_MDR_OCREATE && rtype != OMF_MDR_OUPDATE) {
+		mp_pr_warn("mpool %s, wrong rec type %u packing layout", mp->pds_name, rtype);
+		return -EINVAL;
+	}
+
+	data_rec_sz = sizeof(*ocre_omf);
+
+	ocre_omf = (struct mdcrec_data_ocreate_omf *)outbuf;
+	omf_set_pdrc_rtype(ocre_omf, rtype);
+	omf_set_pdrc_mclass(ocre_omf, mp->pds_pdv[ecl->eld_ld.ol_pdh].pdi_mclass);
+	omf_set_pdrc_objid(ocre_omf, ecl->eld_objid);
+	omf_set_pdrc_gen(ocre_omf, ecl->eld_gen);
+	omf_set_pdrc_mblen(ocre_omf, ecl->eld_mblen);
+
+	if (objid_type(ecl->eld_objid) == OMF_OBJ_MLOG) {
+		memcpy(ocre_omf->pdrc_uuid, ecl->eld_uuid.uuid, OMF_UUID_PACKLEN);
+		data_rec_sz += OMF_UUID_PACKLEN;
+	}
+
+	omf_layout_pack_htole(&(ecl->eld_ld), (char *)&(ocre_omf->pdrc_ld));
+
+	return data_rec_sz;
+}
+
+/**
+ * omf_pmd_layout_unpack_letoh_v1() - Unpack little-endian mdc obj record and optional obj layout
+ * @out:
+ * @inbuf:
+ *
+ * For version 1 of OMF_MDR_OCREATE record (struct layout_descriptor_omf)
+ *
+ * Return:
+ *   0 if successful
+ *   -EINVAL if invalid record type or format
+ *   -ENOMEM if cannot alloc memory for metadata conversion
+ */
+static int omf_pmd_layout_unpack_letoh_v1(void *out, const char *inbuf)
+{
+	struct mdcrec_data_ocreate_omf *ocre_omf;
+	struct omf_mdcrec_data *cdr = out;
+	int rc;
+
+	ocre_omf = (struct mdcrec_data_ocreate_omf *)inbuf;
+
+	cdr->omd_rtype = omf_pdrc_rtype(ocre_omf);
+	if (cdr->omd_rtype != OMF_MDR_OCREATE && cdr->omd_rtype == OMF_MDR_OUPDATE) {
+		rc = -EINVAL;
+		mp_pr_err("Unpacking layout failed, wrong record type %d", rc, cdr->omd_rtype);
+		return rc;
+	}
+
+	cdr->u.obj.omd_mclass = omf_pdrc_mclass(ocre_omf);
+	cdr->u.obj.omd_objid = omf_pdrc_objid(ocre_omf);
+	cdr->u.obj.omd_gen   = omf_pdrc_gen(ocre_omf);
+	cdr->u.obj.omd_mblen = omf_pdrc_mblen(ocre_omf);
+
+	if (objid_type(cdr->u.obj.omd_objid) == OMF_OBJ_MLOG)
+		memcpy(cdr->u.obj.omd_uuid.uuid, ocre_omf->pdrc_uuid, OMF_UUID_PACKLEN);
+
+	rc = omf_layout_unpack_letoh(&cdr->u.obj.omd_old, (char *)&(ocre_omf->pdrc_ld),
+				     OMF_SB_DESC_V1, NULL, UNPACKONLY);
+
+	return rc;
+}
+
+
+/**
+ * omf_pmd_layout_unpack_letoh() - Unpack little-endian mdc obj record and optional obj layout
+ * @mp:
+ * @mdcver: version of the mpool MDC content being unpacked.
+ * @cdr: output
+ * @inbuf:
+ *
+ * Allocate object layout.
+ * For version 1 of OMF_MDR_OCREATE record (strut layout_descriptor_omf)
+ *
+ * Return:
+ *   0 if successful
+ *   -EINVAL if invalid record type or format
+ *   -ENOMEM if cannot alloc memory to return an object layout
+ *   -ENOENT if cannot convert a devid to a device handle (pdh)
+ */
+static int omf_pmd_layout_unpack_letoh(struct mpool_descriptor *mp, struct omf_mdcver *mdcver,
+				       struct omf_mdcrec_data *cdr, const char *inbuf)
+{
+	struct pmd_layout *ecl = NULL;
+	int rc, i;
+
+	rc = omf_unpack_letoh_and_convert(cdr, sizeof(*cdr), inbuf, mdcrec_data_ocreate_table,
+					  ARRAY_SIZE(mdcrec_data_ocreate_table),
+					  OMF_SB_DESC_UNDEF, mdcver);
+	if (rc) {
+		char buf[MAX_MDCVERSTR];
+
+		omfu_mdcver_to_str(mdcver, buf, sizeof(buf));
+		mp_pr_err("mpool %s, unpacking layout failed for mdc content version %s",
+			  rc, mp->pds_name, buf);
+		return rc;
+	}
+
+#ifdef COMP_PMD_ENABLED
+	ecl = pmd_layout_alloc(&cdr->u.obj.omd_uuid, cdr->u.obj.omd_objid, cdr->u.obj.omd_gen,
+			       cdr->u.obj.omd_mblen, cdr->u.obj.omd_old.ol_zcnt);
+#endif
+	if (!ecl) {
+		rc = -ENOMEM;
+		mp_pr_err("mpool %s, unpacking layout failed, could not allocate layout structure",
+			  rc, mp->pds_name);
+		return rc;
+	}
+
+	ecl->eld_ld.ol_zaddr = cdr->u.obj.omd_old.ol_zaddr;
+
+	for (i = 0; i < mp->pds_pdvcnt; i++) {
+		if (mp->pds_pdv[i].pdi_mclass == cdr->u.obj.omd_mclass) {
+			ecl->eld_ld.ol_pdh = i;
+			break;
+		}
+	}
+
+	if (i >= mp->pds_pdvcnt) {
+		pmd_obj_put(ecl);
+
+		rc = -ENOENT;
+		mp_pr_err("mpool %s, unpacking layout failed, mclass %u not in mpool",
+			  rc, mp->pds_name, cdr->u.obj.omd_mclass);
+		return rc;
+	}
+
+	cdr->u.obj.omd_layout = ecl;
+
+	return rc;
+}
+
+
+/*
+ * sb_descriptor
+ */
+
+/**
+ * omf_cksum_crc32c_le() - compute checksum
+ * @dbuf: data buf
+ * @dlen: data length
+ * @obuf: output buf
+ *
+ * Compute 4-byte checksum of type CRC32C for data buffer dbuf with length dlen
+ * and store in obuf little-endian; CRC32C is the only crypto algorithm we
+ * currently support.
+ *
+ * Return: 0 if successful, -EINVAL otherwise...
+ */
+static int omf_cksum_crc32c_le(const char *dbuf, u64 dlen, u8 *obuf)
+{
+	struct shash_desc *desc;
+	size_t descsz;
+	int rc;
+
+	memset(obuf, 0, 4);
+
+	descsz = sizeof(*desc) + crypto_shash_descsize(mpool_tfm);
+	desc = kzalloc(descsz, GFP_KERNEL);
+	if (!desc)
+		return -ENOMEM;
+	desc->tfm = mpool_tfm;
+
+	rc = crypto_shash_digest(desc, (u8 *)dbuf, dlen, obuf);
+
+	kfree(desc);
+
+	return rc;
+}
+
+struct omf_mdcver *omf_sbver_to_mdcver(enum sb_descriptor_ver_omf sbver)
+{
+	struct upgrade_history *uhtab;
+
+	uhtab = omf_find_upgrade_hist(sb_descriptor_table, ARRAY_SIZE(sb_descriptor_table),
+				      sbver, NULL);
+	if (uhtab) {
+		ASSERT(uhtab->uh_sbver == sbver);
+		return &uhtab->uh_mdcver;
+	}
+
+	return NULL;
+}
+
+int omf_sb_pack_htole(struct omf_sb_descriptor *sb, char *outbuf)
+{
+	struct sb_descriptor_omf *sb_omf;
+	u8 cksum[4];
+	int rc;
+
+	if (sb->osb_vers != OMF_SB_DESC_VER_LAST)
+		return -EINVAL; /* Not a valid header version */
+
+	sb_omf = (struct sb_descriptor_omf *)outbuf;
+
+	/* Pack drive-specific info */
+	omf_set_psb_magic(sb_omf, sb->osb_magic);
+	omf_set_psb_name(sb_omf, sb->osb_name, MPOOL_NAMESZ_MAX);
+	omf_set_psb_poolid(sb_omf, sb->osb_poolid.uuid, MPOOL_UUID_SIZE);
+	omf_set_psb_vers(sb_omf, sb->osb_vers);
+	omf_set_psb_gen(sb_omf, sb->osb_gen);
+
+	omf_dparm_pack_htole(&(sb->osb_parm), (char *)&(sb_omf->psb_parm));
+
+	omf_set_psb_mdc01gen(sb_omf, sb->osb_mdc01gen);
+	omf_set_psb_mdc01uuid(sb_omf, sb->osb_mdc01uuid.uuid, MPOOL_UUID_SIZE);
+	omf_layout_pack_htole(&(sb->osb_mdc01desc), (char *)&(sb_omf->psb_mdc01desc));
+	omf_set_psb_mdc01devid(sb_omf, sb->osb_mdc01devid.uuid, MPOOL_UUID_SIZE);
+
+	omf_set_psb_mdc02gen(sb_omf, sb->osb_mdc02gen);
+	omf_set_psb_mdc02uuid(sb_omf, sb->osb_mdc02uuid.uuid, MPOOL_UUID_SIZE);
+	omf_layout_pack_htole(&(sb->osb_mdc02desc), (char *)&(sb_omf->psb_mdc02desc));
+	omf_set_psb_mdc02devid(sb_omf, sb->osb_mdc02devid.uuid, MPOOL_UUID_SIZE);
+
+	outbuf = (char *)&sb_omf->psb_mdc0dev;
+	omf_dparm_pack_htole(&sb->osb_mdc0dev, outbuf);
+
+	/* Add CKSUM1 */
+	rc = omf_cksum_crc32c_le((char *)sb_omf, offsetof(struct sb_descriptor_omf, psb_cksum1),
+				 cksum);
+	if (rc)
+		return -EINVAL;
+
+	omf_set_psb_cksum1(sb_omf, cksum, 4);
+
+	/* Add CKSUM2 */
+	rc = omf_cksum_crc32c_le((char *)&sb_omf->psb_parm, sizeof(sb_omf->psb_parm), cksum);
+	if (rc)
+		return -EINVAL;
+
+	omf_set_psb_cksum2(sb_omf, cksum, 4);
+
+	return 0;
+}
+
+/**
+ * omf_sb_unpack_letoh_v1()- unpack version 1 omf sb descriptor into in-memory format
+ * @out: in-memory format
+ * @inbuf: omf format
+ */
+int omf_sb_unpack_letoh_v1(void *out, const char *inbuf)
+{
+	struct sb_descriptor_omf *sb_omf;
+	struct omf_sb_descriptor *sb;
+	u8 cksum[4], omf_cksum[4];
+	int rc;
+
+	sb_omf = (struct sb_descriptor_omf *)inbuf;
+	sb = (struct omf_sb_descriptor *)out;
+
+	/* Verify CKSUM2 */
+	rc = omf_cksum_crc32c_le((char *) &(sb_omf->psb_parm), sizeof(sb_omf->psb_parm), cksum);
+	omf_psb_cksum2(sb_omf, omf_cksum, 4);
+
+	if (rc || memcmp(cksum, omf_cksum, 4))
+		return -EINVAL;
+
+
+	sb->osb_magic = omf_psb_magic(sb_omf);
+
+	omf_psb_name(sb_omf, sb->osb_name, MPOOL_NAMESZ_MAX);
+
+	sb->osb_vers = omf_psb_vers(sb_omf);
+	ASSERT(sb->osb_vers == OMF_SB_DESC_V1);
+
+	omf_psb_poolid(sb_omf, sb->osb_poolid.uuid, MPOOL_UUID_SIZE);
+
+	sb->osb_gen = omf_psb_gen(sb_omf);
+	omf_dparm_unpack_letoh(&(sb->osb_parm), (char *)&(sb_omf->psb_parm),
+			       OMF_SB_DESC_V1, NULL, UNPACKONLY);
+
+	sb->osb_mdc01gen  = omf_psb_mdc01gen(sb_omf);
+	omf_psb_mdc01uuid(sb_omf, sb->osb_mdc01uuid.uuid, MPOOL_UUID_SIZE);
+	omf_layout_unpack_letoh(&(sb->osb_mdc01desc), (char *)&(sb_omf->psb_mdc01desc),
+				OMF_SB_DESC_V1, NULL, UNPACKONLY);
+	omf_psb_mdc01devid(sb_omf, sb->osb_mdc01devid.uuid, MPOOL_UUID_SIZE);
+
+	sb->osb_mdc02gen = omf_psb_mdc02gen(sb_omf);
+	omf_psb_mdc02uuid(sb_omf, sb->osb_mdc02uuid.uuid, MPOOL_UUID_SIZE);
+	omf_layout_unpack_letoh(&(sb->osb_mdc02desc), (char *)&(sb_omf->psb_mdc02desc),
+				OMF_SB_DESC_V1, NULL, UNPACKONLY);
+	omf_psb_mdc02devid(sb_omf, sb->osb_mdc02devid.uuid, MPOOL_UUID_SIZE);
+
+	inbuf = (char *)&sb_omf->psb_mdc0dev;
+	omf_dparm_unpack_letoh(&sb->osb_mdc0dev, inbuf, OMF_SB_DESC_V1, NULL, UNPACKONLY);
+
+	return 0;
+}
+
+int omf_sb_unpack_letoh(struct omf_sb_descriptor *sb, const char *inbuf, u16 *omf_ver)
+{
+	struct sb_descriptor_omf *sb_omf;
+	u8 cksum[4], omf_cksum[4];
+	u64 magic = 0;
+	int rc;
+
+	sb_omf = (struct sb_descriptor_omf *)inbuf;
+
+	magic = omf_psb_magic(sb_omf);
+
+	*omf_ver = OMF_SB_DESC_UNDEF;
+
+	if (magic != OMF_SB_MAGIC)
+		return -EBADF;
+
+	/* Verify CKSUM1 */
+	rc = omf_cksum_crc32c_le(inbuf, offsetof(struct sb_descriptor_omf, psb_cksum1), cksum);
+	omf_psb_cksum1(sb_omf, omf_cksum, 4);
+	if (rc || memcmp(cksum, omf_cksum, 4))
+		return -EINVAL;
+
+	*omf_ver = omf_psb_vers(sb_omf);
+
+	if (*omf_ver > OMF_SB_DESC_VER_LAST) {
+		rc = -EPROTONOSUPPORT;
+		mp_pr_err("Unsupported sb version %d", rc, *omf_ver);
+		return rc;
+	}
+
+	rc = omf_unpack_letoh_and_convert(sb, sizeof(*sb), inbuf, sb_descriptor_table,
+					  ARRAY_SIZE(sb_descriptor_table), *omf_ver, NULL);
+	if (rc)
+		mp_pr_err("Unpacking superblock failed for version %u", rc, *omf_ver);
+
+	return rc;
+}
+
+bool omf_sb_has_magic_le(const char *inbuf)
+{
+	struct sb_descriptor_omf *sb_omf;
+	u64 magic;
+
+	sb_omf = (struct sb_descriptor_omf *)inbuf;
+	magic  = omf_psb_magic(sb_omf);
+
+	return magic == OMF_SB_MAGIC;
+}
+
+
+/*
+ * mdcrec_objcmn
+ */
+
+/**
+ * omf_mdcrec_objcmn_pack_htole() - pack mdc obj record
+ * @mp:
+ * @cdr:
+ * @outbuf:
+ *
+ * Pack mdc obj record and optional obj layout into outbuf little-endian.
+ *
+ * Return: bytes packed if successful, -EINVAL otherwise
+ */
+static u64 omf_mdcrec_objcmn_pack_htole(struct mpool_descriptor *mp,
+					struct omf_mdcrec_data *cdr, char *outbuf)
+{
+	struct pmd_layout *layout = cdr->u.obj.omd_layout;
+	struct mdcrec_data_odelete_omf *odel_omf;
+	struct mdcrec_data_oerase_omf *oera_omf;
+	s64 bytes = 0;
+
+	switch (cdr->omd_rtype) {
+	case OMF_MDR_ODELETE:
+	case OMF_MDR_OIDCKPT:
+		odel_omf = (struct mdcrec_data_odelete_omf *)outbuf;
+		omf_set_pdro_rtype(odel_omf, cdr->omd_rtype);
+		omf_set_pdro_objid(odel_omf, cdr->u.obj.omd_objid);
+		return sizeof(*odel_omf);
+
+	case OMF_MDR_OERASE:
+		oera_omf = (struct mdcrec_data_oerase_omf *)outbuf;
+		omf_set_pdrt_rtype(oera_omf, cdr->omd_rtype);
+		omf_set_pdrt_objid(oera_omf, cdr->u.obj.omd_objid);
+		omf_set_pdrt_gen(oera_omf, cdr->u.obj.omd_gen);
+		return sizeof(*oera_omf);
+
+	default:
+		break;
+	}
+
+	if (cdr->omd_rtype != OMF_MDR_OCREATE && cdr->omd_rtype != OMF_MDR_OUPDATE) {
+		mp_pr_warn("mpool %s, packing object, unknown rec type %d",
+			   mp->pds_name, cdr->omd_rtype);
+		return -EINVAL;
+	}
+
+	/* OCREATE or OUPDATE: pack object in provided layout descriptor */
+	if (!layout) {
+		mp_pr_warn("mpool %s, invalid layout", mp->pds_name);
+		return -EINVAL;
+	}
+
+	bytes = omf_pmd_layout_pack_htole(mp, cdr->omd_rtype, layout, outbuf);
+	if (bytes < 0)
+		return -EINVAL;
+
+	return bytes;
+}
+
+/**
+ * omf_mdcrec_objcmn_unpack_letoh() - Unpack little-endian mdc record and optional obj layout
+ * @mp:
+ * @mdcver:
+ * @cdr:
+ * @inbuf:
+ *
+ * Return:
+ *   0 if successful
+ *   -EINVAL if invalid record type or format
+ *   -ENOMEM if cannot alloc memory to return an object layout
+ *   -ENOENT if cannot convert a devid to a device handle (pdh)
+ */
+static int omf_mdcrec_objcmn_unpack_letoh(struct mpool_descriptor *mp, struct omf_mdcver *mdcver,
+					  struct omf_mdcrec_data *cdr, const char *inbuf)
+{
+	struct mdcrec_data_odelete_omf *odel_omf;
+	struct mdcrec_data_oerase_omf *oera_omf;
+	enum mdcrec_type_omf rtype;
+	int rc = 0;
+
+	/*
+	 * The data record type is always the first field of all the
+	 * data records.
+	 */
+	rtype = omf_pdro_rtype((struct mdcrec_data_odelete_omf *)inbuf);
+
+	switch (rtype) {
+	case OMF_MDR_ODELETE:
+	case OMF_MDR_OIDCKPT:
+		odel_omf = (struct mdcrec_data_odelete_omf *)inbuf;
+		cdr->omd_rtype = omf_pdro_rtype(odel_omf);
+		cdr->u.obj.omd_objid = omf_pdro_objid(odel_omf);
+		break;
+
+	case OMF_MDR_OERASE:
+		oera_omf = (struct mdcrec_data_oerase_omf *)inbuf;
+		cdr->omd_rtype = omf_pdrt_rtype(oera_omf);
+		cdr->u.obj.omd_objid = omf_pdrt_objid(oera_omf);
+		cdr->u.obj.omd_gen = omf_pdrt_gen(oera_omf);
+		break;
+
+	case OMF_MDR_OCREATE:
+	case OMF_MDR_OUPDATE:
+		rc = omf_pmd_layout_unpack_letoh(mp, mdcver, cdr, inbuf);
+		break;
+
+	default:
+		mp_pr_warn("mpool %s, invalid rtype %d", mp->pds_name, rtype);
+		return -EINVAL;
+	}
+
+	return rc;
+}
+
+
+/*
+ * mdcrec_mcconfig
+ */
+
+/**
+ * omf_mdcrec_mcconfig_pack_htole() - Pack mdc mclass config record into outbuf little-endian.
+ * @cdr:
+ * @outbuf:
+ *
+ * Return: bytes packed.
+ */
+static u64 omf_mdcrec_mcconfig_pack_htole(struct omf_mdcrec_data *cdr, char *outbuf)
+{
+	struct mdcrec_data_mcconfig_omf *mc_omf;
+
+	mc_omf = (struct mdcrec_data_mcconfig_omf *)outbuf;
+	omf_set_pdrs_rtype(mc_omf, cdr->omd_rtype);
+	omf_dparm_pack_htole(&(cdr->u.dev.omd_parm), (char *)&(mc_omf->pdrs_parm));
+
+	return sizeof(*mc_omf);
+}
+
+/**
+ * omf_mdcrec_mcconfig_unpack_letoh() - Unpack little-endian mdc mcconfig record from inbuf.
+ * @cdr:
+ * @inbuf:
+ */
+static int omf_mdcrec_mcconfig_unpack_letoh(struct omf_mdcver *mdcver, struct omf_mdcrec_data *cdr,
+					    const char *inbuf)
+{
+	struct mdcrec_data_mcconfig_omf *mc_omf;
+
+	mc_omf = (struct mdcrec_data_mcconfig_omf *)inbuf;
+
+	cdr->omd_rtype = omf_pdrs_rtype(mc_omf);
+
+	return omf_dparm_unpack_letoh(&(cdr->u.dev.omd_parm), (char *)&(mc_omf->pdrs_parm),
+				      OMF_SB_DESC_UNDEF, mdcver, UNPACKCONVERT);
+}
+
+
+/*
+ * mdcrec_version
+ */
+
+/**
+ * omf_mdcver_pack_htole() - Pack mdc content version record into outbuf little-endian.
+ * @cdr:
+ * @outbuf:
+ *
+ * Return: bytes packed.
+ */
+static u64 omf_mdcver_pack_htole(struct omf_mdcrec_data *cdr, char *outbuf)
+{
+	struct mdcver_omf *pv_omf = (struct mdcver_omf *)outbuf;
+
+	omf_set_pv_rtype(pv_omf, cdr->omd_rtype);
+	omf_set_pv_mdcv_major(pv_omf, cdr->u.omd_version.mdcv_major);
+	omf_set_pv_mdcv_minor(pv_omf, cdr->u.omd_version.mdcv_minor);
+	omf_set_pv_mdcv_patch(pv_omf, cdr->u.omd_version.mdcv_patch);
+	omf_set_pv_mdcv_dev(pv_omf, cdr->u.omd_version.mdcv_dev);
+
+	return sizeof(*pv_omf);
+}
+
+void omf_mdcver_unpack_letoh(struct omf_mdcrec_data *cdr, const char *inbuf)
+{
+	struct mdcver_omf *pv_omf = (struct mdcver_omf *)inbuf;
+
+	cdr->omd_rtype = omf_pv_rtype(pv_omf);
+	cdr->u.omd_version.mdcv_major = omf_pv_mdcv_major(pv_omf);
+	cdr->u.omd_version.mdcv_minor = omf_pv_mdcv_minor(pv_omf);
+	cdr->u.omd_version.mdcv_patch = omf_pv_mdcv_patch(pv_omf);
+	cdr->u.omd_version.mdcv_dev   = omf_pv_mdcv_dev(pv_omf);
+}
+
+
+/*
+ * mdcrec_mcspare
+ */
+static u64 omf_mdcrec_mcspare_pack_htole(struct omf_mdcrec_data *cdr, char *outbuf)
+{
+	struct mdcrec_data_mcspare_omf *mcs_omf;
+
+	mcs_omf = (struct mdcrec_data_mcspare_omf *)outbuf;
+	omf_set_pdra_rtype(mcs_omf, cdr->omd_rtype);
+	omf_set_pdra_mclassp(mcs_omf, cdr->u.mcs.omd_mclassp);
+	omf_set_pdra_spzone(mcs_omf, cdr->u.mcs.omd_spzone);
+
+	return sizeof(*mcs_omf);
+}
+
+/**
+ * omf_mdcrec_mcspare_unpack_letoh_v1() - Unpack little-endian mdc media class spare record
+ * @cdr:
+ * @inbuf:
+ */
+static int omf_mdcrec_mcspare_unpack_letoh_v1(void *out, const char *inbuf)
+{
+	struct mdcrec_data_mcspare_omf *mcs_omf;
+	struct omf_mdcrec_data *cdr = out;
+
+	mcs_omf = (struct mdcrec_data_mcspare_omf *)inbuf;
+
+	cdr->omd_rtype = omf_pdra_rtype(mcs_omf);
+	cdr->u.mcs.omd_mclassp = omf_pdra_mclassp(mcs_omf);
+	cdr->u.mcs.omd_spzone = omf_pdra_spzone(mcs_omf);
+
+	return 0;
+}
+
+/**
+ * omf_mdcrec_mcspare_unpack_letoh() - Unpack little-endian mdc media class spare record
+ * @cdr:
+ * @inbuf:
+ */
+static int omf_mdcrec_mcspare_unpack_letoh(struct omf_mdcrec_data *cdr, const char *inbuf,
+					   enum sb_descriptor_ver_omf sbver,
+					   struct omf_mdcver *mdcver)
+{
+	return omf_unpack_letoh_and_convert(cdr, sizeof(*cdr), inbuf, mdcrec_data_mcspare_table,
+					    ARRAY_SIZE(mdcrec_data_mcspare_table), sbver, mdcver);
+}
+
+
+/*
+ * mdcrec_mpconfig
+ */
+
+/**
+ * omf_mdcrec_mpconfig_pack_htole() - Pack an mpool config record
+ * @cdr:
+ * @outbuf:
+ *
+ * Return: bytes packed.
+ */
+static u64 omf_mdcrec_mpconfig_pack_htole(struct omf_mdcrec_data *cdr, char *outbuf)
+{
+	struct mdcrec_data_mpconfig_omf *cfg_omf;
+	struct mpool_config *cfg;
+
+	cfg = &cdr->u.omd_cfg;
+
+	cfg_omf = (struct mdcrec_data_mpconfig_omf *)outbuf;
+	omf_set_pdmc_rtype(cfg_omf, cdr->omd_rtype);
+	omf_set_pdmc_oid1(cfg_omf, cfg->mc_oid1);
+	omf_set_pdmc_oid2(cfg_omf, cfg->mc_oid2);
+	omf_set_pdmc_uid(cfg_omf, cfg->mc_uid);
+	omf_set_pdmc_gid(cfg_omf, cfg->mc_gid);
+	omf_set_pdmc_mode(cfg_omf, cfg->mc_mode);
+	omf_set_pdmc_rsvd0(cfg_omf, cfg->mc_rsvd0);
+	omf_set_pdmc_captgt(cfg_omf, cfg->mc_captgt);
+	omf_set_pdmc_ra_pages_max(cfg_omf, cfg->mc_ra_pages_max);
+	omf_set_pdmc_vma_size_max(cfg_omf, cfg->mc_vma_size_max);
+	omf_set_pdmc_rsvd1(cfg_omf, cfg->mc_rsvd1);
+	omf_set_pdmc_rsvd2(cfg_omf, cfg->mc_rsvd2);
+	omf_set_pdmc_rsvd3(cfg_omf, cfg->mc_rsvd3);
+	omf_set_pdmc_rsvd4(cfg_omf, cfg->mc_rsvd4);
+	omf_set_pdmc_utype(cfg_omf, &cfg->mc_utype, sizeof(cfg->mc_utype));
+	omf_set_pdmc_label(cfg_omf, cfg->mc_label, sizeof(cfg->mc_label));
+
+	return sizeof(*cfg_omf);
+}
+
+/**
+ * omf_mdcrec_mpconfig_unpack_letoh() - Unpack an mpool config record
+ * @cdr:
+ * @inbuf:
+ *
+ * Return: bytes packed.
+ */
+static void omf_mdcrec_mpconfig_unpack_letoh(struct omf_mdcrec_data *cdr, const char *inbuf)
+{
+	struct mdcrec_data_mpconfig_omf *cfg_omf;
+	struct mpool_config *cfg;
+
+	cfg = &cdr->u.omd_cfg;
+
+	cfg_omf = (struct mdcrec_data_mpconfig_omf *)inbuf;
+	cdr->omd_rtype = omf_pdmc_rtype(cfg_omf);
+	cfg->mc_oid1 = omf_pdmc_oid1(cfg_omf);
+	cfg->mc_oid2 = omf_pdmc_oid2(cfg_omf);
+	cfg->mc_uid = omf_pdmc_uid(cfg_omf);
+	cfg->mc_gid = omf_pdmc_gid(cfg_omf);
+	cfg->mc_mode = omf_pdmc_mode(cfg_omf);
+	cfg->mc_rsvd0 = omf_pdmc_rsvd0(cfg_omf);
+	cfg->mc_captgt = omf_pdmc_captgt(cfg_omf);
+	cfg->mc_ra_pages_max = omf_pdmc_ra_pages_max(cfg_omf);
+	cfg->mc_vma_size_max = omf_pdmc_vma_size_max(cfg_omf);
+	cfg->mc_rsvd1 = omf_pdmc_rsvd1(cfg_omf);
+	cfg->mc_rsvd2 = omf_pdmc_rsvd2(cfg_omf);
+	cfg->mc_rsvd3 = omf_pdmc_rsvd3(cfg_omf);
+	cfg->mc_rsvd4 = omf_pdmc_rsvd4(cfg_omf);
+	omf_pdmc_utype(cfg_omf, &cfg->mc_utype, sizeof(cfg->mc_utype));
+	omf_pdmc_label(cfg_omf, cfg->mc_label, sizeof(cfg->mc_label));
+}
+
+/**
+ * mdcrec_type_objcmn() - Determine if the data record type corresponds to an object.
+ * @rtype: record type
+ *
+ * Return: true if the type is of an object data record.
+ */
+static bool mdcrec_type_objcmn(enum mdcrec_type_omf rtype)
+{
+	return (rtype == OMF_MDR_OCREATE || rtype == OMF_MDR_OUPDATE || rtype == OMF_MDR_ODELETE ||
+		rtype == OMF_MDR_OIDCKPT || rtype == OMF_MDR_OERASE);
+}
+
+int omf_mdcrec_isobj_le(const char *inbuf)
+{
+	const u8 rtype = inbuf[0]; /* rtype is byte so no endian conversion */
+
+	return mdcrec_type_objcmn(rtype);
+}
+
+
+/*
+ * mdcrec
+ */
+int omf_mdcrec_pack_htole(struct mpool_descriptor *mp, struct omf_mdcrec_data *cdr, char *outbuf)
+{
+	u8 rtype = (char)cdr->omd_rtype;
+
+	if (mdcrec_type_objcmn(rtype))
+		return omf_mdcrec_objcmn_pack_htole(mp, cdr, outbuf);
+	else if (rtype == OMF_MDR_VERSION)
+		return omf_mdcver_pack_htole(cdr, outbuf);
+	else if (rtype == OMF_MDR_MCCONFIG)
+		return omf_mdcrec_mcconfig_pack_htole(cdr, outbuf);
+	else if (rtype == OMF_MDR_MCSPARE)
+		return omf_mdcrec_mcspare_pack_htole(cdr, outbuf);
+	else if (rtype == OMF_MDR_MPCONFIG)
+		return omf_mdcrec_mpconfig_pack_htole(cdr, outbuf);
+
+	mp_pr_warn("mpool %s, invalid record type %u in mdc log", mp->pds_name, rtype);
+
+	return -EINVAL;
+}
+
+int omf_mdcrec_unpack_letoh(struct omf_mdcver *mdcver, struct mpool_descriptor *mp,
+			    struct omf_mdcrec_data *cdr, const char *inbuf)
+{
+	u8 rtype = (u8)*inbuf;
+
+	/* rtype is byte so no endian conversion */
+
+	if (mdcrec_type_objcmn(rtype))
+		return omf_mdcrec_objcmn_unpack_letoh(mp, mdcver, cdr, inbuf);
+	else if (rtype == OMF_MDR_VERSION)
+		omf_mdcver_unpack_letoh(cdr, inbuf);
+	else if (rtype == OMF_MDR_MCCONFIG)
+		omf_mdcrec_mcconfig_unpack_letoh(mdcver, cdr, inbuf);
+	else if (rtype == OMF_MDR_MCSPARE)
+		omf_mdcrec_mcspare_unpack_letoh(cdr, inbuf, OMF_SB_DESC_UNDEF, mdcver);
+	else if (rtype == OMF_MDR_MPCONFIG)
+		omf_mdcrec_mpconfig_unpack_letoh(cdr, inbuf);
+	else {
+		mp_pr_warn("mpool %s, unknown record type %u in mdc log", mp->pds_name, rtype);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+u8 omf_mdcrec_unpack_type_letoh(const char *inbuf)
+{
+	/* rtype is byte so no endian conversion */
+	return (u8)*inbuf;
+}
+
+
+/*
+ * logblock_header
+ */
+
+int omf_logblock_header_pack_htole(struct omf_logblock_header *lbh, char *outbuf)
+{
+	struct logblock_header_omf *lbh_omf;
+
+	lbh_omf = (struct logblock_header_omf *)outbuf;
+
+	if (lbh->olh_vers != OMF_LOGBLOCK_VERS)
+		return -EINVAL;
+
+	omf_set_polh_vers(lbh_omf, lbh->olh_vers);
+	omf_set_polh_magic(lbh_omf, lbh->olh_magic.uuid, MPOOL_UUID_SIZE);
+	omf_set_polh_gen(lbh_omf, lbh->olh_gen);
+	omf_set_polh_pfsetid(lbh_omf, lbh->olh_pfsetid);
+	omf_set_polh_cfsetid(lbh_omf, lbh->olh_cfsetid);
+
+	return 0;
+}
+
+int omf_logblock_header_unpack_letoh(struct omf_logblock_header *lbh, const char *inbuf)
+{
+	struct logblock_header_omf *lbh_omf;
+
+	lbh_omf = (struct logblock_header_omf *)inbuf;
+
+	lbh->olh_vers    = omf_polh_vers(lbh_omf);
+	omf_polh_magic(lbh_omf, lbh->olh_magic.uuid, MPOOL_UUID_SIZE);
+	lbh->olh_gen     = omf_polh_gen(lbh_omf);
+	lbh->olh_pfsetid = omf_polh_pfsetid(lbh_omf);
+	lbh->olh_cfsetid = omf_polh_cfsetid(lbh_omf);
+
+	return 0;
+}
+
+int omf_logblock_header_len_le(char *lbuf)
+{
+	struct logblock_header_omf *lbh_omf;
+
+	lbh_omf = (struct logblock_header_omf *)lbuf;
+
+	if (omf_polh_vers(lbh_omf) == OMF_LOGBLOCK_VERS)
+		return OMF_LOGBLOCK_HDR_PACKLEN;
+
+	return -EINVAL;
+}
+
+
+/*
+ * logrec_descriptor
+ */
+static bool logrec_type_valid(enum logrec_type_omf rtype)
+{
+	return rtype <= OMF_LOGREC_CEND;
+}
+
+bool logrec_type_datarec(enum logrec_type_omf rtype)
+{
+	return rtype && rtype <= OMF_LOGREC_DATALAST;
+}
+
+int omf_logrec_desc_pack_htole(struct omf_logrec_descriptor *lrd, char *outbuf)
+{
+	struct logrec_descriptor_omf *lrd_omf;
+
+	if (!logrec_type_valid(lrd->olr_rtype))
+		return -EINVAL;
+
+	lrd_omf = (struct logrec_descriptor_omf *)outbuf;
+	omf_set_polr_tlen(lrd_omf, lrd->olr_tlen);
+	omf_set_polr_rlen(lrd_omf, lrd->olr_rlen);
+	omf_set_polr_rtype(lrd_omf, lrd->olr_rtype);
+
+	return 0;
+}
+
+void omf_logrec_desc_unpack_letoh(struct omf_logrec_descriptor *lrd, const char *inbuf)
+{
+	struct logrec_descriptor_omf *lrd_omf;
+
+	lrd_omf = (struct logrec_descriptor_omf *)inbuf;
+	lrd->olr_tlen  = omf_polr_tlen(lrd_omf);
+	lrd->olr_rlen  = omf_polr_rlen(lrd_omf);
+	lrd->olr_rtype = omf_polr_rtype(lrd_omf);
+}
+
+int omf_init(void)
+{
+	const char *algo = "crc32c";
+	int rc = 0;
+
+	mpool_tfm = crypto_alloc_shash(algo, 0, 0);
+	if (!mpool_tfm) {
+		rc = -ENOMEM;
+		mp_pr_err("crypto_alloc_shash(%s) failed", rc, algo);
+	}
+
+	return rc;
+}
+
+void omf_exit(void)
+{
+	if (mpool_tfm)
+		crypto_free_shash(mpool_tfm);
+}
diff --git a/drivers/mpool/upgrade.c b/drivers/mpool/upgrade.c
new file mode 100644
index 000000000000..1b6b692e58f4
--- /dev/null
+++ b/drivers/mpool/upgrade.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * DOC: Module info.
+ *
+ * Pool metadata upgrade module.
+ *
+ * Defines functions used to upgrade the mpool metadata.
+ *
+ */
+
+#include "omf_if.h"
+#include "upgrade.h"
+
+/*
+ * Latest mpool MDC content version understood by this binary.
+ * Also version used to write MDC content by this binary.
+ */
+#define MDCVER_MAJOR       1
+#define MDCVER_MINOR       0
+#define MDCVER_PATCH       0
+#define MDCVER_DEV         0
+
+/**
+ * struct mdcver_info - mpool MDC content version and its information.
+ * @mi_mdcver:  version of a mpool MDC content. It is the version of the
+ *              first binary that introduced that content semantic/format.
+ * @mi_types:   types used by this release (when writing MDC0-N content)
+ * @mi_ntypes:  no. of types are used by this release.
+ * @mi_comment: comment about that version
+ *
+ * Such a structure instance is added each time the mpool MDCs content
+ * semantic/format changes (making it incompatible with earlier binary
+ * versions).
+ */
+struct mdcver_info {
+	struct omf_mdcver   mi_mdcver;
+	uint8_t            *mi_types;
+	uint8_t             mi_ntypes;
+	const char         *mi_comment;
+};
+
+/*
+ * mpool MDC types used when MDC content is written at version 1.0.0.0.
+ */
+static uint8_t mdcver_1_0_0_0_types[] = {
+	OMF_MDR_OCREATE, OMF_MDR_OUPDATE, OMF_MDR_ODELETE, OMF_MDR_OIDCKPT,
+	OMF_MDR_OERASE, OMF_MDR_MCCONFIG, OMF_MDR_MCSPARE, OMF_MDR_VERSION,
+	OMF_MDR_MPCONFIG};
+
+
+/*
+ * mdcver_info mdcvtab[] - table of versions of mpool MDCs content.
+ *
+ * Each time MDC content semantic/format changes (making it incompatible
+ * with earlier binary versions) an entry is added in this table.
+ * The entry at the end of the array (highest index) is the version placed
+ * in the mpool MDC version record written to media when this binary writes
+ * the mpool MDCs.
+ * This entry is also the last mpool MDC content format/semantic that this
+ * binary understands.
+ *
+ * Example:
+ * - Initial binary 1.0.0.0 generates first ever MDCs content.
+ *   There is one entry in the table with its mi_mdcver being 1.0.0.0.
+ * - binary 1.0.0.1 is released and changes mpool MDC content semantic (for
+ *   example chenge the meaning of media class enum). This release adds the
+ *   entry 1.0.0.1 in this table.
+ * - binary 1.0.1.0 is released and doesn't change MDCs content semantic/format,
+ *   MDCs content generated by 1.0.1.0 binary is still compatible with a
+ *   1.0.0.1 binary reading it.
+ *   No entry is added in the table.
+ * - binary 2.0.0.0 is released and it changes MDCs content semantic.
+ *   A third entry is added in the table with its mi_mdcver being 2.0.0.0.
+ */
+static struct mdcver_info mdcvtab[] = {
+	{{ {MDCVER_MAJOR, MDCVER_MINOR, MDCVER_PATCH, MDCVER_DEV} },
+	mdcver_1_0_0_0_types, sizeof(mdcver_1_0_0_0_types),
+	"Initial mpool MDCs content"},
+};
+
+struct omf_mdcver *omfu_mdcver_cur(void)
+{
+	return &mdcvtab[ARRAY_SIZE(mdcvtab) - 1].mi_mdcver;
+}
+
+const char *omfu_mdcver_comment(struct omf_mdcver *mdcver)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(mdcvtab); i++)
+		if (omfu_mdcver_cmp(mdcver, "==", &mdcvtab[i].mi_mdcver))
+			return mdcvtab[i].mi_comment;
+
+	return NULL;
+}
+
+char *omfu_mdcver_to_str(struct omf_mdcver *mdcver, char *buf, size_t sz)
+{
+	snprintf(buf, sz, "%u.%u.%u.%u", mdcver->mdcv_major,
+		 mdcver->mdcv_minor, mdcver->mdcv_patch, mdcver->mdcv_dev);
+
+	return buf;
+}
+
+bool omfu_mdcver_cmp(struct omf_mdcver *a, char *op, struct omf_mdcver *b)
+{
+	size_t cnt = ARRAY_SIZE(a->mdcver);
+	int res = 0, i;
+
+	for (i = 0; i < cnt; i++) {
+		if (a->mdcver[i] != b->mdcver[i]) {
+			res = (a->mdcver[i] > b->mdcver[i]) ? 1 : -1;
+			break;
+		}
+	}
+
+	if (((op[1] == '=') && (res == 0)) || ((op[0] == '>') && (res > 0)) ||
+	    ((op[0] == '<') && (res < 0)))
+		return true;
+
+	return false;
+}
+
+bool omfu_mdcver_cmp2(struct omf_mdcver *a, char *op, u16 major, u16 minor, u16 patch, u16 dev)
+{
+	struct omf_mdcver b;
+
+	b.mdcv_major = major;
+	b.mdcv_minor = minor;
+	b.mdcv_patch = patch;
+	b.mdcv_dev   = dev;
+
+	return omfu_mdcver_cmp(a, op, &b);
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:26 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:26 -0500
Subject: [PATCH v2 12/22] mpool: add metadata container or mlog-pair framework
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-13-nmeeramohide@micron.com>

Metadata containers are used for storing and maintaining metadata.

MDC APIs are implemented as helper functions built on a pair of
mlogs per MDC. It embodies the concept of compaction to deal with
one of the mlog pairs filling, what it means to compact is
use-case dependent.

The MDC APIs make it easy for a client to:

- Append metadata update records to the active mlog of an MDC
  until it is full (or exceeds some client-specific threshold)
- Flag the start of a compaction which marks the other mlog of
  the MDC as active
- Re-serialize its metadata by appending it to the (newly)
  active mlog of the MDC
- Flag the end of the compaction
- Continue appending metadata update records to the MDC until
  the above process repeats

The MDC API functions handle all failures, including crash
recovery, by using special markers recognized by the mlog
implementation.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mdc.c | 486 ++++++++++++++++++++++++++++++++++++++++++++
 drivers/mpool/mdc.h | 106 ++++++++++
 2 files changed, 592 insertions(+)
 create mode 100644 drivers/mpool/mdc.c
 create mode 100644 drivers/mpool/mdc.h

diff --git a/drivers/mpool/mdc.c b/drivers/mpool/mdc.c
new file mode 100644
index 000000000000..288e6ee4670b
--- /dev/null
+++ b/drivers/mpool/mdc.c
@@ -0,0 +1,486 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include "mpool_printk.h"
+#include "mpool_ioctl.h"
+#include "mpcore.h"
+#include "mp.h"
+#include "mlog.h"
+#include "mdc.h"
+
+#define mdc_logerr(_mpname, _msg, _mlh, _objid, _gen1, _gen2, _err)     \
+	mp_pr_err("mpool %s, mdc open, %s "			        \
+		  "mlog %p objid 0x%lx gen1 %lu gen2 %lu",		\
+		  (_err), (_mpname), (_msg),		                \
+		  (_mlh), (ulong)(_objid), (ulong)(_gen1),		\
+		  (ulong)(_gen2))					\
+
+#define OP_COMMIT      0
+#define OP_DELETE      1
+
+/**
+ * mdc_acquire() - Validate mdc handle and acquire mdc_lock
+ * @mlh: MDC handle
+ * @rw:  read/append?
+ */
+static inline int mdc_acquire(struct mp_mdc *mdc, bool rw)
+{
+	if (!mdc || mdc->mdc_magic != MPC_MDC_MAGIC || !mdc->mdc_valid)
+		return -EINVAL;
+
+	if (rw && (mdc->mdc_flags & MDC_OF_SKIP_SER))
+		return 0;
+
+	mutex_lock(&mdc->mdc_lock);
+
+	/* Validate again after acquiring lock */
+	if (!mdc->mdc_valid) {
+		mutex_unlock(&mdc->mdc_lock);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+/**
+ * mdc_release() - Release mdc_lock
+ * @mlh: MDC handle
+ * @rw:  read/append?
+ */
+static inline void mdc_release(struct mp_mdc *mdc, bool rw)
+{
+	if (rw && (mdc->mdc_flags & MDC_OF_SKIP_SER))
+		return;
+
+	mutex_unlock(&mdc->mdc_lock);
+}
+
+/**
+ * mdc_invalidate() - Invalidates MDC handle by resetting the magic
+ * @mdc: MDC handle
+ */
+static inline void mdc_invalidate(struct mp_mdc *mdc)
+{
+	mdc->mdc_magic = MPC_NO_MAGIC;
+}
+
+/**
+ * mdc_get_mpname() - Get mpool name from mpool descriptor
+ * @mp:     mpool descriptor
+ * @mpname: buffer to store the mpool name (output)
+ * @mplen:  buffer len
+ */
+static int mdc_get_mpname(struct mpool_descriptor *mp, char *mpname, size_t mplen)
+{
+	if (!mp || !mpname)
+		return -EINVAL;
+
+	return mpool_get_mpname(mp, mpname, mplen);
+}
+
+/**
+ * mdc_find_get() - Wrapper around get for mlog pair.
+ */
+static void mdc_find_get(struct mpool_descriptor *mp, u64 *logid, bool do_put,
+			 struct mlog_props *props, struct mlog_descriptor **mlh, int *ferr)
+{
+	int i;
+
+	for (i = 0; i < 2; ++i)
+		ferr[i] = mlog_find_get(mp, logid[i], 0, &props[i], &mlh[i]);
+
+	if (do_put && ((ferr[0] && !ferr[1]) || (ferr[1] && !ferr[0]))) {
+		if (ferr[0])
+			mlog_put(mlh[1]);
+		else
+			mlog_put(mlh[0]);
+	}
+}
+
+/**
+ * mdc_put() - Wrapper around put for mlog pair.
+ */
+static void mdc_put(struct mlog_descriptor *mlh1, struct mlog_descriptor *mlh2)
+{
+	mlog_put(mlh1);
+	mlog_put(mlh2);
+}
+
+int mp_mdc_open(struct mpool_descriptor *mp, u64 logid1, u64 logid2, u8 flags,
+		struct mp_mdc **mdc_out)
+{
+	struct mlog_descriptor *mlh[2];
+	struct mlog_props *props = NULL;
+	struct mp_mdc *mdc;
+
+	int     err = 0, err1 = 0, err2 = 0;
+	int     ferr[2] = {0};
+	u64     gen1 = 0, gen2 = 0;
+	bool    empty = false;
+	u8      mlflags = 0;
+	u64     id[2];
+	char   *mpname;
+
+	if (!mp || !mdc_out)
+		return -EINVAL;
+
+	mdc = kzalloc(sizeof(*mdc), GFP_KERNEL);
+	if (!mdc)
+		return -ENOMEM;
+
+	mdc->mdc_valid = 0;
+	mdc->mdc_mp    = mp;
+	mdc_get_mpname(mp, mdc->mdc_mpname, sizeof(mdc->mdc_mpname));
+
+	mpname = mdc->mdc_mpname;
+
+	if (logid1 == logid2) {
+		err = -EINVAL;
+		goto exit;
+	}
+
+	props = kcalloc(2, sizeof(*props), GFP_KERNEL);
+	if (!props) {
+		err = -ENOMEM;
+		goto exit;
+	}
+
+	/*
+	 * This mdc_find_get can go away once mp_mdc_open is modified to
+	 * operate on handles.
+	 */
+	id[0] = logid1;
+	id[1] = logid2;
+	mdc_find_get(mp, id, true, props, mlh, ferr);
+	if (ferr[0] || ferr[1]) {
+		err = ferr[0] ? : ferr[1];
+		goto exit;
+	}
+	mdc->mdc_logh1 = mlh[0];
+	mdc->mdc_logh2 = mlh[1];
+
+	if (flags & MDC_OF_SKIP_SER)
+		mlflags |= MLOG_OF_SKIP_SER;
+
+	mlflags |= MLOG_OF_COMPACT_SEM;
+
+	err1 = mlog_open(mp, mdc->mdc_logh1, mlflags, &gen1);
+	err2 = mlog_open(mp, mdc->mdc_logh2, mlflags, &gen2);
+
+	if (err1 && err1 != -EMSGSIZE && err1 != -EBUSY) {
+		err = err1;
+	} else if (err2 && err2 != -EMSGSIZE && err2 != -EBUSY) {
+		err = err2;
+	} else if ((err1 && err2) || (!err1 && !err2 && gen1 && gen1 == gen2)) {
+
+		err = -EINVAL;
+
+		/* Bad pair; both have failed erases/compactions or equal non-0 gens. */
+		mp_pr_err("mpool %s, mdc open, bad mlog handle, mlog1 %p logid1 0x%lx errno %d gen1 %lu, mlog2 %p logid2 0x%lx errno %d gen2 %lu",
+			err, mpname, mdc->mdc_logh1, (ulong)logid1, err1, (ulong)gen1,
+			mdc->mdc_logh2, (ulong)logid2, err2, (ulong)gen2);
+	} else {
+		/* Active log is valid log with smallest gen */
+		if (err1 || (!err2 && gen2 < gen1)) {
+			mdc->mdc_alogh = mdc->mdc_logh2;
+			if (!err1) {
+				err = mlog_empty(mp, mdc->mdc_logh1, &empty);
+				if (err)
+					mdc_logerr(mpname, "mlog1 empty check failed",
+						   mdc->mdc_logh1, logid1, gen1, gen2, err);
+			}
+			if (!err && (err1 || !empty)) {
+				err = mlog_erase(mp, mdc->mdc_logh1, gen2 + 1);
+				if (!err) {
+					err = mlog_open(mp, mdc->mdc_logh1, mlflags, &gen1);
+					if (err)
+						mdc_logerr(mpname, "mlog1 open failed",
+							   mdc->mdc_logh1, logid1, gen1, gen2, err);
+				} else {
+					mdc_logerr(mpname, "mlog1 erase failed", mdc->mdc_logh1,
+						   logid1, gen1, gen2, err);
+				}
+			}
+		} else {
+			mdc->mdc_alogh = mdc->mdc_logh1;
+			if (!err2) {
+				err = mlog_empty(mp, mdc->mdc_logh2, &empty);
+				if (err)
+					mdc_logerr(mpname, "mlog2 empty check failed",
+						   mdc->mdc_logh2, logid2, gen1, gen2, err);
+			}
+			if (!err && (err2 || gen2 == gen1 || !empty)) {
+				err = mlog_erase(mp, mdc->mdc_logh2, gen1 + 1);
+				if (!err) {
+					err = mlog_open(mp, mdc->mdc_logh2, mlflags, &gen2);
+					if (err)
+						mdc_logerr(mpname, "mlog2 open failed",
+							   mdc->mdc_logh2, logid2, gen1, gen2, err);
+				} else {
+					mdc_logerr(mpname, "mlog2 erase failed", mdc->mdc_logh2,
+						   logid2, gen1, gen2, err);
+				}
+			}
+		}
+
+		if (!err) {
+			err = mlog_empty(mp, mdc->mdc_alogh, &empty);
+			if (!err && empty) {
+				/*
+				 * First use of log pair so need to add
+				 * cstart/cend recs; above handles case of
+				 * failure between adding cstart and cend
+				 */
+				err = mlog_append_cstart(mp, mdc->mdc_alogh);
+				if (!err) {
+					err = mlog_append_cend(mp, mdc->mdc_alogh);
+					if (err)
+						mdc_logerr(mpname,
+							   "adding cend to active mlog failed",
+							   mdc->mdc_alogh,
+							   mdc->mdc_alogh == mdc->mdc_logh1 ?
+							   logid1 : logid2, gen1, gen2, err);
+				} else {
+					mdc_logerr(mpname, "adding cstart to active mlog failed",
+						   mdc->mdc_alogh,
+						   mdc->mdc_alogh == mdc->mdc_logh1 ?
+						   logid1 : logid2, gen1, gen2, err);
+				}
+
+			} else if (err) {
+				mdc_logerr(mpname, "active mlog empty check failed",
+					   mdc->mdc_alogh, mdc->mdc_alogh == mdc->mdc_logh1 ?
+					   logid1 : logid2, gen1, gen2, err);
+			}
+		}
+	}
+
+	if (!err) {
+		/*
+		 * Inform pre-compaction of the size of the active
+		 * mlog and how much is used. This is applicable
+		 * only for mpool core's internal MDCs.
+		 */
+		mlog_precompact_alsz(mp, mdc->mdc_alogh);
+
+		mdc->mdc_valid = 1;
+		mdc->mdc_magic = MPC_MDC_MAGIC;
+		mdc->mdc_flags = flags;
+		mutex_init(&mdc->mdc_lock);
+
+		*mdc_out = mdc;
+	} else {
+		err1 = mlog_close(mp, mdc->mdc_logh1);
+		err2 = mlog_close(mp, mdc->mdc_logh2);
+
+		mdc_put(mdc->mdc_logh1, mdc->mdc_logh2);
+	}
+
+exit:
+	if (err)
+		kfree(mdc);
+
+	kfree(props);
+
+	return err;
+}
+
+int mp_mdc_cstart(struct mp_mdc *mdc)
+{
+	struct mlog_descriptor *tgth = NULL;
+	struct mpool_descriptor *mp;
+	bool rw = false;
+	int rc;
+
+	if (!mdc)
+		return -EINVAL;
+
+	rc = mdc_acquire(mdc, rw);
+	if (rc)
+		return rc;
+
+	mp = mdc->mdc_mp;
+
+	if (mdc->mdc_alogh == mdc->mdc_logh1)
+		tgth = mdc->mdc_logh2;
+	else
+		tgth = mdc->mdc_logh1;
+
+	rc = mlog_append_cstart(mp, tgth);
+	if (rc) {
+		mdc_release(mdc, rw);
+
+		mp_pr_err("mpool %s, mdc %p cstart failed, mlog %p",
+			  rc, mdc->mdc_mpname, mdc, tgth);
+
+		(void)mp_mdc_close(mdc);
+
+		return rc;
+	}
+
+	mdc->mdc_alogh = tgth;
+	mdc_release(mdc, rw);
+
+	return 0;
+}
+
+int mp_mdc_cend(struct mp_mdc *mdc)
+{
+	struct mlog_descriptor *srch = NULL;
+	struct mlog_descriptor *tgth = NULL;
+	struct mpool_descriptor *mp;
+	u64 gentgt = 0;
+	bool rw = false;
+	int rc;
+
+	if (!mdc)
+		return -EINVAL;
+
+	rc = mdc_acquire(mdc, rw);
+	if (rc)
+		return rc;
+
+	mp = mdc->mdc_mp;
+
+	if (mdc->mdc_alogh == mdc->mdc_logh1) {
+		tgth = mdc->mdc_logh1;
+		srch = mdc->mdc_logh2;
+	} else {
+		tgth = mdc->mdc_logh2;
+		srch = mdc->mdc_logh1;
+	}
+
+	rc = mlog_append_cend(mp, tgth);
+	if (!rc) {
+		rc = mlog_gen(tgth, &gentgt);
+		if (!rc)
+			rc = mlog_erase(mp, srch, gentgt + 1);
+	}
+
+	if (rc) {
+		mdc_release(mdc, rw);
+
+		mp_pr_err("mpool %s, mdc %p cend failed, mlog %p",
+			  rc, mdc->mdc_mpname, mdc, tgth);
+
+		mp_mdc_close(mdc);
+
+		return rc;
+	}
+
+	mdc_release(mdc, rw);
+
+	return rc;
+}
+
+int mp_mdc_close(struct mp_mdc *mdc)
+{
+	struct mpool_descriptor *mp;
+	int rval = 0, rc;
+	bool rw = false;
+
+	if (!mdc)
+		return -EINVAL;
+
+	rc = mdc_acquire(mdc, rw);
+	if (rc)
+		return rc;
+
+	mp = mdc->mdc_mp;
+
+	mdc->mdc_valid = 0;
+
+	rc = mlog_close(mp, mdc->mdc_logh1);
+	if (rc) {
+		mp_pr_err("mpool %s, mdc %p close failed, mlog1 %p",
+			  rc, mdc->mdc_mpname, mdc, mdc->mdc_logh1);
+		rval = rc;
+	}
+
+	rc = mlog_close(mp, mdc->mdc_logh2);
+	if (rc) {
+		mp_pr_err("mpool %s, mdc %p close failed, mlog2 %p",
+			  rc, mdc->mdc_mpname, mdc, mdc->mdc_logh2);
+		rval = rc;
+	}
+
+	mdc_put(mdc->mdc_logh1, mdc->mdc_logh2);
+
+	mdc_invalidate(mdc);
+	mdc_release(mdc, false);
+
+	kfree(mdc);
+
+	return rval;
+}
+
+int mp_mdc_rewind(struct mp_mdc *mdc)
+{
+	bool rw = false;
+	int rc;
+
+	if (!mdc)
+		return -EINVAL;
+
+	rc = mdc_acquire(mdc, rw);
+	if (rc)
+		return rc;
+
+	rc = mlog_read_data_init(mdc->mdc_alogh);
+	if (rc)
+		mp_pr_err("mpool %s, mdc %p rewind failed, mlog %p",
+			  rc, mdc->mdc_mpname, mdc, mdc->mdc_alogh);
+
+	mdc_release(mdc, rw);
+
+	return rc;
+}
+
+int mp_mdc_read(struct mp_mdc *mdc, void *data, size_t len, size_t *rdlen)
+{
+	bool rw = true;
+	int rc;
+
+	if (!mdc || !data)
+		return -EINVAL;
+
+	rc = mdc_acquire(mdc, rw);
+	if (rc)
+		return rc;
+
+	rc = mlog_read_data_next(mdc->mdc_mp, mdc->mdc_alogh, data, (u64)len, (u64 *)rdlen);
+	if (rc && rc != -EOVERFLOW)
+		mp_pr_err("mpool %s, mdc %p read failed, mlog %p len %lu",
+			  rc, mdc->mdc_mpname, mdc, mdc->mdc_alogh, (ulong)len);
+
+	mdc_release(mdc, rw);
+
+	return rc;
+}
+
+int mp_mdc_append(struct mp_mdc *mdc, void *data, ssize_t len, bool sync)
+{
+	bool rw = true;
+	int rc;
+
+	if (!mdc || !data)
+		return -EINVAL;
+
+	rc = mdc_acquire(mdc, rw);
+	if (rc)
+		return rc;
+
+	rc = mlog_append_data(mdc->mdc_mp, mdc->mdc_alogh, data, (u64)len, sync);
+	if (rc)
+		mp_pr_rl("mpool %s, mdc %p append failed, mlog %p, len %lu sync %d",
+			 rc, mdc->mdc_mpname, mdc, mdc->mdc_alogh, (ulong)len, sync);
+
+	mdc_release(mdc, rw);
+
+	return rc;
+}
diff --git a/drivers/mpool/mdc.h b/drivers/mpool/mdc.h
new file mode 100644
index 000000000000..7ab1de261eff
--- /dev/null
+++ b/drivers/mpool/mdc.h
@@ -0,0 +1,106 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_MDC_PRIV_H
+#define MPOOL_MDC_PRIV_H
+
+#include <linux/mutex.h>
+
+#define MPC_MDC_MAGIC           0xFEEDFEED
+#define MPC_NO_MAGIC            0xFADEFADE
+
+struct mpool_descriptor;
+struct mlog_descriptor;
+
+/**
+ * struct mp_mdc - MDC handle
+ * @mdc_mp:     mpool handle
+ * @mdc_logh1:  mlog 1 handle
+ * @mdc_logh2:  mlog 2 handle
+ * @mdc_alogh:  active mlog handle
+ * @mdc_lock:   mdc mutex
+ * @mdc_mpname: mpool name
+ * @mdc_valid:  is the handle valid?
+ * @mdc_magic:  MDC handle magic
+ * @mdc_flags:	MDC flags
+ */
+struct mp_mdc {
+	struct mpool_descriptor    *mdc_mp;
+	struct mlog_descriptor     *mdc_logh1;
+	struct mlog_descriptor     *mdc_logh2;
+	struct mlog_descriptor     *mdc_alogh;
+	struct mutex                mdc_lock;
+	char                        mdc_mpname[MPOOL_NAMESZ_MAX];
+	int                         mdc_valid;
+	int                         mdc_magic;
+	u8                          mdc_flags;
+};
+
+/* MDC (Metadata Container) APIs */
+
+/**
+ * mp_mdc_open() - Open MDC by OIDs
+ * @mp:       mpool handle
+ * @logid1:   Mlog ID 1
+ * @logid2:   Mlog ID 2
+ * @flags:    MDC Open flags (enum mdc_open_flags)
+ * @mdc_out:  MDC handle
+ */
+int
+mp_mdc_open(struct mpool_descriptor *mp, u64 logid1, u64 logid2, u8 flags, struct mp_mdc **mdc_out);
+
+/**
+ * mp_mdc_close() - Close MDC
+ * @mdc:      MDC handle
+ */
+int mp_mdc_close(struct mp_mdc *mdc);
+
+/**
+ * mp_mdc_rewind() - Rewind MDC to first record
+ * @mdc:      MDC handle
+ */
+int mp_mdc_rewind(struct mp_mdc *mdc);
+
+/**
+ * mp_mdc_read() - Read next record from MDC
+ * @mdc:      MDC handle
+ * @data:     buffer to receive data
+ * @len:      length of supplied buffer
+ * @rdlen:    number of bytes read
+ *
+ * Return:
+ *   If the return value is -EOVERFLOW, then the receive buffer "data"
+ *   is too small and must be resized according to the value returned
+ *   in "rdlen".
+ */
+int mp_mdc_read(struct mp_mdc *mdc, void *data, size_t len, size_t *rdlen);
+
+/**
+ * mp_mdc_append() - append record to MDC
+ * @mdc:      MDC handle
+ * @data:     data to write
+ * @len:      length of data
+ * @sync:     flag to defer return until IO is complete
+ */
+int mp_mdc_append(struct mp_mdc *mdc, void *data, ssize_t len, bool sync);
+
+/**
+ * mp_mdc_cstart() - Initiate MDC compaction
+ * @mdc:      MDC handle
+ *
+ * Swap active (ostensibly full) and inactive (empty) mlogs
+ * Append a compaction start marker to newly active mlog
+ */
+int mp_mdc_cstart(struct mp_mdc *mdc);
+
+/**
+ * mp_mdc_cend() - End MDC compactions
+ * @mdc:      MDC handle
+ *
+ * Append a compaction end marker to the active mlog
+ */
+int mp_mdc_cend(struct mp_mdc *mdc);
+
+#endif /* MPOOL_MDC_PRIV_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:25 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:25 -0500
Subject: [PATCH v2 11/22] mpool: add mlog lifecycle management and IO routines
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-12-nmeeramohide@micron.com>

This implements the mlog lifecycle management functions:
allocate, commit, abort, destroy, append, read etc.

Mlog objects are containers for record logging. Mlogs can be
appended with arbitrary sized records and once full, an mlog
must be erased before additional records can be appended.
Mlog records can be read sequentially from the beginning at
any time. Mlogs in a media class are always a multiple of
the mblock size for that media class.

The mlog APIs implement a pattern whereby an mlog is allocated
and then committed or aborted. An mlog is not persistent or
accessible until committed, and a system failure prior to
commit results in the same logical mpool state as if the mlog
had never been allocated. An mlog allocation returns an OID
that is used to commit, append, flush, erase, or read as needed,
and delete the mlog.

At mlog open, the read buffer is fully loaded and parsed to
identify the end-of-log and the next flush set ID, to detect
media corruption, to detect bad record formatting, and to
optionally enforce compaction semantics. At mlog close, the
dirty data is flushed and all memory resources are freed.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mlog.c | 1667 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 1667 insertions(+)
 create mode 100644 drivers/mpool/mlog.c

diff --git a/drivers/mpool/mlog.c b/drivers/mpool/mlog.c
new file mode 100644
index 000000000000..6ccca00735c1
--- /dev/null
+++ b/drivers/mpool/mlog.c
@@ -0,0 +1,1667 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#include <linux/mm.h>
+#include <linux/log2.h>
+#include <linux/blk_types.h>
+#include <asm/page.h>
+
+#include "assert.h"
+#include "mpool_printk.h"
+
+#include "omf_if.h"
+#include "mpcore.h"
+#include "mlog_utils.h"
+
+/**
+ * mlog_alloc_cmn() - Allocate mlog with specified parameters using new or specified objid.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+static int mlog_alloc_cmn(struct mpool_descriptor *mp, u64 objid,
+			  struct mlog_capacity *capreq, enum mp_media_classp mclassp,
+			  struct mlog_props *prop, struct mlog_descriptor **mlh)
+{
+	struct pmd_obj_capacity ocap;
+	struct pmd_layout *layout;
+	int rc;
+
+	layout = NULL;
+	*mlh = NULL;
+
+	ocap.moc_captgt = capreq->lcp_captgt;
+	ocap.moc_spare  = capreq->lcp_spare;
+
+	if (!objid) {
+		rc = pmd_obj_alloc(mp, OMF_OBJ_MLOG, &ocap, mclassp, &layout);
+		if (rc || !layout) {
+			if (rc != -ENOENT)
+				mp_pr_err("mpool %s, allocating mlog failed", rc, mp->pds_name);
+		}
+	} else {
+		rc = pmd_obj_realloc(mp, objid, &ocap, mclassp, &layout);
+		if (rc || !layout) {
+			if (rc != -ENOENT)
+				mp_pr_err("mpool %s, re-allocating mlog 0x%lx failed",
+					  rc, mp->pds_name, (ulong)objid);
+		}
+	}
+	if (rc)
+		return rc;
+
+	/*
+	 * Mlogs rarely created and usually committed immediately so erase in-line;
+	 * mlog not committed so pmd_obj_erase() not needed to make atomic
+	 */
+	pmd_obj_wrlock(layout);
+	rc = pmd_layout_erase(mp, layout);
+	if (!rc)
+		mlog_getprops_cmn(mp, layout, prop);
+	pmd_obj_wrunlock(layout);
+
+	if (rc) {
+		pmd_obj_abort(mp, layout);
+		mp_pr_err("mpool %s, mlog 0x%lx alloc, erase failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	*mlh = layout2mlog(layout);
+
+	return 0;
+}
+
+/**
+ * mlog_alloc() - Allocate mlog with the capacity params specified in capreq.
+ *
+ * Allocate mlog with the capacity params specified in capreq on drives in a
+ * media class mclassp.
+ * If successful mlh is a handle for the mlog and prop contains its properties.
+ *
+ * Note: mlog is not persistent until committed; allocation can be aborted.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+int mlog_alloc(struct mpool_descriptor *mp, struct mlog_capacity *capreq,
+	       enum mp_media_classp mclassp, struct mlog_props *prop,
+	       struct mlog_descriptor **mlh)
+{
+	return mlog_alloc_cmn(mp, 0, capreq, mclassp, prop, mlh);
+}
+
+
+/**
+ * mlog_realloc() - Allocate mlog with specified objid to support crash recovery.
+ *
+ * Allocate mlog with specified objid to support crash recovery; otherwise
+ * is equivalent to mlog_alloc().
+ *
+ * Returns: 0 if successful, -errno otherwise
+ * One of the possible errno values:
+ * -EEXISTS - if objid exists
+ */
+int mlog_realloc(struct mpool_descriptor *mp, u64 objid,
+		 struct mlog_capacity *capreq, enum mp_media_classp mclassp,
+		 struct mlog_props *prop, struct mlog_descriptor **mlh)
+{
+	if (!mlog_objid(objid))
+		return -EINVAL;
+
+	return mlog_alloc_cmn(mp, objid, capreq, mclassp, prop, mlh);
+}
+
+/**
+ * mlog_find_get() - Get handle and properties for existing mlog with specified objid.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+int mlog_find_get(struct mpool_descriptor *mp, u64 objid, int which,
+		  struct mlog_props *prop, struct mlog_descriptor **mlh)
+{
+	struct pmd_layout *layout;
+
+	*mlh = NULL;
+
+	if (!mlog_objid(objid))
+		return -EINVAL;
+
+	layout = pmd_obj_find_get(mp, objid, which);
+	if (!layout)
+		return -ENOENT;
+
+	if (prop) {
+		pmd_obj_rdlock(layout);
+		mlog_getprops_cmn(mp, layout, prop);
+		pmd_obj_rdunlock(layout);
+	}
+
+	*mlh = layout2mlog(layout);
+
+	return 0;
+}
+
+/**
+ * mlog_put() - Put a reference for mlog with specified objid.
+ */
+void mlog_put(struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout;
+
+	layout = mlog2layout(mlh);
+	if (layout)
+		pmd_obj_put(layout);
+}
+
+/**
+ * mlog_lookup_rootids() - Return OIDs of mpctl root MDC.
+ * @id1: (output): OID of one of the mpctl root MDC mlogs.
+ * @id2: (output): OID of the other mpctl root MDC mlogs.
+ */
+void mlog_lookup_rootids(u64 *id1, u64 *id2)
+{
+	if (id1)
+		*id1 = UROOT_OBJID_LOG1;
+
+	if (id2)
+		*id2 = UROOT_OBJID_LOG2;
+}
+
+/**
+ * mlog_commit() - Make allocated mlog persistent.
+ *
+ * If fails mlog still exists in an uncommitted state so can retry commit or abort.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+int mlog_commit(struct mpool_descriptor *mp, struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout;
+
+	layout = mlog2layout(mlh);
+	if (!layout)
+		return -EINVAL;
+
+	return pmd_obj_commit(mp, layout);
+}
+
+/**
+ * mlog_abort() - Discard uncommitted mlog; if successful mlh is invalid after call.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+int mlog_abort(struct mpool_descriptor *mp, struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout;
+
+	layout = mlog2layout(mlh);
+	if (!layout)
+		return -EINVAL;
+
+	return pmd_obj_abort(mp, layout);
+}
+
+/**
+ * mlog_delete() - Delete committed mlog.
+
+ * If successful mlh is invalid after call; if fails mlog is closed.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+int mlog_delete(struct mpool_descriptor *mp, struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout;
+
+	layout = mlog2layout(mlh);
+	if (!layout)
+		return -EINVAL;
+
+	/* Remove from open list and discard buffered log data */
+	pmd_obj_wrlock(layout);
+	oml_layout_lock(mp);
+	oml_layout_remove(mp, layout->eld_objid);
+	oml_layout_unlock(mp);
+
+	mlog_stat_free(layout);
+	pmd_obj_wrunlock(layout);
+
+	return pmd_obj_delete(mp, layout);
+}
+
+/**
+ * mlog_logrecs_validate() - Validate records in lstat.rbuf relative to lstat state.
+ *
+ * Validate records in lstat.rbuf relative to lstat state where midrec
+ * indicates if mid data record from previous log block; updates lstate to
+ * reflect valid markers found (if any).
+ *
+ * Returns:
+ *   0 if successful; -errno otherwise
+ *
+ *   In the output param, i.e., midrec, we store:
+ *   1 if log records are valid and ended mid data record
+ *   0 if log records are valid and did NOT end mid data record
+ */
+static int mlog_logrecs_validate(struct mlog_stat *lstat, int *midrec, u16 rbidx, u16 lbidx)
+{
+	struct omf_logrec_descriptor lrd;
+	u64 recnum = 0;
+	int recoff;
+	int rc = 0;
+	char *rbuf;
+	u16 sectsz = 0;
+
+	sectsz = MLOG_SECSZ(lstat);
+	rbuf   = lstat->lst_rbuf[rbidx] + lbidx * sectsz;
+
+	recoff = omf_logblock_header_len_le(rbuf);
+	if (recoff < 0)
+		return -ENODATA;
+
+	while (sectsz - recoff >= OMF_LOGREC_DESC_PACKLEN) {
+		omf_logrec_desc_unpack_letoh(&lrd, &rbuf[recoff]);
+
+		if (lrd.olr_rtype == OMF_LOGREC_CSTART) {
+			if (!lstat->lst_csem || lstat->lst_rsoff || recnum) {
+				rc = -ENODATA;
+
+				/* No compaction or not first rec in first log block */
+				mp_pr_err("no compact marker nor first rec %u %ld %u %u %lu",
+					  rc, lstat->lst_csem, lstat->lst_rsoff,
+					  rbidx, lbidx, (ulong)recnum);
+				return rc;
+			}
+			lstat->lst_cstart = 1;
+			*midrec = 0;
+		} else if (lrd.olr_rtype == OMF_LOGREC_CEND) {
+			if (!lstat->lst_csem || !lstat->lst_cstart || lstat->lst_cend || *midrec) {
+				rc = -ENODATA;
+
+				/*
+				 * No compaction or cend before cstart or more than one cend
+				 * or cend mid-record.
+				 */
+				mp_pr_err("inconsistent compaction recs %u %u %u %d", rc,
+					  lstat->lst_csem, lstat->lst_cstart, lstat->lst_cend,
+					  *midrec);
+				return rc;
+			}
+			lstat->lst_cend = 1;
+		} else if (lrd.olr_rtype == OMF_LOGREC_EOLB) {
+			if (*midrec || !recnum) {
+				/* EOLB mid-record or first record. */
+				rc = -ENODATA;
+				mp_pr_err("end of log block marker at wrong place %d %lu",
+					  rc, *midrec, (ulong)recnum);
+				return rc;
+			}
+			/* No more records in log buffer */
+			break;
+		} else if (lrd.olr_rtype == OMF_LOGREC_DATAFULL) {
+			if (*midrec && recnum) {
+				rc = -ENODATA;
+
+				/*
+				 * Can occur mid data rec only if is first rec in log block
+				 * indicating partial data rec at end of last log block
+				 * which is a valid failure mode; otherwise is a logging
+				 * error.
+				 */
+				mp_pr_err("data full marker at wrong place %d %lu",
+					  rc, *midrec, (ulong)recnum);
+				return rc;
+			}
+			*midrec = 0;
+		} else if (lrd.olr_rtype == OMF_LOGREC_DATAFIRST) {
+			if (*midrec && recnum) {
+				rc = -ENODATA;
+
+				/* See comment for DATAFULL */
+				mp_pr_err("data first marker at wrong place %d %lu",
+					  rc, *midrec, (ulong)recnum);
+				return rc;
+			}
+			*midrec = 1;
+		} else if (lrd.olr_rtype == OMF_LOGREC_DATAMID) {
+			if (!*midrec) {
+				rc = -ENODATA;
+
+				/* Must occur mid data record. */
+				mp_pr_err("data mid marker at wrong place %d %lu",
+					  rc, *midrec, (ulong)recnum);
+				return rc;
+			}
+		} else if (lrd.olr_rtype == OMF_LOGREC_DATALAST) {
+			if (!(*midrec)) {
+				rc = -ENODATA;
+
+				/* Must occur mid data record */
+				mp_pr_err("data last marker at wrong place %d %lu",
+					  rc, *midrec, (ulong)recnum);
+				return rc;
+			}
+			*midrec = 0;
+		} else {
+			rc = -ENODATA;
+			mp_pr_err("unknown record type %d %lu", rc, lrd.olr_rtype, (ulong)recnum);
+			return rc;
+		}
+
+		recnum = recnum + 1;
+		recoff = recoff + OMF_LOGREC_DESC_PACKLEN + lrd.olr_rlen;
+	}
+
+	return rc;
+}
+
+static inline void max_cfsetid(struct omf_logblock_header *lbh,
+			       struct pmd_layout *layout, u32 *fsetid)
+{
+	if (!mpool_uuid_compare(&lbh->olh_magic, &layout->eld_uuid) &&
+	    (lbh->olh_gen == layout->eld_gen))
+		*fsetid  = max_t(u32, *fsetid, lbh->olh_cfsetid);
+}
+
+/**
+ * mlog_logpage_validate() - Validate log records at log page index 'rbidx' in the read buffer.
+ * @mlh:        mlog_descriptor
+ * @lstat:      mlog_stat
+ * @rbidx:      log page index in the read buffer to validate
+ * @nseclpg:    number of sectors in the log page @rbidx
+ * @midrec:     refer to mlog_logrecs_validate
+ * @leol_found: true, if LEOL found. false, if LEOL not found/log full (output)
+ * @fsetidmax:  maximum flush set ID found in the log (output)
+ * @pfsetid:    previous flush set ID, if LEOL found (output)
+ */
+static int mlog_logpage_validate(struct mlog_descriptor *mlh, struct mlog_stat *lstat,
+				 u16 rbidx, u16 nseclpg, int *midrec,
+				 bool *leol_found, u32 *fsetidmax, u32 *pfsetid)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	char *rbuf;
+	u16 lbidx;
+	u16 sectsz;
+
+	sectsz = MLOG_SECSZ(lstat);
+	rbuf   = lstat->lst_rbuf[rbidx];
+
+	/* Loop through nseclpg sectors in the log page @rbidx. */
+	for (lbidx = 0; lbidx < nseclpg; lbidx++) {
+		struct omf_logblock_header lbh;
+		int rc;
+
+		memset(&lbh, 0, sizeof(lbh));
+
+		(void)omf_logblock_header_unpack_letoh(&lbh, rbuf);
+
+		/*
+		 * If LEOL is already found, then this loop determines
+		 * fsetidmax, i.e., scans through the sectors to determine
+		 * any stale flush set id from a prior failed CFS flush.
+		 */
+		if (*leol_found) {
+			max_cfsetid(&lbh, layout, fsetidmax);
+			rbuf += sectsz;
+			continue;
+		}
+
+		/*
+		 * Check for LEOL based on prev and cur flush set ID.
+		 * If LEOL is detected, then no need to validate this and
+		 * the log blocks that follow.
+		 *
+		 * We issue DISCARD commands to erase mlogs. However the data
+		 * read from a discarded block is non-determinstic. It could be
+		 * all 0s, all 1s or last written data.
+		 *
+		 * We could read following 5 types of data from mlog:
+		 * 1) Garbage
+		 * 2) Stale logs with different log block gen
+		 * 3) Stale logs with different flushset ID
+		 * 4) Stale logs with different magic (UUID)
+		 * 5) Valid logs
+		 */
+		if (mpool_uuid_compare(&lbh.olh_magic, &layout->eld_uuid) ||
+		    (lbh.olh_gen != layout->eld_gen) || (lbh.olh_pfsetid != *fsetidmax)) {
+			*leol_found = true;
+			*pfsetid    = *fsetidmax;
+			rbuf       += sectsz;
+			max_cfsetid(&lbh, layout, fsetidmax);
+			continue;
+		}
+
+		*fsetidmax = lbh.olh_cfsetid;
+
+		/* Validate the log block at lbidx. */
+		rc = mlog_logrecs_validate(lstat, midrec, rbidx, lbidx);
+		if (rc) {
+			mp_pr_err("mlog %p,, midrec %d, log pg idx %u, sector idx %u",
+				  rc, mlh, *midrec, rbidx, lbidx);
+
+			return rc;
+		}
+
+		++lstat->lst_wsoff;
+		rbuf += sectsz;
+	}
+
+	return 0;
+}
+
+/**
+ * mlog_read_and_validate() - Read and validate mlog records
+ * @mp:     mpool descriptor
+ * @layout: layout descriptor
+ * @lempty: is the log empty? (output)
+ *
+ * Called by mlog_open() to read and validate log records in the mlog.
+ * In addition, determine the previous and current flush
+ * set ID to be used by the next flush.
+ *
+ * Note: this function reads the entire mlog. Doing so allows us to confirm that
+ * the mlog's contents are completely legit, and also to recognize the case
+ * where a compaction started but failed to complete (CSTART with no CEND) -
+ * for which the recovery is to use the other mlog of the mlpair.
+ * If the mlog is huge, or if there are a bazillion of them, this could be an
+ * issue to revisit in future performance or functionality optimizations.
+ *
+ * Transactional logs are expensive; this does some "extra" reading at open
+ * time, with some serious benefits.
+ *
+ * Caller must hold the write lock on the layout, which protects the mutation
+ * of the read buffer.
+ */
+static int mlog_read_and_validate(struct mpool_descriptor *mp,
+				  struct pmd_layout *layout, bool *lempty)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	off_t leol_off = 0, rsoff;
+	int midrec = 0, remsec;
+	bool leol_found = false;
+	bool fsetid_loop = false;
+	bool skip_ser = false;
+	u32 fsetidmax = 0;
+	u32 pfsetid = 0;
+	u16 maxsec, nsecs;
+	u16 nlpgs, nseclpg;
+	int rc = 0;
+
+	remsec = MLOG_TOTSEC(lstat);
+	maxsec = MLOG_NSECMB(lstat);
+	rsoff  = lstat->lst_wsoff;
+
+	while (remsec > 0) {
+		u16 rbidx;
+
+		nseclpg = MLOG_NSECLPG(lstat);
+		nsecs   = min_t(u32, maxsec, remsec);
+
+		rc = mlog_populate_rbuf(mp, layout, &nsecs, &rsoff, skip_ser);
+		if (rc) {
+			mp_pr_err("mpool %s, mlog 0x%lx validate failed, nsecs: %u, rsoff: 0x%lx",
+				  rc, mp->pds_name, (ulong)layout->eld_objid, nsecs, rsoff);
+
+			goto exit;
+		}
+
+		nlpgs = (nsecs + nseclpg - 1) / nseclpg;
+		lstat->lst_rsoff = rsoff;
+
+		/* Validate the read buffer, one log page at a time. */
+		for (rbidx = 0; rbidx < nlpgs; rbidx++) {
+
+			/* No. of sectors in the last log page. */
+			if (rbidx == nlpgs - 1) {
+				nseclpg = nsecs % nseclpg;
+				nseclpg = nseclpg > 0 ? nseclpg : MLOG_NSECLPG(lstat);
+			}
+
+			/* Validate the log block(s) in the log page @rbidx. */
+			rc = mlog_logpage_validate(layout2mlog(layout), lstat, rbidx, nseclpg,
+						   &midrec, &leol_found, &fsetidmax, &pfsetid);
+			if (rc) {
+				mp_pr_err("mpool %s, mlog 0x%lx rbuf validate failed, leol: %d, fsetidmax: %u, pfsetid: %u",
+					  rc, mp->pds_name, (ulong)layout->eld_objid, leol_found,
+					  fsetidmax, pfsetid);
+
+				mlog_free_rbuf(lstat, rbidx, nlpgs - 1);
+				goto exit;
+			}
+
+			mlog_free_rbuf(lstat, rbidx, rbidx);
+
+			/*
+			 * If LEOL is found, then note down the LEOL offset
+			 * and kick off the scan to identify any stale flush
+			 * set id from a prior failed flush. If there's one,
+			 * then the next flush set ID must be set one greater
+			 * than the stale fsetid.
+			 */
+			if (leol_found && !fsetid_loop) {
+				leol_off    = lstat->lst_wsoff;
+				fsetid_loop = true;
+			}
+		}
+
+		remsec -= nsecs;
+		if (remsec == 0)
+			break;
+		ASSERT(remsec > 0);
+
+		if (fsetid_loop) {
+			u16    compsec;
+			off_t  endoff;
+			/*
+			 * To determine the new flush set ID, we need to
+			 * scan only through the next min(MLOG_NSECMB, remsec)
+			 * sectors. This is because of the max flush size being
+			 * 1 MB and hence a failed flush wouldn't have touched
+			 * any sectors beyond 1 MB from LEOL.
+			 */
+			endoff  = rsoff + nsecs - 1;
+			compsec = endoff - leol_off + 1;
+			remsec  = min_t(u32, remsec, maxsec - compsec);
+			ASSERT(remsec >= 0);
+
+			rsoff = endoff + 1;
+		} else {
+			rsoff = lstat->lst_wsoff;
+		}
+	}
+
+	/* LEOL wouldn't have been set for a full log. */
+	if (!leol_found)
+		pfsetid = fsetidmax;
+
+	if (pfsetid != 0)
+		*lempty = false;
+
+	lstat->lst_pfsetid = pfsetid;
+	lstat->lst_cfsetid = fsetidmax + 1;
+
+exit:
+	lstat->lst_rsoff = -1;
+
+	return rc;
+}
+
+int mlog_open(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, u8 flags, u64 *gen)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	bool lempty, csem, skip_ser;
+	int rc = 0;
+
+	lempty = csem = skip_ser = false;
+	lstat = NULL;
+	*gen = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_wrlock(layout);
+
+	flags &= MLOG_OF_SKIP_SER | MLOG_OF_COMPACT_SEM;
+
+	if (flags & MLOG_OF_COMPACT_SEM)
+		csem = true;
+
+	if (flags & MLOG_OF_SKIP_SER)
+		skip_ser = true;
+
+	lstat = &layout->eld_lstat;
+
+	if (lstat->lst_abuf) {
+		/* Mlog already open */
+		if (csem && !lstat->lst_csem) {
+			pmd_obj_wrunlock(layout);
+
+			/* Re-open has inconsistent csem flag */
+			rc = -EINVAL;
+			mp_pr_err("mpool %s, re-opening of mlog 0x%lx, inconsistent csem %u %u",
+				  rc, mp->pds_name, (ulong)layout->eld_objid,
+				  csem, lstat->lst_csem);
+		} else if (skip_ser && !(layout->eld_flags & MLOG_OF_SKIP_SER)) {
+			pmd_obj_wrunlock(layout);
+
+			/* Re-open has inconsistent seralization flag */
+			rc = -EINVAL;
+			mp_pr_err("mpool %s, re-opening of mlog 0x%lx, inconsistent ser %u %u",
+				  rc, mp->pds_name, (ulong)layout->eld_objid, skip_ser,
+				  layout->eld_flags & MLOG_OF_SKIP_SER);
+		} else {
+			*gen = layout->eld_gen;
+			pmd_obj_wrunlock(layout);
+		}
+		return rc;
+	}
+
+	if (!(layout->eld_state & PMD_LYT_COMMITTED)) {
+		*gen = 0;
+		pmd_obj_wrunlock(layout);
+
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, mlog 0x%lx, not committed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	if (skip_ser)
+		layout->eld_flags |= MLOG_OF_SKIP_SER;
+
+	rc = mlog_stat_init(mp, mlh, csem);
+	if (rc) {
+		*gen = 0;
+		pmd_obj_wrunlock(layout);
+
+		mp_pr_err("mpool %s, mlog 0x%lx, mlog status initialization failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	lempty = true;
+
+	rc = mlog_read_and_validate(mp, layout, &lempty);
+	if (rc) {
+		mlog_stat_free(layout);
+		pmd_obj_wrunlock(layout);
+
+		mp_pr_err("mpool %s, mlog 0x%lx, mlog content validation failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	} else if (!lempty && csem) {
+		if (!lstat->lst_cstart) {
+			mlog_stat_free(layout);
+			pmd_obj_wrunlock(layout);
+
+			rc = -ENODATA;
+			mp_pr_err("mpool %s, mlog 0x%lx, compaction start missing",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+			return rc;
+		} else if (!lstat->lst_cend) {
+			mlog_stat_free(layout);
+			pmd_obj_wrunlock(layout);
+
+			/* Incomplete compaction */
+			rc = -EMSGSIZE;
+			mp_pr_err("mpool %s, mlog 0x%lx, incomplete compaction",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+			return rc;
+		}
+	}
+
+	*gen = layout->eld_gen;
+
+	/* TODO: Verify that the insert succeeded... */
+	oml_layout_lock(mp);
+	oml_layout_insert(mp, &layout->eld_mlpriv);
+	oml_layout_unlock(mp);
+
+	pmd_obj_wrunlock(layout);
+
+	return rc;
+}
+
+/**
+ * mlog_close() - Flush and close log and release resources; no op if log is not open.
+ *
+ * Returns: 0 on success; -errno otherwise
+ */
+int mlog_close(struct mpool_descriptor *mp, struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	bool skip_ser = false;
+	int rc = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	/*
+	 * Inform pre-compaction that there is no need to try to compact
+	 * an mpool MDC that would contain this mlog because it is closed.
+	 */
+	pmd_precompact_alsz(mp, layout->eld_objid, 0, 0);
+
+	pmd_obj_wrlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (!lstat->lst_abuf) {
+		pmd_obj_wrunlock(layout);
+
+		return 0; /* Log already closed */
+	}
+
+	/* Flush log if potentially dirty and remove layout from open list */
+	if (lstat->lst_abdirty) {
+		rc = mlog_logblocks_flush(mp, layout, skip_ser);
+		lstat->lst_abdirty = false;
+		if (rc)
+			mp_pr_err("mpool %s, mlog 0x%lx close, log block flush failed",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+	}
+
+	oml_layout_lock(mp);
+	oml_layout_remove(mp, layout->eld_objid);
+	oml_layout_unlock(mp);
+
+	mlog_stat_free(layout);
+
+	/* Reset Mlog flags */
+	layout->eld_flags &= (~MLOG_OF_SKIP_SER);
+
+	pmd_obj_wrunlock(layout);
+
+	return rc;
+}
+
+/**
+ * mlog_gen() - Get generation number for log; log can be open or closed.
+ *
+ * Returns: 0 if successful; -errno otherwise
+ */
+int mlog_gen(struct mlog_descriptor *mlh, u64 *gen)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+
+	*gen = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_rdlock(layout);
+	*gen = layout->eld_gen;
+	pmd_obj_rdunlock(layout);
+
+	return 0;
+}
+
+/**
+ * mlog_empty() - Determine if log is empty; log must be open.
+ *
+ * Returns: 0 if successful; -errno otherwise
+ */
+int mlog_empty(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, bool *empty)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	int rc = 0;
+
+	*empty = false;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_rdlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (lstat->lst_abuf) {
+		if ((!lstat->lst_wsoff &&
+		     (lstat->lst_aoff == OMF_LOGBLOCK_HDR_PACKLEN)))
+			*empty = true;
+	} else {
+		rc = -ENOENT;
+	}
+
+	pmd_obj_rdunlock(layout);
+
+	if (rc)
+		mp_pr_err("mpool %s, mlog 0x%lx empty: no mlog status",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+
+	return rc;
+}
+
+/**
+ * mlog_len() - Returns the raw mlog bytes consumed. log must be open.
+ *
+ * Need to account for both metadata and user bytes while computing the log length.
+ */
+static int mlog_len(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, u64 *len)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	int rc = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_rdlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (lstat->lst_abuf)
+		*len = ((u64) lstat->lst_wsoff * MLOG_SECSZ(lstat)) + lstat->lst_aoff;
+	else
+		rc = -ENOENT;
+
+	pmd_obj_rdunlock(layout);
+
+	if (rc)
+		mp_pr_err("mpool %s, mlog 0x%lx bytes consumed: no mlog status",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+
+	return rc;
+}
+
+/**
+ * mlog_erase() - Erase log setting generation number to max(current gen + 1, mingen).
+ *
+ * Log can be open or closed, but must be committed; operation is idempotent
+ * and can be retried if fails.
+ *
+ * Returns: 0 on success; -errno otherwise
+ */
+int mlog_erase(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, u64 mingen)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat = NULL;
+	u64 newgen = 0;
+	int rc = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_wrlock(layout);
+
+	/* Must be committed to log erase start/end markers */
+	if (!(layout->eld_state & PMD_LYT_COMMITTED)) {
+		pmd_obj_wrunlock(layout);
+
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, erasing mlog 0x%lx, mlog not committed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	newgen = max(layout->eld_gen + 1, mingen);
+
+	/* If successful updates state and gen in layout */
+	rc = pmd_obj_erase(mp, layout, newgen);
+	if (rc) {
+		pmd_obj_wrunlock(layout);
+
+		mp_pr_err("mpool %s, erasing mlog 0x%lx, logging erase start failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	rc = pmd_layout_erase(mp, layout);
+	if (rc) {
+		/*
+		 * Log the failure as a debugging message, but ignore the
+		 * failure, since discarding blocks here is only advisory
+		 */
+		mp_pr_debug("mpool %s, erasing mlog 0x%lx, erase failed ",
+			    rc, mp->pds_name, (ulong)layout->eld_objid);
+		rc = 0;
+	}
+
+	/* If successful updates state in layout */
+	lstat = &layout->eld_lstat;
+	if (lstat->lst_abuf) {
+		/* Log is open so need to update lstat info */
+		mlog_free_abuf(lstat, 0, lstat->lst_abidx);
+		mlog_free_rbuf(lstat, 0, MLOG_NLPGMB(lstat) - 1);
+
+		mlog_stat_init_common(layout, lstat);
+	}
+
+	pmd_obj_wrunlock(layout);
+
+	return rc;
+}
+
+/**
+ * mlog_append_marker() - Append a marker (log rec with zero-length data field) of type mtype.
+ *
+ * Returns: 0 on success; -errno otherwise
+ * One of the possible errno values:
+ * -EFBIG - if no room in log
+ */
+static int mlog_append_marker(struct mpool_descriptor *mp, struct pmd_layout *layout,
+			      enum logrec_type_omf mtype)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	struct omf_logrec_descriptor lrd;
+	u16 sectsz, abidx, aoff;
+	u16 asidx, nseclpg;
+	bool skip_ser = false;
+	char *abuf;
+	off_t lpgoff;
+	int rc;
+
+	sectsz  = MLOG_SECSZ(lstat);
+	nseclpg = MLOG_NSECLPG(lstat);
+
+	if (mlog_append_dmax(layout) == -1) {
+		/* Mlog is already full, flush whatever we can */
+		if (lstat->lst_abdirty) {
+			(void)mlog_logblocks_flush(mp, layout, skip_ser);
+			lstat->lst_abdirty = false;
+		}
+
+		return -EFBIG;
+	}
+
+	rc = mlog_update_append_idx(mp, layout, skip_ser);
+	if (rc)
+		return rc;
+
+	abidx  = lstat->lst_abidx;
+	abuf   = lstat->lst_abuf[abidx];
+	asidx  = lstat->lst_wsoff - ((nseclpg * abidx) + lstat->lst_asoff);
+	lpgoff = asidx * sectsz;
+	aoff   = lstat->lst_aoff;
+
+	lrd.olr_tlen  = 0;
+	lrd.olr_rlen  = 0;
+	lrd.olr_rtype = mtype;
+
+	ASSERT(abuf != NULL);
+
+	rc = omf_logrec_desc_pack_htole(&lrd, &abuf[lpgoff + aoff]);
+	if (!rc) {
+		lstat->lst_aoff = aoff + OMF_LOGREC_DESC_PACKLEN;
+
+		rc = mlog_logblocks_flush(mp, layout, skip_ser);
+		lstat->lst_abdirty = false;
+		if (rc)
+			mp_pr_err("mpool %s, mlog 0x%lx log block flush failed",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+	} else {
+		mp_pr_err("mpool %s, mlog 0x%lx log record descriptor packing failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+	}
+
+	return rc;
+}
+
+/**
+ * mlog_append_cstart() - Append compaction start marker; log must be open with csem flag true.
+ *
+ * Returns: 0 on success; -errno otherwise
+ * One of the possible errno values:
+ * -EFBIG - if no room in log
+ */
+int mlog_append_cstart(struct mpool_descriptor *mp, struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	int rc = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_wrlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (!lstat->lst_abuf) {
+		pmd_obj_wrunlock(layout);
+
+		rc = -ENOENT;
+		mp_pr_err("mpool %s, in mlog 0x%lx, inconsistency: no mlog status",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	if (!lstat->lst_csem || lstat->lst_cstart) {
+		pmd_obj_wrunlock(layout);
+
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, in mlog 0x%lx, inconsistent state %u %u",
+			  rc, mp->pds_name,
+			  (ulong)layout->eld_objid, lstat->lst_csem, lstat->lst_cstart);
+		return rc;
+	}
+
+	rc = mlog_append_marker(mp, layout, OMF_LOGREC_CSTART);
+	if (rc) {
+		pmd_obj_wrunlock(layout);
+
+		mp_pr_err("mpool %s, in mlog 0x%lx, marker append failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	lstat->lst_cstart = 1;
+	pmd_obj_wrunlock(layout);
+
+	return 0;
+}
+
+/**
+ * mlog_append_cend() - Append compaction start marker; log must be open with csem flag true.
+ *
+ * Returns: 0 on success; -errno otherwise
+ * One of the possible errno values:
+ * -EFBIG - if no room in log
+ */
+int mlog_append_cend(struct mpool_descriptor *mp, struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	int rc = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_wrlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (!lstat->lst_abuf) {
+		pmd_obj_wrunlock(layout);
+
+		rc = -ENOENT;
+		mp_pr_err("mpool %s, mlog 0x%lx, inconsistency: no mlog status",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	if (!lstat->lst_csem || !lstat->lst_cstart || lstat->lst_cend) {
+		pmd_obj_wrunlock(layout);
+
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, mlog 0x%lx, inconsistent state %u %u %u",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, lstat->lst_csem,
+			  lstat->lst_cstart, lstat->lst_cend);
+		return rc;
+	}
+
+	rc = mlog_append_marker(mp, layout, OMF_LOGREC_CEND);
+	if (rc) {
+		pmd_obj_wrunlock(layout);
+
+		mp_pr_err("mpool %s, mlog 0x%lx, marker append failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+		return rc;
+	}
+
+	lstat->lst_cend = 1;
+	pmd_obj_wrunlock(layout);
+
+	return 0;
+}
+
+/**
+ * memcpy_from_iov() - Moves contents from an iovec to one or more destination buffers.
+ * @iov    : One or more source buffers in the form of an iovec
+ * @buf    : Destination buffer
+ * @buflen : The length of either source or destination whichever is minimum
+ * @nextidx: The next index in iov if the copy requires multiple invocations
+ *           of memcpy_from_iov.
+ *
+ * No bounds check is done on iov. The caller is expected to give the minimum
+ * of source and destination buffers as the length (buflen) here.
+ */
+static void memcpy_from_iov(struct kvec *iov, char *buf, size_t buflen, int *nextidx)
+{
+	int i = *nextidx, cp;
+
+	if ((buflen > 0) && (iov[i].iov_len == 0))
+		i++;
+
+	while (buflen > 0) {
+		cp = (buflen < iov[i].iov_len) ? buflen : iov[i].iov_len;
+
+		if (iov[i].iov_base)
+			memcpy(buf, iov[i].iov_base, cp);
+
+		iov[i].iov_len  -= cp;
+		iov[i].iov_base += cp;
+		buflen          -= cp;
+		buf             += cp;
+
+		if (iov[i].iov_len == 0)
+			i++;
+	}
+
+	*nextidx = i;
+}
+
+/**
+ * mlog_append_data_internal() - Append data record with buflen data bytes from buf.
+ * @mp:       mpool descriptor
+ * @mlh:      mlog descriptor
+ * @iov:      iovec containing user data
+ * @buflen:   length of the user buffer
+ * @sync:     if true, then we do not return until data is on media
+ * @skip_ser: client guarantees serialization
+ *
+ * Log must be open; if log opened with csem true then a compaction
+ * start marker must be in place;
+ *
+ * Returns: 0 on success; -errno otherwise
+ * One of the possible errno values:
+ * -EFBIG - if no room in log
+ */
+static int mlog_append_data_internal(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+				     struct kvec *iov, u64 buflen, int sync, bool skip_ser)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	struct omf_logrec_descriptor lrd;
+	int rc = 0, dfirst, cpidx;
+	u32 datasec;
+	u64 bufoff, rlenmax;
+	u16 aoff, abidx, asidx;
+	u16 nseclpg, sectsz;
+	off_t lpgoff;
+	char *abuf;
+
+	mlog_extract_fsetparms(lstat, &sectsz, &datasec, NULL, &nseclpg);
+
+	bufoff = 0;
+	dfirst = 1;
+	cpidx  = 0;
+
+	lrd.olr_tlen = buflen;
+
+	while (true) {
+		if ((bufoff != buflen) && (mlog_append_dmax(layout) == -1)) {
+
+			/* Mlog is full and there's more to write;
+			 * mlog_append_dmax() should prevent this, but it lied.
+			 */
+			mp_pr_warn("mpool %s, mlog 0x%lx append, mlog free space incorrect",
+				   mp->pds_name, (ulong)layout->eld_objid);
+
+			return -EFBIG;
+		}
+
+		rc = mlog_update_append_idx(mp, layout, skip_ser);
+		if (rc)
+			return rc;
+
+		abidx  = lstat->lst_abidx;
+		abuf   = lstat->lst_abuf[abidx];
+		asidx  = lstat->lst_wsoff - ((nseclpg * abidx) + lstat->lst_asoff);
+		lpgoff = asidx * sectsz;
+		aoff   = lstat->lst_aoff;
+
+		ASSERT(abuf != NULL);
+
+		rlenmax = min((u64)(sectsz - aoff - OMF_LOGREC_DESC_PACKLEN),
+			      (u64)OMF_LOGREC_DESC_RLENMAX);
+
+		if (buflen - bufoff <= rlenmax) {
+			lrd.olr_rlen = buflen - bufoff;
+			if (dfirst)
+				lrd.olr_rtype = OMF_LOGREC_DATAFULL;
+			else
+				lrd.olr_rtype = OMF_LOGREC_DATALAST;
+		} else {
+			lrd.olr_rlen = rlenmax;
+			if (dfirst) {
+				lrd.olr_rtype = OMF_LOGREC_DATAFIRST;
+				dfirst = 0;
+			} else {
+				lrd.olr_rtype = OMF_LOGREC_DATAMID;
+			}
+		}
+
+		rc = omf_logrec_desc_pack_htole(&lrd, &abuf[lpgoff + aoff]);
+		if (rc) {
+			mp_pr_err("mpool %s, mlog 0x%lx, log record packing failed",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+			break;
+		}
+
+		lstat->lst_abdirty = true;
+
+		aoff = aoff + OMF_LOGREC_DESC_PACKLEN;
+		if (lrd.olr_rlen) {
+			memcpy_from_iov(iov, &abuf[lpgoff + aoff], lrd.olr_rlen, &cpidx);
+			aoff   = aoff + lrd.olr_rlen;
+			bufoff = bufoff + lrd.olr_rlen;
+		}
+		lstat->lst_aoff = aoff;
+
+		/*
+		 * Flush log block if sync and no more to write (or)
+		 * if the CFS is full.
+		 */
+		if ((sync && buflen == bufoff) ||
+			(abidx == MLOG_NLPGMB(lstat) - 1 && asidx == nseclpg - 1 &&
+			 sectsz - aoff < OMF_LOGREC_DESC_PACKLEN)) {
+
+			rc = mlog_logblocks_flush(mp, layout, skip_ser);
+			lstat->lst_abdirty = false;
+			if (rc) {
+				mp_pr_err("mpool %s, mlog 0x%lx, log block flush failed",
+					  rc, mp->pds_name, (ulong)layout->eld_objid);
+				break;
+			}
+		}
+
+		ASSERT(rc == 0);
+
+		if (bufoff == buflen)
+			break;
+	}
+
+	return rc;
+}
+
+static int mlog_append_datav(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+			     struct kvec *iov, u64 buflen, int sync)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	s64 dmax = 0;
+	bool skip_ser = false;
+	int rc = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	if (layout->eld_flags & MLOG_OF_SKIP_SER)
+		skip_ser = true;
+
+	if (!skip_ser)
+		pmd_obj_wrlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (!lstat->lst_abuf) {
+		rc = -ENOENT;
+		mp_pr_err("mpool %s, mlog 0x%lx, inconsistency: no mlog status",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+	} else if (lstat->lst_csem && !lstat->lst_cstart) {
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, mlog 0x%lx, inconsistent state %u %u", rc, mp->pds_name,
+			  (ulong)layout->eld_objid, lstat->lst_csem, lstat->lst_cstart);
+	} else {
+		dmax = mlog_append_dmax(layout);
+		if (dmax < 0 || buflen > dmax) {
+			rc = -EFBIG;
+			mp_pr_debug("mpool %s, mlog 0x%lx mlog full %ld",
+				    rc, mp->pds_name, (ulong)layout->eld_objid, (long)dmax);
+
+			/* Flush whatever we can. */
+			if (lstat->lst_abdirty) {
+				(void)mlog_logblocks_flush(mp, layout, skip_ser);
+				lstat->lst_abdirty = false;
+			}
+		}
+	}
+
+	if (rc) {
+		if (!skip_ser)
+			pmd_obj_wrunlock(layout);
+		return rc;
+	}
+
+	rc = mlog_append_data_internal(mp, mlh, iov, buflen, sync, skip_ser);
+	if (rc) {
+		mp_pr_err("mpool %s, mlog 0x%lx append failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+
+		/* Flush whatever we can. */
+		if (lstat->lst_abdirty) {
+			(void)mlog_logblocks_flush(mp, layout, skip_ser);
+			lstat->lst_abdirty = false;
+		}
+	}
+
+	if (!skip_ser)
+		pmd_obj_wrunlock(layout);
+
+	return rc;
+}
+
+int mlog_append_data(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+		     char *buf, u64 buflen, int sync)
+{
+	struct kvec iov;
+
+	iov.iov_base = buf;
+	iov.iov_len  = buflen;
+
+	return mlog_append_datav(mp, mlh, &iov, buflen, sync);
+}
+
+/**
+ * mlog_read_data_init() - Initialize iterator for reading data records from log.
+ *
+ * Log must be open; skips non-data records (markers).
+ *
+ * Returns: 0 on success; -errno otherwise
+ */
+int mlog_read_data_init(struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_stat *lstat;
+	struct mlog_read_iter *lri;
+	int rc = 0;
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_wrlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (!lstat->lst_abuf) {
+		rc = -ENOENT;
+	} else {
+		lri = &lstat->lst_citr;
+
+		mlog_read_iter_init(layout, lstat, lri);
+	}
+
+	pmd_obj_wrunlock(layout);
+
+	return rc;
+}
+
+/**
+ * mlog_read_data_next_impl() -
+ * @mp:
+ * @mlh:
+ * @skip:
+ * @buf:
+ * @buflen:
+ * @rdlen:
+ *
+ * Return:
+ *   -EOVERFLOW: the caller must retry with a larger receive buffer,
+ *   the length of an adequate receive buffer is returned in "rdlen".
+ */
+static int mlog_read_data_next_impl(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+				    bool skip, char *buf, u64 buflen, u64 *rdlen)
+{
+	struct omf_logrec_descriptor lrd;
+	struct mlog_read_iter *lri = NULL;
+	struct pmd_layout *layout;
+	struct mlog_stat *lstat;
+
+	u64 bufoff = 0, midrec = 0;
+	bool recfirst = false;
+	bool skip_ser = false;
+	char *inbuf = NULL;
+	u32 sectsz = 0;
+	int rc = 0;
+
+	layout = mlog2layout(mlh);
+	if (!layout)
+		return -EINVAL;
+
+	if (!mlog_objid(layout->eld_objid))
+		return -EINVAL;
+
+	if (layout->eld_flags & MLOG_OF_SKIP_SER)
+		skip_ser = true;
+	/*
+	 * Need write lock because loading log block to read updates lstat.
+	 * Currently have no use case requiring support for concurrent readers.
+	 */
+	if (!skip_ser)
+		pmd_obj_wrlock(layout);
+
+	lstat = &layout->eld_lstat;
+	if (lstat->lst_abuf) {
+		sectsz = MLOG_SECSZ(lstat);
+		lri    = &lstat->lst_citr;
+
+		if (!lri->lri_valid) {
+			if (!skip_ser)
+				pmd_obj_wrunlock(layout);
+
+			rc = -EINVAL;
+			mp_pr_err("mpool %s, mlog 0x%lx, invalid iterator",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+			return rc;
+		}
+	}
+
+	if (!lstat || !lri) {
+		rc = -ENOENT;
+		mp_pr_err("mpool %s, mlog 0x%lx, inconsistency: no mlog status",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+	} else if (lri->lri_gen != layout->eld_gen ||
+		   lri->lri_soff > lstat->lst_wsoff ||
+		   (lri->lri_soff == lstat->lst_wsoff && lri->lri_roff > lstat->lst_aoff) ||
+		   lri->lri_roff > sectsz) {
+
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, mlog 0x%lx, invalid args gen %lu %lu offsets %ld %ld %u %u %u",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, (ulong)lri->lri_gen,
+			  (ulong)layout->eld_gen, lri->lri_soff, lstat->lst_wsoff, lri->lri_roff,
+			  lstat->lst_aoff, sectsz);
+	} else if (lri->lri_soff == lstat->lst_wsoff && lri->lri_roff == lstat->lst_aoff) {
+		/* Hit end of log - do not error count */
+		rc = -ENOMSG;
+	}
+
+	if (rc) {
+		if (!skip_ser)
+			pmd_obj_wrunlock(layout);
+		if (rc == -ENOMSG) {
+			rc = 0;
+			if (rdlen)
+				*rdlen = 0;
+		}
+
+		return rc;
+	}
+
+	bufoff = 0;
+	midrec = 0;
+
+	while (true) {
+		/* Get log block referenced by lri which can be accumulating buffer */
+		rc = mlog_logblock_load(mp, lri, &inbuf, &recfirst);
+		if (rc) {
+			if (rc == -ENOMSG) {
+				if (!skip_ser)
+					pmd_obj_wrunlock(layout);
+				rc = 0;
+				if (rdlen)
+					*rdlen = 0;
+
+				return rc;
+			}
+
+			mp_pr_err("mpool %s, mlog 0x%lx, getting log block failed",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+			break;
+		}
+
+		if ((sectsz - lri->lri_roff) < OMF_LOGREC_DESC_PACKLEN) {
+			/* No more records in current log block */
+			if (lri->lri_soff < lstat->lst_wsoff) {
+
+				/* Move to next log block */
+				lri->lri_soff = lri->lri_soff + 1;
+				lri->lri_roff = 0;
+				continue;
+			} else {
+				/*
+				 * hit end of log; return EOF even in case
+				 * of a partial data record which is a valid
+				 * failure mode and must be ignored
+				 */
+				if (bufoff)
+					rc = -ENODATA;
+
+				bufoff = 0;	/* Force EOF on partials! */
+				break;
+			}
+		}
+
+		/* Parse next record in log block */
+		omf_logrec_desc_unpack_letoh(&lrd, &inbuf[lri->lri_roff]);
+
+		if (logrec_type_datarec(lrd.olr_rtype)) {
+			/* Data record */
+			if (lrd.olr_rtype == OMF_LOGREC_DATAFULL ||
+			    lrd.olr_rtype == OMF_LOGREC_DATAFIRST) {
+				if (midrec && !recfirst) {
+					rc = -ENODATA;
+
+					/*
+					 * Can occur mid data rec only if is first rec in log
+					 * block indicating partial data rec at end of last
+					 * block which is a valid failure mode,
+					 * Otherwise is a logging error
+					 */
+					mp_pr_err("mpool %s, mlog 0x%lx, inconsistent 1 data rec",
+						  rc, mp->pds_name, (ulong)layout->eld_objid);
+					break;
+				}
+				/*
+				 * Reset copy-out; set midrec which is needed for DATAFIRST
+				 */
+				bufoff = 0;
+				midrec = 1;
+			} else if (lrd.olr_rtype == OMF_LOGREC_DATAMID ||
+				   lrd.olr_rtype == OMF_LOGREC_DATALAST) {
+				if (!midrec) {
+					rc = -ENODATA;
+
+					/* Must occur mid data record. */
+					mp_pr_err("mpool %s, mlog 0x%lx, inconsistent 2 data rec",
+						  rc, mp->pds_name, (ulong)layout->eld_objid);
+					break;
+				}
+			}
+
+			/*
+			 * This is inside a loop, but it is invariant;
+			 * (and it cannot be done until after the unpack)
+			 *
+			 * Return the necessary length to caller.
+			 */
+			if (buflen < lrd.olr_tlen) {
+				if (rdlen)
+					*rdlen = lrd.olr_tlen;
+
+				rc = -EOVERFLOW;
+				break;
+			}
+
+			/* Copy-out data */
+			lri->lri_roff = lri->lri_roff + OMF_LOGREC_DESC_PACKLEN;
+
+			if (!skip)
+				memcpy(&buf[bufoff], &inbuf[lri->lri_roff], lrd.olr_rlen);
+
+			lri->lri_roff = lri->lri_roff + lrd.olr_rlen;
+			bufoff = bufoff + lrd.olr_rlen;
+
+			if (lrd.olr_rtype == OMF_LOGREC_DATAFULL ||
+			    lrd.olr_rtype == OMF_LOGREC_DATALAST)
+				break;
+		} else {
+			/*
+			 * Non data record; just skip unless midrec which is a logging error
+			 */
+			if (midrec) {
+				rc = -ENODATA;
+				mp_pr_err("mpool %s, mlog 0x%lx, inconsistent non-data record",
+					  rc, mp->pds_name, (ulong)layout->eld_objid);
+				break;
+			}
+			if (lrd.olr_rtype == OMF_LOGREC_EOLB)
+				lri->lri_roff = sectsz;
+			else
+				lri->lri_roff = lri->lri_roff + OMF_LOGREC_DESC_PACKLEN +
+					lrd.olr_rlen;
+		}
+	}
+	if (!rc && rdlen)
+		*rdlen = bufoff;
+	else if (rc != -EOVERFLOW && rc != -ENOMEM)
+		/* Handle only remains valid if buffer too small */
+		lri->lri_valid = 0;
+
+	if (!skip_ser)
+		pmd_obj_wrunlock(layout);
+
+	return rc;
+}
+
+/**
+ * mlog_read_data_next() - Read next data record into buffer buf of length buflen bytes.
+ *
+ * Log must be open; skips non-data records (markers).
+ *
+ * Iterator lri must be re-init if returns any error except ENOMEM
+ *
+ * Returns:
+ *   0 on success; The following errno values on failure:
+ *   -EOVERFLOW if buflen is insufficient to hold data record; can retry
+ *   errno otherwise
+ *
+ *   Bytes read on success in the output param rdlen (can be 0 if appended a
+ *   zero-length data record)
+ */
+int mlog_read_data_next(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+			char *buf, u64 buflen, u64 *rdlen)
+{
+	return mlog_read_data_next_impl(mp, mlh, false, buf, buflen, rdlen);
+}
+
+/**
+ * mlog_get_props() - Return basic mlog properties in prop.
+ *
+ * Returns: 0 if successful; -errno otherwise
+ */
+static int mlog_get_props(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+			  struct mlog_props *prop)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+
+	if (!layout)
+		return -EINVAL;
+
+	pmd_obj_rdlock(layout);
+	mlog_getprops_cmn(mp, layout, prop);
+	pmd_obj_rdunlock(layout);
+
+	return 0;
+}
+
+/**
+ * mlog_get_props_ex() - Return extended mlog properties in prop.
+ *
+ * Returns: 0 if successful; -errno otherwise
+ */
+int mlog_get_props_ex(struct mpool_descriptor *mp, struct mlog_descriptor  *mlh,
+		      struct mlog_props_ex *prop)
+{
+	struct pmd_layout *layout;
+	struct pd_prop *pdp;
+
+	layout = mlog2layout(mlh);
+	if (!layout)
+		return -EINVAL;
+
+	pdp = &mp->pds_pdv[layout->eld_ld.ol_pdh].pdi_prop;
+
+	pmd_obj_rdlock(layout);
+	mlog_getprops_cmn(mp, layout, &prop->lpx_props);
+	prop->lpx_zonecnt  = layout->eld_ld.ol_zcnt;
+	prop->lpx_state    = layout->eld_state;
+	prop->lpx_secshift = PD_SECTORSZ(pdp);
+	prop->lpx_totsec   = pmd_layout_cap_get(mp, layout) >> prop->lpx_secshift;
+	pmd_obj_rdunlock(layout);
+
+	return 0;
+}
+
+void mlog_precompact_alsz(struct mpool_descriptor *mp, struct mlog_descriptor *mlh)
+{
+	struct mlog_props prop;
+	u64 len;
+	int rc;
+
+	rc = mlog_get_props(mp, mlh, &prop);
+	if (rc)
+		return;
+
+	rc = mlog_len(mp, mlh, &len);
+	if (rc)
+		return;
+
+	pmd_precompact_alsz(mp, prop.lpr_objid, len, prop.lpr_alloc_cap);
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:24 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:24 -0500
Subject: [PATCH v2 10/22] mpool: add mlog IO utility routines
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-11-nmeeramohide@micron.com>

Adds buffer management routines used by the mlog IO path.

Mlog objects are containers for record logging. An mlog is
structured as a series of consecutive log blocks where each
log block is exactly one sector in size. Records of
arbitrary size can be appended to an mlog until it's full.

Mlog uses a flush set algorithm for implementing multi-sector
append and read buffers. A flush set is one or more sector-
size memory buffers containing consecutive log block data
with newly appended records written together in a single
logical I/O. The flush set algorithm maintains flush set
identifiers in the headers of the log blocks comprising the
append buffer and handles failures when writing these log
blocks.

The flush set grows up to the append buffer size providing
better throughput for async appends. Once the flush set is
full or when the client issues a synchronous append or mlog
flush, the flush set is written to media and the buffer pages
are freed/prepared for the next append depending on the I/O
outcome.

The read buffer is also provisioned like the append buffer.
The read buffer management is optimized for the expected use
cases, which is open a log, read once to load persisted state
into memory and thereafter only append while open. The log
blocks are loaded into the read buffer up to but not including
the offset of the first log block in the append buffer.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mlog_utils.c | 1352 ++++++++++++++++++++++++++++++++++++
 drivers/mpool/mlog_utils.h |   63 ++
 2 files changed, 1415 insertions(+)
 create mode 100644 drivers/mpool/mlog_utils.c
 create mode 100644 drivers/mpool/mlog_utils.h

diff --git a/drivers/mpool/mlog_utils.c b/drivers/mpool/mlog_utils.c
new file mode 100644
index 000000000000..3fae46b5a1c3
--- /dev/null
+++ b/drivers/mpool/mlog_utils.c
@@ -0,0 +1,1352 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#include <linux/mm.h>
+#include <linux/log2.h>
+#include <linux/blk_types.h>
+#include <linux/rbtree.h>
+#include <linux/slab.h>
+#include <asm/page.h>
+
+#include "assert.h"
+#include "mpool_printk.h"
+
+#include "pmd_obj.h"
+#include "mpcore.h"
+#include "mlog.h"
+#include "mlog_utils.h"
+
+#define mlpriv2layout(_ptr) \
+	((struct pmd_layout *)((char *)(_ptr) - offsetof(struct pmd_layout, eld_priv)))
+
+bool mlog_objid(u64 objid)
+{
+	return objid && pmd_objid_type(objid) == OMF_OBJ_MLOG;
+}
+
+/**
+ * mlog2layout() - convert opaque mlog handle to pmd_layout
+ *
+ * This function converts the opaque handle (mlog_descriptor) used by
+ * clients to the internal representation (pmd_layout).  The
+ * conversion is a simple cast, followed by a sanity check to verify the
+ * layout object is an mlog object.  If the validation fails, a NULL
+ * pointer is returned.
+ */
+struct pmd_layout *mlog2layout(struct mlog_descriptor *mlh)
+{
+	struct pmd_layout *layout = (void *)mlh;
+
+	return mlog_objid(layout->eld_objid) ? layout : NULL;
+}
+
+/**
+ * layout2mlog() - convert pmd_layout to opaque mlog_descriptor
+ *
+ * This function converts the internally used pmd_layout to
+ * the externally used opaque mlog_descriptor.
+ */
+struct mlog_descriptor *layout2mlog(struct pmd_layout *layout)
+{
+	return (struct mlog_descriptor *)layout;
+}
+
+static struct pmd_layout_mlpriv *oml_layout_find(struct mpool_descriptor *mp, u64 key)
+{
+	struct pmd_layout_mlpriv *this;
+	struct pmd_layout *layout;
+	struct rb_node *node;
+
+	node = mp->pds_oml_root.rb_node;
+	while (node) {
+		this = rb_entry(node, typeof(*this), mlp_nodeoml);
+		layout = mlpriv2layout(this);
+
+		if (key < layout->eld_objid)
+			node = node->rb_left;
+		else if (key > layout->eld_objid)
+			node = node->rb_right;
+		else
+			return this;
+	}
+
+	return NULL;
+}
+
+struct pmd_layout_mlpriv *oml_layout_insert(struct mpool_descriptor *mp,
+					    struct pmd_layout_mlpriv *item)
+{
+	struct pmd_layout_mlpriv *this;
+	struct pmd_layout *layout;
+	struct rb_node **pos, *parent;
+	struct rb_root *root;
+	u64 key;
+
+	root = &mp->pds_oml_root;
+	pos = &root->rb_node;
+	parent = NULL;
+
+	key = mlpriv2layout(item)->eld_objid;
+
+	while (*pos) {
+		this = rb_entry(*pos, typeof(*this), mlp_nodeoml);
+		layout = mlpriv2layout(this);
+
+		parent = *pos;
+		if (key < layout->eld_objid)
+			pos = &(*pos)->rb_left;
+		else if (key > layout->eld_objid)
+			pos = &(*pos)->rb_right;
+		else
+			return this;
+	}
+
+	/* Add new node and rebalance tree. */
+	rb_link_node(&item->mlp_nodeoml, parent, pos);
+	rb_insert_color(&item->mlp_nodeoml, root);
+
+	return NULL;
+}
+
+struct pmd_layout_mlpriv *oml_layout_remove(struct mpool_descriptor *mp, u64 key)
+{
+	struct pmd_layout_mlpriv *found;
+
+	found = oml_layout_find(mp, key);
+	if (found)
+		rb_erase(&found->mlp_nodeoml, &mp->pds_oml_root);
+
+	return found;
+}
+
+/**
+ * mlog_free_abuf() - Free log pages in the append buffer, range:[start, end].
+ * @lstat: mlog_stat
+ * @start: start log page index, inclusive
+ * @end:   end log page index, inclusive
+ */
+void mlog_free_abuf(struct mlog_stat *lstat, int start, int end)
+{
+	int i;
+
+	for (i = start; i <= end; i++) {
+		if (lstat->lst_abuf[i]) {
+			free_page((unsigned long)lstat->lst_abuf[i]);
+			lstat->lst_abuf[i] = NULL;
+		}
+	}
+}
+
+/**
+ * mlog_free_rbuf() - Free log pages in the read buffer, range:[start, end].
+ * @lstat: mlog_stat
+ * @start: start log page index, inclusive
+ * @end:   end log page index, inclusive
+ */
+void mlog_free_rbuf(struct mlog_stat *lstat, int start, int end)
+{
+	int i;
+
+	for (i = start; i <= end; i++) {
+		if (lstat->lst_rbuf[i]) {
+			free_page((unsigned long)lstat->lst_rbuf[i]);
+			lstat->lst_rbuf[i] = NULL;
+		}
+	}
+}
+
+/**
+ * mlog_init_fsetparms() - Initialize frequently used mlog & flush set parameters.
+ * @mp:     mpool descriptor
+ * @layout: layout descriptor
+ * @mfp:    fset parameters (output)
+ */
+static void mlog_init_fsetparms(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+				struct mlog_fsetparms *mfp)
+{
+	struct pmd_layout *layout;
+	struct pd_prop *pdp;
+	u8 secshift;
+	u16 sectsz;
+
+	layout = mlog2layout(mlh);
+	ASSERT(layout);
+
+	pdp = &mp->pds_pdv[layout->eld_ld.ol_pdh].pdi_prop;
+	secshift = PD_SECTORSZ(pdp);
+	mfp->mfp_totsec = pmd_layout_cap_get(mp, layout) >> secshift;
+
+	sectsz = 1 << secshift;
+	ASSERT((sectsz == PAGE_SIZE) || (sectsz == 512));
+
+	mfp->mfp_sectsz  = sectsz;
+	mfp->mfp_lpgsz   = PAGE_SIZE;
+	mfp->mfp_secpga  = IS_ALIGNED(mfp->mfp_sectsz, mfp->mfp_lpgsz);
+	mfp->mfp_nlpgmb  = MB >> PAGE_SHIFT;
+	mfp->mfp_nsecmb  = MB >> secshift;
+	mfp->mfp_nseclpg = mfp->mfp_lpgsz >> secshift;
+}
+
+/**
+ * mlog_extract_fsetparms() - Helper to extract flush set parameters.
+ * @lstat:   mlog stat
+ * @sectsz:  sector size
+ * @totsec:  total number of sectors in the mlog
+ * @nsecmb:  number of sectors in 1 MiB
+ * @nseclpg: number of sectors in a log page
+ */
+void
+mlog_extract_fsetparms(struct mlog_stat *lstat, u16 *sectsz, u32 *totsec, u16 *nsecmb, u16 *nseclpg)
+{
+	if (sectsz)
+		*sectsz = MLOG_SECSZ(lstat);
+	if (totsec)
+		*totsec = MLOG_TOTSEC(lstat);
+	if (nsecmb)
+		*nsecmb = MLOG_NSECMB(lstat);
+	if (nseclpg)
+		*nseclpg = MLOG_NSECLPG(lstat);
+}
+
+/**
+ * mlog_stat_free() - Deallocate log stat struct for mlog layout (if any)
+ */
+void mlog_stat_free(struct pmd_layout *layout)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+
+	if (!lstat->lst_abuf)
+		return;
+
+	mlog_free_rbuf(lstat, 0, MLOG_NLPGMB(lstat) - 1);
+	mlog_free_abuf(lstat, 0, MLOG_NLPGMB(lstat) - 1);
+
+	kfree(lstat->lst_abuf);
+	lstat->lst_abuf = NULL;
+}
+
+/**
+ * mlog_read_iter_init() - Initialize read iterator
+ *
+ * @layout: mlog layout
+ * @lstat:  mlog stat
+ * @lri"    mlog read iterator
+ */
+void
+mlog_read_iter_init(struct pmd_layout *layout, struct mlog_stat *lstat, struct mlog_read_iter *lri)
+{
+	lri->lri_layout = layout;
+	lri->lri_gen    = layout->eld_gen;
+	lri->lri_soff   = 0;
+	lri->lri_roff   = 0;
+	lri->lri_valid  = 1;
+	lri->lri_rbidx  = 0;
+	lri->lri_sidx   = 0;
+
+	lstat->lst_rsoff  = -1;
+	lstat->lst_rseoff = -1;
+}
+
+/**
+ * mlog_stat_init_common() - Initialize mlog_stat fields.
+ * @layout:
+ * @lstat: mlog_stat
+ */
+void mlog_stat_init_common(struct pmd_layout *layout, struct mlog_stat *lstat)
+{
+	struct mlog_read_iter *lri;
+
+	lstat->lst_pfsetid = 0;
+	lstat->lst_cfsetid = 1;
+	lstat->lst_abidx   = 0;
+	lstat->lst_asoff   = -1;
+	lstat->lst_cfssoff = OMF_LOGBLOCK_HDR_PACKLEN;
+	lstat->lst_aoff    = OMF_LOGBLOCK_HDR_PACKLEN;
+	lstat->lst_abdirty = false;
+	lstat->lst_wsoff   = 0;
+	lstat->lst_cstart  = 0;
+	lstat->lst_cend    = 0;
+
+	lri = &lstat->lst_citr;
+	mlog_read_iter_init(layout, lstat, lri);
+}
+
+/**
+ * mlog_rw_raw() - Called by mpctl kernel for mlog IO.
+ * @mp:     mpool descriptor
+ * @mlh:    mlog descriptor
+ * @iov:    iovec
+ * @iovcnt: iov cnt
+ * @boff:   IO offset
+ * @rw:     MPOOL_OP_READ or MPOOL_OP_WRITE
+ *
+ * The scatter-gather buffer must contain
+ * framed mlog data (this is done in user space for user space mlogs).
+ */
+int mlog_rw_raw(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+		const struct kvec *iov, int iovcnt, u64 boff, u8 rw)
+{
+	struct pmd_layout *layout;
+	int flags;
+	int rc;
+
+	layout = mlog2layout(mlh);
+	if (!layout)
+		return -EINVAL;
+
+	flags = (rw == MPOOL_OP_WRITE) ? REQ_FUA : 0;
+
+	pmd_obj_wrlock(layout);
+	rc = pmd_layout_rw(mp, layout, iov, iovcnt, boff, flags, rw);
+	pmd_obj_wrunlock(layout);
+
+	return rc;
+}
+
+/**
+ * mlog_rw() -
+ * @mp:       mpool descriptor
+ * @mlh:      mlog descriptor
+ * @iov:      iovec
+ * @iovcnt:   iov cnt
+ * @boff:     IO offset
+ * @rw:       MPOOL_OP_READ or MPOOL_OP_WRITE
+ * @skip_ser: client guarantees serialization
+ */
+static int mlog_rw(struct mpool_descriptor *mp, struct mlog_descriptor *mlh,
+		   struct kvec *iov, int iovcnt, u64 boff, u8 rw, bool skip_ser)
+{
+	struct pmd_layout *layout;
+
+	layout = mlog2layout(mlh);
+	if (!layout)
+		return -EINVAL;
+
+	if (!skip_ser) {
+		int flags = (rw == MPOOL_OP_WRITE) ? REQ_FUA : 0;
+
+		return pmd_layout_rw(mp, layout, iov, iovcnt, boff, flags, rw);
+	}
+
+	return mlog_rw_raw(mp, mlh, iov, iovcnt, boff, rw);
+}
+
+/**
+ * mlog_stat_init() - Allocate and init log stat struct for mlog layout.
+ *
+ * Returns: 0 if successful, -errno otherwise
+ */
+int mlog_stat_init(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, bool csem)
+{
+	struct pmd_layout *layout = mlog2layout(mlh);
+	struct mlog_fsetparms mfp;
+	struct mlog_stat *lstat;
+	size_t bufsz;
+	int rc;
+
+	if (!layout)
+		return -EINVAL;
+
+	lstat = &layout->eld_lstat;
+
+	mlog_stat_init_common(layout, lstat);
+	mlog_init_fsetparms(mp, mlh, &mfp);
+
+	bufsz = mfp.mfp_nlpgmb * sizeof(char *) * 2;
+
+	lstat->lst_abuf = kzalloc(bufsz, GFP_KERNEL);
+	if (!lstat->lst_abuf) {
+		rc = -ENOMEM;
+		mp_pr_err("mpool %s, allocating mlog 0x%lx status failed %zu",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, bufsz);
+		return rc;
+	}
+
+	lstat->lst_rbuf = lstat->lst_abuf + mfp.mfp_nlpgmb;
+	lstat->lst_mfp  = mfp;
+	lstat->lst_csem = csem;
+
+	return 0;
+}
+
+/**
+ * mlog_setup_buf() - Build an iovec list to read into an mlog read buffer, or write from
+ * an mlog append buffer.
+ * @lstat:   mlog_stat
+ * @riov:    iovec (output)
+ * @iovcnt:  number of iovecs
+ * @l_iolen: IO length for the last log page in the buffer
+ * @op:      MPOOL_OP_READ or MPOOL_OP_WRITE
+ *
+ * In the read case, the read buffer pages will be allocated if not already populated.
+ */
+static int
+mlog_setup_buf(struct mlog_stat *lstat, struct kvec **riov, u16 iovcnt, u32 l_iolen, u8 op)
+{
+	struct kvec *iov = *riov;
+	u32 len = MLOG_LPGSZ(lstat);
+	bool alloc_iov = false;
+	u16 i;
+	char *buf;
+
+	ASSERT(len == PAGE_SIZE);
+	ASSERT(l_iolen <= PAGE_SIZE);
+
+	if (!iov) {
+		ASSERT((iovcnt * sizeof(*iov)) <= PAGE_SIZE);
+
+		iov = kcalloc(iovcnt, sizeof(*iov), GFP_KERNEL);
+		if (!iov)
+			return -ENOMEM;
+
+		alloc_iov = true;
+		*riov = iov;
+	}
+
+	for (i = 0; i < iovcnt; i++, iov++) {
+		buf = ((op == MPOOL_OP_READ) ? lstat->lst_rbuf[i] : lstat->lst_abuf[i]);
+
+		/* iov_len for the last log page in read/write buffer. */
+		if (i == iovcnt - 1 && l_iolen != 0)
+			len = l_iolen;
+
+		ASSERT(IS_ALIGNED(len, MLOG_SECSZ(lstat)));
+
+		if (op == MPOOL_OP_WRITE && buf) {
+			iov->iov_base = buf;
+			iov->iov_len  = len;
+			continue;
+		}
+
+		/*
+		 * Pages for the append buffer are allocated in
+		 * mlog_append_*(), so we shouldn't be here for MPOOL_OP_WRITE.
+		 */
+		ASSERT(op == MPOOL_OP_READ);
+
+		/*
+		 * If the read buffer contains stale log pages from a prior
+		 * iterator, reuse them. No need to zero these pages for
+		 * the same reason provided in the following comment.
+		 */
+		if (buf) {
+			iov->iov_base = buf;
+			iov->iov_len  = len;
+			continue;
+		}
+
+		/*
+		 * No need to zero the read buffer as we never read more than
+		 * what's needed and do not consume beyond what's read.
+		 */
+		buf = (char *)__get_free_page(GFP_KERNEL);
+		if (!buf) {
+			mlog_free_rbuf(lstat, 0, i - 1);
+			if (alloc_iov) {
+				kfree(iov);
+				*riov = NULL;
+			}
+
+			return -ENOMEM;
+		}
+
+		/*
+		 * Must be a page-aligned buffer so that it can be used
+		 * in bio_add_page().
+		 */
+		ASSERT(PAGE_ALIGNED(buf));
+
+		lstat->lst_rbuf[i] = iov->iov_base = buf;
+		iov->iov_len = len;
+	}
+
+	return 0;
+}
+
+/**
+ * mlog_populate_abuf() - Makes append offset page-aligned and performs the
+ * read operation in the read-modify-write cycle.
+ * @mp:       mpool descriptor
+ * @layout:   layout descriptor
+ * @soff:     sector/LB offset
+ * @buf:      buffer to populate. Size of this buffer must be MLOG_LPGSZ(lstat).
+ * @skip_ser: client guarantees serialization
+ *
+ * This is to ensure that IO-requests to the device are always 4K-aligned.
+ * The read-modify-write cycle happens *only* if the first append post mlog
+ * open lands on a non-page aligned sector offset. For any further appends,
+ * read-modify-write cycle doesn't happen, as the 4k-aligned version of the
+ * flush set algorithm ensures 4k-alignment of sector offsets at the start
+ * of each log page.
+ */
+static int mlog_populate_abuf(struct mpool_descriptor *mp, struct pmd_layout *layout,
+			      off_t *soff, char *buf, bool skip_ser)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	struct kvec iov;
+	u16 sectsz, iovcnt, leading;
+	off_t off;
+	u32 leadb;
+	int rc;
+
+	sectsz = MLOG_SECSZ(lstat);
+
+	/* Find the leading number of sectors to make it page-aligned. */
+	leading = ((*soff * sectsz) & ~PAGE_MASK) >> ilog2(sectsz);
+	if (leading == 0)
+		return 0; /* Nothing to do */
+
+	*soff = *soff - leading;
+	leadb = leading * sectsz;
+
+	iovcnt       = 1;
+	iov.iov_base = buf;
+	iov.iov_len  = MLOG_LPGSZ(lstat);
+
+	off = *soff * sectsz;
+	ASSERT(IS_ALIGNED(off, MLOG_LPGSZ(lstat)));
+
+	rc = mlog_rw(mp, layout2mlog(layout), &iov, iovcnt, off, MPOOL_OP_READ, skip_ser);
+	if (rc) {
+		mp_pr_err("mpool %s, mlog 0x%lx, read IO failed, iovcnt: %u, off: 0x%lx",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, iovcnt, off);
+		return rc;
+	}
+
+	memset(&buf[leadb], 0, MLOG_LPGSZ(lstat) - leadb);
+
+	return 0;
+}
+
+/**
+ * mlog_populate_rbuf() - Fill the read buffer after aligning the read offset to page boundary.
+ * @mp:       mpool descriptor
+ * @layout:   layout descriptor
+ * @nsec:     number of sectors to populate
+ * @soff:     start sector/LB offset
+ * @skip_ser: client guarantees serialization
+ *
+ * Having the read offsets page-aligned avoids unnecessary
+ * complexity at the pd layer.
+ *
+ * In the worst case, for 512 byte sectors, we would end up reading 7
+ * additional sectors, which is acceptable. There won't be any overhead for
+ * 4 KiB sectors as they are naturally page-aligned.
+ *
+ * Caller must hold the write lock on the layout.
+ */
+int mlog_populate_rbuf(struct mpool_descriptor *mp, struct pmd_layout *layout,
+		       u16 *nsec, off_t *soff, bool skip_ser)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	struct kvec *iov = NULL;
+	u16 maxsec, sectsz, iovcnt, nseclpg, leading;
+	off_t off;
+	u32 l_iolen;
+	int rc;
+
+	mlog_extract_fsetparms(lstat, &sectsz, NULL, &maxsec, &nseclpg);
+
+	/* Find the leading number of sectors to make it page-aligned. */
+	leading = ((*soff * sectsz) & ~PAGE_MASK) >> ilog2(sectsz);
+	*soff   = *soff - leading;
+	*nsec  += leading;
+
+	*nsec   = min_t(u32, maxsec, *nsec);
+	iovcnt  = (*nsec + nseclpg - 1) / nseclpg;
+	l_iolen = MLOG_LPGSZ(lstat);
+
+	rc = mlog_setup_buf(lstat, &iov, iovcnt, l_iolen, MPOOL_OP_READ);
+	if (rc) {
+		mp_pr_err("mpool %s, mlog 0x%lx setup failed, iovcnt: %u, last iolen: %u",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, iovcnt, l_iolen);
+		return rc;
+	}
+
+	off = *soff * sectsz;
+	ASSERT(IS_ALIGNED(off, MLOG_LPGSZ(lstat)));
+
+	rc = mlog_rw(mp, layout2mlog(layout), iov, iovcnt, off, MPOOL_OP_READ, skip_ser);
+	if (rc) {
+		mp_pr_err("mpool %s, mlog 0x%lx populate rbuf, IO failed iovcnt: %u, off: 0x%lx",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, iovcnt, off);
+
+		mlog_free_rbuf(lstat, 0, MLOG_NLPGMB(lstat) - 1);
+		kfree(iov);
+		return rc;
+	}
+
+	/*
+	 * If there're any unused buffers beyond iovcnt, free it. This is
+	 * likely to happen when there're multiple threads reading from
+	 * the same mlog simultaneously, using their own iterator.
+	 */
+	mlog_free_rbuf(lstat, iovcnt, MLOG_NLPGMB(lstat) - 1);
+
+	kfree(iov);
+
+	return 0;
+}
+
+/**
+ * mlog_alloc_abufpg() - Allocate a log page at append buffer index 'abidx'.
+ * @mp:       mpool descriptor
+ * @layout:   layout descriptor
+ * @abidx:    allocate log page at index 'abidx'.
+ * @skip_ser: client guarantees serialization
+ *
+ * If the sector size is 512B AND 4K-alignment is forced AND the append offset
+ * at buffer index '0' is not 4K-aligned, then call mlog_populate_abuf().
+ */
+static int mlog_alloc_abufpg(struct mpool_descriptor *mp, struct pmd_layout *layout,
+			     u16 abidx, bool skip_ser)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	char *abuf;
+
+	ASSERT(MLOG_LPGSZ(lstat) == PAGE_SIZE);
+
+	abuf = (char *)get_zeroed_page(GFP_KERNEL);
+	if (!abuf)
+		return -ENOMEM;
+
+	lstat->lst_abuf[abidx] = abuf;
+
+	if (abidx == 0) {
+		off_t asoff, wsoff;
+		u16 aoff, sectsz;
+		int rc;
+
+		/* This path is taken *only* for the first append following an mlog_open(). */
+		sectsz = MLOG_SECSZ(lstat);
+		wsoff  = lstat->lst_wsoff;
+		aoff   = lstat->lst_aoff;
+
+		if (IS_SECPGA(lstat) || (IS_ALIGNED(wsoff * sectsz, MLOG_LPGSZ(lstat)))) {
+			/* This is the common path */
+			lstat->lst_asoff = wsoff;
+			return 0;
+		}
+
+		/*
+		 * This path is taken *only* if,
+		 * - the log block size is 512B AND
+		 * - lst_wsoff is non page-aligned, which is possible for the
+		 *   first append post mlog_open.
+		 */
+		asoff = wsoff;
+		rc = mlog_populate_abuf(mp, layout, &asoff, abuf, skip_ser);
+		if (rc) {
+			mp_pr_err("mpool %s, mlog 0x%lx, making write offset %ld 4K-aligned failed",
+				  rc, mp->pds_name, (ulong)layout->eld_objid, wsoff);
+			mlog_free_abuf(lstat, abidx, abidx);
+			return rc;
+		}
+
+		ASSERT(asoff <= wsoff);
+		ASSERT(IS_ALIGNED(asoff * sectsz, MLOG_LPGSZ(lstat)));
+
+		lstat->lst_cfssoff = ((wsoff - asoff) * sectsz) + aoff;
+		lstat->lst_asoff   = asoff;
+	}
+
+	return 0;
+}
+
+/**
+ * mlog_flush_abuf() - Set up iovec and flush the append buffer to media.
+ * @mp:       mpool descriptor
+ * @layout:   layout descriptor
+ * @skip_ser: client guarantees serialization
+ */
+static int mlog_flush_abuf(struct mpool_descriptor *mp, struct pmd_layout *layout, bool skip_ser)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	struct kvec *iov = NULL;
+	u16 abidx, sectsz, nseclpg;
+	off_t off;
+	u32 l_iolen;
+	int rc;
+
+	mlog_extract_fsetparms(lstat, &sectsz, NULL, NULL, &nseclpg);
+
+	abidx   = lstat->lst_abidx;
+	l_iolen = MLOG_LPGSZ(lstat);
+
+	rc = mlog_setup_buf(lstat, &iov, abidx + 1, l_iolen, MPOOL_OP_WRITE);
+	if (rc) {
+		mp_pr_err("mpool %s, mlog 0x%lx flush, buf setup failed, iovcnt: %u, iolen: %u",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, abidx + 1, l_iolen);
+		return rc;
+	}
+
+	off = lstat->lst_asoff * sectsz;
+
+	ASSERT((IS_ALIGNED(off, MLOG_LPGSZ(lstat))) ||
+		(IS_SECPGA(lstat) && IS_ALIGNED(off, MLOG_SECSZ(lstat))));
+
+	rc = mlog_rw(mp, layout2mlog(layout), iov, abidx + 1, off, MPOOL_OP_WRITE, skip_ser);
+	if (rc)
+		mp_pr_err("mpool %s, mlog 0x%lx flush append buf, IO failed iovcnt %u, off 0x%lx",
+			  rc, mp->pds_name, (ulong)layout->eld_objid, abidx + 1, off);
+
+	kfree(iov);
+
+	return rc;
+}
+
+/**
+ * mlog_flush_posthdlr_4ka() - Handles both successful and failed flush for
+ *                             512B sectors with 4K-Alignment.
+ * @layout: layout descriptor
+ * @fsucc:  flush status
+ */
+static void mlog_flush_posthdlr_4ka(struct pmd_layout *layout, bool fsucc)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	u16 abidx, sectsz, asidx;
+	off_t asoff, wsoff;
+	char *abuf;
+	u32 nsecwr;
+
+	sectsz = MLOG_SECSZ(lstat);
+	abidx = lstat->lst_abidx;
+	asoff = lstat->lst_asoff;
+	wsoff = lstat->lst_wsoff;
+
+	asidx  = wsoff - ((MLOG_NSECLPG(lstat) * abidx) + asoff);
+
+	/* Set the current filling log page index to 0. */
+	lstat->lst_abidx = 0;
+	abuf = lstat->lst_abuf[0];
+
+	if (!fsucc) {
+		u32 cfssoff;
+
+		/*
+		 * Last CFS flush or header packing failed.
+		 * Retain the pfsetid of the first log block.
+		 */
+		cfssoff = lstat->lst_cfssoff;
+		memset(&abuf[cfssoff], 0, MLOG_LPGSZ(lstat) - cfssoff);
+		asidx = (cfssoff >> ilog2(sectsz));
+		lstat->lst_aoff  = cfssoff - (asidx * sectsz);
+		lstat->lst_wsoff = asoff + asidx;
+
+		goto exit2;
+	}
+
+	/* Last CFS flush succeded. */
+	if (abidx != 0) {
+		/* Reorganize buffers if the active log page not at index 0. */
+		abuf = lstat->lst_abuf[abidx];
+		lstat->lst_abuf[abidx] = NULL;
+	}
+
+	nsecwr = wsoff - (asoff + (lstat->lst_cfssoff >> ilog2(sectsz)));
+	asoff  = wsoff - asidx;
+
+	/* The last logblock of the just-written CFS is not full. */
+	if (sectsz - lstat->lst_aoff >= OMF_LOGREC_DESC_PACKLEN) {
+		if (nsecwr != 0)
+			/* Set pfsetid to the cfsetid of just-written CFS. */
+			lstat->lst_pfsetid  = lstat->lst_cfsetid;
+
+		goto exit1;
+	}
+
+	/* The last logblock of the just-written CFS is full. */
+	lstat->lst_aoff = OMF_LOGBLOCK_HDR_PACKLEN;
+	++wsoff;
+	if ((wsoff - asoff) == MLOG_NSECLPG(lstat)) {
+		memset(&abuf[0], 0, MLOG_LPGSZ(lstat));
+		asoff = wsoff;
+	}
+	/* Set pfsetid to the cfsetid of just-written CFS. */
+	lstat->lst_pfsetid  = lstat->lst_cfsetid;
+
+exit1:
+	asidx              = wsoff - asoff;
+	lstat->lst_cfssoff = (asidx * sectsz) + lstat->lst_aoff;
+	lstat->lst_asoff   = asoff;
+	lstat->lst_wsoff   = wsoff;
+
+exit2:
+	/* Increment cfsetid in all cases. */
+	++lstat->lst_cfsetid;
+
+	lstat->lst_abuf[0] = abuf;
+}
+
+/**
+ * mlog_flush_posthdlr() - Handles both successful and failed flush for
+ * 512B and 4K-sectors with native alignment, i.e., 512B and 4K resply.
+ *
+ * @layout: layout descriptor
+ * @fsucc:  flush status
+ */
+static void mlog_flush_posthdlr(struct pmd_layout *layout, bool fsucc)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	u16 abidx, sectsz, asidx;
+	off_t asoff, lpgoff;
+	char *abuf;
+
+	sectsz = MLOG_SECSZ(lstat);
+	abidx  = lstat->lst_abidx;
+	asoff  = lstat->lst_asoff;
+
+	asidx  = lstat->lst_wsoff - ((MLOG_NSECLPG(lstat) * abidx) + asoff);
+	lpgoff = asidx * sectsz;
+
+	/* Set the current filling log page index to 0. */
+	lstat->lst_abidx = 0;
+	abuf = lstat->lst_abuf[0];
+
+	if (!fsucc) {
+		u32 cfssoff;
+
+		/*
+		 * Last CFS flush or header packing failed.
+		 * Retain the pfsetid of the first log block.
+		 */
+		cfssoff = lstat->lst_cfssoff;
+		memset(&abuf[cfssoff], 0, MLOG_LPGSZ(lstat) - cfssoff);
+		lstat->lst_aoff  = cfssoff;
+		lstat->lst_wsoff = asoff;
+
+		goto exit2;
+	}
+
+	/* Last CFS flush succeded. */
+	if (abidx != 0) {
+		/* Reorganize buffers if the active log page not at index 0. */
+		abuf = lstat->lst_abuf[abidx];
+		lstat->lst_abuf[abidx] = NULL;
+	}
+
+	/* The last logblock of the just-written CFS is not full. */
+	if (sectsz - lstat->lst_aoff >= OMF_LOGREC_DESC_PACKLEN) {
+		/*
+		 * If the last logblock in the just-written CFS is
+		 * first in the append buffer at abidx.
+		 */
+		if (lpgoff == 0) {
+			if (abidx != 0)
+				lstat->lst_pfsetid = lstat->lst_cfsetid;
+
+			goto exit1;
+		}
+
+		memcpy(&abuf[0], &abuf[lpgoff], sectsz);
+		memset(&abuf[sectsz], 0, lpgoff - sectsz + lstat->lst_aoff);
+	} else { /* The last logblock of the just-written CFS is full. */
+		memset(&abuf[0], 0, lpgoff + sectsz);
+		lstat->lst_aoff = OMF_LOGBLOCK_HDR_PACKLEN;
+		++lstat->lst_wsoff;
+	}
+	/* Set pfsetid to the cfsetid of just-written CFS. */
+	lstat->lst_pfsetid  = lstat->lst_cfsetid;
+
+exit1:
+	lstat->lst_cfssoff = lstat->lst_aoff;
+	lstat->lst_asoff   = lstat->lst_wsoff;
+
+exit2:
+	/* Increment cfsetid in all cases. */
+	++lstat->lst_cfsetid;
+
+	lstat->lst_abuf[0] = abuf;
+}
+
+/**
+ * mlog_logblocks_hdrpack() -  Pack log block header in all log blocks in the append buffer.
+ * @layout: object layout
+ *
+ * Called prior to CFS flush
+ */
+static int mlog_logblocks_hdrpack(struct pmd_layout *layout)
+{
+	struct omf_logblock_header lbh;
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	off_t lpgoff;
+	u32 pfsetid, cfsetid;
+	u16 sectsz, nseclpg;
+	u16 idx, abidx;
+	u16 sec, start;
+
+	sectsz  = MLOG_SECSZ(lstat);
+	nseclpg = MLOG_NSECLPG(lstat);
+	abidx   = lstat->lst_abidx;
+	pfsetid = lstat->lst_pfsetid;
+	cfsetid = lstat->lst_cfsetid;
+
+	lbh.olh_vers = OMF_LOGBLOCK_VERS;
+
+	for (idx = 0; idx <= abidx; idx++) {
+		start = 0;
+
+		if (!IS_SECPGA(lstat) && idx == 0)
+			start = (lstat->lst_cfssoff >> ilog2(sectsz));
+
+		if (idx == abidx)
+			nseclpg = lstat->lst_wsoff - (nseclpg * abidx + lstat->lst_asoff) + 1;
+
+		for (sec = start; sec < nseclpg; sec++) {
+			int rc;
+
+			lbh.olh_pfsetid = pfsetid;
+			lbh.olh_cfsetid = cfsetid;
+			mpool_uuid_copy(&lbh.olh_magic, &layout->eld_uuid);
+			lbh.olh_gen = layout->eld_gen;
+			lpgoff = sec * sectsz;
+
+			/* Pack the log block header. */
+			rc = omf_logblock_header_pack_htole(&lbh, &lstat->lst_abuf[idx][lpgoff]);
+			if (rc) {
+				mp_pr_err("mlog packing lbh failed, log pg idx %u, vers %u failed",
+					  rc, idx, lbh.olh_vers);
+
+				return rc;
+			}
+
+			/* If there's more than one sector to flush, pfsetid is set to cfsetid. */
+			pfsetid = cfsetid;
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * mlog_logblocks_flush() - Flush CFS and handle both successful and failed flush.
+ * @mp:       mpool descriptor
+ * @layout:   layout descriptor
+ * @skip_ser: client guarantees serialization
+ */
+int mlog_logblocks_flush(struct mpool_descriptor *mp, struct pmd_layout *layout, bool skip_ser)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	int start, end, rc;
+	bool fsucc = true;
+	u16 abidx;
+
+	abidx = lstat->lst_abidx;
+
+	/* Pack log block header in all the log blocks. */
+	rc = mlog_logblocks_hdrpack(layout);
+	if (rc) {
+		mp_pr_err("mpool %s, mlog 0x%lx packing header failed",
+			  rc, mp->pds_name, (ulong)layout->eld_objid);
+
+	} else {
+		rc = mlog_flush_abuf(mp, layout, skip_ser);
+		if (rc)
+			mp_pr_err("mpool %s, mlog 0x%lx log block flush failed",
+				  rc, mp->pds_name, (ulong)layout->eld_objid);
+	}
+
+	if (rc) {
+		/* If flush failed, free all log pages except the first one. */
+		start = 1;
+		end   = abidx;
+		fsucc = false;
+	} else {
+		/* If flush succeeded, free all log pages except the last one.*/
+		start = 0;
+		end   = abidx - 1;
+
+		/*
+		 * Inform pre-compaction of the size of the active mlog and
+		 * how much is used.
+		 */
+		pmd_precompact_alsz(mp, layout->eld_objid, lstat->lst_wsoff * MLOG_SECSZ(lstat),
+				    lstat->lst_mfp.mfp_totsec * MLOG_SECSZ(lstat));
+	}
+	mlog_free_abuf(lstat, start, end);
+
+	if (!IS_SECPGA(lstat))
+		mlog_flush_posthdlr_4ka(layout, fsucc);
+	else
+		mlog_flush_posthdlr(layout, fsucc);
+
+	return rc;
+}
+
+/**
+ * mlog_append_dmax() -
+ *
+ * Max data record that can be appended to log in bytes; -1 if no room
+ * for a 0 byte data record due to record descriptor length.
+ */
+s64 mlog_append_dmax(struct pmd_layout *layout)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	u64 lbmax, lbrest;
+	u32 sectsz, datalb;
+
+	sectsz = MLOG_SECSZ(lstat);
+	datalb = MLOG_TOTSEC(lstat);
+
+	if (lstat->lst_wsoff >= datalb) {
+		/* Mlog already full */
+		return -1;
+	}
+
+	lbmax  = (sectsz - OMF_LOGBLOCK_HDR_PACKLEN - OMF_LOGREC_DESC_PACKLEN);
+	lbrest = (datalb - lstat->lst_wsoff - 1) * lbmax;
+
+	if ((sectsz - lstat->lst_aoff) < OMF_LOGREC_DESC_PACKLEN) {
+		/* Current log block cannot hold even a record descriptor */
+		if (lbrest)
+			return lbrest;
+
+		return -1;
+	}
+
+	/*
+	 * Can start in current log block and spill over to others (if any)
+	 */
+	return sectsz - lstat->lst_aoff - OMF_LOGREC_DESC_PACKLEN + lbrest;
+}
+
+/**
+ * mlog_update_append_idx() - Check if active log block is full and update the append offsets
+ *
+ * Returns: 0 on success; -errno otherwise
+ */
+int mlog_update_append_idx(struct mpool_descriptor *mp, struct pmd_layout *layout, bool skip_ser)
+{
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	u16 sectsz, nseclpg, abidx, asidx;
+	int rc;
+
+	sectsz  = MLOG_SECSZ(lstat);
+	nseclpg = MLOG_NSECLPG(lstat);
+
+	if (sectsz - lstat->lst_aoff < OMF_LOGREC_DESC_PACKLEN) {
+		/* If the log block is full, move to the next log block in the buffer. */
+		abidx = lstat->lst_abidx;
+		asidx = lstat->lst_wsoff - ((nseclpg * abidx) + lstat->lst_asoff);
+		if (asidx == nseclpg - 1)
+			++lstat->lst_abidx;
+		++lstat->lst_wsoff;
+		lstat->lst_aoff = OMF_LOGBLOCK_HDR_PACKLEN;
+	}
+
+	abidx = lstat->lst_abidx;
+	if (!lstat->lst_abuf[abidx]) {
+		/* Allocate a log page at 'abidx' */
+		rc = mlog_alloc_abufpg(mp, layout, abidx, skip_ser);
+		if (rc)
+			return rc;
+	}
+
+	return 0;
+}
+
+/**
+ * mlog_logblocks_load_media() - Read log blocks from media, upto a maximum of 1 MiB.
+ * @mp:    mpool descriptor
+ * @lri:   read iterator
+ * @inbuf: buffer to into (output)
+ */
+static int mlog_logblocks_load_media(struct mpool_descriptor *mp, struct mlog_read_iter *lri,
+				     char **inbuf)
+{
+	struct pmd_layout *layout = lri->lri_layout;
+	struct mlog_stat *lstat = &layout->eld_lstat;
+	u16 maxsec, nsecs, sectsz;
+	bool skip_ser = false;
+	off_t rsoff;
+	int remsec, rc;
+
+	mlog_extract_fsetparms(lstat, &sectsz, NULL, &maxsec, NULL);
+
+	/*
+	 * The read and append buffer must never overlap. So, the read buffer
+	 * can only hold sector offsets in the range [0, lstat->lst_asoff - 1].
+	 */
+	if (lstat->lst_asoff < 0)
+		remsec = lstat->lst_wsoff;
+	else
+		remsec = lstat->lst_asoff;
+
+	if (remsec == 0) {
+		rc = -ENOTRECOVERABLE;
+		mp_pr_err("mpool %s, objid 0x%lx, mlog read cannot be served from read buffer",
+			  rc, mp->pds_name, (ulong)lri->lri_layout->eld_objid);
+		return rc;
+	}
+
+	lri->lri_rbidx = 0;
+	lri->lri_sidx = 0;
+
+	rsoff = lri->lri_soff;
+	remsec -= rsoff;
+	ASSERT(remsec > 0);
+
+	nsecs = min_t(u32, maxsec, remsec);
+
+	if (layout->eld_flags & MLOG_OF_SKIP_SER)
+		skip_ser = true;
+
+	rc = mlog_populate_rbuf(mp, lri->lri_layout, &nsecs, &rsoff, skip_ser);
+	if (rc) {
+		mp_pr_err("mpool %s, objid 0x%lx, mlog read failed, nsecs: %u, rsoff: 0x%lx",
+			  rc, mp->pds_name, (ulong)lri->lri_layout->eld_objid, nsecs, rsoff);
+
+		lstat->lst_rsoff = lstat->lst_rseoff = -1;
+
+		return rc;
+	}
+
+	/*
+	 * 'nsecs' and 'rsoff' can be changed by mlog_populate_rbuf, if the
+	 * read offset is not page-aligned. Adjust lri_sidx and lst_rsoff
+	 * accordingly.
+	 */
+	lri->lri_sidx     = lri->lri_soff - rsoff;
+	lstat->lst_rsoff  = rsoff;
+	lstat->lst_rseoff = rsoff + nsecs - 1;
+
+	*inbuf = lstat->lst_rbuf[lri->lri_rbidx];
+	*inbuf += lri->lri_sidx * sectsz;
+
+	return 0;
+}
+
+/**
+ * mlog_logblock_load_internal() - Read log blocks from either the read buffer or media.
+ * @mp:    mpool descriptor
+ * @lri:   read iterator
+ * @inbuf: buffer to load into (output)
+ */
+static int mlog_logblock_load_internal(struct mpool_descriptor *mp, struct mlog_read_iter *lri,
+				       char **inbuf)
+{
+	struct mlog_stat *lstat;
+	off_t rsoff, rseoff, soff;
+	u16 nsecs, rbidx, rsidx;
+	u16 nlpgs, nseclpg;
+	int rc;
+
+	lstat = &lri->lri_layout->eld_lstat;
+
+	nseclpg = MLOG_NSECLPG(lstat);
+	rbidx   = lri->lri_rbidx;
+	rsidx   = lri->lri_sidx;
+	soff    = lri->lri_soff;
+	rsoff   = lstat->lst_rsoff;
+	rseoff  = lstat->lst_rseoff;
+
+	if (rsoff < 0)
+		goto media_read;
+
+	/*
+	 * If the read offset doesn't fall within the read buffer range,
+	 * then media read.
+	 */
+	if ((soff < rsoff) || (soff > rseoff))
+		goto media_read;
+
+	do {
+		/* If this is not the start of log block. */
+		if (lri->lri_roff != 0)
+			break;
+
+		/* Check if there's unconsumed data in rbuf. */
+		nsecs = rseoff - rsoff + 1;
+		nlpgs = (nsecs + nseclpg - 1) / nseclpg;
+
+		/* No. of sectors in the last log page. */
+		if (rbidx == nlpgs - 1) {
+			nseclpg = nsecs % nseclpg;
+			nseclpg = nseclpg > 0 ? nseclpg : MLOG_NSECLPG(lstat);
+		}
+		/* Remaining sectors in the active log page? */
+		if (rsidx < nseclpg - 1) {
+			++rsidx;
+			break;
+		}
+		/* Remaining log pages in the read buffer? */
+		if (rbidx >= nlpgs - 1)
+			goto media_read;
+
+		/* Free the active log page and move to next one. */
+		mlog_free_rbuf(lstat, rbidx, rbidx);
+		++rbidx;
+		rsidx = 0;
+
+		break;
+	} while (0);
+
+	/* Serve data from the read buffer. */
+	*inbuf  = lstat->lst_rbuf[rbidx];
+	*inbuf += rsidx * MLOG_SECSZ(lstat);
+
+	lri->lri_rbidx = rbidx;
+	lri->lri_sidx  = rsidx;
+
+	return 0;
+
+media_read:
+	rc = mlog_logblocks_load_media(mp, lri, inbuf);
+	if (rc) {
+		mp_pr_err("mpool %s, objid 0x%lx, mlog new read failed",
+			  rc, mp->pds_name, (ulong)lri->lri_layout->eld_objid);
+
+		return rc;
+	}
+
+	return 0;
+}
+
+/**
+ * mlog_loopback_load() - Load log block referenced by lri into lstat.
+ *
+ * Load log block referenced by lri into lstat, update lri if first read
+ * from this log block, and return a pointer to the log block and a flag
+ * indicating if lri references first record in log block.
+ *
+ * Note: lri can reference the log block currently accumulating in lstat
+ *
+ * Returns: 0 on success; -errno otherwise
+ * One of the possible errno values:
+ * -ENOMSG - if at end of log -- NB: requires an API change to signal without
+ */
+int mlog_logblock_load(struct mpool_descriptor *mp, struct mlog_read_iter *lri,
+		       char **buf, bool *first)
+{
+	struct mlog_stat *lstat = NULL;
+	int lbhlen = 0;
+	int rc = 0;
+
+	*buf = NULL;
+	*first = false;
+	lstat  = &lri->lri_layout->eld_lstat;
+
+	if (!lri->lri_valid || lri->lri_soff > lstat->lst_wsoff) {
+		/* lri is invalid; prior checks should prevent this */
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, invalid offset %u %ld %ld",
+			  rc, mp->pds_name, lri->lri_valid, lri->lri_soff, lstat->lst_wsoff);
+	} else if ((lri->lri_soff == lstat->lst_wsoff) || (lstat->lst_asoff > -1 &&
+			lri->lri_soff >= lstat->lst_asoff &&
+			lri->lri_soff <= lstat->lst_wsoff)) {
+		u16 abidx, sectsz, asidx, nseclpg;
+
+		/*
+		 * lri refers to the currently accumulating log block
+		 * in lstat
+		 */
+		if (!lri->lri_roff)
+			/* First read with handle from this log block. */
+			lri->lri_roff = OMF_LOGBLOCK_HDR_PACKLEN;
+
+		if (lri->lri_soff == lstat->lst_wsoff && lri->lri_roff > lstat->lst_aoff) {
+			/* lri is invalid; prior checks should prevent this */
+			rc = -EINVAL;
+			mp_pr_err("mpool %s, invalid next offset %u %u",
+				  rc, mp->pds_name, lri->lri_roff, lstat->lst_aoff);
+			goto out;
+		} else if (lri->lri_soff == lstat->lst_wsoff && lri->lri_roff == lstat->lst_aoff) {
+			/* Hit end of log */
+			rc = -ENOMSG;
+			goto out;
+		} else if (lri->lri_roff == OMF_LOGBLOCK_HDR_PACKLEN)
+			*first = true;
+
+		sectsz  = MLOG_SECSZ(lstat);
+		nseclpg = MLOG_NSECLPG(lstat);
+
+		abidx = (lri->lri_soff - lstat->lst_asoff) / nseclpg;
+		asidx = lri->lri_soff - ((nseclpg * abidx) + lstat->lst_asoff);
+
+		*buf = &lstat->lst_abuf[abidx][asidx * sectsz];
+	} else {
+		/* lri refers to an existing log block; fetch it if not cached. */
+		rc = mlog_logblock_load_internal(mp, lri, buf);
+		if (!rc) {
+			/*
+			 * NOTE: log block header length must be based
+			 * on version since not guaranteed to be the latest
+			 */
+			lbhlen = omf_logblock_header_len_le(*buf);
+
+			if (lbhlen < 0) {
+				rc = -ENODATA;
+				mp_pr_err("mpool %s, getting header length failed %ld",
+					  rc, mp->pds_name, (long)lbhlen);
+			} else {
+				if (!lri->lri_roff)
+					/* First read with handle from this log block. */
+					lri->lri_roff = lbhlen;
+
+				if (lri->lri_roff == lbhlen)
+					*first = true;
+			}
+		}
+	}
+
+out:
+	if (rc) {
+		*buf = NULL;
+		*first = false;
+	}
+
+	return rc;
+}
+
+/**
+ * mlogutil_closeall() - Close all open user (non-mdc) mlogs in mpool and release resources.
+ *
+ * This is an mpool deactivation utility and not part of the mlog user API.
+ */
+void mlogutil_closeall(struct mpool_descriptor *mp)
+{
+	struct pmd_layout_mlpriv *this, *tmp;
+	struct pmd_layout *layout;
+
+	oml_layout_lock(mp);
+
+	rbtree_postorder_for_each_entry_safe(
+		this, tmp, &mp->pds_oml_root, mlp_nodeoml) {
+
+		layout = mlpriv2layout(this);
+
+		if (pmd_objid_type(layout->eld_objid) != OMF_OBJ_MLOG) {
+			mp_pr_warn("mpool %s, non-mlog object 0x%lx in open mlog layout tree",
+				   mp->pds_name, (ulong)layout->eld_objid);
+			continue;
+		}
+
+		if (!pmd_objid_isuser(layout->eld_objid))
+			continue;
+
+		/* Remove layout from open list and discard log data. */
+		rb_erase(&this->mlp_nodeoml, &mp->pds_oml_root);
+		mlog_stat_free(layout);
+	}
+
+	oml_layout_unlock(mp);
+}
+
+/**
+ * mlog_getprops_cmn() - Retrieve basic mlog properties from layout.
+ * @mp:
+ * @layout:
+ * @prop:
+ */
+void
+mlog_getprops_cmn(struct mpool_descriptor *mp, struct pmd_layout *layout, struct mlog_props *prop)
+{
+	memcpy(prop->lpr_uuid.b, layout->eld_uuid.uuid, MPOOL_UUID_SIZE);
+	prop->lpr_objid       = layout->eld_objid;
+	prop->lpr_alloc_cap   = pmd_layout_cap_get(mp, layout);
+	prop->lpr_gen         = layout->eld_gen;
+	prop->lpr_iscommitted = layout->eld_state & PMD_LYT_COMMITTED;
+	prop->lpr_mclassp    = mp->pds_pdv[layout->eld_ld.ol_pdh].pdi_mclass;
+}
diff --git a/drivers/mpool/mlog_utils.h b/drivers/mpool/mlog_utils.h
new file mode 100644
index 000000000000..560548069c11
--- /dev/null
+++ b/drivers/mpool/mlog_utils.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * Defines functions for writing, reading, and managing the lifecycle of mlogs.
+ */
+
+#ifndef MPOOL_MLOG_UTILS_H
+#define MPOOL_MLOG_UTILS_H
+
+#include <linux/uio.h>
+#include <linux/mutex.h>
+
+struct pmd_layout;
+struct pmd_layout_mlpriv;
+struct mpool_descriptor;
+struct mlog_descriptor;
+struct mlog_read_iter;
+struct mlog_stat;
+struct mlog_props;
+
+/* "open mlog" rbtree operations... */
+#define oml_layout_lock(_mp)        mutex_lock(&(_mp)->pds_oml_lock)
+#define oml_layout_unlock(_mp)      mutex_unlock(&(_mp)->pds_oml_lock)
+
+struct pmd_layout_mlpriv *
+oml_layout_insert(struct mpool_descriptor *mp, struct pmd_layout_mlpriv *item);
+
+struct pmd_layout_mlpriv *oml_layout_remove(struct mpool_descriptor *mp, u64 key);
+
+void mlog_free_abuf(struct mlog_stat *lstat, int start, int end);
+
+void mlog_free_rbuf(struct mlog_stat *lstat, int start, int end);
+
+void mlog_extract_fsetparms(struct mlog_stat *lstat, u16 *sectsz, u32 *totsec,
+			    u16 *nsecmb, u16 *nseclpg);
+
+int mlog_stat_init(struct mpool_descriptor *mp, struct mlog_descriptor *mlh, bool csem);
+
+void mlog_stat_free(struct pmd_layout *layout);
+
+void mlog_read_iter_init(struct pmd_layout *layout, struct mlog_stat *lstat,
+			 struct mlog_read_iter *lri);
+
+void mlog_stat_init_common(struct pmd_layout *layout, struct mlog_stat *lstat);
+
+int mlog_populate_rbuf(struct mpool_descriptor *mp, struct pmd_layout *layout,
+		       u16 *nsec, off_t *soff, bool skip_ser);
+
+void mlog_getprops_cmn(struct mpool_descriptor *mp, struct pmd_layout *layout,
+		       struct mlog_props *prop);
+
+int mlog_logblocks_flush(struct mpool_descriptor *mp, struct pmd_layout *layout, bool skip_ser);
+
+s64 mlog_append_dmax(struct pmd_layout *layout);
+
+int mlog_update_append_idx(struct mpool_descriptor *mp, struct pmd_layout *layout, bool skip_ser);
+
+int mlog_logblock_load(struct mpool_descriptor *mp, struct mlog_read_iter *lri,
+		       char **buf, bool *first);
+
+#endif /* MPOOL_MLOG_UTILS_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:35 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:35 -0500
Subject: [PATCH v2 21/22] mpool: add documentation
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-22-nmeeramohide@micron.com>

This adds locking hierarchy documentation for mpool and
updates ioctl-number.rst with mpool driver's ioctl code.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 .../userspace-api/ioctl/ioctl-number.rst      |  3 +-
 drivers/mpool/mpool-locking.rst               | 90 +++++++++++++++++++
 2 files changed, 92 insertions(+), 1 deletion(-)
 create mode 100644 drivers/mpool/mpool-locking.rst

diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 2a198838fca9..1928606ff447 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -97,7 +97,8 @@ Code  Seq#    Include File                                           Comments
 '&'   00-07  drivers/firewire/nosy-user.h
 '1'   00-1F  linux/timepps.h                                         PPS kit from Ulrich Windl
                                                                      <ftp://ftp.de.kernel.org/pub/linux/daemons/ntp/PPS/>
-'2'   01-04  linux/i2o.h
+'2'   01-04  linux/i2o.h                                             conflict!
+'2'   00-8F  drivers/mpool/mpool_ioctl.h                             conflict!
 '3'   00-0F  drivers/s390/char/raw3270.h                             conflict!
 '3'   00-1F  linux/suspend_ioctls.h,                                 conflict!
              kernel/power/user.c
diff --git a/drivers/mpool/mpool-locking.rst b/drivers/mpool/mpool-locking.rst
new file mode 100644
index 000000000000..6a5da727f2fb
--- /dev/null
+++ b/drivers/mpool/mpool-locking.rst
@@ -0,0 +1,90 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+=============
+Mpool Locking
+=============
+
+Hierarchy
+---------
+::
+
+  mpool_s_lock
+  pmd_s_lock
+  eld_rwlock          object layout r/w lock (per layout)
+  pds_oml_lock        "open mlog" rbtree lock
+  mdi_slotvlock
+  mmi_uqlock          unique ID generator lock
+  mmi_compactlock     compaction lock (per MDC)
+  mmi_uc_lock         uncommitted objects rbtree lock (per MDC)
+  mmi_co_lock         committed objects rbtree lock (per MDC)
+  pds_pdvlock
+  pdi_rmlock[]
+  sda_dalock
+
+Nesting
+-------
+
+There are three nesting levels for mblocks, mlogs, and mpcore's own
+metadata containers (MDCs):
+
+1. PMD_OBJ_CLIENT for client mblocks and mlogs.
+2. PMD_MDC_NORMAL for MDC-1/255 and their underlying mlog pairs.
+3. PMD_MDC_ZERO for MDC-0 and its underlying mlog pair.
+
+A thread of execution may obtain at most one instance of a given lock-class
+at each nesting level, and must do so in the order specified above.
+
+The following helper functions determine the nesting level and use the
+appropriate _nested() primitive or lock pool::
+
+  pmd_obj_rdlock() and _rdunlock()
+  pmd_obj_wrlock() and _wrunlock()
+  pmd_mdc_rdlock() and _rdunlock()
+  pmd_mdc_wrlock() and _wrunlock()
+  pmd_mdc_lock() and _unlock()
+
+For additional information on the _nested() primitives, see
+https://www.kernel.org/doc/Documentation/locking/lockdep-design.txt.
+
+MDC Compaction Locking Patterns
+-------------------------------
+
+In addition to obeying the lock hierarchy and lock-class nesting levels, the
+following locking rules must also be followed for object layouts and all
+mpool properties stored in MDC-0 (e.g., the list of mpool drives pds_pdv[]).
+
+Object layouts (struct pmd_layout):
+
+- Readers must read-lock the layout using pmd_obj_rdlock().
+- Updaters must both write-lock the layout using pmd_obj_wrlock() and lock
+  the mmi_compactlock for the object's MDC using pmd_mdc_lock() before
+  first logging the update in that MDC and then updating the layout.
+
+Mpool properties stored in MDC-0:
+
+- Readers must read-lock the data structure(s) associated with the property.
+- Updaters must both write-lock the data structure(s) associated with the
+  property and lock the mmi_compactlock for MDC-0 using pmd_mdc_lock() before
+  first logging the update in MDC-0 and then updating the data structure(s).
+
+This locking pattern achieves the following:
+
+- For objects associated with a given MDC-0/255, layout readers can execute
+  concurrent with compacting that MDC, whereas layout updaters cannot.
+- For mpool properties stored in MDC-0, property readers can execute
+  concurrent with compacting MDC-0, whereas property updaters cannot.
+- To compact a given MDC-0/255, all in-memory and on-media state to be
+  written is frozen by simply locking the mmi_compactlock for that MDC
+  (because updates to the committed objects tree may take place only while
+  holding both both the compaction mutex and the mmi_co_lock write lock).
+
+Furthermore, taking the mmi_compactlock does not reduce concurrency for
+object or property updaters because these are inherently serialized by the
+requirement to synchronously append log records in the associated MDC.
+
+Object Layout Reference Counts
+------------------------------
+
+The reference counts for an object layout (eld_ref) are protected
+by mmi_co_lock or mmi_uc_lock of the object's MDC dependiing upon
+which tree it is in at the time of acquisition.
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:30 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:30 -0500
Subject: [PATCH v2 16/22] mpool: add mpool control plane utility routines
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-17-nmeeramohide@micron.com>

This adds the mpool control plane infrastructure to manage
mpools.

There is a unit object instance for each device object
created by the mpool driver. A minor number is reserved
for each unit object.

The mpool control device (/dev/mpoolctl) gets a minor
number of 0. An mpool device (/dev/mpool/<mpool_name>)
gets a minor number > 0. Utility routines exist to lookup
an mpool unit given its minor number or name.

All units are born with a reference count of two -
one for the caller and a birth reference that can be released
only by either destroying the unit or unloading the module.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/init.c  |  20 ++
 drivers/mpool/init.h  |   3 +
 drivers/mpool/mpctl.c | 465 ++++++++++++++++++++++++++++++++++++++++++
 drivers/mpool/mpctl.h |  49 +++++
 drivers/mpool/sysfs.c |  48 +++++
 drivers/mpool/sysfs.h |  48 +++++
 6 files changed, 633 insertions(+)
 create mode 100644 drivers/mpool/mpctl.c
 create mode 100644 drivers/mpool/mpctl.h
 create mode 100644 drivers/mpool/sysfs.c
 create mode 100644 drivers/mpool/sysfs.h

diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
index eb1217f63746..126c6c7142b5 100644
--- a/drivers/mpool/init.c
+++ b/drivers/mpool/init.c
@@ -12,10 +12,23 @@
 #include "smap.h"
 #include "pmd_obj.h"
 #include "sb.h"
+#include "mpctl.h"
 
 /*
  * Module params...
  */
+unsigned int maxunits __read_mostly = 1024;
+module_param(maxunits, uint, 0444);
+MODULE_PARM_DESC(maxunits, " max mpools");
+
+unsigned int rwsz_max_mb __read_mostly = 32;
+module_param(rwsz_max_mb, uint, 0444);
+MODULE_PARM_DESC(rwsz_max_mb, " max mblock/mlog r/w size (mB)");
+
+unsigned int rwconc_max __read_mostly = 8;
+module_param(rwconc_max, uint, 0444);
+MODULE_PARM_DESC(rwconc_max, " max mblock/mlog large r/w concurrency");
+
 unsigned int rsvd_bios_max __read_mostly = 16;
 module_param(rsvd_bios_max, uint, 0444);
 MODULE_PARM_DESC(rsvd_bios_max, "max reserved bios in mpool bioset");
@@ -26,6 +39,7 @@ MODULE_PARM_DESC(chunk_size_kb, "Chunk size (in KiB) for device I/O");
 
 static void mpool_exit_impl(void)
 {
+	mpctl_exit();
 	pmd_exit();
 	smap_exit();
 	sb_exit();
@@ -68,6 +82,12 @@ static __init int mpool_init(void)
 		goto errout;
 	}
 
+	rc = mpctl_init();
+	if (rc) {
+		errmsg = "mpctl init failed";
+		goto errout;
+	}
+
 errout:
 	if (rc) {
 		mp_pr_err("%s", rc, errmsg);
diff --git a/drivers/mpool/init.h b/drivers/mpool/init.h
index e02a9672e727..3d8f809a5e45 100644
--- a/drivers/mpool/init.h
+++ b/drivers/mpool/init.h
@@ -6,6 +6,9 @@
 #ifndef MPOOL_INIT_H
 #define MPOOL_INIT_H
 
+extern unsigned int maxunits;
+extern unsigned int rwsz_max_mb;
+extern unsigned int rwconc_max;
 extern unsigned int rsvd_bios_max;
 extern int chunk_size_kb;
 
diff --git a/drivers/mpool/mpctl.c b/drivers/mpool/mpctl.c
new file mode 100644
index 000000000000..21eb7ac4610b
--- /dev/null
+++ b/drivers/mpool/mpctl.c
@@ -0,0 +1,465 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/cdev.h>
+#include <linux/log2.h>
+#include <linux/idr.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/blkdev.h>
+#include <linux/vmalloc.h>
+#include <linux/memcontrol.h>
+#include <linux/pagemap.h>
+#include <linux/kobject.h>
+#include <linux/mm_inline.h>
+#include <linux/version.h>
+#include <linux/kref.h>
+
+#include <linux/backing-dev.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+#include <linux/rbtree.h>
+#include <linux/migrate.h>
+#include <linux/delay.h>
+#include <linux/ctype.h>
+#include <linux/uio.h>
+
+#include "mpool_printk.h"
+#include "assert.h"
+
+#include "mpool_ioctl.h"
+#include "mlog.h"
+#include "mp.h"
+#include "mpctl.h"
+#include "sysfs.h"
+#include "init.h"
+
+
+#define NODEV               MKDEV(0, 0)    /* Non-existent device */
+
+/* mpc pseudo-driver instance data (i.e., all globals live here). */
+struct mpc_softstate {
+	struct mutex        ss_lock;        /* Protects ss_unitmap */
+	struct idr          ss_unitmap;     /* minor-to-unit map */
+
+	____cacheline_aligned
+	struct semaphore    ss_op_sema;     /* Serialize mgmt. ops */
+	dev_t               ss_devno;       /* Control device devno */
+	struct cdev         ss_cdev;
+	struct class       *ss_class;
+	bool                ss_inited;
+};
+
+/* Unit-type specific information. */
+struct mpc_uinfo {
+	const char     *ui_typename;
+	const char     *ui_subdirfmt;
+};
+
+/* One mpc_mpool object per mpool. */
+struct mpc_mpool {
+	struct kref                 mp_ref;
+	struct rw_semaphore         mp_lock;
+	struct mpool_descriptor    *mp_desc;
+	struct mp_mdc              *mp_mdc;
+	uint                        mp_dpathc;
+	char                      **mp_dpathv;
+	char                        mp_name[];
+};
+
+/* The following structures are initialized at the end of this file. */
+static const struct file_operations mpc_fops_default;
+
+static struct mpc_softstate mpc_softstate;
+
+static unsigned int mpc_ctl_uid __read_mostly;
+static unsigned int mpc_ctl_gid __read_mostly = 6;
+static unsigned int mpc_ctl_mode __read_mostly = 0664;
+
+static const struct mpc_uinfo mpc_uinfo_ctl = {
+	.ui_typename = "mpoolctl",
+	.ui_subdirfmt = "%s",
+};
+
+static const struct mpc_uinfo mpc_uinfo_mpool = {
+	.ui_typename = "mpool",
+	.ui_subdirfmt = "mpool/%s",
+};
+
+static inline bool mpc_unit_isctldev(const struct mpc_unit *unit)
+{
+	return (unit->un_uinfo == &mpc_uinfo_ctl);
+}
+
+static inline bool mpc_unit_ismpooldev(const struct mpc_unit *unit)
+{
+	return (unit->un_uinfo == &mpc_uinfo_mpool);
+}
+
+static inline uid_t mpc_current_uid(void)
+{
+	return from_kuid(current_user_ns(), current_uid());
+}
+
+static inline gid_t mpc_current_gid(void)
+{
+	return from_kgid(current_user_ns(), current_gid());
+}
+
+/**
+ * mpc_mpool_release() - release kref handler for mpc_mpool object
+ * @refp:  kref pointer
+ */
+static void mpc_mpool_release(struct kref *refp)
+{
+	struct mpc_mpool *mpool = container_of(refp, struct mpc_mpool, mp_ref);
+	int rc;
+
+	if (mpool->mp_desc) {
+		rc = mpool_deactivate(mpool->mp_desc);
+		if (rc)
+			mp_pr_err("mpool %s deactivate failed", rc, mpool->mp_name);
+	}
+
+	kfree(mpool->mp_dpathv);
+	kfree(mpool);
+
+	module_put(THIS_MODULE);
+}
+
+static void mpc_mpool_put(struct mpc_mpool *mpool)
+{
+	kref_put(&mpool->mp_ref, mpc_mpool_release);
+}
+
+/**
+ * mpc_unit_create() - Create and install a unit object
+ * @path:         device path under "/dev/" to create
+ * @mpool:        mpool ptr
+ * @unitp:        unit ptr
+ *
+ * Create a unit object and install a NULL ptr for it in the units map,
+ * thereby reserving a minor number.  The unit cannot be found by any
+ * of the lookup routines until the NULL ptr is replaced by the actual
+ * ptr to the unit.
+ *
+ * A unit maps an mpool device (.e.g., /dev/mpool/foo)  to an mpool object
+ * created by mpool_create().
+ *
+ * All units are born with two references, one for the caller and one that
+ * can only be released by destroying the unit or unloading the module.
+ *
+ * Return:  Returns 0 if successful and sets *unitp.
+ *          Returns -errno on error.
+ */
+static int mpc_unit_create(const char *name, struct mpc_mpool *mpool, struct mpc_unit **unitp)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpc_unit *unit;
+	size_t unitsz;
+	int minor;
+
+	if (!ss || !name || !unitp)
+		return -EINVAL;
+
+	unitsz = sizeof(*unit) + strlen(name) + 1;
+
+	unit = kzalloc(unitsz, GFP_KERNEL);
+	if (!unit)
+		return -ENOMEM;
+
+	strcpy(unit->un_name, name);
+
+	sema_init(&unit->un_open_lock, 1);
+	unit->un_open_excl = false;
+	unit->un_open_cnt = 0;
+	unit->un_devno = NODEV;
+	kref_init(&unit->un_ref);
+	unit->un_mpool = mpool;
+
+	mutex_lock(&ss->ss_lock);
+	minor = idr_alloc(&ss->ss_unitmap, NULL, 0, -1, GFP_KERNEL);
+	mutex_unlock(&ss->ss_lock);
+
+	if (minor < 0) {
+		kfree(unit);
+		return minor;
+	}
+
+	kref_get(&unit->un_ref); /* acquire additional ref for the caller */
+
+	unit->un_devno = MKDEV(MAJOR(ss->ss_cdev.dev), minor);
+	*unitp = unit;
+
+	return 0;
+}
+
+/**
+ * mpc_unit_release() - Destroy a unit object created by mpc_unit_create().
+ * @unit:
+ */
+static void mpc_unit_release(struct kref *refp)
+{
+	struct mpc_unit *unit = container_of(refp, struct mpc_unit, un_ref);
+	struct mpc_softstate *ss = &mpc_softstate;
+
+	mutex_lock(&ss->ss_lock);
+	idr_remove(&ss->ss_unitmap, MINOR(unit->un_devno));
+	mutex_unlock(&ss->ss_lock);
+
+	if (unit->un_mpool)
+		mpc_mpool_put(unit->un_mpool);
+
+	if (unit->un_device)
+		device_destroy(ss->ss_class, unit->un_devno);
+
+	kfree(unit);
+}
+
+static void mpc_unit_put(struct mpc_unit *unit)
+{
+	if (unit)
+		kref_put(&unit->un_ref, mpc_unit_release);
+}
+
+/**
+ * mpc_unit_setup() - Create a device unit object and special file
+ * @uinfo:
+ * @name:
+ * @cfg:
+ * @mpool:
+ * @unitp: unitp can be NULL. *unitp is updated only if unitp is not NULL
+ *	and no error is returned.
+ *
+ * If successful, this function adopts mpool.  On failure, mpool
+ * remains the responsibility of the caller.
+ *
+ * All units are born with two references, one for the caller and one
+ * that can only be released by destroying the unit or unloading the
+ * module. If the caller passes in nil for unitp then this function
+ * will drop the caller's "caller reference" on his behalf.
+ *
+ * Return:  Returns 0 on success, -errno otherwise...
+ */
+static int mpc_unit_setup(const struct mpc_uinfo *uinfo, const char *name,
+			  const struct mpool_config *cfg, struct mpc_mpool *mpool,
+			  struct mpc_unit **unitp)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpc_unit *unit;
+	struct device *device;
+	int rc;
+
+	if (!ss || !uinfo || !name || !name[0] || !cfg || !unitp)
+		return -EINVAL;
+
+	if (cfg->mc_uid == -1 || cfg->mc_gid == -1 || cfg->mc_mode == -1)
+		return -EINVAL;
+
+	if (!capable(CAP_MKNOD))
+		return -EPERM;
+
+	if (cfg->mc_uid != mpc_current_uid() && !capable(CAP_CHOWN))
+		return -EPERM;
+
+	if (cfg->mc_gid != mpc_current_gid() && !capable(CAP_CHOWN))
+		return -EPERM;
+
+	if (mpool && strcmp(mpool->mp_name, name))
+		return -EINVAL;
+
+	*unitp = NULL;
+	unit = NULL;
+
+	/*
+	 * Try to create a new unit object.  If successful, then all error
+	 * handling beyond this point must route through the errout label
+	 * to ensure the unit is fully destroyed.
+	 */
+	rc = mpc_unit_create(name, mpool, &unit);
+	if (rc)
+		return rc;
+
+	unit->un_uid = cfg->mc_uid;
+	unit->un_gid = cfg->mc_gid;
+	unit->un_mode = cfg->mc_mode;
+
+	unit->un_mdc_captgt = cfg->mc_captgt;
+	memcpy(&unit->un_utype, &cfg->mc_utype, sizeof(unit->un_utype));
+	strlcpy(unit->un_label, cfg->mc_label, sizeof(unit->un_label));
+	unit->un_ds_oidv[0] = cfg->mc_oid1;
+	unit->un_ds_oidv[1] = cfg->mc_oid2;
+	unit->un_ra_pages_max = cfg->mc_ra_pages_max;
+
+	device = device_create(ss->ss_class, NULL, unit->un_devno, unit, uinfo->ui_subdirfmt, name);
+	if (IS_ERR(device)) {
+		rc = PTR_ERR(device);
+		mp_pr_err("device_create %s failed", rc, name);
+		goto errout;
+	}
+
+	unit->un_device = device;
+	unit->un_uinfo = uinfo;
+
+	dev_info(unit->un_device, "minor %u, uid %u, gid %u, mode 0%02o",
+		 MINOR(unit->un_devno), cfg->mc_uid, cfg->mc_gid, cfg->mc_mode);
+
+	*unitp = unit;
+
+errout:
+	if (rc) {
+		/*
+		 * Acquire an additional reference on mpool so that it is not
+		 * errantly destroyed along with the unit, then release both
+		 * the unit's birth and caller's references which should
+		 * destroy the unit.
+		 */
+		kref_get(&mpool->mp_ref);
+		mpc_unit_put(unit);
+		mpc_unit_put(unit);
+	}
+
+	return rc;
+}
+
+/**
+ * mpc_uevent() - Hook to intercept and modify uevents before they're posted to udev
+ * @dev:    mpc driver device
+ * @env:
+ *
+ * See man 7 udev for more info.
+ */
+static int mpc_uevent(struct device *dev, struct kobj_uevent_env *env)
+{
+	struct mpc_unit *unit = dev_get_drvdata(dev);
+
+	if (unit) {
+		add_uevent_var(env, "DEVMODE=%#o", unit->un_mode);
+		add_uevent_var(env, "DEVUID=%u", unit->un_uid);
+		add_uevent_var(env, "DEVGID=%u", unit->un_gid);
+	}
+
+	return 0;
+}
+
+static int mpc_exit_unit(int minor, void *item, void *arg)
+{
+	mpc_unit_put(item);
+
+	return ITERCB_NEXT;
+}
+
+/**
+ * mpctl_exit() - Tear down and unload the mpool control module.
+ */
+void mpctl_exit(void)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+
+	if (ss->ss_inited) {
+		idr_for_each(&ss->ss_unitmap, mpc_exit_unit, NULL);
+		idr_destroy(&ss->ss_unitmap);
+
+		if (ss->ss_devno != NODEV) {
+			if (ss->ss_class) {
+				if (ss->ss_cdev.ops)
+					cdev_del(&ss->ss_cdev);
+				class_destroy(ss->ss_class);
+			}
+			unregister_chrdev_region(ss->ss_devno, maxunits);
+		}
+
+		ss->ss_inited = false;
+	}
+}
+
+/**
+ * mpctl_init() - Load and initialize the mpool control module.
+ */
+int mpctl_init(void)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpool_config *cfg = NULL;
+	struct mpc_unit *ctlunit;
+	const char *errmsg = NULL;
+	int rc;
+
+	if (ss->ss_inited)
+		return -EBUSY;
+
+	ctlunit = NULL;
+
+	maxunits = clamp_t(uint, maxunits, 8, 8192);
+
+	cdev_init(&ss->ss_cdev, &mpc_fops_default);
+	ss->ss_cdev.owner = THIS_MODULE;
+
+	mutex_init(&ss->ss_lock);
+	idr_init(&ss->ss_unitmap);
+	ss->ss_class = NULL;
+	ss->ss_devno = NODEV;
+	sema_init(&ss->ss_op_sema, 1);
+	ss->ss_inited = true;
+
+	rc = alloc_chrdev_region(&ss->ss_devno, 0, maxunits, "mpool");
+	if (rc) {
+		errmsg = "cannot allocate control device major";
+		ss->ss_devno = NODEV;
+		goto errout;
+	}
+
+	ss->ss_class = class_create(THIS_MODULE, module_name(THIS_MODULE));
+	if (IS_ERR(ss->ss_class)) {
+		errmsg = "class_create() failed";
+		rc = PTR_ERR(ss->ss_class);
+		ss->ss_class = NULL;
+		goto errout;
+	}
+
+	ss->ss_class->dev_uevent = mpc_uevent;
+
+	rc = cdev_add(&ss->ss_cdev, ss->ss_devno, maxunits);
+	if (rc) {
+		errmsg = "cdev_add() failed";
+		ss->ss_cdev.ops = NULL;
+		goto errout;
+	}
+
+	cfg = kzalloc(sizeof(*cfg), GFP_KERNEL);
+	if (!cfg) {
+		errmsg = "cfg alloc failed";
+		rc = -ENOMEM;
+		goto errout;
+	}
+
+	cfg->mc_uid = mpc_ctl_uid;
+	cfg->mc_gid = mpc_ctl_gid;
+	cfg->mc_mode = mpc_ctl_mode;
+
+	rc = mpc_unit_setup(&mpc_uinfo_ctl, MPC_DEV_CTLNAME, cfg, NULL, &ctlunit);
+	if (rc) {
+		errmsg = "cannot create control device";
+		goto errout;
+	}
+
+	mutex_lock(&ss->ss_lock);
+	idr_replace(&ss->ss_unitmap, ctlunit, MINOR(ctlunit->un_devno));
+	mutex_unlock(&ss->ss_lock);
+
+	mpc_unit_put(ctlunit);
+
+errout:
+	if (rc) {
+		mp_pr_err("%s", rc, errmsg);
+		mpctl_exit();
+	}
+
+	kfree(cfg);
+
+	return rc;
+}
diff --git a/drivers/mpool/mpctl.h b/drivers/mpool/mpctl.h
new file mode 100644
index 000000000000..412a6a491c15
--- /dev/null
+++ b/drivers/mpool/mpctl.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_MPCTL_H
+#define MPOOL_MPCTL_H
+
+#include <linux/rbtree.h>
+#include <linux/kref.h>
+#include <linux/device.h>
+#include <linux/semaphore.h>
+
+#define ITERCB_NEXT     (0)
+#define ITERCB_DONE     (1)
+
+/* There is one unit object for each device object created by the driver. */
+struct mpc_unit {
+	struct kref                 un_ref;
+	int                         un_open_cnt;    /* Unit open count */
+	struct semaphore            un_open_lock;   /* Protects un_open_* */
+	bool                        un_open_excl;   /* Unit exclusively open */
+	uid_t                       un_uid;
+	gid_t                       un_gid;
+	mode_t                      un_mode;
+	dev_t                       un_devno;
+	const struct mpc_uinfo     *un_uinfo;
+	struct mpc_mpool           *un_mpool;
+	struct address_space       *un_mapping;
+	struct device              *un_device;
+	struct mpc_attr            *un_attr;
+	uint                        un_rawio;       /* log2(max_mblock_size) */
+	u64                         un_ds_oidv[2];
+	u32                         un_ra_pages_max;
+	u64                         un_mdc_captgt;
+	uuid_le                     un_utype;
+	u8                          un_label[MPOOL_LABELSZ_MAX];
+	char                        un_name[];
+};
+
+static inline struct mpc_unit *dev_to_unit(struct device *dev)
+{
+	return dev_get_drvdata(dev);
+}
+
+int mpctl_init(void) __cold;
+void mpctl_exit(void) __cold;
+
+#endif /* MPOOL_MPCTL_H */
diff --git a/drivers/mpool/sysfs.c b/drivers/mpool/sysfs.c
new file mode 100644
index 000000000000..638106a77669
--- /dev/null
+++ b/drivers/mpool/sysfs.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#include <linux/slab.h>
+
+#include "sysfs.h"
+
+struct mpc_attr *mpc_attr_create(struct device *dev, const char *name, int acnt)
+{
+	struct mpc_attr *attr;
+	int i;
+
+	attr = kzalloc(sizeof(*attr) + acnt * sizeof(*attr->a_dattr) +
+		       (acnt + 1) * sizeof(*attr->a_attrs), GFP_KERNEL);
+	if (!attr)
+		return NULL;
+
+	attr->a_kobj = &dev->kobj;
+
+	attr->a_dattr = (void *)(attr + 1);
+
+	attr->a_attrs = (void *)(attr->a_dattr + acnt);
+	for (i = 0; i < acnt; i++)
+		attr->a_attrs[i] = &attr->a_dattr[i].attr;
+	attr->a_attrs[i] = NULL;
+
+	attr->a_group.attrs = attr->a_attrs;
+	attr->a_group.name = name;
+
+	return attr;
+}
+
+void mpc_attr_destroy(struct mpc_attr *attr)
+{
+	kfree(attr);
+}
+
+int mpc_attr_group_create(struct mpc_attr *attr)
+{
+	return sysfs_create_group(attr->a_kobj, &attr->a_group);
+}
+
+void mpc_attr_group_destroy(struct mpc_attr *attr)
+{
+	sysfs_remove_group(attr->a_kobj, &attr->a_group);
+}
diff --git a/drivers/mpool/sysfs.h b/drivers/mpool/sysfs.h
new file mode 100644
index 000000000000..b161eceec75f
--- /dev/null
+++ b/drivers/mpool/sysfs.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_SYSFS_H
+#define MPOOL_SYSFS_H
+
+#include <linux/device.h>
+#include <linux/sysfs.h>
+
+
+#define MPC_ATTR(_da, _name, _mode)                \
+	(_da)->attr.name = __stringify(_name);     \
+	(_da)->attr.mode = (_mode);                \
+	(_da)->show      = mpc_##_name##_show      \
+
+#define MPC_ATTR_RO(_dattr, _name)                 \
+	do {                                       \
+		__typeof(_dattr) da = (_dattr);    \
+		MPC_ATTR(da, _name, 0444);         \
+		da->store = NULL;                  \
+	} while (0)
+
+#define MPC_ATTR_RW(_dattr, _name)                 \
+	do {                                       \
+		__typeof(_dattr) da = (_dattr);    \
+		MPC_ATTR(da, _name, 0644);         \
+		da->store = mpc_##_name##_store;   \
+	} while (0)
+
+
+struct mpc_attr {
+	struct attribute_group       a_group;
+	struct kobject              *a_kobj;
+	struct device_attribute     *a_dattr;
+	struct attribute           **a_attrs;
+};
+
+struct mpc_attr *mpc_attr_create(struct device *d, const char *name, int acnt);
+
+void mpc_attr_destroy(struct mpc_attr *attr);
+
+int mpc_attr_group_create(struct mpc_attr *attr);
+
+void mpc_attr_group_destroy(struct mpc_attr *attr);
+
+#endif /* MPOOL_SYSFS_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:22 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:22 -0500
Subject: [PATCH v2 08/22] mpool: add pool metadata routines to manage object
 lifecycle and IO
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-9-nmeeramohide@micron.com>

Metadata manager interface to allocate, commit, abort, erase, read,
write, and destroy objects.

Metadata containers (MDC-1 through MDC-255) store the metadata for
accessing client allocated objects (mblocks and mlogs). An object
identifier for an mblock or mlog encodes the MDC storing its
metadata for faster lookup.

The layout descriptor (struct pmd_layout) is the in-memory
representation of an object. It is a refcounted structure storing
the object ID, state, device ID, zone start address and the number
of consecutive zones, and a rw sempahore protecting the layout
fields.

An object is in uncommitted state on allocation with its storage
reserved from the free space map. An uncommitted object lives only
in memory, i.e., no metadata records are logged. An object's
metadata is persisted in the metadata container only at commit
time. The uncommitted and committed objects are tracked using
separate rbtrees stored in each metadata container. Support for
object persistence is added in a future patch.

The metadata manager interfaces with the PD layer to discard the
zones at object abort/delete, flush the device cache at object
commit and issue read/write requests to the block layer.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/init.c    |    8 +
 drivers/mpool/omf.c     |    2 -
 drivers/mpool/pmd_obj.c | 1577 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 1585 insertions(+), 2 deletions(-)
 create mode 100644 drivers/mpool/pmd_obj.c

diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
index 261ce67e94dd..eb1217f63746 100644
--- a/drivers/mpool/init.c
+++ b/drivers/mpool/init.c
@@ -10,6 +10,7 @@
 #include "omf_if.h"
 #include "pd.h"
 #include "smap.h"
+#include "pmd_obj.h"
 #include "sb.h"
 
 /*
@@ -25,6 +26,7 @@ MODULE_PARM_DESC(chunk_size_kb, "Chunk size (in KiB) for device I/O");
 
 static void mpool_exit_impl(void)
 {
+	pmd_exit();
 	smap_exit();
 	sb_exit();
 	omf_exit();
@@ -60,6 +62,12 @@ static __init int mpool_init(void)
 		goto errout;
 	}
 
+	rc = pmd_init();
+	if (rc) {
+		errmsg = "pmd init failed";
+		goto errout;
+	}
+
 errout:
 	if (rc) {
 		mp_pr_err("%s", rc, errmsg);
diff --git a/drivers/mpool/omf.c b/drivers/mpool/omf.c
index 8e612d35b19a..9f83fb20c4a8 100644
--- a/drivers/mpool/omf.c
+++ b/drivers/mpool/omf.c
@@ -580,10 +580,8 @@ static int omf_pmd_layout_unpack_letoh(struct mpool_descriptor *mp, struct omf_m
 		return rc;
 	}
 
-#ifdef COMP_PMD_ENABLED
 	ecl = pmd_layout_alloc(&cdr->u.obj.omd_uuid, cdr->u.obj.omd_objid, cdr->u.obj.omd_gen,
 			       cdr->u.obj.omd_mblen, cdr->u.obj.omd_old.ol_zcnt);
-#endif
 	if (!ecl) {
 		rc = -ENOMEM;
 		mp_pr_err("mpool %s, unpacking layout failed, could not allocate layout structure",
diff --git a/drivers/mpool/pmd_obj.c b/drivers/mpool/pmd_obj.c
new file mode 100644
index 000000000000..8966fc0abd0e
--- /dev/null
+++ b/drivers/mpool/pmd_obj.c
@@ -0,0 +1,1577 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * DOC: Module info.
+ *
+ * Pool metadata (pmd) module.
+ *
+ * Defines functions for probing, reading, and writing drives in an mpool.
+ *
+ */
+
+#include <linux/slab.h>
+#include <linux/workqueue.h>
+#include <linux/mutex.h>
+#include <linux/rwsem.h>
+#include <linux/atomic.h>
+#include <linux/delay.h>
+
+#include "mpool_printk.h"
+#include "uuid.h"
+#include "assert.h"
+
+#include "pd.h"
+#include "omf_if.h"
+#include "sb.h"
+#include "mclass.h"
+#include "smap.h"
+#include "mpcore.h"
+#include "pmd.h"
+
+static struct kmem_cache *pmd_obj_erase_work_cache __read_mostly;
+static struct kmem_cache *pmd_layout_priv_cache __read_mostly;
+static struct kmem_cache *pmd_layout_cache __read_mostly;
+
+static int pmd_mdc0_meta_update(struct mpool_descriptor *mp, struct pmd_layout *layout);
+static struct pmd_layout *pmd_layout_find(struct rb_root *root, u64 key);
+static struct pmd_layout *pmd_layout_insert(struct rb_root *root, struct pmd_layout *item);
+
+/* Committed object tree operations... */
+void pmd_co_rlock(struct pmd_mdc_info *cinfo, u8 slot)
+{
+	down_read_nested(&cinfo->mmi_co_lock, slot > 0 ? PMD_MDC_NORMAL : PMD_MDC_ZERO);
+}
+
+void pmd_co_runlock(struct pmd_mdc_info *cinfo)
+{
+	up_read(&cinfo->mmi_co_lock);
+}
+
+static void pmd_co_wlock(struct pmd_mdc_info *cinfo, u8 slot)
+{
+	down_write_nested(&cinfo->mmi_co_lock, slot > 0 ? PMD_MDC_NORMAL : PMD_MDC_ZERO);
+}
+
+static void pmd_co_wunlock(struct pmd_mdc_info *cinfo)
+{
+	up_write(&cinfo->mmi_co_lock);
+}
+
+struct pmd_layout *pmd_co_find(struct pmd_mdc_info *cinfo, u64 objid)
+{
+	return pmd_layout_find(&cinfo->mmi_co_root, objid);
+}
+
+struct pmd_layout *pmd_co_insert(struct pmd_mdc_info *cinfo, struct pmd_layout *layout)
+{
+	return pmd_layout_insert(&cinfo->mmi_co_root, layout);
+}
+
+struct pmd_layout *pmd_co_remove(struct pmd_mdc_info *cinfo, struct pmd_layout *layout)
+{
+	struct pmd_layout *found;
+
+	found = pmd_co_find(cinfo, layout->eld_objid);
+	if (found)
+		rb_erase(&found->eld_nodemdc, &cinfo->mmi_co_root);
+
+	return found;
+}
+
+/* Uncommitted object tree operations... */
+static void pmd_uc_lock(struct pmd_mdc_info *cinfo, u8 slot)
+{
+	mutex_lock_nested(&cinfo->mmi_uc_lock, slot > 0 ? PMD_MDC_NORMAL : PMD_MDC_ZERO);
+}
+
+static void pmd_uc_unlock(struct pmd_mdc_info *cinfo)
+{
+	mutex_unlock(&cinfo->mmi_uc_lock);
+}
+
+static struct pmd_layout *pmd_uc_find(struct pmd_mdc_info *cinfo, u64 objid)
+{
+	return pmd_layout_find(&cinfo->mmi_uc_root, objid);
+}
+
+static struct pmd_layout *pmd_uc_insert(struct pmd_mdc_info *cinfo, struct pmd_layout *layout)
+{
+	return pmd_layout_insert(&cinfo->mmi_uc_root, layout);
+}
+
+static struct pmd_layout *pmd_uc_remove(struct pmd_mdc_info *cinfo, struct pmd_layout *layout)
+{
+	struct pmd_layout *found;
+
+	found = pmd_uc_find(cinfo, layout->eld_objid);
+	if (found)
+		rb_erase(&found->eld_nodemdc, &cinfo->mmi_uc_root);
+
+	return found;
+}
+
+/*
+ * General object operations for both internal and external callers...
+ *
+ * See pmd.h for the various nesting levels for a locking class.
+ */
+void pmd_obj_rdlock(struct pmd_layout *layout)
+{
+	enum pmd_lock_class lc __maybe_unused = PMD_MDC_NORMAL;
+
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+	if (objid_slot(layout->eld_objid))
+		lc = PMD_OBJ_CLIENT;
+	else if (objid_mdc0log(layout->eld_objid))
+		lc = PMD_MDC_ZERO;
+#endif
+
+	down_read_nested(&layout->eld_rwlock, lc);
+}
+
+void pmd_obj_rdunlock(struct pmd_layout *layout)
+{
+	up_read(&layout->eld_rwlock);
+}
+
+void pmd_obj_wrlock(struct pmd_layout *layout)
+{
+	enum pmd_lock_class lc __maybe_unused = PMD_MDC_NORMAL;
+
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+	if (objid_slot(layout->eld_objid))
+		lc = PMD_OBJ_CLIENT;
+	else if (objid_mdc0log(layout->eld_objid))
+		lc = PMD_MDC_ZERO;
+#endif
+
+	down_write_nested(&layout->eld_rwlock, lc);
+}
+
+void pmd_obj_wrunlock(struct pmd_layout *layout)
+{
+	up_write(&layout->eld_rwlock);
+}
+
+/*
+ * Alloc and init object layout; non-arg fields and all strip descriptor
+ * fields are set to 0/UNDEF/NONE; no auxiliary object info is allocated.
+ *
+ * Returns NULL if allocation fails.
+ */
+struct pmd_layout *pmd_layout_alloc(struct mpool_uuid *uuid, u64 objid,
+				    u64 gen, u64 mblen, u32 zcnt)
+{
+	struct kmem_cache *cache = pmd_layout_cache;
+	struct pmd_layout *layout;
+
+	if (pmd_objid_type(objid) == OMF_OBJ_MLOG)
+		cache = pmd_layout_priv_cache;
+
+	layout = kmem_cache_zalloc(cache, GFP_KERNEL);
+	if (!layout)
+		return NULL;
+
+	layout->eld_objid     = objid;
+	layout->eld_gen       = gen;
+	layout->eld_mblen     = mblen;
+	layout->eld_ld.ol_zcnt = zcnt;
+	kref_init(&layout->eld_ref);
+	init_rwsem(&layout->eld_rwlock);
+
+	if (pmd_objid_type(objid) == OMF_OBJ_MLOG)
+		mpool_uuid_copy(&layout->eld_uuid, uuid);
+
+	return layout;
+}
+
+/*
+ * Deallocate all memory associated with object layout.
+ */
+void pmd_layout_release(struct kref *refp)
+{
+	struct kmem_cache *cache = pmd_layout_cache;
+	struct pmd_layout *layout;
+
+	layout = container_of(refp, typeof(*layout), eld_ref);
+
+	ASSERT(layout->eld_objid > 0);
+	ASSERT(kref_read(&layout->eld_ref) == 0);
+
+	if (pmd_objid_type(layout->eld_objid) == OMF_OBJ_MLOG)
+		cache = pmd_layout_priv_cache;
+
+	layout->eld_objid = 0;
+
+	kmem_cache_free(cache, layout);
+}
+
+static struct pmd_layout *pmd_layout_find(struct rb_root *root, u64 key)
+{
+	struct rb_node *node = root->rb_node;
+	struct pmd_layout *this;
+
+	while (node) {
+		this = rb_entry(node, typeof(*this), eld_nodemdc);
+
+		if (key < this->eld_objid)
+			node = node->rb_left;
+		else if (key > this->eld_objid)
+			node = node->rb_right;
+		else
+			return this;
+	}
+
+	return NULL;
+}
+
+static struct pmd_layout *pmd_layout_insert(struct rb_root *root, struct pmd_layout *item)
+{
+	struct rb_node **pos = &root->rb_node, *parent = NULL;
+	struct pmd_layout *this;
+
+	/*
+	 * Figure out where to insert given layout, or return the colliding
+	 * layout if there's already a layout in the tree with the given ID.
+	 */
+	while (*pos) {
+		this = rb_entry(*pos, typeof(*this), eld_nodemdc);
+		parent = *pos;
+
+		if (item->eld_objid < this->eld_objid)
+			pos = &(*pos)->rb_left;
+		else if (item->eld_objid > this->eld_objid)
+			pos = &(*pos)->rb_right;
+		else
+			return this;
+	}
+
+	/* Add new node and rebalance tree. */
+	rb_link_node(&item->eld_nodemdc, parent, pos);
+	rb_insert_color(&item->eld_nodemdc, root);
+
+	return NULL;
+}
+
+static void pmd_layout_unprovision(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	int rc;
+	u16 pdh;
+
+	pdh = layout->eld_ld.ol_pdh;
+
+	/* smap_free() should never fail */
+
+	rc = smap_free(mp, pdh, layout->eld_ld.ol_zaddr, layout->eld_ld.ol_zcnt);
+	if (rc)
+		mp_pr_err("releasing %s drive %s space for layout failed, objid 0x%lx",
+			  rc, mp->pds_name, mp->pds_pdv[pdh].pdi_name, (ulong)layout->eld_objid);
+
+	/* Drop birth reference... */
+	pmd_obj_put(layout);
+}
+
+static void pmd_layout_calculate(struct mpool_descriptor *mp, struct pmd_obj_capacity *ocap,
+				 struct media_class *mc, u64 *zcnt)
+{
+	u32 zonepg;
+
+	if (!ocap->moc_captgt) {
+		/* Obj capacity not specified; use one zone. */
+		*zcnt = 1;
+		return;
+	}
+
+	zonepg = mp->pds_pdv[mc->mc_pdmc].pdi_parm.dpr_zonepg;
+	*zcnt = 1 + ((ocap->moc_captgt - 1) / (zonepg << PAGE_SHIFT));
+}
+
+/**
+ * pmd_layout_provision() - provision storage for the given layout
+ * @mp:
+ * @ocap:
+ * @otype:
+ * @layoutp:
+ * @mc:		media class
+ * @zcnt:
+ */
+static int pmd_layout_provision(struct mpool_descriptor *mp, struct pmd_obj_capacity *ocap,
+				struct pmd_layout *layout, struct media_class *mc, u64 zcnt)
+{
+	enum smap_space_type spctype;
+	struct mc_smap_parms mcsp;
+	u64 zoneaddr, align;
+	u8 pdh;
+	int rc;
+
+	spctype = SMAP_SPC_USABLE_ONLY;
+	if (ocap->moc_spare)
+		spctype = SMAP_SPC_SPARE_2_USABLE;
+
+	/* To reduce/eliminate fragmenation, make sure the alignment is a power of 2. */
+	rc = mc_smap_parms_get(&mp->pds_mc[mc->mc_parms.mcp_classp], &mp->pds_params, &mcsp);
+	if (rc)
+		return rc;
+
+	align = min_t(u64, zcnt, mcsp.mcsp_align);
+	align = roundup_pow_of_two(align);
+
+	pdh = mc->mc_pdmc;
+	rc = smap_alloc(mp, pdh, zcnt, spctype, &zoneaddr, align);
+	if (rc)
+		return rc;
+
+	layout->eld_ld.ol_pdh = pdh;
+	layout->eld_ld.ol_zaddr = zoneaddr;
+
+	return 0;
+}
+
+int pmd_layout_rw(struct mpool_descriptor *mp, struct pmd_layout *layout,
+		  const struct kvec *iov, int iovcnt, u64 boff, int flags, u8 rw)
+{
+	struct mpool_dev_info *pd;
+	u64 zaddr;
+	int rc;
+
+	if (!mp || !layout || !iov)
+		return -EINVAL;
+
+	if (rw != MPOOL_OP_READ && rw != MPOOL_OP_WRITE)
+		return -EINVAL;
+
+	pd = &mp->pds_pdv[layout->eld_ld.ol_pdh];
+	if (mpool_pd_status_get(pd) != PD_STAT_ONLINE)
+		return -EIO;
+
+	if (iovcnt == 0)
+		return 0;
+
+	zaddr = layout->eld_ld.ol_zaddr;
+	if (rw == MPOOL_OP_READ)
+		rc = pd_zone_preadv(&pd->pdi_parm, iov, iovcnt, zaddr, boff);
+	else
+		rc = pd_zone_pwritev(&pd->pdi_parm, iov, iovcnt, zaddr, boff, flags);
+
+	if (rc)
+		mpool_pd_status_set(pd, PD_STAT_OFFLINE);
+
+	return rc;
+}
+
+int pmd_layout_erase(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct mpool_dev_info *pd;
+	int rc;
+
+	if (!mp || !layout)
+		return -EINVAL;
+
+	pd = &mp->pds_pdv[layout->eld_ld.ol_pdh];
+	if (mpool_pd_status_get(pd) != PD_STAT_ONLINE)
+		return -EIO;
+
+	rc = pd_zone_erase(&pd->pdi_parm, layout->eld_ld.ol_zaddr, layout->eld_ld.ol_zcnt,
+			   pmd_objid_type(layout->eld_objid) == OMF_OBJ_MLOG);
+	if (rc)
+		mpool_pd_status_set(pd, PD_STAT_OFFLINE);
+
+	return rc;
+}
+
+u64 pmd_layout_cap_get(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	enum obj_type_omf otype = pmd_objid_type(layout->eld_objid);
+	u32 zonepg;
+
+	switch (otype) {
+	case OMF_OBJ_MBLOCK:
+	case OMF_OBJ_MLOG:
+		zonepg = mp->pds_pdv[layout->eld_ld.ol_pdh].pdi_parm.dpr_zonepg;
+		return ((u64)zonepg * layout->eld_ld.ol_zcnt) << PAGE_SHIFT;
+
+	case OMF_OBJ_UNDEF:
+		break;
+	}
+
+	mp_pr_warn("mpool %s objid 0x%lx, undefined object type %d",
+		   mp->pds_name, (ulong)layout->eld_objid, otype);
+
+	return 0;
+}
+
+struct mpool_dev_info *pmd_layout_pd_get(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	return &mp->pds_pdv[layout->eld_ld.ol_pdh];
+}
+
+int pmd_smap_insert(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	int rc;
+	u16 pdh;
+
+	pdh = layout->eld_ld.ol_pdh;
+
+	/* Insert should never fail */
+
+	rc = smap_insert(mp, pdh, layout->eld_ld.ol_zaddr, layout->eld_ld.ol_zcnt);
+	if (rc)
+		mp_pr_err("mpool %s, allocating drive %s space for layout failed, objid 0x%lx",
+			  rc, mp->pds_name, mp->pds_pdv[pdh].pdi_name, (ulong)layout->eld_objid);
+
+	return rc;
+}
+
+struct pmd_layout *pmd_obj_find_get(struct mpool_descriptor *mp, u64 objid, int which)
+{
+	struct pmd_mdc_info *cinfo;
+	struct pmd_layout *found;
+	u8 cslot;
+
+	if (!objtype_user(objid_type(objid)))
+		return NULL;
+
+	cslot = objid_slot(objid);
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	found = NULL;
+
+	/*
+	 * which < 0  - search uncommitted tree only
+	 * which > 0  - search tree only
+	 * which == 0 - search both trees
+	 */
+	if (which <= 0) {
+		pmd_uc_lock(cinfo, cslot);
+		found = pmd_uc_find(cinfo, objid);
+		if (found)
+			kref_get(&found->eld_ref);
+		pmd_uc_unlock(cinfo);
+	}
+
+	if (!found && which >= 0) {
+		pmd_co_rlock(cinfo, cslot);
+		found = pmd_co_find(cinfo, objid);
+		if (found)
+			kref_get(&found->eld_ref);
+		pmd_co_runlock(cinfo);
+	}
+
+	return found;
+}
+
+int pmd_obj_alloc(struct mpool_descriptor *mp, enum obj_type_omf otype,
+		  struct pmd_obj_capacity *ocap, enum mp_media_classp mclassp,
+		  struct pmd_layout **layoutp)
+{
+	return pmd_obj_alloc_cmn(mp, 0, otype, ocap, mclassp, 0, true, layoutp);
+}
+
+int pmd_obj_realloc(struct mpool_descriptor *mp, u64 objid, struct pmd_obj_capacity *ocap,
+		    enum mp_media_classp mclassp, struct pmd_layout **layoutp)
+{
+	if (!pmd_objid_isuser(objid)) {
+		*layoutp = NULL;
+		return -EINVAL;
+	}
+
+	return pmd_obj_alloc_cmn(mp, objid, objid_type(objid), ocap, mclassp, 1, true, layoutp);
+}
+
+int pmd_obj_commit(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct pmd_mdc_info *cinfo;
+	struct pmd_layout *found;
+	int rc;
+	u8 cslot;
+
+	if (!objtype_user(objid_type(layout->eld_objid)))
+		return -EINVAL;
+
+	pmd_obj_wrlock(layout);
+	if (layout->eld_state & PMD_LYT_COMMITTED) {
+		pmd_obj_wrunlock(layout);
+		return 0;
+	}
+
+	/*
+	 * must log create before marking object committed to guarantee it will
+	 * exist after a crash; must hold cinfo.compactclock while log create,
+	 * update layout.state, and add to list of committed objects to prevent
+	 * a race with mdc compaction
+	 */
+	cslot = objid_slot(layout->eld_objid);
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+	pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
+
+#ifdef OBJ_PERSISTENCE_ENABLED
+	rc = pmd_log_create(mp, layout);
+#endif
+	if (!rc) {
+		pmd_uc_lock(cinfo, cslot);
+		found = pmd_uc_remove(cinfo, layout);
+		pmd_uc_unlock(cinfo);
+
+		pmd_co_wlock(cinfo, cslot);
+		found = pmd_co_insert(cinfo, layout);
+		if (!found)
+			layout->eld_state |= PMD_LYT_COMMITTED;
+		pmd_co_wunlock(cinfo);
+
+		if (found) {
+			rc = -EEXIST;
+
+			/*
+			 * if objid exists in committed object list this is a
+			 * SERIOUS bug; need to log a warning message; should
+			 * never happen. Note in this case we are stuck because
+			 * we just logged a second create for an existing
+			 * object.  If mdc compaction runs before a restart this
+			 * extraneous create record will be eliminated,
+			 * otherwise pmd_objs_load() will see the conflict and
+			 * fail the next mpool activation.  We could make
+			 * pmd_objs_load() tolerate this but for now it is
+			 * better to get an activation failure so that
+			 * it's obvious this bug occurred. Best we can do is put
+			 * the layout back in the uncommitted object list so the
+			 * caller can abort after getting the commit failure.
+			 */
+			mp_pr_crit("mpool %s, obj 0x%lx collided during commit",
+				   rc, mp->pds_name, (ulong)layout->eld_objid);
+
+			/* Put the object back in the uncommitted objects tree */
+			pmd_uc_lock(cinfo, cslot);
+			pmd_uc_insert(cinfo, layout);
+			pmd_uc_unlock(cinfo);
+		} else {
+			atomic_inc(&cinfo->mmi_pco_cnt.pcc_cr);
+			atomic_inc(&cinfo->mmi_pco_cnt.pcc_cobj);
+		}
+	}
+
+	pmd_mdc_unlock(&cinfo->mmi_compactlock);
+	pmd_obj_wrunlock(layout);
+
+	if (!rc)
+		pmd_update_obj_stats(mp, layout, cinfo, PMD_OBJ_COMMIT);
+
+	return rc;
+}
+
+static void pmd_obj_erase_cb(struct work_struct *work)
+{
+	struct pmd_obj_erase_work *oef;
+	struct mpool_descriptor *mp;
+	struct pmd_layout *layout;
+
+	oef = container_of(work, struct pmd_obj_erase_work, oef_wqstruct);
+	mp = oef->oef_mp;
+	layout = oef->oef_layout;
+
+	pmd_layout_erase(mp, layout);
+
+	if (oef->oef_cache)
+		kmem_cache_free(oef->oef_cache, oef);
+
+	pmd_layout_unprovision(mp, layout);
+}
+
+static void pmd_obj_erase_start(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct pmd_obj_erase_work oefbuf, *oef;
+	bool async = true;
+
+	oef = kmem_cache_zalloc(pmd_obj_erase_work_cache, GFP_KERNEL);
+	if (!oef) {
+		oef = &oefbuf;
+		async = false;
+	}
+
+	/* If async oef will be freed in pmd_obj_erase_and_free() */
+	oef->oef_mp = mp;
+	oef->oef_layout = layout;
+	oef->oef_cache = async ? pmd_obj_erase_work_cache : NULL;
+	INIT_WORK(&oef->oef_wqstruct, pmd_obj_erase_cb);
+
+	queue_work(mp->pds_erase_wq, &oef->oef_wqstruct);
+
+	if (!async)
+		flush_work(&oef->oef_wqstruct);
+}
+
+int pmd_obj_abort(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct pmd_mdc_info *cinfo;
+	struct pmd_layout *found;
+	long refcnt;
+	u8 cslot;
+
+	if (!objtype_user(objid_type(layout->eld_objid)))
+		return -EINVAL;
+
+	cslot = objid_slot(layout->eld_objid);
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	found = NULL;
+
+	pmd_obj_wrlock(layout);
+
+	pmd_uc_lock(cinfo, cslot);
+	refcnt = kref_read(&layout->eld_ref);
+	if (refcnt == 2) {
+		found = pmd_uc_remove(cinfo, layout);
+		if (found)
+			found->eld_state |= PMD_LYT_REMOVED;
+	}
+	pmd_uc_unlock(cinfo);
+
+	pmd_obj_wrunlock(layout);
+
+	if (!found)
+		return (refcnt > 2) ? -EBUSY : -EINVAL;
+
+	pmd_update_obj_stats(mp, layout, cinfo, PMD_OBJ_ABORT);
+	pmd_obj_erase_start(mp, layout);
+
+	/* Drop caller's reference... */
+	pmd_obj_put(layout);
+
+	return 0;
+}
+
+int pmd_obj_delete(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct pmd_mdc_info *cinfo;
+	struct pmd_layout *found;
+	long refcnt;
+	u64 objid;
+	u8 cslot;
+	int rc;
+
+	if (!objtype_user(objid_type(layout->eld_objid)))
+		return -EINVAL;
+
+	objid = layout->eld_objid;
+	cslot = objid_slot(objid);
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	found = NULL;
+
+	/*
+	 * Must log delete record before removing object for crash recovery.
+	 * Must hold cinfo.compactlock while logging delete record and
+	 * removing object from the list of committed objects to prevent
+	 * race with MDC compaction
+	 */
+	pmd_obj_wrlock(layout);
+	pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
+
+	refcnt = kref_read(&layout->eld_ref);
+	if (refcnt != 2) {
+		pmd_mdc_unlock(&cinfo->mmi_compactlock);
+		pmd_obj_wrunlock(layout);
+
+		return (refcnt > 2) ? -EBUSY : -EINVAL;
+	}
+
+#ifdef OBJ_PERSISTENCE_ENABLED
+	rc = pmd_log_delete(mp, objid);
+#endif
+	if (!rc) {
+		pmd_co_wlock(cinfo, cslot);
+		found = pmd_co_remove(cinfo, layout);
+		if (found)
+			found->eld_state |= PMD_LYT_REMOVED;
+		pmd_co_wunlock(cinfo);
+	}
+
+	pmd_mdc_unlock(&cinfo->mmi_compactlock);
+	pmd_obj_wrunlock(layout);
+
+	if (!found) {
+		mp_pr_rl("mpool %s, objid 0x%lx, pmd_log_del failed",
+			 rc, mp->pds_name, (ulong)objid);
+		return rc;
+	}
+
+	atomic_inc(&cinfo->mmi_pco_cnt.pcc_del);
+	atomic_dec(&cinfo->mmi_pco_cnt.pcc_cobj);
+	pmd_update_obj_stats(mp, layout, cinfo, PMD_OBJ_DELETE);
+	pmd_obj_erase_start(mp, layout);
+
+	/* Drop caller's reference... */
+	pmd_obj_put(layout);
+
+	return 0;
+}
+
+int pmd_obj_erase(struct mpool_descriptor *mp, struct pmd_layout *layout, u64 gen)
+{
+	u64 objid = layout->eld_objid;
+	int rc;
+
+	if ((pmd_objid_type(objid) != OMF_OBJ_MLOG) ||
+	     (!(layout->eld_state & PMD_LYT_COMMITTED)) ||
+	     (layout->eld_state & PMD_LYT_REMOVED) || (gen <= layout->eld_gen)) {
+		mp_pr_warn("mpool %s, object erase failed to start, objid 0x%lx state 0x%x gen %lu",
+			   mp->pds_name, (ulong)objid, layout->eld_state, (ulong)gen);
+
+		return -EINVAL;
+	}
+
+	/*
+	 * Must log the higher gen number for the old active mlog before
+	 * updating object state (layout->eld_gen of the old active mlog).
+	 * It is to guarantee that a activate after crash will know which is the
+	 * new active mlog.
+	 */
+
+	if (objid_mdc0log(objid)) {
+		/* Compact lock is held by the caller */
+
+		/*
+		 * Change MDC0 metadata image in RAM
+		 */
+		if (objid == MDC0_OBJID_LOG1)
+			mp->pds_sbmdc0.osb_mdc01gen = gen;
+		else
+			mp->pds_sbmdc0.osb_mdc02gen = gen;
+
+		/*
+		 * Write the updated MDC0 metadata in the super blocks of the
+		 * drives holding MDC0 metadata.
+		 * Note: for 1.0, there is only one drive.
+		 */
+		rc = pmd_mdc0_meta_update(mp, layout);
+		if (!rc)
+			/*
+			 * Update in-memory eld_gen, only if on-media
+			 * gen gets successfully updated
+			 */
+			layout->eld_gen = gen;
+	} else {
+		struct pmd_mdc_info *cinfo;
+		u8                   cslot;
+
+		/*
+		 * Take the MDC0 (or mlog MDCi for user MDC) compact lock to
+		 * avoid a race with MDC0 (or mlog MDCi) compaction).
+		 */
+		cslot = objid_slot(layout->eld_objid);
+		cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+		pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
+
+#ifdef OBJ_PERSISTENCE_ENABLED
+		rc = pmd_log_erase(mp, layout->eld_objid, gen);
+#endif
+		if (!rc) {
+			layout->eld_gen = gen;
+			if (cslot)
+				atomic_inc(&cinfo->mmi_pco_cnt.pcc_er);
+
+		}
+		pmd_mdc_unlock(&cinfo->mmi_compactlock);
+	}
+
+	return rc;
+}
+
+/**
+ * pmd_alloc_idgen() - generate an id for an allocated object.
+ * @mp:
+ * @otype:
+ * @objid: outpout
+ *
+ * Does a round robin on the MDC1/255 avoiding the ones that are candidate
+ * for pre compaction.
+ *
+ * The round robin has a bias toward the MDCs with the smaller number of
+ * objects. This is to recover from rare and very big allocation bursts.
+ * During an allocation, the MDC[s] candidate for pre compaction are avoided.
+ * If the allocation is a big burst, the result is that these MDC[s] have much
+ * less objects in them as compared to the other ones.
+ * After the burst if a relatively constant allocation rate takes place, the
+ * deficit in objects of the MDCs avoided during the burst, is never recovered.
+ * The bias in the round robin allows to recover. After a while all MDCs ends
+ * up again with about the same number of objects.
+ */
+static int pmd_alloc_idgen(struct mpool_descriptor *mp, enum obj_type_omf otype, u64 *objid)
+{
+	struct pmd_mdc_info *cinfo = NULL;
+	int rc = 0;
+	u8 cslot;
+	u32 tidx;
+
+	if (mp->pds_mda.mdi_slotvcnt < 2) {
+		/* No mdc available to assign object to; cannot use mdc0 */
+		rc = -ENOSPC;
+		mp_pr_err("mpool %s, no MDCi with i>0", rc, mp->pds_name);
+		*objid = 0;
+		return rc;
+	}
+
+	/* Get next mdc for allocation */
+	tidx = atomic_inc_return(&mp->pds_mda.mdi_sel.mds_tbl_idx) % MDC_TBL_SZ;
+	ASSERT(tidx <= MDC_TBL_SZ);
+
+	cslot = mp->pds_mda.mdi_sel.mds_tbl[tidx];
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+	pmd_mdc_lock(&cinfo->mmi_uqlock, cslot);
+	*objid = objid_make(cinfo->mmi_luniq + 1, otype, cslot);
+	if (objid_ckpt(*objid)) {
+
+		/*
+		 * Must checkpoint objid before assigning it to an object
+		 * to guarantee it will not reissue objid after a crash.
+		 * Must hold cinfo.compactlock while log checkpoint to mdc
+		 * to prevent a race with mdc compaction.
+		 */
+		pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
+#ifdef OBJ_PERSISTENCE_ENABLED
+		rc = pmd_log_idckpt(mp, *objid);
+#endif
+		if (!rc)
+			cinfo->mmi_lckpt = *objid;
+		pmd_mdc_unlock(&cinfo->mmi_compactlock);
+	}
+
+	if (!rc)
+		cinfo->mmi_luniq = cinfo->mmi_luniq + 1;
+	pmd_mdc_unlock(&cinfo->mmi_uqlock);
+
+	if (rc) {
+		mp_pr_rl("mpool %s, checkpoint append for objid 0x%lx failed",
+			 rc, mp->pds_name, (ulong)*objid);
+		*objid = 0;
+		return rc;
+	}
+
+	return 0;
+}
+
+static int pmd_realloc_idvalidate(struct mpool_descriptor *mp, u64 objid)
+{
+	struct pmd_mdc_info *cinfo = NULL;
+	u8 cslot = objid_slot(objid);
+	u64 uniq = objid_uniq(objid);
+	int rc = 0;
+
+	/* We never realloc objects in mdc0 */
+	if (!cslot) {
+		rc = -EINVAL;
+		mp_pr_err("mpool %s, can't re-allocate an object 0x%lx associated to MDC0",
+			  rc, mp->pds_name, (ulong)objid);
+		return rc;
+	}
+
+	spin_lock(&mp->pds_mda.mdi_slotvlock);
+	if (cslot >= mp->pds_mda.mdi_slotvcnt)
+		rc = -EINVAL;
+	spin_unlock(&mp->pds_mda.mdi_slotvlock);
+
+	if (rc) {
+		mp_pr_err("mpool %s, realloc failed, slot number %u is too big %u 0x%lx",
+			  rc, mp->pds_name, cslot, mp->pds_mda.mdi_slotvcnt, (ulong)objid);
+	} else {
+		cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+		pmd_mdc_lock(&cinfo->mmi_uqlock, cslot);
+		if (uniq > cinfo->mmi_luniq)
+			rc = -EINVAL;
+		pmd_mdc_unlock(&cinfo->mmi_uqlock);
+
+		if (rc) {
+			mp_pr_err("mpool %s, realloc failed, unique id %lu too big %lu 0x%lx",
+				  rc, mp->pds_name, (ulong)uniq,
+				  (ulong)cinfo->mmi_luniq, (ulong)objid);
+		}
+	}
+
+	return rc;
+}
+
+/**
+ * pmd_alloc_argcheck() -
+ * @mp:      Mpool descriptor
+ * @objid:   Object ID
+ * @otype:   Object type
+ * @mclassp: Media class
+ */
+static int pmd_alloc_argcheck(struct mpool_descriptor *mp, u64 objid,
+			      enum obj_type_omf otype, enum mp_media_classp mclassp)
+{
+	int rc = -EINVAL;
+
+	if (!mp)
+		return rc;
+
+	if (!objtype_user(otype) || !mclass_isvalid(mclassp)) {
+		mp_pr_err("mpool %s, unknown object type or media class %d %d",
+			  rc, mp->pds_name, otype, mclassp);
+		return rc;
+	}
+
+	if (objid && objid_type(objid) != otype) {
+		mp_pr_err("mpool %s, unknown object type mismatch %d %d",
+			  rc, mp->pds_name, objid_type(objid), otype);
+		return rc;
+	}
+
+	return 0;
+}
+
+int pmd_obj_alloc_cmn(struct mpool_descriptor *mp, u64 objid, enum obj_type_omf otype,
+		      struct pmd_obj_capacity *ocap, enum mp_media_classp mclass,
+		      int realloc, bool needref, struct pmd_layout **layoutp)
+{
+	struct pmd_mdc_info *cinfo;
+	struct media_class *mc;
+	struct pmd_layout *layout;
+	struct mpool_uuid uuid;
+	int retries, flush, rc;
+	u64 zcnt = 0;
+	u8  cslot;
+
+	*layoutp = NULL;
+
+	rc = pmd_alloc_argcheck(mp, objid, otype, mclass);
+	if (rc)
+		return rc;
+
+	if (!objid) {
+		/*
+		 * alloc: generate objid, checkpoint as needed to
+		 * support realloc of uncommitted objects after crash and to
+		 * guarantee objids never reuse
+		 */
+		rc = pmd_alloc_idgen(mp, otype, &objid);
+	} else if (realloc) {
+		/* realloc: validate objid */
+		rc = pmd_realloc_idvalidate(mp, objid);
+	}
+	if (rc)
+		return rc;
+
+	if (otype == OMF_OBJ_MLOG)
+		mpool_generate_uuid(&uuid);
+
+	/*
+	 * Retry from 128 to 256ms with a flush every 1/8th of the retries.
+	 * This is a workaround for the async mblock trim problem.
+	 */
+	retries = 1024;
+	flush   = retries >> 3;
+
+retry:
+	down_read(&mp->pds_pdvlock);
+
+	mc = &mp->pds_mc[mclass];
+	if (mc->mc_pdmc < 0) {
+		up_read(&mp->pds_pdvlock);
+		return -ENOENT;
+	}
+
+	/* Calculate the height (zcnt) of layout. */
+	pmd_layout_calculate(mp, ocap, mc, &zcnt);
+
+	layout = pmd_layout_alloc(&uuid, objid, 0, 0, zcnt);
+	if (!layout) {
+		up_read(&mp->pds_pdvlock);
+		return -ENOMEM;
+	}
+
+	/* Try to allocate zones from drives in media class */
+	rc = pmd_layout_provision(mp, ocap, layout, mc, zcnt);
+	up_read(&mp->pds_pdvlock);
+
+	if (rc) {
+		pmd_obj_put(layout);
+
+		/* TODO: Retry only if mperasewq is busy... */
+		if (retries-- > 0) {
+			usleep_range(128, 256);
+
+			if (flush && (retries % flush == 0))
+				flush_workqueue(mp->pds_erase_wq);
+
+			goto retry;
+		}
+
+		mp_pr_rl("mpool %s, layout alloc failed: objid 0x%lx %lu %u",
+			 rc, mp->pds_name, (ulong)objid, (ulong)zcnt, otype);
+
+		return rc;
+	}
+
+	cslot = objid_slot(objid);
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+	pmd_update_obj_stats(mp, layout, cinfo, PMD_OBJ_ALLOC);
+
+	if (needref)
+		kref_get(&layout->eld_ref);
+
+	/*
+	 * If realloc, we MUST confirm (while holding the uncommitted obj
+	 * tree lock) that objid is not in the committed obj tree in order
+	 * to protect against an invalid *_realloc() call.
+	 */
+	pmd_uc_lock(cinfo, cslot);
+	if (realloc) {
+		pmd_co_rlock(cinfo, cslot);
+		if (pmd_co_find(cinfo, objid))
+			rc = -EEXIST;
+		pmd_co_runlock(cinfo);
+	}
+
+	/*
+	 * For both alloc and realloc, confirm that objid is not in the
+	 * uncommitted obj tree and insert it.  Note that a reallocated
+	 * objid can collide, but a generated objid should never collide.
+	 */
+	if (!rc && pmd_uc_insert(cinfo, layout))
+		rc = -EEXIST;
+	pmd_uc_unlock(cinfo);
+
+	if (rc) {
+		mp_pr_err("mpool %s, %sallocated obj 0x%lx should not be in the %scommitted tree",
+			  rc, mp->pds_name, realloc ? "re-" : "",
+			  (ulong)objid, realloc ? "" : "un");
+
+		if (needref)
+			pmd_obj_put(layout);
+
+		/*
+		 * Since object insertion failed, we need to undo the
+		 * per-mdc stats update we did earlier in this routine
+		 */
+		pmd_update_obj_stats(mp, layout, cinfo, PMD_OBJ_ABORT);
+		pmd_layout_unprovision(mp, layout);
+		layout = NULL;
+	}
+
+	*layoutp = layout;
+
+	return rc;
+}
+
+void pmd_mpool_usage(struct mpool_descriptor *mp, struct mpool_usage *usage)
+{
+	int sidx;
+	u16 slotvcnt;
+
+	/*
+	 * Get a local copy of MDC count (slotvcnt), and then drop the lock
+	 * It's okay another MDC is added concurrently, since pds_ds_info
+	 * is always stale by design
+	 */
+	spin_lock(&mp->pds_mda.mdi_slotvlock);
+	slotvcnt = mp->pds_mda.mdi_slotvcnt;
+	spin_unlock(&mp->pds_mda.mdi_slotvlock);
+
+	for (sidx = 1; sidx < slotvcnt; sidx++) {
+		struct pmd_mdc_stats *pms;
+		struct pmd_mdc_info *cinfo;
+
+		cinfo = &mp->pds_mda.mdi_slotv[sidx];
+		pms   = &cinfo->mmi_stats;
+
+		mutex_lock(&cinfo->mmi_stats_lock);
+		usage->mpu_mblock_alen += pms->pms_mblock_alen;
+		usage->mpu_mblock_wlen += pms->pms_mblock_wlen;
+		usage->mpu_mlog_alen   += pms->pms_mlog_alen;
+		usage->mpu_mblock_cnt  += pms->pms_mblock_cnt;
+		usage->mpu_mlog_cnt    += pms->pms_mlog_cnt;
+		mutex_unlock(&cinfo->mmi_stats_lock);
+	}
+
+	if (slotvcnt < 2)
+		return;
+
+	usage->mpu_alen = (usage->mpu_mblock_alen + usage->mpu_mlog_alen);
+	usage->mpu_wlen = (usage->mpu_mblock_wlen + usage->mpu_mlog_alen);
+}
+
+/**
+ * pmd_mdc0_meta_update_update() - update on media the MDC0 metadata.
+ * @mp:
+ * @layout: Used to know on which drives to write the MDC0 metadata.
+ *
+ * For now write the whole super block, but only the MDC0 metadata needs
+ * to be updated, the rest of the superblock doesn't change.
+ *
+ * In 1.0 the MDC0 metadata is replicated on the 4 superblocks of the drive.
+ * In case of failure, the SBs of a same drive may end up having different
+ * values for the MDC0 metadata.
+ * To address this situation voting could be used along with the SB gen number
+ * psb_gen. But for 1.0 a simpler approach is taken: SB gen number is not used
+ * and SB0 is the authoritative replica. The other 3 replicas of MDC0 metadata
+ * are not used when the mpool activates.
+ */
+static int pmd_mdc0_meta_update(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct omf_sb_descriptor *sb;
+	struct mpool_dev_info *pd;
+	struct mc_parms mc_parms;
+	int rc;
+
+	pd = &(mp->pds_pdv[layout->eld_ld.ol_pdh]);
+	if (mpool_pd_status_get(pd) != PD_STAT_ONLINE) {
+		rc = -EIO;
+		mp_pr_err("%s: pd %s unavailable or offline, status %d",
+			  rc, mp->pds_name, pd->pdi_name, mpool_pd_status_get(pd));
+		return rc;
+	}
+
+	sb = kzalloc(sizeof(*sb), GFP_KERNEL);
+	if (!sb)
+		return -ENOMEM;
+
+	/*
+	 * set superblock values common to all new drives in pool
+	 * (new or extant)
+	 */
+	sb->osb_magic = OMF_SB_MAGIC;
+	strlcpy((char *) sb->osb_name, mp->pds_name, sizeof(sb->osb_name));
+	sb->osb_vers = OMF_SB_DESC_VER_LAST;
+	mpool_uuid_copy(&sb->osb_poolid, &mp->pds_poolid);
+	sb->osb_gen = 1;
+
+	/* Set superblock values specific to this drive */
+	mpool_uuid_copy(&sb->osb_parm.odp_devid, &pd->pdi_devid);
+	sb->osb_parm.odp_devsz = pd->pdi_parm.dpr_devsz;
+	sb->osb_parm.odp_zonetot = pd->pdi_parm.dpr_zonetot;
+	mc_pd_prop2mc_parms(&pd->pdi_parm.dpr_prop, &mc_parms);
+	mc_parms2omf_devparm(&mc_parms, &sb->osb_parm);
+
+	sbutil_mdc0_copy(sb, &mp->pds_sbmdc0);
+
+	mp_pr_debug("MDC0 compaction gen1 %lu gen2 %lu",
+		    0, (ulong)sb->osb_mdc01gen, (ulong)sb->osb_mdc02gen);
+
+	/*
+	 * sb_write_update() succeeds if at least SB0 is written. It is
+	 * not a problem to have SB1 not written because the authoritative
+	 * MDC0 metadata replica is the one in SB0.
+	 */
+	rc = sb_write_update(&pd->pdi_parm, sb);
+	if (rc)
+		mp_pr_err("compacting %s MDC0, writing superblock on drive %s failed",
+			  rc, mp->pds_name, pd->pdi_name);
+
+	kfree(sb);
+	return rc;
+}
+
+/**
+ * pmd_update_obj_stats() - update per-MDC space usage
+ * @mp:
+ * @layout:
+ * @cinfo:
+ * @op: object opcode
+ */
+void pmd_update_obj_stats(struct mpool_descriptor *mp, struct pmd_layout *layout,
+			  struct pmd_mdc_info *cinfo, enum pmd_obj_op op)
+{
+	struct pmd_mdc_stats *pms;
+	enum obj_type_omf otype;
+	u64 cap;
+
+	otype = pmd_objid_type(layout->eld_objid);
+
+	mutex_lock(&cinfo->mmi_stats_lock);
+	pms = &cinfo->mmi_stats;
+
+	/* Update space usage and mblock/mlog count */
+	switch (op) {
+	case PMD_OBJ_LOAD:
+		if (otype == OMF_OBJ_MBLOCK)
+			pms->pms_mblock_wlen += layout->eld_mblen;
+		fallthrough;
+
+	case PMD_OBJ_ALLOC:
+		cap = pmd_layout_cap_get(mp, layout);
+		if (otype == OMF_OBJ_MLOG) {
+			pms->pms_mlog_cnt++;
+			pms->pms_mlog_alen += cap;
+		} else if (otype == OMF_OBJ_MBLOCK) {
+			pms->pms_mblock_cnt++;
+			pms->pms_mblock_alen += cap;
+		}
+		break;
+
+	case PMD_OBJ_COMMIT:
+		if (otype == OMF_OBJ_MBLOCK)
+			pms->pms_mblock_wlen += layout->eld_mblen;
+		break;
+
+	case PMD_OBJ_DELETE:
+		if (otype == OMF_OBJ_MBLOCK)
+			pms->pms_mblock_wlen -= layout->eld_mblen;
+		fallthrough;
+
+	case PMD_OBJ_ABORT:
+		cap = pmd_layout_cap_get(mp, layout);
+		if (otype == OMF_OBJ_MLOG) {
+			pms->pms_mlog_cnt--;
+			pms->pms_mlog_alen -= cap;
+		} else if (otype == OMF_OBJ_MBLOCK) {
+			pms->pms_mblock_cnt--;
+			pms->pms_mblock_alen -= cap;
+		}
+		break;
+
+	default:
+		ASSERT(0);
+		break;
+	}
+
+	mutex_unlock(&cinfo->mmi_stats_lock);
+}
+
+/**
+ * pmd_compare_free_space() - compare free space between MDCs
+ * @f:  First  MDC
+ * @s:  Second MDC
+ *
+ * Arrange MDCs in descending order of free space
+ */
+static int pmd_compare_free_space(const void *first, const void *second)
+{
+	const struct pmd_mdc_info *f = *(const struct pmd_mdc_info **)first;
+	const struct pmd_mdc_info *s = *(const struct pmd_mdc_info **)second;
+
+	/* return < 0 - first member should be ahead for second */
+	if (f->mmi_credit.ci_free > s->mmi_credit.ci_free)
+		return -1;
+
+	/* return > 0 - first member should be after second */
+	if (f->mmi_credit.ci_free < s->mmi_credit.ci_free)
+		return 1;
+
+	return 0;
+
+}
+
+/**
+ * pmd_update_ms_tbl() - udpates mds_tlb with MDC slot numbers
+ * @mp:  mpool descriptor
+ * @slotnum:  array of slot numbers
+ *
+ * This function creates an array of mdc slot and credit sets by interleaving
+ * MDC slots. Interleave maximize the interval at which the slots appear in
+ * the mds_tbl.
+ *
+ * The first set in the array is reference set with only 1 member and has max
+ * assigned credits. Subsequent sets are formed to match the reference set and
+ * may contain one or more member such that total credit of the set will match
+ * the reference set. The last set may have fewer credit than the reference set
+ *
+ * Locking: no lock need to be held when calling this function.
+ *
+ */
+static void pmd_update_mds_tbl(struct mpool_descriptor *mp, u8 num_mdc, u8 *slotnum)
+{
+	struct mdc_credit_set *cset, *cs;
+	struct pmd_mdc_info *cinfo;
+	u16 refcredit, neededcredit, tidx, totalcredit = 0;
+	u8 csidx, csmidx, num_cset, i;
+
+	cset = kcalloc(num_mdc, sizeof(*cset), GFP_KERNEL);
+	if (!cset)
+		return;
+
+	cinfo = &mp->pds_mda.mdi_slotv[slotnum[0]];
+	refcredit = cinfo->mmi_credit.ci_credit;
+
+	csidx = 0; /* creditset index */
+	i     = 0; /* slotnum index   */
+	while (i < num_mdc) {
+		cs = &cset[csidx++];
+		neededcredit = refcredit;
+
+		csmidx = 0;
+		/* Setup members of the credit set */
+		while (csmidx < MPOOL_MDC_SET_SZ  && i < num_mdc) {
+			/* slot 0 should never be there */
+			ASSERT(slotnum[i] != 0);
+
+			cinfo = &mp->pds_mda.mdi_slotv[slotnum[i]];
+			cs->cs_num_csm = csmidx + 1;
+			cs->csm[csmidx].m_slot = slotnum[i];
+
+			if (neededcredit <= cinfo->mmi_credit.ci_credit) {
+				/*
+				 * More than required credit is available,
+				 * leftover will be assigned to the next set.
+				 */
+				cs->csm[csmidx].m_credit    += neededcredit;
+				cinfo->mmi_credit.ci_credit -= neededcredit;
+				totalcredit += neededcredit; /* Debug */
+				neededcredit = 0;
+
+				/* Some credit available stay at this mdc */
+				if (cinfo->mmi_credit.ci_credit == 0)
+					i++;
+				break;
+			}
+
+			/*
+			 * Available credit is < needed, assign all
+			 * the available credit and move to the next
+			 * mdc slot.
+			 */
+			cs->csm[csmidx].m_credit += cinfo->mmi_credit.ci_credit;
+			neededcredit -= cinfo->mmi_credit.ci_credit;
+			totalcredit  += cinfo->mmi_credit.ci_credit;
+			cinfo->mmi_credit.ci_credit = 0;
+
+			/* Move to the next mdcslot and set member */
+			i++;
+			csmidx++;
+		}
+	}
+
+	ASSERT(totalcredit == MDC_TBL_SZ);
+	num_cset = csidx;
+
+	tidx  = 0;
+	csidx = 0;
+	while (tidx < MDC_TBL_SZ) {
+		cs = &cset[csidx];
+		if (cs->cs_idx < cs->cs_num_csm) {
+			csmidx = cs->cs_idx;
+			if (cs->csm[csmidx].m_credit) {
+				cs->csm[csmidx].m_credit--;
+				mp->pds_mda.mdi_sel.mds_tbl[tidx] = cs->csm[csmidx].m_slot;
+				totalcredit--;
+
+				if (cs->csm[csmidx].m_credit == 0)
+					cs->cs_idx += 1;
+
+				tidx++;
+			}
+		}
+		/* Loop over the sets */
+		csidx = (csidx + 1) % num_cset;
+	}
+
+	ASSERT(totalcredit == 0);
+
+	kfree(cset);
+}
+
+/**
+ * pmd_update_credit() - udpates MDC credit if new MDCs should be created
+ * @mp:  mpool descriptor
+ *
+ * Credits are assigned as a ratio between MDC such that MDC with least free
+ * space will fill up at the same time as other MDC.
+ *
+ * Locking: no lock need to be held when calling this function.
+ */
+void pmd_update_credit(struct mpool_descriptor *mp)
+{
+	struct pre_compact_ctrs *pco_cnt;
+	struct pmd_mdc_info *cinfo;
+	u64 cap, used, free, nmtoc;
+	u16 credit, cslot;
+	u8 sidx, nidx, num_mdc;
+	u8 *slotnum;
+	void **sarray = mp->pds_mda.mdi_sel.mds_smdc;
+	u32 nbnoalloc = (u32)mp->pds_params.mp_pconbnoalloc;
+
+	if (mp->pds_mda.mdi_slotvcnt < 2) {
+		mp_pr_warn("Not enough MDCn %u", mp->pds_mda.mdi_slotvcnt - 1);
+		return;
+	}
+
+	slotnum = kcalloc(MDC_SLOTS, sizeof(*slotnum), GFP_KERNEL);
+	if (!slotnum)
+		return;
+
+	nmtoc = atomic_read(&mp->pds_pco.pco_nmtoc);
+	nmtoc = nmtoc % (mp->pds_mda.mdi_slotvcnt - 1) + 1;
+
+	/*
+	 * slotvcnt includes MDC 0 and MDCn that are in precompaction
+	 * list and should be excluded. If there are less than (nbnoalloc
+	 * +2) MDCs exclusion is not possible. 2 is added to account for
+	 * MDC0 and the MDC pointed to by pco_nmtoc.
+	 *
+	 * MDC that is in pre-compacting state and two MDCs that follows
+	 * are excluded from allocation. This is done to prevent stall/
+	 * delays for a sync that follows an allocation as both
+	 * take a compaction lock.
+	 */
+	if (mp->pds_mda.mdi_slotvcnt < (nbnoalloc + 2)) {
+		num_mdc = mp->pds_mda.mdi_slotvcnt - 1;
+		cslot  = 1;
+		mp_pr_debug("MDCn cnt %u, cannot skip %u num_mdc %u",
+			    0, mp->pds_mda.mdi_slotvcnt - 1, (u32)nmtoc, num_mdc);
+	} else {
+		num_mdc = mp->pds_mda.mdi_slotvcnt - (nbnoalloc + 2);
+		cslot = (nmtoc + nbnoalloc) % (mp->pds_mda.mdi_slotvcnt - 1);
+	}
+
+
+	/* Walkthrough all MDCs and exclude MDCs that are almost full */
+	for (nidx = 0, sidx = 0; nidx < num_mdc; nidx++) {
+		cslot = cslot % (mp->pds_mda.mdi_slotvcnt - 1) + 1;
+
+		if (cslot == 0)
+			cslot = 1;
+
+		cinfo = &mp->pds_mda.mdi_slotv[cslot];
+		pco_cnt = &(cinfo->mmi_pco_cnt);
+
+		cap  = atomic64_read(&pco_cnt->pcc_cap);
+		used = atomic64_read(&pco_cnt->pcc_len);
+
+		if ((cap - used) < (cap / 400)) {
+			/* Consider < .25% free space as full */
+			mp_pr_warn("MDC slot %u almost full", cslot);
+			continue;
+		}
+		sarray[sidx++] = cinfo;
+		cinfo->mmi_credit.ci_free = cap - used;
+	}
+
+	/* Sort the array with decreasing order of space */
+	sort((void *)sarray, sidx, sizeof(sarray[0]), pmd_compare_free_space, NULL);
+	num_mdc = sidx;
+
+	/* Calculate total free space across the chosen MDC set */
+	for (sidx = 0, free = 0; sidx < num_mdc; sidx++) {
+		cinfo = sarray[sidx];
+		free += cinfo->mmi_credit.ci_free;
+		slotnum[sidx] = cinfo->mmi_credit.ci_slot;
+	}
+
+	/*
+	 * Assign credit to MDCs in the MDC set. Credit is relative and
+	 * will not exceed the total slots in mds_tbl
+	 */
+	for (sidx = 0, credit = 0; sidx < num_mdc; sidx++) {
+		cinfo = &mp->pds_mda.mdi_slotv[slotnum[sidx]];
+		cinfo->mmi_credit.ci_credit = (MDC_TBL_SZ * cinfo->mmi_credit.ci_free) / free;
+		credit += cinfo->mmi_credit.ci_credit;
+	}
+
+	ASSERT(credit <= MDC_TBL_SZ);
+
+	/*
+	 * If the credit is not equal to the table size, assign
+	 * credits so table can be filled all the way.
+	 */
+	if (credit < MDC_TBL_SZ) {
+		credit = MDC_TBL_SZ - credit;
+
+		sidx = 0;
+		while (credit > 0) {
+			sidx = (sidx % num_mdc);
+			cinfo = &mp->pds_mda.mdi_slotv[slotnum[sidx]];
+			cinfo->mmi_credit.ci_credit += 1;
+			sidx++;
+			credit--;
+		}
+	}
+
+	pmd_update_mds_tbl(mp, num_mdc, slotnum);
+
+	kfree(slotnum);
+}
+
+/*
+ * pmd_mlogid2cslot() - Given an mlog object ID which makes one of the mpool
+ *	core MDCs (MDCi with i >0), it returns i.
+ *	Given an client created object ID (mblock or mlog), it returns -1.
+ * @mlogid:
+ */
+static int pmd_mlogid2cslot(u64 mlogid)
+{
+	u64 uniq;
+
+	if (pmd_objid_type(mlogid) != OMF_OBJ_MLOG)
+		return -1;
+	if (objid_slot(mlogid))
+		return -1;
+	uniq = objid_uniq(mlogid);
+	if (uniq > (2 * MDC_SLOTS) - 1)
+		return -1;
+
+	return(uniq/2);
+}
+
+void pmd_precompact_alsz(struct mpool_descriptor *mp, u64 objid, u64 len, u64 cap)
+{
+	struct pre_compact_ctrs *pco_cnt;
+	struct pmd_mdc_info *cinfo;
+	int ret;
+	u8 cslot;
+
+	ret = pmd_mlogid2cslot(objid);
+	if (ret <= 0)
+		return;
+
+	cslot = ret;
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	pco_cnt = &(cinfo->mmi_pco_cnt);
+	atomic64_set(&pco_cnt->pcc_len, len);
+	atomic64_set(&pco_cnt->pcc_cap, cap);
+}
+
+int pmd_init(void)
+{
+	int rc = 0;
+
+	/* Initialize the slab caches. */
+	pmd_layout_cache = kmem_cache_create("mpool_pmd_layout", sizeof(struct pmd_layout),
+					     0, SLAB_HWCACHE_ALIGN | SLAB_POISON, NULL);
+	if (!pmd_layout_cache) {
+		rc = -ENOMEM;
+		mp_pr_err("kmem_cache_create(pmd_layout, %zu) failed",
+			  rc, sizeof(struct pmd_layout));
+		goto errout;
+	}
+
+	pmd_layout_priv_cache = kmem_cache_create("mpool_pmd_layout_priv",
+				sizeof(struct pmd_layout) + sizeof(union pmd_layout_priv),
+				0, SLAB_HWCACHE_ALIGN | SLAB_POISON, NULL);
+	if (!pmd_layout_priv_cache) {
+		rc = -ENOMEM;
+		mp_pr_err("kmem_cache_create(pmd priv, %zu) failed",
+			  rc, sizeof(union pmd_layout_priv));
+		goto errout;
+	}
+
+	pmd_obj_erase_work_cache = kmem_cache_create("mpool_pmd_obj_erase_work",
+						     sizeof(struct pmd_obj_erase_work),
+						     0, SLAB_HWCACHE_ALIGN | SLAB_POISON, NULL);
+	if (!pmd_obj_erase_work_cache) {
+		rc = -ENOMEM;
+		mp_pr_err("kmem_cache_create(pmd_obj_erase, %zu) failed",
+			  rc, sizeof(struct pmd_obj_erase_work));
+		goto errout;
+	}
+
+errout:
+	if (rc)
+		pmd_exit();
+
+	return rc;
+}
+
+void pmd_exit(void)
+{
+	kmem_cache_destroy(pmd_obj_erase_work_cache);
+	kmem_cache_destroy(pmd_layout_priv_cache);
+	kmem_cache_destroy(pmd_layout_cache);
+
+	pmd_obj_erase_work_cache = NULL;
+	pmd_layout_priv_cache = NULL;
+	pmd_layout_cache = NULL;
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:33 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:33 -0500
Subject: [PATCH v2 19/22] mpool: add support to mmap arbitrary collection of
 mblocks
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-20-nmeeramohide@micron.com>

This implements the mmap file operation for the mpool driver.

Mblock and mlog writes avoid the Linux page cache. Mblocks are
written, committed, and made immutable before they can be read
either directly (avoiding the page cache) or memory-mapped.
Mlogs are always read and updated directly and cannot be
memory-mapped.

Mblocks are memory-mapped by creating an mcache map.
The mcache map APIs allow an arbitrary collection of mblocks
(specified as a vector of mblock OIDs) to be mapped linearly
into the virtual address space of an mpool client.

An extended VMA instance (mpc_xvm) is created for each
mcache map. The xvm takes a ref on all the mblocks it maps.
The size of an xvm region is 1G by default and can be tuned
using the module parameter 'xvm_size_max'. When an mcache
map is created, its xvm instance is assigned with the next
available device offset range. The device offset -> xvm
mapping is inserted into a region map stored in the mpool's
unit object.

The mmap driver entry point uses the vma page offset to deduce
the device offset of the corresponding mcache map. The xvm
instance is looked up from the region map using device offset
as the key. A reference to this xvm instance is stored in the
vma private data, which enables the page fault handler to find
the xvm for the faulting range in constant time. The offset into
the mcache map can then be used to determine the mblock id and
offset to be read for filling the page.

Readahead is enabled for the mpool device by providing the
fadvise file op and by initializing file_ra_state.ra_pages.
Readahead requests are dispatched to one of the four
readahead workqueues and served by the workqueue threads.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/init.c   |   16 +
 drivers/mpool/init.h   |    2 +
 drivers/mpool/mcache.c | 1029 ++++++++++++++++++++++++++++++++++++++++
 drivers/mpool/mcache.h |   96 ++++
 drivers/mpool/mpctl.c  |   45 +-
 drivers/mpool/mpctl.h  |    3 +
 6 files changed, 1190 insertions(+), 1 deletion(-)
 create mode 100644 drivers/mpool/mcache.c
 create mode 100644 drivers/mpool/mcache.h

diff --git a/drivers/mpool/init.c b/drivers/mpool/init.c
index 126c6c7142b5..b1fe3286773a 100644
--- a/drivers/mpool/init.c
+++ b/drivers/mpool/init.c
@@ -12,11 +12,20 @@
 #include "smap.h"
 #include "pmd_obj.h"
 #include "sb.h"
+#include "mcache.h"
 #include "mpctl.h"
 
 /*
  * Module params...
  */
+unsigned int xvm_max __read_mostly = 1048576 * 128;
+module_param(xvm_max, uint, 0444);
+MODULE_PARM_DESC(xvm_max, " max extended VMA regions");
+
+unsigned int xvm_size_max __read_mostly = 30;
+module_param(xvm_size_max, uint, 0444);
+MODULE_PARM_DESC(xvm_size_max, " max extended VMA size log2");
+
 unsigned int maxunits __read_mostly = 1024;
 module_param(maxunits, uint, 0444);
 MODULE_PARM_DESC(maxunits, " max mpools");
@@ -40,6 +49,7 @@ MODULE_PARM_DESC(chunk_size_kb, "Chunk size (in KiB) for device I/O");
 static void mpool_exit_impl(void)
 {
 	mpctl_exit();
+	mcache_exit();
 	pmd_exit();
 	smap_exit();
 	sb_exit();
@@ -82,6 +92,12 @@ static __init int mpool_init(void)
 		goto errout;
 	}
 
+	rc = mcache_init();
+	if (rc) {
+		errmsg = "mcache init failed";
+		goto errout;
+	}
+
 	rc = mpctl_init();
 	if (rc) {
 		errmsg = "mpctl init failed";
diff --git a/drivers/mpool/init.h b/drivers/mpool/init.h
index 3d8f809a5e45..507d43a55c01 100644
--- a/drivers/mpool/init.h
+++ b/drivers/mpool/init.h
@@ -6,6 +6,8 @@
 #ifndef MPOOL_INIT_H
 #define MPOOL_INIT_H
 
+extern unsigned int xvm_max;
+extern unsigned int xvm_size_max;
 extern unsigned int maxunits;
 extern unsigned int rwsz_max_mb;
 extern unsigned int rwconc_max;
diff --git a/drivers/mpool/mcache.c b/drivers/mpool/mcache.c
new file mode 100644
index 000000000000..07c79615ecf1
--- /dev/null
+++ b/drivers/mpool/mcache.c
@@ -0,0 +1,1029 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/memcontrol.h>
+#include <linux/kref.h>
+#include <linux/migrate.h>
+#include <linux/delay.h>
+#include <linux/uio.h>
+#include <linux/fadvise.h>
+#include <linux/prefetch.h>
+
+#include "mpool_ioctl.h"
+
+#include "mp.h"
+#include "mpool_printk.h"
+#include "assert.h"
+
+#include "mpctl.h"
+
+#ifndef lru_to_page
+#define lru_to_page(_head)  (list_entry((_head)->prev, struct page, lru))
+#endif
+
+/*
+ * MPC_RA_IOV_MAX - Max pages per call to mblock read by a readahead
+ * request.  Be careful about increasing this as it directly adds
+ * (n * 24) bytes to the stack frame of mpc_readpages_cb().
+ */
+#define MPC_RA_IOV_MAX      (8)
+
+#define NODEV               MKDEV(0, 0)    /* Non-existent device */
+
+/*
+ * Arguments required to initiate an asynchronous call to mblock_read()
+ * and which must also be preserved across that call.
+ *
+ * Note: We could make things more efficient by changing a_pagev[]
+ * to a struct kvec if mblock_read() would guarantee that it will
+ * not alter the given iovec.
+ */
+struct readpage_args {
+	void                       *a_xvm;
+	struct mblock_descriptor   *a_mbdesc;
+	u64                         a_mboffset;
+	int                         a_pagec;
+	struct page                *a_pagev[];
+};
+
+struct readpage_work {
+	struct work_struct      w_work;
+	struct readpage_args    w_args;
+};
+
+static void mpc_xvm_put(struct mpc_xvm *xvm);
+
+static int mpc_readpage_impl(struct page *page, struct mpc_xvm *map);
+
+/* The following structures are initialized at the end of this file. */
+static const struct vm_operations_struct mpc_vops_default;
+const struct address_space_operations mpc_aops_default;
+
+static struct workqueue_struct *mpc_wq_trunc __read_mostly;
+static struct workqueue_struct *mpc_wq_rav[4] __read_mostly;
+
+static size_t mpc_xvm_cachesz[2] __read_mostly;
+static struct kmem_cache *mpc_xvm_cache[2] __read_mostly;
+
+static struct workqueue_struct *mpc_rgn2wq(uint rgn)
+{
+	return mpc_wq_rav[rgn % ARRAY_SIZE(mpc_wq_rav)];
+}
+
+static int mpc_rgnmap_isorphan(int rgn, void *item, void *data)
+{
+	struct mpc_xvm *xvm = item;
+	void **headp = data;
+
+	if (xvm && kref_read(&xvm->xvm_ref) == 1 && !atomic_read(&xvm->xvm_opened)) {
+		idr_replace(&xvm->xvm_rgnmap->rm_root, NULL, rgn);
+		xvm->xvm_next = *headp;
+		*headp = xvm;
+	}
+
+	return ITERCB_NEXT;
+}
+
+void mpc_rgnmap_flush(struct mpc_rgnmap *rm)
+{
+	struct mpc_xvm *head = NULL, *xvm;
+
+	if (!rm)
+		return;
+
+	/* Wait for all mpc_xvm_free_cb() callbacks to complete... */
+	flush_workqueue(mpc_wq_trunc);
+
+	/*
+	 * Build a list of all orphaned XVMs and release their birth
+	 * references (i.e., XVMs that were created but never mmapped).
+	 */
+	mutex_lock(&rm->rm_lock);
+	idr_for_each(&rm->rm_root, mpc_rgnmap_isorphan, &head);
+	mutex_unlock(&rm->rm_lock);
+
+	while ((xvm = head)) {
+		head = xvm->xvm_next;
+		mpc_xvm_put(xvm);
+	}
+}
+
+static struct mpc_xvm *mpc_xvm_lookup(struct mpc_rgnmap *rm, uint key)
+{
+	struct mpc_xvm *xvm;
+
+	mutex_lock(&rm->rm_lock);
+	xvm = idr_find(&rm->rm_root, key);
+	if (xvm && !kref_get_unless_zero(&xvm->xvm_ref))
+		xvm = NULL;
+	mutex_unlock(&rm->rm_lock);
+
+	return xvm;
+}
+
+void mpc_xvm_free(struct mpc_xvm *xvm)
+{
+	struct mpc_rgnmap *rm;
+
+	ASSERT((u32)(uintptr_t)xvm == xvm->xvm_magic);
+
+	rm = xvm->xvm_rgnmap;
+
+	mutex_lock(&rm->rm_lock);
+	idr_remove(&rm->rm_root, xvm->xvm_rgn);
+	mutex_unlock(&rm->rm_lock);
+
+	xvm->xvm_magic = 0xbadcafe;
+	xvm->xvm_rgn = -1;
+
+	kmem_cache_free(xvm->xvm_cache, xvm);
+
+	atomic_dec(&rm->rm_rgncnt);
+}
+
+static void mpc_xvm_free_cb(struct work_struct *work)
+{
+	struct mpc_xvm *xvm = container_of(work, typeof(*xvm), xvm_work);
+
+	mpc_xvm_free(xvm);
+}
+
+static void mpc_xvm_get(struct mpc_xvm *xvm)
+{
+	kref_get(&xvm->xvm_ref);
+}
+
+static void mpc_xvm_release(struct kref *kref)
+{
+	struct mpc_xvm *xvm = container_of(kref, struct mpc_xvm, xvm_ref);
+	struct mpc_rgnmap *rm = xvm->xvm_rgnmap;
+	int i;
+
+	ASSERT((u32)(uintptr_t)xvm == xvm->xvm_magic);
+
+	mutex_lock(&rm->rm_lock);
+	ASSERT(kref_read(kref) == 0);
+	idr_replace(&rm->rm_root, NULL, xvm->xvm_rgn);
+	mutex_unlock(&rm->rm_lock);
+
+	/*
+	 * Wait for all in-progress readaheads to complete
+	 * before we drop our mblock references.
+	 */
+	if (atomic_add_return(WQ_MAX_ACTIVE, &xvm->xvm_rabusy) > WQ_MAX_ACTIVE)
+		flush_workqueue(mpc_rgn2wq(xvm->xvm_rgn));
+
+	for (i = 0; i < xvm->xvm_mbinfoc; ++i)
+		mblock_put(xvm->xvm_mbinfov[i].mbdesc);
+
+	INIT_WORK(&xvm->xvm_work, mpc_xvm_free_cb);
+	queue_work(mpc_wq_trunc, &xvm->xvm_work);
+}
+
+static void mpc_xvm_put(struct mpc_xvm *xvm)
+{
+	kref_put(&xvm->xvm_ref, mpc_xvm_release);
+}
+
+/*
+ * VM operations
+ */
+
+static void mpc_vm_open(struct vm_area_struct *vma)
+{
+	mpc_xvm_get(vma->vm_private_data);
+}
+
+static void mpc_vm_close(struct vm_area_struct *vma)
+{
+	mpc_xvm_put(vma->vm_private_data);
+}
+
+static int mpc_alloc_and_readpage(struct vm_area_struct *vma, pgoff_t offset, gfp_t gfp)
+{
+	struct address_space *mapping;
+	struct file *file;
+	struct page *page;
+	int rc;
+
+	page = __page_cache_alloc(gfp | __GFP_NOWARN);
+	if (!page)
+		return -ENOMEM;
+
+	file    = vma->vm_file;
+	mapping = file->f_mapping;
+
+	rc = add_to_page_cache_lru(page, mapping, offset, gfp & GFP_KERNEL);
+	if (rc == 0)
+		rc = mpc_readpage_impl(page, vma->vm_private_data);
+	else if (rc == -EEXIST)
+		rc = 0;
+
+	put_page(page);
+
+	return rc;
+}
+
+static bool mpc_lock_page_or_retry(struct page *page, struct mm_struct *mm, uint flags)
+{
+	might_sleep();
+
+	if (trylock_page(page))
+		return true;
+
+	if (flags & FAULT_FLAG_ALLOW_RETRY) {
+		if (flags & FAULT_FLAG_RETRY_NOWAIT)
+			return false;
+
+		mmap_read_unlock(mm);
+		/* _killable version is not exported by the kernel. */
+		wait_on_page_locked(page);
+		return false;
+	}
+
+	if (flags & FAULT_FLAG_KILLABLE) {
+		int rc;
+
+		rc = lock_page_killable(page);
+		if (rc) {
+			mmap_read_unlock(mm);
+			return false;
+		}
+	} else {
+		lock_page(page);
+	}
+
+	return true;
+}
+
+static int mpc_handle_page_error(struct page *page, struct vm_area_struct *vma)
+{
+	int rc;
+
+	ClearPageError(page);
+
+	rc = mpc_readpage_impl(page, vma->vm_private_data);
+	if (rc == 0) {
+		wait_on_page_locked(page);
+		if (!PageUptodate(page))
+			rc = -EIO;
+	}
+
+	put_page(page);
+
+	return rc;
+}
+
+static vm_fault_t mpc_vm_fault_impl(struct vm_area_struct *vma, struct vm_fault *vmf)
+{
+	struct address_space *mapping;
+	struct inode *inode;
+	struct page *page;
+	vm_fault_t vmfrc;
+	pgoff_t offset;
+	loff_t size;
+
+	mapping = vma->vm_file->f_mapping;
+	inode   = mapping->host;
+	offset  = vmf->pgoff;
+	vmfrc   = 0;
+
+	size = round_up(i_size_read(inode), PAGE_SIZE);
+	if (offset >= (size >> PAGE_SHIFT))
+		return VM_FAULT_SIGBUS;
+
+retry_find:
+	page = find_get_page(mapping, offset);
+	if (!page) {
+		int rc = mpc_alloc_and_readpage(vma, offset, mapping_gfp_mask(mapping));
+
+		if (rc < 0)
+			return (rc == -ENOMEM) ? VM_FAULT_OOM : VM_FAULT_SIGBUS;
+
+		vmfrc = VM_FAULT_MAJOR;
+		goto retry_find;
+	}
+
+	/* At this point, page is not locked but has a ref. */
+	if (vmfrc == VM_FAULT_MAJOR)
+		count_vm_event(PGMAJFAULT);
+
+	if (!mpc_lock_page_or_retry(page, vma->vm_mm, vmf->flags)) {
+		put_page(page);
+		return vmfrc | VM_FAULT_RETRY;
+	}
+
+	/* At this point, page is locked with a ref. */
+	if (unlikely(page->mapping != mapping)) {
+		unlock_page(page);
+		put_page(page);
+		goto retry_find;
+	}
+
+	VM_BUG_ON_PAGE(page->index != offset, page);
+
+	if (unlikely(!PageUptodate(page))) {
+		int rc = mpc_handle_page_error(page, vma);
+
+		/* At this point, page is not locked and has no ref. */
+		if (rc)
+			return VM_FAULT_SIGBUS;
+		goto retry_find;
+	}
+
+	/* Page is locked with a ref. */
+	vmf->page = page;
+
+	return vmfrc | VM_FAULT_LOCKED;
+}
+
+static vm_fault_t mpc_vm_fault(struct vm_fault *vmf)
+{
+	return mpc_vm_fault_impl(vmf->vma, vmf);
+}
+
+/*
+ * MPCTL address-space operations.
+ */
+
+static int mpc_readpage_impl(struct page *page, struct mpc_xvm *xvm)
+{
+	struct mpc_mbinfo *mbinfo;
+	struct kvec iov[1];
+	off_t offset;
+	uint mbnum;
+	int rc;
+
+	offset  = page->index << PAGE_SHIFT;
+	offset %= (1ul << xvm_size_max);
+
+	mbnum = offset / xvm->xvm_bktsz;
+	if (mbnum >= xvm->xvm_mbinfoc) {
+		unlock_page(page);
+		return -EINVAL;
+	}
+
+	mbinfo = xvm->xvm_mbinfov + mbnum;
+	offset %= xvm->xvm_bktsz;
+
+	if (offset >= mbinfo->mblen) {
+		unlock_page(page);
+		return -EINVAL;
+	}
+
+	iov[0].iov_base = page_address(page);
+	iov[0].iov_len = PAGE_SIZE;
+
+	rc = mblock_read(xvm->xvm_mpdesc, mbinfo->mbdesc, iov, 1, offset, PAGE_SIZE);
+	if (rc) {
+		unlock_page(page);
+		return rc;
+	}
+
+	if (xvm->xvm_hcpagesp)
+		atomic64_inc(xvm->xvm_hcpagesp);
+	atomic64_inc(&xvm->xvm_nrpages);
+
+	SetPagePrivate(page);
+	set_page_private(page, (ulong)xvm);
+	SetPageUptodate(page);
+	unlock_page(page);
+
+	return 0;
+}
+
+#define MPC_RPARGSBUFSZ \
+	(sizeof(struct readpage_args) + MPC_RA_IOV_MAX * sizeof(void *))
+
+/**
+ * mpc_readpages_cb() - mpc_readpages() callback
+ * @work:   w_work.work from struct readpage_work
+ *
+ * The incoming arguments are in the first page (a_pagev[0]) which
+ * we are about to overwrite, so we copy them to the stack.
+ */
+static void mpc_readpages_cb(struct work_struct *work)
+{
+	struct kvec iovbuf[MPC_RA_IOV_MAX];
+	char argsbuf[MPC_RPARGSBUFSZ];
+	struct readpage_args *args = (void *)argsbuf;
+	struct readpage_work *w;
+	struct mpc_xvm *xvm;
+	struct kvec *iov = iovbuf;
+	size_t argssz;
+	int pagec, rc, i;
+
+	w = container_of(work, struct readpage_work, w_work);
+
+	pagec = w->w_args.a_pagec;
+	argssz = sizeof(*args) + sizeof(args->a_pagev[0]) * pagec;
+
+	ASSERT(pagec <= ARRAY_SIZE(iovbuf));
+	ASSERT(argssz <= sizeof(argsbuf));
+
+	memcpy(args, &w->w_args, argssz);
+	w = NULL; /* Do not touch! */
+
+	xvm = args->a_xvm;
+
+	/*
+	 * Synchronize with mpc_xvm_put() to prevent dropping our
+	 * mblock references while there are reads in progress.
+	 */
+	if (atomic_inc_return(&xvm->xvm_rabusy) > WQ_MAX_ACTIVE) {
+		rc = -ENXIO;
+		goto errout;
+	}
+
+	for (i = 0; i < pagec; ++i) {
+		iov[i].iov_base = page_address(args->a_pagev[i]);
+		iov[i].iov_len = PAGE_SIZE;
+	}
+
+	rc = mblock_read(xvm->xvm_mpdesc, args->a_mbdesc, iov,
+			 pagec, args->a_mboffset, pagec << PAGE_SHIFT);
+	if (rc)
+		goto errout;
+
+	if (xvm->xvm_hcpagesp)
+		atomic64_add(pagec, xvm->xvm_hcpagesp);
+	atomic64_add(pagec, &xvm->xvm_nrpages);
+	atomic_dec(&xvm->xvm_rabusy);
+
+	for (i = 0; i < pagec; ++i) {
+		struct page *page = args->a_pagev[i];
+
+		SetPagePrivate(page);
+		set_page_private(page, (ulong)xvm);
+		SetPageUptodate(page);
+
+		unlock_page(page);
+		put_page(page);
+	}
+
+	return;
+
+errout:
+	atomic_dec(&xvm->xvm_rabusy);
+
+	for (i = 0; i < pagec; ++i) {
+		unlock_page(args->a_pagev[i]);
+		put_page(args->a_pagev[i]);
+	}
+}
+
+static int mpc_readpages(struct file *file, struct address_space *mapping,
+			 struct list_head *pages, uint nr_pages)
+{
+	struct workqueue_struct *wq;
+	struct readpage_work *w;
+	struct work_struct *work;
+	struct mpc_mbinfo *mbinfo;
+	struct mpc_unit *unit;
+	struct mpc_xvm *xvm;
+	struct page *page;
+	off_t offset, mbend;
+	uint mbnum, iovmax, i;
+	uint ra_pages_max;
+	ulong index;
+	gfp_t gfp;
+	u32 key;
+	int rc;
+
+	unit = file->private_data;
+
+	ra_pages_max = unit->un_ra_pages_max;
+	if (ra_pages_max < 1)
+		return 0;
+
+	page   = lru_to_page(pages);
+	offset = page->index << PAGE_SHIFT;
+	index  = page->index;
+	work   = NULL;
+	w      = NULL;
+
+	key = offset >> xvm_size_max;
+
+	/*
+	 * The idr value here (xvm) is pinned for the lifetime of the address map.
+	 * Therefore, we can exit the rcu read-side critsec without worry that xvm will be
+	 * destroyed before put_page() has been called on each and every page in the given
+	 * list of pages.
+	 */
+	rcu_read_lock();
+	xvm = idr_find(&unit->un_rgnmap.rm_root, key);
+	rcu_read_unlock();
+
+	if (!xvm)
+		return 0;
+
+	offset %= (1ul << xvm_size_max);
+
+	mbnum = offset / xvm->xvm_bktsz;
+	if (mbnum >= xvm->xvm_mbinfoc)
+		return 0;
+
+	mbinfo = xvm->xvm_mbinfov + mbnum;
+
+	mbend = mbnum * xvm->xvm_bktsz + mbinfo->mblen;
+	iovmax = MPC_RA_IOV_MAX;
+
+	gfp = mapping_gfp_mask(mapping) & GFP_KERNEL;
+	wq = mpc_rgn2wq(xvm->xvm_rgn);
+
+	nr_pages = min_t(uint, nr_pages, ra_pages_max);
+
+	for (i = 0; i < nr_pages; ++i) {
+		page    = lru_to_page(pages);
+		offset  = page->index << PAGE_SHIFT;
+		offset %= (1ul << xvm_size_max);
+
+		/* Don't read past the end of the mblock. */
+		if (offset >= mbend)
+			break;
+
+		/* mblock reads must be logically contiguous. */
+		if (page->index != index && work) {
+			queue_work(wq, work);
+			work = NULL;
+		}
+
+		index = page->index + 1; /* next expected page index */
+
+		prefetchw(&page->flags);
+		list_del(&page->lru);
+
+		rc = add_to_page_cache_lru(page, mapping, page->index, gfp);
+		if (rc) {
+			if (work) {
+				queue_work(wq, work);
+				work = NULL;
+			}
+			put_page(page);
+			continue;
+		}
+
+		if (!work) {
+			w = page_address(page);
+			INIT_WORK(&w->w_work, mpc_readpages_cb);
+			w->w_args.a_xvm = xvm;
+			w->w_args.a_mbdesc = mbinfo->mbdesc;
+			w->w_args.a_mboffset = offset % xvm->xvm_bktsz;
+			w->w_args.a_pagec = 0;
+			work = &w->w_work;
+
+			iovmax = MPC_RA_IOV_MAX;
+			iovmax -= page->index % MPC_RA_IOV_MAX;
+		}
+
+		w->w_args.a_pagev[w->w_args.a_pagec++] = page;
+
+		/*
+		 * Restrict batch size to the number of struct kvecs
+		 * that will fit into a page (minus our header).
+		 */
+		if (w->w_args.a_pagec >= iovmax) {
+			queue_work(wq, work);
+			work = NULL;
+		}
+	}
+
+	if (work)
+		queue_work(wq, work);
+
+	return 0;
+}
+
+/**
+ * mpc_releasepage() - Linux VM calls the release page when pages are released.
+ * @page:
+ * @gfp:
+ *
+ * The function is added as part of tracking incoming and outgoing pages.
+ */
+static int mpc_releasepage(struct page *page, gfp_t gfp)
+{
+	struct mpc_xvm *xvm;
+
+	if (!PagePrivate(page))
+		return 0;
+
+	xvm = (void *)page_private(page);
+	if (!xvm)
+		return 0;
+
+	ClearPagePrivate(page);
+	set_page_private(page, 0);
+
+	ASSERT((u32)(uintptr_t)xvm == xvm->xvm_magic);
+
+	if (xvm->xvm_hcpagesp)
+		atomic64_dec(xvm->xvm_hcpagesp);
+	atomic64_dec(&xvm->xvm_nrpages);
+
+	return 1;
+}
+
+static void mpc_invalidatepage(struct page *page, uint offset, uint length)
+{
+	mpc_releasepage(page, 0);
+}
+
+/**
+ * mpc_migratepage() -  Callback for handling page migration.
+ * @mapping:
+ * @newpage:
+ * @page:
+ * @mode:
+ *
+ * The drivers having private pages are supplying this callback.
+ * Not sure the page migration releases or invalidates the page being migrated,
+ * or else the tracking of incoming and outgoing pages will be in trouble. The
+ * callback is added to deal with uncertainties around migration. The migration
+ * will be declined so long as the page is private and it belongs to mpctl.
+ */
+static int mpc_migratepage(struct address_space *mapping, struct page *newpage,
+			   struct page *page, enum migrate_mode mode)
+{
+	if (page_has_private(page) &&
+	    !try_to_release_page(page, GFP_KERNEL))
+		return -EAGAIN;
+
+	ASSERT(PageLocked(page));
+
+	return migrate_page(mapping, newpage, page, mode);
+}
+
+int mpc_mmap(struct file *fp, struct vm_area_struct *vma)
+{
+	struct mpc_unit *unit = fp->private_data;
+	struct mpc_xvm *xvm;
+	off_t off;
+	ulong len;
+	u32 key;
+
+	off = vma->vm_pgoff << PAGE_SHIFT;
+	len = vma->vm_end - vma->vm_start - 1;
+
+	/* Verify that the request does not cross an xvm region boundary. */
+	if ((off >> xvm_size_max) != ((off + len) >> xvm_size_max))
+		return -EINVAL;
+
+	/* Acquire a reference on the region map for this region. */
+	key = off >> xvm_size_max;
+
+	xvm = mpc_xvm_lookup(&unit->un_rgnmap, key);
+	if (!xvm)
+		return -EINVAL;
+
+	/*
+	 * Drop the birth ref on first open so that the final call
+	 * to mpc_vm_close() will cause the vma to be destroyed.
+	 */
+	if (atomic_inc_return(&xvm->xvm_opened) == 1)
+		mpc_xvm_put(xvm);
+
+	vma->vm_ops = &mpc_vops_default;
+
+	vma->vm_flags &= ~(VM_RAND_READ | VM_SEQ_READ);
+	vma->vm_flags &= ~(VM_MAYWRITE | VM_MAYEXEC);
+
+	vma->vm_flags = (VM_DONTEXPAND | VM_DONTDUMP | VM_NORESERVE);
+	vma->vm_flags |= VM_MAYREAD | VM_READ | VM_RAND_READ;
+
+	vma->vm_private_data = xvm;
+
+	fp->f_ra.ra_pages = unit->un_ra_pages_max;
+	fp->f_mode |= FMODE_RANDOM;
+
+	return 0;
+}
+
+/**
+ * mpc_fadvise() -
+ *
+ * mpc_fadvise() currently handles only POSIX_FADV_WILLNEED.
+ *
+ * The code path that leads here is: madvise_willneed() -> vfs_fadvise() -> mpc_fadvise()
+ */
+int mpc_fadvise(struct file *file, loff_t offset, loff_t len, int advice)
+{
+	pgoff_t start, end;
+
+	if (!file)
+		return -EINVAL;
+
+	if (advice != POSIX_FADV_WILLNEED)
+		return -EOPNOTSUPP;
+
+	start = offset >> PAGE_SHIFT;
+	end = (offset + len - 1) >> PAGE_SHIFT;
+
+	if (end < start)
+		return -EINVAL;
+
+	/* To force page cache readahead */
+	spin_lock(&file->f_lock);
+	file->f_mode |= FMODE_RANDOM;
+	spin_unlock(&file->f_lock);
+
+	page_cache_sync_readahead(file->f_mapping, &file->f_ra, file, start, end - start + 1);
+
+	return 0;
+}
+
+/**
+ * mpioc_xvm_create() - create an extended VMA map (AKA mcache map)
+ * @unit:
+ * @arg:
+ */
+int mpioc_xvm_create(struct mpc_unit *unit, struct mpool_descriptor *mp, struct mpioc_vma *ioc)
+{
+	struct mpc_mbinfo *mbinfov;
+	struct kmem_cache *cache;
+	struct mpc_rgnmap *rm;
+	struct mpc_xvm *xvm;
+	size_t largest, sz;
+	uint mbidc, mult;
+	u64 *mbidv;
+	int rc, i;
+
+	if (!unit || !unit->un_mapping || !ioc)
+		return -EINVAL;
+
+	if (ioc->im_mbidc < 1)
+		return -EINVAL;
+
+	if (ioc->im_advice > MPC_VMA_PINNED)
+		return -EINVAL;
+
+	mult = 1;
+	if (ioc->im_advice == MPC_VMA_WARM)
+		mult = 10;
+	else if (ioc->im_advice == MPC_VMA_HOT)
+		mult = 100;
+
+	mbidc = ioc->im_mbidc;
+
+	sz = sizeof(*xvm) + sizeof(*mbinfov) * mbidc;
+	if (sz > mpc_xvm_cachesz[1])
+		return -EINVAL;
+	else if (sz > mpc_xvm_cachesz[0])
+		cache = mpc_xvm_cache[1];
+	else
+		cache = mpc_xvm_cache[0];
+
+	sz = mbidc * sizeof(mbidv[0]);
+
+	mbidv = kmalloc(sz, GFP_KERNEL);
+	if (!mbidv)
+		return -ENOMEM;
+
+	rc = copy_from_user(mbidv, ioc->im_mbidv, sz);
+	if (rc) {
+		kfree(mbidv);
+		return -EFAULT;
+	}
+
+	xvm = kmem_cache_zalloc(cache, GFP_KERNEL);
+	if (!xvm) {
+		kfree(mbidv);
+		return -ENOMEM;
+	}
+
+	xvm->xvm_magic = (u32)(uintptr_t)xvm;
+	xvm->xvm_mbinfoc = mbidc;
+	xvm->xvm_mpdesc = mp;
+
+	xvm->xvm_mapping = unit->un_mapping;
+	xvm->xvm_rgnmap = &unit->un_rgnmap;
+	xvm->xvm_advice = ioc->im_advice;
+	kref_init(&xvm->xvm_ref);
+	xvm->xvm_cache = cache;
+	atomic_set(&xvm->xvm_opened, 0);
+
+	atomic64_set(&xvm->xvm_nrpages, 0);
+	atomic_set(&xvm->xvm_rabusy, 0);
+
+	largest = 0;
+
+	mbinfov = xvm->xvm_mbinfov;
+
+	for (i = 0; i < mbidc; ++i) {
+		struct mpc_mbinfo *mbinfo = mbinfov + i;
+		struct mblock_props props;
+
+		rc = mblock_find_get(mp, mbidv[i], 1, &props, &mbinfo->mbdesc);
+		if (rc) {
+			mbidc = i;
+			goto errout;
+		}
+
+		mbinfo->mblen = ALIGN(props.mpr_write_len, PAGE_SIZE);
+		mbinfo->mbmult = mult;
+		atomic64_set(&mbinfo->mbatime, 0);
+
+		largest = max_t(size_t, largest, mbinfo->mblen);
+	}
+
+	xvm->xvm_bktsz = roundup_pow_of_two(largest);
+
+	if (xvm->xvm_bktsz * mbidc > (1ul << xvm_size_max)) {
+		rc = -E2BIG;
+		goto errout;
+	}
+
+	rm = &unit->un_rgnmap;
+
+	mutex_lock(&rm->rm_lock);
+	xvm->xvm_rgn = idr_alloc(&rm->rm_root, NULL, 1, -1, GFP_KERNEL);
+	if (xvm->xvm_rgn < 1) {
+		mutex_unlock(&rm->rm_lock);
+
+		rc = xvm->xvm_rgn ?: -EINVAL;
+		goto errout;
+	}
+
+	ioc->im_offset = (ulong)xvm->xvm_rgn << xvm_size_max;
+	ioc->im_bktsz = xvm->xvm_bktsz;
+	ioc->im_len = xvm->xvm_bktsz * mbidc;
+	ioc->im_len = ALIGN(ioc->im_len, (1ul << xvm_size_max));
+
+	atomic_inc(&rm->rm_rgncnt);
+
+	idr_replace(&rm->rm_root, xvm, xvm->xvm_rgn);
+	mutex_unlock(&rm->rm_lock);
+
+errout:
+	if (rc) {
+		for (i = 0; i < mbidc; ++i)
+			mblock_put(mbinfov[i].mbdesc);
+		kmem_cache_free(cache, xvm);
+	}
+
+	kfree(mbidv);
+
+	return rc;
+}
+
+/**
+ * mpioc_xvm_destroy() - destroy an extended VMA
+ * @unit:
+ * @arg:
+ */
+int mpioc_xvm_destroy(struct mpc_unit *unit, struct mpioc_vma *ioc)
+{
+	struct mpc_rgnmap *rm;
+	struct mpc_xvm *xvm;
+	u64 rgn;
+
+	if (!unit || !ioc)
+		return -EINVAL;
+
+	rgn = ioc->im_offset >> xvm_size_max;
+	rm = &unit->un_rgnmap;
+
+	mutex_lock(&rm->rm_lock);
+	xvm = idr_find(&rm->rm_root, rgn);
+	if (xvm && kref_read(&xvm->xvm_ref) == 1 && !atomic_read(&xvm->xvm_opened))
+		idr_remove(&rm->rm_root, rgn);
+	else
+		xvm = NULL;
+	mutex_unlock(&rm->rm_lock);
+
+	if (xvm)
+		mpc_xvm_put(xvm);
+
+	return 0;
+}
+
+int mpioc_xvm_purge(struct mpc_unit *unit, struct mpioc_vma *ioc)
+{
+	struct mpc_xvm *xvm;
+	u64 rgn;
+
+	if (!unit || !ioc)
+		return -EINVAL;
+
+	rgn = ioc->im_offset >> xvm_size_max;
+
+	xvm = mpc_xvm_lookup(&unit->un_rgnmap, rgn);
+	if (!xvm)
+		return -ENOENT;
+
+	mpc_xvm_put(xvm);
+
+	return 0;
+}
+
+int mpioc_xvm_vrss(struct mpc_unit *unit, struct mpioc_vma *ioc)
+{
+	struct mpc_xvm *xvm;
+	u64 rgn;
+
+	if (!unit || !ioc)
+		return -EINVAL;
+
+	rgn = ioc->im_offset >> xvm_size_max;
+
+	xvm = mpc_xvm_lookup(&unit->un_rgnmap, rgn);
+	if (!xvm)
+		return -ENOENT;
+
+	ioc->im_vssp = mpc_xvm_pglen(xvm);
+	ioc->im_rssp = atomic64_read(&xvm->xvm_nrpages);
+
+	mpc_xvm_put(xvm);
+
+	return 0;
+}
+
+int mcache_init(void)
+{
+	size_t sz;
+	int rc, i;
+
+	xvm_max = clamp_t(uint, xvm_max, 1024, 1u << 30);
+	xvm_size_max = clamp_t(ulong, xvm_size_max, 27, 32);
+
+	sz = sizeof(struct mpc_mbinfo) * 8;
+	mpc_xvm_cachesz[0] = sizeof(struct mpc_xvm) + sz;
+
+	mpc_xvm_cache[0] = kmem_cache_create("mpool_xvm_0", mpc_xvm_cachesz[0], 0,
+					     SLAB_HWCACHE_ALIGN | SLAB_POISON, NULL);
+	if (!mpc_xvm_cache[0]) {
+		rc = -ENOMEM;
+		mp_pr_err("mpc xvm cache 0 create failed", rc);
+		return rc;
+	}
+
+	sz = sizeof(struct mpc_mbinfo) * 32;
+	mpc_xvm_cachesz[1] = sizeof(struct mpc_xvm) + sz;
+
+	mpc_xvm_cache[1] = kmem_cache_create("mpool_xvm_1", mpc_xvm_cachesz[1], 0,
+					     SLAB_HWCACHE_ALIGN | SLAB_POISON, NULL);
+	if (!mpc_xvm_cache[1]) {
+		rc = -ENOMEM;
+		mp_pr_err("mpc xvm cache 1 create failed", rc);
+		goto errout;
+	}
+
+	mpc_wq_trunc = alloc_workqueue("mpc_wq_trunc", WQ_UNBOUND, 16);
+	if (!mpc_wq_trunc) {
+		rc = -ENOMEM;
+		mp_pr_err("trunc workqueue alloc failed", rc);
+		goto errout;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(mpc_wq_rav); ++i) {
+		int     maxactive = 16;
+		char    name[16];
+
+		snprintf(name, sizeof(name), "mpc_wq_ra%d", i);
+
+		mpc_wq_rav[i] = alloc_workqueue(name, 0, maxactive);
+		if (!mpc_wq_rav[i]) {
+			rc = -ENOMEM;
+			mp_pr_err("mpctl ra workqueue alloc failed", rc);
+			goto errout;
+		}
+	}
+
+	return 0;
+
+errout:
+	mcache_exit();
+	return rc;
+}
+
+void mcache_exit(void)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(mpc_wq_rav); ++i) {
+		if (mpc_wq_rav[i])
+			destroy_workqueue(mpc_wq_rav[i]);
+		mpc_wq_rav[i] = NULL;
+	}
+
+	if (mpc_wq_trunc)
+		destroy_workqueue(mpc_wq_trunc);
+	kmem_cache_destroy(mpc_xvm_cache[1]);
+	kmem_cache_destroy(mpc_xvm_cache[0]);
+}
+
+static const struct vm_operations_struct mpc_vops_default = {
+	.open           = mpc_vm_open,
+	.close          = mpc_vm_close,
+	.fault          = mpc_vm_fault,
+};
+
+const struct address_space_operations mpc_aops_default = {
+	.readpages      = mpc_readpages,
+	.releasepage    = mpc_releasepage,
+	.invalidatepage = mpc_invalidatepage,
+	.migratepage    = mpc_migratepage,
+};
diff --git a/drivers/mpool/mcache.h b/drivers/mpool/mcache.h
new file mode 100644
index 000000000000..fe6f45a05494
--- /dev/null
+++ b/drivers/mpool/mcache.h
@@ -0,0 +1,96 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_MCACHE_H
+#define MPOOL_MCACHE_H
+
+#include <linux/kref.h>
+
+#include "init.h"
+#include "mblock.h"
+
+struct mpc_unit;
+
+/**
+ * struct mpc_rgnmap - extended vma (xvm) region management
+ * @rm_lock:    protects rm_root
+ * @rm_root:    root of the region map
+ * @rm_rgncnt;  number of active regions
+ *
+ * Note that this is not a ref-counted object, its lifetime
+ * is tied to struct mpc_unit.
+ */
+struct mpc_rgnmap {
+	struct mutex    rm_lock;
+	struct idr      rm_root;
+	atomic_t        rm_rgncnt;
+} ____cacheline_aligned;
+
+
+struct mpc_mbinfo {
+	struct mblock_descriptor   *mbdesc;
+	u32                         mblen;
+	u32                         mbmult;
+	atomic64_t                  mbatime;
+} __aligned(32);
+
+struct mpc_xvm {
+	size_t                      xvm_bktsz;
+	uint                        xvm_mbinfoc;
+	uint                        xvm_rgn;
+	struct kref                 xvm_ref;
+	u32                         xvm_magic;
+	struct mpool_descriptor    *xvm_mpdesc;
+
+	atomic64_t                 *xvm_hcpagesp;
+	struct address_space       *xvm_mapping;
+	struct mpc_rgnmap          *xvm_rgnmap;
+
+	enum mpc_vma_advice         xvm_advice;
+	atomic_t                    xvm_opened;
+	struct kmem_cache          *xvm_cache;
+	struct mpc_xvm             *xvm_next;
+
+	____cacheline_aligned
+	atomic64_t                  xvm_nrpages;
+	atomic_t                    xvm_rabusy;
+	struct work_struct          xvm_work;
+
+	____cacheline_aligned
+	struct mpc_mbinfo           xvm_mbinfov[];
+};
+
+extern const struct address_space_operations mpc_aops_default;
+
+void mpc_rgnmap_flush(struct mpc_rgnmap *rm);
+
+int mpc_mmap(struct file *fp, struct vm_area_struct *vma);
+
+int mpc_fadvise(struct file *file, loff_t offset, loff_t len, int advice);
+
+int mpioc_xvm_create(struct mpc_unit *unit, struct mpool_descriptor *mp, struct mpioc_vma *ioc);
+
+int mpioc_xvm_destroy(struct mpc_unit *unit, struct mpioc_vma *ioc);
+
+int mpioc_xvm_purge(struct mpc_unit *unit, struct mpioc_vma *ioc);
+
+int mpioc_xvm_vrss(struct mpc_unit *unit, struct mpioc_vma *ioc);
+
+void mpc_xvm_free(struct mpc_xvm *xvm);
+
+int mcache_init(void) __cold;
+void mcache_exit(void) __cold;
+
+static inline pgoff_t mpc_xvm_pgoff(struct mpc_xvm *xvm)
+{
+	return ((ulong)xvm->xvm_rgn << xvm_size_max) >> PAGE_SHIFT;
+}
+
+static inline size_t mpc_xvm_pglen(struct mpc_xvm *xvm)
+{
+	return (xvm->xvm_bktsz * xvm->xvm_mbinfoc) >> PAGE_SHIFT;
+}
+
+#endif /* MPOOL_MCACHE_H */
diff --git a/drivers/mpool/mpctl.c b/drivers/mpool/mpctl.c
index 3a231cf982b3..f11f522ec90c 100644
--- a/drivers/mpool/mpctl.c
+++ b/drivers/mpool/mpctl.c
@@ -142,6 +142,11 @@ static ssize_t mpc_label_show(struct device *dev, struct device_attribute *da, c
 	return scnprintf(buf, PAGE_SIZE, "%s\n", dev_to_unit(dev)->un_label);
 }
 
+static ssize_t mpc_vma_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%u\n", xvm_size_max);
+}
+
 static ssize_t mpc_type_show(struct device *dev, struct device_attribute *da, char *buf)
 {
 	struct mpool_uuid  uuid;
@@ -160,6 +165,7 @@ static void mpc_mpool_params_add(struct device_attribute *dattr)
 	MPC_ATTR_RO(dattr++, mode);
 	MPC_ATTR_RO(dattr++, ra);
 	MPC_ATTR_RO(dattr++, label);
+	MPC_ATTR_RO(dattr++, vma);
 	MPC_ATTR_RO(dattr,   type);
 }
 
@@ -243,6 +249,8 @@ static void mpool_params_merge_defaults(struct mpool_params *params)
 	if (params->mp_mode != -1)
 		params->mp_mode &= 0777;
 
+	params->mp_vma_size_max = xvm_size_max;
+
 	params->mp_rsvd0 = 0;
 	params->mp_rsvd1 = 0;
 	params->mp_rsvd2 = 0;
@@ -382,6 +390,10 @@ static int mpc_unit_create(const char *name, struct mpc_mpool *mpool, struct mpc
 	kref_init(&unit->un_ref);
 	unit->un_mpool = mpool;
 
+	mutex_init(&unit->un_rgnmap.rm_lock);
+	idr_init(&unit->un_rgnmap.rm_root);
+	atomic_set(&unit->un_rgnmap.rm_rgncnt, 0);
+
 	mutex_lock(&ss->ss_lock);
 	minor = idr_alloc(&ss->ss_unitmap, NULL, 0, -1, GFP_KERNEL);
 	mutex_unlock(&ss->ss_lock);
@@ -421,6 +433,8 @@ static void mpc_unit_release(struct kref *refp)
 	if (unit->un_device)
 		device_destroy(ss->ss_class, unit->un_devno);
 
+	idr_destroy(&unit->un_rgnmap.rm_root);
+
 	kfree(unit);
 }
 
@@ -633,6 +647,7 @@ static int mpc_cf_journal(struct mpc_unit *unit)
 	cfg.mc_oid2 = unit->un_ds_oidv[1];
 	cfg.mc_captgt = unit->un_mdc_captgt;
 	cfg.mc_ra_pages_max = unit->un_ra_pages_max;
+	cfg.mc_vma_size_max = xvm_size_max;
 	memcpy(&cfg.mc_utype, &unit->un_utype, sizeof(cfg.mc_utype));
 	strlcpy(cfg.mc_label, unit->un_label, sizeof(cfg.mc_label));
 
@@ -742,6 +757,7 @@ static int mpioc_params_get(struct mpc_unit *unit, struct mpioc_params *get)
 	params->mp_oidv[0] = unit->un_ds_oidv[0];
 	params->mp_oidv[1] = unit->un_ds_oidv[1];
 	params->mp_ra_pages_max = unit->un_ra_pages_max;
+	params->mp_vma_size_max = xvm_size_max;
 	memcpy(&params->mp_utype, &unit->un_utype, sizeof(params->mp_utype));
 	strlcpy(params->mp_label, unit->un_label, sizeof(params->mp_label));
 	strlcpy(params->mp_name, unit->un_name, sizeof(params->mp_name));
@@ -785,6 +801,8 @@ static int mpioc_params_set(struct mpc_unit *unit, struct mpioc_params *set)
 
 	params = &set->mps_params;
 
+	params->mp_vma_size_max = xvm_size_max;
+
 	mutex_lock(&ss->ss_lock);
 	if (params->mp_uid != -1 || params->mp_gid != -1 || params->mp_mode != -1) {
 		err = mpc_mp_chown(unit, params);
@@ -919,6 +937,7 @@ static void mpioc_prop_get(struct mpc_unit *unit, struct mpioc_prop *kprop)
 	params->mp_oidv[0] = unit->un_ds_oidv[0];
 	params->mp_oidv[1] = unit->un_ds_oidv[1];
 	params->mp_ra_pages_max = unit->un_ra_pages_max;
+	params->mp_vma_size_max = xvm_size_max;
 	memcpy(&params->mp_utype, &unit->un_utype, sizeof(params->mp_utype));
 	strlcpy(params->mp_label, unit->un_label, sizeof(params->mp_label));
 	strlcpy(params->mp_name, unit->un_name, sizeof(params->mp_name));
@@ -1171,6 +1190,7 @@ static int mpioc_mp_create(struct mpc_unit *ctl, struct mpioc_mpool *mp,
 	cfg.mc_rsvd0 = mp->mp_params.mp_rsvd0;
 	cfg.mc_captgt = MPOOL_ROOT_LOG_CAP;
 	cfg.mc_ra_pages_max = mp->mp_params.mp_ra_pages_max;
+	cfg.mc_vma_size_max = mp->mp_params.mp_vma_size_max;
 	cfg.mc_rsvd1 = mp->mp_params.mp_rsvd1;
 	cfg.mc_rsvd2 = mp->mp_params.mp_rsvd2;
 	cfg.mc_rsvd3 = mp->mp_params.mp_rsvd3;
@@ -1300,6 +1320,7 @@ static int mpioc_mp_activate(struct mpc_unit *ctl, struct mpioc_mpool *mp,
 	mp->mp_params.mp_oidv[0] = cfg.mc_oid1;
 	mp->mp_params.mp_oidv[1] = cfg.mc_oid2;
 	mp->mp_params.mp_ra_pages_max = cfg.mc_ra_pages_max;
+	mp->mp_params.mp_vma_size_max = cfg.mc_vma_size_max;
 	memcpy(&mp->mp_params.mp_utype, &cfg.mc_utype, sizeof(mp->mp_params.mp_utype));
 	strlcpy(mp->mp_params.mp_label, cfg.mc_label, sizeof(mp->mp_params.mp_label));
 
@@ -2313,6 +2334,7 @@ static int mpc_open(struct inode *ip, struct file *fp)
 	}
 
 	fp->f_op = &mpc_fops_default;
+	fp->f_mapping->a_ops = &mpc_aops_default;
 
 	unit->un_mapping = fp->f_mapping;
 
@@ -2361,8 +2383,11 @@ static int mpc_release(struct inode *ip, struct file *fp)
 	if (!lastclose)
 		goto errout;
 
-	if (mpc_unit_ismpooldev(unit))
+	if (mpc_unit_ismpooldev(unit)) {
+		mpc_rgnmap_flush(&unit->un_rgnmap);
+
 		unit->un_mapping = NULL;
+	}
 
 	unit->un_open_excl = false;
 
@@ -2530,6 +2555,22 @@ static long mpc_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
 		rc = mpioc_mlog_erase(unit, argp);
 		break;
 
+	case MPIOC_VMA_CREATE:
+		rc = mpioc_xvm_create(unit, unit->un_mpool->mp_desc, argp);
+		break;
+
+	case MPIOC_VMA_DESTROY:
+		rc = mpioc_xvm_destroy(unit, argp);
+		break;
+
+	case MPIOC_VMA_PURGE:
+		rc = mpioc_xvm_purge(unit, argp);
+		break;
+
+	case MPIOC_VMA_VRSS:
+		rc = mpioc_xvm_vrss(unit, argp);
+		break;
+
 	default:
 		rc = -ENOTTY;
 		mp_pr_rl("invalid command %x: dir=%u type=%c nr=%u size=%u",
@@ -2553,6 +2594,8 @@ static const struct file_operations mpc_fops_default = {
 	.open		= mpc_open,
 	.release	= mpc_release,
 	.unlocked_ioctl	= mpc_ioctl,
+	.mmap           = mpc_mmap,
+	.fadvise        = mpc_fadvise,
 };
 
 static int mpc_exit_unit(int minor, void *item, void *arg)
diff --git a/drivers/mpool/mpctl.h b/drivers/mpool/mpctl.h
index 412a6a491c15..b93e44248f03 100644
--- a/drivers/mpool/mpctl.h
+++ b/drivers/mpool/mpctl.h
@@ -11,6 +11,8 @@
 #include <linux/device.h>
 #include <linux/semaphore.h>
 
+#include "mcache.h"
+
 #define ITERCB_NEXT     (0)
 #define ITERCB_DONE     (1)
 
@@ -23,6 +25,7 @@ struct mpc_unit {
 	uid_t                       un_uid;
 	gid_t                       un_gid;
 	mode_t                      un_mode;
+	struct mpc_rgnmap           un_rgnmap;
 	dev_t                       un_devno;
 	const struct mpc_uinfo     *un_uinfo;
 	struct mpc_mpool           *un_mpool;
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:36 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:36 -0500
Subject: [PATCH v2 22/22] mpool: add Kconfig and Makefile
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-23-nmeeramohide@micron.com>

This adds the Kconfig and Makefile for mpool.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/Kconfig        |  2 ++
 drivers/Makefile       |  1 +
 drivers/mpool/Kconfig  | 28 ++++++++++++++++++++++++++++
 drivers/mpool/Makefile | 11 +++++++++++
 4 files changed, 42 insertions(+)
 create mode 100644 drivers/mpool/Kconfig
 create mode 100644 drivers/mpool/Makefile

diff --git a/drivers/Kconfig b/drivers/Kconfig
index dcecc9f6e33f..547ac47a10eb 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -235,4 +235,6 @@ source "drivers/interconnect/Kconfig"
 source "drivers/counter/Kconfig"
 
 source "drivers/most/Kconfig"
+
+source "drivers/mpool/Kconfig"
 endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index c0cd1b9075e3..e2477288e761 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -188,3 +188,4 @@ obj-$(CONFIG_GNSS)		+= gnss/
 obj-$(CONFIG_INTERCONNECT)	+= interconnect/
 obj-$(CONFIG_COUNTER)		+= counter/
 obj-$(CONFIG_MOST)		+= most/
+obj-$(CONFIG_MPOOL)		+= mpool/
diff --git a/drivers/mpool/Kconfig b/drivers/mpool/Kconfig
new file mode 100644
index 000000000000..33380f497473
--- /dev/null
+++ b/drivers/mpool/Kconfig
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Object Storage Media Pool (mpool) configuration
+#
+
+config MPOOL
+	tristate "Object Storage Media Pool"
+	depends on BLOCK
+	default n
+	help
+	  This module implements a simple transactional object store on top of
+	  block storage devices.
+
+	  Mpool provides a high-performance alternative to file systems or
+	  raw block devices for applications that can benefit from its simple
+	  object storage model and unique features.
+
+	  If you want to use mpool, choose M here: the module will be called mpool.
+
+config MPOOL_ASSERT
+	bool "Object Storage Media Pool assert support"
+	depends on MPOOL
+	default n
+	help
+	  Enables runtime assertion checking for mpool.
+
+	  This is a developer only config. If this config is enabled and any of the
+	  asserts trigger, it results in a panic.
diff --git a/drivers/mpool/Makefile b/drivers/mpool/Makefile
new file mode 100644
index 000000000000..374bbe5bcfa0
--- /dev/null
+++ b/drivers/mpool/Makefile
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# Makefile for Object Storage Media Pool (mpool)
+#
+
+obj-$(CONFIG_MPOOL) += mpool.o
+
+mpool-y             := init.o pd.o mclass.o smap.o omf.o \
+		       upgrade.o sb.o pmd_obj.o mblock.o  \
+		       mlog_utils.o mlog.o mdc.o mpcore.o pmd.o \
+		       mp.o mpctl.o sysfs.o mcache.o reaper.o
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:34 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:34 -0500
Subject: [PATCH v2 20/22] mpool: add support to proactively evict cached
 mblock data from the page-cache
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-21-nmeeramohide@micron.com>

This adds a mechanism to track object-level usage metrics and use
that metrics to proactively evict mblock data from the page cache.

The proactive reaping is employed just before the onset of memory
pressure, which greatly improves throughput and reduces tail
latencies for read and memory intensive workloads.

The reaper component tracks residency of pages from specified xvms
and weighs the memory used against system free memory. The reaper
begins evicting pages from the specified xvm when the free memory
falls below a predetermined low watermark, stopping when the free
memory rises above a high watermark.

The reaper maintains several lists of xvms and cycles through them
in a round-robin fashion to select pages to evict. Each xvm is
comprised of one or more contiguous virtual subranges of pages,
where each subrange is delineated by an mblock. Each mblock has an
associated access time which is updated on each page fault to any
page in the mblock.  The reaper leverages the atime to decide
whether or not to evict all the pages in the subrange based upon
the current TTL, where the current TTL grows shorter as the urgency
to evict pages grows stronger.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mcache.c |  43 +++
 drivers/mpool/mcache.h |   8 +
 drivers/mpool/mpctl.c  |  15 +
 drivers/mpool/mpctl.h  |   6 +
 drivers/mpool/reaper.c | 686 +++++++++++++++++++++++++++++++++++++++++
 drivers/mpool/reaper.h |  71 +++++
 6 files changed, 829 insertions(+)
 create mode 100644 drivers/mpool/reaper.c
 create mode 100644 drivers/mpool/reaper.h

diff --git a/drivers/mpool/mcache.c b/drivers/mpool/mcache.c
index 07c79615ecf1..1f1b9173a2b4 100644
--- a/drivers/mpool/mcache.c
+++ b/drivers/mpool/mcache.c
@@ -19,6 +19,7 @@
 #include "assert.h"
 
 #include "mpctl.h"
+#include "reaper.h"
 
 #ifndef lru_to_page
 #define lru_to_page(_head)  (list_entry((_head)->prev, struct page, lru))
@@ -64,6 +65,7 @@ const struct address_space_operations mpc_aops_default;
 
 static struct workqueue_struct *mpc_wq_trunc __read_mostly;
 static struct workqueue_struct *mpc_wq_rav[4] __read_mostly;
+struct mpc_reap *mpc_reap __read_mostly;
 
 static size_t mpc_xvm_cachesz[2] __read_mostly;
 static struct kmem_cache *mpc_xvm_cache[2] __read_mostly;
@@ -109,6 +111,10 @@ void mpc_rgnmap_flush(struct mpc_rgnmap *rm)
 		head = xvm->xvm_next;
 		mpc_xvm_put(xvm);
 	}
+
+	/* Wait for reaper to prune its lists... */
+	while (atomic_read(&rm->rm_rgncnt) > 0)
+		usleep_range(100000, 150000);
 }
 
 static struct mpc_xvm *mpc_xvm_lookup(struct mpc_rgnmap *rm, uint key)
@@ -129,6 +135,22 @@ void mpc_xvm_free(struct mpc_xvm *xvm)
 	struct mpc_rgnmap *rm;
 
 	ASSERT((u32)(uintptr_t)xvm == xvm->xvm_magic);
+	ASSERT(atomic_read(&xvm->xvm_reapref) > 0);
+
+again:
+	mpc_reap_xvm_evict(xvm);
+
+	if (atomic_dec_return(&xvm->xvm_reapref) > 0) {
+		atomic_inc(xvm->xvm_freedp);
+		return;
+	}
+
+	if (atomic64_read(&xvm->xvm_nrpages) > 0) {
+		atomic_cmpxchg(&xvm->xvm_evicting, 1, 0);
+		atomic_inc(&xvm->xvm_reapref);
+		usleep_range(10000, 30000);
+		goto again;
+	}
 
 	rm = xvm->xvm_rgnmap;
 
@@ -337,6 +359,8 @@ static vm_fault_t mpc_vm_fault_impl(struct vm_area_struct *vma, struct vm_fault
 	/* Page is locked with a ref. */
 	vmf->page = page;
 
+	mpc_reap_xvm_touch(vma->vm_private_data, page->index);
+
 	return vmfrc | VM_FAULT_LOCKED;
 }
 
@@ -534,6 +558,9 @@ static int mpc_readpages(struct file *file, struct address_space *mapping,
 	gfp = mapping_gfp_mask(mapping) & GFP_KERNEL;
 	wq = mpc_rgn2wq(xvm->xvm_rgn);
 
+	if (mpc_reap_xvm_duress(xvm))
+		nr_pages = min_t(uint, nr_pages, 8);
+
 	nr_pages = min_t(uint, nr_pages, ra_pages_max);
 
 	for (i = 0; i < nr_pages; ++i) {
@@ -603,6 +630,8 @@ static int mpc_readpages(struct file *file, struct address_space *mapping,
  * @gfp:
  *
  * The function is added as part of tracking incoming and outgoing pages.
+ * When the number of pages owned exceeds the limit (if defined) reap function
+ * will get invoked to trim down the usage.
  */
 static int mpc_releasepage(struct page *page, gfp_t gfp)
 {
@@ -699,6 +728,8 @@ int mpc_mmap(struct file *fp, struct vm_area_struct *vma)
 	fp->f_ra.ra_pages = unit->un_ra_pages_max;
 	fp->f_mode |= FMODE_RANDOM;
 
+	mpc_reap_xvm_add(unit->un_ds_reap, xvm);
+
 	return 0;
 }
 
@@ -805,6 +836,9 @@ int mpioc_xvm_create(struct mpc_unit *unit, struct mpool_descriptor *mp, struct
 	xvm->xvm_cache = cache;
 	atomic_set(&xvm->xvm_opened, 0);
 
+	INIT_LIST_HEAD(&xvm->xvm_list);
+	atomic_set(&xvm->xvm_evicting, 0);
+	atomic_set(&xvm->xvm_reapref, 1);
 	atomic64_set(&xvm->xvm_nrpages, 0);
 	atomic_set(&xvm->xvm_rabusy, 0);
 
@@ -914,6 +948,8 @@ int mpioc_xvm_purge(struct mpc_unit *unit, struct mpioc_vma *ioc)
 	if (!xvm)
 		return -ENOENT;
 
+	mpc_reap_xvm_evict(xvm);
+
 	mpc_xvm_put(xvm);
 
 	return 0;
@@ -978,6 +1014,12 @@ int mcache_init(void)
 		goto errout;
 	}
 
+	rc = mpc_reap_create(&mpc_reap);
+	if (rc) {
+		mp_pr_err("reap create failed", rc);
+		goto errout;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(mpc_wq_rav); ++i) {
 		int     maxactive = 16;
 		char    name[16];
@@ -1009,6 +1051,7 @@ void mcache_exit(void)
 		mpc_wq_rav[i] = NULL;
 	}
 
+	mpc_reap_destroy(mpc_reap);
 	if (mpc_wq_trunc)
 		destroy_workqueue(mpc_wq_trunc);
 	kmem_cache_destroy(mpc_xvm_cache[1]);
diff --git a/drivers/mpool/mcache.h b/drivers/mpool/mcache.h
index fe6f45a05494..8ddb407c862b 100644
--- a/drivers/mpool/mcache.h
+++ b/drivers/mpool/mcache.h
@@ -47,12 +47,19 @@ struct mpc_xvm {
 	atomic64_t                 *xvm_hcpagesp;
 	struct address_space       *xvm_mapping;
 	struct mpc_rgnmap          *xvm_rgnmap;
+	struct mpc_reap            *xvm_reap;
 
 	enum mpc_vma_advice         xvm_advice;
 	atomic_t                    xvm_opened;
 	struct kmem_cache          *xvm_cache;
 	struct mpc_xvm             *xvm_next;
 
+	____cacheline_aligned
+	struct list_head            xvm_list;
+	atomic_t                    xvm_evicting;
+	atomic_t                    xvm_reapref;
+	atomic_t                   *xvm_freedp;
+
 	____cacheline_aligned
 	atomic64_t                  xvm_nrpages;
 	atomic_t                    xvm_rabusy;
@@ -62,6 +69,7 @@ struct mpc_xvm {
 	struct mpc_mbinfo           xvm_mbinfov[];
 };
 
+extern struct mpc_reap *mpc_reap;
 extern const struct address_space_operations mpc_aops_default;
 
 void mpc_rgnmap_flush(struct mpc_rgnmap *rm);
diff --git a/drivers/mpool/mpctl.c b/drivers/mpool/mpctl.c
index f11f522ec90c..e42abffdcc14 100644
--- a/drivers/mpool/mpctl.c
+++ b/drivers/mpool/mpctl.c
@@ -38,6 +38,7 @@
 #include "mp.h"
 #include "mpctl.h"
 #include "sysfs.h"
+#include "reaper.h"
 #include "init.h"
 
 
@@ -185,6 +186,10 @@ static int mpc_params_register(struct mpc_unit *unit, int cnt)
 	if (mpc_unit_ismpooldev(unit))
 		mpc_mpool_params_add(dattr);
 
+	/* Common parameters */
+	if (mpc_unit_isctldev(unit))
+		mpc_reap_params_add(dattr);
+
 	rc = mpc_attr_group_create(attr);
 	if (rc) {
 		mpc_attr_destroy(attr);
@@ -2337,6 +2342,7 @@ static int mpc_open(struct inode *ip, struct file *fp)
 	fp->f_mapping->a_ops = &mpc_aops_default;
 
 	unit->un_mapping = fp->f_mapping;
+	unit->un_ds_reap = mpc_reap;
 
 	inode_lock(ip);
 	i_size_write(ip, 1ul << (__BITS_PER_LONG - 1));
@@ -2386,6 +2392,7 @@ static int mpc_release(struct inode *ip, struct file *fp)
 	if (mpc_unit_ismpooldev(unit)) {
 		mpc_rgnmap_flush(&unit->un_rgnmap);
 
+		unit->un_ds_reap = NULL;
 		unit->un_mapping = NULL;
 	}
 
@@ -2714,6 +2721,14 @@ int mpctl_init(void)
 		goto errout;
 	}
 
+	/* The reaper component has already been initialized before mpctl. */
+	ctlunit->un_ds_reap = mpc_reap;
+	rc = mpc_params_register(ctlunit, MPC_REAP_PARAMS_CNT);
+	if (rc) {
+		errmsg = "cannot register common parameters";
+		goto errout;
+	}
+
 	mutex_lock(&ss->ss_lock);
 	idr_replace(&ss->ss_unitmap, ctlunit, MINOR(ctlunit->un_devno));
 	mutex_unlock(&ss->ss_lock);
diff --git a/drivers/mpool/mpctl.h b/drivers/mpool/mpctl.h
index b93e44248f03..1a582bf1a474 100644
--- a/drivers/mpool/mpctl.h
+++ b/drivers/mpool/mpctl.h
@@ -30,6 +30,7 @@ struct mpc_unit {
 	const struct mpc_uinfo     *un_uinfo;
 	struct mpc_mpool           *un_mpool;
 	struct address_space       *un_mapping;
+	struct mpc_reap            *un_ds_reap;
 	struct device              *un_device;
 	struct mpc_attr            *un_attr;
 	uint                        un_rawio;       /* log2(max_mblock_size) */
@@ -46,6 +47,11 @@ static inline struct mpc_unit *dev_to_unit(struct device *dev)
 	return dev_get_drvdata(dev);
 }
 
+static inline struct mpc_reap *dev_to_reap(struct device *dev)
+{
+	return dev_to_unit(dev)->un_ds_reap;
+}
+
 int mpctl_init(void) __cold;
 void mpctl_exit(void) __cold;
 
diff --git a/drivers/mpool/reaper.c b/drivers/mpool/reaper.c
new file mode 100644
index 000000000000..364d692a71ee
--- /dev/null
+++ b/drivers/mpool/reaper.c
@@ -0,0 +1,686 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+/*
+ * The reaper subsystem tracks residency of pages from specified VMAs and
+ * weighs the memory used against system free memory.  Should the relative
+ * residency vs free memory fall below a predetermined low watermark the
+ * reaper begins evicting pages from the specified VMAs, stopping when free
+ * memory rises above a high watermark that is slightly higher than the low
+ * watermark.
+ *
+ * The reaper maintains several lists of VMAs and cycles through them in a
+ * round-robin fashion to select pages to evict.  Each VMA is comprised of
+ * one or more contiguous virtual subranges of pages, where each subrange
+ * is delineated by an mblock (typically no larger than 32M).  Each mblock
+ * has an associated access time which is updated on each page fault to any
+ * page in the mblock.  The reaper leverages the atime to decide whether or
+ * not to evict all the pages in the subrange based upon the current TTL,
+ * where the current TTL grows shorter as the urgency to evict pages grows
+ * stronger.
+ */
+
+#include <linux/kernel.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/delay.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/sched/clock.h>
+
+#include "mpool_printk.h"
+#include "assert.h"
+
+#include "sysfs.h"
+#include "mpctl.h"
+#include "reaper.h"
+
+#define REAP_ELEM_MAX       3
+
+/**
+ * struct mpc_reap_elem -
+ * @reap_lock:      lock to protect reap_list
+ * @reap_list:      list of inodes
+ * @reap_active:    reaping in progress
+ * @reap_hpages:    total hot pages which are mapped
+ * @reap_cpages:    total coldpages which are mapped
+ */
+struct mpc_reap_elem {
+	struct mutex            reap_lock;
+	struct list_head        reap_list;
+
+	____cacheline_aligned
+	atomic_t                reap_running;
+	struct work_struct      reap_work;
+	struct mpc_reap        *reap_reap;
+
+	____cacheline_aligned
+	atomic64_t              reap_hpages;
+	atomic_t                reap_nfreed;
+
+	____cacheline_aligned
+	atomic64_t              reap_wpages;
+
+	____cacheline_aligned
+	atomic64_t              reap_cpages;
+};
+
+/**
+ * struct mpc_reap -
+ * @reap_lwm:     Low water mark
+ * @reap_ttl_cur: Current time-to-live
+ * @reap_wq:
+ *
+ * @reap_hdr:     sysctl table header
+ * @reap_tab:     sysctl table components
+ * @reap_mempct:
+ * @reap_ttl:
+ * @reap_debug:
+ *
+ * @reap_eidx:    Pruner element index
+ * @reap_emit:    Pruner debug message control
+ * @reap_dwork:
+ * @reap_elem:    Array of reaper lists (reaper pool)
+ */
+struct mpc_reap {
+	atomic_t                    reap_lwm;
+	atomic_t                    reap_ttl_cur;
+	struct workqueue_struct    *reap_wq;
+
+	____cacheline_aligned
+	unsigned int                reap_mempct;
+	unsigned int                reap_ttl;
+	unsigned int                reap_debug;
+
+	____cacheline_aligned
+	atomic_t                    reap_eidx;
+	atomic_t                    reap_emit;
+	struct delayed_work         reap_dwork;
+
+	struct mpc_reap_elem        reap_elem[REAP_ELEM_MAX];
+};
+
+/**
+ * mpc_reap_meminfo() - Get current system-wide memory usage
+ * @freep:    ptr to return bytes of free memory
+ * @availp:   ptr to return bytes of available memory
+ * @shift:    shift results by %shift bits
+ *
+ * %mpc_reap_meminfo() returns current free and available memory
+ * sizes obtained from /proc/meminfo in userland and si_meminfo()
+ * in the kernel.  The resulting sizes are in bytes, but the
+ * caller can supply a non-zero %shift argment to obtain results
+ * in different units (e.g., for MiB shift=20, for GiB shift=30).
+ *
+ * %freep and/or %availp may be NULL.
+ */
+static void mpc_reap_meminfo(ulong *freep, ulong *availp, uint shift)
+{
+	struct sysinfo si;
+
+	si_meminfo(&si);
+
+	if (freep)
+		*freep = (si.freeram * si.mem_unit) >> shift;
+
+	if (availp)
+		*availp = (si_mem_available() * si.mem_unit) >> shift;
+}
+
+static void mpc_reap_evict_vma(struct mpc_xvm *xvm)
+{
+	struct address_space *mapping = xvm->xvm_mapping;
+	struct mpc_reap *reap = xvm->xvm_reap;
+	pgoff_t off, bktsz, len;
+	u64 ttl, xtime, now;
+	int i;
+
+	bktsz = xvm->xvm_bktsz >> PAGE_SHIFT;
+	off = mpc_xvm_pgoff(xvm);
+
+	ttl = atomic_read(&reap->reap_ttl_cur) * 1000ul;
+	now = local_clock();
+
+	for (i = 0; i < xvm->xvm_mbinfoc; ++i, off += bktsz) {
+		struct mpc_mbinfo *mbinfo = xvm->xvm_mbinfov + i;
+
+		xtime = now - (ttl * mbinfo->mbmult);
+		len = mbinfo->mblen >> PAGE_SHIFT;
+
+		if (atomic64_read(&mbinfo->mbatime) > xtime)
+			continue;
+
+		atomic64_set(&mbinfo->mbatime, U64_MAX);
+
+		invalidate_inode_pages2_range(mapping, off, off + len);
+
+		if (atomic64_read(&xvm->xvm_nrpages) < 32)
+			break;
+
+		if (need_resched())
+			cond_resched();
+
+		ttl = atomic_read(&reap->reap_ttl_cur) * 1000ul;
+		now = local_clock();
+	}
+}
+
+/**
+ * mpc_reap_evict() - Evict "cold" pages from the given XVMs
+ * @process:    A list of one of more XVMs to be reaped
+ */
+static void mpc_reap_evict(struct list_head *process)
+{
+	struct mpc_xvm *xvm, *next;
+
+	list_for_each_entry_safe(xvm, next, process, xvm_list) {
+		if (atomic_read(&xvm->xvm_reap->reap_lwm))
+			mpc_reap_evict_vma(xvm);
+
+		atomic_cmpxchg(&xvm->xvm_evicting, 1, 0);
+	}
+}
+
+/**
+ * mpc_reap_scan() - Scan for pages to purge
+ * @elem:
+ * @idx:    reap list index
+ */
+static void mpc_reap_scan(struct mpc_reap_elem *elem)
+{
+	struct list_head *list, process;
+	struct mpc_xvm *xvm, *next;
+	u64 nrpages, n;
+
+	INIT_LIST_HEAD(&process);
+
+	mutex_lock(&elem->reap_lock);
+	list = &elem->reap_list;
+	n = 0;
+
+	list_for_each_entry_safe(xvm, next, list, xvm_list) {
+		nrpages = atomic64_read(&xvm->xvm_nrpages);
+
+		if (nrpages < 32)
+			continue;
+
+		if (atomic_read(&xvm->xvm_reapref) == 1)
+			continue;
+
+		if (atomic_cmpxchg(&xvm->xvm_evicting, 0, 1))
+			continue;
+
+		list_del(&xvm->xvm_list);
+		list_add(&xvm->xvm_list, &process);
+
+		if (++n > 4)
+			break;
+	}
+	mutex_unlock(&elem->reap_lock);
+
+	mpc_reap_evict(&process);
+
+	mutex_lock(&elem->reap_lock);
+	list_splice_tail(&process, list);
+	mutex_unlock(&elem->reap_lock);
+
+	usleep_range(300, 700);
+}
+
+static void mpc_reap_run(struct work_struct *work)
+{
+	struct mpc_reap_elem *elem;
+	struct mpc_reap *reap;
+
+	elem = container_of(work, struct mpc_reap_elem, reap_work);
+	reap = elem->reap_reap;
+
+	while (atomic_read(&reap->reap_lwm))
+		mpc_reap_scan(elem);
+
+	atomic_cmpxchg(&elem->reap_running, 1, 0);
+}
+
+/**
+ * mpc_reap_tune() - Dynamic tuning of reap knobs.
+ * @reap:
+ */
+static void mpc_reap_tune(struct mpc_reap *reap)
+{
+	ulong total_pages, hpages, wpages, cpages, mfree;
+	uint freepct, hwm, lwm, ttl, debug, i;
+
+	hpages = wpages = cpages = 0;
+
+	/*
+	 * Take a live snapshot of the current memory usage.  Disable
+	 * preemption so that the result is reasonably accurate.
+	 */
+	preempt_disable();
+	mpc_reap_meminfo(&mfree, NULL, PAGE_SHIFT);
+
+	for (i = 0; i < REAP_ELEM_MAX; ++i) {
+		struct mpc_reap_elem *elem = &reap->reap_elem[i];
+
+		hpages += atomic64_read(&elem->reap_hpages);
+		wpages += atomic64_read(&elem->reap_wpages);
+		cpages += atomic64_read(&elem->reap_cpages);
+	}
+	preempt_enable();
+
+	total_pages = mfree + hpages + wpages + cpages;
+
+	/*
+	 * Determine the current percentage of free memory relative to the
+	 * number of hot+warm+cold pages tracked by the reaper.  freepct,
+	 * lwm, and hwm are scaled to 10000 for finer resolution.
+	 */
+	freepct = ((hpages + wpages + cpages) * 10000) / total_pages;
+	freepct = 10000 - freepct;
+
+	lwm = (100 - reap->reap_mempct) * 100;
+	hwm = (lwm * 10300) / 10000;
+	hwm = min_t(u32, hwm, 9700);
+	ttl = reap->reap_ttl;
+
+	if (freepct >= hwm) {
+		if (atomic_read(&reap->reap_ttl_cur) != ttl)
+			atomic_set(&reap->reap_ttl_cur, ttl);
+		if (atomic_read(&reap->reap_lwm))
+			atomic_set(&reap->reap_lwm, 0);
+	} else if (freepct < lwm || atomic_read(&reap->reap_lwm) > 0) {
+		ulong x = 10000 - (freepct * 10000) / hwm;
+
+		if (atomic_read(&reap->reap_lwm) != x) {
+			atomic_set(&reap->reap_lwm, x);
+
+			x = (ttl * (500ul * 500)) / (x * x);
+			if (x > ttl)
+				x = ttl;
+
+			atomic_set(&reap->reap_ttl_cur, x);
+		}
+	}
+
+	debug = reap->reap_debug;
+	if (!debug || (debug == 1 && freepct > hwm))
+		return;
+
+	if (atomic_inc_return(&reap->reap_emit) % REAP_ELEM_MAX > 0)
+		return;
+
+	mp_pr_info(
+		"%lu %lu, hot %lu, warm %lu, cold %lu, freepct %u, lwm %u, hwm %u, %2u, ttl %u",
+		mfree >> (20 - PAGE_SHIFT), total_pages >> (20 - PAGE_SHIFT),
+		hpages >> (20 - PAGE_SHIFT), wpages >> (20 - PAGE_SHIFT),
+		cpages >> (20 - PAGE_SHIFT), freepct, lwm, hwm,
+		atomic_read(&reap->reap_lwm), atomic_read(&reap->reap_ttl_cur) / 1000);
+}
+
+static void mpc_reap_prune(struct work_struct *work)
+{
+	struct mpc_reap_elem *elem;
+	struct mpc_xvm *xvm, *next;
+	struct mpc_reap *reap;
+	struct list_head freeme;
+	uint nfreed, eidx;
+	ulong delay;
+
+	reap = container_of(work, struct mpc_reap, reap_dwork.work);
+
+	/*
+	 * First, assesss the current memory situation.  If free
+	 * memory is below the low watermark then try to start a
+	 * reaper to evict some pages.
+	 */
+	mpc_reap_tune(reap);
+
+	if (atomic_read(&reap->reap_lwm)) {
+		eidx = atomic_read(&reap->reap_eidx) % REAP_ELEM_MAX;
+		elem = reap->reap_elem + eidx;
+
+		if (!atomic_cmpxchg(&elem->reap_running, 0, 1))
+			queue_work(reap->reap_wq, &elem->reap_work);
+	}
+
+	/* Next, advance to the next elem and prune VMAs that have been freed. */
+	eidx = atomic_inc_return(&reap->reap_eidx) % REAP_ELEM_MAX;
+
+	elem = reap->reap_elem + eidx;
+	INIT_LIST_HEAD(&freeme);
+
+	nfreed = atomic_read(&elem->reap_nfreed);
+
+	if (nfreed && mutex_trylock(&elem->reap_lock)) {
+		struct list_head   *list = &elem->reap_list;
+		uint                npruned = 0;
+
+		list_for_each_entry_safe(xvm, next, list, xvm_list) {
+			if (atomic_read(&xvm->xvm_reapref) > 1)
+				continue;
+
+			list_del(&xvm->xvm_list);
+			list_add_tail(&xvm->xvm_list, &freeme);
+
+			if (++npruned >= nfreed)
+				break;
+		}
+		mutex_unlock(&elem->reap_lock);
+
+		list_for_each_entry_safe(xvm, next, &freeme, xvm_list)
+			mpc_xvm_free(xvm);
+
+		atomic_sub(npruned, &elem->reap_nfreed);
+	}
+
+	delay = reap->reap_mempct < 100 ? 1000 / REAP_ELEM_MAX : 1000;
+	delay = msecs_to_jiffies(delay);
+
+	queue_delayed_work(reap->reap_wq, &reap->reap_dwork, delay);
+}
+
+#define REAP_MEMPCT_MIN    5
+#define REAP_MEMPCT_MAX    100
+#define REAP_TTL_MIN       100
+#define REAP_DEBUG_MAX     3
+
+static ssize_t mpc_reap_mempct_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%d\n", dev_to_reap(dev)->reap_mempct);
+}
+
+static ssize_t mpc_reap_mempct_store(struct device *dev, struct device_attribute *da,
+				     const char *buf, size_t count)
+{
+	struct mpc_reap *reap;
+	unsigned int val;
+	int rc;
+
+	rc = kstrtouint(buf, 10, &val);
+	if (rc || (val < REAP_MEMPCT_MIN || val > REAP_MEMPCT_MAX))
+		return -EINVAL;
+
+	reap = dev_to_reap(dev);
+	reap->reap_mempct = val;
+
+	return count;
+}
+
+static ssize_t mpc_reap_debug_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%d\n", dev_to_reap(dev)->reap_debug);
+}
+
+static ssize_t mpc_reap_debug_store(struct device *dev, struct device_attribute *da,
+				    const char *buf, size_t count)
+{
+	struct mpc_reap *reap;
+	unsigned int val;
+	int rc;
+
+	rc = kstrtouint(buf, 10, &val);
+	if (rc || val > REAP_DEBUG_MAX)
+		return -EINVAL;
+
+	reap = dev_to_reap(dev);
+	reap->reap_debug = val;
+
+	return count;
+}
+
+static ssize_t mpc_reap_ttl_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%d\n", dev_to_reap(dev)->reap_ttl);
+}
+
+static ssize_t mpc_reap_ttl_store(struct device *dev, struct device_attribute *da,
+				  const char *buf, size_t count)
+{
+	struct mpc_reap *reap;
+	unsigned int val;
+	int rc;
+
+	rc = kstrtouint(buf, 10, &val);
+	if (rc || val < REAP_TTL_MIN)
+		return -EINVAL;
+
+	reap = dev_to_reap(dev);
+	reap->reap_ttl = val;
+
+	return count;
+}
+
+void mpc_reap_params_add(struct device_attribute *dattr)
+{
+	MPC_ATTR_RW(dattr++, reap_mempct);
+	MPC_ATTR_RW(dattr++, reap_debug);
+	MPC_ATTR_RW(dattr, reap_ttl);
+}
+
+static void mpc_reap_mempct_init(struct mpc_reap *reap)
+{
+	ulong mavail;
+	uint pct = 60;
+
+	mpc_reap_meminfo(NULL, &mavail, 30);
+
+	if (mavail > 256)
+		pct = 10;
+	else if (mavail > 128)
+		pct = 13;
+	else if (mavail > 64)
+		pct = 25;
+	else if (mavail > 32)
+		pct = 40;
+
+	reap->reap_mempct = clamp_t(unsigned int, 100 - pct, 1, 100);
+}
+
+/**
+ * mpc_reap_create() - Allocate and initialize reap data strctures
+ * @reapout: Initialized reap structure.
+ *
+ * Returns -ENOMEM if the allocation fails.
+ */
+int mpc_reap_create(struct mpc_reap **reapp)
+{
+	struct mpc_reap_elem *elem;
+	struct mpc_reap *reap;
+	uint flags, i;
+
+	flags = WQ_UNBOUND | WQ_HIGHPRI | WQ_CPU_INTENSIVE;
+	*reapp = NULL;
+
+	reap = kzalloc(roundup_pow_of_two(sizeof(*reap)), GFP_KERNEL);
+	if (!reap)
+		return -ENOMEM;
+
+	reap->reap_wq = alloc_workqueue("mpc_reap", flags, REAP_ELEM_MAX + 1);
+	if (!reap->reap_wq) {
+		kfree(reap);
+		return -ENOMEM;
+	}
+
+	atomic_set(&reap->reap_lwm, 0);
+	atomic_set(&reap->reap_ttl_cur, 0);
+	atomic_set(&reap->reap_eidx, 0);
+	atomic_set(&reap->reap_emit, 0);
+
+	for (i = 0; i < REAP_ELEM_MAX; ++i) {
+		elem = &reap->reap_elem[i];
+
+		mutex_init(&elem->reap_lock);
+		INIT_LIST_HEAD(&elem->reap_list);
+
+		INIT_WORK(&elem->reap_work, mpc_reap_run);
+		atomic_set(&elem->reap_running, 0);
+		elem->reap_reap = reap;
+
+		atomic64_set(&elem->reap_hpages, 0);
+		atomic64_set(&elem->reap_wpages, 0);
+		atomic64_set(&elem->reap_cpages, 0);
+		atomic_set(&elem->reap_nfreed, 0);
+	}
+
+	reap->reap_ttl   = 10 * 1000 * 1000;
+	reap->reap_debug = 0;
+	mpc_reap_mempct_init(reap);
+
+	INIT_DELAYED_WORK(&reap->reap_dwork, mpc_reap_prune);
+	queue_delayed_work(reap->reap_wq, &reap->reap_dwork, 1);
+
+	*reapp = reap;
+
+	return 0;
+}
+
+void mpc_reap_destroy(struct mpc_reap *reap)
+{
+	struct mpc_reap_elem *elem;
+	int i;
+
+	if (!reap)
+		return;
+
+	cancel_delayed_work_sync(&reap->reap_dwork);
+
+	/*
+	 * There shouldn't be any reapers running at this point,
+	 * but perform a flush/wait for good measure...
+	 */
+	atomic_set(&reap->reap_lwm, 0);
+
+	if (reap->reap_wq)
+		flush_workqueue(reap->reap_wq);
+
+	for (i = 0; i < REAP_ELEM_MAX; ++i) {
+		elem = &reap->reap_elem[i];
+
+		ASSERT(atomic64_read(&elem->reap_hpages) == 0);
+		ASSERT(atomic64_read(&elem->reap_wpages) == 0);
+		ASSERT(atomic64_read(&elem->reap_cpages) == 0);
+		ASSERT(atomic_read(&elem->reap_nfreed) == 0);
+		ASSERT(list_empty(&elem->reap_list));
+
+		mutex_destroy(&elem->reap_lock);
+	}
+
+	if (reap->reap_wq)
+		destroy_workqueue(reap->reap_wq);
+	kfree(reap);
+}
+
+void mpc_reap_xvm_add(struct mpc_reap *reap, struct mpc_xvm *xvm)
+{
+	struct mpc_reap_elem *elem;
+	uint idx;
+
+	if (!reap || !xvm)
+		return;
+
+	if (xvm->xvm_advice == MPC_VMA_PINNED)
+		return;
+
+	/* Acquire a reference on xvm for the reaper... */
+	atomic_inc(&xvm->xvm_reapref);
+	xvm->xvm_reap = reap;
+
+	idx = (get_cycles() >> 1) % REAP_ELEM_MAX;
+
+	elem = &reap->reap_elem[idx];
+
+	mutex_lock(&elem->reap_lock);
+	xvm->xvm_freedp = &elem->reap_nfreed;
+
+	if (xvm->xvm_advice == MPC_VMA_HOT)
+		xvm->xvm_hcpagesp = &elem->reap_hpages;
+	else if (xvm->xvm_advice == MPC_VMA_WARM)
+		xvm->xvm_hcpagesp = &elem->reap_wpages;
+	else
+		xvm->xvm_hcpagesp = &elem->reap_cpages;
+
+	list_add_tail(&xvm->xvm_list, &elem->reap_list);
+	mutex_unlock(&elem->reap_lock);
+}
+
+void mpc_reap_xvm_evict(struct mpc_xvm *xvm)
+{
+	pgoff_t start, end, bktsz;
+
+	if (atomic_cmpxchg(&xvm->xvm_evicting, 0, 1))
+		return;
+
+	start = mpc_xvm_pgoff(xvm);
+	end = mpc_xvm_pglen(xvm) + start;
+	bktsz = xvm->xvm_bktsz >> PAGE_SHIFT;
+
+	if (bktsz < 1024)
+		bktsz = end - start;
+
+	/* Evict in chunks to improve mmap_sem interleaving... */
+	for (; start < end; start += bktsz)
+		invalidate_inode_pages2_range(xvm->xvm_mapping, start, start + bktsz);
+
+	atomic_cmpxchg(&xvm->xvm_evicting, 1, 0);
+}
+
+void mpc_reap_xvm_touch(struct mpc_xvm *xvm, int index)
+{
+	struct mpc_reap *reap;
+	atomic64_t *atimep;
+	uint mbnum, lwm;
+	pgoff_t offset;
+	ulong delay;
+	u64 now;
+
+	reap = xvm->xvm_reap;
+	if (!reap)
+		return;
+
+	offset = (index << PAGE_SHIFT) % (1ul << xvm_size_max);
+	mbnum = offset / xvm->xvm_bktsz;
+
+	atimep = &xvm->xvm_mbinfov[mbnum].mbatime;
+	now = local_clock();
+
+	/*
+	 * Don't update atime too frequently.  If we set atime to
+	 * U64_MAX in mpc_reap_evict_vma() then the addition here
+	 * will roll over and atime will be updated.
+	 */
+	if (atomic64_read(atimep) + (10 * USEC_PER_SEC) < now)
+		atomic64_set(atimep, now);
+
+	/* Sleep a bit if the reaper is having trouble meeting the free memory target. */
+	lwm = atomic_read(&reap->reap_lwm);
+	if (lwm < 3333)
+		return;
+
+	delay = 500000 / (10001 - lwm) - (500000 / 10001);
+	delay = min_t(ulong, delay, 3000);
+
+	usleep_range(delay, delay * 2);
+}
+
+bool mpc_reap_xvm_duress(struct mpc_xvm *xvm)
+{
+	struct mpc_reap *reap;
+	uint lwm;
+
+	if (xvm->xvm_advice == MPC_VMA_HOT)
+		return false;
+
+	reap = xvm->xvm_reap;
+	if (!reap)
+		return false;
+
+	lwm = atomic_read(&reap->reap_lwm);
+	if (lwm < 1500)
+		return false;
+
+	if (lwm > 3000)
+		return true;
+
+	return (xvm->xvm_advice == MPC_VMA_COLD);
+}
diff --git a/drivers/mpool/reaper.h b/drivers/mpool/reaper.h
new file mode 100644
index 000000000000..d3af6aef918d
--- /dev/null
+++ b/drivers/mpool/reaper.h
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+#ifndef MPOOL_REAPER_H
+#define MPOOL_REAPER_H
+
+#define MPC_REAP_PARAMS_CNT    3
+
+struct mpc_reap;
+struct mpc_xvm;
+
+/**
+ * mpc_reap_create() - Allocate and initialize reap data strctures
+ * @reapp: Ptr to initialized reap structure.
+ *
+ * Return: -ENOMEM if the allocaiton fails.
+ */
+int mpc_reap_create(struct mpc_reap **reapp);
+
+/**
+ * mpc_reap_destroy() - Destroy the given reaper
+ * @reap:
+ */
+void mpc_reap_destroy(struct mpc_reap *reap);
+
+/**
+ * mpc_reap_xvm_add() - Add an extended VMA to the reap list
+ * @xvm: extended VMA
+ */
+void mpc_reap_xvm_add(struct mpc_reap *reap, struct mpc_xvm *xvm);
+
+/**
+ * mpc_reap_xvm_evict() - Evict all pages of given extended VMA
+ * @xvm: extended VMA
+ */
+void mpc_reap_xvm_evict(struct mpc_xvm *xvm);
+
+/**
+ * mpc_reap_xvm_touch() - Update extended VMA mblock atime
+ * @xvm:    extended VMA
+ * @index:  valid page index within the extended VMA
+ *
+ * Update the access time stamp of the mblock given by the valid
+ * page %index within the VMA.  Might sleep for some number of
+ * microseconds if the reaper is under duress (i.e., the more
+ * urgent the duress the longer the sleep).
+ *
+ * This function is called only by mpc_vm_fault_impl(), once
+ * for each successful page fault.
+ */
+void mpc_reap_xvm_touch(struct mpc_xvm *xvm, int index);
+
+/**
+ * mpc_reap_xvm_duress() - Check to see if reaper is under duress
+ * @xvm:   extended VMA
+ *
+ * Return: %false if the VMA is marked MPC_XVM_HOT.
+ * Return: %false if reaper is not enabled nor under duress.
+ * Return: %true depending upon the urgency of duress and the
+ * VMA advice (MPC_XVM_WARM or MPC_XVM_COLD).
+ *
+ * This function is called only by mpc_readpages() to decide whether
+ * or not to reduce the size of a speculative readahead request.
+ */
+bool mpc_reap_xvm_duress(struct mpc_xvm *xvm);
+
+void mpc_reap_params_add(struct device_attribute *dattr);
+
+#endif /* MPOOL_REAPER_H */
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:31 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:31 -0500
Subject: [PATCH v2 17/22] mpool: add mpool lifecycle management ioctls
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-18-nmeeramohide@micron.com>

This adds the open, release and mpool management ioctls for
the mpool driver.

The create, destroy, activate, deactivate and rename ioctls
are issued to the mpool control device (/dev/mpoolctl),
and the rest are issued to the mpool device
(/dev/mpool/<mpool_name>).

The mpool control device is owned by (root, disk) with
mode 0664. Non-default uid, gid and mode can be assigned to
an mpool device either at create time or post creation using
the params set ioctl.

Both the per-mpool and common parameters are available in
the sysfs device tree path created by the kernel for each mpool
device minor (/sys/devices/virtual/mpool). Mpool parameters
cannot be changed via the sysfs tree at this point.

The mpool management ioctl handlers invoke the mpool lifecycle
management routines to administer mpools. Activating an mpool
creates a unit object which stores some key information like
reference to the device object, reference to the mpc_mpool
instance containing the per-mpool private data, device props,
ownership and mode bits, device open count, flags etc. The
per-mpool parameters are persisted in MDC0 at activation.

Deactivating an mpool tears down the unit object and releases
all its associated resources.

An mpool can be renamed only when it's deactivated.  Renaming
an mpool updates the superblock on all its constituent storage
volumes with the new mpool name.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mpctl.c | 1560 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 1560 insertions(+)

diff --git a/drivers/mpool/mpctl.c b/drivers/mpool/mpctl.c
index 21eb7ac4610b..de62f9a5524d 100644
--- a/drivers/mpool/mpctl.c
+++ b/drivers/mpool/mpctl.c
@@ -80,6 +80,9 @@ static struct mpc_softstate mpc_softstate;
 static unsigned int mpc_ctl_uid __read_mostly;
 static unsigned int mpc_ctl_gid __read_mostly = 6;
 static unsigned int mpc_ctl_mode __read_mostly = 0664;
+static unsigned int mpc_default_uid __read_mostly;
+static unsigned int mpc_default_gid __read_mostly = 6;
+static unsigned int mpc_default_mode __read_mostly = 0660;
 
 static const struct mpc_uinfo mpc_uinfo_ctl = {
 	.ui_typename = "mpoolctl",
@@ -111,6 +114,202 @@ static inline gid_t mpc_current_gid(void)
 	return from_kgid(current_user_ns(), current_gid());
 }
 
+#define MPC_MPOOL_PARAMS_CNT     7
+
+static ssize_t mpc_uid_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%d\n", dev_to_unit(dev)->un_uid);
+}
+
+static ssize_t mpc_gid_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%d\n", dev_to_unit(dev)->un_gid);
+}
+
+static ssize_t mpc_mode_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "0%o\n", dev_to_unit(dev)->un_mode);
+}
+
+static ssize_t mpc_ra_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%u\n", dev_to_unit(dev)->un_ra_pages_max);
+}
+
+static ssize_t mpc_label_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	return scnprintf(buf, PAGE_SIZE, "%s\n", dev_to_unit(dev)->un_label);
+}
+
+static ssize_t mpc_type_show(struct device *dev, struct device_attribute *da, char *buf)
+{
+	struct mpool_uuid  uuid;
+	char               uuid_str[MPOOL_UUID_STRING_LEN + 1] = { };
+
+	memcpy(uuid.uuid, dev_to_unit(dev)->un_utype.b, MPOOL_UUID_SIZE);
+	mpool_unparse_uuid(&uuid, uuid_str);
+
+	return scnprintf(buf, PAGE_SIZE, "%s\n", uuid_str);
+}
+
+static void mpc_mpool_params_add(struct device_attribute *dattr)
+{
+	MPC_ATTR_RO(dattr++, uid);
+	MPC_ATTR_RO(dattr++, gid);
+	MPC_ATTR_RO(dattr++, mode);
+	MPC_ATTR_RO(dattr++, ra);
+	MPC_ATTR_RO(dattr++, label);
+	MPC_ATTR_RO(dattr,   type);
+}
+
+static int mpc_params_register(struct mpc_unit *unit, int cnt)
+{
+	struct device_attribute *dattr;
+	struct mpc_attr *attr;
+	int rc;
+
+	attr = mpc_attr_create(unit->un_device, "parameters", cnt);
+	if (!attr)
+		return -ENOMEM;
+
+	dattr = attr->a_dattr;
+
+	/* Per-mpool parameters */
+	if (mpc_unit_ismpooldev(unit))
+		mpc_mpool_params_add(dattr);
+
+	rc = mpc_attr_group_create(attr);
+	if (rc) {
+		mpc_attr_destroy(attr);
+		return rc;
+	}
+
+	unit->un_attr = attr;
+
+	return 0;
+}
+
+static void mpc_params_unregister(struct mpc_unit *unit)
+{
+	mpc_attr_group_destroy(unit->un_attr);
+	mpc_attr_destroy(unit->un_attr);
+	unit->un_attr = NULL;
+}
+
+/**
+ * mpc_toascii() - convert string to restricted ASCII
+ *
+ * Zeroes out the remainder of str[] and returns the length.
+ */
+static size_t mpc_toascii(char *str, size_t sz)
+{
+	size_t len = 0;
+	int i;
+
+	if (!str || sz < 1)
+		return 0;
+
+	if (str[0] == '-')
+		str[0] = '_';
+
+	for (i = 0; i < (sz - 1) && str[i]; ++i) {
+		if (isalnum(str[i]) || strchr("_.-", str[i]))
+			continue;
+
+		str[i] = '_';
+	}
+
+	len = i;
+
+	while (i < sz)
+		str[i++] = '\000';
+
+	return len;
+}
+
+static void mpool_params_merge_defaults(struct mpool_params *params)
+{
+	if (params->mp_spare_cap == MPOOL_SPARES_INVALID)
+		params->mp_spare_cap = MPOOL_SPARES_DEFAULT;
+
+	if (params->mp_spare_stg == MPOOL_SPARES_INVALID)
+		params->mp_spare_stg = MPOOL_SPARES_DEFAULT;
+
+	if (params->mp_ra_pages_max == U32_MAX)
+		params->mp_ra_pages_max = MPOOL_RA_PAGES_MAX;
+	params->mp_ra_pages_max = clamp_t(u32, params->mp_ra_pages_max, 0, MPOOL_RA_PAGES_MAX);
+
+	if (params->mp_mode != -1)
+		params->mp_mode &= 0777;
+
+	params->mp_rsvd0 = 0;
+	params->mp_rsvd1 = 0;
+	params->mp_rsvd2 = 0;
+	params->mp_rsvd3 = 0;
+	params->mp_rsvd4 = 0;
+
+	if (!strcmp(params->mp_label, MPOOL_LABEL_INVALID))
+		strcpy(params->mp_label, MPOOL_LABEL_DEFAULT);
+
+	mpc_toascii(params->mp_label, sizeof(params->mp_label));
+}
+
+static void mpool_to_mpcore_params(struct mpool_params *params, struct mpcore_params *mpc_params)
+{
+	u64 mdc0cap, mdcncap;
+	u32 mdcnum;
+
+	mpcore_params_defaults(mpc_params);
+
+	mdc0cap = (u64)params->mp_mdc0cap << 20;
+	mdcncap = (u64)params->mp_mdcncap << 20;
+	mdcnum  = params->mp_mdcnum;
+
+	if (mdc0cap != 0)
+		mpc_params->mp_mdc0cap = mdc0cap;
+
+	if (mdcncap != 0)
+		mpc_params->mp_mdcncap = mdcncap;
+
+	if (mdcnum != 0)
+		mpc_params->mp_mdcnum = mdcnum;
+}
+
+static bool mpool_params_merge_config(struct mpool_params *params, struct mpool_config *cfg)
+{
+	uuid_le uuidnull = { };
+	bool changed = false;
+
+	if (params->mp_uid != -1 && params->mp_uid != cfg->mc_uid) {
+		cfg->mc_uid = params->mp_uid;
+		changed = true;
+	}
+
+	if (params->mp_gid != -1 && params->mp_gid != cfg->mc_gid) {
+		cfg->mc_gid = params->mp_gid;
+		changed = true;
+	}
+
+	if (params->mp_mode != -1 && params->mp_mode != cfg->mc_mode) {
+		cfg->mc_mode = params->mp_mode;
+		changed = true;
+	}
+
+	if (memcmp(&uuidnull, &params->mp_utype, sizeof(uuidnull)) &&
+	    memcmp(&params->mp_utype, &cfg->mc_utype, sizeof(params->mp_utype))) {
+		memcpy(&cfg->mc_utype, &params->mp_utype, sizeof(cfg->mc_utype));
+		changed = true;
+	}
+
+	if (strcmp(params->mp_label, MPOOL_LABEL_DEFAULT) &&
+	    strncmp(params->mp_label, cfg->mc_label, sizeof(params->mp_label))) {
+		strlcpy(cfg->mc_label, params->mp_label, sizeof(cfg->mc_label));
+		changed = true;
+	}
+
+	return changed;
+}
+
 /**
  * mpc_mpool_release() - release kref handler for mpc_mpool object
  * @refp:  kref pointer
@@ -215,6 +414,9 @@ static void mpc_unit_release(struct kref *refp)
 	if (unit->un_mpool)
 		mpc_mpool_put(unit->un_mpool);
 
+	if (unit->un_attr)
+		mpc_params_unregister(unit);
+
 	if (unit->un_device)
 		device_destroy(ss->ss_class, unit->un_devno);
 
@@ -227,6 +429,89 @@ static void mpc_unit_put(struct mpc_unit *unit)
 		kref_put(&unit->un_ref, mpc_unit_release);
 }
 
+/**
+ * mpc_unit_lookup() - Look up a unit by minor number.
+ * @minor:  minor number
+ * @unitp:  unit ptr
+ *
+ * Returns a referenced ptr to the unit (via *unitp) if found,
+ * otherwise it sets *unitp to NULL.
+ */
+static void mpc_unit_lookup(int minor, struct mpc_unit **unitp)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpc_unit *unit;
+
+	*unitp = NULL;
+
+	mutex_lock(&ss->ss_lock);
+	unit = idr_find(&ss->ss_unitmap, minor);
+	if (unit) {
+		kref_get(&unit->un_ref);
+		*unitp = unit;
+	}
+	mutex_unlock(&ss->ss_lock);
+}
+
+/**
+ * mpc_unit_lookup_by_name_itercb() - Test to see if unit matches arg.
+ * @item:   unit ptr
+ * @arg:    argument vector base ptr
+ *
+ * This iterator callback is called by mpc_unit_lookup_by_name()
+ * for each unit in the units table.
+ *
+ * Return: If the unit matching the given name is found returns
+ * the referenced unit pointer in argv[2], otherwise NULL.
+ */
+static int mpc_unit_lookup_by_name_itercb(int minor, void *item, void *arg)
+{
+	struct mpc_unit *unit = item;
+	void **argv = arg;
+	struct mpc_unit *parent = argv[0];
+	const char *name = argv[1];
+
+	if (!unit)
+		return ITERCB_NEXT;
+
+	if (mpc_unit_isctldev(parent) && !mpc_unit_ismpooldev(unit))
+		return ITERCB_NEXT;
+
+	if (parent->un_mpool && unit->un_mpool != parent->un_mpool)
+		return ITERCB_NEXT;
+
+	if (strcmp(unit->un_name, name) == 0) {
+		kref_get(&unit->un_ref);
+		argv[2] = unit;
+		return ITERCB_DONE;
+	}
+
+	return ITERCB_NEXT;
+}
+
+/**
+ * mpc_unit_lookup_by_name() - Look up an mpool unit by name.
+ * @parent: parent unit
+ * @name:   unit name. This is not the mpool name.
+ * @unitp:  unit ptr
+ *
+ * If a unit exists in the system which has the given name and parent
+ * then it is referenced and returned via *unitp.  Otherwise, *unitp
+ * is set to NULL.
+ */
+static void mpc_unit_lookup_by_name(struct mpc_unit *parent, const char *name,
+				    struct mpc_unit **unitp)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	void *argv[] = { parent, (void *)name, NULL };
+
+	mutex_lock(&ss->ss_lock);
+	idr_for_each(&ss->ss_unitmap, mpc_unit_lookup_by_name_itercb, argv);
+	mutex_unlock(&ss->ss_lock);
+
+	*unitp = argv[2];
+}
+
 /**
  * mpc_unit_setup() - Create a device unit object and special file
  * @uinfo:
@@ -327,6 +612,36 @@ static int mpc_unit_setup(const struct mpc_uinfo *uinfo, const char *name,
 	return rc;
 }
 
+
+static int mpc_cf_journal(struct mpc_unit *unit)
+{
+	struct mpool_config cfg = { };
+	struct mpc_mpool *mpool;
+	int rc;
+
+	mpool = unit->un_mpool;
+	if (!mpool)
+		return -EINVAL;
+
+	down_write(&mpool->mp_lock);
+
+	cfg.mc_uid = unit->un_uid;
+	cfg.mc_gid = unit->un_gid;
+	cfg.mc_mode = unit->un_mode;
+	cfg.mc_oid1 = unit->un_ds_oidv[0];
+	cfg.mc_oid2 = unit->un_ds_oidv[1];
+	cfg.mc_captgt = unit->un_mdc_captgt;
+	cfg.mc_ra_pages_max = unit->un_ra_pages_max;
+	memcpy(&cfg.mc_utype, &unit->un_utype, sizeof(cfg.mc_utype));
+	strlcpy(cfg.mc_label, unit->un_label, sizeof(cfg.mc_label));
+
+	rc = mpool_config_store(mpool->mp_desc, &cfg);
+
+	up_write(&mpool->mp_lock);
+
+	return rc;
+}
+
 /**
  * mpc_uevent() - Hook to intercept and modify uevents before they're posted to udev
  * @dev:    mpc driver device
@@ -347,6 +662,1251 @@ static int mpc_uevent(struct device *dev, struct kobj_uevent_env *env)
 	return 0;
 }
 
+/**
+ * mpc_mp_chown() - Change ownership of an mpool.
+ * @unit: mpool unit ptr
+ * @mps:
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpc_mp_chown(struct mpc_unit *unit, struct mpool_params *params)
+{
+	mode_t mode;
+	uid_t uid;
+	gid_t gid;
+	int rc = 0;
+
+	if (!mpc_unit_ismpooldev(unit))
+		return -EINVAL;
+
+	uid  = params->mp_uid;
+	gid  = params->mp_gid;
+	mode = params->mp_mode;
+
+	if (mode != -1)
+		mode &= 0777;
+
+	if (uid != -1 && uid != unit->un_uid && !capable(CAP_CHOWN))
+		return -EPERM;
+
+	if (gid != -1 && gid != unit->un_gid && !capable(CAP_CHOWN))
+		return -EPERM;
+
+	if (mode != -1 && mode != unit->un_mode && !capable(CAP_FOWNER))
+		return -EPERM;
+
+	if (-1 != uid)
+		unit->un_uid = uid;
+	if (-1 != gid)
+		unit->un_gid = gid;
+	if (-1 != mode)
+		unit->un_mode = mode;
+
+	if (uid != -1 || gid != -1 || mode != -1)
+		rc = kobject_uevent(&unit->un_device->kobj, KOBJ_CHANGE);
+
+	return rc;
+}
+
+/**
+ * mpioc_params_get() - get parameters of an activated mpool
+ * @unit:   mpool unit ptr
+ * @get:    mpool params
+ *
+ * MPIOC_PARAMS_GET ioctl handler to get mpool parameters
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_params_get(struct mpc_unit *unit, struct mpioc_params *get)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpool_descriptor *desc;
+	struct mpool_params *params;
+	struct mpool_xprops xprops = { };
+	u8 mclass;
+
+	if (!mpc_unit_ismpooldev(unit))
+		return -EINVAL;
+
+	desc = unit->un_mpool->mp_desc;
+
+	mutex_lock(&ss->ss_lock);
+
+	params = &get->mps_params;
+	memset(params, 0, sizeof(*params));
+	params->mp_uid = unit->un_uid;
+	params->mp_gid = unit->un_gid;
+	params->mp_mode = unit->un_mode;
+	params->mp_mdc_captgt = MPOOL_ROOT_LOG_CAP;
+	params->mp_oidv[0] = unit->un_ds_oidv[0];
+	params->mp_oidv[1] = unit->un_ds_oidv[1];
+	params->mp_ra_pages_max = unit->un_ra_pages_max;
+	memcpy(&params->mp_utype, &unit->un_utype, sizeof(params->mp_utype));
+	strlcpy(params->mp_label, unit->un_label, sizeof(params->mp_label));
+	strlcpy(params->mp_name, unit->un_name, sizeof(params->mp_name));
+
+	/* Get mpool properties.. */
+	mpool_get_xprops(desc, &xprops);
+
+	for (mclass = 0; mclass < MP_MED_NUMBER; mclass++)
+		params->mp_mblocksz[mclass] = xprops.ppx_params.mp_mblocksz[mclass];
+
+	params->mp_spare_cap = xprops.ppx_drive_spares[MP_MED_CAPACITY];
+	params->mp_spare_stg = xprops.ppx_drive_spares[MP_MED_STAGING];
+
+	memcpy(params->mp_poolid.b, xprops.ppx_params.mp_poolid.b, MPOOL_UUID_SIZE);
+
+	mutex_unlock(&ss->ss_lock);
+
+	return 0;
+}
+
+/**
+ * mpioc_params_set() - set parameters of an activated mpool
+ * @unit:   mpool unit ptr
+ * @set:    mpool params
+ *
+ * MPIOC_PARAMS_SET ioctl handler to set mpool parameters
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_params_set(struct mpc_unit *unit, struct mpioc_params *set)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpool_descriptor *mp;
+	struct mpool_params *params;
+	uuid_le uuidnull = { };
+	int rerr = 0, err = 0;
+	bool journal = false;
+
+	if (!mpc_unit_ismpooldev(unit))
+		return -EINVAL;
+
+	params = &set->mps_params;
+
+	mutex_lock(&ss->ss_lock);
+	if (params->mp_uid != -1 || params->mp_gid != -1 || params->mp_mode != -1) {
+		err = mpc_mp_chown(unit, params);
+		if (err) {
+			mutex_unlock(&ss->ss_lock);
+			return err;
+		}
+		journal = true;
+	}
+
+	if (params->mp_label[0]) {
+		mpc_toascii(params->mp_label, sizeof(params->mp_label));
+		strlcpy(unit->un_label, params->mp_label, sizeof(unit->un_label));
+		journal = true;
+	}
+
+	if (memcmp(&uuidnull, &params->mp_utype, sizeof(uuidnull))) {
+		memcpy(&unit->un_utype, &params->mp_utype, sizeof(unit->un_utype));
+		journal = true;
+	}
+
+	if (params->mp_ra_pages_max != U32_MAX) {
+		unit->un_ra_pages_max = clamp_t(u32, params->mp_ra_pages_max,
+						0, MPOOL_RA_PAGES_MAX);
+		journal = true;
+	}
+
+	if (journal)
+		err = mpc_cf_journal(unit);
+	mutex_unlock(&ss->ss_lock);
+
+	if (err) {
+		mp_pr_err("%s: params commit failed", err, unit->un_name);
+		return err;
+	}
+
+	mp = unit->un_mpool->mp_desc;
+
+	if (params->mp_spare_cap != MPOOL_SPARES_INVALID) {
+		err = mpool_drive_spares(mp, MP_MED_CAPACITY, params->mp_spare_cap);
+		if (err && err != -ENOENT)
+			rerr = err;
+	}
+
+	if (params->mp_spare_stg != MPOOL_SPARES_INVALID) {
+		err = mpool_drive_spares(mp, MP_MED_STAGING, params->mp_spare_stg);
+		if (err && err != -ENOENT)
+			rerr = err;
+	}
+
+	return rerr;
+}
+
+/**
+ * mpioc_mp_mclass_get() - get information regarding an mpool's mclasses
+ * @unit:   mpool unit ptr
+ * @mcl:    mclass info struct
+ *
+ * MPIOC_MP_MCLASS_GET ioctl handler to get mclass information
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_mp_mclass_get(struct mpc_unit *unit, struct mpioc_mclass *mcl)
+{
+	struct mpool_descriptor *desc = unit->un_mpool->mp_desc;
+	struct mpool_mclass_xprops mcxv[MP_MED_NUMBER];
+	uint32_t mcxc = ARRAY_SIZE(mcxv);
+	int rc;
+
+	if (!mcl || !desc)
+		return -EINVAL;
+
+	if (!mcl->mcl_xprops) {
+		mpool_mclass_get_cnt(desc, &mcl->mcl_cnt);
+		return 0;
+	}
+
+	memset(mcxv, 0, sizeof(mcxv));
+
+	rc = mpool_mclass_get(desc, &mcxc, mcxv);
+	if (rc)
+		return rc;
+
+	if (mcxc > mcl->mcl_cnt)
+		mcxc = mcl->mcl_cnt;
+	mcl->mcl_cnt = mcxc;
+
+	rc = copy_to_user(mcl->mcl_xprops, mcxv, sizeof(mcxv[0]) * mcxc);
+
+	return rc ? -EFAULT : 0;
+}
+
+/**
+ * mpioc_devprops_get() - Get device properties
+ * @unit:   mpool unit ptr
+ *
+ * MPIOC_PROP_GET ioctl handler to retrieve properties for the specified device.
+ */
+static int mpioc_devprops_get(struct mpc_unit *unit, struct mpioc_devprops *devprops)
+{
+	int rc = 0;
+
+	if (unit->un_mpool) {
+		struct mpool_descriptor *mp = unit->un_mpool->mp_desc;
+
+		rc = mpool_get_devprops_by_name(mp, devprops->dpr_pdname, &devprops->dpr_devprops);
+	}
+
+	return rc;
+}
+
+/**
+ * mpioc_prop_get() - Get mpool properties.
+ * @unit:   mpool unit ptr
+ *
+ * MPIOC_PROP_GET ioctl handler to retrieve properties for the specified device.
+ */
+static void mpioc_prop_get(struct mpc_unit *unit, struct mpioc_prop *kprop)
+{
+	struct mpool_descriptor *desc = unit->un_mpool->mp_desc;
+	struct mpool_params *params;
+	struct mpool_xprops *xprops;
+
+	memset(kprop, 0, sizeof(*kprop));
+
+	/* Get unit properties.. */
+	params = &kprop->pr_xprops.ppx_params;
+	params->mp_uid = unit->un_uid;
+	params->mp_gid = unit->un_gid;
+	params->mp_mode = unit->un_mode;
+	params->mp_mdc_captgt = unit->un_mdc_captgt;
+	params->mp_oidv[0] = unit->un_ds_oidv[0];
+	params->mp_oidv[1] = unit->un_ds_oidv[1];
+	params->mp_ra_pages_max = unit->un_ra_pages_max;
+	memcpy(&params->mp_utype, &unit->un_utype, sizeof(params->mp_utype));
+	strlcpy(params->mp_label, unit->un_label, sizeof(params->mp_label));
+	strlcpy(params->mp_name, unit->un_name, sizeof(params->mp_name));
+
+	/* Get mpool properties.. */
+	xprops = &kprop->pr_xprops;
+	mpool_get_xprops(desc, xprops);
+	mpool_get_usage(desc, MP_MED_ALL, &kprop->pr_usage);
+
+	params->mp_spare_cap = xprops->ppx_drive_spares[MP_MED_CAPACITY];
+	params->mp_spare_stg = xprops->ppx_drive_spares[MP_MED_STAGING];
+
+	kprop->pr_mcxc = ARRAY_SIZE(kprop->pr_mcxv);
+	mpool_mclass_get(desc, &kprop->pr_mcxc, kprop->pr_mcxv);
+}
+
+/**
+ * mpioc_proplist_get_itercb() - Get properties iterator callback.
+ * @item:   unit ptr
+ * @arg:    argument list
+ *
+ * Return: Returns properties for each unit matching the input criteria.
+ */
+static int mpioc_proplist_get_itercb(int minor, void *item, void *arg)
+{
+	struct mpc_unit *unit = item;
+	struct mpioc_prop __user *uprop;
+	struct mpioc_prop kprop;
+	struct mpc_unit *match;
+	struct mpioc_list *ls;
+	void **argv = arg;
+	int *cntp, rc;
+	int *errp;
+
+	if (!unit)
+		return ITERCB_NEXT;
+
+	match = argv[0];
+	ls = argv[1];
+
+	if (mpc_unit_isctldev(match) && !mpc_unit_ismpooldev(unit) &&
+	    ls->ls_cmd != MPIOC_LIST_CMD_PROP_GET)
+		return ITERCB_NEXT;
+
+	if (mpc_unit_ismpooldev(match) && !mpc_unit_ismpooldev(unit) &&
+	    ls->ls_cmd != MPIOC_LIST_CMD_PROP_GET)
+		return ITERCB_NEXT;
+
+	if (mpc_unit_ismpooldev(match) && unit->un_mpool != match->un_mpool)
+		return ITERCB_NEXT;
+
+	cntp = argv[2];
+	errp = argv[3];
+
+	mpioc_prop_get(unit, &kprop);
+
+	uprop = (struct mpioc_prop __user *)ls->ls_listv + *cntp;
+
+	rc = copy_to_user(uprop, &kprop, sizeof(*uprop));
+	if (rc) {
+		*errp = -EFAULT;
+		return ITERCB_DONE;
+	}
+
+	return (++(*cntp) >= ls->ls_listc) ? ITERCB_DONE : ITERCB_NEXT;
+}
+
+/**
+ * mpioc_proplist_get() - Get mpool properties.
+ * @unit:   mpool unit ptr
+ * @ls:     properties parameter block
+ *
+ * MPIOC_PROP_GET ioctl handler to retrieve properties for one
+ * or more mpools.
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_proplist_get(struct mpc_unit *unit, struct mpioc_list *ls)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	int err = 0;
+	int cnt = 0;
+	void *argv[] = { unit, ls, &cnt, &err };
+
+	if (!ls || ls->ls_listc < 1 || ls->ls_cmd == MPIOC_LIST_CMD_INVALID)
+		return -EINVAL;
+
+	mutex_lock(&ss->ss_lock);
+	idr_for_each(&ss->ss_unitmap, mpioc_proplist_get_itercb, argv);
+	mutex_unlock(&ss->ss_lock);
+
+	ls->ls_listc = cnt;
+
+	return err;
+}
+
+/**
+ * mpc_mpool_open() - Open the mpool specified by the given drive paths,
+ *                    and then create an mpool object to track the
+ *                    underlying mpool.
+ * @dpathc: drive count
+ * @dpathv: drive path name vector
+ * @mpoolp: mpool ptr. Set only if success.
+ * @pd_prop: PDs properties
+ *
+ * Return:  Returns 0 if successful and sets *mpoolp.
+ *          Returns -errno on error.
+ */
+static int mpc_mpool_open(uint dpathc, char **dpathv, struct mpc_mpool **mpoolp,
+			  struct pd_prop *pd_prop, struct mpool_params *params, u32 flags)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpcore_params mpc_params;
+	struct mpc_mpool *mpool;
+	size_t mpoolsz, len;
+	int rc;
+
+	if (!ss || !dpathv || !mpoolp || !params)
+		return -EINVAL;
+
+	len = mpc_toascii(params->mp_name, sizeof(params->mp_name));
+	if (len < 1 || len >= MPOOL_NAMESZ_MAX)
+		return (len < 1) ? -EINVAL : -ENAMETOOLONG;
+
+	mpoolsz = sizeof(*mpool) + len + 1;
+
+	mpool = kzalloc(mpoolsz, GFP_KERNEL);
+	if (!mpool)
+		return -ENOMEM;
+
+	if (!try_module_get(THIS_MODULE)) {
+		kfree(mpool);
+		return -EBUSY;
+	}
+
+	mpool_to_mpcore_params(params, &mpc_params);
+
+	rc = mpool_activate(dpathc, dpathv, pd_prop, MPOOL_ROOT_LOG_CAP,
+			    &mpc_params, flags, &mpool->mp_desc);
+	if (rc) {
+		mp_pr_err("Activating %s failed", rc, params->mp_name);
+		module_put(THIS_MODULE);
+		kfree(mpool);
+		return rc;
+	}
+
+	kref_init(&mpool->mp_ref);
+	init_rwsem(&mpool->mp_lock);
+	mpool->mp_dpathc = dpathc;
+	mpool->mp_dpathv = dpathv;
+	strcpy(mpool->mp_name, params->mp_name);
+
+	*mpoolp = mpool;
+
+	return 0;
+}
+
+/**
+ * mpioc_mp_create() - create an mpool.
+ * @mp:      mpool parameter block
+ * @pd_prop:
+ * @dpathv:
+ *
+ * MPIOC_MP_CREATE ioctl handler to create an mpool.
+ *
+ * Return:  Returns 0 if the mpool is created, -errno otherwise...
+ */
+static int mpioc_mp_create(struct mpc_unit *ctl, struct mpioc_mpool *mp,
+			   struct pd_prop *pd_prop, char ***dpathv)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpcore_params mpc_params;
+	struct mpool_config cfg = { };
+	struct mpc_mpool *mpool = NULL;
+	struct mpc_unit *unit = NULL;
+	size_t len;
+	mode_t mode;
+	uid_t uid;
+	gid_t gid;
+	int rc;
+
+	if (!ctl || !mp || !pd_prop || !dpathv)
+		return -EINVAL;
+
+	len = mpc_toascii(mp->mp_params.mp_name, sizeof(mp->mp_params.mp_name));
+	if (len < 1 || len >= MPOOL_NAMESZ_MAX)
+		return (len < 1) ? -EINVAL : -ENAMETOOLONG;
+
+	mpool_params_merge_defaults(&mp->mp_params);
+
+	uid  = mp->mp_params.mp_uid;
+	gid  = mp->mp_params.mp_gid;
+	mode = mp->mp_params.mp_mode;
+
+	if (uid == -1)
+		uid = mpc_default_uid;
+	if (gid == -1)
+		gid = mpc_default_gid;
+	if (mode == -1)
+		mode = mpc_default_mode;
+
+	mode &= 0777;
+
+	if (uid != mpc_current_uid() && !capable(CAP_CHOWN)) {
+		rc = -EPERM;
+		mp_pr_err("chown permission denied, uid %d", rc, uid);
+		return rc;
+	}
+
+	if (gid != mpc_current_gid() && !capable(CAP_CHOWN)) {
+		rc = -EPERM;
+		mp_pr_err("chown permission denied, gid %d", rc, gid);
+		return rc;
+	}
+
+	if (!capable(CAP_SYS_ADMIN)) {
+		rc = -EPERM;
+		mp_pr_err("chmod/activate permission denied", rc);
+		return rc;
+	}
+
+	mpool_to_mpcore_params(&mp->mp_params, &mpc_params);
+
+	rc = mpool_create(mp->mp_params.mp_name, mp->mp_flags, *dpathv,
+			  pd_prop, &mpc_params, MPOOL_ROOT_LOG_CAP);
+	if (rc) {
+		mp_pr_err("%s: create failed", rc, mp->mp_params.mp_name);
+		return rc;
+	}
+
+	/*
+	 * Create an mpc_mpool object through which we can (re)open and manage
+	 * the mpool.  If successful, mpc_mpool_open() adopts dpathv.
+	 */
+	mpool_params_merge_defaults(&mp->mp_params);
+
+	rc = mpc_mpool_open(mp->mp_dpathc, *dpathv, &mpool, pd_prop, &mp->mp_params, mp->mp_flags);
+	if (rc) {
+		mp_pr_err("%s: mpc_mpool_open failed", rc, mp->mp_params.mp_name);
+		mpool_destroy(mp->mp_dpathc, *dpathv, pd_prop, mp->mp_flags);
+		return rc;
+	}
+
+	*dpathv = NULL;
+
+	mlog_lookup_rootids(&cfg.mc_oid1, &cfg.mc_oid2);
+	cfg.mc_uid = uid;
+	cfg.mc_gid = gid;
+	cfg.mc_mode = mode;
+	cfg.mc_rsvd0 = mp->mp_params.mp_rsvd0;
+	cfg.mc_captgt = MPOOL_ROOT_LOG_CAP;
+	cfg.mc_ra_pages_max = mp->mp_params.mp_ra_pages_max;
+	cfg.mc_rsvd1 = mp->mp_params.mp_rsvd1;
+	cfg.mc_rsvd2 = mp->mp_params.mp_rsvd2;
+	cfg.mc_rsvd3 = mp->mp_params.mp_rsvd3;
+	cfg.mc_rsvd4 = mp->mp_params.mp_rsvd4;
+	memcpy(&cfg.mc_utype, &mp->mp_params.mp_utype, sizeof(cfg.mc_utype));
+	strlcpy(cfg.mc_label, mp->mp_params.mp_label, sizeof(cfg.mc_label));
+
+	rc = mpool_config_store(mpool->mp_desc, &cfg);
+	if (rc) {
+		mp_pr_err("%s: config store failed", rc, mp->mp_params.mp_name);
+		goto errout;
+	}
+
+	/* A unit is born with two references:  A birth reference, and one for the caller. */
+	rc = mpc_unit_setup(&mpc_uinfo_mpool, mp->mp_params.mp_name,
+			    &cfg, mpool, &unit);
+	if (rc) {
+		mp_pr_err("%s: unit setup failed", rc, mp->mp_params.mp_name);
+		goto errout;
+	}
+
+	/* Return resolved params to caller. */
+	mp->mp_params.mp_uid = uid;
+	mp->mp_params.mp_gid = gid;
+	mp->mp_params.mp_mode = mode;
+	mp->mp_params.mp_mdc_captgt = cfg.mc_captgt;
+	mp->mp_params.mp_oidv[0] = cfg.mc_oid1;
+	mp->mp_params.mp_oidv[1] = cfg.mc_oid2;
+
+	rc = mpc_params_register(unit, MPC_MPOOL_PARAMS_CNT);
+	if (rc) {
+		mpc_unit_put(unit); /* drop birth ref */
+		goto errout;
+	}
+
+	mutex_lock(&ss->ss_lock);
+	idr_replace(&ss->ss_unitmap, unit, MINOR(unit->un_devno));
+	mutex_unlock(&ss->ss_lock);
+
+	mpool = NULL;
+
+errout:
+	if (mpool) {
+		mpool_deactivate(mpool->mp_desc);
+		mpool->mp_desc = NULL;
+		mpool_destroy(mp->mp_dpathc, mpool->mp_dpathv, pd_prop, mp->mp_flags);
+	}
+
+	/*
+	 * For failures after mpc_unit_setup() (i.e., mpool != NULL)
+	 * dropping the final unit ref will release the mpool ref.
+	 */
+	if (unit)
+		mpc_unit_put(unit); /* Drop caller's ref */
+	else if (mpool)
+		mpc_mpool_put(mpool);
+
+	return rc;
+}
+
+/**
+ * mpioc_mp_activate() - activate an mpool.
+ * @mp:      mpool parameter block
+ * @pd_prop:
+ * @dpathv:
+ *
+ * MPIOC_MP_ACTIVATE ioctl handler to activate an mpool.
+ *
+ * Return:  Returns 0 if the mpool is activated, -errno otherwise...
+ */
+static int mpioc_mp_activate(struct mpc_unit *ctl, struct mpioc_mpool *mp,
+			     struct pd_prop *pd_prop, char ***dpathv)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpool_config cfg;
+	struct mpc_mpool *mpool = NULL;
+	struct mpc_unit *unit = NULL;
+	size_t len;
+	int rc;
+
+	if (!capable(CAP_SYS_ADMIN))
+		return -EPERM;
+
+	if (!ctl || !mp || !pd_prop || !dpathv)
+		return -EINVAL;
+
+	len = mpc_toascii(mp->mp_params.mp_name, sizeof(mp->mp_params.mp_name));
+	if (len < 1 || len >= MPOOL_NAMESZ_MAX)
+		return (len < 1) ? -EINVAL : -ENAMETOOLONG;
+
+	mpool_params_merge_defaults(&mp->mp_params);
+
+	/*
+	 * Create an mpc_mpool object through which we can (re)open and manage
+	 * the mpool.  If successful, mpc_mpool_open() adopts dpathv.
+	 */
+	rc = mpc_mpool_open(mp->mp_dpathc, *dpathv, &mpool, pd_prop, &mp->mp_params, mp->mp_flags);
+	if (rc) {
+		mp_pr_err("%s: mpc_mpool_open failed", rc, mp->mp_params.mp_name);
+		return rc;
+	}
+
+	*dpathv = NULL; /* Was adopted by successful mpc_mpool_open() */
+
+	rc = mpool_config_fetch(mpool->mp_desc, &cfg);
+	if (rc) {
+		mp_pr_err("%s config fetch failed", rc, mp->mp_params.mp_name);
+		goto errout;
+	}
+
+	if (mpool_params_merge_config(&mp->mp_params, &cfg))
+		mpool_config_store(mpool->mp_desc, &cfg);
+
+	/* A unit is born with two references:  A birth reference, and one for the caller. */
+	rc = mpc_unit_setup(&mpc_uinfo_mpool, mp->mp_params.mp_name,
+			    &cfg, mpool, &unit);
+	if (rc) {
+		mp_pr_err("%s unit setup failed", rc, mp->mp_params.mp_name);
+		goto errout;
+	}
+
+	/* Return resolved params to caller. */
+	mp->mp_params.mp_uid = cfg.mc_uid;
+	mp->mp_params.mp_gid = cfg.mc_gid;
+	mp->mp_params.mp_mode = cfg.mc_mode;
+	mp->mp_params.mp_mdc_captgt = cfg.mc_captgt;
+	mp->mp_params.mp_oidv[0] = cfg.mc_oid1;
+	mp->mp_params.mp_oidv[1] = cfg.mc_oid2;
+	mp->mp_params.mp_ra_pages_max = cfg.mc_ra_pages_max;
+	mp->mp_params.mp_vma_size_max = cfg.mc_vma_size_max;
+	memcpy(&mp->mp_params.mp_utype, &cfg.mc_utype, sizeof(mp->mp_params.mp_utype));
+	strlcpy(mp->mp_params.mp_label, cfg.mc_label, sizeof(mp->mp_params.mp_label));
+
+	rc = mpc_params_register(unit, MPC_MPOOL_PARAMS_CNT);
+	if (rc) {
+		mpc_unit_put(unit); /* drop birth ref */
+		goto errout;
+	}
+
+	mutex_lock(&ss->ss_lock);
+	idr_replace(&ss->ss_unitmap, unit, MINOR(unit->un_devno));
+	mutex_unlock(&ss->ss_lock);
+
+	mpool = NULL;
+
+errout:
+	/*
+	 * For failures after mpc_unit_setup() (i.e., mpool != NULL)
+	 * dropping the final unit ref will release the mpool ref.
+	 */
+	if (unit)
+		mpc_unit_put(unit); /* drop caller's ref */
+	else if (mpool)
+		mpc_mpool_put(mpool);
+
+	return rc;
+}
+
+/**
+ * mpioc_mp_deactivate_impl() - deactivate an mpool.
+ * @unit:   control device unit ptr
+ * @mp:     mpool parameter block
+ *
+ * MPIOC_MP_DEACTIVATE ioctl handler to deactivate an mpool.
+ */
+static int mp_deactivate_impl(struct mpc_unit *ctl, struct mpioc_mpool *mp, bool locked)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpc_unit *unit = NULL;
+	size_t len;
+	int rc;
+
+	if (!ctl || !mp)
+		return -EINVAL;
+
+	if (!mpc_unit_isctldev(ctl))
+		return -ENOTTY;
+
+	len = mpc_toascii(mp->mp_params.mp_name, sizeof(mp->mp_params.mp_name));
+	if (len < 1 || len >= MPOOL_NAMESZ_MAX)
+		return (len < 1) ? -EINVAL : -ENAMETOOLONG;
+
+	if (!locked) {
+		rc = down_interruptible(&ss->ss_op_sema);
+		if (rc)
+			return rc;
+	}
+
+	mpc_unit_lookup_by_name(ctl, mp->mp_params.mp_name, &unit);
+	if (!unit) {
+		rc = -ENXIO;
+		goto errout;
+	}
+
+	/*
+	 * In order to be determined idle, a unit shall not be open
+	 * and shall have a ref count of exactly two (the birth ref
+	 * and the lookup ref from above).
+	 */
+	mutex_lock(&ss->ss_lock);
+	if (unit->un_open_cnt > 0 || kref_read(&unit->un_ref) != 2) {
+		rc = -EBUSY;
+		mp_pr_err("%s: busy, cannot deactivate", rc, unit->un_name);
+	} else {
+		idr_replace(&ss->ss_unitmap, NULL, MINOR(unit->un_devno));
+		rc = 0;
+	}
+	mutex_unlock(&ss->ss_lock);
+
+	if (!rc)
+		mpc_unit_put(unit); /* drop birth ref */
+
+	mpc_unit_put(unit); /* drop lookup ref */
+
+errout:
+	if (!locked)
+		up(&ss->ss_op_sema);
+
+	return rc;
+}
+
+static int mpioc_mp_deactivate(struct mpc_unit *ctl, struct mpioc_mpool *mp)
+{
+	return mp_deactivate_impl(ctl, mp, false);
+}
+
+static int mpioc_mp_cmd(struct mpc_unit *ctl, uint cmd, struct mpioc_mpool *mp)
+{
+	struct mpc_softstate *ss = &mpc_softstate;
+	struct mpc_unit *unit = NULL;
+	struct pd_prop *pd_prop = NULL;
+	char **dpathv = NULL, *dpaths;
+	size_t dpathvsz, pd_prop_sz;
+	const char *action;
+	size_t len;
+	int rc, i;
+
+	if (!ctl || !mp)
+		return -EINVAL;
+
+	if (!mpc_unit_isctldev(ctl))
+		return -EOPNOTSUPP;
+
+	if (mp->mp_dpathc < 1 || mp->mp_dpathc > MPOOL_DRIVES_MAX)
+		return -EDOM;
+
+	len = mpc_toascii(mp->mp_params.mp_name, sizeof(mp->mp_params.mp_name));
+	if (len < 1 || len >= MPOOL_NAMESZ_MAX)
+		return (len < 1) ? -EINVAL : -ENAMETOOLONG;
+
+	switch (cmd) {
+	case MPIOC_MP_CREATE:
+		action = "create";
+		break;
+
+	case MPIOC_MP_DESTROY:
+		action = "destroy";
+		break;
+
+	case MPIOC_MP_ACTIVATE:
+		action = "activate";
+		break;
+
+	case MPIOC_MP_RENAME:
+		action = "rename";
+		break;
+
+	default:
+		return -EINVAL;
+	}
+
+	if (!mp->mp_pd_prop || !mp->mp_dpaths) {
+		rc = -EINVAL;
+		mp_pr_err("%s: %s, (%d drives), drives names %p or PD props %p invalid",
+			  rc, mp->mp_params.mp_name, action, mp->mp_dpathc,
+			  mp->mp_dpaths, mp->mp_pd_prop);
+
+		return rc;
+	}
+
+	if (mp->mp_dpathssz > (mp->mp_dpathc + 1) * PATH_MAX)
+		return -EINVAL;
+
+	rc = down_interruptible(&ss->ss_op_sema);
+	if (rc)
+		return rc;
+
+	/*
+	 * If mpc_unit_lookup_by_name() succeeds it will have acquired
+	 * a reference on unit.  We release that reference at the
+	 * end of this function.
+	 */
+	mpc_unit_lookup_by_name(ctl, mp->mp_params.mp_name, &unit);
+
+	if (unit && cmd != MPIOC_MP_DESTROY) {
+		if (cmd == MPIOC_MP_ACTIVATE)
+			goto errout;
+		rc = -EEXIST;
+		mp_pr_err("%s: mpool already activated", rc, mp->mp_params.mp_name);
+		goto errout;
+	}
+
+	/*
+	 * The device path names are in one long string separated by
+	 * newlines.  Here we allocate one chunk of memory to hold
+	 * all the device paths and a vector of ptrs to them.
+	 */
+	dpathvsz = mp->mp_dpathc * sizeof(*dpathv) + mp->mp_dpathssz;
+	if (dpathvsz > MPOOL_DRIVES_MAX * (PATH_MAX + sizeof(*dpathv))) {
+		rc = -E2BIG;
+		mp_pr_err("%s: %s, too many member drives %zu",
+			  rc, mp->mp_params.mp_name, action, dpathvsz);
+		goto errout;
+	}
+
+	dpathv = kmalloc(dpathvsz, GFP_KERNEL);
+	if (!dpathv) {
+		rc = -ENOMEM;
+		goto errout;
+	}
+
+	dpaths = (char *)dpathv + mp->mp_dpathc * sizeof(*dpathv);
+
+	rc = copy_from_user(dpaths, mp->mp_dpaths, mp->mp_dpathssz);
+	if (rc) {
+		rc = -EFAULT;
+		goto errout;
+	}
+
+	for (i = 0; i < mp->mp_dpathc; ++i) {
+		dpathv[i] = strsep(&dpaths, "\n");
+		if (!dpathv[i]) {
+			rc = -EINVAL;
+			goto errout;
+		}
+	}
+
+	/* Get the PDs properties from user space buffer. */
+	pd_prop_sz = mp->mp_dpathc * sizeof(*pd_prop);
+	pd_prop = kmalloc(pd_prop_sz, GFP_KERNEL);
+	if (!pd_prop) {
+		rc = -ENOMEM;
+		mp_pr_err("%s: %s, alloc pd prop %zu failed",
+			  rc, mp->mp_params.mp_name, action, pd_prop_sz);
+		goto errout;
+	}
+
+	rc = copy_from_user(pd_prop, mp->mp_pd_prop, pd_prop_sz);
+	if (rc) {
+		rc = -EFAULT;
+		mp_pr_err("%s: %s, copyin pd prop %zu failed",
+			  rc, mp->mp_params.mp_name, action, pd_prop_sz);
+		goto errout;
+	}
+
+	switch (cmd) {
+	case MPIOC_MP_CREATE:
+		rc = mpioc_mp_create(ctl, mp, pd_prop, &dpathv);
+		break;
+
+	case MPIOC_MP_ACTIVATE:
+		rc = mpioc_mp_activate(ctl, mp, pd_prop, &dpathv);
+		break;
+
+	case MPIOC_MP_DESTROY:
+		if (unit) {
+			mpc_unit_put(unit);
+			unit = NULL;
+
+			rc = mp_deactivate_impl(ctl, mp, true);
+			if (rc) {
+				action = "deactivate";
+				break;
+			}
+		}
+		rc = mpool_destroy(mp->mp_dpathc, dpathv, pd_prop, mp->mp_flags);
+		break;
+
+	case MPIOC_MP_RENAME:
+		rc = mpool_rename(mp->mp_dpathc, dpathv, pd_prop, mp->mp_flags,
+				   mp->mp_params.mp_name);
+		break;
+	}
+
+	if (rc)
+		mp_pr_err("%s: %s failed", rc, mp->mp_params.mp_name, action);
+
+errout:
+	mpc_unit_put(unit);
+	up(&ss->ss_op_sema);
+
+	kfree(pd_prop);
+	kfree(dpathv);
+
+	return rc;
+}
+
+/**
+ * mpioc_mp_add() - add a device to an existing mpool
+ * @unit:   mpool unit ptr
+ * @drv:    mpool device parameter block
+ *
+ * MPIOC_MP_ADD ioctl handler to add a drive to a activated mpool
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_mp_add(struct mpc_unit *unit, struct mpioc_drive *drv)
+{
+	struct mpool_descriptor *desc = unit->un_mpool->mp_desc;
+	size_t pd_prop_sz, dpathvsz;
+	struct pd_prop *pd_prop;
+	char **dpathv, *dpaths;
+	int rc, i;
+
+	/*
+	 * The device path names are in one long string separated by
+	 * newlines.  Here we allocate one chunk of memory to hold
+	 * all the device paths and a vector of ptrs to them.
+	 */
+	dpathvsz = drv->drv_dpathc * sizeof(*dpathv) + drv->drv_dpathssz;
+	if (drv->drv_dpathc > MPOOL_DRIVES_MAX ||
+	    dpathvsz > MPOOL_DRIVES_MAX * (PATH_MAX + sizeof(*dpathv))) {
+		rc = -E2BIG;
+		mp_pr_err("%s: invalid pathc %u, pathsz %zu",
+			  rc, unit->un_name, drv->drv_dpathc, dpathvsz);
+		return rc;
+	}
+
+	dpathv = kmalloc(dpathvsz, GFP_KERNEL);
+	if (!dpathv) {
+		rc = -ENOMEM;
+		mp_pr_err("%s: alloc dpathv %zu failed", rc, unit->un_name, dpathvsz);
+		return rc;
+	}
+
+	dpaths = (char *)dpathv + drv->drv_dpathc * sizeof(*dpathv);
+	rc = copy_from_user(dpaths, drv->drv_dpaths, drv->drv_dpathssz);
+	if (rc) {
+		rc = -EFAULT;
+		mp_pr_err("%s: copyin dpaths %u failed", rc, unit->un_name, drv->drv_dpathssz);
+		kfree(dpathv);
+		return rc;
+	}
+
+	for (i = 0; i < drv->drv_dpathc; ++i) {
+		dpathv[i] = strsep(&dpaths, "\n");
+		if (!dpathv[i] || (strlen(dpathv[i]) > PATH_MAX - 1)) {
+			rc = -EINVAL;
+			mp_pr_err("%s: ill-formed dpathv list ", rc, unit->un_name);
+			kfree(dpathv);
+			return rc;
+		}
+	}
+
+	/* Get the PDs properties from user space buffer. */
+	pd_prop_sz = drv->drv_dpathc * sizeof(*pd_prop);
+
+	pd_prop = kmalloc(pd_prop_sz, GFP_KERNEL);
+	if (!pd_prop) {
+		rc = -ENOMEM;
+		mp_pr_err("%s: alloc pd prop %zu failed", rc, unit->un_name, pd_prop_sz);
+		kfree(dpathv);
+		return rc;
+	}
+
+	rc = copy_from_user(pd_prop, drv->drv_pd_prop, pd_prop_sz);
+	if (rc) {
+		rc = -EFAULT;
+		mp_pr_err("%s: copyin pd prop %zu failed", rc, unit->un_name, pd_prop_sz);
+		kfree(pd_prop);
+		kfree(dpathv);
+		return rc;
+	}
+
+	for (i = 0; i < drv->drv_dpathc; ++i) {
+		rc = mpool_drive_add(desc, dpathv[i], &pd_prop[i]);
+		if (rc)
+			break;
+	}
+
+	kfree(pd_prop);
+	kfree(dpathv);
+
+	return rc;
+}
+
+static struct mpc_softstate *mpc_cdev2ss(struct cdev *cdev)
+{
+	if (!cdev || cdev->owner != THIS_MODULE) {
+		mp_pr_crit("module dissociated", -EINVAL);
+		return NULL;
+	}
+
+	return container_of(cdev, struct mpc_softstate, ss_cdev);
+}
+
+/*
+ * MPCTL file operations.
+ */
+
+/**
+ * mpc_open() - Open an mpool device.
+ * @ip: inode ptr
+ * @fp: file ptr
+ *
+ * Return:  Returns 0 on success, -errno otherwise...
+ */
+static int mpc_open(struct inode *ip, struct file *fp)
+{
+	struct mpc_softstate *ss;
+	struct mpc_unit *unit;
+	bool firstopen;
+	int rc = 0;
+
+	ss = mpc_cdev2ss(ip->i_cdev);
+	if (!ss || ss != &mpc_softstate)
+		return -EBADFD;
+
+	/* Acquire a reference on the unit object.  We'll release it in mpc_release(). */
+	mpc_unit_lookup(iminor(fp->f_inode), &unit);
+	if (!unit)
+		return -ENODEV;
+
+	if (down_trylock(&unit->un_open_lock)) {
+		rc = (fp->f_flags & O_NONBLOCK) ? -EWOULDBLOCK :
+			down_interruptible(&unit->un_open_lock);
+
+		if (rc)
+			goto errout;
+	}
+
+	firstopen = (unit->un_open_cnt == 0);
+
+	if (!firstopen) {
+		if (fp->f_mapping != unit->un_mapping)
+			rc = -EBUSY;
+		else if (unit->un_open_excl || (fp->f_flags & O_EXCL))
+			rc = -EBUSY;
+		goto unlock;
+	}
+
+	if (!mpc_unit_ismpooldev(unit)) {
+		unit->un_open_excl = !!(fp->f_flags & O_EXCL);
+		goto unlock; /* control device */
+	}
+
+	/* First open of an mpool unit (not the control device). */
+	if (!fp->f_mapping || fp->f_mapping != ip->i_mapping) {
+		rc = -EINVAL;
+		goto unlock;
+	}
+
+	fp->f_op = &mpc_fops_default;
+
+	unit->un_mapping = fp->f_mapping;
+
+	inode_lock(ip);
+	i_size_write(ip, 1ul << (__BITS_PER_LONG - 1));
+	inode_unlock(ip);
+
+	unit->un_open_excl = !!(fp->f_flags & O_EXCL);
+
+unlock:
+	if (!rc) {
+		fp->private_data = unit;
+		nonseekable_open(ip, fp);
+		++unit->un_open_cnt;
+	}
+	up(&unit->un_open_lock);
+
+errout:
+	if (rc) {
+		if (rc != -EBUSY)
+			mp_pr_err("open %s failed", rc, unit->un_name);
+		mpc_unit_put(unit);
+	}
+
+	return rc;
+}
+
+/**
+ * mpc_release() - Close the specified mpool device.
+ * @ip: inode ptr
+ * @fp: file ptr
+ *
+ * Return:  Returns 0 on success, -errno otherwise...
+ */
+static int mpc_release(struct inode *ip, struct file *fp)
+{
+	struct mpc_unit *unit;
+	bool lastclose;
+
+	unit = fp->private_data;
+	if (!unit)
+		return -EBADFD;
+
+	down(&unit->un_open_lock);
+	lastclose = (--unit->un_open_cnt == 0);
+	if (!lastclose)
+		goto errout;
+
+	if (mpc_unit_ismpooldev(unit))
+		unit->un_mapping = NULL;
+
+	unit->un_open_excl = false;
+
+errout:
+	up(&unit->un_open_lock);
+
+	mpc_unit_put(unit);
+
+	return 0;
+}
+
+/**
+ * mpc_ioctl() - mpc driver ioctl entry point
+ * @fp:     file pointer
+ * @cmd:    an mpool ioctl command (i.e.,  MPIOC_*)
+ * @arg:    varies..
+ *
+ * Perform the specified mpool ioctl command.
+ *
+ * Return:  Returns 0 on success, -errno otherwise...
+ */
+static long mpc_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
+{
+	char argbuf[256] __aligned(16);
+	struct mpc_unit *unit;
+	size_t argbufsz;
+	void *argp;
+	ulong iosz;
+	int rc;
+
+	if (_IOC_TYPE(cmd) != MPIOC_MAGIC)
+		return -ENOTTY;
+
+	if ((fp->f_flags & O_ACCMODE) == O_RDONLY) {
+		switch (cmd) {
+		case MPIOC_PROP_GET:
+		case MPIOC_DEVPROPS_GET:
+		case MPIOC_MP_MCLASS_GET:
+			break;
+
+		default:
+			return -EINVAL;
+		}
+	}
+
+	unit = fp->private_data;
+	argbufsz = sizeof(argbuf);
+	iosz = _IOC_SIZE(cmd);
+	argp = (void *)arg;
+
+	if (!unit || (iosz > sizeof(union mpioc_union)))
+		return -EINVAL;
+
+	/* Set up argp/argbuf for read/write requests. */
+	if (_IOC_DIR(cmd) & (_IOC_READ | _IOC_WRITE)) {
+		argp = argbuf;
+		if (iosz > argbufsz) {
+			argbufsz = roundup_pow_of_two(iosz);
+
+			argp = kzalloc(argbufsz, GFP_KERNEL);
+			if (!argp)
+				return -ENOMEM;
+		}
+
+		if (_IOC_DIR(cmd) & _IOC_WRITE) {
+			if (copy_from_user(argp, (const void __user *)arg, iosz)) {
+				if (argp != argbuf)
+					kfree(argp);
+				return -EFAULT;
+			}
+		}
+	}
+
+	switch (cmd) {
+	case MPIOC_MP_CREATE:
+	case MPIOC_MP_ACTIVATE:
+	case MPIOC_MP_DESTROY:
+	case MPIOC_MP_RENAME:
+		rc = mpioc_mp_cmd(unit, cmd, argp);
+		break;
+
+	case MPIOC_MP_DEACTIVATE:
+		rc = mpioc_mp_deactivate(unit, argp);
+		break;
+
+	case MPIOC_DRV_ADD:
+		rc = mpioc_mp_add(unit, argp);
+		break;
+
+	case MPIOC_PARAMS_SET:
+		rc = mpioc_params_set(unit, argp);
+		break;
+
+	case MPIOC_PARAMS_GET:
+		rc = mpioc_params_get(unit, argp);
+		break;
+
+	case MPIOC_MP_MCLASS_GET:
+		rc = mpioc_mp_mclass_get(unit, argp);
+		break;
+
+	case MPIOC_PROP_GET:
+		rc = mpioc_proplist_get(unit, argp);
+		break;
+
+	case MPIOC_DEVPROPS_GET:
+		rc = mpioc_devprops_get(unit, argp);
+		break;
+
+	default:
+		rc = -ENOTTY;
+		mp_pr_rl("invalid command %x: dir=%u type=%c nr=%u size=%u",
+			 rc, cmd, _IOC_DIR(cmd), _IOC_TYPE(cmd), _IOC_NR(cmd), _IOC_SIZE(cmd));
+		break;
+	}
+
+	if (!rc && _IOC_DIR(cmd) & _IOC_READ) {
+		if (copy_to_user((void __user *)arg, argp, iosz))
+			rc = -EFAULT;
+	}
+
+	if (argp != argbuf)
+		kfree(argp);
+
+	return rc;
+}
+
+static const struct file_operations mpc_fops_default = {
+	.owner		= THIS_MODULE,
+	.open		= mpc_open,
+	.release	= mpc_release,
+	.unlocked_ioctl	= mpc_ioctl,
+};
+
 static int mpc_exit_unit(int minor, void *item, void *arg)
 {
 	mpc_unit_put(item);
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:28 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:28 -0500
Subject: [PATCH v2 14/22] mpool: add pool metadata routines to create
 persistent mpools
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-15-nmeeramohide@micron.com>

Mpool metadata is stored in metadata containers (MDC). An
mpool can have a maximum of 256 MDCs, MDC-0 through MDC-255.
The following metadata manager functionality is added here
for object persistence:

- Initialize and validate MDC0
- Allocate and initialize MDC 1-N. An mpool is created with
  16 MDCs to provide the requisite concurrency.
- Dynamically scale up the number of MDCs when running low
  on space and the garbage is below a certain threshold
  across all MDCs
- Deserialize metadata records from MDC 0-N at mpool activation
  and setup the corresponding in-memory structures
- Pre-compact MDC-K based on its usage and if the garbage in
  MDC-K is above a certain threshold. A pre-compacting MDC is
  not chosen for object allocation.

MDC0 is a distinguished container that stores both the metadata
for accessing MDC-1 through MDC-255 and all mpool properties.
MDC-1 through MDC-255 store the metadata for accessing client
allocated mblocks and mlogs. Metadata for accessing the mlogs
comprising MDC-0 is in the superblock for the capacity media
class.

In the context of MDC-1/255, compacting MDC-K is simply
serializing the in-memory metadata for accessing the still-live
client objects associated with MDC-K. In the context of MDC-0,
compacting is simply serializing the in-memory mpool properties
and in-memory metadata for accessing MDC-1/255.

An instance of struct pmd_mdc_info is created for each MDC in
an mpool. This struct hosts both the uncommitted and committed
object trees and a lock protecting each of the two trees.
Compacting an MDC requires freezing both the list of committed
objects in that MDC and the metadata for those objects,
which is facilitated by the compact lock in each MDC instance.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/pmd.c     | 2046 +++++++++++++++++++++++++++++++++++++++
 drivers/mpool/pmd_obj.c |    8 -
 2 files changed, 2046 insertions(+), 8 deletions(-)
 create mode 100644 drivers/mpool/pmd.c

diff --git a/drivers/mpool/pmd.c b/drivers/mpool/pmd.c
new file mode 100644
index 000000000000..07e08b5eed43
--- /dev/null
+++ b/drivers/mpool/pmd.c
@@ -0,0 +1,2046 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * DOC: Module info.
+ *
+ * Pool metadata (pmd) module.
+ *
+ * Defines functions for probing, reading, and writing drives in an mpool.
+ *
+ */
+
+#include <linux/workqueue.h>
+#include <linux/atomic.h>
+#include <linux/rwsem.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+
+#include "assert.h"
+#include "mpool_printk.h"
+
+#include "mpool_ioctl.h"
+#include "mdc.h"
+#include "upgrade.h"
+#include "smap.h"
+#include "omf_if.h"
+#include "mpcore.h"
+#include "pmd.h"
+
+static DEFINE_MUTEX(pmd_s_lock);
+
+#define pmd_co_foreach(_cinfo, _node) \
+	for ((_node) = rb_first(&(_cinfo)->mmi_co_root); (_node); (_node) = rb_next((_node)))
+
+static int pmd_mdc0_validate(struct mpool_descriptor *mp, int activation);
+
+static void pmd_mda_init(struct mpool_descriptor *mp)
+{
+	int i;
+
+	spin_lock_init(&mp->pds_mda.mdi_slotvlock);
+	mp->pds_mda.mdi_slotvcnt = 0;
+
+	for (i = 0; i < MDC_SLOTS; ++i) {
+		struct pmd_mdc_info *pmi = mp->pds_mda.mdi_slotv + i;
+
+		mutex_init(&pmi->mmi_compactlock);
+		mutex_init(&pmi->mmi_uc_lock);
+		pmi->mmi_uc_root = RB_ROOT;
+		init_rwsem(&pmi->mmi_co_lock);
+		pmi->mmi_co_root = RB_ROOT;
+		mutex_init(&pmi->mmi_uqlock);
+		pmi->mmi_luniq = 0;
+		pmi->mmi_recbuf = NULL;
+		pmi->mmi_lckpt = objid_make(0, OMF_OBJ_UNDEF, i);
+		memset(&pmi->mmi_stats, 0, sizeof(pmi->mmi_stats));
+
+		/*
+		 * Initial mpool metadata content version.
+		 */
+		pmi->mmi_mdcver.mdcv_major = 1;
+		pmi->mmi_mdcver.mdcv_minor = 0;
+		pmi->mmi_mdcver.mdcv_patch = 0;
+		pmi->mmi_mdcver.mdcv_dev   = 0;
+
+		pmi->mmi_credit.ci_slot = i;
+
+		mutex_init(&pmi->mmi_stats_lock);
+	}
+
+	mp->pds_mda.mdi_slotv[1].mmi_luniq = UROOT_OBJID_MAX;
+	mp->pds_mda.mdi_sel.mds_tbl_idx.counter = 0;
+}
+
+static void pmd_mda_free(struct mpool_descriptor *mp)
+{
+	int sidx;
+
+	/*
+	 * close mdc0 last because closing other mdc logs can result in
+	 * mdc0 updates
+	 */
+	for (sidx = mp->pds_mda.mdi_slotvcnt - 1; sidx > -1; sidx--) {
+		struct pmd_layout      *layout, *tmp;
+		struct pmd_mdc_info    *cinfo;
+
+		cinfo = &mp->pds_mda.mdi_slotv[sidx];
+
+		mp_mdc_close(cinfo->mmi_mdc);
+		kfree(cinfo->mmi_recbuf);
+		cinfo->mmi_recbuf = NULL;
+
+		/* Release committed objects... */
+		rbtree_postorder_for_each_entry_safe(
+			layout, tmp, &cinfo->mmi_co_root, eld_nodemdc) {
+
+			pmd_obj_put(layout);
+		}
+
+		/* Release uncommitted objects... */
+		rbtree_postorder_for_each_entry_safe(
+			layout, tmp, &cinfo->mmi_uc_root, eld_nodemdc) {
+
+			pmd_obj_put(layout);
+		}
+	}
+}
+
+static int pmd_mdc0_init(struct mpool_descriptor *mp, struct pmd_layout *mdc01,
+		     struct pmd_layout *mdc02)
+{
+	struct pmd_mdc_info *cinfo = &mp->pds_mda.mdi_slotv[0];
+	int rc;
+
+	cinfo->mmi_recbuf = kzalloc(OMF_MDCREC_PACKLEN_MAX, GFP_KERNEL);
+	if (!cinfo->mmi_recbuf) {
+		rc = -ENOMEM;
+		mp_pr_err("mpool %s, log rec buffer alloc %zu failed",
+			  rc, mp->pds_name, OMF_MDCREC_PACKLEN_MAX);
+		return rc;
+	}
+
+	/*
+	 * we put the mdc0 mlog layouts in mdc 0 because mdc0 mlog objids have a
+	 * slot # of 0 so the rest of the code expects to find the layout there.
+	 * this allows the majority of the code to treat mdc0 mlog metadata
+	 * exactly the same as for mdcN (and user mlogs), even though mdc0
+	 * metadata is actually stored in superblocks.  however there are a few
+	 * places that need to recognize mdc0 mlogs are special, including
+	 * pmd_mdc_compact() and pmd_obj_erase().
+	 */
+
+	mp->pds_mda.mdi_slotvcnt = 1;
+	pmd_co_insert(cinfo, mdc01);
+	pmd_co_insert(cinfo, mdc02);
+
+	rc = mp_mdc_open(mp, mdc01->eld_objid, mdc02->eld_objid, MDC_OF_SKIP_SER, &cinfo->mmi_mdc);
+	if (rc) {
+		mp_pr_err("mpool %s, MDC0 open failed", rc, mp->pds_name);
+
+		pmd_co_remove(cinfo, mdc01);
+		pmd_co_remove(cinfo, mdc02);
+
+		kfree(cinfo->mmi_recbuf);
+		cinfo->mmi_recbuf = NULL;
+
+		mp->pds_mda.mdi_slotvcnt = 0;
+	}
+
+	return rc;
+}
+
+/**
+ * pmd_mdc0_validate() -
+ * @mp:
+ * @activation:
+ *
+ * Called during mpool activation and mdc alloc because a failed
+ * mdc alloc can result in extraneous mdc mlog objects which if
+ * found we attempt to clean-up here. when called during activation
+ * we may need to adjust mp.mda. this is not so when called from
+ * mdc alloc and in fact decreasing slotvcnt post activation would
+ * violate a key invariant.
+ */
+static int pmd_mdc0_validate(struct mpool_descriptor *mp, int activation)
+{
+	struct pmd_mdc_info *cinfo;
+	struct pmd_layout *layout;
+	struct rb_node *node;
+	int err = 0, err1, err2, i;
+	u64 mdcn, mdcmax = 0;
+	u64 logid1, logid2;
+	u16 slotvcnt;
+	u8 *lcnt;
+
+	/*
+	 * Activation is single-threaded and mdc alloc is serialized
+	 * so the number of active mdc (slotvcnt) will not change.
+	 */
+	spin_lock(&mp->pds_mda.mdi_slotvlock);
+	slotvcnt = mp->pds_mda.mdi_slotvcnt;
+	spin_unlock(&mp->pds_mda.mdi_slotvlock);
+
+	if (!slotvcnt) {
+		/* Must be at least mdc0 */
+		err = -EINVAL;
+		mp_pr_err("mpool %s, no MDC0", err, mp->pds_name);
+		return err;
+	}
+
+	cinfo = &mp->pds_mda.mdi_slotv[0];
+
+	lcnt = kcalloc(MDC_SLOTS, sizeof(*lcnt), GFP_KERNEL);
+	if (!lcnt) {
+		err = -ENOMEM;
+		mp_pr_err("mpool %s, lcnt alloc failed", err, mp->pds_name);
+		return err;
+	}
+
+	pmd_co_rlock(cinfo, 0);
+
+	pmd_co_foreach(cinfo, node) {
+		layout = rb_entry(node, typeof(*layout), eld_nodemdc);
+
+		mdcn = objid_uniq(layout->eld_objid) >> 1;
+		if (mdcn < MDC_SLOTS) {
+			lcnt[mdcn] = lcnt[mdcn] + 1;
+			mdcmax = max(mdcmax, mdcn);
+		}
+		if (mdcn >= MDC_SLOTS || lcnt[mdcn] > 2 ||
+		    objid_type(layout->eld_objid) != OMF_OBJ_MLOG ||
+		    objid_slot(layout->eld_objid)) {
+			err = -EINVAL;
+			mp_pr_err("mpool %s, MDC0 number of MDCs %lu %u or bad otype, objid 0x%lx",
+				  err, mp->pds_name, (ulong)mdcn,
+				  lcnt[mdcn], (ulong)layout->eld_objid);
+			break;
+		}
+	}
+
+	pmd_co_runlock(cinfo);
+
+	if (err)
+		goto exit;
+
+	if (!mdcmax) {
+		/*
+		 * trivial case of mdc0 only; no mdc alloc failure to
+		 * clean-up
+		 */
+		if (lcnt[0] != 2 || slotvcnt != 1) {
+			err = -EINVAL;
+			mp_pr_err("mpool %s, inconsistent number of MDCs or slots %d %d",
+				  err, mp->pds_name, lcnt[0], slotvcnt);
+		}
+
+		goto exit;
+	}
+
+	if ((mdcmax != (slotvcnt - 1)) && mdcmax != slotvcnt) {
+		err = -EINVAL;
+
+		/*
+		 * mdcmax is normally slotvcnt-1; can be slotvcnt if
+		 * mdc alloc failed
+		 */
+		mp_pr_err("mpool %s, inconsistent max number of MDCs %lu %u",
+			  err, mp->pds_name, (ulong)mdcmax, slotvcnt);
+		goto exit;
+	}
+
+	/* Both logs must always exist below mdcmax */
+	for (i = 0; i < mdcmax; i++) {
+		if (lcnt[i] != 2) {
+			err = -ENOENT;
+			mp_pr_err("mpool %s, MDC0 missing mlogs %lu %d %u",
+				  err, mp->pds_name, (ulong)mdcmax, i, lcnt[i]);
+			goto exit;
+		}
+	}
+
+	/* Clean-up from failed mdc alloc if needed */
+	if (lcnt[mdcmax] != 2 || mdcmax == slotvcnt) {
+		/* Note: if activation then mdcmax == slotvcnt-1 always */
+		err1 = 0;
+		err2 = 0;
+		logid1 = logid_make(2 * mdcmax, 0);
+		logid2 = logid_make(2 * mdcmax + 1, 0);
+
+		layout = pmd_obj_find_get(mp, logid1, 1);
+		if (layout) {
+			err1 = pmd_obj_delete(mp, layout);
+			if (err1)
+				mp_pr_err("mpool %s, MDC0 %d, can't delete mlog %lu %lu %u %u",
+					  err1, mp->pds_name, activation, (ulong)logid1,
+					  (ulong)mdcmax, lcnt[mdcmax], slotvcnt);
+		}
+
+		layout = pmd_obj_find_get(mp, logid2, 1);
+		if (layout) {
+			err2 = pmd_obj_delete(mp, layout);
+			if (err2)
+				mp_pr_err("mpool %s, MDC0 %d, can't delete mlog %lu %lu %u %u",
+					  err2, mp->pds_name, activation, (ulong)logid2,
+					  (ulong)mdcmax, lcnt[mdcmax], slotvcnt);
+		}
+
+		if (activation) {
+			/*
+			 * Mpool activation can ignore mdc alloc clean-up
+			 * failures; single-threaded; don't need slotvlock
+			 * or uqlock to adjust mda
+			 */
+			cinfo->mmi_luniq = mdcmax - 1;
+			mp->pds_mda.mdi_slotvcnt = mdcmax;
+			mp_pr_warn("mpool %s, MDC0 alloc recovery: uniq %llu slotvcnt %d",
+				   mp->pds_name, (unsigned long long)cinfo->mmi_luniq,
+				   mp->pds_mda.mdi_slotvcnt);
+		} else {
+			/* MDC alloc cannot tolerate clean-up failures */
+			if (err1)
+				err = err1;
+			else if (err2)
+				err = err2;
+
+			if (err)
+				mp_pr_err("mpool %s, MDC0 alloc recovery, cleanup failed %lu %u %u",
+					  err, mp->pds_name, (ulong)mdcmax, lcnt[mdcmax], slotvcnt);
+			else
+				mp_pr_warn("mpool %s, MDC0 alloc recovery", mp->pds_name);
+
+		}
+	}
+
+exit:
+	kfree(lcnt);
+
+	return err;
+}
+
+int pmd_mdc_alloc(struct mpool_descriptor *mp, u64 mincap, u32 iter)
+{
+	struct pmd_obj_capacity ocap;
+	enum mp_media_classp mclassp;
+	struct pmd_mdc_info *cinfo, *cinew;
+	struct pmd_layout *layout1, *layout2;
+	const char *msg = "(no detail)";
+	u64 mdcslot, logid1, logid2;
+	bool reverse = false;
+	u32 pdcnt;
+	int err;
+
+	/*
+	 * serialize to prevent gap in mdc slot space in event of failure
+	 */
+	mutex_lock(&pmd_s_lock);
+
+	/*
+	 * recover previously failed mdc alloc if needed; cannot continue
+	 * if fails
+	 * note: there is an unlikely corner case where we logically delete an
+	 * mlog from a previously failed mdc alloc but a background op is
+	 * preventing its full removal; this will show up later in this
+	 * fn as a failed alloc.
+	 */
+	err = pmd_mdc0_validate(mp, 0);
+	if (err) {
+		mutex_unlock(&pmd_s_lock);
+
+		mp_pr_err("mpool %s, allocating an MDC, inconsistent MDC0", err, mp->pds_name);
+		return err;
+	}
+
+	/* MDC0 exists by definition; created as part of mpool creation */
+	cinfo = &mp->pds_mda.mdi_slotv[0];
+
+	pmd_mdc_lock(&cinfo->mmi_uqlock, 0);
+	mdcslot = cinfo->mmi_luniq;
+	pmd_mdc_unlock(&cinfo->mmi_uqlock);
+
+	if (mdcslot >= MDC_SLOTS - 1) {
+		mutex_unlock(&pmd_s_lock);
+
+		err = -ENOSPC;
+		mp_pr_err("mpool %s, allocating an MDC, too many %lu",
+			  err, mp->pds_name, (ulong)mdcslot);
+		return err;
+	}
+	mdcslot = mdcslot + 1;
+
+	/*
+	 * Alloc rec buf for new mdc slot; not visible so don't need to
+	 * lock fields.
+	 */
+	cinew = &mp->pds_mda.mdi_slotv[mdcslot];
+	cinew->mmi_recbuf = kzalloc(OMF_MDCREC_PACKLEN_MAX, GFP_KERNEL);
+	if (!cinew->mmi_recbuf) {
+		mutex_unlock(&pmd_s_lock);
+
+		mp_pr_warn("mpool %s, MDC%lu pack/unpack buf alloc failed %lu",
+			   mp->pds_name, (ulong)mdcslot, (ulong)OMF_MDCREC_PACKLEN_MAX);
+		return -ENOMEM;
+	}
+	cinew->mmi_credit.ci_slot = mdcslot;
+
+	mclassp = MP_MED_CAPACITY;
+	pdcnt = 1;
+
+	/*
+	 * Create new mdcs with same parameters and on same media class
+	 * as mdc0.
+	 */
+	ocap.moc_captgt = mincap;
+	ocap.moc_spare  = false;
+
+	logid1 = logid_make(2 * mdcslot, 0);
+	logid2 = logid_make(2 * mdcslot + 1, 0);
+
+	if (!(pdcnt & 0x1) && ((iter * 2 / pdcnt) & 0x1)) {
+		/*
+		 * Reverse the allocation order.
+		 * The goal is to have active mlogs on all the mpool PDs.
+		 * If 2 PDs, no parity, no reserve, the active mlogs
+		 * will be on PDs 0,1,0,1,0,1,0,1 etc
+		 * instead of 0,0,0,0,0 etc without reversing.
+		 * No need to reverse if the number of PDs is odd.
+		 */
+		reverse = true;
+	}
+
+	/*
+	 * Each mlog must meet mincap since only one is active at a
+	 * time.
+	 */
+	layout1 = NULL;
+	err = pmd_obj_alloc_cmn(mp, reverse ? logid2 : logid1, OMF_OBJ_MLOG,
+				&ocap, mclassp, 0, false, &layout1);
+	if (err) {
+		if (err != -ENOENT)
+			msg = "allocation of first mlog failed";
+		goto exit;
+	}
+
+	layout2 = NULL;
+	err = pmd_obj_alloc_cmn(mp, reverse ? logid1 : logid2, OMF_OBJ_MLOG,
+				&ocap, mclassp, 0, false, &layout2);
+	if (err) {
+		pmd_obj_abort(mp, layout1);
+		if (err != -ENOENT)
+			msg = "allocation of second mlog failed";
+		goto exit;
+	}
+
+	/*
+	 * Must erase before commit to guarantee new mdc logs start
+	 * empty; mlogs not committed so pmd_obj_erase()
+	 * not needed to make atomic.
+	 */
+	pmd_obj_wrlock(layout1);
+	err = pmd_layout_erase(mp, layout1);
+	pmd_obj_wrunlock(layout1);
+
+	if (err) {
+		msg = "erase of first mlog failed";
+	} else {
+		pmd_obj_wrlock(layout2);
+		err = pmd_layout_erase(mp, layout2);
+		pmd_obj_wrunlock(layout2);
+
+		if (err)
+			msg = "erase of second mlog failed";
+	}
+	if (err) {
+		pmd_obj_abort(mp, layout1);
+		pmd_obj_abort(mp, layout2);
+		goto exit;
+	}
+
+	/*
+	 * don't need to commit logid1 and logid2 atomically; mdc0
+	 * validation deletes non-paired mdc logs to handle failing part
+	 * way through this process
+	 */
+	err = pmd_obj_commit(mp, layout1);
+	if (err) {
+		pmd_obj_abort(mp, layout1);
+		pmd_obj_abort(mp, layout2);
+		msg = "commit of first mlog failed";
+		goto exit;
+	} else {
+		err = pmd_obj_commit(mp, layout2);
+		if (err) {
+			pmd_obj_delete(mp, layout1);
+			pmd_obj_abort(mp, layout2);
+			msg = "commit of second mlog failed";
+			goto exit;
+		}
+	}
+
+	/*
+	 * Finalize new mdc slot before making visible; don't need to
+	 * lock fields.
+	 */
+	err = mp_mdc_open(mp, logid1, logid2, MDC_OF_SKIP_SER, &cinew->mmi_mdc);
+	if (err) {
+		msg = "mdc open failed";
+
+		/* Failed open so just delete logid1/2; don't
+		 * need to delete atomically since mdc0 validation
+		 * will cleanup any detritus
+		 */
+		pmd_obj_delete(mp, layout1);
+		pmd_obj_delete(mp, layout2);
+		goto exit;
+	}
+
+	/*
+	 * Append the version record.
+	 */
+	if (omfu_mdcver_cmp2(omfu_mdcver_cur(), ">=", 1, 0, 0, 1)) {
+		err = pmd_mdc_addrec_version(mp, mdcslot);
+		if (err) {
+			msg = "error adding the version record";
+			/*
+			 * No version record in a MDC will trigger a MDC
+			 * compaction if a activate is attempted later with this
+			 * empty MDC.
+			 * The compaction will add the version record in that
+			 * empty MDC.
+			 * Same error handling as above.
+			 */
+			pmd_obj_delete(mp, layout1);
+			pmd_obj_delete(mp, layout2);
+			goto exit;
+		}
+	}
+
+	/* Make new mdc visible */
+	pmd_mdc_lock(&cinfo->mmi_uqlock, 0);
+
+	spin_lock(&mp->pds_mda.mdi_slotvlock);
+	cinfo->mmi_luniq = mdcslot;
+	mp->pds_mda.mdi_slotvcnt = mdcslot + 1;
+	spin_unlock(&mp->pds_mda.mdi_slotvlock);
+
+	pmd_mdc_unlock(&cinfo->mmi_uqlock);
+
+exit:
+	if (err) {
+		kfree(cinew->mmi_recbuf);
+		cinew->mmi_recbuf = NULL;
+	}
+
+	mutex_unlock(&pmd_s_lock);
+
+	mp_pr_debug("new mdc logid1 %llu logid2 %llu",
+		    0, (unsigned long long)logid1, (unsigned long long)logid2);
+
+	if (err) {
+		mp_pr_err("mpool %s, MDC%lu: %s", err, mp->pds_name, (ulong)mdcslot, msg);
+
+	} else {
+		mp_pr_debug("mpool %s, delta slotvcnt from %u to %llu", 0, mp->pds_name,
+			    mp->pds_mda.mdi_slotvcnt, (unsigned long long)mdcslot + 1);
+
+	}
+	return err;
+}
+
+/**
+ * pmd_mdc_alloc_set() - allocates a set of MDCs
+ * @mp: mpool descriptor
+ *
+ * Creates MDCs in multiple of MPOOL_MDC_SET_SZ. If allocation had
+ * failed in prior iteration allocate MDCs to make it even multiple
+ * of MPOOL_MDC_SET_SZ.
+ *
+ * Locking: lock should not be held when calling this function.
+ */
+static void pmd_mdc_alloc_set(struct mpool_descriptor *mp)
+{
+	u8 mdc_cnt, sidx;
+	int rc;
+
+	/*
+	 * MDCs are created in multiple of MPOOL_MDC_SET_SZ.
+	 * However, if past allocation had failed there may not be an
+	 * even multiple of MDCs in that case create any remaining
+	 * MDCs to get an even multiple.
+	 */
+	mdc_cnt =  MPOOL_MDC_SET_SZ - ((mp->pds_mda.mdi_slotvcnt - 1) % MPOOL_MDC_SET_SZ);
+
+	mdc_cnt = min(mdc_cnt, (u8)(MDC_SLOTS - (mp->pds_mda.mdi_slotvcnt)));
+
+	for (sidx = 1; sidx <= mdc_cnt; sidx++) {
+		rc = pmd_mdc_alloc(mp, mp->pds_params.mp_mdcncap, 0);
+		if (rc) {
+			mp_pr_err("mpool %s, only %u of %u MDCs created",
+				  rc, mp->pds_name, sidx-1, mdc_cnt);
+
+			/*
+			 * For MDCN creation failure ignore the error.
+			 * Attempt to create any remaining MDC next time
+			 * next time new mdcs are required.
+			 */
+			rc = 0;
+			break;
+		}
+	}
+}
+
+/**
+ * pmd_cmp_drv_mdc0() - compare the drive info read from the MDC0 drive list
+ *	to what is obtained from the drive itself or from the configuration.
+ * @mp:
+ * @pdh:
+ * @omd:
+ *
+ * The drive is in list passed to mpool open or an UNAVAIL mdc0 drive.
+ */
+static int pmd_cmp_drv_mdc0(struct mpool_descriptor *mp, u8 pdh,
+			    struct omf_devparm_descriptor *omd)
+{
+	const char *msg __maybe_unused;
+	struct mc_parms mcp_mdc0list, mcp_pd;
+	struct mpool_dev_info *pd;
+
+	pd = &mp->pds_pdv[pdh];
+
+	mc_pd_prop2mc_parms(&(pd->pdi_parm.dpr_prop), &mcp_pd);
+	mc_omf_devparm2mc_parms(omd, &mcp_mdc0list);
+
+	if (!memcmp(&mcp_pd, &mcp_mdc0list, sizeof(mcp_pd)))
+		return 0;
+
+	if (mpool_pd_status_get(pd) == PD_STAT_UNAVAIL)
+		msg = "UNAVAIL mdc0 drive parms don't match those in drive list record";
+	else
+		msg = "mismatch between MDC0 drive list record and drive parms";
+
+	mp_pr_warn("mpool %s, %s for %s, mclassp %d %d zonepg %u %u sectorsz %u %u devtype %u %u features %lu %lu",
+		   mp->pds_name, msg, pd->pdi_name, mcp_pd.mcp_classp, mcp_mdc0list.mcp_classp,
+		   mcp_pd.mcp_zonepg, mcp_mdc0list.mcp_zonepg, mcp_pd.mcp_sectorsz,
+		   mcp_mdc0list.mcp_sectorsz, mcp_pd.mcp_devtype, mcp_mdc0list.mcp_devtype,
+		   (ulong)mcp_pd.mcp_features, (ulong)mcp_mdc0list.mcp_features);
+
+	return -EINVAL;
+}
+
+static const char *msg_unavail1 __maybe_unused =
+	"defunct and unavailable drive still belong to the mpool";
+
+static const char *msg_unavail2 __maybe_unused =
+	"defunct and available drive still belong to the mpool";
+
+static int pmd_props_load(struct mpool_descriptor *mp)
+{
+	struct omf_devparm_descriptor netdev[MP_MED_NUMBER] = { };
+	struct omf_mdcrec_data *cdr;
+	enum mp_media_classp mclassp;
+	struct pmd_mdc_info *cinfo;
+	struct media_class *mc;
+	bool zombie[MPOOL_DRIVES_MAX];
+	int spzone[MP_MED_NUMBER], i;
+	size_t rlen = 0;
+	u64 pdh, buflen;
+	int err;
+
+	cinfo = &mp->pds_mda.mdi_slotv[0];
+	buflen = OMF_MDCREC_PACKLEN_MAX;
+
+	/*  Note: single threaded here so don't need any locks */
+
+	/* Set mpool properties to defaults; overwritten by property records (if any). */
+	for (mclassp = 0; mclassp < MP_MED_NUMBER; mclassp++)
+		spzone[mclassp] = -1;
+
+	/*
+	 * read mdc0 to capture net of drives, content version & other
+	 * properties; ignore obj records
+	 */
+	err = mp_mdc_rewind(cinfo->mmi_mdc);
+	if (err) {
+		mp_pr_err("mpool %s, MDC0 init for read properties failed", err, mp->pds_name);
+		return err;
+	}
+
+	cdr = kzalloc(sizeof(*cdr), GFP_KERNEL);
+	if (!cdr) {
+		err = -ENOMEM;
+		mp_pr_err("mpool %s, cdr alloc failed", err, mp->pds_name);
+		return err;
+	}
+
+	while (true) {
+		err = mp_mdc_read(cinfo->mmi_mdc, cinfo->mmi_recbuf, buflen, &rlen);
+		if (err) {
+			mp_pr_err("mpool %s, MDC0 read next failed %lu",
+				  err, mp->pds_name, (ulong)rlen);
+			break;
+		}
+		if (rlen == 0)
+			/* Hit end of log */
+			break;
+
+		/*
+		 * skip object-related mdcrec in mdc0; not ready to unpack
+		 * these yet
+		 */
+		if (omf_mdcrec_isobj_le(cinfo->mmi_recbuf))
+			continue;
+
+		err = omf_mdcrec_unpack_letoh(&(cinfo->mmi_mdcver), mp, cdr, cinfo->mmi_recbuf);
+		if (err) {
+			mp_pr_err("mpool %s, MDC0 property unpack failed", err, mp->pds_name);
+			break;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_MCCONFIG) {
+			struct omf_devparm_descriptor *src;
+
+			src = &cdr->u.dev.omd_parm;
+			ASSERT(src->odp_mclassp < MP_MED_NUMBER);
+
+			memcpy(&netdev[src->odp_mclassp], src, sizeof(*src));
+			continue;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_MCSPARE) {
+			mclassp = cdr->u.mcs.omd_mclassp;
+			if (mclass_isvalid(mclassp)) {
+				spzone[mclassp] = cdr->u.mcs.omd_spzone;
+			} else {
+				err = -EINVAL;
+
+				/* Should never happen */
+				mp_pr_err("mpool %s, MDC0 mclass spare record, invalid mclassp %u",
+					  err, mp->pds_name, mclassp);
+				break;
+			}
+			continue;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_VERSION) {
+			cinfo->mmi_mdcver = cdr->u.omd_version;
+			if (omfu_mdcver_cmp(&cinfo->mmi_mdcver, ">", omfu_mdcver_cur())) {
+				char *buf1, *buf2 = NULL;
+
+				buf1 = kmalloc(2 * MAX_MDCVERSTR, GFP_KERNEL);
+				if (buf1) {
+					buf2 = buf1 + MAX_MDCVERSTR;
+					omfu_mdcver_to_str(&cinfo->mmi_mdcver, buf1, sizeof(buf1));
+					omfu_mdcver_to_str(omfu_mdcver_cur(), buf2, sizeof(buf2));
+				}
+
+				err = -EOPNOTSUPP;
+				mp_pr_err("mpool %s, MDC0 version %s, binary version %s",
+					  err, mp->pds_name, buf1, buf2);
+				kfree(buf1);
+				break;
+			}
+			continue;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_MPCONFIG)
+			mp->pds_cfg = cdr->u.omd_cfg;
+	}
+
+	if (err) {
+		kfree(cdr);
+		return err;
+	}
+
+	/* Reconcile net drive list with those in mpool descriptor */
+	for (i = 0; i < mp->pds_pdvcnt; i++)
+		zombie[i] = true;
+
+	for (i = 0; i < MP_MED_NUMBER; i++) {
+		struct omf_devparm_descriptor *omd;
+		int    j;
+
+		omd = &netdev[i];
+
+		if (mpool_uuid_is_null(&omd->odp_devid))
+			continue;
+
+		j = mp->pds_pdvcnt;
+		while (j--) {
+			if (mpool_uuid_compare(&mp->pds_pdv[j].pdi_devid, &omd->odp_devid) == 0)
+				break;
+		}
+
+		if (j >= 0) {
+			zombie[j] = false;
+			err = pmd_cmp_drv_mdc0(mp, j, omd);
+			if (err)
+				break;
+		} else {
+			err = mpool_desc_unavail_add(mp, omd);
+			if (err)
+				break;
+			zombie[mp->pds_pdvcnt - 1] = false;
+		}
+	}
+
+	/* Check for zombie drives and recompute uacnt[] */
+	if (!err) {
+		for (i = 0; i < MP_MED_NUMBER; i++) {
+			mc = &mp->pds_mc[i];
+			mc->mc_uacnt = 0;
+		}
+
+		for (pdh = 0; pdh < mp->pds_pdvcnt; pdh++) {
+			struct mpool_dev_info  *pd;
+
+			mc = &mp->pds_mc[mp->pds_pdv[pdh].pdi_mclass];
+			pd = &mp->pds_pdv[pdh];
+			if (zombie[pdh]) {
+				char uuid_str[40];
+
+				mpool_unparse_uuid(&pd->pdi_devid, uuid_str);
+				err = -ENXIO;
+
+				if (mpool_pd_status_get(pd) == PD_STAT_UNAVAIL)
+					mp_pr_err("mpool %s, drive %s %s %s", err, mp->pds_name,
+						   uuid_str, pd->pdi_name, msg_unavail1);
+				else
+					mp_pr_err("mpool %s, drive %s %s %s", err, mp->pds_name,
+						  uuid_str, pd->pdi_name, msg_unavail2);
+				break;
+			} else if (mpool_pd_status_get(pd) == PD_STAT_UNAVAIL) {
+				mc->mc_uacnt += 1;
+			}
+		}
+	}
+
+	/*
+	 * Now it is possible to update the percent spare because all
+	 * the media classes of the mpool have been created because all
+	 * the mpool PDs have been added in their classes.
+	 */
+	if (!err) {
+		for (mclassp = 0; mclassp < MP_MED_NUMBER; mclassp++) {
+			if (spzone[mclassp] >= 0) {
+				err = mc_set_spzone(&mp->pds_mc[mclassp], spzone[mclassp]);
+				/*
+				 * Should never happen, it should exist a class
+				 * with perf. level mclassp with a least 1 PD.
+				 */
+				if (err)
+					break;
+			}
+		}
+		if (err)
+			mp_pr_err("mpool %s, can't set spare %u because the class %u has no PD",
+				  err, mp->pds_name, spzone[mclassp], mclassp);
+	}
+
+	kfree(cdr);
+
+	return err;
+}
+
+static int pmd_objs_load(struct mpool_descriptor *mp, u8 cslot)
+{
+	struct omf_mdcrec_data *cdr = NULL;
+	struct pmd_mdc_info *cinfo;
+	struct rb_node *node;
+	u64 argv[2] = { 0 };
+	const char *msg;
+	size_t recbufsz;
+	char *recbuf;
+	u64 mdcmax;
+	int err;
+
+	/* Note: single threaded here so don't need any locks */
+
+	recbufsz = OMF_MDCREC_PACKLEN_MAX;
+	msg = "(no detail)";
+	mdcmax = 0;
+
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+	/* Initialize mdc if not mdc0. */
+	if (cslot) {
+		u64 logid1 = logid_make(2 * cslot, 0);
+		u64 logid2 = logid_make(2 * cslot + 1, 0);
+
+		/* Freed in pmd_mda_free() */
+		cinfo->mmi_recbuf = kmalloc(recbufsz, GFP_KERNEL);
+		if (!cinfo->mmi_recbuf) {
+			msg = "MDC recbuf alloc failed";
+			err = -ENOMEM;
+			goto errout;
+		}
+
+		err = mp_mdc_open(mp, logid1, logid2, MDC_OF_SKIP_SER, &cinfo->mmi_mdc);
+		if (err) {
+			msg = "mdc open failed";
+			goto errout;
+		}
+	}
+
+	/* Read mdc and capture net result of object data records. */
+	err = mp_mdc_rewind(cinfo->mmi_mdc);
+	if (err) {
+		msg = "mdc rewind failed";
+		goto errout;
+	}
+
+	/* Cache these pointers to simplify the ensuing code. */
+	recbuf = cinfo->mmi_recbuf;
+
+	cdr = kzalloc(sizeof(*cdr), GFP_KERNEL);
+	if (!cdr) {
+		msg = "cdr alloc failed";
+		goto errout;
+	}
+
+	while (true) {
+		struct pmd_layout *layout, *found;
+		size_t rlen = 0;
+		u64 objid;
+
+		err = mp_mdc_read(cinfo->mmi_mdc, recbuf, recbufsz, &rlen);
+		if (err) {
+			msg = "mdc read data failed";
+			break;
+		}
+		if (rlen == 0)
+			break; /* Hit end of log */
+
+		/*
+		 * Version record, if present, must be first.
+		 */
+		if (omf_mdcrec_unpack_type_letoh(recbuf) == OMF_MDR_VERSION) {
+			omf_mdcver_unpack_letoh(cdr, recbuf);
+			cinfo->mmi_mdcver = cdr->u.omd_version;
+
+			if (omfu_mdcver_cmp(&cinfo->mmi_mdcver, ">", omfu_mdcver_cur())) {
+				char *buf1, *buf2 = NULL;
+
+				buf1 = kmalloc(2 * MAX_MDCVERSTR, GFP_KERNEL);
+				if (buf1) {
+					buf2 = buf1 + MAX_MDCVERSTR;
+					omfu_mdcver_to_str(&cinfo->mmi_mdcver, buf1, sizeof(buf1));
+					omfu_mdcver_to_str(omfu_mdcver_cur(), buf2, sizeof(buf2));
+				}
+
+				err = -EOPNOTSUPP;
+				mp_pr_err("mpool %s, MDC%u version %s, binary version %s",
+					  err, mp->pds_name, cslot, buf1, buf2);
+				kfree(buf1);
+				break;
+			}
+			continue;
+		}
+
+		/* Skip non object-related mdcrec in mdc0; i.e., property
+		 * records.
+		 */
+		if (!cslot && !omf_mdcrec_isobj_le(recbuf))
+			continue;
+
+		err = omf_mdcrec_unpack_letoh(&cinfo->mmi_mdcver, mp, cdr, recbuf);
+		if (err) {
+			msg = "mlog record unpack failed";
+			break;
+		}
+
+		objid = cdr->u.obj.omd_objid;
+
+		if (objid_slot(objid) != cslot) {
+			msg = "mlog record wrong slot";
+			err = -EBADSLT;
+			break;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_OCREATE) {
+			layout = cdr->u.obj.omd_layout;
+			layout->eld_state = PMD_LYT_COMMITTED;
+
+			found = pmd_co_insert(cinfo, layout);
+			if (found) {
+				msg = "OCREATE duplicate object ID";
+				pmd_obj_put(layout);
+				err = -EEXIST;
+				break;
+			}
+
+			atomic_inc(&cinfo->mmi_pco_cnt.pcc_cr);
+			atomic_inc(&cinfo->mmi_pco_cnt.pcc_cobj);
+
+			continue;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_ODELETE) {
+			found = pmd_co_find(cinfo, objid);
+			if (!found) {
+				msg = "ODELETE object not found";
+				err = -ENOENT;
+				break;
+			}
+
+			pmd_co_remove(cinfo, found);
+			pmd_obj_put(found);
+
+			atomic_inc(&cinfo->mmi_pco_cnt.pcc_del);
+			atomic_dec(&cinfo->mmi_pco_cnt.pcc_cobj);
+
+			continue;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_OIDCKPT) {
+			/*
+			 * objid == mmi_lckpt == 0 is legit. Such records
+			 * are appended by mpool MDC compaction due to a
+			 * mpool metadata upgrade on an empty mpool.
+			 */
+			if ((objid_uniq(objid) || objid_uniq(cinfo->mmi_lckpt))
+				&& (objid_uniq(objid) <= objid_uniq(cinfo->mmi_lckpt))) {
+				msg = "OIDCKPT cdr ckpt %lu <= cinfo ckpt %lu";
+				argv[0] = objid_uniq(objid);
+				argv[1] = objid_uniq(cinfo->mmi_lckpt);
+				err = -EINVAL;
+				break;
+			}
+
+			cinfo->mmi_lckpt = objid;
+			continue;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_OERASE) {
+			layout = pmd_co_find(cinfo, objid);
+			if (!layout) {
+				msg = "OERASE object not found";
+				err = -ENOENT;
+				break;
+			}
+
+			/* Note: OERASE gen can equal layout gen after a compaction. */
+			if (cdr->u.obj.omd_gen < layout->eld_gen) {
+				msg = "OERASE cdr gen %lu < layout gen %lu";
+				argv[0] = cdr->u.obj.omd_gen;
+				argv[1] = layout->eld_gen;
+				err = -EINVAL;
+				break;
+			}
+
+			layout->eld_gen = cdr->u.obj.omd_gen;
+
+			atomic_inc(&cinfo->mmi_pco_cnt.pcc_er);
+			continue;
+		}
+
+		if (cdr->omd_rtype == OMF_MDR_OUPDATE) {
+			layout = cdr->u.obj.omd_layout;
+
+			found = pmd_co_find(cinfo, objid);
+			if (!found) {
+				msg = "OUPDATE object not found";
+				pmd_obj_put(layout);
+				err = -ENOENT;
+				break;
+			}
+
+			pmd_co_remove(cinfo, found);
+			pmd_obj_put(found);
+
+			layout->eld_state = PMD_LYT_COMMITTED;
+			pmd_co_insert(cinfo, layout);
+
+			atomic_inc(&cinfo->mmi_pco_cnt.pcc_up);
+
+			continue;
+		}
+	}
+
+	if (err)
+		goto errout;
+
+	/*
+	 * Add all existing objects to space map.
+	 * Also add/update per-mpool space usage stats
+	 */
+	pmd_co_foreach(cinfo, node) {
+		struct pmd_layout *layout;
+
+		layout = rb_entry(node, typeof(*layout), eld_nodemdc);
+
+		/* Remember objid and gen in case of error... */
+		cdr->u.obj.omd_objid = layout->eld_objid;
+		cdr->u.obj.omd_gen = layout->eld_gen;
+
+		if (objid_slot(layout->eld_objid) != cslot) {
+			msg = "layout wrong slot";
+			err = -EBADSLT;
+			break;
+		}
+
+		err = pmd_smap_insert(mp, layout);
+		if (err) {
+			msg = "smap insert failed";
+			break;
+		}
+
+		pmd_update_obj_stats(mp, layout, cinfo, PMD_OBJ_LOAD);
+
+		/* For mdc0 track last logical mdc created. */
+		if (!cslot)
+			mdcmax = max(mdcmax, (objid_uniq(layout->eld_objid) >> 1));
+	}
+
+	if (err)
+		goto errout;
+
+	cdr->u.obj.omd_objid = 0;
+	cdr->u.obj.omd_gen = 0;
+
+	if (!cslot) {
+		/* MDC0: finish initializing mda */
+		cinfo->mmi_luniq = mdcmax;
+		mp->pds_mda.mdi_slotvcnt = mdcmax + 1;
+
+		/* MDC0 only: validate other mdc metadata; may make adjustments to mp.mda. */
+		err = pmd_mdc0_validate(mp, 1);
+		if (err)
+			msg = "MDC0 validation failed";
+	} else {
+		/*
+		 * other mdc: set luniq to guaranteed max value
+		 * previously used and ensure next objid allocation
+		 * will be checkpointed; supports realloc of
+		 * uncommitted objects after a crash
+		 */
+		cinfo->mmi_luniq = objid_uniq(cinfo->mmi_lckpt) + OBJID_UNIQ_DELTA - 1;
+	}
+
+errout:
+	if (err) {
+		char *msgbuf;
+
+		msgbuf = kmalloc(64, GFP_KERNEL);
+		if (msgbuf)
+			snprintf(msgbuf, 64, msg, argv[0], argv[1]);
+
+		mp_pr_err("mpool %s, %s: cslot %u, ckpt %lx, %lx/%lu",
+			  err, mp->pds_name, msgbuf, cslot, (ulong)cinfo->mmi_lckpt,
+			  (ulong)cdr->u.obj.omd_objid, (ulong)cdr->u.obj.omd_gen);
+
+		kfree(msgbuf);
+	}
+
+	kfree(cdr);
+
+	return err;
+}
+
+/**
+ * pmd_objs_load_worker() -
+ * @ws:
+ *
+ * worker thread for loading user MDC 1~N
+ * Each worker instance will do the following (not counting errors):
+ * * grab an MDC number atomically from olw->olw_progress
+ * * If the MDC number is invalid, exit
+ * * load the objects from that MDC
+ *
+ * If an error occurs in this or any other worker, don't load any more MDCs
+ */
+static void pmd_objs_load_worker(struct work_struct *ws)
+{
+	struct pmd_obj_load_work *olw;
+	int sidx, rc;
+
+	olw = container_of(ws, struct pmd_obj_load_work, olw_work);
+
+	while (atomic_read(olw->olw_err) == 0) {
+		sidx = atomic_fetch_add(1, olw->olw_progress);
+		if (sidx >= olw->olw_mp->pds_mda.mdi_slotvcnt)
+			break; /* No more MDCs to load */
+
+		rc = pmd_objs_load(olw->olw_mp, sidx);
+		if (rc)
+			atomic_set(olw->olw_err, rc);
+	}
+}
+
+/**
+ * pmd_objs_load_parallel() - load MDC 1~N in parallel
+ * @mp:
+ *
+ * By loading user MDCs in parallel, we can reduce the mpool activate
+ * time, since the jobs of loading MDC 1~N are independent.
+ * On the other hand, we don't want to start all the jobs at once.
+ * If any one fails, we don't have to start others.
+ */
+static int pmd_objs_load_parallel(struct mpool_descriptor *mp)
+{
+	struct pmd_obj_load_work *olwv;
+	atomic_t err = ATOMIC_INIT(0);
+	atomic_t progress = ATOMIC_INIT(1);
+	uint njobs, inc, cpu, i;
+
+	if (mp->pds_mda.mdi_slotvcnt < 2)
+		return 0; /* No user MDCs allocated */
+
+	njobs = mp->pds_params.mp_objloadjobs;
+	njobs = clamp_t(uint, njobs, 1, mp->pds_mda.mdi_slotvcnt - 1);
+
+	if (mp->pds_mda.mdi_slotvcnt / njobs >= 4 && num_online_cpus() > njobs)
+		njobs *= 2;
+
+	olwv = kcalloc(njobs, sizeof(*olwv), GFP_KERNEL);
+	if (!olwv)
+		return -ENOMEM;
+
+	inc = (num_online_cpus() / njobs) & ~1u;
+	cpu = raw_smp_processor_id();
+
+	/*
+	 * Each of njobs workers will atomically grab MDC numbers from &progress
+	 * and load them, until all valid user MDCs have been loaded.
+	 */
+	for (i = 0; i < njobs; ++i) {
+		INIT_WORK(&olwv[i].olw_work, pmd_objs_load_worker);
+		olwv[i].olw_progress = &progress;
+		olwv[i].olw_err = &err;
+		olwv[i].olw_mp = mp;
+
+		/*
+		 * Try to distribute work across all NUMA nodes.
+		 * queue_work_node() would be preferable, but
+		 * it's not available on older kernels.
+		 */
+		cpu = (cpu + inc) % nr_cpumask_bits;
+		cpu = cpumask_next_wrap(cpu, cpu_online_mask, nr_cpumask_bits, false);
+		queue_work_on(cpu, mp->pds_workq, &olwv[i].olw_work);
+	}
+
+	/* Wait for all worker threads to complete */
+	flush_workqueue(mp->pds_workq);
+
+	kfree(olwv);
+
+	return atomic_read(&err);
+}
+
+static int pmd_mdc_append(struct mpool_descriptor *mp, u8 cslot,
+			  struct omf_mdcrec_data *cdr, int sync)
+{
+	struct pmd_mdc_info *cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	s64 plen;
+
+	plen = omf_mdcrec_pack_htole(mp, cdr, cinfo->mmi_recbuf);
+	if (plen < 0) {
+		mp_pr_warn("mpool %s, MDC%u append failed", mp->pds_name, cslot);
+		return plen;
+	}
+
+	return mp_mdc_append(cinfo->mmi_mdc, cinfo->mmi_recbuf, plen, sync);
+}
+
+/**
+ * pmd_log_all_mdc_cobjs() - write in the new active mlog the object records.
+ * @mp:
+ * @cslot:
+ * @compacted: output
+ * @total: output
+ */
+static int pmd_log_all_mdc_cobjs(struct mpool_descriptor *mp, u8 cslot,
+				 struct omf_mdcrec_data *cdr, u32 *compacted, u32 *total)
+{
+	struct pmd_mdc_info *cinfo;
+	struct pmd_layout *layout;
+	struct rb_node *node;
+	int rc;
+
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	rc = 0;
+
+	pmd_co_foreach(cinfo, node) {
+		layout = rb_entry(node, typeof(*layout), eld_nodemdc);
+
+		if (!objid_mdc0log(layout->eld_objid)) {
+			cdr->omd_rtype = OMF_MDR_OCREATE;
+			cdr->u.obj.omd_layout = layout;
+
+			rc = pmd_mdc_append(mp, cslot, cdr, 0);
+			if (rc) {
+				mp_pr_err("mpool %s, MDC%u log committed obj failed, objid 0x%lx",
+					  rc, mp->pds_name, cslot, (ulong)layout->eld_objid);
+				break;
+			}
+
+			++(*compacted);
+		}
+		++(*total);
+	}
+
+	for (; node; node = rb_next(node))
+		++(*total);
+
+	return rc;
+}
+
+/**
+ * pmd_log_mdc0_cobjs() - write in the new active mlog (of MDC0) the MDC0
+ *	records that are particular to MDC0.
+ * @mp:
+ */
+static int pmd_log_mdc0_cobjs(struct mpool_descriptor *mp)
+{
+	struct mpool_dev_info *pd;
+	int rc = 0, i;
+
+	/*
+	 * Log a drive record (OMF_MDR_MCCONFIG) for every drive in pds_pdv[]
+	 * that is not defunct.
+	 */
+	for (i = 0; i < mp->pds_pdvcnt; i++) {
+		pd = &(mp->pds_pdv[i]);
+		rc = pmd_prop_mcconfig(mp, pd, true);
+		if (rc)
+			return rc;
+	}
+
+	/*
+	 * Log a media class spare record (OMF_MDR_MCSPARE) for every media
+	 * class.
+	 * mc count can't change now. Because the MDC0 compact lock is held
+	 * and that blocks the addition of PDs in the  mpool.
+	 */
+	for (i = 0; i < MP_MED_NUMBER; i++) {
+		struct media_class *mc;
+
+		mc = &mp->pds_mc[i];
+		if (mc->mc_pdmc >= 0) {
+			rc = pmd_prop_mcspare(mp, mc->mc_parms.mcp_classp,
+					       mc->mc_sparms.mcsp_spzone, true);
+			if (rc)
+				return rc;
+		}
+	}
+
+	return pmd_prop_mpconfig(mp, &mp->pds_cfg, true);
+}
+
+/**
+ * pmd_log_non_mdc0_cobjs() - write in the new active mlog (of MDCi i>0) the
+ *	MDCi records that are particular to MDCi (not used by MDC0).
+ * @mp:
+ * @cslot:
+ */
+static int pmd_log_non_mdc0_cobjs(struct mpool_descriptor *mp, u8 cslot,
+				  struct omf_mdcrec_data *cdr)
+{
+	struct pmd_mdc_info *cinfo;
+
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+	/*
+	 * if not mdc0 log last objid checkpoint to support realloc of
+	 * uncommitted objects after a crash and to guarantee objids are
+	 * never reused.
+	 */
+	cdr->omd_rtype = OMF_MDR_OIDCKPT;
+	cdr->u.obj.omd_objid = cinfo->mmi_lckpt;
+
+	return pmd_mdc_append(mp, cslot, cdr, 0);
+}
+
+/**
+ * pmd_pre_compact_reset() - called on MDCi i>0
+ * @cinfo:
+ * @compacted: object create records appended in the new active mlog.
+ *
+ * Locking:
+ *	MDCi compact lock is held by the caller.
+ */
+static void pmd_pre_compact_reset(struct pmd_mdc_info *cinfo, u32 compacted)
+{
+	struct pre_compact_ctrs *pco_cnt;
+
+	pco_cnt = &cinfo->mmi_pco_cnt;
+	ASSERT(pco_cnt->pcc_cobj.counter == compacted);
+
+	atomic_set(&pco_cnt->pcc_cr, compacted);
+	atomic_set(&pco_cnt->pcc_cobj, compacted);
+	atomic_set(&pco_cnt->pcc_up, 0);
+	atomic_set(&pco_cnt->pcc_del, 0);
+	atomic_set(&pco_cnt->pcc_er, 0);
+}
+
+/**
+ * pmd_mdc_compact() - compact an mpool MDCi with i >= 0.
+ * @mp:
+ * @cslot: the "i" of MDCi
+ *
+ * Locking:
+ * 1) caller must hold MDCi compact lock
+ * 2) MDC compaction freezes the state of all MDCs objects [and for MDC0
+ *    also freezes all mpool properties] by simply holding MDC
+ *    mmi_compactlock mutex. Hence, MDC compaction does not need to
+ *    read-lock individual object layouts or mpool property data
+ *    structures to read them. It is why this function and its callees don't
+ *    take any lock.
+ *
+ * Note: this function or its callees must call pmd_mdc_append() with no sync
+ *	instead of pmd_mdc_addrec() to avoid triggering nested compaction of
+ *	a same MDCi.
+ *	The sync/flush is done by append of cend, no need to sync before that.
+ */
+static int pmd_mdc_compact(struct mpool_descriptor *mp, u8 cslot)
+{
+	struct pmd_mdc_info *cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	u64 logid1 = logid_make(2 * cslot, 0);
+	u64 logid2 = logid_make(2 * cslot + 1, 0);
+	struct omf_mdcrec_data *cdr;
+	int retry = 0, rc = 0;
+
+	cdr = kzalloc(sizeof(*cdr), GFP_KERNEL);
+	if (!cdr) {
+		rc = -ENOMEM;
+		mp_pr_crit("mpool %s, alloc failure during compact", rc, mp->pds_name);
+		return rc;
+	}
+
+	for (retry = 0; retry < MPOOL_MDC_COMPACT_RETRY_DEFAULT; retry++) {
+		u32 compacted = 0;
+		u32 total = 0;
+
+		if (rc) {
+			rc = mp_mdc_open(mp, logid1, logid2, MDC_OF_SKIP_SER, &cinfo->mmi_mdc);
+			if (rc)
+				continue;
+		}
+
+		mp_pr_debug("mpool %s, MDC%u start: mlog1 gen %lu mlog2 gen %lu",
+			    rc, mp->pds_name, cslot,
+			    (ulong)((struct pmd_layout *)cinfo->mmi_mdc->mdc_logh1)->eld_gen,
+			    (ulong)((struct pmd_layout *)cinfo->mmi_mdc->mdc_logh2)->eld_gen);
+
+		rc = mp_mdc_cstart(cinfo->mmi_mdc);
+		if (rc)
+			continue;
+
+		if (omfu_mdcver_cmp2(omfu_mdcver_cur(), ">=", 1, 0, 0, 1)) {
+			rc = pmd_mdc_addrec_version(mp, cslot);
+			if (rc) {
+				mp_mdc_close(cinfo->mmi_mdc);
+				continue;
+			}
+		}
+
+		if (cslot)
+			rc = pmd_log_non_mdc0_cobjs(mp, cslot, cdr);
+		else
+			rc = pmd_log_mdc0_cobjs(mp);
+		if (rc)
+			continue;
+
+		rc = pmd_log_all_mdc_cobjs(mp, cslot, cdr, &compacted, &total);
+
+		mp_pr_debug("mpool %s, MDC%u compacted %u of %u objects: retry=%d",
+			    rc, mp->pds_name, cslot, compacted, total, retry);
+
+		/*
+		 * Append the compaction end record in the new active
+		 * mlog, and flush/sync all the previous records
+		 * appended in the new active log by the compaction
+		 * above.
+		 */
+		if (!rc)
+			rc = mp_mdc_cend(cinfo->mmi_mdc);
+
+		if (!rc) {
+			if (cslot) {
+				/*
+				 * MDCi i>0 compacted successfully
+				 * MDCi compact lock is held.
+				 */
+				pmd_pre_compact_reset(cinfo, compacted);
+			}
+
+			mp_pr_debug("mpool %s, MDC%u end: mlog1 gen %lu mlog2 gen %lu",
+				    rc, mp->pds_name, cslot,
+				    (ulong)((struct pmd_layout *)
+					    cinfo->mmi_mdc->mdc_logh1)->eld_gen,
+				    (ulong)((struct pmd_layout *)
+					    cinfo->mmi_mdc->mdc_logh2)->eld_gen);
+			break;
+		}
+	}
+
+	if (rc)
+		mp_pr_crit("mpool %s, MDC%u compaction failed", rc, mp->pds_name, cslot);
+
+	kfree(cdr);
+
+	return rc;
+}
+
+static int pmd_mdc_addrec(struct mpool_descriptor *mp, u8 cslot, struct omf_mdcrec_data *cdr)
+{
+	int rc;
+
+	rc = pmd_mdc_append(mp, cslot, cdr, 1);
+
+	if (rc == -EFBIG) {
+		rc = pmd_mdc_compact(mp, cslot);
+		if (!rc)
+			rc = pmd_mdc_append(mp, cslot, cdr, 1);
+	}
+
+	if (rc)
+		mp_pr_rl("mpool %s, MDC%u append failed%s", rc, mp->pds_name, cslot,
+			 (rc == -EFBIG) ? " post compaction" : "");
+
+	return rc;
+}
+
+int pmd_mdc_addrec_version(struct mpool_descriptor *mp, u8 cslot)
+{
+	struct omf_mdcrec_data cdr;
+	struct omf_mdcver *ver;
+
+	cdr.omd_rtype = OMF_MDR_VERSION;
+
+	ver = omfu_mdcver_cur();
+	cdr.u.omd_version = *ver;
+
+	return pmd_mdc_addrec(mp, cslot, &cdr);
+}
+
+int pmd_prop_mcconfig(struct mpool_descriptor *mp, struct mpool_dev_info *pd, bool compacting)
+{
+	struct omf_mdcrec_data cdr;
+	struct mc_parms mc_parms;
+
+	cdr.omd_rtype = OMF_MDR_MCCONFIG;
+	mpool_uuid_copy(&cdr.u.dev.omd_parm.odp_devid, &pd->pdi_devid);
+	mc_pd_prop2mc_parms(&pd->pdi_parm.dpr_prop, &mc_parms);
+	mc_parms2omf_devparm(&mc_parms, &cdr.u.dev.omd_parm);
+	cdr.u.dev.omd_parm.odp_zonetot = pd->pdi_parm.dpr_zonetot;
+	cdr.u.dev.omd_parm.odp_devsz = pd->pdi_parm.dpr_devsz;
+
+	/* If compacting no sync needed and don't trigger another compaction. */
+	if (compacting)
+		return pmd_mdc_append(mp, 0, &cdr, 0);
+
+	return pmd_mdc_addrec(mp, 0, &cdr);
+}
+
+int pmd_prop_mcspare(struct mpool_descriptor *mp, enum mp_media_classp mclassp,
+		     u8 spzone, bool compacting)
+{
+	struct omf_mdcrec_data cdr;
+	int rc;
+
+	if (!mclass_isvalid(mclassp) || spzone > 100) {
+		rc = -EINVAL;
+		mp_pr_err("persisting %s spare zone info, invalid arguments %d %u",
+			  rc, mp->pds_name, mclassp, spzone);
+		return rc;
+	}
+
+	cdr.omd_rtype = OMF_MDR_MCSPARE;
+	cdr.u.mcs.omd_mclassp = mclassp;
+	cdr.u.mcs.omd_spzone = spzone;
+
+	/* If compacting no sync needed and don't trigger another compaction. */
+	if (compacting)
+		return pmd_mdc_append(mp, 0, &cdr, 0);
+
+	return pmd_mdc_addrec(mp, 0, &cdr);
+}
+
+int pmd_log_delete(struct mpool_descriptor *mp, u64 objid)
+{
+	struct omf_mdcrec_data cdr;
+
+	cdr.omd_rtype = OMF_MDR_ODELETE;
+	cdr.u.obj.omd_objid = objid;
+
+	return pmd_mdc_addrec(mp, objid_slot(objid), &cdr);
+}
+
+int pmd_log_create(struct mpool_descriptor *mp, struct pmd_layout *layout)
+{
+	struct omf_mdcrec_data cdr;
+
+	cdr.omd_rtype = OMF_MDR_OCREATE;
+	cdr.u.obj.omd_layout = layout;
+
+	return pmd_mdc_addrec(mp, objid_slot(layout->eld_objid), &cdr);
+}
+
+int pmd_log_erase(struct mpool_descriptor *mp, u64 objid, u64 gen)
+{
+	struct omf_mdcrec_data cdr;
+
+	cdr.omd_rtype = OMF_MDR_OERASE;
+	cdr.u.obj.omd_objid = objid;
+	cdr.u.obj.omd_gen = gen;
+
+	return pmd_mdc_addrec(mp, objid_slot(objid), &cdr);
+}
+
+int pmd_log_idckpt(struct mpool_descriptor *mp, u64 objid)
+{
+	struct omf_mdcrec_data cdr;
+
+	cdr.omd_rtype = OMF_MDR_OIDCKPT;
+	cdr.u.obj.omd_objid = objid;
+
+	return pmd_mdc_addrec(mp, objid_slot(objid), &cdr);
+}
+
+int pmd_prop_mpconfig(struct mpool_descriptor *mp, const struct mpool_config *cfg, bool compacting)
+{
+	struct omf_mdcrec_data cdr = { };
+
+	cdr.omd_rtype = OMF_MDR_MPCONFIG;
+	cdr.u.omd_cfg = *cfg;
+
+	if (compacting)
+		return pmd_mdc_append(mp, 0, &cdr, 0);
+
+	return pmd_mdc_addrec(mp, 0, &cdr);
+}
+
+/**
+ * pmd_need_compact() - determine if MDCi corresponding to cslot need compaction of not.
+ * @mp:
+ * @cslot:
+ *
+ * The MDCi needs compaction if the active mlog is above some threshold and
+ * if there is enough garbage (that can be eliminated by the compaction).
+ *
+ * Locking: not lock need to be held when calling this function.
+ *	as a result of not holding lock the result may be off if a compaction
+ *	of MDCi (with i = cslot) is taking place at the same time.
+ */
+static bool pmd_need_compact(struct mpool_descriptor *mp, u8 cslot, char *msgbuf, size_t msgsz)
+{
+	struct pre_compact_ctrs *pco_cnt;
+	struct pmd_mdc_info *cinfo;
+	u64 rec, cobj, len, cap;
+	u32 garbage, pct;
+
+	ASSERT(cslot > 0);
+
+	cinfo = &mp->pds_mda.mdi_slotv[cslot];
+	pco_cnt = &(cinfo->mmi_pco_cnt);
+
+	cap = atomic64_read(&pco_cnt->pcc_cap);
+	if (cap == 0)
+		return false; /* MDC closed for now. */
+
+	len = atomic64_read(&pco_cnt->pcc_len);
+	rec = atomic_read(&pco_cnt->pcc_cr) + atomic_read(&pco_cnt->pcc_up) +
+		atomic_read(&pco_cnt->pcc_del) + atomic_read(&pco_cnt->pcc_er);
+	cobj = atomic_read(&pco_cnt->pcc_cobj);
+
+	pct = (len * 100) / cap;
+	if (pct < mp->pds_params.mp_pcopctfull)
+		return false; /* Active mlog not filled enough */
+
+	if (rec > cobj) {
+		garbage = (rec - cobj) * 100;
+		garbage /= rec;
+	} else {
+
+		/*
+		 * We may arrive here rarely if the caller doesn't
+		 * hold the compact lock. In that case, the update of
+		 * the counters may be seen out of order or a compaction
+		 * may take place at the same time.
+		 */
+		garbage = 0;
+	}
+
+	if (garbage < mp->pds_params.mp_pcopctgarbage)
+		return false;
+
+	if (msgbuf)
+		snprintf(msgbuf, msgsz,
+			 "bytes used %lu, total %lu, pct %u, records %lu, objects %lu, garbage %u",
+			 (ulong)len, (ulong)cap, pct, (ulong)rec, (ulong)cobj, garbage);
+
+	return true;
+}
+
+/**
+ * pmd_mdc_needed() - determines if new MDCns should be created
+ * @mp:  mpool descriptor
+ *
+ * New MDC's are created if total free space across all MDC's
+ * is above a threshold value and the garbage to reclaim space
+ * is below a garbage threshold.
+ *
+ * Locking: no lock needs to be held when calling this function.
+ *
+ * NOTES:
+ * - Skip non-active MDC
+ * - Accumulate total capacity, total garbage and total in-use capacity
+ *   across all active MDCs.
+ * - Return true if total used capacity across all MDCs is threshold and
+ *   garbage is < a threshold that would yield significant free space upon
+ *   compaction.
+ */
+static bool pmd_mdc_needed(struct mpool_descriptor *mp)
+{
+	struct pre_compact_ctrs *pco_cnt;
+	struct pmd_mdc_info *cinfo;
+	u64 cap, tcap, used, garbage, record, rec, cobj;
+	u32 pct, pctg, mdccnt;
+	u16 cslot;
+
+	ASSERT(mp->pds_mda.mdi_slotvcnt <= MDC_SLOTS);
+
+	cap = used = garbage = record = pctg = 0;
+
+	if (mp->pds_mda.mdi_slotvcnt == MDC_SLOTS)
+		return false;
+
+	for (cslot = 1, mdccnt = 0; cslot < mp->pds_mda.mdi_slotvcnt; cslot++) {
+
+		cinfo = &mp->pds_mda.mdi_slotv[cslot];
+		pco_cnt = &(cinfo->mmi_pco_cnt);
+
+		tcap = atomic64_read(&pco_cnt->pcc_cap);
+		if (tcap == 0) {
+			/*
+			 * MDC closed for now and will not be considered
+			 * in making a decision to create new MDC.
+			 */
+			mp_pr_warn("MDC %u not open", cslot);
+			continue;
+		}
+		cap += tcap;
+
+		mdccnt++;
+
+		used += atomic64_read(&pco_cnt->pcc_len);
+		rec = atomic_read(&pco_cnt->pcc_cr) + atomic_read(&pco_cnt->pcc_up) +
+			atomic_read(&pco_cnt->pcc_del) + atomic_read(&pco_cnt->pcc_er);
+
+		cobj = atomic_read(&pco_cnt->pcc_cobj);
+
+		if (rec > cobj)
+			garbage += (rec - cobj);
+
+		record += rec;
+	}
+
+	if (mdccnt == 0) {
+		mp_pr_warn("No mpool MDCs available");
+		return false;
+	}
+
+	/* Percentage capacity used across all MDCs */
+	pct  = (used  * 100) / cap;
+
+	/* Percentage garbage available across all MDCs */
+	if (garbage)
+		pctg = (garbage * 100) / record;
+
+	if (pct > mp->pds_params.mp_crtmdcpctfull && pctg < mp->pds_params.mp_crtmdcpctgrbg) {
+		mp_pr_debug("MDCn %u cap %u used %u rec %u grbg %u pct used %u grbg %u Thres %u-%u",
+			    0, mdccnt, (u32)cap, (u32)used, (u32)record, (u32)garbage, pct, pctg,
+			    (u32)mp->pds_params.mp_crtmdcpctfull,
+			    (u32)mp->pds_params.mp_crtmdcpctgrbg);
+		return true;
+	}
+
+	return false;
+}
+
+
+/**
+ * pmd_precompact() - precompact an mpool MDC
+ * @work:
+ *
+ * The goal of this thread is to minimize the application objects commit time.
+ * This thread pre compacts the MDC1/255. As a consequence MDC1/255 compaction
+ * does not occurs in the context of an application object commit.
+ */
+static void pmd_precompact(struct work_struct *work)
+{
+	struct pre_compact_ctrl *pco;
+	struct mpool_descriptor *mp;
+	struct pmd_mdc_info *cinfo;
+	char msgbuf[128];
+	uint nmtoc, delay;
+	bool compact;
+	u8 cslot;
+
+	pco = container_of(work, typeof(*pco), pco_dwork.work);
+	mp = pco->pco_mp;
+
+	nmtoc = atomic_fetch_add(1, &pco->pco_nmtoc);
+
+	/* Only compact MDC1/255 not MDC0. */
+	cslot = (nmtoc % (mp->pds_mda.mdi_slotvcnt - 1)) + 1;
+
+	/*
+	 * Check if the next mpool mdc to compact needs compaction.
+	 *
+	 * Note that this check is done without taking any lock.
+	 * This is safe because the mpool MDCs don't go away as long as
+	 * the mpool is activated. The mpool can't deactivate before
+	 * this thread exit.
+	 */
+	compact = pmd_need_compact(mp, cslot, NULL, 0);
+	if (compact) {
+		cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+		/*
+		 * Check a second time while we hold the compact lock
+		 * to avoid doing a useless compaction.
+		 */
+		pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
+		compact = pmd_need_compact(mp, cslot, msgbuf, sizeof(msgbuf));
+		if (compact)
+			pmd_mdc_compact(mp, cslot);
+		pmd_mdc_unlock(&cinfo->mmi_compactlock);
+
+		if (compact)
+			mp_pr_info("mpool %s, MDC%u %s", mp->pds_name, cslot, msgbuf);
+	}
+
+	/* If running low on MDC space create new MDCs */
+	if (pmd_mdc_needed(mp))
+		pmd_mdc_alloc_set(mp);
+
+	pmd_update_credit(mp);
+
+	delay = clamp_t(uint, mp->pds_params.mp_pcoperiod, 1, 3600);
+
+	queue_delayed_work(mp->pds_workq, &pco->pco_dwork, msecs_to_jiffies(delay * 1000));
+}
+
+void pmd_precompact_start(struct mpool_descriptor *mp)
+{
+	struct pre_compact_ctrl *pco;
+
+	pco = &mp->pds_pco;
+	pco->pco_mp = mp;
+	atomic_set(&pco->pco_nmtoc, 0);
+
+	INIT_DELAYED_WORK(&pco->pco_dwork, pmd_precompact);
+	queue_delayed_work(mp->pds_workq, &pco->pco_dwork, 1);
+}
+
+void pmd_precompact_stop(struct mpool_descriptor *mp)
+{
+	cancel_delayed_work_sync(&mp->pds_pco.pco_dwork);
+}
+
+static int pmd_write_meta_to_latest_version(struct mpool_descriptor *mp, bool permitted)
+{
+	struct pmd_mdc_info *cinfo_converted = NULL, *cinfo;
+	char buf1[MAX_MDCVERSTR] __maybe_unused;
+	char buf2[MAX_MDCVERSTR] __maybe_unused;
+	u32 cslot;
+	int rc;
+
+	/*
+	 * Compact MDC0 first (before MDC1-255 compaction appends in MDC0) to
+	 * avoid having a potential mix of new and old records in MDC0.
+	 */
+	for (cslot = 0; cslot < mp->pds_mda.mdi_slotvcnt; cslot++) {
+		cinfo = &mp->pds_mda.mdi_slotv[cslot];
+
+		/*
+		 * At that point the version on media should be smaller or
+		 * equal to the latest version supported by this binary.
+		 * If it is not the case, the activate fails earlier.
+		 */
+		if (omfu_mdcver_cmp(&cinfo->mmi_mdcver, "==", omfu_mdcver_cur()))
+			continue;
+
+		omfu_mdcver_to_str(&cinfo->mmi_mdcver, buf1, sizeof(buf1));
+		omfu_mdcver_to_str(omfu_mdcver_cur(), buf2, sizeof(buf2));
+
+		if (!permitted) {
+			rc = -EPERM;
+			mp_pr_err("mpool %s, MDC%u upgrade needed from version %s to %s",
+				  rc, mp->pds_name, cslot, buf1, buf2);
+			return rc;
+		}
+
+		mp_pr_info("mpool %s, MDC%u upgraded from version %s to %s",
+			   mp->pds_name, cslot, buf1, buf2);
+
+		cinfo_converted = cinfo;
+
+		pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
+		rc = pmd_mdc_compact(mp, cslot);
+		pmd_mdc_unlock(&cinfo->mmi_compactlock);
+
+		if (rc) {
+			mp_pr_err("mpool %s, failed to compact MDC %u post upgrade from %s to %s",
+				  rc, mp->pds_name, cslot, buf1, buf2);
+			return rc;
+		}
+	}
+
+	if (cinfo_converted != NULL)
+		mp_pr_info("mpool %s, converted MDC from version %s to %s", mp->pds_name,
+			   omfu_mdcver_to_str(&cinfo_converted->mmi_mdcver, buf1, sizeof(buf1)),
+			   omfu_mdcver_to_str(omfu_mdcver_cur(), buf2, sizeof(buf2)));
+
+	return 0;
+}
+
+void pmd_mdc_cap(struct mpool_descriptor *mp, u64 *mdcmax, u64 *mdccap, u64 *mdc0cap)
+{
+	struct pmd_mdc_info *cinfo = NULL;
+	struct pmd_layout *layout = NULL;
+	struct rb_node *node = NULL;
+	u64 mlogsz;
+	u32 zonepg = 0;
+	u16 mdcn = 0;
+
+	if (!mdcmax || !mdccap || !mdc0cap)
+		return;
+
+	/* Serialize to prevent race with pmd_mdc_alloc() */
+	mutex_lock(&pmd_s_lock);
+
+	/*
+	 * exclude mdc0 from stats because not used for mpool user
+	 * object metadata
+	 */
+	cinfo = &mp->pds_mda.mdi_slotv[0];
+
+	pmd_mdc_lock(&cinfo->mmi_uqlock, 0);
+	*mdcmax = cinfo->mmi_luniq;
+	pmd_mdc_unlock(&cinfo->mmi_uqlock);
+
+	/*  Taking compactlock to freeze all object layout metadata in mdc0 */
+	pmd_mdc_lock(&cinfo->mmi_compactlock, 0);
+	pmd_co_rlock(cinfo, 0);
+
+	pmd_co_foreach(cinfo, node) {
+		layout = rb_entry(node, typeof(*layout), eld_nodemdc);
+
+		mdcn = objid_uniq(layout->eld_objid) >> 1;
+
+		if (mdcn > *mdcmax)
+			/* Ignore detritus from failed pmd_mdc_alloc() */
+			continue;
+
+		zonepg = mp->pds_pdv[layout->eld_ld.ol_pdh].pdi_parm.dpr_zonepg;
+		mlogsz = (layout->eld_ld.ol_zcnt * zonepg) << PAGE_SHIFT;
+
+		if (!mdcn)
+			*mdc0cap = *mdc0cap + mlogsz;
+		else
+			*mdccap = *mdccap + mlogsz;
+	}
+
+	pmd_co_runlock(cinfo);
+	pmd_mdc_unlock(&cinfo->mmi_compactlock);
+	mutex_unlock(&pmd_s_lock);
+
+	/* Only count capacity of one mlog in each mdc mlog pair */
+	*mdccap  = *mdccap >> 1;
+	*mdc0cap = *mdc0cap >> 1;
+}
+
+int pmd_mpool_activate(struct mpool_descriptor *mp, struct pmd_layout *mdc01,
+		       struct pmd_layout *mdc02, int create)
+{
+	int rc;
+
+	mp_pr_debug("mdc01: %lu mdc02: %lu", 0, (ulong)mdc01->eld_objid, (ulong)mdc02->eld_objid);
+
+	/* Activation is intense; serialize it when have multiple mpools */
+	mutex_lock(&pmd_s_lock);
+
+	/* Init metadata array for mpool */
+	pmd_mda_init(mp);
+
+	/* Initialize mdc0 for mpool */
+	rc = pmd_mdc0_init(mp, mdc01, mdc02);
+	if (rc) {
+		/*
+		 * pmd_mda_free() will dealloc mdc01/2 on subsequent
+		 * activation failures
+		 */
+		pmd_obj_put(mdc01);
+		pmd_obj_put(mdc02);
+		goto exit;
+	}
+
+	/* Load mpool properties from mdc0 including drive list and states */
+	if (!create) {
+		rc = pmd_props_load(mp);
+		if (rc)
+			goto exit;
+	}
+
+	/*
+	 * initialize smaps for all drives in mpool (now that list
+	 * is finalized)
+	 */
+	rc = smap_mpool_init(mp);
+	if (rc)
+		goto exit;
+
+	/* Load mdc layouts from mdc0 and finalize mda initialization */
+	rc = pmd_objs_load(mp, 0);
+	if (rc)
+		goto exit;
+
+	/* Load user object layouts from all other mdc */
+	rc = pmd_objs_load_parallel(mp);
+	if (rc) {
+		mp_pr_err("mpool %s, failed to load user MDCs", rc, mp->pds_name);
+		goto exit;
+	}
+
+	/*
+	 * If the format of the mpool metadata read from media during activate
+	 * is not the latest, it is time to write the metadata on media with
+	 * the latest format.
+	 */
+	if (!create) {
+		rc = pmd_write_meta_to_latest_version(mp, true);
+		if (rc) {
+			mp_pr_err("mpool %s, failed to compact MDCs (metadata conversion)",
+				  rc, mp->pds_name);
+			goto exit;
+		}
+	}
+exit:
+	if (rc) {
+		/* Activation failed; cleanup */
+		pmd_mda_free(mp);
+		smap_mpool_free(mp);
+	}
+
+	mutex_unlock(&pmd_s_lock);
+
+	return rc;
+}
+
+void pmd_mpool_deactivate(struct mpool_descriptor *mp)
+{
+	/* Deactivation is intense; serialize it when have multiple mpools */
+	mutex_lock(&pmd_s_lock);
+
+	/* Close all open user (non-mdc) mlogs */
+	mlogutil_closeall(mp);
+
+	pmd_mda_free(mp);
+	smap_mpool_free(mp);
+
+	mutex_unlock(&pmd_s_lock);
+}
diff --git a/drivers/mpool/pmd_obj.c b/drivers/mpool/pmd_obj.c
index 8966fc0abd0e..18157fecccfb 100644
--- a/drivers/mpool/pmd_obj.c
+++ b/drivers/mpool/pmd_obj.c
@@ -507,9 +507,7 @@ int pmd_obj_commit(struct mpool_descriptor *mp, struct pmd_layout *layout)
 
 	pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
 
-#ifdef OBJ_PERSISTENCE_ENABLED
 	rc = pmd_log_create(mp, layout);
-#endif
 	if (!rc) {
 		pmd_uc_lock(cinfo, cslot);
 		found = pmd_uc_remove(cinfo, layout);
@@ -675,9 +673,7 @@ int pmd_obj_delete(struct mpool_descriptor *mp, struct pmd_layout *layout)
 		return (refcnt > 2) ? -EBUSY : -EINVAL;
 	}
 
-#ifdef OBJ_PERSISTENCE_ENABLED
 	rc = pmd_log_delete(mp, objid);
-#endif
 	if (!rc) {
 		pmd_co_wlock(cinfo, cslot);
 		found = pmd_co_remove(cinfo, layout);
@@ -763,9 +759,7 @@ int pmd_obj_erase(struct mpool_descriptor *mp, struct pmd_layout *layout, u64 ge
 
 		pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
 
-#ifdef OBJ_PERSISTENCE_ENABLED
 		rc = pmd_log_erase(mp, layout->eld_objid, gen);
-#endif
 		if (!rc) {
 			layout->eld_gen = gen;
 			if (cslot)
@@ -830,9 +824,7 @@ static int pmd_alloc_idgen(struct mpool_descriptor *mp, enum obj_type_omf otype,
 		 * to prevent a race with mdc compaction.
 		 */
 		pmd_mdc_lock(&cinfo->mmi_compactlock, cslot);
-#ifdef OBJ_PERSISTENCE_ENABLED
 		rc = pmd_log_idckpt(mp, *objid);
-#endif
 		if (!rc)
 			cinfo->mmi_lckpt = *objid;
 		pmd_mdc_unlock(&cinfo->mmi_compactlock);
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:27 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:27 -0500
Subject: [PATCH v2 13/22] mpool: add utility routines for mpool lifecycle
 management
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-14-nmeeramohide@micron.com>

This adds utility routines to:

- Create and initialize a media class with an mpool volume
- Initialize and validate superblocks on all media class
  volumes
- Open and initialize all media class volumes
- Allocate metadata container 0 (MDC0) and update the
  superblock on capacity media class volume with metadata for
  accessing MDC0
- Create and initialize root MDC
- Initialize mpool descriptor and track the mapping between an
  mpool UUID and its descriptor in a rbtree

When an mpool is created, a pair of mlogs are instantiated with
well-known OIDs comprising the root MDC of the mpool. The root
MDC provides a location for mpool clients to store whatever
metadata they need for start-up.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mpcore.c | 987 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 987 insertions(+)
 create mode 100644 drivers/mpool/mpcore.c

diff --git a/drivers/mpool/mpcore.c b/drivers/mpool/mpcore.c
new file mode 100644
index 000000000000..246baedcdcec
--- /dev/null
+++ b/drivers/mpool/mpcore.c
@@ -0,0 +1,987 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * Media pool (mpool) manager module.
+ *
+ * Defines functions to create and maintain mpools comprising multiple drives
+ * in multiple media classes used for storing mblocks and mlogs.
+ */
+
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <linux/sort.h>
+#include <linux/slab.h>
+#include <linux/kref.h>
+#include <linux/rbtree.h>
+
+#include "mpool_ioctl.h"
+
+#include "mpool_printk.h"
+#include "assert.h"
+#include "uuid.h"
+
+#include "mp.h"
+#include "omf.h"
+#include "omf_if.h"
+#include "pd.h"
+#include "smap.h"
+#include "mclass.h"
+#include "pmd_obj.h"
+#include "mpcore.h"
+#include "sb.h"
+#include "upgrade.h"
+
+struct omf_devparm_descriptor;
+struct mpool_descriptor;
+
+/* Rbtree mapping mpool UUID to mpool descriptor node: uuid_to_mpdesc_rb */
+struct rb_root mpool_pools = { NULL };
+
+int uuid_to_mpdesc_insert(struct rb_root *root, struct mpool_descriptor *data)
+{
+	struct rb_node **new = &(root->rb_node), *parent = NULL;
+
+	/* Figure out where to put new node */
+	while (*new) {
+		struct mpool_descriptor *this = rb_entry(*new, struct mpool_descriptor, pds_node);
+
+		int result = mpool_uuid_compare(&data->pds_poolid, &this->pds_poolid);
+
+		parent = *new;
+		if (result < 0)
+			new = &((*new)->rb_left);
+		else if (result > 0)
+			new = &((*new)->rb_right);
+		else
+			return false;
+	}
+
+	/* Add new node and rebalance tree. */
+	rb_link_node(&data->pds_node, parent, new);
+	rb_insert_color(&data->pds_node, root);
+
+	return true;
+}
+
+static struct mpool_descriptor *
+uuid_to_mpdesc_search(struct rb_root *root, struct mpool_uuid *key_uuid)
+{
+	struct rb_node *node = root->rb_node;
+
+	while (node) {
+		struct mpool_descriptor *data = rb_entry(node, struct mpool_descriptor, pds_node);
+
+		int  result = mpool_uuid_compare(key_uuid, &data->pds_poolid);
+
+		if (result < 0)
+			node = node->rb_left;
+		else if (result > 0)
+			node = node->rb_right;
+		else
+			return data;
+	}
+	return NULL;
+}
+
+int mpool_dev_sbwrite(struct mpool_descriptor *mp, struct mpool_dev_info *pd,
+		      struct omf_sb_descriptor *sbmdc0)
+{
+	struct omf_sb_descriptor *sb = NULL;
+	struct mc_parms mc_parms;
+	int rc;
+
+	if (mpool_pd_status_get(pd) != PD_STAT_ONLINE) {
+		rc = -EIO;
+		mp_pr_err("%s:%s unavailable or offline, status %d",
+			  rc, mp->pds_name, pd->pdi_name, mpool_pd_status_get(pd));
+		return rc;
+	}
+
+	sb = kzalloc(sizeof(struct omf_sb_descriptor), GFP_KERNEL);
+	if (!sb) {
+		rc = -ENOMEM;
+		mp_pr_err("mpool %s, writing superblock on drive %s, alloc of superblock descriptor failed %lu",
+			  rc, mp->pds_name, pd->pdi_name, sizeof(struct omf_sb_descriptor));
+		return rc;
+	}
+
+	/*
+	 * Set superblock values common to all new drives in pool
+	 * (new or extant)
+	 */
+	sb->osb_magic = OMF_SB_MAGIC;
+	strlcpy((char *) sb->osb_name, mp->pds_name, sizeof(sb->osb_name));
+	sb->osb_vers = OMF_SB_DESC_VER_LAST;
+	mpool_uuid_copy(&sb->osb_poolid, &mp->pds_poolid);
+	sb->osb_gen = 1;
+
+	/* Set superblock values specific to this drive */
+	mpool_uuid_copy(&sb->osb_parm.odp_devid, &pd->pdi_devid);
+	sb->osb_parm.odp_devsz = pd->pdi_parm.dpr_devsz;
+	sb->osb_parm.odp_zonetot = pd->pdi_parm.dpr_zonetot;
+	mc_pd_prop2mc_parms(&pd->pdi_parm.dpr_prop, &mc_parms);
+	mc_parms2omf_devparm(&mc_parms, &sb->osb_parm);
+
+	if (sbmdc0)
+		sbutil_mdc0_copy(sb, sbmdc0);
+	else
+		sbutil_mdc0_clear(sb);
+
+	rc = sb_write_new(&pd->pdi_parm, sb);
+	if (rc) {
+		mp_pr_err("mpool %s, writing superblock on drive %s, write failed",
+			  rc, mp->pds_name, pd->pdi_name);
+	}
+
+	kfree(sb);
+	return rc;
+}
+
+/**
+ * mpool_mdc0_alloc() - Allocate space for the two MDC0 mlogs
+ * @mp:
+ * @sb:
+ *
+ * In the context of a mpool create, allocate space for the two MDC0 mlogs
+ *	and update the sb structure with the position of MDC0.
+ *
+ * Note: this function assumes that the media classes have already been
+ *	created.
+ */
+static int mpool_mdc0_alloc(struct mpool_descriptor *mp, struct omf_sb_descriptor *sb)
+{
+	struct mpool_dev_info *pd;
+	struct media_class *mc;
+	struct mpool_uuid uuid;
+	u64 zcnt, zonelen;
+	u32 cnt;
+	int rc;
+
+	sbutil_mdc0_clear(sb);
+
+	ASSERT(mp->pds_mdparm.md_mclass < MP_MED_NUMBER);
+
+	mc = &mp->pds_mc[mp->pds_mdparm.md_mclass];
+	if (mc->mc_pdmc < 0) {
+		rc = -ENOSPC;
+		mp_pr_err("%s: sb update memory image MDC0 information, not enough drives",
+			  rc, mp->pds_name);
+		return rc;
+	}
+
+	pd = &mp->pds_pdv[mc->mc_pdmc];
+
+	zonelen = (u64)pd->pdi_parm.dpr_zonepg << PAGE_SHIFT;
+	zcnt = 1 + ((mp->pds_params.mp_mdc0cap - 1) / zonelen);
+
+	cnt = sb_zones_for_sbs(&(pd->pdi_prop));
+	if (cnt < 1) {
+		rc = -EINVAL;
+		mp_pr_err("%s: sb MDC0, getting sb range failed for drive %s %u",
+			  rc, mp->pds_name, pd->pdi_name, cnt);
+		return rc;
+	}
+
+	if ((pd->pdi_zonetot - cnt) < zcnt * 2) {
+		rc = -ENOSPC;
+		mp_pr_err("%s: sb MDC0, no room for MDC0 on drive %s %lu %u %lu",
+			  rc, mp->pds_name, pd->pdi_name,
+			  (ulong)pd->pdi_zonetot, cnt, (ulong)zcnt);
+		return rc;
+	}
+
+	/*
+	 * mdc0 log1/2 alloced on first 2 * zcnt zone's
+	 */
+	rc = pd_zone_erase(&pd->pdi_parm, cnt, zcnt * 2, true);
+	if (rc) {
+		mp_pr_err("%s: sb MDC0, erase failed on %s %u %lu",
+			  rc, mp->pds_name, pd->pdi_name, cnt, (ulong)zcnt);
+		return rc;
+	}
+
+	/*
+	 * Fill in common mdc0 log1/2 and drive info.
+	 */
+	sb->osb_mdc01gen = 1;
+	sb->osb_mdc01desc.ol_zcnt = zcnt;
+	mpool_generate_uuid(&uuid);
+	mpool_uuid_copy(&sb->osb_mdc01uuid, &uuid);
+
+	sb->osb_mdc02gen = 2;
+	sb->osb_mdc02desc.ol_zcnt = zcnt;
+	mpool_generate_uuid(&uuid);
+	mpool_uuid_copy(&sb->osb_mdc02uuid, &uuid);
+
+	mpool_uuid_copy(&sb->osb_mdc01devid, &pd->pdi_devid);
+	sb->osb_mdc01desc.ol_zaddr = cnt;
+
+	mpool_uuid_copy(&sb->osb_mdc02devid, &pd->pdi_devid);
+	sb->osb_mdc02desc.ol_zaddr = cnt + zcnt;
+
+	mpool_uuid_copy(&sb->osb_mdc0dev.odp_devid, &pd->pdi_devid);
+	sb->osb_mdc0dev.odp_devsz = pd->pdi_parm.dpr_devsz;
+	sb->osb_mdc0dev.odp_zonetot = pd->pdi_parm.dpr_zonetot;
+	mc_parms2omf_devparm(&mc->mc_parms, &sb->osb_mdc0dev);
+
+	return 0;
+}
+
+int mpool_dev_sbwrite_newpool(struct mpool_descriptor *mp, struct omf_sb_descriptor *sbmdc0)
+{
+	struct mpool_dev_info *pd = NULL;
+	u64 pdh = 0;
+	int rc;
+
+	/* Alloc mdc0 and generate mdc0 info for superblocks */
+	rc = mpool_mdc0_alloc(mp, sbmdc0);
+	if (rc) {
+		mp_pr_err("%s: MDC0 allocation failed", rc, mp->pds_name);
+		return rc;
+	}
+
+	for (pdh = 0; pdh < mp->pds_pdvcnt; pdh++) {
+		pd = &mp->pds_pdv[pdh];
+
+		if (pd->pdi_mclass == mp->pds_mdparm.md_mclass)
+			rc = mpool_dev_sbwrite(mp, pd, sbmdc0);
+		else
+			rc = mpool_dev_sbwrite(mp, pd, NULL);
+		if (rc) {
+			mp_pr_err("%s: sb write %s failed, %d %d", rc, mp->pds_name,
+				  pd->pdi_name, pd->pdi_mclass, mp->pds_mdparm.md_mclass);
+			break;
+		}
+	}
+
+	return rc;
+}
+
+int mpool_mdc0_sb2obj(struct mpool_descriptor *mp, struct omf_sb_descriptor *sb,
+		      struct pmd_layout **l1, struct pmd_layout **l2)
+{
+	int rc, i;
+
+	/* MDC0 mlog1 layout */
+	*l1 = pmd_layout_alloc(&sb->osb_mdc01uuid, MDC0_OBJID_LOG1, sb->osb_mdc01gen, 0,
+			       sb->osb_mdc01desc.ol_zcnt);
+	if (!*l1) {
+		*l1 = *l2 = NULL;
+
+		rc = -ENOMEM;
+		mp_pr_err("mpool %s, MDC0 mlog1 allocation failed", rc, mp->pds_name);
+		return rc;
+	}
+
+	(*l1)->eld_state = PMD_LYT_COMMITTED;
+
+	for (i = 0; i < mp->pds_pdvcnt; i++) {
+		if (mpool_uuid_compare(&mp->pds_pdv[i].pdi_devid, &sb->osb_mdc01devid) == 0) {
+			(*l1)->eld_ld.ol_pdh = i;
+			(*l1)->eld_ld.ol_zaddr = sb->osb_mdc01desc.ol_zaddr;
+			break;
+		}
+	}
+
+	if (i >= mp->pds_pdvcnt) {
+		char uuid_str[40];
+
+		/* Should never happen */
+		pmd_obj_put(*l1);
+		*l1 = *l2 = NULL;
+
+		mpool_unparse_uuid(&sb->osb_mdc01devid, uuid_str);
+		rc = -ENOENT;
+		mp_pr_err("mpool %s, allocating MDC0 mlog1, can't find handle for pd uuid %s,",
+			  rc, mp->pds_name, uuid_str);
+
+		return rc;
+	}
+
+	/* MDC0 mlog2 layout */
+	*l2 = pmd_layout_alloc(&sb->osb_mdc02uuid, MDC0_OBJID_LOG2, sb->osb_mdc02gen, 0,
+			       sb->osb_mdc02desc.ol_zcnt);
+	if (!*l2) {
+		pmd_obj_put(*l1);
+
+		*l1 = *l2 = NULL;
+
+		rc = -ENOMEM;
+		mp_pr_err("mpool %s, MDC0 mlog2 allocation failed", rc, mp->pds_name);
+		return rc;
+	}
+
+	(*l2)->eld_state = PMD_LYT_COMMITTED;
+
+	for (i = 0; i < mp->pds_pdvcnt; i++) {
+		if (mpool_uuid_compare(&mp->pds_pdv[i].pdi_devid, &sb->osb_mdc02devid) == 0) {
+			(*l2)->eld_ld.ol_pdh = i;
+			(*l2)->eld_ld.ol_zaddr = sb->osb_mdc02desc.ol_zaddr;
+			break;
+		}
+	}
+
+	if (i >= mp->pds_pdvcnt) {
+		char uuid_str[40];
+
+		/* Should never happen */
+		pmd_obj_put(*l1);
+		pmd_obj_put(*l2);
+		*l1 = *l2 = NULL;
+
+		mpool_unparse_uuid(&sb->osb_mdc02devid, uuid_str);
+		rc = -ENOENT;
+		mp_pr_err("mpool %s, allocating MDC0 mlog2, can't find handle for pd uuid %s",
+			  rc, mp->pds_name, uuid_str);
+
+		return rc;
+	}
+
+	return 0;
+}
+
+/**
+ * mpool_dev_check_new() - check if a drive is ready to be added in an mpool.
+ * @mp:
+ * @pd:
+ */
+int mpool_dev_check_new(struct mpool_descriptor *mp, struct mpool_dev_info *pd)
+{
+	int rval, rc;
+
+	if (mpool_pd_status_get(pd) != PD_STAT_ONLINE) {
+		rc = -EIO;
+		mp_pr_err("%s:%s unavailable or offline, status %d",
+			  rc, mp->pds_name, pd->pdi_name, mpool_pd_status_get(pd));
+		return rc;
+	}
+
+	/* Confirm drive does not contain mpool magic value */
+	rval = sb_magic_check(&pd->pdi_parm);
+	if (rval) {
+		if (rval < 0) {
+			rc = rval;
+			mp_pr_err("%s:%s read sb magic failed", rc, mp->pds_name, pd->pdi_name);
+			return rc;
+		}
+
+		rc = -EBUSY;
+		mp_pr_err("%s:%s sb magic already exists", rc, mp->pds_name, pd->pdi_name);
+		return rc;
+	}
+
+	return 0;
+}
+
+int mpool_desc_pdmc_add(struct mpool_descriptor *mp, u16 pdh,
+			struct omf_devparm_descriptor *omf_devparm, bool check_only)
+{
+	struct mpool_dev_info *pd = NULL;
+	struct media_class *mc;
+	struct mc_parms mc_parms;
+	int rc;
+
+	pd = &mp->pds_pdv[pdh];
+	if (omf_devparm == NULL)
+		mc_pd_prop2mc_parms(&pd->pdi_parm.dpr_prop, &mc_parms);
+	else
+		mc_omf_devparm2mc_parms(omf_devparm, &mc_parms);
+
+	if (!mclass_isvalid(mc_parms.mcp_classp)) {
+		rc = -EINVAL;
+		mp_pr_err("%s: media class %u of %s is undefined",  rc, mp->pds_name,
+			  mc_parms.mcp_classp, pd->pdi_name);
+		return rc;
+	}
+
+	/*
+	 * Devices that do not support updatable sectors can't be included
+	 * in an mpool. Do not check if in the context of an unavailable PD
+	 * during activate, because it is impossible to determine the PD
+	 * properties.
+	 */
+	if ((omf_devparm == NULL) && !(pd->pdi_cmdopt & PD_CMD_SECTOR_UPDATABLE)) {
+		rc = -EINVAL;
+		mp_pr_err("%s: device %s sectors not updatable", rc, mp->pds_name, pd->pdi_name);
+		return rc;
+	}
+
+	mc = &mp->pds_mc[mc_parms.mcp_classp];
+	if (mc->mc_pdmc < 0) {
+		struct mc_smap_parms mcsp;
+
+		/*
+		 * No media class corresponding to the PD class yet, create one.
+		 */
+		rc = mc_smap_parms_get(&mp->pds_mc[mc_parms.mcp_classp], &mp->pds_params, &mcsp);
+		if (rc)
+			return rc;
+
+		if (!check_only)
+			mc_init_class(mc, &mc_parms, &mcsp);
+	} else {
+		rc = -EINVAL;
+		mp_pr_err("%s: add %s, only 1 device allowed per media class",
+			  rc, mp->pds_name, pd->pdi_name);
+		return rc;
+	}
+
+	if (check_only)
+		return 0;
+
+	mc->mc_pdmc = pdh;
+
+	return 0;
+}
+
+/**
+ * mpool_desc_init_newpool() - Create the media classes and add all the mpool PDs
+ * @mp:
+ * @flags: enum mp_mgmt_flags
+ *
+ * Called on mpool create.
+ * Create the media classes and add all the mpool PDs in their media class.
+ * Update the metadata media class in mp->pds_mdparm
+ *
+ * Note: the PD properties (pd->pdi_parm.dpr_prop) must be updated
+ * and correct when entering this function.
+ */
+int mpool_desc_init_newpool(struct mpool_descriptor *mp, u32 flags)
+{
+	u64 pdh = 0;
+	int rc;
+
+	if (!(flags & (1 << MP_FLAGS_FORCE))) {
+		rc = mpool_dev_check_new(mp, &mp->pds_pdv[pdh]);
+		if (rc)
+			return rc;
+	}
+
+	/*
+	 * Add drive in its media class. That may create the class
+	 * if first drive of the class.
+	 */
+	rc = mpool_desc_pdmc_add(mp, pdh, NULL, false);
+	if (rc) {
+		struct mpool_dev_info *pd __maybe_unused;
+
+		pd = &mp->pds_pdv[pdh];
+
+		mp_pr_err("mpool %s, mpool desc init, adding drive %s in a media class failed",
+			  rc, mp->pds_name, pd->pdi_name);
+		return rc;
+	}
+
+	mp->pds_mdparm.md_mclass = mp->pds_pdv[pdh].pdi_mclass;
+
+	return 0;
+}
+
+int mpool_dev_init_all(struct mpool_dev_info *pdv, u64 dcnt, char **dpaths,
+		       struct pd_prop *pd_prop)
+{
+	char *pdname;
+	int idx, rc;
+
+	if (dcnt == 0)
+		return -EINVAL;
+
+	for (rc = 0, idx = 0; idx < dcnt; idx++, pd_prop++) {
+		rc = pd_dev_open(dpaths[idx], &pdv[idx].pdi_parm, pd_prop);
+		if (rc) {
+			mp_pr_err("opening device %s failed", rc, dpaths[idx]);
+			break;
+		}
+
+		pdname = strrchr(dpaths[idx], '/');
+		pdname = pdname ? pdname + 1 : dpaths[idx];
+		strlcpy(pdv[idx].pdi_name, pdname, sizeof(pdv[idx].pdi_name));
+
+		mpool_pd_status_set(&pdv[idx], PD_STAT_ONLINE);
+	}
+
+	while (rc && idx-- > 0)
+		pd_dev_close(&pdv[idx].pdi_parm);
+
+	return rc;
+}
+
+void mpool_mdc_cap_init(struct mpool_descriptor *mp, struct mpool_dev_info *pd)
+{
+	u64 zonesz, defmbsz;
+
+	zonesz = (pd->pdi_zonepg << PAGE_SHIFT) >> 20;
+	defmbsz = MPOOL_MBSIZE_MB_DEFAULT;
+
+	if (mp->pds_params.mp_mdc0cap == 0) {
+		mp->pds_params.mp_mdc0cap = max_t(u64, defmbsz, zonesz);
+		mp->pds_params.mp_mdc0cap <<= 20;
+	}
+
+	if (mp->pds_params.mp_mdcncap == 0) {
+		mp->pds_params.mp_mdcncap = max_t(u64, zonesz, (256 / zonesz));
+		mp->pds_params.mp_mdcncap <<= 20;
+	}
+}
+
+/**
+ * mpool_desc_init_sb() - Read the super blocks of the PDs.
+ * @mp:
+ * @sbmdc0: output. MDC0 information stored in the super blocks.
+ * @flags:
+ *
+ * Adjust the discovered PD properties stored in pd->pdi_parm.dpr_prop with
+ * PD parameters from the super block. Some of discovered PD properties are
+ * default (like zone size) and need to be adjusted to what the PD actually
+ * use.
+ */
+int mpool_desc_init_sb(struct mpool_descriptor *mp, struct omf_sb_descriptor *sbmdc0,
+		       u32 flags, bool *mc_resize)
+{
+	struct omf_sb_descriptor *sb = NULL;
+	struct mpool_dev_info *pd = NULL;
+	u16 omf_ver = OMF_SB_DESC_UNDEF;
+	bool mdc0found = false;
+	bool force = ((flags & (1 << MP_FLAGS_FORCE)) != 0);
+	u8 pdh = 0;
+	int rc;
+
+	sb = kzalloc(sizeof(*sb), GFP_KERNEL);
+	if (!sb) {
+		rc = -ENOMEM;
+		mp_pr_err("sb desc alloc failed %lu", rc, (ulong)sizeof(*sb));
+		return rc;
+	}
+
+	for (pdh = 0; pdh < mp->pds_pdvcnt; pdh++) {
+		struct omf_devparm_descriptor *dparm;
+		bool resize = false;
+		int i;
+
+		pd = &mp->pds_pdv[pdh];
+		if (mpool_pd_status_get(pd) != PD_STAT_ONLINE) {
+			rc = -EIO;
+			mp_pr_err("pd %s unavailable or offline, status %d",
+				  rc, pd->pdi_name, mpool_pd_status_get(pd));
+			kfree(sb);
+			return rc;
+		}
+
+		/*
+		 * Read superblock; init and validate pool drive info
+		 * from device parameters stored in the super block.
+		 */
+		rc = sb_read(&pd->pdi_parm, sb, &omf_ver, force);
+		if (rc) {
+			mp_pr_err("sb read from %s failed", rc, pd->pdi_name);
+			kfree(sb);
+			return rc;
+		}
+
+		if (!pdh) {
+			size_t n __maybe_unused;
+
+			/*
+			 * First drive; confirm pool not open; set pool-wide
+			 * properties
+			 */
+			if (uuid_to_mpdesc_search(&mpool_pools, &sb->osb_poolid)) {
+				char *uuid_str;
+
+				uuid_str = kmalloc(MPOOL_UUID_STRING_LEN + 1, GFP_KERNEL);
+				if (uuid_str)
+					mpool_unparse_uuid(&sb->osb_poolid, uuid_str);
+
+				rc = -EBUSY;
+				mp_pr_err("%s: mpool already activated, id %s, pd name %s",
+					  rc, sb->osb_name, uuid_str, pd->pdi_name);
+				kfree(sb);
+				kfree(uuid_str);
+				return rc;
+			}
+			mpool_uuid_copy(&mp->pds_poolid, &sb->osb_poolid);
+
+			n = strlcpy(mp->pds_name, (char *)sb->osb_name, sizeof(mp->pds_name));
+			ASSERT(n < sizeof(mp->pds_name));
+		} else {
+			/* Second or later drive; validate pool-wide properties */
+			if (mpool_uuid_compare(&sb->osb_poolid, &mp->pds_poolid) != 0) {
+				char *uuid_str1, *uuid_str2 = NULL;
+
+				uuid_str1 = kmalloc(2 * (MPOOL_UUID_STRING_LEN + 1), GFP_KERNEL);
+				if (uuid_str1) {
+					uuid_str2 = uuid_str1 + MPOOL_UUID_STRING_LEN + 1;
+					mpool_unparse_uuid(&sb->osb_poolid, uuid_str1);
+					mpool_unparse_uuid(&mp->pds_poolid, uuid_str2);
+				}
+
+				rc = -EINVAL;
+				mp_pr_err("%s: pd %s, mpool id %s different from prior id %s",
+					  rc, mp->pds_name, pd->pdi_name, uuid_str1, uuid_str2);
+				kfree(sb);
+				kfree(uuid_str1);
+				return rc;
+			}
+		}
+
+		dparm = &sb->osb_parm;
+		if (!force && pd->pdi_devsz > dparm->odp_devsz) {
+			mp_pr_info("%s: pd %s, discovered size %lu > on-media size %lu",
+				mp->pds_name, pd->pdi_name,
+				(ulong)pd->pdi_devsz, (ulong)dparm->odp_devsz);
+
+			if ((flags & (1 << MP_FLAGS_RESIZE)) == 0) {
+				pd->pdi_devsz = dparm->odp_devsz;
+			} else {
+				dparm->odp_devsz  = pd->pdi_devsz;
+				dparm->odp_zonetot = pd->pdi_devsz / (pd->pdi_zonepg << PAGE_SHIFT);
+
+				pd->pdi_zonetot = dparm->odp_zonetot;
+				resize = true;
+			}
+		}
+
+		/* Validate mdc0 info in superblock if present */
+		if (!sbutil_mdc0_isclear(sb)) {
+			if (!force && !sbutil_mdc0_isvalid(sb)) {
+				rc = -EINVAL;
+				mp_pr_err("%s: pd %s, invalid sb MDC0",
+					  rc, mp->pds_name, pd->pdi_name);
+				kfree(sb);
+				return rc;
+			}
+
+			dparm = &sb->osb_mdc0dev;
+			if (resize) {
+				ASSERT(pd->pdi_devsz > dparm->odp_devsz);
+
+				dparm->odp_devsz = pd->pdi_devsz;
+				dparm->odp_zonetot = pd->pdi_devsz / (pd->pdi_zonepg << PAGE_SHIFT);
+			}
+
+			sbutil_mdc0_copy(sbmdc0, sb);
+			mdc0found = true;
+		}
+
+		/* Set drive info confirming devid is unique and zone parms match */
+		for (i = 0; i < pdh; i++) {
+			if (mpool_uuid_compare(&mp->pds_pdv[i].pdi_devid,
+					       &sb->osb_parm.odp_devid) == 0) {
+				char *uuid_str;
+
+				uuid_str = kmalloc(MPOOL_UUID_STRING_LEN + 1, GFP_KERNEL);
+				if (uuid_str)
+					mpool_unparse_uuid(&sb->osb_parm.odp_devid, uuid_str);
+				rc = -EINVAL;
+				mp_pr_err("%s: pd %s, duplicate devices, uuid %s",
+					  rc, mp->pds_name, pd->pdi_name, uuid_str);
+				kfree(uuid_str);
+				kfree(sb);
+				return rc;
+			}
+		}
+
+		if (omf_ver > OMF_SB_DESC_VER_LAST) {
+			rc = -EOPNOTSUPP;
+			mp_pr_err("%s: unsupported sb version %d", rc, mp->pds_name, omf_ver);
+			kfree(sb);
+			return rc;
+		} else if (!force && (omf_ver < OMF_SB_DESC_VER_LAST || resize)) {
+			if ((flags & (1 << MP_FLAGS_PERMIT_META_CONV)) == 0) {
+				struct omf_mdcver *mdcver;
+				char *buf1, *buf2 = NULL;
+
+				/*
+				 * We have to get the permission from users
+				 * to update mpool meta data
+				 */
+				mdcver = omf_sbver_to_mdcver(omf_ver);
+				ASSERT(mdcver != NULL);
+
+				buf1 = kmalloc(2 * MAX_MDCVERSTR, GFP_KERNEL);
+				if (buf1) {
+					buf2 = buf1 + MAX_MDCVERSTR;
+					omfu_mdcver_to_str(mdcver, buf1, sizeof(buf1));
+					omfu_mdcver_to_str(omfu_mdcver_cur(), buf2, sizeof(buf2));
+				}
+
+				rc = -EPERM;
+				mp_pr_err("%s: reqd sb upgrade from version %s (%s) to %s (%s)",
+					  rc, mp->pds_name,
+					  buf1, omfu_mdcver_comment(mdcver) ?: "",
+					  buf2, omfu_mdcver_comment(omfu_mdcver_cur()));
+				kfree(buf1);
+				kfree(sb);
+				return rc;
+			}
+
+			/* We need to overwrite the old version superblock on the device */
+			rc = sb_write_update(&pd->pdi_parm, sb);
+			if (rc) {
+				mp_pr_err("%s: pd %s, failed to convert or overwrite mpool sb",
+					  rc, mp->pds_name, pd->pdi_name);
+				kfree(sb);
+				return rc;
+			}
+
+			if (!resize)
+				mp_pr_info("%s: pd %s, Convert mpool sb, oldv %d newv %d",
+					   mp->pds_name, pd->pdi_name, omf_ver, sb->osb_vers);
+		}
+
+		mpool_uuid_copy(&pd->pdi_devid, &sb->osb_parm.odp_devid);
+
+		/* Add drive in its media class. Create the media class if not yet created. */
+		rc = mpool_desc_pdmc_add(mp, pdh, NULL, false);
+		if (rc) {
+			mp_pr_err("%s: pd %s, adding drive in a media class failed",
+				  rc, mp->pds_name, pd->pdi_name);
+
+			kfree(sb);
+			return rc;
+		}
+
+		/*
+		 * Record the media class used by the MDC0 metadata.
+		 */
+		if (mdc0found)
+			mp->pds_mdparm.md_mclass = pd->pdi_mclass;
+
+		if (resize && mc_resize)
+			mc_resize[pd->pdi_mclass] = resize;
+	}
+
+	if (!mdc0found) {
+		rc = -EINVAL;
+		mp_pr_err("%s: MDC0 not found", rc, mp->pds_name);
+		kfree(sb);
+		return rc;
+	}
+
+	kfree(sb);
+
+	return 0;
+}
+
+static int comp_func(const void *c1, const void *c2)
+{
+	return strcmp(*(char **)c1, *(char **)c2);
+}
+
+int check_for_dups(char **listv, int cnt, int *dup, int *offset)
+{
+	const char **sortedv;
+	const char *prev;
+	int rc, i;
+
+	*dup = 0;
+	*offset = -1;
+
+	if (0 == cnt || 1 == cnt)
+		return 0;
+
+	sortedv = kcalloc(cnt + 1, sizeof(char *), GFP_KERNEL);
+	if (!sortedv) {
+		rc = -ENOMEM;
+		mp_pr_err("kcalloc failed for %d paths, first path %s", rc, cnt, *listv);
+		return rc;
+	}
+
+	/* Make a shallow copy */
+	for (i = 0; i < cnt; i++)
+		sortedv[i] = listv[i];
+
+	sortedv[i] = NULL;
+
+	sort(sortedv, cnt, sizeof(char *), comp_func, NULL);
+
+	prev = sortedv[0];
+	for (i = 1; i < cnt; i++) {
+		if (strcmp(sortedv[i], prev) == 0) {
+			mp_pr_info("path %s is duplicated", prev);
+			*dup = 1;
+			break;
+		}
+
+		prev = sortedv[i];
+	}
+
+	/* Find offset, prev points to first dup */
+	if (*dup) {
+		for (i = 0; i < cnt; i++) {
+			if (prev == listv[i]) {
+				*offset = i;
+				break;
+			}
+		}
+	}
+
+	kfree(sortedv);
+	return 0;
+}
+
+void fill_in_devprops(struct mpool_descriptor *mp, u64 pdh, struct mpool_devprops *dprop)
+{
+	struct mpool_dev_info *pd;
+	struct media_class *mc;
+	int rc;
+
+	pd = &mp->pds_pdv[pdh];
+	memcpy(dprop->pdp_devid.b, pd->pdi_devid.uuid, MPOOL_UUID_SIZE);
+
+	mc = &mp->pds_mc[pd->pdi_mclass];
+	dprop->pdp_mclassp = mc->mc_parms.mcp_classp;
+	dprop->pdp_status  = mpool_pd_status_get(pd);
+
+	rc = smap_drive_usage(mp, pdh, dprop);
+	if (rc) {
+		mp_pr_err("mpool %s, can't get drive usage, media class %d",
+			  rc, mp->pds_name, dprop->pdp_mclassp);
+	}
+}
+
+int mpool_desc_unavail_add(struct mpool_descriptor *mp, struct omf_devparm_descriptor *omf_devparm)
+{
+	struct mpool_dev_info *pd = NULL;
+	char uuid_str[40];
+	int rc;
+
+	mpool_unparse_uuid(&omf_devparm->odp_devid, uuid_str);
+
+	mp_pr_warn("Activating mpool %s, adding unavailable drive %s", mp->pds_name, uuid_str);
+
+	if (mp->pds_pdvcnt >= MPOOL_DRIVES_MAX) {
+		rc = -EINVAL;
+		mp_pr_err("Activating mpool %s, adding an unavailable drive, too many drives",
+			  rc, mp->pds_name);
+		return rc;
+	}
+
+	pd = &mp->pds_pdv[mp->pds_pdvcnt];
+
+	mpool_uuid_copy(&pd->pdi_devid, &omf_devparm->odp_devid);
+
+	/* Update the PD properties from the metadata record. */
+	mpool_pd_status_set(pd, PD_STAT_UNAVAIL);
+	pd_dev_set_unavail(&pd->pdi_parm, omf_devparm);
+
+	/* Add the PD in its media class. */
+	rc = mpool_desc_pdmc_add(mp, mp->pds_pdvcnt, omf_devparm, false);
+	if (rc)
+		return rc;
+
+	mp->pds_pdvcnt = mp->pds_pdvcnt + 1;
+
+	return 0;
+}
+
+int mpool_create_rmlogs(struct mpool_descriptor *mp, u64 mlog_cap)
+{
+	struct mlog_descriptor *ml_desc;
+	struct mlog_capacity mlcap = {
+		.lcp_captgt = mlog_cap,
+	};
+	struct mlog_props mlprops;
+	u64 root_mlog_id[2];
+	int rc, i;
+
+	mlog_lookup_rootids(&root_mlog_id[0], &root_mlog_id[1]);
+
+	for (i = 0; i < 2; ++i) {
+		rc = mlog_find_get(mp, root_mlog_id[i], 1, NULL, &ml_desc);
+		if (!rc) {
+			mlog_put(ml_desc);
+			continue;
+		}
+
+		if (rc != -ENOENT) {
+			mp_pr_err("mpool %s, root mlog find 0x%lx failed",
+				  rc, mp->pds_name, (ulong)root_mlog_id[i]);
+			return rc;
+		}
+
+		rc = mlog_realloc(mp, root_mlog_id[i], &mlcap,
+				  MP_MED_CAPACITY, &mlprops, &ml_desc);
+		if (rc) {
+			mp_pr_err("mpool %s, root mlog realloc 0x%lx failed",
+				  rc, mp->pds_name, (ulong)root_mlog_id[i]);
+			return rc;
+		}
+
+		if (mlprops.lpr_objid != root_mlog_id[i]) {
+			mlog_put(ml_desc);
+			rc = -ENOENT;
+			mp_pr_err("mpool %s, root mlog mismatch 0x%lx 0x%lx", rc,
+				  mp->pds_name, (ulong)root_mlog_id[i], (ulong)mlprops.lpr_objid);
+			return rc;
+		}
+
+		rc = mlog_commit(mp, ml_desc);
+		if (rc) {
+			if (mlog_abort(mp, ml_desc))
+				mlog_put(ml_desc);
+
+			mp_pr_err("mpool %s, root mlog commit 0x%lx failed",
+				  rc, mp->pds_name, (ulong)root_mlog_id[i]);
+			return rc;
+		}
+
+		mlog_put(ml_desc);
+	}
+
+	return rc;
+}
+
+struct mpool_descriptor *mpool_desc_alloc(void)
+{
+	struct mpool_descriptor *mp;
+	int i;
+
+	mp = kzalloc(sizeof(*mp), GFP_KERNEL);
+	if (!mp)
+		return NULL;
+
+	init_rwsem(&mp->pds_pdvlock);
+
+	mutex_init(&mp->pds_oml_lock);
+	mp->pds_oml_root = RB_ROOT;
+
+	mp->pds_mdparm.md_mclass = MP_MED_INVALID;
+
+	mpcore_params_defaults(&mp->pds_params);
+
+	for (i = 0; i < MP_MED_NUMBER; i++)
+		mp->pds_mc[i].mc_pdmc = -1;
+
+	return mp;
+}
+
+/*
+ * remove mp from mpool_pools; close all dev; dealloc mp.
+ */
+void mpool_desc_free(struct mpool_descriptor *mp)
+{
+	struct mpool_descriptor *found_mp = NULL;
+	struct mpool_uuid uuid_zero;
+	int i;
+
+	mpool_uuid_clear(&uuid_zero);
+
+	/*
+	 * Handle case where poolid and devid not in mappings
+	 * which can happen when cleaning up from failed create/open.
+	 */
+	found_mp = uuid_to_mpdesc_search(&mpool_pools, &mp->pds_poolid);
+	if (found_mp)
+		rb_erase(&found_mp->pds_node, &mpool_pools);
+
+	for (i = 0; i < mp->pds_pdvcnt; i++) {
+		if (mpool_pd_status_get(&mp->pds_pdv[i]) != PD_STAT_UNAVAIL)
+			pd_dev_close(&mp->pds_pdv[i].pdi_parm);
+	}
+
+	kfree(mp);
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:29 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:29 -0500
Subject: [PATCH v2 15/22] mpool: add mpool lifecycle management routines
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-16-nmeeramohide@micron.com>

This adds mpool lifecycle management functions: create,
activate, deactivate, destroy, rename, add a new media
class, fetch properties etc.

An mpool is created with a mandatory capacity media class
volume. A pool drive (PD) instance is initialized for each
media class volume using the attributes pushed from the mpool
user library. The metadata manager interfaces are then
invoked to activate this mpool and allocate the initial set
of metadata containers. The media class attributes, spare
percent, mpool configuration etc. are persisted in MDC-0.

At mpool activation, the records from MDC-0 containing the
mpool properties and metadata for accessing MDC-1 through
MDC-N are first loaded into memory, initializing all the
necessary in-core structures using the metadata manager and
space map interfaces.  Then the records from MDC-1 through
MDC-N containing the metadata for accessing client mblock
and mlog objects are loaded into memory, again initializing
all the necessary in-core structures using the metadata
manager and space map interfaces.

An mpool is destroyed by erasing the superblock on all its
constituent media class volumes. Renaming an mpool updates
the superblock on all the media class volumes with the new
name.  Adding a new media class volume to an activated mpool
is handled like initializing a volume at mpool create.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mp.c | 1086 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 1086 insertions(+)
 create mode 100644 drivers/mpool/mp.c

diff --git a/drivers/mpool/mp.c b/drivers/mpool/mp.c
new file mode 100644
index 000000000000..6b8c51c23fec
--- /dev/null
+++ b/drivers/mpool/mp.c
@@ -0,0 +1,1086 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2015-2020 Micron Technology, Inc.  All rights reserved.
+ */
+
+/*
+ * Media pool (mpool) manager module.
+ *
+ * Defines functions to create and maintain mpools comprising multiple drives
+ * in multiple media classes used for storing mblocks and mlogs.
+ */
+
+#include <linux/string.h>
+#include <linux/mutex.h>
+#include <crypto/hash.h>
+
+#include "assert.h"
+#include "mpool_printk.h"
+
+#include "sb.h"
+#include "upgrade.h"
+#include "mpcore.h"
+#include "mp.h"
+
+/*
+ * Lock for serializing certain mpool ops where required/desirable; could be per
+ * mpool in some cases but no meaningful performance benefit for these rare ops;
+ * also protects mpool_pools and certain mpool_descriptor fields.
+ */
+static DEFINE_MUTEX(mpool_s_lock);
+
+int mpool_create(const char *mpname, u32 flags, char **dpaths, struct pd_prop *pd_prop,
+		 struct mpcore_params *params, u64 mlog_cap)
+{
+	struct omf_sb_descriptor *sbmdc0;
+	struct mpool_descriptor *mp;
+	struct pmd_layout *mdc01, *mdc02;
+	bool active, sbvalid;
+	u16 sidx;
+	int err;
+
+	if (!mpname || !*mpname || !dpaths || !pd_prop)
+		return -EINVAL;
+
+	mdc01 = mdc02 = NULL;
+	active = sbvalid = false;
+
+	mp = mpool_desc_alloc();
+	if (!mp) {
+		err = -ENOMEM;
+		mp_pr_err("mpool %s, alloc desc failed", err, mpname);
+		return err;
+	}
+
+	sbmdc0 = &(mp->pds_sbmdc0);
+	strlcpy((char *)mp->pds_name, mpname, sizeof(mp->pds_name));
+	mpool_generate_uuid(&mp->pds_poolid);
+
+	if (params)
+		mp->pds_params = *params;
+
+	mp->pds_pdvcnt = 0;
+
+	mutex_lock(&mpool_s_lock);
+
+	/*
+	 * Allocate the per-mpool workqueue.
+	 * TODO: Make this per-driver
+	 */
+	mp->pds_erase_wq = alloc_workqueue("mperasewq", WQ_HIGHPRI, 0);
+	if (!mp->pds_erase_wq) {
+		err = -ENOMEM;
+		mp_pr_err("mpool %s, alloc per-mpool wq failed", err, mpname);
+		goto errout;
+	}
+
+	/*
+	 * Set the devices parameters from the ones placed by the discovery
+	 * in pd_prop.
+	 */
+	err = mpool_dev_init_all(mp->pds_pdv, 1, dpaths, pd_prop);
+	if (err) {
+		mp_pr_err("mpool %s, failed to get device parameters", err, mpname);
+		goto errout;
+	}
+
+	mp->pds_pdvcnt = 1;
+
+	mpool_mdc_cap_init(mp, &mp->pds_pdv[0]);
+
+	/* Init new pool drives uuid and mclassp */
+	mpool_generate_uuid(&mp->pds_pdv[0].pdi_devid);
+
+	/*
+	 * Init mpool descriptor from new drive info.
+	 * Creates the media classes and place the PDs in them.
+	 * Determine the media class used for the metadata.
+	 */
+	err = mpool_desc_init_newpool(mp, flags);
+	if (err) {
+		mp_pr_err("mpool %s, desc init from new drive info failed", err, mpname);
+		goto errout;
+	}
+
+	/*
+	 * Alloc empty mdc0 and write superblocks to all drives; if
+	 * crash drives with superblocks will not be recognized as mpool
+	 * members because there are not yet any drive state records in mdc0
+	 */
+	sbvalid = true;
+	err = mpool_dev_sbwrite_newpool(mp, sbmdc0);
+	if (err) {
+		mp_pr_err("mpool %s, couldn't write superblocks", err, mpname);
+		goto errout;
+	}
+
+	/* Alloc mdc0 mlog layouts and activate mpool with empty mdc0 */
+	err = mpool_mdc0_sb2obj(mp, sbmdc0, &mdc01, &mdc02);
+	if (err) {
+		mp_pr_err("mpool %s, alloc of MDC0 mlogs failed", err, mpname);
+		goto errout;
+	}
+
+	err = pmd_mpool_activate(mp, mdc01, mdc02, 1);
+	if (err) {
+		mp_pr_err("mpool %s, activation failed", err, mpname);
+		goto errout;
+	}
+
+	active = true;
+
+	/*
+	 * Add the version record (always first record) in MDC0.
+	 * The version record is used only from version 1.0.0.1.
+	 */
+	if (omfu_mdcver_cmp2(omfu_mdcver_cur(), ">=", 1, 0, 0, 1)) {
+		err = pmd_mdc_addrec_version(mp, 0);
+		if (err) {
+			mp_pr_err("mpool %s, writing MDC version record in MDC0 failed",
+				  err, mpname);
+			goto errout;
+		}
+	}
+
+	/*
+	 * Add drive state records to mdc0; if crash before complete will
+	 * detect if attempt to open same drive list; it may be possible to
+	 * open the subset of the drive list for which state records were
+	 * written without detection, in which case the other drives can be
+	 * added
+	 */
+	err = pmd_prop_mcconfig(mp, &mp->pds_pdv[0], false);
+	if (err) {
+		mp_pr_err("mpool %s, add drive state to MDC0 failed", err, mpname);
+		goto errout;
+	}
+
+	/*
+	 * Create mdcs so user can create mlog/mblock objects;
+	 * if crash before all the configured mdcs are created, or if create
+	 * fails, will detect in activate and re-try.
+	 *
+	 * mp_cmdcn corresponds to the number of MDCNs used for client
+	 * objects, i.e., [1 - mp_cmdcn]
+	 */
+	for (sidx = 1; sidx <= mp->pds_params.mp_mdcnum; sidx++) {
+		err = pmd_mdc_alloc(mp, mp->pds_params.mp_mdcncap, sidx - 1);
+		if (err) {
+			mp_pr_info("mpool %s, only %u MDCs out of %lu MDCs were created",
+				  mpname, sidx - 1, (ulong)mp->pds_params.mp_mdcnum);
+			/*
+			 * For MDCN creation failure, mask the error and
+			 * continue further with create.
+			 */
+			err = 0;
+			break;
+		}
+	}
+	pmd_update_credit(mp);
+
+	/*
+	 * Attempt root mlog creation only if MDC1 was successfully created.
+	 * If MDC1 doesn't exist, it will be re-created during activate.
+	 */
+	if (sidx > 1) {
+		err = mpool_create_rmlogs(mp, mlog_cap);
+		if (err) {
+			mp_pr_info("mpool %s, root mlog creation failed", mpname);
+			/*
+			 * If root mlog creation fails, mask the error and
+			 * proceed with create. root mlogs will be re-created
+			 * during activate.
+			 */
+			err = 0;
+		}
+	}
+
+	/* Add mp to the list of all open mpools */
+	uuid_to_mpdesc_insert(&mpool_pools, mp);
+
+errout:
+
+	if (mp->pds_erase_wq)
+		destroy_workqueue(mp->pds_erase_wq);
+
+	if (active)
+		pmd_mpool_deactivate(mp);
+
+	if (err && sbvalid) {
+		struct mpool_dev_info *pd;
+		int err1;
+
+		/* Erase super blocks on the drives */
+		pd = &mp->pds_pdv[0];
+		if (mpool_pd_status_get(pd) != PD_STAT_ONLINE) {
+			err1 = -EIO;
+			mp_pr_err("%s:%s unavailable or offline, status %d",
+				  err1, mp->pds_name, pd->pdi_name, mpool_pd_status_get(pd));
+		} else {
+			err1 = sb_erase(&pd->pdi_parm);
+			if (err1)
+				mp_pr_info("%s: cleanup, sb erase failed on device %s",
+					   mp->pds_name, pd->pdi_name);
+		}
+	}
+
+	mpool_desc_free(mp);
+
+	mutex_unlock(&mpool_s_lock);
+
+	return err;
+}
+
+int mpool_activate(u64 dcnt, char **dpaths, struct pd_prop *pd_prop, u64 mlog_cap,
+		   struct mpcore_params *params, u32 flags, struct mpool_descriptor **mpp)
+{
+	struct omf_sb_descriptor *sbmdc0;
+	struct mpool_descriptor *mp;
+	struct pmd_layout *mdc01 = NULL;
+	struct pmd_layout *mdc02 = NULL;
+	struct media_class *mcmeta;
+	u64 mdcmax, mdcnum, mdcncap, mdc0cap;
+	bool force = ((flags & (1 << MP_FLAGS_FORCE)) != 0);
+	bool mc_resize[MP_MED_NUMBER] = { };
+	bool active;
+	int dup, doff, err, i;
+	u8  pdh;
+
+	active = false;
+	*mpp = NULL;
+
+	if (dcnt > MPOOL_DRIVES_MAX) {
+		err = -EINVAL;
+		mp_pr_err("too many drives in input %lu, first drive path %s",
+			  err, (ulong)dcnt, dpaths[0]);
+		return err;
+	}
+
+	/*
+	 * Verify no duplicate drive paths
+	 */
+	err = check_for_dups(dpaths, dcnt, &dup, &doff);
+	if (err) {
+		mp_pr_err("duplicate drive check failed", err);
+		return err;
+	} else if (dup) {
+		err = -EINVAL;
+		mp_pr_err("duplicate drive path %s", err, (doff == -1) ? "" : dpaths[doff]);
+		return err;
+	}
+
+	/* Alloc mpool descriptor and fill in device-indepdendent values */
+	mp = mpool_desc_alloc();
+	if (!mp) {
+		err = -ENOMEM;
+		mp_pr_err("alloc mpool desc failed", err);
+		return err;
+	}
+
+	sbmdc0 = &(mp->pds_sbmdc0);
+
+	mp->pds_pdvcnt = 0;
+
+	if (params)
+		mp->pds_params = *params;
+
+	mutex_lock(&mpool_s_lock);
+
+	mp->pds_workq = alloc_workqueue("mpoolwq", WQ_UNBOUND, 0);
+	if (!mp->pds_workq) {
+		err = -ENOMEM;
+		mp_pr_err("alloc mpoolwq failed, first drive path %s", err, dpaths[0]);
+		goto errout;
+	}
+
+	mp->pds_erase_wq = alloc_workqueue("mperasewq", WQ_HIGHPRI, 0);
+	if (!mp->pds_erase_wq) {
+		err = -ENOMEM;
+		mp_pr_err("alloc mperasewq failed, first drive path %s", err, dpaths[0]);
+		goto errout;
+	}
+
+	/* Get device parm for all drive paths */
+	err = mpool_dev_init_all(mp->pds_pdv, dcnt, dpaths, pd_prop);
+	if (err) {
+		mp_pr_err("can't get drive device params, first drive path %s", err, dpaths[0]);
+		goto errout;
+	}
+
+	/* Set mp.pdvcnt so dpaths will get closed in cleanup if activate fails. */
+	mp->pds_pdvcnt = dcnt;
+
+	/* Init mpool descriptor from superblocks on drives */
+	err = mpool_desc_init_sb(mp, sbmdc0, flags, mc_resize);
+	if (err) {
+		mp_pr_err("mpool_desc_init_sb failed, first drive path %s", err, dpaths[0]);
+		goto errout;
+	}
+
+	mcmeta = &mp->pds_mc[mp->pds_mdparm.md_mclass];
+	if (mcmeta->mc_pdmc < 0) {
+		err = -ENODEV;
+		mp_pr_err("mpool %s, too many unavailable drives", err, mp->pds_name);
+		goto errout;
+	}
+
+	/* Alloc mdc0 mlog layouts from superblock and activate mpool */
+	err = mpool_mdc0_sb2obj(mp, sbmdc0, &mdc01, &mdc02);
+	if (err) {
+		mp_pr_err("mpool %s, allocation of MDC0 mlogs layouts failed", err, mp->pds_name);
+		goto errout;
+	}
+
+	err = pmd_mpool_activate(mp, mdc01, mdc02, 0);
+	if (err) {
+		mp_pr_err("mpool %s, activation failed", err, mp->pds_name);
+		goto errout;
+	}
+
+	active = true;
+
+	for (pdh = 0; pdh < mp->pds_pdvcnt; pdh++) {
+		struct mpool_dev_info  *pd;
+
+		pd = &mp->pds_pdv[pdh];
+
+		if (mc_resize[pd->pdi_mclass]) {
+			err = pmd_prop_mcconfig(mp, pd, false);
+			if (err) {
+				mp_pr_err("mpool %s, updating MCCONFIG record for resize failed",
+					  err, mp->pds_name);
+				goto errout;
+			}
+		}
+
+		if (pd->pdi_mclass == MP_MED_CAPACITY)
+			mpool_mdc_cap_init(mp, pd);
+	}
+
+	/* Tolerate unavailable drives only if force flag specified */
+	for (i = 0; !force && i < MP_MED_NUMBER; i++) {
+		struct media_class *mc;
+
+		mc = &mp->pds_mc[i];
+		if (mc->mc_uacnt) {
+			err = -ENODEV;
+			mp_pr_err("mpool %s, unavailable drives present", err, mp->pds_name);
+			goto errout;
+		}
+	}
+
+	/*
+	 * Create mdcs if needed so user can create mlog/mblock objects;
+	 * Only needed if the configured number of mdcs did not get created
+	 * during mpool create due to crash or failure.
+	 */
+	mdcmax = mdcncap = mdc0cap = 0;
+	mdcnum = mp->pds_params.mp_mdcnum;
+
+	pmd_mdc_cap(mp, &mdcmax, &mdcncap, &mdc0cap);
+
+	if (mdc0cap)
+		mp->pds_params.mp_mdc0cap = mdc0cap;
+
+	if (mdcncap && mdcmax) {
+		mdcncap = mdcncap / mdcmax;
+		mp->pds_params.mp_mdcncap = mdcncap;
+		mp->pds_params.mp_mdcnum  = mdcmax;
+	}
+
+	if (mdcmax < mdcnum) {
+		mp_pr_info("mpool %s, detected missing MDCs %lu %lu",
+			   mp->pds_name, (ulong)mdcnum, (ulong)mdcmax);
+
+		for (mdcmax++; mdcmax <= mdcnum; mdcmax++) {
+
+			err = pmd_mdc_alloc(mp, mp->pds_params.mp_mdcncap,
+					    mdcmax);
+			if (!err)
+				continue;
+
+			/* MDC1 creation failure - non-functional mpool */
+			if (mdcmax < 2) {
+				mp_pr_err("mpool %s, MDC1 can't be created", err, mp->pds_name);
+				goto errout;
+			}
+
+			mp_pr_notice("mpool %s, couldn't create %lu MDCs out of %lu MDCs",
+				     mp->pds_name, (ulong)(mdcnum - mdcmax + 1), (ulong)mdcnum);
+
+			/*
+			 * For MDCN (N > 1) creation failure, log a warning,
+			 * mask the error and continue with activate. Mpool
+			 * only needs a minimum of 1 MDC to be functional.
+			 */
+			err = 0;
+
+			break;
+		}
+		mp->pds_params.mp_mdcnum = mdcmax - 1;
+	}
+
+	pmd_update_credit(mp);
+
+	/*
+	 * If we reach here, then MDC1 must exist. Now, make sure that the
+	 * root mlogs also exist and if they don't, re-create them.
+	 */
+	err = mpool_create_rmlogs(mp, mlog_cap);
+	if (err) {
+		/* Root mlogs creation failure - non-functional mpool */
+		mp_pr_err("mpool %s, root mlogs creation failed", err, mp->pds_name);
+		goto errout;
+	}
+
+	/* Add mp to the list of all activated mpools */
+	uuid_to_mpdesc_insert(&mpool_pools, mp);
+
+	/* Start the background thread doing pre-compaction of MDC1/255 */
+	pmd_precompact_start(mp);
+
+errout:
+	if (err) {
+		if (mp->pds_workq)
+			destroy_workqueue(mp->pds_workq);
+		if (mp->pds_erase_wq)
+			destroy_workqueue(mp->pds_erase_wq);
+
+		if (active)
+			pmd_mpool_deactivate(mp);
+
+		mpool_desc_free(mp);
+		mp = NULL;
+	}
+
+	mutex_unlock(&mpool_s_lock);
+
+	*mpp = mp;
+
+	if (!err) {
+		/*
+		 * Start the periodic background job which logs a message
+		 * when an mpool's usable space is close to its limits.
+		 */
+		struct smap_usage_work *usagew;
+
+		usagew = &mp->pds_smap_usage_work;
+
+		INIT_DELAYED_WORK(&usagew->smapu_wstruct, smap_log_mpool_usage);
+		usagew->smapu_mp = mp;
+		smap_log_mpool_usage(&usagew->smapu_wstruct.work);
+	}
+
+	return err;
+}
+
+int mpool_deactivate(struct mpool_descriptor *mp)
+{
+	pmd_precompact_stop(mp);
+	smap_wait_usage_done(mp);
+
+	mutex_lock(&mpool_s_lock);
+	destroy_workqueue(mp->pds_workq);
+	destroy_workqueue(mp->pds_erase_wq);
+
+	pmd_mpool_deactivate(mp);
+
+	mpool_desc_free(mp);
+	mutex_unlock(&mpool_s_lock);
+
+	return 0;
+}
+
+int mpool_destroy(u64 dcnt, char **dpaths, struct pd_prop *pd_prop, u32 flags)
+{
+	struct omf_sb_descriptor *sbmdc0;
+	struct mpool_descriptor *mp;
+	int dup, doff;
+	int err, i;
+
+	if (dcnt > MPOOL_DRIVES_MAX) {
+		err = -EINVAL;
+		mp_pr_err("first pd %s, too many drives %lu %d",
+			  err, dpaths[0], (ulong)dcnt, MPOOL_DRIVES_MAX);
+		return err;
+	} else if (dcnt == 0) {
+		return -EINVAL;
+	}
+
+	/*
+	 * Verify no duplicate drive paths
+	 */
+	err = check_for_dups(dpaths, dcnt, &dup, &doff);
+	if (err) {
+		mp_pr_err("check_for_dups failed, dcnt %lu", err, (ulong)dcnt);
+		return err;
+	} else if (dup) {
+		err = -ENOMEM;
+		mp_pr_err("duplicate drives found", err);
+		return err;
+	}
+
+	sbmdc0 = kzalloc(sizeof(*sbmdc0), GFP_KERNEL);
+	if (!sbmdc0) {
+		err = -ENOMEM;
+		mp_pr_err("alloc sb %zu failed", err, sizeof(*sbmdc0));
+		return err;
+	}
+
+	mp = mpool_desc_alloc();
+	if (!mp) {
+		err = -ENOMEM;
+		mp_pr_err("alloc mpool desc failed", err);
+		kfree(sbmdc0);
+		return err;
+	}
+
+	mp->pds_pdvcnt = 0;
+
+	mutex_lock(&mpool_s_lock);
+
+	/* Get device parm for all drive paths */
+	err = mpool_dev_init_all(mp->pds_pdv, dcnt, dpaths, pd_prop);
+	if (err) {
+		mp_pr_err("first pd %s, get device params failed", err, dpaths[0]);
+		goto errout;
+	}
+
+	/* Set pdvcnt so dpaths will get closed in cleanup if open fails. */
+	mp->pds_pdvcnt = dcnt;
+
+	/* Init mpool descriptor from superblocks on drives */
+	err = mpool_desc_init_sb(mp, sbmdc0, flags, NULL);
+	if (err) {
+		mp_pr_err("mpool %s, first pd %s, mpool desc init from sb failed",
+			  err, (mp->pds_name == NULL) ? "" : mp->pds_name, dpaths[0]);
+		goto errout;
+	}
+
+	/* Erase super blocks on the drives */
+	for (i = 0; i < mp->pds_pdvcnt; i++) {
+		struct mpool_dev_info *pd;
+
+		pd = &mp->pds_pdv[i];
+		if (mpool_pd_status_get(pd) != PD_STAT_ONLINE) {
+			err = -EIO;
+			mp_pr_err("pd %s unavailable or offline, status %d",
+				  err, pd->pdi_name, mpool_pd_status_get(pd));
+		} else {
+			err = sb_erase(&pd->pdi_parm);
+			if (err)
+				mp_pr_err("pd %s, sb erase failed", err, pd->pdi_name);
+		}
+
+		if (err)
+			break;
+	}
+
+errout:
+	mpool_desc_free(mp);
+
+	mutex_unlock(&mpool_s_lock);
+
+	kfree(sbmdc0);
+
+	return err;
+}
+
+int mpool_rename(u64 dcnt, char **dpaths, struct pd_prop *pd_prop,
+		 u32 flags, const char *mp_newname)
+{
+	struct omf_sb_descriptor *sb;
+	struct mpool_descriptor *mp;
+	struct mpool_dev_info *pd = NULL;
+	u16 omf_ver = OMF_SB_DESC_UNDEF;
+	bool force = ((flags & (1 << MP_FLAGS_FORCE)) != 0);
+	u8 pdh;
+	int dup, doff;
+	int err = 0;
+
+	if (!mp_newname || dcnt == 0)
+		return -EINVAL;
+
+	if (dcnt > MPOOL_DRIVES_MAX) {
+		err = -EINVAL;
+		mp_pr_err("first pd %s, too many drives %lu %d",
+			  err, dpaths[0], (ulong)dcnt, MPOOL_DRIVES_MAX);
+		return err;
+	}
+
+	/*
+	 * Verify no duplicate drive paths
+	 */
+	err = check_for_dups(dpaths, dcnt, &dup, &doff);
+	if (err) {
+		mp_pr_err("check_for_dups failed, dcnt %lu", err, (ulong)dcnt);
+		return err;
+	} else if (dup) {
+		err = -ENOMEM;
+		mp_pr_err("duplicate drives found", err);
+		return err;
+	}
+
+	sb = kzalloc(sizeof(*sb), GFP_KERNEL);
+	if (!sb) {
+		err = -ENOMEM;
+		mp_pr_err("alloc sb %zu failed", err, sizeof(*sb));
+		return err;
+	}
+
+	mp = mpool_desc_alloc();
+	if (!mp) {
+		err = -ENOMEM;
+		mp_pr_err("alloc mpool desc failed", err);
+		kfree(sb);
+		return err;
+	}
+
+	mp->pds_pdvcnt = 0;
+
+	mutex_lock(&mpool_s_lock);
+
+	/* Get device parm for all drive paths */
+	err = mpool_dev_init_all(mp->pds_pdv, dcnt, dpaths, pd_prop);
+	if (err) {
+		mp_pr_err("first pd %s, get device params failed", err, dpaths[0]);
+		goto errout;
+	}
+
+	/* Set pdvcnt so dpaths will get closed in cleanup if open fails.
+	 */
+	mp->pds_pdvcnt = dcnt;
+
+	for (pdh = 0; pdh < mp->pds_pdvcnt; pdh++) {
+		pd = &mp->pds_pdv[pdh];
+
+		if (mpool_pd_status_get(pd) != PD_STAT_ONLINE) {
+			err = -EIO;
+			mp_pr_err("pd %s unavailable or offline, status %d",
+				  err, pd->pdi_name, mpool_pd_status_get(pd));
+			goto errout;
+		}
+
+		/*
+		 * Read superblock; init and validate pool drive info
+		 * from device parameters stored in the super block.
+		 */
+		err = sb_read(&pd->pdi_parm, sb, &omf_ver, force);
+		if (err) {
+			mp_pr_err("pd %s, sb read failed", err, pd->pdi_name);
+			goto errout;
+		}
+
+		if (omf_ver > OMF_SB_DESC_VER_LAST ||
+		    omf_ver < OMF_SB_DESC_VER_LAST) {
+			err = -EOPNOTSUPP;
+			mp_pr_err("pd %s, invalid sb version %d %d",
+				  err, pd->pdi_name, omf_ver, OMF_SB_DESC_VER_LAST);
+			goto errout;
+		}
+
+		if (!strcmp(mp_newname, sb->osb_name))
+			continue;
+
+		strlcpy(sb->osb_name, mp_newname, sizeof(sb->osb_name));
+
+		err = sb_write_update(&pd->pdi_parm, sb);
+		if (err) {
+			mp_pr_err("Failed to rename mpool %s on device %s",
+				  err, mp->pds_name, pd->pdi_name);
+			goto errout;
+		}
+	}
+
+errout:
+	mutex_unlock(&mpool_s_lock);
+
+	mpool_desc_free(mp);
+	kfree(sb);
+
+	return err;
+}
+
+int mpool_drive_add(struct mpool_descriptor *mp, char *dpath, struct pd_prop *pd_prop)
+{
+	struct mpool_dev_info *pd;
+	struct mc_smap_parms mcsp;
+	char *dpathv[1] = { dpath };
+	bool erase = false;
+	bool smap = false;
+	int err;
+
+	/*
+	 * All device list changes are serialized via mpool_s_lock so
+	 * don't need to acquire mp.pdvlock until ready to update mpool
+	 * descriptor
+	 */
+	mutex_lock(&mpool_s_lock);
+
+	if (mp->pds_pdvcnt >= MPOOL_DRIVES_MAX) {
+		mutex_unlock(&mpool_s_lock);
+
+		mp_pr_warn("%s: pd %s, too many drives %u %d",
+			   mp->pds_name, dpath, mp->pds_pdvcnt, MPOOL_DRIVES_MAX);
+		return -EINVAL;
+	}
+
+	/*
+	 * get device parm for dpath; use next slot in mp.pdv which won't
+	 * be visible until we update mp.pdvcnt
+	 */
+	pd = &mp->pds_pdv[mp->pds_pdvcnt];
+
+	/*
+	 * Some leftover may be present due to a previous try to add a PD
+	 * at this position. Clear up.
+	 */
+	memset(pd, 0, sizeof(*pd));
+
+	err = mpool_dev_init_all(pd, 1, dpathv, pd_prop);
+	if (err) {
+		mutex_unlock(&mpool_s_lock);
+
+		mp_pr_err("%s: pd %s, getting drive params failed", err, mp->pds_name, dpath);
+		return err;
+	}
+
+	/* Confirm drive meets all criteria for adding to this mpool */
+	err = mpool_dev_check_new(mp, pd);
+	if (err) {
+		mp_pr_err("%s: pd %s, drive doesn't pass criteria", err, mp->pds_name, dpath);
+		goto errout;
+	}
+
+	/*
+	 * Check that the drive can be added in a media class.
+	 */
+	down_read(&mp->pds_pdvlock);
+	err = mpool_desc_pdmc_add(mp, mp->pds_pdvcnt, NULL, true);
+	up_read(&mp->pds_pdvlock);
+	if (err) {
+		mp_pr_err("%s: pd %s, can't place in any media class", err, mp->pds_name, dpath);
+		goto errout;
+	}
+
+
+	mpool_generate_uuid(&pd->pdi_devid);
+
+	/* Write mpool superblock to drive */
+	erase = true;
+	err = mpool_dev_sbwrite(mp, pd, NULL);
+	if (err) {
+		mp_pr_err("%s: pd %s, sb write failed", err, mp->pds_name, dpath);
+		goto errout;
+	}
+
+	/* Get percent spare */
+	down_read(&mp->pds_pdvlock);
+	err = mc_smap_parms_get(&mp->pds_mc[pd->pdi_mclass], &mp->pds_params, &mcsp);
+	up_read(&mp->pds_pdvlock);
+	if (err)
+		goto errout;
+
+	/* Alloc space map for drive */
+	err = smap_drive_init(mp, &mcsp, mp->pds_pdvcnt);
+	if (err) {
+		mp_pr_err("%s: pd %s, smap init failed", err, mp->pds_name, dpath);
+		goto errout;
+	}
+	smap = true;
+
+	/*
+	 * Take MDC0 compact lock to prevent race with MDC0 compaction.
+	 * Take it across memory and media update.
+	 */
+	PMD_MDC0_COMPACTLOCK(mp);
+
+	/*
+	 * Add drive state record to mdc0; if crash any time prior to adding
+	 * this record the drive will not be recognized as an mpool member
+	 * on next open
+	 */
+	err = pmd_prop_mcconfig(mp, pd, false);
+	if (err) {
+		PMD_MDC0_COMPACTUNLOCK(mp);
+		mp_pr_err("%s: pd %s, adding drive state to MDC0 failed", err, mp->pds_name, dpath);
+		goto errout;
+	}
+
+	/* Make new drive visible in mpool */
+	down_write(&mp->pds_pdvlock);
+	mp->pds_pdvcnt++;
+
+	/*
+	 * Add the PD in its class. That should NOT fail because we already
+	 * checked that the drive can be added in a media class.
+	 */
+	err = mpool_desc_pdmc_add(mp, mp->pds_pdvcnt - 1, NULL, false);
+	if (err)
+		mp->pds_pdvcnt--;
+
+	up_write(&mp->pds_pdvlock);
+	PMD_MDC0_COMPACTUNLOCK(mp);
+
+errout:
+	if (err) {
+		/*
+		 * No pd could have been be added at mp->pds_pdvcnt since we
+		 * dropped pds_pdvlock because mpool_s_lock is held.
+		 */
+		if (smap)
+			smap_drive_free(mp, mp->pds_pdvcnt);
+
+		/*
+		 * Erase the pd super blocks only if the pd doesn't already
+		 * belong to this mpool or another one.
+		 */
+		if (erase)
+			sb_erase(&pd->pdi_parm);
+
+		pd_dev_close(&pd->pdi_parm);
+	}
+
+	mutex_unlock(&mpool_s_lock);
+
+	return err;
+}
+
+void mpool_mclass_get_cnt(struct mpool_descriptor *mp, u32 *cnt)
+{
+	int i;
+
+	*cnt = 0;
+
+	down_read(&mp->pds_pdvlock);
+	for (i = 0; i < MP_MED_NUMBER; i++) {
+		struct media_class *mc;
+
+		mc = &mp->pds_mc[i];
+		if (mc->mc_pdmc >= 0)
+			(*cnt)++;
+	}
+	up_read(&mp->pds_pdvlock);
+}
+
+int mpool_mclass_get(struct mpool_descriptor *mp, u32 *mcxc, struct mpool_mclass_xprops *mcxv)
+{
+	int i, n;
+
+	if (!mp || !mcxc || !mcxv)
+		return -EINVAL;
+
+	mutex_lock(&mpool_s_lock);
+	down_read(&mp->pds_pdvlock);
+
+	for (n = i = 0; i < MP_MED_NUMBER && n < *mcxc; i++) {
+		struct media_class *mc;
+
+		mc = &mp->pds_mc[i];
+		if (mc->mc_pdmc < 0)
+			continue;
+
+		mcxv->mc_mclass = mc->mc_parms.mcp_classp;
+		mcxv->mc_devtype = mc->mc_parms.mcp_devtype;
+		mcxv->mc_spare = mc->mc_sparms.mcsp_spzone;
+
+		mcxv->mc_zonepg = mc->mc_parms.mcp_zonepg;
+		mcxv->mc_sectorsz = mc->mc_parms.mcp_sectorsz;
+		mcxv->mc_features = mc->mc_parms.mcp_features;
+		mcxv->mc_uacnt = mc->mc_uacnt;
+		smap_mclass_usage(mp, i, &mcxv->mc_usage);
+
+		++mcxv;
+		++n;
+	}
+
+	up_read(&mp->pds_pdvlock);
+	mutex_unlock(&mpool_s_lock);
+
+	*mcxc = n;
+
+	return 0;
+}
+
+int mpool_drive_spares(struct mpool_descriptor *mp, enum mp_media_classp mclassp, u8 drive_spares)
+{
+	struct media_class *mc;
+	int err;
+
+	if (!mclass_isvalid(mclassp) || drive_spares > 100) {
+		err = -EINVAL;
+		mp_pr_err("mpool %s, setting percent %u spare for drives in media class %d failed",
+			  err, mp->pds_name, drive_spares, mclassp);
+		return err;
+	}
+
+	/*
+	 * Do not write the spare record or try updating spare if there are
+	 * no PDs in the specified media class.
+	 */
+	down_read(&mp->pds_pdvlock);
+	mc = &mp->pds_mc[mclassp];
+	up_read(&mp->pds_pdvlock);
+
+	if (mc->mc_pdmc < 0) {
+		err = -ENOENT;
+		goto skip_update;
+	}
+
+	mutex_lock(&mpool_s_lock);
+
+	/*
+	 * Take mdc0 compact lock to prevent race with mdc0 compaction.
+	 * Also make memory and media update to look atomic to compaction.
+	 */
+	PMD_MDC0_COMPACTLOCK(mp);
+
+	/*
+	 * update media class spare record in mdc0; no effect if crash before
+	 * complete
+	 */
+	err = pmd_prop_mcspare(mp, mclassp, drive_spares, false);
+	if (err) {
+		mp_pr_err("mpool %s, setting spare %u mclass %d failed, could not record in MDC0",
+			  err, mp->pds_name, drive_spares, mclassp);
+	} else {
+		/* Update spare zone accounting for media class */
+		down_write(&mp->pds_pdvlock);
+
+		err = mc_set_spzone(&mp->pds_mc[mclassp], drive_spares);
+		if (err)
+			mp_pr_err("mpool %s, setting spare %u mclass %d failed",
+				  err, mp->pds_name, drive_spares, mclassp);
+		else
+			/*
+			 * smap accounting update always succeeds when
+			 * mclassp/zone are valid
+			 */
+			smap_drive_spares(mp, mclassp, drive_spares);
+
+		up_write(&mp->pds_pdvlock);
+	}
+
+	PMD_MDC0_COMPACTUNLOCK(mp);
+
+	mutex_unlock(&mpool_s_lock);
+
+skip_update:
+	return err;
+}
+
+void mpool_get_xprops(struct mpool_descriptor *mp, struct mpool_xprops *xprops)
+{
+	struct media_class *mc;
+	int mclassp, i;
+	u16 ftmax;
+
+	mutex_lock(&mpool_s_lock);
+	down_read(&mp->pds_pdvlock);
+
+	memcpy(xprops->ppx_params.mp_poolid.b, mp->pds_poolid.uuid, MPOOL_UUID_SIZE);
+	ftmax = 0;
+
+	for (mclassp = 0; mclassp < MP_MED_NUMBER; mclassp++) {
+		xprops->ppx_pd_mclassv[mclassp] = MP_MED_INVALID;
+
+		mc = &mp->pds_mc[mclassp];
+		if (mc->mc_pdmc < 0) {
+			xprops->ppx_drive_spares[mclassp] = 0;
+			xprops->ppx_uacnt[mclassp] = 0;
+
+			xprops->ppx_params.mp_mblocksz[mclassp] = 0;
+			continue;
+		}
+
+		xprops->ppx_drive_spares[mclassp] = mc->mc_sparms.mcsp_spzone;
+		xprops->ppx_uacnt[mclassp] = mc->mc_uacnt;
+		ftmax = max((u16)ftmax, (u16)(xprops->ppx_uacnt[mclassp]));
+		xprops->ppx_params.mp_mblocksz[mclassp] =
+			(mc->mc_parms.mcp_zonepg << PAGE_SHIFT) >> 20;
+	}
+
+	for (i = 0; i < mp->pds_pdvcnt; ++i) {
+		mc = &mp->pds_mc[mp->pds_pdv[i].pdi_mclass];
+		if (mc->mc_pdmc < 0)
+			continue;
+
+		xprops->ppx_pd_mclassv[i] = mc->mc_parms.mcp_classp;
+
+		strlcpy(xprops->ppx_pd_namev[i], mp->pds_pdv[i].pdi_name,
+			sizeof(xprops->ppx_pd_namev[i]));
+	}
+
+	up_read(&mp->pds_pdvlock);
+	mutex_unlock(&mpool_s_lock);
+
+	xprops->ppx_params.mp_stat = ftmax ? MPOOL_STAT_FAULTED : MPOOL_STAT_OPTIMAL;
+}
+
+int mpool_get_devprops_by_name(struct mpool_descriptor *mp, char *pdname,
+			       struct mpool_devprops *dprop)
+{
+	int i;
+
+	down_read(&mp->pds_pdvlock);
+
+	for (i = 0; i < mp->pds_pdvcnt; i++) {
+		if (!strcmp(pdname, mp->pds_pdv[i].pdi_name))
+			fill_in_devprops(mp, i, dprop);
+	}
+
+	up_read(&mp->pds_pdvlock);
+
+	return 0;
+}
+
+void mpool_get_usage(struct mpool_descriptor *mp, enum mp_media_classp mclassp,
+		     struct mpool_usage *usage)
+{
+	memset(usage, 0, sizeof(*usage));
+
+	down_read(&mp->pds_pdvlock);
+	if (mclassp != MP_MED_ALL) {
+		struct media_class *mc;
+
+		ASSERT(mclassp < MP_MED_NUMBER);
+
+		mc = &mp->pds_mc[mclassp];
+		if (mc->mc_pdmc < 0) {
+			/* Not an error, this media class is empty. */
+			up_read(&mp->pds_pdvlock);
+			return;
+		}
+	}
+	smap_mpool_usage(mp, mclassp, usage);
+	up_read(&mp->pds_pdvlock);
+
+	if (mclassp == MP_MED_ALL)
+		pmd_mpool_usage(mp, usage);
+}
+
+int mpool_config_store(struct mpool_descriptor *mp, const struct mpool_config *cfg)
+{
+	int err;
+
+	if (!mp || !cfg)
+		return -EINVAL;
+
+	mp->pds_cfg = *cfg;
+
+	err = pmd_prop_mpconfig(mp, cfg, false);
+	if (err)
+		mp_pr_err("mpool %s, logging config record failed", err, mp->pds_name);
+
+	return err;
+}
+
+int mpool_config_fetch(struct mpool_descriptor *mp, struct mpool_config *cfg)
+{
+	if (!mp || !cfg)
+		return -EINVAL;
+
+	*cfg = mp->pds_cfg;
+
+	return 0;
+}
-- 
2.17.2



From nmeeramohide at micron.com  Mon Oct 12 12:27:32 2020
From: nmeeramohide at micron.com (Nabeel M Mohamed)
Date: Mon, 12 Oct 2020 11:27:32 -0500
Subject: [PATCH v2 18/22] mpool: add object lifecycle management ioctls
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201012162736.65241-19-nmeeramohide@micron.com>

This adds the mblock and mlog management ioctls: alloc, commit,
abort, destroy, read, write, fetch properties etc.

The mblock and mlog management ioctl handlers are thin wrappers
around the core mblock/mlog lifecycle management and IO routines
introduced in an earlier patch.

The object read/write ioctl handlers utilizes vcache, which is a
small cache of iovec objects and page pointers. This cache is
used for large mblock/mlog IO. It acts as an emergency memory
pool for handling IO requests under memory pressure thereby
reducing tail latencies.

Co-developed-by: Greg Becker <gbecker at micron.com>
Signed-off-by: Greg Becker <gbecker at micron.com>
Co-developed-by: Pierre Labat <plabat at micron.com>
Signed-off-by: Pierre Labat <plabat at micron.com>
Co-developed-by: John Groves <jgroves at micron.com>
Signed-off-by: John Groves <jgroves at micron.com>
Signed-off-by: Nabeel M Mohamed <nmeeramohide at micron.com>
---
 drivers/mpool/mpctl.c | 670 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 667 insertions(+), 3 deletions(-)

diff --git a/drivers/mpool/mpctl.c b/drivers/mpool/mpctl.c
index de62f9a5524d..3a231cf982b3 100644
--- a/drivers/mpool/mpctl.c
+++ b/drivers/mpool/mpctl.c
@@ -33,6 +33,7 @@
 #include "assert.h"
 
 #include "mpool_ioctl.h"
+#include "mblock.h"
 #include "mlog.h"
 #include "mp.h"
 #include "mpctl.h"
@@ -1299,7 +1300,6 @@ static int mpioc_mp_activate(struct mpc_unit *ctl, struct mpioc_mpool *mp,
 	mp->mp_params.mp_oidv[0] = cfg.mc_oid1;
 	mp->mp_params.mp_oidv[1] = cfg.mc_oid2;
 	mp->mp_params.mp_ra_pages_max = cfg.mc_ra_pages_max;
-	mp->mp_params.mp_vma_size_max = cfg.mc_vma_size_max;
 	memcpy(&mp->mp_params.mp_utype, &cfg.mc_utype, sizeof(mp->mp_params.mp_utype));
 	strlcpy(mp->mp_params.mp_label, cfg.mc_label, sizeof(mp->mp_params.mp_label));
 
@@ -1656,6 +1656,596 @@ static int mpioc_mp_add(struct mpc_unit *unit, struct mpioc_drive *drv)
 	return rc;
 }
 
+
+/**
+ * struct vcache -  very-large-buffer cache...
+ */
+struct vcache {
+	spinlock_t  vc_lock;
+	void       *vc_head;
+	size_t      vc_size;
+} ____cacheline_aligned;
+
+static struct vcache mpc_physio_vcache;
+
+static void *mpc_vcache_alloc(struct vcache *vc, size_t sz)
+{
+	void *p;
+
+	if (!vc || sz > vc->vc_size)
+		return NULL;
+
+	spin_lock(&vc->vc_lock);
+	p = vc->vc_head;
+	if (p)
+		vc->vc_head = *(void **)p;
+	spin_unlock(&vc->vc_lock);
+
+	return p;
+}
+
+static void mpc_vcache_free(struct vcache *vc, void *p)
+{
+	if (!vc || !p)
+		return;
+
+	spin_lock(&vc->vc_lock);
+	*(void **)p = vc->vc_head;
+	vc->vc_head = p;
+	spin_unlock(&vc->vc_lock);
+}
+
+static int mpc_vcache_init(struct vcache *vc, size_t sz, size_t n)
+{
+	if (!vc || sz < PAGE_SIZE || n < 1)
+		return -EINVAL;
+
+	spin_lock_init(&vc->vc_lock);
+	vc->vc_head = NULL;
+	vc->vc_size = sz;
+
+	while (n-- > 0)
+		mpc_vcache_free(vc, vmalloc(sz));
+
+	return vc->vc_head ? 0 : -ENOMEM;
+}
+
+static void mpc_vcache_fini(struct vcache *vc)
+{
+	void *p;
+
+	while ((p = mpc_vcache_alloc(vc, PAGE_SIZE)))
+		vfree(p);
+}
+
+/**
+ * mpc_physio() - Generic raw device mblock read/write routine.
+ * @mpd:      mpool descriptor
+ * @desc:     mblock or mlog descriptor
+ * @uiov:     vector of iovecs that describe user-space segments
+ * @uioc:     count of elements in uiov[]
+ * @offset:   offset into the mblock at which to start reading
+ * @objtype:  mblock or mlog
+ * @rw:       READ or WRITE in regards to the media.
+ * @stkbuf:   caller provided scratch space
+ * @stkbufsz: size of stkbuf
+ *
+ * This function creates an array of iovec objects each of which
+ * map a portion of the user request into kernel space so that
+ * mpool can directly access the user data.  Note that this is
+ * a zero-copy operation.
+ *
+ * Requires that each user-space segment be page aligned and of an
+ * integral number of pages.
+ *
+ * See http://www.makelinux.net/ldd3/chp-15-sect-3 for more detail.
+ */
+static int mpc_physio(struct mpool_descriptor *mpd, void *desc, struct iovec *uiov,
+		      int uioc, off_t offset, enum mp_obj_type objtype, int rw,
+		      void *stkbuf, size_t stkbufsz)
+{
+	struct kvec *iov_base, *iov;
+	struct iov_iter iter;
+	struct page **pagesv;
+	size_t pagesvsz, pgbase, length;
+	int pagesc, niov, rc, i;
+	ssize_t cc;
+
+	iov = NULL;
+	niov = 0;
+	rc = 0;
+
+	length = iov_length(uiov, uioc);
+
+	if (length < PAGE_SIZE || !IS_ALIGNED(length, PAGE_SIZE))
+		return -EINVAL;
+
+	if (length > (rwsz_max_mb << 20))
+		return -EINVAL;
+
+	/*
+	 * Allocate an array of page pointers for iov_iter_get_pages()
+	 * and an array of iovecs for mblock_read() and mblock_write().
+	 *
+	 * Note: the only way we can calculate the number of required
+	 * iovecs in advance is to assume that we need one per page.
+	 */
+	pagesc = length / PAGE_SIZE;
+	pagesvsz = (sizeof(*pagesv) + sizeof(*iov)) * pagesc;
+
+	/*
+	 * pagesvsz may be big, and it will not be used as the iovec_list
+	 * for the block stack - pd will chunk it up to the underlying
+	 * devices (with another iovec list per pd).
+	 */
+	if (pagesvsz > stkbufsz) {
+		pagesv = NULL;
+
+		if (pagesvsz <= PAGE_SIZE * 2)
+			pagesv = kmalloc(pagesvsz, GFP_NOIO);
+
+		while (!pagesv) {
+			pagesv = mpc_vcache_alloc(&mpc_physio_vcache, pagesvsz);
+			if (!pagesv)
+				usleep_range(750, 1250);
+		}
+	} else {
+		pagesv = stkbuf;
+	}
+
+	if (!pagesv)
+		return -ENOMEM;
+
+	iov_base = (struct kvec *)((char *)pagesv + (sizeof(*pagesv) * pagesc));
+
+	iov_iter_init(&iter, rw, uiov, uioc, length);
+
+	for (i = 0, cc = 0; i < pagesc; i += (cc / PAGE_SIZE)) {
+
+		/* Get struct page vector for the user buffers. */
+		cc = iov_iter_get_pages(&iter, &pagesv[i], length - (i * PAGE_SIZE),
+					pagesc - i, &pgbase);
+		if (cc < 0) {
+			rc = cc;
+			pagesc = i;
+			goto errout;
+		}
+
+		/*
+		 * pgbase is the offset into the 1st iovec - our alignment
+		 * requirements force it to be 0
+		 */
+		if (cc < PAGE_SIZE || pgbase != 0) {
+			rc = -EINVAL;
+			pagesc = i + 1;
+			goto errout;
+		}
+
+		iov_iter_advance(&iter, cc);
+	}
+
+	/* Build an array of iovecs for mpool so that it can directly access the user data. */
+	for (i = 0, iov = iov_base; i < pagesc; ++i, ++iov, ++niov) {
+		iov->iov_len = PAGE_SIZE;
+		iov->iov_base = kmap(pagesv[i]);
+
+		if (!iov->iov_base) {
+			rc = -EINVAL;
+			pagesc = i + 1;
+			goto errout;
+		}
+	}
+
+	switch (objtype) {
+	case MP_OBJ_MBLOCK:
+		if (rw == WRITE)
+			rc = mblock_write(mpd, desc, iov_base, niov, pagesc << PAGE_SHIFT);
+		else
+			rc = mblock_read(mpd, desc, iov_base, niov, offset, pagesc << PAGE_SHIFT);
+		break;
+
+	case MP_OBJ_MLOG:
+		rc = mlog_rw_raw(mpd, desc, iov_base, niov, offset, rw);
+		break;
+
+	default:
+		rc = -EINVAL;
+		goto errout;
+	}
+
+errout:
+	for (i = 0, iov = iov_base; i < pagesc; ++i, ++iov) {
+		if (i < niov)
+			kunmap(pagesv[i]);
+		put_page(pagesv[i]);
+	}
+
+	if (pagesvsz > stkbufsz) {
+		if (pagesvsz > PAGE_SIZE * 2)
+			mpc_vcache_free(&mpc_physio_vcache, pagesv);
+		else
+			kfree(pagesv);
+	}
+
+	return rc;
+}
+
+/**
+ * mpioc_mb_alloc() - Allocate an mblock object.
+ * @unit:   mpool unit ptr
+ * @mb:     mblock parameter block
+ *
+ * MPIOC_MB_ALLOC ioctl handler to allocate a single mblock.
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_mb_alloc(struct mpc_unit *unit, struct mpioc_mblock *mb)
+{
+	struct mblock_descriptor *mblock;
+	struct mpool_descriptor *mpool;
+	struct mblock_props props;
+	int rc;
+
+	if (!unit || !mb || !unit->un_mpool)
+		return -EINVAL;
+
+	mpool = unit->un_mpool->mp_desc;
+
+	rc = mblock_alloc(mpool, mb->mb_mclassp, mb->mb_spare, &mblock, &props);
+	if (rc)
+		return rc;
+
+	mblock_get_props_ex(mpool, mblock, &mb->mb_props);
+	mblock_put(mblock);
+
+	mb->mb_objid  = props.mpr_objid;
+	mb->mb_offset = -1;
+
+	return 0;
+}
+
+/**
+ * mpioc_mb_find() - Find an mblock object by its objid
+ * @unit:   mpool unit ptr
+ * @mb:     mblock parameter block
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_mb_find(struct mpc_unit *unit, struct mpioc_mblock *mb)
+{
+	struct mblock_descriptor *mblock;
+	struct mpool_descriptor *mpool;
+	int rc;
+
+	if (!unit || !mb || !unit->un_mpool)
+		return -EINVAL;
+
+	if (!mblock_objid(mb->mb_objid))
+		return -EINVAL;
+
+	mpool = unit->un_mpool->mp_desc;
+
+	rc = mblock_find_get(mpool, mb->mb_objid, 0, NULL, &mblock);
+	if (rc)
+		return rc;
+
+	(void)mblock_get_props_ex(mpool, mblock, &mb->mb_props);
+
+	mblock_put(mblock);
+
+	mb->mb_offset = -1;
+
+	return 0;
+}
+
+/**
+ * mpioc_mb_abcomdel() - Abort, commit, or delete an mblock.
+ * @unit:   mpool unit ptr
+ * @cmd     MPIOC_MB_ABORT, MPIOC_MB_COMMIT, or MPIOC_MB_DELETE
+ * @mi:     mblock parameter block
+ *
+ * MPIOC_MB_ACD ioctl handler to either abort, commit, or delete
+ * the specified mblock.
+ *
+ * Return:  Returns 0 if successful, -errno otherwise...
+ */
+static int mpioc_mb_abcomdel(struct mpc_unit *unit, uint cmd, struct mpioc_mblock_id *mi)
+{
+	struct mblock_descriptor *mblock;
+	struct mpool_descriptor *mpool;
+	int which, rc;
+	bool drop;
+
+	if (!unit || !mi || !unit->un_mpool)
+		return -EINVAL;
+
+	if (!mblock_objid(mi->mi_objid))
+		return -EINVAL;
+
+	which = (cmd == MPIOC_MB_DELETE) ? 1 : -1;
+	mpool = unit->un_mpool->mp_desc;
+	drop = true;
+
+	rc = mblock_find_get(mpool, mi->mi_objid, which, NULL, &mblock);
+	if (rc)
+		return rc;
+
+	switch (cmd) {
+	case MPIOC_MB_COMMIT:
+		rc = mblock_commit(mpool, mblock);
+		break;
+
+	case MPIOC_MB_ABORT:
+		rc = mblock_abort(mpool, mblock);
+		drop = !!rc;
+		break;
+
+	case MPIOC_MB_DELETE:
+		rc = mblock_delete(mpool, mblock);
+		drop = !!rc;
+		break;
+
+	default:
+		rc = -ENOTTY;
+		break;
+	}
+
+	if (drop)
+		mblock_put(mblock);
+
+	return rc;
+}
+
+/**
+ * mpioc_mb_rw() - read/write mblock ioctl handler
+ * @unit:   mpool unit ptr
+ * @cmd:    MPIOC_MB_READ or MPIOC_MB_WRITE
+ * @mbiov:  mblock parameter block
+ */
+static int mpioc_mb_rw(struct mpc_unit *unit, uint cmd, struct mpioc_mblock_rw *mbrw,
+		       void *stkbuf, size_t stkbufsz)
+{
+	struct mblock_descriptor *mblock;
+	struct mpool_descriptor *mpool;
+	struct iovec *kiov;
+	bool xfree = false;
+	int which, rc;
+	size_t kiovsz;
+
+	if (!unit || !mbrw || !unit->un_mpool)
+		return -EINVAL;
+
+	if (!mblock_objid(mbrw->mb_objid))
+		return -EINVAL;
+
+	/*
+	 * For small iovec counts we simply copyin the array of iovecs
+	 * to local storage (stkbuf).  Otherwise, we must kmalloc a
+	 * buffer into which to perform the copyin.
+	 */
+	if (mbrw->mb_iov_cnt > MPIOC_KIOV_MAX)
+		return -EINVAL;
+
+	kiovsz = mbrw->mb_iov_cnt * sizeof(*kiov);
+
+	if (kiovsz > stkbufsz) {
+		kiov = kmalloc(kiovsz, GFP_KERNEL);
+		if (!kiov)
+			return -ENOMEM;
+
+		xfree = true;
+	} else {
+		kiov = stkbuf;
+		stkbuf += kiovsz;
+		stkbufsz -= kiovsz;
+	}
+
+	which = (cmd == MPIOC_MB_READ) ? 1 : -1;
+	mpool = unit->un_mpool->mp_desc;
+
+	rc = mblock_find_get(mpool, mbrw->mb_objid, which, NULL, &mblock);
+	if (rc)
+		goto errout;
+
+	if (copy_from_user(kiov, mbrw->mb_iov, kiovsz)) {
+		rc = -EFAULT;
+	} else {
+		rc = mpc_physio(mpool, mblock, kiov, mbrw->mb_iov_cnt, mbrw->mb_offset,
+				MP_OBJ_MBLOCK, (cmd == MPIOC_MB_READ) ? READ : WRITE,
+				stkbuf, stkbufsz);
+	}
+
+	mblock_put(mblock);
+
+errout:
+	if (xfree)
+		kfree(kiov);
+
+	return rc;
+}
+
+/*
+ * Mpctl mlog ioctl handlers
+ */
+static int mpioc_mlog_alloc(struct mpc_unit *unit, struct mpioc_mlog *ml)
+{
+	struct mpool_descriptor *mpool;
+	struct mlog_descriptor *mlog;
+	struct mlog_props props;
+	int rc;
+
+	if (!unit || !unit->un_mpool || !ml)
+		return -EINVAL;
+
+	mpool = unit->un_mpool->mp_desc;
+
+	rc = mlog_alloc(mpool, &ml->ml_cap, ml->ml_mclassp, &props, &mlog);
+	if (rc)
+		return rc;
+
+	mlog_get_props_ex(mpool, mlog, &ml->ml_props);
+	mlog_put(mlog);
+
+	ml->ml_objid = props.lpr_objid;
+
+	return 0;
+}
+
+static int mpioc_mlog_find(struct mpc_unit *unit, struct mpioc_mlog *ml)
+{
+	struct mpool_descriptor    *mpool;
+	struct mlog_descriptor     *mlog;
+	int rc;
+
+	if (!unit || !unit->un_mpool || !ml || !mlog_objid(ml->ml_objid))
+		return -EINVAL;
+
+	mpool = unit->un_mpool->mp_desc;
+
+	rc = mlog_find_get(mpool, ml->ml_objid, 0, NULL, &mlog);
+	if (!rc) {
+		rc = mlog_get_props_ex(mpool, mlog, &ml->ml_props);
+		mlog_put(mlog);
+	}
+
+	return rc;
+}
+
+static int mpioc_mlog_abcomdel(struct mpc_unit *unit, uint cmd, struct mpioc_mlog_id *mi)
+{
+	struct mpool_descriptor *mpool;
+	struct mlog_descriptor *mlog;
+	struct mlog_props_ex props;
+	int which, rc;
+	bool drop;
+
+	if (!unit || !unit->un_mpool || !mi || !mlog_objid(mi->mi_objid))
+		return -EINVAL;
+
+	which = (cmd == MPIOC_MLOG_DELETE) ? 1 : -1;
+	mpool = unit->un_mpool->mp_desc;
+	drop = true;
+
+	rc = mlog_find_get(mpool, mi->mi_objid, which, NULL, &mlog);
+	if (rc)
+		return rc;
+
+	switch (cmd) {
+	case MPIOC_MLOG_COMMIT:
+		rc = mlog_commit(mpool, mlog);
+		if (!rc) {
+			mlog_get_props_ex(mpool, mlog, &props);
+			mi->mi_gen   = props.lpx_props.lpr_gen;
+			mi->mi_state = props.lpx_state;
+		}
+		break;
+
+	case MPIOC_MLOG_ABORT:
+		rc = mlog_abort(mpool, mlog);
+		drop = !!rc;
+		break;
+
+	case MPIOC_MLOG_DELETE:
+		rc = mlog_delete(mpool, mlog);
+		drop = !!rc;
+		break;
+
+	default:
+		rc = -ENOTTY;
+		break;
+	}
+
+	if (drop)
+		mlog_put(mlog);
+
+	return rc;
+}
+
+static int mpioc_mlog_rw(struct mpc_unit *unit, struct mpioc_mlog_io *mi,
+			 void *stkbuf, size_t stkbufsz)
+{
+	struct mpool_descriptor *mpool;
+	struct mlog_descriptor *mlog;
+	struct iovec *kiov;
+	bool xfree = false;
+	size_t kiovsz;
+	int rc;
+
+	if (!unit || !unit->un_mpool || !mi || !mlog_objid(mi->mi_objid))
+		return -EINVAL;
+
+	/*
+	 * For small iovec counts we simply copyin the array of iovecs
+	 * to the stack (kiov_buf). Otherwise, we must kmalloc a
+	 * buffer into which to perform the copyin.
+	 */
+	if (mi->mi_iovc > MPIOC_KIOV_MAX)
+		return -EINVAL;
+
+	kiovsz = mi->mi_iovc * sizeof(*kiov);
+
+	if (kiovsz > stkbufsz) {
+		kiov = kmalloc(kiovsz, GFP_KERNEL);
+		if (!kiov)
+			return -ENOMEM;
+
+		xfree = true;
+	} else {
+		kiov = stkbuf;
+		stkbuf += kiovsz;
+		stkbufsz -= kiovsz;
+	}
+
+	mpool = unit->un_mpool->mp_desc;
+
+	rc = mlog_find_get(mpool, mi->mi_objid, 1, NULL, &mlog);
+	if (rc)
+		goto errout;
+
+	if (copy_from_user(kiov, mi->mi_iov, kiovsz)) {
+		rc = -EFAULT;
+	} else {
+		rc = mpc_physio(mpool, mlog, kiov, mi->mi_iovc, mi->mi_off, MP_OBJ_MLOG,
+				(mi->mi_op == MPOOL_OP_READ) ? READ : WRITE, stkbuf, stkbufsz);
+	}
+
+	mlog_put(mlog);
+
+errout:
+	if (xfree)
+		kfree(kiov);
+
+	return rc;
+}
+
+static int mpioc_mlog_erase(struct mpc_unit *unit, struct mpioc_mlog_id *mi)
+{
+	struct mpool_descriptor *mpool;
+	struct mlog_descriptor *mlog;
+	struct mlog_props_ex props;
+	int rc;
+
+	if (!unit || !unit->un_mpool || !mi || !mlog_objid(mi->mi_objid))
+		return -EINVAL;
+
+	mpool = unit->un_mpool->mp_desc;
+
+	rc = mlog_find_get(mpool, mi->mi_objid, 0, NULL, &mlog);
+	if (rc)
+		return rc;
+
+	rc = mlog_erase(mpool, mlog, mi->mi_gen);
+	if (!rc) {
+		mlog_get_props_ex(mpool, mlog, &props);
+		mi->mi_gen   = props.lpx_props.lpr_gen;
+		mi->mi_state = props.lpx_state;
+	}
+
+	mlog_put(mlog);
+
+	return rc;
+}
+
 static struct mpc_softstate *mpc_cdev2ss(struct cdev *cdev)
 {
 	if (!cdev || cdev->owner != THIS_MODULE) {
@@ -1798,8 +2388,8 @@ static long mpc_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
 {
 	char argbuf[256] __aligned(16);
 	struct mpc_unit *unit;
-	size_t argbufsz;
-	void *argp;
+	size_t argbufsz, stkbufsz;
+	void *argp, *stkbuf;
 	ulong iosz;
 	int rc;
 
@@ -1810,7 +2400,12 @@ static long mpc_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
 		switch (cmd) {
 		case MPIOC_PROP_GET:
 		case MPIOC_DEVPROPS_GET:
+		case MPIOC_MB_FIND:
+		case MPIOC_MB_READ:
 		case MPIOC_MP_MCLASS_GET:
+		case MPIOC_MLOG_FIND:
+		case MPIOC_MLOG_READ:
+		case MPIOC_MLOG_PROPS:
 			break;
 
 		default:
@@ -1882,6 +2477,59 @@ static long mpc_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
 		rc = mpioc_devprops_get(unit, argp);
 		break;
 
+	case MPIOC_MB_ALLOC:
+		rc = mpioc_mb_alloc(unit, argp);
+		break;
+
+	case MPIOC_MB_FIND:
+		rc = mpioc_mb_find(unit, argp);
+		break;
+
+	case MPIOC_MB_COMMIT:
+	case MPIOC_MB_DELETE:
+	case MPIOC_MB_ABORT:
+		rc = mpioc_mb_abcomdel(unit, cmd, argp);
+		break;
+
+	case MPIOC_MB_READ:
+	case MPIOC_MB_WRITE:
+		ASSERT(roundup(iosz, 16) < argbufsz);
+
+		stkbufsz = argbufsz - roundup(iosz, 16);
+		stkbuf = argbuf + roundup(iosz, 16);
+
+		rc = mpioc_mb_rw(unit, cmd, argp, stkbuf, stkbufsz);
+		break;
+
+	case MPIOC_MLOG_ALLOC:
+		rc = mpioc_mlog_alloc(unit, argp);
+		break;
+
+	case MPIOC_MLOG_FIND:
+	case MPIOC_MLOG_PROPS:
+		rc = mpioc_mlog_find(unit, argp);
+		break;
+
+	case MPIOC_MLOG_ABORT:
+	case MPIOC_MLOG_COMMIT:
+	case MPIOC_MLOG_DELETE:
+		rc = mpioc_mlog_abcomdel(unit, cmd, argp);
+		break;
+
+	case MPIOC_MLOG_READ:
+	case MPIOC_MLOG_WRITE:
+		ASSERT(roundup(iosz, 16) < argbufsz);
+
+		stkbufsz = argbufsz - roundup(iosz, 16);
+		stkbuf = argbuf + roundup(iosz, 16);
+
+		rc = mpioc_mlog_rw(unit, argp, stkbuf, stkbufsz);
+		break;
+
+	case MPIOC_MLOG_ERASE:
+		rc = mpioc_mlog_erase(unit, argp);
+		break;
+
 	default:
 		rc = -ENOTTY;
 		mp_pr_rl("invalid command %x: dir=%u type=%c nr=%u size=%u",
@@ -1936,6 +2584,8 @@ void mpctl_exit(void)
 
 		ss->ss_inited = false;
 	}
+
+	mpc_vcache_fini(&mpc_physio_vcache);
 }
 
 /**
@@ -1947,6 +2597,7 @@ int mpctl_init(void)
 	struct mpool_config *cfg = NULL;
 	struct mpc_unit *ctlunit;
 	const char *errmsg = NULL;
+	size_t sz;
 	int rc;
 
 	if (ss->ss_inited)
@@ -1956,6 +2607,19 @@ int mpctl_init(void)
 
 	maxunits = clamp_t(uint, maxunits, 8, 8192);
 
+	rwsz_max_mb = clamp_t(ulong, rwsz_max_mb, 1, 128);
+	rwconc_max = clamp_t(ulong, rwconc_max, 1, 32);
+
+	/* Must be same as mpc_physio() pagesvsz calculation. */
+	sz = (rwsz_max_mb << 20) / PAGE_SIZE;
+	sz *= (sizeof(void *) + sizeof(struct iovec));
+
+	rc = mpc_vcache_init(&mpc_physio_vcache, sz, rwconc_max);
+	if (rc) {
+		errmsg = "vcache init failed";
+		goto errout;
+	}
+
 	cdev_init(&ss->ss_cdev, &mpc_fops_default);
 	ss->ss_cdev.owner = THIS_MODULE;
 
-- 
2.17.2



From rdunlap at infradead.org  Mon Oct 12 12:45:52 2020
From: rdunlap at infradead.org (Randy Dunlap)
Date: Mon, 12 Oct 2020 09:45:52 -0700
Subject: [PATCH v2 01/22] mpool: add utility routines and ioctl definitions
In-Reply-To: <20201012162736.65241-2-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201012162736.65241-2-nmeeramohide@micron.com>
Message-ID: <954153c4-ee2f-2a2f-8643-8ff67d8cd649@infradead.org>

On 10/12/20 9:27 AM, Nabeel M Mohamed wrote:
> +#define MPIOC_MAGIC             ('2')

Hi,

That value should be documented in
Documentation/userspace-api/ioctl/ioctl-number.rst.

thanks.
-- 
~Randy



From rdunlap at infradead.org  Mon Oct 12 12:48:25 2020
From: rdunlap at infradead.org (Randy Dunlap)
Date: Mon, 12 Oct 2020 09:48:25 -0700
Subject: [PATCH v2 01/22] mpool: add utility routines and ioctl definitions
In-Reply-To: <954153c4-ee2f-2a2f-8643-8ff67d8cd649@infradead.org>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201012162736.65241-2-nmeeramohide@micron.com>
 <954153c4-ee2f-2a2f-8643-8ff67d8cd649@infradead.org>
Message-ID: <bcc66e64-a19a-cb40-34cd-e71fa983b861@infradead.org>

On 10/12/20 9:45 AM, Randy Dunlap wrote:
> On 10/12/20 9:27 AM, Nabeel M Mohamed wrote:
>> +#define MPIOC_MAGIC             ('2')
> 
> Hi,
> 
> That value should be documented in
> Documentation/userspace-api/ioctl/ioctl-number.rst.


Sorry, I see it now.

thanks.

-- 
~Randy



From rdunlap at infradead.org  Mon Oct 12 12:53:31 2020
From: rdunlap at infradead.org (Randy Dunlap)
Date: Mon, 12 Oct 2020 09:53:31 -0700
Subject: [PATCH v2 21/22] mpool: add documentation
In-Reply-To: <20201012162736.65241-22-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201012162736.65241-22-nmeeramohide@micron.com>
Message-ID: <57fef732-f63d-9c9c-e368-660c654a7438@infradead.org>


> diff --git a/drivers/mpool/mpool-locking.rst b/drivers/mpool/mpool-locking.rst
> new file mode 100644
> index 000000000000..6a5da727f2fb
> --- /dev/null
> +++ b/drivers/mpool/mpool-locking.rst
> @@ -0,0 +1,90 @@

> +Object Layout Reference Counts
> +------------------------------
> +
> +The reference counts for an object layout (eld_ref) are protected
> +by mmi_co_lock or mmi_uc_lock of the object's MDC dependiing upon

                                                     depending

> +which tree it is in at the time of acquisition.

-- 
~Randy



From kbusch at kernel.org  Mon Oct 12 13:13:33 2020
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 12 Oct 2020 10:13:33 -0700
Subject: NVME, isolcpus, and irq affinity
In-Reply-To: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
References: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
Message-ID: <20201012171333.GA1032168@dhcp-10-100-145-180.wdl.wdc.com>

On Mon, Oct 12, 2020 at 09:49:38AM -0600, Chris Friesen wrote:
> I've got a linux system running the RT kernel with threaded irqs.? On
> startup we affine the various irq threads to the housekeeping CPUs, but I
> recently hit a scenario where after some days of uptime we ended up with a
> number of NVME irq threads affined to application cores instead (not good
> when we're trying to run low-latency applications).
> 
> Looking at the code, it appears that the NVME driver can in some scenarios
> call nvme_setup_io_queues() after the initial setup and thus allocate new
> IRQ threads at runtime.? It appears that this will then call
> pci_alloc_irq_vectors_affinity(), 

Yes, the driver will re-run interrupt setup on a controller reset.

> which seems to determine affinity without any regard for things like
> "isolcpus" or "cset shield".
>
> There seem to be other reports of similar issues:
> 
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1831566
> 
> Am I worried about nothing, or is there a risk that those irq threads would
> actually need to do real work (which would cause unacceptable jitter in my
> application)?
> 
> Assuming I'm reading the code correctly, how does it make sense for the NVME
> driver to affine interrupts to CPUs which have explicitly been designated as
> "isolated"?

The driver allocates interrupts, but doesn't affine them. The driver
lets the kernel handle that instead.


From chris.friesen at windriver.com  Mon Oct 12 14:50:49 2020
From: chris.friesen at windriver.com (Chris Friesen)
Date: Mon, 12 Oct 2020 12:50:49 -0600
Subject: NVME, isolcpus, and irq affinity
In-Reply-To: <20201012171333.GA1032168@dhcp-10-100-145-180.wdl.wdc.com>
References: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
 <20201012171333.GA1032168@dhcp-10-100-145-180.wdl.wdc.com>
Message-ID: <00547cf5-0df0-d027-bffa-adfbe247f09d@windriver.com>

On 10/12/2020 11:13 AM, Keith Busch wrote:
> On Mon, Oct 12, 2020 at 09:49:38AM -0600, Chris Friesen wrote:

>> Assuming I'm reading the code correctly, how does it make sense for the NVME
>> driver to affine interrupts to CPUs which have explicitly been designated as
>> "isolated"?
> 
> The driver allocates interrupts, but doesn't affine them. The driver
> lets the kernel handle that instead.

Okay, thanks for the quick reply.

The interrupts in question have names of the form "nvmeXqY", where X and 
Y are integers.  Some of them are affined to individual CPUs while 
others are affined to two or more CPUs.

If no tasks on the CPUs in question do any disk I/O is it possible these 
interrupts could still get triggered by activity instigated by tasks on 
other CPUs?  Or would they basically be idle and inert in that case?

Thanks,

Chris


From kbusch at kernel.org  Mon Oct 12 15:05:11 2020
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 12 Oct 2020 12:05:11 -0700
Subject: NVME, isolcpus, and irq affinity
In-Reply-To: <00547cf5-0df0-d027-bffa-adfbe247f09d@windriver.com>
References: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
 <20201012171333.GA1032168@dhcp-10-100-145-180.wdl.wdc.com>
 <00547cf5-0df0-d027-bffa-adfbe247f09d@windriver.com>
Message-ID: <20201012190511.GA1032142@dhcp-10-100-145-180.wdl.wdc.com>

On Mon, Oct 12, 2020 at 12:50:49PM -0600, Chris Friesen wrote:
> On 10/12/2020 11:13 AM, Keith Busch wrote:
> > On Mon, Oct 12, 2020 at 09:49:38AM -0600, Chris Friesen wrote:
> 
> > > Assuming I'm reading the code correctly, how does it make sense for the NVME
> > > driver to affine interrupts to CPUs which have explicitly been designated as
> > > "isolated"?
> > 
> > The driver allocates interrupts, but doesn't affine them. The driver
> > lets the kernel handle that instead.
> 
> Okay, thanks for the quick reply.
> 
> The interrupts in question have names of the form "nvmeXqY", where X and Y
> are integers.  Some of them are affined to individual CPUs while others are
> affined to two or more CPUs.
> 
> If no tasks on the CPUs in question do any disk I/O is it possible these
> interrupts could still get triggered by activity instigated by tasks on
> other CPUs?  Or would they basically be idle and inert in that case?

In order to wake the thread, the top half handler needs to see
completion entries on that interrupt's associated queue. If you never
use the CPUs in an interrupt's affinity mask to dispatch IO, there won't
be completions, so the thread never wakes.


From tom.leiming at gmail.com  Mon Oct 12 20:51:43 2020
From: tom.leiming at gmail.com (Ming Lei)
Date: Tue, 13 Oct 2020 08:51:43 +0800
Subject: NVME, isolcpus, and irq affinity
In-Reply-To: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
References: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
Message-ID: <CACVXFVM=UPgJ0K4iZOG=DJNmfovHFjg-u5zQkEFgwG7np5V3PQ@mail.gmail.com>

On Mon, Oct 12, 2020 at 11:52 PM Chris Friesen
<chris.friesen at windriver.com> wrote:
>
> Hi,
>
> I'm not subscribed to the list so please CC me on replies.
>
> I've got a linux system running the RT kernel with threaded irqs.  On
> startup we affine the various irq threads to the housekeeping CPUs, but
> I recently hit a scenario where after some days of uptime we ended up
> with a number of NVME irq threads affined to application cores instead
> (not good when we're trying to run low-latency applications).
>
> Looking at the code, it appears that the NVME driver can in some
> scenarios call nvme_setup_io_queues() after the initial setup and thus
> allocate new IRQ threads at runtime.  It appears that this will then
> call pci_alloc_irq_vectors_affinity(), which seems to determine affinity
> without any regard for things like "isolcpus" or "cset shield".
>
> There seem to be other reports of similar issues:
>
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1831566
>
> Am I worried about nothing, or is there a risk that those irq threads
> would actually need to do real work (which would cause unacceptable
> jitter in my application)?
>
> Assuming I'm reading the code correctly, how does it make sense for the
> NVME driver to affine interrupts to CPUs which have explicitly been
> designated as "isolated"?

You may pass 'isolcpus=managed_irq,...' for this kind of isolation, see details
in 'isolcpus=' part of Documentation/admin-guide/kernel-parameters.txt.

And this feature is added since v5.6.

Thanks,
Ming Lei


From pizhenwei at bytedance.com  Mon Oct 12 23:52:33 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Tue, 13 Oct 2020 11:52:33 +0800
Subject: [PATCH] nvmet: fix uninitialized work for zero kato
Message-ID: <20201013035233.923889-1-pizhenwei@bytedance.com>

Hit a warning:
WARNING: CPU: 1 PID: 241 at kernel/workqueue.c:1627 __queue_delayed_work+0x6d/0x90
with trace:
  mod_delayed_work_on+0x59/0x90
  nvmet_update_cc+0xee/0x100 [nvmet]
  nvmet_execute_prop_set+0x72/0x80 [nvmet]
  nvmet_tcp_try_recv_pdu+0x2f7/0x770 [nvmet_tcp]
  nvmet_tcp_io_work+0x63f/0xb2d [nvmet_tcp]
  ...

This could be reproduced easily with a keep alive time 0:
nvme connect -t tcp -n NQN -a ADDR -s PORT --keep-alive-tmo=0

The reason is:
Starting an uninitialized work when initiator connects with zero
kato. Althrough keep-alive timer is disabled during allocating a ctrl
(fix in 0d3b6a8d213a), ka_work still has a chance to run
(called by nvmet_start_ctrl to detect dead host).

Initilize ka_work during allocating ctrl, and set a reasonable kato
before scheduling ka_work.

Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/target/core.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index b7b63330b5ef..3c5b2b065476 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -19,6 +19,8 @@ struct workqueue_struct *buffered_io_wq;
 static const struct nvmet_fabrics_ops *nvmet_transports[NVMF_TRTYPE_MAX];
 static DEFINE_IDA(cntlid_ida);
 
+#define NVMET_DEFAULT_KATO	5
+
 /*
  * This read/write semaphore is used to synchronize access to configuration
  * information on a target system that will result in discovery log page
@@ -385,6 +387,11 @@ static void nvmet_keep_alive_timer(struct work_struct *work)
 	if (cmd_seen) {
 		pr_debug("ctrl %d reschedule traffic based keep-alive timer\n",
 			ctrl->cntlid);
+
+		/* run once, trigger from nvmet_start_ctrl to detect dead link */
+		if (!ctrl->kato)
+			return;
+
 		schedule_delayed_work(&ctrl->ka_work, ctrl->kato * HZ);
 		return;
 	}
@@ -403,15 +410,11 @@ static void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl)
 	pr_debug("ctrl %d start keep-alive timer for %d secs\n",
 		ctrl->cntlid, ctrl->kato);
 
-	INIT_DELAYED_WORK(&ctrl->ka_work, nvmet_keep_alive_timer);
 	schedule_delayed_work(&ctrl->ka_work, ctrl->kato * HZ);
 }
 
 static void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl)
 {
-	if (unlikely(ctrl->kato == 0))
-		return;
-
 	pr_debug("ctrl %d stop keep-alive\n", ctrl->cntlid);
 
 	cancel_delayed_work_sync(&ctrl->ka_work);
@@ -1107,6 +1110,8 @@ static inline u8 nvmet_cc_iocqes(u32 cc)
 
 static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
 {
+	u32 kato = ctrl->kato ? ctrl->kato : NVMET_DEFAULT_KATO;
+
 	lockdep_assert_held(&ctrl->lock);
 
 	if (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
@@ -1126,7 +1131,7 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
 	 * in case a host died before it enabled the controller.  Hence, simply
 	 * reset the keep alive timer when the controller is enabled.
 	 */
-	mod_delayed_work(system_wq, &ctrl->ka_work, ctrl->kato * HZ);
+	mod_delayed_work(system_wq, &ctrl->ka_work, kato * HZ);
 }
 
 static void nvmet_clear_ctrl(struct nvmet_ctrl *ctrl)
@@ -1378,6 +1383,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 
 	/* keep-alive timeout in seconds */
 	ctrl->kato = DIV_ROUND_UP(kato, 1000);
+	INIT_DELAYED_WORK(&ctrl->ka_work, nvmet_keep_alive_timer);
 
 	ctrl->err_counter = 0;
 	spin_lock_init(&ctrl->error_lock);
-- 
2.11.0



From chris.friesen at windriver.com  Tue Oct 13 02:24:03 2020
From: chris.friesen at windriver.com (Chris Friesen)
Date: Tue, 13 Oct 2020 00:24:03 -0600
Subject: NVME, isolcpus, and irq affinity
In-Reply-To: <CACVXFVM=UPgJ0K4iZOG=DJNmfovHFjg-u5zQkEFgwG7np5V3PQ@mail.gmail.com>
References: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
 <CACVXFVM=UPgJ0K4iZOG=DJNmfovHFjg-u5zQkEFgwG7np5V3PQ@mail.gmail.com>
Message-ID: <c199e17e-2c7d-8010-7648-3d16ed964c2f@windriver.com>

On 10/12/2020 6:51 PM, Ming Lei wrote:
> On Mon, Oct 12, 2020 at 11:52 PM Chris Friesen
> <chris.friesen at windriver.com> wrote:
>>
>> Hi,
>>
>> I'm not subscribed to the list so please CC me on replies.
>>
>> I've got a linux system running the RT kernel with threaded irqs.  On
>> startup we affine the various irq threads to the housekeeping CPUs, but
>> I recently hit a scenario where after some days of uptime we ended up
>> with a number of NVME irq threads affined to application cores instead
>> (not good when we're trying to run low-latency applications).
>>
>> Looking at the code, it appears that the NVME driver can in some
>> scenarios call nvme_setup_io_queues() after the initial setup and thus
>> allocate new IRQ threads at runtime.  It appears that this will then
>> call pci_alloc_irq_vectors_affinity(), which seems to determine affinity
>> without any regard for things like "isolcpus" or "cset shield".
>>
>> There seem to be other reports of similar issues:
>>
>> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1831566
>>
>> Am I worried about nothing, or is there a risk that those irq threads
>> would actually need to do real work (which would cause unacceptable
>> jitter in my application)?
>>
>> Assuming I'm reading the code correctly, how does it make sense for the
>> NVME driver to affine interrupts to CPUs which have explicitly been
>> designated as "isolated"?
> 
> You may pass 'isolcpus=managed_irq,...' for this kind of isolation, see details
> in 'isolcpus=' part of Documentation/admin-guide/kernel-parameters.txt.
> 
> And this feature is added since v5.6.

I suspect that might work, unfortunately it's not available in our 
kernel and jumping to a brand new kernel will mean a lot of additional 
validation work so it's not something we can do on a whim.

I'm definitely looking forward to moving to something newer though.

Chris


From tom.leiming at gmail.com  Tue Oct 13 03:50:29 2020
From: tom.leiming at gmail.com (Ming Lei)
Date: Tue, 13 Oct 2020 15:50:29 +0800
Subject: NVME, isolcpus, and irq affinity
In-Reply-To: <c199e17e-2c7d-8010-7648-3d16ed964c2f@windriver.com>
References: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
 <CACVXFVM=UPgJ0K4iZOG=DJNmfovHFjg-u5zQkEFgwG7np5V3PQ@mail.gmail.com>
 <c199e17e-2c7d-8010-7648-3d16ed964c2f@windriver.com>
Message-ID: <CACVXFVOSR7UXDC52A-2sE1xeZNZ1+KUxGPXUJrvu_FpZ0qT4hA@mail.gmail.com>

On Tue, Oct 13, 2020 at 2:24 PM Chris Friesen
<chris.friesen at windriver.com> wrote:
>
> On 10/12/2020 6:51 PM, Ming Lei wrote:
> > On Mon, Oct 12, 2020 at 11:52 PM Chris Friesen
> > <chris.friesen at windriver.com> wrote:
> >>
> >> Hi,
> >>
> >> I'm not subscribed to the list so please CC me on replies.
> >>
> >> I've got a linux system running the RT kernel with threaded irqs.  On
> >> startup we affine the various irq threads to the housekeeping CPUs, but
> >> I recently hit a scenario where after some days of uptime we ended up
> >> with a number of NVME irq threads affined to application cores instead
> >> (not good when we're trying to run low-latency applications).
> >>
> >> Looking at the code, it appears that the NVME driver can in some
> >> scenarios call nvme_setup_io_queues() after the initial setup and thus
> >> allocate new IRQ threads at runtime.  It appears that this will then
> >> call pci_alloc_irq_vectors_affinity(), which seems to determine affinity
> >> without any regard for things like "isolcpus" or "cset shield".
> >>
> >> There seem to be other reports of similar issues:
> >>
> >> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1831566
> >>
> >> Am I worried about nothing, or is there a risk that those irq threads
> >> would actually need to do real work (which would cause unacceptable
> >> jitter in my application)?
> >>
> >> Assuming I'm reading the code correctly, how does it make sense for the
> >> NVME driver to affine interrupts to CPUs which have explicitly been
> >> designated as "isolated"?
> >
> > You may pass 'isolcpus=managed_irq,...' for this kind of isolation, see details
> > in 'isolcpus=' part of Documentation/admin-guide/kernel-parameters.txt.
> >
> > And this feature is added since v5.6.
>
> I suspect that might work, unfortunately it's not available in our

Did you look at '11ea68f553e2 genirq, sched/isolation: Isolate from
handling managed interrupts'?
Which supposes to address this kind of issue exactly.

> kernel and jumping to a brand new kernel will mean a lot of additional
> validation work so it's not something we can do on a whim.

You can backport that patch to your kernel.

>
> I'm definitely looking forward to moving to something newer though.

What is the something newer you want?


Thanks,
Ming Lei


From kai.heng.feng at canonical.com  Tue Oct 13 04:34:45 2020
From: kai.heng.feng at canonical.com (Kai-Heng Feng)
Date: Tue, 13 Oct 2020 16:34:45 +0800
Subject: [PATCH] nvme-pci: Disable Write Zeroes on Sandisk Skyhawk
In-Reply-To: <20200723155731.22313-1-kai.heng.feng@canonical.com>
References: <20200723155731.22313-1-kai.heng.feng@canonical.com>
Message-ID: <20201013083445.12317-1-kai.heng.feng@canonical.com>

Like commit 5611ec2b9814 ("nvme-pci: prevent SK hynix PC400 from using
Write Zeroes command"), Sandisk Skyhawk has the same issue:
[ 6305.633887] blk_update_request: operation not supported error, dev nvme0n1, sector 340812032 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0

So also disable Write Zeroes command on Sandisk Skyhawk.

BugLink: https://bugs.launchpad.net/bugs/1899503
Signed-off-by: Kai-Heng Feng <kai.heng.feng at canonical.com>
---
 drivers/nvme/host/pci.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 8984796db0c8..d310d7317e2a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3187,6 +3187,8 @@ static const struct pci_device_id nvme_id_table[] = {
 				NVME_QUIRK_IGNORE_DEV_SUBNQN, },
 	{ PCI_DEVICE(0x1c5c, 0x1504),   /* SK Hynix PC400 */
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+	{ PCI_DEVICE(0x15b7, 0x2001),   /*  Sandisk Skyhawk */
+		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE_CLASS(PCI_CLASS_STORAGE_EXPRESS, 0xffffff) },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2001),
 		.driver_data = NVME_QUIRK_SINGLE_VECTOR },
-- 
2.17.1



From chris.friesen at windriver.com  Tue Oct 13 11:19:16 2020
From: chris.friesen at windriver.com (Chris Friesen)
Date: Tue, 13 Oct 2020 09:19:16 -0600
Subject: NVME, isolcpus, and irq affinity
In-Reply-To: <CACVXFVOSR7UXDC52A-2sE1xeZNZ1+KUxGPXUJrvu_FpZ0qT4hA@mail.gmail.com>
References: <2a4a5174-d1b9-86b7-f3c8-03b041fa093e@windriver.com>
 <CACVXFVM=UPgJ0K4iZOG=DJNmfovHFjg-u5zQkEFgwG7np5V3PQ@mail.gmail.com>
 <c199e17e-2c7d-8010-7648-3d16ed964c2f@windriver.com>
 <CACVXFVOSR7UXDC52A-2sE1xeZNZ1+KUxGPXUJrvu_FpZ0qT4hA@mail.gmail.com>
Message-ID: <635acc40-d76b-3f2d-45aa-38ec1d0865a7@windriver.com>

On 10/13/2020 1:50 AM, Ming Lei wrote:
> On Tue, Oct 13, 2020 at 2:24 PM Chris Friesen
> <chris.friesen at windriver.com> wrote:

>> kernel and jumping to a brand new kernel will mean a lot of additional
>> validation work so it's not something we can do on a whim.
> 
> You can backport that patch to your kernel.

Right, we might end up having to do something like that.

>> I'm definitely looking forward to moving to something newer though.
> 
> What is the something newer you want?

Sorry, I meant something newer than what we have currently. :)

Chris


From kbusch at kernel.org  Tue Oct 13 13:16:10 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 13 Oct 2020 10:16:10 -0700
Subject: [PATCHv4 1/3] block: add zone specific block statuses
In-Reply-To: <20201009152927.GA1023921@dhcp-10-100-145-180.wdl.wdc.com>
References: <20200924205330.4043232-1-kbusch@kernel.org>
 <20200924205330.4043232-2-kbusch@kernel.org>
 <20201009152927.GA1023921@dhcp-10-100-145-180.wdl.wdc.com>
Message-ID: <20201013171610.GB1033288@dhcp-10-100-145-180.wdl.wdc.com>

Hi Jens,

I'm going to try one more time in the hope you've just been too busy to
notice these patches. The series has been unchanged for a month now, and
has reviews from all the right people. Can this be considered for
inclusion?


From axboe at kernel.dk  Tue Oct 13 15:25:59 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Tue, 13 Oct 2020 13:25:59 -0600
Subject: [PATCHv4 1/3] block: add zone specific block statuses
In-Reply-To: <20201013171610.GB1033288@dhcp-10-100-145-180.wdl.wdc.com>
References: <20200924205330.4043232-1-kbusch@kernel.org>
 <20200924205330.4043232-2-kbusch@kernel.org>
 <20201009152927.GA1023921@dhcp-10-100-145-180.wdl.wdc.com>
 <20201013171610.GB1033288@dhcp-10-100-145-180.wdl.wdc.com>
Message-ID: <52669c98-ddc3-188e-ee2b-aff1823e8e96@kernel.dk>

On 10/13/20 11:16 AM, Keith Busch wrote:
> Hi Jens,
> 
> I'm going to try one more time in the hope you've just been too busy to
> notice these patches. The series has been unchanged for a month now, and
> has reviews from all the right people. Can this be considered for
> inclusion?

Thanks for the pings, finally got them picked up. Sorry about the
delay!

-- 
Jens Axboe



From sagi at grimberg.me  Tue Oct 13 18:18:51 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 13 Oct 2020 15:18:51 -0700
Subject: [PATCH] nvmet: fix uninitialized work for zero kato
In-Reply-To: <20201013035233.923889-1-pizhenwei@bytedance.com>
References: <20201013035233.923889-1-pizhenwei@bytedance.com>
Message-ID: <a75e067a-d0ea-8402-b883-cedd36bb7cf7@grimberg.me>

> Hit a warning:
> WARNING: CPU: 1 PID: 241 at kernel/workqueue.c:1627 __queue_delayed_work+0x6d/0x90
> with trace:
>    mod_delayed_work_on+0x59/0x90
>    nvmet_update_cc+0xee/0x100 [nvmet]
>    nvmet_execute_prop_set+0x72/0x80 [nvmet]
>    nvmet_tcp_try_recv_pdu+0x2f7/0x770 [nvmet_tcp]
>    nvmet_tcp_io_work+0x63f/0xb2d [nvmet_tcp]
>    ...
> 
> This could be reproduced easily with a keep alive time 0:
> nvme connect -t tcp -n NQN -a ADDR -s PORT --keep-alive-tmo=0
> 
> The reason is:
> Starting an uninitialized work when initiator connects with zero
> kato. Althrough keep-alive timer is disabled during allocating a ctrl
> (fix in 0d3b6a8d213a), ka_work still has a chance to run
> (called by nvmet_start_ctrl to detect dead host).

This should have a "Fixes:" tag.

> 
> Initilize ka_work during allocating ctrl, and set a reasonable kato
> before scheduling ka_work.
> 
> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
> ---
>   drivers/nvme/target/core.c | 16 +++++++++++-----
>   1 file changed, 11 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
> index b7b63330b5ef..3c5b2b065476 100644
> --- a/drivers/nvme/target/core.c
> +++ b/drivers/nvme/target/core.c
> @@ -19,6 +19,8 @@ struct workqueue_struct *buffered_io_wq;
>   static const struct nvmet_fabrics_ops *nvmet_transports[NVMF_TRTYPE_MAX];
>   static DEFINE_IDA(cntlid_ida);
>   
> +#define NVMET_DEFAULT_KATO	5
> +
>   /*
>    * This read/write semaphore is used to synchronize access to configuration
>    * information on a target system that will result in discovery log page
> @@ -385,6 +387,11 @@ static void nvmet_keep_alive_timer(struct work_struct *work)
>   	if (cmd_seen) {
>   		pr_debug("ctrl %d reschedule traffic based keep-alive timer\n",
>   			ctrl->cntlid);
> +
> +		/* run once, trigger from nvmet_start_ctrl to detect dead link */
> +		if (!ctrl->kato)
> +			return;
> +
>   		schedule_delayed_work(&ctrl->ka_work, ctrl->kato * HZ);

It will be better to just schedule/mod the ka_work if kato != 0, other
changes in the patch aren't needed IMO.

>   		return;
>   	}
> @@ -403,15 +410,11 @@ static void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl)
>   	pr_debug("ctrl %d start keep-alive timer for %d secs\n",
>   		ctrl->cntlid, ctrl->kato);
>   
> -	INIT_DELAYED_WORK(&ctrl->ka_work, nvmet_keep_alive_timer);
>   	schedule_delayed_work(&ctrl->ka_work, ctrl->kato * HZ);
>   }
>   
>   static void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl)
>   {
> -	if (unlikely(ctrl->kato == 0))
> -		return;
> -
>   	pr_debug("ctrl %d stop keep-alive\n", ctrl->cntlid);
>   
>   	cancel_delayed_work_sync(&ctrl->ka_work);
> @@ -1107,6 +1110,8 @@ static inline u8 nvmet_cc_iocqes(u32 cc)
>   
>   static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
>   {
> +	u32 kato = ctrl->kato ? ctrl->kato : NVMET_DEFAULT_KATO;
> +

The controller shouldn't have a default value, it should receive
the desired value from the host.

>   	lockdep_assert_held(&ctrl->lock);
>   
>   	if (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
> @@ -1126,7 +1131,7 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
>   	 * in case a host died before it enabled the controller.  Hence, simply
>   	 * reset the keep alive timer when the controller is enabled.
>   	 */
> -	mod_delayed_work(system_wq, &ctrl->ka_work, ctrl->kato * HZ);
> +	mod_delayed_work(system_wq, &ctrl->ka_work, kato * HZ);
>   }
>   
>   static void nvmet_clear_ctrl(struct nvmet_ctrl *ctrl)
> @@ -1378,6 +1383,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
>   
>   	/* keep-alive timeout in seconds */
>   	ctrl->kato = DIV_ROUND_UP(kato, 1000);
> +	INIT_DELAYED_WORK(&ctrl->ka_work, nvmet_keep_alive_timer);
>   
>   	ctrl->err_counter = 0;
>   	spin_lock_init(&ctrl->error_lock);
> 


From sagi at grimberg.me  Tue Oct 13 18:23:07 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 13 Oct 2020 15:23:07 -0700
Subject: [PATCH] nvme-rdma: fix crash due to incorrect cqe
In-Reply-To: <20201012085537.2599-1-lengchao@huawei.com>
References: <20201012085537.2599-1-lengchao@huawei.com>
Message-ID: <b42c0b68-c1fe-1e14-e887-d157a919510d@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From sagi at grimberg.me  Tue Oct 13 18:25:20 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 13 Oct 2020 15:25:20 -0700
Subject: [PATCH] nvme-rdma: fix crash when connect rejected
In-Reply-To: <20201012081040.31741-1-lengchao@huawei.com>
References: <20201012081040.31741-1-lengchao@huawei.com>
Message-ID: <cbbe61ba-4061-3150-911d-db4aa0baf85d@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From sagi at grimberg.me  Tue Oct 13 18:26:07 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 13 Oct 2020 15:26:07 -0700
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201009231816.1524-1-logang@deltatee.com>
References: <20201009231816.1524-1-logang@deltatee.com>
Message-ID: <80fedfa3-f7f2-f918-5f7c-5973b59b9e0a@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From sagi at grimberg.me  Tue Oct 13 18:31:18 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 13 Oct 2020 15:31:18 -0700
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201012081306.GB556731@T590>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
Message-ID: <0b5bfc44-d925-960d-b2a2-d0ba88b51111@grimberg.me>


>>>>>> -- 
>>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>>> index 629b025685d1..46428ff0b0fc 100644
>>>>>> --- a/drivers/nvme/host/tcp.c
>>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>>>>>  ??????? /* fence other contexts that may complete the command */
>>>>>>  ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>>>  ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>>>> -?????? if (!blk_mq_request_completed(rq)) {
>>>>>> +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>>>>>>  ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>>>>  ??????????????? blk_mq_complete_request_sync(rq);
>>>>>>  ??????? }
>> This may just reduce the probability. The concurrency of timeout and teardown will cause the same request
>> be treated repeatly, this is not we expected.
> 
> That is right, not like SCSI, NVME doesn't apply atomic request completion, so
> request may be completed/freed from both timeout & nvme_cancel_request().
> 
> .teardown_lock still may cover the race with Sagi's patch because teardown
> actually cancels requests in sync style.
> 
>> In the teardown process, after quiesced queues delete the timer and cancel the timeout work maybe a better option.
> 
> Seems better solution, given it is aligned with NVME PCI's reset
> handling. nvme_sync_queues() may be called in nvme_tcp_teardown_io_queues() to
> avoid this race.

We can't call nvme_sync_queues, that flushes the timeout work that is
serializing with the teardown, it will deadlock.


From sagi at grimberg.me  Tue Oct 13 18:36:08 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 13 Oct 2020 15:36:08 -0700
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
Message-ID: <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>


>>> This may just reduce the probability. The concurrency of timeout and 
>>> teardown will cause the same request
>>> be treated repeatly, this is not we expected.
>>
>> That is right, not like SCSI, NVME doesn't apply atomic request 
>> completion, so
>> request may be completed/freed from both timeout & nvme_cancel_request().
>>
>> .teardown_lock still may cover the race with Sagi's patch because 
>> teardown
>> actually cancels requests in sync style.
> In extreme scenarios, the request may be already retry success(rq state 
> change to inflight).
> Timeout processing may wrongly stop the queue and abort the request.
> teardown_lock serialize the process of timeout and teardown, but do not 
> avoid the race.
> It might not be safe.

Not sure I understand the scenario you are describing.

what do you mean by "In extreme scenarios, the request may be already 
retry success(rq state change to inflight)"?

What will retry the request? only when the host will reconnect
the request will be retried.

We can call nvme_sync_queues in the last part of the teardown, but
I still don't understand the race here.


From dgilbert at interlog.com  Tue Oct 13 18:38:46 2020
From: dgilbert at interlog.com (Douglas Gilbert)
Date: Tue, 13 Oct 2020 18:38:46 -0400
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201009231816.1524-1-logang@deltatee.com>
References: <20201009231816.1524-1-logang@deltatee.com>
Message-ID: <1b6ced06-dd44-ac20-cafa-b4724d9a188d@interlog.com>

On 2020-10-09 7:18 p.m., Logan Gunthorpe wrote:
> Clean up some confusing elements of nvmet_passthru_map_sg() by returning
> early if the request is greater than the maximum bio size. This allows
> us to drop the sg_cnt variable.
> 
> This should not result in any functional change but makes the code
> clearer and more understandable. The original code allocated a truncated
> bio then would return EINVAL when bio_add_pc_page() filled that bio. The
> new code just returns EINVAL early if this would happen.
> 
> Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
> Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
> Cc: Christoph Hellwig <hch at lst.de>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>

Reviewed-by: Douglas Gilbert <dgilbert at interlog.com>


I have commented offline to Logan about the return from bio_alloc()
not being checked for NULL. He assures me that can't happen but I believe
that is true only for small scatter-gather lists. So I guess it depends
on the size of the data this is planned to be sent through this
pass-through.

> ---
>   drivers/nvme/target/passthru.c | 7 ++++---
>   1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
> index dacfa7435d0b..076e829490a1 100644
> --- a/drivers/nvme/target/passthru.c
> +++ b/drivers/nvme/target/passthru.c
> @@ -180,18 +180,20 @@ static void nvmet_passthru_req_done(struct request *rq,
>   
>   static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>   {
> -	int sg_cnt = req->sg_cnt;
>   	struct scatterlist *sg;
>   	int op_flags = 0;
>   	struct bio *bio;
>   	int i, ret;
>   
> +	if (req->sg_cnt > BIO_MAX_PAGES)
> +		return -EINVAL;
> +
>   	if (req->cmd->common.opcode == nvme_cmd_flush)
>   		op_flags = REQ_FUA;
>   	else if (nvme_is_write(req->cmd))
>   		op_flags = REQ_SYNC | REQ_IDLE;
>   
> -	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
> +	bio = bio_alloc(GFP_KERNEL, req->sg_cnt);
>   	bio->bi_end_io = bio_put;
>   	bio->bi_opf = req_op(rq) | op_flags;
>   
> @@ -201,7 +203,6 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>   			bio_put(bio);
>   			return -EINVAL;
>   		}
> -		sg_cnt--;
>   	}
>   
>   	ret = blk_rq_append_bio(rq, &bio);
> 
> base-commit: 549738f15da0e5a00275977623be199fbbf7df50
> 



From Chaitanya.Kulkarni at wdc.com  Tue Oct 13 20:16:19 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 14 Oct 2020 00:16:19 +0000
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
References: <20201009231816.1524-1-logang@deltatee.com>
Message-ID: <BYAPR04MB496556A3CBC239BC57EE588B86050@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/9/20 16:18, Logan Gunthorpe wrote:
> Clean up some confusing elements of nvmet_passthru_map_sg() by returning
> early if the request is greater than the maximum bio size. This allows
> us to drop the sg_cnt variable.
>
> This should not result in any functional change but makes the code
> clearer and more understandable. The original code allocated a truncated
> bio then would return EINVAL when bio_add_pc_page() filled that bio. The
> new code just returns EINVAL early if this would happen.
>
> Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
> Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
> Cc: Christoph Hellwig <hch at lst.de>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>

why the prefix is nvmet-passthru ?



From Chaitanya.Kulkarni at wdc.com  Tue Oct 13 20:20:02 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 14 Oct 2020 00:20:02 +0000
Subject: [PATCH] nvme-pci: Disable Write Zeroes on Sandisk Skyhawk
References: <20200723155731.22313-1-kai.heng.feng@canonical.com>
 <20201013083445.12317-1-kai.heng.feng@canonical.com>
Message-ID: <BYAPR04MB496518A4C9FC65964C58453386050@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/13/20 01:45, Kai-Heng Feng wrote:
> Like commit 5611ec2b9814 ("nvme-pci: prevent SK hynix PC400 from using
> Write Zeroes command"), Sandisk Skyhawk has the same issue:
> [ 6305.633887] blk_update_request: operation not supported error, dev nvme0n1, sector 340812032 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
>
> So also disable Write Zeroes command on Sandisk Skyhawk.
>
> BugLink: https://bugs.launchpad.net/bugs/1899503
> Signed-off-by: Kai-Heng Feng <kai.heng.feng at canonical.com>

Are you sure this happens all the devices of the same model

and not a firmware bug on specific device ?

If yes then looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>





From logang at deltatee.com  Tue Oct 13 20:20:03 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 13 Oct 2020 18:20:03 -0600
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <BYAPR04MB496556A3CBC239BC57EE588B86050@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20201009231816.1524-1-logang@deltatee.com>
 <BYAPR04MB496556A3CBC239BC57EE588B86050@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <e7d76d3e-955b-1827-fd2f-9b7db49646ed@deltatee.com>



On 2020-10-13 6:16 p.m., Chaitanya Kulkarni wrote:
> On 10/9/20 16:18, Logan Gunthorpe wrote:
>> Clean up some confusing elements of nvmet_passthru_map_sg() by returning
>> early if the request is greater than the maximum bio size. This allows
>> us to drop the sg_cnt variable.
>>
>> This should not result in any functional change but makes the code
>> clearer and more understandable. The original code allocated a truncated
>> bio then would return EINVAL when bio_add_pc_page() filled that bio. The
>> new code just returns EINVAL early if this would happen.
>>
>> Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
>> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
>> Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
>> Cc: Christoph Hellwig <hch at lst.de>
>> Cc: Sagi Grimberg <sagi at grimberg.me>
>> Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> 
> why the prefix is nvmet-passthru ?

Well that's the prefix I used in the first patch, which follows other
similar conventions like nvmet-tcp, nvmet-fc, nvmet-rdma, nvmet-loop, etc.

Logan



From Chaitanya.Kulkarni at wdc.com  Tue Oct 13 20:25:58 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 14 Oct 2020 00:25:58 +0000
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
References: <20201009231816.1524-1-logang@deltatee.com>
 <BYAPR04MB496556A3CBC239BC57EE588B86050@BYAPR04MB4965.namprd04.prod.outlook.com>
 <e7d76d3e-955b-1827-fd2f-9b7db49646ed@deltatee.com>
Message-ID: <BYAPR04MB49654B340081F257BF9C4BB486050@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/13/20 17:20, Logan Gunthorpe wrote:
>
> On 2020-10-13 6:16 p.m., Chaitanya Kulkarni wrote:
>> On 10/9/20 16:18, Logan Gunthorpe wrote:
>>> Clean up some confusing elements of nvmet_passthru_map_sg() by returning
>>> early if the request is greater than the maximum bio size. This allows
>>> us to drop the sg_cnt variable.
>>>
>>> This should not result in any functional change but makes the code
>>> clearer and more understandable. The original code allocated a truncated
>>> bio then would return EINVAL when bio_add_pc_page() filled that bio. The
>>> new code just returns EINVAL early if this would happen.
>>>
>>> Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
>>> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
>>> Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
>>> Cc: Christoph Hellwig <hch at lst.de>
>>> Cc: Sagi Grimberg <sagi at grimberg.me>
>>> Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
>> why the prefix is nvmet-passthru ?
> Well that's the prefix I used in the first patch, which follows other
> similar conventions like nvmet-tcp, nvmet-fc, nvmet-rdma, nvmet-loop, etc.
>
> Logan
>
>
Those are transport types and passthru is a target backend.

I found this [1] in the git log, only one patch has that prefix.

The convention is to not add backend prefixes for target patches (see
bdev/file),

I'm not sure we want to start adding prefixes unless there is a specific

reason.

 # git log --oneline  drivers/nvme/target/passthru.c
dc1890b99602 nvmet: add passthru ZNS support
df8dd70a9db8 nvme: lift the file open code from nvme_ctrl_get_by_path
3a6b076168e2 (tag: nvme-5.9-2020-09-17) nvmet: get transport reference
for passthru ctrl
7ee51cf60a90 nvmet: call blk_mq_free_request() directly
a2138fd49467 nvmet: fix oops in pt cmd execution
4db69a3d7cfe nvmet: add ns tear down label for pt-cmd handling
0ceeab96ba59 nvmet-passthru: Reject commands with non-sgl flags set
ba76af676cd0 nvmet: Add passthru enable/disable helpers
c1fef73f793b nvmet: add passthru code to process commands



From ming.lei at redhat.com  Tue Oct 13 21:08:13 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed, 14 Oct 2020 09:08:13 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
References: <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
Message-ID: <20201014010813.GA775684@T590>

On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
> 
> > > > This may just reduce the probability. The concurrency of timeout
> > > > and teardown will cause the same request
> > > > be treated repeatly, this is not we expected.
> > > 
> > > That is right, not like SCSI, NVME doesn't apply atomic request
> > > completion, so
> > > request may be completed/freed from both timeout & nvme_cancel_request().
> > > 
> > > .teardown_lock still may cover the race with Sagi's patch because
> > > teardown
> > > actually cancels requests in sync style.
> > In extreme scenarios, the request may be already retry success(rq state
> > change to inflight).
> > Timeout processing may wrongly stop the queue and abort the request.
> > teardown_lock serialize the process of timeout and teardown, but do not
> > avoid the race.
> > It might not be safe.
> 
> Not sure I understand the scenario you are describing.
> 
> what do you mean by "In extreme scenarios, the request may be already retry
> success(rq state change to inflight)"?
> 
> What will retry the request? only when the host will reconnect
> the request will be retried.
> 
> We can call nvme_sync_queues in the last part of the teardown, but
> I still don't understand the race here.

Not like SCSI, NVME doesn't complete request atomically, so double
completion/free can be done from both timeout & nvme_cancel_request()(via teardown).

Given request is completed remotely or asynchronously in the two code paths,
the teardown_lock can't protect the case.

One solution is to apply the introduced blk_mq_complete_request_sync()
in both two code paths.

Another candidate is to use nvme_sync_queues() before teardown, such as
the following change:

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index d6a3e1487354..dc3561ca0074 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1909,6 +1909,7 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_start_freeze(ctrl);
 	nvme_stop_queues(ctrl);
+	nvme_sync_queues(ctrl);
 	nvme_tcp_stop_io_queues(ctrl);
 	if (ctrl->tagset) {
 		blk_mq_tagset_busy_iter(ctrl->tagset,
@@ -2171,14 +2172,11 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
 	struct nvme_ctrl *ctrl = &req->queue->ctrl->ctrl;
 
-	/* fence other contexts that may complete the command */
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
 	if (!blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static enum blk_eh_timer_return


Thanks,
Ming



From lengchao at huawei.com  Tue Oct 13 21:25:43 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 14 Oct 2020 09:25:43 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <0b5bfc44-d925-960d-b2a2-d0ba88b51111@grimberg.me>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <0b5bfc44-d925-960d-b2a2-d0ba88b51111@grimberg.me>
Message-ID: <dba6f24d-7d4f-a89e-047d-9d63837084dd@huawei.com>



On 2020/10/14 6:31, Sagi Grimberg wrote:
> 
>>>>>>> -- 
>>>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>>>> index 629b025685d1..46428ff0b0fc 100644
>>>>>>> --- a/drivers/nvme/host/tcp.c
>>>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>>>>>> ???????? /* fence other contexts that may complete the command */
>>>>>>> ???????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>>>> ???????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>>>>> -?????? if (!blk_mq_request_completed(rq)) {
>>>>>>> +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>>>>>>> ???????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>>>>>>> ???????????????? blk_mq_complete_request_sync(rq);
>>>>>>> ???????? }
>>> This may just reduce the probability. The concurrency of timeout and teardown will cause the same request
>>> be treated repeatly, this is not we expected.
>>
>> That is right, not like SCSI, NVME doesn't apply atomic request completion, so
>> request may be completed/freed from both timeout & nvme_cancel_request().
>>
>> .teardown_lock still may cover the race with Sagi's patch because teardown
>> actually cancels requests in sync style.
>>
>>> In the teardown process, after quiesced queues delete the timer and cancel the timeout work maybe a better option.
>>
>> Seems better solution, given it is aligned with NVME PCI's reset
>> handling. nvme_sync_queues() may be called in nvme_tcp_teardown_io_queues() to
>> avoid this race.
> 
> We can't call nvme_sync_queues, that flushes the timeout work that is
> serializing with the teardown, it will deadlock.
yes, the teardown_lock is not needed if we sync_queue. We is testing the patch with nvme over roce.
> .


From lengchao at huawei.com  Tue Oct 13 21:32:39 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 14 Oct 2020 09:32:39 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
References: <20201008213750.899462-1-sagi@grimberg.me>
 <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
Message-ID: <6f9f59be-5d83-390e-b3cf-b5955ffa7259@huawei.com>



On 2020/10/14 6:36, Sagi Grimberg wrote:
> 
>>>> This may just reduce the probability. The concurrency of timeout and teardown will cause the same request
>>>> be treated repeatly, this is not we expected.
>>>
>>> That is right, not like SCSI, NVME doesn't apply atomic request completion, so
>>> request may be completed/freed from both timeout & nvme_cancel_request().
>>>
>>> .teardown_lock still may cover the race with Sagi's patch because teardown
>>> actually cancels requests in sync style.
>> In extreme scenarios, the request may be already retry success(rq state change to inflight).
>> Timeout processing may wrongly stop the queue and abort the request.
>> teardown_lock serialize the process of timeout and teardown, but do not avoid the race.
>> It might not be safe.
> 
> Not sure I understand the scenario you are describing.
> 
> what do you mean by "In extreme scenarios, the request may be already retry success(rq state change to inflight)"?
> 
> What will retry the request? only when the host will reconnect
> the request will be retried.
If irq interrupt the timeout work, and cause the timeout work pause long time(more than 100ms).
The reconnect may already success, and start requests. And then timeout work continue run to
wrongly stop queue and cancel the request.
The probability of this happening is very low.
> 
> We can call nvme_sync_queues in the last part of the teardown, but
> I still don't understand the race here.
> .


From lengchao at huawei.com  Tue Oct 13 21:37:07 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 14 Oct 2020 09:37:07 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201014010813.GA775684@T590>
References: <20201009043938.GC27356@T590>
 <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590>
Message-ID: <a1221820-55db-e550-0a9c-684ab96608e3@huawei.com>

rdma also need do this patch.
We do test this patch with nvme over roce a few days, now work well.

On 2020/10/14 9:08, Ming Lei wrote:
> On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
>>
>>>>> This may just reduce the probability. The concurrency of timeout
>>>>> and teardown will cause the same request
>>>>> be treated repeatly, this is not we expected.
>>>>
>>>> That is right, not like SCSI, NVME doesn't apply atomic request
>>>> completion, so
>>>> request may be completed/freed from both timeout & nvme_cancel_request().
>>>>
>>>> .teardown_lock still may cover the race with Sagi's patch because
>>>> teardown
>>>> actually cancels requests in sync style.
>>> In extreme scenarios, the request may be already retry success(rq state
>>> change to inflight).
>>> Timeout processing may wrongly stop the queue and abort the request.
>>> teardown_lock serialize the process of timeout and teardown, but do not
>>> avoid the race.
>>> It might not be safe.
>>
>> Not sure I understand the scenario you are describing.
>>
>> what do you mean by "In extreme scenarios, the request may be already retry
>> success(rq state change to inflight)"?
>>
>> What will retry the request? only when the host will reconnect
>> the request will be retried.
>>
>> We can call nvme_sync_queues in the last part of the teardown, but
>> I still don't understand the race here.
> 
> Not like SCSI, NVME doesn't complete request atomically, so double
> completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
> 
> Given request is completed remotely or asynchronously in the two code paths,
> the teardown_lock can't protect the case.
> 
> One solution is to apply the introduced blk_mq_complete_request_sync()
> in both two code paths.
> 
> Another candidate is to use nvme_sync_queues() before teardown, such as
> the following change:
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index d6a3e1487354..dc3561ca0074 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -1909,6 +1909,7 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>   	blk_mq_quiesce_queue(ctrl->admin_q);
>   	nvme_start_freeze(ctrl);
>   	nvme_stop_queues(ctrl);
> +	nvme_sync_queues(ctrl);
nvme_sync_queues now sync io queues and admin queues, so we may need like this:
nvme_sync_io_queues(struct nvme_ctrl *ctrl)
{
	down_read(&ctrl->namespaces_rwsem);
	list_for_each_entry(ns, &ctrl->namespaces, list)
		blk_sync_queue(ns->queue);
	up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_sync_io_queues);

void nvme_sync_queues(struct nvme_ctrl *ctrl)
{
	struct nvme_ns *ns;

     nvme_sync_io_queues(ctrl);

	if (ctrl->admin_q)
		blk_sync_queue(ctrl->admin_q);
}
EXPORT_SYMBOL_GPL(nvme_sync_queues);
>   	nvme_tcp_stop_io_queues(ctrl);
>   	if (ctrl->tagset) {
>   		blk_mq_tagset_busy_iter(ctrl->tagset,
> @@ -2171,14 +2172,11 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>   	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
>   	struct nvme_ctrl *ctrl = &req->queue->ctrl->ctrl;
>   
> -	/* fence other contexts that may complete the command */
> -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>   	if (!blk_mq_request_completed(rq)) {
>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>   		blk_mq_complete_request(rq);
>   	}
> -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   }
>   
>   static enum blk_eh_timer_return
> 
> 
> Thanks,
> Ming
> 
> .
> 


From ming.lei at redhat.com  Tue Oct 13 22:02:57 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed, 14 Oct 2020 10:02:57 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <a1221820-55db-e550-0a9c-684ab96608e3@huawei.com>
References: <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590>
 <a1221820-55db-e550-0a9c-684ab96608e3@huawei.com>
Message-ID: <20201014020257.GB775684@T590>

On Wed, Oct 14, 2020 at 09:37:07AM +0800, Chao Leng wrote:
> rdma also need do this patch.
> We do test this patch with nvme over roce a few days, now work well.
> 
> On 2020/10/14 9:08, Ming Lei wrote:
> > On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
> > > 
> > > > > > This may just reduce the probability. The concurrency of timeout
> > > > > > and teardown will cause the same request
> > > > > > be treated repeatly, this is not we expected.
> > > > > 
> > > > > That is right, not like SCSI, NVME doesn't apply atomic request
> > > > > completion, so
> > > > > request may be completed/freed from both timeout & nvme_cancel_request().
> > > > > 
> > > > > .teardown_lock still may cover the race with Sagi's patch because
> > > > > teardown
> > > > > actually cancels requests in sync style.
> > > > In extreme scenarios, the request may be already retry success(rq state
> > > > change to inflight).
> > > > Timeout processing may wrongly stop the queue and abort the request.
> > > > teardown_lock serialize the process of timeout and teardown, but do not
> > > > avoid the race.
> > > > It might not be safe.
> > > 
> > > Not sure I understand the scenario you are describing.
> > > 
> > > what do you mean by "In extreme scenarios, the request may be already retry
> > > success(rq state change to inflight)"?
> > > 
> > > What will retry the request? only when the host will reconnect
> > > the request will be retried.
> > > 
> > > We can call nvme_sync_queues in the last part of the teardown, but
> > > I still don't understand the race here.
> > 
> > Not like SCSI, NVME doesn't complete request atomically, so double
> > completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
> > 
> > Given request is completed remotely or asynchronously in the two code paths,
> > the teardown_lock can't protect the case.
> > 
> > One solution is to apply the introduced blk_mq_complete_request_sync()
> > in both two code paths.
> > 
> > Another candidate is to use nvme_sync_queues() before teardown, such as
> > the following change:
> > 
> > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > index d6a3e1487354..dc3561ca0074 100644
> > --- a/drivers/nvme/host/tcp.c
> > +++ b/drivers/nvme/host/tcp.c
> > @@ -1909,6 +1909,7 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
> >   	blk_mq_quiesce_queue(ctrl->admin_q);
> >   	nvme_start_freeze(ctrl);
> >   	nvme_stop_queues(ctrl);
> > +	nvme_sync_queues(ctrl);
> nvme_sync_queues now sync io queues and admin queues, so we may need like this:
> nvme_sync_io_queues(struct nvme_ctrl *ctrl)
> {
> 	down_read(&ctrl->namespaces_rwsem);
> 	list_for_each_entry(ns, &ctrl->namespaces, list)
> 		blk_sync_queue(ns->queue);
> 	up_read(&ctrl->namespaces_rwsem);
> }

Looks not necessary to do that, because admin queue is quiesced in
nvme_tcp_teardown_io_queues(), so it is safe to sync admin queue here too.


Thanks,
Ming



From lengchao at huawei.com  Tue Oct 13 22:32:15 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 14 Oct 2020 10:32:15 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201014020257.GB775684@T590>
References: <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590>
 <a1221820-55db-e550-0a9c-684ab96608e3@huawei.com>
 <20201014020257.GB775684@T590>
Message-ID: <645a5cd7-fba5-5bf7-fe43-596fdcc61366@huawei.com>



On 2020/10/14 10:02, Ming Lei wrote:
> On Wed, Oct 14, 2020 at 09:37:07AM +0800, Chao Leng wrote:
>> rdma also need do this patch.
>> We do test this patch with nvme over roce a few days, now work well.
>>
>> On 2020/10/14 9:08, Ming Lei wrote:
>>> On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
>>>>
>>>>>>> This may just reduce the probability. The concurrency of timeout
>>>>>>> and teardown will cause the same request
>>>>>>> be treated repeatly, this is not we expected.
>>>>>>
>>>>>> That is right, not like SCSI, NVME doesn't apply atomic request
>>>>>> completion, so
>>>>>> request may be completed/freed from both timeout & nvme_cancel_request().
>>>>>>
>>>>>> .teardown_lock still may cover the race with Sagi's patch because
>>>>>> teardown
>>>>>> actually cancels requests in sync style.
>>>>> In extreme scenarios, the request may be already retry success(rq state
>>>>> change to inflight).
>>>>> Timeout processing may wrongly stop the queue and abort the request.
>>>>> teardown_lock serialize the process of timeout and teardown, but do not
>>>>> avoid the race.
>>>>> It might not be safe.
>>>>
>>>> Not sure I understand the scenario you are describing.
>>>>
>>>> what do you mean by "In extreme scenarios, the request may be already retry
>>>> success(rq state change to inflight)"?
>>>>
>>>> What will retry the request? only when the host will reconnect
>>>> the request will be retried.
>>>>
>>>> We can call nvme_sync_queues in the last part of the teardown, but
>>>> I still don't understand the race here.
>>>
>>> Not like SCSI, NVME doesn't complete request atomically, so double
>>> completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
>>>
>>> Given request is completed remotely or asynchronously in the two code paths,
>>> the teardown_lock can't protect the case.
>>>
>>> One solution is to apply the introduced blk_mq_complete_request_sync()
>>> in both two code paths.
>>>
>>> Another candidate is to use nvme_sync_queues() before teardown, such as
>>> the following change:
>>>
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index d6a3e1487354..dc3561ca0074 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -1909,6 +1909,7 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>>>    	blk_mq_quiesce_queue(ctrl->admin_q);
>>>    	nvme_start_freeze(ctrl);
>>>    	nvme_stop_queues(ctrl);
>>> +	nvme_sync_queues(ctrl);
>> nvme_sync_queues now sync io queues and admin queues, so we may need like this:
>> nvme_sync_io_queues(struct nvme_ctrl *ctrl)
>> {
>> 	down_read(&ctrl->namespaces_rwsem);
>> 	list_for_each_entry(ns, &ctrl->namespaces, list)
>> 		blk_sync_queue(ns->queue);
>> 	up_read(&ctrl->namespaces_rwsem);
>> }
> 
> Looks not necessary to do that, because admin queue is quiesced in
> nvme_tcp_teardown_io_queues(), so it is safe to sync admin queue here too.Sync admin queue is not necessary. It should do in tear down admin queue.
And rdma do not quiesce the admin queue in tear down io queue.
So separate nvme_sync_io_queues may be a better choice.
> 
> 
> Thanks,
> Ming
> 
> .
> 


From lengchao at huawei.com  Tue Oct 13 22:41:48 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 14 Oct 2020 10:41:48 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201014020257.GB775684@T590>
References: <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590>
 <a1221820-55db-e550-0a9c-684ab96608e3@huawei.com>
 <20201014020257.GB775684@T590>
Message-ID: <6a2ea98a-0ab7-32e1-a342-da0a68dd54ac@huawei.com>



On 2020/10/14 10:02, Ming Lei wrote:
> On Wed, Oct 14, 2020 at 09:37:07AM +0800, Chao Leng wrote:
>> rdma also need do this patch.
>> We do test this patch with nvme over roce a few days, now work well.
>>
>> On 2020/10/14 9:08, Ming Lei wrote:
>>> On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
>>>>
>>>>>>> This may just reduce the probability. The concurrency of timeout
>>>>>>> and teardown will cause the same request
>>>>>>> be treated repeatly, this is not we expected.
>>>>>>
>>>>>> That is right, not like SCSI, NVME doesn't apply atomic request
>>>>>> completion, so
>>>>>> request may be completed/freed from both timeout & nvme_cancel_request().
>>>>>>
>>>>>> .teardown_lock still may cover the race with Sagi's patch because
>>>>>> teardown
>>>>>> actually cancels requests in sync style.
>>>>> In extreme scenarios, the request may be already retry success(rq state
>>>>> change to inflight).
>>>>> Timeout processing may wrongly stop the queue and abort the request.
>>>>> teardown_lock serialize the process of timeout and teardown, but do not
>>>>> avoid the race.
>>>>> It might not be safe.
>>>>
>>>> Not sure I understand the scenario you are describing.
>>>>
>>>> what do you mean by "In extreme scenarios, the request may be already retry
>>>> success(rq state change to inflight)"?
>>>>
>>>> What will retry the request? only when the host will reconnect
>>>> the request will be retried.
>>>>
>>>> We can call nvme_sync_queues in the last part of the teardown, but
>>>> I still don't understand the race here.
>>>
>>> Not like SCSI, NVME doesn't complete request atomically, so double
>>> completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
>>>
>>> Given request is completed remotely or asynchronously in the two code paths,
>>> the teardown_lock can't protect the case.
>>>
>>> One solution is to apply the introduced blk_mq_complete_request_sync()
>>> in both two code paths.
>>>
>>> Another candidate is to use nvme_sync_queues() before teardown, such as
>>> the following change:
>>>
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index d6a3e1487354..dc3561ca0074 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -1909,6 +1909,7 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>>>    	blk_mq_quiesce_queue(ctrl->admin_q);
>>>    	nvme_start_freeze(ctrl);
>>>    	nvme_stop_queues(ctrl);
>>> +	nvme_sync_queues(ctrl);
>> nvme_sync_queues now sync io queues and admin queues, so we may need like this:
>> nvme_sync_io_queues(struct nvme_ctrl *ctrl)
>> {
>> 	down_read(&ctrl->namespaces_rwsem);
>> 	list_for_each_entry(ns, &ctrl->namespaces, list)
>> 		blk_sync_queue(ns->queue);
>> 	up_read(&ctrl->namespaces_rwsem);
>> }
> 
> Looks not necessary to do that, because admin queue is quiesced in
> nvme_tcp_teardown_io_queues(), so it is safe to sync admin queue here too.
sync admin queue is not necessary. It should be done in teardown admin queue.
And rdma do not quiesce the admin queue in tear down io queue.
So separate nvme_sync_io_queues may be a better choice.
> 
> 
> Thanks,
> Ming
> 
> .
> 


From ming.lei at redhat.com  Tue Oct 13 23:34:34 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed, 14 Oct 2020 11:34:34 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201014010813.GA775684@T590>
References: <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590>
Message-ID: <20201014033434.GC775684@T590>

On Wed, Oct 14, 2020 at 09:08:28AM +0800, Ming Lei wrote:
> On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
> > 
> > > > > This may just reduce the probability. The concurrency of timeout
> > > > > and teardown will cause the same request
> > > > > be treated repeatly, this is not we expected.
> > > > 
> > > > That is right, not like SCSI, NVME doesn't apply atomic request
> > > > completion, so
> > > > request may be completed/freed from both timeout & nvme_cancel_request().
> > > > 
> > > > .teardown_lock still may cover the race with Sagi's patch because
> > > > teardown
> > > > actually cancels requests in sync style.
> > > In extreme scenarios, the request may be already retry success(rq state
> > > change to inflight).
> > > Timeout processing may wrongly stop the queue and abort the request.
> > > teardown_lock serialize the process of timeout and teardown, but do not
> > > avoid the race.
> > > It might not be safe.
> > 
> > Not sure I understand the scenario you are describing.
> > 
> > what do you mean by "In extreme scenarios, the request may be already retry
> > success(rq state change to inflight)"?
> > 
> > What will retry the request? only when the host will reconnect
> > the request will be retried.
> > 
> > We can call nvme_sync_queues in the last part of the teardown, but
> > I still don't understand the race here.
> 
> Not like SCSI, NVME doesn't complete request atomically, so double
> completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
> 
> Given request is completed remotely or asynchronously in the two code paths,
> the teardown_lock can't protect the case.

Thinking of the issue further, the race shouldn't be between timeout and
teardown.

Both nvme_cancel_request() and nvme_tcp_complete_timed_out() are called
with .teardown_lock, and both check if the request is completed before
calling blk_mq_complete_request() which marks the request as COMPLETE state.
So the request shouldn't be double-freed in the two code paths.

Another possible reason is that between timeout and normal completion(fail
fast pending requests after ctrl state is updated to CONNECTING).

Yi, can you try the following patch and see if the issue is fixed?

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index d6a3e1487354..fab9220196bd 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1886,7 +1886,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
 	if (ctrl->admin_tagset) {
@@ -1897,15 +1896,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 	if (remove)
 		blk_mq_unquiesce_queue(ctrl->admin_q);
 	nvme_tcp_destroy_admin_queue(ctrl, remove);
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	if (ctrl->queue_count <= 1)
-		goto out;
+		return;
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_start_freeze(ctrl);
 	nvme_stop_queues(ctrl);
@@ -1918,8 +1915,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 	if (remove)
 		nvme_start_queues(ctrl);
 	nvme_tcp_destroy_io_queues(ctrl, remove);
-out:
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
@@ -2030,11 +2025,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
 	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
 
 	nvme_stop_keep_alive(ctrl);
+
+	mutex_lock(&tcp_ctrl->teardown_lock);
 	nvme_tcp_teardown_io_queues(ctrl, false);
-	/* unquiesce to fail fast pending requests */
-	nvme_start_queues(ctrl);
 	nvme_tcp_teardown_admin_queue(ctrl, false);
-	blk_mq_unquiesce_queue(ctrl->admin_q);
+	mutex_unlock(&tcp_ctrl->teardown_lock);
 
 	if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_CONNECTING)) {
 		/* state change failure is ok if we started ctrl delete */
@@ -2043,6 +2038,9 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
 		return;
 	}
 
+	/* drain pending timeout for avoiding race with normal completion */
+	nvme_sync_queues(ctrl);
+
 	nvme_tcp_reconnect_or_remove(ctrl);
 }
 
@@ -2051,6 +2049,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
 	cancel_work_sync(&to_tcp_ctrl(ctrl)->err_work);
 	cancel_delayed_work_sync(&to_tcp_ctrl(ctrl)->connect_work);
 
+	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_teardown_io_queues(ctrl, shutdown);
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	if (shutdown)
@@ -2058,6 +2057,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
 	else
 		nvme_disable_ctrl(ctrl);
 	nvme_tcp_teardown_admin_queue(ctrl, shutdown);
+	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_delete_ctrl(struct nvme_ctrl *ctrl)
@@ -2080,6 +2080,9 @@ static void nvme_reset_ctrl_work(struct work_struct *work)
 		return;
 	}
 
+	/* drain pending timeout for avoiding race with normal completion */
+	nvme_sync_queues(ctrl);
+
 	if (nvme_tcp_setup_ctrl(ctrl, false))
 		goto out_fail;
 

Thanks,
Ming



From pizhenwei at bytedance.com  Tue Oct 13 23:36:50 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Wed, 14 Oct 2020 11:36:50 +0800
Subject: [PATCH v2] nvmet: fix uninitialized work for zero kato
Message-ID: <20201014033650.953656-1-pizhenwei@bytedance.com>

Hit a warning:
WARNING: CPU: 1 PID: 241 at kernel/workqueue.c:1627 __queue_delayed_work+0x6d/0x90
with trace:
  mod_delayed_work_on+0x59/0x90
  nvmet_update_cc+0xee/0x100 [nvmet]
  nvmet_execute_prop_set+0x72/0x80 [nvmet]
  nvmet_tcp_try_recv_pdu+0x2f7/0x770 [nvmet_tcp]
  nvmet_tcp_io_work+0x63f/0xb2d [nvmet_tcp]
  ...

This could be reproduced easily with a keep alive time 0:
nvme connect -t tcp -n NQN -a ADDR -s PORT --keep-alive-tmo=0

Starting an uninitialized work when initiator connects with zero
kato. Althrough keep-alive timer is disabled during allocating a ctrl
(fix in 0d3b6a8d213a), ka_work still has a chance to run
(called by nvmet_start_ctrl).

Fixes:
Don't run keep alive work with zero kato.

Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/target/core.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index b7b63330b5ef..90e0c84df2af 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1126,7 +1126,8 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
 	 * in case a host died before it enabled the controller.  Hence, simply
 	 * reset the keep alive timer when the controller is enabled.
 	 */
-	mod_delayed_work(system_wq, &ctrl->ka_work, ctrl->kato * HZ);
+	if (ctrl->kato)
+		mod_delayed_work(system_wq, &ctrl->ka_work, ctrl->kato * HZ);
 }
 
 static void nvmet_clear_ctrl(struct nvmet_ctrl *ctrl)
-- 
2.11.0



From lengchao at huawei.com  Wed Oct 14 05:39:12 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 14 Oct 2020 17:39:12 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201014033434.GC775684@T590>
References: <1711488120.3435389.1602219830518.JavaMail.zimbra@redhat.com>
 <e39b711e-ebbd-8955-ca19-07c1dad26fa2@grimberg.me>
 <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590> <20201014033434.GC775684@T590>
Message-ID: <f5870b91-28c5-ea99-59df-cdcc8c482011@huawei.com>



On 2020/10/14 11:34, Ming Lei wrote:
> On Wed, Oct 14, 2020 at 09:08:28AM +0800, Ming Lei wrote:
>> On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
>>>
>>>>>> This may just reduce the probability. The concurrency of timeout
>>>>>> and teardown will cause the same request
>>>>>> be treated repeatly, this is not we expected.
>>>>>
>>>>> That is right, not like SCSI, NVME doesn't apply atomic request
>>>>> completion, so
>>>>> request may be completed/freed from both timeout & nvme_cancel_request().
>>>>>
>>>>> .teardown_lock still may cover the race with Sagi's patch because
>>>>> teardown
>>>>> actually cancels requests in sync style.
>>>> In extreme scenarios, the request may be already retry success(rq state
>>>> change to inflight).
>>>> Timeout processing may wrongly stop the queue and abort the request.
>>>> teardown_lock serialize the process of timeout and teardown, but do not
>>>> avoid the race.
>>>> It might not be safe.
>>>
>>> Not sure I understand the scenario you are describing.
>>>
>>> what do you mean by "In extreme scenarios, the request may be already retry
>>> success(rq state change to inflight)"?
>>>
>>> What will retry the request? only when the host will reconnect
>>> the request will be retried.
>>>
>>> We can call nvme_sync_queues in the last part of the teardown, but
>>> I still don't understand the race here.
>>
>> Not like SCSI, NVME doesn't complete request atomically, so double
>> completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
>>
>> Given request is completed remotely or asynchronously in the two code paths,
>> the teardown_lock can't protect the case.
> 
> Thinking of the issue further, the race shouldn't be between timeout and
> teardown.
> 
> Both nvme_cancel_request() and nvme_tcp_complete_timed_out() are called
> with .teardown_lock, and both check if the request is completed before
> calling blk_mq_complete_request() which marks the request as COMPLETE state.
> So the request shouldn't be double-freed in the two code paths.
> 
> Another possible reason is that between timeout and normal completion(fail
> fast pending requests after ctrl state is updated to CONNECTING).
> 
> Yi, can you try the following patch and see if the issue is fixed?
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index d6a3e1487354..fab9220196bd 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -1886,7 +1886,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
>   static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
>   		bool remove)
>   {
> -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   	blk_mq_quiesce_queue(ctrl->admin_q);
>   	nvme_tcp_stop_queue(ctrl, 0);
>   	if (ctrl->admin_tagset) {
> @@ -1897,15 +1896,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
>   	if (remove)
>   		blk_mq_unquiesce_queue(ctrl->admin_q);
>   	nvme_tcp_destroy_admin_queue(ctrl, remove);
> -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   }
>   
>   static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>   		bool remove)
>   {
> -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   	if (ctrl->queue_count <= 1)
> -		goto out;
> +		return;
>   	blk_mq_quiesce_queue(ctrl->admin_q);
>   	nvme_start_freeze(ctrl);
>   	nvme_stop_queues(ctrl);
> @@ -1918,8 +1915,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>   	if (remove)
>   		nvme_start_queues(ctrl);
>   	nvme_tcp_destroy_io_queues(ctrl, remove);
> -out:
> -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   }
>   
>   static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
> @@ -2030,11 +2025,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
>   	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
>   
>   	nvme_stop_keep_alive(ctrl);
> +
> +	mutex_lock(&tcp_ctrl->teardown_lock);
>   	nvme_tcp_teardown_io_queues(ctrl, false);
> -	/* unquiesce to fail fast pending requests */
> -	nvme_start_queues(ctrl);
>   	nvme_tcp_teardown_admin_queue(ctrl, false);
> -	blk_mq_unquiesce_queue(ctrl->admin_q);
Delete blk_mq_unquiesce_queue will cause a bug which may cause reconnect failed.
Delete nvme_start_queues may cause another bug.
> +	mutex_unlock(&tcp_ctrl->teardown_lock);
>   
>   	if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_CONNECTING)) {
>   		/* state change failure is ok if we started ctrl delete */
> @@ -2043,6 +2038,9 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
>   		return;
>   	}
>   
> +	/* drain pending timeout for avoiding race with normal completion */
> +	nvme_sync_queues(ctrl);
> +
>   	nvme_tcp_reconnect_or_remove(ctrl);
>   }
>   
> @@ -2051,6 +2049,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
>   	cancel_work_sync(&to_tcp_ctrl(ctrl)->err_work);
>   	cancel_delayed_work_sync(&to_tcp_ctrl(ctrl)->connect_work);
>   
> +	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   	nvme_tcp_teardown_io_queues(ctrl, shutdown);
>   	blk_mq_quiesce_queue(ctrl->admin_q);
>   	if (shutdown)
> @@ -2058,6 +2057,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
>   	else
>   		nvme_disable_ctrl(ctrl);
>   	nvme_tcp_teardown_admin_queue(ctrl, shutdown);
> +	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   }
>   
>   static void nvme_tcp_delete_ctrl(struct nvme_ctrl *ctrl)
> @@ -2080,6 +2080,9 @@ static void nvme_reset_ctrl_work(struct work_struct *work)
>   		return;
>   	}
>   
> +	/* drain pending timeout for avoiding race with normal completion */
> +	nvme_sync_queues(ctrl);
> +
>   	if (nvme_tcp_setup_ctrl(ctrl, false))
>   		goto out_fail;
>   
> 
> Thanks,
> Ming
> 
> .
> 


From ming.lei at redhat.com  Wed Oct 14 05:56:42 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed, 14 Oct 2020 17:56:42 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <f5870b91-28c5-ea99-59df-cdcc8c482011@huawei.com>
References: <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590> <20201014033434.GC775684@T590>
 <f5870b91-28c5-ea99-59df-cdcc8c482011@huawei.com>
Message-ID: <20201014095642.GE775684@T590>

On Wed, Oct 14, 2020 at 05:39:12PM +0800, Chao Leng wrote:
> 
> 
> On 2020/10/14 11:34, Ming Lei wrote:
> > On Wed, Oct 14, 2020 at 09:08:28AM +0800, Ming Lei wrote:
> > > On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
> > > > 
> > > > > > > This may just reduce the probability. The concurrency of timeout
> > > > > > > and teardown will cause the same request
> > > > > > > be treated repeatly, this is not we expected.
> > > > > > 
> > > > > > That is right, not like SCSI, NVME doesn't apply atomic request
> > > > > > completion, so
> > > > > > request may be completed/freed from both timeout & nvme_cancel_request().
> > > > > > 
> > > > > > .teardown_lock still may cover the race with Sagi's patch because
> > > > > > teardown
> > > > > > actually cancels requests in sync style.
> > > > > In extreme scenarios, the request may be already retry success(rq state
> > > > > change to inflight).
> > > > > Timeout processing may wrongly stop the queue and abort the request.
> > > > > teardown_lock serialize the process of timeout and teardown, but do not
> > > > > avoid the race.
> > > > > It might not be safe.
> > > > 
> > > > Not sure I understand the scenario you are describing.
> > > > 
> > > > what do you mean by "In extreme scenarios, the request may be already retry
> > > > success(rq state change to inflight)"?
> > > > 
> > > > What will retry the request? only when the host will reconnect
> > > > the request will be retried.
> > > > 
> > > > We can call nvme_sync_queues in the last part of the teardown, but
> > > > I still don't understand the race here.
> > > 
> > > Not like SCSI, NVME doesn't complete request atomically, so double
> > > completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
> > > 
> > > Given request is completed remotely or asynchronously in the two code paths,
> > > the teardown_lock can't protect the case.
> > 
> > Thinking of the issue further, the race shouldn't be between timeout and
> > teardown.
> > 
> > Both nvme_cancel_request() and nvme_tcp_complete_timed_out() are called
> > with .teardown_lock, and both check if the request is completed before
> > calling blk_mq_complete_request() which marks the request as COMPLETE state.
> > So the request shouldn't be double-freed in the two code paths.
> > 
> > Another possible reason is that between timeout and normal completion(fail
> > fast pending requests after ctrl state is updated to CONNECTING).
> > 
> > Yi, can you try the following patch and see if the issue is fixed?
> > 
> > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > index d6a3e1487354..fab9220196bd 100644
> > --- a/drivers/nvme/host/tcp.c
> > +++ b/drivers/nvme/host/tcp.c
> > @@ -1886,7 +1886,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
> >   static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
> >   		bool remove)
> >   {
> > -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
> >   	blk_mq_quiesce_queue(ctrl->admin_q);
> >   	nvme_tcp_stop_queue(ctrl, 0);
> >   	if (ctrl->admin_tagset) {
> > @@ -1897,15 +1896,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
> >   	if (remove)
> >   		blk_mq_unquiesce_queue(ctrl->admin_q);
> >   	nvme_tcp_destroy_admin_queue(ctrl, remove);
> > -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
> >   }
> >   static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
> >   		bool remove)
> >   {
> > -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
> >   	if (ctrl->queue_count <= 1)
> > -		goto out;
> > +		return;
> >   	blk_mq_quiesce_queue(ctrl->admin_q);
> >   	nvme_start_freeze(ctrl);
> >   	nvme_stop_queues(ctrl);
> > @@ -1918,8 +1915,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
> >   	if (remove)
> >   		nvme_start_queues(ctrl);
> >   	nvme_tcp_destroy_io_queues(ctrl, remove);
> > -out:
> > -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
> >   }
> >   static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
> > @@ -2030,11 +2025,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
> >   	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
> >   	nvme_stop_keep_alive(ctrl);
> > +
> > +	mutex_lock(&tcp_ctrl->teardown_lock);
> >   	nvme_tcp_teardown_io_queues(ctrl, false);
> > -	/* unquiesce to fail fast pending requests */
> > -	nvme_start_queues(ctrl);
> >   	nvme_tcp_teardown_admin_queue(ctrl, false);
> > -	blk_mq_unquiesce_queue(ctrl->admin_q);
> Delete blk_mq_unquiesce_queue will cause a bug which may cause reconnect failed.
> Delete nvme_start_queues may cause another bug.

nvme_tcp_setup_ctrl() will re-start io and admin queue, and only .connect_q
and .fabrics_q are required during reconnect.

So can you explain in detail about the bug?

Thanks,
Ming



From logang at deltatee.com  Wed Oct 14 11:47:58 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Wed, 14 Oct 2020 09:47:58 -0600
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <BYAPR04MB49654B340081F257BF9C4BB486050@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20201009231816.1524-1-logang@deltatee.com>
 <BYAPR04MB496556A3CBC239BC57EE588B86050@BYAPR04MB4965.namprd04.prod.outlook.com>
 <e7d76d3e-955b-1827-fd2f-9b7db49646ed@deltatee.com>
 <BYAPR04MB49654B340081F257BF9C4BB486050@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <542a1ba5-3a6a-1a5d-6b35-de5c2d1ff8ac@deltatee.com>



On 2020-10-13 6:25 p.m., Chaitanya Kulkarni wrote:
> On 10/13/20 17:20, Logan Gunthorpe wrote:
>>
>> On 2020-10-13 6:16 p.m., Chaitanya Kulkarni wrote:
>>> On 10/9/20 16:18, Logan Gunthorpe wrote:
>>>> Clean up some confusing elements of nvmet_passthru_map_sg() by returning
>>>> early if the request is greater than the maximum bio size. This allows
>>>> us to drop the sg_cnt variable.
>>>>
>>>> This should not result in any functional change but makes the code
>>>> clearer and more understandable. The original code allocated a truncated
>>>> bio then would return EINVAL when bio_add_pc_page() filled that bio. The
>>>> new code just returns EINVAL early if this would happen.
>>>>
>>>> Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
>>>> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
>>>> Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
>>>> Cc: Christoph Hellwig <hch at lst.de>
>>>> Cc: Sagi Grimberg <sagi at grimberg.me>
>>>> Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
>>> why the prefix is nvmet-passthru ?
>> Well that's the prefix I used in the first patch, which follows other
>> similar conventions like nvmet-tcp, nvmet-fc, nvmet-rdma, nvmet-loop, etc.
>>
>> Logan
>>
>>
> Those are transport types and passthru is a target backend.
> 
> I found this [1] in the git log, only one patch has that prefix.
> 
> The convention is to not add backend prefixes for target patches (see
> bdev/file),
> 
> I'm not sure we want to start adding prefixes unless there is a specific
> 
> reason.

Well, I don't see the harm in it, but if whomever merges it wants to
change the prefix, I won't object.

Logan


From kbusch at kernel.org  Wed Oct 14 12:10:48 2020
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 14 Oct 2020 09:10:48 -0700
Subject: [PATCH v2] nvmet: fix uninitialized work for zero kato
In-Reply-To: <20201014033650.953656-1-pizhenwei@bytedance.com>
References: <20201014033650.953656-1-pizhenwei@bytedance.com>
Message-ID: <20201014161048.GA1267726@dhcp-10-100-145-180.wdl.wdc.com>

On Wed, Oct 14, 2020 at 11:36:50AM +0800, zhenwei pi wrote:
> Fixes:
> Don't run keep alive work with zero kato.

"Fixes" tags need to have a git commit id followed by the commit
subject. I can't find any commit with that subject, though.


From sagi at grimberg.me  Wed Oct 14 15:00:36 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 14 Oct 2020 12:00:36 -0700
Subject: [PATCH v2] nvmet: fix uninitialized work for zero kato
In-Reply-To: <20201014161048.GA1267726@dhcp-10-100-145-180.wdl.wdc.com>
References: <20201014033650.953656-1-pizhenwei@bytedance.com>
 <20201014161048.GA1267726@dhcp-10-100-145-180.wdl.wdc.com>
Message-ID: <517a0b11-5bd0-4d8e-da0e-e0e0a4109897@grimberg.me>


>> Fixes:
>> Don't run keep alive work with zero kato.
> 
> "Fixes" tags need to have a git commit id followed by the commit
> subject. I can't find any commit with that subject, though.

Fixes: 0d3b6a8d213a ("nvmet: Disable keep-alive timer when kato is 
cleared to 0h")


From kbusch at kernel.org  Wed Oct 14 16:38:45 2020
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 14 Oct 2020 13:38:45 -0700
Subject: [PATCH] nvme: set passthrough command gendisk
Message-ID: <20201014203845.1400053-1-kbusch@kernel.org>

The nvme event traces for IO commands are reporting the wrong QID
because we never set the request's gendisk. Set this parameter if the
passthrough command is being sent through an IO queue.

Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/core.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 95ef4943d8bd..ab5921e824d5 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -894,9 +894,14 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		unsigned timeout, int qid, int at_head,
 		blk_mq_req_flags_t flags, bool poll)
 {
+	struct nvme_ns *ns = q->queuedata;
+	struct gendisk *disk = NULL;
 	struct request *req;
 	int ret;
 
+	if (ns)
+		disk = ns->disk;
+
 	req = nvme_alloc_request(q, cmd, flags, qid);
 	if (IS_ERR(req))
 		return PTR_ERR(req);
@@ -910,9 +915,9 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 	}
 
 	if (poll)
-		nvme_execute_rq_polled(req->q, NULL, req, at_head);
+		nvme_execute_rq_polled(req->q, disk, req, at_head);
 	else
-		blk_execute_rq(req->q, NULL, req, at_head);
+		blk_execute_rq(req->q, disk, req, at_head);
 	if (result)
 		*result = nvme_req(req)->result;
 	if (nvme_req(req)->flags & NVME_REQ_CANCELLED)
-- 
2.24.1



From pizhenwei at bytedance.com  Wed Oct 14 21:51:40 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Thu, 15 Oct 2020 09:51:40 +0800
Subject: [PATCH v3] nvmet: fix uninitialized work for zero kato
Message-ID: <20201015015140.1061936-1-pizhenwei@bytedance.com>

Hit a warning:
WARNING: CPU: 1 PID: 241 at kernel/workqueue.c:1627 __queue_delayed_work+0x6d/0x90
with trace:
  mod_delayed_work_on+0x59/0x90
  nvmet_update_cc+0xee/0x100 [nvmet]
  nvmet_execute_prop_set+0x72/0x80 [nvmet]
  nvmet_tcp_try_recv_pdu+0x2f7/0x770 [nvmet_tcp]
  nvmet_tcp_io_work+0x63f/0xb2d [nvmet_tcp]
  ...

This could be reproduced easily with a keep alive time 0:
nvme connect -t tcp -n NQN -a ADDR -s PORT --keep-alive-tmo=0

Starting an uninitialized work when initiator connects with zero
kato. Althrough keep-alive timer is disabled during allocating a ctrl
(fix in 0d3b6a8d213a), ka_work still has a chance to run
(called by nvmet_start_ctrl).

Fixes: 0d3b6a8d213a ("nvmet: Disable keep-alive timer when kato is cleared to 0h") 
Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/target/core.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 25d62d867563..aafcbc424b7a 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1126,7 +1126,8 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
 	 * in case a host died before it enabled the controller.  Hence, simply
 	 * reset the keep alive timer when the controller is enabled.
 	 */
-	mod_delayed_work(system_wq, &ctrl->ka_work, ctrl->kato * HZ);
+	if (ctrl->kato)
+		mod_delayed_work(system_wq, &ctrl->ka_work, ctrl->kato * HZ);
 }
 
 static void nvmet_clear_ctrl(struct nvmet_ctrl *ctrl)
-- 
2.11.0



From kai.heng.feng at canonical.com  Thu Oct 15 01:09:30 2020
From: kai.heng.feng at canonical.com (Kai-Heng Feng)
Date: Thu, 15 Oct 2020 13:09:30 +0800
Subject: [PATCH] nvme-pci: Disable Write Zeroes on Sandisk Skyhawk
In-Reply-To: <BYAPR04MB496518A4C9FC65964C58453386050@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20200723155731.22313-1-kai.heng.feng@canonical.com>
 <20201013083445.12317-1-kai.heng.feng@canonical.com>
 <BYAPR04MB496518A4C9FC65964C58453386050@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <73A03A07-05E3-40FA-991F-5B98C05B51D1@canonical.com>



> On Oct 14, 2020, at 08:20, Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com> wrote:
> 
> On 10/13/20 01:45, Kai-Heng Feng wrote:
>> Like commit 5611ec2b9814 ("nvme-pci: prevent SK hynix PC400 from using
>> Write Zeroes command"), Sandisk Skyhawk has the same issue:
>> [ 6305.633887] blk_update_request: operation not supported error, dev nvme0n1, sector 340812032 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
>> 
>> So also disable Write Zeroes command on Sandisk Skyhawk.
>> 
>> BugLink: https://bugs.launchpad.net/bugs/1899503
>> Signed-off-by: Kai-Heng Feng <kai.heng.feng at canonical.com>
> 
> Are you sure this happens all the devices of the same model
> 
> and not a firmware bug on specific device ?
> 
> If yes then looks good.

Yes, and this is the reply from WD, requested by the user:
"These are newest WD branded drives and WD already confirmed they are
the newest firmware / no upgrades are available."

Kai-Heng

> 
> Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> 
> 
> 



From lengchao at huawei.com  Thu Oct 15 02:05:01 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 15 Oct 2020 14:05:01 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201014095642.GE775684@T590>
References: <23f19725-f46b-7de7-915d-b97fd6d69cdc@redhat.com>
 <ced685bf-ad48-ac41-8089-8c5ba09abfcb@grimberg.me>
 <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590> <20201014033434.GC775684@T590>
 <f5870b91-28c5-ea99-59df-cdcc8c482011@huawei.com>
 <20201014095642.GE775684@T590>
Message-ID: <c9cf7168-d8ce-276f-de01-739199ed4258@huawei.com>



On 2020/10/14 17:56, Ming Lei wrote:
> On Wed, Oct 14, 2020 at 05:39:12PM +0800, Chao Leng wrote:
>>
>>
>> On 2020/10/14 11:34, Ming Lei wrote:
>>> On Wed, Oct 14, 2020 at 09:08:28AM +0800, Ming Lei wrote:
>>>> On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
>>>>>
>>>>>>>> This may just reduce the probability. The concurrency of timeout
>>>>>>>> and teardown will cause the same request
>>>>>>>> be treated repeatly, this is not we expected.
>>>>>>>
>>>>>>> That is right, not like SCSI, NVME doesn't apply atomic request
>>>>>>> completion, so
>>>>>>> request may be completed/freed from both timeout & nvme_cancel_request().
>>>>>>>
>>>>>>> .teardown_lock still may cover the race with Sagi's patch because
>>>>>>> teardown
>>>>>>> actually cancels requests in sync style.
>>>>>> In extreme scenarios, the request may be already retry success(rq state
>>>>>> change to inflight).
>>>>>> Timeout processing may wrongly stop the queue and abort the request.
>>>>>> teardown_lock serialize the process of timeout and teardown, but do not
>>>>>> avoid the race.
>>>>>> It might not be safe.
>>>>>
>>>>> Not sure I understand the scenario you are describing.
>>>>>
>>>>> what do you mean by "In extreme scenarios, the request may be already retry
>>>>> success(rq state change to inflight)"?
>>>>>
>>>>> What will retry the request? only when the host will reconnect
>>>>> the request will be retried.
>>>>>
>>>>> We can call nvme_sync_queues in the last part of the teardown, but
>>>>> I still don't understand the race here.
>>>>
>>>> Not like SCSI, NVME doesn't complete request atomically, so double
>>>> completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
>>>>
>>>> Given request is completed remotely or asynchronously in the two code paths,
>>>> the teardown_lock can't protect the case.
>>>
>>> Thinking of the issue further, the race shouldn't be between timeout and
>>> teardown.
>>>
>>> Both nvme_cancel_request() and nvme_tcp_complete_timed_out() are called
>>> with .teardown_lock, and both check if the request is completed before
>>> calling blk_mq_complete_request() which marks the request as COMPLETE state.
>>> So the request shouldn't be double-freed in the two code paths.
>>>
>>> Another possible reason is that between timeout and normal completion(fail
>>> fast pending requests after ctrl state is updated to CONNECTING).
>>>
>>> Yi, can you try the following patch and see if the issue is fixed?
>>>
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index d6a3e1487354..fab9220196bd 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -1886,7 +1886,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
>>>    static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
>>>    		bool remove)
>>>    {
>>> -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>    	blk_mq_quiesce_queue(ctrl->admin_q);
>>>    	nvme_tcp_stop_queue(ctrl, 0);
>>>    	if (ctrl->admin_tagset) {
>>> @@ -1897,15 +1896,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
>>>    	if (remove)
>>>    		blk_mq_unquiesce_queue(ctrl->admin_q);
>>>    	nvme_tcp_destroy_admin_queue(ctrl, remove);
>>> -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>    }
>>>    static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>>>    		bool remove)
>>>    {
>>> -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>    	if (ctrl->queue_count <= 1)
>>> -		goto out;
>>> +		return;
>>>    	blk_mq_quiesce_queue(ctrl->admin_q);
>>>    	nvme_start_freeze(ctrl);
>>>    	nvme_stop_queues(ctrl);
>>> @@ -1918,8 +1915,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>>>    	if (remove)
>>>    		nvme_start_queues(ctrl);
>>>    	nvme_tcp_destroy_io_queues(ctrl, remove);
>>> -out:
>>> -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>    }
>>>    static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
>>> @@ -2030,11 +2025,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
>>>    	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
>>>    	nvme_stop_keep_alive(ctrl);
>>> +
>>> +	mutex_lock(&tcp_ctrl->teardown_lock);
>>>    	nvme_tcp_teardown_io_queues(ctrl, false);
>>> -	/* unquiesce to fail fast pending requests */
>>> -	nvme_start_queues(ctrl);
>>>    	nvme_tcp_teardown_admin_queue(ctrl, false);
>>> -	blk_mq_unquiesce_queue(ctrl->admin_q);
>> Delete blk_mq_unquiesce_queue will cause a bug which may cause reconnect failed.
>> Delete nvme_start_queues may cause another bug.
> 
> nvme_tcp_setup_ctrl() will re-start io and admin queue, and only .connect_q
> and .fabrics_q are required during reconnect.I check the code. Unquiesce the admin queue in nvme_tcp_configure_admin_queue, so reconnect can work well.
> 
> So can you explain in detail about the bug?
First if reconnect failed, quiesce the io queue and admin queue will cause IO pause long time.
Second if reconnect failed more than max_reconnects, delete ctrl will hang.
Second, I'm not sure. you can check it.
The hang function is nvme_remove_namespaces in nvme_do_delete_ctrl.
> 
> Thanks,
> Ming
> 
> .
> 


From hch at lst.de  Thu Oct 15 03:42:22 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 15 Oct 2020 09:42:22 +0200
Subject: [PATCH] nvme: set passthrough command gendisk
In-Reply-To: <20201014203845.1400053-1-kbusch@kernel.org>
References: <20201014203845.1400053-1-kbusch@kernel.org>
Message-ID: <20201015074222.GA14082@lst.de>

On Wed, Oct 14, 2020 at 01:38:45PM -0700, Keith Busch wrote:
> The nvme event traces for IO commands are reporting the wrong QID
> because we never set the request's gendisk. Set this parameter if the
> passthrough command is being sent through an IO queue.

Why do we event need the check for a non-NULL disk in nvme_req_qid?


From hch at lst.de  Thu Oct 15 03:45:32 2020
From: hch at lst.de (hch at lst.de)
Date: Thu, 15 Oct 2020 09:45:32 +0200
Subject: NVME driver throw out NULL pointer during initialization and
 system will be reboot
In-Reply-To: <MN2PR12MB43048EF6642F885022011626F6070@MN2PR12MB4304.namprd12.prod.outlook.com>
References: <MN2PR12MB43048EF6642F885022011626F6070@MN2PR12MB4304.namprd12.prod.outlook.com>
Message-ID: <20201015074532.GE14082@lst.de>

On Mon, Oct 12, 2020 at 11:52:41AM +0000, Yang, Bear wrote:
> [AMD Official Use Only - Internal Distribution Only]
> 
> Hi ALL,
> I encountered a problem and not sure if this is a NVME driver bug, so want to ask you to help to see if there is a solution already.
> 
> Server: AMD 7502(32core) * 2, total 128 threads
> OS: RHEL 7.6.6

Please report the problem to Red Hat.


From ming.lei at redhat.com  Thu Oct 15 03:50:20 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Thu, 15 Oct 2020 15:50:20 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <c9cf7168-d8ce-276f-de01-739199ed4258@huawei.com>
References: <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590> <20201014033434.GC775684@T590>
 <f5870b91-28c5-ea99-59df-cdcc8c482011@huawei.com>
 <20201014095642.GE775684@T590>
 <c9cf7168-d8ce-276f-de01-739199ed4258@huawei.com>
Message-ID: <20201015075020.GA1099950@T590>

On Thu, Oct 15, 2020 at 02:05:01PM +0800, Chao Leng wrote:
> 
> 
> On 2020/10/14 17:56, Ming Lei wrote:
> > On Wed, Oct 14, 2020 at 05:39:12PM +0800, Chao Leng wrote:
> > > 
> > > 
> > > On 2020/10/14 11:34, Ming Lei wrote:
> > > > On Wed, Oct 14, 2020 at 09:08:28AM +0800, Ming Lei wrote:
> > > > > On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
> > > > > > 
> > > > > > > > > This may just reduce the probability. The concurrency of timeout
> > > > > > > > > and teardown will cause the same request
> > > > > > > > > be treated repeatly, this is not we expected.
> > > > > > > > 
> > > > > > > > That is right, not like SCSI, NVME doesn't apply atomic request
> > > > > > > > completion, so
> > > > > > > > request may be completed/freed from both timeout & nvme_cancel_request().
> > > > > > > > 
> > > > > > > > .teardown_lock still may cover the race with Sagi's patch because
> > > > > > > > teardown
> > > > > > > > actually cancels requests in sync style.
> > > > > > > In extreme scenarios, the request may be already retry success(rq state
> > > > > > > change to inflight).
> > > > > > > Timeout processing may wrongly stop the queue and abort the request.
> > > > > > > teardown_lock serialize the process of timeout and teardown, but do not
> > > > > > > avoid the race.
> > > > > > > It might not be safe.
> > > > > > 
> > > > > > Not sure I understand the scenario you are describing.
> > > > > > 
> > > > > > what do you mean by "In extreme scenarios, the request may be already retry
> > > > > > success(rq state change to inflight)"?
> > > > > > 
> > > > > > What will retry the request? only when the host will reconnect
> > > > > > the request will be retried.
> > > > > > 
> > > > > > We can call nvme_sync_queues in the last part of the teardown, but
> > > > > > I still don't understand the race here.
> > > > > 
> > > > > Not like SCSI, NVME doesn't complete request atomically, so double
> > > > > completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
> > > > > 
> > > > > Given request is completed remotely or asynchronously in the two code paths,
> > > > > the teardown_lock can't protect the case.
> > > > 
> > > > Thinking of the issue further, the race shouldn't be between timeout and
> > > > teardown.
> > > > 
> > > > Both nvme_cancel_request() and nvme_tcp_complete_timed_out() are called
> > > > with .teardown_lock, and both check if the request is completed before
> > > > calling blk_mq_complete_request() which marks the request as COMPLETE state.
> > > > So the request shouldn't be double-freed in the two code paths.
> > > > 
> > > > Another possible reason is that between timeout and normal completion(fail
> > > > fast pending requests after ctrl state is updated to CONNECTING).
> > > > 
> > > > Yi, can you try the following patch and see if the issue is fixed?
> > > > 
> > > > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > > > index d6a3e1487354..fab9220196bd 100644
> > > > --- a/drivers/nvme/host/tcp.c
> > > > +++ b/drivers/nvme/host/tcp.c
> > > > @@ -1886,7 +1886,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
> > > >    static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
> > > >    		bool remove)
> > > >    {
> > > > -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
> > > >    	blk_mq_quiesce_queue(ctrl->admin_q);
> > > >    	nvme_tcp_stop_queue(ctrl, 0);
> > > >    	if (ctrl->admin_tagset) {
> > > > @@ -1897,15 +1896,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
> > > >    	if (remove)
> > > >    		blk_mq_unquiesce_queue(ctrl->admin_q);
> > > >    	nvme_tcp_destroy_admin_queue(ctrl, remove);
> > > > -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
> > > >    }
> > > >    static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
> > > >    		bool remove)
> > > >    {
> > > > -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
> > > >    	if (ctrl->queue_count <= 1)
> > > > -		goto out;
> > > > +		return;
> > > >    	blk_mq_quiesce_queue(ctrl->admin_q);
> > > >    	nvme_start_freeze(ctrl);
> > > >    	nvme_stop_queues(ctrl);
> > > > @@ -1918,8 +1915,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
> > > >    	if (remove)
> > > >    		nvme_start_queues(ctrl);
> > > >    	nvme_tcp_destroy_io_queues(ctrl, remove);
> > > > -out:
> > > > -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
> > > >    }
> > > >    static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
> > > > @@ -2030,11 +2025,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
> > > >    	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
> > > >    	nvme_stop_keep_alive(ctrl);
> > > > +
> > > > +	mutex_lock(&tcp_ctrl->teardown_lock);
> > > >    	nvme_tcp_teardown_io_queues(ctrl, false);
> > > > -	/* unquiesce to fail fast pending requests */
> > > > -	nvme_start_queues(ctrl);
> > > >    	nvme_tcp_teardown_admin_queue(ctrl, false);
> > > > -	blk_mq_unquiesce_queue(ctrl->admin_q);
> > > Delete blk_mq_unquiesce_queue will cause a bug which may cause reconnect failed.
> > > Delete nvme_start_queues may cause another bug.
> > 
> > nvme_tcp_setup_ctrl() will re-start io and admin queue, and only .connect_q
> > and .fabrics_q are required during reconnect.I check the code. Unquiesce the admin queue in nvme_tcp_configure_admin_queue, so reconnect can work well.
> > 
> > So can you explain in detail about the bug?
> First if reconnect failed, quiesce the io queue and admin queue will cause IO pause long time.

Any normal IO can't make progress until reconnect is successful, so this
change won't increase IO pause. This way is exactly what NVMe PCI takes,
see nvme_start_queues() called from nvme_reset_work().

> Second if reconnect failed more than max_reconnects, delete ctrl will hang.

No, delete ctrl won't hang, because 'shutdown' parameter is true in case
of deleting ctrl, which will unquiesce both admin_q and io queues in
nvme_tcp_teardown_io_queues() and nvme_tcp_teardown_admin_queue().


Thanks, 
Ming



From hch at lst.de  Thu Oct 15 03:56:40 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 15 Oct 2020 09:56:40 +0200
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201009231816.1524-1-logang@deltatee.com>
References: <20201009231816.1524-1-logang@deltatee.com>
Message-ID: <20201015075640.GJ14082@lst.de>

On Fri, Oct 09, 2020 at 05:18:16PM -0600, Logan Gunthorpe wrote:
>  static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>  {
> -	int sg_cnt = req->sg_cnt;
>  	struct scatterlist *sg;
>  	int op_flags = 0;
>  	struct bio *bio;
>  	int i, ret;
>  
> +	if (req->sg_cnt > BIO_MAX_PAGES)
> +		return -EINVAL;

Don't you need to handle larger requests as well?  Or at least
limit MDTS?


From hch at infradead.org  Thu Oct 15 04:02:54 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 15 Oct 2020 09:02:54 +0100
Subject: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <20201012162736.65241-1-nmeeramohide@micron.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
Message-ID: <20201015080254.GA31136@infradead.org>

I don't think this belongs into the kernel.  It is a classic case for
infrastructure that should be built in userspace.  If anything is
missing to implement it in userspace with equivalent performance we
need to improve out interfaces, although io_uring should cover pretty
much everything you need.


From hch at lst.de  Thu Oct 15 04:10:43 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 15 Oct 2020 10:10:43 +0200
Subject: [PATCH] nvme-rdma: fix crash when connect rejected
In-Reply-To: <20201012081040.31741-1-lengchao@huawei.com>
References: <20201012081040.31741-1-lengchao@huawei.com>
Message-ID: <20201015081043.GA15299@lst.de>

Applied to nvme-5.10, thanks.


From hch at lst.de  Thu Oct 15 04:10:48 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 15 Oct 2020 10:10:48 +0200
Subject: [PATCH] nvme-rdma: fix crash due to incorrect cqe
In-Reply-To: <20201012085537.2599-1-lengchao@huawei.com>
References: <20201012085537.2599-1-lengchao@huawei.com>
Message-ID: <20201015081048.GB15299@lst.de>

Applied to nvme-5.10, thanks.


From hch at lst.de  Thu Oct 15 04:10:54 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 15 Oct 2020 10:10:54 +0200
Subject: [PATCH v3] nvmet: fix uninitialized work for zero kato
In-Reply-To: <20201015015140.1061936-1-pizhenwei@bytedance.com>
References: <20201015015140.1061936-1-pizhenwei@bytedance.com>
Message-ID: <20201015081054.GC15299@lst.de>

Applied to nvme-5.10, thanks.


From lengchao at huawei.com  Thu Oct 15 06:05:52 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 15 Oct 2020 18:05:52 +0800
Subject: [PATCH] block: re-introduce blk_mq_complete_request_sync
In-Reply-To: <20201015075020.GA1099950@T590>
References: <7a7aca6e-30f5-0754-fb7f-599699b97108@redhat.com>
 <6f2a5ae2-2e6a-0386-691c-baefeecb5478@huawei.com>
 <20201012081306.GB556731@T590>
 <5e05fc3b-ad81-aacc-1f8e-7ff0d1ad58fe@huawei.com>
 <e19073e4-06da-ce3c-519c-ece2c4d942fa@grimberg.me>
 <20201014010813.GA775684@T590> <20201014033434.GC775684@T590>
 <f5870b91-28c5-ea99-59df-cdcc8c482011@huawei.com>
 <20201014095642.GE775684@T590>
 <c9cf7168-d8ce-276f-de01-739199ed4258@huawei.com>
 <20201015075020.GA1099950@T590>
Message-ID: <c8cb03f2-0dce-3c67-7700-526f26111a7e@huawei.com>



On 2020/10/15 15:50, Ming Lei wrote:
> On Thu, Oct 15, 2020 at 02:05:01PM +0800, Chao Leng wrote:
>>
>>
>> On 2020/10/14 17:56, Ming Lei wrote:
>>> On Wed, Oct 14, 2020 at 05:39:12PM +0800, Chao Leng wrote:
>>>>
>>>>
>>>> On 2020/10/14 11:34, Ming Lei wrote:
>>>>> On Wed, Oct 14, 2020 at 09:08:28AM +0800, Ming Lei wrote:
>>>>>> On Tue, Oct 13, 2020 at 03:36:08PM -0700, Sagi Grimberg wrote:
>>>>>>>
>>>>>>>>>> This may just reduce the probability. The concurrency of timeout
>>>>>>>>>> and teardown will cause the same request
>>>>>>>>>> be treated repeatly, this is not we expected.
>>>>>>>>>
>>>>>>>>> That is right, not like SCSI, NVME doesn't apply atomic request
>>>>>>>>> completion, so
>>>>>>>>> request may be completed/freed from both timeout & nvme_cancel_request().
>>>>>>>>>
>>>>>>>>> .teardown_lock still may cover the race with Sagi's patch because
>>>>>>>>> teardown
>>>>>>>>> actually cancels requests in sync style.
>>>>>>>> In extreme scenarios, the request may be already retry success(rq state
>>>>>>>> change to inflight).
>>>>>>>> Timeout processing may wrongly stop the queue and abort the request.
>>>>>>>> teardown_lock serialize the process of timeout and teardown, but do not
>>>>>>>> avoid the race.
>>>>>>>> It might not be safe.
>>>>>>>
>>>>>>> Not sure I understand the scenario you are describing.
>>>>>>>
>>>>>>> what do you mean by "In extreme scenarios, the request may be already retry
>>>>>>> success(rq state change to inflight)"?
>>>>>>>
>>>>>>> What will retry the request? only when the host will reconnect
>>>>>>> the request will be retried.
>>>>>>>
>>>>>>> We can call nvme_sync_queues in the last part of the teardown, but
>>>>>>> I still don't understand the race here.
>>>>>>
>>>>>> Not like SCSI, NVME doesn't complete request atomically, so double
>>>>>> completion/free can be done from both timeout & nvme_cancel_request()(via teardown).
>>>>>>
>>>>>> Given request is completed remotely or asynchronously in the two code paths,
>>>>>> the teardown_lock can't protect the case.
>>>>>
>>>>> Thinking of the issue further, the race shouldn't be between timeout and
>>>>> teardown.
>>>>>
>>>>> Both nvme_cancel_request() and nvme_tcp_complete_timed_out() are called
>>>>> with .teardown_lock, and both check if the request is completed before
>>>>> calling blk_mq_complete_request() which marks the request as COMPLETE state.
>>>>> So the request shouldn't be double-freed in the two code paths.
>>>>>
>>>>> Another possible reason is that between timeout and normal completion(fail
>>>>> fast pending requests after ctrl state is updated to CONNECTING).
>>>>>
>>>>> Yi, can you try the following patch and see if the issue is fixed?
>>>>>
>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>> index d6a3e1487354..fab9220196bd 100644
>>>>> --- a/drivers/nvme/host/tcp.c
>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>> @@ -1886,7 +1886,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
>>>>>     static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
>>>>>     		bool remove)
>>>>>     {
>>>>> -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>>     	blk_mq_quiesce_queue(ctrl->admin_q);
>>>>>     	nvme_tcp_stop_queue(ctrl, 0);
>>>>>     	if (ctrl->admin_tagset) {
>>>>> @@ -1897,15 +1896,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
>>>>>     	if (remove)
>>>>>     		blk_mq_unquiesce_queue(ctrl->admin_q);
>>>>>     	nvme_tcp_destroy_admin_queue(ctrl, remove);
>>>>> -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>>     }
>>>>>     static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>>>>>     		bool remove)
>>>>>     {
>>>>> -	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>>     	if (ctrl->queue_count <= 1)
>>>>> -		goto out;
>>>>> +		return;
>>>>>     	blk_mq_quiesce_queue(ctrl->admin_q);
>>>>>     	nvme_start_freeze(ctrl);
>>>>>     	nvme_stop_queues(ctrl);
>>>>> @@ -1918,8 +1915,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
>>>>>     	if (remove)
>>>>>     		nvme_start_queues(ctrl);
>>>>>     	nvme_tcp_destroy_io_queues(ctrl, remove);
>>>>> -out:
>>>>> -	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>>     }
>>>>>     static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
>>>>> @@ -2030,11 +2025,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
>>>>>     	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
>>>>>     	nvme_stop_keep_alive(ctrl);
>>>>> +
>>>>> +	mutex_lock(&tcp_ctrl->teardown_lock);
>>>>>     	nvme_tcp_teardown_io_queues(ctrl, false);
>>>>> -	/* unquiesce to fail fast pending requests */
>>>>> -	nvme_start_queues(ctrl);
>>>>>     	nvme_tcp_teardown_admin_queue(ctrl, false);
>>>>> -	blk_mq_unquiesce_queue(ctrl->admin_q);
>>>> Delete blk_mq_unquiesce_queue will cause a bug which may cause reconnect failed.
>>>> Delete nvme_start_queues may cause another bug.
>>>
>>> nvme_tcp_setup_ctrl() will re-start io and admin queue, and only .connect_q
>>> and .fabrics_q are required during reconnect.I check the code. Unquiesce the admin queue in nvme_tcp_configure_admin_queue, so reconnect can work well.
>>>
>>> So can you explain in detail about the bug?
>> First if reconnect failed, quiesce the io queue and admin queue will cause IO pause long time.
> 
> Any normal IO can't make progress until reconnect is successful, so this
> change won't increase IO pause. This way is exactly what NVMe PCI takes,
> see nvme_start_queues() called from nvme_reset_work().
now is ok. Now the patch which fix the long pause time is discussing.
> 
>> Second if reconnect failed more than max_reconnects, delete ctrl will hang.
> 
> No, delete ctrl won't hang, because 'shutdown' parameter is true in case
> of deleting ctrl, which will unquiesce both admin_q and io queues in
> nvme_tcp_teardown_io_queues() and nvme_tcp_teardown_admin_queue().
No, now nvme_remove_namespaces is before tear down queues.
tear down queues is in ctrl->ops->delete_ctrl.
static void nvme_do_delete_ctrl(struct nvme_ctrl *ctrl)
{
	dev_info(ctrl->device,
		 "Removing ctrl: NQN \"%s\"\n", ctrl->opts->subsysnqn);

	flush_work(&ctrl->reset_work);
	nvme_stop_ctrl(ctrl);
	nvme_remove_namespaces(ctrl);
	ctrl->ops->delete_ctrl(ctrl);
	nvme_uninit_ctrl(ctrl);
}
> 
> 
> Thanks,
> Ming
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
> .
> 


From kbusch at kernel.org  Thu Oct 15 10:45:48 2020
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 15 Oct 2020 07:45:48 -0700
Subject: [PATCH] nvme: set passthrough command gendisk
In-Reply-To: <20201015074222.GA14082@lst.de>
References: <20201014203845.1400053-1-kbusch@kernel.org>
 <20201015074222.GA14082@lst.de>
Message-ID: <20201015144548.GA1401094@dhcp-10-100-145-180.wdl.wdc.com>

On Thu, Oct 15, 2020 at 09:42:22AM +0200, Christoph Hellwig wrote:
> On Wed, Oct 14, 2020 at 01:38:45PM -0700, Keith Busch wrote:
> > The nvme event traces for IO commands are reporting the wrong QID
> > because we never set the request's gendisk. Set this parameter if the
> > passthrough command is being sent through an IO queue.
> 
> Why do we event need the check for a non-NULL disk in nvme_req_qid?

It's the criteria we use to know if we're tracing an Admin vs IO
command. I suppose we could change the criteria we check rather than set
the disk parameter. Maybe rq->queue->queuedata instead of rq->bd_disk.
Would you prefer that?


From logang at deltatee.com  Thu Oct 15 12:01:30 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu, 15 Oct 2020 10:01:30 -0600
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201015075640.GJ14082@lst.de>
References: <20201009231816.1524-1-logang@deltatee.com>
 <20201015075640.GJ14082@lst.de>
Message-ID: <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>



On 2020-10-15 1:56 a.m., Christoph Hellwig wrote:
> On Fri, Oct 09, 2020 at 05:18:16PM -0600, Logan Gunthorpe wrote:
>>  static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>>  {
>> -	int sg_cnt = req->sg_cnt;
>>  	struct scatterlist *sg;
>>  	int op_flags = 0;
>>  	struct bio *bio;
>>  	int i, ret;
>>  
>> +	if (req->sg_cnt > BIO_MAX_PAGES)
>> +		return -EINVAL;
> 
> Don't you need to handle larger requests as well?  Or at least
> limit MDTS?

No and Yes: there is already code in nvmet_passthru_override_id_ctrl()
to limit MDTS based on max_segments and max_hw_sectors. So any request
that doesn't pass this check should be from a misbehaving client and an
error should be appropriate.

Logan



From dgilbert at interlog.com  Thu Oct 15 13:24:34 2020
From: dgilbert at interlog.com (Douglas Gilbert)
Date: Thu, 15 Oct 2020 13:24:34 -0400
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>
References: <20201009231816.1524-1-logang@deltatee.com>
 <20201015075640.GJ14082@lst.de>
 <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>
Message-ID: <7913172f-0125-e327-562a-aa6b842ab193@interlog.com>

On 2020-10-15 12:01 p.m., Logan Gunthorpe wrote:
> 
> 
> On 2020-10-15 1:56 a.m., Christoph Hellwig wrote:
>> On Fri, Oct 09, 2020 at 05:18:16PM -0600, Logan Gunthorpe wrote:
>>>   static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>>>   {
>>> -	int sg_cnt = req->sg_cnt;
>>>   	struct scatterlist *sg;
>>>   	int op_flags = 0;
>>>   	struct bio *bio;
>>>   	int i, ret;
>>>   
>>> +	if (req->sg_cnt > BIO_MAX_PAGES)
>>> +		return -EINVAL;
>>
>> Don't you need to handle larger requests as well?  Or at least
>> limit MDTS?
> 
> No and Yes: there is already code in nvmet_passthru_override_id_ctrl()
> to limit MDTS based on max_segments and max_hw_sectors. So any request
> that doesn't pass this check should be from a misbehaving client and an
> error should be appropriate.

Running the numbers: with PAGE_SIZE of 4096 bytes and BIO_MAX_PAGES at
256 gives 1 MiB. From memory the block layer won't accept single requests
bigger than 4 MiB (or 8 MiB). Then it is possible that the sgl was
built with sgl_alloc_order(order > 0, chainable=false) in which case
the maximum (unchained) bio carrying size goes up to:
    PAGE_SIZE * (2^order) * 256

I'm using order=3 by default in my driver. So one (unchained) bio will
hold as much (or more) than the block layer will take.

Doug Gilbert




From hch at lst.de  Thu Oct 15 13:59:43 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 15 Oct 2020 19:59:43 +0200
Subject: [PATCH] nvme: set passthrough command gendisk
In-Reply-To: <20201015144548.GA1401094@dhcp-10-100-145-180.wdl.wdc.com>
References: <20201014203845.1400053-1-kbusch@kernel.org>
 <20201015074222.GA14082@lst.de>
 <20201015144548.GA1401094@dhcp-10-100-145-180.wdl.wdc.com>
Message-ID: <20201015175943.GA23361@lst.de>

On Thu, Oct 15, 2020 at 07:45:48AM -0700, Keith Busch wrote:
> On Thu, Oct 15, 2020 at 09:42:22AM +0200, Christoph Hellwig wrote:
> > On Wed, Oct 14, 2020 at 01:38:45PM -0700, Keith Busch wrote:
> > > The nvme event traces for IO commands are reporting the wrong QID
> > > because we never set the request's gendisk. Set this parameter if the
> > > passthrough command is being sent through an IO queue.
> > 
> > Why do we event need the check for a non-NULL disk in nvme_req_qid?
> 
> It's the criteria we use to know if we're tracing an Admin vs IO
> command. I suppose we could change the criteria we check rather than set
> the disk parameter. Maybe rq->queue->queuedata instead of rq->bd_disk.
> Would you prefer that?

Yes, I think that makes much more sense.


From hch at lst.de  Thu Oct 15 14:01:48 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 15 Oct 2020 20:01:48 +0200
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>
References: <20201009231816.1524-1-logang@deltatee.com>
 <20201015075640.GJ14082@lst.de>
 <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>
Message-ID: <20201015180148.GA23377@lst.de>

On Thu, Oct 15, 2020 at 10:01:30AM -0600, Logan Gunthorpe wrote:
> 
> 
> On 2020-10-15 1:56 a.m., Christoph Hellwig wrote:
> > On Fri, Oct 09, 2020 at 05:18:16PM -0600, Logan Gunthorpe wrote:
> >>  static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
> >>  {
> >> -	int sg_cnt = req->sg_cnt;
> >>  	struct scatterlist *sg;
> >>  	int op_flags = 0;
> >>  	struct bio *bio;
> >>  	int i, ret;
> >>  
> >> +	if (req->sg_cnt > BIO_MAX_PAGES)
> >> +		return -EINVAL;
> > 
> > Don't you need to handle larger requests as well?  Or at least
> > limit MDTS?
> 
> No and Yes: there is already code in nvmet_passthru_override_id_ctrl()
> to limit MDTS based on max_segments and max_hw_sectors.

But those are entirely unrelated to the bio size.  BIO_MAX_PAGES is
256, so with 4k pages and assuming none can't be merged that is 1MB,
while max_segments/max_hw_sectors could be something much larger.


From kbusch at kernel.org  Thu Oct 15 14:36:29 2020
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 15 Oct 2020 11:36:29 -0700
Subject: [PATCHv2] nvme: use queuedata for nvme_req_qid
Message-ID: <20201015183629.1424705-1-kbusch@kernel.org>

The request's rq_disk isn't set for passthrough IO commands, so tracing
uses qid 0 for these which incorrectly decodes as an admin command. Use
the request_queue's queuedata instead since that value is always set for
the IO queues, and never set for the admin queue.

Signed-off-by: Keith Busch <kbusch at kernel.org>
---
v1->v2

  Don't bother setting the rq_disk request parameter; use the existing
  queuedata criteria instead

 drivers/nvme/host/nvme.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index e7c88b40f5bb..cc111136a981 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -176,7 +176,7 @@ static inline struct nvme_request *nvme_req(struct request *req)
 
 static inline u16 nvme_req_qid(struct request *req)
 {
-	if (!req->rq_disk)
+	if (!req->q->queuedata)
 		return 0;
 	return blk_mq_unique_tag_to_hwq(blk_mq_unique_tag(req)) + 1;
 }
-- 
2.24.1



From logang at deltatee.com  Thu Oct 15 14:40:52 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu, 15 Oct 2020 12:40:52 -0600
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201015180148.GA23377@lst.de>
References: <20201009231816.1524-1-logang@deltatee.com>
 <20201015075640.GJ14082@lst.de>
 <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>
 <20201015180148.GA23377@lst.de>
Message-ID: <b798e7f6-0afc-551c-f6d9-f7900b400137@deltatee.com>



On 2020-10-15 12:01 p.m., Christoph Hellwig wrote:
> On Thu, Oct 15, 2020 at 10:01:30AM -0600, Logan Gunthorpe wrote:
>>
>>
>> On 2020-10-15 1:56 a.m., Christoph Hellwig wrote:
>>> On Fri, Oct 09, 2020 at 05:18:16PM -0600, Logan Gunthorpe wrote:
>>>>  static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>>>>  {
>>>> -	int sg_cnt = req->sg_cnt;
>>>>  	struct scatterlist *sg;
>>>>  	int op_flags = 0;
>>>>  	struct bio *bio;
>>>>  	int i, ret;
>>>>  
>>>> +	if (req->sg_cnt > BIO_MAX_PAGES)
>>>> +		return -EINVAL;
>>>
>>> Don't you need to handle larger requests as well?  Or at least
>>> limit MDTS?
>>
>> No and Yes: there is already code in nvmet_passthru_override_id_ctrl()
>> to limit MDTS based on max_segments and max_hw_sectors.
> 
> But those are entirely unrelated to the bio size.  BIO_MAX_PAGES is
> 256, so with 4k pages and assuming none can't be merged that is 1MB,
> while max_segments/max_hw_sectors could be something much larger.

Isn't it constrained by max_segments which is set to NVME_MAX_SEGS=127
(for PCI)... less than BIO_MAX_PAGES...

Would the NVME driver even work if max_segments was greater than
BIO_MAX_PAGES? Correct me if I'm wrong, but it looks like
blk_rq_map_sg() will only map one bio within a request. So there has to
be one bio per request by the time it hits nvme_map_data()...

So I'm not really sure how we could even construct a valid passthrough
request in nvmet_passthru_map_sg() that is larger than BIO_MAX_PAGES.

If you want me to send a patch to future proof the MDTS limit with
BIO_MAX_PAGES, I can do that, but it doesn't look like it will have any
effect right now unless big things change.

Logan


From lkp at intel.com  Thu Oct 15 16:37:50 2020
From: lkp at intel.com (kernel test robot)
Date: Fri, 16 Oct 2020 04:37:50 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 f2ac6321ceaa4d8ea39aa4d5279e7f52c9f8a53b
Message-ID: <5f88b31e.USOIUpyFF/NQMOrJ%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: f2ac6321ceaa4d8ea39aa4d5279e7f52c9f8a53b  nvme-rdma: fix crash due to incorrect cqe

elapsed time: 721m

configs tested: 176
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
arm                        shmobile_defconfig
m68k                        m5407c3_defconfig
arm                              zx_defconfig
sh                           se7343_defconfig
arm                           viper_defconfig
arm                          pxa168_defconfig
nios2                            alldefconfig
powerpc                      bamboo_defconfig
powerpc                   currituck_defconfig
mips                         cobalt_defconfig
arm                        trizeps4_defconfig
arm                           sama5_defconfig
sparc                       sparc32_defconfig
arm                        multi_v5_defconfig
mips                            gpr_defconfig
powerpc                 mpc834x_mds_defconfig
powerpc                      pcm030_defconfig
m68k                         apollo_defconfig
ia64                             alldefconfig
mips                        bcm63xx_defconfig
mips                         tb0287_defconfig
sh                   sh7724_generic_defconfig
arm                       netwinder_defconfig
xtensa                              defconfig
mips                            e55_defconfig
arm                         cm_x300_defconfig
powerpc                 mpc8560_ads_defconfig
powerpc                     tqm8555_defconfig
arm                              alldefconfig
mips                        jmr3927_defconfig
arm                  colibri_pxa300_defconfig
arm                            xcep_defconfig
alpha                               defconfig
powerpc                       eiger_defconfig
mips                           ip32_defconfig
powerpc                   motionpro_defconfig
mips                          malta_defconfig
arm                         lpc32xx_defconfig
arm                          simpad_defconfig
riscv                    nommu_k210_defconfig
mips                           ip28_defconfig
i386                             alldefconfig
arc                        vdk_hs38_defconfig
powerpc                      pmac32_defconfig
powerpc                     sbc8548_defconfig
sh                              ul2_defconfig
mips                   sb1250_swarm_defconfig
arc                                 defconfig
powerpc                 linkstation_defconfig
ia64                          tiger_defconfig
mips                    maltaup_xpa_defconfig
arm                        multi_v7_defconfig
mips                     decstation_defconfig
sh                          r7785rp_defconfig
arm                         lubbock_defconfig
arm                        mini2440_defconfig
sh                   rts7751r2dplus_defconfig
powerpc                      mgcoge_defconfig
arm                      tct_hammer_defconfig
arm                        vexpress_defconfig
powerpc                     tqm8540_defconfig
arm                         nhk8815_defconfig
arm                             pxa_defconfig
arm                         bcm2835_defconfig
m68k                         amcore_defconfig
sh                           se7206_defconfig
sh                          r7780mp_defconfig
ia64                                defconfig
nios2                            allyesconfig
ia64                         bigsur_defconfig
powerpc                      ppc44x_defconfig
sparc                            allyesconfig
sparc                       sparc64_defconfig
arm                      pxa255-idp_defconfig
s390                       zfcpdump_defconfig
sh                           se7705_defconfig
powerpc               mpc834x_itxgp_defconfig
arm                          ep93xx_defconfig
arm                        spear3xx_defconfig
powerpc                     mpc5200_defconfig
arm                       imx_v4_v5_defconfig
powerpc                       ebony_defconfig
arm                            pleb_defconfig
sh                   sh7770_generic_defconfig
sh                          lboxre2_defconfig
arm                       omap2plus_defconfig
m68k                            q40_defconfig
powerpc                      cm5200_defconfig
arm                             rpc_defconfig
arm                           tegra_defconfig
ia64                             allmodconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
csky                                defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                               defconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
x86_64               randconfig-a004-20201015
x86_64               randconfig-a002-20201015
x86_64               randconfig-a006-20201015
x86_64               randconfig-a001-20201015
x86_64               randconfig-a005-20201015
x86_64               randconfig-a003-20201015
i386                 randconfig-a005-20201015
i386                 randconfig-a006-20201015
i386                 randconfig-a001-20201015
i386                 randconfig-a003-20201015
i386                 randconfig-a004-20201015
i386                 randconfig-a002-20201015
i386                 randconfig-a005-20201014
i386                 randconfig-a006-20201014
i386                 randconfig-a001-20201014
i386                 randconfig-a003-20201014
i386                 randconfig-a004-20201014
i386                 randconfig-a002-20201014
x86_64               randconfig-a016-20201014
x86_64               randconfig-a012-20201014
x86_64               randconfig-a015-20201014
x86_64               randconfig-a013-20201014
x86_64               randconfig-a014-20201014
x86_64               randconfig-a011-20201014
i386                 randconfig-a016-20201015
i386                 randconfig-a013-20201015
i386                 randconfig-a015-20201015
i386                 randconfig-a011-20201015
i386                 randconfig-a012-20201015
i386                 randconfig-a014-20201015
i386                 randconfig-a016-20201014
i386                 randconfig-a013-20201014
i386                 randconfig-a015-20201014
i386                 randconfig-a011-20201014
i386                 randconfig-a012-20201014
i386                 randconfig-a014-20201014
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a004-20201014
x86_64               randconfig-a002-20201014
x86_64               randconfig-a006-20201014
x86_64               randconfig-a001-20201014
x86_64               randconfig-a005-20201014
x86_64               randconfig-a003-20201014

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From hch at lst.de  Fri Oct 16 09:57:59 2020
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 16 Oct 2020 15:57:59 +0200
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <b798e7f6-0afc-551c-f6d9-f7900b400137@deltatee.com>
References: <20201009231816.1524-1-logang@deltatee.com>
 <20201015075640.GJ14082@lst.de>
 <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>
 <20201015180148.GA23377@lst.de>
 <b798e7f6-0afc-551c-f6d9-f7900b400137@deltatee.com>
Message-ID: <20201016135759.GA4685@lst.de>

On Thu, Oct 15, 2020 at 12:40:52PM -0600, Logan Gunthorpe wrote:
> > But those are entirely unrelated to the bio size.  BIO_MAX_PAGES is
> > 256, so with 4k pages and assuming none can't be merged that is 1MB,
> > while max_segments/max_hw_sectors could be something much larger.
> 
> Isn't it constrained by max_segments which is set to NVME_MAX_SEGS=127
> (for PCI)... less than BIO_MAX_PAGES...

nvmet-passthrough is not limited to work on PCIe controllers.  And even
if it did relying on an implicit limit like that is a time bomb.  I
have work pending for one of the next merge windows to lift that limit
for example, and I would not have thought that nvmet-passthrough
relies on it in any way.

> Would the NVME driver even work if max_segments was greater than
> BIO_MAX_PAGES? Correct me if I'm wrong, but it looks like
> blk_rq_map_sg() will only map one bio within a request. So there has to
> be one bio per request by the time it hits nvme_map_data()...

blk_rq_map_sg maps the bio chain.  Take a look at the for_each_bio
loop in __blk_bios_map_sg.

> If you want me to send a patch to future proof the MDTS limit with
> BIO_MAX_PAGES, I can do that, but it doesn't look like it will have any
> effect right now unless big things change.

Yes, please do.


From hch at lst.de  Fri Oct 16 09:58:36 2020
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 16 Oct 2020 15:58:36 +0200
Subject: [PATCHv2] nvme: use queuedata for nvme_req_qid
In-Reply-To: <20201015183629.1424705-1-kbusch@kernel.org>
References: <20201015183629.1424705-1-kbusch@kernel.org>
Message-ID: <20201016135836.GA4928@lst.de>

Thanks,

applied to nvme-5.10.


From hch at lst.de  Fri Oct 16 10:04:08 2020
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 16 Oct 2020 16:04:08 +0200
Subject: [PATCH] nvme-pci: Disable Write Zeroes on Sandisk Skyhawk
In-Reply-To: <20201013083445.12317-1-kai.heng.feng@canonical.com>
References: <20200723155731.22313-1-kai.heng.feng@canonical.com>
 <20201013083445.12317-1-kai.heng.feng@canonical.com>
Message-ID: <20201016140408.GA5165@lst.de>

Thanks,

applied to nvme-5.10.


From ming.lei at redhat.com  Fri Oct 16 10:28:07 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Fri, 16 Oct 2020 22:28:07 +0800
Subject: [PATCH 0/4] blk-mq/nvme-tcp: fix timed out related races 
Message-ID: <20201016142811.1262214-1-ming.lei@redhat.com>

Hi,

The 1st 2 patches fixes request completion related races.

The 2nd 3 patches fixes/improves nvme-tcp error recovery.

With the 4 patches, nvme/012 can pass on nvme-tcp in Zhang Yi's test
machine.


Ming Lei (4):
  blk-mq: check rq->state explicitly in
    blk_mq_tagset_count_completed_rqs
  blk-mq: think request as completed if it isn't IN_FLIGHT.
  nvme: tcp: fix race between timeout and normal completion
  nvme: tcp: complete non-IO requests atomically

 block/blk-flush.c       |  2 ++
 block/blk-mq-tag.c      |  2 +-
 drivers/nvme/host/tcp.c | 76 +++++++++++++++++++++++++++++------------
 include/linux/blk-mq.h  |  8 ++++-
 4 files changed, 65 insertions(+), 23 deletions(-)

CC: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Yi Zhang <yi.zhang at redhat.com>


-- 
2.25.2



From ming.lei at redhat.com  Fri Oct 16 10:28:08 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Fri, 16 Oct 2020 22:28:08 +0800
Subject: [PATCH 1/4] blk-mq: check rq->state explicitly in
 blk_mq_tagset_count_completed_rqs
In-Reply-To: <20201016142811.1262214-1-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
Message-ID: <20201016142811.1262214-2-ming.lei@redhat.com>

blk_mq_tagset_count_completed_rqs() is called from
blk_mq_tagset_wait_completed_request() for draining requests to be
completed remotely. What we need to do is to see request->state to
switch to non-MQ_RQ_COMPLETE.

So check MQ_RQ_COMPLETE explicitly in blk_mq_tagset_count_completed_rqs().

Meantime mark flush request as IDLE in its .end_io() for aligning to
end normal request because flush request may stay in inflight tags in case
of !elevator, so we need to change its state into IDLE.

Cc: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 block/blk-flush.c  | 2 ++
 block/blk-mq-tag.c | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/block/blk-flush.c b/block/blk-flush.c
index 53abb5c73d99..31a2ae04d3f0 100644
--- a/block/blk-flush.c
+++ b/block/blk-flush.c
@@ -231,6 +231,8 @@ static void flush_end_io(struct request *flush_rq, blk_status_t error)
 		return;
 	}
 
+	WRITE_ONCE(rq->state, MQ_RQ_IDLE);
+
 	if (fq->rq_status != BLK_STS_OK)
 		error = fq->rq_status;
 
diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index 9c92053e704d..10ff8968b93b 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -367,7 +367,7 @@ static bool blk_mq_tagset_count_completed_rqs(struct request *rq,
 {
 	unsigned *count = data;
 
-	if (blk_mq_request_completed(rq))
+	if (blk_mq_rq_state(rq) == MQ_RQ_COMPLETE)
 		(*count)++;
 	return true;
 }
-- 
2.25.2



From ming.lei at redhat.com  Fri Oct 16 10:28:09 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Fri, 16 Oct 2020 22:28:09 +0800
Subject: [PATCH 2/4] blk-mq: think request as completed if it isn't IN_FLIGHT.
In-Reply-To: <20201016142811.1262214-1-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
Message-ID: <20201016142811.1262214-3-ming.lei@redhat.com>

MQ_RQ_COMPLETE is one transient state, because the .complete callback
ends or requeues this request, then the request state is updated to
IDLE.

blk_mq_request_completed() is often used by driver for avoiding
double completion with help of driver's specific sync approach. Such as,
NVMe TCP calls blk_mq_request_completed() in its timeout handler
and abort handler for avoiding double completion. If request's state
is updated to IDLE in either one, another code path may think this
request as not completed, and will complete it one more time. Then
double completion is triggered.

Yi reported[1] that 'refcount_t: underflow; use-after-free' of rq->ref
is triggered in blktests(nvme/012) on one very slow machine.

Fixes this issue by thinking request as completed if its state becomes
not IN_FLIGHT.

Reported-by: Yi Zhang <yi.zhang at redhat.com>
Cc: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 include/linux/blk-mq.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index b23eeca4d677..9a0c1f8ac42d 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -486,9 +486,15 @@ static inline int blk_mq_request_started(struct request *rq)
 	return blk_mq_rq_state(rq) != MQ_RQ_IDLE;
 }
 
+/*
+ * It is often called in abort handler for avoiding double completion,
+ * MQ_RQ_COMPLETE is one transient state because .complete callback
+ * may end or requeue this request, in either way the request is marked
+ * as IDLE. So return true if this request's state become not IN_FLIGHT.
+ */
 static inline int blk_mq_request_completed(struct request *rq)
 {
-	return blk_mq_rq_state(rq) == MQ_RQ_COMPLETE;
+	return blk_mq_rq_state(rq) != MQ_RQ_IN_FLIGHT;
 }
 
 void blk_mq_start_request(struct request *rq);
-- 
2.25.2



From ming.lei at redhat.com  Fri Oct 16 10:28:10 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Fri, 16 Oct 2020 22:28:10 +0800
Subject: [PATCH 3/4] nvme: tcp: fix race between timeout and normal completion
In-Reply-To: <20201016142811.1262214-1-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
Message-ID: <20201016142811.1262214-4-ming.lei@redhat.com>

NVMe TCP timeout handler allows to abort request directly when the
controller isn't in LIVE state. nvme_tcp_error_recovery() updates
controller state as RESETTING, and schedule reset work function. If
new timeout comes before the work function is called, the new timedout
request will be aborted directly, however at that time, the controller
isn't shut down yet, then timeout abort vs. normal completion race
will be triggered.

Fix the race by the following approach:

1) aborting timed out request directly only in case that controller is in
CONNECTING and DELETING state. In the two states, controller has been shutdown,
so it is safe to do so; Also, it is enough to recovery controller in this way,
because we only stop/destroy queues during RESETTING, and cancel all in-flight
requests, no new request is required in RESETTING.

2) delay unquiesce io queues and admin queue until controller is LIVE
because it isn't necessary to start queues during RESETTING. Instead,
this way may risk timeout vs. normal completion race because we need
to abort timed-out request directly during CONNECTING state for setting
up controller.

CC: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Reported-by: Yi Zhang <yi.zhang at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 drivers/nvme/host/tcp.c | 36 ++++++++++++++++++------------------
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index d6a3e1487354..56ac61a90c1b 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1886,7 +1886,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
 	if (ctrl->admin_tagset) {
@@ -1897,15 +1896,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 	if (remove)
 		blk_mq_unquiesce_queue(ctrl->admin_q);
 	nvme_tcp_destroy_admin_queue(ctrl, remove);
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	if (ctrl->queue_count <= 1)
-		goto out;
+		return;
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_start_freeze(ctrl);
 	nvme_stop_queues(ctrl);
@@ -1918,8 +1915,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 	if (remove)
 		nvme_start_queues(ctrl);
 	nvme_tcp_destroy_io_queues(ctrl, remove);
-out:
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
@@ -1938,6 +1933,8 @@ static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
 				ctrl->opts->reconnect_delay * HZ);
 	} else {
 		dev_info(ctrl->device, "Removing controller...\n");
+		nvme_start_queues(ctrl);
+		blk_mq_unquiesce_queue(ctrl->admin_q);
 		nvme_delete_ctrl(ctrl);
 	}
 }
@@ -2030,11 +2027,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
 	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
 
 	nvme_stop_keep_alive(ctrl);
+
+	mutex_lock(&tcp_ctrl->teardown_lock);
 	nvme_tcp_teardown_io_queues(ctrl, false);
-	/* unquiesce to fail fast pending requests */
-	nvme_start_queues(ctrl);
 	nvme_tcp_teardown_admin_queue(ctrl, false);
-	blk_mq_unquiesce_queue(ctrl->admin_q);
+	mutex_unlock(&tcp_ctrl->teardown_lock);
 
 	if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_CONNECTING)) {
 		/* state change failure is ok if we started ctrl delete */
@@ -2051,6 +2048,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
 	cancel_work_sync(&to_tcp_ctrl(ctrl)->err_work);
 	cancel_delayed_work_sync(&to_tcp_ctrl(ctrl)->connect_work);
 
+	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_teardown_io_queues(ctrl, shutdown);
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	if (shutdown)
@@ -2058,6 +2056,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
 	else
 		nvme_disable_ctrl(ctrl);
 	nvme_tcp_teardown_admin_queue(ctrl, shutdown);
+	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_delete_ctrl(struct nvme_ctrl *ctrl)
@@ -2192,19 +2191,20 @@ nvme_tcp_timeout(struct request *rq, bool reserved)
 		"queue %d: timeout request %#x type %d\n",
 		nvme_tcp_queue_id(req->queue), rq->tag, pdu->hdr.type);
 
-	if (ctrl->state != NVME_CTRL_LIVE) {
+	/*
+	 * During CONNECTING or DELETING, the controller has been shutdown,
+	 * so it is safe to abort the request directly, otherwise timeout
+	 * vs. normal completion will be triggered.
+	 */
+	if (ctrl->state == NVME_CTRL_CONNECTING ||
+			ctrl->state == NVME_CTRL_DELETING ||
+			ctrl->state == NVME_CTRL_DELETING_NOIO) {
 		/*
-		 * If we are resetting, connecting or deleting we should
-		 * complete immediately because we may block controller
-		 * teardown or setup sequence
+		 * If we are connecting we should complete immediately because
+		 * we may block controller setup sequence
 		 * - ctrl disable/shutdown fabrics requests
 		 * - connect requests
 		 * - initialization admin requests
-		 * - I/O requests that entered after unquiescing and
-		 *   the controller stopped responding
-		 *
-		 * All other requests should be cancelled by the error
-		 * recovery work, so it's fine that we fail it here.
 		 */
 		nvme_tcp_complete_timed_out(rq);
 		return BLK_EH_DONE;
-- 
2.25.2



From ming.lei at redhat.com  Fri Oct 16 10:28:11 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Fri, 16 Oct 2020 22:28:11 +0800
Subject: [PATCH 4/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <20201016142811.1262214-1-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
Message-ID: <20201016142811.1262214-5-ming.lei@redhat.com>

During controller's CONNECTING state, admin/fabric/connect requests
are submitted for recovery, and we allow to abort this request directly
in time out handler for not blocking the setup step.

So timout vs. normal completion race may be triggered on these requests.

Add atomic completion for requests from connect/fabric/admin queue for
avoiding the race.

CC: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Reported-by: Yi Zhang <yi.zhang at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 37 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 56ac61a90c1b..654061abdc5a 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -30,6 +30,8 @@ static int so_priority;
 module_param(so_priority, int, 0644);
 MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
 
+#define REQ_STATE_COMPLETE     0
+
 enum nvme_tcp_send_state {
 	NVME_TCP_SEND_CMD_PDU = 0,
 	NVME_TCP_SEND_H2C_PDU,
@@ -56,6 +58,8 @@ struct nvme_tcp_request {
 	size_t			offset;
 	size_t			data_sent;
 	enum nvme_tcp_send_state state;
+
+	unsigned long		comp_state;
 };
 
 enum nvme_tcp_queue_flags {
@@ -469,6 +473,33 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
 	queue_work(nvme_reset_wq, &to_tcp_ctrl(ctrl)->err_work);
 }
 
+/*
+ * requests originated from admin, fabrics and connect_q have to be
+ * completed atomically because we don't cover the race between timeout
+ * and normal completion for these queues.
+ */
+static inline bool nvme_tcp_need_atomic_complete(struct request *rq)
+{
+	return !rq->rq_disk;
+}
+
+static inline void nvme_tcp_clear_rq_complete(struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+
+	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
+		clear_bit(REQ_STATE_COMPLETE, &req->comp_state);
+}
+
+static inline bool nvme_tcp_mark_rq_complete(struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+
+	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
+		return !test_and_set_bit(REQ_STATE_COMPLETE, &req->comp_state);
+	return true;
+}
+
 static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		struct nvme_completion *cqe)
 {
@@ -483,7 +514,8 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		return -EINVAL;
 	}
 
-	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
+	if (nvme_tcp_mark_rq_complete(rq) &&
+			!nvme_try_complete_req(rq, cqe->status, cqe->result))
 		nvme_complete_rq(rq);
 	queue->nr_cqe++;
 
@@ -674,7 +706,8 @@ static inline void nvme_tcp_end_request(struct request *rq, u16 status)
 {
 	union nvme_result res = {};
 
-	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
+	if (nvme_tcp_mark_rq_complete(rq) &&
+			!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
 		nvme_complete_rq(rq);
 }
 
@@ -2173,7 +2206,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	/* fence other contexts that may complete the command */
 	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
-	if (!blk_mq_request_completed(rq)) {
+	if (nvme_tcp_mark_rq_complete(rq) && !blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
@@ -2315,6 +2348,7 @@ static blk_status_t nvme_tcp_queue_rq(struct blk_mq_hw_ctx *hctx,
 	if (unlikely(ret))
 		return ret;
 
+	nvme_tcp_clear_rq_complete(rq);
 	blk_mq_start_request(rq);
 
 	nvme_tcp_queue_request(req, true, bd->last);
-- 
2.25.2



From logang at deltatee.com  Fri Oct 16 12:49:04 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Fri, 16 Oct 2020 10:49:04 -0600
Subject: [PATCH] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201016135759.GA4685@lst.de>
References: <20201009231816.1524-1-logang@deltatee.com>
 <20201015075640.GJ14082@lst.de>
 <bc51aadc-eee3-cf52-0623-b3f82e1958f4@deltatee.com>
 <20201015180148.GA23377@lst.de>
 <b798e7f6-0afc-551c-f6d9-f7900b400137@deltatee.com>
 <20201016135759.GA4685@lst.de>
Message-ID: <9cb1b799-63a4-a597-896a-72cd5589d99c@deltatee.com>



On 2020-10-16 7:57 a.m., Christoph Hellwig wrote:
> On Thu, Oct 15, 2020 at 12:40:52PM -0600, Logan Gunthorpe wrote:
> blk_rq_map_sg maps the bio chain.  Take a look at the for_each_bio
> loop in __blk_bios_map_sg.

Ah, yes, ok... I originally had a loop that created a chain of bios but
used bio_add_page() instead of bio_add_pc_page(). This was combined with
a separate check for max_segments and max_hw_sectors. When I switched to
bio_add_pc_page() to drop the separate check, it no longer made sense to
do the chain because there was no way to tell if bio_add_pc_page()
failed because of bio_full() or because it exceeded
max_segments/hw_sectors. If the bio was full, we'd want to create the
chain, but if it exceeded the maximums we'd want to error out...

>> If you want me to send a patch to future proof the MDTS limit with
>> BIO_MAX_PAGES, I can do that, but it doesn't look like it will have any
>> effect right now unless big things change.
> 
> Yes, please do.

Will do.




From james.smart at broadcom.com  Fri Oct 16 17:06:04 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:06:04 -0700
Subject: [PATCH] nvme-fc: fix error loop in create_hw_io_queues
Message-ID: <20201016210604.48542-1-james.smart@broadcom.com>

The loop that backs out of hw io queue creation continues through index
0, which corresponds to the admin queue as well.

Fix the loop so it only proceeds through indexes 1..n which correspond to
io queues.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index e2e09e25c056..f002522146e2 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2314,7 +2314,7 @@ nvme_fc_create_hw_io_queues(struct nvme_fc_ctrl *ctrl, u16 qsize)
 	return 0;
 
 delete_queues:
-	for (; i >= 0; i--)
+	for (; i > 0; i--)
 		__nvme_fc_delete_hw_queue(ctrl, &ctrl->queues[i], i);
 	return ret;
 }
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/a83f342f/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:06:27 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:06:27 -0700
Subject: [PATCH] nvme-fc: fix io timeout to abort io
Message-ID: <20201016210627.48595-1-james.smart@broadcom.com>

Currently, an io timeout unconditionally invokes nvme_fc_error_recovery()
which checks for LIVE or CONNECTING state. If live, the routine resets
the controller which initiates a reconnect - which is valid. If CONNECTING,
err_work is scheduled. Err_work then calls the terminate_io routine, which
also checks for CONNECTING and noops any further action on outstanding io.
The result is nothing happened to the timed out io. As such, if the
command was dropped on the wire, it will never timeout/complete, and the
connect process will hang.

Change the behavior of the io timeout routine to unconditionally abort the
io. Io completion handling will note that an io failed due to an abort and
will terminate the connection/association as needed. If the abort was
unable to happen, continue with a call to nvme_fc_error_recovery(). To
ensure something different happens in nvme_fc_error_recovery() rework it
so at it will abort all ios on the association to force a failure.

As io aborts now may occur outside of delete_association, counting for
completion must be wary and only count those aborted during
delete_association when TERMIO is set on the controller.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 108 ++++++++++++++++++++++++++---------------
 1 file changed, 69 insertions(+), 39 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index f002522146e2..c81ebe94d8fe 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -1837,8 +1837,10 @@ __nvme_fc_abort_op(struct nvme_fc_ctrl *ctrl, struct nvme_fc_fcp_op *op)
 	opstate = atomic_xchg(&op->state, FCPOP_STATE_ABORTED);
 	if (opstate != FCPOP_STATE_ACTIVE)
 		atomic_set(&op->state, opstate);
-	else if (test_bit(FCCTRL_TERMIO, &ctrl->flags))
+	else if (test_bit(FCCTRL_TERMIO, &ctrl->flags)) {
+		op->flags |= FCOP_FLAGS_TERMIO;
 		ctrl->iocnt++;
+	}
 	spin_unlock_irqrestore(&ctrl->lock, flags);
 
 	if (opstate != FCPOP_STATE_ACTIVE)
@@ -1874,7 +1876,8 @@ __nvme_fc_fcpop_chk_teardowns(struct nvme_fc_ctrl *ctrl,
 
 	if (opstate == FCPOP_STATE_ABORTED) {
 		spin_lock_irqsave(&ctrl->lock, flags);
-		if (test_bit(FCCTRL_TERMIO, &ctrl->flags)) {
+		if (test_bit(FCCTRL_TERMIO, &ctrl->flags) &&
+		    op->flags & FCOP_FLAGS_TERMIO) {
 			if (!--ctrl->iocnt)
 				wake_up(&ctrl->ioabort_wait);
 		}
@@ -2446,15 +2449,20 @@ nvme_fc_timeout(struct request *rq, bool reserved)
 {
 	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(rq);
 	struct nvme_fc_ctrl *ctrl = op->ctrl;
+	struct nvme_fc_cmd_iu *cmdiu = &op->cmd_iu;
+	struct nvme_command *sqe = &cmdiu->sqe;
 
 	/*
-	 * we can't individually ABTS an io without affecting the queue,
-	 * thus killing the queue, and thus the association.
-	 * So resolve by performing a controller reset, which will stop
-	 * the host/io stack, terminate the association on the link,
-	 * and recreate an association on the link.
+	 * Attempt to abort the offending command. Command completion
+	 * will detect the aborted io and will fail the connection.
 	 */
-	nvme_fc_error_recovery(ctrl, "io timeout error");
+	dev_info(ctrl->ctrl.device,
+		"NVME-FC{%d.%d}: io timeout: opcode %d fctype %d w10/11: "
+		"x%08x/x%08x\n",
+		ctrl->cnum, op->queue->qnum, sqe->common.opcode,
+		sqe->connect.fctype, sqe->common.cdw10, sqe->common.cdw11);
+	if (__nvme_fc_abort_op(ctrl, op))
+		nvme_fc_error_recovery(ctrl, "io timeout abort failed");
 
 	/*
 	 * the io abort has been initiated. Have the reset timer
@@ -2726,6 +2734,7 @@ nvme_fc_complete_rq(struct request *rq)
 	struct nvme_fc_ctrl *ctrl = op->ctrl;
 
 	atomic_set(&op->state, FCPOP_STATE_IDLE);
+	op->flags &= ~FCOP_FLAGS_TERMIO;
 
 	nvme_fc_unmap_data(ctrl, rq, op);
 	nvme_complete_rq(rq);
@@ -3090,26 +3099,19 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 	return ret;
 }
 
+
 /*
- * This routine stops operation of the controller on the host side.
- * On the host os stack side: Admin and IO queues are stopped,
- *   outstanding ios on them terminated via FC ABTS.
- * On the link side: the association is terminated.
+ * This routine runs through all outstanding commands on the association
+ * and aborts them.  This routine is typically be called by the
+ * delete_association routine. It is also called due to an error during
+ * reconnect. In that scenario, it is most likely a command that initializes
+ * the controller, including fabric Connect commands on io queues, that
+ * may have timed out or failed thus the io must be killed for the connect
+ * thread to see the error.
  */
 static void
-nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl)
+__nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
 {
-	struct nvmefc_ls_rcv_op *disls = NULL;
-	unsigned long flags;
-
-	if (!test_and_clear_bit(ASSOC_ACTIVE, &ctrl->flags))
-		return;
-
-	spin_lock_irqsave(&ctrl->lock, flags);
-	set_bit(FCCTRL_TERMIO, &ctrl->flags);
-	ctrl->iocnt = 0;
-	spin_unlock_irqrestore(&ctrl->lock, flags);
-
 	/*
 	 * If io queues are present, stop them and terminate all outstanding
 	 * ios on them. As FC allocates FC exchange for each io, the
@@ -3127,6 +3129,8 @@ nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl)
 		blk_mq_tagset_busy_iter(&ctrl->tag_set,
 				nvme_fc_terminate_exchange, &ctrl->ctrl);
 		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
+		if (start_queues)
+			nvme_start_queues(&ctrl->ctrl);
 	}
 
 	/*
@@ -3143,13 +3147,34 @@ nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl)
 
 	/*
 	 * clean up the admin queue. Same thing as above.
-	 * use blk_mq_tagset_busy_itr() and the transport routine to
-	 * terminate the exchanges.
 	 */
 	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
 	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
 				nvme_fc_terminate_exchange, &ctrl->ctrl);
 	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
+}
+
+/*
+ * This routine stops operation of the controller on the host side.
+ * On the host os stack side: Admin and IO queues are stopped,
+ *   outstanding ios on them terminated via FC ABTS.
+ * On the link side: the association is terminated.
+ */
+static void
+nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl)
+{
+	struct nvmefc_ls_rcv_op *disls = NULL;
+	unsigned long flags;
+
+	if (!test_and_clear_bit(ASSOC_ACTIVE, &ctrl->flags))
+		return;
+
+	spin_lock_irqsave(&ctrl->lock, flags);
+	set_bit(FCCTRL_TERMIO, &ctrl->flags);
+	ctrl->iocnt = 0;
+	spin_unlock_irqrestore(&ctrl->lock, flags);
+
+	__nvme_fc_abort_outstanding_ios(ctrl, false);
 
 	/* kill the aens as they are a separate path */
 	nvme_fc_abort_aen_ops(ctrl);
@@ -3263,22 +3288,27 @@ static void
 __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
 {
 	/*
-	 * if state is connecting - the error occurred as part of a
-	 * reconnect attempt. The create_association error paths will
-	 * clean up any outstanding io.
-	 *
-	 * if it's a different state - ensure all pending io is
-	 * terminated. Given this can delay while waiting for the
-	 * aborted io to return, we recheck adapter state below
-	 * before changing state.
+	 * if state is CONNECTING - the error occurred as part of a
+	 * reconnect attempt. Abort any ios on the association and
+	 * let the create_association error paths resolve things.
 	 */
-	if (ctrl->ctrl.state != NVME_CTRL_CONNECTING) {
-		nvme_stop_keep_alive(&ctrl->ctrl);
-
-		/* will block will waiting for io to terminate */
-		nvme_fc_delete_association(ctrl);
+	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
+		__nvme_fc_abort_outstanding_ios(ctrl, true);
+		return;
 	}
 
+	/*
+	 * For any other state, kill the association. As this routine
+	 * is a common io abort routine for resetting and such, after
+	 * the association is terminated, ensure that the state is set
+	 * to CONNECTING.
+	 */
+
+	nvme_stop_keep_alive(&ctrl->ctrl);
+
+	/* will block will waiting for io to terminate */
+	nvme_fc_delete_association(ctrl);
+
 	if (ctrl->ctrl.state != NVME_CTRL_CONNECTING &&
 	    !nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
 		dev_err(ctrl->ctrl.device,
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/4549c508/attachment-0001.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:17:24 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:17:24 -0700
Subject: [PATCH] nvme-fc: wait for queues to freeze before calling
 update_hr_hw_queues
Message-ID: <20201016211724.48707-1-james.smart@broadcom.com>

On reconnect, the code currently does not freeze the controller before
possibly updating the number hw queues for the controller.

Add the freeze before updating the number of hw queues. Note: the queues
are already started and remain started through the reconnect.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index c81ebe94d8fe..7067aaf50bf7 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2885,11 +2885,14 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)
 	if (ret)
 		goto out_delete_hw_queues;
 
-	if (prior_ioq_cnt != nr_io_queues)
+	if (prior_ioq_cnt != nr_io_queues) {
 		dev_info(ctrl->ctrl.device,
 			"reconnect: revising io queue count from %d to %d\n",
 			prior_ioq_cnt, nr_io_queues);
-	blk_mq_update_nr_hw_queues(&ctrl->tag_set, nr_io_queues);
+		nvme_wait_freeze(&ctrl->ctrl);
+		blk_mq_update_nr_hw_queues(&ctrl->tag_set, nr_io_queues);
+		nvme_unfreeze(&ctrl->ctrl);
+	}
 
 	return 0;
 
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/862de498/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:27:25 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:27:25 -0700
Subject: [PATCH 0/4] nvme-fc: clean up error recovery implementation
Message-ID: <20201016212729.49138-1-james.smart@broadcom.com>

After reviewing the error_recovery paths to address the abort on the
io to cover timeouts on connecting ios, it became clear that some of the
thing in the way it was coded were unnecessary.

This patch set cleans up those items and refactors. It then addresses the
failure case of io errors during reconnect which wants to take the create
failure cases, but there is no status back from the core routines to
trigger that path.

James Smart (4):
  nvme-fc: remove err_work work item
  nvme-fc: eliminate terminate_io use by nvme_fc_error_recovery
  nvme-fc: remove nvme_fc_terminate_io()
  nvme_fc: track error_recovery while connecting

 drivers/nvme/host/fc.c | 270 +++++++++++++++++------------------------
 1 file changed, 109 insertions(+), 161 deletions(-)

-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/c843f183/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:27:26 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:27:26 -0700
Subject: [PATCH 1/4] nvme-fc: remove err_work work item
In-Reply-To: <20201016212729.49138-1-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
Message-ID: <20201016212729.49138-2-james.smart@broadcom.com>

err_work was created to handle errors (mainly io timeout) while in
CONNECTING state. The flag for err_work_active is also unneeded.

Remove err_work_active and err_work.  The actions to abort ios are moved
inline to nvme_error_recovery().

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 40 ++++++++++------------------------------
 1 file changed, 10 insertions(+), 30 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 7067aaf50bf7..06fb208ab350 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -153,7 +153,6 @@ struct nvme_fc_ctrl {
 	u32			cnum;
 
 	bool			ioq_live;
-	atomic_t		err_work_active;
 	u64			association_id;
 	struct nvmefc_ls_rcv_op	*rcv_disconn;
 
@@ -163,7 +162,6 @@ struct nvme_fc_ctrl {
 	struct blk_mq_tag_set	tag_set;
 
 	struct delayed_work	connect_work;
-	struct work_struct	err_work;
 
 	struct kref		ref;
 	unsigned long		flags;
@@ -2410,11 +2408,11 @@ nvme_fc_nvme_ctrl_freed(struct nvme_ctrl *nctrl)
 	nvme_fc_ctrl_put(ctrl);
 }
 
+static void __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl);
+
 static void
 nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
 {
-	int active;
-
 	/*
 	 * if an error (io timeout, etc) while (re)connecting,
 	 * it's an error on creating the new association.
@@ -2423,11 +2421,14 @@ nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
 	 * ios hitting this path before things are cleaned up.
 	 */
 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
-		active = atomic_xchg(&ctrl->err_work_active, 1);
-		if (!active && !queue_work(nvme_fc_wq, &ctrl->err_work)) {
-			atomic_set(&ctrl->err_work_active, 0);
-			WARN_ON(1);
-		}
+		__nvme_fc_terminate_io(ctrl);
+
+		/*
+		 * Rescheduling the connection after recovering
+		 * from the io error is left to the reconnect work
+		 * item, which is what should have stalled waiting on
+		 * the io that had the error that scheduled this work.
+		 */
 		return;
 	}
 
@@ -3233,7 +3234,6 @@ nvme_fc_delete_ctrl(struct nvme_ctrl *nctrl)
 {
 	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
 
-	cancel_work_sync(&ctrl->err_work);
 	cancel_delayed_work_sync(&ctrl->connect_work);
 	/*
 	 * kill the association on the link side.  this will block
@@ -3343,23 +3343,6 @@ nvme_fc_reset_ctrl_work(struct work_struct *work)
 			ctrl->cnum);
 }
 
-static void
-nvme_fc_connect_err_work(struct work_struct *work)
-{
-	struct nvme_fc_ctrl *ctrl =
-			container_of(work, struct nvme_fc_ctrl, err_work);
-
-	__nvme_fc_terminate_io(ctrl);
-
-	atomic_set(&ctrl->err_work_active, 0);
-
-	/*
-	 * Rescheduling the connection after recovering
-	 * from the io error is left to the reconnect work
-	 * item, which is what should have stalled waiting on
-	 * the io that had the error that scheduled this work.
-	 */
-}
 
 static const struct nvme_ctrl_ops nvme_fc_ctrl_ops = {
 	.name			= "fc",
@@ -3474,7 +3457,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 	ctrl->dev = lport->dev;
 	ctrl->cnum = idx;
 	ctrl->ioq_live = false;
-	atomic_set(&ctrl->err_work_active, 0);
 	init_waitqueue_head(&ctrl->ioabort_wait);
 
 	get_device(ctrl->dev);
@@ -3482,7 +3464,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 
 	INIT_WORK(&ctrl->ctrl.reset_work, nvme_fc_reset_ctrl_work);
 	INIT_DELAYED_WORK(&ctrl->connect_work, nvme_fc_connect_ctrl_work);
-	INIT_WORK(&ctrl->err_work, nvme_fc_connect_err_work);
 	spin_lock_init(&ctrl->lock);
 
 	/* io queue count */
@@ -3575,7 +3556,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 fail_ctrl:
 	nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING);
 	cancel_work_sync(&ctrl->ctrl.reset_work);
-	cancel_work_sync(&ctrl->err_work);
 	cancel_delayed_work_sync(&ctrl->connect_work);
 
 	ctrl->ctrl.opts = NULL;
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/6ca655a0/attachment-0001.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:27:27 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:27:27 -0700
Subject: [PATCH 2/4] nvme-fc: eliminate terminate_io use by
 nvme_fc_error_recovery
In-Reply-To: <20201016212729.49138-1-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
Message-ID: <20201016212729.49138-3-james.smart@broadcom.com>

nvme_fc_error_recovery() special cases handling when in CONNECTING state
and calls __nvme_fc_terminate_io(). __nvme_fc_terminate_io() itself
special cases CONNECTING state and calls the routine to abort outstanding
ios.

Simplify the sequence by putting the call to abort outstanding ios directly
in nvme_fc_error_recovery.

Move the location of __nvme_fc_abort_outstanding_ios(), and
nvme_fc_terminate_exchange() which is called by it, to avoid adding
function prototypes for nvme_fc_error_recovery().

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 185 ++++++++++++++++++-----------------------
 1 file changed, 83 insertions(+), 102 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 06fb208ab350..d65a4a9f4808 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2408,27 +2408,96 @@ nvme_fc_nvme_ctrl_freed(struct nvme_ctrl *nctrl)
 	nvme_fc_ctrl_put(ctrl);
 }
 
-static void __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl);
+/*
+ * This routine is used by the transport when it needs to find active
+ * io on a queue that is to be terminated. The transport uses
+ * blk_mq_tagset_busy_itr() to find the busy requests, which then invoke
+ * this routine to kill them on a 1 by 1 basis.
+ *
+ * As FC allocates FC exchange for each io, the transport must contact
+ * the LLDD to terminate the exchange, thus releasing the FC exchange.
+ * After terminating the exchange the LLDD will call the transport's
+ * normal io done path for the request, but it will have an aborted
+ * status. The done path will return the io request back to the block
+ * layer with an error status.
+ */
+static bool
+nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
+{
+	struct nvme_ctrl *nctrl = data;
+	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
+	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
+
+	__nvme_fc_abort_op(ctrl, op);
+	return true;
+}
+
+/*
+ * This routine runs through all outstanding commands on the association
+ * and aborts them.  This routine is typically be called by the
+ * delete_association routine. It is also called due to an error during
+ * reconnect. In that scenario, it is most likely a command that initializes
+ * the controller, including fabric Connect commands on io queues, that
+ * may have timed out or failed thus the io must be killed for the connect
+ * thread to see the error.
+ */
+static void
+__nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
+{
+	/*
+	 * If io queues are present, stop them and terminate all outstanding
+	 * ios on them. As FC allocates FC exchange for each io, the
+	 * transport must contact the LLDD to terminate the exchange,
+	 * thus releasing the FC exchange. We use blk_mq_tagset_busy_itr()
+	 * to tell us what io's are busy and invoke a transport routine
+	 * to kill them with the LLDD.  After terminating the exchange
+	 * the LLDD will call the transport's normal io done path, but it
+	 * will have an aborted status. The done path will return the
+	 * io requests back to the block layer as part of normal completions
+	 * (but with error status).
+	 */
+	if (ctrl->ctrl.queue_count > 1) {
+		nvme_stop_queues(&ctrl->ctrl);
+		blk_mq_tagset_busy_iter(&ctrl->tag_set,
+				nvme_fc_terminate_exchange, &ctrl->ctrl);
+		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
+		if (start_queues)
+			nvme_start_queues(&ctrl->ctrl);
+	}
+
+	/*
+	 * Other transports, which don't have link-level contexts bound
+	 * to sqe's, would try to gracefully shutdown the controller by
+	 * writing the registers for shutdown and polling (call
+	 * nvme_shutdown_ctrl()). Given a bunch of i/o was potentially
+	 * just aborted and we will wait on those contexts, and given
+	 * there was no indication of how live the controlelr is on the
+	 * link, don't send more io to create more contexts for the
+	 * shutdown. Let the controller fail via keepalive failure if
+	 * its still present.
+	 */
+
+	/*
+	 * clean up the admin queue. Same thing as above.
+	 */
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
+				nvme_fc_terminate_exchange, &ctrl->ctrl);
+	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
+}
 
 static void
 nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
 {
 	/*
-	 * if an error (io timeout, etc) while (re)connecting,
-	 * it's an error on creating the new association.
-	 * Start the error recovery thread if it hasn't already
-	 * been started. It is expected there could be multiple
-	 * ios hitting this path before things are cleaned up.
+	 * if an error (io timeout, etc) while (re)connecting, the remote
+	 * port requested terminating of the association (disconnect_ls)
+	 * or an error (timeout or abort) occurred on an io while creating
+	 * the controller.  Abort any ios on the association and let the
+	 * create_association error path resolve things.
 	 */
 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
-		__nvme_fc_terminate_io(ctrl);
-
-		/*
-		 * Rescheduling the connection after recovering
-		 * from the io error is left to the reconnect work
-		 * item, which is what should have stalled waiting on
-		 * the io that had the error that scheduled this work.
-		 */
+		__nvme_fc_abort_outstanding_ios(ctrl, true);
 		return;
 	}
 
@@ -2742,30 +2811,6 @@ nvme_fc_complete_rq(struct request *rq)
 	nvme_fc_ctrl_put(ctrl);
 }
 
-/*
- * This routine is used by the transport when it needs to find active
- * io on a queue that is to be terminated. The transport uses
- * blk_mq_tagset_busy_itr() to find the busy requests, which then invoke
- * this routine to kill them on a 1 by 1 basis.
- *
- * As FC allocates FC exchange for each io, the transport must contact
- * the LLDD to terminate the exchange, thus releasing the FC exchange.
- * After terminating the exchange the LLDD will call the transport's
- * normal io done path for the request, but it will have an aborted
- * status. The done path will return the io request back to the block
- * layer with an error status.
- */
-static bool
-nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
-{
-	struct nvme_ctrl *nctrl = data;
-	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
-	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
-
-	__nvme_fc_abort_op(ctrl, op);
-	return true;
-}
-
 
 static const struct blk_mq_ops nvme_fc_mq_ops = {
 	.queue_rq	= nvme_fc_queue_rq,
@@ -3104,60 +3149,6 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 }
 
 
-/*
- * This routine runs through all outstanding commands on the association
- * and aborts them.  This routine is typically be called by the
- * delete_association routine. It is also called due to an error during
- * reconnect. In that scenario, it is most likely a command that initializes
- * the controller, including fabric Connect commands on io queues, that
- * may have timed out or failed thus the io must be killed for the connect
- * thread to see the error.
- */
-static void
-__nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
-{
-	/*
-	 * If io queues are present, stop them and terminate all outstanding
-	 * ios on them. As FC allocates FC exchange for each io, the
-	 * transport must contact the LLDD to terminate the exchange,
-	 * thus releasing the FC exchange. We use blk_mq_tagset_busy_itr()
-	 * to tell us what io's are busy and invoke a transport routine
-	 * to kill them with the LLDD.  After terminating the exchange
-	 * the LLDD will call the transport's normal io done path, but it
-	 * will have an aborted status. The done path will return the
-	 * io requests back to the block layer as part of normal completions
-	 * (but with error status).
-	 */
-	if (ctrl->ctrl.queue_count > 1) {
-		nvme_stop_queues(&ctrl->ctrl);
-		blk_mq_tagset_busy_iter(&ctrl->tag_set,
-				nvme_fc_terminate_exchange, &ctrl->ctrl);
-		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
-		if (start_queues)
-			nvme_start_queues(&ctrl->ctrl);
-	}
-
-	/*
-	 * Other transports, which don't have link-level contexts bound
-	 * to sqe's, would try to gracefully shutdown the controller by
-	 * writing the registers for shutdown and polling (call
-	 * nvme_shutdown_ctrl()). Given a bunch of i/o was potentially
-	 * just aborted and we will wait on those contexts, and given
-	 * there was no indication of how live the controlelr is on the
-	 * link, don't send more io to create more contexts for the
-	 * shutdown. Let the controller fail via keepalive failure if
-	 * its still present.
-	 */
-
-	/*
-	 * clean up the admin queue. Same thing as above.
-	 */
-	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
-	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
-				nvme_fc_terminate_exchange, &ctrl->ctrl);
-	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
-}
-
 /*
  * This routine stops operation of the controller on the host side.
  * On the host os stack side: Admin and IO queues are stopped,
@@ -3290,16 +3281,6 @@ nvme_fc_reconnect_or_delete(struct nvme_fc_ctrl *ctrl, int status)
 static void
 __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
 {
-	/*
-	 * if state is CONNECTING - the error occurred as part of a
-	 * reconnect attempt. Abort any ios on the association and
-	 * let the create_association error paths resolve things.
-	 */
-	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
-		__nvme_fc_abort_outstanding_ios(ctrl, true);
-		return;
-	}
-
 	/*
 	 * For any other state, kill the association. As this routine
 	 * is a common io abort routine for resetting and such, after
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/3974cf91/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:27:28 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:27:28 -0700
Subject: [PATCH 3/4] nvme-fc: remove nvme_fc_terminate_io()
In-Reply-To: <20201016212729.49138-1-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
Message-ID: <20201016212729.49138-4-james.smart@broadcom.com>

__nvme_fc_terminate_io() is now called by only 1 place, in reset_work.
Consoldate and move the functionality of terminate_io into reset_work.

In reset_work, rather than calling the create_association directly,
schedule the connect work element to do its thing. After scheduling, flush
the connect work element to continue with semantic of not returning until
connect has been attempted at least once.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 49 ++++++++++++++----------------------------
 1 file changed, 16 insertions(+), 33 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index d65a4a9f4808..b2f9b3752df7 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3279,49 +3279,32 @@ nvme_fc_reconnect_or_delete(struct nvme_fc_ctrl *ctrl, int status)
 }
 
 static void
-__nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
+nvme_fc_reset_ctrl_work(struct work_struct *work)
 {
-	/*
-	 * For any other state, kill the association. As this routine
-	 * is a common io abort routine for resetting and such, after
-	 * the association is terminated, ensure that the state is set
-	 * to CONNECTING.
-	 */
+	struct nvme_fc_ctrl *ctrl =
+		container_of(work, struct nvme_fc_ctrl, ctrl.reset_work);
 
-	nvme_stop_keep_alive(&ctrl->ctrl);
+	nvme_stop_ctrl(&ctrl->ctrl);
 
 	/* will block will waiting for io to terminate */
 	nvme_fc_delete_association(ctrl);
 
-	if (ctrl->ctrl.state != NVME_CTRL_CONNECTING &&
-	    !nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
+	if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
 		dev_err(ctrl->ctrl.device,
 			"NVME-FC{%d}: error_recovery: Couldn't change state "
 			"to CONNECTING\n", ctrl->cnum);
-}
-
-static void
-nvme_fc_reset_ctrl_work(struct work_struct *work)
-{
-	struct nvme_fc_ctrl *ctrl =
-		container_of(work, struct nvme_fc_ctrl, ctrl.reset_work);
-	int ret;
-
-	__nvme_fc_terminate_io(ctrl);
 
-	nvme_stop_ctrl(&ctrl->ctrl);
-
-	if (ctrl->rport->remoteport.port_state == FC_OBJSTATE_ONLINE)
-		ret = nvme_fc_create_association(ctrl);
-	else
-		ret = -ENOTCONN;
-
-	if (ret)
-		nvme_fc_reconnect_or_delete(ctrl, ret);
-	else
-		dev_info(ctrl->ctrl.device,
-			"NVME-FC{%d}: controller reset complete\n",
-			ctrl->cnum);
+	if (ctrl->rport->remoteport.port_state == FC_OBJSTATE_ONLINE) {
+		if (!queue_delayed_work(nvme_wq, &ctrl->connect_work, 0)) {
+			dev_err(ctrl->ctrl.device,
+				"NVME-FC{%d}: failed to schedule connect "
+				"after reset\n", ctrl->cnum);
+		} else {
+			flush_delayed_work(&ctrl->connect_work);
+		}
+	} else {
+		nvme_fc_reconnect_or_delete(ctrl, -ENOTCONN);
+	}
 }
 
 
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/5e311a96/attachment-0001.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:27:29 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:27:29 -0700
Subject: [PATCH 4/4] nvme_fc: track error_recovery while connecting
In-Reply-To: <20201016212729.49138-1-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
Message-ID: <20201016212729.49138-5-james.smart@broadcom.com>

Whenever there are errors during CONNECTING, the driver recovers by
aborting all outstanding ios and counts on the io completion to fail them
and thus the connection/association they are on.  However, the connection
failure depends on a failure state from the core routines.  Not all
commands that are issued by the core routine are guaranteed to cause a
failure of the core routine. They may be treated as a failure status and
the status is then ignored.

As such, whenever the transport enters error_recovery while CONNECTING,
it will set a new flag indicating an association failed. The
create_association routine which creates and initializes the controller,
will monitor the state of the flag as well as the core routine error
status and ensure the association fails if there was an error.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index b2f9b3752df7..6352068c0c4a 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -142,7 +142,8 @@ struct nvme_fc_rport {
 
 /* fc_ctrl flags values - specified as bit positions */
 #define ASSOC_ACTIVE		0
-#define FCCTRL_TERMIO		1
+#define ASSOC_FAILED		1
+#define FCCTRL_TERMIO		2
 
 struct nvme_fc_ctrl {
 	spinlock_t		lock;
@@ -2498,6 +2499,7 @@ nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
 	 */
 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
 		__nvme_fc_abort_outstanding_ios(ctrl, true);
+		set_bit(ASSOC_FAILED, &ctrl->flags);
 		return;
 	}
 
@@ -3030,6 +3032,8 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 		ctrl->cnum, ctrl->lport->localport.port_name,
 		ctrl->rport->remoteport.port_name, ctrl->ctrl.opts->subsysnqn);
 
+	clear_bit(ASSOC_FAILED, &ctrl->flags);
+
 	/*
 	 * Create the admin queue
 	 */
@@ -3058,7 +3062,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 	 */
 
 	ret = nvme_enable_ctrl(&ctrl->ctrl);
-	if (ret)
+	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
 		goto out_disconnect_admin_queue;
 
 	ctrl->ctrl.max_segments = ctrl->lport->ops->max_sgl_segments;
@@ -3068,7 +3072,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 	blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
 	ret = nvme_init_identify(&ctrl->ctrl);
-	if (ret)
+	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
 		goto out_disconnect_admin_queue;
 
 	/* sanity checks */
@@ -3113,9 +3117,9 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 			ret = nvme_fc_create_io_queues(ctrl);
 		else
 			ret = nvme_fc_recreate_io_queues(ctrl);
-		if (ret)
-			goto out_term_aen_ops;
 	}
+	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
+		goto out_term_aen_ops;
 
 	changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
 
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/9ac31f0d/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:28:38 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:28:38 -0700
Subject: [PATCH] nvme-fcloop: add sysfs attribute to inject command drop
Message-ID: <20201016212838.49298-1-james.smart@broadcom.com>

Add sysfs attribute to specify parameters for dropping a command.
The attribute takes a string of
  <opcode>:<starting a what instance>:<number of times>
Opcode is formatted as lower 8 bits are opcode. If a fabrics opcode,
a bit above bits 7:0 will be set.

Once set, each sqe is looked at. If the opcode matches the running
instance count is updated. If the instance count is in the range of
where to drop (based on starting and # of times), then drop the
command by not passing it to the target layer.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/target/fcloop.c | 81 +++++++++++++++++++++++++++++++++++-
 1 file changed, 79 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
index 3da067a8311e..ae46c8402f67 100644
--- a/drivers/nvme/target/fcloop.c
+++ b/drivers/nvme/target/fcloop.c
@@ -564,6 +564,50 @@ fcloop_call_host_done(struct nvmefc_fcp_req *fcpreq,
 	fcloop_tfcp_req_put(tfcp_req);
 }
 
+static bool drop_fabric_opcode;
+#define DROP_OPCODE_MASK	0x00FF
+/* fabrics opcode will have a bit set above 1st byte */
+static int drop_opcode = -1;
+static int drop_instance;
+static int drop_amount;
+static int drop_current_cnt;
+
+/*
+ * Routine to parse io and determine if the io is to be dropped.
+ * Returns:
+ *  0 if io is not obstructed
+ *  1 if io was dropped
+ */
+int check_for_drop(struct fcloop_fcpreq *tfcp_req)
+{
+	struct nvmefc_fcp_req *fcpreq = tfcp_req->fcpreq;
+	struct nvme_fc_cmd_iu *cmdiu = fcpreq->cmdaddr;
+	struct nvme_command *sqe = &cmdiu->sqe;
+
+	if (drop_opcode == -1)
+		return 0;
+
+	pr_info("%s: seq opcd x%02x fctype x%02x: drop F %s op x%02x "
+		"inst %d start %d amt %d\n",
+		__func__, sqe->common.opcode, sqe->fabrics.fctype,
+		drop_fabric_opcode ? "y" : "n",
+		drop_opcode, drop_current_cnt, drop_instance, drop_amount);
+
+	if ((drop_fabric_opcode &&
+	     (sqe->common.opcode != nvme_fabrics_command ||
+	      sqe->fabrics.fctype != drop_opcode)) ||
+	    (!drop_fabric_opcode && sqe->common.opcode != drop_opcode))
+		return 0;
+
+	if (++drop_current_cnt >= drop_instance) {
+		if (drop_current_cnt >= drop_instance + drop_amount)
+			drop_opcode = -1;
+		return 1;
+	}
+
+	return 0;
+}
+
 static void
 fcloop_fcp_recv_work(struct work_struct *work)
 {
@@ -590,10 +634,14 @@ fcloop_fcp_recv_work(struct work_struct *work)
 
 	if (unlikely(aborted))
 		ret = -ECANCELED;
-	else
-		ret = nvmet_fc_rcv_fcp_req(tfcp_req->tport->targetport,
+	else {
+		if (likely(!check_for_drop(tfcp_req)))
+			ret = nvmet_fc_rcv_fcp_req(tfcp_req->tport->targetport,
 				&tfcp_req->tgt_fcp_req,
 				fcpreq->cmdaddr, fcpreq->cmdlen);
+		else
+			pr_info("%s: dropped command ********\n", __func__);
+	}
 	if (ret)
 		fcloop_call_host_done(fcpreq, tfcp_req, ret);
 
@@ -1449,6 +1497,33 @@ fcloop_delete_target_port(struct device *dev, struct device_attribute *attr,
 	return ret ? ret : count;
 }
 
+static ssize_t
+fcloop_set_cmd_drop(struct device *dev, struct device_attribute *attr,
+		const char *buf, size_t count)
+{
+	int opcode, starting, amount;
+
+	if (sscanf(buf, "%x:%d:%d", &opcode, &starting, &amount) != 3)
+		return -EBADRQC;
+
+	drop_current_cnt = 0;
+	drop_fabric_opcode = (opcode & ~DROP_OPCODE_MASK) ? true : false;
+	drop_opcode = (opcode & DROP_OPCODE_MASK);
+	drop_instance = starting;
+	/* the check to drop routine uses instance + count to know when
+	 * to end. Thus, if dropping 1 instance, count should be 0.
+	 * so subtract 1 from the count.
+	 */
+	drop_amount = amount - 1;
+
+	pr_info("%s: DROP: Starting at instance %d of%s opcode x%x drop +%d "
+		"instances\n",
+		__func__, drop_instance, drop_fabric_opcode ? " fabric" : "",
+		drop_opcode, drop_amount);
+
+	return count;
+}
+
 
 static DEVICE_ATTR(add_local_port, 0200, NULL, fcloop_create_local_port);
 static DEVICE_ATTR(del_local_port, 0200, NULL, fcloop_delete_local_port);
@@ -1456,6 +1531,7 @@ static DEVICE_ATTR(add_remote_port, 0200, NULL, fcloop_create_remote_port);
 static DEVICE_ATTR(del_remote_port, 0200, NULL, fcloop_delete_remote_port);
 static DEVICE_ATTR(add_target_port, 0200, NULL, fcloop_create_target_port);
 static DEVICE_ATTR(del_target_port, 0200, NULL, fcloop_delete_target_port);
+static DEVICE_ATTR(set_cmd_drop, 0200, NULL, fcloop_set_cmd_drop);
 
 static struct attribute *fcloop_dev_attrs[] = {
 	&dev_attr_add_local_port.attr,
@@ -1464,6 +1540,7 @@ static struct attribute *fcloop_dev_attrs[] = {
 	&dev_attr_del_remote_port.attr,
 	&dev_attr_add_target_port.attr,
 	&dev_attr_del_target_port.attr,
+	&dev_attr_set_cmd_drop.attr,
 	NULL
 };
 
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/a323112e/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 16 17:29:28 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 16 Oct 2020 14:29:28 -0700
Subject: [PATCH] nvme-fc: shorten reconnect delay if possible for FC
Message-ID: <20201016212928.49429-1-james.smart@broadcom.com>

We've had several complaints about a 10s reconnect delay (the default)
when there was an error while there is connectivity to a subsystem.
The max_reconnects and reconnect_delay are set in common code prior to
calling the transport to create the controller.

This change checks if the default reconnect delay is being used, and if
so, it adjusts it to a shorter period (2s) for the nvme-fc transport.
It does so by calculating the controller loss tmo window, changing the
value of the reconnect delay, and then recalculating the maximum number
of reconnect attempts allowed.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 6352068c0c4a..1af7c6f358f5 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -26,6 +26,10 @@ enum nvme_fc_queue_flags {
 };
 
 #define NVME_FC_DEFAULT_DEV_LOSS_TMO	60	/* seconds */
+#define NVME_FC_DEFAULT_RECONNECT_TMO	2	/* delay between reconnects
+						 * when connected and a
+						 * connection failure.
+						 */
 
 struct nvme_fc_queue {
 	struct nvme_fc_ctrl	*ctrl;
@@ -3387,7 +3391,7 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 {
 	struct nvme_fc_ctrl *ctrl;
 	unsigned long flags;
-	int ret, idx;
+	int ret, idx, ctrl_loss_tmo;
 
 	if (!(rport->remoteport.port_role &
 	    (FC_PORT_ROLE_NVME_DISCOVERY | FC_PORT_ROLE_NVME_TARGET))) {
@@ -3413,6 +3417,19 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 		goto out_free_ctrl;
 	}
 
+	/*
+	 * if ctrl_loss_tmo is being enforced and the default reconnect delay
+	 * is being used, change to a shorter reconnect delay for FC.
+	 */
+	if (opts->max_reconnects != -1 &&
+	    opts->reconnect_delay == NVMF_DEF_RECONNECT_DELAY &&
+	    opts->reconnect_delay > NVME_FC_DEFAULT_RECONNECT_TMO) {
+		ctrl_loss_tmo = opts->max_reconnects * opts->reconnect_delay;
+		opts->reconnect_delay = NVME_FC_DEFAULT_RECONNECT_TMO;
+		opts->max_reconnects = DIV_ROUND_UP(ctrl_loss_tmo,
+						opts->reconnect_delay);
+	}
+
 	ctrl->ctrl.opts = opts;
 	ctrl->ctrl.nr_reconnects = 0;
 	if (lport->dev)
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201016/303c4969/attachment-0001.p7s>

From nmeeramohide at micron.com  Fri Oct 16 17:58:32 2020
From: nmeeramohide at micron.com (Nabeel Meeramohideen Mohamed (nmeeramohide))
Date: Fri, 16 Oct 2020 21:58:32 +0000
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <20201015080254.GA31136@infradead.org>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
Message-ID: <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>

On Thursday, October 15, 2020 2:03 AM, Christoph Hellwig <hch at infradead.org> wrote:
> I don't think this belongs into the kernel.  It is a classic case for
> infrastructure that should be built in userspace.  If anything is
> missing to implement it in userspace with equivalent performance we
> need to improve out interfaces, although io_uring should cover pretty
> much everything you need.

Hi Christoph,

We previously considered moving the mpool object store code to user-space.
However, by implementing mpool as a device driver, we get several benefits
in terms of scalability, performance, and functionality. In doing so, we relied
only on standard interfaces and did not make any changes to the kernel.

(1)  mpool's "mcache map" facility allows us to memory-map (and later unmap)
a collection of logically related objects with a single system call. The objects in
such a collection are created at different times, physically disparate, and may
even reside on different media class volumes.

For our HSE storage engine application, there are commonly 10's to 100's of
objects in a given mcache map, and 75,000 total objects mapped at a given time.

Compared to memory-mapping objects individually, the mcache map facility
scales well because it requires only a single system call and single vm_area_struct
to memory-map a complete collection of objects.

(2) The mcache map reaper mechanism proactively evicts object data from the page
cache based on object-level metrics. This provides significant performance benefit
for many workloads.

For example, we ran YCSB workloads B (95/5 read/write mix)  and C (100% read)
against our HSE storage engine using the mpool driver in a 5.9 kernel.
For each workload, we ran with the reaper turned-on and turned-off.

For workload B, the reaper increased throughput 1.77x, while reducing 99.99% tail
latency for reads by 39% and updates by 99%. For workload C, the reaper increased
throughput by 1.84x, while reducing the 99.99% read tail latency by 63%. These
improvements are even more dramatic with earlier kernels.

(3) The mcache map facility can memory-map objects on NVMe ZNS drives that were
created using the Zone Append command. This patch set does not support ZNS, but
that work is in progress and we will be demonstrating our HSE storage engine
running on mpool with ZNS drives at FMS 2020.

(4) mpool's immutable object model allows the driver to support concurrent reading
of object data directly and memory-mapped without a performance penalty to verify
coherence. This allows background operations, such as LSM-tree compaction, to
operate efficiently and without polluting the page cache.

(5) Representing an mpool as a /dev/mpool/<mpool-name> device file provides a
convenient mechanism for controlling access to and managing the multiple storage
volumes, and in the future pmem devices, that may comprise an logical mpool.

Thanks,
Nabeel


From dan.j.williams at intel.com  Fri Oct 16 18:11:48 2020
From: dan.j.williams at intel.com (Dan Williams)
Date: Fri, 16 Oct 2020 15:11:48 -0700
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
 <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
Message-ID: <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>

On Fri, Oct 16, 2020 at 2:59 PM Nabeel Meeramohideen Mohamed
(nmeeramohide) <nmeeramohide at micron.com> wrote:
>
> On Thursday, October 15, 2020 2:03 AM, Christoph Hellwig <hch at infradead.org> wrote:
> > I don't think this belongs into the kernel.  It is a classic case for
> > infrastructure that should be built in userspace.  If anything is
> > missing to implement it in userspace with equivalent performance we
> > need to improve out interfaces, although io_uring should cover pretty
> > much everything you need.
>
> Hi Christoph,
>
> We previously considered moving the mpool object store code to user-space.
> However, by implementing mpool as a device driver, we get several benefits
> in terms of scalability, performance, and functionality. In doing so, we relied
> only on standard interfaces and did not make any changes to the kernel.
>
> (1)  mpool's "mcache map" facility allows us to memory-map (and later unmap)
> a collection of logically related objects with a single system call. The objects in
> such a collection are created at different times, physically disparate, and may
> even reside on different media class volumes.
>
> For our HSE storage engine application, there are commonly 10's to 100's of
> objects in a given mcache map, and 75,000 total objects mapped at a given time.
>
> Compared to memory-mapping objects individually, the mcache map facility
> scales well because it requires only a single system call and single vm_area_struct
> to memory-map a complete collection of objects.

Why can't that be a batch of mmap calls on io_uring?

> (2) The mcache map reaper mechanism proactively evicts object data from the page
> cache based on object-level metrics. This provides significant performance benefit
> for many workloads.
>
> For example, we ran YCSB workloads B (95/5 read/write mix)  and C (100% read)
> against our HSE storage engine using the mpool driver in a 5.9 kernel.
> For each workload, we ran with the reaper turned-on and turned-off.
>
> For workload B, the reaper increased throughput 1.77x, while reducing 99.99% tail
> latency for reads by 39% and updates by 99%. For workload C, the reaper increased
> throughput by 1.84x, while reducing the 99.99% read tail latency by 63%. These
> improvements are even more dramatic with earlier kernels.

What metrics proved useful and can the vanilla page cache / page
reclaim mechanism be augmented with those metrics?

>
> (3) The mcache map facility can memory-map objects on NVMe ZNS drives that were
> created using the Zone Append command. This patch set does not support ZNS, but
> that work is in progress and we will be demonstrating our HSE storage engine
> running on mpool with ZNS drives at FMS 2020.
>
> (4) mpool's immutable object model allows the driver to support concurrent reading
> of object data directly and memory-mapped without a performance penalty to verify
> coherence. This allows background operations, such as LSM-tree compaction, to
> operate efficiently and without polluting the page cache.
>

How is this different than existing background operations / defrag
that filesystems perform today? Where are the opportunities to improve
those operations?

> (5) Representing an mpool as a /dev/mpool/<mpool-name> device file provides a
> convenient mechanism for controlling access to and managing the multiple storage
> volumes, and in the future pmem devices, that may comprise an logical mpool.

Christoph and I have talked about replacing the pmem driver's
dependence on device-mapper for pooling. What extensions would be
needed for the existing driver arch?


From logang at deltatee.com  Fri Oct 16 18:19:04 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Fri, 16 Oct 2020 16:19:04 -0600
Subject: [PATCH v2 1/2] nvmet-passthru: Limit mdts by BIO_MAX_PAGES
Message-ID: <20201016221905.4686-1-logang@deltatee.com>

nvmet_passthru_map_sg() only supports mapping a single BIO, not a chain
so the effective maximum transfer should also be limitted by BIO_MAX_PAGES
(presently this works out to 1MB).

For PCI passthru devices the max_sectors would typically be more limitting
than BIO_MAX_PAGES, but this may not be true for all passthru devices.

Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
Suggested-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index dacfa7435d0b..1ab88df3310f 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -26,7 +26,7 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
 	struct nvme_ctrl *pctrl = ctrl->subsys->passthru_ctrl;
 	u16 status = NVME_SC_SUCCESS;
 	struct nvme_id_ctrl *id;
-	u32 max_hw_sectors;
+	int max_hw_sectors;
 	int page_shift;
 
 	id = kzalloc(sizeof(*id), GFP_KERNEL);
@@ -48,6 +48,13 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
 	max_hw_sectors = min_not_zero(pctrl->max_segments << (PAGE_SHIFT - 9),
 				      pctrl->max_hw_sectors);
 
+	/*
+	 * nvmet_passthru_map_sg is limitted to using a single bio so limit
+	 * the mdts based on BIO_MAX_PAGES as well
+	 */
+	max_hw_sectors = min_not_zero(BIO_MAX_PAGES << (PAGE_SHIFT - 9),
+				      max_hw_sectors);
+
 	page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12;
 
 	id->mdts = ilog2(max_hw_sectors) + 9 - page_shift;

base-commit: bbf5c979011a099af5dc76498918ed7df445635b
-- 
2.20.1



From logang at deltatee.com  Fri Oct 16 18:19:05 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Fri, 16 Oct 2020 16:19:05 -0600
Subject: [PATCH v2 2/2] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201016221905.4686-1-logang@deltatee.com>
References: <20201016221905.4686-1-logang@deltatee.com>
Message-ID: <20201016221905.4686-2-logang@deltatee.com>

Clean up some confusing elements of nvmet_passthru_map_sg() by returning
early if the request is greater than the maximum bio size. This allows
us to drop the sg_cnt variable.

This should not result in any functional change but makes the code
clearer and more understandable. The original code allocated a truncated
bio then would return EINVAL when bio_add_pc_page() filled that bio. The
new code just returns EINVAL early if this would happen.

Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 1ab88df3310f..8ea9b173f285 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -187,18 +187,20 @@ static void nvmet_passthru_req_done(struct request *rq,
 
 static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 {
-	int sg_cnt = req->sg_cnt;
 	struct scatterlist *sg;
 	int op_flags = 0;
 	struct bio *bio;
 	int i, ret;
 
+	if (req->sg_cnt > BIO_MAX_PAGES)
+		return -EINVAL;
+
 	if (req->cmd->common.opcode == nvme_cmd_flush)
 		op_flags = REQ_FUA;
 	else if (nvme_is_write(req->cmd))
 		op_flags = REQ_SYNC | REQ_IDLE;
 
-	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
+	bio = bio_alloc(GFP_KERNEL, req->sg_cnt);
 	bio->bi_end_io = bio_put;
 	bio->bi_opf = req_op(rq) | op_flags;
 
@@ -208,7 +210,6 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 			bio_put(bio);
 			return -EINVAL;
 		}
-		sg_cnt--;
 	}
 
 	ret = blk_rq_append_bio(rq, &bio);
-- 
2.20.1



From lkp at intel.com  Fri Oct 16 22:16:59 2020
From: lkp at intel.com (kernel test robot)
Date: Sat, 17 Oct 2020 10:16:59 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 d987eab65bf7f917472996aeda2754bac0f5fd33
Message-ID: <5f8a541b.Bc1VJP+Rzts4j13G%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: d987eab65bf7f917472996aeda2754bac0f5fd33  nvme-pci: disable Write Zeroes on Sandisk Skyhawk

elapsed time: 723m

configs tested: 106
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
m68k                          hp300_defconfig
sparc64                             defconfig
powerpc                     sequoia_defconfig
mips                  decstation_64_defconfig
sh                         ap325rxa_defconfig
powerpc                 mpc832x_mds_defconfig
sh                           se7712_defconfig
arm                          pcm027_defconfig
s390                             alldefconfig
xtensa                         virt_defconfig
powerpc                          g5_defconfig
csky                             alldefconfig
arm                       aspeed_g4_defconfig
sh                           se7206_defconfig
arm                          prima2_defconfig
mips                        qi_lb60_defconfig
powerpc                 mpc837x_mds_defconfig
mips                        vocore2_defconfig
arm                        clps711x_defconfig
mips                           ip27_defconfig
arm                           sama5_defconfig
openrisc                            defconfig
powerpc                    gamecube_defconfig
xtensa                  nommu_kc705_defconfig
sh                           se7722_defconfig
mips                           jazz_defconfig
arm                        mvebu_v5_defconfig
arm                        mini2440_defconfig
h8300                            alldefconfig
arm                          lpd270_defconfig
arc                                 defconfig
arm                          iop32x_defconfig
arm                        trizeps4_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a005-20201016
i386                 randconfig-a006-20201016
i386                 randconfig-a001-20201016
i386                 randconfig-a003-20201016
i386                 randconfig-a004-20201016
i386                 randconfig-a002-20201016
x86_64               randconfig-a016-20201016
x86_64               randconfig-a012-20201016
x86_64               randconfig-a015-20201016
x86_64               randconfig-a013-20201016
x86_64               randconfig-a014-20201016
x86_64               randconfig-a011-20201016
i386                 randconfig-a016-20201016
i386                 randconfig-a013-20201016
i386                 randconfig-a015-20201016
i386                 randconfig-a011-20201016
i386                 randconfig-a012-20201016
i386                 randconfig-a014-20201016
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a004-20201016
x86_64               randconfig-a002-20201016
x86_64               randconfig-a006-20201016
x86_64               randconfig-a001-20201016
x86_64               randconfig-a005-20201016
x86_64               randconfig-a003-20201016

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From ming.lei at redhat.com  Sun Oct 18 20:50:29 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Mon, 19 Oct 2020 08:50:29 +0800
Subject: [PATCH 1/4] blk-mq: check rq->state explicitly in
 blk_mq_tagset_count_completed_rqs
In-Reply-To: <20201016142811.1262214-2-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
 <20201016142811.1262214-2-ming.lei@redhat.com>
Message-ID: <20201019005029.GB1301072@T590>

On Fri, Oct 16, 2020 at 10:28:08PM +0800, Ming Lei wrote:
> blk_mq_tagset_count_completed_rqs() is called from
> blk_mq_tagset_wait_completed_request() for draining requests to be
> completed remotely. What we need to do is to see request->state to
> switch to non-MQ_RQ_COMPLETE.
> 
> So check MQ_RQ_COMPLETE explicitly in blk_mq_tagset_count_completed_rqs().
> 
> Meantime mark flush request as IDLE in its .end_io() for aligning to
> end normal request because flush request may stay in inflight tags in case
> of !elevator, so we need to change its state into IDLE.
> 
> Cc: Chao Leng <lengchao at huawei.com>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Signed-off-by: Ming Lei <ming.lei at redhat.com>
> ---
>  block/blk-flush.c  | 2 ++
>  block/blk-mq-tag.c | 2 +-
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/block/blk-flush.c b/block/blk-flush.c
> index 53abb5c73d99..31a2ae04d3f0 100644
> --- a/block/blk-flush.c
> +++ b/block/blk-flush.c
> @@ -231,6 +231,8 @@ static void flush_end_io(struct request *flush_rq, blk_status_t error)
>  		return;
>  	}
>  
> +	WRITE_ONCE(rq->state, MQ_RQ_IDLE);

oops, the above line should have been:

	WRITE_ONCE(flush_rq->state, MQ_RQ_IDLE);

Thanks,
Ming



From hare at suse.de  Mon Oct 19 06:04:00 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 19 Oct 2020 12:04:00 +0200
Subject: [PATCH] nvme-fc: fix error loop in create_hw_io_queues
In-Reply-To: <20201016210604.48542-1-james.smart@broadcom.com>
References: <20201016210604.48542-1-james.smart@broadcom.com>
Message-ID: <a49f995b-d256-51f9-e004-e975fab81104@suse.de>

On 10/16/20 11:06 PM, James Smart wrote:
> The loop that backs out of hw io queue creation continues through index
> 0, which corresponds to the admin queue as well.
> 
> Fix the loop so it only proceeds through indexes 1..n which correspond to
> io queues.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
>   drivers/nvme/host/fc.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index e2e09e25c056..f002522146e2 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2314,7 +2314,7 @@ nvme_fc_create_hw_io_queues(struct nvme_fc_ctrl *ctrl, u16 qsize)
>   	return 0;
>   
>   delete_queues:
> -	for (; i >= 0; i--)
> +	for (; i > 0; i--)
>   		__nvme_fc_delete_hw_queue(ctrl, &ctrl->queues[i], i);
>   	return ret;
>   }
> 
> 
Reviewed-by: Hannes Reinecke <hare at suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Mon Oct 19 06:06:16 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 19 Oct 2020 12:06:16 +0200
Subject: [PATCH] nvme-fc: wait for queues to freeze before calling
 update_hr_hw_queues
In-Reply-To: <20201016211724.48707-1-james.smart@broadcom.com>
References: <20201016211724.48707-1-james.smart@broadcom.com>
Message-ID: <112e9185-bd5e-24f6-d893-163c011c6b60@suse.de>

On 10/16/20 11:17 PM, James Smart wrote:
> On reconnect, the code currently does not freeze the controller before
> possibly updating the number hw queues for the controller.
> 
> Add the freeze before updating the number of hw queues. Note: the queues
> are already started and remain started through the reconnect.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
>   drivers/nvme/host/fc.c | 7 +++++--
>   1 file changed, 5 insertions(+), 2 deletions(-)
> Reviewed-by: Hannes Reinecke <hare at suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Mon Oct 19 06:07:43 2020
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 19 Oct 2020 12:07:43 +0200
Subject: [PATCH] nvme-fc: shorten reconnect delay if possible for FC
In-Reply-To: <20201016212928.49429-1-james.smart@broadcom.com>
References: <20201016212928.49429-1-james.smart@broadcom.com>
Message-ID: <c59178ea-da88-3d07-3e16-c6e15b5e2028@suse.de>

On 10/16/20 11:29 PM, James Smart wrote:
> We've had several complaints about a 10s reconnect delay (the default)
> when there was an error while there is connectivity to a subsystem.
> The max_reconnects and reconnect_delay are set in common code prior to
> calling the transport to create the controller.
> 
> This change checks if the default reconnect delay is being used, and if
> so, it adjusts it to a shorter period (2s) for the nvme-fc transport.
> It does so by calculating the controller loss tmo window, changing the
> value of the reconnect delay, and then recalculating the maximum number
> of reconnect attempts allowed.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
>   drivers/nvme/host/fc.c | 19 ++++++++++++++++++-
>   1 file changed, 18 insertions(+), 1 deletion(-)
> 
Reviewed-by: Hannes Reinecke <hare at suse.de

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From mgurtovoy at nvidia.com  Mon Oct 19 07:20:52 2020
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Mon, 19 Oct 2020 14:20:52 +0300
Subject: [PATCH v2 1/2] nvmet-passthru: Limit mdts by BIO_MAX_PAGES
In-Reply-To: <20201016221905.4686-1-logang@deltatee.com>
References: <20201016221905.4686-1-logang@deltatee.com>
Message-ID: <7fbd6b56-cda5-a9b3-7323-99d9704fbec5@nvidia.com>


On 10/17/2020 1:19 AM, Logan Gunthorpe wrote:
> nvmet_passthru_map_sg() only supports mapping a single BIO, not a chain
> so the effective maximum transfer should also be limitted by BIO_MAX_PAGES
> (presently this works out to 1MB).
>
> For PCI passthru devices the max_sectors would typically be more limitting
> than BIO_MAX_PAGES, but this may not be true for all passthru devices.
>
> Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
> Suggested-by: Christoph Hellwig <hch at lst.de>
> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
> Cc: Christoph Hellwig <hch at lst.de>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> ---
>   drivers/nvme/target/passthru.c | 9 ++++++++-
>   1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
> index dacfa7435d0b..1ab88df3310f 100644
> --- a/drivers/nvme/target/passthru.c
> +++ b/drivers/nvme/target/passthru.c
> @@ -26,7 +26,7 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
>   	struct nvme_ctrl *pctrl = ctrl->subsys->passthru_ctrl;
>   	u16 status = NVME_SC_SUCCESS;
>   	struct nvme_id_ctrl *id;
> -	u32 max_hw_sectors;
> +	int max_hw_sectors;
>   	int page_shift;
>   
>   	id = kzalloc(sizeof(*id), GFP_KERNEL);
> @@ -48,6 +48,13 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
>   	max_hw_sectors = min_not_zero(pctrl->max_segments << (PAGE_SHIFT - 9),
>   				      pctrl->max_hw_sectors);
>   
> +	/*
> +	 * nvmet_passthru_map_sg is limitted to using a single bio so limit
> +	 * the mdts based on BIO_MAX_PAGES as well
> +	 */
> +	max_hw_sectors = min_not_zero(BIO_MAX_PAGES << (PAGE_SHIFT - 9),
> +				      max_hw_sectors);

can 9 be replaced with SECTOR_SHIFT ?


> +
>   	page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12;
>   
>   	id->mdts = ilog2(max_hw_sectors) + 9 - page_shift;
>
> base-commit: bbf5c979011a099af5dc76498918ed7df445635b


From muneendra.kumar at broadcom.com  Mon Oct 19 03:42:56 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:12:56 +0530
Subject: [RFC v2 01/18] cgroup: Added cgroup_get_from_kernfs_id
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-2-git-send-email-muneendra.kumar@broadcom.com>

Added a new function cgroup_get_from_kernfs_id  to retrieve the cgroup
associated with cgroup id.
It takes cgroupid as an argument and returns cgrp and on failure
it returns NULL.
Exported the same as this can be used by blk-cgorup.c

Added function declaration of cgroup_get_from_kernfs_id in cgorup.h

Signed-off-by: Muneendra <muneendra.kumar at broadcom.com>

---
v2:
New patch
---
 include/linux/cgroup.h |  6 ++++++
 kernel/cgroup/cgroup.c | 25 +++++++++++++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index 618838c48313..1741f02a41d6 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -696,6 +696,7 @@ static inline void cgroup_kthread_ready(void)
 }
 
 void cgroup_path_from_kernfs_id(u64 id, char *buf, size_t buflen);
+struct cgroup *cgroup_get_from_kernfs_id(u64 id);
 #else /* !CONFIG_CGROUPS */
 
 struct cgroup_subsys_state;
@@ -743,6 +744,11 @@ static inline bool task_under_cgroup_hierarchy(struct task_struct *task,
 
 static inline void cgroup_path_from_kernfs_id(u64 id, char *buf, size_t buflen)
 {}
+
+static struct cgroup *cgroup_get_from_kernfs_id(u64 id)
+{
+	return NULL;
+}
 #endif /* !CONFIG_CGROUPS */
 
 #ifdef CONFIG_CGROUPS
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index dd247747ec14..f34ca94b04c0 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -5803,6 +5803,31 @@ void cgroup_path_from_kernfs_id(u64 id, char *buf, size_t buflen)
 	kernfs_put(kn);
 }
 
+/*
+ * cgroup_get_from_kernfs_id : get the cgroup associated with cgroup id
+ * @id: cgroup id
+ * On success it returns the cgrp on failure it returns NULL
+ */
+struct cgroup *cgroup_get_from_kernfs_id(u64 id)
+{
+	struct kernfs_node *kn;
+	struct cgroup *cgrp = NULL;
+
+	mutex_lock(&cgroup_mutex);
+	kn = kernfs_find_and_get_node_by_id(cgrp_dfl_root.kf_root, id);
+	if (!kn)
+		goto out_unlock;
+
+	cgrp = kn->priv;
+	if (cgroup_is_dead(cgrp) || !cgroup_tryget(cgrp))
+		cgrp = NULL;
+	kernfs_put(kn);
+out_unlock:
+	mutex_unlock(&cgroup_mutex);
+	return cgrp;
+}
+EXPORT_SYMBOL_GPL(cgroup_get_from_kernfs_id);
+
 /*
  * proc_cgroup_show()
  *  - Print task's cgroup paths into seq_file, one line for each hierarchy
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/7969e947/attachment-0001.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:42:55 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:12:55 +0530
Subject: [RFC v2 00/18] blkcg:Support to track FC storage blk io traffic
Message-ID: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>

This Patch added a unique application identifier i.e
app_id  knob to  blkcg which allows identification of traffic
sources at an individual cgroup based Applications
(ex:virtual machine (VM))level in both host and
fabric infrastructure.

Added a new sysfs attribute appid_store to set the application identfier
in  the blkcg associted with cgroup id
/sys/class/fc/fc_udev_device/* 
With this new interface the user can set the application identfier
in  the blkcg associted with cgroup id.

This capability can be utilized by multiple block transport infrastructure
like fc,iscsi,roce.

Existing FC fabric will use this feature and the description of
the use case is below.

Various virtualization technologies used in Fibre Channel
SAN deployments have created the opportunity to identify
and associate traffic with specific virtualized applications.
The concepts behind the T11 Application Services standard is
to provide the general mechanisms needed to identify
virtualized services.
It enables the Fabric and the storage targets to
identify, monitor, and handle FC traffic
based on vm tags by inserting application specific identification
into the FC frame.

The patches were cut against  5.10/scsi-queue tree

V2:
renamed app_identifier to app_id.
removed the  sysfs interface blkio.app_identifie under
/sys/fs/cgroup/blkio
Ported the patch on top of 5.10/scsi-queue.
Removed redundant code due to changes since last submit.
Added a fix for issuing QFPA command.

Gaurav Srivastava (15):
  lpfc: vmid: Add the datastructure for supporting VMID in lpfc
  lpfc: vmid: API to check if VMID is enabled.
  lpfc: vmid: Supplementary data structures for vmid
  lpfc: vmid: Forward declarations for APIs
  lpfc: vmid: Add support for vmid in mailbox command
  lpfc: vmid: VMID params initialization
  lpfc: vmid: vmid resource allocation
  lpfc: vmid: cleanup vmid resources
  lpfc: vmid: Implements ELS commands for appid patch
  lpfc: vmid: Functions to manage vmids
  lpfc: vmid: Implements CT commands for appid.
  lpfc: vmid: Appends the vmid in the wqe before sending request
  lpfc: vmid: Timeout implementation for vmid
  lpfc: vmid: Adding qfpa and vmid timeout check in worker thread
  lpfc: vmid: Introducing vmid in io path.

Muneendra (3):
  cgroup: Added cgroup_get_from_kernfs_id
  blkcg: Added a app identifier support for blkcg
  nvme: Added a newsysfs attribute appid_store

 block/blk-cgroup.c               |  31 +++
 drivers/nvme/host/fc.c           |  73 ++++++-
 drivers/scsi/lpfc/lpfc.h         | 121 +++++++++++
 drivers/scsi/lpfc/lpfc_attr.c    |  47 ++++
 drivers/scsi/lpfc/lpfc_crtn.h    |  11 +
 drivers/scsi/lpfc/lpfc_ct.c      | 249 +++++++++++++++++++++
 drivers/scsi/lpfc/lpfc_disc.h    |   1 +
 drivers/scsi/lpfc/lpfc_els.c     | 356 ++++++++++++++++++++++++++++++-
 drivers/scsi/lpfc/lpfc_hbadisc.c | 151 +++++++++++++
 drivers/scsi/lpfc/lpfc_hw.h      | 124 ++++++++++-
 drivers/scsi/lpfc/lpfc_hw4.h     |  12 ++
 drivers/scsi/lpfc/lpfc_init.c    | 108 ++++++++++
 drivers/scsi/lpfc/lpfc_mbox.c    |   6 +
 drivers/scsi/lpfc/lpfc_scsi.c    | 325 ++++++++++++++++++++++++++++
 drivers/scsi/lpfc/lpfc_sli.c     |  65 +++++-
 drivers/scsi/lpfc/lpfc_sli.h     |   8 +
 include/linux/blk-cgroup.h       |  22 ++
 include/linux/cgroup.h           |   6 +
 kernel/cgroup/cgroup.c           |  25 +++
 19 files changed, 1728 insertions(+), 13 deletions(-)

-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/4902261c/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:42:57 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:12:57 +0530
Subject: [RFC v2 02/18] blkcg: Added a app identifier support for blkcg
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-3-git-send-email-muneendra.kumar@broadcom.com>

This Patch added a unique application identifier i.e
app_id  knob to  blkcg which allows identification of traffic
sources at an individual cgroup based Applications
(ex:virtual machine (VM))level in both host and
fabric infrastructure.

Provided the interface blkcg_get_app_identifier to
grab the app identifier associated with a bio.

Provided the interface blkcg_set_app_identifier to
set the app identifier in a blkcgrp associated with cgroup id

Signed-off-by: Muneendra <muneendra.kumar at broadcom.com>

---
v2:
renamed app_identifier to app_id
removed the  sysfs interface blkio.app_identifie under
/sys/fs/cgroup/blkio
Added a new interface blkcg_set_app_identifier
---
 block/blk-cgroup.c         | 31 +++++++++++++++++++++++++++++++
 include/linux/blk-cgroup.h | 22 ++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index 619a79b51068..672971521010 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -546,6 +546,37 @@ static struct blkcg_gq *blkg_lookup_check(struct blkcg *blkcg,
 	return __blkg_lookup(blkcg, q, true /* update_hint */);
 }
 
+/*
+ * Sets the app_identifier field associted to blkcg
+ * @buf: application identifier
+ * @id: cgrp id
+ * @len: size of appid
+ */
+int blkcg_set_app_identifier(char *buf, u64 id, size_t len)
+{
+	struct cgroup *cgrp = NULL;
+	struct cgroup_subsys_state *css = NULL;
+	struct blkcg *blkcg = NULL;
+
+	cgrp = cgroup_get_from_kernfs_id(id);
+	if (!cgrp)
+		return -ENOENT;
+
+	css = cgroup_get_e_css(cgrp, &io_cgrp_subsys);
+	if (!css)
+		return -ENOENT;
+
+	blkcg = css_to_blkcg(css);
+	if (!blkcg)
+		return -ENOENT;
+
+	if (len > APPID_LEN)
+		return -EINVAL;
+	strlcpy(blkcg->app_id, buf, len);
+	return 0;
+}
+EXPORT_SYMBOL(blkcg_set_app_identifier);
+
 /**
  * blkg_conf_prep - parse and prepare for per-blkg config update
  * @inputp: input string pointer
diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h
index c8fc9792ac77..5bd3f9f397ac 100644
--- a/include/linux/blk-cgroup.h
+++ b/include/linux/blk-cgroup.h
@@ -30,6 +30,8 @@
 
 /* Max limits for throttle policy */
 #define THROTL_IOPS_MAX		UINT_MAX
+#define APPID_LEN              128
+
 
 #ifdef CONFIG_BLK_CGROUP
 
@@ -55,6 +57,8 @@ struct blkcg {
 	struct blkcg_policy_data	*cpd[BLKCG_MAX_POLS];
 
 	struct list_head		all_blkcgs_node;
+	char                            app_id[APPID_LEN];
+
 #ifdef CONFIG_CGROUP_WRITEBACK
 	struct list_head		cgwb_list;
 #endif
@@ -206,6 +210,24 @@ struct gendisk *blkcg_conf_get_disk(char **inputp);
 int blkg_conf_prep(struct blkcg *blkcg, const struct blkcg_policy *pol,
 		   char *input, struct blkg_conf_ctx *ctx);
 void blkg_conf_finish(struct blkg_conf_ctx *ctx);
+int blkcg_set_app_identifier(char *buf, u64 id, size_t len);
+
+/**
+ * blkcg_get_app_identifier - grab the app identifier associated with a bio
+ * @bio: target bio
+ *
+ * This returns the app identifier associated with a bio,
+ * %NULL if not associated.
+ * Callers are expected to either handle %NULL or know association has been
+ * done prior to calling this.
+ */
+static inline char *blkcg_get_app_identifier(struct bio *bio)
+{
+	if (bio && bio->bi_blkg &&
+			strlen(bio->bi_blkg->blkcg->app_id))
+		return bio->bi_blkg->blkcg->app_id;
+	return NULL;
+}
 
 /**
  * blkcg_css - find the current css
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/3080e66a/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:42:58 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:12:58 +0530
Subject: [RFC v2 03/18] nvme: Added a newsysfs attribute appid_store
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-4-git-send-email-muneendra.kumar@broadcom.com>

Added a new sysfs attribute appid_store under
/sys/class/fc/fc_udev_device/*

With this new interface the user can set the application identfier
in  the blkcg associted with cgroup id.

Once the application identifer has set with this interface it allows
identification of traffic sources at an individual cgroup based
Applications (ex:virtual machine (VM))level in both host and
fabric infrastructure(FC).

Below is the interface provided to set the app_id

echo "<cgroupid>:<appid>" >> /sys/class/fc/fc_udev_device/appid_store
echo "457E:100000109b521d27" >> /sys/class/fc/fc_udev_device/appid_store

Signed-off-by: Muneendra <muneendra.kumar at broadcom.com>

---
v2:
New Patch
---
 drivers/nvme/host/fc.c | 73 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index eae43bb444e0..6d6cc06fd54a 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -9,7 +9,7 @@
 #include <uapi/scsi/fc/fc_els.h>
 #include <linux/delay.h>
 #include <linux/overflow.h>
-
+#include <linux/blk-cgroup.h>
 #include "nvme.h"
 #include "fabrics.h"
 #include <linux/nvme-fc-driver.h>
@@ -3768,10 +3768,81 @@ static ssize_t nvme_fc_nvme_discovery_store(struct device *dev,
 
 	return count;
 }
+
+/*parse the Cgroup id from a buf and returns the length of cgrpid*/
+static int fc_parse_cgrpid(const char *buf, u64 *id)
+{
+	char cgrp_id[16+1];
+	int cgrpid_len, j;
+
+	memset(cgrp_id, 0x0, sizeof(cgrp_id));
+	for (cgrpid_len = 0, j = 0; cgrpid_len < 17; cgrpid_len++) {
+		if (buf[cgrpid_len] != ':')
+			cgrp_id[cgrpid_len] = buf[cgrpid_len];
+		else {
+			j = 1;
+			break;
+		}
+	}
+	if (!j)
+		return -EINVAL;
+	if (kstrtou64(cgrp_id, 16, id) < 0)
+		return -EINVAL;
+	return cgrpid_len;
+}
+
+/*
+ * fc_update_appid :parses and updates the appid in the blkcg associated with
+ * cgroupid.
+ * @buf: buf contains both cgrpid and appid info
+ * @count: size of the buffer
+ */
+static int fc_update_appid(const char *buf, size_t count)
+{
+	u64 cgrp_id;
+	int appid_len = 0;
+	int cgrpid_len = 0;
+	char app_id[APPID_LEN];
+	int ret = 0;
+
+	if (buf[count-1] == '\n')
+		count--;
+
+	if ((count > (16+1+APPID_LEN)) || (!strchr(buf, ':')))
+		return -EINVAL;
+
+	cgrpid_len = fc_parse_cgrpid(buf, &cgrp_id);
+	if (cgrpid_len < 0)
+		return -EINVAL;
+	/*appid len is count - cgrpid_len -1 (: + \n) */
+	appid_len = count - cgrpid_len - 1;
+	if (appid_len > APPID_LEN)
+		return -EINVAL;
+
+	memset(app_id, 0x0, sizeof(app_id));
+	memcpy(app_id, &buf[cgrpid_len+1], appid_len);
+	ret = blkcg_set_app_identifier(app_id, cgrp_id, sizeof(app_id));
+	if (ret < 0)
+		return ret;
+	return count;
+}
+
+static ssize_t fc_appid_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	int ret  = 0;
+
+	ret = fc_update_appid(buf, count);
+	if (ret < 0)
+		return -EINVAL;
+	return count;
+}
 static DEVICE_ATTR(nvme_discovery, 0200, NULL, nvme_fc_nvme_discovery_store);
+static DEVICE_ATTR(appid_store, 0200, NULL, fc_appid_store);
 
 static struct attribute *nvme_fc_attrs[] = {
 	&dev_attr_nvme_discovery.attr,
+	&dev_attr_appid_store.attr,
 	NULL
 };
 
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/150d7bf2/attachment-0001.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:42:59 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:12:59 +0530
Subject: [RFC v2 04/18] lpfc: vmid: Add the datastructure for supporting VMID
 in lpfc
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-5-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch adds the primary datastructures needed to implement VMID in lpfc
driver. It maintains the capability, current state, hash table for the
vmid/appid along with other information.

Signed-off-by: Gaurav Srivastava <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
Removed unused variable.
---
 drivers/scsi/lpfc/lpfc.h | 97 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 97 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index 549adfaa97ce..8f0062d2b891 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -303,6 +303,63 @@ struct lpfc_stats {
 struct lpfc_hba;
 
 
+#define LPFC_VMID_TIMER   300	/* timer interval in seconds. */
+
+#define LPFC_MAX_VMID_SIZE      256
+#define LPFC_COMPRESS_VMID_SIZE 16
+
+union lpfc_vmid_io_tag {
+	u32 app_id;	/* App Id vmid */
+	u8 cs_ctl_vmid;	/* Priority tag vmid */
+};
+
+#define JIFFIES_PER_HR	(HZ * 60 * 60)
+
+struct lpfc_vmid {
+	u8 flag;
+#define LPFC_VMID_SLOT_FREE     0x0
+#define LPFC_VMID_SLOT_USED     0x1
+#define LPFC_VMID_REQ_REGISTER  0x2
+#define LPFC_VMID_REGISTERED    0x4
+#define LPFC_VMID_DE_REGISTER   0x8
+	u8 host_vmid[LPFC_MAX_VMID_SIZE];
+	union lpfc_vmid_io_tag un;
+	u64 io_rd_cnt;
+	u64 io_wr_cnt;
+	u8 vmid_len;
+	u8 delete_inactive; /* Delete if inactive flag 0 = no, 1 = yes */
+	u32 hash_index;
+	u64 __percpu *last_io_time;
+};
+
+#define lpfc_vmid_is_type_priority_tag(vport)\
+	(vport->vmid_priority_tagging ? 1 : 0)
+
+#define LPFC_VMID_HASH_SIZE     256
+#define LPFC_VMID_HASH_MASK     255
+#define LPFC_VMID_HASH_SHIFT    6
+
+struct lpfc_vmid_context {
+	struct lpfc_vmid *vmp;
+	struct lpfc_nodelist *nlp;
+	u8 instantiated;
+};
+
+struct lpfc_vmid_priority_range {
+	u8 low;
+	u8 high;
+	u8 qos;
+};
+
+struct lpfc_vmid_priority_info {
+	u32 num_descriptors;
+	struct lpfc_vmid_priority_range *vmid_range;
+};
+
+#define QFPA_EVEN_ONLY 0x01
+#define QFPA_ODD_ONLY  0x02
+#define QFPA_EVEN_ODD  0x03
+
 enum discovery_state {
 	LPFC_VPORT_UNKNOWN     =  0,    /* vport state is unknown */
 	LPFC_VPORT_FAILED      =  1,    /* vport has failed */
@@ -442,6 +499,9 @@ struct lpfc_vport {
 #define WORKER_RAMP_DOWN_QUEUE         0x800	/* hba: Decrease Q depth */
 #define WORKER_RAMP_UP_QUEUE           0x1000	/* hba: Increase Q depth */
 #define WORKER_SERVICE_TXQ             0x2000	/* hba: IOCBs on the txq */
+#define WORKER_CHECK_INACTIVE_VMID     0x4000	/* hba: check inactive vmids */
+#define WORKER_CHECK_VMID_ISSUE_QFPA   0x8000	/* vport: Check if qfpa need */
+						/* to issue */
 
 	struct timer_list els_tmofunc;
 	struct timer_list delayed_disc_tmo;
@@ -452,6 +512,8 @@ struct lpfc_vport {
 #define FC_LOADING		0x1	/* HBA in process of loading drvr */
 #define FC_UNLOADING		0x2	/* HBA in process of unloading drvr */
 #define FC_ALLOW_FDMI		0x4	/* port is ready for FDMI requests */
+#define FC_ALLOW_VMID		0x8	/* Allow VMID IO's */
+#define FC_DEREGISTER_ALL_APP_ID	0x10	/* Deregister all vmid's */
 	/* Vport Config Parameters */
 	uint32_t cfg_scan_down;
 	uint32_t cfg_lun_queue_depth;
@@ -470,9 +532,36 @@ struct lpfc_vport {
 	uint32_t cfg_tgt_queue_depth;
 	uint32_t cfg_first_burst_size;
 	uint32_t dev_loss_tmo_changed;
+	/* VMID parameters */
+	u8 lpfc_vmid_host_uuid[LPFC_COMPRESS_VMID_SIZE];
+	u32 max_vmid;	/* maximum VMIDs allowed per port */
+	u32 cur_vmid_cnt;	/* Current VMID count */
+#define LPFC_MIN_VMID	4
+#define LPFC_MAX_VMID	255
+	u32 vmid_inactivity_timeout;	/* Time after which the VMID */
+						/* deregisters from switch */
+	u32 vmid_priority_tagging;
+#define LPFC_VMID_PRIO_TAG_DISABLE	0 /* Disable */
+#define LPFC_VMID_PRIO_TAG_SUP_TARGETS	1 /* Allow supported targets only */
+#define LPFC_VMID_PRIO_TAG_ALL_TARGETS	2 /* Allow all targets */
+	unsigned long *vmid_priority_range;
+#define LPFC_VMID_MAX_PRIORITY_RANGE    256
+#define LPFC_VMID_PRIORITY_BITMAP_SIZE  32
+	u8 vmid_flag;
+#define LPFC_VMID_IN_USE		0x1
+#define LPFC_VMID_ISSUE_QFPA		0x2
+#define LPFC_VMID_QFPA_CMPL		0x4
+#define LPFC_VMID_QOS_ENABLED		0x8
+#define LPFC_VMID_TIMER_ENBLD		0x10
+	struct fc_qfpa_res *qfpa_res;
 
 	struct fc_vport *fc_vport;
 
+	struct lpfc_vmid *vmid;
+	struct lpfc_vmid *hash_table[LPFC_VMID_HASH_SIZE];
+	rwlock_t vmid_lock;
+	struct lpfc_vmid_priority_info vmid_priority;
+
 #ifdef CONFIG_SCSI_LPFC_DEBUG_FS
 	struct dentry *debug_disc_trc;
 	struct dentry *debug_nodelist;
@@ -925,6 +1014,13 @@ struct lpfc_hba {
 	struct nvmet_fc_target_port *targetport;
 	lpfc_vpd_t vpd;		/* vital product data */
 
+	u32 cfg_max_vmid;	/* maximum VMIDs allowed per port */
+	u32 cfg_vmid_app_header;
+#define LPFC_VMID_APP_HEADER_DISABLE	0
+#define LPFC_VMID_APP_HEADER_ENABLE	1
+	u32 cfg_vmid_priority_tagging;
+	u32 cfg_vmid_inactivity_timeout;	/* Time after which the VMID */
+						/*  deregisters from switch */
 	struct pci_dev *pcidev;
 	struct list_head      work_list;
 	uint32_t              work_ha;      /* Host Attention Bits for WT */
@@ -1168,6 +1264,7 @@ struct lpfc_hba {
 	struct list_head ct_ev_waiters;
 	struct unsol_rcv_ct_ctx ct_ctx[LPFC_CT_CTX_MAX];
 	uint32_t ctx_idx;
+	struct timer_list inactive_vmid_poll;
 
 	/* RAS Support */
 	struct lpfc_ras_fwlog ras_fwlog;
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/45917564/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:00 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:00 +0530
Subject: [RFC v2 05/18] lpfc: vmid: API to check if VMID is enabled.
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-6-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This API will determine if VMID is enabled by the user or not.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index 8f0062d2b891..a62e985aa180 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -1506,3 +1506,27 @@ static const char *routine(enum enum_name table_key)			\
 	}								\
 	return name;							\
 }
+
+/**
+ * lpfc_is_vmid_enabled - returns if VMID is enabled for either switch types
+ * @phba: Pointer to HBA context object.
+ *
+ * Relationship between the enable, target support and if vmid tag is required
+ * for the particular combination
+ * ---------------------------------------------------
+ * Switch    Enable Flag  Target Support  VMID Needed
+ * ---------------------------------------------------
+ * App Id     0              NA              N
+ * App Id     1               0              N
+ * App Id     1               1              Y
+ * Pr Tag     0              NA              N
+ * Pr Tag     1               0              N
+ * Pr Tag     1               1              Y
+ * Pr Tag     2               *              Y
+ ---------------------------------------------------
+ *
+ **/
+static inline int lpfc_is_vmid_enabled(struct lpfc_hba *phba)
+{
+	return phba->cfg_vmid_app_header || phba->cfg_vmid_priority_tagging;
+}
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/9950b91e/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:01 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:01 +0530
Subject: [RFC v2 06/18] lpfc: vmid: Supplementary data structures for vmid
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-7-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch adds additional data structures for supporting the two
versions of vmid implementation. First type uses app header while the
other types uses priority tagging mechanism. These data structures
are used mostly for ELS and CT commands for the two vmid implementation.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_disc.h |   1 +
 drivers/scsi/lpfc/lpfc_hw.h   | 124 ++++++++++++++++++++++++++++++++--
 drivers/scsi/lpfc/lpfc_sli.h  |   8 +++
 3 files changed, 129 insertions(+), 4 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc_disc.h b/drivers/scsi/lpfc/lpfc_disc.h
index 482e4a888dae..c38313ee17dd 100644
--- a/drivers/scsi/lpfc/lpfc_disc.h
+++ b/drivers/scsi/lpfc/lpfc_disc.h
@@ -113,6 +113,7 @@ struct lpfc_nodelist {
 	uint8_t         nlp_fcp_info;	        /* class info, bits 0-3 */
 #define NLP_FCP_2_DEVICE   0x10			/* FCP-2 device */
 	u8		nlp_nvme_info;	        /* NVME NSLER Support */
+	u8		vmid_support;		/* destination VMID support */
 #define NLP_NVME_NSLER     0x1			/* NVME NSLER device */
 
 	uint16_t        nlp_usg_map;	/* ndlp management usage bitmap */
diff --git a/drivers/scsi/lpfc/lpfc_hw.h b/drivers/scsi/lpfc/lpfc_hw.h
index c20034b3101c..c6b252fbea40 100644
--- a/drivers/scsi/lpfc/lpfc_hw.h
+++ b/drivers/scsi/lpfc/lpfc_hw.h
@@ -275,6 +275,7 @@ struct lpfc_sli_ct_request {
 #define  SLI_CT_ACCESS_DENIED             0x10
 #define  SLI_CT_INVALID_PORT_ID           0x11
 #define  SLI_CT_DATABASE_EMPTY            0x12
+#define  SLI_CT_APP_ID_NOT_AVAILABLE      0x40
 
 /*
  * Name Server Command Codes
@@ -400,16 +401,16 @@ struct csp {
 	uint16_t altBbCredit:1;	/* FC Word 1, bit 27 */
 	uint16_t edtovResolution:1;	/* FC Word 1, bit 26 */
 	uint16_t multicast:1;	/* FC Word 1, bit 25 */
-	uint16_t broadcast:1;	/* FC Word 1, bit 24 */
+	u16 app_hdr_support:1;	/* FC Word 1, bit 24 */
 
-	uint16_t huntgroup:1;	/* FC Word 1, bit 23 */
+	uint16_t priority_tagging:1;	/* FC Word 1, bit 23 */
 	uint16_t simplex:1;	/* FC Word 1, bit 22 */
 	uint16_t word1Reserved1:3;	/* FC Word 1, bit 21:19 */
 	uint16_t dhd:1;		/* FC Word 1, bit 18 */
 	uint16_t contIncSeqCnt:1;	/* FC Word 1, bit 17 */
 	uint16_t payloadlength:1;	/* FC Word 1, bit 16 */
 #else	/*  __LITTLE_ENDIAN_BITFIELD */
-	uint16_t broadcast:1;	/* FC Word 1, bit 24 */
+	u16 app_hdr_support:1;	/* FC Word 1, bit 24 */
 	uint16_t multicast:1;	/* FC Word 1, bit 25 */
 	uint16_t edtovResolution:1;	/* FC Word 1, bit 26 */
 	uint16_t altBbCredit:1;	/* FC Word 1, bit 27 */
@@ -423,7 +424,7 @@ struct csp {
 	uint16_t dhd:1;		/* FC Word 1, bit 18 */
 	uint16_t word1Reserved1:3;	/* FC Word 1, bit 21:19 */
 	uint16_t simplex:1;	/* FC Word 1, bit 22 */
-	uint16_t huntgroup:1;	/* FC Word 1, bit 23 */
+	uint16_t priority_tagging:1;	/* FC Word 1, bit 23 */
 #endif
 
 	uint8_t bbRcvSizeMsb;	/* Upper nibble is reserved */
@@ -607,6 +608,8 @@ struct fc_vft_header {
 #define ELS_CMD_LIRR      0x7A000000
 #define ELS_CMD_LCB	  0x81000000
 #define ELS_CMD_FPIN	  0x16000000
+#define ELS_CMD_QFPA      0xB0000000
+#define ELS_CMD_UVEM      0xB1000000
 #else	/*  __LITTLE_ENDIAN_BITFIELD */
 #define ELS_CMD_MASK      0xffff
 #define ELS_RSP_MASK      0xff
@@ -649,6 +652,8 @@ struct fc_vft_header {
 #define ELS_CMD_LIRR      0x7A
 #define ELS_CMD_LCB	  0x81
 #define ELS_CMD_FPIN	  ELS_FPIN
+#define ELS_CMD_QFPA      0xB0
+#define ELS_CMD_UVEM      0xB1
 #endif
 
 /*
@@ -1317,6 +1322,117 @@ struct fc_rdp_res_frame {
 };
 
 
+/* UVEM */
+
+#define LPFC_UVEM_SIZE 60
+#define LPFC_UVEM_VEM_ID_DESC_SIZE 16
+#define LPFC_UVEM_VE_MAP_DESC_SIZE 20
+
+#define VEM_ID_DESC_TAG  0x0001000A
+struct lpfc_vem_id_desc {
+	u32 tag;
+	u32 length;
+	u8 vem_id[16];
+};
+
+#define LPFC_QFPA_SIZE	4
+
+#define INSTANTIATED_VE_DESC_TAG  0x0001000B
+struct instantiated_ve_desc {
+	u32 tag;
+	u32 length;
+	u8 global_vem_id[16];
+	u32 word6;
+#define lpfc_instantiated_local_id_SHIFT   0
+#define lpfc_instantiated_local_id_MASK    0x000000ff
+#define lpfc_instantiated_local_id_WORD    word6
+#define lpfc_instantiated_nport_id_SHIFT   8
+#define lpfc_instantiated_nport_id_MASK    0x00ffffff
+#define lpfc_instantiated_nport_id_WORD    word6
+};
+
+#define DEINSTANTIATED_VE_DESC_TAG  0x0001000C
+struct deinstantiated_ve_desc {
+	u32 tag;
+	u32 length;
+	u8 global_vem_id[16];
+	u32 word6;
+#define lpfc_deinstantiated_nport_id_SHIFT   0
+#define lpfc_deinstantiated_nport_id_MASK    0x000000ff
+#define lpfc_deinstantiated_nport_id_WORD    word6
+#define lpfc_deinstantiated_local_id_SHIFT   24
+#define lpfc_deinstantiated_local_id_MASK    0x00ffffff
+#define lpfc_deinstantiated_local_id_WORD    word6
+};
+
+/* Query Fabric Priority Allocation Response */
+#define LPFC_PRIORITY_RANGE_DESC_SIZE 12
+
+struct priority_range_desc {
+	u32 tag;
+	u32 length;
+	u8 lo_range;
+	u8 hi_range;
+	u8 qos_priority;
+	u8 local_ve_id;
+};
+
+struct fc_qfpa_res {
+	u32 reply_sequence;	/* LS_ACC or LS_RJT */
+	u32 length;	/* FC Word 1    */
+	struct priority_range_desc desc[1];
+};
+
+/* Application Server command code */
+/* VMID               */
+
+#define SLI_CT_APP_SEV_Subtypes     0x20	/* Application Server subtype */
+
+#define SLI_CTAS_GAPPIA_ENT    0x0100	/* Get Application Identifier */
+#define SLI_CTAS_GALLAPPIA     0x0101	/* Get All Application Identifier */
+#define SLI_CTAS_GALLAPPIA_ID  0x0102	/* Get All Application Identifier */
+					/* for Nport */
+#define SLI_CTAS_GAPPIA_IDAPP  0x0103	/* Get Application Identifier */
+					/* for Nport */
+#define SLI_CTAS_RAPP_IDENT    0x0200	/* Register Application Identifier */
+#define SLI_CTAS_DAPP_IDENT    0x0300	/* Deregister Application */
+					/* Identifier */
+#define SLI_CTAS_DALLAPP_ID    0x0301	/* Deregister All Application */
+					/* Identifier */
+
+struct entity_id_object {
+	u8 entity_id_len;
+	u8 entity_id[255];	/* VM UUID */
+};
+
+struct app_id_object {
+	u32 port_id;
+	u32 app_id;
+	struct entity_id_object obj;
+};
+
+struct lpfc_vmid_rapp_ident_list {
+	u32 no_of_objects;
+	struct entity_id_object obj[1];
+};
+
+struct lpfc_vmid_dapp_ident_list {
+	u32 no_of_objects;
+	struct entity_id_object obj[1];
+};
+
+#define GALLAPPIA_ID_LAST  0x80
+struct lpfc_vmid_gallapp_ident_list {
+	u8 control;
+	u8 reserved[3];
+	struct app_id_object app_id;
+};
+
+#define RAPP_IDENT_OFFSET  (offsetof(struct lpfc_sli_ct_request, un) + 4)
+#define DAPP_IDENT_OFFSET  (offsetof(struct lpfc_sli_ct_request, un) + 4)
+#define GALLAPPIA_ID_SIZE  (offsetof(struct lpfc_sli_ct_request, un) + 4)
+#define DALLAPP_ID_SIZE    (offsetof(struct lpfc_sli_ct_request, un) + 4)
+
 /******** FDMI ********/
 
 /* lpfc_sli_ct_request defines the CT_IU preamble for FDMI commands */
diff --git a/drivers/scsi/lpfc/lpfc_sli.h b/drivers/scsi/lpfc/lpfc_sli.h
index 93d976ea8c5d..6dd45885df4f 100644
--- a/drivers/scsi/lpfc/lpfc_sli.h
+++ b/drivers/scsi/lpfc/lpfc_sli.h
@@ -35,6 +35,12 @@ typedef enum _lpfc_ctx_cmd {
 	LPFC_CTX_HOST
 } lpfc_ctx_cmd;
 
+union lpfc_vmid_iocb_tag {
+	u32 app_id;
+	u8 cs_ctl_vmid;
+	struct lpfc_vmid_context *vmid_context;	/* UVEM context information */
+};
+
 struct lpfc_cq_event {
 	struct list_head list;
 	uint16_t hdwq;
@@ -100,6 +106,7 @@ struct lpfc_iocbq {
 #define LPFC_IO_NVME	        0x200000 /* NVME FCP command */
 #define LPFC_IO_NVME_LS		0x400000 /* NVME LS command */
 #define LPFC_IO_NVMET		0x800000 /* NVMET command */
+#define LPFC_IO_VMID            0x1000000 /* VMID tagged IO */
 
 	uint32_t drvrTimeout;	/* driver timeout in seconds */
 	struct lpfc_vport *vport;/* virtual port pointer */
@@ -114,6 +121,7 @@ struct lpfc_iocbq {
 		struct lpfc_node_rrq *rrq;
 	} context_un;
 
+	union lpfc_vmid_iocb_tag vmid_tag;
 	void (*fabric_iocb_cmpl)(struct lpfc_hba *, struct lpfc_iocbq *,
 			   struct lpfc_iocbq *);
 	void (*wait_iocb_cmpl)(struct lpfc_hba *, struct lpfc_iocbq *,
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/cc2b1133/attachment-0001.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:02 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:02 +0530
Subject: [RFC v2 07/18] lpfc: vmid: Forward declarations for APIs
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-8-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch contains the forward declarations of commonly used APIs which
are used outside the scope of the file.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_crtn.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h
index 782f6f76f18a..74ca5860ca8e 100644
--- a/drivers/scsi/lpfc/lpfc_crtn.h
+++ b/drivers/scsi/lpfc/lpfc_crtn.h
@@ -600,3 +600,14 @@ extern int lpfc_enable_nvmet_cnt;
 extern unsigned long long lpfc_enable_nvmet[];
 extern int lpfc_no_hba_reset_cnt;
 extern unsigned long lpfc_no_hba_reset[];
+
+/* vmid interface */
+int lpfc_vmid_uvem(struct lpfc_vport *vport, struct lpfc_vmid *vmid, bool ins);
+u32 lpfc_vmid_get_cs_ctl(struct lpfc_vport *vport);
+int lpfc_vmid_cmd(struct lpfc_vport *vport,
+		  int cmdcode, struct lpfc_vmid *vmid);
+int lpfc_vmid_hash_fn(char *vmid, int len);
+struct lpfc_vmid *lpfc_get_vmid_from_hastable(struct lpfc_vport *vport,
+					      u32 hash, u8 *buf);
+void lpfc_vmid_vport_cleanup(struct lpfc_vport *vport);
+int lpfc_issue_els_qfpa(struct lpfc_vport *vport);
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/b001eb3a/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:03 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:03 +0530
Subject: [RFC v2 08/18] lpfc: vmid: Add support for vmid in mailbox command
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-9-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch adds supporting datastructures for mailbox command which helps
in determining if the firmware supports appid or not.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_hw4.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h
index 12e4e76233e6..76c072366a16 100644
--- a/drivers/scsi/lpfc/lpfc_hw4.h
+++ b/drivers/scsi/lpfc/lpfc_hw4.h
@@ -272,6 +272,9 @@ struct lpfc_sli4_flags {
 #define lpfc_vfi_rsrc_rdy_MASK		0x00000001
 #define lpfc_vfi_rsrc_rdy_WORD		word0
 #define LPFC_VFI_RSRC_RDY		1
+#define lpfc_ftr_ashdr_SHIFT            4
+#define lpfc_ftr_ashdr_MASK             0x00000001
+#define lpfc_ftr_ashdr_WORD             word0
 };
 
 struct sli4_bls_rsp {
@@ -2943,6 +2946,9 @@ struct lpfc_mbx_request_features {
 #define lpfc_mbx_rq_ftr_rq_mrqp_SHIFT		16
 #define lpfc_mbx_rq_ftr_rq_mrqp_MASK		0x00000001
 #define lpfc_mbx_rq_ftr_rq_mrqp_WORD		word2
+#define lpfc_mbx_rq_ftr_rq_ashdr_SHIFT          17
+#define lpfc_mbx_rq_ftr_rq_ashdr_MASK           0x00000001
+#define lpfc_mbx_rq_ftr_rq_ashdr_WORD           word2
 	uint32_t word3;
 #define lpfc_mbx_rq_ftr_rsp_iaab_SHIFT		0
 #define lpfc_mbx_rq_ftr_rsp_iaab_MASK		0x00000001
@@ -2974,6 +2980,9 @@ struct lpfc_mbx_request_features {
 #define lpfc_mbx_rq_ftr_rsp_mrqp_SHIFT		16
 #define lpfc_mbx_rq_ftr_rsp_mrqp_MASK		0x00000001
 #define lpfc_mbx_rq_ftr_rsp_mrqp_WORD		word3
+#define lpfc_mbx_rq_ftr_rsp_ashdr_SHIFT         17
+#define lpfc_mbx_rq_ftr_rsp_ashdr_MASK          0x00000001
+#define lpfc_mbx_rq_ftr_rsp_ashdr_WORD          word3
 };
 
 struct lpfc_mbx_supp_pages {
@@ -4383,6 +4392,9 @@ struct wqe_common {
 #define wqe_nvme_SHIFT        4
 #define wqe_nvme_MASK         0x00000001
 #define wqe_nvme_WORD         word10
+#define wqe_appid_SHIFT       5
+#define wqe_appid_MASK        0x00000001
+#define wqe_appid_WORD        word10
 #define wqe_oas_SHIFT         6
 #define wqe_oas_MASK          0x00000001
 #define wqe_oas_WORD          word10
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/93ef56ec/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:04 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:04 +0530
Subject: [RFC v2 09/18] lpfc: vmid: VMID params initialization
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-10-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch initializes the VMID parameters like the type of vmid, max
number of vmids supported and timeout value for the vmid registration
based on the user input.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_attr.c | 47 +++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index e94eac194676..e73997bce65c 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -6138,6 +6138,44 @@ LPFC_BBCR_ATTR_RW(enable_bbcr, 1, 0, 1, "Enable BBC Recovery");
  */
 LPFC_ATTR_RW(enable_dpp, 1, 0, 1, "Enable Direct Packet Push");
 
+/*
+ * lpfc_max_vmid: Maximum number of VMs to be tagged. This is valid only if
+ * either vmid_app_header or vmid_priority_tagging is enabled.
+ *       4 - 255  = vmid support enabled for 4-255 VMs
+ *       Value range is [4,255].
+ */
+LPFC_ATTR_RW(max_vmid, LPFC_MIN_VMID, LPFC_MIN_VMID, LPFC_MAX_VMID,
+	     "Maximum number of VMs supported");
+
+/*
+ * lpfc_vmid_inactivity_timeout: Inactivity timeout duration in hours
+ *       0  = Timeout is disabled
+ * Value range is [0,24].
+ */
+LPFC_ATTR_RW(vmid_inactivity_timeout, 4, 0, 24,
+	     "Inactivity timeout in hours");
+
+/*
+ * lpfc_vmid_app_header: Enable App Header VMID support
+ *       0  = Support is disabled (default)
+ *       1  = Support is enabled
+ * Value range is [0,1].
+ */
+LPFC_ATTR_RW(vmid_app_header, LPFC_VMID_APP_HEADER_DISABLE,
+	     LPFC_VMID_APP_HEADER_DISABLE, LPFC_VMID_APP_HEADER_ENABLE,
+	     "Enable App Header VMID support");
+
+/*
+ * lpfc_vmid_priority_tagging: Enable Priority Tagging VMID support
+ *       0  = Support is disabled (default)
+ *       1  = Support is enabled
+ * Value range is [0,1]..
+ */
+LPFC_ATTR_RW(vmid_priority_tagging, LPFC_VMID_PRIO_TAG_DISABLE,
+	     LPFC_VMID_PRIO_TAG_DISABLE,
+	     LPFC_VMID_PRIO_TAG_ALL_TARGETS,
+	     "Enable Priority Tagging VMID support");
+
 struct device_attribute *lpfc_hba_attrs[] = {
 	&dev_attr_nvme_info,
 	&dev_attr_scsi_stat,
@@ -6255,6 +6293,10 @@ struct device_attribute *lpfc_hba_attrs[] = {
 	&dev_attr_lpfc_ras_fwlog_func,
 	&dev_attr_lpfc_enable_bbcr,
 	&dev_attr_lpfc_enable_dpp,
+	&dev_attr_lpfc_max_vmid,
+	&dev_attr_lpfc_vmid_inactivity_timeout,
+	&dev_attr_lpfc_vmid_app_header,
+	&dev_attr_lpfc_vmid_priority_tagging,
 	NULL,
 };
 
@@ -7314,6 +7356,11 @@ lpfc_get_cfgparam(struct lpfc_hba *phba)
 	lpfc_enable_hba_heartbeat_init(phba, lpfc_enable_hba_heartbeat);
 
 	lpfc_EnableXLane_init(phba, lpfc_EnableXLane);
+	/* VMID Inits */
+	lpfc_max_vmid_init(phba, lpfc_max_vmid);
+	lpfc_vmid_inactivity_timeout_init(phba, lpfc_vmid_inactivity_timeout);
+	lpfc_vmid_app_header_init(phba, lpfc_vmid_app_header);
+	lpfc_vmid_priority_tagging_init(phba, lpfc_vmid_priority_tagging);
 	if (phba->sli_rev != LPFC_SLI_REV4)
 		phba->cfg_EnableXLane = 0;
 	lpfc_XLanePriority_init(phba, lpfc_XLanePriority);
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/322892d1/attachment-0001.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:05 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:05 +0530
Subject: [RFC v2 10/18] lpfc: vmid: vmid resource allocation
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-11-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch allocates the resource for vmid and checks if the firmware
supports the feature or not.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_init.c | 64 +++++++++++++++++++++++++++++++++++
 drivers/scsi/lpfc/lpfc_mbox.c |  6 ++++
 drivers/scsi/lpfc/lpfc_sli.c  |  9 +++++
 3 files changed, 79 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index ca25e54bb782..e32d69515586 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -4284,6 +4284,62 @@ lpfc_get_wwpn(struct lpfc_hba *phba)
 		return rol64(wwn, 32);
 }
 
+/**
+ * lpfc_vmid_res_alloc - Allocates resources for VMID
+ * @phba: pointer to lpfc hba data structure.
+ * @vport: pointer to vport data structure
+ *
+ * This routine allocated the resources needed for the vmid.
+ *
+ * Return codes
+ *	0 on Succeess
+ *	Non-0 on Failure
+ */
+u8
+lpfc_vmid_res_alloc(struct lpfc_hba *phba, struct lpfc_vport *vport)
+{
+	u16 i;
+
+	/* vmid feature is supported only on SLI4 */
+	if (phba->sli_rev == LPFC_SLI_REV3) {
+		phba->cfg_vmid_app_header = 0;
+		phba->cfg_vmid_priority_tagging = 0;
+	}
+
+	/* if enabled, then allocated the resources */
+	if (lpfc_is_vmid_enabled(phba)) {
+		vport->vmid =
+		    kmalloc_array(phba->cfg_max_vmid, sizeof(struct lpfc_vmid),
+				  GFP_KERNEL);
+		if (!vport->vmid)
+			return 1;
+
+		memset(vport->vmid, 0,
+		       phba->cfg_max_vmid * sizeof(struct lpfc_vmid));
+
+		rwlock_init(&vport->vmid_lock);
+
+		/* setting the VMID parameters for the vport */
+		vport->vmid_priority_tagging = phba->cfg_vmid_priority_tagging;
+		vport->vmid_inactivity_timeout =
+		    phba->cfg_vmid_inactivity_timeout;
+		vport->max_vmid = phba->cfg_max_vmid;
+		vport->cur_vmid_cnt = 0;
+
+		for (i = 0; i < LPFC_VMID_HASH_SIZE; i++)
+			vport->hash_table[i] = NULL;
+
+		vport->vmid_priority_range = bitmap_zalloc
+			(LPFC_VMID_MAX_PRIORITY_RANGE, GFP_KERNEL);
+
+		if (!vport->vmid_priority_range) {
+			kfree(vport->vmid);
+			return 1;
+		}
+	}
+	return 0;
+}
+
 /**
  * lpfc_create_port - Create an FC port
  * @phba: pointer to lpfc hba data structure.
@@ -4439,6 +4495,12 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev)
 			vport->port_type, shost->sg_tablesize,
 			phba->cfg_scsi_seg_cnt, phba->cfg_sg_seg_cnt);
 
+	/* allocate the resources for vmid */
+	rc = lpfc_vmid_res_alloc(phba, vport);
+
+	if (rc)
+		goto out;
+
 	/* Initialize all internally managed lists. */
 	INIT_LIST_HEAD(&vport->fc_nodes);
 	INIT_LIST_HEAD(&vport->rcv_buffer_list);
@@ -4463,6 +4525,8 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev)
 	return vport;
 
 out_put_shost:
+	kfree(vport->vmid);
+	bitmap_free(vport->vmid_priority_range);
 	scsi_host_put(shost);
 out:
 	return NULL;
diff --git a/drivers/scsi/lpfc/lpfc_mbox.c b/drivers/scsi/lpfc/lpfc_mbox.c
index 3414ffcb26fe..78a9b9baecf3 100644
--- a/drivers/scsi/lpfc/lpfc_mbox.c
+++ b/drivers/scsi/lpfc/lpfc_mbox.c
@@ -2100,6 +2100,12 @@ lpfc_request_features(struct lpfc_hba *phba, struct lpfcMboxq *mboxq)
 		bf_set(lpfc_mbx_rq_ftr_rq_iaab, &mboxq->u.mqe.un.req_ftrs, 0);
 		bf_set(lpfc_mbx_rq_ftr_rq_iaar, &mboxq->u.mqe.un.req_ftrs, 0);
 	}
+
+	/* Enable Application Services Header for apphedr VMID */
+	if (phba->cfg_vmid_app_header) {
+		bf_set(lpfc_mbx_rq_ftr_rq_ashdr, &mboxq->u.mqe.un.req_ftrs, 1);
+		bf_set(lpfc_ftr_ashdr, &phba->sli4_hba.sli4_flags, 1);
+	}
 	return;
 }
 
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 4cd7ded656b7..51b99b7beaf9 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -7558,6 +7558,15 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba)
 		goto out_free_mbox;
 	}
 
+	/* Disable vmid if app header is not supported */
+	if (phba->cfg_vmid_app_header && !(bf_get(lpfc_mbx_rq_ftr_rsp_ashdr,
+						  &mqe->un.req_ftrs))) {
+		bf_set(lpfc_ftr_ashdr, &phba->sli4_hba.sli4_flags, 0);
+		phba->cfg_vmid_app_header = 0;
+		lpfc_printf_log(phba, KERN_DEBUG, LOG_SLI,
+				"1242 vmid feature not supported");
+	}
+
 	/*
 	 * The port must support FCP initiator mode as this is the
 	 * only mode running in the host.
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/04bdb67b/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:06 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:06 +0530
Subject: [RFC v2 11/18] lpfc: vmid: cleanup vmid resources
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-12-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

The patch cleans up the vmid resources and stops the timer.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_init.c |  4 ++++
 drivers/scsi/lpfc/lpfc_scsi.c | 21 +++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index e32d69515586..88777875f4b8 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -2843,6 +2843,10 @@ lpfc_cleanup(struct lpfc_vport *vport)
 	if (phba->link_state > LPFC_LINK_DOWN)
 		lpfc_port_link_failure(vport);
 
+	/* cleanup vmid resources */
+	if (lpfc_is_vmid_enabled(phba))
+		lpfc_vmid_vport_cleanup(vport);
+
 	list_for_each_entry_safe(ndlp, next_ndlp, &vport->fc_nodes, nlp_listp) {
 		if (!NLP_CHK_NODE_ACT(ndlp)) {
 			ndlp = lpfc_enable_node(vport, ndlp,
diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index 5e802c8b22a9..7bc1fd69b715 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -4711,6 +4711,27 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd)
 	return 0;
 }
 
+/*
+ * lpfc_vmid_vport_cleanup - cleans up the resources associated with a vports
+ * @vport: The virtual port for which this call is being executed.
+ */
+void lpfc_vmid_vport_cleanup(struct lpfc_vport *vport)
+{
+	/* delete the timer */
+	if (vport->port_type == LPFC_PHYSICAL_PORT)
+		del_timer_sync(&vport->phba->inactive_vmid_poll);
+
+	/* free the resources */
+	kfree(vport->qfpa_res);
+	kfree(vport->vmid_priority.vmid_range);
+	kfree(vport->vmid);
+
+	/* reset variables */
+	vport->qfpa_res = NULL;
+	vport->vmid_priority.vmid_range = NULL;
+	vport->vmid = NULL;
+	vport->cur_vmid_cnt = 0;
+}
 
 /**
  * lpfc_abort_handler - scsi_host_template eh_abort_handler entry point
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/26a188fd/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:07 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:07 +0530
Subject: [RFC v2 12/18] lpfc: vmid: Implements ELS commands for appid patch
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-13-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch implements ELS command like QFPA and UVEM for the priority
tagging appid support. Other supporting functions are also part of this
patch.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_els.c | 356 ++++++++++++++++++++++++++++++++++-
 1 file changed, 349 insertions(+), 7 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index f4e274eb6c9c..4f61d2edb89b 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -25,6 +25,7 @@
 #include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/interrupt.h>
+#include <linux/delay.h>
 
 #include <scsi/scsi.h>
 #include <scsi/scsi_device.h>
@@ -55,6 +56,8 @@ static int lpfc_issue_els_fdisc(struct lpfc_vport *vport,
 				struct lpfc_nodelist *ndlp, uint8_t retry);
 static int lpfc_issue_fabric_iocb(struct lpfc_hba *phba,
 				  struct lpfc_iocbq *iocb);
+static void lpfc_cmpl_els_uvem(struct lpfc_hba *, struct lpfc_iocbq *,
+			       struct lpfc_iocbq *);
 
 static int lpfc_max_els_tries = 3;
 
@@ -316,12 +319,12 @@ lpfc_prep_els_iocb(struct lpfc_vport *vport, uint8_t expectRsp,
 	if (expectRsp) {
 		/* Xmit ELS command <elsCmd> to remote NPORT <did> */
 		lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS,
-				 "0116 Xmit ELS command x%x to remote "
-				 "NPORT x%x I/O tag: x%x, port state:x%x "
-				 "rpi x%x fc_flag:x%x\n",
+				 "0116 Xmit ELS command x%x to remote\n"
+				 "NPORT x%x I/O tag: x%x, port state:x%x\n"
+				 "rpi x%x fc_flag:x%x nlp_flag:x%x vport:x%p\n",
 				 elscmd, did, elsiocb->iotag,
 				 vport->port_state, ndlp->nlp_rpi,
-				 vport->fc_flag);
+				 vport->fc_flag, ndlp->nlp_flag, vport);
 	} else {
 		/* Xmit ELS response <elsCmd> to remote NPORT <did> */
 		lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS,
@@ -1116,12 +1119,16 @@ lpfc_cmpl_els_flogi(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 
 	/* FLOGI completes successfully */
 	lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS,
-			 "0101 FLOGI completes successfully, I/O tag:x%x, "
-			 "xri x%x Data: x%x x%x x%x x%x x%x %x\n",
+			 "0101 FLOGI completes successfully, I/O tag:x%x,\n"
+			 "xri x%x Data: x%x x%x x%x x%x x%x %x %x\n",
 			 cmdiocb->iotag, cmdiocb->sli4_xritag,
 			 irsp->un.ulpWord[4], sp->cmn.e_d_tov,
 			 sp->cmn.w2.r_a_tov, sp->cmn.edtovResolution,
-			 vport->port_state, vport->fc_flag);
+			 vport->port_state, vport->fc_flag,
+			 sp->cmn.priority_tagging);
+
+	if (sp->cmn.priority_tagging)
+		vport->vmid_flag |= LPFC_VMID_ISSUE_QFPA;
 
 	if (vport->port_state == LPFC_FLOGI) {
 		/*
@@ -1302,6 +1309,18 @@ lpfc_issue_els_flogi(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
 	if (sp->cmn.fcphHigh < FC_PH3)
 		sp->cmn.fcphHigh = FC_PH3;
 
+	/* to deterine if switch supports priority tagging */
+	if (phba->cfg_vmid_priority_tagging) {
+		sp->cmn.priority_tagging = 1;
+		/* lpfc_vmid_host_uuid is combination of wwpn and wwnn */
+		if (vport->lpfc_vmid_host_uuid[0] == 0) {
+			memcpy(vport->lpfc_vmid_host_uuid, phba->wwpn,
+			       sizeof(phba->wwpn));
+			memcpy(&vport->lpfc_vmid_host_uuid[8], phba->wwnn,
+			       sizeof(phba->wwnn));
+		}
+	}
+
 	if  (phba->sli_rev == LPFC_SLI_REV4) {
 		if (bf_get(lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) ==
 		    LPFC_SLI_INTF_IF_TYPE_0) {
@@ -1999,6 +2018,7 @@ lpfc_cmpl_els_plogi(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 	struct lpfc_nodelist *ndlp;
 	struct lpfc_dmabuf *prsp;
 	int disc;
+	struct serv_parm *sp = NULL;
 
 	/* we pass cmdiocb to state machine which needs rspiocb as well */
 	cmdiocb->context_un.rsp_iocb = rspiocb;
@@ -2074,6 +2094,23 @@ lpfc_cmpl_els_plogi(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 				   cmdiocb->context2)->list.next,
 				  struct lpfc_dmabuf, list);
 		ndlp = lpfc_plogi_confirm_nport(phba, prsp->virt, ndlp);
+
+		sp = (struct serv_parm *)((u8 *)prsp->virt +
+					  sizeof(u32));
+
+		ndlp->vmid_support = 0;
+		if ((phba->cfg_vmid_app_header && sp->cmn.app_hdr_support) ||
+		    (phba->cfg_vmid_priority_tagging &&
+		     sp->cmn.priority_tagging)) {
+			lpfc_printf_log(phba, KERN_DEBUG, LOG_ELS,
+					"4018 app_hdr_support %d tagging %d DID x%x",
+					sp->cmn.app_hdr_support,
+					sp->cmn.priority_tagging,
+					ndlp->nlp_DID);
+			/* if the dest port supports VMID, mark it in ndlp */
+			ndlp->vmid_support = 1;
+		}
+
 		lpfc_disc_state_machine(vport, ndlp, cmdiocb,
 					     NLP_EVT_CMPL_PLOGI);
 	}
@@ -2192,6 +2229,14 @@ lpfc_issue_els_plogi(struct lpfc_vport *vport, uint32_t did, uint8_t retry)
 	memset(sp->un.vendorVersion, 0, sizeof(sp->un.vendorVersion));
 	sp->cmn.bbRcvSizeMsb &= 0xF;
 
+	/* check if the destination port supports VMID */
+	ndlp->vmid_support = 0;
+	if (vport->vmid_priority_tagging)
+		sp->cmn.priority_tagging = 1;
+	else if (phba->cfg_vmid_app_header &&
+		 bf_get(lpfc_ftr_ashdr, &phba->sli4_hba.sli4_flags))
+		sp->cmn.app_hdr_support = 1;
+
 	lpfc_debugfs_disc_trc(vport, LPFC_DISC_TRC_ELS_CMD,
 		"Issue PLOGI:     did:x%x",
 		did, 0, 0);
@@ -10162,3 +10207,300 @@ lpfc_sli_abts_recover_port(struct lpfc_vport *vport,
 	lpfc_unreg_rpi(vport, ndlp);
 }
 
+void lpfc_init_cs_ctl_bitmap(struct lpfc_vport *vport)
+{
+	bitmap_zero(vport->vmid_priority_range, LPFC_VMID_MAX_PRIORITY_RANGE);
+}
+
+void
+lpfc_vmid_set_cs_ctl_range(struct lpfc_vport *vport, u32 min, u32 max)
+{
+	u32 i;
+
+	if ((min > max) || (max > LPFC_VMID_MAX_PRIORITY_RANGE))
+		return;
+
+	for (i = min; i <= max; i++)
+		set_bit(i, vport->vmid_priority_range);
+}
+
+void lpfc_vmid_put_cs_ctl(struct lpfc_vport *vport, u32 ctcl_vmid)
+{
+	set_bit(ctcl_vmid, vport->vmid_priority_range);
+}
+
+u32 lpfc_vmid_get_cs_ctl(struct lpfc_vport *vport)
+{
+	u32 i;
+
+	i = find_first_bit(vport->vmid_priority_range,
+			   LPFC_VMID_MAX_PRIORITY_RANGE);
+
+	if (i == LPFC_VMID_MAX_PRIORITY_RANGE)
+		return 0;
+
+	clear_bit(i, vport->vmid_priority_range);
+	return i;
+}
+
+#define MAX_PRIORITY_DESC	255
+
+static void
+lpfc_cmpl_els_qfpa(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
+		   struct lpfc_iocbq *rspiocb)
+{
+	struct lpfc_vport *vport = cmdiocb->vport;
+	struct priority_range_desc *desc;
+	struct lpfc_dmabuf *prsp = NULL;
+	struct lpfc_vmid_priority_range *vmid_range = NULL;
+	u32 *data;
+	struct lpfc_dmabuf *dmabuf = cmdiocb->context2;
+	IOCB_t *irsp = &rspiocb->iocb;
+	u8 *pcmd;
+	u32 len, i;
+
+	prsp = list_get_first(&dmabuf->list, struct lpfc_dmabuf, list);
+	if (!prsp)
+		goto out;
+
+	pcmd = prsp->virt;
+	data = (u32 *)pcmd;
+	if (*((u32 *)(pcmd)) == ELS_CMD_LS_RJT) {
+		lpfc_printf_vlog(vport, KERN_WARNING, LOG_SLI,
+				 "6528 QFPA LS_RJT %x  %x ", data[0], data[1]);
+		goto out;
+	}
+	if (irsp->ulpStatus) {
+		lpfc_printf_vlog(vport, KERN_ERR, LOG_SLI,
+				 "6529 QFPA failed with status %x  %x ",
+				 irsp->ulpStatus, irsp->un.ulpWord[4]);
+		goto out;
+	}
+
+	if (!vport->qfpa_res) {
+		vport->qfpa_res = kmalloc(FCELSSIZE, GFP_KERNEL);
+		if (!vport->qfpa_res)
+			goto out;
+		memset(vport->qfpa_res, 0, FCELSSIZE);
+	}
+
+	len = *((u32 *)(pcmd + 4));
+	len = be32_to_cpu(len);
+	memcpy(vport->qfpa_res, pcmd, len + 8);
+	len = len / LPFC_PRIORITY_RANGE_DESC_SIZE;
+
+	desc = (struct priority_range_desc *)(pcmd + 8);
+	vmid_range = vport->vmid_priority.vmid_range;
+	if (!vmid_range) {
+		vmid_range = kmalloc_array(MAX_PRIORITY_DESC,
+					   sizeof
+					   (struct lpfc_vmid_priority_range),
+					   GFP_KERNEL);
+		if (!vmid_range)
+			goto out;
+		memset(vmid_range, 0, MAX_PRIORITY_DESC *
+		       sizeof(struct lpfc_vmid_priority_range));
+		vport->vmid_priority.vmid_range = vmid_range;
+	}
+	vport->vmid_priority.num_descriptors = len;
+
+	for (i = 0; i < len; i++, vmid_range++, desc++) {
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_ELS,
+				 "6539 vmid values low=%d, high=%d, qos=%d,\n"
+				 " local ve id=%d\n", desc->lo_range,
+				 desc->hi_range, desc->qos_priority,
+				 desc->local_ve_id);
+
+		vmid_range->low = desc->lo_range << 1;
+		if (desc->local_ve_id == QFPA_ODD_ONLY)
+			vmid_range->low++;
+		if (desc->qos_priority)
+			vport->vmid_flag |= LPFC_VMID_QOS_ENABLED;
+		vmid_range->qos = desc->qos_priority;
+
+		vmid_range->high = desc->hi_range << 1;
+		if ((desc->local_ve_id == QFPA_ODD_ONLY) ||
+		    (desc->local_ve_id == QFPA_EVEN_ODD))
+			vmid_range->high++;
+	}
+	lpfc_init_cs_ctl_bitmap(vport);
+	for (i = 0; i < vport->vmid_priority.num_descriptors; i++) {
+		lpfc_vmid_set_cs_ctl_range(vport,
+				vport->vmid_priority.vmid_range[i].low,
+				vport->vmid_priority.vmid_range[i].high);
+	}
+
+	vport->vmid_flag |= LPFC_VMID_QFPA_CMPL;
+ out:
+	lpfc_els_free_iocb(phba, cmdiocb);
+}
+
+int lpfc_issue_els_qfpa(struct lpfc_vport *vport)
+{
+	struct lpfc_hba *phba = vport->phba;
+	IOCB_t *icmd;
+	struct lpfc_nodelist *ndlp;
+	struct lpfc_iocbq *elsiocb;
+	struct lpfc_sli *psli;
+	u8 *pcmd;
+	int ret;
+
+	psli = &phba->sli;
+
+	ndlp = lpfc_findnode_did(phba->pport, Fabric_DID);
+	if (ndlp && !NLP_CHK_NODE_ACT(ndlp))
+		return 1;
+
+	if (!ndlp)
+		return 1;
+
+	elsiocb = lpfc_prep_els_iocb(vport, 1, LPFC_QFPA_SIZE, 2, ndlp,
+				     ndlp->nlp_DID, ELS_CMD_QFPA);
+	if (!elsiocb)
+		return 1;
+
+	icmd = &elsiocb->iocb;
+	pcmd = (u8 *)(((struct lpfc_dmabuf *)elsiocb->context2)->virt);
+
+	*((u32 *)(pcmd)) = ELS_CMD_QFPA;
+	pcmd += 4;
+
+	elsiocb->iocb_cmpl = lpfc_cmpl_els_qfpa;
+	ret = lpfc_sli_issue_iocb(phba, LPFC_ELS_RING, elsiocb, 2);
+	if (ret != IOCB_SUCCESS) {
+		lpfc_els_free_iocb(phba, elsiocb);
+		return 1;
+	}
+	vport->vmid_flag &= ~LPFC_VMID_QOS_ENABLED;
+	return 0;
+}
+
+int
+lpfc_vmid_uvem(struct lpfc_vport *vport,
+	       struct lpfc_vmid *vmid, bool instantiated)
+{
+	struct lpfc_vem_id_desc *vem_id_desc;
+	struct lpfc_nodelist *ndlp;
+	IOCB_t *icmd;
+	struct lpfc_iocbq *elsiocb;
+	struct instantiated_ve_desc *inst_desc;
+	struct lpfc_vmid_context *vmid_context;
+	u8 *pcmd;
+	u32 *len;
+	int ret = 0;
+
+	ndlp = lpfc_findnode_did(vport, Fabric_DID);
+	if (ndlp && !NLP_CHK_NODE_ACT(ndlp))
+		return 1;
+
+	vmid_context = kmalloc(sizeof(*vmid_context), GFP_KERNEL);
+	if (!vmid_context)
+		return 1;
+	elsiocb = lpfc_prep_els_iocb(vport, 1, LPFC_UVEM_SIZE, 2,
+				     ndlp, Fabric_DID, ELS_CMD_UVEM);
+	if (!elsiocb)
+		goto out;
+
+	lpfc_printf_vlog(vport, KERN_DEBUG, LOG_ELS,
+			 "3427 %s %d", vmid->host_vmid, instantiated);
+	vmid_context->vmp = vmid;
+	vmid_context->nlp = ndlp;
+	vmid_context->instantiated = instantiated;
+	elsiocb->vmid_tag.vmid_context = vmid_context;
+	icmd = &elsiocb->iocb;
+	pcmd = (u8 *)(((struct lpfc_dmabuf *)elsiocb->context2)->virt);
+
+	if (vport->lpfc_vmid_host_uuid[0] == 0)
+		memcpy(vport->lpfc_vmid_host_uuid, vmid->host_vmid,
+		       LPFC_COMPRESS_VMID_SIZE);
+
+	*((u32 *)(pcmd)) = ELS_CMD_UVEM;
+	len = (u32 *)(pcmd + 4);
+	*len = cpu_to_be32(LPFC_UVEM_SIZE - 8);
+
+	vem_id_desc = (struct lpfc_vem_id_desc *)(pcmd + 8);
+	vem_id_desc->tag = be32_to_cpu(VEM_ID_DESC_TAG);
+	vem_id_desc->length = be32_to_cpu(LPFC_UVEM_VEM_ID_DESC_SIZE);
+	memcpy(vem_id_desc->vem_id, vport->lpfc_vmid_host_uuid,
+	       LPFC_COMPRESS_VMID_SIZE);
+
+	inst_desc = (struct instantiated_ve_desc *)(pcmd + 32);
+	inst_desc->tag = be32_to_cpu(INSTANTIATED_VE_DESC_TAG);
+	inst_desc->length = be32_to_cpu(LPFC_UVEM_VE_MAP_DESC_SIZE);
+	memcpy(inst_desc->global_vem_id, vmid->host_vmid,
+	       LPFC_COMPRESS_VMID_SIZE);
+
+	bf_set(lpfc_instantiated_nport_id, inst_desc, vport->fc_myDID);
+	bf_set(lpfc_instantiated_local_id, inst_desc,
+	       vmid->un.cs_ctl_vmid);
+	if (instantiated) {
+		inst_desc->tag = be32_to_cpu(INSTANTIATED_VE_DESC_TAG);
+	} else {
+		inst_desc->tag = be32_to_cpu(DEINSTANTIATED_VE_DESC_TAG);
+		lpfc_vmid_put_cs_ctl(vport, vmid->un.cs_ctl_vmid);
+	}
+	inst_desc->word6 = cpu_to_be32(inst_desc->word6);
+
+	elsiocb->iocb_cmpl = lpfc_cmpl_els_uvem;
+	ret = lpfc_sli_issue_iocb(vport->phba, LPFC_ELS_RING, elsiocb, 0);
+	if (ret != IOCB_SUCCESS) {
+		lpfc_els_free_iocb(vport->phba, elsiocb);
+		goto out;
+	}
+
+	return 0;
+ out:
+	kfree(vmid_context);
+	return 1;
+}
+
+static void
+lpfc_cmpl_els_uvem(struct lpfc_hba *phba, struct lpfc_iocbq *icmdiocb,
+		   struct lpfc_iocbq *rspiocb)
+{
+	struct lpfc_vport *vport = icmdiocb->vport;
+	struct lpfc_dmabuf *prsp = NULL;
+	struct lpfc_vmid_context *vmid_context =
+	    icmdiocb->vmid_tag.vmid_context;
+	struct lpfc_nodelist *ndlp = icmdiocb->context1;
+	u8 *pcmd;
+	u32 *data;
+	IOCB_t *irsp = &rspiocb->iocb;
+	struct lpfc_dmabuf *dmabuf = icmdiocb->context2;
+	struct lpfc_vmid *vmid;
+
+	vmid = vmid_context->vmp;
+	if (ndlp && !NLP_CHK_NODE_ACT(ndlp))
+		ndlp = NULL;
+
+	prsp = list_get_first(&dmabuf->list, struct lpfc_dmabuf, list);
+	if (!prsp)
+		goto out;
+	pcmd = prsp->virt;
+	data = (u32 *)pcmd;
+	if (*((u32 *)(pcmd)) == ELS_CMD_LS_RJT) {
+		lpfc_printf_vlog(vport, KERN_WARNING, LOG_SLI,
+				 "4532 UVEM LS_RJT %x %x ", data[0], data[1]);
+		goto out;
+	}
+	if (irsp->ulpStatus) {
+		lpfc_printf_vlog(vport, KERN_WARNING, LOG_SLI,
+				 "4533 UVEM error status %x: %x ",
+				 irsp->ulpStatus, irsp->un.ulpWord[4]);
+		goto out;
+	}
+	spin_lock(&phba->hbalock);
+	/* Set IN USE flag */
+	vport->vmid_flag |= LPFC_VMID_IN_USE;
+	phba->pport->vmid_flag |= LPFC_VMID_IN_USE;
+	spin_unlock(&phba->hbalock);
+
+	if (vmid_context->instantiated) {
+		vmid->flag |= LPFC_VMID_REGISTERED;
+		vmid->flag &= ~LPFC_VMID_REQ_REGISTER;
+	}
+
+ out:
+	kfree(vmid_context);
+	lpfc_els_free_iocb(phba, icmdiocb);
+}
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/33d32ceb/attachment-0001.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:08 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:08 +0530
Subject: [RFC v2 13/18] lpfc: vmid: Functions to manage vmids
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-14-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch contains the routines to save, retrieve and remove the vmids
from the data structure. A hash table is used to save the vmids and
the corresponding UUIDs associated with the application/VMs.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_scsi.c | 139 ++++++++++++++++++++++++++++++++++
 1 file changed, 139 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index 7bc1fd69b715..e5a1056cc575 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -28,6 +28,7 @@
 #include <asm/unaligned.h>
 #include <linux/t10-pi.h>
 #include <linux/crc-t10dif.h>
+#include <linux/blk-cgroup.h>
 #include <net/checksum.h>
 
 #include <scsi/scsi.h>
@@ -4485,6 +4486,144 @@ void lpfc_poll_timeout(struct timer_list *t)
 	}
 }
 
+/*
+ * lpfc_get_vmid_from_hastable - search the UUID in the hash table
+ * @vport: The virtual port for which this call is being executed.
+ * @hash: calculated hash value
+ * @buf: uuid associated with the VE
+ * Returns the vmid entry associated with the UUID
+ * Make sure to acquire the appropriate lock before invoking this routine.
+ */
+struct lpfc_vmid *lpfc_get_vmid_from_hastable(struct lpfc_vport *vport,
+					      u32 hash, u8 *buf)
+{
+	struct lpfc_vmid *vmp;
+	u16 count = 0;
+
+	while (count < LPFC_VMID_HASH_SIZE) {
+		vmp = vport->hash_table[hash];
+		if (vmp) {
+			if (strncmp(&vmp->host_vmid[0], buf, 16) == 0)
+				return vmp;
+		} else {
+			return NULL;
+		}
+		/* search the next available slot and continue till entry */
+		/* is found */
+		count++;
+		hash++;
+
+		/* or the end is reached */
+		if (hash == LPFC_VMID_HASH_SIZE)
+			hash = 0;
+	}
+	return NULL;
+}
+
+/*
+ * lpfc_put_vmid_from_hastable - put the VMID in the hash table
+ * @vport: The virtual port for which this call is being executed.
+ * @hash - calculated hash value
+ * @vmp: Pointer to a VMID entry representing a VM sending IO
+ *
+ * This routine will insert the newly acquired vmid entity in the hash table.
+ * Make sure to acquire the appropriate lock before invoking this routine.
+ */
+int
+lpfc_put_vmid_in_hashtable(struct lpfc_vport *vport, u32 hash,
+			   struct lpfc_vmid *vmp)
+{
+	int count = 0;
+
+	while (count < LPFC_VMID_HASH_SIZE) {
+		if (!vport->hash_table[hash]) {
+			vport->hash_table[hash] = vmp;
+			vmp->hash_index = hash;
+			return 1;
+		}
+		/* if the slot is already occupied, a collision has occurred. */
+		/* Store in the next available slot */
+		count++;
+		hash++;
+		/* table is full */
+		if (hash == LPFC_VMID_HASH_SIZE)
+			hash = 0;
+	}
+	return 0;
+}
+
+/*
+ * lpfc_vmid_hash_fn- creates a hash value of the UUID
+ * @uuid: uuid associated with the VE
+ * @len: length of the UUID
+ * Returns the calculated hash value
+ */
+int lpfc_vmid_hash_fn(char *vmid, int len)
+{
+	int c;
+	int hash = 0;
+
+	if (len == 0)
+		return 0;
+	while (len--) {
+		c = *vmid++;
+		if (c >= 'A' && c <= 'Z')
+			c += 'a' - 'A';
+
+		hash = (hash + (c << LPFC_VMID_HASH_SHIFT) +
+			(c >> LPFC_VMID_HASH_SHIFT)) * 19;
+	}
+
+	return hash & LPFC_VMID_HASH_MASK;
+}
+
+/*
+ * lpfc_vmid_update_entry - update the vmid entry in the hash table
+ * @vport: The virtual port for which this call is being executed.
+ * @cmd: address of scsi cmmd descriptor
+ * @vmp: Pointer to a VMID entry representing a VM sending IO
+ * @tag: VMID tag
+ */
+void lpfc_vmid_update_entry(struct lpfc_vport *vport, struct scsi_cmnd
+				   *cmd, struct lpfc_vmid *vmp,
+				   union lpfc_vmid_io_tag *tag)
+{
+	u64 *lta;
+
+	if (vport->vmid_priority_tagging)
+		tag->cs_ctl_vmid = vmp->un.cs_ctl_vmid;
+	else
+		tag->app_id = vmp->un.app_id;
+
+	if (cmd->sc_data_direction == DMA_TO_DEVICE)
+		vmp->io_wr_cnt++;
+	else
+		vmp->io_rd_cnt++;
+
+	/* update the last access timestamp in the table */
+	lta = per_cpu_ptr(vmp->last_io_time, raw_smp_processor_id());
+	*lta = jiffies;
+}
+
+void lpfc_vmid_assign_cs_ctl(struct lpfc_vport *vport, struct lpfc_vmid *vmid)
+{
+	u32 hash;
+	struct lpfc_vmid *pvmid;
+
+	if (vport->port_type == LPFC_PHYSICAL_PORT) {
+		vmid->un.cs_ctl_vmid = lpfc_vmid_get_cs_ctl(vport);
+	} else {
+		hash = lpfc_vmid_hash_fn(vmid->host_vmid, vmid->vmid_len);
+		pvmid =
+		    lpfc_get_vmid_from_hastable(vport->phba->pport, hash,
+						vmid->host_vmid);
+		if (!pvmid)
+			vmid->un.cs_ctl_vmid = pvmid->un.cs_ctl_vmid;
+		else
+			vmid->un.cs_ctl_vmid = lpfc_vmid_get_cs_ctl(vport);
+	}
+}
+
 /**
  * lpfc_queuecommand - scsi_host_template queuecommand entry point
  * @cmnd: Pointer to scsi_cmnd data structure.
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/d91b5edb/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:09 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:09 +0530
Subject: [RFC v2 14/18] lpfc: vmid: Implements CT commands for appid.
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-15-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

The patch implements CT commands for registering and deregistering the
appid for the application. Also, a small change in decrementing the ndlp
ref counter has been added.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
Removed redundant code due to changes since last submit
---
 drivers/scsi/lpfc/lpfc_ct.c | 249 ++++++++++++++++++++++++++++++++++++
 1 file changed, 249 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_ct.c b/drivers/scsi/lpfc/lpfc_ct.c
index c201686d3815..691f78340f26 100644
--- a/drivers/scsi/lpfc/lpfc_ct.c
+++ b/drivers/scsi/lpfc/lpfc_ct.c
@@ -3406,3 +3406,252 @@ lpfc_decode_firmware_rev(struct lpfc_hba *phba, char *fwrevision, int flag)
 	}
 	return;
 }
+
+void
+lpfc_cmpl_ct_cmd_vmid(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
+		      struct lpfc_iocbq *rspiocb)
+{
+	struct lpfc_vport *vport = cmdiocb->vport;
+	struct lpfc_dmabuf *inp = cmdiocb->context1;
+	struct lpfc_dmabuf *outp = cmdiocb->context2;
+	struct lpfc_sli_ct_request *ctcmd = inp->virt;
+	struct lpfc_sli_ct_request *ctrsp = outp->virt;
+	u16 rsp = ctrsp->CommandResponse.bits.CmdRsp;
+	struct app_id_object *app;
+	u32 cmd, hash;
+	struct lpfc_vmid *vmp;
+	u8 *data = outp->virt;
+	int i;
+
+	cmd = be16_to_cpu(ctcmd->CommandResponse.bits.CmdRsp);
+	if (cmd == SLI_CTAS_DALLAPP_ID)
+		lpfc_ct_free_iocb(phba, cmdiocb);
+
+	if (lpfc_els_chk_latt(vport) || rspiocb->iocb.ulpStatus) {
+		if (cmd != SLI_CTAS_DALLAPP_ID)
+			return;
+	}
+	/* Check for a CT LS_RJT response */
+	if (rsp == be16_to_cpu(SLI_CT_RESPONSE_FS_RJT)) {
+		if (cmd != SLI_CTAS_DALLAPP_ID)
+			lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+					 "3306 VMID FS_RJT Data: x%x  %x %x\n",
+				 cmd, ctrsp->ReasonCode, ctrsp->Explanation);
+		if ((cmd != SLI_CTAS_DALLAPP_ID) ||
+		    (ctrsp->ReasonCode != SLI_CT_UNABLE_TO_PERFORM_REQ) ||
+		    (ctrsp->Explanation != SLI_CT_APP_ID_NOT_AVAILABLE)) {
+			/* If DALLAPP_ID failed retry later */
+			if (cmd == SLI_CTAS_DALLAPP_ID)
+				vport->load_flag |= FC_DEREGISTER_ALL_APP_ID;
+			return;
+		}
+	}
+
+	switch (cmd) {
+	case SLI_CTAS_RAPP_IDENT:
+		app = (struct app_id_object *)(RAPP_IDENT_OFFSET + data);
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+				 "6712 RAPP_IDENT app id %d  port id %x id\n"
+				 "len %d\n", be32_to_cpu(app->app_id),
+				 be32_to_cpu(app->port_id),
+				 app->obj.entity_id_len);
+
+		if (app->obj.entity_id_len == 0 || app->port_id == 0)
+			return;
+
+		hash = lpfc_vmid_hash_fn(app->obj.entity_id,
+					 app->obj.entity_id_len);
+		vmp = lpfc_get_vmid_from_hastable(vport, hash,
+						  app->obj.entity_id);
+		if (vmp) {
+			vmp->un.app_id = be32_to_cpu(app->app_id);
+			vmp->flag |= LPFC_VMID_REGISTERED;
+			vmp->flag &= ~LPFC_VMID_REQ_REGISTER;
+			/* Set IN USE flag */
+			vport->vmid_flag |= LPFC_VMID_IN_USE;
+		} else {
+			lpfc_printf_vlog(vport, KERN_DEBUG,
+					 LOG_DISCOVERY, "6901 No entry found\n"
+					 "%s hash %d\n", app->obj.entity_id,
+					 hash);
+		}
+		break;
+	case SLI_CTAS_DAPP_IDENT:
+		app = (struct app_id_object *)(DAPP_IDENT_OFFSET + data);
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+				 "6713 DAPP_IDENT app id %d  port id %x",
+				 cpu_to_be32(app->app_id),
+				 cpu_to_be32(app->port_id));
+		break;
+	case SLI_CTAS_DALLAPP_ID:
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+				 "8856 Deregistered all app ids");
+		for (i = 0; i < phba->cfg_max_vmid; i++) {
+			vmp = &vport->vmid[i];
+			if (vmp->flag != LPFC_VMID_SLOT_FREE)
+				memset(vmp, 0, sizeof(struct lpfc_vmid));
+		}
+		for (i = 0; i < LPFC_VMID_HASH_SIZE; i++)
+			vport->hash_table[i] = NULL;
+		vport->load_flag |= FC_ALLOW_VMID;
+		break;
+	default:
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+				 "8857 Invalid command code");
+	}
+}
+
+/**
+ * lpfc_vmid_cmd - Build and send a FDMI cmd to the specified NPort
+ * @vport: pointer to a host virtual N_Port data structure.
+ * @ndlp: ndlp to send FDMI cmd to (if NULL use FDMI_DID)
+ * cmdcode: FDMI command to send
+ * mask: Mask of HBA or PORT Attributes to send
+ *
+ * Builds and sends a FDMI command using the CT subsystem.
+ */
+int
+lpfc_vmid_cmd(struct lpfc_vport *vport,
+	      int cmdcode, struct lpfc_vmid *vmid)
+{
+	struct lpfc_hba *phba = vport->phba;
+	struct lpfc_dmabuf *mp, *bmp;
+	struct lpfc_sli_ct_request *ctreq;
+	struct ulp_bde64 *bpl;
+	u32 size;
+	u32 rsp_size;
+	u8 *data;
+	struct lpfc_vmid_rapp_ident_list *rap;
+	struct lpfc_vmid_dapp_ident_list *dap;
+	u8 retry = 0;
+	struct lpfc_nodelist *ndlp;
+
+	void (*cmpl)(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
+		     struct lpfc_iocbq *rspiocb);
+
+	ndlp = lpfc_findnode_did(vport, FDMI_DID);
+	if (!ndlp || !NLP_CHK_NODE_ACT(ndlp))
+		return 0;
+
+	cmpl = lpfc_cmpl_ct_cmd_vmid;
+
+	/* fill in BDEs for command */
+	/* Allocate buffer for command payload */
+	mp = kmalloc(sizeof(*mp), GFP_KERNEL);
+	if (!mp)
+		goto vmid_free_mp_exit;
+
+	mp->virt = lpfc_mbuf_alloc(phba, 0, &mp->phys);
+	if (!mp->virt)
+		goto vmid_free_mp_virt_exit;
+
+	/* Allocate buffer for Buffer ptr list */
+	bmp = kmalloc(sizeof(*bmp), GFP_KERNEL);
+	if (!bmp)
+		goto vmid_free_bmp_exit;
+
+	bmp->virt = lpfc_mbuf_alloc(phba, 0, &bmp->phys);
+	if (!bmp->virt)
+		goto vmid_free_bmp_virt_exit;
+
+	INIT_LIST_HEAD(&mp->list);
+	INIT_LIST_HEAD(&bmp->list);
+
+	lpfc_printf_vlog(vport, KERN_INFO, LOG_DISCOVERY,
+			 "0218 VMID Request Data: x%x x%x x%x",
+			 vport->fc_flag, vport->port_state, cmdcode);
+	ctreq = (struct lpfc_sli_ct_request *)mp->virt;
+	data = mp->virt;
+	/* First populate the CT_IU preamble */
+	memset(data, 0, LPFC_BPL_SIZE);
+	ctreq->RevisionId.bits.Revision = SLI_CT_REVISION;
+	ctreq->RevisionId.bits.InId = 0;
+
+	ctreq->FsType = SLI_CT_MANAGEMENT_SERVICE;
+	ctreq->FsSubType = SLI_CT_APP_SEV_Subtypes;
+
+	ctreq->CommandResponse.bits.CmdRsp = cpu_to_be16(cmdcode);
+	rsp_size = LPFC_BPL_SIZE;
+	size = 0;
+
+	switch (cmdcode) {
+	case SLI_CTAS_RAPP_IDENT:
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+				 "1329 RAPP_IDENT for %s", vmid->host_vmid);
+		ctreq->un.PortID = cpu_to_be32(vport->fc_myDID);
+		rap = (struct lpfc_vmid_rapp_ident_list *)
+			(DAPP_IDENT_OFFSET + data);
+		rap->no_of_objects = cpu_to_be32(1);
+		rap->obj[0].entity_id_len = vmid->vmid_len;
+		memcpy(rap->obj[0].entity_id, vmid->host_vmid, vmid->vmid_len);
+		size = RAPP_IDENT_OFFSET +
+			sizeof(struct lpfc_vmid_rapp_ident_list);
+		retry = 1;
+		break;
+
+	case SLI_CTAS_GALLAPPIA_ID:
+		ctreq->un.PortID = cpu_to_be32(vport->fc_myDID);
+		size = GALLAPPIA_ID_SIZE;
+		break;
+
+	case SLI_CTAS_DAPP_IDENT:
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+				 "1469 DAPP_IDENT for %s", vmid->host_vmid);
+		ctreq->un.PortID = cpu_to_be32(vport->fc_myDID);
+		dap = (struct lpfc_vmid_dapp_ident_list *)
+			(DAPP_IDENT_OFFSET + data);
+		dap->no_of_objects = cpu_to_be32(1);
+		dap->obj[0].entity_id_len = vmid->vmid_len;
+		memcpy(dap->obj[0].entity_id, vmid->host_vmid, vmid->vmid_len);
+		size = DAPP_IDENT_OFFSET +
+			sizeof(struct lpfc_vmid_dapp_ident_list);
+		vmid->flag &= ~LPFC_VMID_REGISTERED;
+		retry = 1;
+		break;
+
+	case SLI_CTAS_DALLAPP_ID:
+		ctreq->un.PortID = cpu_to_be32(vport->fc_myDID);
+		size = DALLAPP_ID_SIZE;
+		break;
+
+	default:
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+				 "7062 VMID cmdcode x%x not supported",
+				 cmdcode);
+		goto vmid_free_all_mem;
+	}
+
+	ctreq->CommandResponse.bits.Size = cpu_to_be16(rsp_size);
+
+	bpl = (struct ulp_bde64 *)bmp->virt;
+	bpl->addrHigh = putPaddrHigh(mp->phys);
+	bpl->addrLow = putPaddrLow(mp->phys);
+	bpl->tus.f.bdeFlags = 0;
+	bpl->tus.f.bdeSize = size;
+
+	/* The lpfc_ct_cmd/lpfc_get_req shall increment ndlp reference count
+	 * to hold ndlp reference for the corresponding callback function.
+	 */
+	if (!lpfc_ct_cmd(vport, mp, bmp, ndlp, cmpl, rsp_size, retry))
+		return 0;
+
+	/* Decrement ndlp reference count to release ndlp reference held
+	 * for the failed command's callback function.
+	 */
+	lpfc_nlp_put(ndlp);
+
+ vmid_free_all_mem:
+	lpfc_mbuf_free(phba, bmp->virt, bmp->phys);
+ vmid_free_bmp_virt_exit:
+	kfree(bmp);
+ vmid_free_bmp_exit:
+	lpfc_mbuf_free(phba, mp->virt, mp->phys);
+ vmid_free_mp_virt_exit:
+	kfree(mp);
+ vmid_free_mp_exit:
+
+	/* Issue FDMI request failed */
+	lpfc_printf_vlog(vport, KERN_DEBUG, LOG_DISCOVERY,
+			 "0244 Issue FDMI request failed Data: x%x", cmdcode);
+	return 1;
+}
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/6c06439b/attachment-0001.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:10 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:10 +0530
Subject: [RFC v2 15/18] lpfc: vmid: Appends the vmid in the wqe before sending
 request
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-16-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch adds the vmid in wqe before sending out the request.
The type of vmid depends on the configured type and is checked before
being appended.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_sli.c | 56 +++++++++++++++++++++++++++++++++++-
 1 file changed, 55 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 51b99b7beaf9..53dbd6a3f460 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -3724,7 +3724,7 @@ lpfc_sli_sp_handle_rspiocb(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
 
 		if (irsp->ulpStatus) {
 			/* Rsp ring <ringno> error: IOCB */
-			lpfc_printf_log(phba, KERN_WARNING, LOG_SLI,
+			lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
 					"0328 Rsp Ring %d error: "
 					"IOCB Data: "
 					"x%x x%x x%x x%x "
@@ -9625,6 +9625,8 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 				*pcmd == ELS_CMD_RSCN_XMT ||
 				*pcmd == ELS_CMD_FDISC ||
 				*pcmd == ELS_CMD_LOGO ||
+				*pcmd == ELS_CMD_QFPA ||
+				*pcmd == ELS_CMD_UVEM ||
 				*pcmd == ELS_CMD_PLOGI)) {
 				bf_set(els_req64_sp, &wqe->els_req, 1);
 				bf_set(els_req64_sid, &wqe->els_req,
@@ -9756,6 +9758,24 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 			ptr = &wqe->words[22];
 			memcpy(ptr, fcp_cmnd, sizeof(struct fcp_cmnd));
 		}
+
+		/* add the VMID tags as per the switch */
+		if (iocbq->iocb_flag & LPFC_IO_VMID) {
+			union lpfc_wqe128 *wqe128;
+
+			if (phba->pport->vmid_priority_tagging) {
+				bf_set(wqe_ccpe, &wqe->fcp_iwrite.wqe_com, 1);
+				bf_set(wqe_ccp, &wqe->fcp_iwrite.wqe_com,
+				       (iocbq->vmid_tag.cs_ctl_vmid));
+				/* Bit 0 must be 0 */
+			} else {
+				wqe128 = (union lpfc_wqe128 *)wqe;
+				bf_set(wqe_appid, &wqe->fcp_iwrite.wqe_com, 1);
+				bf_set(wqe_wqes, &wqe128->fcp_iwrite.wqe_com,
+				       1);
+				wqe128->words[31] = iocbq->vmid_tag.app_id;
+			}
+		}
 		break;
 	case CMD_FCP_IREAD64_CR:
 		/* word3 iocb=iotag wqe=payload_offset_len */
@@ -9820,6 +9840,23 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 			ptr = &wqe->words[22];
 			memcpy(ptr, fcp_cmnd, sizeof(struct fcp_cmnd));
 		}
+
+		/* add the VMID tags as per the switch */
+		if (iocbq->iocb_flag & LPFC_IO_VMID) {
+			union lpfc_wqe128 *wqe128;
+
+			if (phba->pport->vmid_priority_tagging) {
+				bf_set(wqe_ccpe, &wqe->fcp_iread.wqe_com, 1);
+				bf_set(wqe_ccp, &wqe->fcp_iread.wqe_com,
+				       (iocbq->vmid_tag.cs_ctl_vmid));
+				/* Bit 0 must be 0 */
+			} else {
+				wqe128 = (union lpfc_wqe128 *)wqe;
+				bf_set(wqe_appid, &wqe->fcp_iread.wqe_com, 1);
+				bf_set(wqe_wqes, &wqe128->fcp_iread.wqe_com, 1);
+				wqe128->words[31] = iocbq->vmid_tag.app_id;
+			}
+		}
 		break;
 	case CMD_FCP_ICMND64_CR:
 		/* word3 iocb=iotag wqe=payload_offset_len */
@@ -9877,6 +9914,23 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 			ptr = &wqe->words[22];
 			memcpy(ptr, fcp_cmnd, sizeof(struct fcp_cmnd));
 		}
+
+		/* add the VMID tags as per the switch */
+		if (iocbq->iocb_flag & LPFC_IO_VMID) {
+			union lpfc_wqe128 *wqe128;
+
+			if (phba->pport->vmid_priority_tagging) {
+				bf_set(wqe_ccpe, &wqe->fcp_icmd.wqe_com, 1);
+				bf_set(wqe_ccp, &wqe->fcp_icmd.wqe_com,
+				       (iocbq->vmid_tag.cs_ctl_vmid));
+				/* Bit 0 must be 0 */
+			} else {
+				wqe128 = (union lpfc_wqe128 *)wqe;
+				bf_set(wqe_appid, &wqe->fcp_icmd.wqe_com, 1);
+				bf_set(wqe_wqes, &wqe128->fcp_icmd.wqe_com, 1);
+				wqe128->words[31] = iocbq->vmid_tag.app_id;
+			}
+		}
 		break;
 	case CMD_GEN_REQUEST64_CR:
 		/* For this command calculate the xmit length of the
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/8dda6a98/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:11 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:11 +0530
Subject: [RFC v2 16/18] lpfc: vmid: Timeout implementation for vmid
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-17-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch implements the timeout functionality for the vmid. After the
set time period of inactivity, the vmid is deregistered from the switch.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_hbadisc.c | 109 +++++++++++++++++++++++++++++++
 drivers/scsi/lpfc/lpfc_init.c    |  40 ++++++++++++
 2 files changed, 149 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c
index 142a02114479..b20013866942 100644
--- a/drivers/scsi/lpfc/lpfc_hbadisc.c
+++ b/drivers/scsi/lpfc/lpfc_hbadisc.c
@@ -190,6 +190,115 @@ lpfc_dev_loss_tmo_callbk(struct fc_rport *rport)
 	return;
 }
 
+/**
+ * lpfc_check_inactive_vmid_one - VMID inactivity checker for a vport
+ * @vport: Pointer to vport context object.
+ *
+ * This function checks for idle vmid entries related to a particular vport. If
+ * found unused/idle, it frees them accordingly.
+ **/
+static void lpfc_check_inactive_vmid_one(struct lpfc_vport *vport)
+{
+	u16 i, keep;
+	u32 difftime = 0, r;
+	u64 *lta;
+	int cpu;
+
+	write_lock(&vport->vmid_lock);
+
+	if (!vport->cur_vmid_cnt)
+		goto out;
+
+	/* iterate through the table */
+	for (i = 0; i < LPFC_VMID_HASH_SIZE; ++i) {
+		if (vport->hash_table[i] && (vport->hash_table[i]->flag &
+					     LPFC_VMID_REGISTERED)) {
+			/* check if the particular vmid is in use */
+			/* for all available per cpu variable */
+			for_each_possible_cpu(cpu) {
+				/* if last access time is less than timeout */
+				lta = per_cpu_ptr(
+					vport->hash_table[i]->last_io_time,
+					cpu);
+				if (!lta)
+					continue;
+				difftime = (jiffies) - (*lta);
+				if ((vport->vmid_inactivity_timeout *
+				     JIFFIES_PER_HR) > difftime) {
+					keep = 1;
+					break;
+				}
+			}
+
+			/* if none of the cpus have been used by the vm, */
+			/*  remove the entry if already registered */
+			if (!keep) {
+				/* mark the entry for deregistration */
+				vport->hash_table[i]->flag =
+					LPFC_VMID_DE_REGISTER;
+				write_unlock(&vport->vmid_lock);
+				if (vport->vmid_priority_tagging)
+					r = lpfc_vmid_uvem(vport,
+							   vport->hash_table[i],
+							   false);
+				else
+					r = lpfc_vmid_cmd(vport,
+							  SLI_CTAS_DAPP_IDENT,
+							  vport->hash_table[i]);
+
+				/* decrement number of active vms and mark */
+				/* entry in slot as free */
+				write_lock(&vport->vmid_lock);
+				if (!r) {
+					struct lpfc_vmid *ht =
+							vport->hash_table[i];
+					vport->cur_vmid_cnt--;
+					ht->flag = LPFC_VMID_SLOT_FREE;
+					free_percpu(ht->last_io_time);
+					ht->last_io_time = NULL;
+					vport->hash_table[i] = NULL;
+				}
+			}
+		}
+		keep = 0;
+	}
+ out:
+	write_unlock(&vport->vmid_lock);
+}
+
+/**
+ * lpfc_check_inactive_vmid - VMID inactivity checker
+ * @phba: Pointer to hba context object.
+ *
+ * This function is called from the worker thread to determine if an entry in
+ * the vmid table can be released since there was no IO activity seen from that
+ * particular VM for the specified time. When this happens, the entry in the
+ * table is released and also the resources on the switch cleared.
+ **/
+
+void lpfc_check_inactive_vmid(struct lpfc_hba *phba)
+{
+	struct lpfc_vport *vport;
+	struct lpfc_vport **vports;
+	int i;
+
+	vports = lpfc_create_vport_work_array(phba);
+	if (!vports)
+		return;
+
+	for (i = 0; i <= phba->max_vports; i++) {
+		if ((!vports[i]) && (i == 0))
+			vport = phba->pport;
+		else
+			vport = vports[i];
+		if (!vport)
+			break;
+
+		lpfc_check_inactive_vmid_one(vport);
+	}
+	lpfc_destroy_vport_work_array(phba, vports);
+}
+
 /**
  * lpfc_dev_loss_tmo_handler - Remote node devloss timeout handler
  * @ndlp: Pointer to remote node object.
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 88777875f4b8..7e93049fba21 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -4829,6 +4829,42 @@ lpfc_sli4_fcf_redisc_wait_tmo(struct timer_list *t)
 	lpfc_worker_wake_up(phba);
 }
 
+/**
+ * lpfc_vmid_poll - VMID timeout detection
+ * @ptr: Map to lpfc_hba data structure pointer.
+ *
+ * This routine is invoked when there is no IO on by a VM for the specified
+ * amount of time. When this situation is detected, the VMID has to be
+ * deregistered from the switch and all the local resources freed. The VMID
+ * will be reassigned to the VM once the IO begins.
+ **/
+static void
+lpfc_vmid_poll(struct timer_list *t)
+{
+	struct lpfc_hba *phba = from_timer(phba, t, inactive_vmid_poll);
+	u32 wake_up = 0;
+
+	/* check if there is a need to issue QFPA */
+	if (phba->pport->vmid_priority_tagging) {
+		wake_up = 1;
+		phba->pport->work_port_events |= WORKER_CHECK_VMID_ISSUE_QFPA;
+	}
+
+	/* Is the vmid inactivity timer enabled */
+	if (phba->pport->vmid_inactivity_timeout ||
+	    phba->pport->load_flag & FC_DEREGISTER_ALL_APP_ID) {
+		wake_up = 1;
+		phba->pport->work_port_events |= WORKER_CHECK_INACTIVE_VMID;
+	}
+
+	if (wake_up)
+		lpfc_worker_wake_up(phba);
+
+	/* restart the timer for the next iteration */
+	mod_timer(&phba->inactive_vmid_poll, jiffies + msecs_to_jiffies(1000 *
+							LPFC_VMID_TIMER));
+}
+
 /**
  * lpfc_sli4_parse_latt_fault - Parse sli4 link-attention link fault code
  * @phba: pointer to lpfc hba data structure.
@@ -6679,6 +6715,10 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
 	phba->hbqs[LPFC_ELS_HBQ].hbq_alloc_buffer = lpfc_sli4_rb_alloc;
 	phba->hbqs[LPFC_ELS_HBQ].hbq_free_buffer = lpfc_sli4_rb_free;
 
+	/* for VMID idle timeout if VMID is enabled */
+	if (lpfc_is_vmid_enabled(phba))
+		timer_setup(&phba->inactive_vmid_poll, lpfc_vmid_poll, 0);
+
 	/*
 	 * Initialize the SLI Layer to run with lpfc SLI4 HBAs.
 	 */
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/777b8a19/attachment-0001.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:12 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:12 +0530
Subject: [RFC v2 17/18] lpfc: vmid: Adding qfpa and vmid timeout check in
 worker thread
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-18-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

This patch add the periodic check for issuing of qfpa command and vmid
timeout in the worker thread. The inactivity timeout check is added via
the timer function.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
---
 drivers/scsi/lpfc/lpfc_hbadisc.c | 42 ++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c
index b20013866942..38df3c4341f9 100644
--- a/drivers/scsi/lpfc/lpfc_hbadisc.c
+++ b/drivers/scsi/lpfc/lpfc_hbadisc.c
@@ -433,6 +433,32 @@ lpfc_dev_loss_tmo_handler(struct lpfc_nodelist *ndlp)
 	return fcf_inuse;
 }
 
+void lpfc_check_vmid_qfpa_issue(struct lpfc_hba *phba)
+{
+	struct lpfc_vport *vport;
+	struct lpfc_vport **vports;
+	int i, ret;
+
+	vports = lpfc_create_vport_work_array(phba);
+	if (!vports)
+		return;
+
+	for (i = 0; i <= phba->max_vports; i++) {
+		if ((!vports[i]) && (i == 0))
+			vport = phba->pport;
+		else
+			vport = vports[i];
+		if (!vport)
+			break;
+
+		if (vport->vmid_flag & LPFC_VMID_ISSUE_QFPA) {
+			ret = lpfc_issue_els_qfpa(vport);
+			vport->vmid_flag &= ~LPFC_VMID_ISSUE_QFPA;
+		}
+	}
+	lpfc_destroy_vport_work_array(phba, vports);
+}
+
 /**
  * lpfc_sli4_post_dev_loss_tmo_handler - SLI4 post devloss timeout handler
  * @phba: Pointer to hba context object.
@@ -744,6 +770,22 @@ lpfc_work_done(struct lpfc_hba *phba)
 	if (ha_copy & HA_LATT)
 		lpfc_handle_latt(phba);
 
+	/* Handle VMID Events */
+	if (lpfc_is_vmid_enabled(phba)) {
+		if (phba->pport->work_port_events &
+		    WORKER_CHECK_VMID_ISSUE_QFPA) {
+			lpfc_check_vmid_qfpa_issue(phba);
+			phba->pport->work_port_events &=
+				~WORKER_CHECK_VMID_ISSUE_QFPA;
+		}
+		if (phba->pport->work_port_events &
+		    WORKER_CHECK_INACTIVE_VMID) {
+			lpfc_check_inactive_vmid(phba);
+			phba->pport->work_port_events &=
+			    ~WORKER_CHECK_INACTIVE_VMID;
+		}
+	}
+
 	/* Process SLI4 events */
 	if (phba->pci_dev_grp == LPFC_PCI_DEV_OC) {
 		if (phba->hba_flag & HBA_RRQ_ACTIVE)
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/c717bcee/attachment.p7s>

From muneendra.kumar at broadcom.com  Mon Oct 19 03:43:13 2020
From: muneendra.kumar at broadcom.com (Muneendra)
Date: Mon, 19 Oct 2020 13:13:13 +0530
Subject: [RFC v2 18/18] lpfc: vmid: Introducing vmid in io path.
In-Reply-To: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <1603093393-12875-19-git-send-email-muneendra.kumar@broadcom.com>

From: Gaurav Srivastava <gaurav.srivastava at broadcom.com>

The patch introduces the vmid in the io path. It checks if the vmid is
enabled and if io belongs to a vm or not and acts accordingly. Other
supporing APIs are also included in the patch.

Signed-off-by: Gaurav Srivastava  <gaurav.srivastava at broadcom.com>
Signed-off-by: James Smart <jsmart2021 at gmail.com>

---
v2:
Ported the patch on top of 5.10/scsi-queue
Added a fix for issuing QFPA command which was not included in the
last submit
---
 drivers/scsi/lpfc/lpfc_scsi.c | 165 ++++++++++++++++++++++++++++++++++
 1 file changed, 165 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index e5a1056cc575..421ec6ba981e 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -4624,6 +4624,151 @@ void lpfc_vmid_assign_cs_ctl(struct lpfc_vport *vport, struct lpfc_vmid *vmid)
 	}
 }
 
+/*
+ * lpfc_vmid_get_appid- get the vmid associated with the uuid
+ * @vport: The virtual port for which this call is being executed.
+ * @uuid: uuid associated with the VE
+ * @cmd: address of scsi cmmd descriptor
+ * @tag: VMID tag
+ * Returns status of the function
+ */
+static int lpfc_vmid_get_appid(struct lpfc_vport *vport, char *uuid, struct
+			       scsi_cmnd * cmd, union lpfc_vmid_io_tag *tag)
+{
+	struct lpfc_vmid *vmp = NULL;
+	int hash, len, rc = 1, i;
+	u8 pending = 0;
+
+	/* check if QFPA is complete */
+	if (lpfc_vmid_is_type_priority_tag(vport) && !(vport->vmid_flag &
+	      LPFC_VMID_QFPA_CMPL)) {
+		vport->work_port_events |= WORKER_CHECK_VMID_ISSUE_QFPA;
+		return 1;
+	}
+
+	/* search if the uuid has already been mapped to the vmid */
+	len = strlen(uuid);
+	hash = lpfc_vmid_hash_fn(uuid, len);
+
+	/* search for the VMID in the table */
+	read_lock(&vport->vmid_lock);
+	vmp = lpfc_get_vmid_from_hastable(vport, hash, uuid);
+	read_unlock(&vport->vmid_lock);
+
+	/* if found, check if its already registered  */
+	if (vmp  && vmp->flag & LPFC_VMID_REGISTERED) {
+		lpfc_vmid_update_entry(vport, cmd, vmp, tag);
+		rc = 0;
+	} else if (vmp && (vmp->flag & LPFC_VMID_REQ_REGISTER ||
+			   vmp->flag & LPFC_VMID_DE_REGISTER)) {
+		/* else if register or dereg request has already been sent */
+		/* Hence vmid tag will not be added for this IO */
+		rc = 1;
+	} else {
+		/* else, start the process to obtain one as per the */
+		/* switch connected */
+		write_lock(&vport->vmid_lock);
+		vmp = lpfc_get_vmid_from_hastable(vport, hash, uuid);
+
+		/* while the read lock was released, in case the entry was */
+		/* added by other context or is in process of being added */
+		if (vmp && vmp->flag & LPFC_VMID_REGISTERED) {
+			lpfc_vmid_update_entry(vport, cmd, vmp, tag);
+			write_unlock(&vport->vmid_lock);
+			return 0;
+		} else if (vmp && vmp->flag & LPFC_VMID_REQ_REGISTER) {
+			write_unlock(&vport->vmid_lock);
+			return 1;
+		}
+
+		/* else search and allocate a free slot in the hash table */
+		if (vport->cur_vmid_cnt < vport->max_vmid) {
+			for (i = 0; i < vport->max_vmid; ++i) {
+				vmp = vport->vmid + i;
+				if (vmp->flag == LPFC_VMID_SLOT_FREE) {
+					vmp = vport->vmid + i;
+					break;
+				}
+			}
+		} else {
+			write_unlock(&vport->vmid_lock);
+			return 1;
+		}
+
+		if (vmp && (vmp->flag == LPFC_VMID_SLOT_FREE)) {
+			vmp->vmid_len = len;
+
+			/* Add the vmid and register  */
+			memcpy(vmp->host_vmid, uuid, vmp->vmid_len);
+			vmp->io_rd_cnt = 0;
+			vmp->io_wr_cnt = 0;
+			vmp->flag = LPFC_VMID_SLOT_USED;
+			lpfc_put_vmid_in_hashtable(vport, hash, vmp);
+
+			vmp->delete_inactive =
+			    vport->vmid_inactivity_timeout ? 1 : 0;
+
+			/* if type priority tag, get next available vmid */
+			if (lpfc_vmid_is_type_priority_tag(vport))
+				lpfc_vmid_assign_cs_ctl(vport, vmp);
+
+			/* allocate the per cpu variable for holding */
+			/* the last access time stamp only if vmid is enabled */
+			if (!vmp->last_io_time)
+				vmp->last_io_time =
+				    __alloc_percpu(sizeof(u64),
+						   __alignof__(struct
+							       lpfc_vmid));
+
+			/* registration pending */
+			pending = 1;
+			rc = 1;
+		}
+		write_unlock(&vport->vmid_lock);
+
+		/* complete transaction with switch */
+		if (pending) {
+			if (lpfc_vmid_is_type_priority_tag(vport))
+				rc = lpfc_vmid_uvem(vport, vmp, true);
+			else
+				rc = lpfc_vmid_cmd(vport,
+						   SLI_CTAS_RAPP_IDENT,
+						   vmp);
+			if (!rc) {
+				write_lock(&vport->vmid_lock);
+				vport->cur_vmid_cnt++;
+				vmp->flag |= LPFC_VMID_REQ_REGISTER;
+				write_unlock(&vport->vmid_lock);
+			}
+		}
+
+		/* finally, enable the idle timer once */
+		if (!(vport->phba->pport->vmid_flag & LPFC_VMID_TIMER_ENBLD)) {
+			mod_timer(&vport->phba->inactive_vmid_poll,
+				  jiffies +
+				  msecs_to_jiffies(1000 * LPFC_VMID_TIMER));
+			vport->phba->pport->vmid_flag |= LPFC_VMID_TIMER_ENBLD;
+		}
+	}
+	return rc;
+}
+
+/*
+ * lpfc_is_command_vm_io - get the uuid from blk cgroup
+ * @cmd:Pointer to scsi_cmnd data structure
+ * Returns uuid if present if not null
+ */
+static char *lpfc_is_command_vm_io(struct scsi_cmnd *cmd)
+{
+	char *uuid = NULL;
+
+	if (cmd->request) {
+		if (cmd->request->bio)
+			uuid = blkcg_get_app_identifier(cmd->request->bio);
+	}
+	return uuid;
+}
+
 /**
  * lpfc_queuecommand - scsi_host_template queuecommand entry point
  * @cmnd: Pointer to scsi_cmnd data structure.
@@ -4649,6 +4794,7 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd)
 	int err, idx;
 #ifdef CONFIG_SCSI_LPFC_DEBUG_FS
 	uint64_t start = 0L;
+	u8 *uuid = NULL;
 
 	if (phba->ktime_on)
 		start = ktime_get_ns();
@@ -4772,6 +4918,25 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd)
 
 	lpfc_scsi_prep_cmnd(vport, lpfc_cmd, ndlp);
 
+	/* check the necessary and sufficient condition to support VMID */
+	if (lpfc_is_vmid_enabled(phba) &&
+	    (ndlp->vmid_support ||
+	     phba->pport->vmid_priority_tagging ==
+	     LPFC_VMID_PRIO_TAG_ALL_TARGETS)) {
+		/* is the IO generated by a VM, get the associated virtual */
+		/* entity id */
+		uuid = lpfc_is_command_vm_io(cmnd);
+
+		if (uuid) {
+			err = lpfc_vmid_get_appid(vport, uuid, cmnd,
+				(union lpfc_vmid_io_tag *)
+					&lpfc_cmd->cur_iocbq.vmid_tag);
+			if (!err)
+				lpfc_cmd->cur_iocbq.iocb_flag |= LPFC_IO_VMID;
+		}
+	}
+
+	atomic_inc(&ndlp->cmd_pending);
 #ifdef CONFIG_SCSI_LPFC_DEBUG_FS
 	if (unlikely(phba->hdwqstat_on & LPFC_CHECK_SCSI_IO))
 		this_cpu_inc(phba->sli4_hba.c_stat->xmt_io);
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/8a8ee776/attachment.p7s>

From tj at kernel.org  Mon Oct 19 10:43:16 2020
From: tj at kernel.org (Tejun Heo)
Date: Mon, 19 Oct 2020 10:43:16 -0400
Subject: [RFC v2 01/18] cgroup: Added cgroup_get_from_kernfs_id
In-Reply-To: <1603093393-12875-2-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
 <1603093393-12875-2-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <20201019144316.GB4418@mtj.thefacebook.com>

On Mon, Oct 19, 2020 at 01:12:56PM +0530, Muneendra wrote:
> Added a new function cgroup_get_from_kernfs_id  to retrieve the cgroup
> associated with cgroup id.
> It takes cgroupid as an argument and returns cgrp and on failure
> it returns NULL.
> Exported the same as this can be used by blk-cgorup.c
> 
> Added function declaration of cgroup_get_from_kernfs_id in cgorup.h
> 
> Signed-off-by: Muneendra <muneendra.kumar at broadcom.com>

Acked-by: Tejun Heo <tj at kernel.org>

Thanks.

-- 
tejun


From himanshu.madhani at oracle.com  Mon Oct 19 10:43:16 2020
From: himanshu.madhani at oracle.com (Himanshu Madhani)
Date: Mon, 19 Oct 2020 09:43:16 -0500
Subject: [PATCH 1/4] nvme-fc: remove err_work work item
In-Reply-To: <20201016212729.49138-2-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
 <20201016212729.49138-2-james.smart@broadcom.com>
Message-ID: <60D3B57C-3F14-4E39-9709-A4316105BBC9@oracle.com>



> On Oct 16, 2020, at 4:27 PM, James Smart <james.smart at broadcom.com> wrote:
> 
> err_work was created to handle errors (mainly io timeout) while in
> CONNECTING state. The flag for err_work_active is also unneeded.
> 
> Remove err_work_active and err_work.  The actions to abort ios are moved
> inline to nvme_error_recovery().
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
> drivers/nvme/host/fc.c | 40 ++++++++++------------------------------
> 1 file changed, 10 insertions(+), 30 deletions(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 7067aaf50bf7..06fb208ab350 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -153,7 +153,6 @@ struct nvme_fc_ctrl {
> 	u32			cnum;
> 
> 	bool			ioq_live;
> -	atomic_t		err_work_active;
> 	u64			association_id;
> 	struct nvmefc_ls_rcv_op	*rcv_disconn;
> 
> @@ -163,7 +162,6 @@ struct nvme_fc_ctrl {
> 	struct blk_mq_tag_set	tag_set;
> 
> 	struct delayed_work	connect_work;
> -	struct work_struct	err_work;
> 
> 	struct kref		ref;
> 	unsigned long		flags;
> @@ -2410,11 +2408,11 @@ nvme_fc_nvme_ctrl_freed(struct nvme_ctrl *nctrl)
> 	nvme_fc_ctrl_put(ctrl);
> }
> 
> +static void __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl);
> +
> static void
> nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
> {
> -	int active;
> -
> 	/*
> 	 * if an error (io timeout, etc) while (re)connecting,
> 	 * it's an error on creating the new association.
> @@ -2423,11 +2421,14 @@ nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
> 	 * ios hitting this path before things are cleaned up.
> 	 */
> 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
> -		active = atomic_xchg(&ctrl->err_work_active, 1);
> -		if (!active && !queue_work(nvme_fc_wq, &ctrl->err_work)) {
> -			atomic_set(&ctrl->err_work_active, 0);
> -			WARN_ON(1);
> -		}
> +		__nvme_fc_terminate_io(ctrl);
> +
> +		/*
> +		 * Rescheduling the connection after recovering
> +		 * from the io error is left to the reconnect work
> +		 * item, which is what should have stalled waiting on
> +		 * the io that had the error that scheduled this work.
> +		 */
> 		return;
> 	}
> 
> @@ -3233,7 +3234,6 @@ nvme_fc_delete_ctrl(struct nvme_ctrl *nctrl)
> {
> 	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
> 
> -	cancel_work_sync(&ctrl->err_work);
> 	cancel_delayed_work_sync(&ctrl->connect_work);
> 	/*
> 	 * kill the association on the link side.  this will block
> @@ -3343,23 +3343,6 @@ nvme_fc_reset_ctrl_work(struct work_struct *work)
> 			ctrl->cnum);
> }
> 
> -static void
> -nvme_fc_connect_err_work(struct work_struct *work)
> -{
> -	struct nvme_fc_ctrl *ctrl =
> -			container_of(work, struct nvme_fc_ctrl, err_work);
> -
> -	__nvme_fc_terminate_io(ctrl);
> -
> -	atomic_set(&ctrl->err_work_active, 0);
> -
> -	/*
> -	 * Rescheduling the connection after recovering
> -	 * from the io error is left to the reconnect work
> -	 * item, which is what should have stalled waiting on
> -	 * the io that had the error that scheduled this work.
> -	 */
> -}
> 
> static const struct nvme_ctrl_ops nvme_fc_ctrl_ops = {
> 	.name			= "fc",
> @@ -3474,7 +3457,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
> 	ctrl->dev = lport->dev;
> 	ctrl->cnum = idx;
> 	ctrl->ioq_live = false;
> -	atomic_set(&ctrl->err_work_active, 0);
> 	init_waitqueue_head(&ctrl->ioabort_wait);
> 
> 	get_device(ctrl->dev);
> @@ -3482,7 +3464,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
> 
> 	INIT_WORK(&ctrl->ctrl.reset_work, nvme_fc_reset_ctrl_work);
> 	INIT_DELAYED_WORK(&ctrl->connect_work, nvme_fc_connect_ctrl_work);
> -	INIT_WORK(&ctrl->err_work, nvme_fc_connect_err_work);
> 	spin_lock_init(&ctrl->lock);
> 
> 	/* io queue count */
> @@ -3575,7 +3556,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
> fail_ctrl:
> 	nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING);
> 	cancel_work_sync(&ctrl->ctrl.reset_work);
> -	cancel_work_sync(&ctrl->err_work);
> 	cancel_delayed_work_sync(&ctrl->connect_work);
> 
> 	ctrl->ctrl.opts = NULL;
> -- 
> 2.26.2
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Looks Good. 

Reviewed-by: Himanshu Madhani <himanshu.madhani at oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering



From tj at kernel.org  Mon Oct 19 10:46:19 2020
From: tj at kernel.org (Tejun Heo)
Date: Mon, 19 Oct 2020 10:46:19 -0400
Subject: [RFC v2 02/18] blkcg: Added a app identifier support for blkcg
In-Reply-To: <1603093393-12875-3-git-send-email-muneendra.kumar@broadcom.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
 <1603093393-12875-3-git-send-email-muneendra.kumar@broadcom.com>
Message-ID: <20201019144619.GC4418@mtj.thefacebook.com>

On Mon, Oct 19, 2020 at 01:12:57PM +0530, Muneendra wrote:
> This Patch added a unique application identifier i.e
> app_id  knob to  blkcg which allows identification of traffic
> sources at an individual cgroup based Applications
> (ex:virtual machine (VM))level in both host and
> fabric infrastructure.
> 
> Provided the interface blkcg_get_app_identifier to
> grab the app identifier associated with a bio.
> 
> Provided the interface blkcg_set_app_identifier to
> set the app identifier in a blkcgrp associated with cgroup id
> 
> Signed-off-by: Muneendra <muneendra.kumar at broadcom.com>
> 
> ---
> v2:
> renamed app_identifier to app_id
> removed the  sysfs interface blkio.app_identifie under
> /sys/fs/cgroup/blkio
> Added a new interface blkcg_set_app_identifier
> ---
>  block/blk-cgroup.c         | 31 +++++++++++++++++++++++++++++++
>  include/linux/blk-cgroup.h | 22 ++++++++++++++++++++++
>  2 files changed, 53 insertions(+)
> 
> diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
> index 619a79b51068..672971521010 100644
> --- a/block/blk-cgroup.c
> +++ b/block/blk-cgroup.c
> @@ -546,6 +546,37 @@ static struct blkcg_gq *blkg_lookup_check(struct blkcg *blkcg,
>  	return __blkg_lookup(blkcg, q, true /* update_hint */);
>  }
>  
> +/*
> + * Sets the app_identifier field associted to blkcg
> + * @buf: application identifier
> + * @id: cgrp id
> + * @len: size of appid
> + */
> +int blkcg_set_app_identifier(char *buf, u64 id, size_t len)

Can you please use a name which is more specific?

> +{
> +	struct cgroup *cgrp = NULL;
> +	struct cgroup_subsys_state *css = NULL;
> +	struct blkcg *blkcg = NULL;
> +
> +	cgrp = cgroup_get_from_kernfs_id(id);
> +	if (!cgrp)
> +		return -ENOENT;
> +
> +	css = cgroup_get_e_css(cgrp, &io_cgrp_subsys);
> +	if (!css)
> +		return -ENOENT;
> +
> +	blkcg = css_to_blkcg(css);
> +	if (!blkcg)
> +		return -ENOENT;
> +
> +	if (len > APPID_LEN)
> +		return -EINVAL;
> +	strlcpy(blkcg->app_id, buf, len);

Shouldn't the cgrp and css be put before exit?

> +	return 0;
> +}
> +EXPORT_SYMBOL(blkcg_set_app_identifier);
> +
>  /**
>   * blkg_conf_prep - parse and prepare for per-blkg config update
>   * @inputp: input string pointer
> diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h
> index c8fc9792ac77..5bd3f9f397ac 100644
> --- a/include/linux/blk-cgroup.h
> +++ b/include/linux/blk-cgroup.h
> @@ -30,6 +30,8 @@
>  
>  /* Max limits for throttle policy */
>  #define THROTL_IOPS_MAX		UINT_MAX
> +#define APPID_LEN              128
> +
>  
>  #ifdef CONFIG_BLK_CGROUP
>  
> @@ -55,6 +57,8 @@ struct blkcg {
>  	struct blkcg_policy_data	*cpd[BLKCG_MAX_POLS];
>  
>  	struct list_head		all_blkcgs_node;
> +	char                            app_id[APPID_LEN];

Please use a more specific name and gate them behind a CONFIG.

Thanks.

-- 
tejun


From himanshu.madhani at oracle.com  Mon Oct 19 10:49:27 2020
From: himanshu.madhani at oracle.com (Himanshu Madhani)
Date: Mon, 19 Oct 2020 09:49:27 -0500
Subject: [PATCH 2/4] nvme-fc: eliminate terminate_io use by
 nvme_fc_error_recovery
In-Reply-To: <20201016212729.49138-3-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
 <20201016212729.49138-3-james.smart@broadcom.com>
Message-ID: <E68ABA84-3EEA-4F82-8C5C-4F2E93ADCA65@oracle.com>



> On Oct 16, 2020, at 4:27 PM, James Smart <james.smart at broadcom.com> wrote:
> 
> nvme_fc_error_recovery() special cases handling when in CONNECTING state
> and calls __nvme_fc_terminate_io(). __nvme_fc_terminate_io() itself
> special cases CONNECTING state and calls the routine to abort outstanding
> ios.
> 
> Simplify the sequence by putting the call to abort outstanding ios directly
> in nvme_fc_error_recovery.
> 
> Move the location of __nvme_fc_abort_outstanding_ios(), and
> nvme_fc_terminate_exchange() which is called by it, to avoid adding
> function prototypes for nvme_fc_error_recovery().
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
> drivers/nvme/host/fc.c | 185 ++++++++++++++++++-----------------------
> 1 file changed, 83 insertions(+), 102 deletions(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 06fb208ab350..d65a4a9f4808 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2408,27 +2408,96 @@ nvme_fc_nvme_ctrl_freed(struct nvme_ctrl *nctrl)
> 	nvme_fc_ctrl_put(ctrl);
> }
> 
> -static void __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl);
> +/*
> + * This routine is used by the transport when it needs to find active
> + * io on a queue that is to be terminated. The transport uses
> + * blk_mq_tagset_busy_itr() to find the busy requests, which then invoke
> + * this routine to kill them on a 1 by 1 basis.
> + *
> + * As FC allocates FC exchange for each io, the transport must contact
> + * the LLDD to terminate the exchange, thus releasing the FC exchange.
> + * After terminating the exchange the LLDD will call the transport's
> + * normal io done path for the request, but it will have an aborted
> + * status. The done path will return the io request back to the block
> + * layer with an error status.
> + */
> +static bool
> +nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
> +{
> +	struct nvme_ctrl *nctrl = data;
> +	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
> +	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
> +
> +	__nvme_fc_abort_op(ctrl, op);
> +	return true;
> +}
> +
> +/*
> + * This routine runs through all outstanding commands on the association
> + * and aborts them.  This routine is typically be called by the
> + * delete_association routine. It is also called due to an error during
> + * reconnect. In that scenario, it is most likely a command that initializes
> + * the controller, including fabric Connect commands on io queues, that
> + * may have timed out or failed thus the io must be killed for the connect
> + * thread to see the error.
> + */
> +static void
> +__nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
> +{
> +	/*
> +	 * If io queues are present, stop them and terminate all outstanding
> +	 * ios on them. As FC allocates FC exchange for each io, the
> +	 * transport must contact the LLDD to terminate the exchange,
> +	 * thus releasing the FC exchange. We use blk_mq_tagset_busy_itr()
> +	 * to tell us what io's are busy and invoke a transport routine
> +	 * to kill them with the LLDD.  After terminating the exchange
> +	 * the LLDD will call the transport's normal io done path, but it
> +	 * will have an aborted status. The done path will return the
> +	 * io requests back to the block layer as part of normal completions
> +	 * (but with error status).
> +	 */
> +	if (ctrl->ctrl.queue_count > 1) {
> +		nvme_stop_queues(&ctrl->ctrl);
> +		blk_mq_tagset_busy_iter(&ctrl->tag_set,
> +				nvme_fc_terminate_exchange, &ctrl->ctrl);
> +		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
> +		if (start_queues)
> +			nvme_start_queues(&ctrl->ctrl);
> +	}
> +
> +	/*
> +	 * Other transports, which don't have link-level contexts bound
> +	 * to sqe's, would try to gracefully shutdown the controller by
> +	 * writing the registers for shutdown and polling (call
> +	 * nvme_shutdown_ctrl()). Given a bunch of i/o was potentially
> +	 * just aborted and we will wait on those contexts, and given
> +	 * there was no indication of how live the controlelr is on the
> +	 * link, don't send more io to create more contexts for the
> +	 * shutdown. Let the controller fail via keepalive failure if
> +	 * its still present.
> +	 */
> +
> +	/*
> +	 * clean up the admin queue. Same thing as above.
> +	 */
> +	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
> +	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
> +				nvme_fc_terminate_exchange, &ctrl->ctrl);
> +	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
> +}
> 
> static void
> nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
> {
> 	/*
> -	 * if an error (io timeout, etc) while (re)connecting,
> -	 * it's an error on creating the new association.
> -	 * Start the error recovery thread if it hasn't already
> -	 * been started. It is expected there could be multiple
> -	 * ios hitting this path before things are cleaned up.
> +	 * if an error (io timeout, etc) while (re)connecting, the remote
> +	 * port requested terminating of the association (disconnect_ls)
> +	 * or an error (timeout or abort) occurred on an io while creating
> +	 * the controller.  Abort any ios on the association and let the
> +	 * create_association error path resolve things.
> 	 */
> 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
> -		__nvme_fc_terminate_io(ctrl);
> -
> -		/*
> -		 * Rescheduling the connection after recovering
> -		 * from the io error is left to the reconnect work
> -		 * item, which is what should have stalled waiting on
> -		 * the io that had the error that scheduled this work.
> -		 */
> +		__nvme_fc_abort_outstanding_ios(ctrl, true);
> 		return;
> 	}
> 
> @@ -2742,30 +2811,6 @@ nvme_fc_complete_rq(struct request *rq)
> 	nvme_fc_ctrl_put(ctrl);
> }
> 
> -/*
> - * This routine is used by the transport when it needs to find active
> - * io on a queue that is to be terminated. The transport uses
> - * blk_mq_tagset_busy_itr() to find the busy requests, which then invoke
> - * this routine to kill them on a 1 by 1 basis.
> - *
> - * As FC allocates FC exchange for each io, the transport must contact
> - * the LLDD to terminate the exchange, thus releasing the FC exchange.
> - * After terminating the exchange the LLDD will call the transport's
> - * normal io done path for the request, but it will have an aborted
> - * status. The done path will return the io request back to the block
> - * layer with an error status.
> - */
> -static bool
> -nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
> -{
> -	struct nvme_ctrl *nctrl = data;
> -	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
> -	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
> -
> -	__nvme_fc_abort_op(ctrl, op);
> -	return true;
> -}
> -
> 
> static const struct blk_mq_ops nvme_fc_mq_ops = {
> 	.queue_rq	= nvme_fc_queue_rq,
> @@ -3104,60 +3149,6 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
> }
> 
> 
> -/*
> - * This routine runs through all outstanding commands on the association
> - * and aborts them.  This routine is typically be called by the
> - * delete_association routine. It is also called due to an error during
> - * reconnect. In that scenario, it is most likely a command that initializes
> - * the controller, including fabric Connect commands on io queues, that
> - * may have timed out or failed thus the io must be killed for the connect
> - * thread to see the error.
> - */
> -static void
> -__nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
> -{
> -	/*
> -	 * If io queues are present, stop them and terminate all outstanding
> -	 * ios on them. As FC allocates FC exchange for each io, the
> -	 * transport must contact the LLDD to terminate the exchange,
> -	 * thus releasing the FC exchange. We use blk_mq_tagset_busy_itr()
> -	 * to tell us what io's are busy and invoke a transport routine
> -	 * to kill them with the LLDD.  After terminating the exchange
> -	 * the LLDD will call the transport's normal io done path, but it
> -	 * will have an aborted status. The done path will return the
> -	 * io requests back to the block layer as part of normal completions
> -	 * (but with error status).
> -	 */
> -	if (ctrl->ctrl.queue_count > 1) {
> -		nvme_stop_queues(&ctrl->ctrl);
> -		blk_mq_tagset_busy_iter(&ctrl->tag_set,
> -				nvme_fc_terminate_exchange, &ctrl->ctrl);
> -		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
> -		if (start_queues)
> -			nvme_start_queues(&ctrl->ctrl);
> -	}
> -
> -	/*
> -	 * Other transports, which don't have link-level contexts bound
> -	 * to sqe's, would try to gracefully shutdown the controller by
> -	 * writing the registers for shutdown and polling (call
> -	 * nvme_shutdown_ctrl()). Given a bunch of i/o was potentially
> -	 * just aborted and we will wait on those contexts, and given
> -	 * there was no indication of how live the controlelr is on the
> -	 * link, don't send more io to create more contexts for the
> -	 * shutdown. Let the controller fail via keepalive failure if
> -	 * its still present.
> -	 */
> -
> -	/*
> -	 * clean up the admin queue. Same thing as above.
> -	 */
> -	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
> -	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
> -				nvme_fc_terminate_exchange, &ctrl->ctrl);
> -	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
> -}
> -
> /*
>  * This routine stops operation of the controller on the host side.
>  * On the host os stack side: Admin and IO queues are stopped,
> @@ -3290,16 +3281,6 @@ nvme_fc_reconnect_or_delete(struct nvme_fc_ctrl *ctrl, int status)
> static void
> __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
> {
> -	/*
> -	 * if state is CONNECTING - the error occurred as part of a
> -	 * reconnect attempt. Abort any ios on the association and
> -	 * let the create_association error paths resolve things.
> -	 */
> -	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
> -		__nvme_fc_abort_outstanding_ios(ctrl, true);
> -		return;
> -	}
> -
> 	/*
> 	 * For any other state, kill the association. As this routine
> 	 * is a common io abort routine for resetting and such, after
> -- 
> 2.26.2
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Looks Okay. 

Reviewed-by: Himanshu Madhani <himanshu.madhani at oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering



From himanshu.madhani at oracle.com  Mon Oct 19 10:51:36 2020
From: himanshu.madhani at oracle.com (Himanshu Madhani)
Date: Mon, 19 Oct 2020 09:51:36 -0500
Subject: [PATCH 3/4] nvme-fc: remove nvme_fc_terminate_io()
In-Reply-To: <20201016212729.49138-4-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
 <20201016212729.49138-4-james.smart@broadcom.com>
Message-ID: <00E40C1E-9374-4E3D-BB63-BDE1DFFAB46F@oracle.com>



> On Oct 16, 2020, at 4:27 PM, James Smart <james.smart at broadcom.com> wrote:
> 
> __nvme_fc_terminate_io() is now called by only 1 place, in reset_work.
> Consoldate and move the functionality of terminate_io into reset_work.
> 
> In reset_work, rather than calling the create_association directly,
> schedule the connect work element to do its thing. After scheduling, flush
> the connect work element to continue with semantic of not returning until
> connect has been attempted at least once.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
> drivers/nvme/host/fc.c | 49 ++++++++++++++----------------------------
> 1 file changed, 16 insertions(+), 33 deletions(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index d65a4a9f4808..b2f9b3752df7 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -3279,49 +3279,32 @@ nvme_fc_reconnect_or_delete(struct nvme_fc_ctrl *ctrl, int status)
> }
> 
> static void
> -__nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
> +nvme_fc_reset_ctrl_work(struct work_struct *work)
> {
> -	/*
> -	 * For any other state, kill the association. As this routine
> -	 * is a common io abort routine for resetting and such, after
> -	 * the association is terminated, ensure that the state is set
> -	 * to CONNECTING.
> -	 */
> +	struct nvme_fc_ctrl *ctrl =
> +		container_of(work, struct nvme_fc_ctrl, ctrl.reset_work);
> 
> -	nvme_stop_keep_alive(&ctrl->ctrl);
> +	nvme_stop_ctrl(&ctrl->ctrl);
> 
> 	/* will block will waiting for io to terminate */
> 	nvme_fc_delete_association(ctrl);
> 
> -	if (ctrl->ctrl.state != NVME_CTRL_CONNECTING &&
> -	    !nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
> +	if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
> 		dev_err(ctrl->ctrl.device,
> 			"NVME-FC{%d}: error_recovery: Couldn't change state "
> 			"to CONNECTING\n", ctrl->cnum);
> -}
> -
> -static void
> -nvme_fc_reset_ctrl_work(struct work_struct *work)
> -{
> -	struct nvme_fc_ctrl *ctrl =
> -		container_of(work, struct nvme_fc_ctrl, ctrl.reset_work);
> -	int ret;
> -
> -	__nvme_fc_terminate_io(ctrl);
> 
> -	nvme_stop_ctrl(&ctrl->ctrl);
> -
> -	if (ctrl->rport->remoteport.port_state == FC_OBJSTATE_ONLINE)
> -		ret = nvme_fc_create_association(ctrl);
> -	else
> -		ret = -ENOTCONN;
> -
> -	if (ret)
> -		nvme_fc_reconnect_or_delete(ctrl, ret);
> -	else
> -		dev_info(ctrl->ctrl.device,
> -			"NVME-FC{%d}: controller reset complete\n",
> -			ctrl->cnum);
> +	if (ctrl->rport->remoteport.port_state == FC_OBJSTATE_ONLINE) {
> +		if (!queue_delayed_work(nvme_wq, &ctrl->connect_work, 0)) {
> +			dev_err(ctrl->ctrl.device,
> +				"NVME-FC{%d}: failed to schedule connect "
> +				"after reset\n", ctrl->cnum);
> +		} else {
> +			flush_delayed_work(&ctrl->connect_work);
> +		}
> +	} else {
> +		nvme_fc_reconnect_or_delete(ctrl, -ENOTCONN);
> +	}
> }
> 
> 
> -- 
> 2.26.2
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Looks Okay. 

Reviewed-by: Himanshu Madhani <himanshu.madhani at oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering



From himanshu.madhani at oracle.com  Mon Oct 19 10:55:56 2020
From: himanshu.madhani at oracle.com (Himanshu Madhani)
Date: Mon, 19 Oct 2020 09:55:56 -0500
Subject: [PATCH] nvme-fc: fix error loop in create_hw_io_queues
In-Reply-To: <20201016210604.48542-1-james.smart@broadcom.com>
References: <20201016210604.48542-1-james.smart@broadcom.com>
Message-ID: <CDFD0554-F758-4944-87C1-DA3154DC5065@oracle.com>



> On Oct 16, 2020, at 4:06 PM, James Smart <james.smart at broadcom.com> wrote:
> 
> The loop that backs out of hw io queue creation continues through index
> 0, which corresponds to the admin queue as well.
> 
> Fix the loop so it only proceeds through indexes 1..n which correspond to
> io queues.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
> drivers/nvme/host/fc.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index e2e09e25c056..f002522146e2 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2314,7 +2314,7 @@ nvme_fc_create_hw_io_queues(struct nvme_fc_ctrl *ctrl, u16 qsize)
> 	return 0;
> 
> delete_queues:
> -	for (; i >= 0; i--)
> +	for (; i > 0; i--)
> 		__nvme_fc_delete_hw_queue(ctrl, &ctrl->queues[i], i);
> 	return ret;
> }
> -- 
> 2.26.2
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme


Reviewed-by: Himanshu Madhani <himanshu.madhani at oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering



From himanshu.madhani at oracle.com  Mon Oct 19 10:54:33 2020
From: himanshu.madhani at oracle.com (Himanshu Madhani)
Date: Mon, 19 Oct 2020 09:54:33 -0500
Subject: [PATCH 4/4] nvme_fc: track error_recovery while connecting
In-Reply-To: <20201016212729.49138-5-james.smart@broadcom.com>
References: <20201016212729.49138-1-james.smart@broadcom.com>
 <20201016212729.49138-5-james.smart@broadcom.com>
Message-ID: <8332227D-4103-42D6-B40F-772EE1C2871C@oracle.com>



> On Oct 16, 2020, at 4:27 PM, James Smart <james.smart at broadcom.com> wrote:
> 
> Whenever there are errors during CONNECTING, the driver recovers by
> aborting all outstanding ios and counts on the io completion to fail them
> and thus the connection/association they are on.  However, the connection
> failure depends on a failure state from the core routines.  Not all
> commands that are issued by the core routine are guaranteed to cause a
> failure of the core routine. They may be treated as a failure status and
> the status is then ignored.
> 
> As such, whenever the transport enters error_recovery while CONNECTING,
> it will set a new flag indicating an association failed. The
> create_association routine which creates and initializes the controller,
> will monitor the state of the flag as well as the core routine error
> status and ensure the association fails if there was an error.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
> drivers/nvme/host/fc.c | 14 +++++++++-----
> 1 file changed, 9 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index b2f9b3752df7..6352068c0c4a 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -142,7 +142,8 @@ struct nvme_fc_rport {
> 
> /* fc_ctrl flags values - specified as bit positions */
> #define ASSOC_ACTIVE		0
> -#define FCCTRL_TERMIO		1
> +#define ASSOC_FAILED		1
> +#define FCCTRL_TERMIO		2
> 
> struct nvme_fc_ctrl {
> 	spinlock_t		lock;
> @@ -2498,6 +2499,7 @@ nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
> 	 */
> 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
> 		__nvme_fc_abort_outstanding_ios(ctrl, true);
> +		set_bit(ASSOC_FAILED, &ctrl->flags);
> 		return;
> 	}
> 
> @@ -3030,6 +3032,8 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
> 		ctrl->cnum, ctrl->lport->localport.port_name,
> 		ctrl->rport->remoteport.port_name, ctrl->ctrl.opts->subsysnqn);
> 
> +	clear_bit(ASSOC_FAILED, &ctrl->flags);
> +
> 	/*
> 	 * Create the admin queue
> 	 */
> @@ -3058,7 +3062,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
> 	 */
> 
> 	ret = nvme_enable_ctrl(&ctrl->ctrl);
> -	if (ret)
> +	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
> 		goto out_disconnect_admin_queue;
> 
> 	ctrl->ctrl.max_segments = ctrl->lport->ops->max_sgl_segments;
> @@ -3068,7 +3072,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
> 	blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
> 
> 	ret = nvme_init_identify(&ctrl->ctrl);
> -	if (ret)
> +	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
> 		goto out_disconnect_admin_queue;
> 
> 	/* sanity checks */
> @@ -3113,9 +3117,9 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
> 			ret = nvme_fc_create_io_queues(ctrl);
> 		else
> 			ret = nvme_fc_recreate_io_queues(ctrl);
> -		if (ret)
> -			goto out_term_aen_ops;
> 	}
> +	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
> +		goto out_term_aen_ops;
> 
> 	changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
> 
> -- 
> 2.26.2
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Looks fine. 

Reviewed-by: Himanshu Madhani <himanshu.madhani at oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering



From himanshu.madhani at oracle.com  Mon Oct 19 10:57:14 2020
From: himanshu.madhani at oracle.com (Himanshu Madhani)
Date: Mon, 19 Oct 2020 09:57:14 -0500
Subject: [PATCH] nvme-fc: wait for queues to freeze before calling
 update_hr_hw_queues
In-Reply-To: <20201016211724.48707-1-james.smart@broadcom.com>
References: <20201016211724.48707-1-james.smart@broadcom.com>
Message-ID: <14FF47F2-A7A4-4335-82D0-86615D1920DF@oracle.com>



> On Oct 16, 2020, at 4:17 PM, James Smart <james.smart at broadcom.com> wrote:
> 
> On reconnect, the code currently does not freeze the controller before
> possibly updating the number hw queues for the controller.
> 
> Add the freeze before updating the number of hw queues. Note: the queues
> are already started and remain started through the reconnect.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
> drivers/nvme/host/fc.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index c81ebe94d8fe..7067aaf50bf7 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2885,11 +2885,14 @@ nvme_fc_recreate_io_queues(struct nvme_fc_ctrl *ctrl)
> 	if (ret)
> 		goto out_delete_hw_queues;
> 
> -	if (prior_ioq_cnt != nr_io_queues)
> +	if (prior_ioq_cnt != nr_io_queues) {
> 		dev_info(ctrl->ctrl.device,
> 			"reconnect: revising io queue count from %d to %d\n",
> 			prior_ioq_cnt, nr_io_queues);
> -	blk_mq_update_nr_hw_queues(&ctrl->tag_set, nr_io_queues);
> +		nvme_wait_freeze(&ctrl->ctrl);
> +		blk_mq_update_nr_hw_queues(&ctrl->tag_set, nr_io_queues);
> +		nvme_unfreeze(&ctrl->ctrl);
> +	}
> 
> 	return 0;
> 
> -- 
> 2.26.2
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Looks Good. 

Reviewed-by: Himanshu Madhani <himanshu.madhani at oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering



From himanshu.madhani at oracle.com  Mon Oct 19 10:59:51 2020
From: himanshu.madhani at oracle.com (Himanshu Madhani)
Date: Mon, 19 Oct 2020 09:59:51 -0500
Subject: [PATCH] nvme-fc: shorten reconnect delay if possible for FC
In-Reply-To: <20201016212928.49429-1-james.smart@broadcom.com>
References: <20201016212928.49429-1-james.smart@broadcom.com>
Message-ID: <11A701E9-B011-402F-9B78-BD5CD265D27B@oracle.com>



> On Oct 16, 2020, at 4:29 PM, James Smart <james.smart at broadcom.com> wrote:
> 
> We've had several complaints about a 10s reconnect delay (the default)
> when there was an error while there is connectivity to a subsystem.
> The max_reconnects and reconnect_delay are set in common code prior to
> calling the transport to create the controller.
> 
> This change checks if the default reconnect delay is being used, and if
> so, it adjusts it to a shorter period (2s) for the nvme-fc transport.
> It does so by calculating the controller loss tmo window, changing the
> value of the reconnect delay, and then recalculating the maximum number
> of reconnect attempts allowed.
> 
> Signed-off-by: James Smart <james.smart at broadcom.com>
> ---
> drivers/nvme/host/fc.c | 19 ++++++++++++++++++-
> 1 file changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 6352068c0c4a..1af7c6f358f5 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -26,6 +26,10 @@ enum nvme_fc_queue_flags {
> };
> 
> #define NVME_FC_DEFAULT_DEV_LOSS_TMO	60	/* seconds */
> +#define NVME_FC_DEFAULT_RECONNECT_TMO	2	/* delay between reconnects
> +						 * when connected and a
> +						 * connection failure.
> +						 */
> 
> struct nvme_fc_queue {
> 	struct nvme_fc_ctrl	*ctrl;
> @@ -3387,7 +3391,7 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
> {
> 	struct nvme_fc_ctrl *ctrl;
> 	unsigned long flags;
> -	int ret, idx;
> +	int ret, idx, ctrl_loss_tmo;
> 
> 	if (!(rport->remoteport.port_role &
> 	    (FC_PORT_ROLE_NVME_DISCOVERY | FC_PORT_ROLE_NVME_TARGET))) {
> @@ -3413,6 +3417,19 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
> 		goto out_free_ctrl;
> 	}
> 
> +	/*
> +	 * if ctrl_loss_tmo is being enforced and the default reconnect delay
> +	 * is being used, change to a shorter reconnect delay for FC.
> +	 */
> +	if (opts->max_reconnects != -1 &&
> +	    opts->reconnect_delay == NVMF_DEF_RECONNECT_DELAY &&
> +	    opts->reconnect_delay > NVME_FC_DEFAULT_RECONNECT_TMO) {
> +		ctrl_loss_tmo = opts->max_reconnects * opts->reconnect_delay;
> +		opts->reconnect_delay = NVME_FC_DEFAULT_RECONNECT_TMO;
> +		opts->max_reconnects = DIV_ROUND_UP(ctrl_loss_tmo,
> +						opts->reconnect_delay);
> +	}
> +
> 	ctrl->ctrl.opts = opts;
> 	ctrl->ctrl.nr_reconnects = 0;
> 	if (lport->dev)
> -- 
> 2.26.2
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Looks good.

Reviewed-by: Himanshu Madhani <himanshu.madhani at oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering



From muneendra.kumar at broadcom.com  Mon Oct 19 11:13:05 2020
From: muneendra.kumar at broadcom.com (Muneendra Kumar M)
Date: Mon, 19 Oct 2020 20:43:05 +0530
Subject: [RFC v2 02/18] blkcg: Added a app identifier support for blkcg
In-Reply-To: <20201019144619.GC4418@mtj.thefacebook.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
 <1603093393-12875-3-git-send-email-muneendra.kumar@broadcom.com>
 <20201019144619.GC4418@mtj.thefacebook.com>
Message-ID: <ab237522d9110888aaaa538519f3af7d@mail.gmail.com>

Hi Tejun,
Thanks for the input.
>> +     cgrp = cgroup_get_from_kernfs_id(id);
>> +     if (!cgrp)
>> +             return -ENOENT;
>> +
> >+     css = cgroup_get_e_css(cgrp, &io_cgrp_subsys);
>> +     if (!css)
>> +             return -ENOENT;
>> +
>> +     blkcg = css_to_blkcg(css);
>> +     if (!blkcg)
>> +             return -ENOENT;
> >+		return -EINVAL;
> >+	strlcpy(blkcg->app_id, buf, len);

>Shouldn't the cgrp and css be put before exit?
[Muneendra]Correct me if iam wrong.
You mean to add cgroup_put(cgrp) and css_put(css) before exit ?

> +
>
>  	struct list_head		all_blkcgs_node;
> +	char                            app_id[APPID_LEN];

>Please use a more specific name and gate them behind a CONFIG.
[Muneendra] Agree will do the changes.

Regards,
Muneendra.

Thanks.

--
tejun
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/128376d6/attachment.p7s>

From tj at kernel.org  Mon Oct 19 11:19:35 2020
From: tj at kernel.org (Tejun Heo)
Date: Mon, 19 Oct 2020 11:19:35 -0400
Subject: [RFC v2 02/18] blkcg: Added a app identifier support for blkcg
In-Reply-To: <ab237522d9110888aaaa538519f3af7d@mail.gmail.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
 <1603093393-12875-3-git-send-email-muneendra.kumar@broadcom.com>
 <20201019144619.GC4418@mtj.thefacebook.com>
 <ab237522d9110888aaaa538519f3af7d@mail.gmail.com>
Message-ID: <20201019151935.GD4418@mtj.thefacebook.com>

On Mon, Oct 19, 2020 at 08:43:05PM +0530, Muneendra Kumar M wrote:
> Hi Tejun,
> Thanks for the input.
> >> +     cgrp = cgroup_get_from_kernfs_id(id);
> >> +     if (!cgrp)
> >> +             return -ENOENT;
> >> +
> > >+     css = cgroup_get_e_css(cgrp, &io_cgrp_subsys);
> >> +     if (!css)
> >> +             return -ENOENT;
> >> +
> >> +     blkcg = css_to_blkcg(css);
> >> +     if (!blkcg)
> >> +             return -ENOENT;
> > >+		return -EINVAL;
> > >+	strlcpy(blkcg->app_id, buf, len);
> 
> >Shouldn't the cgrp and css be put before exit?
> [Muneendra]Correct me if iam wrong.
> You mean to add cgroup_put(cgrp) and css_put(css) before exit ?

Yeah, as-is it'd leak references each time the function is called.

Thanks.

-- 
tejun


From muneendra.kumar at broadcom.com  Mon Oct 19 11:20:59 2020
From: muneendra.kumar at broadcom.com (Muneendra Kumar M)
Date: Mon, 19 Oct 2020 20:50:59 +0530
Subject: [RFC v2 02/18] blkcg: Added a app identifier support for blkcg
In-Reply-To: <20201019151935.GD4418@mtj.thefacebook.com>
References: <1603093393-12875-1-git-send-email-muneendra.kumar@broadcom.com>
 <1603093393-12875-3-git-send-email-muneendra.kumar@broadcom.com>
 <20201019144619.GC4418@mtj.thefacebook.com>
 <ab237522d9110888aaaa538519f3af7d@mail.gmail.com>
 <20201019151935.GD4418@mtj.thefacebook.com>
Message-ID: <4ccbf473fc32df26e90ee494aabe754b@mail.gmail.com>

Hi Tejun,
Thanks for the input.
I will add it my next version.

Regards,
Muneendra.

-----Original Message-----
From: Tejun Heo [mailto:htejun at gmail.com] On Behalf Of Tejun Heo
Sent: Monday, October 19, 2020 8:50 PM
To: Muneendra Kumar M <muneendra.kumar at broadcom.com>
Cc: linux-block at vger.kernel.org; linux-scsi at vger.kernel.org;
linux-nvme at lists.infradead.org; jsmart2021 at gmail.com; emilne at redhat.com;
mkumar at redhat.com; pbonzini at redhat.com
Subject: Re: [RFC v2 02/18] blkcg: Added a app identifier support for
blkcg

On Mon, Oct 19, 2020 at 08:43:05PM +0530, Muneendra Kumar M wrote:
> Hi Tejun,
> Thanks for the input.
> >> +     cgrp = cgroup_get_from_kernfs_id(id);
> >> +     if (!cgrp)
> >> +             return -ENOENT;
> >> +
> > >+     css = cgroup_get_e_css(cgrp, &io_cgrp_subsys);
> >> +     if (!css)
> >> +             return -ENOENT;
> >> +
> >> +     blkcg = css_to_blkcg(css);
> >> +     if (!blkcg)
> >> +             return -ENOENT;
> > >+		return -EINVAL;
> > >+	strlcpy(blkcg->app_id, buf, len);
>
> >Shouldn't the cgrp and css be put before exit?
> [Muneendra]Correct me if iam wrong.
> You mean to add cgroup_put(cgrp) and css_put(css) before exit ?

Yeah, as-is it'd leak references each time the function is called.

Thanks.

-- 
tejun
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4177 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201019/aae8aa80/attachment.p7s>

From logang at deltatee.com  Mon Oct 19 11:40:03 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Mon, 19 Oct 2020 09:40:03 -0600
Subject: [PATCH v2 1/2] nvmet-passthru: Limit mdts by BIO_MAX_PAGES
In-Reply-To: <7fbd6b56-cda5-a9b3-7323-99d9704fbec5@nvidia.com>
References: <20201016221905.4686-1-logang@deltatee.com>
 <7fbd6b56-cda5-a9b3-7323-99d9704fbec5@nvidia.com>
Message-ID: <32db0b68-8e22-4503-e480-d7070e08617c@deltatee.com>



On 2020-10-19 5:20 a.m., Max Gurtovoy wrote:
> 
> On 10/17/2020 1:19 AM, Logan Gunthorpe wrote:
>> nvmet_passthru_map_sg() only supports mapping a single BIO, not a chain
>> so the effective maximum transfer should also be limitted by BIO_MAX_PAGES
>> (presently this works out to 1MB).
>>
>> For PCI passthru devices the max_sectors would typically be more limitting
>> than BIO_MAX_PAGES, but this may not be true for all passthru devices.
>>
>> Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
>> Suggested-by: Christoph Hellwig <hch at lst.de>
>> Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
>> Cc: Christoph Hellwig <hch at lst.de>
>> Cc: Sagi Grimberg <sagi at grimberg.me>
>> Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
>> ---
>>   drivers/nvme/target/passthru.c | 9 ++++++++-
>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
>> index dacfa7435d0b..1ab88df3310f 100644
>> --- a/drivers/nvme/target/passthru.c
>> +++ b/drivers/nvme/target/passthru.c
>> @@ -26,7 +26,7 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
>>   	struct nvme_ctrl *pctrl = ctrl->subsys->passthru_ctrl;
>>   	u16 status = NVME_SC_SUCCESS;
>>   	struct nvme_id_ctrl *id;
>> -	u32 max_hw_sectors;
>> +	int max_hw_sectors;
>>   	int page_shift;
>>   
>>   	id = kzalloc(sizeof(*id), GFP_KERNEL);
>> @@ -48,6 +48,13 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
>>   	max_hw_sectors = min_not_zero(pctrl->max_segments << (PAGE_SHIFT - 9),
>>   				      pctrl->max_hw_sectors);
>>   
>> +	/*
>> +	 * nvmet_passthru_map_sg is limitted to using a single bio so limit
>> +	 * the mdts based on BIO_MAX_PAGES as well
>> +	 */
>> +	max_hw_sectors = min_not_zero(BIO_MAX_PAGES << (PAGE_SHIFT - 9),
>> +				      max_hw_sectors);
> 
> can 9 be replaced with SECTOR_SHIFT ?

Oh, yes, that does sound better. Thanks.

Logan



From borispismenny at gmail.com  Mon Oct 19 14:28:03 2020
From: borispismenny at gmail.com (Boris Pismenny)
Date: Mon, 19 Oct 2020 21:28:03 +0300
Subject: [PATCH net-next RFC v1 05/10] nvme-tcp: Add DDP offload control
 path
In-Reply-To: <c6bb16cc-fdda-3c4e-41f6-9155911aa2c8@grimberg.me>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-6-borisp@mellanox.com>
 <c6bb16cc-fdda-3c4e-41f6-9155911aa2c8@grimberg.me>
Message-ID: <4e31f3bf-ba79-079d-a181-d7f14c843a95@gmail.com>

On 09/10/2020 1:19, Sagi Grimberg wrote:
> On 9/30/20 9:20 AM, Boris Pismenny wrote:
>> This commit introduces direct data placement offload to NVME
>> TCP. There is a context per queue, which is established after the
>> handshake
>> using the tcp_ddp_sk_add/del NDOs.
>>
>> Additionally, a resynchronization routine is used to assist
>> hardware recovery from TCP OOO, and continue the offload.
>> Resynchronization operates as follows:
>> 1. TCP OOO causes the NIC HW to stop the offload
>> 2. NIC HW identifies a PDU header at some TCP sequence number,
>> and asks NVMe-TCP to confirm it.
>> This request is delivered from the NIC driver to NVMe-TCP by first
>> finding the socket for the packet that triggered the request, and
>> then fiding the nvme_tcp_queue that is used by this routine.
>> Finally, the request is recorded in the nvme_tcp_queue.
>> 3. When NVMe-TCP observes the requested TCP sequence, it will compare
>> it with the PDU header TCP sequence, and report the result to the
>> NIC driver (tcp_ddp_resync), which will update the HW,
>> and resume offload when all is successful.
>>
>> Furthermore, we let the offloading driver advertise what is the max hw
>> sectors/segments via tcp_ddp_limits.
>>
>> A follow-up patch introduces the data-path changes required for this
>> offload.
>>
>> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
>> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
>> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
>> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
>> ---
>>   drivers/nvme/host/tcp.c  | 188 +++++++++++++++++++++++++++++++++++++++
>>   include/linux/nvme-tcp.h |   2 +
>>   2 files changed, 190 insertions(+)
>>
>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>> index 8f4f29f18b8c..06711ac095f2 100644
>> --- a/drivers/nvme/host/tcp.c
>> +++ b/drivers/nvme/host/tcp.c
>> @@ -62,6 +62,7 @@ enum nvme_tcp_queue_flags {
>>   	NVME_TCP_Q_ALLOCATED	= 0,
>>   	NVME_TCP_Q_LIVE		= 1,
>>   	NVME_TCP_Q_POLLING	= 2,
>> +	NVME_TCP_Q_OFFLOADS     = 3,
>>   };
>>   
>>   enum nvme_tcp_recv_state {
>> @@ -110,6 +111,8 @@ struct nvme_tcp_queue {
>>   	void (*state_change)(struct sock *);
>>   	void (*data_ready)(struct sock *);
>>   	void (*write_space)(struct sock *);
>> +
>> +	atomic64_t  resync_req;
>>   };
>>   
>>   struct nvme_tcp_ctrl {
>> @@ -129,6 +132,8 @@ struct nvme_tcp_ctrl {
>>   	struct delayed_work	connect_work;
>>   	struct nvme_tcp_request async_req;
>>   	u32			io_queues[HCTX_MAX_TYPES];
>> +
>> +	struct net_device       *offloading_netdev;
>>   };
>>   
>>   static LIST_HEAD(nvme_tcp_ctrl_list);
>> @@ -223,6 +228,159 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>>   	return nvme_tcp_pdu_data_left(req) <= len;
>>   }
>>   
>> +#ifdef CONFIG_TCP_DDP
>> +
>> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
>> +const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops __read_mostly = {
>> +	.resync_request		= nvme_tcp_resync_request,
>> +};
>> +
>> +static
>> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,
>> +			    struct nvme_tcp_config *config)
>> +{
>> +	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
>> +	struct tcp_ddp_config *ddp_config = (struct tcp_ddp_config *)config;
>> +	int ret;
>> +
>> +	if (unlikely(!netdev)) {
> Let's remove unlikely from non datapath routines, its slightly
> confusing.
>
>> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
> dev_info_ratelimited with queue->ctrl->ctrl.device ?
> Also, lets remove __func__. This usually is not very helpful.
>
>> +		return -EINVAL;
>> +	}
>> +
>> +	if (!(netdev->features & NETIF_F_HW_TCP_DDP)) {
>> +		dev_put(netdev);
>> +		return -EINVAL;
> EINVAL or ENODEV?
ENODEV seems more appropriate, we'll use it. Thanks
>> +	}
>> +
>> +	ret = netdev->tcp_ddp_ops->tcp_ddp_sk_add(netdev,
>> +						 queue->sock->sk,
>> +						 ddp_config);
>> +	if (!ret)
>> +		inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = &nvme_tcp_ddp_ulp_ops;
>> +	else
>> +		dev_put(netdev);
>> +	return ret;
>> +}
>> +
>> +static
>> +void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
>> +{
>> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
>> +
>> +	if (unlikely(!netdev)) {
>> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
> Same here.
>
>> +		return;
>> +	}
>> +
>> +	netdev->tcp_ddp_ops->tcp_ddp_sk_del(netdev, queue->sock->sk);
>> +
>> +	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = NULL;
> Just a general question, why is this needed?
This assignment is symmetric with the nvme_tcp_offload_socket assignment. The idea was to ensure that the functions established during offload cannot be used from this moment.

>> +	dev_put(netdev); /* put the queue_init get_netdev_for_sock() */
>> +}
>> +
>> +static
>> +int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue,
>> +			    struct tcp_ddp_limits *limits)
>> +{
>> +	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
>> +	int ret = 0;
>> +
>> +	if (unlikely(!netdev)) {
>> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
>> +		return -EINVAL;
> Same here
>
>> +	}
>> +
>> +	if (netdev->features & NETIF_F_HW_TCP_DDP &&
>> +	    netdev->tcp_ddp_ops &&
>> +	    netdev->tcp_ddp_ops->tcp_ddp_limits)
>> +			ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, limits);
>> +	else
>> +			ret = -EOPNOTSUPP;
>> +
>> +	if (!ret) {
>> +		queue->ctrl->offloading_netdev = netdev;
>> +		pr_info("%s netdev %s offload limits: max_ddp_sgl_len %d\n",
>> +			__func__, netdev->name, limits->max_ddp_sgl_len);
> dev_info, and given that it per-queue, please make it dev_dbg.
>
>> +		queue->ctrl->ctrl.max_segments = limits->max_ddp_sgl_len;
>> +		queue->ctrl->ctrl.max_hw_sectors =
>> +			limits->max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
>> +	} else {
>> +		queue->ctrl->offloading_netdev = NULL;
> Maybe nullify in the controller setup intead?
It is already set to zero after allocation (i.e., kzalloc). The goal here is to ensure it is zero in case there is a reset and it was non zero due to offload being used in the past.?

>> +	}
>> +
>> +	dev_put(netdev);
>> +
>> +	return ret;
>> +}
>> +
>> +static
>> +void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
>> +			      unsigned int pdu_seq)
>> +{
>> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
>> +	u64 resync_val;
>> +	u32 resync_seq;
>> +
>> +	if (unlikely(!netdev)) {
>> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
>> +		return;
> What happens now, fallback to SW? Maybe dev_warn then..
> will the SW keep seeing these responses after one failed?
As long as there is no resync response the system falls back to software, and this message is emitted. I'll move it to below to display it only when it is relevant and not every time this function is called.

>> +	}
>> +
>> +	resync_val = atomic64_read(&queue->resync_req);
>> +	if ((resync_val & TCP_DDP_RESYNC_REQ) == 0)
>> +		return;
>> +
>> +	resync_seq = resync_val >> 32;
>> +	if (before(pdu_seq, resync_seq))
>> +		return;
> I think it will be better to pass the skb to this func and keep the
> pdu_seq contained locally.

This requires passing the offset to obtain the sequence:
u64 pdu_seq = TCP_SKB_CB(skb)->seq + *offset - queue->pdu_offset;

It makes the interface a bit ugly, IMO.

>> +
>> +	if (atomic64_cmpxchg(&queue->resync_req, resync_val, (resync_val - 1)))
>> +		netdev->tcp_ddp_ops->tcp_ddp_resync(netdev, queue->sock->sk, pdu_seq);
> A small comment on this manipulation may help the reader.
>
>> +}
>> +
>> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
>> +{
>> +	struct nvme_tcp_queue *queue = sk->sk_user_data;
>> +
>> +	atomic64_set(&queue->resync_req,
>> +		     (((uint64_t)seq << 32) | flags));
>> +
>> +	return true;
>> +}
>> +
>> +#else
>> +
>> +static
>> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue,
>> +			    struct nvme_tcp_config *config)
>> +{
>> +	return -EINVAL;
>> +}
>> +
>> +static
>> +void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
>> +{}
>> +
>> +static
>> +int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue,
>> +			    struct tcp_ddp_limits *limits)
>> +{
>> +	return -EINVAL;
>> +}
>> +
>> +static
>> +void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
>> +			      unsigned int pdu_seq)
>> +{}
>> +
>> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
>> +{
>> +	return false;
>> +}
>> +
>> +#endif
>> +
>>   static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>>   		unsigned int dir)
>>   {
>> @@ -628,6 +786,11 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>>   	size_t rcv_len = min_t(size_t, *len, queue->pdu_remaining);
>>   	int ret;
>>   
>> +	u64 pdu_seq = TCP_SKB_CB(skb)->seq + *offset - queue->pdu_offset;
>> +
>> +	if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags))
>> +		nvme_tcp_resync_response(queue, pdu_seq);
> Here, just pass in (queue, skb)

Do you mean (queue, skb, offset)?
We need the offset for the pdu_seq calculation above.

>> +
>>   	ret = skb_copy_bits(skb, *offset,
>>   		&pdu[queue->pdu_offset], rcv_len);
>>   	if (unlikely(ret))
>> @@ -1370,6 +1533,8 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl,
>>   {
>>   	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
>>   	struct nvme_tcp_queue *queue = &ctrl->queues[qid];
>> +	struct nvme_tcp_config config;
>> +	struct tcp_ddp_limits limits;
>>   	int ret, rcv_pdu_size;
>>   
>>   	queue->ctrl = ctrl;
>> @@ -1487,6 +1652,26 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl,
>>   #endif
>>   	write_unlock_bh(&queue->sock->sk->sk_callback_lock);
>>   
>> +	if (nvme_tcp_queue_id(queue) != 0) {
> 	if (!nvme_tcp_admin_queue(queue)) {
>
>> +		config.cfg.type		= TCP_DDP_NVME;
>> +		config.pfv		= NVME_TCP_PFV_1_0;
>> +		config.cpda		= 0;
>> +		config.dgst		= queue->hdr_digest ?
>> +						NVME_TCP_HDR_DIGEST_ENABLE : 0;
>> +		config.dgst		|= queue->data_digest ?
>> +						NVME_TCP_DATA_DIGEST_ENABLE : 0;
>> +		config.queue_size	= queue->queue_size;
>> +		config.queue_id		= nvme_tcp_queue_id(queue);
>> +		config.io_cpu		= queue->io_cpu;
> Can the config initialization move to nvme_tcp_offload_socket?

Definitely. The original idea of placing it here was that the nvme-tcp handshake may influence the parameters. But, at this moment, I realize that it is orthogonal.

>> +
>> +		ret = nvme_tcp_offload_socket(queue, &config);
>> +		if (!ret)
>> +			set_bit(NVME_TCP_Q_OFFLOADS, &queue->flags);
>> +	} else {
>> +		ret = nvme_tcp_offload_limits(queue, &limits);
>> +	}
> I'm thinking that instead of this conditional, we want to place
> nvme nvme_tcp_alloc_admin_queue in nvme_tcp_alloc_admin_queue, and
> also move nvme_tcp_alloc_admin_queue to __nvme_tcp_alloc_io_queues
> loop.

We need to do it only after the socket is connected to obtain the 5-tuple and the appropriate netdev, and preferably after the protocol handshake, as there is nothing to offload there.

>> +	/* offload is opportunistic - failure is non-critical */
> Than make it void...
>
>> +
>>   	return 0;
>>   
>>   err_init_connect:
>> @@ -1519,6 +1704,9 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
>>   	kernel_sock_shutdown(queue->sock, SHUT_RDWR);
>>   	nvme_tcp_restore_sock_calls(queue);
>>   	cancel_work_sync(&queue->io_work);
>> +
>> +	if (test_bit(NVME_TCP_Q_OFFLOADS, &queue->flags))
>> +		nvme_tcp_unoffload_socket(queue);
> Why not in nvme_tcp_free_queue, symmetric to the alloc?

Sure. I've tried to keep it close to the socket disconnect, which isn't symmetrical for some reason, which suggests that these are interchangable?!

>>   }
>>   
>>   static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
>> diff --git a/include/linux/nvme-tcp.h b/include/linux/nvme-tcp.h
>> index 959e0bd9a913..65df64c34ecd 100644
>> --- a/include/linux/nvme-tcp.h
>> +++ b/include/linux/nvme-tcp.h
>> @@ -8,6 +8,8 @@
>>   #define _LINUX_NVME_TCP_H
>>   
>>   #include <linux/nvme.h>
>> +#include <net/sock.h>
>> +#include <net/tcp_ddp.h>
> Why is this needed? I think we want to place this in tcp.c no?
Not needed. It's probably leftover from previous iterations on the code. Removed for the next iteration of the patchset.


From logang at deltatee.com  Mon Oct 19 18:04:05 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Mon, 19 Oct 2020 16:04:05 -0600
Subject: [PATCH v3 1/2] nvmet-passthru: Limit mdts by BIO_MAX_PAGES
Message-ID: <20201019220406.19708-1-logang@deltatee.com>

nvmet_passthru_map_sg() only supports mapping a single BIO, not a chain
so the effective maximum transfer should also be limitted by BIO_MAX_PAGES
(presently this works out to 1MB).

For PCI passthru devices the max_sectors would typically be more limitting
than BIO_MAX_PAGES, but this may not be true for all passthru devices.

Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
Suggested-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Cc: Max Gurtovoy <mgurtovoy at nvidia.com>
---

v3: Added PG_TO_SECTOR constant which uses SECTOR_SHIFT (per Max)

 drivers/nvme/target/passthru.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index dacfa7435d0b..ed6b0586173e 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -22,11 +22,12 @@ static DEFINE_XARRAY(passthru_subsystems);

 static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
 {
+	const int PG_TO_SECTOR = PAGE_SHIFT - SECTOR_SHIFT;
 	struct nvmet_ctrl *ctrl = req->sq->ctrl;
 	struct nvme_ctrl *pctrl = ctrl->subsys->passthru_ctrl;
 	u16 status = NVME_SC_SUCCESS;
 	struct nvme_id_ctrl *id;
-	u32 max_hw_sectors;
+	int max_hw_sectors;
 	int page_shift;

 	id = kzalloc(sizeof(*id), GFP_KERNEL);
@@ -45,9 +46,16 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
 	 * which depends on the host's memory fragementation. To solve this,
 	 * ensure mdts is limited to the pages equal to the number of segments.
 	 */
-	max_hw_sectors = min_not_zero(pctrl->max_segments << (PAGE_SHIFT - 9),
+	max_hw_sectors = min_not_zero(pctrl->max_segments << PG_TO_SECTOR,
 				      pctrl->max_hw_sectors);

+	/*
+	 * nvmet_passthru_map_sg is limitted to using a single bio so limit
+	 * the mdts based on BIO_MAX_PAGES as well
+	 */
+	max_hw_sectors = min_not_zero(BIO_MAX_PAGES << PG_TO_SECTOR,
+				      max_hw_sectors);
+
 	page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12;

 	id->mdts = ilog2(max_hw_sectors) + 9 - page_shift;

base-commit: bbf5c979011a099af5dc76498918ed7df445635b
--
2.20.1


From logang at deltatee.com  Mon Oct 19 18:04:06 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Mon, 19 Oct 2020 16:04:06 -0600
Subject: [PATCH v3 2/2] nvmet-passthru: Cleanup nvmet_passthru_map_sg()
In-Reply-To: <20201019220406.19708-1-logang@deltatee.com>
References: <20201019220406.19708-1-logang@deltatee.com>
Message-ID: <20201019220406.19708-2-logang@deltatee.com>

Clean up some confusing elements of nvmet_passthru_map_sg() by returning
early if the request is greater than the maximum bio size. This allows
us to drop the sg_cnt variable.

This should not result in any functional change but makes the code
clearer and more understandable. The original code allocated a truncated
bio then would return EINVAL when bio_add_pc_page() filled that bio. The
new code just returns EINVAL early if this would happen.

Fixes: c1fef73f793b ("nvmet: add passthru code to process commands")
Signed-off-by: Logan Gunthorpe <logang at deltatee.com>
Suggested-by: Douglas Gilbert <dgilbert at interlog.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index ed6b0586173e..0e891a286fd8 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -188,18 +188,20 @@ static void nvmet_passthru_req_done(struct request *rq,
 
 static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 {
-	int sg_cnt = req->sg_cnt;
 	struct scatterlist *sg;
 	int op_flags = 0;
 	struct bio *bio;
 	int i, ret;
 
+	if (req->sg_cnt > BIO_MAX_PAGES)
+		return -EINVAL;
+
 	if (req->cmd->common.opcode == nvme_cmd_flush)
 		op_flags = REQ_FUA;
 	else if (nvme_is_write(req->cmd))
 		op_flags = REQ_SYNC | REQ_IDLE;
 
-	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
+	bio = bio_alloc(GFP_KERNEL, req->sg_cnt);
 	bio->bi_end_io = bio_put;
 	bio->bi_opf = req_op(rq) | op_flags;
 
@@ -209,7 +211,6 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 			bio_put(bio);
 			return -EINVAL;
 		}
-		sg_cnt--;
 	}
 
 	ret = blk_rq_append_bio(rq, &bio);
-- 
2.20.1



From nmeeramohide at micron.com  Mon Oct 19 18:30:02 2020
From: nmeeramohide at micron.com (Nabeel Meeramohideen Mohamed (nmeeramohide))
Date: Mon, 19 Oct 2020 22:30:02 +0000
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
 <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
Message-ID: <SN6PR08MB420843C280D54D7B5B76EB78B31E0@SN6PR08MB4208.namprd08.prod.outlook.com>

Hi Dan,

On Friday, October 16, 2020 4:12 PM, Dan Williams <dan.j.williams at intel.com> wrote:
> 
> On Fri, Oct 16, 2020 at 2:59 PM Nabeel Meeramohideen Mohamed
> (nmeeramohide) <nmeeramohide at micron.com> wrote:
> >
> > On Thursday, October 15, 2020 2:03 AM, Christoph Hellwig
> <hch at infradead.org> wrote:
> > > I don't think this belongs into the kernel.  It is a classic case for
> > > infrastructure that should be built in userspace.  If anything is
> > > missing to implement it in userspace with equivalent performance we
> > > need to improve out interfaces, although io_uring should cover pretty
> > > much everything you need.
> >
> > Hi Christoph,
> >
> > We previously considered moving the mpool object store code to user-space.
> > However, by implementing mpool as a device driver, we get several benefits
> > in terms of scalability, performance, and functionality. In doing so, we relied
> > only on standard interfaces and did not make any changes to the kernel.
> >
> > (1)  mpool's "mcache map" facility allows us to memory-map (and later unmap)
> > a collection of logically related objects with a single system call. The objects in
> > such a collection are created at different times, physically disparate, and may
> > even reside on different media class volumes.
> >
> > For our HSE storage engine application, there are commonly 10's to 100's of
> > objects in a given mcache map, and 75,000 total objects mapped at a given
> time.
> >
> > Compared to memory-mapping objects individually, the mcache map facility
> > scales well because it requires only a single system call and single
> vm_area_struct
> > to memory-map a complete collection of objects.

> Why can't that be a batch of mmap calls on io_uring?

Agreed, we could add the capability to invoke mmap via io_uring to help mitigate the
system call overhead of memory-mapping individual objects, versus our mache map
mechanism. However, there is still the scalability issue of having a vm_area_struct
for each object (versus one for each mache map).

We ran YCSB workload C in two different configurations -
Config 1: memory-mapping each individual object
Config 2: memory-mapping a collection of related objects using mcache map

- Config 1 incurred ~3.3x additional kernel memory for the vm_area_struct slab -
24.8 MB (127188 objects) for config 1, versus 7.3 MB (37482 objects) for config 2.

- Workload C exhibited around 10-25% better tail latencies (4-nines) for config 2,
not sure if it's due the reduced complexity of searching VMAs during page faults.

> > (2) The mcache map reaper mechanism proactively evicts object data from the
> page
> > cache based on object-level metrics. This provides significant performance
> benefit
> > for many workloads.
> >
> > For example, we ran YCSB workloads B (95/5 read/write mix)  and C (100% read)
> > against our HSE storage engine using the mpool driver in a 5.9 kernel.
> > For each workload, we ran with the reaper turned-on and turned-off.
> >
> > For workload B, the reaper increased throughput 1.77x, while reducing 99.99%
> tail
> > latency for reads by 39% and updates by 99%. For workload C, the reaper
> increased
> > throughput by 1.84x, while reducing the 99.99% read tail latency by 63%. These
> > improvements are even more dramatic with earlier kernels.

> What metrics proved useful and can the vanilla page cache / page
> reclaim mechanism be augmented with those metrics?

The mcache map facility is designed to cache a collection of related immutable objects
with similar lifetimes. It is best suited for storage applications that run queries against
organized collections of immutable objects, such as storage engines and DBs based on
SSTables.

Each mcache map is associated with a temperature (pinned, hot, warm, cold) and is left
to the application to tag it appropriately. For our HSE storage engine application,
the SSTables in the root/intermediate levels acts as a routing table to redirect queries to
an appropriate leaf level SSTable, in which case, the mcache map corresponding to the
root/intermediate level SSTables can be tagged as pinned/hot.

The mcache reaper tracks the access time of each object in an mcache map. On memory
pressure, the access time is compared to a time-to-live metric that?s set based on the
map?s temperature, how close is the free memory to the low and high watermarks etc.
If the object was last accessed outside the ttl window, its pages are evicted from the
page cache.

We also apply a few other techniques like throttling the readaheads and adding a delay
in the page fault handler to not overwhelm the page cache during memory pressure.

In the workloads that we run, we have noticed stalls when kswapd does the reclaim and
that impacts throughput and tail latencies as described in our last email. The mcache
reaper runs proactively and can make better reclaim decisions as it is designed to
address a specific class of workloads.

We doubt whether the same mechanisms can be employed in the vanilla page cache as
it is designed to work for a wide variety of workloads.

> > (4) mpool's immutable object model allows the driver to support concurrent
> reading
> > of object data directly and memory-mapped without a performance penalty to
> verify
> > coherence. This allows background operations, such as LSM-tree compaction,
> to
> > operate efficiently and without polluting the page cache.

> How is this different than existing background operations / defrag
> that filesystems perform today? Where are the opportunities to improve
> those operations?

We haven?t measured the benefit of eliminating the coherence check, which isn?t needed
in our case because objects are immutable. However the open(2) documentation makes
the statement that ?applications should avoid mixing mmap(2) of files with direct I/O to
the same files?, which is what we are effectively doing when we directly read from an
object that is also in an mcache map.

> > (5) Representing an mpool as a /dev/mpool/<mpool-name> device file
> provides a
> > convenient mechanism for controlling access to and managing the multiple
> storage
> > volumes, and in the future pmem devices, that may comprise an logical mpool.

> Christoph and I have talked about replacing the pmem driver's
> dependence on device-mapper for pooling. What extensions would be
> needed for the existing driver arch?

mpool doesn?t extend any of the existing driver arch to manage multiple storage volumes.

Mpool implements the concept of media classes, where each media class corresponds
to a different storage volume. Clients specify a media class when creating an object in
an mpool. mpool currently supports only two media classes, ?capacity? for storing bulk
of the objects backed by, for instance, QLC SSDs and ?staging? for storing objects
requiring lower latency/higher throughput backed by, for instance, 3DXP SSDs. 

An mpool is accessed via the /dev/mpool/<mpool-name> device file and the
mpool descriptor attached to this device file instance tracks all its associated media
class volumes. mpool relies on device mapper to provide physical device aggregation
within a media class volume.

From chaitanya.kulkarni at wdc.com  Mon Oct 19 21:44:16 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon, 19 Oct 2020 18:44:16 -0700
Subject: [PATCH 0/1] nvmet: allow user to set req alloc flag
Message-ID: <20201020014417.3604-1-chaitanya.kulkarni@wdc.com>

Hi,

When using NVMeOF target in the passthru mode we allocate the request
with BLK_MQ_REQ_NOWAIT flag. This allocates the request in the following
manner :-

nvme_alloc_request()
 blk_mq_alloc_rquest()
  blk_mq_queue_enter()
   if (flag & BLK_MQ_REQ_NOWAIT)
  	return -EBUSY; <-- return if busy.

On the NVMe controller which I've the fio random write workload running
parallel on 32 namespaces with higher queue depth results in I/O error,
where blk_mq_queue_enter() returning -EBUSY as shown above. This problem
is not easy to reproduce but occurs once in a while with following error
(See 1 for detailed log) :-

test1: (groupid=0, jobs=32): err= 5 
(file:io_u.c:1744, func=io_u error, error=Input/output error):

When the flag BLK_MQ_REQ_NOWAIT is removed from the allocation the
workload doen't result in the error.

This patch fixes the problem with the request allocation by adding
a new configfs attribute so that user can optionally decide whether
to use BLK_MQ_REQ_NOWAIT or not. We retain the default behavior by
using BLK_MQ_REQ_NOWAIT when creating the nvmet passthru subsystem.

Regards,
Chaitanya

Chaitanya Kulkarni (1):
  nvmet: allow user to set req alloc flag

 drivers/nvme/target/configfs.c | 36 ++++++++++++++++++++++++++++++++++
 drivers/nvme/target/core.c     |  3 +++
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c |  3 ++-
 4 files changed, 42 insertions(+), 1 deletion(-)

1. FIO workload resulting in the I/O error with default behavior :-
-------------------------------------------------------------------

# cat nvmet-req-nowait-1.fio.log 
fio-3.8-5-g464b
Starting 32 processes

test1: (groupid=0, jobs=32): err= 5 (file:io_u.c:1744, func=io_u error, error=Input/output error): pid=4580: Mon Oct 19 16:07:42 2020
  write: IOPS=871k, BW=3403MiB/s (3569MB/s)(1664MiB/489msec)
    slat (usec): min=6, max=17846, avg=33.13, stdev=228.94
    clat (usec): min=50, max=40690, avg=4618.04, stdev=1154.78
     lat (usec): min=194, max=40702, avg=4651.20, stdev=1170.03
    clat percentiles (usec):
     |  1.00th=[ 3195],  5.00th=[ 3621], 10.00th=[ 3720], 20.00th=[ 3752],
     | 30.00th=[ 3818], 40.00th=[ 3884], 50.00th=[ 3982], 60.00th=[ 4359],
     | 70.00th=[ 5604], 80.00th=[ 5735], 90.00th=[ 5997], 95.00th=[ 6194],
     | 99.00th=[ 6915], 99.50th=[ 7504], 99.90th=[16057], 99.95th=[17695],
     | 99.99th=[22938]
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.21%, 4=50.40%, 10=49.19%, 20=0.09%, 50=0.04%
  cpu          : usr=4.07%, sys=34.86%, ctx=5296, majf=0, minf=393
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.2%, >=64=99.5%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,426154,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=3403MiB/s (3569MB/s), 3403MiB/s-3403MiB/s (3569MB/s-3569MB/s), io=1664MiB (1745MB), run=489-489msec

Disk stats (read/write):
  nvme2n1: ios=0/9329, merge=0/0, ticks=0/16477, in_queue=16478, util=1.81%
  nvme2n2: ios=40/7807, merge=0/0, ticks=3/13759, in_queue=13763, util=1.92%
  nvme2n3: ios=40/7577, merge=0/0, ticks=7/17245, in_queue=17253, util=1.98%
  nvme2n4: ios=40/7647, merge=0/0, ticks=7/27139, in_queue=27147, util=2.14%
  nvme2n5: ios=40/8678, merge=0/0, ticks=5/20387, in_queue=20393, util=2.16%
  nvme2n6: ios=40/9069, merge=0/0, ticks=7/26973, in_queue=26981, util=2.29%
  nvme2n7: ios=40/6509, merge=0/0, ticks=8/19087, in_queue=19095, util=2.34%
  nvme2n8: ios=40/9491, merge=0/0, ticks=4/19053, in_queue=19058, util=2.42%
  nvme2n9: ios=40/6604, merge=0/0, ticks=4/27047, in_queue=27051, util=2.43%
  nvme2n10: ios=40/9525, merge=0/0, ticks=8/27259, in_queue=27268, util=2.60%
  nvme2n11: ios=40/8255, merge=0/0, ticks=7/27218, in_queue=27226, util=2.65%
  nvme2n12: ios=40/7239, merge=0/0, ticks=5/26904, in_queue=26910, util=2.75%
  nvme2n13: ios=40/7588, merge=0/0, ticks=6/18549, in_queue=18556, util=2.75%
  nvme2n14: ios=40/7474, merge=0/0, ticks=5/18387, in_queue=18392, util=2.91%
  nvme2n15: ios=40/7293, merge=0/0, ticks=4/14218, in_queue=14223, util=3.23%
  nvme2n16: ios=0/8255, merge=0/0, ticks=0/27107, in_queue=27108, util=3.39%
  nvme2n17: ios=0/8383, merge=0/0, ticks=0/27563, in_queue=27563, util=3.79%
  nvme2n18: ios=0/7900, merge=0/0, ticks=0/27229, in_queue=27229, util=3.94%
  nvme2n19: ios=0/9406, merge=0/0, ticks=0/14151, in_queue=14152, util=4.37%
  nvme2n20: ios=0/9022, merge=0/0, ticks=0/27223, in_queue=27223, util=4.89%
  nvme2n21: ios=0/7231, merge=0/0, ticks=0/14102, in_queue=14102, util=4.98%
  nvme2n22: ios=0/8891, merge=0/0, ticks=0/27090, in_queue=27090, util=5.52%
  nvme2n23: ios=0/6223, merge=0/0, ticks=0/27066, in_queue=27067, util=5.54%
  nvme2n24: ios=0/7366, merge=0/0, ticks=0/27291, in_queue=27291, util=6.09%
  nvme2n25: ios=0/7301, merge=0/0, ticks=0/26984, in_queue=26985, util=7.55%
  nvme2n26: ios=0/7716, merge=0/0, ticks=0/19097, in_queue=19098, util=8.43%
  nvme2n27: ios=0/9144, merge=0/0, ticks=0/27355, in_queue=27356, util=10.98%
  nvme2n28: ios=0/6591, merge=0/0, ticks=0/14058, in_queue=14058, util=12.28%
  nvme2n29: ios=0/8915, merge=0/0, ticks=0/27237, in_queue=27237, util=17.74%
  nvme2n30: ios=0/7098, merge=0/0, ticks=0/25610, in_queue=25610, util=28.49%
  nvme2n31: ios=0/6404, merge=0/0, ticks=0/21010, in_queue=21011, util=31.86%
  nvme2n32: ios=0/9678, merge=0/0, ticks=0/19048, in_queue=19049, util=73.70%

1. FIO workload without any error when not using BLK_MQ_REQ_NOWAIT :-
---------------------------------------------------------------------

# cat nvmet-req-nowait-0.fio.log 
fio-3.8-5-g464b
Starting 32 processes

test1: (groupid=0, jobs=32): err= 0: pid=4665: Mon Oct 19 16:09:06 2020
  write: IOPS=832k, BW=3250MiB/s (3408MB/s)(195GiB/61570msec)
    slat (usec): min=6, max=1984.3k, avg=34.57, stdev=1293.12
    clat (usec): min=42, max=3012.5k, avg=4804.90, stdev=16251.84
     lat (usec): min=80, max=3012.5k, avg=4839.58, stdev=16320.43
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    4], 10.00th=[    4], 20.00th=[    5],
     | 30.00th=[    5], 40.00th=[    5], 50.00th=[    5], 60.00th=[    5],
     | 70.00th=[    5], 80.00th=[    5], 90.00th=[    7], 95.00th=[    7],
     | 99.00th=[    7], 99.50th=[    8], 99.90th=[   13], 99.95th=[   19],
     | 99.99th=[  776]
   bw (  KiB/s): min=   56, max=132007, per=3.23%, avg=107439.18, stdev=20819.41, samples=3790
   iops        : min=   14, max=33001, avg=26859.75, stdev=5204.86, samples=3790
  lat (usec)   : 50=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  lat (usec)   : 1000=0.01%
  lat (msec)   : 2=0.01%, 4=13.50%, 10=86.26%, 20=0.19%, 50=0.01%
  lat (msec)   : 100=0.01%, 250=0.01%, 500=0.02%, 750=0.01%, 1000=0.01%
  cpu          : usr=3.89%, sys=33.38%, ctx=5596245, majf=0, minf=438
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,51228543,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=3250MiB/s (3408MB/s), 3250MiB/s-3250MiB/s (3408MB/s-3408MB/s), io=195GiB (210GB), run=61570-61570msec

Disk stats (read/write):
  nvme2n1: ios=96/1583859, merge=0/0, ticks=13/431764, in_queue=431778, util=74.77%
  nvme2n2: ios=80/1626714, merge=0/0, ticks=9/441432, in_queue=441441, util=77.34%
  nvme2n3: ios=0/1656183, merge=0/0, ticks=0/468030, in_queue=468030, util=79.37%
  nvme2n4: ios=80/1557939, merge=0/0, ticks=8/424431, in_queue=424439, util=73.46%
  nvme2n5: ios=80/1649598, merge=0/0, ticks=8/428050, in_queue=428059, util=79.85%
  nvme2n6: ios=40/1609485, merge=0/0, ticks=4/426484, in_queue=426488, util=79.08%
  nvme2n7: ios=89/1583523, merge=0/0, ticks=11/428733, in_queue=428744, util=76.78%
  nvme2n8: ios=96/1582927, merge=0/0, ticks=11/423911, in_queue=423922, util=77.75%
  nvme2n9: ios=96/1666151, merge=0/0, ticks=13/447659, in_queue=447671, util=81.52%
  nvme2n10: ios=96/1524152, merge=0/0, ticks=12/420001, in_queue=420013, util=76.71%
  nvme2n11: ios=96/1630334, merge=0/0, ticks=14/449232, in_queue=449246, util=80.11%
  nvme2n12: ios=96/1582044, merge=0/0, ticks=11/433235, in_queue=433246, util=79.40%
  nvme2n13: ios=96/1663883, merge=0/0, ticks=10/477186, in_queue=477196, util=84.84%
  nvme2n14: ios=96/1522140, merge=0/0, ticks=14/442828, in_queue=442842, util=76.87%
  nvme2n15: ios=80/1564809, merge=0/0, ticks=11/437801, in_queue=437812, util=80.18%
  nvme2n16: ios=80/1596635, merge=0/0, ticks=10/441174, in_queue=441183, util=82.13%
  nvme2n17: ios=80/1646794, merge=0/0, ticks=7/467660, in_queue=467667, util=86.82%
  nvme2n18: ios=80/1589903, merge=0/0, ticks=11/449100, in_queue=449112, util=83.04%
  nvme2n19: ios=80/1613331, merge=0/0, ticks=12/433444, in_queue=433456, util=86.20%
  nvme2n20: ios=80/1594185, merge=0/0, ticks=9/433980, in_queue=433988, util=87.45%
  nvme2n21: ios=80/1566221, merge=0/0, ticks=11/430998, in_queue=431009, util=86.10%
  nvme2n22: ios=80/1636843, merge=0/0, ticks=8/462546, in_queue=462554, util=87.66%
  nvme2n23: ios=80/1522876, merge=0/0, ticks=13/441547, in_queue=441560, util=84.79%
  nvme2n24: ios=80/1636153, merge=0/0, ticks=8/465240, in_queue=465248, util=90.78%
  nvme2n25: ios=80/1507958, merge=0/0, ticks=9/419422, in_queue=419431, util=83.93%
  nvme2n26: ios=0/1536856, merge=0/0, ticks=0/428050, in_queue=428050, util=88.16%
  nvme2n27: ios=0/1608849, merge=0/0, ticks=0/457471, in_queue=457470, util=91.70%
  nvme2n28: ios=0/1607794, merge=0/0, ticks=0/467126, in_queue=467126, util=94.52%
  nvme2n29: ios=0/1623665, merge=0/0, ticks=0/432039, in_queue=432039, util=93.98%
  nvme2n30: ios=0/1633017, merge=0/0, ticks=0/441836, in_queue=441836, util=97.04%
  nvme2n31: ios=0/1644927, merge=0/0, ticks=0/455211, in_queue=455212, util=96.53%
  nvme2n32: ios=0/1582765, merge=0/0, ticks=0/447856, in_queue=447856, util=94.97%
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Oct 19 21:44:17 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon, 19 Oct 2020 18:44:17 -0700
Subject: [PATCH 1/1] nvmet: allow user to set req alloc flag
In-Reply-To: <20201020014417.3604-1-chaitanya.kulkarni@wdc.com>
References: <20201020014417.3604-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201020014417.3604-2-chaitanya.kulkarni@wdc.com>

By default, we set the passthru request allocation flag such that it
returns the error in the following code patch and we fail the I/O when
BLK_MQ_REQ_NOWAIT is set:-

nvme_alloc_request()
??blk_mq_alloc_request()
?? blk_mq_queue_enter()
?? ??if (flag & BLK_MQ_REQ_NOWAIT)
?? ?? ?? ?? return -EBUSY; <-- return if busy.

On some controllers using BLK_MQ_REQ_NOWAIT ends up in I/O error where
the controller is perfectly healthy and not in a degraded state.

Block layer request allocation does allow us to wait instead of
immediately returning the error when we BLK_MQ_REQ_NOWAIT flag is not 
used. This has shown to fix the I/O error problem reported under
heavy random write workload.

This patch fixes the problem with the request allocation by adding
a new configfs attribute so that user can optionally decide whether
to use BLK_MQ_REQ_NOWAIT or not. We retain the default behavior by
using BLK_MQ_REQ_NOWAIT when creating the nvmet passthru subsystem.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/configfs.c | 36 ++++++++++++++++++++++++++++++++++
 drivers/nvme/target/core.c     |  3 +++
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c |  3 ++-
 4 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 37e1d7784e17..3892b8fb0ff6 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -6,6 +6,7 @@
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/blk-mq.h>
 #include <linux/slab.h>
 #include <linux/stat.h>
 #include <linux/ctype.h>
@@ -736,9 +737,44 @@ static ssize_t nvmet_passthru_enable_store(struct config_item *item,
 }
 CONFIGFS_ATTR(nvmet_passthru_, enable);
 
+static ssize_t nvmet_passthru_req_nowait_show(struct config_item *item,
+						char *page)
+{
+	struct nvmet_subsys *subsys = to_subsys(item->ci_parent);
+
+	return snprintf(page, PAGE_SIZE, "%d\n", subsys->req_nowait);
+}
+
+static ssize_t nvmet_passthru_req_nowait_store(struct config_item *item,
+		const char *page, size_t count)
+{
+	struct nvmet_subsys *subsys = to_subsys(item->ci_parent);
+	bool req_nowait;
+	int ret = 0;
+
+	mutex_lock(&subsys->lock);
+	if (subsys->passthru_ctrl) {
+		pr_err("disable passthru ctrl before setting req_nowait\n");
+		ret = -EBUSY;
+		goto out;
+	}
+
+	if (strtobool(page, &req_nowait)) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	subsys->req_nowait = req_nowait ? BLK_MQ_REQ_NOWAIT : 0;
+out:
+	mutex_unlock(&subsys->lock);
+	return ret ? ret : count;
+}
+CONFIGFS_ATTR(nvmet_passthru_, req_nowait);
+
 static struct configfs_attribute *nvmet_passthru_attrs[] = {
 	&nvmet_passthru_attr_device_path,
 	&nvmet_passthru_attr_enable,
+	&nvmet_passthru_attr_req_nowait,
 	NULL,
 };
 
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index aafcbc424b7a..50634945bd42 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1511,6 +1511,9 @@ struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
 	}
 	subsys->cntlid_min = NVME_CNTLID_MIN;
 	subsys->cntlid_max = NVME_CNTLID_MAX;
+#ifdef NVME_TARGET_PASSTHRU
+	subsys->req_nowait = BLK_MQ_REQ_NOWAIT;
+#endif
 	kref_init(&subsys->ref);
 
 	mutex_init(&subsys->lock);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 559a15ccc322..a7ca8f5860c9 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -249,6 +249,7 @@ struct nvmet_subsys {
 	struct nvme_ctrl	*passthru_ctrl;
 	char			*passthru_ctrl_path;
 	struct config_group	passthru_group;
+	blk_mq_req_flags_t	req_nowait;
 #endif /* CONFIG_NVME_TARGET_PASSTHRU */
 };
 
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 56c571052216..7a2ce00f7a25 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -215,6 +215,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 
 static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 {
+	blk_mq_req_flags_t req_nowait = req->sq->ctrl->subsys->req_nowait;
 	struct nvme_ctrl *ctrl = nvmet_req_passthru_ctrl(req);
 	struct request_queue *q = ctrl->admin_q;
 	struct nvme_ns *ns = NULL;
@@ -236,7 +237,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		q = ns->queue;
 	}
 
-	rq = nvme_alloc_request(q, req->cmd, BLK_MQ_REQ_NOWAIT, NVME_QID_ANY);
+	rq = nvme_alloc_request(q, req->cmd, req_nowait, NVME_QID_ANY);
 	if (IS_ERR(rq)) {
 		status = NVME_SC_INTERNAL;
 		goto out_put_ns;
-- 
2.22.1



From logang at deltatee.com  Mon Oct 19 22:05:36 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Mon, 19 Oct 2020 20:05:36 -0600
Subject: [PATCH v2 1/2] nvmet-passthru: Limit mdts by BIO_MAX_PAGES
In-Reply-To: <BYAPR04MB4965B1360F5F6A1D13FF1DD0861F0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20201016221905.4686-1-logang@deltatee.com>
 <7fbd6b56-cda5-a9b3-7323-99d9704fbec5@nvidia.com>
 <32db0b68-8e22-4503-e480-d7070e08617c@deltatee.com>
 <BYAPR04MB4965B1360F5F6A1D13FF1DD0861F0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <efa5f46a-cb8f-11c5-1faf-c9a4cf3e0792@deltatee.com>



On 2020-10-19 8:03 p.m., Chaitanya Kulkarni wrote:
> On 10/19/20 08:40, Logan Gunthorpe wrote:
>>>> +	max_hw_sectors = min_not_zero(BIO_MAX_PAGES << (PAGE_SHIFT - 9),
>>>> +				      max_hw_sectors);
>>> can 9 be replaced with SECTOR_SHIFT ?
>> Oh, yes, that does sound better. Thanks.
>>
>> Logan
>>
>>
> Although it is a good suggestion we are not using the SECTOR_SIZE in
> 
> the target code, I don't think we should change that for passthru. Also,
> 
> I remember that on the block layer mailing list SECTOR_SIZE macro been
> 
> discouraged over use of 9.

Well in that case v2 or v3 can be merged depending on maintainer preference.

Logan


From yi.zhang at redhat.com  Tue Oct 20 03:32:44 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Tue, 20 Oct 2020 03:32:44 -0400 (EDT)
Subject: [PATCH 0/4] blk-mq/nvme-tcp: fix timed out related races
In-Reply-To: <20201016142811.1262214-1-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
Message-ID: <826927458.4718534.1603179164356.JavaMail.zimbra@redhat.com>

Thanks Ming, feel free to add:
Tested-by: Yi Zhang <yi.zhang at redhat.com>

For the timeout issue, I've filed below issue to track it, thanks.
https://bugzilla.kernel.org/show_bug.cgi?id=209763

Best Regards,
  Yi Zhang


----- Original Message -----
From: "Ming Lei" <ming.lei at redhat.com>
To: "Jens Axboe" <axboe at kernel.dk>, linux-block at vger.kernel.org, linux-nvme at lists.infradead.org, "Christoph Hellwig" <hch at lst.de>, "Keith Busch" <kbusch at kernel.org>
Cc: "Yi Zhang" <yi.zhang at redhat.com>, "Sagi Grimberg" <sagi at grimberg.me>, "Chao Leng" <lengchao at huawei.com>, "Ming Lei" <ming.lei at redhat.com>
Sent: Friday, October 16, 2020 10:28:07 PM
Subject: [PATCH 0/4] blk-mq/nvme-tcp: fix timed out related races

Hi,

The 1st 2 patches fixes request completion related races.

The 2nd 3 patches fixes/improves nvme-tcp error recovery.

With the 4 patches, nvme/012 can pass on nvme-tcp in Zhang Yi's test
machine.


Ming Lei (4):
  blk-mq: check rq->state explicitly in
    blk_mq_tagset_count_completed_rqs
  blk-mq: think request as completed if it isn't IN_FLIGHT.
  nvme: tcp: fix race between timeout and normal completion
  nvme: tcp: complete non-IO requests atomically

 block/blk-flush.c       |  2 ++
 block/blk-mq-tag.c      |  2 +-
 drivers/nvme/host/tcp.c | 76 +++++++++++++++++++++++++++++------------
 include/linux/blk-mq.h  |  8 ++++-
 4 files changed, 65 insertions(+), 23 deletions(-)

CC: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Yi Zhang <yi.zhang at redhat.com>


-- 
2.25.2


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme



From sagi at grimberg.me  Tue Oct 20 04:11:11 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 20 Oct 2020 01:11:11 -0700
Subject: [PATCH 3/4] nvme: tcp: fix race between timeout and normal
 completion
In-Reply-To: <20201016142811.1262214-4-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
 <20201016142811.1262214-4-ming.lei@redhat.com>
Message-ID: <e9d2e28e-fb55-358c-3e8c-6f3e9dd91c25@grimberg.me>


> NVMe TCP timeout handler allows to abort request directly when the
> controller isn't in LIVE state. nvme_tcp_error_recovery() updates
> controller state as RESETTING, and schedule reset work function. If
> new timeout comes before the work function is called, the new timedout
> request will be aborted directly, however at that time, the controller
> isn't shut down yet, then timeout abort vs. normal completion race
> will be triggered.

This assertion is incorrect, the before completing the request from
the timeout handler, we call nvme_tcp_stop_queue, which guarantees upon
return that no more completions will be seen from this queue.

> Fix the race by the following approach:
> 
> 1) aborting timed out request directly only in case that controller is in
> CONNECTING and DELETING state. In the two states, controller has been shutdown,
> so it is safe to do so; Also, it is enough to recovery controller in this way,
> because we only stop/destroy queues during RESETTING, and cancel all in-flight
> requests, no new request is required in RESETTING.

Unfortunately RESETTING also requires direct completion because this
state may include I/O that may timeout and unless we complete it
the reset flow cannot make forward progress
(nvme_disable_ctrl/nvme_shutdown_ctrl generate I/O in fabrics).

> 
> 2) delay unquiesce io queues and admin queue until controller is LIVE
> because it isn't necessary to start queues during RESETTING. Instead,
> this way may risk timeout vs. normal completion race because we need
> to abort timed-out request directly during CONNECTING state for setting
> up controller.

We can't unquisce I/O only when the controller is LIVE because I/O needs
to be able to failover for multipath, which should not be tied with
the controller becoming LIVE again what-so-ever...


From sagi at grimberg.me  Tue Oct 20 04:14:54 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 20 Oct 2020 01:14:54 -0700
Subject: [PATCH 4/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <20201016142811.1262214-5-ming.lei@redhat.com>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
 <20201016142811.1262214-5-ming.lei@redhat.com>
Message-ID: <7a63d842-a1d3-6452-9961-f22c212bf545@grimberg.me>


> During controller's CONNECTING state, admin/fabric/connect requests
> are submitted for recovery, and we allow to abort this request directly
> in time out handler for not blocking the setup step.
> 
> So timout vs. normal completion race may be triggered on these requests.
> 
> Add atomic completion for requests from connect/fabric/admin queue for
> avoiding the race.

I don't understand why this is needed... I don't understand the race,
and I don't understand why fencing the queue with nvme_tcp_stop_queue
and serialization against the teardown flow is not sufficient.

> 
> CC: Chao Leng <lengchao at huawei.com>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Reported-by: Yi Zhang <yi.zhang at redhat.com>
> Signed-off-by: Ming Lei <ming.lei at redhat.com>
> ---
>   drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
>   1 file changed, 37 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 56ac61a90c1b..654061abdc5a 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -30,6 +30,8 @@ static int so_priority;
>   module_param(so_priority, int, 0644);
>   MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
>   
> +#define REQ_STATE_COMPLETE     0
> +
>   enum nvme_tcp_send_state {
>   	NVME_TCP_SEND_CMD_PDU = 0,
>   	NVME_TCP_SEND_H2C_PDU,
> @@ -56,6 +58,8 @@ struct nvme_tcp_request {
>   	size_t			offset;
>   	size_t			data_sent;
>   	enum nvme_tcp_send_state state;
> +
> +	unsigned long		comp_state;
>   };
>   
>   enum nvme_tcp_queue_flags {
> @@ -469,6 +473,33 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
>   	queue_work(nvme_reset_wq, &to_tcp_ctrl(ctrl)->err_work);
>   }
>   
> +/*
> + * requests originated from admin, fabrics and connect_q have to be
> + * completed atomically because we don't cover the race between timeout
> + * and normal completion for these queues.
> + */
> +static inline bool nvme_tcp_need_atomic_complete(struct request *rq)
> +{
> +	return !rq->rq_disk;
> +}
> +
> +static inline void nvme_tcp_clear_rq_complete(struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +
> +	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
> +		clear_bit(REQ_STATE_COMPLETE, &req->comp_state);
> +}
> +
> +static inline bool nvme_tcp_mark_rq_complete(struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +
> +	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
> +		return !test_and_set_bit(REQ_STATE_COMPLETE, &req->comp_state);
> +	return true;
> +}
> +
>   static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		struct nvme_completion *cqe)
>   {
> @@ -483,7 +514,8 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		return -EINVAL;
>   	}
>   
> -	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> +	if (nvme_tcp_mark_rq_complete(rq) &&
> +			!nvme_try_complete_req(rq, cqe->status, cqe->result))
>   		nvme_complete_rq(rq);
>   	queue->nr_cqe++;
>   
> @@ -674,7 +706,8 @@ static inline void nvme_tcp_end_request(struct request *rq, u16 status)
>   {
>   	union nvme_result res = {};
>   
> -	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> +	if (nvme_tcp_mark_rq_complete(rq) &&
> +			!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
>   		nvme_complete_rq(rq);
>   }
>   
> @@ -2173,7 +2206,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>   	/* fence other contexts that may complete the command */
>   	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
> -	if (!blk_mq_request_completed(rq)) {
> +	if (nvme_tcp_mark_rq_complete(rq) && !blk_mq_request_completed(rq)) {
>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>   		blk_mq_complete_request(rq);
>   	}
> @@ -2315,6 +2348,7 @@ static blk_status_t nvme_tcp_queue_rq(struct blk_mq_hw_ctx *hctx,
>   	if (unlikely(ret))
>   		return ret;
>   
> +	nvme_tcp_clear_rq_complete(rq);
>   	blk_mq_start_request(rq);
>   
>   	nvme_tcp_queue_request(req, true, bd->last);
> 


From sagi at grimberg.me  Tue Oct 20 04:18:04 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 20 Oct 2020 01:18:04 -0700
Subject: [PATCH v3 1/2] nvmet-passthru: Limit mdts by BIO_MAX_PAGES
In-Reply-To: <20201019220406.19708-1-logang@deltatee.com>
References: <20201019220406.19708-1-logang@deltatee.com>
Message-ID: <55a1515e-6051-c7d2-dc7a-003a0f61f98f@grimberg.me>


> diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
> index dacfa7435d0b..ed6b0586173e 100644
> --- a/drivers/nvme/target/passthru.c
> +++ b/drivers/nvme/target/passthru.c
> @@ -22,11 +22,12 @@ static DEFINE_XARRAY(passthru_subsystems);
> 
>   static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
>   {
> +	const int PG_TO_SECTOR = PAGE_SHIFT - SECTOR_SHIFT;
>   	struct nvmet_ctrl *ctrl = req->sq->ctrl;
>   	struct nvme_ctrl *pctrl = ctrl->subsys->passthru_ctrl;
>   	u16 status = NVME_SC_SUCCESS;
>   	struct nvme_id_ctrl *id;
> -	u32 max_hw_sectors;
> +	int max_hw_sectors;
>   	int page_shift;
> 
>   	id = kzalloc(sizeof(*id), GFP_KERNEL);
> @@ -45,9 +46,16 @@ static u16 nvmet_passthru_override_id_ctrl(struct nvmet_req *req)
>   	 * which depends on the host's memory fragementation. To solve this,
>   	 * ensure mdts is limited to the pages equal to the number of segments.
>   	 */
> -	max_hw_sectors = min_not_zero(pctrl->max_segments << (PAGE_SHIFT - 9),
> +	max_hw_sectors = min_not_zero(pctrl->max_segments << PG_TO_SECTOR,
>   				      pctrl->max_hw_sectors);

I also liked the previous one better...


From sagi at grimberg.me  Tue Oct 20 04:22:35 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 20 Oct 2020 01:22:35 -0700
Subject: [PATCH 0/1] nvmet: allow user to set req alloc flag
In-Reply-To: <20201020014417.3604-1-chaitanya.kulkarni@wdc.com>
References: <20201020014417.3604-1-chaitanya.kulkarni@wdc.com>
Message-ID: <74e2bc85-3752-08fe-4f2c-07130ab4411d@grimberg.me>


> Hi,
> 
> When using NVMeOF target in the passthru mode we allocate the request
> with BLK_MQ_REQ_NOWAIT flag. This allocates the request in the following
> manner :-
> 
> nvme_alloc_request()
>   blk_mq_alloc_rquest()
>    blk_mq_queue_enter()
>     if (flag & BLK_MQ_REQ_NOWAIT)
>    	return -EBUSY; <-- return if busy.
> 
> On the NVMe controller which I've the fio random write workload running
> parallel on 32 namespaces with higher queue depth results in I/O error,
> where blk_mq_queue_enter() returning -EBUSY as shown above. This problem
> is not easy to reproduce but occurs once in a while with following error
> (See 1 for detailed log) :-
> 
> test1: (groupid=0, jobs=32): err= 5
> (file:io_u.c:1744, func=io_u error, error=Input/output error):
> 
> When the flag BLK_MQ_REQ_NOWAIT is removed from the allocation the
> workload doen't result in the error.
> 
> This patch fixes the problem with the request allocation by adding
> a new configfs attribute so that user can optionally decide whether
> to use BLK_MQ_REQ_NOWAIT or not. We retain the default behavior by
> using BLK_MQ_REQ_NOWAIT when creating the nvmet passthru subsystem.

Why should we ever set REQ_NOWAIT at all? Nothing prevents the
host(s) queue depth from exceeding the controller queue depth...


From ming.lei at redhat.com  Tue Oct 20 04:52:57 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:52:57 +0800
Subject: [PATCH V2 0/4] blk-mq/nvme-tcp: fix timed out related races 
Message-ID: <20201020085301.1553959-1-ming.lei@redhat.com>

Hi,

The 1st 2 patches fixes request completion related races.

The 2nd 3 patches fixes/improves nvme-tcp error recovery.

With the 4 patches, nvme/012 can pass on nvme-tcp in Zhang Yi's test
machine.

V2:
	- re-order patch3 and patch4
	- fix comment
	- improve patch "nvme: tcp: fix race between timeout and normal completion"


Ming Lei (4):
  blk-mq: check rq->state explicitly in
    blk_mq_tagset_count_completed_rqs
  blk-mq: fix blk_mq_request_completed
  nvme: tcp: complete non-IO requests atomically
  nvme: tcp: fix race between timeout and normal completion

 block/blk-flush.c       |  2 +
 block/blk-mq-tag.c      |  2 +-
 drivers/nvme/host/tcp.c | 98 ++++++++++++++++++++++++++++++++---------
 include/linux/blk-mq.h  |  8 +++-
 4 files changed, 86 insertions(+), 24 deletions(-)

CC: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Yi Zhang <yi.zhang at redhat.com>
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:52:58 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:52:58 +0800
Subject: [PATCH V2 1/4] blk-mq: check rq->state explicitly in
 blk_mq_tagset_count_completed_rqs
In-Reply-To: <20201020085301.1553959-1-ming.lei@redhat.com>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
Message-ID: <20201020085301.1553959-2-ming.lei@redhat.com>

blk_mq_tagset_count_completed_rqs() is called from
blk_mq_tagset_wait_completed_request() for draining requests being
completed remotely. What we need to do is to make sure that request->state
is switched to non-MQ_RQ_COMPLETE.

So check MQ_RQ_COMPLETE explicitly in blk_mq_tagset_count_completed_rqs().

Meantime mark flush request as IDLE in its .end_io() for aligning to
end normal request because flush request may stay in inflight tags in case
of !elevator, so we need to change its state into IDLE.

Cc: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Reported-by: Yi Zhang <yi.zhang at redhat.com>
Tested-by: Yi Zhang <yi.zhang at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 block/blk-flush.c  | 2 ++
 block/blk-mq-tag.c | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/block/blk-flush.c b/block/blk-flush.c
index 53abb5c73d99..f6a07ae533c9 100644
--- a/block/blk-flush.c
+++ b/block/blk-flush.c
@@ -231,6 +231,8 @@ static void flush_end_io(struct request *flush_rq, blk_status_t error)
 		return;
 	}
 
+	WRITE_ONCE(flush_rq->state, MQ_RQ_IDLE);
+
 	if (fq->rq_status != BLK_STS_OK)
 		error = fq->rq_status;
 
diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index 9c92053e704d..10ff8968b93b 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -367,7 +367,7 @@ static bool blk_mq_tagset_count_completed_rqs(struct request *rq,
 {
 	unsigned *count = data;
 
-	if (blk_mq_request_completed(rq))
+	if (blk_mq_rq_state(rq) == MQ_RQ_COMPLETE)
 		(*count)++;
 	return true;
 }
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:52:59 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:52:59 +0800
Subject: [PATCH V2 2/4] blk-mq: fix blk_mq_request_completed
In-Reply-To: <20201020085301.1553959-1-ming.lei@redhat.com>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
Message-ID: <20201020085301.1553959-3-ming.lei@redhat.com>

MQ_RQ_COMPLETE is one transient state, because the .complete callback
ends or requeues this request, then the request state is updated to
IDLE from the .complete callback.

blk_mq_request_completed() is often used by driver for avoiding
double completion with help of driver's specific sync approach. Such as,
NVMe TCP calls blk_mq_request_completed() in its timeout handler
and abort handler for avoiding double completion. If request's state
is updated to IDLE in either one, another code path may think this
request as not completed, and will complete it one more time. Then
double completion is triggered.

Yi reported[1] that 'refcount_t: underflow; use-after-free' of rq->ref
is triggered in blktests(nvme/012) on one very slow machine.

Fixes this issue by thinking request as completed if its state becomes
not IN_FLIGHT.

Reported-by: Yi Zhang <yi.zhang at redhat.com>
Tested-by: Yi Zhang <yi.zhang at redhat.com>
Cc: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 include/linux/blk-mq.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 90da3582b91d..9a67408f79d9 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -478,9 +478,15 @@ static inline int blk_mq_request_started(struct request *rq)
 	return blk_mq_rq_state(rq) != MQ_RQ_IDLE;
 }
 
+/*
+ * It is often called in abort handler for avoiding double completion,
+ * MQ_RQ_COMPLETE is one transient state because .complete callback
+ * may end or requeue this request, in either way the request is marked
+ * as IDLE. So return true if this request's state become not IN_FLIGHT.
+ */
 static inline int blk_mq_request_completed(struct request *rq)
 {
-	return blk_mq_rq_state(rq) == MQ_RQ_COMPLETE;
+	return blk_mq_rq_state(rq) != MQ_RQ_IN_FLIGHT;
 }
 
 void blk_mq_start_request(struct request *rq);
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:53:00 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:53:00 +0800
Subject: [PATCH V2 3/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <20201020085301.1553959-1-ming.lei@redhat.com>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
Message-ID: <20201020085301.1553959-4-ming.lei@redhat.com>

During controller's CONNECTING state, admin/fabric/connect requests
are submitted for recovery controller, and we allow to abort this request
directly in time out handler for not blocking setup procedure.

So timout vs. normal completion race exists on these requests since
admin/fabirc/connect queues won't be shutdown before handling timeout
during CONNECTING state.

Add atomic completion for requests from connect/fabric/admin queue for
avoiding the race.

CC: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Reported-by: Yi Zhang <yi.zhang at redhat.com>
Tested-by: Yi Zhang <yi.zhang at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 37 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index d6a3e1487354..7e85bd4a8d1b 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -30,6 +30,8 @@ static int so_priority;
 module_param(so_priority, int, 0644);
 MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
 
+#define REQ_STATE_COMPLETE     0
+
 enum nvme_tcp_send_state {
 	NVME_TCP_SEND_CMD_PDU = 0,
 	NVME_TCP_SEND_H2C_PDU,
@@ -56,6 +58,8 @@ struct nvme_tcp_request {
 	size_t			offset;
 	size_t			data_sent;
 	enum nvme_tcp_send_state state;
+
+	unsigned long		comp_state;
 };
 
 enum nvme_tcp_queue_flags {
@@ -469,6 +473,33 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
 	queue_work(nvme_reset_wq, &to_tcp_ctrl(ctrl)->err_work);
 }
 
+/*
+ * requests originated from admin, fabrics and connect_q have to be
+ * completed atomically because we don't cover the race between timeout
+ * and normal completion for these queues.
+ */
+static inline bool nvme_tcp_need_atomic_complete(struct request *rq)
+{
+	return !rq->rq_disk;
+}
+
+static inline void nvme_tcp_clear_rq_complete(struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+
+	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
+		clear_bit(REQ_STATE_COMPLETE, &req->comp_state);
+}
+
+static inline bool nvme_tcp_mark_rq_complete(struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+
+	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
+		return !test_and_set_bit(REQ_STATE_COMPLETE, &req->comp_state);
+	return true;
+}
+
 static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		struct nvme_completion *cqe)
 {
@@ -483,7 +514,8 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		return -EINVAL;
 	}
 
-	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
+	if (nvme_tcp_mark_rq_complete(rq) &&
+			!nvme_try_complete_req(rq, cqe->status, cqe->result))
 		nvme_complete_rq(rq);
 	queue->nr_cqe++;
 
@@ -674,7 +706,8 @@ static inline void nvme_tcp_end_request(struct request *rq, u16 status)
 {
 	union nvme_result res = {};
 
-	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
+	if (nvme_tcp_mark_rq_complete(rq) &&
+			!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
 		nvme_complete_rq(rq);
 }
 
@@ -2174,7 +2207,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	/* fence other contexts that may complete the command */
 	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
-	if (!blk_mq_request_completed(rq)) {
+	if (nvme_tcp_mark_rq_complete(rq) && !blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
@@ -2315,6 +2348,7 @@ static blk_status_t nvme_tcp_queue_rq(struct blk_mq_hw_ctx *hctx,
 	if (unlikely(ret))
 		return ret;
 
+	nvme_tcp_clear_rq_complete(rq);
 	blk_mq_start_request(rq);
 
 	nvme_tcp_queue_request(req, true, bd->last);
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:53:01 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:53:01 +0800
Subject: [PATCH V2 4/4] nvme: tcp: fix race between timeout and normal
 completion
In-Reply-To: <20201020085301.1553959-1-ming.lei@redhat.com>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
Message-ID: <20201020085301.1553959-5-ming.lei@redhat.com>

NVMe TCP timeout handler allows to abort request directly when the
controller isn't in LIVE state. nvme_tcp_error_recovery() updates
controller state as RESETTING, and schedule reset work function. If
new timeout comes before the work function is called, the new timedout
request will be aborted directly, however at that time, the controller
isn't shut down yet, then timeout abort vs. normal completion race
will be triggered.

Fix the race by the following approach:

1) delay unquiesce io queues and admin queue until controller is LIVE
because it isn't necessary to start queues during RESETTING. Instead,
this way may risk timeout vs. normal completion race because we need
to abort timed-out request directly during CONNECTING state for setting
up controller.

2) aborting timed out request directly only in case that controller is in
CONNECTING and DELETING state. In CONNECTING state, requests are only
submitted for recovering controller, and normal IO requests aren't
allowed, so it is safe to do so. In DELETING state, teardown controller
if IO request timeout happens.

CC: Chao Leng <lengchao at huawei.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Reported-by: Yi Zhang <yi.zhang at redhat.com>
Tested-by: Yi Zhang <yi.zhang at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
 drivers/nvme/host/tcp.c | 58 +++++++++++++++++++++++++++--------------
 1 file changed, 39 insertions(+), 19 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 7e85bd4a8d1b..3a137631b2b3 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1919,7 +1919,6 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
 	if (ctrl->admin_tagset) {
@@ -1930,15 +1929,13 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 	if (remove)
 		blk_mq_unquiesce_queue(ctrl->admin_q);
 	nvme_tcp_destroy_admin_queue(ctrl, remove);
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	if (ctrl->queue_count <= 1)
-		goto out;
+		return;
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_start_freeze(ctrl);
 	nvme_stop_queues(ctrl);
@@ -1951,8 +1948,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 	if (remove)
 		nvme_start_queues(ctrl);
 	nvme_tcp_destroy_io_queues(ctrl, remove);
-out:
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
@@ -1971,6 +1966,10 @@ static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
 				ctrl->opts->reconnect_delay * HZ);
 	} else {
 		dev_info(ctrl->device, "Removing controller...\n");
+
+		/* start queues for not blocking removing path */
+		nvme_start_queues(ctrl);
+		blk_mq_unquiesce_queue(ctrl->admin_q);
 		nvme_delete_ctrl(ctrl);
 	}
 }
@@ -2063,11 +2062,11 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)
 	struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
 
 	nvme_stop_keep_alive(ctrl);
+
+	mutex_lock(&tcp_ctrl->teardown_lock);
 	nvme_tcp_teardown_io_queues(ctrl, false);
-	/* unquiesce to fail fast pending requests */
-	nvme_start_queues(ctrl);
 	nvme_tcp_teardown_admin_queue(ctrl, false);
-	blk_mq_unquiesce_queue(ctrl->admin_q);
+	mutex_unlock(&tcp_ctrl->teardown_lock);
 
 	if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_CONNECTING)) {
 		/* state change failure is ok if we started ctrl delete */
@@ -2084,6 +2083,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
 	cancel_work_sync(&to_tcp_ctrl(ctrl)->err_work);
 	cancel_delayed_work_sync(&to_tcp_ctrl(ctrl)->connect_work);
 
+	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_teardown_io_queues(ctrl, shutdown);
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	if (shutdown)
@@ -2091,6 +2091,7 @@ static void nvme_tcp_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
 	else
 		nvme_disable_ctrl(ctrl);
 	nvme_tcp_teardown_admin_queue(ctrl, shutdown);
+	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_delete_ctrl(struct nvme_ctrl *ctrl)
@@ -2225,22 +2226,41 @@ nvme_tcp_timeout(struct request *rq, bool reserved)
 		"queue %d: timeout request %#x type %d\n",
 		nvme_tcp_queue_id(req->queue), rq->tag, pdu->hdr.type);
 
-	if (ctrl->state != NVME_CTRL_LIVE) {
+	/*
+	 * During CONNECTING or DELETING, the controller has been shutdown,
+	 * so it is safe to abort the request directly, otherwise timeout
+	 * vs. normal completion will be triggered.
+	 */
+	if (ctrl->state == NVME_CTRL_CONNECTING ||
+			ctrl->state == NVME_CTRL_DELETING ||
+			ctrl->state == NVME_CTRL_DELETING_NOIO) {
 		/*
-		 * If we are resetting, connecting or deleting we should
-		 * complete immediately because we may block controller
-		 * teardown or setup sequence
+		 * If we are connecting we should complete immediately because
+		 * we may block controller setup sequence
 		 * - ctrl disable/shutdown fabrics requests
 		 * - connect requests
 		 * - initialization admin requests
-		 * - I/O requests that entered after unquiescing and
-		 *   the controller stopped responding
+		 */
+		if (!rq->rq_disk) {
+			nvme_tcp_complete_timed_out(rq);
+			return BLK_EH_DONE;
+		}
+
+		/*
+		 * During CONNECTING, any in-flight requests are aborted, and
+		 * queue is stopped, so in theory not possible to see timed out
+		 * requests. And it might happen when one IO timeout is triggered
+		 * before changing to CONNECTING, but the timeout handling is
+		 * scheduled after updating to CONNECTING, so safe to ignore
+		 * this case.
 		 *
-		 * All other requests should be cancelled by the error
-		 * recovery work, so it's fine that we fail it here.
+		 * During DELETING, tear down controller and make forward
+		 * progress.
 		 */
-		nvme_tcp_complete_timed_out(rq);
-		return BLK_EH_DONE;
+		if (ctrl->state != NVME_CTRL_CONNECTING) {
+			nvme_tcp_teardown_ctrl(ctrl, false);
+			return BLK_EH_DONE;
+		}
 	}
 
 	/*
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:55:51 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:55:51 +0800
Subject: [PATCH V8 0/4] blk-mq: implement queue quiesce via percpu_ref for
 BLK_MQ_F_BLOCKING
Message-ID: <20201020085555.1554255-1-ming.lei@redhat.com>

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce,
and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

The 3rd patch adds tagset quiesce interface.

The 4th patch applies tagset quiesce interface for NVMe subsystem.

V8:
	- rebase on latest linus tree, only there is small fuzz change on 2/4

V7:
	- base on latest for-5.10/block, only there is small change on 2/4

V6:
	- base on for-5.10/block directly, instead of being against on patchset of
	'percpu_ref & block: reduce memory footprint of percpu_ref in fast path',
	because these patches don't depend on that patchset. 

V5:
	- warn once in case that driver unquiesces its queue being
	  quiesce and not done, only patch 2 is modified

V4:
	- remove .mq_quiesce_mutex, and switch to test_and_[set|clear] for
	avoiding duplicated quiesce action
	- pass blktests(block, nvme)

V3:
	- add tagset quiesce interface
	- apply tagset quiesce interface for NVMe
	- pass blktests(block, nvme)

V2:
	- add .mq_quiesce_lock
	- add comment on patch 2 wrt. handling hctx_lock() failure
	- trivial patch style change


Ming Lei (3):
  block: use test_and_{clear|test}_bit to set/clear QUEUE_FLAG_QUIESCED
  blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING
  blk-mq: add tagset quiesce interface

Sagi Grimberg (1):
  nvme: use blk_mq_[un]quiesce_tagset

 block/blk-core.c         |  13 +++
 block/blk-mq-sysfs.c     |   2 -
 block/blk-mq.c           | 182 +++++++++++++++++++++++++--------------
 block/blk-sysfs.c        |   6 +-
 block/blk.h              |   2 +
 drivers/nvme/host/core.c |  19 ++--
 include/linux/blk-mq.h   |  10 +--
 include/linux/blkdev.h   |   4 +
 8 files changed, 154 insertions(+), 84 deletions(-)

Cc: Hannes Reinecke <hare at suse.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:55:52 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:55:52 +0800
Subject: [PATCH V8 1/4] block: use test_and_{clear|test}_bit to set/clear
 QUEUE_FLAG_QUIESCED
In-Reply-To: <20201020085555.1554255-1-ming.lei@redhat.com>
References: <20201020085555.1554255-1-ming.lei@redhat.com>
Message-ID: <20201020085555.1554255-2-ming.lei@redhat.com>

Prepare for replacing srcu with percpu-refcount for implementing queue
quiesce.

The following patch needs to avoid duplicated quiesce action for
BLK_MQ_F_BLOCKING, so use test_and_{clear|test}_bit to set/clear
QUEUE_FLAG_QUIESCED.

Signed-off-by: Ming Lei <ming.lei at redhat.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 block/blk-core.c | 13 +++++++++++++
 block/blk-mq.c   | 11 ++++++++---
 block/blk.h      |  2 ++
 3 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index ac00d2fa4eb4..abdcc0a57c75 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -107,6 +107,19 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_queue_flag_test_and_set);
 
+/**
+ * blk_queue_flag_test_and_clear - atomically test and clear a queue flag
+ * @flag: flag to be clear
+ * @q: request queue
+ *
+ * Returns the previous value of @flag - 0 if the flag was not set and 1 if
+ * the flag was set.
+ */
+bool blk_queue_flag_test_and_clear(unsigned int flag, struct request_queue *q)
+{
+	return test_and_clear_bit(flag, &q->queue_flags);
+}
+
 void blk_rq_init(struct request_queue *q, struct request *rq)
 {
 	memset(rq, 0, sizeof(*rq));
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 696450257ac1..6ff538d74b62 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -199,13 +199,18 @@ void blk_mq_unfreeze_queue(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_mq_unfreeze_queue);
 
+static bool __blk_mq_quiesce_queue_nowait(struct request_queue *q)
+{
+	return blk_queue_flag_test_and_set(QUEUE_FLAG_QUIESCED, q);
+}
+
 /*
  * FIXME: replace the scsi_internal_device_*block_nowait() calls in the
  * mpt3sas driver such that this function can be removed.
  */
 void blk_mq_quiesce_queue_nowait(struct request_queue *q)
 {
-	blk_queue_flag_set(QUEUE_FLAG_QUIESCED, q);
+	__blk_mq_quiesce_queue_nowait(q);
 }
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_nowait);
 
@@ -224,7 +229,7 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	unsigned int i;
 	bool rcu = false;
 
-	blk_mq_quiesce_queue_nowait(q);
+	__blk_mq_quiesce_queue_nowait(q);
 
 	queue_for_each_hw_ctx(q, hctx, i) {
 		if (hctx->flags & BLK_MQ_F_BLOCKING)
@@ -246,7 +251,7 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
  */
 void blk_mq_unquiesce_queue(struct request_queue *q)
 {
-	blk_queue_flag_clear(QUEUE_FLAG_QUIESCED, q);
+	blk_queue_flag_test_and_clear(QUEUE_FLAG_QUIESCED, q);
 
 	/* dispatch requests which are inserted during quiescing */
 	blk_mq_run_hw_queues(q, true);
diff --git a/block/blk.h b/block/blk.h
index dfab98465db9..f925268ce7fd 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -444,4 +444,6 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio,
 		struct page *page, unsigned int len, unsigned int offset,
 		unsigned int max_sectors, bool *same_page);
 
+bool blk_queue_flag_test_and_clear(unsigned int flag, struct request_queue *q);
+
 #endif /* BLK_INTERNAL_H */
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:55:53 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:55:53 +0800
Subject: [PATCH V8 2/4] blk-mq: implement queue quiesce via percpu_ref for
 BLK_MQ_F_BLOCKING
In-Reply-To: <20201020085555.1554255-1-ming.lei@redhat.com>
References: <20201020085555.1554255-1-ming.lei@redhat.com>
Message-ID: <20201020085555.1554255-3-ming.lei@redhat.com>

In case of BLK_MQ_F_BLOCKING, blk-mq uses SRCU to mark read critical
section during dispatching request, then request queue quiesce is based on
SRCU. What we want to get is low cost added in fast path.

With percpu-ref, it is cleaner and simpler & enough for implementing queue
quiesce. The main requirement is to make sure all read sections to observe
QUEUE_FLAG_QUIESCED once blk_mq_quiesce_queue() returns.

Also it becomes much easier to add interface of async queue quiesce.

Meantime memory footprint can be reduced with per-request-queue percpu-ref.

>From implementation viewpoint, in fast path, not see percpu_ref is
slower than SRCU, and srcu tree(default option in most distributions)
could be slower since memory barrier is required in both lock & unlock,
and rcu_read_lock()/rcu_read_unlock() should be much cheap than
smp_mb().

1) percpu_ref just hold the rcu_read_lock, then run a check &
   increase/decrease on the percpu variable:

   rcu_read_lock()
   if (__ref_is_percpu(ref, &percpu_count))
	this_cpu_inc(*percpu_count);
   rcu_read_unlock()

2) srcu tree:
        idx = READ_ONCE(ssp->srcu_idx) & 0x1;
        this_cpu_inc(ssp->sda->srcu_lock_count[idx]);
        smp_mb(); /* B */  /* Avoid leaking the critical section. */

Also from my test on null_blk(blocking), not observe percpu-ref performs
worse than srcu, see the following test:

1) test steps:

rmmod null_blk > /dev/null 2>&1
modprobe null_blk nr_devices=1 submit_queues=1 blocking=1
fio --bs=4k --size=512G  --rw=randread --norandommap --direct=1 --ioengine=libaio \
	--iodepth=64 --runtime=60 --group_reporting=1  --name=nullb0 \
	--filename=/dev/nullb0 --numjobs=32

test machine: HP DL380, 16 cpu cores, 2 threads per core, dual
sockets/numa, Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz

2) test result:
- srcu quiesce: 6063K IOPS
- percpu-ref quiesce: 6113K IOPS

Signed-off-by: Ming Lei <ming.lei at redhat.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 128 ++++++++++++++++++++++-------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   8 ---
 include/linux/blkdev.h |   4 ++
 5 files changed, 79 insertions(+), 69 deletions(-)

diff --git a/block/blk-mq-sysfs.c b/block/blk-mq-sysfs.c
index 7b52e7657b2d..7b41a8b72a3d 100644
--- a/block/blk-mq-sysfs.c
+++ b/block/blk-mq-sysfs.c
@@ -36,8 +36,6 @@ static void blk_mq_hw_sysfs_release(struct kobject *kobj)
 	struct blk_mq_hw_ctx *hctx = container_of(kobj, struct blk_mq_hw_ctx,
 						  kobj);
 
-	if (hctx->flags & BLK_MQ_F_BLOCKING)
-		cleanup_srcu_struct(hctx->srcu);
 	blk_free_flush_queue(hctx->fq);
 	sbitmap_free(&hctx->ctx_map);
 	free_cpumask_var(hctx->cpumask);
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 6ff538d74b62..b79bbf938fcb 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -225,19 +225,23 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_nowait);
  */
 void blk_mq_quiesce_queue(struct request_queue *q)
 {
-	struct blk_mq_hw_ctx *hctx;
-	unsigned int i;
-	bool rcu = false;
+	bool blocking = !!(q->tag_set->flags & BLK_MQ_F_BLOCKING);
+	bool was_quiesced =__blk_mq_quiesce_queue_nowait(q);
 
-	__blk_mq_quiesce_queue_nowait(q);
+	if (!was_quiesced && blocking)
+		percpu_ref_kill(&q->dispatch_counter);
 
-	queue_for_each_hw_ctx(q, hctx, i) {
-		if (hctx->flags & BLK_MQ_F_BLOCKING)
-			synchronize_srcu(hctx->srcu);
-		else
-			rcu = true;
-	}
-	if (rcu)
+	/*
+	 * In case of F_BLOCKING, if driver unquiesces its queue being
+	 * quiesced, it can cause bigger trouble, and we simply return &
+	 * warn once for avoiding hang here.
+	 */
+	if (blocking)
+		wait_event(q->mq_quiesce_wq,
+				percpu_ref_is_zero(&q->dispatch_counter) ||
+				WARN_ON_ONCE(!percpu_ref_is_dying(
+						&q->dispatch_counter)));
+	else
 		synchronize_rcu();
 }
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
@@ -251,7 +255,10 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
  */
 void blk_mq_unquiesce_queue(struct request_queue *q)
 {
-	blk_queue_flag_test_and_clear(QUEUE_FLAG_QUIESCED, q);
+	if (blk_queue_flag_test_and_clear(QUEUE_FLAG_QUIESCED, q)) {
+		if (q->tag_set->flags & BLK_MQ_F_BLOCKING)
+			percpu_ref_resurrect(&q->dispatch_counter);
+	}
 
 	/* dispatch requests which are inserted during quiescing */
 	blk_mq_run_hw_queues(q, true);
@@ -704,24 +711,21 @@ void blk_mq_complete_request(struct request *rq)
 }
 EXPORT_SYMBOL(blk_mq_complete_request);
 
-static void hctx_unlock(struct blk_mq_hw_ctx *hctx, int srcu_idx)
-	__releases(hctx->srcu)
+static void hctx_unlock(struct blk_mq_hw_ctx *hctx)
 {
-	if (!(hctx->flags & BLK_MQ_F_BLOCKING))
-		rcu_read_unlock();
+	if (hctx->flags & BLK_MQ_F_BLOCKING)
+		percpu_ref_put(&hctx->queue->dispatch_counter);
 	else
-		srcu_read_unlock(hctx->srcu, srcu_idx);
+		rcu_read_unlock();
 }
 
-static void hctx_lock(struct blk_mq_hw_ctx *hctx, int *srcu_idx)
-	__acquires(hctx->srcu)
+/* Returning false means that queue is being quiesced */
+static inline bool hctx_lock(struct blk_mq_hw_ctx *hctx)
 {
-	if (!(hctx->flags & BLK_MQ_F_BLOCKING)) {
-		/* shut up gcc false positive */
-		*srcu_idx = 0;
-		rcu_read_lock();
-	} else
-		*srcu_idx = srcu_read_lock(hctx->srcu);
+	if (hctx->flags & BLK_MQ_F_BLOCKING)
+		return percpu_ref_tryget_live(&hctx->queue->dispatch_counter);
+	rcu_read_lock();
+	return true;
 }
 
 /**
@@ -1498,8 +1502,6 @@ bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx *hctx, struct list_head *list,
  */
 static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
 {
-	int srcu_idx;
-
 	/*
 	 * We should be running this queue from one of the CPUs that
 	 * are mapped to it.
@@ -1533,9 +1535,10 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
 
 	might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING);
 
-	hctx_lock(hctx, &srcu_idx);
-	blk_mq_sched_dispatch_requests(hctx);
-	hctx_unlock(hctx, srcu_idx);
+	if (hctx_lock(hctx)) {
+		blk_mq_sched_dispatch_requests(hctx);
+		hctx_unlock(hctx);
+	}
 }
 
 static inline int blk_mq_first_mapped_cpu(struct blk_mq_hw_ctx *hctx)
@@ -1647,7 +1650,6 @@ EXPORT_SYMBOL(blk_mq_delay_run_hw_queue);
  */
 void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async)
 {
-	int srcu_idx;
 	bool need_run;
 
 	/*
@@ -1658,10 +1660,12 @@ void blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async)
 	 * And queue will be rerun in blk_mq_unquiesce_queue() if it is
 	 * quiesced.
 	 */
-	hctx_lock(hctx, &srcu_idx);
+	if (!hctx_lock(hctx))
+		return;
+
 	need_run = !blk_queue_quiesced(hctx->queue) &&
 		blk_mq_hctx_has_pending(hctx);
-	hctx_unlock(hctx, srcu_idx);
+	hctx_unlock(hctx);
 
 	if (need_run)
 		__blk_mq_delay_run_hw_queue(hctx, async, 0);
@@ -2006,7 +2010,7 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx,
 	bool run_queue = true;
 
 	/*
-	 * RCU or SRCU read lock is needed before checking quiesced flag.
+	 * hctx_lock() is needed before checking quiesced flag.
 	 *
 	 * When queue is stopped or quiesced, ignore 'bypass_insert' from
 	 * blk_mq_request_issue_directly(), and return BLK_STS_OK to caller,
@@ -2054,11 +2058,14 @@ static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx,
 		struct request *rq, blk_qc_t *cookie)
 {
 	blk_status_t ret;
-	int srcu_idx;
 
 	might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING);
 
-	hctx_lock(hctx, &srcu_idx);
+	/* Insert request to queue in case of being quiesced */
+	if (!hctx_lock(hctx)) {
+		blk_mq_sched_insert_request(rq, false, false, false);
+		return;
+	}
 
 	ret = __blk_mq_try_issue_directly(hctx, rq, cookie, false, true);
 	if (ret == BLK_STS_RESOURCE || ret == BLK_STS_DEV_RESOURCE)
@@ -2066,19 +2073,22 @@ static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx,
 	else if (ret != BLK_STS_OK)
 		blk_mq_end_request(rq, ret);
 
-	hctx_unlock(hctx, srcu_idx);
+	hctx_unlock(hctx);
 }
 
 blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last)
 {
 	blk_status_t ret;
-	int srcu_idx;
 	blk_qc_t unused_cookie;
 	struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
 
-	hctx_lock(hctx, &srcu_idx);
+	/* Insert request to queue in case of being quiesced */
+	if (!hctx_lock(hctx)) {
+		blk_mq_sched_insert_request(rq, false, false, false);
+		return BLK_STS_OK;
+	}
 	ret = __blk_mq_try_issue_directly(hctx, rq, &unused_cookie, true, last);
-	hctx_unlock(hctx, srcu_idx);
+	hctx_unlock(hctx);
 
 	return ret;
 }
@@ -2611,20 +2621,6 @@ static void blk_mq_exit_hw_queues(struct request_queue *q,
 	}
 }
 
-static int blk_mq_hw_ctx_size(struct blk_mq_tag_set *tag_set)
-{
-	int hw_ctx_size = sizeof(struct blk_mq_hw_ctx);
-
-	BUILD_BUG_ON(ALIGN(offsetof(struct blk_mq_hw_ctx, srcu),
-			   __alignof__(struct blk_mq_hw_ctx)) !=
-		     sizeof(struct blk_mq_hw_ctx));
-
-	if (tag_set->flags & BLK_MQ_F_BLOCKING)
-		hw_ctx_size += sizeof(struct srcu_struct);
-
-	return hw_ctx_size;
-}
-
 static int blk_mq_init_hctx(struct request_queue *q,
 		struct blk_mq_tag_set *set,
 		struct blk_mq_hw_ctx *hctx, unsigned hctx_idx)
@@ -2662,7 +2658,7 @@ blk_mq_alloc_hctx(struct request_queue *q, struct blk_mq_tag_set *set,
 	struct blk_mq_hw_ctx *hctx;
 	gfp_t gfp = GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY;
 
-	hctx = kzalloc_node(blk_mq_hw_ctx_size(set), gfp, node);
+	hctx = kzalloc_node(sizeof(struct blk_mq_hw_ctx), gfp, node);
 	if (!hctx)
 		goto fail_alloc_hctx;
 
@@ -2705,8 +2701,6 @@ blk_mq_alloc_hctx(struct request_queue *q, struct blk_mq_tag_set *set,
 	if (!hctx->fq)
 		goto free_bitmap;
 
-	if (hctx->flags & BLK_MQ_F_BLOCKING)
-		init_srcu_struct(hctx->srcu);
 	blk_mq_hctx_kobj_init(hctx);
 
 	return hctx;
@@ -3186,6 +3180,13 @@ static void blk_mq_realloc_hw_ctxs(struct blk_mq_tag_set *set,
 	mutex_unlock(&q->sysfs_lock);
 }
 
+static void blk_mq_dispatch_counter_release(struct percpu_ref *ref)
+{
+	struct request_queue *q = container_of(ref, struct request_queue,
+				dispatch_counter);
+	wake_up_all(&q->mq_quiesce_wq);
+}
+
 struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 						  struct request_queue *q,
 						  bool elevator_init)
@@ -3202,6 +3203,14 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 	if (blk_mq_alloc_ctxs(q))
 		goto err_poll;
 
+	if (set->flags & BLK_MQ_F_BLOCKING) {
+		init_waitqueue_head(&q->mq_quiesce_wq);
+		if (percpu_ref_init(&q->dispatch_counter,
+					blk_mq_dispatch_counter_release,
+					PERCPU_REF_ALLOW_REINIT, GFP_KERNEL))
+			goto err_hctxs;
+	}
+
 	/* init q->mq_kobj and sw queues' kobjects */
 	blk_mq_sysfs_init(q);
 
@@ -3210,7 +3219,7 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 
 	blk_mq_realloc_hw_ctxs(set, q);
 	if (!q->nr_hw_queues)
-		goto err_hctxs;
+		goto err_dispatch_counter;
 
 	INIT_WORK(&q->timeout_work, blk_mq_timeout_work);
 	blk_queue_rq_timeout(q, set->timeout ? set->timeout : 30 * HZ);
@@ -3244,6 +3253,9 @@ struct request_queue *blk_mq_init_allocated_queue(struct blk_mq_tag_set *set,
 
 	return q;
 
+err_dispatch_counter:
+	if (set->flags & BLK_MQ_F_BLOCKING)
+		percpu_ref_exit(&q->dispatch_counter);
 err_hctxs:
 	kfree(q->queue_hw_ctx);
 	q->nr_hw_queues = 0;
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index b513f1683af0..21daf13a47c5 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -804,9 +804,13 @@ static void blk_release_queue(struct kobject *kobj)
 
 	blk_queue_free_zone_bitmaps(q);
 
-	if (queue_is_mq(q))
+	if (queue_is_mq(q)) {
 		blk_mq_release(q);
 
+		if (q->tag_set->flags & BLK_MQ_F_BLOCKING)
+			percpu_ref_exit(&q->dispatch_counter);
+	}
+
 	blk_trace_shutdown(q);
 	mutex_lock(&q->debugfs_mutex);
 	debugfs_remove_recursive(q->debugfs_dir);
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index b23eeca4d677..df642055f02c 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -4,7 +4,6 @@
 
 #include <linux/blkdev.h>
 #include <linux/sbitmap.h>
-#include <linux/srcu.h>
 
 struct blk_mq_tags;
 struct blk_flush_queue;
@@ -173,13 +172,6 @@ struct blk_mq_hw_ctx {
 	 * q->unused_hctx_list.
 	 */
 	struct list_head	hctx_list;
-
-	/**
-	 * @srcu: Sleepable RCU. Use as lock when type of the hardware queue is
-	 * blocking (BLK_MQ_F_BLOCKING). Must be the last member - see also
-	 * blk_mq_hw_ctx_size().
-	 */
-	struct srcu_struct	srcu[];
 };
 
 /**
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 639cae2c158b..236d5247da18 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -575,6 +575,10 @@ struct request_queue {
 	 */
 	struct mutex		mq_freeze_lock;
 
+	/* only used for BLK_MQ_F_BLOCKING */
+	struct percpu_ref	dispatch_counter;
+	wait_queue_head_t	mq_quiesce_wq;
+
 	struct blk_mq_tag_set	*tag_set;
 	struct list_head	tag_set_list;
 	struct bio_set		bio_split;
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:55:54 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:55:54 +0800
Subject: [PATCH V8 3/4] blk-mq: add tagset quiesce interface
In-Reply-To: <20201020085555.1554255-1-ming.lei@redhat.com>
References: <20201020085555.1554255-1-ming.lei@redhat.com>
Message-ID: <20201020085555.1554255-4-ming.lei@redhat.com>

drivers that have shared tagsets may need to quiesce potentially a lot
of request queues that all share a single tagset (e.g. nvme). Add an interface
to quiesce all the queues on a given tagset. This interface is useful because
it can speedup the quiesce by doing it in parallel.

For tagsets that have BLK_MQ_F_BLOCKING set, we kill request queue's dispatch
percpu-refcount such that all of them wait for the counter becoming zero. For
tagsets that don't have BLK_MQ_F_BLOCKING set, we simply call a single
synchronize_rcu as this is sufficient.

This patch is against Sagi's original post.
Signed-off-by: Ming Lei <ming.lei at redhat.com>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>
Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 block/blk-mq.c         | 59 +++++++++++++++++++++++++++++++++++-------
 include/linux/blk-mq.h |  2 ++
 2 files changed, 51 insertions(+), 10 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index b79bbf938fcb..c98943f8c223 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -214,16 +214,7 @@ void blk_mq_quiesce_queue_nowait(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_nowait);
 
-/**
- * blk_mq_quiesce_queue() - wait until all ongoing dispatches have finished
- * @q: request queue.
- *
- * Note: this function does not prevent that the struct request end_io()
- * callback function is invoked. Once this function is returned, we make
- * sure no dispatch can happen until the queue is unquiesced via
- * blk_mq_unquiesce_queue().
- */
-void blk_mq_quiesce_queue(struct request_queue *q)
+static void __blk_mq_quiesce_queue(struct request_queue *q, bool wait)
 {
 	bool blocking = !!(q->tag_set->flags & BLK_MQ_F_BLOCKING);
 	bool was_quiesced =__blk_mq_quiesce_queue_nowait(q);
@@ -231,6 +222,9 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	if (!was_quiesced && blocking)
 		percpu_ref_kill(&q->dispatch_counter);
 
+	if (!wait)
+		return;
+
 	/*
 	 * In case of F_BLOCKING, if driver unquiesces its queue being
 	 * quiesced, it can cause bigger trouble, and we simply return &
@@ -244,6 +238,20 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	else
 		synchronize_rcu();
 }
+
+/*
+ * blk_mq_quiesce_queue() - wait until all ongoing dispatches have finished
+ * @q: request queue.
+ *
+ * Note: this function does not prevent that the struct request end_io()
+ * callback function is invoked. Once this function is returned, we make
+ * sure no dispatch can happen until the queue is unquiesced via
+ * blk_mq_unquiesce_queue().
+ */
+void blk_mq_quiesce_queue(struct request_queue *q)
+{
+	__blk_mq_quiesce_queue(q, true);
+}
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
 
 /*
@@ -265,6 +273,37 @@ void blk_mq_unquiesce_queue(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_mq_unquiesce_queue);
 
+void blk_mq_quiesce_tagset(struct blk_mq_tag_set *set)
+{
+	struct request_queue *q;
+
+	mutex_lock(&set->tag_list_lock);
+	list_for_each_entry(q, &set->tag_list, tag_set_list)
+		__blk_mq_quiesce_queue(q, false);
+
+	/* wait until all queues' quiesce is done */
+	if (set->flags & BLK_MQ_F_BLOCKING) {
+		list_for_each_entry(q, &set->tag_list, tag_set_list)
+			wait_event(q->mq_quiesce_wq,
+				   percpu_ref_is_zero(&q->dispatch_counter));
+	} else {
+		synchronize_rcu();
+	}
+	mutex_unlock(&set->tag_list_lock);
+}
+EXPORT_SYMBOL_GPL(blk_mq_quiesce_tagset);
+
+void blk_mq_unquiesce_tagset(struct blk_mq_tag_set *set)
+{
+	struct request_queue *q;
+
+	mutex_lock(&set->tag_list_lock);
+	list_for_each_entry(q, &set->tag_list, tag_set_list)
+		blk_mq_unquiesce_queue(q);
+	mutex_unlock(&set->tag_list_lock);
+}
+EXPORT_SYMBOL_GPL(blk_mq_unquiesce_tagset);
+
 void blk_mq_wake_waiters(struct request_queue *q)
 {
 	struct blk_mq_hw_ctx *hctx;
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index df642055f02c..90da3582b91d 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -519,6 +519,8 @@ int blk_mq_map_queues(struct blk_mq_queue_map *qmap);
 void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues);
 
 void blk_mq_quiesce_queue_nowait(struct request_queue *q);
+void blk_mq_quiesce_tagset(struct blk_mq_tag_set *set);
+void blk_mq_unquiesce_tagset(struct blk_mq_tag_set *set);
 
 unsigned int blk_mq_rq_cpu(struct request *rq);
 
-- 
2.25.2



From ming.lei at redhat.com  Tue Oct 20 04:55:55 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 16:55:55 +0800
Subject: [PATCH V8 4/4] nvme: use blk_mq_[un]quiesce_tagset
In-Reply-To: <20201020085555.1554255-1-ming.lei@redhat.com>
References: <20201020085555.1554255-1-ming.lei@redhat.com>
Message-ID: <20201020085555.1554255-5-ming.lei@redhat.com>

From: Sagi Grimberg <sagi at grimberg.me>

All controller namespaces share the same tagset, so we
can use this interface which does the optimal operation
for parallel quiesce based on the tagset type (e.g.
blocking tagsets and non-blocking tagsets).

Reviewed-by: Hannes Reinecke <hare at suse.de>
Tested-by: Sagi Grimberg <sagi at grimberg.me>
Reviewed-by: Keith Busch <kbusch at kernel.org>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Bart Van Assche <bvanassche at acm.org>
Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
Cc: Chao Leng <lengchao at huawei.com>

Add code to unquiesce ctrl->connect_q in nvme_stop_queues(), meantime
avoid to call blk_mq_quiesce_tagset()/blk_mq_unquiesce_tagset() if
this tagset isn't initialized.
Signed-off-by: Ming Lei <ming.lei at redhat.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/core.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 56e2a22e8a02..1deab659db4f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4558,23 +4558,22 @@ EXPORT_SYMBOL_GPL(nvme_start_freeze);
 
 void nvme_stop_queues(struct nvme_ctrl *ctrl)
 {
-	struct nvme_ns *ns;
+	if (list_empty_careful(&ctrl->namespaces))
+		return;
 
-	down_read(&ctrl->namespaces_rwsem);
-	list_for_each_entry(ns, &ctrl->namespaces, list)
-		blk_mq_quiesce_queue(ns->queue);
-	up_read(&ctrl->namespaces_rwsem);
+	blk_mq_quiesce_tagset(ctrl->tagset);
+
+	if (ctrl->connect_q)
+		blk_mq_unquiesce_queue(ctrl->connect_q);
 }
 EXPORT_SYMBOL_GPL(nvme_stop_queues);
 
 void nvme_start_queues(struct nvme_ctrl *ctrl)
 {
-	struct nvme_ns *ns;
+	if (list_empty_careful(&ctrl->namespaces))
+		return;
 
-	down_read(&ctrl->namespaces_rwsem);
-	list_for_each_entry(ns, &ctrl->namespaces, list)
-		blk_mq_unquiesce_queue(ns->queue);
-	up_read(&ctrl->namespaces_rwsem);
+	blk_mq_unquiesce_tagset(ctrl->tagset);
 }
 EXPORT_SYMBOL_GPL(nvme_start_queues);
 
-- 
2.25.2



From lengchao at huawei.com  Tue Oct 20 05:04:29 2020
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 20 Oct 2020 17:04:29 +0800
Subject: [PATCH V2 3/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <20201020085301.1553959-4-ming.lei@redhat.com>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
 <20201020085301.1553959-4-ming.lei@redhat.com>
Message-ID: <e1d53ce3-ea27-439e-5e7a-ba790742908e@huawei.com>



On 2020/10/20 16:53, Ming Lei wrote:
> During controller's CONNECTING state, admin/fabric/connect requests
> are submitted for recovery controller, and we allow to abort this request
> directly in time out handler for not blocking setup procedure.
> 
> So timout vs. normal completion race exists on these requests since
> admin/fabirc/connect queues won't be shutdown before handling timeout
> during CONNECTING state.
> 
> Add atomic completion for requests from connect/fabric/admin queue for
> avoiding the race.
> 
> CC: Chao Leng <lengchao at huawei.com>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Reported-by: Yi Zhang <yi.zhang at redhat.com>
> Tested-by: Yi Zhang <yi.zhang at redhat.com>
> Signed-off-by: Ming Lei <ming.lei at redhat.com>
> ---
>   drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
>   1 file changed, 37 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index d6a3e1487354..7e85bd4a8d1b 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -30,6 +30,8 @@ static int so_priority;
>   module_param(so_priority, int, 0644);
>   MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
>   
> +#define REQ_STATE_COMPLETE     0
> +
>   enum nvme_tcp_send_state {
>   	NVME_TCP_SEND_CMD_PDU = 0,
>   	NVME_TCP_SEND_H2C_PDU,
> @@ -56,6 +58,8 @@ struct nvme_tcp_request {
>   	size_t			offset;
>   	size_t			data_sent;
>   	enum nvme_tcp_send_state state;
> +
> +	unsigned long		comp_state;
I do not think adding state is a good idea.
It is similar to rq->state.
In the teardown process, after quiesced queues delete the timer and
cancel the timeout work maybe a better option.
I will send the patch later.
The patch is already tested with roce more than one week.
>   };
>   
>   enum nvme_tcp_queue_flags {
> @@ -469,6 +473,33 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
>   	queue_work(nvme_reset_wq, &to_tcp_ctrl(ctrl)->err_work);
>   }
>   
> +/*
> + * requests originated from admin, fabrics and connect_q have to be
> + * completed atomically because we don't cover the race between timeout
> + * and normal completion for these queues.
> + */
> +static inline bool nvme_tcp_need_atomic_complete(struct request *rq)
> +{
> +	return !rq->rq_disk;
> +}
> +
> +static inline void nvme_tcp_clear_rq_complete(struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +
> +	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
> +		clear_bit(REQ_STATE_COMPLETE, &req->comp_state);
> +}
> +
> +static inline bool nvme_tcp_mark_rq_complete(struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +
> +	if (unlikely(nvme_tcp_need_atomic_complete(rq)))
> +		return !test_and_set_bit(REQ_STATE_COMPLETE, &req->comp_state);
> +	return true;
> +}
> +
>   static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		struct nvme_completion *cqe)
>   {
> @@ -483,7 +514,8 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		return -EINVAL;
>   	}
>   
> -	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> +	if (nvme_tcp_mark_rq_complete(rq) &&
> +			!nvme_try_complete_req(rq, cqe->status, cqe->result))
>   		nvme_complete_rq(rq);
>   	queue->nr_cqe++;
>   
> @@ -674,7 +706,8 @@ static inline void nvme_tcp_end_request(struct request *rq, u16 status)
>   {
>   	union nvme_result res = {};
>   
> -	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> +	if (nvme_tcp_mark_rq_complete(rq) &&
> +			!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
>   		nvme_complete_rq(rq);
>   }
>   
> @@ -2174,7 +2207,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>   	/* fence other contexts that may complete the command */
>   	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>   	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
> -	if (!blk_mq_request_completed(rq)) {
> +	if (nvme_tcp_mark_rq_complete(rq) && !blk_mq_request_completed(rq)) {
>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>   		blk_mq_complete_request(rq);
>   	}
> @@ -2315,6 +2348,7 @@ static blk_status_t nvme_tcp_queue_rq(struct blk_mq_hw_ctx *hctx,
>   	if (unlikely(ret))
>   		return ret;
>   
> +	nvme_tcp_clear_rq_complete(rq);
>   	blk_mq_start_request(rq);
>   
>   	nvme_tcp_queue_request(req, true, bd->last);
> 


From lengchao at huawei.com  Tue Oct 20 05:08:08 2020
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 20 Oct 2020 17:08:08 +0800
Subject: [PATCH 0/3] avoid race between time out and tear down
Message-ID: <20201020090808.27702-1-lengchao@huawei.com>

Avoid race between time out and tear down for rdma and tcp.

Chao Leng (3):
  nvme-core: introduce sync io queues
  nvme-rdma: avoid race between time out and tear down
  nvme-tcp: avoid race between time out and tear down

 drivers/nvme/host/core.c |  8 ++++++--
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/rdma.c | 12 ++----------
 drivers/nvme/host/tcp.c  | 14 +++-----------
 4 files changed, 12 insertions(+), 23 deletions(-)

-- 
2.16.4



From lengchao at huawei.com  Tue Oct 20 05:08:17 2020
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 20 Oct 2020 17:08:17 +0800
Subject: [PATCH 1/3] nvme-core: introduce sync io queues
Message-ID: <20201020090817.27755-1-lengchao@huawei.com>

Introduce sync io queues for some scenarios which just only need sync
io queues not sync all queues.

Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/core.c | 8 ++++++--
 drivers/nvme/host/nvme.h | 1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e85f6304efd7..2211cf155e89 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4573,8 +4573,7 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
 }
 EXPORT_SYMBOL_GPL(nvme_start_queues);
 
-
-void nvme_sync_queues(struct nvme_ctrl *ctrl)
+void nvme_sync_io_queues(struct nvme_ctrl *ctrl)
 {
 	struct nvme_ns *ns;
 
@@ -4582,7 +4581,12 @@ void nvme_sync_queues(struct nvme_ctrl *ctrl)
 	list_for_each_entry(ns, &ctrl->namespaces, list)
 		blk_sync_queue(ns->queue);
 	up_read(&ctrl->namespaces_rwsem);
+}
+EXPORT_SYMBOL_GPL(nvme_sync_io_queues);
 
+void nvme_sync_queues(struct nvme_ctrl *ctrl)
+{
+	nvme_sync_io_queues(ctrl);
 	if (ctrl->admin_q)
 		blk_sync_queue(ctrl->admin_q);
 }
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 566776100126..49d7f43511d4 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -602,6 +602,7 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl);
 void nvme_start_queues(struct nvme_ctrl *ctrl);
 void nvme_kill_queues(struct nvme_ctrl *ctrl);
 void nvme_sync_queues(struct nvme_ctrl *ctrl);
+void nvme_sync_io_queues(struct nvme_ctrl *ctrl);
 void nvme_unfreeze(struct nvme_ctrl *ctrl);
 void nvme_wait_freeze(struct nvme_ctrl *ctrl);
 int nvme_wait_freeze_timeout(struct nvme_ctrl *ctrl, long timeout);
-- 
2.16.4



From lengchao at huawei.com  Tue Oct 20 05:08:25 2020
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 20 Oct 2020 17:08:25 +0800
Subject: [PATCH 2/3] nvme-rdma: avoid race between time out and tear down
Message-ID: <20201020090825.27815-1-lengchao@huawei.com>

Now use teardown_lock to serialize for time out and tear down. This may
cause abnormal: first cancel all request in tear down, then time out may
complete the request again, but the request may already be freed or
restarted.

To avoid race between time out and tear down, in tear down process,
first we quiesce the queue, and then delete the timer and cancel
the time out work for the queue. At the same time we need to delete
teardown_lock.

Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/rdma.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..fde8ad07326e 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -122,7 +122,6 @@ struct nvme_rdma_ctrl {
 	struct sockaddr_storage src_addr;
 
 	struct nvme_ctrl	ctrl;
-	struct mutex		teardown_lock;
 	bool			use_inline_data;
 	u32			io_queues[HCTX_MAX_TYPES];
 };
@@ -1010,8 +1009,8 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
 static void nvme_rdma_teardown_admin_queue(struct nvme_rdma_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&ctrl->teardown_lock);
 	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
 	if (ctrl->ctrl.admin_tagset) {
 		blk_mq_tagset_busy_iter(ctrl->ctrl.admin_tagset,
@@ -1021,16 +1020,15 @@ static void nvme_rdma_teardown_admin_queue(struct nvme_rdma_ctrl *ctrl,
 	if (remove)
 		blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 	nvme_rdma_destroy_admin_queue(ctrl, remove);
-	mutex_unlock(&ctrl->teardown_lock);
 }
 
 static void nvme_rdma_teardown_io_queues(struct nvme_rdma_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&ctrl->teardown_lock);
 	if (ctrl->ctrl.queue_count > 1) {
 		nvme_start_freeze(&ctrl->ctrl);
 		nvme_stop_queues(&ctrl->ctrl);
+		nvme_sync_io_queues(&ctrl->ctrl);
 		nvme_rdma_stop_io_queues(ctrl);
 		if (ctrl->ctrl.tagset) {
 			blk_mq_tagset_busy_iter(ctrl->ctrl.tagset,
@@ -1041,7 +1039,6 @@ static void nvme_rdma_teardown_io_queues(struct nvme_rdma_ctrl *ctrl,
 			nvme_start_queues(&ctrl->ctrl);
 		nvme_rdma_destroy_io_queues(ctrl, remove);
 	}
-	mutex_unlock(&ctrl->teardown_lock);
 }
 
 static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
@@ -1968,16 +1965,12 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
 {
 	struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq);
 	struct nvme_rdma_queue *queue = req->queue;
-	struct nvme_rdma_ctrl *ctrl = queue->ctrl;
 
-	/* fence other contexts that may complete the command */
-	mutex_lock(&ctrl->teardown_lock);
 	nvme_rdma_stop_queue(queue);
 	if (!blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
-	mutex_unlock(&ctrl->teardown_lock);
 }
 
 static enum blk_eh_timer_return
@@ -2312,7 +2305,6 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
 		return ERR_PTR(-ENOMEM);
 	ctrl->ctrl.opts = opts;
 	INIT_LIST_HEAD(&ctrl->list);
-	mutex_init(&ctrl->teardown_lock);
 
 	if (!(opts->mask & NVMF_OPT_TRSVCID)) {
 		opts->trsvcid =
-- 
2.16.4



From lengchao at huawei.com  Tue Oct 20 05:08:31 2020
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 20 Oct 2020 17:08:31 +0800
Subject: [PATCH 3/3] nvme-tcp: avoid race between time out and tear down
Message-ID: <20201020090831.27891-1-lengchao@huawei.com>

Now use teardown_lock to serialize for time out and tear down. This may
cause abnormal: first cancel all request in tear down, then time out may
complete the request again, but the request may already be freed or
restarted.

To avoid race between time out and tear down, in tear down process,
first we quiesce the queue, and then delete the timer and cancel
the time out work for the queue. At the same time we need to delete
teardown_lock.

Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/tcp.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8f4f29f18b8c..0b229a2f59d5 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -124,7 +124,6 @@ struct nvme_tcp_ctrl {
 	struct sockaddr_storage src_addr;
 	struct nvme_ctrl	ctrl;
 
-	struct mutex		teardown_lock;
 	struct work_struct	err_work;
 	struct delayed_work	connect_work;
 	struct nvme_tcp_request async_req;
@@ -1887,8 +1886,8 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
 	if (ctrl->admin_tagset) {
 		blk_mq_tagset_busy_iter(ctrl->admin_tagset,
@@ -1898,18 +1897,17 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 	if (remove)
 		blk_mq_unquiesce_queue(ctrl->admin_q);
 	nvme_tcp_destroy_admin_queue(ctrl, remove);
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	if (ctrl->queue_count <= 1)
-		goto out;
+		return;
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_start_freeze(ctrl);
 	nvme_stop_queues(ctrl);
+	nvme_sync_io_queues(ctrl);
 	nvme_tcp_stop_io_queues(ctrl);
 	if (ctrl->tagset) {
 		blk_mq_tagset_busy_iter(ctrl->tagset,
@@ -1919,8 +1917,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 	if (remove)
 		nvme_start_queues(ctrl);
 	nvme_tcp_destroy_io_queues(ctrl, remove);
-out:
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
@@ -2172,14 +2168,11 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
 	struct nvme_ctrl *ctrl = &req->queue->ctrl->ctrl;
 
-	/* fence other contexts that may complete the command */
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
 	if (!blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static enum blk_eh_timer_return
@@ -2456,7 +2449,6 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
 			nvme_tcp_reconnect_ctrl_work);
 	INIT_WORK(&ctrl->err_work, nvme_tcp_error_recovery_work);
 	INIT_WORK(&ctrl->ctrl.reset_work, nvme_reset_ctrl_work);
-	mutex_init(&ctrl->teardown_lock);
 
 	if (!(opts->mask & NVMF_OPT_TRSVCID)) {
 		opts->trsvcid =
-- 
2.16.4



From ming.lei at redhat.com  Tue Oct 20 05:44:20 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 20 Oct 2020 17:44:20 +0800
Subject: [PATCH 3/4] nvme: tcp: fix race between timeout and normal
 completion
In-Reply-To: <e9d2e28e-fb55-358c-3e8c-6f3e9dd91c25@grimberg.me>
References: <20201016142811.1262214-1-ming.lei@redhat.com>
 <20201016142811.1262214-4-ming.lei@redhat.com>
 <e9d2e28e-fb55-358c-3e8c-6f3e9dd91c25@grimberg.me>
Message-ID: <20201020094420.GD1429635@T590>

On Tue, Oct 20, 2020 at 01:11:11AM -0700, Sagi Grimberg wrote:
> 
> > NVMe TCP timeout handler allows to abort request directly when the
> > controller isn't in LIVE state. nvme_tcp_error_recovery() updates
> > controller state as RESETTING, and schedule reset work function. If
> > new timeout comes before the work function is called, the new timedout
> > request will be aborted directly, however at that time, the controller
> > isn't shut down yet, then timeout abort vs. normal completion race
> > will be triggered.
> 
> This assertion is incorrect, the before completing the request from
> the timeout handler, we call nvme_tcp_stop_queue, which guarantees upon
> return that no more completions will be seen from this queue.

OK, then looks the issue can be fixed by patch 1 & 2 only.

Yi, can you test again and see if the issue can be fixed by patch 1 & 2?


Thanks,
Ming



From hhan at redhat.com  Tue Oct 20 08:45:31 2020
From: hhan at redhat.com (Han Han)
Date: Tue, 20 Oct 2020 20:45:31 +0800
Subject: [PATCH libnvme] examples: Add src/ into the include and library path
Message-ID: <20201020124531.238269-1-hhan@redhat.com>

That will fix the compiling errors like:
     CC telemetry-listen
telemetry-listen.c:21:10: fatal error: libnvme.h: No such file or directory
   21 | #include <libnvme.h>
      |          ^~~~~~~~~~~

Signed-off-by: Han Han <hhan at redhat.com>
---
 examples/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/Makefile b/examples/Makefile
index be867a7..24f8574 100644
--- a/examples/Makefile
+++ b/examples/Makefile
@@ -1,5 +1,5 @@
 CFLAGS ?= -g -O2
-override CFLAGS += -Wall -D_GNU_SOURCE
+override CFLAGS += -Wall -D_GNU_SOURCE -I ../src -L ../src
 
 include ../Makefile.quiet
 
-- 
2.26.2



From hhan at redhat.com  Tue Oct 20 09:49:19 2020
From: hhan at redhat.com (Han Han)
Date: Tue, 20 Oct 2020 21:49:19 +0800
Subject: [PATCH libnvme 0/2] Two fixes for the rpmbuild spec
Message-ID: <20201020134921.270042-1-hhan@redhat.com>

Han Han (2):
  spec: Use the correct url of libnvme
  spec: Fix the sub-include dir name to nvme

 libnvme.spec | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
2.26.2



From hhan at redhat.com  Tue Oct 20 09:49:20 2020
From: hhan at redhat.com (Han Han)
Date: Tue, 20 Oct 2020 21:49:20 +0800
Subject: [PATCH 1/2] spec: Use the correct url of libnvme
In-Reply-To: <20201020134921.270042-1-hhan@redhat.com>
References: <20201020134921.270042-1-hhan@redhat.com>
Message-ID: <20201020134921.270042-2-hhan@redhat.com>

Signed-off-by: Han Han <hhan at redhat.com>
---
 libnvme.spec | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libnvme.spec b/libnvme.spec
index f3407d8..80d66e9 100644
--- a/libnvme.spec
+++ b/libnvme.spec
@@ -5,7 +5,7 @@ Summary: Linux-native nvme device management library
 License: LGPLv2+
 Source: %{name}-%{version}.tar.gz
 BuildRoot: %{_tmppath}/%{name}-root
-URL: http://github.com/linux-nvme/nvme-cli
+URL: http://github.com/linux-nvme/libnvme
 BuildRequires: gcc
 
 %description
-- 
2.26.2



From hhan at redhat.com  Tue Oct 20 09:49:21 2020
From: hhan at redhat.com (Han Han)
Date: Tue, 20 Oct 2020 21:49:21 +0800
Subject: [PATCH 2/2] spec: Fix the sub-include dir name to nvme
In-Reply-To: <20201020134921.270042-1-hhan@redhat.com>
References: <20201020134921.270042-1-hhan@redhat.com>
Message-ID: <20201020134921.270042-3-hhan@redhat.com>

It will fix the error of "libnvme/ dir is not found":
$ rpmbuild -ba ~/rpmbuild/SPECS/libnvme.spec
...
error: Directory not found:
/home/hhan/rpmbuild/BUILDROOT/libnvme-0.1-0.x86_64/usr/include/libnvme

Signed-off-by: Han Han <hhan at redhat.com>
---
 libnvme.spec | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libnvme.spec b/libnvme.spec
index 80d66e9..9708084 100644
--- a/libnvme.spec
+++ b/libnvme.spec
@@ -39,7 +39,7 @@ for Linux-native nvme device maangement.
 
 %files devel
 %defattr(-,root,root)
-%attr(-,root,root) %{_includedir}/libnvme/
+%attr(-,root,root) %{_includedir}/nvme/
 %attr(0644,root,root) %{_includedir}/libnvme.h
 %attr(0755,root,root) %{_libdir}/libnvme.so
 %attr(0644,root,root) %{_libdir}/libnvme.a
-- 
2.26.2



From kbusch at kernel.org  Tue Oct 20 10:58:24 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 20 Oct 2020 07:58:24 -0700
Subject: [PATCH libnvme 0/2] Two fixes for the rpmbuild spec
In-Reply-To: <20201020134921.270042-1-hhan@redhat.com>
References: <20201020134921.270042-1-hhan@redhat.com>
Message-ID: <20201020145824.GI1435260@dhcp-10-100-145-180.wdc.com>

Thank you, patches applied.


From kbusch at kernel.org  Tue Oct 20 11:14:26 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 20 Oct 2020 08:14:26 -0700
Subject: [PATCH libnvme] examples: Add src/ into the include and library
 path
In-Reply-To: <20201020124531.238269-1-hhan@redhat.com>
References: <20201020124531.238269-1-hhan@redhat.com>
Message-ID: <20201020151426.GJ1435260@dhcp-10-100-145-180.wdc.com>

On Tue, Oct 20, 2020 at 08:45:31PM +0800, Han Han wrote:
> diff --git a/examples/Makefile b/examples/Makefile
> index be867a7..24f8574 100644
> --- a/examples/Makefile
> +++ b/examples/Makefile
> @@ -1,5 +1,5 @@
>  CFLAGS ?= -g -O2
> -override CFLAGS += -Wall -D_GNU_SOURCE
> +override CFLAGS += -Wall -D_GNU_SOURCE -I ../src -L ../src

I guess the examples assumed the library was installed, but that doesn't
make sense if you're testing within the repo. Patch applied, thank you.


From logang at deltatee.com  Tue Oct 20 11:39:28 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 20 Oct 2020 09:39:28 -0600
Subject: [PATCH 0/1] nvmet: allow user to set req alloc flag
In-Reply-To: <74e2bc85-3752-08fe-4f2c-07130ab4411d@grimberg.me>
References: <20201020014417.3604-1-chaitanya.kulkarni@wdc.com>
 <74e2bc85-3752-08fe-4f2c-07130ab4411d@grimberg.me>
Message-ID: <a37ee655-c215-2354-4d55-267b11d92ae0@deltatee.com>



On 2020-10-20 2:22 a.m., Sagi Grimberg wrote:
> 
>> Hi,
>>
>> When using NVMeOF target in the passthru mode we allocate the request
>> with BLK_MQ_REQ_NOWAIT flag. This allocates the request in the following
>> manner :-
>>
>> nvme_alloc_request()
>>   blk_mq_alloc_rquest()
>>    blk_mq_queue_enter()
>>     if (flag & BLK_MQ_REQ_NOWAIT)
>>    	return -EBUSY; <-- return if busy.
>>
>> On the NVMe controller which I've the fio random write workload running
>> parallel on 32 namespaces with higher queue depth results in I/O error,
>> where blk_mq_queue_enter() returning -EBUSY as shown above. This problem
>> is not easy to reproduce but occurs once in a while with following error
>> (See 1 for detailed log) :-
>>
>> test1: (groupid=0, jobs=32): err= 5
>> (file:io_u.c:1744, func=io_u error, error=Input/output error):
>>
>> When the flag BLK_MQ_REQ_NOWAIT is removed from the allocation the
>> workload doen't result in the error.
>>
>> This patch fixes the problem with the request allocation by adding
>> a new configfs attribute so that user can optionally decide whether
>> to use BLK_MQ_REQ_NOWAIT or not. We retain the default behavior by
>> using BLK_MQ_REQ_NOWAIT when creating the nvmet passthru subsystem.
> 
> Why should we ever set REQ_NOWAIT at all? Nothing prevents the
> host(s) queue depth from exceeding the controller queue depth...

I agree... I certainly found adding a configfs attribute for this rather
off-putting. Why would the user want an option that turns on random errors?

Logan



From Jeff.Lien at wdc.com  Tue Oct 20 11:52:25 2020
From: Jeff.Lien at wdc.com (Jeffrey Lien)
Date: Tue, 20 Oct 2020 15:52:25 +0000
Subject: NVMe CLI Version Tag
Message-ID: <BYAPR04MB5656B49BFE47211B3A2F1748EA1F0@BYAPR04MB5656.namprd04.prod.outlook.com>

Keith,
I'd like to request that the nvme cli version be updated to 1.13 and then tagged.   It's been 4 months since it was last changed and tagged.  

Thanks

Jeff Lien
eSSD Core SW Tools & Drivers

Western Digital
2900 37th St NW
Building 108-1
Rochester, MN 55901
Email:  Jeff.Lien at wdc.com
Office: +1-507-322-2416



From kbusch at kernel.org  Tue Oct 20 11:53:12 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 20 Oct 2020 08:53:12 -0700
Subject: [PATCH 0/1] nvmet: allow user to set req alloc flag
In-Reply-To: <a37ee655-c215-2354-4d55-267b11d92ae0@deltatee.com>
References: <20201020014417.3604-1-chaitanya.kulkarni@wdc.com>
 <74e2bc85-3752-08fe-4f2c-07130ab4411d@grimberg.me>
 <a37ee655-c215-2354-4d55-267b11d92ae0@deltatee.com>
Message-ID: <20201020155312.GB1437971@dhcp-10-100-145-180.wdc.com>

On Tue, Oct 20, 2020 at 09:39:28AM -0600, Logan Gunthorpe wrote:
> 
> 
> On 2020-10-20 2:22 a.m., Sagi Grimberg wrote:
> > 
> >> Hi,
> >>
> >> When using NVMeOF target in the passthru mode we allocate the request
> >> with BLK_MQ_REQ_NOWAIT flag. This allocates the request in the following
> >> manner :-
> >>
> >> nvme_alloc_request()
> >>   blk_mq_alloc_rquest()
> >>    blk_mq_queue_enter()
> >>     if (flag & BLK_MQ_REQ_NOWAIT)
> >>    	return -EBUSY; <-- return if busy.
> >>
> >> On the NVMe controller which I've the fio random write workload running
> >> parallel on 32 namespaces with higher queue depth results in I/O error,
> >> where blk_mq_queue_enter() returning -EBUSY as shown above. This problem
> >> is not easy to reproduce but occurs once in a while with following error
> >> (See 1 for detailed log) :-
> >>
> >> test1: (groupid=0, jobs=32): err= 5
> >> (file:io_u.c:1744, func=io_u error, error=Input/output error):
> >>
> >> When the flag BLK_MQ_REQ_NOWAIT is removed from the allocation the
> >> workload doen't result in the error.
> >>
> >> This patch fixes the problem with the request allocation by adding
> >> a new configfs attribute so that user can optionally decide whether
> >> to use BLK_MQ_REQ_NOWAIT or not. We retain the default behavior by
> >> using BLK_MQ_REQ_NOWAIT when creating the nvmet passthru subsystem.
> > 
> > Why should we ever set REQ_NOWAIT at all? Nothing prevents the
> > host(s) queue depth from exceeding the controller queue depth...
> 
> I agree... I certainly found adding a configfs attribute for this rather
> off-putting. Why would the user want an option that turns on random errors?

Right, you really only want to use NOWAIT when waiting can lead to bad
things like a deadlock. I don't think the target passthrough has any
such issue, so you should be able to safely leave that flag out.


From Chaitanya.Kulkarni at wdc.com  Tue Oct 20 13:06:53 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 20 Oct 2020 17:06:53 +0000
Subject: [PATCH 1/3] nvme-core: introduce sync io queues
References: <20201020090817.27755-1-lengchao@huawei.com>
Message-ID: <BYAPR04MB49659A83BC2C4763B1CF2E6C861F0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/20/20 02:12, Chao Leng wrote:
> Introduce sync io queues for some scenarios which just only need sync
> io queues not sync all queues.
>
> Signed-off-by: Chao Leng <lengchao at huawei.com>

Can you please explain the scenario in detail ?



From dwagner at suse.de  Tue Oct 20 13:50:17 2020
From: dwagner at suse.de (Daniel Wagner)
Date: Tue, 20 Oct 2020 19:50:17 +0200
Subject: NVMe CLI Version Tag
In-Reply-To: <BYAPR04MB5656B49BFE47211B3A2F1748EA1F0@BYAPR04MB5656.namprd04.prod.outlook.com>
References: <BYAPR04MB5656B49BFE47211B3A2F1748EA1F0@BYAPR04MB5656.namprd04.prod.outlook.com>
Message-ID: <20201020175017.2avzkm2bwyeg2t3a@beryllium.lan>

Hi,

On Tue, Oct 20, 2020 at 03:52:25PM +0000, Jeffrey Lien wrote:
> Keith,
> I'd like to request that the nvme cli version be updated to 1.13 and
> then tagged.   It's been 4 months since it was last changed and
> tagged.

FWIW, I would also appreciate to have a fresh release of nvme-cli.

Thanks,
Daniel


From kbusch at kernel.org  Tue Oct 20 13:56:44 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 20 Oct 2020 10:56:44 -0700
Subject: NVMe CLI Version Tag
In-Reply-To: <20201020175017.2avzkm2bwyeg2t3a@beryllium.lan>
References: <BYAPR04MB5656B49BFE47211B3A2F1748EA1F0@BYAPR04MB5656.namprd04.prod.outlook.com>
 <20201020175017.2avzkm2bwyeg2t3a@beryllium.lan>
Message-ID: <20201020175644.GA1438267@dhcp-10-100-145-180.wdc.com>

On Tue, Oct 20, 2020 at 07:50:17PM +0200, Daniel Wagner wrote:
> On Tue, Oct 20, 2020 at 03:52:25PM +0000, Jeffrey Lien wrote:
> > I'd like to request that the nvme cli version be updated to 1.13 and
> > then tagged.   It's been 4 months since it was last changed and
> > tagged.
> 
> FWIW, I would also appreciate to have a fresh release of nvme-cli.

Sure thing, I'll just run it through some sanity tests this afternoon
and set a new tag this evening if everything checks out.


From sagi at grimberg.me  Tue Oct 20 14:43:26 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 20 Oct 2020 11:43:26 -0700
Subject: [PATCH 0/3] avoid race between time out and tear down
In-Reply-To: <20201020090808.27702-1-lengchao@huawei.com>
References: <20201020090808.27702-1-lengchao@huawei.com>
Message-ID: <9e1966b0-d471-0043-7fdb-e0f535571feb@grimberg.me>


> Avoid race between time out and tear down for rdma and tcp.

This patchset overall looks good, but we still need the patch that
avoids double completion:

--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 629b025685d1..46428ff0b0fc 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct 
request *rq)
         /* fence other contexts that may complete the command */
         mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
         nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
-       if (!blk_mq_request_completed(rq)) {
+       if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
                 nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
                 blk_mq_complete_request_sync(rq);
         }
--


From dan.j.williams at intel.com  Tue Oct 20 17:35:44 2020
From: dan.j.williams at intel.com (Dan Williams)
Date: Tue, 20 Oct 2020 14:35:44 -0700
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <SN6PR08MB420843C280D54D7B5B76EB78B31E0@SN6PR08MB4208.namprd08.prod.outlook.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
 <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
 <SN6PR08MB420843C280D54D7B5B76EB78B31E0@SN6PR08MB4208.namprd08.prod.outlook.com>
Message-ID: <CAPcyv4iYOk3i0pPgXxDTy47BxWCXqqXS0J6mrY5o+1M_41XoAw@mail.gmail.com>

On Mon, Oct 19, 2020 at 3:30 PM Nabeel Meeramohideen Mohamed
(nmeeramohide) <nmeeramohide at micron.com> wrote:
>
> Hi Dan,
>
> On Friday, October 16, 2020 4:12 PM, Dan Williams <dan.j.williams at intel.com> wrote:
> >
> > On Fri, Oct 16, 2020 at 2:59 PM Nabeel Meeramohideen Mohamed
> > (nmeeramohide) <nmeeramohide at micron.com> wrote:
> > >
> > > On Thursday, October 15, 2020 2:03 AM, Christoph Hellwig
> > <hch at infradead.org> wrote:
> > > > I don't think this belongs into the kernel.  It is a classic case for
> > > > infrastructure that should be built in userspace.  If anything is
> > > > missing to implement it in userspace with equivalent performance we
> > > > need to improve out interfaces, although io_uring should cover pretty
> > > > much everything you need.
> > >
> > > Hi Christoph,
> > >
> > > We previously considered moving the mpool object store code to user-space.
> > > However, by implementing mpool as a device driver, we get several benefits
> > > in terms of scalability, performance, and functionality. In doing so, we relied
> > > only on standard interfaces and did not make any changes to the kernel.
> > >
> > > (1)  mpool's "mcache map" facility allows us to memory-map (and later unmap)
> > > a collection of logically related objects with a single system call. The objects in
> > > such a collection are created at different times, physically disparate, and may
> > > even reside on different media class volumes.
> > >
> > > For our HSE storage engine application, there are commonly 10's to 100's of
> > > objects in a given mcache map, and 75,000 total objects mapped at a given
> > time.
> > >
> > > Compared to memory-mapping objects individually, the mcache map facility
> > > scales well because it requires only a single system call and single
> > vm_area_struct
> > > to memory-map a complete collection of objects.
>
> > Why can't that be a batch of mmap calls on io_uring?
>
> Agreed, we could add the capability to invoke mmap via io_uring to help mitigate the
> system call overhead of memory-mapping individual objects, versus our mache map
> mechanism. However, there is still the scalability issue of having a vm_area_struct
> for each object (versus one for each mache map).
>
> We ran YCSB workload C in two different configurations -
> Config 1: memory-mapping each individual object
> Config 2: memory-mapping a collection of related objects using mcache map
>
> - Config 1 incurred ~3.3x additional kernel memory for the vm_area_struct slab -
> 24.8 MB (127188 objects) for config 1, versus 7.3 MB (37482 objects) for config 2.
>
> - Workload C exhibited around 10-25% better tail latencies (4-nines) for config 2,
> not sure if it's due the reduced complexity of searching VMAs during page faults.

So this gets to the meta question that is giving me pause on this
whole proposal:

    What does Linux get from merging mpool?

What you have above is a decent scalability bug report. That type of
pressure to meet new workload needs is how Linux interfaces evolve.
However, rather than evolve those interfaces mpool is a revolutionary
replacement that leaves the bugs intact for everyone that does not
switch over to mpool.

Consider io_uring as an example where the kernel resisted trends
towards userspace I/O engines and instead evolved a solution that
maintained kernel control while also achieving similar performance
levels.

The exercise is useful to identify places where Linux has
deficiencies, but wholesale replacing an entire I/O submission model
is a direction that leaves the old apis to rot.


From chaitanya.kulkarni at wdc.com  Tue Oct 20 19:14:03 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 20 Oct 2020 16:14:03 -0700
Subject: [PATCH V2 0/1] nvmet: don't use BLK_MQ_REQ_NOWAIT for passthru
Message-ID: <20201020231404.9365-1-chaitanya.kulkarni@wdc.com>

MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi,

When using NVMeOF target in the passthru mode we allocate the request
with BLK_MQ_REQ_NOWAIT flag. This allocates the request in the following
manner :-

nvme_alloc_request()
 blk_mq_alloc_request()
  blk_mq_queue_enter()
   if (flag & BLK_MQ_REQ_NOWAIT)
  	return -EBUSY; <-- return if busy.

On the NVMe controller which I've the fio random write workload running
parallel on 32 namespaces with higher queue depth results in I/O error,
where blk_mq_queue_enter() returning -EBUSY as shown above. This problem
is not easy to reproduce but occurs once in a while with following error
(See 1 for detailed log) :-

fio: io_u error on file /dev/nvme2n27: Input/output error:
write offset=1417535488, buflen=4096

Removing the BLK_MQ_REQ_NOWAIT flag fixes the error See [2].

Regards,
Chaitanya

* Changes from V1:-

1. Remove the configfs param which was added to retain the default
   behaviour.

Chaitanya Kulkarni (1):
  nvmet: don't use BLK_MQ_REQ_NOWAIT for passthru

 drivers/nvme/target/passthru.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

1. FIO workload resulting in the I/O error with default behavior :-
-------------------------------------------------------------------
fio-3.8-5-g464b
Starting 32 processes
fio: io_u error on file /dev/nvme2n27: Input/output error: write offset=1417535488, buflen=4096
fio: io_u error on file /dev/nvme2n27: Input/output error: write offset=3086512128, buflen=4096
fio: pid=7351, err=5/file:io_u.c:1744, func=io_u error, error=Input/output error

test1: (groupid=0, jobs=32): err= 5 (file:io_u.c:1744, func=io_u error, error=Input/output error): pid=7231: Tue Oct 20 15:58:39 2020
  write: IOPS=371k, BW=1449MiB/s (1519MB/s)(28.3GiB/20008msec)
    slat (usec): min=7, max=4395.1k, avg=80.28, stdev=7346.51
    clat (usec): min=59, max=4950.2k, avg=10959.88, stdev=85948.79
     lat (usec): min=269, max=4950.3k, avg=11040.29, stdev=86297.65
    clat percentiles (msec):
     |  1.00th=[    5],  5.00th=[    7], 10.00th=[    8], 20.00th=[    8],
     | 30.00th=[    8], 40.00th=[    8], 50.00th=[    9], 60.00th=[    9],
     | 70.00th=[    9], 80.00th=[   11], 90.00th=[   12], 95.00th=[   13],
     | 99.00th=[   14], 99.50th=[   14], 99.90th=[  266], 99.95th=[ 2198],
     | 99.99th=[ 3977]
   bw (  KiB/s): min=   56, max=135872, per=3.63%, avg=53903.52, stdev=14740.26, samples=1096
   iops        : min=   14, max=33968, avg=13475.80, stdev=3685.05, samples=1096
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.69%, 10=77.71%, 20=21.43%, 50=0.05%
  lat (msec)   : 250=0.01%, 500=0.02%, 750=0.01%, 1000=0.01%
  cpu          : usr=2.13%, sys=29.35%, ctx=615306, majf=0, minf=425
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,7420387,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=1449MiB/s (1519MB/s), 1449MiB/s-1449MiB/s (1519MB/s-1519MB/s), io=28.3GiB (30.4GB), run=20008-20008msec

Disk stats (read/write):
  nvme2n1: ios=0/197898, merge=0/0, ticks=0/228729, in_queue=228730, util=33.25%
  nvme2n2: ios=40/224130, merge=0/0, ticks=4/228306, in_queue=228310, util=39.41%
  nvme2n3: ios=40/239787, merge=0/0, ticks=4/228579, in_queue=228584, util=41.12%
  nvme2n4: ios=40/233169, merge=0/0, ticks=7/238798, in_queue=238806, util=41.18%
  nvme2n5: ios=40/211806, merge=0/0, ticks=8/213539, in_queue=213547, util=38.52%
  nvme2n6: ios=40/274195, merge=0/0, ticks=6/228864, in_queue=228871, util=47.11%
  nvme2n7: ios=40/233491, merge=0/0, ticks=6/231474, in_queue=231481, util=43.22%
  nvme2n8: ios=40/218498, merge=0/0, ticks=8/231410, in_queue=231419, util=42.28%
  nvme2n9: ios=40/212493, merge=0/0, ticks=6/224277, in_queue=224284, util=40.88%
  nvme2n10: ios=0/228695, merge=0/0, ticks=0/219852, in_queue=219853, util=45.08%
  nvme2n11: ios=0/231399, merge=0/0, ticks=0/223938, in_queue=223938, util=46.55%
  nvme2n12: ios=0/235817, merge=0/0, ticks=0/210005, in_queue=210005, util=47.22%
  nvme2n13: ios=0/206752, merge=0/0, ticks=0/217374, in_queue=217375, util=42.48%
  nvme2n14: ios=0/238433, merge=0/0, ticks=0/231576, in_queue=231576, util=49.51%
  nvme2n15: ios=0/243670, merge=0/0, ticks=0/242351, in_queue=242352, util=52.40%
  nvme2n16: ios=0/236602, merge=0/0, ticks=0/235182, in_queue=235183, util=51.65%
  nvme2n17: ios=0/221305, merge=0/0, ticks=0/221968, in_queue=221968, util=51.56%
  nvme2n18: ios=0/243352, merge=0/0, ticks=0/226829, in_queue=226829, util=55.10%
  nvme2n19: ios=0/228293, merge=0/0, ticks=0/240925, in_queue=240926, util=55.47%
  nvme2n20: ios=0/228712, merge=0/0, ticks=0/234262, in_queue=234262, util=57.27%
  nvme2n21: ios=0/222757, merge=0/0, ticks=0/218464, in_queue=218465, util=56.75%
  nvme2n22: ios=0/221461, merge=0/0, ticks=0/230017, in_queue=230017, util=59.04%
  nvme2n23: ios=0/251562, merge=0/0, ticks=0/238324, in_queue=238324, util=63.18%
  nvme2n24: ios=0/227673, merge=0/0, ticks=0/227511, in_queue=227512, util=61.54%
  nvme2n25: ios=0/224529, merge=0/0, ticks=0/232201, in_queue=232201, util=64.99%
  nvme2n26: ios=0/213535, merge=0/0, ticks=0/220483, in_queue=220484, util=62.53%
  nvme2n27: ios=0/215235, merge=0/0, ticks=0/226286, in_queue=226286, util=66.75%
  nvme2n28: ios=0/234016, merge=0/0, ticks=0/236245, in_queue=236246, util=71.34%
  nvme2n29: ios=0/263185, merge=0/0, ticks=0/244128, in_queue=244129, util=78.04%
  nvme2n30: ios=0/234893, merge=0/0, ticks=0/242473, in_queue=242473, util=77.01%
  nvme2n31: ios=0/252381, merge=0/0, ticks=0/234862, in_queue=234862, util=79.70%
  nvme2n32: ios=0/229931, merge=0/0, ticks=0/241206, in_queue=241206, util=78.84%

2. FIO workload without any error when not using BLK_MQ_REQ_NOWAIT :-
---------------------------------------------------------------------
fio-3.8-5-g464b
Starting 32 processes
Jobs: 32 (f=32): [w(32)][100.0%][r=0KiB/s,w=1586MiB/s][r=0,w=406k IOPS][eta 00m:00s]
test1: (groupid=0, jobs=32): err= 0: pid=8355: Tue Oct 20 16:01:38 2020
  write: IOPS=379k, BW=1482MiB/s (1554MB/s)(86.8GiB/60001msec)
    slat (usec): min=7, max=2921.9k, avg=77.70, stdev=2888.32
    clat (usec): min=66, max=3151.7k, avg=10718.84, stdev=34419.95
     lat (usec): min=138, max=3151.7k, avg=10796.67, stdev=34562.09
    clat percentiles (msec):
     |  1.00th=[    7],  5.00th=[    8], 10.00th=[    8], 20.00th=[    9],
     | 30.00th=[    9], 40.00th=[    9], 50.00th=[   10], 60.00th=[   10],
     | 70.00th=[   10], 80.00th=[   12], 90.00th=[   14], 95.00th=[   15],
     | 99.00th=[   16], 99.50th=[   17], 99.90th=[  249], 99.95th=[  550],
     | 99.99th=[ 2106]
   bw (  KiB/s): min=   56, max=86480, per=3.24%, avg=49129.37, stdev=10469.48, samples=3676
   iops        : min=   14, max=21620, avg=12282.27, stdev=2617.36, samples=3676
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=70.74%, 20=28.99%, 50=0.13%
  lat (msec)   : 100=0.01%, 250=0.02%, 500=0.04%, 750=0.03%, 1000=0.01%
  cpu          : usr=2.33%, sys=33.88%, ctx=2147232, majf=0, minf=400
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,22760015,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=1482MiB/s (1554MB/s), 1482MiB/s-1482MiB/s (1554MB/s-1554MB/s), io=86.8GiB (93.2GB), run=60001-60001msec

Disk stats (read/write):
  nvme2n1: ios=0/706667, merge=0/0, ticks=0/599438, in_queue=599439, util=67.26%
  nvme2n2: ios=40/708959, merge=0/0, ticks=7/597489, in_queue=597496, util=68.74%
  nvme2n3: ios=40/708004, merge=0/0, ticks=9/597836, in_queue=597846, util=68.69%
  nvme2n4: ios=40/748019, merge=0/0, ticks=5/605254, in_queue=605259, util=72.55%
  nvme2n5: ios=40/730666, merge=0/0, ticks=8/590529, in_queue=590538, util=71.24%
  nvme2n6: ios=40/713818, merge=0/0, ticks=6/557329, in_queue=557336, util=70.82%
  nvme2n7: ios=40/706833, merge=0/0, ticks=8/596236, in_queue=596245, util=70.51%
  nvme2n8: ios=40/701539, merge=0/0, ticks=6/584228, in_queue=584235, util=70.79%
  nvme2n9: ios=40/715091, merge=0/0, ticks=7/566007, in_queue=566014, util=72.27%
  nvme2n10: ios=0/709021, merge=0/0, ticks=0/583441, in_queue=583441, util=72.46%
  nvme2n11: ios=0/682020, merge=0/0, ticks=0/555933, in_queue=555934, util=71.22%
  nvme2n12: ios=0/692200, merge=0/0, ticks=0/595379, in_queue=595379, util=73.53%
  nvme2n13: ios=0/692685, merge=0/0, ticks=0/575632, in_queue=575632, util=73.10%
  nvme2n14: ios=0/712976, merge=0/0, ticks=0/580104, in_queue=580104, util=75.03%
  nvme2n15: ios=0/695618, merge=0/0, ticks=0/572385, in_queue=572386, util=75.13%
  nvme2n16: ios=0/704321, merge=0/0, ticks=0/569438, in_queue=569438, util=76.12%
  nvme2n17: ios=0/682198, merge=0/0, ticks=0/574509, in_queue=574509, util=76.62%
  nvme2n18: ios=0/698455, merge=0/0, ticks=0/580754, in_queue=580755, util=78.41%
  nvme2n19: ios=0/705429, merge=0/0, ticks=0/582465, in_queue=582466, util=79.51%
  nvme2n20: ios=0/719166, merge=0/0, ticks=0/573492, in_queue=573492, util=82.13%
  nvme2n21: ios=0/708253, merge=0/0, ticks=0/572174, in_queue=572174, util=81.47%
  nvme2n22: ios=0/730410, merge=0/0, ticks=0/600699, in_queue=600700, util=83.61%
  nvme2n23: ios=0/718952, merge=0/0, ticks=0/574481, in_queue=574482, util=83.71%
  nvme2n24: ios=0/711723, merge=0/0, ticks=0/582901, in_queue=582902, util=84.12%
  nvme2n25: ios=0/714620, merge=0/0, ticks=0/597742, in_queue=597742, util=85.40%
  nvme2n26: ios=0/713391, merge=0/0, ticks=0/608668, in_queue=608668, util=86.62%
  nvme2n27: ios=0/685253, merge=0/0, ticks=0/583583, in_queue=583584, util=85.96%
  nvme2n28: ios=0/720086, merge=0/0, ticks=0/586675, in_queue=586675, util=87.49%
  nvme2n29: ios=0/722030, merge=0/0, ticks=0/594997, in_queue=594998, util=90.26%
  nvme2n30: ios=0/730508, merge=0/0, ticks=0/601446, in_queue=601446, util=92.39%
  nvme2n31: ios=0/747852, merge=0/0, ticks=0/598947, in_queue=598947, util=93.79%
  nvme2n32: ios=0/698092, merge=0/0, ticks=0/584146, in_queue=584147, util=90.58%
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Oct 20 19:14:04 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 20 Oct 2020 16:14:04 -0700
Subject: [PATCH V2 1/1] nvmet: don't use BLK_MQ_REQ_NOWAIT for passthru
In-Reply-To: <20201020231404.9365-1-chaitanya.kulkarni@wdc.com>
References: <20201020231404.9365-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201020231404.9365-2-chaitanya.kulkarni@wdc.com>

By default, we set the passthru request allocation flag such that it
returns the error in the following code path and we fail the I/O when
BLK_MQ_REQ_NOWAIT is used for request allocation :-

nvme_alloc_request()
?blk_mq_alloc_request()
? blk_mq_queue_enter()
? ?if (flag & BLK_MQ_REQ_NOWAIT)
? ? ? ? return -EBUSY; <-- return if busy.

On some controllers using BLK_MQ_REQ_NOWAIT ends up in I/O error where
the controller is perfectly healthy and not in a degraded state.

Block layer request allocation does allow us to wait instead of
immediately returning the error when we BLK_MQ_REQ_NOWAIT flag is not
used. This has shown to fix the I/O error problem reported under
heavy random write workload.

Remove the BLK_MQ_REQ_NOWAIT parameter for passthru request allocation
which resolves this issue.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 56c571052216..d082351569de 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -236,7 +236,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		q = ns->queue;
 	}
 
-	rq = nvme_alloc_request(q, req->cmd, BLK_MQ_REQ_NOWAIT, NVME_QID_ANY);
+	rq = nvme_alloc_request(q, req->cmd, 0, NVME_QID_ANY);
 	if (IS_ERR(rq)) {
 		status = NVME_SC_INTERNAL;
 		goto out_put_ns;
-- 
2.22.1



From logang at deltatee.com  Tue Oct 20 19:15:52 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Tue, 20 Oct 2020 17:15:52 -0600
Subject: [PATCH V2 1/1] nvmet: don't use BLK_MQ_REQ_NOWAIT for passthru
In-Reply-To: <20201020231404.9365-2-chaitanya.kulkarni@wdc.com>
References: <20201020231404.9365-1-chaitanya.kulkarni@wdc.com>
 <20201020231404.9365-2-chaitanya.kulkarni@wdc.com>
Message-ID: <32159323-7473-c699-ff16-32380ae385e3@deltatee.com>



On 2020-10-20 5:14 p.m., Chaitanya Kulkarni wrote:
> By default, we set the passthru request allocation flag such that it
> returns the error in the following code path and we fail the I/O when
> BLK_MQ_REQ_NOWAIT is used for request allocation :-
> 
> nvme_alloc_request()
> ?blk_mq_alloc_request()
> ? blk_mq_queue_enter()
> ? ?if (flag & BLK_MQ_REQ_NOWAIT)
> ? ? ? ? return -EBUSY; <-- return if busy.
> 
> On some controllers using BLK_MQ_REQ_NOWAIT ends up in I/O error where
> the controller is perfectly healthy and not in a degraded state.
> 
> Block layer request allocation does allow us to wait instead of
> immediately returning the error when we BLK_MQ_REQ_NOWAIT flag is not
> used. This has shown to fix the I/O error problem reported under
> heavy random write workload.
> 
> Remove the BLK_MQ_REQ_NOWAIT parameter for passthru request allocation
> which resolves this issue.
> 
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>

Makes sense to me:

Reviewed-by: Logan Gunthorpe <logang at deltatee.com>


From kbusch at kernel.org  Tue Oct 20 19:54:54 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 20 Oct 2020 16:54:54 -0700
Subject: NVMe CLI Version Tag
In-Reply-To: <20201020175644.GA1438267@dhcp-10-100-145-180.wdc.com>
References: <BYAPR04MB5656B49BFE47211B3A2F1748EA1F0@BYAPR04MB5656.namprd04.prod.outlook.com>
 <20201020175017.2avzkm2bwyeg2t3a@beryllium.lan>
 <20201020175644.GA1438267@dhcp-10-100-145-180.wdc.com>
Message-ID: <20201020235454.GB1438267@dhcp-10-100-145-180.wdc.com>

On Tue, Oct 20, 2020 at 10:56:44AM -0700, Keith Busch wrote:
> On Tue, Oct 20, 2020 at 07:50:17PM +0200, Daniel Wagner wrote:
> > On Tue, Oct 20, 2020 at 03:52:25PM +0000, Jeffrey Lien wrote:
> > > I'd like to request that the nvme cli version be updated to 1.13 and
> > > then tagged.   It's been 4 months since it was last changed and
> > > tagged.
> > 
> > FWIW, I would also appreciate to have a fresh release of nvme-cli.
> 
> Sure thing, I'll just run it through some sanity tests this afternoon
> and set a new tag this evening if everything checks out.

Looks okay on my end, so v1.13 tagged and pushed. I didn't pull any of
the pending requests like I usually do for this release as that
inevitably leads to breakage right before a release. I'll start pulling
new stuff in now that this is hopefully a good stable point with ZNS
support and all the other recent goodies.


From ming.lei at redhat.com  Tue Oct 20 21:22:41 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed, 21 Oct 2020 09:22:41 +0800
Subject: [PATCH V2 3/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <e1d53ce3-ea27-439e-5e7a-ba790742908e@huawei.com>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
 <20201020085301.1553959-4-ming.lei@redhat.com>
 <e1d53ce3-ea27-439e-5e7a-ba790742908e@huawei.com>
Message-ID: <20201021012241.GC1571548@T590>

On Tue, Oct 20, 2020 at 05:04:29PM +0800, Chao Leng wrote:
> 
> 
> On 2020/10/20 16:53, Ming Lei wrote:
> > During controller's CONNECTING state, admin/fabric/connect requests
> > are submitted for recovery controller, and we allow to abort this request
> > directly in time out handler for not blocking setup procedure.
> > 
> > So timout vs. normal completion race exists on these requests since
> > admin/fabirc/connect queues won't be shutdown before handling timeout
> > during CONNECTING state.
> > 
> > Add atomic completion for requests from connect/fabric/admin queue for
> > avoiding the race.
> > 
> > CC: Chao Leng <lengchao at huawei.com>
> > Cc: Sagi Grimberg <sagi at grimberg.me>
> > Reported-by: Yi Zhang <yi.zhang at redhat.com>
> > Tested-by: Yi Zhang <yi.zhang at redhat.com>
> > Signed-off-by: Ming Lei <ming.lei at redhat.com>
> > ---
> >   drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
> >   1 file changed, 37 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > index d6a3e1487354..7e85bd4a8d1b 100644
> > --- a/drivers/nvme/host/tcp.c
> > +++ b/drivers/nvme/host/tcp.c
> > @@ -30,6 +30,8 @@ static int so_priority;
> >   module_param(so_priority, int, 0644);
> >   MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
> > +#define REQ_STATE_COMPLETE     0
> > +
> >   enum nvme_tcp_send_state {
> >   	NVME_TCP_SEND_CMD_PDU = 0,
> >   	NVME_TCP_SEND_H2C_PDU,
> > @@ -56,6 +58,8 @@ struct nvme_tcp_request {
> >   	size_t			offset;
> >   	size_t			data_sent;
> >   	enum nvme_tcp_send_state state;
> > +
> > +	unsigned long		comp_state;
> I do not think adding state is a good idea.
> It is similar to rq->state.
> In the teardown process, after quiesced queues delete the timer and
> cancel the timeout work maybe a better option.
> I will send the patch later.
> The patch is already tested with roce more than one week.

Actually there isn't race between timeout and teardown, and patch 1 and patch
2 are enough to fix the issue reported by Yi.

It is just that rq->state is updated to IDLE in its. complete(), so
either one of code paths may think that this rq isn't completed, and
patch 2 has addressed this issue.

In short, teardown lock is enough to cover the race.


Thanks,
Ming



From lengchao at huawei.com  Tue Oct 20 22:10:29 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 21 Oct 2020 10:10:29 +0800
Subject: [PATCH 1/3] nvme-core: introduce sync io queues
In-Reply-To: <BYAPR04MB49659A83BC2C4763B1CF2E6C861F0@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20201020090817.27755-1-lengchao@huawei.com>
 <BYAPR04MB49659A83BC2C4763B1CF2E6C861F0@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <cf50bebf-270d-09e9-7700-cbb13ee8926e@huawei.com>



On 2020/10/21 1:06, Chaitanya Kulkarni wrote:
> On 10/20/20 02:12, Chao Leng wrote:
>> Introduce sync io queues for some scenarios which just only need sync
>> io queues not sync all queues.
>>
>> Signed-off-by: Chao Leng <lengchao at huawei.com>
> 
> Can you please explain the scenario in detail ?
It is used for avoid race between time out and tear down. see patch 2/3.
The race may cause abnormal:
1. Reported by Yi Zhang <yi.zhang at redhat.com>
detail: https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/
2. BUG_ON in blk_mq_requeue_request
Because error recovery and time out may repeated completion request.
First error recovery cancel request in tear down process, the request
will be retried in completion, rq->state will be changed to IDEL.
And then time out will complete the request again, and samely retry
the request, BUG_ON will happen in blk_mq_requeue_request.
3. abnormal link disconnection
Firt error recovery cancel all request, reconnect success, the request
will be restarted. And then time out will complete the request again,
the queue will be stoped in nvme_rdma(tcp)_complete_timed_out,
Abnormal link diconnection will happen. The condition(time out process
is delayed long time by some reason such as hardware interrupt) is need.
So the probability is low.
> 
> .
> 


From lengchao at huawei.com  Tue Oct 20 22:16:41 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 21 Oct 2020 10:16:41 +0800
Subject: [PATCH 0/3] avoid race between time out and tear down
In-Reply-To: <9e1966b0-d471-0043-7fdb-e0f535571feb@grimberg.me>
References: <20201020090808.27702-1-lengchao@huawei.com>
 <9e1966b0-d471-0043-7fdb-e0f535571feb@grimberg.me>
Message-ID: <3d5b96dd-4e0f-758a-4175-e69fef0b8c34@huawei.com>



On 2020/10/21 2:43, Sagi Grimberg wrote:
> 
>> Avoid race between time out and tear down for rdma and tcp.
> 
> This patchset overall looks good, but we still need the patch that
> avoids double completion:
> 
> -- 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 629b025685d1..46428ff0b0fc 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>  ??????? /* fence other contexts that may complete the command */
>  ??????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>  ??????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
> -?????? if (!blk_mq_request_completed(rq)) {
> +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
Yes, this patch is need. and samely for nvme_cancel_request.
This will fix the race with asynchronous completion.

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e85f6304efd7..1e838d952096 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -338,7 +338,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
                                 "Cancelling I/O %d", req->tag);

         /* don't abort one completed request */
-       if (blk_mq_request_completed(req))
+       if (blk_mq_request_completed(req) || !blk_mq_request_started(rq))
                 return true;

         nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
-- 
>  ??????????????? nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
>  ??????????????? blk_mq_complete_request_sync(rq);
>  ??????? }
> -- 
> .


From lengchao at huawei.com  Tue Oct 20 22:20:11 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 21 Oct 2020 10:20:11 +0800
Subject: [PATCH V2 3/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <20201021012241.GC1571548@T590>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
 <20201020085301.1553959-4-ming.lei@redhat.com>
 <e1d53ce3-ea27-439e-5e7a-ba790742908e@huawei.com>
 <20201021012241.GC1571548@T590>
Message-ID: <7a3cdbdb-8e57-484c-fcb3-e8e72dfe8d13@huawei.com>



On 2020/10/21 9:22, Ming Lei wrote:
> On Tue, Oct 20, 2020 at 05:04:29PM +0800, Chao Leng wrote:
>>
>>
>> On 2020/10/20 16:53, Ming Lei wrote:
>>> During controller's CONNECTING state, admin/fabric/connect requests
>>> are submitted for recovery controller, and we allow to abort this request
>>> directly in time out handler for not blocking setup procedure.
>>>
>>> So timout vs. normal completion race exists on these requests since
>>> admin/fabirc/connect queues won't be shutdown before handling timeout
>>> during CONNECTING state.
>>>
>>> Add atomic completion for requests from connect/fabric/admin queue for
>>> avoiding the race.
>>>
>>> CC: Chao Leng <lengchao at huawei.com>
>>> Cc: Sagi Grimberg <sagi at grimberg.me>
>>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>>> Tested-by: Yi Zhang <yi.zhang at redhat.com>
>>> Signed-off-by: Ming Lei <ming.lei at redhat.com>
>>> ---
>>>    drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
>>>    1 file changed, 37 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index d6a3e1487354..7e85bd4a8d1b 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -30,6 +30,8 @@ static int so_priority;
>>>    module_param(so_priority, int, 0644);
>>>    MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
>>> +#define REQ_STATE_COMPLETE     0
>>> +
>>>    enum nvme_tcp_send_state {
>>>    	NVME_TCP_SEND_CMD_PDU = 0,
>>>    	NVME_TCP_SEND_H2C_PDU,
>>> @@ -56,6 +58,8 @@ struct nvme_tcp_request {
>>>    	size_t			offset;
>>>    	size_t			data_sent;
>>>    	enum nvme_tcp_send_state state;
>>> +
>>> +	unsigned long		comp_state;
>> I do not think adding state is a good idea.
>> It is similar to rq->state.
>> In the teardown process, after quiesced queues delete the timer and
>> cancel the timeout work maybe a better option.
>> I will send the patch later.
>> The patch is already tested with roce more than one week.
> 
> Actually there isn't race between timeout and teardown, and patch 1 and patch
> 2 are enough to fix the issue reported by Yi.
> 
> It is just that rq->state is updated to IDLE in its. complete(), so
> either one of code paths may think that this rq isn't completed, and
> patch 2 has addressed this issue.
> 
> In short, teardown lock is enough to cover the race.
The race may cause abnormals:
1. Reported by Yi Zhang <yi.zhang at redhat.com>
detail: https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/
2. BUG_ON in blk_mq_requeue_request
Because error recovery and time out may repeated completion request.
First error recovery cancel request in tear down process, the request
will be retried in completion, rq->state will be changed to IDEL.
And then time out will complete the request again, and samely retry
the request, BUG_ON will happen in blk_mq_requeue_request.
3. abnormal link disconnection
Firt error recovery cancel all request, reconnect success, the request
will be restarted. And then time out will complete the request again,
the queue will be stoped in nvme_rdma(tcp)_complete_timed_out,
Abnormal link diconnection will happen. The condition(time out process
is delayed long time by some reason such as hardware interrupt) is need.
So the probability is low.

teardown_lock just serialize the race. and add checkint the rq->state can avoid
the 1 and 2 scenario, but 3 scenario can not be fixed.
> 
> 
> Thanks,
> Ming
> 
> .
> 


From ming.lei at redhat.com  Tue Oct 20 22:55:34 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Wed, 21 Oct 2020 10:55:34 +0800
Subject: [PATCH V2 3/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <7a3cdbdb-8e57-484c-fcb3-e8e72dfe8d13@huawei.com>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
 <20201020085301.1553959-4-ming.lei@redhat.com>
 <e1d53ce3-ea27-439e-5e7a-ba790742908e@huawei.com>
 <20201021012241.GC1571548@T590>
 <7a3cdbdb-8e57-484c-fcb3-e8e72dfe8d13@huawei.com>
Message-ID: <20201021025534.GD1571548@T590>

On Wed, Oct 21, 2020 at 10:20:11AM +0800, Chao Leng wrote:
> 
> 
> On 2020/10/21 9:22, Ming Lei wrote:
> > On Tue, Oct 20, 2020 at 05:04:29PM +0800, Chao Leng wrote:
> > > 
> > > 
> > > On 2020/10/20 16:53, Ming Lei wrote:
> > > > During controller's CONNECTING state, admin/fabric/connect requests
> > > > are submitted for recovery controller, and we allow to abort this request
> > > > directly in time out handler for not blocking setup procedure.
> > > > 
> > > > So timout vs. normal completion race exists on these requests since
> > > > admin/fabirc/connect queues won't be shutdown before handling timeout
> > > > during CONNECTING state.
> > > > 
> > > > Add atomic completion for requests from connect/fabric/admin queue for
> > > > avoiding the race.
> > > > 
> > > > CC: Chao Leng <lengchao at huawei.com>
> > > > Cc: Sagi Grimberg <sagi at grimberg.me>
> > > > Reported-by: Yi Zhang <yi.zhang at redhat.com>
> > > > Tested-by: Yi Zhang <yi.zhang at redhat.com>
> > > > Signed-off-by: Ming Lei <ming.lei at redhat.com>
> > > > ---
> > > >    drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
> > > >    1 file changed, 37 insertions(+), 3 deletions(-)
> > > > 
> > > > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > > > index d6a3e1487354..7e85bd4a8d1b 100644
> > > > --- a/drivers/nvme/host/tcp.c
> > > > +++ b/drivers/nvme/host/tcp.c
> > > > @@ -30,6 +30,8 @@ static int so_priority;
> > > >    module_param(so_priority, int, 0644);
> > > >    MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
> > > > +#define REQ_STATE_COMPLETE     0
> > > > +
> > > >    enum nvme_tcp_send_state {
> > > >    	NVME_TCP_SEND_CMD_PDU = 0,
> > > >    	NVME_TCP_SEND_H2C_PDU,
> > > > @@ -56,6 +58,8 @@ struct nvme_tcp_request {
> > > >    	size_t			offset;
> > > >    	size_t			data_sent;
> > > >    	enum nvme_tcp_send_state state;
> > > > +
> > > > +	unsigned long		comp_state;
> > > I do not think adding state is a good idea.
> > > It is similar to rq->state.
> > > In the teardown process, after quiesced queues delete the timer and
> > > cancel the timeout work maybe a better option.
> > > I will send the patch later.
> > > The patch is already tested with roce more than one week.
> > 
> > Actually there isn't race between timeout and teardown, and patch 1 and patch
> > 2 are enough to fix the issue reported by Yi.
> > 
> > It is just that rq->state is updated to IDLE in its. complete(), so
> > either one of code paths may think that this rq isn't completed, and
> > patch 2 has addressed this issue.
> > 
> > In short, teardown lock is enough to cover the race.
> The race may cause abnormals:
> 1. Reported by Yi Zhang <yi.zhang at redhat.com>
> detail: https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/
> 2. BUG_ON in blk_mq_requeue_request
> Because error recovery and time out may repeated completion request.
> First error recovery cancel request in tear down process, the request
> will be retried in completion, rq->state will be changed to IDEL.

Right.

> And then time out will complete the request again, and samely retry
> the request, BUG_ON will happen in blk_mq_requeue_request.

With patch2 in this patchset, timeout handler won't complete the request any
more.

> 3. abnormal link disconnection
> Firt error recovery cancel all request, reconnect success, the request
> will be restarted. And then time out will complete the request again,
> the queue will be stoped in nvme_rdma(tcp)_complete_timed_out,
> Abnormal link diconnection will happen. The condition(time out process
> is delayed long time by some reason such as hardware interrupt) is need.
> So the probability is low.

OK, the timeout handler may just get chance to run after recovery is
done, and it can be fixed by calling nvme_sync_queues() after
updating to CONNECTING or before updating to LIVE together with patch 1 & 2.

> teardown_lock just serialize the race. and add checkint the rq->state can avoid
> the 1 and 2 scenario, but 3 scenario can not be fixed.

I didn't think of scenario 3, which seems not triggered in Yi's test.


thanks,
Ming



From lengchao at huawei.com  Tue Oct 20 23:14:37 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 21 Oct 2020 11:14:37 +0800
Subject: [PATCH V2 3/4] nvme: tcp: complete non-IO requests atomically
In-Reply-To: <20201021025534.GD1571548@T590>
References: <20201020085301.1553959-1-ming.lei@redhat.com>
 <20201020085301.1553959-4-ming.lei@redhat.com>
 <e1d53ce3-ea27-439e-5e7a-ba790742908e@huawei.com>
 <20201021012241.GC1571548@T590>
 <7a3cdbdb-8e57-484c-fcb3-e8e72dfe8d13@huawei.com>
 <20201021025534.GD1571548@T590>
Message-ID: <c710fdd5-c433-16a4-4d78-099a880344ba@huawei.com>



On 2020/10/21 10:55, Ming Lei wrote:
> On Wed, Oct 21, 2020 at 10:20:11AM +0800, Chao Leng wrote:
>>
>>
>> On 2020/10/21 9:22, Ming Lei wrote:
>>> On Tue, Oct 20, 2020 at 05:04:29PM +0800, Chao Leng wrote:
>>>>
>>>>
>>>> On 2020/10/20 16:53, Ming Lei wrote:
>>>>> During controller's CONNECTING state, admin/fabric/connect requests
>>>>> are submitted for recovery controller, and we allow to abort this request
>>>>> directly in time out handler for not blocking setup procedure.
>>>>>
>>>>> So timout vs. normal completion race exists on these requests since
>>>>> admin/fabirc/connect queues won't be shutdown before handling timeout
>>>>> during CONNECTING state.
>>>>>
>>>>> Add atomic completion for requests from connect/fabric/admin queue for
>>>>> avoiding the race.
>>>>>
>>>>> CC: Chao Leng <lengchao at huawei.com>
>>>>> Cc: Sagi Grimberg <sagi at grimberg.me>
>>>>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>>>>> Tested-by: Yi Zhang <yi.zhang at redhat.com>
>>>>> Signed-off-by: Ming Lei <ming.lei at redhat.com>
>>>>> ---
>>>>>     drivers/nvme/host/tcp.c | 40 +++++++++++++++++++++++++++++++++++++---
>>>>>     1 file changed, 37 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>> index d6a3e1487354..7e85bd4a8d1b 100644
>>>>> --- a/drivers/nvme/host/tcp.c
>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>> @@ -30,6 +30,8 @@ static int so_priority;
>>>>>     module_param(so_priority, int, 0644);
>>>>>     MODULE_PARM_DESC(so_priority, "nvme tcp socket optimize priority");
>>>>> +#define REQ_STATE_COMPLETE     0
>>>>> +
>>>>>     enum nvme_tcp_send_state {
>>>>>     	NVME_TCP_SEND_CMD_PDU = 0,
>>>>>     	NVME_TCP_SEND_H2C_PDU,
>>>>> @@ -56,6 +58,8 @@ struct nvme_tcp_request {
>>>>>     	size_t			offset;
>>>>>     	size_t			data_sent;
>>>>>     	enum nvme_tcp_send_state state;
>>>>> +
>>>>> +	unsigned long		comp_state;
>>>> I do not think adding state is a good idea.
>>>> It is similar to rq->state.
>>>> In the teardown process, after quiesced queues delete the timer and
>>>> cancel the timeout work maybe a better option.
>>>> I will send the patch later.
>>>> The patch is already tested with roce more than one week.
>>>
>>> Actually there isn't race between timeout and teardown, and patch 1 and patch
>>> 2 are enough to fix the issue reported by Yi.
>>>
>>> It is just that rq->state is updated to IDLE in its. complete(), so
>>> either one of code paths may think that this rq isn't completed, and
>>> patch 2 has addressed this issue.
>>>
>>> In short, teardown lock is enough to cover the race.
>> The race may cause abnormals:
>> 1. Reported by Yi Zhang <yi.zhang at redhat.com>
>> detail: https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/
>> 2. BUG_ON in blk_mq_requeue_request
>> Because error recovery and time out may repeated completion request.
>> First error recovery cancel request in tear down process, the request
>> will be retried in completion, rq->state will be changed to IDEL.
> 
> Right.
> 
>> And then time out will complete the request again, and samely retry
>> the request, BUG_ON will happen in blk_mq_requeue_request.
> 
> With patch2 in this patchset, timeout handler won't complete the request any
> more.
> 
>> 3. abnormal link disconnection
>> Firt error recovery cancel all request, reconnect success, the request
>> will be restarted. And then time out will complete the request again,
>> the queue will be stoped in nvme_rdma(tcp)_complete_timed_out,
>> Abnormal link diconnection will happen. The condition(time out process
>> is delayed long time by some reason such as hardware interrupt) is need.
>> So the probability is low.
> 
> OK, the timeout handler may just get chance to run after recovery is
> done, and it can be fixed by calling nvme_sync_queues() after
> updating to CONNECTING or before updating to LIVE together with patch 1 & 2.
> 
>> teardown_lock just serialize the race. and add checkint the rq->state can avoid
>> the 1 and 2 scenario, but 3 scenario can not be fixed.
> 
> I didn't think of scenario 3, which seems not triggered in Yi's test.
The scenario 3 is unlikely triggered in normal test.
The trigger condition are harsh. It'll only happen in some extreme situations.
If without scenario 3, Sagi's patch can work well.
> 
> 
> thanks,
> Ming
> 
> .
> 


From sagi at grimberg.me  Wed Oct 21 00:59:01 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 20 Oct 2020 21:59:01 -0700
Subject: [PATCH 0/3] avoid race between time out and tear down
In-Reply-To: <3d5b96dd-4e0f-758a-4175-e69fef0b8c34@huawei.com>
References: <20201020090808.27702-1-lengchao@huawei.com>
 <9e1966b0-d471-0043-7fdb-e0f535571feb@grimberg.me>
 <3d5b96dd-4e0f-758a-4175-e69fef0b8c34@huawei.com>
Message-ID: <b60f34ea-86f1-ef86-8e1f-b156b7681699@grimberg.me>


>>> Avoid race between time out and tear down for rdma and tcp.
>>
>> This patchset overall looks good, but we still need the patch that
>> avoids double completion:
>>
>> -- 
>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>> index 629b025685d1..46428ff0b0fc 100644
>> --- a/drivers/nvme/host/tcp.c
>> +++ b/drivers/nvme/host/tcp.c
>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct 
>> request *rq)
>> ???????? /* fence other contexts that may complete the command */
>> ???????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>> ???????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>> -?????? if (!blk_mq_request_completed(rq)) {
>> +?????? if (blk_mq_request_started(rq) && 
>> !blk_mq_request_completed(rq)) {
> Yes, this patch is need. and samely for nvme_cancel_request.
> This will fix the race with asynchronous completion.
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index e85f6304efd7..1e838d952096 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -338,7 +338,7 @@ bool nvme_cancel_request(struct request *req, void 
> *data, bool reserved)
>  ??????????????????????????????? "Cancelling I/O %d", req->tag);
> 
>  ??????? /* don't abort one completed request */
> -?????? if (blk_mq_request_completed(req))
> +?????? if (blk_mq_request_completed(req) || !blk_mq_request_started(rq))
>  ??????????????? return true;
> 
>  ??????? nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;

This one is unneeded because blk_mq_tagset_busy_iter checks that the
request has started...


From yi.zhang at redhat.com  Wed Oct 21 01:55:06 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Wed, 21 Oct 2020 13:55:06 +0800
Subject: [bug report] rdma_rxe: kernel NULL pointer observed with blktests
 nvme/012 on ppc64le/aarch64
In-Reply-To: <fa25d298-33bc-a110-7074-5f87d7fd418c@gmail.com>
References: <1650006240.4229491.1603034423590.JavaMail.zimbra@redhat.com>
 <fa25d298-33bc-a110-7074-5f87d7fd418c@gmail.com>
Message-ID: <1ab3fd1f-5bfe-9778-4abc-b9e6d20357d9@redhat.com>



On 10/20/20 2:54 AM, Bob Pearson wrote:
> On 10/18/20 10:20 AM, Yi Zhang wrote:
>> Hello
>>
>> I found this bug with blktests nvme/012 on ppc64le/aarch64, could anyone help check it,
>> Let me know if you need any test for further investigation, thanks.
>>
>> ppc64le:
>> [  155.427446] run blktests nvme/012 at 2020-10-18 09:54:03
>> [  156.593195] rdma_rxe: loaded
>> [  156.614836] infiniband rxe0: set active
>> [  156.614843] infiniband rxe0: added env2
>> [  156.617421] lo speed is unknown, defaulting to 1000
>> [  156.617449] lo speed is unknown, defaulting to 1000
>> [  156.617484] lo speed is unknown, defaulting to 1000
>> [  156.619911] infiniband rxe1: set active
>> [  156.619916] infiniband rxe1: added lo
>> [  156.619987] lo speed is unknown, defaulting to 1000
>> [  156.640971] Rounding down aligned max_sectors from 4294967295 to 4294967168
>> [  156.641095] db_root: cannot open: /etc/target
>> [  156.655793] lo speed is unknown, defaulting to 1000
>> [  156.700482] loop: module loaded
>> [  156.744820] Loading iSCSI transport class v2.0-870.
>> [  156.805883] iscsi: registered transport (iser)
>> [  156.833559] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>> [  156.884254] nvmet_rdma: enabling port 0 (10.0.2.182:4420)
>> [  157.005564] RPC: Registered rdma transport module.
>> [  157.005573] RPC: Registered rdma backchannel transport module.
>> [  157.121039] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0c00c1b2-6456-4e5e-a0d4-9677000ca7bc.
>> [  157.121388] nvme nvme0: creating 16 I/O queues.
>> [  157.128894] nvme nvme0: mapped 16/0/0 default/read/poll queues.
>> [  157.130039] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 10.0.2.182:4420
>> [  158.196673] XFS (nvme0n1): Mounting V5 Filesystem
>> [  158.206313] XFS (nvme0n1): Ending clean mount
>> [  158.207141] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
>> [  190.087119] XFS (nvme0n1): Unmounting Filesystem
>> [  190.087284] BUG: Kernel NULL pointer dereference on read at 0x00000000
>> [  190.087289] Faulting instruction address: 0xc0000000000ae400
>> [  190.087294] Oops: Kernel access of bad area, sig: 11 [#1]
>> [  190.087298] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries
>> [  190.087302] Modules linked in: ib_isert iscsi_target_mod rpcrdma ib_iser libiscsi scsi_transport_iscsi loop ib_srpt target_core_mod crc32_generic rdma_rxe ib_srp ib_ipoib rdma_ucm ib_uverbs ip6_udp_tunnel udp_tunnel nvme_rdma nvme_fabrics nvme_core ib_umad nvmet_rdma rdma_cm iw_cm ib_cm ib_core nvmet rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd bonding grace fscache rfkill sunrpc xts pseries_rng uio_pdrv_genirq vmx_crypto uio ip_tables xfs libcrc32c sd_mod t10_pi sg ibmvscsi ibmveth scsi_transport_srp
>> [  190.087335] CPU: 9 PID: 56 Comm: ksoftirqd/9 Not tainted 5.9.0 #1
>> [  190.087339] NIP:  c0000000000ae400 LR: c008000002d1f77c CTR: 0000000000000007
>> [  190.087342] REGS: c00000033b9bf550 TRAP: 0380   Not tainted  (5.9.0)
>> [  190.087345] MSR:  800000000280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  CR: 48088280  XER: 20040000
>> [  190.087351] CFAR: c0000000000ae3a8 IRQMASK: 0
>> [  190.087351] GPR00: c008000002d1f77c c00000033b9bf7e0 c000000001842500 c00000033133b086
>> [  190.087351] GPR04: 0000000000000000 00000000000003c0 0000000000000007 0000000000000001
>> [  190.087351] GPR08: c00000033b9bfa70 c00000033133b086 0000000000000000 c008000002d243f0
>> [  190.087351] GPR12: c0000000000b8680 c00000001eca3200 c00000032d2f4000 0000000000000001
>> [  190.087351] GPR16: c00000032d2f4000 0000000000000400 0000000000000000 c008000007c3ba38
>> [  190.087351] GPR20: 0000000000000000 c000000338a06c00 c00000033b9bfa70 c00000033b9bfa70
>> [  190.087351] GPR24: 0000000000000001 c00000033133b086 c000000334083f00 000000005e7d553d
>> [  190.087351] GPR28: 0000000000000000 00000000000003c0 00000000000003c0 0000000000000001
>> [  190.087383] NIP [c0000000000ae400] memcpy_power7+0xa0/0x7e0
>> [  190.087388] LR [c008000002d1f77c] rxe_mem_copy+0x1f4/0x308 [rdma_rxe]
>> [  190.087391] Call Trace:
>> [  190.087394] [c00000033b9bf7e0] [c000000327d83158] 0xc000000327d83158 (unreliable)
>> [  190.087399] [c00000033b9bf8e0] [c008000002d1f77c] rxe_mem_copy+0x1f4/0x308 [rdma_rxe]
>> [  190.087404] [c00000033b9bf970] [c008000002d1fbe4] copy_data+0x11c/0x460 [rdma_rxe]
>> [  190.087409] [c00000033b9bfa00] [c008000002d131ac] rxe_requester+0x1054/0x1368 [rdma_rxe]
>> [  190.087414] [c00000033b9bfb50] [c008000002d21298] rxe_do_task+0x110/0x1d0 [rdma_rxe]
>> [  190.087418] [c00000033b9bfbe0] [c000000000157f40] tasklet_action_common.isra.18+0x1b0/0x1c0
>> [  190.087424] [c00000033b9bfc40] [c000000000d4e8bc] __do_softirq+0x15c/0x3b4
>> [  190.087427] [c00000033b9bfd30] [c0000000001577f4] run_ksoftirqd+0x64/0x90
>> [  190.087432] [c00000033b9bfd50] [c00000000018a794] smpboot_thread_fn+0x204/0x270
>> [  190.087436] [c00000033b9bfdb0] [c000000000184070] kthread+0x1a0/0x1b0
>> [  190.087440] [c00000033b9bfe20] [c00000000000d3d0] ret_from_kernel_thread+0x5c/0x6c
>> [  190.087443] Instruction dump:
>> [  190.087446] f9c10070 f9e10078 fa010080 fa210088 fa410090 fa610098 fa8100a0 faa100a8
>> [  190.087451] fac100b0 f8010110 78a6c9c2 7cc903a6 <e8040000> e8c40008 e8e40010 e9040018
>> [  190.087458] ---[ end trace 244246d4a62eb74b ]---
>> [  190.089228]
>> [  191.089236] Kernel panic - not syncing: Fatal exception
>>
>>
>> # gdb drivers/infiniband/sw/rxe/rdma_rxe.ko
>> Reading symbols from drivers/infiniband/sw/rxe/rdma_rxe.ko...done.
>> (gdb) l *(rxe_mem_copy+0x1f4)
>> 0xf7bc is in rxe_mem_copy (./include/linux/string.h:406).
>> 401			if (q_size < size)
>> 402				__read_overflow2();
>> 403		}
>> 404		if (p_size < size || q_size < size)
>> 405			fortify_panic(__func__);
>> 406		return __underlying_memcpy(p, q, size);
>> 407	}
>> 408	
>> 409	__FORTIFY_INLINE void *memmove(void *p, const void *q, __kernel_size_t size)
>> 410	{
>> (gdb)
>>
>> aarch64:
>> [  691.557907] r[  691.888082] lo speed is unknown, defaulting to 1000
>> [  691.892016] lo speed is unknown, defaulting to 1000
>> [  691.896920] lo speed is unknown, defaulting to 1000
>> [  691.904834] lo speed is unknown, defaulting to 1000
>> [  691.908762] lo speed is unknown, defaulting to 1000
>> [  691.913610] lo speed is unknown, defaulting to 1000
>> [  693.427251] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
>> [  730.178748] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
>> [  730.186576] Mem abort info:
>> [  730.189356]   ESR = 0x96000006
>> [  730.192385]   EC = 0x25: DABT (current EL), IL = 32 bits
>> [  730.197680]   SET = 0, FnV = 0
>> [  730.200725]   EA = 0, S1PTW = 0
>> [  730.203843] Data abort info:
>> [  730.206707]   ISV = 0, ISS = 0x00000006
>> [  730.210533]   CM = 0, WnR = 0
>> [  730.213479] user pgtable: 64k pages, 42-bit VAs, pgdp=00000017b7950000
>> [  730.219995] [0000000000000000] pgd=0000000000000000, p4d=0000000000000000, pud=0000000000000000, pmd=0000000000000000
>> [  730.230587] Internal error: Oops: 96000006 [#1] SMP
>> [  730.235441] Modules linked in: loop crc32_generic rdma_rxe ip6_udp_tunnel udp_tunnel nvme_rdma nvme_fabrics nvme_core nvmet_rdma nvmet rfkill vfat fat ib_isert iscsi_target_mod rpcrdma ib_srpt target_core_mod sunrpc ib_srp scsi_transport_srp rdma_ucm ib_umad ib_iser ib_ipoib libiscsi rdma_cm scsi_transport_iscsi ib_cm iw_cm dm_multipath mlx5_ib ib_uverbs ib_core acpi_ipmi crct10dif_ce ghash_ce sha2_ce sha256_arm64 ipmi_ssif sha1_ce sbsa_gwdt ipmi_devintf ipmi_msghandler ip_tables xfs libcrc32c sg mlx5_core sdhci_acpi mlxfw tls sdhci qcom_emac mmc_core ahci_platform libahci_platform hdma hdma_mgmt dm_mirror dm_region_hash dm_log dm_mod
>> [  730.291605] CPU: 24 PID: 131 Comm: ksoftirqd/24 Not tainted 5.8.0+ #2
>> [  730.298027] Hardware name: WIWYNN Qualcomm Centriq 2400 Reference Evaluation Platform CV90-LA115-P11/Qualcomm Centriq 2400 Customer Reference Board, BIOS
>> [  730.311830] pstate: 20400005 (nzCv daif +PAN -UAO BTYPE=--)
>> [  730.317392] pc : __memcpy+0x100/0x180
>> [  730.321038] lr : rxe_mem_copy+0x1fc/0x230 [rdma_rxe]
>> [  730.325978] sp : fffffe001588faf0
>> [  730.329277] x29: fffffe001588faf0 x28: fffffe00294b1080
>> [  730.334572] x27: fffffc17b5065086 x26: 00000000000003c0
>> [  730.339867] x25: 0000000000000000 x24: fffffe00113f3000
>> [  730.345162] x23: fffffc17ae07c0f0 x22: 00000000288c9f50
>> [  730.350457] x21: fffffe001588fc60 x20: 0000000000000001
>> [  730.355753] x19: 00000000000003c0 x18: 0000000000000002
>> [  730.361048] x17: 1df0130affff0000 x16: 0000000000000001
>> [  730.366343] x15: 0000000000000002 x14: 0000000000000000
>> [  730.371638] x13: 0000000000000000 x12: 0000000000000000
>> [  730.376933] x11: 0000000000000000 x10: 0000000000000000
>> [  730.382228] x9 : fffffe000a52dd44 x8 : 0000000000000000
>> [  730.387523] x7 : 0000000000000000 x6 : fffffc17b5065086
>> [  730.392818] x5 : fffffe001588fc60 x4 : 0000000000000000
>> [  730.398113] x3 : 00000000000003c0 x2 : 0000000000000340
>> [  730.403409] x1 : 0000000000000000 x0 : fffffc17b5065086
>> [  730.408705] Call trace:
>> [  730.411136]  __memcpy+0x100/0x180
>> [  730.414438]  copy_data+0xc4/0x318 [rdma_rxe]
>> [  730.418691]  rxe_requester+0xa58/0xe38 [rdma_rxe]
>> [  730.423379]  rxe_do_task+0x128/0x200 [rdma_rxe]
>> [  730.427892]  tasklet_action_common.isra.21+0xfc/0x130
>> [  730.432924]  tasklet_action+0x2c/0x38
>> [  730.436570]  __do_softirq+0x128/0x33c
>> [  730.440215]  run_ksoftirqd+0x40/0x58
>> [  730.443777]  smpboot_thread_fn+0x168/0x1b0
>> [  730.447856]  kthread+0x114/0x118
>> [  730.451066]  ret_from_fork+0x10/0x18
>> [  730.454627] Code: d503201f d503201f d503201f d503201f (a8c12027)
>> [  730.460720] ---[ end trace 7d9bff591d1280d9 ]---
>> [  730.465302] Kernel panic - not syncing: Fatal exception in interrupt
>> [  730.471648] SMP: stopping secondary CPUs
>> [  730.475614] Kernel Offset: disabled
>> [  730.479017] CPU features: 0x040002,61800418
>> [  730.483183] Memory Limit: none
>> [  730.486232] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
>>
>>
>> Best Regards,
>>    Yi Zhang
>>
>>
> The code looks different than current. What kernel version are you testing?

The ppc64le log is from 5.9.0, and aarch64 log is from 5.8.0
> Is there tight timing on MR creation/destruction/invalidation and use?
No, I only ran the blktests nvme_trtype=rdma nvme/012

> Bob Pearson
>



From lengchao at huawei.com  Wed Oct 21 02:51:46 2020
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 21 Oct 2020 14:51:46 +0800
Subject: [PATCH 0/3] avoid race between time out and tear down
In-Reply-To: <b60f34ea-86f1-ef86-8e1f-b156b7681699@grimberg.me>
References: <20201020090808.27702-1-lengchao@huawei.com>
 <9e1966b0-d471-0043-7fdb-e0f535571feb@grimberg.me>
 <3d5b96dd-4e0f-758a-4175-e69fef0b8c34@huawei.com>
 <b60f34ea-86f1-ef86-8e1f-b156b7681699@grimberg.me>
Message-ID: <5a48d20f-e9c9-5d37-776a-1fe3b1eb79a8@huawei.com>



On 2020/10/21 12:59, Sagi Grimberg wrote:
> 
>>>> Avoid race between time out and tear down for rdma and tcp.
>>>
>>> This patchset overall looks good, but we still need the patch that
>>> avoids double completion:
>>>
>>> -- 
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index 629b025685d1..46428ff0b0fc 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>> ???????? /* fence other contexts that may complete the command */
>>> ???????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>> ???????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>> -?????? if (!blk_mq_request_completed(rq)) {
>>> +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>> Yes, this patch is need. and samely for nvme_cancel_request.
>> This will fix the race with asynchronous completion.
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index e85f6304efd7..1e838d952096 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -338,7 +338,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
>> ???????????????????????????????? "Cancelling I/O %d", req->tag);
>>
>> ???????? /* don't abort one completed request */
>> -?????? if (blk_mq_request_completed(req))
>> +?????? if (blk_mq_request_completed(req) || !blk_mq_request_started(rq))
>> ???????????????? return true;
>>
>> ???????? nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
> 
> This one is unneeded because blk_mq_tagset_busy_iter checks that the
> request has started...
Yes, it is already checked.
> .


From sagi at grimberg.me  Wed Oct 21 04:53:07 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 21 Oct 2020 01:53:07 -0700
Subject: [PATCH 0/3] avoid race between time out and tear down
In-Reply-To: <5a48d20f-e9c9-5d37-776a-1fe3b1eb79a8@huawei.com>
References: <20201020090808.27702-1-lengchao@huawei.com>
 <9e1966b0-d471-0043-7fdb-e0f535571feb@grimberg.me>
 <3d5b96dd-4e0f-758a-4175-e69fef0b8c34@huawei.com>
 <b60f34ea-86f1-ef86-8e1f-b156b7681699@grimberg.me>
 <5a48d20f-e9c9-5d37-776a-1fe3b1eb79a8@huawei.com>
Message-ID: <db872ac4-662c-1e47-291a-7c64ae0cfa57@grimberg.me>


>>>>> Avoid race between time out and tear down for rdma and tcp.
>>>>
>>>> This patchset overall looks good, but we still need the patch that
>>>> avoids double completion:
>>>>
>>>> -- 
>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>> index 629b025685d1..46428ff0b0fc 100644
>>>> --- a/drivers/nvme/host/tcp.c
>>>> +++ b/drivers/nvme/host/tcp.c
>>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct 
>>>> request *rq)
>>>> ???????? /* fence other contexts that may complete the command */
>>>> ???????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>> ???????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>> -?????? if (!blk_mq_request_completed(rq)) {
>>>> +?????? if (blk_mq_request_started(rq) && 
>>>> !blk_mq_request_completed(rq)) {
>>> Yes, this patch is need. and samely for nvme_cancel_request.
>>> This will fix the race with asynchronous completion.
>>>
>>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>>> index e85f6304efd7..1e838d952096 100644
>>> --- a/drivers/nvme/host/core.c
>>> +++ b/drivers/nvme/host/core.c
>>> @@ -338,7 +338,7 @@ bool nvme_cancel_request(struct request *req, 
>>> void *data, bool reserved)
>>> ???????????????????????????????? "Cancelling I/O %d", req->tag);
>>>
>>> ???????? /* don't abort one completed request */
>>> -?????? if (blk_mq_request_completed(req))
>>> +?????? if (blk_mq_request_completed(req) || 
>>> !blk_mq_request_started(rq))
>>> ???????????????? return true;
>>>
>>> ???????? nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
>>
>> This one is unneeded because blk_mq_tagset_busy_iter checks that the
>> request has started...
> Yes, it is already checked.

Can you add the attached patches and resend?

You can also add my:
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-nvme-rdma-fix-possible-double-completion-for-timed-o.patch
Type: text/x-patch
Size: 2507 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201021/8648afcc/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-nvme-tcp-fix-possible-double-completion-for-timed-ou.patch
Type: text/x-patch
Size: 2542 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201021/8648afcc/attachment-0001.bin>

From snitzer at redhat.com  Wed Oct 21 10:24:05 2020
From: snitzer at redhat.com (Mike Snitzer)
Date: Wed, 21 Oct 2020 10:24:05 -0400
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
 <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
Message-ID: <CAMM=eLf+2VYHB6vZVjn_=GA5uXJWKL-d6PuCpHEBPz=_Loe58A@mail.gmail.com>

Hey Dan,

On Fri, Oct 16, 2020 at 6:38 PM Dan Williams <dan.j.williams at intel.com> wrote:
>
> On Fri, Oct 16, 2020 at 2:59 PM Nabeel Meeramohideen Mohamed
> (nmeeramohide) <nmeeramohide at micron.com> wrote:
>
> > (5) Representing an mpool as a /dev/mpool/<mpool-name> device file provides a
> > convenient mechanism for controlling access to and managing the multiple storage
> > volumes, and in the future pmem devices, that may comprise an logical mpool.
>
> Christoph and I have talked about replacing the pmem driver's
> dependence on device-mapper for pooling.

Was this discussion done publicly or private?  If public please share
a pointer to the thread.

I'd really like to understand the problem statement that is leading to
pursuing a pmem native alternative to existing DM.

Thanks,
Mike


From dan.j.williams at intel.com  Wed Oct 21 12:24:06 2020
From: dan.j.williams at intel.com (Dan Williams)
Date: Wed, 21 Oct 2020 09:24:06 -0700
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <CAMM=eLf+2VYHB6vZVjn_=GA5uXJWKL-d6PuCpHEBPz=_Loe58A@mail.gmail.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
 <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
 <CAMM=eLf+2VYHB6vZVjn_=GA5uXJWKL-d6PuCpHEBPz=_Loe58A@mail.gmail.com>
Message-ID: <CAPcyv4hj2iPmf4YNdJLZqHMh2B10hbkSnk_9BAAACbG_LFKfBQ@mail.gmail.com>

On Wed, Oct 21, 2020 at 7:24 AM Mike Snitzer <snitzer at redhat.com> wrote:
>
> Hey Dan,
>
> On Fri, Oct 16, 2020 at 6:38 PM Dan Williams <dan.j.williams at intel.com> wrote:
> >
> > On Fri, Oct 16, 2020 at 2:59 PM Nabeel Meeramohideen Mohamed
> > (nmeeramohide) <nmeeramohide at micron.com> wrote:
> >
> > > (5) Representing an mpool as a /dev/mpool/<mpool-name> device file provides a
> > > convenient mechanism for controlling access to and managing the multiple storage
> > > volumes, and in the future pmem devices, that may comprise an logical mpool.
> >
> > Christoph and I have talked about replacing the pmem driver's
> > dependence on device-mapper for pooling.
>
> Was this discussion done publicly or private?  If public please share
> a pointer to the thread.
>
> I'd really like to understand the problem statement that is leading to
> pursuing a pmem native alternative to existing DM.
>

IIRC it was during the hallway track at a conference. Some of the
concern is the flexibility to carve physical address space but not
attach a block-device in front of it, and allow pmem/dax-capable
filesystems to mount on something other than a block-device.

DM does fit the bill for block-device concatenation and striping, but
there's some pressure to have a level of provisioning beneath that.

The device-dax facility has already started to grow some physical
address space partitioning capabilities this cycle, see 60e93dc097f7
device-dax: add dis-contiguous resource support, and the question
becomes when / if that support needs to extend across regions is DM
the right tool for that?


From nmeeramohide at micron.com  Wed Oct 21 13:10:59 2020
From: nmeeramohide at micron.com (Nabeel Meeramohideen Mohamed (nmeeramohide))
Date: Wed, 21 Oct 2020 17:10:59 +0000
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <CAPcyv4iYOk3i0pPgXxDTy47BxWCXqqXS0J6mrY5o+1M_41XoAw@mail.gmail.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
 <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
 <SN6PR08MB420843C280D54D7B5B76EB78B31E0@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4iYOk3i0pPgXxDTy47BxWCXqqXS0J6mrY5o+1M_41XoAw@mail.gmail.com>
Message-ID: <SN6PR08MB4208A22F5C94AB05439A2703B31C0@SN6PR08MB4208.namprd08.prod.outlook.com>

On Tuesday, October 20, 2020 3:36 PM, Dan Williams <dan.j.williams at intel.com> wrote:
> 
>     What does Linux get from merging mpool?
> 

What Linux gets from merging mpool is a generic object store target with some
unique and beneficial features:

- the ability to allocate objects from multiple classes of media
- facilities to memory-map (and unmap) collections of related objects with similar
lifetimes in a single call
- proactive eviction of object data from the page cache which takes into account
these object relationships and lifetimes
- concurrent access to object data directly and memory mapped to eliminate
page cache pollution from background operations
- a management model that is intentionally patterned after LVM so as to feel
familiar to Linux users

The HSE storage engine, which is built on mpool, consistently demonstrates
throughputs and latencies in real-world applications that are multiples better
than common alternatives.  We believe this represents a concrete example of
the benefits of the mpool object store.

That said, we are very open to ideas on how we can improve the mpool
implementation to be better aligned with existing Linux I/O mechanisms.

Thanks,
Nabeel

From dan.j.williams at intel.com  Wed Oct 21 13:48:24 2020
From: dan.j.williams at intel.com (Dan Williams)
Date: Wed, 21 Oct 2020 10:48:24 -0700
Subject: [EXT] Re: [PATCH v2 00/22] add Object Storage Media Pool (mpool)
In-Reply-To: <SN6PR08MB4208A22F5C94AB05439A2703B31C0@SN6PR08MB4208.namprd08.prod.outlook.com>
References: <20201012162736.65241-1-nmeeramohide@micron.com>
 <20201015080254.GA31136@infradead.org>
 <SN6PR08MB420880574E0705BBC80EC1A3B3030@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4j7a0gq++rL--2W33fL4+S0asYjYkvfBfs+hY+3J=c_GA@mail.gmail.com>
 <SN6PR08MB420843C280D54D7B5B76EB78B31E0@SN6PR08MB4208.namprd08.prod.outlook.com>
 <CAPcyv4iYOk3i0pPgXxDTy47BxWCXqqXS0J6mrY5o+1M_41XoAw@mail.gmail.com>
 <SN6PR08MB4208A22F5C94AB05439A2703B31C0@SN6PR08MB4208.namprd08.prod.outlook.com>
Message-ID: <CAPcyv4gOGi392Q3knF=cAuvKONnmp2AoKX82VQEQJU0c7o7AKA@mail.gmail.com>

On Wed, Oct 21, 2020 at 10:11 AM Nabeel Meeramohideen Mohamed
(nmeeramohide) <nmeeramohide at micron.com> wrote:
>
> On Tuesday, October 20, 2020 3:36 PM, Dan Williams <dan.j.williams at intel.com> wrote:
> >
> >     What does Linux get from merging mpool?
> >
>
> What Linux gets from merging mpool is a generic object store target with some
> unique and beneficial features:

I'll try to make the point a different way. Mpool points out places
where the existing apis fail to scale. Rather than attempt to fix that
problem it proposes to replace the old apis. However, the old apis are
still there. So now upstream has 2 maintenance burdens when it could
have just had one. So when I ask "what does Linux get" it is in
reference to the fact that Linux gets a compounded maintenance problem
and whether the benefits of mpool outweigh that burden. Historically
Linux has been able to evolve to meet the scaling requirements of new
applications, so I am asking whether you have tried to solve the
application problem by evolving rather than replacing existing
infrastructure? The justification to replace rather than evolve is
high because that's how core Linux stays relevant.


From chaitanya.kulkarni at wdc.com  Wed Oct 21 19:28:59 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 16:28:59 -0700
Subject: [PATCH] nvmet: fix the Oops in the tracing for flush cmd
Message-ID: <20201021232859.4915-1-chaitanya.kulkarni@wdc.com>

When target side trace in turned on and flush command is issued from the
host it results in the following Oops.

[  856.789724] BUG: kernel NULL pointer dereference, address: 0000000000000068
[  856.790686] #PF: supervisor read access in kernel mode
[  856.791262] #PF: error_code(0x0000) - not-present page
[  856.791863] PGD 6d7110067 P4D 6d7110067 PUD 66f0ad067 PMD 0
[  856.792527] Oops: 0000 [#1] SMP NOPTI
[  856.792950] CPU: 15 PID: 7034 Comm: nvme Tainted: G           OE     5.9.0nvme-5.9+ #71
[  856.793790] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-59-gc9ba5276e3214
[  856.794956] RIP: 0010:trace_event_raw_event_nvmet_req_init+0x13e/0x170 [nvmet]
[  856.795734] Code: 41 5c 41 5d c3 31 d2 31 f6 e8 4e 9b b8 e0 e9 0e ff ff ff 49 8b 55 00 48 8b 38 8b 0
[  856.797740] RSP: 0018:ffffc90001be3a60 EFLAGS: 00010246
[  856.798375] RAX: 0000000000000000 RBX: ffff8887e7d2c01c RCX: 0000000000000000
[  856.799234] RDX: 0000000000000020 RSI: 0000000057e70ea2 RDI: ffff8887e7d2c034
[  856.800088] RBP: ffff88869f710578 R08: ffff888807500d40 R09: 00000000fffffffe
[  856.800951] R10: 0000000064c66670 R11: 00000000ef955201 R12: ffff8887e7d2c034
[  856.801807] R13: ffff88869f7105c8 R14: 0000000000000040 R15: ffff88869f710440
[  856.802667] FS:  00007f6a22bd8780(0000) GS:ffff888813a00000(0000) knlGS:0000000000000000
[  856.803635] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  856.804367] CR2: 0000000000000068 CR3: 00000006d73e0000 CR4: 00000000003506e0
[  856.805283] Call Trace:
[  856.805613]  nvmet_req_init+0x27c/0x480 [nvmet]
[  856.806200]  nvme_loop_queue_rq+0xcb/0x1d0 [nvme_loop]
[  856.806862]  blk_mq_dispatch_rq_list+0x123/0x7b0
[  856.807459]  ? kvm_sched_clock_read+0x14/0x30
[  856.808025]  __blk_mq_sched_dispatch_requests+0xc7/0x170
[  856.808708]  blk_mq_sched_dispatch_requests+0x30/0x60
[  856.809372]  __blk_mq_run_hw_queue+0x70/0x100
[  856.809935]  __blk_mq_delay_run_hw_queue+0x156/0x170
[  856.810574]  blk_mq_run_hw_queue+0x86/0xe0
[  856.811104]  blk_mq_sched_insert_request+0xef/0x160
[  856.811733]  blk_execute_rq+0x69/0xc0
[  856.812212]  ? blk_mq_rq_ctx_init+0xd0/0x230
[  856.812784]  nvme_execute_passthru_rq+0x57/0x130 [nvme_core]
[  856.813461]  nvme_submit_user_cmd+0xeb/0x300 [nvme_core]
[  856.814099]  nvme_user_cmd.isra.82+0x11e/0x1a0 [nvme_core]
[  856.814752]  blkdev_ioctl+0x1dc/0x2c0
[  856.815197]  block_ioctl+0x3f/0x50
[  856.815606]  __x64_sys_ioctl+0x84/0xc0
[  856.816074]  do_syscall_64+0x33/0x40
[  856.816533]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  856.817168] RIP: 0033:0x7f6a222ed107
[  856.817617] Code: 44 00 00 48 8b 05 81 cd 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 8
[  856.819901] RSP: 002b:00007ffca848f058 EFLAGS: 00000202 ORIG_RAX: 0000000000000010
[  856.820846] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f6a222ed107
[  856.821726] RDX: 00007ffca848f060 RSI: 00000000c0484e43 RDI: 0000000000000003
[  856.822603] RBP: 0000000000000003 R08: 000000000000003f R09: 0000000000000005
[  856.823478] R10: 00007ffca848ece0 R11: 0000000000000202 R12: 00007ffca84912d3
[  856.824359] R13: 00007ffca848f4d0 R14: 0000000000000002 R15: 000000000067e900
[  856.825236] Modules linked in: nvme_loop(OE) nvmet(OE) nvme_fabrics(OE) null_blk nvme(OE) nvme_corel

Fix this problem by adding a check for NULL namesapce in
drivers/nvme/target/trace.h:__assign_disk_name().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/trace.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/trace.h b/drivers/nvme/target/trace.h
index 0458046d6501..efb79cf898ac 100644
--- a/drivers/nvme/target/trace.h
+++ b/drivers/nvme/target/trace.h
@@ -54,7 +54,8 @@ static inline void __assign_disk_name(char *name, struct nvmet_req *req,
 
 	if ((init && req->sq->qid) || (!init && req->cq->qid)) {
 		ns = nvmet_find_namespace(ctrl, req->cmd->rw.nsid);
-		strncpy(name, ns->device_path, DISK_NAME_LEN);
+		if (ns)
+			strncpy(name, ns->device_path, DISK_NAME_LEN);
 		return;
 	}
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Oct 21 21:02:28 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 18:02:28 -0700
Subject: [PATCH V3 0/6] nvmet: passthru fixes and improvements
Message-ID: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>

Hi,

This patch series has several small trivial fixes and few code
optimizations.

Regards,
Chaitanya

Changes from V2:-

1. Remove "nvme-core: annotate nvme_alloc_request()" patch and
   split the nvme_alloc_request() into nvme_alloc_request_qid_any()
   and nvme_alloc_request_qid() with addition of the prep patch for
   the same.
2. Remove the cleanup patches and trim down the series.
3. Remove the code for setting up the op_flags for passthru.  
4. Rebase and retest on the nvme-5.10.

Changes from V1:-

1. Remove the sg_cnt check and nvmet_passthru_sg_map() annotation.
2. Add annotations and performance numbers for newly added patch #1.
3. Move ctrl refcount and module refcount into nvme_dev_open() and
   nvme_dev_release(). Add prepration patch for the same.
4. Add reviewed-by tags.

Chaitanya Kulkarni (6):
  nvme-core: add a helper to init req from nvme cmd
  nvme-core: split nvme_alloc_request()
  nvmet: remove op_flags for passthru commands
  block: move blk_rq_bio_prep() to linux/blk-mq.h
  nvmet: use minimized version of blk_rq_append_bio
  nvmet: use inline bio for passthru fast path

 block/blk.h                    | 12 --------
 drivers/nvme/host/core.c       | 56 +++++++++++++++++++++++-----------
 drivers/nvme/host/lightnvm.c   |  5 ++-
 drivers/nvme/host/nvme.h       |  4 +--
 drivers/nvme/host/pci.c        |  6 ++--
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c | 38 +++++++++++++----------
 include/linux/blk-mq.h         | 12 ++++++++
 8 files changed, 81 insertions(+), 53 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Oct 21 21:02:29 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 18:02:29 -0700
Subject: [PATCH V3 1/6] nvme-core: add a helper to init req from nvme cmd
In-Reply-To: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022010234.8304-2-chaitanya.kulkarni@wdc.com>

This is a preparation patch which adds a helper that we use in the next
patch that splits the nvme_alloc_request() into
nvme_alloc_request_qid_any() and nvme_alloc_request_qid(). The new
functions shares the code to initialize the allocated request from NVMe
cmd.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 56e2a22e8a02..5bc52594fe63 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -514,6 +514,14 @@ static inline void nvme_clear_nvme_request(struct request *req)
 	}
 }
 
+static inline void nvme_init_req_from_cmd(struct request *req,
+		struct nvme_command *cmd)
+{
+	req->cmd_flags |= REQ_FAILFAST_DRIVER;
+	nvme_clear_nvme_request(req);
+	nvme_req(req)->cmd = cmd;
+}
+
 struct request *nvme_alloc_request(struct request_queue *q,
 		struct nvme_command *cmd, blk_mq_req_flags_t flags, int qid)
 {
@@ -529,9 +537,7 @@ struct request *nvme_alloc_request(struct request_queue *q,
 	if (IS_ERR(req))
 		return req;
 
-	req->cmd_flags |= REQ_FAILFAST_DRIVER;
-	nvme_clear_nvme_request(req);
-	nvme_req(req)->cmd = cmd;
+	nvme_init_req_from_cmd(req, cmd);
 
 	return req;
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Oct 21 21:02:30 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 18:02:30 -0700
Subject: [PATCH V3 2/6] nvme-core: split nvme_alloc_request()
In-Reply-To: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022010234.8304-3-chaitanya.kulkarni@wdc.com>

Right now nvme_alloc_request() allocates a request from block layer
based on the value of the qid. When qid set to NVME_QID_ANY it used
blk_mq_alloc_request() else blk_mq_alloc_request_hctx().

The function nvme_alloc_request() is called from different context, The
only place where it uses non NVME_QID_ANY value is for fabrics connect
commands :-

nvme_submit_sync_cmd()		NVME_QID_ANY
nvme_features()			NVME_QID_ANY
nvme_sec_submit()		NVME_QID_ANY
nvmf_reg_read32()		NVME_QID_ANY
nvmf_reg_read64()		NVME_QID_ANY
nvmf_reg_write32()		NVME_QID_ANY
nvmf_connect_admin_queue()	NVME_QID_ANY
nvme_submit_user_cmd()		NVME_QID_ANY
	nvme_alloc_request()
nvme_keep_alive()		NVME_QID_ANY
	nvme_alloc_request()
nvme_timeout()			NVME_QID_ANY
	nvme_alloc_request()
nvme_delete_queue()		NVME_QID_ANY
	nvme_alloc_request()
nvmet_passthru_execute_cmd()	NVME_QID_ANY
	nvme_alloc_request()
nvmf_connect_io_queue() 	QID
	__nvme_submit_sync_cmd()
		nvme_alloc_request()

With passthru nvme_alloc_request() now falls into the I/O fast path such
that blk_mq_alloc_request_hctx() is never gets called and that adds
additional branch check and the code in fast path.

Split the nvme_alloc_request() into nvme_alloc_request_qid_any() and
nvme_alloc_request_qid().

Replace each call of the nvme_alloc_request() with NVME_QID_ANY param
with a call to newly added nvme_alloc_request_qid_any().

Replace a call to nvme_alloc_request() with QID param with a call to
newly added nvme_alloc_request_qid_any() and nvme_alloc_request_qid()
based on the qid value set in the __nvme_submit_sync_cmd().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c       | 44 +++++++++++++++++++++++-----------
 drivers/nvme/host/lightnvm.c   |  5 ++--
 drivers/nvme/host/nvme.h       |  4 ++--
 drivers/nvme/host/pci.c        |  6 ++---
 drivers/nvme/target/passthru.c |  2 +-
 5 files changed, 38 insertions(+), 23 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 5bc52594fe63..87e56ef48f5d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -522,26 +522,38 @@ static inline void nvme_init_req_from_cmd(struct request *req,
 	nvme_req(req)->cmd = cmd;
 }
 
-struct request *nvme_alloc_request(struct request_queue *q,
+static inline unsigned int nvme_req_op(struct nvme_command *cmd)
+{
+	return nvme_is_write(cmd) ? REQ_OP_DRV_OUT : REQ_OP_DRV_IN;
+}
+
+struct request *nvme_alloc_request_qid_any(struct request_queue *q,
+		struct nvme_command *cmd, blk_mq_req_flags_t flags)
+{
+	struct request *req;
+
+	req = blk_mq_alloc_request(q, nvme_req_op(cmd), flags);
+	if (unlikely(IS_ERR(req)))
+		return req;
+
+	nvme_init_req_from_cmd(req, cmd);
+	return req;
+}
+EXPORT_SYMBOL_GPL(nvme_alloc_request_qid_any);
+
+static struct request *nvme_alloc_request_qid(struct request_queue *q,
 		struct nvme_command *cmd, blk_mq_req_flags_t flags, int qid)
 {
-	unsigned op = nvme_is_write(cmd) ? REQ_OP_DRV_OUT : REQ_OP_DRV_IN;
 	struct request *req;
 
-	if (qid == NVME_QID_ANY) {
-		req = blk_mq_alloc_request(q, op, flags);
-	} else {
-		req = blk_mq_alloc_request_hctx(q, op, flags,
-				qid ? qid - 1 : 0);
-	}
+	req = blk_mq_alloc_request_hctx(q, nvme_req_op(cmd), flags,
+			qid ? qid - 1 : 0);
 	if (IS_ERR(req))
 		return req;
 
 	nvme_init_req_from_cmd(req, cmd);
-
 	return req;
 }
-EXPORT_SYMBOL_GPL(nvme_alloc_request);
 
 static int nvme_toggle_streams(struct nvme_ctrl *ctrl, bool enable)
 {
@@ -899,7 +911,11 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 	struct request *req;
 	int ret;
 
-	req = nvme_alloc_request(q, cmd, flags, qid);
+	if (qid == NVME_QID_ANY)
+		req = nvme_alloc_request_qid_any(q, cmd, flags);
+	else
+		req = nvme_alloc_request_qid(q, cmd, flags, qid);
+
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
@@ -1069,7 +1085,7 @@ static int nvme_submit_user_cmd(struct request_queue *q,
 	void *meta = NULL;
 	int ret;
 
-	req = nvme_alloc_request(q, cmd, 0, NVME_QID_ANY);
+	req = nvme_alloc_request_qid_any(q, cmd, 0);
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
@@ -1143,8 +1159,8 @@ static int nvme_keep_alive(struct nvme_ctrl *ctrl)
 {
 	struct request *rq;
 
-	rq = nvme_alloc_request(ctrl->admin_q, &ctrl->ka_cmd, BLK_MQ_REQ_RESERVED,
-			NVME_QID_ANY);
+	rq = nvme_alloc_request_qid_any(ctrl->admin_q, &ctrl->ka_cmd,
+			BLK_MQ_REQ_RESERVED);
 	if (IS_ERR(rq))
 		return PTR_ERR(rq);
 
diff --git a/drivers/nvme/host/lightnvm.c b/drivers/nvme/host/lightnvm.c
index 8e562d0f2c30..b1ee1a0310f6 100644
--- a/drivers/nvme/host/lightnvm.c
+++ b/drivers/nvme/host/lightnvm.c
@@ -653,7 +653,7 @@ static struct request *nvme_nvm_alloc_request(struct request_queue *q,
 
 	nvme_nvm_rqtocmd(rqd, ns, cmd);
 
-	rq = nvme_alloc_request(q, (struct nvme_command *)cmd, 0, NVME_QID_ANY);
+	rq = nvme_alloc_request_qid_any(q, (struct nvme_command *)cmd, 0);
 	if (IS_ERR(rq))
 		return rq;
 
@@ -767,8 +767,7 @@ static int nvme_nvm_submit_user_cmd(struct request_queue *q,
 	DECLARE_COMPLETION_ONSTACK(wait);
 	int ret = 0;
 
-	rq = nvme_alloc_request(q, (struct nvme_command *)vcmd, 0,
-			NVME_QID_ANY);
+	rq = nvme_alloc_request_qid_any(q, (struct nvme_command *)vcmd, 0);
 	if (IS_ERR(rq)) {
 		ret = -ENOMEM;
 		goto err_cmd;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index cc111136a981..f39a0a387a51 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -608,8 +608,8 @@ int nvme_wait_freeze_timeout(struct nvme_ctrl *ctrl, long timeout);
 void nvme_start_freeze(struct nvme_ctrl *ctrl);
 
 #define NVME_QID_ANY -1
-struct request *nvme_alloc_request(struct request_queue *q,
-		struct nvme_command *cmd, blk_mq_req_flags_t flags, int qid);
+struct request *nvme_alloc_request_qid_any(struct request_queue *q,
+		struct nvme_command *cmd, blk_mq_req_flags_t flags);
 void nvme_cleanup_cmd(struct request *req);
 blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req,
 		struct nvme_command *cmd);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index df8f3612107f..94f329b5f980 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1289,8 +1289,8 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved)
 		"I/O %d QID %d timeout, aborting\n",
 		 req->tag, nvmeq->qid);
 
-	abort_req = nvme_alloc_request(dev->ctrl.admin_q, &cmd,
-			BLK_MQ_REQ_NOWAIT, NVME_QID_ANY);
+	abort_req = nvme_alloc_request_qid_any(dev->ctrl.admin_q, &cmd,
+			BLK_MQ_REQ_NOWAIT);
 	if (IS_ERR(abort_req)) {
 		atomic_inc(&dev->ctrl.abort_limit);
 		return BLK_EH_RESET_TIMER;
@@ -2204,7 +2204,7 @@ static int nvme_delete_queue(struct nvme_queue *nvmeq, u8 opcode)
 	cmd.delete_queue.opcode = opcode;
 	cmd.delete_queue.qid = cpu_to_le16(nvmeq->qid);
 
-	req = nvme_alloc_request(q, &cmd, BLK_MQ_REQ_NOWAIT, NVME_QID_ANY);
+	req = nvme_alloc_request_qid_any(q, &cmd, BLK_MQ_REQ_NOWAIT);
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 56c571052216..76affbc3bd9a 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -236,7 +236,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		q = ns->queue;
 	}
 
-	rq = nvme_alloc_request(q, req->cmd, BLK_MQ_REQ_NOWAIT, NVME_QID_ANY);
+	rq = nvme_alloc_request_qid_any(q, req->cmd, BLK_MQ_REQ_NOWAIT);
 	if (IS_ERR(rq)) {
 		status = NVME_SC_INTERNAL;
 		goto out_put_ns;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Oct 21 21:02:31 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 18:02:31 -0700
Subject: [PATCH V3 3/6] nvmet: remove op_flags for passthru commands
In-Reply-To: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022010234.8304-4-chaitanya.kulkarni@wdc.com>

For passthru commands setting op_flags has no meaning. Remove the code
that sets the op flags in nvmet_passthru_map_sg().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 76affbc3bd9a..e5b5657bcce2 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -182,18 +182,12 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 {
 	int sg_cnt = req->sg_cnt;
 	struct scatterlist *sg;
-	int op_flags = 0;
 	struct bio *bio;
 	int i, ret;
 
-	if (req->cmd->common.opcode == nvme_cmd_flush)
-		op_flags = REQ_FUA;
-	else if (nvme_is_write(req->cmd))
-		op_flags = REQ_SYNC | REQ_IDLE;
-
 	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
 	bio->bi_end_io = bio_put;
-	bio->bi_opf = req_op(rq) | op_flags;
+	bio->bi_opf = req_op(rq);
 
 	for_each_sg(req->sg, sg, req->sg_cnt, i) {
 		if (bio_add_pc_page(rq->q, bio, sg_page(sg), sg->length,
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Oct 21 21:02:32 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 18:02:32 -0700
Subject: [PATCH V3 4/6] block: move blk_rq_bio_prep() to linux/blk-mq.h
In-Reply-To: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022010234.8304-5-chaitanya.kulkarni@wdc.com>

This is a preparation patch to have minimal block layer request bio
append functionality in the context of the NVMeOF Passthru driver which
falls in the fast path and doesn't need calls from blk_rq_append_bio().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Reviewed-by: Logan Gunthorpe <logang at deltatee.com>
---
 block/blk.h            | 12 ------------
 include/linux/blk-mq.h | 12 ++++++++++++
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/block/blk.h b/block/blk.h
index dfab98465db9..e05507a8d1e3 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -91,18 +91,6 @@ static inline bool bvec_gap_to_prev(struct request_queue *q,
 	return __bvec_gap_to_prev(q, bprv, offset);
 }
 
-static inline void blk_rq_bio_prep(struct request *rq, struct bio *bio,
-		unsigned int nr_segs)
-{
-	rq->nr_phys_segments = nr_segs;
-	rq->__data_len = bio->bi_iter.bi_size;
-	rq->bio = rq->biotail = bio;
-	rq->ioprio = bio_prio(bio);
-
-	if (bio->bi_disk)
-		rq->rq_disk = bio->bi_disk;
-}
-
 #ifdef CONFIG_BLK_DEV_INTEGRITY
 void blk_flush_integrity(void);
 bool __bio_integrity_endio(struct bio *);
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index b23eeca4d677..d1d277073761 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -591,6 +591,18 @@ static inline void blk_mq_cleanup_rq(struct request *rq)
 		rq->q->mq_ops->cleanup_rq(rq);
 }
 
+static inline void blk_rq_bio_prep(struct request *rq, struct bio *bio,
+		unsigned int nr_segs)
+{
+	rq->nr_phys_segments = nr_segs;
+	rq->__data_len = bio->bi_iter.bi_size;
+	rq->bio = rq->biotail = bio;
+	rq->ioprio = bio_prio(bio);
+
+	if (bio->bi_disk)
+		rq->rq_disk = bio->bi_disk;
+}
+
 blk_qc_t blk_mq_submit_bio(struct bio *bio);
 
 #endif
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Oct 21 21:02:33 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 18:02:33 -0700
Subject: [PATCH V3 5/6] nvmet: use minimized version of blk_rq_append_bio
In-Reply-To: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022010234.8304-6-chaitanya.kulkarni@wdc.com>

The function blk_rq_append_bio() is a genereric API written for all
types driver (having bounce buffers) and different context (where
request is already having a bio i.e. rq->bio != NULL).

It does mainly three things: calculating the segments, bounce queue and
if req->bio == NULL call blk_rq_bio_prep() or handle low level merge()
case.

The NVMe PCIe and fabrics transports currently does not use queue
bounce mechanism. In order to find this for each request processing
in the passthru blk_rq_append_bio() does extra work in the fast path
for each request.

When I ran I/Os with different block sizes on the passthru controller
I found that we can reuse the req->sg_cnt instead of iterating over the
bvecs to find out nr_segs in blk_rq_append_bio(). This calculation in
blk_rq_append_bio() is a duplication of work given that we have the
value in req->sg_cnt. (correct me here if I'm wrong).

With NVMe passthru request based driver we allocate fresh request each
time, so every call to blk_rq_append_bio() rq->bio will be NULL i.e.
we don't really need the second condition in the blk_rq_append_bio()
and the resulting error condition in the caller of blk_rq_append_bio().

So for NVMeOF passthru driver recalculating the segments, bounce check
and ll_back_merge code is not needed such that we can get away with the
minimal version of the blk_rq_append_bio() which removes the error check
in the fast path along with extra variable in nvmet_passthru_map_sg().

This patch updates the nvmet_passthru_map_sg() such that it does only
appending the bio to the request in the context of the NVMeOF Passthru
driver. Following are perf numbers :-

With current implementation (blk_rq_append_bio()) :-
----------------------------------------------------
+    5.80%     0.02%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd
+    5.44%     0.01%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd
+    4.88%     0.00%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd
+    5.44%     0.01%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd
+    4.86%     0.01%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd
+    5.17%     0.00%  kworker/0:2-eve  [nvmet]  [k] nvmet_passthru_execute_cmd

With this patch using blk_rq_bio_prep() :-
----------------------------------------------------
+    3.14%     0.02%  kworker/0:2-eve  [nvmet]  [k] nvmet_passthru_execute_cmd
+    3.26%     0.01%  kworker/0:2-eve  [nvmet]  [k] nvmet_passthru_execute_cmd
+    5.37%     0.01%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd
+    5.18%     0.02%  kworker/0:2-eve  [nvmet]  [k] nvmet_passthru_execute_cmd
+    4.84%     0.02%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd
+    4.87%     0.01%  kworker/0:2-mm_  [nvmet]  [k] nvmet_passthru_execute_cmd

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Reviewed-by: Logan Gunthorpe <logang at deltatee.com>
---
 drivers/nvme/target/passthru.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index e5b5657bcce2..496ffedb77dc 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -183,7 +183,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 	int sg_cnt = req->sg_cnt;
 	struct scatterlist *sg;
 	struct bio *bio;
-	int i, ret;
+	int i;
 
 	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
 	bio->bi_end_io = bio_put;
@@ -198,11 +198,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 		sg_cnt--;
 	}
 
-	ret = blk_rq_append_bio(rq, &bio);
-	if (unlikely(ret)) {
-		bio_put(bio);
-		return ret;
-	}
+	blk_rq_bio_prep(rq, bio, req->sg_cnt);
 
 	return 0;
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Oct 21 21:02:34 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 21 Oct 2020 18:02:34 -0700
Subject: [PATCH V3 6/6] nvmet: use inline bio for passthru fast path
In-Reply-To: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022010234.8304-7-chaitanya.kulkarni@wdc.com>

In nvmet_passthru_execute_cmd() which is a high frequency function
it uses bio_alloc() which leads to memory allocation from the fs pool
for each I/O.

For NVMeoF nvmet_req we already have inline_bvec allocated as a part of
request allocation that can be used with preallocated bio when we
already know the size of request before bio allocation with bio_alloc(),
which we already do.

Introduce a bio member for the nvmet_req passthru anon union. In the
fast path, check if we can get away with inline bvec and bio from
nvmet_req with bio_init() call before actually allocating from the
bio_alloc().

This will be useful to avoid any new memory allocation under high
memory pressure situation and get rid of any extra work of
allocation (bio_alloc()) vs initialization (bio_init()) when
transfer len is < NVMET_MAX_INLINE_DATA_LEN that user can configure at
compile time.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c | 20 ++++++++++++++++++--
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 559a15ccc322..408a13084fb4 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -330,6 +330,7 @@ struct nvmet_req {
 			struct work_struct      work;
 		} f;
 		struct {
+			struct bio		inline_bio;
 			struct request		*rq;
 			struct work_struct      work;
 			bool			use_workqueue;
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 496ffedb77dc..32498b4302cc 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -178,6 +178,14 @@ static void nvmet_passthru_req_done(struct request *rq,
 	blk_mq_free_request(rq);
 }
 
+static void nvmet_passthru_bio_done(struct bio *bio)
+{
+	struct nvmet_req *req = bio->bi_private;
+
+	if (bio != &req->p.inline_bio)
+		bio_put(bio);
+}
+
 static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 {
 	int sg_cnt = req->sg_cnt;
@@ -186,13 +194,21 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 	int i;
 
 	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
-	bio->bi_end_io = bio_put;
+	if (req->transfer_len <= NVMET_MAX_INLINE_DATA_LEN) {
+		bio = &req->p.inline_bio;
+		bio_init(bio, req->inline_bvec, ARRAY_SIZE(req->inline_bvec));
+	} else {
+		bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
+	}
+
+	bio->bi_end_io = nvmet_passthru_bio_done;
 	bio->bi_opf = req_op(rq);
+	bio->bi_private = req;
 
 	for_each_sg(req->sg, sg, req->sg_cnt, i) {
 		if (bio_add_pc_page(rq->q, bio, sg_page(sg), sg->length,
 				    sg->offset) < sg->length) {
-			bio_put(bio);
+			nvmet_passthru_bio_done(bio);
 			return -EINVAL;
 		}
 		sg_cnt--;
-- 
2.22.1



From lengchao at huawei.com  Wed Oct 21 21:58:25 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 09:58:25 +0800
Subject: [PATCH 0/3] avoid race between time out and tear down
In-Reply-To: <db872ac4-662c-1e47-291a-7c64ae0cfa57@grimberg.me>
References: <20201020090808.27702-1-lengchao@huawei.com>
 <9e1966b0-d471-0043-7fdb-e0f535571feb@grimberg.me>
 <3d5b96dd-4e0f-758a-4175-e69fef0b8c34@huawei.com>
 <b60f34ea-86f1-ef86-8e1f-b156b7681699@grimberg.me>
 <5a48d20f-e9c9-5d37-776a-1fe3b1eb79a8@huawei.com>
 <db872ac4-662c-1e47-291a-7c64ae0cfa57@grimberg.me>
Message-ID: <1903f0fa-7dd3-6e87-0e4e-502628822e89@huawei.com>



On 2020/10/21 16:53, Sagi Grimberg wrote:
> 
>>>>>> Avoid race between time out and tear down for rdma and tcp.
>>>>>
>>>>> This patchset overall looks good, but we still need the patch that
>>>>> avoids double completion:
>>>>>
>>>>> -- 
>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>> index 629b025685d1..46428ff0b0fc 100644
>>>>> --- a/drivers/nvme/host/tcp.c
>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>> @@ -2175,7 +2175,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>>>>> ???????? /* fence other contexts that may complete the command */
>>>>> ???????? mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
>>>>> ???????? nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>>>>> -?????? if (!blk_mq_request_completed(rq)) {
>>>>> +?????? if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>>>> Yes, this patch is need. and samely for nvme_cancel_request.
>>>> This will fix the race with asynchronous completion.
>>>>
>>>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>>>> index e85f6304efd7..1e838d952096 100644
>>>> --- a/drivers/nvme/host/core.c
>>>> +++ b/drivers/nvme/host/core.c
>>>> @@ -338,7 +338,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
>>>> ???????????????????????????????? "Cancelling I/O %d", req->tag);
>>>>
>>>> ???????? /* don't abort one completed request */
>>>> -?????? if (blk_mq_request_completed(req))
>>>> +?????? if (blk_mq_request_completed(req) || !blk_mq_request_started(rq))
>>>> ???????????????? return true;
>>>>
>>>> ???????? nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
>>>
>>> This one is unneeded because blk_mq_tagset_busy_iter checks that the
>>> request has started...
>> Yes, it is already checked.
> 
> Can you add the attached patches and resend?
> 
> You can also add my:
> Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
ok, I will do it later.


From lengchao at huawei.com  Wed Oct 21 22:15:00 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 10:15:00 +0800
Subject: [PATCH v2 1/5] nvme-core: introduce sync io queues
Message-ID: <20201022021500.11022-1-lengchao@huawei.com>

Introduce sync io queues for some scenarios which just only need sync
io queues not sync all queues.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/core.c | 8 ++++++--
 drivers/nvme/host/nvme.h | 1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e85f6304efd7..2211cf155e89 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4573,8 +4573,7 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
 }
 EXPORT_SYMBOL_GPL(nvme_start_queues);
 
-
-void nvme_sync_queues(struct nvme_ctrl *ctrl)
+void nvme_sync_io_queues(struct nvme_ctrl *ctrl)
 {
 	struct nvme_ns *ns;
 
@@ -4582,7 +4581,12 @@ void nvme_sync_queues(struct nvme_ctrl *ctrl)
 	list_for_each_entry(ns, &ctrl->namespaces, list)
 		blk_sync_queue(ns->queue);
 	up_read(&ctrl->namespaces_rwsem);
+}
+EXPORT_SYMBOL_GPL(nvme_sync_io_queues);
 
+void nvme_sync_queues(struct nvme_ctrl *ctrl)
+{
+	nvme_sync_io_queues(ctrl);
 	if (ctrl->admin_q)
 		blk_sync_queue(ctrl->admin_q);
 }
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 566776100126..49d7f43511d4 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -602,6 +602,7 @@ void nvme_stop_queues(struct nvme_ctrl *ctrl);
 void nvme_start_queues(struct nvme_ctrl *ctrl);
 void nvme_kill_queues(struct nvme_ctrl *ctrl);
 void nvme_sync_queues(struct nvme_ctrl *ctrl);
+void nvme_sync_io_queues(struct nvme_ctrl *ctrl);
 void nvme_unfreeze(struct nvme_ctrl *ctrl);
 void nvme_wait_freeze(struct nvme_ctrl *ctrl);
 int nvme_wait_freeze_timeout(struct nvme_ctrl *ctrl, long timeout);
-- 
2.16.4



From lengchao at huawei.com  Wed Oct 21 22:14:52 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 10:14:52 +0800
Subject: [PATCH v2 0/5] avoid race for time out
Message-ID: <20201022021452.10971-1-lengchao@huawei.com>

First avoid race between time out and tear down for rdma and tcp.
Second avoid repeated request completion in time out for rdma and tcp.

V2:
	- add avoiding repeated request completion in time out

Chao Leng (3):
  nvme-core: introduce sync io queues
  nvme-rdma: avoid race between time out and tear down
  nvme-tcp: avoid race between time out and tear down

Sagi Grimberg (2):
  nvme-rdma: avoid repeated request completion
  nvme-tcp: avoid repeated request completion

 drivers/nvme/host/core.c |  8 ++++++--
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/rdma.c | 14 +++-----------
 drivers/nvme/host/tcp.c  | 16 ++++------------
 4 files changed, 14 insertions(+), 25 deletions(-)

-- 
2.16.4



From lengchao at huawei.com  Wed Oct 21 22:15:08 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 10:15:08 +0800
Subject: [PATCH v2 2/5] nvme-rdma: avoid race between time out and tear down
Message-ID: <20201022021508.11091-1-lengchao@huawei.com>

Now use teardown_lock to serialize for time out and tear down. This may
cause abnormal: first cancel all request in tear down, then time out may
complete the request again, but the request may already be freed or
restarted.

To avoid race between time out and tear down, in tear down process,
first we quiesce the queue, and then delete the timer and cancel
the time out work for the queue. At the same time we need to delete
teardown_lock.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/rdma.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..fde8ad07326e 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -122,7 +122,6 @@ struct nvme_rdma_ctrl {
 	struct sockaddr_storage src_addr;
 
 	struct nvme_ctrl	ctrl;
-	struct mutex		teardown_lock;
 	bool			use_inline_data;
 	u32			io_queues[HCTX_MAX_TYPES];
 };
@@ -1010,8 +1009,8 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
 static void nvme_rdma_teardown_admin_queue(struct nvme_rdma_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&ctrl->teardown_lock);
 	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
 	if (ctrl->ctrl.admin_tagset) {
 		blk_mq_tagset_busy_iter(ctrl->ctrl.admin_tagset,
@@ -1021,16 +1020,15 @@ static void nvme_rdma_teardown_admin_queue(struct nvme_rdma_ctrl *ctrl,
 	if (remove)
 		blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 	nvme_rdma_destroy_admin_queue(ctrl, remove);
-	mutex_unlock(&ctrl->teardown_lock);
 }
 
 static void nvme_rdma_teardown_io_queues(struct nvme_rdma_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&ctrl->teardown_lock);
 	if (ctrl->ctrl.queue_count > 1) {
 		nvme_start_freeze(&ctrl->ctrl);
 		nvme_stop_queues(&ctrl->ctrl);
+		nvme_sync_io_queues(&ctrl->ctrl);
 		nvme_rdma_stop_io_queues(ctrl);
 		if (ctrl->ctrl.tagset) {
 			blk_mq_tagset_busy_iter(ctrl->ctrl.tagset,
@@ -1041,7 +1039,6 @@ static void nvme_rdma_teardown_io_queues(struct nvme_rdma_ctrl *ctrl,
 			nvme_start_queues(&ctrl->ctrl);
 		nvme_rdma_destroy_io_queues(ctrl, remove);
 	}
-	mutex_unlock(&ctrl->teardown_lock);
 }
 
 static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
@@ -1968,16 +1965,12 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
 {
 	struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq);
 	struct nvme_rdma_queue *queue = req->queue;
-	struct nvme_rdma_ctrl *ctrl = queue->ctrl;
 
-	/* fence other contexts that may complete the command */
-	mutex_lock(&ctrl->teardown_lock);
 	nvme_rdma_stop_queue(queue);
 	if (!blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
-	mutex_unlock(&ctrl->teardown_lock);
 }
 
 static enum blk_eh_timer_return
@@ -2312,7 +2305,6 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
 		return ERR_PTR(-ENOMEM);
 	ctrl->ctrl.opts = opts;
 	INIT_LIST_HEAD(&ctrl->list);
-	mutex_init(&ctrl->teardown_lock);
 
 	if (!(opts->mask & NVMF_OPT_TRSVCID)) {
 		opts->trsvcid =
-- 
2.16.4



From lengchao at huawei.com  Wed Oct 21 22:15:15 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 10:15:15 +0800
Subject: [PATCH v2 3/5] nvme-tcp: avoid race between time out and tear down
Message-ID: <20201022021515.11143-1-lengchao@huawei.com>

Now use teardown_lock to serialize for time out and tear down. This may
cause abnormal: first cancel all request in tear down, then time out may
complete the request again, but the request may already be freed or
restarted.

To avoid race between time out and tear down, in tear down process,
first we quiesce the queue, and then delete the timer and cancel
the time out work for the queue. At the same time we need to delete
teardown_lock.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/tcp.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8f4f29f18b8c..0b229a2f59d5 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -124,7 +124,6 @@ struct nvme_tcp_ctrl {
 	struct sockaddr_storage src_addr;
 	struct nvme_ctrl	ctrl;
 
-	struct mutex		teardown_lock;
 	struct work_struct	err_work;
 	struct delayed_work	connect_work;
 	struct nvme_tcp_request async_req;
@@ -1887,8 +1886,8 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
 	if (ctrl->admin_tagset) {
 		blk_mq_tagset_busy_iter(ctrl->admin_tagset,
@@ -1898,18 +1897,17 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl,
 	if (remove)
 		blk_mq_unquiesce_queue(ctrl->admin_q);
 	nvme_tcp_destroy_admin_queue(ctrl, remove);
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 		bool remove)
 {
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	if (ctrl->queue_count <= 1)
-		goto out;
+		return;
 	blk_mq_quiesce_queue(ctrl->admin_q);
 	nvme_start_freeze(ctrl);
 	nvme_stop_queues(ctrl);
+	nvme_sync_io_queues(ctrl);
 	nvme_tcp_stop_io_queues(ctrl);
 	if (ctrl->tagset) {
 		blk_mq_tagset_busy_iter(ctrl->tagset,
@@ -1919,8 +1917,6 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,
 	if (remove)
 		nvme_start_queues(ctrl);
 	nvme_tcp_destroy_io_queues(ctrl, remove);
-out:
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)
@@ -2172,14 +2168,11 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
 	struct nvme_ctrl *ctrl = &req->queue->ctrl->ctrl;
 
-	/* fence other contexts that may complete the command */
-	mutex_lock(&to_tcp_ctrl(ctrl)->teardown_lock);
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
 	if (!blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
-	mutex_unlock(&to_tcp_ctrl(ctrl)->teardown_lock);
 }
 
 static enum blk_eh_timer_return
@@ -2456,7 +2449,6 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
 			nvme_tcp_reconnect_ctrl_work);
 	INIT_WORK(&ctrl->err_work, nvme_tcp_error_recovery_work);
 	INIT_WORK(&ctrl->ctrl.reset_work, nvme_reset_ctrl_work);
-	mutex_init(&ctrl->teardown_lock);
 
 	if (!(opts->mask & NVMF_OPT_TRSVCID)) {
 		opts->trsvcid =
-- 
2.16.4



From lengchao at huawei.com  Wed Oct 21 22:15:23 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 10:15:23 +0800
Subject: [PATCH v2 4/5] nvme-rdma: avoid repeated request completion
Message-ID: <20201022021523.11214-1-lengchao@huawei.com>

From: Sagi Grimberg <sagi at grimberg.me>

The request may be executed asynchronously, and rq->state may be
changed to IDLE. To avoid repeated request completion, only
MQ_RQ_COMPLETE of rq->state is checked in nvme_rdma_complete_timed_out.
It is not safe, so need adding check IDLE for rq->state.

Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/rdma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index fde8ad07326e..3f5ebf4c8ac6 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1967,7 +1967,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
 	struct nvme_rdma_queue *queue = req->queue;
 
 	nvme_rdma_stop_queue(queue);
-	if (!blk_mq_request_completed(rq)) {
+	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
-- 
2.16.4



From lengchao at huawei.com  Wed Oct 21 22:15:31 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 10:15:31 +0800
Subject: [PATCH v2 5/5] nvme-tcp: avoid repeated request completion
Message-ID: <20201022021531.11265-1-lengchao@huawei.com>

From: Sagi Grimberg <sagi at grimberg.me>

The request may be executed asynchronously, and rq->state may be
changed to IDLE. To avoid repeated request completion, only
MQ_RQ_COMPLETE of rq->state is checked in nvme_tcp_complete_timed_out.
It is not safe, so need adding check IDLE for rq->state.

Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/tcp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 0b229a2f59d5..830711e39d4d 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2169,7 +2169,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	struct nvme_ctrl *ctrl = &req->queue->ctrl->ctrl;
 
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
-	if (!blk_mq_request_completed(rq)) {
+	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
 		blk_mq_complete_request(rq);
 	}
-- 
2.16.4



From pizhenwei at bytedance.com  Thu Oct 22 04:38:50 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Thu, 22 Oct 2020 16:38:50 +0800
Subject: [PATCH] nvme-rdma: handle nvme completion data length
Message-ID: <20201022083850.1334880-1-pizhenwei@bytedance.com>

Hit a kernel warning:
refcount_t: underflow; use-after-free.
WARNING: CPU: 0 PID: 0 at lib/refcount.c:28

RIP: 0010:refcount_warn_saturate+0xd9/0xe0
Call Trace:
 <IRQ>
 nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
 __ib_process_cq+0x76/0x150 [ib_core]
 ...

The reason is that a zero bytes message received from target, and the
host side continues to process without length checking, then the
previous CQE is processed twice.

Handle data length, ignore zero bytes message, and try to recovery for
corrupted CQE case.

Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/host/rdma.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..9f5112040d43 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1767,6 +1767,17 @@ static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc)
 		return;
 	}
 
+	if (unlikely(!wc->byte_len)) {
+		/* zero bytes message could be ignored */
+		return;
+	} else if (unlikely(wc->byte_len < len)) {
+		/* Corrupted completion, try to recovry */
+		dev_err(queue->ctrl->ctrl.device,
+			"Unexpected nvme completion length(%d)\n", wc->byte_len);
+		nvme_rdma_error_recovery(queue->ctrl);
+		return;
+	}
+
 	ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
 	/*
 	 * AEN requests are special as they don't time out and can
-- 
2.11.0



From lengchao at huawei.com  Thu Oct 22 05:55:01 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 22 Oct 2020 17:55:01 +0800
Subject: [PATCH] nvme-rdma: handle nvme completion data length
In-Reply-To: <20201022083850.1334880-1-pizhenwei@bytedance.com>
References: <20201022083850.1334880-1-pizhenwei@bytedance.com>
Message-ID: <04a97f73-ba13-a4b5-3ea4-fc438391507e@huawei.com>



On 2020/10/22 16:38, zhenwei pi wrote:
> Hit a kernel warning:
> refcount_t: underflow; use-after-free.
> WARNING: CPU: 0 PID: 0 at lib/refcount.c:28
> 
> RIP: 0010:refcount_warn_saturate+0xd9/0xe0
> Call Trace:
>   <IRQ>
>   nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
>   __ib_process_cq+0x76/0x150 [ib_core]
>   ...
> 
> The reason is that a zero bytes message received from target, and the
> host side continues to process without length checking, then the
> previous CQE is processed twice.
> 
> Handle data length, ignore zero bytes message, and try to recovery for
> corrupted CQE case.
> 
> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
> ---
>   drivers/nvme/host/rdma.c | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 9e378d0a0c01..9f5112040d43 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1767,6 +1767,17 @@ static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc)
>   		return;
>   	}
>   
> +	if (unlikely(!wc->byte_len)) {
> +		/* zero bytes message could be ignored */
> +		return;
> +	} else if (unlikely(wc->byte_len < len)) {
> +		/* Corrupted completion, try to recovry */
> +		dev_err(queue->ctrl->ctrl.device,
> +			"Unexpected nvme completion length(%d)\n", wc->byte_len);
> +		nvme_rdma_error_recovery(queue->ctrl);
> +		return;
> +	}
!wc->byte_len and wc->byte_len < len may be the same type of anomaly.
Why do different error handling?
In which scenario zero bytes message received from target? fault inject test or normal test/run?
> +
>   	ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
>   	/*
>   	 * AEN requests are special as they don't time out and can
> 


From pizhenwei at bytedance.com  Thu Oct 22 06:05:45 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Thu, 22 Oct 2020 18:05:45 +0800
Subject: [External] Re: [PATCH] nvme-rdma: handle nvme completion data
 length
In-Reply-To: <04a97f73-ba13-a4b5-3ea4-fc438391507e@huawei.com>
References: <20201022083850.1334880-1-pizhenwei@bytedance.com>
 <04a97f73-ba13-a4b5-3ea4-fc438391507e@huawei.com>
Message-ID: <1c78dbe5-47a4-1590-e064-681cba5fb01d@bytedance.com>

On 10/22/20 5:55 PM, Chao Leng wrote:
> 
> 
> On 2020/10/22 16:38, zhenwei pi wrote:
>> Hit a kernel warning:
>> refcount_t: underflow; use-after-free.
>> WARNING: CPU: 0 PID: 0 at lib/refcount.c:28
>>
>> RIP: 0010:refcount_warn_saturate+0xd9/0xe0
>> Call Trace:
>> ? <IRQ>
>> ? nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
>> ? __ib_process_cq+0x76/0x150 [ib_core]
>> ? ...
>>
>> The reason is that a zero bytes message received from target, and the
>> host side continues to process without length checking, then the
>> previous CQE is processed twice.
>>
>> Handle data length, ignore zero bytes message, and try to recovery for
>> corrupted CQE case.
>>
>> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
>> ---
>> ? drivers/nvme/host/rdma.c | 11 +++++++++++
>> ? 1 file changed, 11 insertions(+)
>>
>> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
>> index 9e378d0a0c01..9f5112040d43 100644
>> --- a/drivers/nvme/host/rdma.c
>> +++ b/drivers/nvme/host/rdma.c
>> @@ -1767,6 +1767,17 @@ static void nvme_rdma_recv_done(struct ib_cq 
>> *cq, struct ib_wc *wc)
>> ????????? return;
>> ????? }
>> +??? if (unlikely(!wc->byte_len)) {
>> +??????? /* zero bytes message could be ignored */
>> +??????? return;
>> +??? } else if (unlikely(wc->byte_len < len)) {
>> +??????? /* Corrupted completion, try to recovry */
>> +??????? dev_err(queue->ctrl->ctrl.device,
>> +??????????? "Unexpected nvme completion length(%d)\n", wc->byte_len);
>> +??????? nvme_rdma_error_recovery(queue->ctrl);
>> +??????? return;
>> +??? }
> !wc->byte_len and wc->byte_len < len may be the same type of anomaly.
> Why do different error handling?
> In which scenario zero bytes message received from target? fault inject 
> test or normal test/run?

Zero bytes message could be used as transport layer keep alive mechanism 
(I's also developing target side transport layer keep alive now. To 
reclaim resource, target side needs to close dead connections even kato 
is set as 0).

>> +
>> ????? ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
>> ????? /*
>> ?????? * AEN requests are special as they don't time out and can
>>

-- 
zhenwei pi


From hch at lst.de  Thu Oct 22 09:22:06 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 22 Oct 2020 15:22:06 +0200
Subject: [PATCH] nvmet: fix the Oops in the tracing for flush cmd
In-Reply-To: <20201021232859.4915-1-chaitanya.kulkarni@wdc.com>
References: <20201021232859.4915-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022132206.GA8781@lst.de>

I think the problem is that the extra namespace lookup doesn't make
sense to start with.  We'll need to reuse req->ns, and for that move
the init tracepoint a bit:

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index aafcbc424b7a94..6b610b68c44226 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -907,8 +907,6 @@ bool nvmet_req_init(struct nvmet_req *req, struct nvmet_cq *cq,
 	req->error_loc = NVMET_NO_ERROR_LOC;
 	req->error_slba = 0;
 
-	trace_nvmet_req_init(req, req->cmd);
-
 	/* no support for fused commands yet */
 	if (unlikely(flags & (NVME_CMD_FUSE_FIRST | NVME_CMD_FUSE_SECOND))) {
 		req->error_loc = offsetof(struct nvme_common_command, flags);
@@ -937,6 +935,8 @@ bool nvmet_req_init(struct nvmet_req *req, struct nvmet_cq *cq,
 
 	if (status)
 		goto fail;
+	
+	trace_nvmet_req_init(req, req->cmd);
 
 	if (unlikely(!percpu_ref_tryget_live(&sq->ref))) {
 		status = NVME_SC_INVALID_FIELD | NVME_SC_DNR;
diff --git a/drivers/nvme/target/trace.h b/drivers/nvme/target/trace.h
index 0458046d650178..c14e3249a14dc8 100644
--- a/drivers/nvme/target/trace.h
+++ b/drivers/nvme/target/trace.h
@@ -46,19 +46,12 @@ static inline struct nvmet_ctrl *nvmet_req_to_ctrl(struct nvmet_req *req)
 	return req->sq->ctrl;
 }
 
-static inline void __assign_disk_name(char *name, struct nvmet_req *req,
-		bool init)
+static inline void __assign_req_name(char *name, struct nvmet_req *req)
 {
-	struct nvmet_ctrl *ctrl = nvmet_req_to_ctrl(req);
-	struct nvmet_ns *ns;
-
-	if ((init && req->sq->qid) || (!init && req->cq->qid)) {
-		ns = nvmet_find_namespace(ctrl, req->cmd->rw.nsid);
-		strncpy(name, ns->device_path, DISK_NAME_LEN);
-		return;
-	}
-
-	memset(name, 0, DISK_NAME_LEN);
+	if (req->ns)
+		strncpy(name, req->ns->device_path, DISK_NAME_LEN);
+	else
+		memset(name, 0, DISK_NAME_LEN);
 }
 #endif
 
@@ -81,7 +74,7 @@ TRACE_EVENT(nvmet_req_init,
 	TP_fast_assign(
 		__entry->cmd = cmd;
 		__entry->ctrl = nvmet_req_to_ctrl(req);
-		__assign_disk_name(__entry->disk, req, true);
+		__assign_req_name(__entry->disk, req);
 		__entry->qid = req->sq->qid;
 		__entry->cid = cmd->common.command_id;
 		__entry->opcode = cmd->common.opcode;
@@ -121,7 +114,7 @@ TRACE_EVENT(nvmet_req_complete,
 		__entry->cid = req->cqe->command_id;
 		__entry->result = le64_to_cpu(req->cqe->result.u64);
 		__entry->status = le16_to_cpu(req->cqe->status) >> 1;
-		__assign_disk_name(__entry->disk, req, false);
+		__assign_req_name(__entry->disk, req);
 	),
 	TP_printk("nvmet%s: %sqid=%d, cmdid=%u, res=%#llx, status=%#x",
 		__print_ctrl_name(__entry->ctrl),


From hch at lst.de  Thu Oct 22 09:29:22 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 22 Oct 2020 15:29:22 +0200
Subject: [PATCH v2 1/2] nvmet-passthru: Limit mdts by BIO_MAX_PAGES
In-Reply-To: <20201016221905.4686-1-logang@deltatee.com>
References: <20201016221905.4686-1-logang@deltatee.com>
Message-ID: <20201022132922.GA8969@lst.de>

Thanks,

applied 1 and 2 to nvme-5.10.


From hch at infradead.org  Thu Oct 22 09:37:07 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 22 Oct 2020 14:37:07 +0100
Subject: [PATCH] nvme-fc: fix error loop in create_hw_io_queues
In-Reply-To: <20201016210604.48542-1-james.smart@broadcom.com>
References: <20201016210604.48542-1-james.smart@broadcom.com>
Message-ID: <20201022133707.GA30500@infradead.org>

Thanks,

applied.


From hch at lst.de  Thu Oct 22 09:29:30 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 22 Oct 2020 15:29:30 +0200
Subject: [PATCH V2 1/1] nvmet: don't use BLK_MQ_REQ_NOWAIT for passthru
In-Reply-To: <20201020231404.9365-2-chaitanya.kulkarni@wdc.com>
References: <20201020231404.9365-1-chaitanya.kulkarni@wdc.com>
 <20201020231404.9365-2-chaitanya.kulkarni@wdc.com>
Message-ID: <20201022132930.GB8969@lst.de>

Thanks,

applied.


From hch at infradead.org  Thu Oct 22 09:37:21 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 22 Oct 2020 14:37:21 +0100
Subject: [PATCH] nvme-fc: fix io timeout to abort io
In-Reply-To: <20201016210627.48595-1-james.smart@broadcom.com>
References: <20201016210627.48595-1-james.smart@broadcom.com>
Message-ID: <20201022133721.GB30500@infradead.org>

Thanks, applied.


From hch at infradead.org  Thu Oct 22 09:37:33 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 22 Oct 2020 14:37:33 +0100
Subject: [PATCH] nvme-fc: wait for queues to freeze before calling
 update_hr_hw_queues
In-Reply-To: <20201016211724.48707-1-james.smart@broadcom.com>
References: <20201016211724.48707-1-james.smart@broadcom.com>
Message-ID: <20201022133733.GC30500@infradead.org>

Thanks,

applied.


From hch at infradead.org  Thu Oct 22 09:37:45 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 22 Oct 2020 14:37:45 +0100
Subject: [PATCH] nvme-fc: shorten reconnect delay if possible for FC
In-Reply-To: <20201016212928.49429-1-james.smart@broadcom.com>
References: <20201016212928.49429-1-james.smart@broadcom.com>
Message-ID: <20201022133745.GD30500@infradead.org>

Thanks,

applied.


From hch at infradead.org  Thu Oct 22 09:38:26 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 22 Oct 2020 14:38:26 +0100
Subject: [PATCH] nvme-fc: soften messages logged by error_recovery
In-Reply-To: <20200911200254.111005-1-james.smart@broadcom.com>
References: <20200911200254.111005-1-james.smart@broadcom.com>
Message-ID: <20201022133826.GA31124@infradead.org>

Both hunk fail to apply to the current tree.


From logang at deltatee.com  Thu Oct 22 11:57:58 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu, 22 Oct 2020 09:57:58 -0600
Subject: [PATCH V3 6/6] nvmet: use inline bio for passthru fast path
In-Reply-To: <20201022010234.8304-7-chaitanya.kulkarni@wdc.com>
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
 <20201022010234.8304-7-chaitanya.kulkarni@wdc.com>
Message-ID: <9ba9c9ba-7caf-e24c-1471-62c199cfcd4a@deltatee.com>




On 2020-10-21 7:02 p.m., Chaitanya Kulkarni wrote:
> In nvmet_passthru_execute_cmd() which is a high frequency function
> it uses bio_alloc() which leads to memory allocation from the fs pool
> for each I/O.
> 
> For NVMeoF nvmet_req we already have inline_bvec allocated as a part of
> request allocation that can be used with preallocated bio when we
> already know the size of request before bio allocation with bio_alloc(),
> which we already do.
> 
> Introduce a bio member for the nvmet_req passthru anon union. In the
> fast path, check if we can get away with inline bvec and bio from
> nvmet_req with bio_init() call before actually allocating from the
> bio_alloc().
> 
> This will be useful to avoid any new memory allocation under high
> memory pressure situation and get rid of any extra work of
> allocation (bio_alloc()) vs initialization (bio_init()) when
> transfer len is < NVMET_MAX_INLINE_DATA_LEN that user can configure at
> compile time.
> 
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> ---
>  drivers/nvme/target/nvmet.h    |  1 +
>  drivers/nvme/target/passthru.c | 20 ++++++++++++++++++--
>  2 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
> index 559a15ccc322..408a13084fb4 100644
> --- a/drivers/nvme/target/nvmet.h
> +++ b/drivers/nvme/target/nvmet.h
> @@ -330,6 +330,7 @@ struct nvmet_req {
>  			struct work_struct      work;
>  		} f;
>  		struct {
> +			struct bio		inline_bio;
>  			struct request		*rq;
>  			struct work_struct      work;
>  			bool			use_workqueue;
> diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
> index 496ffedb77dc..32498b4302cc 100644
> --- a/drivers/nvme/target/passthru.c
> +++ b/drivers/nvme/target/passthru.c
> @@ -178,6 +178,14 @@ static void nvmet_passthru_req_done(struct request *rq,
>  	blk_mq_free_request(rq);
>  }
>  
> +static void nvmet_passthru_bio_done(struct bio *bio)
> +{
> +	struct nvmet_req *req = bio->bi_private;
> +
> +	if (bio != &req->p.inline_bio)
> +		bio_put(bio);
> +}
> +
>  static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>  {
>  	int sg_cnt = req->sg_cnt;
> @@ -186,13 +194,21 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>  	int i;
>  
>  	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
> -	bio->bi_end_io = bio_put;
> +	if (req->transfer_len <= NVMET_MAX_INLINE_DATA_LEN) {
> +		bio = &req->p.inline_bio;
> +		bio_init(bio, req->inline_bvec, ARRAY_SIZE(req->inline_bvec));
> +	} else {
> +		bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
> +	}
> +
> +	bio->bi_end_io = nvmet_passthru_bio_done;

I still think it's cleaner to change bi_endio for the inline/alloc'd
cases by simply setting bi_endi_io to bio_put() only in the bio_alloc
case. This should also be more efficient as it's one less indirect call
and condition for the inline case.

Besides that, the entire series looks good to me.

Reviewed-by: Logan Gunthorpe <logang at deltatee.com>

Logan


From logang at deltatee.com  Thu Oct 22 14:45:25 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu, 22 Oct 2020 12:45:25 -0600
Subject: [PATCH blktests v3 00/11] NVMe Target Passthru Block Tests
In-Reply-To: <20201008164024.12546-1-logang@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
Message-ID: <29f3dc94-50c3-1548-034e-09c5394ef781@deltatee.com>


On 2020-10-08 10:40 a.m., Logan Gunthorpe wrote:
> Hi,
> 
> This series adds blktests for the nvmet passthru feature that was merged
> for 5.9. It's been reconciled with Sagi's blktest series that Omar
> recently merged.

Bump. This has been around for a while now. Omar, can you please
consider picking this up?

Thanks,

Logan


From osandov at osandov.com  Thu Oct 22 18:04:12 2020
From: osandov at osandov.com (Omar Sandoval)
Date: Thu, 22 Oct 2020 15:04:12 -0700
Subject: [PATCH blktests v3 00/11] NVMe Target Passthru Block Tests
In-Reply-To: <29f3dc94-50c3-1548-034e-09c5394ef781@deltatee.com>
References: <20201008164024.12546-1-logang@deltatee.com>
 <29f3dc94-50c3-1548-034e-09c5394ef781@deltatee.com>
Message-ID: <20201022220412.GB1006674@relinquished.localdomain>

On Thu, Oct 22, 2020 at 12:45:25PM -0600, Logan Gunthorpe wrote:
> 
> On 2020-10-08 10:40 a.m., Logan Gunthorpe wrote:
> > Hi,
> > 
> > This series adds blktests for the nvmet passthru feature that was merged
> > for 5.9. It's been reconciled with Sagi's blktest series that Omar
> > recently merged.
> 
> Bump. This has been around for a while now. Omar, can you please
> consider picking this up?

There were a couple of shellcheck errors:

tests/nvme/rc:77:8: warning: Declare and assign separately to avoid masking return values. [SC2155]
tests/nvme/rc:278:7: note: Useless echo? Instead of 'echo $(cmd)', just use 'cmd'. [SC2005]

I fixed those up and applied, thanks.


From logang at deltatee.com  Thu Oct 22 18:25:24 2020
From: logang at deltatee.com (Logan Gunthorpe)
Date: Thu, 22 Oct 2020 16:25:24 -0600
Subject: [PATCH blktests v3 00/11] NVMe Target Passthru Block Tests
In-Reply-To: <20201022220412.GB1006674@relinquished.localdomain>
References: <20201008164024.12546-1-logang@deltatee.com>
 <29f3dc94-50c3-1548-034e-09c5394ef781@deltatee.com>
 <20201022220412.GB1006674@relinquished.localdomain>
Message-ID: <088efbf6-efb4-5a0f-2295-cedba2c150ea@deltatee.com>



On 2020-10-22 4:04 p.m., Omar Sandoval wrote:
> On Thu, Oct 22, 2020 at 12:45:25PM -0600, Logan Gunthorpe wrote:
>>
>> On 2020-10-08 10:40 a.m., Logan Gunthorpe wrote:
>>> Hi,
>>>
>>> This series adds blktests for the nvmet passthru feature that was merged
>>> for 5.9. It's been reconciled with Sagi's blktest series that Omar
>>> recently merged.
>>
>> Bump. This has been around for a while now. Omar, can you please
>> consider picking this up?
> 
> There were a couple of shellcheck errors:
> 
> tests/nvme/rc:77:8: warning: Declare and assign separately to avoid masking return values. [SC2155]
> tests/nvme/rc:278:7: note: Useless echo? Instead of 'echo $(cmd)', just use 'cmd'. [SC2005]
> 
> I fixed those up and applied, thanks.

Oh, oops. I must have introduced those very recently.

Thanks for fixing it up and merging!

Logan


From chaitanya.kulkarni at wdc.com  Thu Oct 22 19:58:21 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 22 Oct 2020 16:58:21 -0700
Subject: [PATCH V2] nvmet: fix the Oops in the tracing for flush cmd
Message-ID: <20201022235821.6760-1-chaitanya.kulkarni@wdc.com>

When target side trace in turned on and flush command is issued from the
host it results in the following Oops.

[  856.789724] BUG: kernel NULL pointer dereference, address: 0000000000000068
[  856.790686] #PF: supervisor read access in kernel mode
[  856.791262] #PF: error_code(0x0000) - not-present page
[  856.791863] PGD 6d7110067 P4D 6d7110067 PUD 66f0ad067 PMD 0
[  856.792527] Oops: 0000 [#1] SMP NOPTI
[  856.792950] CPU: 15 PID: 7034 Comm: nvme Tainted: G           OE     5.9.0nvme-5.9+ #71
[  856.793790] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-59-gc9ba5276e3214
[  856.794956] RIP: 0010:trace_event_raw_event_nvmet_req_init+0x13e/0x170 [nvmet]
[  856.795734] Code: 41 5c 41 5d c3 31 d2 31 f6 e8 4e 9b b8 e0 e9 0e ff ff ff 49 8b 55 00 48 8b 38 8b 0
[  856.797740] RSP: 0018:ffffc90001be3a60 EFLAGS: 00010246
[  856.798375] RAX: 0000000000000000 RBX: ffff8887e7d2c01c RCX: 0000000000000000
[  856.799234] RDX: 0000000000000020 RSI: 0000000057e70ea2 RDI: ffff8887e7d2c034
[  856.800088] RBP: ffff88869f710578 R08: ffff888807500d40 R09: 00000000fffffffe
[  856.800951] R10: 0000000064c66670 R11: 00000000ef955201 R12: ffff8887e7d2c034
[  856.801807] R13: ffff88869f7105c8 R14: 0000000000000040 R15: ffff88869f710440
[  856.802667] FS:  00007f6a22bd8780(0000) GS:ffff888813a00000(0000) knlGS:0000000000000000
[  856.803635] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  856.804367] CR2: 0000000000000068 CR3: 00000006d73e0000 CR4: 00000000003506e0
[  856.805283] Call Trace:
[  856.805613]  nvmet_req_init+0x27c/0x480 [nvmet]
[  856.806200]  nvme_loop_queue_rq+0xcb/0x1d0 [nvme_loop]
[  856.806862]  blk_mq_dispatch_rq_list+0x123/0x7b0
[  856.807459]  ? kvm_sched_clock_read+0x14/0x30
[  856.808025]  __blk_mq_sched_dispatch_requests+0xc7/0x170
[  856.808708]  blk_mq_sched_dispatch_requests+0x30/0x60
[  856.809372]  __blk_mq_run_hw_queue+0x70/0x100
[  856.809935]  __blk_mq_delay_run_hw_queue+0x156/0x170
[  856.810574]  blk_mq_run_hw_queue+0x86/0xe0
[  856.811104]  blk_mq_sched_insert_request+0xef/0x160
[  856.811733]  blk_execute_rq+0x69/0xc0
[  856.812212]  ? blk_mq_rq_ctx_init+0xd0/0x230
[  856.812784]  nvme_execute_passthru_rq+0x57/0x130 [nvme_core]
[  856.813461]  nvme_submit_user_cmd+0xeb/0x300 [nvme_core]
[  856.814099]  nvme_user_cmd.isra.82+0x11e/0x1a0 [nvme_core]
[  856.814752]  blkdev_ioctl+0x1dc/0x2c0
[  856.815197]  block_ioctl+0x3f/0x50
[  856.815606]  __x64_sys_ioctl+0x84/0xc0
[  856.816074]  do_syscall_64+0x33/0x40
[  856.816533]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  856.817168] RIP: 0033:0x7f6a222ed107
[  856.817617] Code: 44 00 00 48 8b 05 81 cd 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 8
[  856.819901] RSP: 002b:00007ffca848f058 EFLAGS: 00000202 ORIG_RAX: 0000000000000010
[  856.820846] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f6a222ed107
[  856.821726] RDX: 00007ffca848f060 RSI: 00000000c0484e43 RDI: 0000000000000003
[  856.822603] RBP: 0000000000000003 R08: 000000000000003f R09: 0000000000000005
[  856.823478] R10: 00007ffca848ece0 R11: 0000000000000202 R12: 00007ffca84912d3
[  856.824359] R13: 00007ffca848f4d0 R14: 0000000000000002 R15: 000000000067e900
[  856.825236] Modules linked in: nvme_loop(OE) nvmet(OE) nvme_fabrics(OE) null_blk nvme(OE) nvme_corel

Move the nvmet_req_init() tracepoint after we parse the command in
nvmet_req_init() so that we can get rid of the duplicate
nvmet_find_namespace() call.
Rename __assign_disk_name() ->  __assign_req_name(). Now that we call
tracepoint after parsing the command simplify the newly added
__assign_req_name() which fixes this bug.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
Changes from V1:-

1. Move the nvmet_req_init() tracepoint after we parse the command in
   nvmet_req_init().
2. Rename __assign_disk_name() -> __assign_req_name().
3. Simplify the newly added __assign_req_name() which fixes this bug.
4. Update the commit log.

---
 drivers/nvme/target/core.c  |  4 ++--
 drivers/nvme/target/trace.h | 21 +++++++--------------
 2 files changed, 9 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index aafcbc424b7a..957b39a82431 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -907,8 +907,6 @@ bool nvmet_req_init(struct nvmet_req *req, struct nvmet_cq *cq,
 	req->error_loc = NVMET_NO_ERROR_LOC;
 	req->error_slba = 0;
 
-	trace_nvmet_req_init(req, req->cmd);
-
 	/* no support for fused commands yet */
 	if (unlikely(flags & (NVME_CMD_FUSE_FIRST | NVME_CMD_FUSE_SECOND))) {
 		req->error_loc = offsetof(struct nvme_common_command, flags);
@@ -938,6 +936,8 @@ bool nvmet_req_init(struct nvmet_req *req, struct nvmet_cq *cq,
 	if (status)
 		goto fail;
 
+	trace_nvmet_req_init(req, req->cmd);
+
 	if (unlikely(!percpu_ref_tryget_live(&sq->ref))) {
 		status = NVME_SC_INVALID_FIELD | NVME_SC_DNR;
 		goto fail;
diff --git a/drivers/nvme/target/trace.h b/drivers/nvme/target/trace.h
index 0458046d6501..c14e3249a14d 100644
--- a/drivers/nvme/target/trace.h
+++ b/drivers/nvme/target/trace.h
@@ -46,19 +46,12 @@ static inline struct nvmet_ctrl *nvmet_req_to_ctrl(struct nvmet_req *req)
 	return req->sq->ctrl;
 }
 
-static inline void __assign_disk_name(char *name, struct nvmet_req *req,
-		bool init)
+static inline void __assign_req_name(char *name, struct nvmet_req *req)
 {
-	struct nvmet_ctrl *ctrl = nvmet_req_to_ctrl(req);
-	struct nvmet_ns *ns;
-
-	if ((init && req->sq->qid) || (!init && req->cq->qid)) {
-		ns = nvmet_find_namespace(ctrl, req->cmd->rw.nsid);
-		strncpy(name, ns->device_path, DISK_NAME_LEN);
-		return;
-	}
-
-	memset(name, 0, DISK_NAME_LEN);
+	if (req->ns)
+		strncpy(name, req->ns->device_path, DISK_NAME_LEN);
+	else
+		memset(name, 0, DISK_NAME_LEN);
 }
 #endif
 
@@ -81,7 +74,7 @@ TRACE_EVENT(nvmet_req_init,
 	TP_fast_assign(
 		__entry->cmd = cmd;
 		__entry->ctrl = nvmet_req_to_ctrl(req);
-		__assign_disk_name(__entry->disk, req, true);
+		__assign_req_name(__entry->disk, req);
 		__entry->qid = req->sq->qid;
 		__entry->cid = cmd->common.command_id;
 		__entry->opcode = cmd->common.opcode;
@@ -121,7 +114,7 @@ TRACE_EVENT(nvmet_req_complete,
 		__entry->cid = req->cqe->command_id;
 		__entry->result = le64_to_cpu(req->cqe->result.u64);
 		__entry->status = le16_to_cpu(req->cqe->status) >> 1;
-		__assign_disk_name(__entry->disk, req, false);
+		__assign_req_name(__entry->disk, req);
 	),
 	TP_printk("nvmet%s: %sqid=%d, cmdid=%u, res=%#llx, status=%#x",
 		__print_ctrl_name(__entry->ctrl),
-- 
2.22.1



From lkp at intel.com  Thu Oct 22 22:00:30 2020
From: lkp at intel.com (kernel test robot)
Date: Fri, 23 Oct 2020 10:00:30 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 157b2dc28599df312d71e1ac93b3306bc5b440b5
Message-ID: <5f92393e.5Y8vQRbDP0gTbYaQ%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: 157b2dc28599df312d71e1ac93b3306bc5b440b5  nvme-fc: shorten reconnect delay if possible for FC

elapsed time: 725m

configs tested: 168
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
sh                 kfr2r09-romimage_defconfig
m68k                       m5208evb_defconfig
m68k                          amiga_defconfig
arm                            dove_defconfig
sh                        sh7785lcr_defconfig
powerpc                 mpc837x_mds_defconfig
mips                          ath79_defconfig
um                            kunit_defconfig
powerpc                     ppa8548_defconfig
sh                           se7206_defconfig
powerpc                     tqm5200_defconfig
arm                           spitz_defconfig
h8300                    h8300h-sim_defconfig
c6x                        evmc6474_defconfig
powerpc                        icon_defconfig
arm                         bcm2835_defconfig
ia64                          tiger_defconfig
arm                        mvebu_v7_defconfig
arm                       spear13xx_defconfig
powerpc                 mpc836x_mds_defconfig
xtensa                    xip_kc705_defconfig
powerpc                      obs600_defconfig
arm                        cerfcube_defconfig
powerpc                      ppc40x_defconfig
mips                           ip27_defconfig
powerpc                      pasemi_defconfig
arm                       aspeed_g5_defconfig
arm                       netwinder_defconfig
arm                        oxnas_v6_defconfig
xtensa                  cadence_csp_defconfig
arc                                 defconfig
sh                               allmodconfig
mips                          rb532_defconfig
m68k                             alldefconfig
um                           x86_64_defconfig
parisc                              defconfig
arm                      footbridge_defconfig
sh                             shx3_defconfig
sh                           se7705_defconfig
sh                          polaris_defconfig
arm                           stm32_defconfig
powerpc                         wii_defconfig
nds32                             allnoconfig
arm                          prima2_defconfig
ia64                      gensparse_defconfig
mips                        nlm_xlp_defconfig
arm                        spear3xx_defconfig
arm                    vt8500_v6_v7_defconfig
powerpc                     redwood_defconfig
sh                             sh03_defconfig
mips                      bmips_stb_defconfig
sparc64                          alldefconfig
sh                     magicpanelr2_defconfig
xtensa                generic_kc705_defconfig
powerpc                 mpc8313_rdb_defconfig
powerpc                     asp8347_defconfig
m68k                          sun3x_defconfig
powerpc                       eiger_defconfig
arm                         s3c6400_defconfig
arm                       imx_v6_v7_defconfig
arm                        mvebu_v5_defconfig
c6x                         dsk6455_defconfig
arm                      pxa255-idp_defconfig
arm                         s5pv210_defconfig
m68k                          hp300_defconfig
arc                         haps_hs_defconfig
mips                        qi_lb60_defconfig
arm                            hisi_defconfig
arm                        multi_v5_defconfig
xtensa                              defconfig
mips                            ar7_defconfig
powerpc                      acadia_defconfig
sh                              ul2_defconfig
mips                         cobalt_defconfig
mips                  cavium_octeon_defconfig
m68k                        m5407c3_defconfig
sh                             espt_defconfig
arm                        realview_defconfig
arc                              allyesconfig
sh                          sdk7780_defconfig
powerpc                     rainier_defconfig
mips                           ip22_defconfig
arm                        trizeps4_defconfig
powerpc                     mpc5200_defconfig
arm                         cm_x300_defconfig
sparc                            allyesconfig
powerpc                    socrates_defconfig
c6x                        evmc6678_defconfig
mips                        omega2p_defconfig
arm                         ebsa110_defconfig
arm                        spear6xx_defconfig
powerpc                    mvme5100_defconfig
arm                             rpc_defconfig
powerpc                      ppc64e_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                               defconfig
i386                                defconfig
nios2                               defconfig
c6x                              allyesconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a002-20201022
i386                 randconfig-a005-20201022
i386                 randconfig-a003-20201022
i386                 randconfig-a001-20201022
i386                 randconfig-a006-20201022
i386                 randconfig-a004-20201022
i386                 randconfig-a002-20201023
i386                 randconfig-a005-20201023
i386                 randconfig-a003-20201023
i386                 randconfig-a001-20201023
i386                 randconfig-a006-20201023
i386                 randconfig-a004-20201023
x86_64               randconfig-a011-20201022
x86_64               randconfig-a013-20201022
x86_64               randconfig-a016-20201022
x86_64               randconfig-a015-20201022
x86_64               randconfig-a012-20201022
x86_64               randconfig-a014-20201022
i386                 randconfig-a016-20201022
i386                 randconfig-a014-20201022
i386                 randconfig-a015-20201022
i386                 randconfig-a012-20201022
i386                 randconfig-a013-20201022
i386                 randconfig-a011-20201022
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a001-20201022
x86_64               randconfig-a002-20201022
x86_64               randconfig-a003-20201022
x86_64               randconfig-a006-20201022
x86_64               randconfig-a004-20201022
x86_64               randconfig-a005-20201022

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From lengchao at huawei.com  Thu Oct 22 22:07:10 2020
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 23 Oct 2020 10:07:10 +0800
Subject: [External] Re: [PATCH] nvme-rdma: handle nvme completion data
 length
In-Reply-To: <1c78dbe5-47a4-1590-e064-681cba5fb01d@bytedance.com>
References: <20201022083850.1334880-1-pizhenwei@bytedance.com>
 <04a97f73-ba13-a4b5-3ea4-fc438391507e@huawei.com>
 <1c78dbe5-47a4-1590-e064-681cba5fb01d@bytedance.com>
Message-ID: <c2891e98-a81c-7f3a-3b19-1e34bd1f212c@huawei.com>



On 2020/10/22 18:05, zhenwei pi wrote:
> On 10/22/20 5:55 PM, Chao Leng wrote:
>>
>>
>> On 2020/10/22 16:38, zhenwei pi wrote:
>>> Hit a kernel warning:
>>> refcount_t: underflow; use-after-free.
>>> WARNING: CPU: 0 PID: 0 at lib/refcount.c:28
>>>
>>> RIP: 0010:refcount_warn_saturate+0xd9/0xe0
>>> Call Trace:
>>> ? <IRQ>
>>> ? nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
>>> ? __ib_process_cq+0x76/0x150 [ib_core]
>>> ? ...
>>>
>>> The reason is that a zero bytes message received from target, and the
>>> host side continues to process without length checking, then the
>>> previous CQE is processed twice.
>>>
>>> Handle data length, ignore zero bytes message, and try to recovery for
>>> corrupted CQE case.
>>>
>>> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
>>> ---
>>> ? drivers/nvme/host/rdma.c | 11 +++++++++++
>>> ? 1 file changed, 11 insertions(+)
>>>
>>> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
>>> index 9e378d0a0c01..9f5112040d43 100644
>>> --- a/drivers/nvme/host/rdma.c
>>> +++ b/drivers/nvme/host/rdma.c
>>> @@ -1767,6 +1767,17 @@ static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc)
>>> ????????? return;
>>> ????? }
>>> +??? if (unlikely(!wc->byte_len)) {
>>> +??????? /* zero bytes message could be ignored */
>>> +??????? return;
Resource leak, need nvme_rdma_post_recv.
>>> +??? } else if (unlikely(wc->byte_len < len)) {
>>> +??????? /* Corrupted completion, try to recovry */
>>> +??????? dev_err(queue->ctrl->ctrl.device,
>>> +??????????? "Unexpected nvme completion length(%d)\n", wc->byte_len);
>>> +??????? nvme_rdma_error_recovery(queue->ctrl);
>>> +??????? return;
>>> +??? }
>> !wc->byte_len and wc->byte_len < len may be the same type of anomaly.
>> Why do different error handling?
>> In which scenario zero bytes message received from target? fault inject test or normal test/run?
> 
> Zero bytes message could be used as transport layer keep alive mechanism (I's also developing target side transport layer keep alive now. To reclaim resource, target side needs to close dead connections even kato is set as 0).
nvme over fabric protocol do not define this.
May be async event is a option for target keep alive(if kato set as 0).
> 
>>> +
>>> ????? ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
>>> ????? /*
>>> ?????? * AEN requests are special as they don't time out and can
>>>
> 


From pizhenwei at bytedance.com  Fri Oct 23 02:59:10 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Fri, 23 Oct 2020 14:59:10 +0800
Subject: [PATCH v2] nvme-rdma: handle nvme completion data length
Message-ID: <20201023065910.1358586-1-pizhenwei@bytedance.com>

Hit a kernel warning:
refcount_t: underflow; use-after-free.
WARNING: CPU: 0 PID: 0 at lib/refcount.c:28

RIP: 0010:refcount_warn_saturate+0xd9/0xe0
Call Trace:
 <IRQ>
 nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
 __ib_process_cq+0x76/0x150 [ib_core]
 ...

The reason is that a zero bytes message received from target, and the
host side continues to process without length checking, then the
previous CQE is processed twice.

Handle data length, ignore zero bytes message, and try to recovery for
corrupted CQE case.

Thanks to Chao Leng for suggestions.

Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/host/rdma.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9e378d0a0c01..2ecadd309f4a 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1767,6 +1767,21 @@ static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc)
 		return;
 	}
 
+	/* received data length checking */
+	if (unlikely(wc->byte_len < len)) {
+		/* zero bytes message could be ignored */
+		if (!wc->byte_len) {
+			nvme_rdma_post_recv(queue, qe);
+			return;
+		}
+
+		/* corrupted completion, try to recovry */
+		dev_err(queue->ctrl->ctrl.device,
+			"Unexpected nvme completion length(%d)\n", wc->byte_len);
+		nvme_rdma_error_recovery(queue->ctrl);
+		return;
+	}
+
 	ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
 	/*
 	 * AEN requests are special as they don't time out and can
-- 
2.11.0



From borispismenny at gmail.com  Sun Oct 11 10:44:07 2020
From: borispismenny at gmail.com (Boris Pismenny)
Date: Sun, 11 Oct 2020 17:44:07 +0300
Subject: [PATCH net-next RFC v1 02/10] net: Introduce direct data
 placement tcp offload
In-Reply-To: <ab791897-6398-f38e-c2a6-f54c5ca5e1c5@grimberg.me>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-3-borisp@mellanox.com>
 <ab791897-6398-f38e-c2a6-f54c5ca5e1c5@grimberg.me>
Message-ID: <c5ad9021-9f44-9ed3-8e41-90a4d561d637@gmail.com>



On 09/10/2020 0:47, Sagi Grimberg wrote:
>> + * tcp_ddp.h
>> + *	Author:	Boris Pismenny <borisp at mellanox.com>
>> + *	Copyright (C) 2020 Mellanox Technologies.
>> + */
>> +#ifndef _TCP_DDP_H
>> +#define _TCP_DDP_H
>> +
>> +#include <linux/blkdev.h>
> Why is blkdev.h needed?

That's a lefotover from a previous iteration over this code. I'll remove it for the next patch.

>
>> +#include <linux/netdevice.h>
>> +#include <net/inet_connection_sock.h>
>> +#include <net/sock.h>
>> +
>> +/* limits returned by the offload driver, zero means don't care */
>> +struct tcp_ddp_limits {
>> +	int	 max_ddp_sgl_len;
>> +};
>> +
>> +enum tcp_ddp_type {
>> +	TCP_DDP_NVME = 1,
>> +};
>> +
>> +struct tcp_ddp_config {
>> +	enum tcp_ddp_type    type;
>> +	unsigned char        buf[];
> A little kdoc may help here as its not exactly clear what is
> buf used for (at this point at least)...

Will add.

>> +};
>> +
>> +struct nvme_tcp_config {
> struct nvme_tcp_ddp_config

Sure.

>> +	struct tcp_ddp_config   cfg;
>> +
>> +	u16			pfv;
>> +	u8			cpda;
>> +	u8			dgst;
>> +	int			queue_size;
>> +	int			queue_id;
>> +	int			io_cpu;
>> +};
>> +
> Other than that this looks good to me.
Thanks Sagi!


From borispismenny at gmail.com  Sun Oct 11 10:58:00 2020
From: borispismenny at gmail.com (Boris Pismenny)
Date: Sun, 11 Oct 2020 17:58:00 +0300
Subject: [PATCH net-next RFC v1 03/10] net: Introduce crc offload for tcp
 ddp ulp
In-Reply-To: <904a5edf-3f50-5f6f-6bc6-3d1a1a664aa5@grimberg.me>
References: <20200930162010.21610-1-borisp@mellanox.com>
 <20200930162010.21610-4-borisp@mellanox.com>
 <904a5edf-3f50-5f6f-6bc6-3d1a1a664aa5@grimberg.me>
Message-ID: <8d8a5e86-c734-7fa3-248b-b7c3c7a5699e@gmail.com>

On 09/10/2020 0:51, Sagi Grimberg wrote:
>> This commit itroduces support for CRC offload to direct data placement
>                introduces
>> ULP on the receive side. Both DDP and CRC share a common API to
>> initialize the offload for a TCP socket. But otherwise, both can
>> be executed independently.
>  From the API it not clear that the offload engine does crc32c, do you
> see this extended to other crc types in the future?

Yes, it is somewhat implicit, and it depends on the tcp ddp configuration. At the moment we only do nvme-tcp, and that has only CRC32C. If in the future, there would be other CRC variants, or data digest algorithms, then the code could be easily extended.

In general, any data digest over TCP can leverage this code and SKB member. Not only other CRC types can benefit from it, but even more complex data digest algorithms like SHA can use this. In essence this bit is similar to the TLS skb->decrypted bit. In TLS, skb->decrypted also indicates that the authentication has no error, exactly like ddp_crc indicates that there is no CRC32C error. The only reason we didn't use the same bit for both is that these two protocol offloads can be combined and that will benefit from two independent bits.

>
> Other than this the patch looks good to me,
> Reviewed-by: Sagi Grimberg <sagi at grimberg.me>



From lengchao at huawei.com  Fri Oct 23 04:45:23 2020
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 23 Oct 2020 16:45:23 +0800
Subject: [PATCH v2] nvme-rdma: handle nvme completion data length
In-Reply-To: <20201023065910.1358586-1-pizhenwei@bytedance.com>
References: <20201023065910.1358586-1-pizhenwei@bytedance.com>
Message-ID: <33381bb5-6daa-5f47-9e3c-a57eeb490950@huawei.com>

Looks good.

Reviewed-by: Chao Leng <lengchao at huawei.com>

On 2020/10/23 14:59, zhenwei pi wrote:
> Hit a kernel warning:
> refcount_t: underflow; use-after-free.
> WARNING: CPU: 0 PID: 0 at lib/refcount.c:28
> 
> RIP: 0010:refcount_warn_saturate+0xd9/0xe0
> Call Trace:
>   <IRQ>
>   nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
>   __ib_process_cq+0x76/0x150 [ib_core]
>   ...
> 
> The reason is that a zero bytes message received from target, and the
> host side continues to process without length checking, then the
> previous CQE is processed twice.
> 
> Handle data length, ignore zero bytes message, and try to recovery for
> corrupted CQE case.
> 
> Thanks to Chao Leng for suggestions.
> 
> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
> ---
>   drivers/nvme/host/rdma.c | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)
> 
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 9e378d0a0c01..2ecadd309f4a 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1767,6 +1767,21 @@ static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc)
>   		return;
>   	}
>   
> +	/* received data length checking */
> +	if (unlikely(wc->byte_len < len)) {
> +		/* zero bytes message could be ignored */
> +		if (!wc->byte_len) {
> +			nvme_rdma_post_recv(queue, qe);
> +			return;
> +		}
> +
> +		/* corrupted completion, try to recovry */
> +		dev_err(queue->ctrl->ctrl.device,
> +			"Unexpected nvme completion length(%d)\n", wc->byte_len);
> +		nvme_rdma_error_recovery(queue->ctrl);
> +		return;
> +	}
> +
>   	ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
>   	/*
>   	 * AEN requests are special as they don't time out and can
> 


From dwagner at suse.de  Fri Oct 23 05:02:28 2020
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 23 Oct 2020 11:02:28 +0200
Subject: NVMe CLI Version Tag
In-Reply-To: <20201020235454.GB1438267@dhcp-10-100-145-180.wdc.com>
References: <BYAPR04MB5656B49BFE47211B3A2F1748EA1F0@BYAPR04MB5656.namprd04.prod.outlook.com>
 <20201020175017.2avzkm2bwyeg2t3a@beryllium.lan>
 <20201020175644.GA1438267@dhcp-10-100-145-180.wdc.com>
 <20201020235454.GB1438267@dhcp-10-100-145-180.wdc.com>
Message-ID: <20201023090228.w2gooi4vpj4aaugt@beryllium.lan>

On Tue, Oct 20, 2020 at 04:54:54PM -0700, Keith Busch wrote:
> Looks okay on my end, so v1.13 tagged and pushed. I didn't pull any of
> the pending requests like I usually do for this release as that
> inevitably leads to breakage right before a release. I'll start pulling
> new stuff in now that this is hopefully a good stable point with ZNS
> support and all the other recent goodies.

Excellent! Many thanks!
Daniel


From hch at infradead.org  Fri Oct 23 07:00:35 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Fri, 23 Oct 2020 13:00:35 +0200
Subject: [GIT PULL] nvme fixes for 5.10-rc1
Message-ID: <20201023110035.GA3504312@infradead.org>

The following changes since commit cb3a92da231bcf55c243d00fa619ee36281b0001:

  block: remove unused members for io_context (2020-10-20 07:10:14 -0600)

are available in the Git repository at:

  git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-23

for you to fetch changes up to f673714a1247669bc90322dfb14a5cf553833796:

  nvme-fc: shorten reconnect delay if possible for FC (2020-10-23 12:54:45 +0200)

----------------------------------------------------------------
nvme fixes for 5.10

 - rdma error handling fixes (Chao Leng)
 - fc error handling and reconnect fixes (James Smart)
 - fix the qid displace when tracing ioctl command (Keith Busch)
 - don't use BLK_MQ_REQ_NOWAIT for passthru (Chaitanya Kulkarni)
 - fix MTDT for passthru (Logan Gunthorpe)
 - blacklist Write Same on more devices (Kai-Heng Feng)
 - fix an uninitialized work struct (zhenwei pi)

----------------------------------------------------------------
Chaitanya Kulkarni (1):
      nvmet: don't use BLK_MQ_REQ_NOWAIT for passthru

Chao Leng (2):
      nvme-rdma: fix crash when connect rejected
      nvme-rdma: fix crash due to incorrect cqe

James Smart (4):
      nvme-fc: fix io timeout to abort I/O
      nvme-fc: fix error loop in create_hw_io_queues
      nvme-fc: wait for queues to freeze before calling update_hr_hw_queues
      nvme-fc: shorten reconnect delay if possible for FC

Kai-Heng Feng (1):
      nvme-pci: disable Write Zeroes on Sandisk Skyhawk

Keith Busch (1):
      nvme: use queuedata for nvme_req_qid

Logan Gunthorpe (2):
      nvmet: limit passthru MTDS by BIO_MAX_PAGES
      nvmet: cleanup nvmet_passthru_map_sg()

zhenwei pi (1):
      nvmet: fix uninitialized work for zero kato

 drivers/nvme/host/fc.c         | 138 ++++++++++++++++++++++++++++-------------
 drivers/nvme/host/nvme.h       |   2 +-
 drivers/nvme/host/pci.c        |   2 +
 drivers/nvme/host/rdma.c       |   6 +-
 drivers/nvme/target/core.c     |   3 +-
 drivers/nvme/target/passthru.c |  18 ++++--
 6 files changed, 115 insertions(+), 54 deletions(-)


From axboe at kernel.dk  Fri Oct 23 09:30:00 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Fri, 23 Oct 2020 07:30:00 -0600
Subject: [GIT PULL] nvme fixes for 5.10-rc1
In-Reply-To: <20201023110035.GA3504312@infradead.org>
References: <20201023110035.GA3504312@infradead.org>
Message-ID: <2e3120d9-7c08-1aba-1481-118131a459b0@kernel.dk>

On 10/23/20 5:00 AM, Christoph Hellwig wrote:
> The following changes since commit cb3a92da231bcf55c243d00fa619ee36281b0001:
> 
>   block: remove unused members for io_context (2020-10-20 07:10:14 -0600)
> 
> are available in the Git repository at:
> 
>   git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-23

Pulled, thanks.

-- 
Jens Axboe



From sagi at grimberg.me  Fri Oct 23 14:01:40 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 23 Oct 2020 11:01:40 -0700
Subject: [PATCH v2] nvme-rdma: handle nvme completion data length
In-Reply-To: <20201023065910.1358586-1-pizhenwei@bytedance.com>
References: <20201023065910.1358586-1-pizhenwei@bytedance.com>
Message-ID: <d23e33e7-d545-2ad0-d163-5182dd5430b1@grimberg.me>


> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 9e378d0a0c01..2ecadd309f4a 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1767,6 +1767,21 @@ static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc)
>   		return;
>   	}
>   
> +	/* received data length checking */
> +	if (unlikely(wc->byte_len < len)) {
> +		/* zero bytes message could be ignored */
> +		if (!wc->byte_len) {
> +			nvme_rdma_post_recv(queue, qe);
> +			return;
> +		}

Nothing in the spec defines zero-length messages, hence we cannot
support something that is not standard. If your array needs this,
please submit a TPAR to the NVMe TWG.


From kbusch at kernel.org  Fri Oct 23 15:16:28 2020
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 23 Oct 2020 12:16:28 -0700
Subject: [PATCH] nvme: ignore zone validate errors on subsequent scans
Message-ID: <20201023191628.1729393-1-kbusch@kernel.org>

Revalidating nvme zoned namespaces requires IO commands, and there are
controller states that prevent IO. For example, a sanitize in progress
is required to fail all IO, but we don't want to remove a namespace
we've previously added just because the controller is in such a state.
Suppress the error in this case.

Reported-by: Michael Nguyen <michael.nguyen at wdc.com>
Signed-off-by: Keith Busch <kbusch at kernel.org>
---
Yet another reason zone management commands should have been in the
admin command set!

 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 95ef4943d8bd..376096bfc54a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2125,7 +2125,7 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_id_ns *id)
 
 	if (blk_queue_is_zoned(ns->queue)) {
 		ret = nvme_revalidate_zones(ns);
-		if (ret)
+		if (ret && !nvme_first_scan(ns->disk))
 			return ret;
 	}
 
-- 
2.24.1



From chaitanya.kulkarni at wdc.com  Fri Oct 23 16:25:21 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Fri, 23 Oct 2020 13:25:21 -0700
Subject: [PATCH] nvme-cli: remove default ctrl_loss_tmo for connect
Message-ID: <20201023202521.9610-1-chaitanya.kulkarni@wdc.com>

Since nvme-cli adds a default ctrl_loss_tmo for connect command
all the testcases in the blktests are failing. Thjs addition of
the parameter breaks the nvme-cli command-line compatibility with older
versions.

Remove the default ctrl loss tmo parameter addition for the connect
command user can specify that from the command line.

Without this patch :-

# ./check tests/nvme/008
nvme/008 (create an NVMeOF host with a block device-backed ns) [failed]
    runtime  0.157s  ...  0.165s
    --- tests/nvme/008.out	2019-09-13 21:45:10.353000000 -0700
    +++ /root/blktests/results/nodev/nvme/008.out.bad	2020-10-23 10:08:16.910000000 -0700
    @@ -1,5 +1,7 @@
     Running nvme/008
    -91fdba0d-f87b-4c25-b80f-db7be1418b9e
    -uuid.91fdba0d-f87b-4c25-b80f-db7be1418b9e
    -NQN:blktests-subsystem-1 disconnected 1 controller(s)
    +Failed to write to /dev/nvme-fabrics: Invalid argument
    +cat: /sys/class/nvme/nvme*/subsysnqn: No such file or directory
    +cat: /sys/block/n1/uuid: No such file or directory
    ...
    (Run 'diff -u tests/nvme/008.out /root/blktests/results/nodev/nvme/008.out.bad' to see the entire diff)

With this patch :-

# ./check tests/nvme/008
nvme/008 (create an NVMeOF host with a block device-backed ns) [passed]
    runtime  0.165s  ...  2.857s


Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 fabrics.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/fabrics.c b/fabrics.c
index f8a055b..9c33c02 100644
--- a/fabrics.c
+++ b/fabrics.c
@@ -896,8 +896,6 @@ static int build_options(char *argstr, int max_len, bool discover)
 				cfg.keep_alive_tmo, false)) ||
 	    add_int_argument(&argstr, &max_len, "reconnect_delay",
 				cfg.reconnect_delay, false) ||
-	    add_int_argument(&argstr, &max_len, "ctrl_loss_tmo",
-				cfg.ctrl_loss_tmo, true) ||
 	    add_int_argument(&argstr, &max_len, "tos",
 				cfg.tos, true) ||
 	    add_bool_argument(&argstr, &max_len, "duplicate_connect",
-- 
2.22.1



From Chaitanya.Kulkarni at wdc.com  Fri Oct 23 16:34:46 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Fri, 23 Oct 2020 20:34:46 +0000
Subject: [PATCH] nvme: ignore zone validate errors on subsequent scans
References: <20201023191628.1729393-1-kbusch@kernel.org>
Message-ID: <BYAPR04MB4965696830CA41FFF35D2139861A0@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/23/20 12:22, Keith Busch wrote:
> Revalidating nvme zoned namespaces requires IO commands, and there are
> controller states that prevent IO. For example, a sanitize in progress
> is required to fail all IO, but we don't want to remove a namespace
> we've previously added just because the controller is in such a state.
> Suppress the error in this case.
>
> Reported-by: Michael Nguyen <michael.nguyen at wdc.com>
> Signed-off-by: Keith Busch <kbusch at kernel.org>
> ---
> Yet another reason zone management commands should have been in the
> admin command set!

Agree, I've raised this question.

Looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>



From sagi at grimberg.me  Fri Oct 23 17:13:57 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 23 Oct 2020 14:13:57 -0700
Subject: [PATCH] nvme-cli: remove default ctrl_loss_tmo for connect
In-Reply-To: <20201023202521.9610-1-chaitanya.kulkarni@wdc.com>
References: <20201023202521.9610-1-chaitanya.kulkarni@wdc.com>
Message-ID: <58b2a430-4ea8-174e-ab7c-ce93d45c4bc2@grimberg.me>


> Since nvme-cli adds a default ctrl_loss_tmo for connect command
> all the testcases in the blktests are failing. Thjs addition of
> the parameter breaks the nvme-cli command-line compatibility with older
> versions.
> 
> Remove the default ctrl loss tmo parameter addition for the connect
> command user can specify that from the command line.

Please CC the patch author that made ctrl_loss_tmo to be passed
by default, the patch description provided does not specify
a good reason why the patch was given.

Also, please add a Fixes: tag.

--
commit 4b607779af71d23bc5ff7fcc14a7e1fde8507ed7
Author: Aleksei Marov <alekseymmm at mail.ru>
Date:   Fri Jun 19 18:10:07 2020 +0300

     Allow 0 value of ctrl_loss_tmo.
--


From sagi at grimberg.me  Fri Oct 23 17:16:01 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 23 Oct 2020 14:16:01 -0700
Subject: [PATCH] nvme-cli: remove default ctrl_loss_tmo for connect
In-Reply-To: <20201023202521.9610-1-chaitanya.kulkarni@wdc.com>
References: <20201023202521.9610-1-chaitanya.kulkarni@wdc.com>
Message-ID: <5c212507-d518-5343-3836-93107cfea53f@grimberg.me>


> diff --git a/fabrics.c b/fabrics.c
> index f8a055b..9c33c02 100644
> --- a/fabrics.c
> +++ b/fabrics.c
> @@ -896,8 +896,6 @@ static int build_options(char *argstr, int max_len, bool discover)
>   				cfg.keep_alive_tmo, false)) ||
>   	    add_int_argument(&argstr, &max_len, "reconnect_delay",
>   				cfg.reconnect_delay, false) ||
> -	    add_int_argument(&argstr, &max_len, "ctrl_loss_tmo",
> -				cfg.ctrl_loss_tmo, true) ||

You just removed the entire option altogether? the user can no longer
pass ctrl_loss_tmo now. You should instead disallow zero back (i.e.
revert the offending patch that made it as such) and figure out
a proper way to allow ctrl_loss_tmo=0 functionality.


From james.smart at broadcom.com  Fri Oct 23 18:26:04 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 23 Oct 2020 15:26:04 -0700
Subject: [PATCH] nvme_fc: track error_recovery while connecting
Message-ID: <20201023222604.90435-1-james.smart@broadcom.com>

Whenever there are errors during CONNECTING, the driver recovers by
aborting all outstanding ios and counts on the io completion to fail them
and thus the connection/association they are on.  However, the connection
failure depends on a failure state from the core routines.  Not all
commands that are issued by the core routine are guaranteed to cause a
failure of the core routine. They may be treated as a failure status and
the status is then ignored.

As such, whenever the transport enters error_recovery while CONNECTING,
it will set a new flag indicating an association failed. The
create_association routine which creates and initializes the controller,
will monitor the state of the flag as well as the core routine error
status and ensure the association fails if there was an error.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 3c002bdcace3..cfb6ef74c742 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -146,7 +146,8 @@ struct nvme_fc_rport {
 
 /* fc_ctrl flags values - specified as bit positions */
 #define ASSOC_ACTIVE		0
-#define FCCTRL_TERMIO		1
+#define ASSOC_FAILED		1
+#define FCCTRL_TERMIO		2
 
 struct nvme_fc_ctrl {
 	spinlock_t		lock;
@@ -2988,6 +2989,8 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 		ctrl->cnum, ctrl->lport->localport.port_name,
 		ctrl->rport->remoteport.port_name, ctrl->ctrl.opts->subsysnqn);
 
+	clear_bit(ASSOC_FAILED, &ctrl->flags);
+
 	/*
 	 * Create the admin queue
 	 */
@@ -3016,7 +3019,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 	 */
 
 	ret = nvme_enable_ctrl(&ctrl->ctrl);
-	if (ret)
+	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
 		goto out_disconnect_admin_queue;
 
 	ctrl->ctrl.max_segments = ctrl->lport->ops->max_sgl_segments;
@@ -3026,7 +3029,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 	blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
 	ret = nvme_init_identify(&ctrl->ctrl);
-	if (ret)
+	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
 		goto out_disconnect_admin_queue;
 
 	/* sanity checks */
@@ -3071,9 +3074,9 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 			ret = nvme_fc_create_io_queues(ctrl);
 		else
 			ret = nvme_fc_recreate_io_queues(ctrl);
-		if (ret)
-			goto out_term_aen_ops;
 	}
+	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
+		goto out_term_aen_ops;
 
 	changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
 
@@ -3301,6 +3304,7 @@ __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
 	 */
 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
 		__nvme_fc_abort_outstanding_ios(ctrl, true);
+		set_bit(ASSOC_FAILED, &ctrl->flags);
 		return;
 	}
 
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201023/d7c33a4b/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 23 18:27:49 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 23 Oct 2020 15:27:49 -0700
Subject: [PATCH v2 0/3] nvme-fc: clean up error recovery implementation
Message-ID: <20201023222752.90621-1-james.smart@broadcom.com>

After reviewing the error_recovery paths to address the abort on the
io to cover timeouts on connecting ios, it became clear that some of the
thing in the way it was coded were unnecessary.

This patch set cleans up those items and refactors. It then addresses the
failure case of io errors during reconnect which wants to take the create
failure cases, but there is no status back from the core routines to
trigger that path.

Note: this assumes the patch "nvme_fc: track error_recovery while
connecting" has been applied already

V2:
 Extracted v1's patch4 (the track error_recovery patch) into it's own
 patch as it is a fix, not a cleanup. That patch was posted on its own.

James Smart (3):
  nvme-fc: remove err_work work item
  nvme-fc: eliminate terminate_io use by nvme_fc_error_recovery
  nvme-fc: remove nvme_fc_terminate_io()

 drivers/nvme/host/fc.c | 258 ++++++++++++++++-------------------------
 1 file changed, 101 insertions(+), 157 deletions(-)

-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201023/7b96312d/attachment.p7s>

From james.smart at broadcom.com  Fri Oct 23 18:27:50 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 23 Oct 2020 15:27:50 -0700
Subject: [PATCH v2 1/3] nvme-fc: remove err_work work item
In-Reply-To: <20201023222752.90621-1-james.smart@broadcom.com>
References: <20201023222752.90621-1-james.smart@broadcom.com>
Message-ID: <20201023222752.90621-2-james.smart@broadcom.com>

err_work was created to handle errors (mainly io timeout) while in
CONNECTING state. The flag for err_work_active is also unneeded.

Remove err_work_active and err_work.  The actions to abort ios are moved
inline to nvme_error_recovery().

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 40 ++++++++++------------------------------
 1 file changed, 10 insertions(+), 30 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index cfb6ef74c742..9cdb116b545b 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -158,7 +158,6 @@ struct nvme_fc_ctrl {
 	u32			cnum;
 
 	bool			ioq_live;
-	atomic_t		err_work_active;
 	u64			association_id;
 	struct nvmefc_ls_rcv_op	*rcv_disconn;
 
@@ -168,7 +167,6 @@ struct nvme_fc_ctrl {
 	struct blk_mq_tag_set	tag_set;
 
 	struct delayed_work	connect_work;
-	struct work_struct	err_work;
 
 	struct kref		ref;
 	unsigned long		flags;
@@ -2415,11 +2413,11 @@ nvme_fc_nvme_ctrl_freed(struct nvme_ctrl *nctrl)
 	nvme_fc_ctrl_put(ctrl);
 }
 
+static void __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl);
+
 static void
 nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
 {
-	int active;
-
 	/*
 	 * if an error (io timeout, etc) while (re)connecting,
 	 * it's an error on creating the new association.
@@ -2428,11 +2426,14 @@ nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
 	 * ios hitting this path before things are cleaned up.
 	 */
 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
-		active = atomic_xchg(&ctrl->err_work_active, 1);
-		if (!active && !queue_work(nvme_fc_wq, &ctrl->err_work)) {
-			atomic_set(&ctrl->err_work_active, 0);
-			WARN_ON(1);
-		}
+		__nvme_fc_terminate_io(ctrl);
+
+		/*
+		 * Rescheduling the connection after recovering
+		 * from the io error is left to the reconnect work
+		 * item, which is what should have stalled waiting on
+		 * the io that had the error that scheduled this work.
+		 */
 		return;
 	}
 
@@ -3240,7 +3241,6 @@ nvme_fc_delete_ctrl(struct nvme_ctrl *nctrl)
 {
 	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
 
-	cancel_work_sync(&ctrl->err_work);
 	cancel_delayed_work_sync(&ctrl->connect_work);
 	/*
 	 * kill the association on the link side.  this will block
@@ -3351,23 +3351,6 @@ nvme_fc_reset_ctrl_work(struct work_struct *work)
 			ctrl->cnum);
 }
 
-static void
-nvme_fc_connect_err_work(struct work_struct *work)
-{
-	struct nvme_fc_ctrl *ctrl =
-			container_of(work, struct nvme_fc_ctrl, err_work);
-
-	__nvme_fc_terminate_io(ctrl);
-
-	atomic_set(&ctrl->err_work_active, 0);
-
-	/*
-	 * Rescheduling the connection after recovering
-	 * from the io error is left to the reconnect work
-	 * item, which is what should have stalled waiting on
-	 * the io that had the error that scheduled this work.
-	 */
-}
 
 static const struct nvme_ctrl_ops nvme_fc_ctrl_ops = {
 	.name			= "fc",
@@ -3495,7 +3478,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 	ctrl->dev = lport->dev;
 	ctrl->cnum = idx;
 	ctrl->ioq_live = false;
-	atomic_set(&ctrl->err_work_active, 0);
 	init_waitqueue_head(&ctrl->ioabort_wait);
 
 	get_device(ctrl->dev);
@@ -3503,7 +3485,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 
 	INIT_WORK(&ctrl->ctrl.reset_work, nvme_fc_reset_ctrl_work);
 	INIT_DELAYED_WORK(&ctrl->connect_work, nvme_fc_connect_ctrl_work);
-	INIT_WORK(&ctrl->err_work, nvme_fc_connect_err_work);
 	spin_lock_init(&ctrl->lock);
 
 	/* io queue count */
@@ -3596,7 +3577,6 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 fail_ctrl:
 	nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING);
 	cancel_work_sync(&ctrl->ctrl.reset_work);
-	cancel_work_sync(&ctrl->err_work);
 	cancel_delayed_work_sync(&ctrl->connect_work);
 
 	ctrl->ctrl.opts = NULL;
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201023/19bc38b1/attachment-0001.p7s>

From james.smart at broadcom.com  Fri Oct 23 18:27:51 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 23 Oct 2020 15:27:51 -0700
Subject: [PATCH v2 2/3] nvme-fc: eliminate terminate_io use by
 nvme_fc_error_recovery
In-Reply-To: <20201023222752.90621-1-james.smart@broadcom.com>
References: <20201023222752.90621-1-james.smart@broadcom.com>
Message-ID: <20201023222752.90621-3-james.smart@broadcom.com>

nvme_fc_error_recovery() special cases handling when in CONNECTING state
and calls __nvme_fc_terminate_io(). __nvme_fc_terminate_io() itself
special cases CONNECTING state and calls the routine to abort outstanding
ios.

Simplify the sequence by putting the call to abort outstanding ios directly
in nvme_fc_error_recovery.

Move the location of __nvme_fc_abort_outstanding_ios(), and
nvme_fc_terminate_exchange() which is called by it, to avoid adding
function prototypes for nvme_fc_error_recovery().

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 187 ++++++++++++++++++-----------------------
 1 file changed, 84 insertions(+), 103 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 9cdb116b545b..ffbfb0533cac 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2413,27 +2413,97 @@ nvme_fc_nvme_ctrl_freed(struct nvme_ctrl *nctrl)
 	nvme_fc_ctrl_put(ctrl);
 }
 
-static void __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl);
+/*
+ * This routine is used by the transport when it needs to find active
+ * io on a queue that is to be terminated. The transport uses
+ * blk_mq_tagset_busy_itr() to find the busy requests, which then invoke
+ * this routine to kill them on a 1 by 1 basis.
+ *
+ * As FC allocates FC exchange for each io, the transport must contact
+ * the LLDD to terminate the exchange, thus releasing the FC exchange.
+ * After terminating the exchange the LLDD will call the transport's
+ * normal io done path for the request, but it will have an aborted
+ * status. The done path will return the io request back to the block
+ * layer with an error status.
+ */
+static bool
+nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
+{
+	struct nvme_ctrl *nctrl = data;
+	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
+	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
+
+	__nvme_fc_abort_op(ctrl, op);
+	return true;
+}
+
+/*
+ * This routine runs through all outstanding commands on the association
+ * and aborts them.  This routine is typically be called by the
+ * delete_association routine. It is also called due to an error during
+ * reconnect. In that scenario, it is most likely a command that initializes
+ * the controller, including fabric Connect commands on io queues, that
+ * may have timed out or failed thus the io must be killed for the connect
+ * thread to see the error.
+ */
+static void
+__nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
+{
+	/*
+	 * If io queues are present, stop them and terminate all outstanding
+	 * ios on them. As FC allocates FC exchange for each io, the
+	 * transport must contact the LLDD to terminate the exchange,
+	 * thus releasing the FC exchange. We use blk_mq_tagset_busy_itr()
+	 * to tell us what io's are busy and invoke a transport routine
+	 * to kill them with the LLDD.  After terminating the exchange
+	 * the LLDD will call the transport's normal io done path, but it
+	 * will have an aborted status. The done path will return the
+	 * io requests back to the block layer as part of normal completions
+	 * (but with error status).
+	 */
+	if (ctrl->ctrl.queue_count > 1) {
+		nvme_stop_queues(&ctrl->ctrl);
+		blk_mq_tagset_busy_iter(&ctrl->tag_set,
+				nvme_fc_terminate_exchange, &ctrl->ctrl);
+		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
+		if (start_queues)
+			nvme_start_queues(&ctrl->ctrl);
+	}
+
+	/*
+	 * Other transports, which don't have link-level contexts bound
+	 * to sqe's, would try to gracefully shutdown the controller by
+	 * writing the registers for shutdown and polling (call
+	 * nvme_shutdown_ctrl()). Given a bunch of i/o was potentially
+	 * just aborted and we will wait on those contexts, and given
+	 * there was no indication of how live the controlelr is on the
+	 * link, don't send more io to create more contexts for the
+	 * shutdown. Let the controller fail via keepalive failure if
+	 * its still present.
+	 */
+
+	/*
+	 * clean up the admin queue. Same thing as above.
+	 */
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
+				nvme_fc_terminate_exchange, &ctrl->ctrl);
+	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
+}
 
 static void
 nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
 {
 	/*
-	 * if an error (io timeout, etc) while (re)connecting,
-	 * it's an error on creating the new association.
-	 * Start the error recovery thread if it hasn't already
-	 * been started. It is expected there could be multiple
-	 * ios hitting this path before things are cleaned up.
+	 * if an error (io timeout, etc) while (re)connecting, the remote
+	 * port requested terminating of the association (disconnect_ls)
+	 * or an error (timeout or abort) occurred on an io while creating
+	 * the controller.  Abort any ios on the association and let the
+	 * create_association error path resolve things.
 	 */
 	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
-		__nvme_fc_terminate_io(ctrl);
-
-		/*
-		 * Rescheduling the connection after recovering
-		 * from the io error is left to the reconnect work
-		 * item, which is what should have stalled waiting on
-		 * the io that had the error that scheduled this work.
-		 */
+		__nvme_fc_abort_outstanding_ios(ctrl, true);
+		set_bit(ASSOC_FAILED, &ctrl->flags);
 		return;
 	}
 
@@ -2747,30 +2817,6 @@ nvme_fc_complete_rq(struct request *rq)
 	nvme_fc_ctrl_put(ctrl);
 }
 
-/*
- * This routine is used by the transport when it needs to find active
- * io on a queue that is to be terminated. The transport uses
- * blk_mq_tagset_busy_itr() to find the busy requests, which then invoke
- * this routine to kill them on a 1 by 1 basis.
- *
- * As FC allocates FC exchange for each io, the transport must contact
- * the LLDD to terminate the exchange, thus releasing the FC exchange.
- * After terminating the exchange the LLDD will call the transport's
- * normal io done path for the request, but it will have an aborted
- * status. The done path will return the io request back to the block
- * layer with an error status.
- */
-static bool
-nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
-{
-	struct nvme_ctrl *nctrl = data;
-	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
-	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
-
-	__nvme_fc_abort_op(ctrl, op);
-	return true;
-}
-
 
 static const struct blk_mq_ops nvme_fc_mq_ops = {
 	.queue_rq	= nvme_fc_queue_rq,
@@ -3111,60 +3157,6 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 }
 
 
-/*
- * This routine runs through all outstanding commands on the association
- * and aborts them.  This routine is typically be called by the
- * delete_association routine. It is also called due to an error during
- * reconnect. In that scenario, it is most likely a command that initializes
- * the controller, including fabric Connect commands on io queues, that
- * may have timed out or failed thus the io must be killed for the connect
- * thread to see the error.
- */
-static void
-__nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
-{
-	/*
-	 * If io queues are present, stop them and terminate all outstanding
-	 * ios on them. As FC allocates FC exchange for each io, the
-	 * transport must contact the LLDD to terminate the exchange,
-	 * thus releasing the FC exchange. We use blk_mq_tagset_busy_itr()
-	 * to tell us what io's are busy and invoke a transport routine
-	 * to kill them with the LLDD.  After terminating the exchange
-	 * the LLDD will call the transport's normal io done path, but it
-	 * will have an aborted status. The done path will return the
-	 * io requests back to the block layer as part of normal completions
-	 * (but with error status).
-	 */
-	if (ctrl->ctrl.queue_count > 1) {
-		nvme_stop_queues(&ctrl->ctrl);
-		blk_mq_tagset_busy_iter(&ctrl->tag_set,
-				nvme_fc_terminate_exchange, &ctrl->ctrl);
-		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
-		if (start_queues)
-			nvme_start_queues(&ctrl->ctrl);
-	}
-
-	/*
-	 * Other transports, which don't have link-level contexts bound
-	 * to sqe's, would try to gracefully shutdown the controller by
-	 * writing the registers for shutdown and polling (call
-	 * nvme_shutdown_ctrl()). Given a bunch of i/o was potentially
-	 * just aborted and we will wait on those contexts, and given
-	 * there was no indication of how live the controlelr is on the
-	 * link, don't send more io to create more contexts for the
-	 * shutdown. Let the controller fail via keepalive failure if
-	 * its still present.
-	 */
-
-	/*
-	 * clean up the admin queue. Same thing as above.
-	 */
-	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
-	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
-				nvme_fc_terminate_exchange, &ctrl->ctrl);
-	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
-}
-
 /*
  * This routine stops operation of the controller on the host side.
  * On the host os stack side: Admin and IO queues are stopped,
@@ -3297,17 +3289,6 @@ nvme_fc_reconnect_or_delete(struct nvme_fc_ctrl *ctrl, int status)
 static void
 __nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
 {
-	/*
-	 * if state is CONNECTING - the error occurred as part of a
-	 * reconnect attempt. Abort any ios on the association and
-	 * let the create_association error paths resolve things.
-	 */
-	if (ctrl->ctrl.state == NVME_CTRL_CONNECTING) {
-		__nvme_fc_abort_outstanding_ios(ctrl, true);
-		set_bit(ASSOC_FAILED, &ctrl->flags);
-		return;
-	}
-
 	/*
 	 * For any other state, kill the association. As this routine
 	 * is a common io abort routine for resetting and such, after
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201023/aef5754d/attachment-0001.p7s>

From james.smart at broadcom.com  Fri Oct 23 18:27:52 2020
From: james.smart at broadcom.com (James Smart)
Date: Fri, 23 Oct 2020 15:27:52 -0700
Subject: [PATCH v2 3/3] nvme-fc: remove nvme_fc_terminate_io()
In-Reply-To: <20201023222752.90621-1-james.smart@broadcom.com>
References: <20201023222752.90621-1-james.smart@broadcom.com>
Message-ID: <20201023222752.90621-4-james.smart@broadcom.com>

__nvme_fc_terminate_io() is now called by only 1 place, in reset_work.
Consoldate and move the functionality of terminate_io into reset_work.

In reset_work, rather than calling the create_association directly,
schedule the connect work element to do its thing. After scheduling, flush
the connect work element to continue with semantic of not returning until
connect has been attempted at least once.

Signed-off-by: James Smart <james.smart at broadcom.com>
---
 drivers/nvme/host/fc.c | 49 ++++++++++++++----------------------------
 1 file changed, 16 insertions(+), 33 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index ffbfb0533cac..f4c246462658 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3287,49 +3287,32 @@ nvme_fc_reconnect_or_delete(struct nvme_fc_ctrl *ctrl, int status)
 }
 
 static void
-__nvme_fc_terminate_io(struct nvme_fc_ctrl *ctrl)
+nvme_fc_reset_ctrl_work(struct work_struct *work)
 {
-	/*
-	 * For any other state, kill the association. As this routine
-	 * is a common io abort routine for resetting and such, after
-	 * the association is terminated, ensure that the state is set
-	 * to CONNECTING.
-	 */
+	struct nvme_fc_ctrl *ctrl =
+		container_of(work, struct nvme_fc_ctrl, ctrl.reset_work);
 
-	nvme_stop_keep_alive(&ctrl->ctrl);
+	nvme_stop_ctrl(&ctrl->ctrl);
 
 	/* will block will waiting for io to terminate */
 	nvme_fc_delete_association(ctrl);
 
-	if (ctrl->ctrl.state != NVME_CTRL_CONNECTING &&
-	    !nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
+	if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
 		dev_err(ctrl->ctrl.device,
 			"NVME-FC{%d}: error_recovery: Couldn't change state "
 			"to CONNECTING\n", ctrl->cnum);
-}
-
-static void
-nvme_fc_reset_ctrl_work(struct work_struct *work)
-{
-	struct nvme_fc_ctrl *ctrl =
-		container_of(work, struct nvme_fc_ctrl, ctrl.reset_work);
-	int ret;
-
-	__nvme_fc_terminate_io(ctrl);
 
-	nvme_stop_ctrl(&ctrl->ctrl);
-
-	if (ctrl->rport->remoteport.port_state == FC_OBJSTATE_ONLINE)
-		ret = nvme_fc_create_association(ctrl);
-	else
-		ret = -ENOTCONN;
-
-	if (ret)
-		nvme_fc_reconnect_or_delete(ctrl, ret);
-	else
-		dev_info(ctrl->ctrl.device,
-			"NVME-FC{%d}: controller reset complete\n",
-			ctrl->cnum);
+	if (ctrl->rport->remoteport.port_state == FC_OBJSTATE_ONLINE) {
+		if (!queue_delayed_work(nvme_wq, &ctrl->connect_work, 0)) {
+			dev_err(ctrl->ctrl.device,
+				"NVME-FC{%d}: failed to schedule connect "
+				"after reset\n", ctrl->cnum);
+		} else {
+			flush_delayed_work(&ctrl->connect_work);
+		}
+	} else {
+		nvme_fc_reconnect_or_delete(ctrl, -ENOTCONN);
+	}
 }
 
 
-- 
2.26.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20201023/6686cc07/attachment-0001.p7s>

From lkp at intel.com  Fri Oct 23 19:00:47 2020
From: lkp at intel.com (kernel test robot)
Date: Sat, 24 Oct 2020 07:00:47 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 f673714a1247669bc90322dfb14a5cf553833796
Message-ID: <5f93609f.sAqs90OmcCMrXE3S%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: f673714a1247669bc90322dfb14a5cf553833796  nvme-fc: shorten reconnect delay if possible for FC

elapsed time: 725m

configs tested: 185
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm                              allyesconfig
arm                              allmodconfig
arm64                               defconfig
arc                        vdk_hs38_defconfig
mips                      bmips_stb_defconfig
powerpc                      katmai_defconfig
sh                      rts7751r2d1_defconfig
nios2                            allyesconfig
powerpc                     ep8248e_defconfig
arm                            mmp2_defconfig
arm                         palmz72_defconfig
arm                          tango4_defconfig
powerpc                  mpc885_ads_defconfig
arm                         s3c6400_defconfig
mips                     loongson1c_defconfig
mips                        jmr3927_defconfig
powerpc                     ppa8548_defconfig
sh                               allmodconfig
powerpc                          g5_defconfig
arm                         s5pv210_defconfig
powerpc                     tqm8548_defconfig
sh                           se7619_defconfig
arm                         orion5x_defconfig
arm                        spear6xx_defconfig
riscv                            alldefconfig
arm                          exynos_defconfig
arm                        multi_v5_defconfig
powerpc                     ksi8560_defconfig
mips                     cu1830-neo_defconfig
powerpc                   bluestone_defconfig
powerpc                 mpc8272_ads_defconfig
arm                      tct_hammer_defconfig
c6x                              allyesconfig
powerpc                 canyonlands_defconfig
mips                        maltaup_defconfig
arm                           omap1_defconfig
arm                           spitz_defconfig
sh                           se7751_defconfig
powerpc                 linkstation_defconfig
arm                     am200epdkit_defconfig
microblaze                          defconfig
m68k                        m5307c3_defconfig
s390                             allyesconfig
powerpc                   currituck_defconfig
arm                          pxa910_defconfig
mips                           ip27_defconfig
powerpc                      pasemi_defconfig
arm                       aspeed_g5_defconfig
arm                       netwinder_defconfig
arm                        oxnas_v6_defconfig
xtensa                  cadence_csp_defconfig
arm                            zeus_defconfig
sh                           se7722_defconfig
mips                     decstation_defconfig
arm                      jornada720_defconfig
sh                          lboxre2_defconfig
sh                           se7705_defconfig
sh                          polaris_defconfig
arm                           stm32_defconfig
powerpc                         wii_defconfig
arm                    vt8500_v6_v7_defconfig
powerpc                     redwood_defconfig
sh                             sh03_defconfig
sparc64                          alldefconfig
sh                     magicpanelr2_defconfig
powerpc                    sam440ep_defconfig
xtensa                         virt_defconfig
m68k                          atari_defconfig
powerpc                      bamboo_defconfig
arm                       aspeed_g4_defconfig
arm                          moxart_defconfig
ia64                             allyesconfig
arm                          lpd270_defconfig
powerpc                   motionpro_defconfig
nios2                         10m50_defconfig
mips                          ath79_defconfig
powerpc                 mpc834x_mds_defconfig
powerpc                     akebono_defconfig
sh                           se7343_defconfig
arm                            dove_defconfig
arm                      integrator_defconfig
powerpc                 mpc836x_mds_defconfig
powerpc                  iss476-smp_defconfig
m68k                             alldefconfig
mips                      fuloong2e_defconfig
arc                 nsimosci_hs_smp_defconfig
powerpc                 mpc85xx_cds_defconfig
riscv                               defconfig
sh                        edosk7705_defconfig
powerpc                      ppc6xx_defconfig
arm                       versatile_defconfig
h8300                               defconfig
m68k                        stmark2_defconfig
powerpc                        icon_defconfig
nios2                               defconfig
powerpc                          allyesconfig
powerpc                 mpc8313_rdb_defconfig
arm                   milbeaut_m10v_defconfig
ia64                             allmodconfig
ia64                                defconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
arc                              allyesconfig
nds32                             allnoconfig
nds32                               defconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
h8300                            allyesconfig
arc                                 defconfig
xtensa                           allyesconfig
parisc                              defconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
x86_64               randconfig-a001-20201023
x86_64               randconfig-a003-20201023
x86_64               randconfig-a006-20201023
x86_64               randconfig-a004-20201023
x86_64               randconfig-a005-20201023
i386                 randconfig-a002-20201022
i386                 randconfig-a005-20201022
i386                 randconfig-a003-20201022
i386                 randconfig-a001-20201022
i386                 randconfig-a006-20201022
i386                 randconfig-a004-20201022
i386                 randconfig-a002-20201023
i386                 randconfig-a005-20201023
i386                 randconfig-a003-20201023
i386                 randconfig-a001-20201023
i386                 randconfig-a006-20201023
i386                 randconfig-a004-20201023
x86_64               randconfig-a002-20201023
x86_64               randconfig-a011-20201022
x86_64               randconfig-a013-20201022
x86_64               randconfig-a016-20201022
x86_64               randconfig-a015-20201022
x86_64               randconfig-a012-20201022
x86_64               randconfig-a014-20201022
i386                 randconfig-a016-20201022
i386                 randconfig-a014-20201022
i386                 randconfig-a015-20201022
i386                 randconfig-a012-20201022
i386                 randconfig-a013-20201022
i386                 randconfig-a011-20201022
i386                 randconfig-a016-20201023
i386                 randconfig-a014-20201023
i386                 randconfig-a015-20201023
i386                 randconfig-a012-20201023
i386                 randconfig-a013-20201023
i386                 randconfig-a011-20201023
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a001-20201022
x86_64               randconfig-a002-20201022
x86_64               randconfig-a003-20201022
x86_64               randconfig-a006-20201022
x86_64               randconfig-a004-20201022
x86_64               randconfig-a005-20201022
x86_64               randconfig-a011-20201023
x86_64               randconfig-a013-20201023
x86_64               randconfig-a016-20201023
x86_64               randconfig-a015-20201023
x86_64               randconfig-a012-20201023
x86_64               randconfig-a014-20201023

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From hch at lst.de  Sat Oct 24 03:36:21 2020
From: hch at lst.de (Christoph Hellwig)
Date: Sat, 24 Oct 2020 09:36:21 +0200
Subject: [PATCH v2] nvme-rdma: handle nvme completion data length
In-Reply-To: <d23e33e7-d545-2ad0-d163-5182dd5430b1@grimberg.me>
References: <20201023065910.1358586-1-pizhenwei@bytedance.com>
 <d23e33e7-d545-2ad0-d163-5182dd5430b1@grimberg.me>
Message-ID: <20201024073621.GA28151@lst.de>

On Fri, Oct 23, 2020 at 11:01:40AM -0700, Sagi Grimberg wrote:
>>   +	/* received data length checking */
>> +	if (unlikely(wc->byte_len < len)) {
>> +		/* zero bytes message could be ignored */
>> +		if (!wc->byte_len) {
>> +			nvme_rdma_post_recv(queue, qe);
>> +			return;
>> +		}
>
> Nothing in the spec defines zero-length messages, hence we cannot
> support something that is not standard. If your array needs this,
> please submit a TPAR to the NVMe TWG.

We'll still need sanity checking instead of the recount underflow,
I think tearing down the connection here and kicking off error recovery
is probably the best idea.


From pizhenwei at bytedance.com  Sun Oct 25 07:51:24 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Sun, 25 Oct 2020 19:51:24 +0800
Subject: [PATCH v3] nvme-rdma: handle nvme completion data length
Message-ID: <20201025115124.1430678-1-pizhenwei@bytedance.com>

Hit a kernel warning:
refcount_t: underflow; use-after-free.
WARNING: CPU: 0 PID: 0 at lib/refcount.c:28

RIP: 0010:refcount_warn_saturate+0xd9/0xe0
Call Trace:
 <IRQ>
 nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
 __ib_process_cq+0x76/0x150 [ib_core]
 ...

The reason is that a zero bytes message received from target, and the
host side continues to process without length checking, then the
previous CQE is processed twice.

Do sanity check on received data length, try to recovery for corrupted
CQE case.

Because zero bytes message in not defined in spec, using zero bytes
message to detect dead connections on transport layer is not
standard, currently still treat it as illegal.

Thanks to Chao Leng & Sagi for suggestions.

Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/host/rdma.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index aad829a2b50d..40a0a3b6476c 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1768,6 +1768,14 @@ static void nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc)
 		return;
 	}
 
+	/* sanity checking for received data length */
+	if (unlikely(wc->byte_len < len)) {
+		dev_err(queue->ctrl->ctrl.device,
+			"Unexpected nvme completion length(%d)\n", wc->byte_len);
+		nvme_rdma_error_recovery(queue->ctrl);
+		return;
+	}
+
 	ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
 	/*
 	 * AEN requests are special as they don't time out and can
-- 
2.11.0



From sagi at grimberg.me  Mon Oct 26 03:40:06 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 26 Oct 2020 00:40:06 -0700
Subject: [PATCH v3] nvme-rdma: handle nvme completion data length
In-Reply-To: <20201025115124.1430678-1-pizhenwei@bytedance.com>
References: <20201025115124.1430678-1-pizhenwei@bytedance.com>
Message-ID: <146e7680-264e-7b70-0a5c-a6e7dd520a2d@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From guoqing.jiang at cloud.ionos.com  Mon Oct 26 04:19:37 2020
From: guoqing.jiang at cloud.ionos.com (Guoqing Jiang)
Date: Mon, 26 Oct 2020 09:19:37 +0100
Subject: Lock issue with 2a7cec53816 ("RDMA/cma: Fix locking for the
 RDMA_CM_CONNECT state")
Message-ID: <3b1f7767-98e2-93e0-b718-16d1c5346140@cloud.ionos.com>

Hi Jason & Leon,


[Sorry to send again since the previous mail didn't hit the list]

With the mentioned commit, I got the below calltraces.

[ 1209.544359] INFO: task kworker/u8:3:615 blocked for more than 120 
seconds.
[ 1209.545170]?????? Tainted: G?????????? O????? 5.9.0-rc8+ #30
[ 1209.545341] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 1209.545814] task:kworker/u8:3??? state:D stack:??? 0 pid:? 615 
ppid:???? 2 flags:0x00004000
[ 1209.547601] Workqueue: rdma_cm cma_work_handler [rdma_cm]
[ 1209.548645] Call Trace:
[ 1209.552120]? __schedule+0x363/0x7b0
[ 1209.552903]? schedule+0x44/0xb0
[ 1209.553093]? schedule_preempt_disabled+0xe/0x10
[ 1209.553253]? __mutex_lock.isra.10+0x26d/0x4e0
[ 1209.553421]? __mutex_lock_slowpath+0x13/0x20
[ 1209.553596]? ? __mutex_lock_slowpath+0x13/0x20
[ 1209.553751]? mutex_lock+0x2f/0x40
[ 1209.554045]? rdma_connect+0x52/0x770 [rdma_cm]
[ 1209.554215]? ? vprintk_default+0x1f/0x30
[ 1209.554359]? ? vprintk_func+0x62/0x100
[ 1209.554532]? ? cma_cm_event_handler+0x2b/0xe0 [rdma_cm]
[ 1209.554688]? ? printk+0x52/0x6e
[ 1209.554852]? rtrs_clt_rdma_cm_handler+0x393/0x8b0 [rtrs_client]
[ 1209.555056]? cma_cm_event_handler+0x2b/0xe0 [rdma_cm]
[ 1209.561823]? ? cma_comp_exch+0x4e/0x60 [rdma_cm]
[ 1209.561883]? ? cma_cm_event_handler+0x2b/0xe0 [rdma_cm]
[ 1209.561940]? cma_work_handler+0x89/0xc0 [rdma_cm]
[ 1209.561988]? process_one_work+0x20c/0x400
[...]
[ 1209.562323] INFO: task bash:630 blocked for more than 120 seconds.
[ 1209.562368]?????? Tainted: G?????????? O????? 5.9.0-rc8+ #30
[ 1209.562404] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 1209.562456] task:bash??????????? state:D stack:??? 0 pid:? 630 
ppid:?? 618 flags:0x00004000
[ 1209.562515] Call Trace:
[ 1209.562559]? __schedule+0x363/0x7b0
[ 1209.562603]? schedule+0x44/0xb0
[ 1209.562642]? schedule_preempt_disabled+0xe/0x10
[ 1209.562685]? __mutex_lock.isra.10+0x26d/0x4e0
[ 1209.562727]? ? vprintk_func+0x62/0x100
[ 1209.562765]? ? vprintk_default+0x1f/0x30
[ 1209.562806]? __mutex_lock_slowpath+0x13/0x20
[ 1209.562847]? ? __mutex_lock_slowpath+0x13/0x20
[ 1209.562887]? mutex_lock+0x2f/0x40
[ 1209.562936]? rdma_destroy_id+0x31/0x60 [rdma_cm]
[ 1209.562983]? destroy_cm.isra.24+0x2d/0x50 [rtrs_client]
[ 1209.563586]? init_sess+0x23b/0xac0 [rtrs_client]
[ 1209.563691]? ? wait_woken+0x80/0x80
[ 1209.563742]? rtrs_clt_open+0x27f/0x540 [rtrs_client]
[ 1209.563791]? ? remap_devs+0x150/0x150 [rnbd_client]
[ 1209.563867]? find_and_get_or_create_sess+0x4a1/0x5e0 [rnbd_client]
[ 1209.563921]? ? remap_devs+0x150/0x150 [rnbd_client]
[ 1209.563966]? ? atomic_notifier_call_chain+0x1a/0x20
[ 1209.564011]? ? vt_console_print+0x203/0x3d0
[ 1209.564051]? ? up+0x32/0x50
[ 1209.564093]? ? __irq_work_queue_local+0x4f/0x60
[ 1209.564132]? ? irq_work_queue+0x1a/0x30
[ 1209.564169]? ? wake_up_klogd.part.28+0x34/0x40
[ 1209.564208]? ? vprintk_emit+0x126/0x2b0
[ 1209.564249]? ? vprintk_default+0x1f/0x30
[ 1209.564292]? rnbd_clt_map_device+0x74/0x7b0 [rnbd_client]
[ 1209.564341]? rnbd_clt_map_device_store+0x39e/0x6c0 [rnbd_client]

 From my understanding, it is caused by the rdma_connect need to hold 
the handler_mutex
again in the path.

cma_ib_handler -> mutex_lock(&id_priv->handler_mutex)
-> cma_cm_event_handler
 ??? ??? ??? ??? ??? -> id_priv->id.event_handler = rtrs_clt_rdma_cm_handler
-> rdma_connect
-> mutex_lock(&id_priv->handler_mutex)


And seems nvme rdma could have the same issue since rdma_connect is 
called by
nvme_rdma_cm_handler -> nvme_rdma_route_resolved.

Thanks,
Guoqing


From haris.iqbal at cloud.ionos.com  Mon Oct 26 06:27:53 2020
From: haris.iqbal at cloud.ionos.com (Haris Iqbal)
Date: Mon, 26 Oct 2020 15:57:53 +0530
Subject: Lock issue with 2a7cec53816 ("RDMA/cma: Fix locking for the
 RDMA_CM_CONNECT state")
In-Reply-To: <3b1f7767-98e2-93e0-b718-16d1c5346140@cloud.ionos.com>
References: <3b1f7767-98e2-93e0-b718-16d1c5346140@cloud.ionos.com>
Message-ID: <CAJpMwyhONwT6mYzOazjuWS01AVSd4bu=K33-LYk=ZygP021V4w@mail.gmail.com>

On Mon, Oct 26, 2020 at 1:50 PM Guoqing Jiang
<guoqing.jiang at cloud.ionos.com> wrote:
>
> Hi Jason & Leon,
>
>
> [Sorry to send again since the previous mail didn't hit the list]
>
> With the mentioned commit, I got the below calltraces.
>
> [ 1209.544359] INFO: task kworker/u8:3:615 blocked for more than 120
> seconds.
> [ 1209.545170]       Tainted: G           O      5.9.0-rc8+ #30
> [ 1209.545341] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 1209.545814] task:kworker/u8:3    state:D stack:    0 pid:  615
> ppid:     2 flags:0x00004000
> [ 1209.547601] Workqueue: rdma_cm cma_work_handler [rdma_cm]
> [ 1209.548645] Call Trace:
> [ 1209.552120]  __schedule+0x363/0x7b0
> [ 1209.552903]  schedule+0x44/0xb0
> [ 1209.553093]  schedule_preempt_disabled+0xe/0x10
> [ 1209.553253]  __mutex_lock.isra.10+0x26d/0x4e0
> [ 1209.553421]  __mutex_lock_slowpath+0x13/0x20
> [ 1209.553596]  ? __mutex_lock_slowpath+0x13/0x20
> [ 1209.553751]  mutex_lock+0x2f/0x40
> [ 1209.554045]  rdma_connect+0x52/0x770 [rdma_cm]
> [ 1209.554215]  ? vprintk_default+0x1f/0x30
> [ 1209.554359]  ? vprintk_func+0x62/0x100
> [ 1209.554532]  ? cma_cm_event_handler+0x2b/0xe0 [rdma_cm]
> [ 1209.554688]  ? printk+0x52/0x6e
> [ 1209.554852]  rtrs_clt_rdma_cm_handler+0x393/0x8b0 [rtrs_client]
> [ 1209.555056]  cma_cm_event_handler+0x2b/0xe0 [rdma_cm]
> [ 1209.561823]  ? cma_comp_exch+0x4e/0x60 [rdma_cm]
> [ 1209.561883]  ? cma_cm_event_handler+0x2b/0xe0 [rdma_cm]
> [ 1209.561940]  cma_work_handler+0x89/0xc0 [rdma_cm]
> [ 1209.561988]  process_one_work+0x20c/0x400
> [...]
> [ 1209.562323] INFO: task bash:630 blocked for more than 120 seconds.
> [ 1209.562368]       Tainted: G           O      5.9.0-rc8+ #30
> [ 1209.562404] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 1209.562456] task:bash            state:D stack:    0 pid:  630
> ppid:   618 flags:0x00004000
> [ 1209.562515] Call Trace:
> [ 1209.562559]  __schedule+0x363/0x7b0
> [ 1209.562603]  schedule+0x44/0xb0
> [ 1209.562642]  schedule_preempt_disabled+0xe/0x10
> [ 1209.562685]  __mutex_lock.isra.10+0x26d/0x4e0
> [ 1209.562727]  ? vprintk_func+0x62/0x100
> [ 1209.562765]  ? vprintk_default+0x1f/0x30
> [ 1209.562806]  __mutex_lock_slowpath+0x13/0x20
> [ 1209.562847]  ? __mutex_lock_slowpath+0x13/0x20
> [ 1209.562887]  mutex_lock+0x2f/0x40
> [ 1209.562936]  rdma_destroy_id+0x31/0x60 [rdma_cm]
> [ 1209.562983]  destroy_cm.isra.24+0x2d/0x50 [rtrs_client]
> [ 1209.563586]  init_sess+0x23b/0xac0 [rtrs_client]
> [ 1209.563691]  ? wait_woken+0x80/0x80
> [ 1209.563742]  rtrs_clt_open+0x27f/0x540 [rtrs_client]
> [ 1209.563791]  ? remap_devs+0x150/0x150 [rnbd_client]
> [ 1209.563867]  find_and_get_or_create_sess+0x4a1/0x5e0 [rnbd_client]
> [ 1209.563921]  ? remap_devs+0x150/0x150 [rnbd_client]
> [ 1209.563966]  ? atomic_notifier_call_chain+0x1a/0x20
> [ 1209.564011]  ? vt_console_print+0x203/0x3d0
> [ 1209.564051]  ? up+0x32/0x50
> [ 1209.564093]  ? __irq_work_queue_local+0x4f/0x60
> [ 1209.564132]  ? irq_work_queue+0x1a/0x30
> [ 1209.564169]  ? wake_up_klogd.part.28+0x34/0x40
> [ 1209.564208]  ? vprintk_emit+0x126/0x2b0
> [ 1209.564249]  ? vprintk_default+0x1f/0x30
> [ 1209.564292]  rnbd_clt_map_device+0x74/0x7b0 [rnbd_client]
> [ 1209.564341]  rnbd_clt_map_device_store+0x39e/0x6c0 [rnbd_client]
>
>  From my understanding, it is caused by the rdma_connect need to hold
> the handler_mutex
> again in the path.
>
> cma_ib_handler -> mutex_lock(&id_priv->handler_mutex)
> -> cma_cm_event_handler
>                      -> id_priv->id.event_handler = rtrs_clt_rdma_cm_handler
> -> rdma_connect
> -> mutex_lock(&id_priv->handler_mutex)

I think its coming from

cma_work_handler() -> cma_cm_event_handler() ->
id_priv->id.event_handler = rtrs_clt_rdma_cm_handler
rtrs_rdma_route_resolved
-> rdma_connect tries to get the same lock again
                                       -> mutex_lock(&id_priv->handler_mutex)

Since, cma_ib_handler() doesn't handle and forward the
RDMA_CM_EVENT_ROUTE_RESOLVED event.
And cma_work_handler() also holds the mutex before calling
cma_cm_event_handler()

It probably starts with,

After RDMA_CM_EVENT_ADDR_RESOLVED comes to rtrs_clt_rdma_cm_handler,
rdma_resolve_route() -> cma_resolve_ib_route() -> cma_query_ib_route() --> ...
(callback) cma_work_handler()

>
>
> And seems nvme rdma could have the same issue since rdma_connect is
> called by
> nvme_rdma_cm_handler -> nvme_rdma_route_resolved.
>
> Thanks,
> Guoqing



-- 

Regards
-Haris


From jgg at nvidia.com  Mon Oct 26 10:25:49 2020
From: jgg at nvidia.com (Jason Gunthorpe)
Date: Mon, 26 Oct 2020 11:25:49 -0300
Subject: [PATCH] RDMA: Add rdma_connect_locked()
Message-ID: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>

There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
handler triggers a completion and another thread does rdma_connect() or
the handler directly calls rdma_connect().

In all cases rdma_connect() needs to hold the handler_mutex, but when
handler's are invoked this is already held by the core code. This causes
ULPs using the 2nd method to deadlock.

Provide a rdma_connect_locked() and have all ULPs call it from their
handlers.

Reported-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state"
Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
---
 drivers/infiniband/core/cma.c            | 39 +++++++++++++++++++++---
 drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
 drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
 drivers/nvme/host/rdma.c                 | 10 +++---
 include/rdma/rdma_cm.h                   | 13 +-------
 net/rds/ib_cm.c                          |  5 +--
 6 files changed, 47 insertions(+), 26 deletions(-)

Seems people are not testing these four ULPs against rdma-next.. Here is a
quick fix for the issue:

https://lore.kernel.org/r/3b1f7767-98e2-93e0-b718-16d1c5346140 at cloud.ionos.com

Jason

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 7c2ab1f2fbea37..2eaaa1292fb847 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv,
 	/*
 	 * The FSM uses a funny double locking where state is protected by both
 	 * the handler_mutex and the spinlock. State is not allowed to change
-	 * away from a handler_mutex protected value without also holding
+	 * to/from a handler_mutex protected value without also holding
 	 * handler_mutex.
 	 */
-	if (comp == RDMA_CM_CONNECT)
+	if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT)
 		lockdep_assert_held(&id_priv->handler_mutex);
 
 	spin_lock_irqsave(&id_priv->lock, flags);
@@ -4038,13 +4038,20 @@ static int cma_connect_iw(struct rdma_id_private *id_priv,
 	return ret;
 }
 
-int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+/**
+ * rdma_connect_locked - Initiate an active connection request.
+ * @id: Connection identifier to connect.
+ * @conn_param: Connection information used for connected QPs.
+ *
+ * Same as rdma_connect() but can only be called from the
+ * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback.
+ */
+int rdma_connect_locked(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
 {
 	struct rdma_id_private *id_priv =
 		container_of(id, struct rdma_id_private, id);
 	int ret;
 
-	mutex_lock(&id_priv->handler_mutex);
 	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
 		ret = -EINVAL;
 		goto err_unlock;
@@ -4071,6 +4078,30 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
 err_state:
 	cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);
 err_unlock:
+	return ret;
+}
+EXPORT_SYMBOL(rdma_connect_locked);
+
+/**
+ * rdma_connect - Initiate an active connection request.
+ * @id: Connection identifier to connect.
+ * @conn_param: Connection information used for connected QPs.
+ *
+ * Users must have resolved a route for the rdma_cm_id to connect with by having
+ * called rdma_resolve_route before calling this routine.
+ *
+ * This call will either connect to a remote QP or obtain remote QP information
+ * for unconnected rdma_cm_id's.  The actual operation is based on the
+ * rdma_cm_id's port space.
+ */
+int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+{
+	struct rdma_id_private *id_priv =
+		container_of(id, struct rdma_id_private, id);
+	int ret;
+
+	mutex_lock(&id_priv->handler_mutex);
+	ret = rdma_connect_locked(id, conn_param);
 	mutex_unlock(&id_priv->handler_mutex);
 	return ret;
 }
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 2f3ebc0a75d924..2bd18b00689341 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -620,7 +620,7 @@ static void iser_route_handler(struct rdma_cm_id *cma_id)
 	conn_param.private_data	= (void *)&req_hdr;
 	conn_param.private_data_len = sizeof(struct iser_cm_hdr);
 
-	ret = rdma_connect(cma_id, &conn_param);
+	ret = rdma_connect_locked(cma_id, &conn_param);
 	if (ret) {
 		iser_err("failure connecting: %d\n", ret);
 		goto failure;
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index 776e89231c52f7..f298adc02acba2 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -1674,9 +1674,9 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con)
 	uuid_copy(&msg.sess_uuid, &sess->s.uuid);
 	uuid_copy(&msg.paths_uuid, &clt->paths_uuid);
 
-	err = rdma_connect(con->c.cm_id, &param);
+	err = rdma_connect_locked(con->c.cm_id, &param);
 	if (err)
-		rtrs_err(clt, "rdma_connect(): %d\n", err);
+		rtrs_err(clt, "rdma_connect_locked(): %d\n", err);
 
 	return err;
 }
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index aad829a2b50d0f..f488dc5f4c2c61 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1730,11 +1730,10 @@ static void nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
 	req->result = cqe->result;
 
 	if (wc->wc_flags & IB_WC_WITH_INVALIDATE) {
-		if (unlikely(!req->mr ||
-			     wc->ex.invalidate_rkey != req->mr->rkey)) {
+		if (unlikely(wc->ex.invalidate_rkey != req->mr->rkey)) {
 			dev_err(queue->ctrl->ctrl.device,
 				"Bogus remote invalidation for rkey %#x\n",
-				req->mr ? req->mr->rkey : 0);
+				req->mr->rkey);
 			nvme_rdma_error_recovery(queue->ctrl);
 		}
 	} else if (req->mr) {
@@ -1890,10 +1889,10 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue)
 		priv.hsqsize = cpu_to_le16(queue->ctrl->ctrl.sqsize);
 	}
 
-	ret = rdma_connect(queue->cm_id, &param);
+	ret = rdma_connect_locked(queue->cm_id, &param);
 	if (ret) {
 		dev_err(ctrl->ctrl.device,
-			"rdma_connect failed (%d).\n", ret);
+			"rdma_connect_locked failed (%d).\n", ret);
 		goto out_destroy_queue_ib;
 	}
 
@@ -1927,6 +1926,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
 		complete(&queue->cm_done);
 		return 0;
 	case RDMA_CM_EVENT_REJECTED:
+		nvme_rdma_destroy_queue_ib(queue);
 		cm_error = nvme_rdma_conn_rejected(queue, ev);
 		break;
 	case RDMA_CM_EVENT_ROUTE_ERROR:
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index c672ae1da26bb5..937d55611cd073 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -227,19 +227,8 @@ void rdma_destroy_qp(struct rdma_cm_id *id);
 int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
 		       int *qp_attr_mask);
 
-/**
- * rdma_connect - Initiate an active connection request.
- * @id: Connection identifier to connect.
- * @conn_param: Connection information used for connected QPs.
- *
- * Users must have resolved a route for the rdma_cm_id to connect with
- * by having called rdma_resolve_route before calling this routine.
- *
- * This call will either connect to a remote QP or obtain remote QP
- * information for unconnected rdma_cm_id's.  The actual operation is
- * based on the rdma_cm_id's port space.
- */
 int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+int rdma_connect_locked(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
 
 int rdma_connect_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param,
 		     struct rdma_ucm_ece *ece);
diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
index 06603dd1c8aa38..b36b60668b1da9 100644
--- a/net/rds/ib_cm.c
+++ b/net/rds/ib_cm.c
@@ -956,9 +956,10 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id, bool isv6)
 	rds_ib_cm_fill_conn_param(conn, &conn_param, &dp,
 				  conn->c_proposed_version,
 				  UINT_MAX, UINT_MAX, isv6);
-	ret = rdma_connect(cm_id, &conn_param);
+	ret = rdma_connect_locked(cm_id, &conn_param);
 	if (ret)
-		rds_ib_conn_error(conn, "rdma_connect failed (%d)\n", ret);
+		rds_ib_conn_error(conn, "rdma_connect_locked failed (%d)\n",
+				  ret);
 
 out:
 	/* Beware - returning non-zero tells the rdma_cm to destroy
-- 
2.28.0



From santosh.shilimkar at oracle.com  Mon Oct 26 12:01:20 2020
From: santosh.shilimkar at oracle.com (santosh.shilimkar at oracle.com)
Date: Mon, 26 Oct 2020 09:01:20 -0700
Subject: [PATCH] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
References: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <ed68ad93-602e-c617-87e4-a713856478a0@oracle.com>



On 10/26/20 7:25 AM, Jason Gunthorpe wrote:
> There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
> handler triggers a completion and another thread does rdma_connect() or
> the handler directly calls rdma_connect().
> 
> In all cases rdma_connect() needs to hold the handler_mutex, but when
> handler's are invoked this is already held by the core code. This causes
> ULPs using the 2nd method to deadlock.
> 
> Provide a rdma_connect_locked() and have all ULPs call it from their
> handlers.
> 
> Reported-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
> Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state"
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> ---

[....]

> diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
> index 06603dd1c8aa38..b36b60668b1da9 100644
> --- a/net/rds/ib_cm.c
> +++ b/net/rds/ib_cm.c
> @@ -956,9 +956,10 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id, bool isv6)
>   	rds_ib_cm_fill_conn_param(conn, &conn_param, &dp,
>   				  conn->c_proposed_version,
>   				  UINT_MAX, UINT_MAX, isv6);
> -	ret = rdma_connect(cm_id, &conn_param);
> +	ret = rdma_connect_locked(cm_id, &conn_param);
>   	if (ret)
> -		rds_ib_conn_error(conn, "rdma_connect failed (%d)\n", ret);
> +		rds_ib_conn_error(conn, "rdma_connect_locked failed (%d)\n",
> +				  ret);
>   
>   out:
>   	/* Beware - returning non-zero tells the rdma_cm to destroy
> 
For RDS part,
Acked-by: Santosh Shilimkar <santosh.shilimkar at oracle.com>


From Chaitanya.Kulkarni at wdc.com  Mon Oct 26 17:02:53 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon, 26 Oct 2020 21:02:53 +0000
Subject: [PATCH] nvme-cli: remove default ctrl_loss_tmo for connect
References: <20201023202521.9610-1-chaitanya.kulkarni@wdc.com>
 <5c212507-d518-5343-3836-93107cfea53f@grimberg.me>
Message-ID: <BYAPR04MB4965DB3544F102E75DE05DB486190@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/23/20 14:16, Sagi Grimberg wrote:
>> diff --git a/fabrics.c b/fabrics.c
>> index f8a055b..9c33c02 100644
>> --- a/fabrics.c
>> +++ b/fabrics.c
>> @@ -896,8 +896,6 @@ static int build_options(char *argstr, int max_len, bool discover)
>>   				cfg.keep_alive_tmo, false)) ||
>>   	    add_int_argument(&argstr, &max_len, "reconnect_delay",
>>   				cfg.reconnect_delay, false) ||
>> -	    add_int_argument(&argstr, &max_len, "ctrl_loss_tmo",
>> -				cfg.ctrl_loss_tmo, true) ||
> You just removed the entire option altogether? the user can no longer
> pass ctrl_loss_tmo now. You should instead disallow zero back (i.e.
> revert the offending patch that made it as such) and figure out
> a proper way to allow ctrl_loss_tmo=0 functionality.
>
Thanks for the comments will send out the V2.



From sashal at kernel.org  Mon Oct 26 19:52:04 2020
From: sashal at kernel.org (Sasha Levin)
Date: Mon, 26 Oct 2020 19:52:04 -0400
Subject: [PATCH AUTOSEL 5.8 132/132] nvme-rdma: fix crash when connect rejected
In-Reply-To: <20201026235205.1023962-1-sashal@kernel.org>
References: <20201026235205.1023962-1-sashal@kernel.org>
Message-ID: <20201026235205.1023962-132-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 43efdb8e870ee0f58633fd579aa5b5185bf5d39e ]

A crash can happened when a connect is rejected.   The host establishes
the connection after received ConnectReply, and then continues to send
the fabrics Connect command.  If the controller does not receive the
ReadyToUse capsule, host may receive a ConnectReject reply.

Call nvme_rdma_destroy_queue_ib after the host received the
RDMA_CM_EVENT_REJECTED event.  Then when the fabrics Connect command
times out, nvme_rdma_timeout calls nvme_rdma_complete_rq to fail the
request.  A crash happenes due to use after free in
nvme_rdma_complete_rq.

nvme_rdma_destroy_queue_ib is redundant when handling the
RDMA_CM_EVENT_REJECTED event as nvme_rdma_destroy_queue_ib is already
called in connection failure handler.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/rdma.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 4a0bc8927048a..99f627b0053f7 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1900,7 +1900,6 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
 		complete(&queue->cm_done);
 		return 0;
 	case RDMA_CM_EVENT_REJECTED:
-		nvme_rdma_destroy_queue_ib(queue);
 		cm_error = nvme_rdma_conn_rejected(queue, ev);
 		break;
 	case RDMA_CM_EVENT_ROUTE_ERROR:
-- 
2.25.1



From sashal at kernel.org  Mon Oct 26 19:55:16 2020
From: sashal at kernel.org (Sasha Levin)
Date: Mon, 26 Oct 2020 19:55:16 -0400
Subject: [PATCH AUTOSEL 5.4 80/80] nvme-rdma: fix crash when connect rejected
In-Reply-To: <20201026235516.1025100-1-sashal@kernel.org>
References: <20201026235516.1025100-1-sashal@kernel.org>
Message-ID: <20201026235516.1025100-80-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 43efdb8e870ee0f58633fd579aa5b5185bf5d39e ]

A crash can happened when a connect is rejected.   The host establishes
the connection after received ConnectReply, and then continues to send
the fabrics Connect command.  If the controller does not receive the
ReadyToUse capsule, host may receive a ConnectReject reply.

Call nvme_rdma_destroy_queue_ib after the host received the
RDMA_CM_EVENT_REJECTED event.  Then when the fabrics Connect command
times out, nvme_rdma_timeout calls nvme_rdma_complete_rq to fail the
request.  A crash happenes due to use after free in
nvme_rdma_complete_rq.

nvme_rdma_destroy_queue_ib is redundant when handling the
RDMA_CM_EVENT_REJECTED event as nvme_rdma_destroy_queue_ib is already
called in connection failure handler.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/rdma.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index abe4fe496d05c..a41ee9feab8e7 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1679,7 +1679,6 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
 		complete(&queue->cm_done);
 		return 0;
 	case RDMA_CM_EVENT_REJECTED:
-		nvme_rdma_destroy_queue_ib(queue);
 		cm_error = nvme_rdma_conn_rejected(queue, ev);
 		break;
 	case RDMA_CM_EVENT_ROUTE_ERROR:
-- 
2.25.1



From sashal at kernel.org  Mon Oct 26 20:04:15 2020
From: sashal at kernel.org (Sasha Levin)
Date: Mon, 26 Oct 2020 20:04:15 -0400
Subject: [PATCH AUTOSEL 4.19 60/60] nvme-rdma: fix crash when connect rejected
In-Reply-To: <20201027000415.1026364-1-sashal@kernel.org>
References: <20201027000415.1026364-1-sashal@kernel.org>
Message-ID: <20201027000415.1026364-60-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 43efdb8e870ee0f58633fd579aa5b5185bf5d39e ]

A crash can happened when a connect is rejected.   The host establishes
the connection after received ConnectReply, and then continues to send
the fabrics Connect command.  If the controller does not receive the
ReadyToUse capsule, host may receive a ConnectReject reply.

Call nvme_rdma_destroy_queue_ib after the host received the
RDMA_CM_EVENT_REJECTED event.  Then when the fabrics Connect command
times out, nvme_rdma_timeout calls nvme_rdma_complete_rq to fail the
request.  A crash happenes due to use after free in
nvme_rdma_complete_rq.

nvme_rdma_destroy_queue_ib is redundant when handling the
RDMA_CM_EVENT_REJECTED event as nvme_rdma_destroy_queue_ib is already
called in connection failure handler.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/rdma.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 077c678166651..134e14e778f8e 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1640,7 +1640,6 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
 		complete(&queue->cm_done);
 		return 0;
 	case RDMA_CM_EVENT_REJECTED:
-		nvme_rdma_destroy_queue_ib(queue);
 		cm_error = nvme_rdma_conn_rejected(queue, ev);
 		break;
 	case RDMA_CM_EVENT_ROUTE_ERROR:
-- 
2.25.1



From sashal at kernel.org  Mon Oct 26 20:09:45 2020
From: sashal at kernel.org (Sasha Levin)
Date: Mon, 26 Oct 2020 20:09:45 -0400
Subject: [PATCH AUTOSEL 4.14 46/46] nvme-rdma: fix crash when connect rejected
In-Reply-To: <20201027000946.1026923-1-sashal@kernel.org>
References: <20201027000946.1026923-1-sashal@kernel.org>
Message-ID: <20201027000946.1026923-46-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 43efdb8e870ee0f58633fd579aa5b5185bf5d39e ]

A crash can happened when a connect is rejected.   The host establishes
the connection after received ConnectReply, and then continues to send
the fabrics Connect command.  If the controller does not receive the
ReadyToUse capsule, host may receive a ConnectReject reply.

Call nvme_rdma_destroy_queue_ib after the host received the
RDMA_CM_EVENT_REJECTED event.  Then when the fabrics Connect command
times out, nvme_rdma_timeout calls nvme_rdma_complete_rq to fail the
request.  A crash happenes due to use after free in
nvme_rdma_complete_rq.

nvme_rdma_destroy_queue_ib is redundant when handling the
RDMA_CM_EVENT_REJECTED event as nvme_rdma_destroy_queue_ib is already
called in connection failure handler.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/rdma.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index c91bfd839cabe..564e457f1345e 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1545,7 +1545,6 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
 		complete(&queue->cm_done);
 		return 0;
 	case RDMA_CM_EVENT_REJECTED:
-		nvme_rdma_destroy_queue_ib(queue);
 		cm_error = nvme_rdma_conn_rejected(queue, ev);
 		break;
 	case RDMA_CM_EVENT_ROUTE_ERROR:
-- 
2.25.1



From lengchao at huawei.com  Mon Oct 26 22:01:00 2020
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 27 Oct 2020 10:01:00 +0800
Subject: [PATCH] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
References: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <e13ec119-3cc6-87ab-bc76-d2d3de7631e4@huawei.com>



On 2020/10/26 22:25, Jason Gunthorpe wrote:
> There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
> handler triggers a completion and another thread does rdma_connect() or
> the handler directly calls rdma_connect().
> 
> In all cases rdma_connect() needs to hold the handler_mutex, but when
> handler's are invoked this is already held by the core code. This causes
> ULPs using the 2nd method to deadlock.
> 
> Provide a rdma_connect_locked() and have all ULPs call it from their
> handlers.
> 
> Reported-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
> Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state"
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> ---
>   drivers/infiniband/core/cma.c            | 39 +++++++++++++++++++++---
>   drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
>   drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
>   drivers/nvme/host/rdma.c                 | 10 +++---
>   include/rdma/rdma_cm.h                   | 13 +-------
>   net/rds/ib_cm.c                          |  5 +--
>   6 files changed, 47 insertions(+), 26 deletions(-)
> 
> Seems people are not testing these four ULPs against rdma-next.. Here is a
> quick fix for the issue:
> 
> https://lore.kernel.org/r/3b1f7767-98e2-93e0-b718-16d1c5346140 at cloud.ionos.com
> 
> Jason
> 
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 7c2ab1f2fbea37..2eaaa1292fb847 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv,
>   	/*
>   	 * The FSM uses a funny double locking where state is protected by both
>   	 * the handler_mutex and the spinlock. State is not allowed to change
> -	 * away from a handler_mutex protected value without also holding
> +	 * to/from a handler_mutex protected value without also holding
>   	 * handler_mutex.
>   	 */
> -	if (comp == RDMA_CM_CONNECT)
> +	if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT)
>   		lockdep_assert_held(&id_priv->handler_mutex);
>   
>   	spin_lock_irqsave(&id_priv->lock, flags);
> @@ -4038,13 +4038,20 @@ static int cma_connect_iw(struct rdma_id_private *id_priv,
>   	return ret;
>   }
>   
> -int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> +/**
> + * rdma_connect_locked - Initiate an active connection request.
> + * @id: Connection identifier to connect.
> + * @conn_param: Connection information used for connected QPs.
> + *
> + * Same as rdma_connect() but can only be called from the
> + * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback.
> + */
> +int rdma_connect_locked(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>   {
>   	struct rdma_id_private *id_priv =
>   		container_of(id, struct rdma_id_private, id);
>   	int ret;
>   
> -	mutex_lock(&id_priv->handler_mutex);
>   	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
>   		ret = -EINVAL;
>   		goto err_unlock;
> @@ -4071,6 +4078,30 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>   err_state:
>   	cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);
>   err_unlock:
> +	return ret;
> +}
> +EXPORT_SYMBOL(rdma_connect_locked);
> +
> +/**
> + * rdma_connect - Initiate an active connection request.
> + * @id: Connection identifier to connect.
> + * @conn_param: Connection information used for connected QPs.
> + *
> + * Users must have resolved a route for the rdma_cm_id to connect with by having
> + * called rdma_resolve_route before calling this routine.
> + *
> + * This call will either connect to a remote QP or obtain remote QP information
> + * for unconnected rdma_cm_id's.  The actual operation is based on the
> + * rdma_cm_id's port space.
> + */
> +int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> +{
> +	struct rdma_id_private *id_priv =
> +		container_of(id, struct rdma_id_private, id);
> +	int ret;
> +
> +	mutex_lock(&id_priv->handler_mutex);
> +	ret = rdma_connect_locked(id, conn_param);
>   	mutex_unlock(&id_priv->handler_mutex);
>   	return ret;
>   }
> diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
> index 2f3ebc0a75d924..2bd18b00689341 100644
> --- a/drivers/infiniband/ulp/iser/iser_verbs.c
> +++ b/drivers/infiniband/ulp/iser/iser_verbs.c
> @@ -620,7 +620,7 @@ static void iser_route_handler(struct rdma_cm_id *cma_id)
>   	conn_param.private_data	= (void *)&req_hdr;
>   	conn_param.private_data_len = sizeof(struct iser_cm_hdr);
>   
> -	ret = rdma_connect(cma_id, &conn_param);
> +	ret = rdma_connect_locked(cma_id, &conn_param);
>   	if (ret) {
>   		iser_err("failure connecting: %d\n", ret);
>   		goto failure;
> diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> index 776e89231c52f7..f298adc02acba2 100644
> --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> @@ -1674,9 +1674,9 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con)
>   	uuid_copy(&msg.sess_uuid, &sess->s.uuid);
>   	uuid_copy(&msg.paths_uuid, &clt->paths_uuid);
>   
> -	err = rdma_connect(con->c.cm_id, &param);
> +	err = rdma_connect_locked(con->c.cm_id, &param);
>   	if (err)
> -		rtrs_err(clt, "rdma_connect(): %d\n", err);
> +		rtrs_err(clt, "rdma_connect_locked(): %d\n", err);
>   
>   	return err;
>   }
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index aad829a2b50d0f..f488dc5f4c2c61 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1730,11 +1730,10 @@ static void nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
>   	req->result = cqe->result;
>   
>   	if (wc->wc_flags & IB_WC_WITH_INVALIDATE) {
> -		if (unlikely(!req->mr ||
> -			     wc->ex.invalidate_rkey != req->mr->rkey)) {
> +		if (unlikely(wc->ex.invalidate_rkey != req->mr->rkey)) {
>   			dev_err(queue->ctrl->ctrl.device,
>   				"Bogus remote invalidation for rkey %#x\n",
> -				req->mr ? req->mr->rkey : 0);
> +				req->mr->rkey);
Maybe the code version is incorrect, cause falsely code rollback.
>   			nvme_rdma_error_recovery(queue->ctrl);
>   		}
>   	} else if (req->mr) {
> @@ -1890,10 +1889,10 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue)
>   		priv.hsqsize = cpu_to_le16(queue->ctrl->ctrl.sqsize);
>   	}
>   
> -	ret = rdma_connect(queue->cm_id, &param);
> +	ret = rdma_connect_locked(queue->cm_id, &param);
>   	if (ret) {
>   		dev_err(ctrl->ctrl.device,
> -			"rdma_connect failed (%d).\n", ret);
> +			"rdma_connect_locked failed (%d).\n", ret);
>   		goto out_destroy_queue_ib;
>   	}
>   
> @@ -1927,6 +1926,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
>   		complete(&queue->cm_done);
>   		return 0;
>   	case RDMA_CM_EVENT_REJECTED:
> +		nvme_rdma_destroy_queue_ib(queue);
Maybe the code version is incorrect, cause falsely code rollback.
>   		cm_error = nvme_rdma_conn_rejected(queue, ev);
>   		break;
>   	case RDMA_CM_EVENT_ROUTE_ERROR:
> diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
> index c672ae1da26bb5..937d55611cd073 100644
> --- a/include/rdma/rdma_cm.h
> +++ b/include/rdma/rdma_cm.h
> @@ -227,19 +227,8 @@ void rdma_destroy_qp(struct rdma_cm_id *id);
>   int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
>   		       int *qp_attr_mask);
>   
> -/**
> - * rdma_connect - Initiate an active connection request.
> - * @id: Connection identifier to connect.
> - * @conn_param: Connection information used for connected QPs.
> - *
> - * Users must have resolved a route for the rdma_cm_id to connect with
> - * by having called rdma_resolve_route before calling this routine.
> - *
> - * This call will either connect to a remote QP or obtain remote QP
> - * information for unconnected rdma_cm_id's.  The actual operation is
> - * based on the rdma_cm_id's port space.
> - */
>   int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
> +int rdma_connect_locked(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
>   
>   int rdma_connect_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param,
>   		     struct rdma_ucm_ece *ece);
> diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
> index 06603dd1c8aa38..b36b60668b1da9 100644
> --- a/net/rds/ib_cm.c
> +++ b/net/rds/ib_cm.c
> @@ -956,9 +956,10 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id, bool isv6)
>   	rds_ib_cm_fill_conn_param(conn, &conn_param, &dp,
>   				  conn->c_proposed_version,
>   				  UINT_MAX, UINT_MAX, isv6);
> -	ret = rdma_connect(cm_id, &conn_param);
> +	ret = rdma_connect_locked(cm_id, &conn_param);
>   	if (ret)
> -		rds_ib_conn_error(conn, "rdma_connect failed (%d)\n", ret);
> +		rds_ib_conn_error(conn, "rdma_connect_locked failed (%d)\n",
> +				  ret);
>   
>   out:
>   	/* Beware - returning non-zero tells the rdma_cm to destroy
> 


From jinpu.wang at cloud.ionos.com  Tue Oct 27 03:33:45 2020
From: jinpu.wang at cloud.ionos.com (Jinpu Wang)
Date: Tue, 27 Oct 2020 08:33:45 +0100
Subject: [PATCH] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
References: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <CAMGffEn-bOgELLb1rTg9W+f2Hqd6A46T1rkDZegKov8TrAkDxA@mail.gmail.com>

On Mon, Oct 26, 2020 at 3:25 PM Jason Gunthorpe <jgg at nvidia.com> wrote:
>
> There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
> handler triggers a completion and another thread does rdma_connect() or
> the handler directly calls rdma_connect().
>
> In all cases rdma_connect() needs to hold the handler_mutex, but when
> handler's are invoked this is already held by the core code. This causes
> ULPs using the 2nd method to deadlock.
>
> Provide a rdma_connect_locked() and have all ULPs call it from their
> handlers.
>
> Reported-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
> Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state"
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> ---
>  drivers/infiniband/core/cma.c            | 39 +++++++++++++++++++++---
>  drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
>  drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
>  drivers/nvme/host/rdma.c                 | 10 +++---
>  include/rdma/rdma_cm.h                   | 13 +-------
>  net/rds/ib_cm.c                          |  5 +--
>  6 files changed, 47 insertions(+), 26 deletions(-)
>
> Seems people are not testing these four ULPs against rdma-next.. Here is a
> quick fix for the issue:
>
> https://lore.kernel.org/r/3b1f7767-98e2-93e0-b718-16d1c5346140 at cloud.ionos.com
>
> Jason
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 7c2ab1f2fbea37..2eaaa1292fb847 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv,
>         /*
>          * The FSM uses a funny double locking where state is protected by both
>          * the handler_mutex and the spinlock. State is not allowed to change
> -        * away from a handler_mutex protected value without also holding
> +        * to/from a handler_mutex protected value without also holding
>          * handler_mutex.
>          */
> -       if (comp == RDMA_CM_CONNECT)
> +       if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT)
>                 lockdep_assert_held(&id_priv->handler_mutex);
>
>         spin_lock_irqsave(&id_priv->lock, flags);
> @@ -4038,13 +4038,20 @@ static int cma_connect_iw(struct rdma_id_private *id_priv,
>         return ret;
>  }
>
> -int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> +/**
> + * rdma_connect_locked - Initiate an active connection request.
> + * @id: Connection identifier to connect.
> + * @conn_param: Connection information used for connected QPs.
> + *
> + * Same as rdma_connect() but can only be called from the
> + * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback.
> + */
> +int rdma_connect_locked(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>  {
>         struct rdma_id_private *id_priv =
>                 container_of(id, struct rdma_id_private, id);
>         int ret;
>
> -       mutex_lock(&id_priv->handler_mutex);
>         if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
>                 ret = -EINVAL;
>                 goto err_unlock;
> @@ -4071,6 +4078,30 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>  err_state:
>         cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);
>  err_unlock:
> +       return ret;
> +}
> +EXPORT_SYMBOL(rdma_connect_locked);
> +
> +/**
> + * rdma_connect - Initiate an active connection request.
> + * @id: Connection identifier to connect.
> + * @conn_param: Connection information used for connected QPs.
> + *
> + * Users must have resolved a route for the rdma_cm_id to connect with by having
> + * called rdma_resolve_route before calling this routine.
> + *
> + * This call will either connect to a remote QP or obtain remote QP information
> + * for unconnected rdma_cm_id's.  The actual operation is based on the
> + * rdma_cm_id's port space.
> + */
> +int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> +{
> +       struct rdma_id_private *id_priv =
> +               container_of(id, struct rdma_id_private, id);
> +       int ret;
> +
> +       mutex_lock(&id_priv->handler_mutex);
> +       ret = rdma_connect_locked(id, conn_param);
>         mutex_unlock(&id_priv->handler_mutex);
>         return ret;
>  }
> diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
> index 2f3ebc0a75d924..2bd18b00689341 100644
> --- a/drivers/infiniband/ulp/iser/iser_verbs.c
> +++ b/drivers/infiniband/ulp/iser/iser_verbs.c
> @@ -620,7 +620,7 @@ static void iser_route_handler(struct rdma_cm_id *cma_id)
>         conn_param.private_data = (void *)&req_hdr;
>         conn_param.private_data_len = sizeof(struct iser_cm_hdr);
>
> -       ret = rdma_connect(cma_id, &conn_param);
> +       ret = rdma_connect_locked(cma_id, &conn_param);
>         if (ret) {
>                 iser_err("failure connecting: %d\n", ret);
>                 goto failure;
> diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> index 776e89231c52f7..f298adc02acba2 100644
> --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> @@ -1674,9 +1674,9 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con)
>         uuid_copy(&msg.sess_uuid, &sess->s.uuid);
>         uuid_copy(&msg.paths_uuid, &clt->paths_uuid);
>
> -       err = rdma_connect(con->c.cm_id, &param);
> +       err = rdma_connect_locked(con->c.cm_id, &param);
>         if (err)
> -               rtrs_err(clt, "rdma_connect(): %d\n", err);
> +               rtrs_err(clt, "rdma_connect_locked(): %d\n", err);
>
>         return err;
>  }
For rtrs, looks good to me!
Thanks for the quick fix.
Acked-by: Jack Wang <jinpu.wang at cloud.ionos.com>


From hch at lst.de  Tue Oct 27 04:04:29 2020
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 27 Oct 2020 09:04:29 +0100
Subject: [PATCH] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
References: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <20201027080429.GA22725@lst.de>

> +int rdma_connect_locked(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)

This adds an overly long line.

> +int rdma_connect_locked(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);

Same here.

Otherwise looks good except for the nvme merge error pointed out by
Chao Leng.

Reviewed-by: Christoph Hellwig <hch at lst.de>


From baolin.wang at linux.alibaba.com  Tue Oct 27 04:15:16 2020
From: baolin.wang at linux.alibaba.com (Baolin Wang)
Date: Tue, 27 Oct 2020 16:15:16 +0800
Subject: [PATCH] nvme: Simplify the nvme_req_qid()
Message-ID: <3ddb02eee7765d63328da3e1499a26da5cd20f1b.1603785673.git.baolin.wang@linux.alibaba.com>

Use the request's '->mq_hctx->queue_num' directly to simplify the
nvme_req_qid() function.

Signed-off-by: Baolin Wang <baolin.wang at linux.alibaba.com>
---
 drivers/nvme/host/nvme.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index cc11113..0b62b62 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -178,7 +178,8 @@ static inline u16 nvme_req_qid(struct request *req)
 {
 	if (!req->q->queuedata)
 		return 0;
-	return blk_mq_unique_tag_to_hwq(blk_mq_unique_tag(req)) + 1;
+
+	return req->mq_hctx->queue_num + 1;
 }
 
 /* The below value is the specific amount of delay needed before checking
-- 
1.8.3.1



From hch at infradead.org  Tue Oct 27 05:07:01 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Tue, 27 Oct 2020 09:07:01 +0000
Subject: [PATCH] nvme_fc: track error_recovery while connecting
In-Reply-To: <20201023222604.90435-1-james.smart@broadcom.com>
References: <20201023222604.90435-1-james.smart@broadcom.com>
Message-ID: <20201027090701.GA17222@infradead.org>

Thanks,

applied to nvme-5.10.


From hch at infradead.org  Tue Oct 27 05:07:15 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Tue, 27 Oct 2020 09:07:15 +0000
Subject: [PATCH v2 0/3] nvme-fc: clean up error recovery implementation
In-Reply-To: <20201023222752.90621-1-james.smart@broadcom.com>
References: <20201023222752.90621-1-james.smart@broadcom.com>
Message-ID: <20201027090715.GB17222@infradead.org>

Thanks,

applied to nvme-5.10.


From hch at lst.de  Tue Oct 27 05:07:24 2020
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 27 Oct 2020 10:07:24 +0100
Subject: [PATCH V2] nvmet: fix the Oops in the tracing for flush cmd
In-Reply-To: <20201022235821.6760-1-chaitanya.kulkarni@wdc.com>
References: <20201022235821.6760-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201027090723.GA26785@lst.de>

Thanks,

applied to nvme-5.10.


From hch at lst.de  Tue Oct 27 05:07:34 2020
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 27 Oct 2020 10:07:34 +0100
Subject: [PATCH] nvme: ignore zone validate errors on subsequent scans
In-Reply-To: <20201023191628.1729393-1-kbusch@kernel.org>
References: <20201023191628.1729393-1-kbusch@kernel.org>
Message-ID: <20201027090734.GB26785@lst.de>

Thanks,

applied to nvme-5.10.


From hch at lst.de  Tue Oct 27 05:07:46 2020
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 27 Oct 2020 10:07:46 +0100
Subject: [PATCH v3] nvme-rdma: handle nvme completion data length
In-Reply-To: <20201025115124.1430678-1-pizhenwei@bytedance.com>
References: <20201025115124.1430678-1-pizhenwei@bytedance.com>
Message-ID: <20201027090746.GC26785@lst.de>

Thanks,

applied to nvme-5.10.


From jgg at nvidia.com  Tue Oct 27 08:00:41 2020
From: jgg at nvidia.com (Jason Gunthorpe)
Date: Tue, 27 Oct 2020 09:00:41 -0300
Subject: [PATCH] RDMA: Add rdma_connect_locked()
In-Reply-To: <e13ec119-3cc6-87ab-bc76-d2d3de7631e4@huawei.com>
References: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
 <e13ec119-3cc6-87ab-bc76-d2d3de7631e4@huawei.com>
Message-ID: <20201027120041.GI1523783@nvidia.com>

On Tue, Oct 27, 2020 at 10:01:00AM +0800, Chao Leng wrote:
> > diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> > index aad829a2b50d0f..f488dc5f4c2c61 100644
> > +++ b/drivers/nvme/host/rdma.c
> > @@ -1730,11 +1730,10 @@ static void nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
> >   	req->result = cqe->result;
> >   	if (wc->wc_flags & IB_WC_WITH_INVALIDATE) {
> > -		if (unlikely(!req->mr ||
> > -			     wc->ex.invalidate_rkey != req->mr->rkey)) {
> > +		if (unlikely(wc->ex.invalidate_rkey != req->mr->rkey)) {
> >   			dev_err(queue->ctrl->ctrl.device,
> >   				"Bogus remote invalidation for rkey %#x\n",
> > -				req->mr ? req->mr->rkey : 0);
> > +				req->mr->rkey);
> Maybe the code version is incorrect, cause falsely code rollback.

Oh wow, thanks for noticing that, I made a git fumble when doing this
:(

Jason


From guoqing.jiang at cloud.ionos.com  Tue Oct 27 08:05:00 2020
From: guoqing.jiang at cloud.ionos.com (Guoqing Jiang)
Date: Tue, 27 Oct 2020 13:05:00 +0100
Subject: [PATCH] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
References: <0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <11bb18bd-a26a-d0e2-9ff6-6d7e2bf3fb86@cloud.ionos.com>



On 10/26/20 15:25, Jason Gunthorpe wrote:
> There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
> handler triggers a completion and another thread does rdma_connect() or
> the handler directly calls rdma_connect().
> 
> In all cases rdma_connect() needs to hold the handler_mutex, but when
> handler's are invoked this is already held by the core code. This causes
> ULPs using the 2nd method to deadlock.
> 
> Provide a rdma_connect_locked() and have all ULPs call it from their
> handlers.
> 
> Reported-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
> Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state"
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> ---
>   drivers/infiniband/core/cma.c            | 39 +++++++++++++++++++++---
>   drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
>   drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
>   drivers/nvme/host/rdma.c                 | 10 +++---
>   include/rdma/rdma_cm.h                   | 13 +-------
>   net/rds/ib_cm.c                          |  5 +--
>   6 files changed, 47 insertions(+), 26 deletions(-)
> 
> Seems people are not testing these four ULPs against rdma-next.. Here is a
> quick fix for the issue:
> 
> https://lore.kernel.org/r/3b1f7767-98e2-93e0-b718-16d1c5346140 at cloud.ionos.com

I can't see the previous calltrace with this patch.

Tested-by: Guoqing Jiang<guoqing.jiang at cloud.ionos.com>


Thanks,
Guoqing


From pizhenwei at bytedance.com  Tue Oct 27 08:15:45 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Tue, 27 Oct 2020 20:15:45 +0800
Subject: [PATCH 1/2] nvmet: introduce transport layer keep-alive
Message-ID: <20201027121546.1776104-1-pizhenwei@bytedance.com>

In the zero KATO scenario, if initiator crashes without transport
layer disconnection, target side would never reclaim resources.

A target could start transport layer keep-alive to detect dead
connection for the admin queue.

Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/target/admin-cmd.c |  2 +-
 drivers/nvme/target/core.c      | 14 +++++++++++---
 drivers/nvme/target/nvmet.h     |  3 ++-
 3 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index dca34489a1dc..53fbd5c193a1 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -729,7 +729,7 @@ u16 nvmet_set_feat_kato(struct nvmet_req *req)
 
 	nvmet_stop_keep_alive_timer(req->sq->ctrl);
 	req->sq->ctrl->kato = DIV_ROUND_UP(val32, 1000);
-	nvmet_start_keep_alive_timer(req->sq->ctrl);
+	nvmet_start_keep_alive_timer(req->sq->ctrl, req);
 
 	nvmet_set_result(req, req->sq->ctrl->kato);
 
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 957b39a82431..451192f7ad6a 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -395,10 +395,18 @@ static void nvmet_keep_alive_timer(struct work_struct *work)
 	nvmet_ctrl_fatal_error(ctrl);
 }
 
-void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl)
+void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl, struct nvmet_req *req)
 {
-	if (unlikely(ctrl->kato == 0))
+	if (unlikely(ctrl->kato == 0)) {
+		if (req->ops->keep_alive)
+			pr_info("ctrl %d starts with transport keep-alive %s\n", ctrl->cntlid,
+				req->ops->keep_alive(req) ? "failed" : "succeed");
+		else
+			pr_info("ctrl %d starts without both NVMeOF and transport keep-alive",
+				ctrl->cntlid);
+
 		return;
+	}
 
 	pr_debug("ctrl %d start keep-alive timer for %d secs\n",
 		ctrl->cntlid, ctrl->kato);
@@ -1383,7 +1391,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 	ctrl->err_counter = 0;
 	spin_lock_init(&ctrl->error_lock);
 
-	nvmet_start_keep_alive_timer(ctrl);
+	nvmet_start_keep_alive_timer(ctrl, req);
 
 	mutex_lock(&subsys->lock);
 	list_add_tail(&ctrl->subsys_entry, &subsys->ctrls);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 559a15ccc322..de1785ce9fcd 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -305,6 +305,7 @@ struct nvmet_fabrics_ops {
 	u16 (*install_queue)(struct nvmet_sq *nvme_sq);
 	void (*discovery_chg)(struct nvmet_port *port);
 	u8 (*get_mdts)(const struct nvmet_ctrl *ctrl);
+	int (*keep_alive)(struct nvmet_req *req);
 };
 
 #define NVMET_MAX_INLINE_BIOVEC	8
@@ -395,7 +396,7 @@ void nvmet_get_feat_async_event(struct nvmet_req *req);
 u16 nvmet_set_feat_kato(struct nvmet_req *req);
 u16 nvmet_set_feat_async_event(struct nvmet_req *req, u32 mask);
 void nvmet_execute_async_event(struct nvmet_req *req);
-void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl);
+void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl, struct nvmet_req *req);
 void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl);
 
 u16 nvmet_parse_connect_cmd(struct nvmet_req *req);
-- 
2.11.0



From pizhenwei at bytedance.com  Tue Oct 27 08:15:46 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Tue, 27 Oct 2020 20:15:46 +0800
Subject: [PATCH 2/2] nvmet-tcp: add keep-alive
In-Reply-To: <20201027121546.1776104-1-pizhenwei@bytedance.com>
References: <20201027121546.1776104-1-pizhenwei@bytedance.com>
Message-ID: <20201027121546.1776104-2-pizhenwei@bytedance.com>

Add tcp keep-alive to detect dead connections for zero KATO case.

Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
---
 drivers/nvme/target/tcp.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index dc1f0f647189..58800e914e61 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1714,6 +1714,30 @@ static void nvmet_tcp_disc_port_addr(struct nvmet_req *req,
 	}
 }
 
+static int nvmet_tcp_keep_alive(struct nvmet_req *req)
+{
+	struct nvmet_tcp_cmd *cmd = container_of(req, struct nvmet_tcp_cmd, req);
+	struct nvmet_tcp_queue *queue = cmd->queue;
+	struct socket *sock = queue->sock;
+	int ret;
+
+	/* TCP Keepalive options, max 5+1*3 = 8s */
+	sock_set_keepalive(sock->sk);
+	ret = tcp_sock_set_keepidle(sock->sk, 5);
+	if (ret)
+		return ret;
+
+	ret = tcp_sock_set_keepintvl(sock->sk, 1);
+	if (ret)
+		return ret;
+
+	ret = tcp_sock_set_keepcnt(sock->sk, 3);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
 static const struct nvmet_fabrics_ops nvmet_tcp_ops = {
 	.owner			= THIS_MODULE,
 	.type			= NVMF_TRTYPE_TCP,
@@ -1724,6 +1748,7 @@ static const struct nvmet_fabrics_ops nvmet_tcp_ops = {
 	.delete_ctrl		= nvmet_tcp_delete_ctrl,
 	.install_queue		= nvmet_tcp_install_queue,
 	.disc_traddr		= nvmet_tcp_disc_port_addr,
+	.keep_alive		= nvmet_tcp_keep_alive,
 };
 
 static int __init nvmet_tcp_init(void)
-- 
2.11.0



From jgg at nvidia.com  Tue Oct 27 08:20:36 2020
From: jgg at nvidia.com (Jason Gunthorpe)
Date: Tue, 27 Oct 2020 09:20:36 -0300
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
Message-ID: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>

There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
handler triggers a completion and another thread does rdma_connect() or
the handler directly calls rdma_connect().

In all cases rdma_connect() needs to hold the handler_mutex, but when
handler's are invoked this is already held by the core code. This causes
ULPs using the 2nd method to deadlock.

Provide a rdma_connect_locked() and have all ULPs call it from their
handlers.

Link: https://lore.kernel.org/r/0-v1-75e124dbad74+b05-rdma_connect_locking_jgg at nvidia.com
Reported-and-tested-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state")
Acked-by: Santosh Shilimkar <santosh.shilimkar at oracle.com>
Acked-by: Jack Wang <jinpu.wang at cloud.ionos.com>
Reviewed-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
---
 drivers/infiniband/core/cma.c            | 40 +++++++++++++++++++++---
 drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
 drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
 drivers/nvme/host/rdma.c                 |  4 +--
 include/rdma/rdma_cm.h                   | 14 ++-------
 net/rds/ib_cm.c                          |  5 +--
 6 files changed, 46 insertions(+), 23 deletions(-)

v2:
 - Remove extra code from nvme (Chao)
 - Fix long lines (CH)

I've applied this version to rdma-rc - expecting to get these ULPs unbroken for rc2
release

Thanks,
Jason

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 7c2ab1f2fbea37..193c8902b9db26 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv,
 	/*
 	 * The FSM uses a funny double locking where state is protected by both
 	 * the handler_mutex and the spinlock. State is not allowed to change
-	 * away from a handler_mutex protected value without also holding
+	 * to/from a handler_mutex protected value without also holding
 	 * handler_mutex.
 	 */
-	if (comp == RDMA_CM_CONNECT)
+	if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT)
 		lockdep_assert_held(&id_priv->handler_mutex);
 
 	spin_lock_irqsave(&id_priv->lock, flags);
@@ -4038,13 +4038,21 @@ static int cma_connect_iw(struct rdma_id_private *id_priv,
 	return ret;
 }
 
-int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+/**
+ * rdma_connect_locked - Initiate an active connection request.
+ * @id: Connection identifier to connect.
+ * @conn_param: Connection information used for connected QPs.
+ *
+ * Same as rdma_connect() but can only be called from the
+ * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback.
+ */
+int rdma_connect_locked(struct rdma_cm_id *id,
+			struct rdma_conn_param *conn_param)
 {
 	struct rdma_id_private *id_priv =
 		container_of(id, struct rdma_id_private, id);
 	int ret;
 
-	mutex_lock(&id_priv->handler_mutex);
 	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
 		ret = -EINVAL;
 		goto err_unlock;
@@ -4071,6 +4079,30 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
 err_state:
 	cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);
 err_unlock:
+	return ret;
+}
+EXPORT_SYMBOL(rdma_connect_locked);
+
+/**
+ * rdma_connect - Initiate an active connection request.
+ * @id: Connection identifier to connect.
+ * @conn_param: Connection information used for connected QPs.
+ *
+ * Users must have resolved a route for the rdma_cm_id to connect with by having
+ * called rdma_resolve_route before calling this routine.
+ *
+ * This call will either connect to a remote QP or obtain remote QP information
+ * for unconnected rdma_cm_id's.  The actual operation is based on the
+ * rdma_cm_id's port space.
+ */
+int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+{
+	struct rdma_id_private *id_priv =
+		container_of(id, struct rdma_id_private, id);
+	int ret;
+
+	mutex_lock(&id_priv->handler_mutex);
+	ret = rdma_connect_locked(id, conn_param);
 	mutex_unlock(&id_priv->handler_mutex);
 	return ret;
 }
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 2f3ebc0a75d924..2bd18b00689341 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -620,7 +620,7 @@ static void iser_route_handler(struct rdma_cm_id *cma_id)
 	conn_param.private_data	= (void *)&req_hdr;
 	conn_param.private_data_len = sizeof(struct iser_cm_hdr);
 
-	ret = rdma_connect(cma_id, &conn_param);
+	ret = rdma_connect_locked(cma_id, &conn_param);
 	if (ret) {
 		iser_err("failure connecting: %d\n", ret);
 		goto failure;
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index 776e89231c52f7..f298adc02acba2 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -1674,9 +1674,9 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con)
 	uuid_copy(&msg.sess_uuid, &sess->s.uuid);
 	uuid_copy(&msg.paths_uuid, &clt->paths_uuid);
 
-	err = rdma_connect(con->c.cm_id, &param);
+	err = rdma_connect_locked(con->c.cm_id, &param);
 	if (err)
-		rtrs_err(clt, "rdma_connect(): %d\n", err);
+		rtrs_err(clt, "rdma_connect_locked(): %d\n", err);
 
 	return err;
 }
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index aad829a2b50d0f..8bbc48cc45dc1d 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1890,10 +1890,10 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue)
 		priv.hsqsize = cpu_to_le16(queue->ctrl->ctrl.sqsize);
 	}
 
-	ret = rdma_connect(queue->cm_id, &param);
+	ret = rdma_connect_locked(queue->cm_id, &param);
 	if (ret) {
 		dev_err(ctrl->ctrl.device,
-			"rdma_connect failed (%d).\n", ret);
+			"rdma_connect_locked failed (%d).\n", ret);
 		goto out_destroy_queue_ib;
 	}
 
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index c672ae1da26bb5..32a67af18415d6 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -227,19 +227,9 @@ void rdma_destroy_qp(struct rdma_cm_id *id);
 int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
 		       int *qp_attr_mask);
 
-/**
- * rdma_connect - Initiate an active connection request.
- * @id: Connection identifier to connect.
- * @conn_param: Connection information used for connected QPs.
- *
- * Users must have resolved a route for the rdma_cm_id to connect with
- * by having called rdma_resolve_route before calling this routine.
- *
- * This call will either connect to a remote QP or obtain remote QP
- * information for unconnected rdma_cm_id's.  The actual operation is
- * based on the rdma_cm_id's port space.
- */
 int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+int rdma_connect_locked(struct rdma_cm_id *id,
+			struct rdma_conn_param *conn_param);
 
 int rdma_connect_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param,
 		     struct rdma_ucm_ece *ece);
diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
index 06603dd1c8aa38..b36b60668b1da9 100644
--- a/net/rds/ib_cm.c
+++ b/net/rds/ib_cm.c
@@ -956,9 +956,10 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id, bool isv6)
 	rds_ib_cm_fill_conn_param(conn, &conn_param, &dp,
 				  conn->c_proposed_version,
 				  UINT_MAX, UINT_MAX, isv6);
-	ret = rdma_connect(cm_id, &conn_param);
+	ret = rdma_connect_locked(cm_id, &conn_param);
 	if (ret)
-		rds_ib_conn_error(conn, "rdma_connect failed (%d)\n", ret);
+		rds_ib_conn_error(conn, "rdma_connect_locked failed (%d)\n",
+				  ret);
 
 out:
 	/* Beware - returning non-zero tells the rdma_cm to destroy
-- 
2.28.0



From mgurtovoy at nvidia.com  Tue Oct 27 09:06:51 2020
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Tue, 27 Oct 2020 15:06:51 +0200
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
References: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <5b4368f9-c231-bcf4-28af-7a9bcac02eb4@nvidia.com>


On 10/27/2020 2:20 PM, Jason Gunthorpe wrote:
> There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
> handler triggers a completion and another thread does rdma_connect() or
> the handler directly calls rdma_connect().
>
> In all cases rdma_connect() needs to hold the handler_mutex, but when
> handler's are invoked this is already held by the core code. This causes
> ULPs using the 2nd method to deadlock.
>
> Provide a rdma_connect_locked() and have all ULPs call it from their
> handlers.
>
> Link: https://lore.kernel.org/r/0-v1-75e124dbad74+b05-rdma_connect_locking_jgg at nvidia.com
> Reported-and-tested-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
> Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state")
> Acked-by: Santosh Shilimkar <santosh.shilimkar at oracle.com>
> Acked-by: Jack Wang <jinpu.wang at cloud.ionos.com>
> Reviewed-by: Christoph Hellwig <hch at lst.de>
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> ---
>   drivers/infiniband/core/cma.c            | 40 +++++++++++++++++++++---
>   drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
>   drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
>   drivers/nvme/host/rdma.c                 |  4 +--
>   include/rdma/rdma_cm.h                   | 14 ++-------
>   net/rds/ib_cm.c                          |  5 +--
>   6 files changed, 46 insertions(+), 23 deletions(-)
>
> v2:
>   - Remove extra code from nvme (Chao)
>   - Fix long lines (CH)
>
> I've applied this version to rdma-rc - expecting to get these ULPs unbroken for rc2
> release
>
> Thanks,
> Jason
>
iser and nvme/rdma looks good to me,

Reviewed-by: Max Gurtovoy <mgurtovoy at nvidia.com>



From leonro at nvidia.com  Tue Oct 27 09:19:36 2020
From: leonro at nvidia.com (Leon Romanovsky)
Date: Tue, 27 Oct 2020 15:19:36 +0200
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
References: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <20201027131936.GD1763578@unreal>

On Tue, Oct 27, 2020 at 09:20:36AM -0300, Jason Gunthorpe wrote:
> There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
> handler triggers a completion and another thread does rdma_connect() or
> the handler directly calls rdma_connect().
>
> In all cases rdma_connect() needs to hold the handler_mutex, but when
> handler's are invoked this is already held by the core code. This causes
> ULPs using the 2nd method to deadlock.
>
> Provide a rdma_connect_locked() and have all ULPs call it from their
> handlers.
>
> Link: https://lore.kernel.org/r/0-v1-75e124dbad74+b05-rdma_connect_locking_jgg at nvidia.com
> Reported-and-tested-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
> Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state")
> Acked-by: Santosh Shilimkar <santosh.shilimkar at oracle.com>
> Acked-by: Jack Wang <jinpu.wang at cloud.ionos.com>
> Reviewed-by: Christoph Hellwig <hch at lst.de>
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> ---
>  drivers/infiniband/core/cma.c            | 40 +++++++++++++++++++++---
>  drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
>  drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
>  drivers/nvme/host/rdma.c                 |  4 +--
>  include/rdma/rdma_cm.h                   | 14 ++-------
>  net/rds/ib_cm.c                          |  5 +--
>  6 files changed, 46 insertions(+), 23 deletions(-)
>
> v2:
>  - Remove extra code from nvme (Chao)
>  - Fix long lines (CH)
>
> I've applied this version to rdma-rc - expecting to get these ULPs unbroken for rc2
> release
>
> Thanks,
> Jason
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 7c2ab1f2fbea37..193c8902b9db26 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv,
>  	/*
>  	 * The FSM uses a funny double locking where state is protected by both
>  	 * the handler_mutex and the spinlock. State is not allowed to change
> -	 * away from a handler_mutex protected value without also holding
> +	 * to/from a handler_mutex protected value without also holding
>  	 * handler_mutex.
>  	 */
> -	if (comp == RDMA_CM_CONNECT)
> +	if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT)
>  		lockdep_assert_held(&id_priv->handler_mutex);
>
>  	spin_lock_irqsave(&id_priv->lock, flags);
> @@ -4038,13 +4038,21 @@ static int cma_connect_iw(struct rdma_id_private *id_priv,
>  	return ret;
>  }
>
> -int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> +/**
> + * rdma_connect_locked - Initiate an active connection request.
> + * @id: Connection identifier to connect.
> + * @conn_param: Connection information used for connected QPs.
> + *
> + * Same as rdma_connect() but can only be called from the
> + * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback.
> + */
> +int rdma_connect_locked(struct rdma_cm_id *id,
> +			struct rdma_conn_param *conn_param)
>  {
>  	struct rdma_id_private *id_priv =
>  		container_of(id, struct rdma_id_private, id);
>  	int ret;
>
> -	mutex_lock(&id_priv->handler_mutex);
>  	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
>  		ret = -EINVAL;
>  		goto err_unlock;

Not a big deal, but his label is not correct anymore.

Thanks


From jgg at nvidia.com  Tue Oct 27 09:23:26 2020
From: jgg at nvidia.com (Jason Gunthorpe)
Date: Tue, 27 Oct 2020 10:23:26 -0300
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
In-Reply-To: <20201027131936.GD1763578@unreal>
References: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
 <20201027131936.GD1763578@unreal>
Message-ID: <20201027132326.GK1523783@nvidia.com>

On Tue, Oct 27, 2020 at 03:19:36PM +0200, Leon Romanovsky wrote:

> > +int rdma_connect_locked(struct rdma_cm_id *id,
> > +			struct rdma_conn_param *conn_param)
> >  {
> >  	struct rdma_id_private *id_priv =
> >  		container_of(id, struct rdma_id_private, id);
> >  	int ret;
> >
> > -	mutex_lock(&id_priv->handler_mutex);
> >  	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
> >  		ret = -EINVAL;
> >  		goto err_unlock;
> 
> Not a big deal, but his label is not correct anymore.

Oh, yep

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 193c8902b9db26..f58d19881524dc 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -4053,10 +4053,8 @@ int rdma_connect_locked(struct rdma_cm_id *id,
 		container_of(id, struct rdma_id_private, id);
 	int ret;
 
-	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
-		ret = -EINVAL;
-		goto err_unlock;
-	}
+	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT))
+		return -EINVAL;
 
 	if (!id->qp) {
 		id_priv->qp_num = conn_param->qp_num;
@@ -4078,7 +4076,6 @@ int rdma_connect_locked(struct rdma_cm_id *id,
 	return 0;
 err_state:
 	cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);
-err_unlock:
 	return ret;
 }
 EXPORT_SYMBOL(rdma_connect_locked);


From jongpuls at gmail.com  Tue Oct 27 11:54:38 2020
From: jongpuls at gmail.com (Jongpil Jung)
Date: Wed, 28 Oct 2020 00:54:38 +0900
Subject: [PATCH 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
Message-ID: <20201027155438.GA111754@jongpil-desktop>

LiteON(SSSTC) CL1 device running FW 220TQ,22001 has bugs with simple
suspend.

When NVMe device receive D3hot from host, NVMe firmware will do
garbage collection. While NVMe device do Garbage collection,
firmware has chance to going incorrect address.
In that case, NVMe storage device goes to no device available state.
Finally, host can't access the device any more.

Quirk devices will not use simple suspend even if HMB is enabled.
In case of poweroff scenario, NVMe receive "PME turn off".
So garbage collection will not be happening.

Liteon(SSSTC) will fix the issue, that's why quirk apply on specific
vendor id and firmware version.

Signed-off-by: Jongpil Jung <jongpuls at gmail.com>
---
 drivers/nvme/host/core.c | 30 ++++++++++++++++++++++++++++++
 drivers/nvme/host/nvme.h |  4 ++++
 drivers/nvme/host/pci.c  |  6 +++++-
 3 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 95ef4943d8bd..9ee520aa0100 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2637,6 +2637,36 @@ static const struct nvme_core_quirk_entry core_quirks[] = {
 		.vid = 0x14a4,
 		.fr = "22301111",
 		.quirks = NVME_QUIRK_SIMPLE_SUSPEND,
+	},
+	{
+		/*
+		 * This LiteON CL1-3D256 CR22001 firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x14a4,
+		.fr = "CR22001",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
+	},
+		{
+		/*
+		 * This LiteON CL1-3D256 CR220TQ firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x14a4,
+		.fr = "CR220TQ",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
+	},
+	{
+		/*
+		 * This SSSTC CL1-3D256 CR22001 firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x1e95,
+		.fr = "CR22001",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
 	}
 };
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index cc111136a981..2fde019dad8e 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -144,6 +144,10 @@ enum nvme_quirks {
 	 * NVMe 1.3 compliance.
 	 */
 	NVME_QUIRK_NO_NS_DESC_LIST		= (1 << 15),
+	/*
+	 * Force noraml suspend/resume path for HMB enabled devices.
+	 */
+	NVME_QUIRK_NORMAL_SUSPEND_HMB		= (1 << 16),
 };
 
 /*
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index df8f3612107f..1b1221cfb257 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3014,10 +3014,14 @@ static int nvme_suspend(struct device *dev)
 	 * specification allows the device to access the host memory buffer in
 	 * host DRAM from all power states, but hosts will fail access to DRAM
 	 * during S3.
+	 *
+	 * If NVME_QUIRK_NORMAL_SUSPEND_HMB is enabled,
+	 * do not use SIMPLE_SUSPEND for HMB device.
 	 */
 	if (pm_suspend_via_firmware() || !ctrl->npss ||
 	    !pcie_aspm_enabled(pdev) ||
-	    ndev->nr_host_mem_descs ||
+	    (ndev->nr_host_mem_descs &&
+	     !(ndev->ctrl.quirks & NVME_QUIRK_NORMAL_SUSPEND_HMB)) ||
 	    (ndev->ctrl.quirks & NVME_QUIRK_SIMPLE_SUSPEND))
 		return nvme_disable_prepare_reset(ndev, true);
 
-- 
2.25.1



From kbusch at kernel.org  Tue Oct 27 16:23:34 2020
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 27 Oct 2020 13:23:34 -0700
Subject: [PATCH 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
In-Reply-To: <20201027155438.GA111754@jongpil-desktop>
References: <20201027155438.GA111754@jongpil-desktop>
Message-ID: <20201027202334.GA1942708@dhcp-10-100-145-180.wdc.com>

On Wed, Oct 28, 2020 at 12:54:38AM +0900, Jongpil Jung wrote:
> suspend.
> 
> When NVMe device receive D3hot from host, NVMe firmware will do
> garbage collection. While NVMe device do Garbage collection,
> firmware has chance to going incorrect address.
> In that case, NVMe storage device goes to no device available state.
> Finally, host can't access the device any more.
> 
> Quirk devices will not use simple suspend even if HMB is enabled.
> In case of poweroff scenario, NVMe receive "PME turn off".
> So garbage collection will not be happening.
> 
> Liteon(SSSTC) will fix the issue, that's why quirk apply on specific
> vendor id and firmware version.

This is a concerning quirk. We use the simple suspend when HMB is
enabled because at least some platforms disable device DMA access in the
runtime suspend state. Many devices continue to access their HMB while
in low power, so we can't let both conditions occur concurrently.
Unless you know for sure this device doesn't access host memory in
low-power, there will be at least some platform combinations where this
quirk will fail.


From sagi at grimberg.me  Tue Oct 27 17:08:13 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 27 Oct 2020 14:08:13 -0700
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
References: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <73296987-3f3e-7d39-b131-f93621d01851@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From lkp at intel.com  Tue Oct 27 17:23:28 2020
From: lkp at intel.com (kernel test robot)
Date: Wed, 28 Oct 2020 05:23:28 +0800
Subject: [linux-nvme:nvme-5.10] BUILD SUCCESS
 3c3751f2daf6675f6b5bee83b792354c272f5bd2
Message-ID: <5f988fd0.OvVX7NIMDc9kXRuL%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git  nvme-5.10
branch HEAD: 3c3751f2daf6675f6b5bee83b792354c272f5bd2  nvmet: fix a NULL pointer dereference when tracing the flush command

elapsed time: 721m

configs tested: 216
configs skipped: 3

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
c6x                        evmc6457_defconfig
sh                            titan_defconfig
powerpc                     tqm8548_defconfig
arm                          pxa168_defconfig
mips                           ip22_defconfig
arm                     davinci_all_defconfig
powerpc                       maple_defconfig
arm                           omap1_defconfig
powerpc                       holly_defconfig
m68k                        mvme147_defconfig
powerpc                     mpc83xx_defconfig
powerpc                      tqm8xx_defconfig
powerpc                     pseries_defconfig
powerpc                     tqm8560_defconfig
sh                             sh03_defconfig
powerpc                     sequoia_defconfig
ia64                             alldefconfig
arm                           sunxi_defconfig
powerpc                        icon_defconfig
sh                     magicpanelr2_defconfig
arm                  colibri_pxa300_defconfig
arm                          pxa3xx_defconfig
powerpc                    mvme5100_defconfig
riscv                    nommu_virt_defconfig
arm                            mps2_defconfig
mips                         tb0219_defconfig
sh                          landisk_defconfig
arm                             mxs_defconfig
ia64                             allyesconfig
arm                          pxa910_defconfig
powerpc                     tqm8541_defconfig
parisc                           alldefconfig
arm                        spear6xx_defconfig
powerpc                   motionpro_defconfig
arm                       mainstone_defconfig
powerpc                 mpc832x_rdb_defconfig
xtensa                generic_kc705_defconfig
arm                            qcom_defconfig
arm                           h3600_defconfig
arm                           sama5_defconfig
arm                         s3c2410_defconfig
powerpc                 mpc85xx_cds_defconfig
sh                         ap325rxa_defconfig
arc                         haps_hs_defconfig
mips                           rs90_defconfig
m68k                       m5275evb_defconfig
powerpc                  iss476-smp_defconfig
arc                     haps_hs_smp_defconfig
arm                      pxa255-idp_defconfig
arm                        neponset_defconfig
xtensa                  nommu_kc705_defconfig
sh                      rts7751r2d1_defconfig
mips                      malta_kvm_defconfig
arm                         assabet_defconfig
arc                              alldefconfig
sh                           se7343_defconfig
powerpc                    adder875_defconfig
m68k                       m5208evb_defconfig
arm                     am200epdkit_defconfig
arm                          lpd270_defconfig
xtensa                         virt_defconfig
powerpc                 linkstation_defconfig
mips                      bmips_stb_defconfig
parisc                generic-32bit_defconfig
arm                      integrator_defconfig
arm                          simpad_defconfig
sh                               allmodconfig
powerpc                      bamboo_defconfig
powerpc                     kmeter1_defconfig
c6x                        evmc6474_defconfig
arc                          axs103_defconfig
h8300                            allyesconfig
mips                     cu1000-neo_defconfig
mips                     loongson1c_defconfig
powerpc                         wii_defconfig
arm                           h5000_defconfig
arc                          axs101_defconfig
c6x                                 defconfig
sh                               j2_defconfig
mips                            gpr_defconfig
sh                            migor_defconfig
powerpc                      walnut_defconfig
c6x                        evmc6472_defconfig
mips                        nlm_xlp_defconfig
arc                        nsimosci_defconfig
sh                           se7780_defconfig
arm                          moxart_defconfig
arm                      footbridge_defconfig
arm                  colibri_pxa270_defconfig
s390                       zfcpdump_defconfig
sh                           se7705_defconfig
mips                        qi_lb60_defconfig
h8300                       h8s-sim_defconfig
powerpc                      pasemi_defconfig
mips                         bigsur_defconfig
xtensa                    smp_lx200_defconfig
arm                          exynos_defconfig
sh                          rsk7264_defconfig
nds32                             allnoconfig
arm                         vf610m4_defconfig
mips                      fuloong2e_defconfig
mips                           mtx1_defconfig
m68k                            mac_defconfig
sh                        dreamcast_defconfig
powerpc                     ppa8548_defconfig
powerpc                 mpc836x_mds_defconfig
powerpc                     redwood_defconfig
arc                      axs103_smp_defconfig
ia64                                defconfig
m68k                        mvme16x_defconfig
powerpc                      acadia_defconfig
powerpc                         ps3_defconfig
c6x                        evmc6678_defconfig
powerpc                       ppc64_defconfig
arm                       versatile_defconfig
arc                     nsimosci_hs_defconfig
powerpc                      chrp32_defconfig
mips                        maltaup_defconfig
mips                         mpc30x_defconfig
sh                          lboxre2_defconfig
mips                           ip28_defconfig
mips                            ar7_defconfig
mips                             allyesconfig
arm                           spitz_defconfig
mips                          rb532_defconfig
ia64                             allmodconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
arc                                 defconfig
parisc                              defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                                defconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a002-20201026
i386                 randconfig-a003-20201026
i386                 randconfig-a005-20201026
i386                 randconfig-a001-20201026
i386                 randconfig-a006-20201026
i386                 randconfig-a004-20201026
i386                 randconfig-a002-20201027
i386                 randconfig-a003-20201027
i386                 randconfig-a005-20201027
i386                 randconfig-a001-20201027
i386                 randconfig-a006-20201027
i386                 randconfig-a004-20201027
x86_64               randconfig-a011-20201026
x86_64               randconfig-a013-20201026
x86_64               randconfig-a016-20201026
x86_64               randconfig-a015-20201026
x86_64               randconfig-a012-20201026
x86_64               randconfig-a014-20201026
i386                 randconfig-a016-20201026
i386                 randconfig-a015-20201026
i386                 randconfig-a014-20201026
i386                 randconfig-a012-20201026
i386                 randconfig-a013-20201026
i386                 randconfig-a011-20201026
i386                 randconfig-a016-20201027
i386                 randconfig-a015-20201027
i386                 randconfig-a014-20201027
i386                 randconfig-a012-20201027
i386                 randconfig-a013-20201027
i386                 randconfig-a011-20201027
x86_64               randconfig-a001-20201027
x86_64               randconfig-a003-20201027
x86_64               randconfig-a002-20201027
x86_64               randconfig-a006-20201027
x86_64               randconfig-a004-20201027
x86_64               randconfig-a005-20201027
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a001-20201026
x86_64               randconfig-a003-20201026
x86_64               randconfig-a002-20201026
x86_64               randconfig-a006-20201026
x86_64               randconfig-a004-20201026
x86_64               randconfig-a005-20201026
x86_64               randconfig-a011-20201027
x86_64               randconfig-a013-20201027
x86_64               randconfig-a016-20201027
x86_64               randconfig-a015-20201027
x86_64               randconfig-a012-20201027
x86_64               randconfig-a014-20201027

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From sagi at grimberg.me  Wed Oct 28 03:15:06 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 28 Oct 2020 00:15:06 -0700
Subject: [PATCH 1/2] nvmet: introduce transport layer keep-alive
In-Reply-To: <20201027121546.1776104-1-pizhenwei@bytedance.com>
References: <20201027121546.1776104-1-pizhenwei@bytedance.com>
Message-ID: <bbbb8d0e-22a3-0e94-9a0a-b7ce1792a1ae@grimberg.me>


On 10/27/20 5:15 AM, zhenwei pi wrote:
> In the zero KATO scenario, if initiator crashes without transport
> layer disconnection, target side would never reclaim resources.
> 
> A target could start transport layer keep-alive to detect dead
> connection for the admin queue.

Not sure why we should worry about kato=0, it's really
more for debug purposes. I'd rather that we block this
option from the host altogether.

> 
> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
> ---
>   drivers/nvme/target/admin-cmd.c |  2 +-
>   drivers/nvme/target/core.c      | 14 +++++++++++---
>   drivers/nvme/target/nvmet.h     |  3 ++-
>   3 files changed, 14 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
> index dca34489a1dc..53fbd5c193a1 100644
> --- a/drivers/nvme/target/admin-cmd.c
> +++ b/drivers/nvme/target/admin-cmd.c
> @@ -729,7 +729,7 @@ u16 nvmet_set_feat_kato(struct nvmet_req *req)
>   
>   	nvmet_stop_keep_alive_timer(req->sq->ctrl);
>   	req->sq->ctrl->kato = DIV_ROUND_UP(val32, 1000);
> -	nvmet_start_keep_alive_timer(req->sq->ctrl);
> +	nvmet_start_keep_alive_timer(req->sq->ctrl, req);
>   
>   	nvmet_set_result(req, req->sq->ctrl->kato);
>   
> diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
> index 957b39a82431..451192f7ad6a 100644
> --- a/drivers/nvme/target/core.c
> +++ b/drivers/nvme/target/core.c
> @@ -395,10 +395,18 @@ static void nvmet_keep_alive_timer(struct work_struct *work)
>   	nvmet_ctrl_fatal_error(ctrl);
>   }
>   
> -void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl)
> +void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl, struct nvmet_req *req)
>   {
> -	if (unlikely(ctrl->kato == 0))
> +	if (unlikely(ctrl->kato == 0)) {
> +		if (req->ops->keep_alive)
> +			pr_info("ctrl %d starts with transport keep-alive %s\n", ctrl->cntlid,
> +				req->ops->keep_alive(req) ? "failed" : "succeed");
> +		else
> +			pr_info("ctrl %d starts without both NVMeOF and transport keep-alive",
> +				ctrl->cntlid);
> +
>   		return;
> +	}
>   
>   	pr_debug("ctrl %d start keep-alive timer for %d secs\n",
>   		ctrl->cntlid, ctrl->kato);
> @@ -1383,7 +1391,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
>   	ctrl->err_counter = 0;
>   	spin_lock_init(&ctrl->error_lock);
>   
> -	nvmet_start_keep_alive_timer(ctrl);
> +	nvmet_start_keep_alive_timer(ctrl, req);
>   
>   	mutex_lock(&subsys->lock);
>   	list_add_tail(&ctrl->subsys_entry, &subsys->ctrls);
> diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
> index 559a15ccc322..de1785ce9fcd 100644
> --- a/drivers/nvme/target/nvmet.h
> +++ b/drivers/nvme/target/nvmet.h
> @@ -305,6 +305,7 @@ struct nvmet_fabrics_ops {
>   	u16 (*install_queue)(struct nvmet_sq *nvme_sq);
>   	void (*discovery_chg)(struct nvmet_port *port);
>   	u8 (*get_mdts)(const struct nvmet_ctrl *ctrl);
> +	int (*keep_alive)(struct nvmet_req *req);
>   };
>   
>   #define NVMET_MAX_INLINE_BIOVEC	8
> @@ -395,7 +396,7 @@ void nvmet_get_feat_async_event(struct nvmet_req *req);
>   u16 nvmet_set_feat_kato(struct nvmet_req *req);
>   u16 nvmet_set_feat_async_event(struct nvmet_req *req, u32 mask);
>   void nvmet_execute_async_event(struct nvmet_req *req);
> -void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl);
> +void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl, struct nvmet_req *req);
>   void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl);
>   
>   u16 nvmet_parse_connect_cmd(struct nvmet_req *req);
> 


From jongpuls at gmail.com  Wed Oct 28 03:40:00 2020
From: jongpuls at gmail.com (Jongpil Jung)
Date: Wed, 28 Oct 2020 16:40:00 +0900
Subject: [PATCH V2 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
Message-ID: <20201028074000.GA320668@image-900X5T-900X5U>

LiteON(SSSTC) CL1 device running FW 220TQ,22001 has bugs with simple
suspend.

When NVMe device receive D3hot from host, NVMe firmware will do
garbage collection. While NVMe device do Garbage collection,
firmware has chance to going incorrect address.
In that case, NVMe storage device goes to no device available state.
Finally, host can't access the device any more.

Quirk devices will not use simple suspend even if HMB is enabled.
In case of poweroff scenario, NVMe receive "PME turn off".
So garbage collection will not be happening.

Liteon(SSSTC) will fix the issue, that's why quirk apply on specific
vendor id and firmware version on specific platform.

Signed-off-by: Jongpil Jung <jongpuls at gmail.com>
---
 drivers/nvme/host/core.c | 42 ++++++++++++++++++++++++++++++++++++++--
 drivers/nvme/host/nvme.h |  4 ++++
 drivers/nvme/host/pci.c  |  6 +++++-
 3 files changed, 49 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 95ef4943d8bd..b7b0b694ee21 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2637,6 +2637,36 @@ static const struct nvme_core_quirk_entry core_quirks[] = {
 		.vid = 0x14a4,
 		.fr = "22301111",
 		.quirks = NVME_QUIRK_SIMPLE_SUSPEND,
+	},
+	{
+		/*
+		 * This LiteON CL1-3D256 CR22001 firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x14a4,
+		.fr = "CR22001",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
+	},
+		{
+		/*
+		 * This LiteON CL1-3D256 CR220TQ firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x14a4,
+		.fr = "CR220TQ",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
+	},
+	{
+		/*
+		 * This SSSTC CL1-3D256 CR22001 firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x1e95,
+		.fr = "CR22001",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
 	}
 };
 
@@ -3007,10 +3037,18 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		 * could re-scan for quirks every time we reinitialize
 		 * the device, but we'd have to make sure that the driver
 		 * behaves intelligently if the quirks change.
+		 * For NVME_QUIRK_NORMAL_SUSPEND_HMB, need to check board
+		 * as well.
 		 */
 		for (i = 0; i < ARRAY_SIZE(core_quirks); i++) {
-			if (quirk_matches(id, &core_quirks[i]))
-				ctrl->quirks |= core_quirks[i].quirks;
+			if (quirk_matches(id, &core_quirks[i])) {
+				if (core_quirks[i].quirks == NVME_QUIRK_NORMAL_SUSPEND_HMB) {
+					if (dmi_match(DMI_BOARD_VENDOR, "Google") &&
+					    dmi_match(DMI_BOARD_NAME, "Kohaku"))
+						ctrl->quirks |= core_quirks[i].quirks;
+				} else
+					ctrl->quirks |= core_quirks[i].quirks;
+			}
 		}
 	}
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index cc111136a981..2fde019dad8e 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -144,6 +144,10 @@ enum nvme_quirks {
 	 * NVMe 1.3 compliance.
 	 */
 	NVME_QUIRK_NO_NS_DESC_LIST		= (1 << 15),
+	/*
+	 * Force noraml suspend/resume path for HMB enabled devices.
+	 */
+	NVME_QUIRK_NORMAL_SUSPEND_HMB		= (1 << 16),
 };
 
 /*
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index df8f3612107f..1b1221cfb257 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3014,10 +3014,14 @@ static int nvme_suspend(struct device *dev)
 	 * specification allows the device to access the host memory buffer in
 	 * host DRAM from all power states, but hosts will fail access to DRAM
 	 * during S3.
+	 *
+	 * If NVME_QUIRK_NORMAL_SUSPEND_HMB is enabled,
+	 * do not use SIMPLE_SUSPEND for HMB device.
 	 */
 	if (pm_suspend_via_firmware() || !ctrl->npss ||
 	    !pcie_aspm_enabled(pdev) ||
-	    ndev->nr_host_mem_descs ||
+	    (ndev->nr_host_mem_descs &&
+	     !(ndev->ctrl.quirks & NVME_QUIRK_NORMAL_SUSPEND_HMB)) ||
 	    (ndev->ctrl.quirks & NVME_QUIRK_SIMPLE_SUSPEND))
 		return nvme_disable_prepare_reset(ndev, true);
 
-- 
2.25.1



From hch at lst.de  Wed Oct 28 03:46:42 2020
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 28 Oct 2020 08:46:42 +0100
Subject: [PATCH V2 1/1] nvme: Add quirk for LiteON CL1 devices running
 FW 220TQ,22001
In-Reply-To: <20201028074000.GA320668@image-900X5T-900X5U>
References: <20201028074000.GA320668@image-900X5T-900X5U>
Message-ID: <20201028074642.GA16651@lst.de>

On Wed, Oct 28, 2020 at 04:40:00PM +0900, Jongpil Jung wrote:
> When NVMe device receive D3hot from host, NVMe firmware will do
> garbage collection. While NVMe device do Garbage collection,
> firmware has chance to going incorrect address.

What does "going incorrect address" mean?


From jongpuls at gmail.com  Wed Oct 28 05:06:03 2020
From: jongpuls at gmail.com (Jongpil Jung)
Date: Wed, 28 Oct 2020 18:06:03 +0900
Subject: [PATCH 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
In-Reply-To: <20201027202334.GA1942708@dhcp-10-100-145-180.wdc.com>
References: <20201027155438.GA111754@jongpil-desktop>
 <20201027202334.GA1942708@dhcp-10-100-145-180.wdc.com>
Message-ID: <20201028090603.GB599972@image-900X5T-900X5U>

On Tue, Oct 27, 2020 at 01:23:34PM -0700, Keith Busch wrote:
> On Wed, Oct 28, 2020 at 12:54:38AM +0900, Jongpil Jung wrote:
> > suspend.
> > 
> > When NVMe device receive D3hot from host, NVMe firmware will do
> > garbage collection. While NVMe device do Garbage collection,
> > firmware has chance to going incorrect address.
> > In that case, NVMe storage device goes to no device available state.
> > Finally, host can't access the device any more.
> > 
> > Quirk devices will not use simple suspend even if HMB is enabled.
> > In case of poweroff scenario, NVMe receive "PME turn off".
> > So garbage collection will not be happening.
> > 
> > Liteon(SSSTC) will fix the issue, that's why quirk apply on specific
> > vendor id and firmware version.
> 
> This is a concerning quirk. We use the simple suspend when HMB is
> enabled because at least some platforms disable device DMA access in the
> runtime suspend state. Many devices continue to access their HMB while
> in low power, so we can't let both conditions occur concurrently.
> Unless you know for sure this device doesn't access host memory in
> low-power, there will be at least some platform combinations where this
> quirk will fail.
I will submit another patch to check platform.


From jongpuls at gmail.com  Wed Oct 28 05:14:21 2020
From: jongpuls at gmail.com (Jongpil Jung)
Date: Wed, 28 Oct 2020 18:14:21 +0900
Subject: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
Message-ID: <20201028091421.GA667673@image-900X5T-900X5U>

LiteON(SSSTC) CL1 device running FW 220TQ,22001 has bugs with simple
suspend.

When NVMe device receive D3hot from host, NVMe firmware will do
garbage collection. While NVMe device do Garbage collection,
firmware has chance to going incorrect address.
In that case, NVMe storage device goes to no device available state.
Finally, host can't access the device any more.

Quirk devices will not use simple suspend even if HMB is enabled.
In case of poweroff scenario, NVMe receive "PME turn off".
So garbage collection will not be happening.

Liteon(SSSTC) will fix the issue, that's why quirk apply on specific
vendor id and firmware version on specific platform.

Signed-off-by: Jongpil Jung <jongpuls at gmail.com>
---
 drivers/nvme/host/core.c | 43 ++++++++++++++++++++++++++++++++++++++--
 drivers/nvme/host/nvme.h |  4 ++++
 drivers/nvme/host/pci.c  |  6 +++++-
 3 files changed, 50 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 95ef4943d8bd..414775a77013 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -7,6 +7,7 @@
 #include <linux/blkdev.h>
 #include <linux/blk-mq.h>
 #include <linux/compat.h>
+#include <linux/dmi.h>
 #include <linux/delay.h>
 #include <linux/errno.h>
 #include <linux/hdreg.h>
@@ -2637,6 +2638,36 @@ static const struct nvme_core_quirk_entry core_quirks[] = {
 		.vid = 0x14a4,
 		.fr = "22301111",
 		.quirks = NVME_QUIRK_SIMPLE_SUSPEND,
+	},
+	{
+		/*
+		 * This LiteON CL1-3D256 CR22001 firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x14a4,
+		.fr = "CR22001",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
+	},
+		{
+		/*
+		 * This LiteON CL1-3D256 CR220TQ firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x14a4,
+		.fr = "CR220TQ",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
+	},
+	{
+		/*
+		 * This SSSTC CL1-3D256 CR22001 firmware version has some
+		 * issue in simple suspend.
+		 * Simple Suspend issue will be fixed in future firmware
+		 */
+		.vid = 0x1e95,
+		.fr = "CR22001",
+		.quirks = NVME_QUIRK_NORMAL_SUSPEND_HMB,
 	}
 };
 
@@ -3007,10 +3038,18 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		 * could re-scan for quirks every time we reinitialize
 		 * the device, but we'd have to make sure that the driver
 		 * behaves intelligently if the quirks change.
+		 * For NVME_QUIRK_NORMAL_SUSPEND_HMB, need to check board
+		 * as well.
 		 */
 		for (i = 0; i < ARRAY_SIZE(core_quirks); i++) {
-			if (quirk_matches(id, &core_quirks[i]))
-				ctrl->quirks |= core_quirks[i].quirks;
+			if (quirk_matches(id, &core_quirks[i])) {
+				if (core_quirks[i].quirks == NVME_QUIRK_NORMAL_SUSPEND_HMB) {
+					if (dmi_match(DMI_BOARD_VENDOR, "Google") &&
+					    dmi_match(DMI_BOARD_NAME, "Kohaku"))
+						ctrl->quirks |= core_quirks[i].quirks;
+				} else
+					ctrl->quirks |= core_quirks[i].quirks;
+			}
 		}
 	}
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index cc111136a981..2fde019dad8e 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -144,6 +144,10 @@ enum nvme_quirks {
 	 * NVMe 1.3 compliance.
 	 */
 	NVME_QUIRK_NO_NS_DESC_LIST		= (1 << 15),
+	/*
+	 * Force noraml suspend/resume path for HMB enabled devices.
+	 */
+	NVME_QUIRK_NORMAL_SUSPEND_HMB		= (1 << 16),
 };
 
 /*
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index df8f3612107f..1b1221cfb257 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3014,10 +3014,14 @@ static int nvme_suspend(struct device *dev)
 	 * specification allows the device to access the host memory buffer in
 	 * host DRAM from all power states, but hosts will fail access to DRAM
 	 * during S3.
+	 *
+	 * If NVME_QUIRK_NORMAL_SUSPEND_HMB is enabled,
+	 * do not use SIMPLE_SUSPEND for HMB device.
 	 */
 	if (pm_suspend_via_firmware() || !ctrl->npss ||
 	    !pcie_aspm_enabled(pdev) ||
-	    ndev->nr_host_mem_descs ||
+	    (ndev->nr_host_mem_descs &&
+	     !(ndev->ctrl.quirks & NVME_QUIRK_NORMAL_SUSPEND_HMB)) ||
 	    (ndev->ctrl.quirks & NVME_QUIRK_SIMPLE_SUSPEND))
 		return nvme_disable_prepare_reset(ndev, true);
 
-- 
2.25.1



From maorg at nvidia.com  Wed Oct 28 05:19:14 2020
From: maorg at nvidia.com (Maor Gottlieb)
Date: Wed, 28 Oct 2020 11:19:14 +0200
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
In-Reply-To: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
References: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
Message-ID: <4401b7b1-5d05-a715-4701-957fd09f34c9@nvidia.com>


On 10/27/2020 2:20 PM, Jason Gunthorpe wrote:
> There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the
> handler triggers a completion and another thread does rdma_connect() or
> the handler directly calls rdma_connect().
>
> In all cases rdma_connect() needs to hold the handler_mutex, but when
> handler's are invoked this is already held by the core code. This causes
> ULPs using the 2nd method to deadlock.
>
> Provide a rdma_connect_locked() and have all ULPs call it from their
> handlers.
>
> Link: https://lore.kernel.org/r/0-v1-75e124dbad74+b05-rdma_connect_locking_jgg at nvidia.com
> Reported-and-tested-by: Guoqing Jiang <guoqing.jiang at cloud.ionos.com>
> Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state")
> Acked-by: Santosh Shilimkar <santosh.shilimkar at oracle.com>
> Acked-by: Jack Wang <jinpu.wang at cloud.ionos.com>
> Reviewed-by: Christoph Hellwig <hch at lst.de>
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> ---
>   drivers/infiniband/core/cma.c            | 40 +++++++++++++++++++++---
>   drivers/infiniband/ulp/iser/iser_verbs.c |  2 +-
>   drivers/infiniband/ulp/rtrs/rtrs-clt.c   |  4 +--
>   drivers/nvme/host/rdma.c                 |  4 +--
>   include/rdma/rdma_cm.h                   | 14 ++-------
>   net/rds/ib_cm.c                          |  5 +--
>   6 files changed, 46 insertions(+), 23 deletions(-)
>
> v2:
>   - Remove extra code from nvme (Chao)
>   - Fix long lines (CH)
>
> I've applied this version to rdma-rc - expecting to get these ULPs unbroken for rc2
> release
>
> Thanks,
> Jason
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 7c2ab1f2fbea37..193c8902b9db26 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv,
>   	/*
>   	 * The FSM uses a funny double locking where state is protected by both
>   	 * the handler_mutex and the spinlock. State is not allowed to change
> -	 * away from a handler_mutex protected value without also holding
> +	 * to/from a handler_mutex protected value without also holding
>   	 * handler_mutex.
>   	 */
> -	if (comp == RDMA_CM_CONNECT)
> +	if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT)
>   		lockdep_assert_held(&id_priv->handler_mutex);
>   
>   	spin_lock_irqsave(&id_priv->lock, flags);
> @@ -4038,13 +4038,21 @@ static int cma_connect_iw(struct rdma_id_private *id_priv,
>   	return ret;
>   }
>   
> -int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> +/**
> + * rdma_connect_locked - Initiate an active connection request.
> + * @id: Connection identifier to connect.
> + * @conn_param: Connection information used for connected QPs.
> + *
> + * Same as rdma_connect() but can only be called from the
> + * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback.
> + */
> +int rdma_connect_locked(struct rdma_cm_id *id,
> +			struct rdma_conn_param *conn_param)
>   {
>   	struct rdma_id_private *id_priv =
>   		container_of(id, struct rdma_id_private, id);
>   	int ret;
>   
> -	mutex_lock(&id_priv->handler_mutex);

You need to delete the mutex_unlock in success path too.
>   	if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) {
>   		ret = -EINVAL;
>   		goto err_unlock;
> @@ -4071,6 +4079,30 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>   err_state:
>   	cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);
>   err_unlock:
> +	return ret;
> +}
> +EXPORT_SYMBOL(rdma_connect_locked);
> +
> +/**
> + * rdma_connect - Initiate an active connection request.
> + * @id: Connection identifier to connect.
> + * @conn_param: Connection information used for connected QPs.
> + *
> + * Users must have resolved a route for the rdma_cm_id to connect with by having
> + * called rdma_resolve_route before calling this routine.
> + *
> + * This call will either connect to a remote QP or obtain remote QP information
> + * for unconnected rdma_cm_id's.  The actual operation is based on the
> + * rdma_cm_id's port space.
> + */
> +int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> +{
> +	struct rdma_id_private *id_priv =
> +		container_of(id, struct rdma_id_private, id);
> +	int ret;
> +
> +	mutex_lock(&id_priv->handler_mutex);
> +	ret = rdma_connect_locked(id, conn_param);
>   	mutex_unlock(&id_priv->handler_mutex);
>   	return ret;
>   }
> diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
> index 2f3ebc0a75d924..2bd18b00689341 100644
> --- a/drivers/infiniband/ulp/iser/iser_verbs.c
> +++ b/drivers/infiniband/ulp/iser/iser_verbs.c
> @@ -620,7 +620,7 @@ static void iser_route_handler(struct rdma_cm_id *cma_id)
>   	conn_param.private_data	= (void *)&req_hdr;
>   	conn_param.private_data_len = sizeof(struct iser_cm_hdr);
>   
> -	ret = rdma_connect(cma_id, &conn_param);
> +	ret = rdma_connect_locked(cma_id, &conn_param);
>   	if (ret) {
>   		iser_err("failure connecting: %d\n", ret);
>   		goto failure;
> diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> index 776e89231c52f7..f298adc02acba2 100644
> --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
> @@ -1674,9 +1674,9 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con)
>   	uuid_copy(&msg.sess_uuid, &sess->s.uuid);
>   	uuid_copy(&msg.paths_uuid, &clt->paths_uuid);
>   
> -	err = rdma_connect(con->c.cm_id, &param);
> +	err = rdma_connect_locked(con->c.cm_id, &param);
>   	if (err)
> -		rtrs_err(clt, "rdma_connect(): %d\n", err);
> +		rtrs_err(clt, "rdma_connect_locked(): %d\n", err);
>   
>   	return err;
>   }
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index aad829a2b50d0f..8bbc48cc45dc1d 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1890,10 +1890,10 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue)
>   		priv.hsqsize = cpu_to_le16(queue->ctrl->ctrl.sqsize);
>   	}
>   
> -	ret = rdma_connect(queue->cm_id, &param);
> +	ret = rdma_connect_locked(queue->cm_id, &param);
>   	if (ret) {
>   		dev_err(ctrl->ctrl.device,
> -			"rdma_connect failed (%d).\n", ret);
> +			"rdma_connect_locked failed (%d).\n", ret);
>   		goto out_destroy_queue_ib;
>   	}
>   
> diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
> index c672ae1da26bb5..32a67af18415d6 100644
> --- a/include/rdma/rdma_cm.h
> +++ b/include/rdma/rdma_cm.h
> @@ -227,19 +227,9 @@ void rdma_destroy_qp(struct rdma_cm_id *id);
>   int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
>   		       int *qp_attr_mask);
>   
> -/**
> - * rdma_connect - Initiate an active connection request.
> - * @id: Connection identifier to connect.
> - * @conn_param: Connection information used for connected QPs.
> - *
> - * Users must have resolved a route for the rdma_cm_id to connect with
> - * by having called rdma_resolve_route before calling this routine.
> - *
> - * This call will either connect to a remote QP or obtain remote QP
> - * information for unconnected rdma_cm_id's.  The actual operation is
> - * based on the rdma_cm_id's port space.
> - */
>   int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
> +int rdma_connect_locked(struct rdma_cm_id *id,
> +			struct rdma_conn_param *conn_param);
>   
>   int rdma_connect_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param,
>   		     struct rdma_ucm_ece *ece);
> diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
> index 06603dd1c8aa38..b36b60668b1da9 100644
> --- a/net/rds/ib_cm.c
> +++ b/net/rds/ib_cm.c
> @@ -956,9 +956,10 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id, bool isv6)
>   	rds_ib_cm_fill_conn_param(conn, &conn_param, &dp,
>   				  conn->c_proposed_version,
>   				  UINT_MAX, UINT_MAX, isv6);
> -	ret = rdma_connect(cm_id, &conn_param);
> +	ret = rdma_connect_locked(cm_id, &conn_param);
>   	if (ret)
> -		rds_ib_conn_error(conn, "rdma_connect failed (%d)\n", ret);
> +		rds_ib_conn_error(conn, "rdma_connect_locked failed (%d)\n",
> +				  ret);
>   
>   out:
>   	/* Beware - returning non-zero tells the rdma_cm to destroy


From pizhenwei at bytedance.com  Wed Oct 28 05:29:00 2020
From: pizhenwei at bytedance.com (zhenwei pi)
Date: Wed, 28 Oct 2020 17:29:00 +0800
Subject: [External] Re: [PATCH 1/2] nvmet: introduce transport layer
 keep-alive
In-Reply-To: <bbbb8d0e-22a3-0e94-9a0a-b7ce1792a1ae@grimberg.me>
References: <20201027121546.1776104-1-pizhenwei@bytedance.com>
 <bbbb8d0e-22a3-0e94-9a0a-b7ce1792a1ae@grimberg.me>
Message-ID: <3f8e5405-152f-d71b-a37f-9bf4f85714c2@bytedance.com>

On 10/28/20 3:15 PM, Sagi Grimberg wrote:
> 
> On 10/27/20 5:15 AM, zhenwei pi wrote:
>> In the zero KATO scenario, if initiator crashes without transport
>> layer disconnection, target side would never reclaim resources.
>>
>> A target could start transport layer keep-alive to detect dead
>> connection for the admin queue.
> 
> Not sure why we should worry about kato=0, it's really
> more for debug purposes. I'd rather that we block this
> option from the host altogether.
> 

A target can't suppose that initiator always runs as expected.
I'm developing a user level NVMeOF library, in my plan, user level KATO 
timer gets a little complex, I'd like to use TCP level keep-alive
to make code simple. So the target side needs to handle zero KATO case.

>>
>> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
>> ---
>> ? drivers/nvme/target/admin-cmd.c |? 2 +-
>> ? drivers/nvme/target/core.c????? | 14 +++++++++++---
>> ? drivers/nvme/target/nvmet.h???? |? 3 ++-
>> ? 3 files changed, 14 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/nvme/target/admin-cmd.c 
>> b/drivers/nvme/target/admin-cmd.c
>> index dca34489a1dc..53fbd5c193a1 100644
>> --- a/drivers/nvme/target/admin-cmd.c
>> +++ b/drivers/nvme/target/admin-cmd.c
>> @@ -729,7 +729,7 @@ u16 nvmet_set_feat_kato(struct nvmet_req *req)
>> ????? nvmet_stop_keep_alive_timer(req->sq->ctrl);
>> ????? req->sq->ctrl->kato = DIV_ROUND_UP(val32, 1000);
>> -??? nvmet_start_keep_alive_timer(req->sq->ctrl);
>> +??? nvmet_start_keep_alive_timer(req->sq->ctrl, req);
>> ????? nvmet_set_result(req, req->sq->ctrl->kato);
>> diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
>> index 957b39a82431..451192f7ad6a 100644
>> --- a/drivers/nvme/target/core.c
>> +++ b/drivers/nvme/target/core.c
>> @@ -395,10 +395,18 @@ static void nvmet_keep_alive_timer(struct 
>> work_struct *work)
>> ????? nvmet_ctrl_fatal_error(ctrl);
>> ? }
>> -void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl)
>> +void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl, struct 
>> nvmet_req *req)
>> ? {
>> -??? if (unlikely(ctrl->kato == 0))
>> +??? if (unlikely(ctrl->kato == 0)) {
>> +??????? if (req->ops->keep_alive)
>> +??????????? pr_info("ctrl %d starts with transport keep-alive %s\n", 
>> ctrl->cntlid,
>> +??????????????? req->ops->keep_alive(req) ? "failed" : "succeed");
>> +??????? else
>> +??????????? pr_info("ctrl %d starts without both NVMeOF and transport 
>> keep-alive",
>> +??????????????? ctrl->cntlid);
>> +
>> ????????? return;
>> +??? }
>> ????? pr_debug("ctrl %d start keep-alive timer for %d secs\n",
>> ????????? ctrl->cntlid, ctrl->kato);
>> @@ -1383,7 +1391,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, 
>> const char *hostnqn,
>> ????? ctrl->err_counter = 0;
>> ????? spin_lock_init(&ctrl->error_lock);
>> -??? nvmet_start_keep_alive_timer(ctrl);
>> +??? nvmet_start_keep_alive_timer(ctrl, req);
>> ????? mutex_lock(&subsys->lock);
>> ????? list_add_tail(&ctrl->subsys_entry, &subsys->ctrls);
>> diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
>> index 559a15ccc322..de1785ce9fcd 100644
>> --- a/drivers/nvme/target/nvmet.h
>> +++ b/drivers/nvme/target/nvmet.h
>> @@ -305,6 +305,7 @@ struct nvmet_fabrics_ops {
>> ????? u16 (*install_queue)(struct nvmet_sq *nvme_sq);
>> ????? void (*discovery_chg)(struct nvmet_port *port);
>> ????? u8 (*get_mdts)(const struct nvmet_ctrl *ctrl);
>> +??? int (*keep_alive)(struct nvmet_req *req);
>> ? };
>> ? #define NVMET_MAX_INLINE_BIOVEC??? 8
>> @@ -395,7 +396,7 @@ void nvmet_get_feat_async_event(struct nvmet_req 
>> *req);
>> ? u16 nvmet_set_feat_kato(struct nvmet_req *req);
>> ? u16 nvmet_set_feat_async_event(struct nvmet_req *req, u32 mask);
>> ? void nvmet_execute_async_event(struct nvmet_req *req);
>> -void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl);
>> +void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl, struct 
>> nvmet_req *req);
>> ? void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl);
>> ? u16 nvmet_parse_connect_cmd(struct nvmet_req *req);
>>

-- 
zhenwei pi


From yi.zhang at redhat.com  Wed Oct 28 07:36:06 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Wed, 28 Oct 2020 19:36:06 +0800
Subject: [PATCH v2 0/5] avoid race for time out
In-Reply-To: <20201022021452.10971-1-lengchao@huawei.com>
References: <20201022021452.10971-1-lengchao@huawei.com>
Message-ID: <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>

Hello

This series fixed the WARNING issue I reported [1], but now the nvme/012 
[2] will be hang there and never finished, here is the log[3]
[1]
https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/

[2]
[root at hpe-xw9400-02 blktests]# nvme_trtype=tcp ./check nvme/012
nvme/012 (run mkfs and data verification fio job on NVMeOF block 
device-backed ns)
 ??? runtime? 1199.651s? ...

[3]
[? 120.550409] run blktests nvme/012 at 2020-10-28 06:50:11
[? 121.138234] loop: module loaded
[? 121.170869] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[? 121.215930] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[? 121.288229] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[? 121.302597] nvme nvme0: creating 12 I/O queues.
[? 121.308361] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[? 121.320030] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
127.0.0.1:4420
[? 123.278903] XFS (nvme0n1): Mounting V5 Filesystem
[? 123.291608] XFS (nvme0n1): Ending clean mount
[? 123.297321] xfs filesystem being mounted at /mnt/blktests supports 
timestamps until 2038 (0x7fffffff)
[? 183.872118] nvme nvme0: queue 1: timeout request 0x6c type 4
[? 183.877792] nvme nvme0: starting error recovery
[? 183.882376] nvme nvme0: queue 8: timeout request 0x11 type 4
[? 183.888149] nvme nvme0: queue 8: timeout request 0x12 type 4
[? 183.893805] nvme nvme0: queue 8: timeout request 0x13 type 4
[? 183.899469] nvme nvme0: queue 8: timeout request 0x14 type 4
[? 183.905130] nvme nvme0: queue 8: timeout request 0x15 type 4
[? 183.910792] nvme nvme0: queue 8: timeout request 0x16 type 4
[? 183.916453] nvme nvme0: queue 8: timeout request 0x17 type 4
[? 183.922114] nvme nvme0: queue 8: timeout request 0x18 type 4
[? 183.927777] nvme nvme0: queue 8: timeout request 0x19 type 4
[? 183.933450] nvme nvme0: queue 8: timeout request 0x1a type 4
[? 183.939110] nvme nvme0: queue 8: timeout request 0x1b type 4
[? 183.944771] nvme nvme0: queue 8: timeout request 0x1c type 4
[? 183.950431] nvme nvme0: queue 8: timeout request 0x1d type 4
[? 183.956095] nvme nvme0: queue 8: timeout request 0x1e type 4
[? 183.961755] nvme nvme0: queue 8: timeout request 0x1f type 4
[? 183.967414] nvme nvme0: queue 8: timeout request 0x20 type 4
[? 183.973218] block nvme0n1: no usable path - requeuing I/O
[? 183.978623] block nvme0n1: no usable path - requeuing I/O
[? 183.982492] nvme nvme0: Reconnecting in 10 seconds...
[? 183.984022] block nvme0n1: no usable path - requeuing I/O
[? 183.994476] block nvme0n1: no usable path - requeuing I/O
[? 183.999870] block nvme0n1: no usable path - requeuing I/O
[? 184.005264] block nvme0n1: no usable path - requeuing I/O
[? 184.010669] block nvme0n1: no usable path - requeuing I/O
[? 184.016080] block nvme0n1: no usable path - requeuing I/O
[? 184.021463] block nvme0n1: no usable path - requeuing I/O
[? 184.026858] block nvme0n1: no usable path - requeuing I/O
[? 209.472647] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
[? 209.478662] nvmet: ctrl 2 fatal error occurred!
[? 213.568765] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[? 213.574782] nvmet: ctrl 1 fatal error occurred!
[? 238.064572] nvmet: creating controller 2 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[? 256.577658] nvme nvme0: queue 0: timeout request 0x0 type 4
[? 256.583333] nvme nvme0: Connect command failed, error wo/DNR bit: 881
[? 256.589806] nvme nvme0: failed to connect queue: 0 ret=881
[? 256.595326] nvme nvme0: Failed reconnect attempt 1
[? 256.600119] nvme nvme0: Reconnecting in 10 seconds...
[? 266.818455] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[? 266.832356] nvme_ns_head_submit_bio: 30 callbacks suppressed
[? 266.832362] block nvme0n1: no usable path - requeuing I/O
[? 266.843443] block nvme0n1: no usable path - requeuing I/O
[? 266.848848] block nvme0n1: no usable path - requeuing I/O
[? 266.854244] block nvme0n1: no usable path - requeuing I/O
[? 266.859663] block nvme0n1: no usable path - requeuing I/O
[? 266.865059] block nvme0n1: no usable path - requeuing I/O
[? 266.870454] block nvme0n1: no usable path - requeuing I/O
[? 266.875845] block nvme0n1: no usable path - requeuing I/O
[? 266.881234] block nvme0n1: no usable path - requeuing I/O
[? 266.886632] block nvme0n1: no usable path - requeuing I/O
[? 266.892237] nvme nvme0: creating 12 I/O queues.
[? 266.903744] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[? 266.911929] nvme nvme0: Successfully reconnected (2 attempt)
[? 327.747177] nvme nvme0: queue 2: timeout request 0x1e type 4
[? 327.752883] nvme nvme0: starting error recovery
[? 327.757450] nvme nvme0: queue 4: timeout request 0x63 type 4
[? 327.763511] nvme_ns_head_submit_bio: 14 callbacks suppressed
[? 327.763520] block nvme0n1: no usable path - requeuing I/O
[? 327.774614] block nvme0n1: no usable path - requeuing I/O
[? 327.780053] block nvme0n1: no usable path - requeuing I/O
[? 327.785450] block nvme0n1: no usable path - requeuing I/O
[? 327.790876] block nvme0n1: no usable path - requeuing I/O
[? 327.796316] block nvme0n1: no usable path - requeuing I/O
[? 327.801727] block nvme0n1: no usable path - requeuing I/O
[? 327.807231] block nvme0n1: no usable path - requeuing I/O
[? 327.812627] block nvme0n1: no usable path - requeuing I/O
[? 327.818025] block nvme0n1: no usable path - requeuing I/O
[? 353.859745] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[? 353.865761] nvmet: ctrl 1 fatal error occurred!


On 10/22/20 10:14 AM, Chao Leng wrote:
> First avoid race between time out and tear down for rdma and tcp.
> Second avoid repeated request completion in time out for rdma and tcp.
>
> V2:
> 	- add avoiding repeated request completion in time out
>
> Chao Leng (3):
>    nvme-core: introduce sync io queues
>    nvme-rdma: avoid race between time out and tear down
>    nvme-tcp: avoid race between time out and tear down
>
> Sagi Grimberg (2):
>    nvme-rdma: avoid repeated request completion
>    nvme-tcp: avoid repeated request completion
>
>   drivers/nvme/host/core.c |  8 ++++++--
>   drivers/nvme/host/nvme.h |  1 +
>   drivers/nvme/host/rdma.c | 14 +++-----------
>   drivers/nvme/host/tcp.c  | 16 ++++------------
>   4 files changed, 14 insertions(+), 25 deletions(-)
>



From jgg at nvidia.com  Wed Oct 28 08:14:37 2020
From: jgg at nvidia.com (Jason Gunthorpe)
Date: Wed, 28 Oct 2020 09:14:37 -0300
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
In-Reply-To: <4401b7b1-5d05-a715-4701-957fd09f34c9@nvidia.com>
References: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
 <4401b7b1-5d05-a715-4701-957fd09f34c9@nvidia.com>
Message-ID: <20201028121437.GU1523783@nvidia.com>

On Wed, Oct 28, 2020 at 11:19:14AM +0200, Maor Gottlieb wrote:
> > +			struct rdma_conn_param *conn_param)
> >   {
> >   	struct rdma_id_private *id_priv =
> >   		container_of(id, struct rdma_id_private, id);
> >   	int ret;
> > -	mutex_lock(&id_priv->handler_mutex);
> 
> You need to delete the mutex_unlock in success path too.

Gaaaaah. Just goes to prove I shouldn't write patches with a child on
my lap :\

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index f58d19881524dc..a77750b8954db0 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -4072,7 +4072,6 @@ int rdma_connect_locked(struct rdma_cm_id *id,
 		ret = -ENOSYS;
 	if (ret)
 		goto err_state;
-	mutex_unlock(&id_priv->handler_mutex);
 	return 0;
 err_state:
 	cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);

Thanks,
Jason


From yi.zhang at redhat.com  Wed Oct 28 08:19:08 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Wed, 28 Oct 2020 20:19:08 +0800
Subject: nvme_tcp: nvme connect failed after execute stress-ng: unshare
In-Reply-To: <c5ad05c0-b0aa-bfda-32db-1fd0e0c74e55@redhat.com>
References: <643455136.711850.1599712518936.JavaMail.zimbra@redhat.com>
 <34570678-f8e7-a3ce-e7a3-be887ec2d949@grimberg.me>
 <c5ad05c0-b0aa-bfda-32db-1fd0e0c74e55@redhat.com>
Message-ID: <a8bb7d88-a99f-684b-aa66-da1447f6d8a0@redhat.com>



On 9/15/20 9:40 AM, Yi Zhang wrote:
> Hi Sagi
>
> On 9/15/20 7:50 AM, Sagi Grimberg wrote:
>>
>>> Hello
>>>
>>> Recently I found nvme-tcp connecting always failed[1] after execute 
>>> stress-ng:unshare[2], by bisecting I finally found it was introduced 
>>> with commit[3], the connecting works well after revert it.
>>> I'm not sure whether it's one test case issue or kernel issue, could 
>>> anyone help check it.
>>
>> Is this failure persistent or transient?
>>
> It's persistent, and most of recent CKI job with 5.8 stable kernel 
> also showed this failure.
>

Found? "WARNING: possible circular locking" with latest 5.10.0-rc1

[? 230.353213] run blktests nvme/003 at 2020-10-28 07:41:08
[? 230.666092] loop: module loaded
[? 230.870124] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[? 230.902252] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[? 246.097201] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[? 246.104886] nvmet: ctrl 1 fatal error occurred!

[? 246.113796] ======================================================
[? 246.120042] WARNING: possible circular locking dependency detected
[? 246.126288] 5.10.0-rc1 #1 Not tainted
[? 246.129987] ------------------------------------------------------
[? 246.136231] kworker/0:2/251 is trying to acquire lock:
[? 246.141423] ffff00012ec0b830 
((work_completion)(&queue->io_work)){+.+.}-{0:0}, at: 
__flush_work+0x48/0x4f0
[? 246.151202]
 ?????????????? but task is already holding lock:
[? 246.157095] ffff8000147e3dd8 
((work_completion)(&queue->release_work)){+.+.}-{0:0}, at: 
process_one_work+0x1bc/0x670
[? 246.167743]
 ?????????????? which lock already depends on the new lock.

[? 246.176007]
 ?????????????? the existing dependency chain (in reverse order) is:
[? 246.183569]
 ?????????????? -> #2 ((work_completion)(&queue->release_work)){+.+.}-{0:0}:
[? 246.191849]??????? validate_chain+0x5f0/0xa00
[? 246.196251]??????? __lock_acquire+0x3b0/0xb70
[? 246.200652]??????? lock_acquire+0x104/0x428
[? 246.204878]??????? process_one_work+0x21c/0x670
[? 246.209456]??????? worker_thread+0x54/0x408
[? 246.213684]??????? kthread+0x11c/0x148
[? 246.217473]??????? ret_from_fork+0x10/0x18
[? 246.221611]
 ?????????????? -> #1 ((wq_completion)events){+.+.}-{0:0}:
[? 246.228602]??????? validate_chain+0x5f0/0xa00
[? 246.233005]??????? __lock_acquire+0x3b0/0xb70
[? 246.237407]??????? lock_acquire+0x104/0x428
[? 246.241634]??????? flush_workqueue+0x8c/0x4a8
[? 246.246041]??????? nvmet_tcp_install_queue+0x100/0x110 [nvmet_tcp]
[? 246.252309]??????? nvmet_install_queue+0xc4/0x150 [nvmet]
[? 246.257775]??????? nvmet_execute_admin_connect+0xf4/0x1e0 [nvmet]
[? 246.263935]??????? nvmet_tcp_io_work+0x8dc/0x970 [nvmet_tcp]
[? 246.269654]??????? process_one_work+0x268/0x670
[? 246.274231]??????? worker_thread+0x54/0x408
[? 246.278458]??????? kthread+0x11c/0x148
[? 246.282246]??????? ret_from_fork+0x10/0x18
[? 246.288493]
 ?????????????? -> #0 ((work_completion)(&queue->io_work)){+.+.}-{0:0}:
[? 246.300559]??????? check_noncircular+0x100/0x128
[? 246.307367]??????? check_prev_add+0xa4/0x668
[? 246.313777]??????? validate_chain+0x5f0/0xa00
[? 246.320230]??????? __lock_acquire+0x3b0/0xb70
[? 246.326652]??????? lock_acquire+0x104/0x428
[? 246.332879]??????? __flush_work+0x70/0x4f0
[? 246.338994]??????? flush_work+0x1c/0x28
[? 246.344833]??????? nvmet_tcp_release_queue_work+0xd0/0x230 [nvmet_tcp]
[? 246.353405]??????? process_one_work+0x268/0x670
[? 246.359955]??????? worker_thread+0x54/0x408
[? 246.366131]??????? kthread+0x11c/0x148
[? 246.371843]??????? ret_from_fork+0x10/0x18
[? 246.377876]
 ?????????????? other info that might help us debug this:

[? 246.391416] Chain exists of:
 ???????????????? (work_completion)(&queue->io_work) --> 
(wq_completion)events --> (work_completion)(&queue->release_work)

[? 246.411895]? Possible unsafe locking scenario:

[? 246.421363]??????? CPU0??????????????????? CPU1
[? 246.427687]??????? ----??????????????????? ----
[? 246.433978] lock((work_completion)(&queue->release_work));
[? 246.441413] lock((wq_completion)events);
[? 246.449816] lock((work_completion)(&queue->release_work));
[? 246.459780]?? lock((work_completion)(&queue->io_work));
[? 246.466808]
 ??????????????? *** DEADLOCK ***

[? 246.478008] 2 locks held by kworker/0:2/251:
[? 246.484053]? #0: ffff000100051b38 
((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x1bc/0x670
[? 246.495355]? #1: ffff8000147e3dd8 
((work_completion)(&queue->release_work)){+.+.}-{0:0}, at: 
process_one_work+0x1bc/0x670
[? 246.508301]
 ?????????????? stack backtrace:
[? 246.516512] CPU: 0 PID: 251 Comm: kworker/0:2 Not tainted 5.10.0-rc1 #1
[? 246.525121] Hardware name: GIGABYTE R120-T32-00/MT30-GS1-00, BIOS F02 
08/06/2019
[? 246.534560] Workqueue: events nvmet_tcp_release_queue_work [nvmet_tcp]
[? 246.543156] Call trace:
[? 246.547634]? dump_backtrace+0x0/0x1c8
[? 246.553325]? show_stack+0x20/0x70
[? 246.558653]? dump_stack+0xf8/0x164
[? 246.564043]? print_circular_bug.isra.0+0x234/0x290
[? 246.570834]? check_noncircular+0x100/0x128
[? 246.576931]? check_prev_add+0xa4/0x668
[? 246.582676]? validate_chain+0x5f0/0xa00
[? 246.588512]? __lock_acquire+0x3b0/0xb70
[? 246.594346]? lock_acquire+0x104/0x428
[? 246.599986]? __flush_work+0x70/0x4f0
[? 246.605506]? flush_work+0x1c/0x28
[? 246.610765]? nvmet_tcp_release_queue_work+0xd0/0x230 [nvmet_tcp]
[? 246.618768]? process_one_work+0x268/0x670
[? 246.624777]? worker_thread+0x54/0x408
[? 246.630438]? kthread+0x11c/0x148
[? 246.635661]? ret_from_fork+0x10/0x18
[? 295.375201] nvme nvme0: queue 0: timeout request 0x0 type 4
[? 295.383074] nvme nvme0: Connect command failed, error wo/DNR bit: 881
[? 295.391912] nvme nvme0: failed to connect queue: 0 ret=881
[? 479.792299] nvmet: creating controller 1 for subsystem 
nqn.2014-08.org.nvmexpress.discovery for NQN 
nqn.2014-08.org.nvmexpress:uuid:0e657e35bc5f402480543a94c7a292ed.


>>>
>>> [1]
>>> # sh test.sh
>>> + ./stress-ng/stress-ng --unshare 0 --timeout 5 --log-file unshare.log
>>> stress-ng: info:? [355534] dispatching hogs: 32 unshare
>>> stress-ng: info:? [355534] successful run completed in 5.04s
>>> + modprobe null-blk nr-devices=1
>>> + modprobe nvmet-tcp
>>> + modprobe nvme-tcp
>>> + nvmetcli restore tcp.json
>>> + nvme connect -t tcp -n nqn.2014-08.org.nvmexpress.discovery -a 
>>> 127.0.0.1 -s 4420
>>> Failed to write to /dev/nvme-fabrics: Input/output error
>>>
>>> # dmesg | tail -9
>>> [? 700.012299] null_blk: module loaded
>>> [? 700.073415] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>>> [? 700.073923] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>>> [? 715.291020] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
>>> [? 715.297031] nvmet: ctrl 1 fatal error occurred!
>>> [? 749.939898] nvmet: creating controller 1 for subsystem 
>>> nqn.2014-08.org.nvmexpress.discovery for NQN 
>>> nqn.2014-08.org.nvmexpress:uuid:e405e6bb-8e28-4a73-b338-3fddb5746b8c.
>>> [? 763.417376] nvme nvme0: queue 0: timeout request 0x0 type 4
>>> [? 763.422979] nvme nvme0: Connect command failed, error wo/DNR bit: 
>>> 881
>>> [? 763.429419] nvme nvme0: failed to connect queue: 0 ret=881
>>>
>>> # uname -r
>>> 5.9.0-rc4
>>>
>>>
>>> [2] stress-ng: unshare case
>>> https://github.com/ColinIanKing/stress-ng.git
>>> https://github.com/ColinIanKing/stress-ng/blob/master/stress-unshare.c
>>>
>>>
>>> [3]
>>> commit e1eb26fa62d04ec0955432be1aa8722a97cb52e7
>>> Author: Giuseppe Scrivano <gscrivan at redhat.com>
>>> Date:?? Sun Jun 7 21:40:10 2020 -0700
>>>
>>> ???? ipc/namespace.c: use a work queue to free_ipc
>>> ???? ???? the reason is to avoid a delay caused by the 
>>> synchronize_rcu() call in
>>> ???? kern_umount() when the mqueue mount is freed.



From tom.leiming at gmail.com  Wed Oct 28 09:25:27 2020
From: tom.leiming at gmail.com (Ming Lei)
Date: Wed, 28 Oct 2020 21:25:27 +0800
Subject: [PATCH v2 0/5] avoid race for time out
In-Reply-To: <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
References: <20201022021452.10971-1-lengchao@huawei.com>
 <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
Message-ID: <CACVXFVMk3MWELDLFv44B=oAgwFnmuDCnSDe4QYE-+ipkxeZzLg@mail.gmail.com>

On Wed, Oct 28, 2020 at 7:38 PM Yi Zhang <yi.zhang at redhat.com> wrote:
>
> Hello
>
> This series fixed the WARNING issue I reported [1], but now the nvme/012
> [2] will be hang there and never finished, here is the log[3]

Hello Yi,

Please try the following patch against Chao's patches and see if the
hang issue can
be fixed:

https://lore.kernel.org/linux-block/20201020085301.1553959-2-ming.lei at redhat.com/T/#u


Thanks,
Ming Lei


From maorg at nvidia.com  Wed Oct 28 09:31:03 2020
From: maorg at nvidia.com (Maor Gottlieb)
Date: Wed, 28 Oct 2020 15:31:03 +0200
Subject: [PATCH rdma v2] RDMA: Add rdma_connect_locked()
In-Reply-To: <20201028121437.GU1523783@nvidia.com>
References: <0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com>
 <4401b7b1-5d05-a715-4701-957fd09f34c9@nvidia.com>
 <20201028121437.GU1523783@nvidia.com>
Message-ID: <8306260b-ce24-3803-7d39-42f5d2209af7@nvidia.com>


On 10/28/2020 2:14 PM, Jason Gunthorpe wrote:
> On Wed, Oct 28, 2020 at 11:19:14AM +0200, Maor Gottlieb wrote:
>>> +			struct rdma_conn_param *conn_param)
>>>    {
>>>    	struct rdma_id_private *id_priv =
>>>    		container_of(id, struct rdma_id_private, id);
>>>    	int ret;
>>> -	mutex_lock(&id_priv->handler_mutex);
>> You need to delete the mutex_unlock in success path too.
> Gaaaaah. Just goes to prove I shouldn't write patches with a child on
> my lap :\
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index f58d19881524dc..a77750b8954db0 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -4072,7 +4072,6 @@ int rdma_connect_locked(struct rdma_cm_id *id,
>   		ret = -ENOSYS;
>   	if (ret)
>   		goto err_state;
> -	mutex_unlock(&id_priv->handler_mutex);
>   	return 0;
>   err_state:
>   	cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED);
>
> Thanks,
> Jason

Reviewed-by: Maor Gottlieb <maorg at nvidia.com>


From mgurtovoy at nvidia.com  Wed Oct 28 12:58:08 2020
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Wed, 28 Oct 2020 18:58:08 +0200
Subject: [PATCH v3] nvme-rdma: handle nvme completion data length
In-Reply-To: <20201025115124.1430678-1-pizhenwei@bytedance.com>
References: <20201025115124.1430678-1-pizhenwei@bytedance.com>
Message-ID: <86fa6106-b969-4bb9-95ee-c1101a61ff03@nvidia.com>


On 10/25/2020 1:51 PM, zhenwei pi wrote:
> Hit a kernel warning:
> refcount_t: underflow; use-after-free.
> WARNING: CPU: 0 PID: 0 at lib/refcount.c:28
>
> RIP: 0010:refcount_warn_saturate+0xd9/0xe0
> Call Trace:
>   <IRQ>
>   nvme_rdma_recv_done+0xf3/0x280 [nvme_rdma]
>   __ib_process_cq+0x76/0x150 [ib_core]
>   ...
>
> The reason is that a zero bytes message received from target, and the
> host side continues to process without length checking, then the
> previous CQE is processed twice.
>
> Do sanity check on received data length, try to recovery for corrupted
> CQE case.
>
> Because zero bytes message in not defined in spec, using zero bytes
> message to detect dead connections on transport layer is not
> standard, currently still treat it as illegal.
>
> Thanks to Chao Leng & Sagi for suggestions.
>
> Signed-off-by: zhenwei pi <pizhenwei at bytedance.com>
> ---
>   drivers/nvme/host/rdma.c | 8 ++++++++
>   1 file changed, 8 insertions(+)
>
Seems strange that the targets sends zero byte packets.

Can you specify which target is this and the scenario ?



From hch at lst.de  Wed Oct 28 13:17:26 2020
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 28 Oct 2020 18:17:26 +0100
Subject: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running
 FW 220TQ,22001
In-Reply-To: <20201028091421.GA667673@image-900X5T-900X5U>
References: <20201028091421.GA667673@image-900X5T-900X5U>
Message-ID: <20201028171726.GA9897@lst.de>

On Wed, Oct 28, 2020 at 06:14:21PM +0900, Jongpil Jung wrote:
> LiteON(SSSTC) CL1 device running FW 220TQ,22001 has bugs with simple
> suspend.
> 
> When NVMe device receive D3hot from host, NVMe firmware will do
> garbage collection. While NVMe device do Garbage collection,
> firmware has chance to going incorrect address.

I'm still lost on what this means.  The device doesn't receive 'D3hot'
from the host, it receives a shutdown request.  And we don't just do
that when hibernating, but also when doing an actual shutdown, a suspend
to disk, unbinding the driver, etc.  So if the device has a problem
with shutdowns we really are in trouble.


From Gloria.Tsai at ssstc.com  Wed Oct 28 22:20:27 2020
From: Gloria.Tsai at ssstc.com (Gloria Tsai)
Date: Thu, 29 Oct 2020 02:20:27 +0000
Subject: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
In-Reply-To: <20201028171726.GA9897@lst.de>
References: <20201028091421.GA667673@image-900X5T-900X5U>
 <20201028171726.GA9897@lst.de>
Message-ID: <HK2PR02MB4004EC30D14A16B59FD22A44EE140@HK2PR02MB4004.apcprd02.prod.outlook.com>

Corrected the description of this bug that SSD will not do GC after receiving shutdown cmd.
Do GC before shutdown -> delete IO Q -> shutdown from host -> breakup GC -> D3hot -> enter PS4 -> have a chance swap block -> use wrong pointer on device SRAM -> over program

SSD gets in low power mode only in suspend, this issue only happens in low power mode. 
As for Hibernate, power off, unbinding the driver or even warm boot, EC boot, SSD won't be in low power state, so there'll be no problem.


Regards,
Gloria Tsai
_____________________________________

Sales PM Division
Solid State Storage Technology Corporation
TEL: +886-3-612-3888 ext. 2201
E-Mail: gloria.tsai at ssstc.com
_____________________________________

-----Original Message-----
From: Christoph Hellwig <hch at lst.de> 
Sent: Thursday, October 29, 2020 1:17 AM
To: Jongpil Jung <jongpuls at gmail.com>
Cc: Keith Busch <kbusch at kernel.org>; Jens Axboe <axboe at fb.com>; Christoph Hellwig <hch at lst.de>; Sagi Grimberg <sagi at grimberg.me>; linux-nvme at lists.infradead.org; linux-kernel at vger.kernel.org; Gloria Tsai <Gloria.Tsai at ssstc.com>; jongpil19.jung at samsung.com; jongheony.kim at samsung.com; dj54.sohn at samsung.com
Subject: Re: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running FW 220TQ,22001



This message was sent from outside of the company. Please do not click links or open attachments unless you recognize the source of this email and know the content is safe.


On Wed, Oct 28, 2020 at 06:14:21PM +0900, Jongpil Jung wrote:
> LiteON(SSSTC) CL1 device running FW 220TQ,22001 has bugs with simple 
> suspend.
>
> When NVMe device receive D3hot from host, NVMe firmware will do 
> garbage collection. While NVMe device do Garbage collection, firmware 
> has chance to going incorrect address.

I'm still lost on what this means.  The device doesn't receive 'D3hot'
from the host, it receives a shutdown request.  And we don't just do that when hibernating, but also when doing an actual shutdown, a suspend to disk, unbinding the driver, etc.  So if the device has a problem with shutdowns we really are in trouble.


From kbusch at kernel.org  Wed Oct 28 22:33:06 2020
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 29 Oct 2020 11:33:06 +0900
Subject: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
In-Reply-To: <HK2PR02MB4004EC30D14A16B59FD22A44EE140@HK2PR02MB4004.apcprd02.prod.outlook.com>
References: <20201028091421.GA667673@image-900X5T-900X5U>
 <20201028171726.GA9897@lst.de>
 <HK2PR02MB4004EC30D14A16B59FD22A44EE140@HK2PR02MB4004.apcprd02.prod.outlook.com>
Message-ID: <20201029023306.GB20928@redsun51.ssa.fujisawa.hgst.com>

On Thu, Oct 29, 2020 at 02:20:27AM +0000, Gloria Tsai wrote:
> Corrected the description of this bug that SSD will not do GC after receiving shutdown cmd.
> Do GC before shutdown -> delete IO Q -> shutdown from host -> breakup GC -> D3hot -> enter PS4 -> have a chance swap block -> use wrong pointer on device SRAM -> over program

What do you mean by "wrong pointer"? At the place in the sequence you're
referring to, the PCI BME is disabled: you can't access *any* host RAM,
so there's no "correct" pointer either.


From kbusch at kernel.org  Wed Oct 28 23:15:21 2020
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 29 Oct 2020 12:15:21 +0900
Subject: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
In-Reply-To: <20201029023306.GB20928@redsun51.ssa.fujisawa.hgst.com>
References: <20201028091421.GA667673@image-900X5T-900X5U>
 <20201028171726.GA9897@lst.de>
 <HK2PR02MB4004EC30D14A16B59FD22A44EE140@HK2PR02MB4004.apcprd02.prod.outlook.com>
 <20201029023306.GB20928@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <20201029031521.GC20928@redsun51.ssa.fujisawa.hgst.com>

On Thu, Oct 29, 2020 at 11:33:06AM +0900, Keith Busch wrote:
> On Thu, Oct 29, 2020 at 02:20:27AM +0000, Gloria Tsai wrote:
> > Corrected the description of this bug that SSD will not do GC after receiving shutdown cmd.
> > Do GC before shutdown -> delete IO Q -> shutdown from host -> breakup GC -> D3hot -> enter PS4 -> have a chance swap block -> use wrong pointer on device SRAM -> over program
> 
> What do you mean by "wrong pointer"? At the place in the sequence you're
> referring to, the PCI BME is disabled: you can't access *any* host RAM,
> so there's no "correct" pointer either.

Re-reading your message, I do see you said "device" rather than "host",
so my response may not be relevant.


From Gloria.Tsai at ssstc.com  Wed Oct 28 23:21:27 2020
From: Gloria.Tsai at ssstc.com (Gloria Tsai)
Date: Thu, 29 Oct 2020 03:21:27 +0000
Subject: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running FW
 220TQ,22001
In-Reply-To: <20201029031521.GC20928@redsun51.ssa.fujisawa.hgst.com>
References: <20201028091421.GA667673@image-900X5T-900X5U>
 <20201028171726.GA9897@lst.de>
 <HK2PR02MB4004EC30D14A16B59FD22A44EE140@HK2PR02MB4004.apcprd02.prod.outlook.com>
 <20201029023306.GB20928@redsun51.ssa.fujisawa.hgst.com>
 <20201029031521.GC20928@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <HK2PR02MB4004E96D4FEF0589232CF694EE140@HK2PR02MB4004.apcprd02.prod.outlook.com>

Correct, it's device SRAM.


Regards,
Gloria Tsai
_____________________________________

Sales PM Division
Solid State Storage Technology Corporation
TEL: +886-3-612-3888 ext. 2201
E-Mail: gloria.tsai at ssstc.com
_____________________________________

-----Original Message-----
From: Keith Busch <kbusch at kernel.org> 
Sent: Thursday, October 29, 2020 11:15 AM
To: Gloria Tsai <Gloria.Tsai at ssstc.com>
Cc: Jongpil Jung <jongpuls at gmail.com>; Sagi Grimberg <sagi at grimberg.me>; linux-kernel at vger.kernel.org; linux-nvme at lists.infradead.org; Jens Axboe <axboe at fb.com>; dj54.sohn at samsung.com; jongpil19.jung at samsung.com; jongheony.kim at samsung.com; Christoph Hellwig <hch at lst.de>
Subject: Re: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running FW 220TQ,22001



This message was sent from outside of the company. Please do not click links or open attachments unless you recognize the source of this email and know the content is safe.


On Thu, Oct 29, 2020 at 11:33:06AM +0900, Keith Busch wrote:
> On Thu, Oct 29, 2020 at 02:20:27AM +0000, Gloria Tsai wrote:
> > Corrected the description of this bug that SSD will not do GC after receiving shutdown cmd.
> > Do GC before shutdown -> delete IO Q -> shutdown from host -> 
> > breakup GC -> D3hot -> enter PS4 -> have a chance swap block -> use 
> > wrong pointer on device SRAM -> over program
>
> What do you mean by "wrong pointer"? At the place in the sequence 
> you're referring to, the PCI BME is disabled: you can't access *any* 
> host RAM, so there's no "correct" pointer either.

Re-reading your message, I do see you said "device" rather than "host", so my response may not be relevant.


From lengchao at huawei.com  Thu Oct 29 02:13:20 2020
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 29 Oct 2020 14:13:20 +0800
Subject: [PATCH v2 0/5] avoid race for time out
In-Reply-To: <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
References: <20201022021452.10971-1-lengchao@huawei.com>
 <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
Message-ID: <a800631e-5a54-fca6-ba70-2fe3c18a7ece@huawei.com>



On 2020/10/28 19:36, Yi Zhang wrote:
> Hello
> 
> This series fixed the WARNING issue I reported [1], but now the nvme/012 [2] will be hang there and never finished, here is the log[3]
This is another bug. In two scenarios may cause request hang:
1. If work with nvme native multipath, all path is not availabble.
request will hang until all controller are deleted.
2. if work without multipath, controller is reconnectting.
request will hang until the controller are deleted.
This patch may fix the request hang.
https://lore.kernel.org/linux-nvme/319b8b1869f34a48b26fbd902883ed71 at kioxia.com/
This patch has been discussed for too long time.

> [1]
> https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/
> 
> [2]
> [root at hpe-xw9400-02 blktests]# nvme_trtype=tcp ./check nvme/012
> nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns)
>  ??? runtime? 1199.651s? ...
> 
> [3]
> [? 120.550409] run blktests nvme/012 at 2020-10-28 06:50:11
> [? 121.138234] loop: module loaded
> [? 121.170869] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
> [? 121.215930] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
> [? 121.288229] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
> [? 121.302597] nvme nvme0: creating 12 I/O queues.
> [? 121.308361] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [? 121.320030] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420
> [? 123.278903] XFS (nvme0n1): Mounting V5 Filesystem
> [? 123.291608] XFS (nvme0n1): Ending clean mount
> [? 123.297321] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff)
> [? 183.872118] nvme nvme0: queue 1: timeout request 0x6c type 4
> [? 183.877792] nvme nvme0: starting error recovery
> [? 183.882376] nvme nvme0: queue 8: timeout request 0x11 type 4
> [? 183.888149] nvme nvme0: queue 8: timeout request 0x12 type 4
> [? 183.893805] nvme nvme0: queue 8: timeout request 0x13 type 4
> [? 183.899469] nvme nvme0: queue 8: timeout request 0x14 type 4
> [? 183.905130] nvme nvme0: queue 8: timeout request 0x15 type 4
> [? 183.910792] nvme nvme0: queue 8: timeout request 0x16 type 4
> [? 183.916453] nvme nvme0: queue 8: timeout request 0x17 type 4
> [? 183.922114] nvme nvme0: queue 8: timeout request 0x18 type 4
> [? 183.927777] nvme nvme0: queue 8: timeout request 0x19 type 4
> [? 183.933450] nvme nvme0: queue 8: timeout request 0x1a type 4
> [? 183.939110] nvme nvme0: queue 8: timeout request 0x1b type 4
> [? 183.944771] nvme nvme0: queue 8: timeout request 0x1c type 4
> [? 183.950431] nvme nvme0: queue 8: timeout request 0x1d type 4
> [? 183.956095] nvme nvme0: queue 8: timeout request 0x1e type 4
> [? 183.961755] nvme nvme0: queue 8: timeout request 0x1f type 4
> [? 183.967414] nvme nvme0: queue 8: timeout request 0x20 type 4
> [? 183.973218] block nvme0n1: no usable path - requeuing I/O
> [? 183.978623] block nvme0n1: no usable path - requeuing I/O
> [? 183.982492] nvme nvme0: Reconnecting in 10 seconds...
> [? 183.984022] block nvme0n1: no usable path - requeuing I/O
> [? 183.994476] block nvme0n1: no usable path - requeuing I/O
> [? 183.999870] block nvme0n1: no usable path - requeuing I/O
> [? 184.005264] block nvme0n1: no usable path - requeuing I/O
> [? 184.010669] block nvme0n1: no usable path - requeuing I/O
> [? 184.016080] block nvme0n1: no usable path - requeuing I/O
> [? 184.021463] block nvme0n1: no usable path - requeuing I/O
> [? 184.026858] block nvme0n1: no usable path - requeuing I/O
> [? 209.472647] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
> [? 209.478662] nvmet: ctrl 2 fatal error occurred!
> [? 213.568765] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
> [? 213.574782] nvmet: ctrl 1 fatal error occurred!
> [? 238.064572] nvmet: creating controller 2 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
> [? 256.577658] nvme nvme0: queue 0: timeout request 0x0 type 4
> [? 256.583333] nvme nvme0: Connect command failed, error wo/DNR bit: 881
> [? 256.589806] nvme nvme0: failed to connect queue: 0 ret=881
> [? 256.595326] nvme nvme0: Failed reconnect attempt 1
> [? 256.600119] nvme nvme0: Reconnecting in 10 seconds...
> [? 266.818455] nvmet: creating controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
> [? 266.832356] nvme_ns_head_submit_bio: 30 callbacks suppressed
> [? 266.832362] block nvme0n1: no usable path - requeuing I/O
> [? 266.843443] block nvme0n1: no usable path - requeuing I/O
> [? 266.848848] block nvme0n1: no usable path - requeuing I/O
> [? 266.854244] block nvme0n1: no usable path - requeuing I/O
> [? 266.859663] block nvme0n1: no usable path - requeuing I/O
> [? 266.865059] block nvme0n1: no usable path - requeuing I/O
> [? 266.870454] block nvme0n1: no usable path - requeuing I/O
> [? 266.875845] block nvme0n1: no usable path - requeuing I/O
> [? 266.881234] block nvme0n1: no usable path - requeuing I/O
> [? 266.886632] block nvme0n1: no usable path - requeuing I/O
> [? 266.892237] nvme nvme0: creating 12 I/O queues.
> [? 266.903744] nvme nvme0: mapped 12/0/0 default/read/poll queues.
> [? 266.911929] nvme nvme0: Successfully reconnected (2 attempt)
> [? 327.747177] nvme nvme0: queue 2: timeout request 0x1e type 4
> [? 327.752883] nvme nvme0: starting error recovery
> [? 327.757450] nvme nvme0: queue 4: timeout request 0x63 type 4
> [? 327.763511] nvme_ns_head_submit_bio: 14 callbacks suppressed
> [? 327.763520] block nvme0n1: no usable path - requeuing I/O
> [? 327.774614] block nvme0n1: no usable path - requeuing I/O
> [? 327.780053] block nvme0n1: no usable path - requeuing I/O
> [? 327.785450] block nvme0n1: no usable path - requeuing I/O
> [? 327.790876] block nvme0n1: no usable path - requeuing I/O
> [? 327.796316] block nvme0n1: no usable path - requeuing I/O
> [? 327.801727] block nvme0n1: no usable path - requeuing I/O
> [? 327.807231] block nvme0n1: no usable path - requeuing I/O
> [? 327.812627] block nvme0n1: no usable path - requeuing I/O
> [? 327.818025] block nvme0n1: no usable path - requeuing I/O
> [? 353.859745] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
> [? 353.865761] nvmet: ctrl 1 fatal error occurred!
> 
> 
> On 10/22/20 10:14 AM, Chao Leng wrote:
>> First avoid race between time out and tear down for rdma and tcp.
>> Second avoid repeated request completion in time out for rdma and tcp.
>>
>> V2:
>> ????- add avoiding repeated request completion in time out
>>
>> Chao Leng (3):
>> ?? nvme-core: introduce sync io queues
>> ?? nvme-rdma: avoid race between time out and tear down
>> ?? nvme-tcp: avoid race between time out and tear down
>>
>> Sagi Grimberg (2):
>> ?? nvme-rdma: avoid repeated request completion
>> ?? nvme-tcp: avoid repeated request completion
>>
>> ? drivers/nvme/host/core.c |? 8 ++++++--
>> ? drivers/nvme/host/nvme.h |? 1 +
>> ? drivers/nvme/host/rdma.c | 14 +++-----------
>> ? drivers/nvme/host/tcp.c? | 16 ++++------------
>> ? 4 files changed, 14 insertions(+), 25 deletions(-)
>>
> 
> .


From hch at infradead.org  Thu Oct 29 10:49:34 2020
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 29 Oct 2020 15:49:34 +0100
Subject: [GIT PULL] nvme fixes for 5.10
Message-ID: <20201029144934.GA143642@infradead.org>

The following changes since commit f255c19b3ab46d3cad3b1b2e1036f4c926cb1d0c:

  blk-cgroup: Pre-allocate tree node on blkg_conf_prep (2020-10-26 07:57:47 -0600)

are available in the Git repository at:

  git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-29

for you to fetch changes up to 3c3751f2daf6675f6b5bee83b792354c272f5bd2:

  nvmet: fix a NULL pointer dereference when tracing the flush command (2020-10-27 10:02:50 +0100)

----------------------------------------------------------------
nvme updates for 5.10:

 - improve zone revalidation (Keith Busch)
 - gracefully handle zero length messages in nvme-rdma (zhenwei pi)
 - nvme-fc error handling fixes (James Smart)
 - nvmet tracing NULL pointer dereference fix (Chaitanya Kulkarni)

----------------------------------------------------------------
Chaitanya Kulkarni (1):
      nvmet: fix a NULL pointer dereference when tracing the flush command

James Smart (4):
      nvme-fc: track error_recovery while connecting
      nvme-fc: remove err_work work item
      nvme-fc: eliminate terminate_io use by nvme_fc_error_recovery
      nvme-fc: remove nvme_fc_terminate_io()

Keith Busch (1):
      nvme: ignore zone validate errors on subsequent scans

zhenwei pi (1):
      nvme-rdma: handle unexpected nvme completion data length

 drivers/nvme/host/core.c    |   2 +-
 drivers/nvme/host/fc.c      | 270 ++++++++++++++++++--------------------------
 drivers/nvme/host/rdma.c    |   8 ++
 drivers/nvme/target/core.c  |   4 +-
 drivers/nvme/target/trace.h |  21 ++--
 5 files changed, 127 insertions(+), 178 deletions(-)


From axboe at kernel.dk  Thu Oct 29 10:54:57 2020
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 29 Oct 2020 08:54:57 -0600
Subject: [GIT PULL] nvme fixes for 5.10
In-Reply-To: <20201029144934.GA143642@infradead.org>
References: <20201029144934.GA143642@infradead.org>
Message-ID: <306d09c9-be4c-d535-c397-920b960a265f@kernel.dk>

On 10/29/20 8:49 AM, Christoph Hellwig wrote:
> The following changes since commit f255c19b3ab46d3cad3b1b2e1036f4c926cb1d0c:
> 
>   blk-cgroup: Pre-allocate tree node on blkg_conf_prep (2020-10-26 07:57:47 -0600)
> 
> are available in the Git repository at:
> 
>   git://git.infradead.org/nvme.git tags/nvme-5.10-2020-10-29

Pulled, thanks.

-- 
Jens Axboe



From hch at lst.de  Thu Oct 29 10:55:29 2020
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 29 Oct 2020 15:55:29 +0100
Subject: [PATCH V3 1/1] nvme: Add quirk for LiteON CL1 devices running
 FW 220TQ,22001
In-Reply-To: <20201028091421.GA667673@image-900X5T-900X5U>
References: <20201028091421.GA667673@image-900X5T-900X5U>
Message-ID: <20201029145529.GA19011@lst.de>

I'm still worried about this.

If power state based suspend does always work despite a HMB and is
preferred for the specific Google board we should have purely a DMI
based quirk for the board independent of the NVMe controller used with
it.

But if these LiteON devices can't properly handle nvme_dev_disable
calls we have much deeper problems, because it can be called in all
kinds of places, including suspending when not on this specific board.

That being said, I still really do not understand this sentence and thus
the problem at all:

> When NVMe device receive D3hot from host, NVMe firmware will do
> garbage collection. While NVMe device do Garbage collection,
> firmware has chance to going incorrect address.


From javier at javigon.com  Thu Oct 29 14:57:53 2020
From: javier at javigon.com (=?UTF-8?q?Javier=20Gonz=C3=A1lez?=)
Date: Thu, 29 Oct 2020 19:57:53 +0100
Subject: [PATCH] nvme: report capacity 0 for non supported ZNS SSDs
Message-ID: <20201029185753.14368-1-javier.gonz@samsung.com>

Allow ZNS SSDs to be presented to the host even when they implement
features that are not supported by the kernel zoned block device.

Instead of rejecting the SSD at the NVMe driver level, deal with this in
the block layer by setting capacity to 0, as we do with other things
such as unsupported PI configurations. This allows to use standard
management tools such as nvme-cli to choose a different format or
firmware slot that is compatible with the Linux zoned block device.

Signed-off-by: Javier Gonz?lez <javier.gonz at samsung.com>
---
 drivers/nvme/host/core.c |  5 +++++
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/zns.c  | 31 ++++++++++++++-----------------
 3 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c190c56bf702..9ca4f0a6ff2c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2026,6 +2026,11 @@ static void nvme_update_disk_info(struct gendisk *disk,
 			capacity = 0;
 	}
 
+#ifdef CONFIG_BLK_DEV_ZONED
+	if (!ns->zone_sup)
+		capacity = 0;
+#endif
+
 	set_capacity_revalidate_and_notify(disk, capacity, false);
 
 	nvme_config_discard(disk, ns);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 87737fa32360..42cbe5bbc518 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -443,6 +443,7 @@ struct nvme_ns {
 	u8 pi_type;
 #ifdef CONFIG_BLK_DEV_ZONED
 	u64 zsze;
+	bool zone_sup;
 #endif
 	unsigned long features;
 	unsigned long flags;
diff --git a/drivers/nvme/host/zns.c b/drivers/nvme/host/zns.c
index 57cfd78731fb..77a7fed508ef 100644
--- a/drivers/nvme/host/zns.c
+++ b/drivers/nvme/host/zns.c
@@ -44,20 +44,23 @@ int nvme_update_zone_info(struct gendisk *disk, struct nvme_ns *ns,
 	struct nvme_id_ns_zns *id;
 	int status;
 
-	/* Driver requires zone append support */
+	ns->zone_sup = true;
+
 	if (!(le32_to_cpu(log->iocs[nvme_cmd_zone_append]) &
 			NVME_CMD_EFFECTS_CSUPP)) {
+		ns->zone_sup = false;
 		dev_warn(ns->ctrl->device,
 			"append not supported for zoned namespace:%d\n",
 			ns->head->ns_id);
-		return -EINVAL;
-	}
-
-	/* Lazily query controller append limit for the first zoned namespace */
-	if (!ns->ctrl->max_zone_append) {
-		status = nvme_set_max_append(ns->ctrl);
-		if (status)
-			return status;
+	} else {
+		/* Lazily query controller append limit for the first
+		 * zoned namespace
+		 */
+		if (!ns->ctrl->max_zone_append) {
+			status = nvme_set_max_append(ns->ctrl);
+			if (status)
+				return status;
+		}
 	}
 
 	id = kzalloc(sizeof(*id), GFP_KERNEL);
@@ -73,25 +76,19 @@ int nvme_update_zone_info(struct gendisk *disk, struct nvme_ns *ns,
 	if (status)
 		goto free_data;
 
-	/*
-	 * We currently do not handle devices requiring any of the zoned
-	 * operation characteristics.
-	 */
 	if (id->zoc) {
+		ns->zone_sup = false;
 		dev_warn(ns->ctrl->device,
 			"zone operations:%x not supported for namespace:%u\n",
 			le16_to_cpu(id->zoc), ns->head->ns_id);
-		status = -EINVAL;
-		goto free_data;
 	}
 
 	ns->zsze = nvme_lba_to_sect(ns, le64_to_cpu(id->lbafe[lbaf].zsze));
 	if (!is_power_of_2(ns->zsze)) {
+		ns->zone_sup = false;
 		dev_warn(ns->ctrl->device,
 			"invalid zone size:%llu for namespace:%u\n",
 			ns->zsze, ns->head->ns_id);
-		status = -EINVAL;
-		goto free_data;
 	}
 
 	q->limits.zoned = BLK_ZONED_HM;
-- 
2.17.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 14:59:56 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 11:59:56 -0700
Subject: [PATCH 0/2] nvme-core: pr related small cleanups
Message-ID: <20201029185958.7013-1-chaitanya.kulkarni@wdc.com>

Hi,

This is purely cleanup patch-series for nvme-core pr related code.

-ck

Chaitanya Kulkarni (2):
  nvme-core: cleanup nvme_pr_type()
  nvme-core: add new line after variable declatation

 drivers/nvme/host/core.c | 23 +++++++----------------
 1 file changed, 7 insertions(+), 16 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 14:59:57 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 11:59:57 -0700
Subject: [PATCH 1/2] nvme-core: cleanup nvme_pr_type()
In-Reply-To: <20201029185958.7013-1-chaitanya.kulkarni@wdc.com>
References: <20201029185958.7013-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029185958.7013-2-chaitanya.kulkarni@wdc.com>

The switch in the nvme_pr_type() returns values which can be mapped 1:1
on the PR_XXX macors. Get rid of the long switch and replace with simple
conditional operator to return the value.

This is purely code cleanup and doesn't change any functionality.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 376096bfc54a..16f590a03486 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2149,23 +2149,11 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_id_ns *id)
 
 static char nvme_pr_type(enum pr_type type)
 {
-	switch (type) {
-	case PR_WRITE_EXCLUSIVE:
-		return 1;
-	case PR_EXCLUSIVE_ACCESS:
-		return 2;
-	case PR_WRITE_EXCLUSIVE_REG_ONLY:
-		return 3;
-	case PR_EXCLUSIVE_ACCESS_REG_ONLY:
-		return 4;
-	case PR_WRITE_EXCLUSIVE_ALL_REGS:
-		return 5;
-	case PR_EXCLUSIVE_ACCESS_ALL_REGS:
-		return 6;
-	default:
+	if (type < PR_WRITE_EXCLUSIVE || type > PR_EXCLUSIVE_ACCESS_ALL_REGS)
 		return 0;
-	}
-};
+
+	return type;
+}
 
 static int nvme_pr_command(struct block_device *bdev, u32 cdw10,
 				u64 key, u64 sa_key, u8 op)
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 14:59:58 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 11:59:58 -0700
Subject: [PATCH 2/2] nvme-core: add new line after variable declatation
In-Reply-To: <20201029185958.7013-1-chaitanya.kulkarni@wdc.com>
References: <20201029185958.7013-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029185958.7013-3-chaitanya.kulkarni@wdc.com>

Add a new line in functions nvme_pr_preempt(), nvme_pr_clear() and
nvme_pr_release() after variable declaration which follows the rest of
the code in the host/core.c.

This is purely code cleanup and doesn't change any functionality.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 16f590a03486..b4fcdc373144 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2212,18 +2212,21 @@ static int nvme_pr_preempt(struct block_device *bdev, u64 old, u64 new,
 		enum pr_type type, bool abort)
 {
 	u32 cdw10 = nvme_pr_type(type) << 8 | (abort ? 2 : 1);
+
 	return nvme_pr_command(bdev, cdw10, old, new, nvme_cmd_resv_acquire);
 }
 
 static int nvme_pr_clear(struct block_device *bdev, u64 key)
 {
 	u32 cdw10 = 1 | (key ? 1 << 3 : 0);
+
 	return nvme_pr_command(bdev, cdw10, key, 0, nvme_cmd_resv_register);
 }
 
 static int nvme_pr_release(struct block_device *bdev, u64 key, enum pr_type type)
 {
 	u32 cdw10 = nvme_pr_type(type) << 8 | (key ? 1 << 3 : 0);
+
 	return nvme_pr_command(bdev, cdw10, key, 0, nvme_cmd_resv_release);
 }
 
-- 
2.22.1



From Chaitanya.Kulkarni at wdc.com  Thu Oct 29 15:02:27 2020
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 19:02:27 +0000
Subject: [PATCH V3 6/6] nvmet: use inline bio for passthru fast path
References: <20201022010234.8304-1-chaitanya.kulkarni@wdc.com>
 <20201022010234.8304-7-chaitanya.kulkarni@wdc.com>
 <9ba9c9ba-7caf-e24c-1471-62c199cfcd4a@deltatee.com>
Message-ID: <BYAPR04MB4965980B745F28E069B8460186140@BYAPR04MB4965.namprd04.prod.outlook.com>

On 10/22/20 08:58, Logan Gunthorpe wrote:
>
>
> On 2020-10-21 7:02 p.m., Chaitanya Kulkarni wrote:
>> In nvmet_passthru_execute_cmd() which is a high frequency function
>> it uses bio_alloc() which leads to memory allocation from the fs pool
>> for each I/O.
>>
>> For NVMeoF nvmet_req we already have inline_bvec allocated as a part of
>> request allocation that can be used with preallocated bio when we
>> already know the size of request before bio allocation with bio_alloc(),
>> which we already do.
>>
>> Introduce a bio member for the nvmet_req passthru anon union. In the
>> fast path, check if we can get away with inline bvec and bio from
>> nvmet_req with bio_init() call before actually allocating from the
>> bio_alloc().
>>
>> This will be useful to avoid any new memory allocation under high
>> memory pressure situation and get rid of any extra work of
>> allocation (bio_alloc()) vs initialization (bio_init()) when
>> transfer len is < NVMET_MAX_INLINE_DATA_LEN that user can configure at
>> compile time.
>>
>> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
>> ---
>>  drivers/nvme/target/nvmet.h    |  1 +
>>  drivers/nvme/target/passthru.c | 20 ++++++++++++++++++--
>>  2 files changed, 19 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
>> index 559a15ccc322..408a13084fb4 100644
>> --- a/drivers/nvme/target/nvmet.h
>> +++ b/drivers/nvme/target/nvmet.h
>> @@ -330,6 +330,7 @@ struct nvmet_req {
>>  			struct work_struct      work;
>>  		} f;
>>  		struct {
>> +			struct bio		inline_bio;
>>  			struct request		*rq;
>>  			struct work_struct      work;
>>  			bool			use_workqueue;
>> diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
>> index 496ffedb77dc..32498b4302cc 100644
>> --- a/drivers/nvme/target/passthru.c
>> +++ b/drivers/nvme/target/passthru.c
>> @@ -178,6 +178,14 @@ static void nvmet_passthru_req_done(struct request *rq,
>>  	blk_mq_free_request(rq);
>>  }
>>  
>> +static void nvmet_passthru_bio_done(struct bio *bio)
>> +{
>> +	struct nvmet_req *req = bio->bi_private;
>> +
>> +	if (bio != &req->p.inline_bio)
>> +		bio_put(bio);
>> +}
>> +
>>  static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>>  {
>>  	int sg_cnt = req->sg_cnt;
>> @@ -186,13 +194,21 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
>>  	int i;
>>  
>>  	bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
>> -	bio->bi_end_io = bio_put;
>> +	if (req->transfer_len <= NVMET_MAX_INLINE_DATA_LEN) {
>> +		bio = &req->p.inline_bio;
>> +		bio_init(bio, req->inline_bvec, ARRAY_SIZE(req->inline_bvec));
>> +	} else {
>> +		bio = bio_alloc(GFP_KERNEL, min(sg_cnt, BIO_MAX_PAGES));
>> +	}
>> +
>> +	bio->bi_end_io = nvmet_passthru_bio_done;
> I still think it's cleaner to change bi_endio for the inline/alloc'd
> cases by simply setting bi_endi_io to bio_put() only in the bio_alloc
> case. This should also be more efficient as it's one less indirect call
> and condition for the inline case.
>
> Besides that, the entire series looks good to me.
>
> Reviewed-by: Logan Gunthorpe <logang at deltatee.com>
>
> Logan
>
Sagi/Christoph, any comments on this one ?

This series been sitting out for a while now.



From sagi at grimberg.me  Thu Oct 29 17:00:41 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 29 Oct 2020 14:00:41 -0700
Subject: [PATCH v2 0/5] avoid race for time out
In-Reply-To: <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
References: <20201022021452.10971-1-lengchao@huawei.com>
 <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
Message-ID: <e913e433-d8ef-c04b-d9fa-3a6d71657136@grimberg.me>


> Hello
> 
> This series fixed the WARNING issue I reported [1], but now the nvme/012 
> [2] will be hang there and never finished, here is the log[3]
> [1]
> https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/ 
> 
> 
> [2]
> [root at hpe-xw9400-02 blktests]# nvme_trtype=tcp ./check nvme/012
> nvme/012 (run mkfs and data verification fio job on NVMeOF block 
> device-backed ns)
>  ??? runtime? 1199.651s? ...

Hey Yi,

This is a different issue, as I said, first of all the test is not
designed to trigger this scenario so it is weird why it even happens.

For debug purposes, can you switch to run blktests with siw for
comparison?

This patch set should move forward for inclusion regardless.


From sagi at grimberg.me  Thu Oct 29 17:01:55 2020
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 29 Oct 2020 14:01:55 -0700
Subject: [PATCH v2 0/5] avoid race for time out
In-Reply-To: <e913e433-d8ef-c04b-d9fa-3a6d71657136@grimberg.me>
References: <20201022021452.10971-1-lengchao@huawei.com>
 <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
 <e913e433-d8ef-c04b-d9fa-3a6d71657136@grimberg.me>
Message-ID: <22167d2f-2b86-6030-8949-eeeab92fcae9@grimberg.me>


> For debug purposes, can you switch to run blktests with siw for
> comparison?

Disregard this comment, forgot which transport we are talking about
for a second...


From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:32 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:32 -0700
Subject: [PATCH V5 0/7] nvme-core: timeout related fixes and cleanup
Message-ID: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>

Hi,

This patch series uses NVME_IO_TIMEOUT for the sync request submission
and user request submission when the timeout is not specified by the 
caller and request queue data is set to NULL which is true for admin 
queue.

Also in this version, I've added timeout values setting for the NVMeOF
passthru controller given that passthru VUCs (both admin and I/O VUCs)
can differ in execution time than general-purpose admin and I/O command
set following is the detailed scenario :-

Consider NVMeOF target is setup in the passthru mode and both target and 
host modules are loaded with the default timeout values present
in the nvme-core.

1. User issues nvme cmd from the host to the target of nvme-cli where
?? ??timeout is taken from the controller log page ??e.g. :-
?? ??# nvme io-passthru --timeout=XXX ... 
?? ??OR ??
?? ??# nvme admin-passthru --timeout=XXX ... 
?? ??The timeout value in the above example is greater than the default
?? ??timeout value present in host-core for both on host machine and the 
?? ??target machine.
2. The host-core on the host side will set the timeout for the host
?? ??request and issues NVMe cmd over transport.
3. In the absence of [1] target will set the timeout value which is
?? ??default timeout and that is smaller than the timeout set on the host
?? ??side.
4. Due to lack of timeout value not passed over the transport leading
?? ??to smaller timeout value for the request on the target than the host
?? ??side, target side command will timeout.
?? ??
?? ??This breaks the scenario where the same command with timeout value
   from nvme-cli is able to execute with success when issued to NVMe
   PCIe ctrl but will fail when it is executed on the NVMeOF Passthru
   ctrl.

Regards,
Chaitanya

[1] nvmet: add passthru admin timeout value attr
    nvmet: add passthru io timeout value attr

* Changes from V4:-

1. Rebase and retest on nvme-5.10.

* Changes from V3:-

1. Rebase and retest on nvme-5.10.
2. Update the cover-letter with detailed scenario why NVMeOF target
   passthru is needed.
3. Wrap the line under 80 char for the last patch.
4. Change the nvme_default_timeout() -> nvme_set_req_default_timeout()
   such that it will not take timeout argument.

* Changes from V2:-

1. Introduce nvme_defalt_timeout() helper and use it in host/core.c.
2. Use nvme_default_timeout() in the lightnvme.c

* Changes from V1:-

1. Instead of using qid to decide IO or ADMIN timeout use request
   queue's queuedata whch we only set for non admin queue
   __nvme_submit_sync_cmd().
2. Add second patch to set IO timeout for nvme_submit_user_cmd().
3. Set the NVMeOF passthru ctrl timeout values with default values from
   nvme-core module.
4. Add admin and I/O timeout configfs attributes for NVMeOF passthru
   controller.

Chaitanya Kulkarni (7):
  nvme-core: use I/O timeout in submit sync cmd
  nvme-core: use I/O timeout in nvme_submit_user_cmd
  lightnvm: use I/O timeout in nvm submit user cmd
  nvmet: set default timeout for passthru requests
  nvmet: add passthru admin timeout value attr
  nvmet: add passthru io timeout value attr
  nvme: use consistent macro name for timeout

 drivers/nvme/host/core.c       | 10 ++++--
 drivers/nvme/host/fc.c         |  2 +-
 drivers/nvme/host/lightnvm.c   |  3 +-
 drivers/nvme/host/nvme.h       | 10 +++++-
 drivers/nvme/host/pci.c        |  8 ++---
 drivers/nvme/host/rdma.c       |  2 +-
 drivers/nvme/host/tcp.c        |  2 +-
 drivers/nvme/target/configfs.c | 64 ++++++++++++++++++++++++++++++++++
 drivers/nvme/target/loop.c     |  2 +-
 drivers/nvme/target/nvmet.h    |  2 ++
 drivers/nvme/target/passthru.c | 11 ++++++
 11 files changed, 103 insertions(+), 13 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:33 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:33 -0700
Subject: [PATCH V5 1/7] nvme-core: use I/O timeout in submit sync cmd
In-Reply-To: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
References: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029234739.21266-2-chaitanya.kulkarni@wdc.com>

In the function __nvme_submit_sync_cmd() it uses ADMIN_TIMEOUT when
caller doesn't specify value for the timeout variable. This function is
also called from the NVMe commands contexts (nvme_pr_command()/
nvme_ns_report_zones()) where NVME_IO_TIMEOUT can be used instead of
ADMIN_TIMEOUT.

For now we don't set the request queue's queuedata for admin command.

Introduce a helper nvme_default_timeout() and when timeout is not set
for the block layer request  based on the request queue's queuedata,
set Admin timeout else I/O timeout.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 3 ++-
 drivers/nvme/host/nvme.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 95ef4943d8bd..42ca1b04b6d9 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -901,7 +901,8 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
-	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	if (!req->timeout)
+		nvme_req_set_default_timeout(req);
 
 	if (buffer && bufflen) {
 		ret = blk_rq_map_kern(q, req, buffer, bufflen, GFP_KERNEL);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index cc111136a981..9eb75731c6d7 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -638,6 +638,14 @@ struct nvme_ns *nvme_get_ns_from_disk(struct gendisk *disk,
 		struct nvme_ns_head **head, int *srcu_idx);
 void nvme_put_ns_from_disk(struct nvme_ns_head *head, int idx);
 
+static inline void nvme_req_set_default_timeout(struct request *req)
+{
+	if (req->q->queuedata)
+		req->timeout = NVME_IO_TIMEOUT;
+	else /* no queuedata implies admin queue */
+		req->timeout = ADMIN_TIMEOUT;
+}
+
 extern const struct attribute_group *nvme_ns_id_attr_groups[];
 extern const struct block_device_operations nvme_ns_head_ops;
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:34 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:34 -0700
Subject: [PATCH V5 2/7] nvme-core: use I/O timeout in nvme_submit_user_cmd
In-Reply-To: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
References: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029234739.21266-3-chaitanya.kulkarni@wdc.com>

In the function nvme_submit_user_cmd() it uses ADMIN_TIMEOUT when
caller doesn't specify value for the timeout variable. This function is
also called from the user I/O command contexts (nvme_submit_io,
nvme_user_cmd_XXX) where NVME_IO_TIMEOUT can be used instead of
ADMIN_TIMEOUT.

For now we don't set the request queue's queuedata for admin command.

Use newly introduced helper function nvme_default_timeout() to set the
block layer request timeout value.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 42ca1b04b6d9..5feb2331df6f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1072,7 +1072,9 @@ static int nvme_submit_user_cmd(struct request_queue *q,
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
-	req->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	if (!req->timeout)
+		nvme_req_set_default_timeout(req);
+
 	nvme_req(req)->flags |= NVME_REQ_USERCMD;
 
 	if (ubuffer && bufflen) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:35 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:35 -0700
Subject: [PATCH V5 3/7] lightnvm: use I/O timeout in nvm submit user cmd
In-Reply-To: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
References: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029234739.21266-4-chaitanya.kulkarni@wdc.com>

In the function nvme_nvm_submit_user_cmd() it uses ADMIN_TIMEOUT when
caller doesn't specify value for the timeout variable.

For now we don't set the request queue's queuedata for admin command.

Use newly introduced helper function nvme_default_timeout() to set the
block layer request timeout value which sets the appropriate I/O or
Admin timeout values when caller doesn't specify the timeout value.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/lightnvm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/lightnvm.c b/drivers/nvme/host/lightnvm.c
index 8e562d0f2c30..784be80ee2f9 100644
--- a/drivers/nvme/host/lightnvm.c
+++ b/drivers/nvme/host/lightnvm.c
@@ -774,7 +774,8 @@ static int nvme_nvm_submit_user_cmd(struct request_queue *q,
 		goto err_cmd;
 	}
 
-	rq->timeout = timeout ? timeout : ADMIN_TIMEOUT;
+	if (!rq->timeout)
+		nvme_req_set_default_timeout(rq);
 
 	if (ppa_buf && ppa_len) {
 		ppa_list = dma_pool_alloc(dev->dma_pool, GFP_KERNEL, &ppa_dma);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:36 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:36 -0700
Subject: [PATCH V5 4/7] nvmet: set default timeout for passthru requests
In-Reply-To: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
References: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029234739.21266-5-chaitanya.kulkarni@wdc.com>

In nvmet_passthru_execute_cmd() we don't set the default timeout values
for passthru requests. Use the default values ro set the passthru
request timeout.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 8ee94f056898..da23485352d2 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -250,6 +250,8 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		goto out_put_ns;
 	}
 
+	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : ADMIN_TIMEOUT;
+
 	if (req->sg_cnt) {
 		ret = nvmet_passthru_map_sg(req, rq);
 		if (unlikely(ret)) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:37 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:37 -0700
Subject: [PATCH V5 5/7] nvmet: add passthru admin timeout value attr
In-Reply-To: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
References: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029234739.21266-6-chaitanya.kulkarni@wdc.com>

NVMeOF controller in the passsthru mode is capable of handling wide set
of admin commands including Vender unique passhtru admin comands (VUAC).

The VUACs are used to read the large driver logs and can take longer
than default NVMe commands, that is for passthru requests the timeout
value may differ from the passthru controller's default timeout values
(nvme-core:admin_timeout).

Add a configfs attribute so that user can set the admin timeout values.
In case if this configfs value is not set default to ADMIN_TIMEOUT
value.

This attribute setting is only allowed when ctrl is disable to avoid
rcu calls in the fast path, in future when needed we can always make it
fast path friendly using RCU.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/configfs.c | 32 ++++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c |  7 ++++++-
 3 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 37e1d7784e17..7625f8892fb5 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -736,9 +736,41 @@ static ssize_t nvmet_passthru_enable_store(struct config_item *item,
 }
 CONFIGFS_ATTR(nvmet_passthru_, enable);
 
+static ssize_t nvmet_passthru_admin_timeout_show(struct config_item *item,
+		char *page)
+{
+	return sprintf(page, "%u\n", to_subsys(item->ci_parent)->admin_timeout);
+}
+
+static ssize_t nvmet_passthru_admin_timeout_store(struct config_item *item,
+		const char *page, size_t count)
+{
+	struct nvmet_subsys *subsys = to_subsys(item->ci_parent);
+	unsigned int admin_timeout;
+	int ret = 0;
+
+	mutex_lock(&subsys->lock);
+	if (subsys->passthru_ctrl) {
+		pr_err("disable passthru ctrl before setting admin_timeout\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (kstrtouint(page, 0, &admin_timeout)) {
+		ret = -EINVAL;
+		goto out;
+	}
+	subsys->admin_timeout = admin_timeout;
+out:
+	mutex_unlock(&subsys->lock);
+	return ret ? ret : count;
+}
+CONFIGFS_ATTR(nvmet_passthru_, admin_timeout);
+
 static struct configfs_attribute *nvmet_passthru_attrs[] = {
 	&nvmet_passthru_attr_device_path,
 	&nvmet_passthru_attr_enable,
+	&nvmet_passthru_attr_admin_timeout,
 	NULL,
 };
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 559a15ccc322..a0c80e5179a2 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -249,6 +249,7 @@ struct nvmet_subsys {
 	struct nvme_ctrl	*passthru_ctrl;
 	char			*passthru_ctrl_path;
 	struct config_group	passthru_group;
+	unsigned int		admin_timeout;
 #endif /* CONFIG_NVME_TARGET_PASSTHRU */
 };
 
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index da23485352d2..d75ca6ddf60a 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -223,6 +223,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 
 static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 {
+	unsigned int admin_timeout = req->sq->ctrl->subsys->admin_timeout;
 	struct nvme_ctrl *ctrl = nvmet_req_passthru_ctrl(req);
 	struct request_queue *q = ctrl->admin_q;
 	struct nvme_ns *ns = NULL;
@@ -250,7 +251,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		goto out_put_ns;
 	}
 
-	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : ADMIN_TIMEOUT;
+	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : admin_timeout;
 
 	if (req->sg_cnt) {
 		ret = nvmet_passthru_map_sg(req, rq);
@@ -548,6 +549,10 @@ int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys)
 			NVME_TERTIARY(subsys->ver));
 		subsys->ver = NVME_VS(1, 2, 1);
 	}
+
+	if (!subsys->admin_timeout)
+		subsys->admin_timeout = ADMIN_TIMEOUT;
+
 	nvme_get_ctrl(ctrl);
 	__module_get(subsys->passthru_ctrl->ops->module);
 	ret = 0;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:38 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:38 -0700
Subject: [PATCH V5 6/7] nvmet: add passthru io timeout value attr
In-Reply-To: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
References: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029234739.21266-7-chaitanya.kulkarni@wdc.com>

NVMe controller in the passsthru mode is capable of handling wide set
of admin commands including Vender unique passhtru I/O comands
(VUICs).

The VUICs can take longer than default NVMe commands, that is for
passthru requests the timeout value may differ from the passthru
controller's default timeout values (nvme-core:io_timeout).

Add configfs attribute so that user can set the I/O timeout values.
In case if this configfs value is not set default to NVME_IO_TIMEOUT
value.

This attribute setting is only allowed when ctrl is disable to avoid
rcu calls in the fast path, in future when needed we can always make it
fast path friendly using RCU.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/configfs.c | 32 ++++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h    |  1 +
 drivers/nvme/target/passthru.c |  6 +++++-
 3 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 7625f8892fb5..5beae49ba6ad 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -767,10 +767,42 @@ static ssize_t nvmet_passthru_admin_timeout_store(struct config_item *item,
 }
 CONFIGFS_ATTR(nvmet_passthru_, admin_timeout);
 
+static ssize_t nvmet_passthru_io_timeout_show(struct config_item *item,
+		char *page)
+{
+	return sprintf(page, "%u\n", to_subsys(item->ci_parent)->io_timeout);
+}
+
+static ssize_t nvmet_passthru_io_timeout_store(struct config_item *item,
+		const char *page, size_t count)
+{
+	struct nvmet_subsys *subsys = to_subsys(item->ci_parent);
+	unsigned int io_timeout;
+	int ret = 0;
+
+	mutex_lock(&subsys->lock);
+	if (subsys->passthru_ctrl) {
+		pr_err("disable passthru ctrl before setting io_timeout\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (kstrtouint(page, 0, &io_timeout)) {
+		ret = -EINVAL;
+		goto out;
+	}
+	subsys->io_timeout = io_timeout;
+out:
+	mutex_unlock(&subsys->lock);
+	return ret ? ret : count;
+}
+CONFIGFS_ATTR(nvmet_passthru_, io_timeout);
+
 static struct configfs_attribute *nvmet_passthru_attrs[] = {
 	&nvmet_passthru_attr_device_path,
 	&nvmet_passthru_attr_enable,
 	&nvmet_passthru_attr_admin_timeout,
+	&nvmet_passthru_attr_io_timeout,
 	NULL,
 };
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index a0c80e5179a2..2f9635273629 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -250,6 +250,7 @@ struct nvmet_subsys {
 	char			*passthru_ctrl_path;
 	struct config_group	passthru_group;
 	unsigned int		admin_timeout;
+	unsigned int		io_timeout;
 #endif /* CONFIG_NVME_TARGET_PASSTHRU */
 };
 
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index d75ca6ddf60a..177ea23561e4 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -224,6 +224,7 @@ static int nvmet_passthru_map_sg(struct nvmet_req *req, struct request *rq)
 static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 {
 	unsigned int admin_timeout = req->sq->ctrl->subsys->admin_timeout;
+	unsigned int io_timeout = req->sq->ctrl->subsys->io_timeout;
 	struct nvme_ctrl *ctrl = nvmet_req_passthru_ctrl(req);
 	struct request_queue *q = ctrl->admin_q;
 	struct nvme_ns *ns = NULL;
@@ -251,7 +252,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		goto out_put_ns;
 	}
 
-	rq->timeout = q->queuedata ? NVME_IO_TIMEOUT : admin_timeout;
+	rq->timeout = q->queuedata ? io_timeout : admin_timeout;
 
 	if (req->sg_cnt) {
 		ret = nvmet_passthru_map_sg(req, rq);
@@ -553,6 +554,9 @@ int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys)
 	if (!subsys->admin_timeout)
 		subsys->admin_timeout = ADMIN_TIMEOUT;
 
+	if (!subsys->io_timeout)
+		subsys->io_timeout = NVME_IO_TIMEOUT;
+
 	nvme_get_ctrl(ctrl);
 	__module_get(subsys->passthru_ctrl->ops->module);
 	ret = 0;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Thu Oct 29 19:47:39 2020
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 29 Oct 2020 16:47:39 -0700
Subject: [PATCH V5 7/7] nvme: use consistent macro name for timeout
In-Reply-To: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
References: <20201029234739.21266-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20201029234739.21266-8-chaitanya.kulkarni@wdc.com>

This is purely a clenaup patch, add prefix NVME to the ADMIN_TIMEOUT
to make consistent with NVME_IO_TIMEOUT.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c       | 3 ++-
 drivers/nvme/host/fc.c         | 2 +-
 drivers/nvme/host/nvme.h       | 4 ++--
 drivers/nvme/host/pci.c        | 8 ++++----
 drivers/nvme/host/rdma.c       | 2 +-
 drivers/nvme/host/tcp.c        | 2 +-
 drivers/nvme/target/loop.c     | 2 +-
 drivers/nvme/target/passthru.c | 2 +-
 8 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 5feb2331df6f..6d50f7cbcfa2 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2267,7 +2267,8 @@ int nvme_sec_submit(void *data, u16 spsp, u8 secp, void *buffer, size_t len,
 	cmd.common.cdw11 = cpu_to_le32(len);
 
 	return __nvme_submit_sync_cmd(ctrl->admin_q, &cmd, NULL, buffer, len,
-				      ADMIN_TIMEOUT, NVME_QID_ANY, 1, 0, false);
+				      NVME_ADMIN_TIMEOUT, NVME_QID_ANY, 1, 0,
+				      false);
 }
 EXPORT_SYMBOL_GPL(nvme_sec_submit);
 #endif /* CONFIG_BLK_SED_OPAL */
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 3c002bdcace3..a0be43edfb3c 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3530,7 +3530,7 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
 			    ctrl->lport->ops->fcprqst_priv_sz);
 	ctrl->admin_tag_set.driver_data = ctrl;
 	ctrl->admin_tag_set.nr_hw_queues = 1;
-	ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
+	ctrl->admin_tag_set.timeout = NVME_ADMIN_TIMEOUT;
 	ctrl->admin_tag_set.flags = BLK_MQ_F_NO_SCHED;
 
 	ret = blk_mq_alloc_tag_set(&ctrl->admin_tag_set);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9eb75731c6d7..37244d376855 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -24,7 +24,7 @@ extern unsigned int nvme_io_timeout;
 #define NVME_IO_TIMEOUT	(nvme_io_timeout * HZ)
 
 extern unsigned int admin_timeout;
-#define ADMIN_TIMEOUT	(admin_timeout * HZ)
+#define NVME_ADMIN_TIMEOUT	(admin_timeout * HZ)
 
 #define NVME_DEFAULT_KATO	5
 #define NVME_KATO_GRACE		10
@@ -643,7 +643,7 @@ static inline void nvme_req_set_default_timeout(struct request *req)
 	if (req->q->queuedata)
 		req->timeout = NVME_IO_TIMEOUT;
 	else /* no queuedata implies admin queue */
-		req->timeout = ADMIN_TIMEOUT;
+		req->timeout = NVME_ADMIN_TIMEOUT;
 }
 
 extern const struct attribute_group *nvme_ns_id_attr_groups[];
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index df8f3612107f..ce882d996e15 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1296,7 +1296,7 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved)
 		return BLK_EH_RESET_TIMER;
 	}
 
-	abort_req->timeout = ADMIN_TIMEOUT;
+	abort_req->timeout = NVME_ADMIN_TIMEOUT;
 	abort_req->end_io_data = NULL;
 	blk_execute_rq_nowait(abort_req->q, NULL, abort_req, 0, abort_endio);
 
@@ -1592,7 +1592,7 @@ static int nvme_alloc_admin_tags(struct nvme_dev *dev)
 		dev->admin_tagset.nr_hw_queues = 1;
 
 		dev->admin_tagset.queue_depth = NVME_AQ_MQ_TAG_DEPTH;
-		dev->admin_tagset.timeout = ADMIN_TIMEOUT;
+		dev->admin_tagset.timeout = NVME_ADMIN_TIMEOUT;
 		dev->admin_tagset.numa_node = dev->ctrl.numa_node;
 		dev->admin_tagset.cmd_size = sizeof(struct nvme_iod);
 		dev->admin_tagset.flags = BLK_MQ_F_NO_SCHED;
@@ -2208,7 +2208,7 @@ static int nvme_delete_queue(struct nvme_queue *nvmeq, u8 opcode)
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
-	req->timeout = ADMIN_TIMEOUT;
+	req->timeout = NVME_ADMIN_TIMEOUT;
 	req->end_io_data = nvmeq;
 
 	init_completion(&nvmeq->delete_done);
@@ -2224,7 +2224,7 @@ static bool __nvme_disable_io_queues(struct nvme_dev *dev, u8 opcode)
 	unsigned long timeout;
 
  retry:
-	timeout = ADMIN_TIMEOUT;
+	timeout = NVME_ADMIN_TIMEOUT;
 	while (nr_queues > 0) {
 		if (nvme_delete_queue(&dev->queues[nr_queues], opcode))
 			break;
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index aad829a2b50d..0bb55d667af6 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -798,7 +798,7 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
 				NVME_RDMA_DATA_SGL_SIZE;
 		set->driver_data = ctrl;
 		set->nr_hw_queues = 1;
-		set->timeout = ADMIN_TIMEOUT;
+		set->timeout = NVME_ADMIN_TIMEOUT;
 		set->flags = BLK_MQ_F_NO_SCHED;
 	} else {
 		set = &ctrl->tag_set;
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index d6a3e1487354..6680e5342fd1 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1569,7 +1569,7 @@ static struct blk_mq_tag_set *nvme_tcp_alloc_tagset(struct nvme_ctrl *nctrl,
 		set->cmd_size = sizeof(struct nvme_tcp_request);
 		set->driver_data = ctrl;
 		set->nr_hw_queues = 1;
-		set->timeout = ADMIN_TIMEOUT;
+		set->timeout = NVME_ADMIN_TIMEOUT;
 	} else {
 		set = &ctrl->tag_set;
 		memset(set, 0, sizeof(*set));
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index 5ab46d3130e7..979dfeb39d9b 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -345,7 +345,7 @@ static int nvme_loop_configure_admin_queue(struct nvme_loop_ctrl *ctrl)
 		NVME_INLINE_SG_CNT * sizeof(struct scatterlist);
 	ctrl->admin_tag_set.driver_data = ctrl;
 	ctrl->admin_tag_set.nr_hw_queues = 1;
-	ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
+	ctrl->admin_tag_set.timeout = NVME_ADMIN_TIMEOUT;
 	ctrl->admin_tag_set.flags = BLK_MQ_F_NO_SCHED;
 
 	ctrl->queues[0].ctrl = ctrl;
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 177ea23561e4..09b03c422ed9 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -552,7 +552,7 @@ int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys)
 	}
 
 	if (!subsys->admin_timeout)
-		subsys->admin_timeout = ADMIN_TIMEOUT;
+		subsys->admin_timeout = NVME_ADMIN_TIMEOUT;
 
 	if (!subsys->io_timeout)
 		subsys->io_timeout = NVME_IO_TIMEOUT;
-- 
2.22.1



From yi.zhang at redhat.com  Thu Oct 29 20:04:07 2020
From: yi.zhang at redhat.com (Yi Zhang)
Date: Fri, 30 Oct 2020 08:04:07 +0800
Subject: [PATCH v2 0/5] avoid race for time out
In-Reply-To: <e913e433-d8ef-c04b-d9fa-3a6d71657136@grimberg.me>
References: <20201022021452.10971-1-lengchao@huawei.com>
 <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
 <e913e433-d8ef-c04b-d9fa-3a6d71657136@grimberg.me>
Message-ID: <eb2dcbb9-241f-e876-e914-3159f64b7351@redhat.com>



On 10/30/20 5:00 AM, Sagi Grimberg wrote:
>
>> Hello
>>
>> This series fixed the WARNING issue I reported [1], but now the 
>> nvme/012 [2] will be hang there and never finished, here is the log[3]
>> [1]
>> https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/ 
>>
>>
>> [2]
>> [root at hpe-xw9400-02 blktests]# nvme_trtype=tcp ./check nvme/012
>> nvme/012 (run mkfs and data verification fio job on NVMeOF block 
>> device-backed ns)
>> ???? runtime? 1199.651s? ...
>
> Hey Yi,
>
> This is a different issue, as I said, first of all the test is not
> designed to trigger this scenario so it is weird why it even happens.
>
OK, I will keep monitor the issue in the future, and the original issue 
also found in stable branch, should we also CC stable?

Hi Ming

just FYI, with your path on top of this series
the test passed 100 times although there are timeout, but finally 
reconnected and test finished.
Here is log:
[47534.756506] run blktests nvme/012 at 2020-10-29 10:34:52
[47534.934534] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[47534.949247] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[47534.965075] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[47534.979730] nvme nvme0: creating 12 I/O queues.
[47534.985596] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[47534.997327] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
127.0.0.1:4420
[47536.885732] XFS (nvme0n1): Mounting V5 Filesystem
[47536.899366] XFS (nvme0n1): Ending clean mount
[47536.905082] xfs filesystem being mounted at /mnt/blktests supports 
timestamps until 2038 (0x7fffffff)
[47597.518197] nvme nvme0: queue 7: timeout request 0x63 type 4
[47597.523884] nvme nvme0: starting error recovery
[47597.528457] nvme nvme0: queue 11: timeout request 0x21 type 4
[47597.534865] nvme nvme0: queue 11: timeout request 0x22 type 4
[47597.540607] nvme nvme0: queue 11: timeout request 0x23 type 4
[47597.546351] nvme nvme0: queue 11: timeout request 0x24 type 4
[47597.552090] nvme nvme0: queue 11: timeout request 0x25 type 4
[47597.557830] nvme nvme0: queue 11: timeout request 0x26 type 4
[47597.563570] nvme nvme0: queue 11: timeout request 0x27 type 4
[47597.569310] nvme nvme0: queue 11: timeout request 0x28 type 4
[47597.575049] nvme nvme0: queue 11: timeout request 0x29 type 4
[47597.580803] nvme nvme0: queue 11: timeout request 0x2a type 4
[47597.586544] nvme nvme0: queue 11: timeout request 0x2b type 4
[47597.592287] nvme nvme0: queue 11: timeout request 0x2c type 4
[47597.598026] nvme nvme0: queue 11: timeout request 0x2d type 4
[47597.603765] nvme nvme0: queue 11: timeout request 0x2e type 4
[47597.609505] nvme nvme0: queue 11: timeout request 0x2f type 4
[47597.615244] nvme nvme0: queue 11: timeout request 0x30 type 4
[47597.621052] block nvme0n1: no usable path - requeuing I/O
[47597.622230] nvme nvme0: Reconnecting in 10 seconds...
[47597.626452] block nvme0n1: no usable path - requeuing I/O
[47597.636885] block nvme0n1: no usable path - requeuing I/O
[47597.642276] block nvme0n1: no usable path - requeuing I/O
[47597.647670] block nvme0n1: no usable path - requeuing I/O
[47597.653076] block nvme0n1: no usable path - requeuing I/O
[47597.658470] block nvme0n1: no usable path - requeuing I/O
[47597.663874] block nvme0n1: no usable path - requeuing I/O
[47597.669272] block nvme0n1: no usable path - requeuing I/O
[47597.674676] block nvme0n1: no usable path - requeuing I/O
[47623.118733] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
[47623.124745] nvmet: ctrl 2 fatal error occurred!
[47627.214836] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[47627.220856] nvmet: ctrl 1 fatal error occurred!
[47647.671350] nvmet: creating controller 2 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[47668.175734] nvme nvme0: queue 0: timeout request 0x0 type 4
[47668.181397] nvme nvme0: Connect command failed, error wo/DNR bit: 881
[47668.187884] nvme nvme0: failed to connect queue: 0 ret=881
[47668.193725] nvme nvme0: Failed reconnect attempt 1
[47668.198510] nvme nvme0: Reconnecting in 10 seconds...
[47678.416555] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[47678.430569] nvme_ns_head_submit_bio: 30 callbacks suppressed
[47678.430577] block nvme0n1: no usable path - requeuing I/O
[47678.441645] block nvme0n1: no usable path - requeuing I/O
[47678.447042] block nvme0n1: no usable path - requeuing I/O
[47678.452446] block nvme0n1: no usable path - requeuing I/O
[47678.457862] block nvme0n1: no usable path - requeuing I/O
[47678.463787] block nvme0n1: no usable path - requeuing I/O
[47678.469180] block nvme0n1: no usable path - requeuing I/O
[47678.474569] block nvme0n1: no usable path - requeuing I/O
[47678.479955] block nvme0n1: no usable path - requeuing I/O
[47678.485344] block nvme0n1: no usable path - requeuing I/O
[47678.490977] nvme nvme0: creating 12 I/O queues.
[47678.503323] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[47678.511558] nvme nvme0: Successfully reconnected (2 attempt)
[47743.953398] nvme nvme0: queue 3: timeout request 0x31 type 4
[47743.959069] nvme nvme0: starting error recovery
[47743.963627] nvme nvme0: queue 3: timeout request 0x32 type 4
[47743.969367] nvme nvme0: queue 3: timeout request 0x33 type 4
[47743.975021] nvme nvme0: queue 3: timeout request 0x34 type 4
[47743.980826] nvme nvme0: queue 3: timeout request 0x35 type 4
[47743.986482] nvme nvme0: queue 3: timeout request 0x36 type 4
[47743.992291] nvme nvme0: queue 3: timeout request 0x37 type 4
[47743.997953] nvme nvme0: queue 3: timeout request 0x38 type 4
[47744.003609] nvme nvme0: queue 3: timeout request 0x39 type 4
[47744.009272] nvme nvme0: queue 3: timeout request 0x3a type 4
[47744.014949] nvme nvme0: queue 3: timeout request 0x3b type 4
[47744.020933] nvme nvme0: queue 3: timeout request 0x3c type 4
[47744.026589] nvme nvme0: queue 3: timeout request 0x3d type 4
[47744.032251] nvme nvme0: queue 3: timeout request 0x3e type 4
[47744.037914] nvme nvme0: queue 3: timeout request 0x3f type 4
[47744.043569] nvme nvme0: queue 3: timeout request 0x40 type 4
[47744.049227] nvme nvme0: queue 3: timeout request 0x41 type 4
[47744.054895] nvme nvme0: queue 5: timeout request 0x11 type 4
[47744.060659] nvme nvme0: queue 5: timeout request 0x12 type 4
[47744.066348] nvme nvme0: queue 5: timeout request 0x13 type 4
[47744.071999] nvme nvme0: queue 5: timeout request 0x14 type 4
[47744.077646] nvme nvme0: queue 5: timeout request 0x15 type 4
[47744.083305] nvme nvme0: queue 5: timeout request 0x1e type 4
[47744.088959] nvme nvme0: queue 5: timeout request 0x1f type 4
[47744.094608] nvme nvme0: queue 5: timeout request 0x20 type 4
[47744.100263] nvme nvme0: queue 11: timeout request 0x24 type 4
[47744.106083] nvme_ns_head_submit_bio: 14 callbacks suppressed
[47744.106087] block nvme0n1: no usable path - requeuing I/O
[47744.107067] nvme nvme0: Reconnecting in 10 seconds...
[47744.111745] block nvme0n1: no usable path - requeuing I/O
[47744.127552] block nvme0n1: no usable path - requeuing I/O
[47744.133104] block nvme0n1: no usable path - requeuing I/O
[47744.138493] block nvme0n1: no usable path - requeuing I/O
[47744.143885] block nvme0n1: no usable path - requeuing I/O
[47744.149280] block nvme0n1: no usable path - requeuing I/O
[47744.154668] block nvme0n1: no usable path - requeuing I/O
[47744.160066] block nvme0n1: no usable path - requeuing I/O
[47744.165459] block nvme0n1: no usable path - requeuing I/O
[47769.553925] nvmet: ctrl 2 keep-alive timer (15 seconds) expired!
[47769.559934] nvmet: ctrl 2 fatal error occurred!
[47770.577940] nvmet: ctrl 1 keep-alive timer (15 seconds) expired!
[47770.583962] nvmet: ctrl 1 fatal error occurred!
[47814.247111] nvmet: creating controller 2 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[47815.634922] nvme nvme0: queue 0: timeout request 0x1 type 4
[47815.640593] nvme nvme0: Connect command failed, error wo/DNR bit: 881
[47815.647088] nvme nvme0: failed to connect queue: 0 ret=881
[47815.653011] nvme nvme0: Failed reconnect attempt 1
[47815.657792] nvme nvme0: Reconnecting in 10 seconds...
[47825.875752] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:ffe2b140e76a45649005853f3b871859.
[47825.889828] nvme_ns_head_submit_bio: 101 callbacks suppressed
[47825.889835] block nvme0n1: no usable path - requeuing I/O
[47825.900998] block nvme0n1: no usable path - requeuing I/O
[47825.906396] block nvme0n1: no usable path - requeuing I/O
[47825.911800] block nvme0n1: no usable path - requeuing I/O
[47825.917213] block nvme0n1: no usable path - requeuing I/O
[47825.922607] block nvme0n1: no usable path - requeuing I/O
[47825.928004] block nvme0n1: no usable path - requeuing I/O
[47825.933387] block nvme0n1: no usable path - requeuing I/O
[47825.938779] block nvme0n1: no usable path - requeuing I/O
[47825.944169] block nvme0n1: no usable path - requeuing I/O
[47825.949765] nvme nvme0: creating 12 I/O queues.
[47825.962904] nvme nvme0: mapped 12/0/0 default/read/poll queues.
[47825.970958] nvme nvme0: Successfully reconnected (2 attempt)
[47831.211728] XFS (nvme0n1): Unmounting Filesystem
[47831.362650] nvme nvme0: Removing ctrl: NQN "blktests-subsystem-1"


> For debug purposes, can you switch to run blktests with siw for
> comparison?
>
> This patch set should move forward for inclusion regardless.
>



From ming.lei at redhat.com  Thu Oct 29 21:00:59 2020
From: ming.lei at redhat.com (Ming Lei)
Date: Fri, 30 Oct 2020 09:00:59 +0800
Subject: [PATCH v2 0/5] avoid race for time out
In-Reply-To: <eb2dcbb9-241f-e876-e914-3159f64b7351@redhat.com>
References: <20201022021452.10971-1-lengchao@huawei.com>
 <aba13cb9-a3d4-490b-4d1d-1deb60eafff6@redhat.com>
 <e913e433-d8ef-c04b-d9fa-3a6d71657136@grimberg.me>
 <eb2dcbb9-241f-e876-e914-3159f64b7351@redhat.com>
Message-ID: <20201030010059.GB2012749@T590>

On Fri, Oct 30, 2020 at 08:04:07AM +0800, Yi Zhang wrote:
> 
> 
> On 10/30/20 5:00 AM, Sagi Grimberg wrote:
> > 
> > > Hello
> > > 
> > > This series fixed the WARNING issue I reported [1], but now the
> > > nvme/012 [2] will be hang there and never finished, here is the
> > > log[3]
> > > [1]
> > > https://lore.kernel.org/linux-nvme/1934331639.3314730.1602152202454.JavaMail.zimbra at redhat.com/
> > > 
> > > 
> > > [2]
> > > [root at hpe-xw9400-02 blktests]# nvme_trtype=tcp ./check nvme/012
> > > nvme/012 (run mkfs and data verification fio job on NVMeOF block
> > > device-backed ns)
> > > ???????? runtime?? 1199.651s?? ...
> > 
> > Hey Yi,
> > 
> > This is a different issue, as I said, first of all the test is not
> > designed to trigger this scenario so it is weird why it even happens.
> > 
> OK, I will keep monitor the issue in the future, and the original issue also
> found in stable branch, should we also CC stable?
> 
> Hi Ming
> 
> just FYI, with your path on top of this series
> the test passed 100 times although there are timeout, but finally
> reconnected and test finished.

Hi Yi,

Thanks for your test, I will re-post the patch out in linux-block today. 


Thanks,
Ming



From brandon.gustav at googlemail.com  Fri Oct 30 00:31:12 2020
From: brandon.gustav at googlemail.com (B.L. Jones)
Date: Thu, 29 Oct 2020 21:31:12 -0700
Subject: NVME controller issue prevents boot on Kernel versions >= 5.8 on
 MacBook 7,1 (MacBook Air 2015)
Message-ID: <CA+ykBt3kGCOdupWSNNiCxQLduq8Bs_t4YN8CsUWYOOGEk884vw@mail.gmail.com>

Hello all,

This is my first time emailing any kernel mail list so I will do my
best to furnish any information you may need regarding this.

To begin, I reported a boot issue against the kernel on the
Fedora/RedHat Bugzilla last month here:
https://bugzilla.redhat.com/show_bug.cgi?id=1878596

Essentially, booting any kernel greater than version 5.8 on my 2015
MacBook Air would cause a soft freeze, with an error stating:

[   34.890739] f-air kernel: nvme nvme0: controller is down; will
reset: CSTS=0x3, PCI_STATUS=0x10
[   34.917820] f-air kernel: nvme nvme0: detected Apple NVMe
controller, set queue depth=2 to work around controller resets

Another individual in the bugzilla thread took a look at the error
logs and identified a potential commit that may have caused the issue:

commit 54b2fcee1db041a83b52b51752dade6090cf952f
Author: Keith Busch <kbusch at kernel.org>
Date:   Mon Apr 27 11:54:46 2020 -0700

    nvme-pci: remove last_sq_tail

    The nvme driver does not have enough tags to wrap the queue, and blk-mq
    will no longer call commit_rqs() when there are no new submissions to
    notify.

    Signed-off-by: Keith Busch <kbusch at kernel.org>
    Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
    Signed-off-by: Christoph Hellwig <hch at lst.de>
    Signed-off-by: Jens Axboe <axboe at kernel.dk>
===

The main/interesting hunk in this patch is as follows:

===
@@ -446,24 +445,11 @@ static int nvme_pci_map_queues(struct blk_mq_tag_set *set)
  return 0;
 }

-/*
- * Write sq tail if we are asked to, or if the next command would wrap.
- */
-static inline void nvme_write_sq_db(struct nvme_queue *nvmeq, bool write_sq)
+static inline void nvme_write_sq_db(struct nvme_queue *nvmeq)
 {
- if (!write_sq) {
- u16 next_tail = nvmeq->sq_tail + 1;
-
- if (next_tail == nvmeq->q_depth)
- next_tail = 0;
- if (next_tail != nvmeq->last_sq_tail)
- return;
- }
-
  if (nvme_dbbuf_update_and_check_event(nvmeq->sq_tail,
  nvmeq->dbbuf_sq_db, nvmeq->dbbuf_sq_ei))
  writel(nvmeq->sq_tail, nvmeq->q_db);
- nvmeq->last_sq_tail = nvmeq->sq_tail;
 }
===

I built kernel version 5.8.16 per the recommendation on the bugzilla
thread, reverting this commit: 54b2fcee1db0 ("nvme-pci: remove
last_sq_tail"), and was able to successfully boot, whereas before no
kernel version greater than 5.8 would boot.

Additionally, another user built this kernel version and packaged it
with the Fedora build service with this commit reversion. That build
also booted successfully and seemed to confirm that this commit was
causing the issue.

More detailed technical information can be found in M. Vefa Bicakci's
comment on the bugzilla thread:
https://bugzilla.redhat.com/show_bug.cgi?id=1878596#c22

It is my hope that this information can lead to a fix in a newer
kernel update for this hardware. If you require any additional
information, please let me know and I will do my best to provide what
you need (I am not quite a developer, just an enthusiastic user).


All the best,
Brandon Jones


From Niklas.Cassel at wdc.com  Fri Oct 30 10:31:47 2020
From: Niklas.Cassel at wdc.com (Niklas Cassel)
Date: Fri, 30 Oct 2020 14:31:47 +0000
Subject: [PATCH] nvme: report capacity 0 for non supported ZNS SSDs
In-Reply-To: <20201029185753.14368-1-javier.gonz@samsung.com>
References: <20201029185753.14368-1-javier.gonz@samsung.com>
Message-ID: <20201030143146.GA105238@localhost.localdomain>

On Thu, Oct 29, 2020 at 07:57:53PM +0100, Javier Gonz?lez wrote:
> Allow ZNS SSDs to be presented to the host even when they implement
> features that are not supported by the kernel zoned block device.
> 
> Instead of rejecting the SSD at the NVMe driver level, deal with this in
> the block layer by setting capacity to 0, as we do with other things
> such as unsupported PI configurations. This allows to use standard
> management tools such as nvme-cli to choose a different format or
> firmware slot that is compatible with the Linux zoned block device.
> 
> Signed-off-by: Javier Gonz?lez <javier.gonz at samsung.com>
> ---
>  drivers/nvme/host/core.c |  5 +++++
>  drivers/nvme/host/nvme.h |  1 +
>  drivers/nvme/host/zns.c  | 31 ++++++++++++++-----------------
>  3 files changed, 20 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index c190c56bf702..9ca4f0a6ff2c 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2026,6 +2026,11 @@ static void nvme_update_disk_info(struct gendisk *disk,
>  			capacity = 0;
>  	}

Hello Javier,

>  
> +#ifdef CONFIG_BLK_DEV_ZONED

if (IS_ENABLED(CONFIG_BLK_DEV_ZONED)

is preferred over ifdefs.

> +	if (!ns->zone_sup)
> +		capacity = 0;
> +#endif
> +
>  	set_capacity_revalidate_and_notify(disk, capacity, false);
>  
>  	nvme_config_discard(disk, ns);
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 87737fa32360..42cbe5bbc518 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -443,6 +443,7 @@ struct nvme_ns {
>  	u8 pi_type;
>  #ifdef CONFIG_BLK_DEV_ZONED
>  	u64 zsze;
> +	bool zone_sup;

Perhaps a more descriptive name? zoned_ns_supp ?

>  #endif
>  	unsigned long features;
>  	unsigned long flags;
> diff --git a/drivers/nvme/host/zns.c b/drivers/nvme/host/zns.c
> index 57cfd78731fb..77a7fed508ef 100644
> --- a/drivers/nvme/host/zns.c
> +++ b/drivers/nvme/host/zns.c
> @@ -44,20 +44,23 @@ int nvme_update_zone_info(struct gendisk *disk, struct nvme_ns *ns,
>  	struct nvme_id_ns_zns *id;
>  	int status;
>  
> -	/* Driver requires zone append support */
> +	ns->zone_sup = true;

I don't think it is wise to assign it to true here.
E.g. if kzalloc() failes, if nvme_submit_sync_cmd() fails,
or if nvme_set_max_append() fails, you have already set this to true,
but zoc or power of 2 checks were never performed.


Perhaps something like this would be more robust:

@@ -53,18 +53,19 @@ int nvme_update_zone_info(struct nvme_ns *ns, unsigned lbaf)
        struct nvme_command c = { };
        struct nvme_id_ns_zns *id;
        int status;
+       bool new_ns_supp = true;
+
+       /* default to NS not supported */
+       ns->zoned_ns_supp = false;

-       /* Driver requires zone append support */
        if (!(le32_to_cpu(log->iocs[nvme_cmd_zone_append]) &
                        NVME_CMD_EFFECTS_CSUPP)) {
                dev_warn(ns->ctrl->device,
                        "append not supported for zoned namespace:%d\n",
                        ns->head->ns_id);
-               return -EINVAL;
-       }
-
-       /* Lazily query controller append limit for the first zoned namespace */
-       if (!ns->ctrl->max_zone_append) {
+               new_ns_supp = false;
+       } else if (!ns->ctrl->max_zone_append) {
+               /* Lazily query controller append limit for the first zoned namespace */
                status = nvme_set_max_append(ns->ctrl);
                if (status)
                        return status;
@@ -80,19 +81,16 @@ int nvme_update_zone_info(struct nvme_ns *ns, unsigned lbaf)
        c.identify.csi = NVME_CSI_ZNS;

        status = nvme_submit_sync_cmd(ns->ctrl->admin_q, &c, id, sizeof(*id));
-       if (status)
-               goto free_data;
+       if (status) {
+               kfree(id);
+               return status;
+       }

-       /*
-        * We currently do not handle devices requiring any of the zoned
-        * operation characteristics.
-        */
        if (id->zoc) {
                dev_warn(ns->ctrl->device,
                        "zone operations:%x not supported for namespace:%u\n",
                        le16_to_cpu(id->zoc), ns->head->ns_id);
-               status = -EINVAL;
-               goto free_data;
+               new_ns_supp = false;
        }

        ns->zsze = nvme_lba_to_sect(ns, le64_to_cpu(id->lbafe[lbaf].zsze));
@@ -100,17 +98,14 @@ int nvme_update_zone_info(struct nvme_ns *ns, unsigned lbaf)
                dev_warn(ns->ctrl->device,
                        "invalid zone size:%llu for namespace:%u\n",
                        ns->zsze, ns->head->ns_id);
-               status = -EINVAL;
-               goto free_data;
+               new_ns_supp = false;
        }

+       ns->zoned_ns_supp = new_ns_supp;
        q->limits.zoned = BLK_ZONED_HM;
        blk_queue_flag_set(QUEUE_FLAG_ZONE_RESETALL, q);
        blk_queue_max_open_zones(q, le32_to_cpu(id->mor) + 1);
        blk_queue_max_active_zones(q, le32_to_cpu(id->mar) + 1);
-free_data:
-       kfree(id);
-       return status;
 }

 static void *nvme_zns_alloc_report_buffer(struct nvme_ns *ns,


Kind regards,
Niklas

From kbusch at kernel.org  Fri Oct 30 11:27:48 2020
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 30 Oct 2020 08:27:48 -0700
Subject: NVME controller issue prevents boot on Kernel versions >= 5.8 on
 MacBook 7,1 (MacBook Air 2015)
In-Reply-To: <CA+ykBt3kGCOdupWSNNiCxQLduq8Bs_t4YN8CsUWYOOGEk884vw@mail.gmail.com>
References: <CA+ykBt3kGCOdupWSNNiCxQLduq8Bs_t4YN8CsUWYOOGEk884vw@mail.gmail.com>
Message-ID: <20201030152748.GA1946804@dhcp-10-100-145-180.wdc.com>

On Thu, Oct 29, 2020 at 09:31:12PM -0700, B.L. Jones wrote:
> Hello all,
> 
> This is my first time emailing any kernel mail list so I will do my
> best to furnish any information you may need regarding this.
> 
> To begin, I reported a boot issue against the kernel on the
> Fedora/RedHat Bugzilla last month here:
> https://bugzilla.redhat.com/show_bug.cgi?id=1878596
> 
> Essentially, booting any kernel greater than version 5.8 on my 2015
> MacBook Air would cause a soft freeze, with an error stating:
> 
> [   34.890739] f-air kernel: nvme nvme0: controller is down; will reset: CSTS=0x3, PCI_STATUS=0x10
> [   34.917820] f-air kernel: nvme nvme0: detected Apple NVMe controller, set queue depth=2 to work around controller resets
> 
> Another individual in the bugzilla thread took a look at the error
> logs and identified a potential commit that may have caused the issue:
> 
> commit 54b2fcee1db041a83b52b51752dade6090cf952f
> Author: Keith Busch <kbusch at kernel.org>
> Date:   Mon Apr 27 11:54:46 2020 -0700
> 
>     nvme-pci: remove last_sq_tail
> 
>     The nvme driver does not have enough tags to wrap the queue, and blk-mq
>     will no longer call commit_rqs() when there are no new submissions to
>     notify.
> 
>     Signed-off-by: Keith Busch <kbusch at kernel.org>

Thanks for the notice. I'll see if I can fix this, but I think a revert
may be necessary.


From kbusch at kernel.org  Fri Oct 30 13:28:54 2020
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 30 Oct 2020 10:28:54 -0700
Subject: [PATCH] Revert "nvme-pci: remove last_sq_tail"
Message-ID: <20201030172854.1961578-1-kbusch@kernel.org>

Multiple CPUs may be mapped to the same hctx, allowing mulitple
submission contexts to attempt commit_rqs(). We need to verify we're
not writing the same doorbell value multiple times since that's a spec
violation.

Revert commit 54b2fcee1db041a83b52b51752dade6090cf952f.

Link: https://bugzilla.redhat.com/show_bug.cgi?id=1878596
Reported-by: "B.L. Jones" <brandon.gustav at googlemail.com>
Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/pci.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index df8f3612107f..0578ff253c47 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -198,6 +198,7 @@ struct nvme_queue {
 	u32 q_depth;
 	u16 cq_vector;
 	u16 sq_tail;
+	u16 last_sq_tail;
 	u16 cq_head;
 	u16 qid;
 	u8 cq_phase;
@@ -455,11 +456,24 @@ static int nvme_pci_map_queues(struct blk_mq_tag_set *set)
 	return 0;
 }
 
-static inline void nvme_write_sq_db(struct nvme_queue *nvmeq)
+/*
+ * Write sq tail if we are asked to, or if the next command would wrap.
+ */
+static inline void nvme_write_sq_db(struct nvme_queue *nvmeq, bool write_sq)
 {
+	if (!write_sq) {
+		u16 next_tail = nvmeq->sq_tail + 1;
+
+		if (next_tail == nvmeq->q_depth)
+			next_tail = 0;
+		if (next_tail != nvmeq->last_sq_tail)
+			return;
+	}
+
 	if (nvme_dbbuf_update_and_check_event(nvmeq->sq_tail,
 			nvmeq->dbbuf_sq_db, nvmeq->dbbuf_sq_ei))
 		writel(nvmeq->sq_tail, nvmeq->q_db);
+	nvmeq->last_sq_tail = nvmeq->sq_tail;
 }
 
 /**
@@ -476,8 +490,7 @@ static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd,
 	       cmd, sizeof(*cmd));
 	if (++nvmeq->sq_tail == nvmeq->q_depth)
 		nvmeq->sq_tail = 0;
-	if (write_sq)
-		nvme_write_sq_db(nvmeq);
+	nvme_write_sq_db(nvmeq, write_sq);
 	spin_unlock(&nvmeq->sq_lock);
 }
 
@@ -486,7 +499,8 @@ static void nvme_commit_rqs(struct blk_mq_hw_ctx *hctx)
 	struct nvme_queue *nvmeq = hctx->driver_data;
 
 	spin_lock(&nvmeq->sq_lock);
-	nvme_write_sq_db(nvmeq);
+	if (nvmeq->sq_tail != nvmeq->last_sq_tail)
+		nvme_write_sq_db(nvmeq, true);
 	spin_unlock(&nvmeq->sq_lock);
 }
 
@@ -1496,6 +1510,7 @@ static void nvme_init_queue(struct nvme_queue *nvmeq, u16 qid)
 	struct nvme_dev *dev = nvmeq->dev;
 
 	nvmeq->sq_tail = 0;
+	nvmeq->last_sq_tail = 0;
 	nvmeq->cq_head = 0;
 	nvmeq->cq_phase = 1;
 	nvmeq->q_db = &dev->dbs[qid * 2 * dev->db_stride];
-- 
2.24.1



