From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:28 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:28 -0800
Subject: [PATCH 00/10] nvmet: fixes and some cleanups
Message-ID: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>

Hi,

This patch-series fixes behavior for id-ns that is against the spec and
does cleanup for nvmet_find_namespace() to get rid of the duplicate code
and remove the inconsistent error behavior of the host, Also this adds
an helper for unhandled commands to have uniform error message
reporting for the bdev, file and passthru backend. Last two patches
fix compilation warnings.

This is based on nvme-5.12. All the blktests are seem to pass on this
series.                

-ck

Chaitanya Kulkarni (10):
  nvmet: zeroout id-ns buffer for invalid nsid
  nvmet: return uniform error for invalid ns
  nvmet: set error in nvmet_find_namespace()
  nvmet: remove nsid param from nvmet_find_namespace()
  nvmet: remove extra variable in is-ns handler
  nvmet: add helper to report invalid opcode
  nvmet: use invalid cmd opcode helper
  nvmet: use invalid cmd opcode helper
  nvmet: use min of device_path and disk len
  nvme-loop: rename variable to get rid of the warn

 drivers/nvme/target/admin-cmd.c   | 63 +++++++++++++++----------------
 drivers/nvme/target/core.c        | 23 +++++------
 drivers/nvme/target/io-cmd-bdev.c |  5 +--
 drivers/nvme/target/io-cmd-file.c |  5 +--
 drivers/nvme/target/loop.c        |  2 +-
 drivers/nvme/target/nvmet.h       | 22 ++++++++++-
 drivers/nvme/target/passthru.c    |  2 +-
 drivers/nvme/target/trace.h       |  4 +-
 8 files changed, 70 insertions(+), 56 deletions(-)

# gitlog -11
6eb10edb698e (HEAD -> nvme-5.12) nvme-loop: rename variable to get rid of the warn
d94df0995f71 nvmet: use min of device_path and disk len
a98b4f9e46e0 nvmet: use invalid cmd opcode helper
a3ca782d82e5 nvmet: use invalid cmd opcode helper
102c6698b403 nvmet: add helper to report invalid opcode
eed0b3baae83 nvmet: remove extra variable in is-ns handler
4b47817fab94 nvmet: remove nsid param from nvmet_find_namespace()
68a97104491a nvmet: set error in nvmet_find_namespace()
e4079ffeb06e nvmet: return uniform error for invalid ns
5102bfe49832 nvmet: zeroout id-ns buffer for invalid nsid
ab9dbdb20926 (origin/nvme-5.12) nvme-tcp: use cancel tagset helper for tear down

blktests (master) # ./check tests/nvme
nvme/002 (create many subsystems and test discovery)         [passed]
    runtime  26.465s  ...  25.746s
nvme/003 (test if we're sending keep-alives to a discovery controller) [passed]
    runtime  10.135s  ...  10.134s
nvme/004 (test nvme and nvmet UUID NS descriptors)           [passed]
    runtime  1.677s  ...  1.708s
nvme/005 (reset local loopback target)                       [not run]
    nvme_core module does not have parameter multipath
nvme/006 (create an NVMeOF target with a block device-backed ns) [passed]
    runtime  0.120s  ...  0.115s
nvme/007 (create an NVMeOF target with a file-backed ns)     [passed]
    runtime  0.071s  ...  0.069s
nvme/008 (create an NVMeOF host with a block device-backed ns) [passed]
    runtime  1.691s  ...  1.693s
nvme/009 (create an NVMeOF host with a file-backed ns)       [passed]
    runtime  1.667s  ...  1.648s
nvme/010 (run data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  26.581s  ...  32.500s
nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  274.955s  ...  256.070s
nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  45.972s  ...  39.458s
nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  274.262s  ...  285.493s
nvme/014 (flush a NVMeOF block device-backed ns)             [passed]
    runtime  21.649s  ...  20.740s
nvme/015 (unit test for NVMe flush for file backed ns)       [passed]
    runtime  19.500s  ...  18.742s
nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [passed]
    runtime  14.570s  ...  14.262s
nvme/017 (create/delete many file-ns and test discovery)     [passed]
    runtime  14.831s  ...  14.774s
nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed]
    runtime  1.646s  ...  1.654s
nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed]
    runtime  1.689s  ...  1.708s
nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed]
    runtime  1.638s  ...  1.651s
nvme/021 (test NVMe list command on NVMeOF file-backed ns)   [passed]
    runtime  1.659s  ...  1.685s
nvme/022 (test NVMe reset command on NVMeOF file-backed ns)  [passed]
    runtime  2.089s  ...  2.078s
nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed]
    runtime  1.670s  ...  1.680s
nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed]
    runtime  1.640s  ...  1.644s
nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed]
    runtime  1.646s  ...  1.655s
nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed]
    runtime  1.654s  ...  1.637s
nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed]
    runtime  1.722s  ...  1.655s
nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed]
    runtime  1.641s  ...  1.679s
nvme/029 (test userspace IO via nvme-cli read/write interface) [passed]
    runtime  1.999s  ...  2.016s
nvme/030 (ensure the discovery generation counter is updated appropriately) [passed]
    runtime  0.305s  ...  0.290s
nvme/031 (test deletion of NVMeOF controllers immediately after setup) [passed]
    runtime  5.488s  ...  5.596s
nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed]
    runtime  0.043s  ...  0.043s

Tracing test :-

nvmet_req_init: nvmet1: qid=0, cmdid=2, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_admin_identify, cns=3, ctrlid=0)
nvmet_req_init: nvmet1: qid=0, cmdid=3, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_admin_identify, cns=0, ctrlid=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=10, cmdid=81, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=0, len=0, ctrl=0x0, dsmgmt=0, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=10, cmdid=82, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=1, len=0, ctrl=0x0, dsmgmt=0, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=113, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709104, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=114, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709118, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=115, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=0, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=116, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=1, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=117, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709119, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=118, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709087, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=119, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709112, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=120, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709088, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=121, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709070, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=122, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709046, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=123, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709035, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=124, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709028, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=125, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709006, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=126, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368708998, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=127, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368708996, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=113, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709001, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=114, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368708734, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=115, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=256, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=116, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=3, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=117, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=7, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=118, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=15, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=119, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=2, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=120, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=4, len=2, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=121, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=8, len=6, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=122, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=16, len=47, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=123, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=64, len=63, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=124, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368708992, len=3, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=125, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368708997, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=126, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368708999, len=1, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=127, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709002, len=3, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=113, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709007, len=20, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=114, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709029, len=5, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=115, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709036, len=9, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=116, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709047, len=8, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=117, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709056, len=13, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=118, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709071, len=15, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=119, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709089, len=14, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=120, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709105, len=6, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=121, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=5368709113, len=4, ctrl=0x8000, dsmgmt=7, reftag=0)
nvmet_req_init: nvmet1: disk=/dev/nullb0, qid=23, cmdid=122, nsid=1, flags=0x40, meta=0x0, cmd=(nvme_cmd_read, slba=512, len=0, ctrl=0x8000, dsmgmt=7, reftag=0)
kkjkj
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:29 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:29 -0800
Subject: [PATCH 01/10] nvmet: zeroout id-ns buffer for invalid nsid
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-2-chaitanya.kulkarni@wdc.com>

According to spec :-
"5.15.2.7 Identify Namespace data structure for an Allocated Namespace
ID (CNS 11h) The Identify Namespace data structure (refer to Figure 245)
is returned to the host for the namespace specified in the Namespace
Identifier (NSID) field if it is an allocated NSID. If the specified
namespace is an unallocated NSID then the controller returns a zero
filled data structure."

Move call to nvmet_find_namespace() in nvmet_execute_identify_ns()
above the id-ns buffer allocation since there is no point in allocating
a buffer if namespace doesn't exist. Call nvmet_zero_sgl() when call to
nvmet_find_namespace() fails and add an explicit comment to specify the
reason for zeroing the buffer which is not common for the NVMe commands.
Remove the done label and we can directly jump to out label from
nvmet_find_namespace() error case.

Fixes: bffcd507780e ("nvmet: set right status on error in id-ns handler")
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 613a4d8feac1..8cc7bb25d10d 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -476,19 +476,25 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 		goto out;
 	}
 
+	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
+	if (!req->ns) {
+		status = NVME_SC_INVALID_NS;
+		/*
+		 * According to spec : If the specified namespace is
+		 * an unallocated NSID then the controller returns a zero filled
+		 * data structure. Also don't override the error status as invalid
+		 * namespace takes priority over the failed zeroout buffer case.
+		 */
+		nvmet_zero_sgl(req, 0, sizeof(*id));
+		goto out;
+	}
+
 	id = kzalloc(sizeof(*id), GFP_KERNEL);
 	if (!id) {
 		status = NVME_SC_INTERNAL;
 		goto out;
 	}
 
-	/* return an all zeroed buffer if we can't find an active namespace */
-	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
-	if (!req->ns) {
-		status = NVME_SC_INVALID_NS;
-		goto done;
-	}
-
 	nvmet_ns_revalidate(req->ns);
 
 	/*
@@ -539,7 +545,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 
 	if (req->ns->readonly)
 		id->nsattr |= (1 << 0);
-done:
+
 	if (!status)
 		status = nvmet_copy_to_sgl(req, 0, id, sizeof(*id));
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:30 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:30 -0800
Subject: [PATCH 02/10] nvmet: return uniform error for invalid ns
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-3-chaitanya.kulkarni@wdc.com>

For nvmet_find_namespace() error case we have inconsistent error code
mapping in the function nvmet_get_smart_log_nsid(),
nvmet_execute_identify_ns() and nvmet_set_feat_write_protect().

There is no point in retrying for the invalid namesapce from the host
side. Set the error code to the NVME_SC_INVALID_NS | NVME_SC_DNR which
matches what we have in nvmet_execute_identify_desclist().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 8cc7bb25d10d..ee3121b447fe 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -82,7 +82,7 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req *req,
 		pr_err("Could not find namespace id : %d\n",
 				le32_to_cpu(req->cmd->get_log_page.nsid));
 		req->error_loc = offsetof(struct nvme_rw_command, nsid);
-		return NVME_SC_INVALID_NS;
+		return NVME_SC_INVALID_NS | NVME_SC_DNR;
 	}
 
 	/* we don't have the right data for file backed ns */
@@ -478,7 +478,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 
 	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
 	if (!req->ns) {
-		status = NVME_SC_INVALID_NS;
+		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
 		/*
 		 * According to spec : If the specified namespace is
 		 * an unallocated NSID then the controller returns a zero filled
@@ -703,6 +703,7 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->rw.nsid);
 	if (unlikely(!req->ns)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
+		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
 		return status;
 	}
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:31 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:31 -0800
Subject: [PATCH 03/10] nvmet: set error in nvmet_find_namespace()
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-4-chaitanya.kulkarni@wdc.com>

The six callers of nvmet_find_namespace() duplicate the error log page
update and status setting code for each call.

All callers are nvmet requests based functions and so we can pass req
to the nvmet_find_namesapce() & derive ctrl from req, that'll allow us
to update the error log page in nvmet_find_namespace(). Now that we
pass the request we can also get rid of the local variable in
nvmet_find_namespace() and use the req->ns and return the error code.

Replace the ctrl parameter with nvmet_req for nvmet_find_namespace(),
centralize the error log page update for non allocated namesapces, and
return uniform error for non-allocated namespace.

With this change now we have ten less lines and most importantly we
get rid of the inconsistent behaviour for different commands when ns is
not allocated.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 45 +++++++++++++--------------------
 drivers/nvme/target/core.c      | 23 ++++++++---------
 drivers/nvme/target/nvmet.h     |  2 +-
 3 files changed, 30 insertions(+), 40 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index ee3121b447fe..fbb5fe18f2d4 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -75,14 +75,13 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req *req,
 		struct nvme_smart_log *slog)
 {
 	u64 host_reads, host_writes, data_units_read, data_units_written;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl,
-				       req->cmd->get_log_page.nsid);
-	if (!req->ns) {
+	status = nvmet_find_namespace(req, req->cmd->get_log_page.nsid);
+	if (status) {
 		pr_err("Could not find namespace id : %d\n",
 				le32_to_cpu(req->cmd->get_log_page.nsid));
-		req->error_loc = offsetof(struct nvme_rw_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
+		return status;
 	}
 
 	/* we don't have the right data for file backed ns */
@@ -476,9 +475,8 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 		goto out;
 	}
 
-	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
-	if (!req->ns) {
-		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
+	status = nvmet_find_namespace(req, req->cmd->identify.nsid);
+	if (status) {
 		/*
 		 * According to spec : If the specified namespace is
 		 * an unallocated NSID then the controller returns a zero filled
@@ -610,15 +608,12 @@ static u16 nvmet_copy_ns_identifier(struct nvmet_req *req, u8 type, u8 len,
 
 static void nvmet_execute_identify_desclist(struct nvmet_req *req)
 {
-	u16 status = 0;
+	u16 status;
 	off_t off = 0;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->identify.nsid);
-	if (!req->ns) {
-		req->error_loc = offsetof(struct nvme_identify, nsid);
-		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
+	status = nvmet_find_namespace(req, req->cmd->identify.nsid);
+	if (status)
 		goto out;
-	}
 
 	if (memchr_inv(&req->ns->uuid, 0, sizeof(req->ns->uuid))) {
 		status = nvmet_copy_ns_identifier(req, NVME_NIDT_UUID,
@@ -698,14 +693,11 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 {
 	u32 write_protect = le32_to_cpu(req->cmd->common.cdw11);
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
-	u16 status = NVME_SC_FEATURE_NOT_CHANGEABLE;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->rw.nsid);
-	if (unlikely(!req->ns)) {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
+	status = nvmet_find_namespace(req, req->cmd->rw.nsid);
+	if (unlikely(status))
 		return status;
-	}
 
 	mutex_lock(&subsys->lock);
 	switch (write_protect) {
@@ -717,9 +709,9 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 		break;
 	case NVME_NS_NO_WRITE_PROTECT:
 		req->ns->readonly = false;
-		status = 0;
 		break;
 	default:
+		status = NVME_SC_FEATURE_NOT_CHANGEABLE;
 		break;
 	}
 
@@ -804,13 +796,12 @@ void nvmet_execute_set_features(struct nvmet_req *req)
 static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 {
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
-	u32 result;
+	u16 result;
+
+	result = nvmet_find_namespace(req, req->cmd->common.nsid);
+	if (result)
+		return result;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->common.nsid);
-	if (!req->ns)  {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
 	mutex_lock(&subsys->lock);
 	if (req->ns->readonly == true)
 		result = NVME_NS_WRITE_PROTECT;
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 8ce4d59cc9e7..6ebcbc637265 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -417,15 +417,16 @@ void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl)
 	cancel_delayed_work_sync(&ctrl->ka_work);
 }
 
-struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl, __le32 nsid)
+u16 nvmet_find_namespace(struct nvmet_req *req, __le32 id)
 {
-	struct nvmet_ns *ns;
-
-	ns = xa_load(&ctrl->subsys->namespaces, le32_to_cpu(nsid));
-	if (ns)
-		percpu_ref_get(&ns->ref);
+	req->ns = xa_load(&req->sq->ctrl->subsys->namespaces, le32_to_cpu(id));
+	if (unlikely(!req->ns)) {
+		req->error_loc = offsetof(struct nvme_common_command, nsid);
+		return NVME_SC_INVALID_NS | NVME_SC_DNR;
+	}
 
-	return ns;
+	percpu_ref_get(&req->ns->ref);
+	return NVME_SC_SUCCESS;
 }
 
 static void nvmet_destroy_namespace(struct percpu_ref *ref)
@@ -862,11 +863,9 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 	if (nvmet_req_passthru_ctrl(req))
 		return nvmet_parse_passthru_io_cmd(req);
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, cmd->rw.nsid);
-	if (unlikely(!req->ns)) {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	ret = nvmet_find_namespace(req, cmd->rw.nsid);
+	if (unlikely(ret))
+		return ret;
 	ret = nvmet_check_ana_state(req->port, req->ns);
 	if (unlikely(ret)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 8776dd1a0490..80811fccb431 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -443,7 +443,7 @@ struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
 void nvmet_subsys_put(struct nvmet_subsys *subsys);
 void nvmet_subsys_del_ctrls(struct nvmet_subsys *subsys);
 
-struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl, __le32 nsid);
+u16 nvmet_find_namespace(struct nvmet_req *req, __le32 id);
 void nvmet_put_namespace(struct nvmet_ns *ns);
 int nvmet_ns_enable(struct nvmet_ns *ns);
 void nvmet_ns_disable(struct nvmet_ns *ns);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:32 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:32 -0800
Subject: [PATCH 04/10] nvmet: remove nsid param from nvmet_find_namespace()
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-5-chaitanya.kulkarni@wdc.com>

The nvmet_find_namespace() takes nsid parameter which is from NVMe
commands structures such as get_log_page, identify, rw and common. All
these commands have same offset for the nsid field.

Derive nsid from req (req->cmd->common.nsid) and remove the extra
parameter from the nvmet_find_namespace().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 10 +++++-----
 drivers/nvme/target/core.c      |  6 ++++--
 drivers/nvme/target/nvmet.h     |  2 +-
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index fbb5fe18f2d4..ec64218db03c 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -77,7 +77,7 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req *req,
 	u64 host_reads, host_writes, data_units_read, data_units_written;
 	u16 status;
 
-	status = nvmet_find_namespace(req, req->cmd->get_log_page.nsid);
+	status = nvmet_find_namespace(req);
 	if (status) {
 		pr_err("Could not find namespace id : %d\n",
 				le32_to_cpu(req->cmd->get_log_page.nsid));
@@ -475,7 +475,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 		goto out;
 	}
 
-	status = nvmet_find_namespace(req, req->cmd->identify.nsid);
+	status = nvmet_find_namespace(req);
 	if (status) {
 		/*
 		 * According to spec : If the specified namespace is
@@ -611,7 +611,7 @@ static void nvmet_execute_identify_desclist(struct nvmet_req *req)
 	u16 status;
 	off_t off = 0;
 
-	status = nvmet_find_namespace(req, req->cmd->identify.nsid);
+	status = nvmet_find_namespace(req);
 	if (status)
 		goto out;
 
@@ -695,7 +695,7 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
 	u16 status;
 
-	status = nvmet_find_namespace(req, req->cmd->rw.nsid);
+	status = nvmet_find_namespace(req);
 	if (unlikely(status))
 		return status;
 
@@ -798,7 +798,7 @@ static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
 	u16 result;
 
-	result = nvmet_find_namespace(req, req->cmd->common.nsid);
+	result = nvmet_find_namespace(req);
 	if (result)
 		return result;
 
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 6ebcbc637265..9c6683f8e790 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -417,8 +417,10 @@ void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl)
 	cancel_delayed_work_sync(&ctrl->ka_work);
 }
 
-u16 nvmet_find_namespace(struct nvmet_req *req, __le32 id)
+u16 nvmet_find_namespace(struct nvmet_req *req)
 {
+	__le32 id = le32_to_cpu(req->cmd->common.nsid);
+
 	req->ns = xa_load(&req->sq->ctrl->subsys->namespaces, le32_to_cpu(id));
 	if (unlikely(!req->ns)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
@@ -863,7 +865,7 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 	if (nvmet_req_passthru_ctrl(req))
 		return nvmet_parse_passthru_io_cmd(req);
 
-	ret = nvmet_find_namespace(req, cmd->rw.nsid);
+	ret = nvmet_find_namespace(req);
 	if (unlikely(ret))
 		return ret;
 	ret = nvmet_check_ana_state(req->port, req->ns);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 80811fccb431..6864fd916bb5 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -443,7 +443,7 @@ struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
 void nvmet_subsys_put(struct nvmet_subsys *subsys);
 void nvmet_subsys_del_ctrls(struct nvmet_subsys *subsys);
 
-u16 nvmet_find_namespace(struct nvmet_req *req, __le32 id);
+u16 nvmet_find_namespace(struct nvmet_req *req);
 void nvmet_put_namespace(struct nvmet_ns *ns);
 int nvmet_ns_enable(struct nvmet_ns *ns);
 void nvmet_ns_disable(struct nvmet_ns *ns);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:33 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:33 -0800
Subject: [PATCH 05/10] nvmet: remove extra variable in is-ns handler
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-6-chaitanya.kulkarni@wdc.com>

In nvmet_execute_identify_ns() local variable ctrl is accessed only in
one place, remove that and directly use it from nvmet_req->sq->ctrl.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index ec64218db03c..d4e8f76ae59a 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -465,7 +465,6 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 
 static void nvmet_execute_identify_ns(struct nvmet_req *req)
 {
-	struct nvmet_ctrl *ctrl = req->sq->ctrl;
 	struct nvme_id_ns *id;
 	u16 status = 0;
 
@@ -531,7 +530,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 
 	id->lbaf[0].ds = req->ns->blksize_shift;
 
-	if (ctrl->pi_support && nvmet_ns_has_pi(req->ns)) {
+	if (req->sq->ctrl->pi_support && nvmet_ns_has_pi(req->ns)) {
 		id->dpc = NVME_NS_DPC_PI_FIRST | NVME_NS_DPC_PI_LAST |
 			  NVME_NS_DPC_PI_TYPE1 | NVME_NS_DPC_PI_TYPE2 |
 			  NVME_NS_DPC_PI_TYPE3;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:34 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:34 -0800
Subject: [PATCH 06/10] nvmet: add helper to report invalid opcode
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-7-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Add an helper and use it in block device backend to keep the code
and error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-bdev.c |  5 +----
 drivers/nvme/target/nvmet.h       | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c
index 23095bdfce06..105ef2b125cf 100644
--- a/drivers/nvme/target/io-cmd-bdev.c
+++ b/drivers/nvme/target/io-cmd-bdev.c
@@ -449,9 +449,6 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req)
 		req->execute = nvmet_bdev_execute_write_zeroes;
 		return 0;
 	default:
-		pr_err("unhandled cmd %d on qid %d\n", cmd->common.opcode,
-		       req->sq->qid);
-		req->error_loc = offsetof(struct nvme_common_command, opcode);
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 6864fd916bb5..2572e386e2b3 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -613,4 +613,24 @@ static inline sector_t nvmet_lba_to_sect(struct nvmet_ns *ns, __le64 lba)
 	return le64_to_cpu(lba) << (ns->blksize_shift - SECTOR_SHIFT);
 }
 
+static inline u16 nvmet_report_invalid_opcode(struct nvmet_req *req)
+{
+	char *backend;
+
+	if (req->ns->bdev)
+		backend = "bdev";
+	else if (req->ns->file)
+		backend = "file";
+	else if (nvmet_req_passthru_ctrl(req))
+		backend = "passthru";
+	else
+		backend = "unknown";
+
+	pr_err("unhandled cmd %d on qid %d for %s backend\n",
+		req->cmd->common.opcode, req->sq->qid, backend);
+
+	req->error_loc = offsetof(struct nvme_common_command, opcode);
+	return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+}
+
 #endif /* _NVMET_H */
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:35 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:35 -0800
Subject: [PATCH 07/10] nvmet: use invalid cmd opcode helper
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-8-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Use the previously introduced helper in file backend to reduce the
duplicate code and make the error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-file.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c
index 0abbefd9925e..715d4376c997 100644
--- a/drivers/nvme/target/io-cmd-file.c
+++ b/drivers/nvme/target/io-cmd-file.c
@@ -400,9 +400,6 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req)
 		req->execute = nvmet_file_execute_write_zeroes;
 		return 0;
 	default:
-		pr_err("unhandled cmd for file ns %d on qid %d\n",
-				cmd->common.opcode, req->sq->qid);
-		req->error_loc = offsetof(struct nvme_common_command, opcode);
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:36 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:36 -0800
Subject: [PATCH 08/10] nvmet: use invalid cmd opcode helper
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-9-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Use the previously introduced helper in the passthru backend to make the
error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index cbc88acdd233..3b22f4a868f4 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -494,7 +494,7 @@ u16 nvmet_parse_passthru_admin_cmd(struct nvmet_req *req)
 		return nvmet_setup_passthru_command(req);
 	default:
 		/* Reject commands not in the allowlist above */
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:37 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:37 -0800
Subject: [PATCH 09/10] nvmet: use min of device_path and disk len
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-10-chaitanya.kulkarni@wdc.com>

In function __assign_req_name() instead of using the DEVICE_NAME_LEN in
strncpy() use min of DISK_NAME_LEN and strlen(req->ns->device_path).

This is needed to turn off the following warnings:-

In file included from drivers/nvme/target/core.c:14:
In function ?__assign_req_name?,
    inlined from ?trace_event_raw_event_nvmet_req_init? at drivers/nvme/target/./trace.h:58:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?perf_trace_nvmet_req_complete? at drivers/nvme/target/./trace.h:100:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?perf_trace_nvmet_req_init? at drivers/nvme/target/./trace.h:58:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?trace_event_raw_event_nvmet_req_complete? at drivers/nvme/target/./trace.h:100:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/trace.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/trace.h b/drivers/nvme/target/trace.h
index c14e3249a14d..958dca7bcfc7 100644
--- a/drivers/nvme/target/trace.h
+++ b/drivers/nvme/target/trace.h
@@ -48,8 +48,10 @@ static inline struct nvmet_ctrl *nvmet_req_to_ctrl(struct nvmet_req *req)
 
 static inline void __assign_req_name(char *name, struct nvmet_req *req)
 {
+	size_t len = min_t(size_t, DISK_NAME_LEN, strlen(req->ns->device_path));
+
 	if (req->ns)
-		strncpy(name, req->ns->device_path, DISK_NAME_LEN);
+		strncpy(name, req->ns->device_path, len);
 	else
 		memset(name, 0, DISK_NAME_LEN);
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  1 00:41:38 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 31 Jan 2021 21:41:38 -0800
Subject: [PATCH 10/10] nvme-loop: rename variable to get rid of the warn
In-Reply-To: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201054138.34324-11-chaitanya.kulkarni@wdc.com>

Rename the nvme_loop_init_request() numa_node parameter to
hw_queue_numa_node so that we can get rid of the following warning :-

drivers/nvme/target/loop.c: In function ?nvme_loop_init_request?:
drivers/nvme/target/loop.c:205:16: warning: declaration of ?numa_node? shadows a global declaration [-Wshadow]
   unsigned int numa_node)

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/loop.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..88d0656ad4a5 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -202,7 +202,7 @@ static int nvme_loop_init_iod(struct nvme_loop_ctrl *ctrl,
 
 static int nvme_loop_init_request(struct blk_mq_tag_set *set,
 		struct request *req, unsigned int hctx_idx,
-		unsigned int numa_node)
+		unsigned int hw_queue_numa_node)
 {
 	struct nvme_loop_ctrl *ctrl = set->driver_data;
 
-- 
2.22.1



From hare at suse.de  Mon Feb  1 02:29:57 2021
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 1 Feb 2021 08:29:57 +0100
Subject: [PATCH v2] nvme-multipath: Early exit if no path is available
In-Reply-To: <6ceff3cb-c9e9-7e74-92f0-dd745987c943@huawei.com>
References: <20210127103033.15318-1-dwagner@suse.de>
 <db9baae0-547c-7ff4-8b2c-0b95f14be67c@huawei.com>
 <20210128075837.u5u56t23fq5gu6ou@beryllium.lan>
 <69575290-200e-b4a1-4269-c71e4c2cc37b@huawei.com>
 <20210128094004.erwnszjqcxlsi2kd@beryllium.lan>
 <ebb1d098-3ded-e592-4419-e905aabe824f@huawei.com>
 <675d3cf7-1ae8-adc5-b6d0-359fe10f6b23@grimberg.me>
 <59cd053e-46cb-0235-141f-4ce919c93f48@huawei.com>
 <65392653-6b03-9195-f686-5fe4b3290bd2@suse.de>
 <81b22bbf-4dd3-6161-e63a-9699690a4e4f@huawei.com>
 <715dd943-0587-be08-2840-e0948cf0bc62@suse.de>
 <eb131d8f-f009-42e7-105d-58b84060f0dd@huawei.com>
 <ac019690-7f02-d28c-ed58-bfc8c1d48879@suse.de>
 <6ceff3cb-c9e9-7e74-92f0-dd745987c943@huawei.com>
Message-ID: <114751ac-1f7d-ce5e-12c5-7d6303bdb999@suse.de>

On 2/1/21 3:16 AM, Chao Leng wrote:
> 
> 
> On 2021/1/29 17:20, Hannes Reinecke wrote:
>> On 1/29/21 9:46 AM, Chao Leng wrote:
>>>
>>>
>>> On 2021/1/29 16:33, Hannes Reinecke wrote:
>>>> On 1/29/21 8:45 AM, Chao Leng wrote:
>>>>>
>>>>>
>>>>> On 2021/1/29 15:06, Hannes Reinecke wrote:
>>>>>> On 1/29/21 4:07 AM, Chao Leng wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 2021/1/29 9:42, Sagi Grimberg wrote:
>>>>>>>>
>>>>>>>>>> You can't see exactly where it dies but I followed the 
>>>>>>>>>> assembly to
>>>>>>>>>> nvme_round_robin_path(). Maybe it's not the initial 
>>>>>>>>>> nvme_next_ns(head,
>>>>>>>>>> old) which returns NULL but nvme_next_ns() is returning NULL 
>>>>>>>>>> eventually
>>>>>>>>>> (list_next_or_null_rcu()).
>>>>>>>>> So there is other bug cause nvme_next_ns abormal.
>>>>>>>>> I review the code about head->list and head->current_path, I 
>>>>>>>>> find 2 bugs
>>>>>>>>> may cause the bug:
>>>>>>>>> First, I already send the patch. see:
>>>>>>>>> https://lore.kernel.org/linux-nvme/20210128033351.22116-1-lengchao at huawei.com/ 
>>>>>>>>>
>>>>>>>>> Second, in nvme_ns_remove, list_del_rcu is before
>>>>>>>>> nvme_mpath_clear_current_path. This may cause "old" is deleted 
>>>>>>>>> from the
>>>>>>>>> "head", but still use "old". I'm not sure there's any other
>>>>>>>>> consideration here, I will check it and try to fix it.
>>>>>>>>
>>>>>>>> The reason why we first remove from head->list and only then clear
>>>>>>>> current_path is because the other way around there is no way
>>>>>>>> to guarantee that that the ns won't be assigned as current_path
>>>>>>>> again (because it is in head->list).
>>>>>>> ok, I see.
>>>>>>>>
>>>>>>>> nvme_ns_remove fences continue of deletion of the ns by 
>>>>>>>> synchronizing
>>>>>>>> the srcu such that for sure the current_path clearance is visible.
>>>>>>> The list will be like this:
>>>>>>> head->next = ns1;
>>>>>>> ns1->next = head;
>>>>>>> old->next = ns1;
>>>>>>
>>>>>> Where does 'old' pointing to?
>>>>>>
>>>>>>> This may cause infinite loop in nvme_round_robin_path.
>>>>>>> for (ns = nvme_next_ns(head, old);
>>>>>>> ?????ns != old;
>>>>>>> ?????ns = nvme_next_ns(head, ns))
>>>>>>> The ns will always be ns1, and then infinite loop.
>>>>>>
>>>>>> No. nvme_next_ns() will return NULL.
>>>>> If there is just one path(the "old") and the "old" is deleted,
>>>>> nvme_next_ns() will return NULL.
>>>>> The list like this:
>>>>> head->next = head;
>>>>> old->next = head;
>>>>> If there is two or more path and the "old" is deleted,
>>>>> "for" will be infinite loop. because nvme_next_ns() will return
>>>>> the path which in the list except the "old", check condition will
>>>>> be true for ever.
>>>>
>>>> But that will be caught by the statement above:
>>>>
>>>> if (list_is_singular(&head->list))
>>>>
>>>> no?
>>> Two path just a sample example.
>>> If there is just two path, will enter it, may cause no path but there is
>>> actually one path. It is falsely assumed that the "old" must be not 
>>> deleted.
>>> If there is more than two path, will cause infinite loop.
>> So you mean we'll need something like this?
>>
>> diff --git a/drivers/nvme/host/multipath.c 
>> b/drivers/nvme/host/multipath.c
>> index 71696819c228..8ffccaf9c19a 100644
>> --- a/drivers/nvme/host/multipath.c
>> +++ b/drivers/nvme/host/multipath.c
>> @@ -202,10 +202,12 @@ static struct nvme_ns *__nvme_find_path(struct 
>> nvme_ns_head *head, int node)
>> ??static struct nvme_ns *nvme_next_ns(struct nvme_ns_head *head,
>> ???????????????? struct nvme_ns *ns)
>> ??{
>> -?????? ns = list_next_or_null_rcu(&head->list, &ns->siblings, struct 
>> nvme_ns,
>> -?????????????????????? siblings);
>> -?????? if (ns)
>> -?????????????? return ns;
>> +?????? if (ns) {
>> +?????????????? ns = list_next_or_null_rcu(&head->list, &ns->siblings,
>> +????????????????????????????????????????? struct nvme_ns, siblings);
>> +?????????????? if (ns)
>> +?????????????????????? return ns;
>> +?????? }
> No, in the scenario, ns should not be NULL.

Why not? 'ns == NULL' is precisely the corner-case this is trying to fix...

> May be we can do like this:
> 
> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> index 282b7a4ea9a9..b895011a2cbd 100644
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -199,30 +199,24 @@ static struct nvme_ns *__nvme_find_path(struct 
> nvme_ns_head *head, int node)
>  ??????? return found;
>  ?}
> 
> -static struct nvme_ns *nvme_next_ns(struct nvme_ns_head *head,
> -?????????????? struct nvme_ns *ns)
> -{
> -?????? ns = list_next_or_null_rcu(&head->list, &ns->siblings, struct 
> nvme_ns,
> -?????????????????????? siblings);
> -?????? if (ns)
> -?????????????? return ns;
> -?????? return list_first_or_null_rcu(&head->list, struct nvme_ns, 
> siblings);
> -}
> +#define nvme_next_ns_condition(head, current, condition) \
> +({ \
> +?????? struct nvme_ns *__ptr = list_next_or_null_rcu(&(head)->list, \
> +?????????????? &(current)->siblings, struct nvme_ns, siblings); \
> +?????? __ptr ? __ptr : (condition) ? (condition) = false, \
> +?????????????? list_first_or_null_rcu(&(head)->list, struct nvme_ns, \
> +?????????????????????? siblings) : NULL; \
> +})
> 
Urgh. Please, no. That is well impossible to debug.
Can you please open-code it to demonstrate where the difference to the 
current (and my fixed) versions is?
I'm still not clear where the problem is once we applied both patches.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From lengchao at huawei.com  Mon Feb  1 03:47:50 2021
From: lengchao at huawei.com (Chao Leng)
Date: Mon, 1 Feb 2021 16:47:50 +0800
Subject: [PATCH v2] nvme-multipath: Early exit if no path is available
In-Reply-To: <114751ac-1f7d-ce5e-12c5-7d6303bdb999@suse.de>
References: <20210127103033.15318-1-dwagner@suse.de>
 <db9baae0-547c-7ff4-8b2c-0b95f14be67c@huawei.com>
 <20210128075837.u5u56t23fq5gu6ou@beryllium.lan>
 <69575290-200e-b4a1-4269-c71e4c2cc37b@huawei.com>
 <20210128094004.erwnszjqcxlsi2kd@beryllium.lan>
 <ebb1d098-3ded-e592-4419-e905aabe824f@huawei.com>
 <675d3cf7-1ae8-adc5-b6d0-359fe10f6b23@grimberg.me>
 <59cd053e-46cb-0235-141f-4ce919c93f48@huawei.com>
 <65392653-6b03-9195-f686-5fe4b3290bd2@suse.de>
 <81b22bbf-4dd3-6161-e63a-9699690a4e4f@huawei.com>
 <715dd943-0587-be08-2840-e0948cf0bc62@suse.de>
 <eb131d8f-f009-42e7-105d-58b84060f0dd@huawei.com>
 <ac019690-7f02-d28c-ed58-bfc8c1d48879@suse.de>
 <6ceff3cb-c9e9-7e74-92f0-dd745987c943@huawei.com>
 <114751ac-1f7d-ce5e-12c5-7d6303bdb999@suse.de>
Message-ID: <aebc6c2d-4711-95ba-daf2-1cd17fc6f0e7@huawei.com>



On 2021/2/1 15:29, Hannes Reinecke wrote:
> On 2/1/21 3:16 AM, Chao Leng wrote:
>>
>>
>> On 2021/1/29 17:20, Hannes Reinecke wrote:
>>> On 1/29/21 9:46 AM, Chao Leng wrote:
>>>>
>>>>
>>>> On 2021/1/29 16:33, Hannes Reinecke wrote:
>>>>> On 1/29/21 8:45 AM, Chao Leng wrote:
>>>>>>
>>>>>>
>>>>>> On 2021/1/29 15:06, Hannes Reinecke wrote:
>>>>>>> On 1/29/21 4:07 AM, Chao Leng wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2021/1/29 9:42, Sagi Grimberg wrote:
>>>>>>>>>
>>>>>>>>>>> You can't see exactly where it dies but I followed the assembly to
>>>>>>>>>>> nvme_round_robin_path(). Maybe it's not the initial nvme_next_ns(head,
>>>>>>>>>>> old) which returns NULL but nvme_next_ns() is returning NULL eventually
>>>>>>>>>>> (list_next_or_null_rcu()).
>>>>>>>>>> So there is other bug cause nvme_next_ns abormal.
>>>>>>>>>> I review the code about head->list and head->current_path, I find 2 bugs
>>>>>>>>>> may cause the bug:
>>>>>>>>>> First, I already send the patch. see:
>>>>>>>>>> https://lore.kernel.org/linux-nvme/20210128033351.22116-1-lengchao at huawei.com/
>>>>>>>>>> Second, in nvme_ns_remove, list_del_rcu is before
>>>>>>>>>> nvme_mpath_clear_current_path. This may cause "old" is deleted from the
>>>>>>>>>> "head", but still use "old". I'm not sure there's any other
>>>>>>>>>> consideration here, I will check it and try to fix it.
>>>>>>>>>
>>>>>>>>> The reason why we first remove from head->list and only then clear
>>>>>>>>> current_path is because the other way around there is no way
>>>>>>>>> to guarantee that that the ns won't be assigned as current_path
>>>>>>>>> again (because it is in head->list).
>>>>>>>> ok, I see.
>>>>>>>>>
>>>>>>>>> nvme_ns_remove fences continue of deletion of the ns by synchronizing
>>>>>>>>> the srcu such that for sure the current_path clearance is visible.
>>>>>>>> The list will be like this:
>>>>>>>> head->next = ns1;
>>>>>>>> ns1->next = head;
>>>>>>>> old->next = ns1;
>>>>>>>
>>>>>>> Where does 'old' pointing to?
>>>>>>>
>>>>>>>> This may cause infinite loop in nvme_round_robin_path.
>>>>>>>> for (ns = nvme_next_ns(head, old);
>>>>>>>> ?????ns != old;
>>>>>>>> ?????ns = nvme_next_ns(head, ns))
>>>>>>>> The ns will always be ns1, and then infinite loop.
>>>>>>>
>>>>>>> No. nvme_next_ns() will return NULL.
>>>>>> If there is just one path(the "old") and the "old" is deleted,
>>>>>> nvme_next_ns() will return NULL.
>>>>>> The list like this:
>>>>>> head->next = head;
>>>>>> old->next = head;
>>>>>> If there is two or more path and the "old" is deleted,
>>>>>> "for" will be infinite loop. because nvme_next_ns() will return
>>>>>> the path which in the list except the "old", check condition will
>>>>>> be true for ever.
>>>>>
>>>>> But that will be caught by the statement above:
>>>>>
>>>>> if (list_is_singular(&head->list))
>>>>>
>>>>> no?
>>>> Two path just a sample example.
>>>> If there is just two path, will enter it, may cause no path but there is
>>>> actually one path. It is falsely assumed that the "old" must be not deleted.
>>>> If there is more than two path, will cause infinite loop.
>>> So you mean we'll need something like this?
>>>
>>> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
>>> index 71696819c228..8ffccaf9c19a 100644
>>> --- a/drivers/nvme/host/multipath.c
>>> +++ b/drivers/nvme/host/multipath.c
>>> @@ -202,10 +202,12 @@ static struct nvme_ns *__nvme_find_path(struct nvme_ns_head *head, int node)
>>> ??static struct nvme_ns *nvme_next_ns(struct nvme_ns_head *head,
>>> ???????????????? struct nvme_ns *ns)
>>> ??{
>>> -?????? ns = list_next_or_null_rcu(&head->list, &ns->siblings, struct nvme_ns,
>>> -?????????????????????? siblings);
>>> -?????? if (ns)
>>> -?????????????? return ns;
>>> +?????? if (ns) {
>>> +?????????????? ns = list_next_or_null_rcu(&head->list, &ns->siblings,
>>> +????????????????????????????????????????? struct nvme_ns, siblings);
>>> +?????????????? if (ns)
>>> +?????????????????????? return ns;
>>> +?????? }
>> No, in the scenario, ns should not be NULL.
> 
> Why not? 'ns == NULL' is precisely the corner-case this is trying to fix...
> 
>> May be we can do like this:
>>
>> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
>> index 282b7a4ea9a9..b895011a2cbd 100644
>> --- a/drivers/nvme/host/multipath.c
>> +++ b/drivers/nvme/host/multipath.c
>> @@ -199,30 +199,24 @@ static struct nvme_ns *__nvme_find_path(struct nvme_ns_head *head, int node)
>> ???????? return found;
>> ??}
>>
>> -static struct nvme_ns *nvme_next_ns(struct nvme_ns_head *head,
>> -?????????????? struct nvme_ns *ns)
>> -{
>> -?????? ns = list_next_or_null_rcu(&head->list, &ns->siblings, struct nvme_ns,
>> -?????????????????????? siblings);
>> -?????? if (ns)
>> -?????????????? return ns;
>> -?????? return list_first_or_null_rcu(&head->list, struct nvme_ns, siblings);
>> -}
>> +#define nvme_next_ns_condition(head, current, condition) \
>> +({ \
>> +?????? struct nvme_ns *__ptr = list_next_or_null_rcu(&(head)->list, \
>> +?????????????? &(current)->siblings, struct nvme_ns, siblings); \
>> +?????? __ptr ? __ptr : (condition) ? (condition) = false, \
>> +?????????????? list_first_or_null_rcu(&(head)->list, struct nvme_ns, \
>> +?????????????????????? siblings) : NULL; \
>> +})
>>
> Urgh. Please, no. That is well impossible to debug.
> Can you please open-code it to demonstrate where the difference to the current (and my fixed) versions is?
> I'm still not clear where the problem is once we applied both patches.
For example assume the list has three path, and all path is not NVME_ANA_OPTIMIZED:
head->next = ns1;
ns1->next = ns2;
ns2->next = head;
old->next = ns2;

My patch work flow:
nvme_next_ns_condition(head, old, true) return ns2;
nvme_next_ns_condition(head, ns2, true) change the condition to false
                                         and return ns1;
nvme_next_ns_condition(head, ns1, false) return ns2;
nvme_next_ns_condition(head, ns2, false) return NULL;
And then the loop end.

Your patch work flow:
nvme_next_ns(head, old) return ns2;
nvme_next_ns(head, ns2) return ns1;
nvme_next_ns(head, ns1) return ns2;
nvme_next_ns(head, ns2) return ns1;
nvme_next_ns(head, ns1) return ns2;
And then the loop is infinite.
> 
> Cheers,
> 
> Hannes


From hare at suse.de  Mon Feb  1 03:57:22 2021
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 1 Feb 2021 09:57:22 +0100
Subject: [PATCH v2] nvme-multipath: Early exit if no path is available
In-Reply-To: <aebc6c2d-4711-95ba-daf2-1cd17fc6f0e7@huawei.com>
References: <20210127103033.15318-1-dwagner@suse.de>
 <db9baae0-547c-7ff4-8b2c-0b95f14be67c@huawei.com>
 <20210128075837.u5u56t23fq5gu6ou@beryllium.lan>
 <69575290-200e-b4a1-4269-c71e4c2cc37b@huawei.com>
 <20210128094004.erwnszjqcxlsi2kd@beryllium.lan>
 <ebb1d098-3ded-e592-4419-e905aabe824f@huawei.com>
 <675d3cf7-1ae8-adc5-b6d0-359fe10f6b23@grimberg.me>
 <59cd053e-46cb-0235-141f-4ce919c93f48@huawei.com>
 <65392653-6b03-9195-f686-5fe4b3290bd2@suse.de>
 <81b22bbf-4dd3-6161-e63a-9699690a4e4f@huawei.com>
 <715dd943-0587-be08-2840-e0948cf0bc62@suse.de>
 <eb131d8f-f009-42e7-105d-58b84060f0dd@huawei.com>
 <ac019690-7f02-d28c-ed58-bfc8c1d48879@suse.de>
 <6ceff3cb-c9e9-7e74-92f0-dd745987c943@huawei.com>
 <114751ac-1f7d-ce5e-12c5-7d6303bdb999@suse.de>
 <aebc6c2d-4711-95ba-daf2-1cd17fc6f0e7@huawei.com>
Message-ID: <47a1b796-9d91-5947-4bac-dd8f397041a3@suse.de>

On 2/1/21 9:47 AM, Chao Leng wrote:
> 
> 
> On 2021/2/1 15:29, Hannes Reinecke wrote:[ .. ]
>> Urgh. Please, no. That is well impossible to debug.
>> Can you please open-code it to demonstrate where the difference to the 
>> current (and my fixed) versions is?
>> I'm still not clear where the problem is once we applied both patches.
> For example assume the list has three path, and all path is not 
> NVME_ANA_OPTIMIZED:
> head->next = ns1;
> ns1->next = ns2;
> ns2->next = head;
> old->next = ns2;
> 
And this is where I have issues with.
Where does 'old' come from?
Clearly it was part of the list at one point; so what happened to it?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From lengchao at huawei.com  Mon Feb  1 04:40:40 2021
From: lengchao at huawei.com (Chao Leng)
Date: Mon, 1 Feb 2021 17:40:40 +0800
Subject: [PATCH v2] nvme-multipath: Early exit if no path is available
In-Reply-To: <47a1b796-9d91-5947-4bac-dd8f397041a3@suse.de>
References: <20210127103033.15318-1-dwagner@suse.de>
 <db9baae0-547c-7ff4-8b2c-0b95f14be67c@huawei.com>
 <20210128075837.u5u56t23fq5gu6ou@beryllium.lan>
 <69575290-200e-b4a1-4269-c71e4c2cc37b@huawei.com>
 <20210128094004.erwnszjqcxlsi2kd@beryllium.lan>
 <ebb1d098-3ded-e592-4419-e905aabe824f@huawei.com>
 <675d3cf7-1ae8-adc5-b6d0-359fe10f6b23@grimberg.me>
 <59cd053e-46cb-0235-141f-4ce919c93f48@huawei.com>
 <65392653-6b03-9195-f686-5fe4b3290bd2@suse.de>
 <81b22bbf-4dd3-6161-e63a-9699690a4e4f@huawei.com>
 <715dd943-0587-be08-2840-e0948cf0bc62@suse.de>
 <eb131d8f-f009-42e7-105d-58b84060f0dd@huawei.com>
 <ac019690-7f02-d28c-ed58-bfc8c1d48879@suse.de>
 <6ceff3cb-c9e9-7e74-92f0-dd745987c943@huawei.com>
 <114751ac-1f7d-ce5e-12c5-7d6303bdb999@suse.de>
 <aebc6c2d-4711-95ba-daf2-1cd17fc6f0e7@huawei.com>
 <47a1b796-9d91-5947-4bac-dd8f397041a3@suse.de>
Message-ID: <bd37abd5-759d-efe2-fdcd-8b004a41c75a@huawei.com>



On 2021/2/1 16:57, Hannes Reinecke wrote:
> On 2/1/21 9:47 AM, Chao Leng wrote:
>>
>>
>> On 2021/2/1 15:29, Hannes Reinecke wrote:[ .. ]
>>> Urgh. Please, no. That is well impossible to debug.
>>> Can you please open-code it to demonstrate where the difference to the current (and my fixed) versions is?
>>> I'm still not clear where the problem is once we applied both patches.
>> For example assume the list has three path, and all path is not NVME_ANA_OPTIMIZED:
>> head->next = ns1;
>> ns1->next = ns2;
>> ns2->next = head;
>> old->next = ns2;
>>
> And this is where I have issues with.
> Where does 'old' come from?
> Clearly it was part of the list at one point; so what happened to it?
I explained this earlier.
In nvme_ns_remove, there is a hole between list_del_rcu and
nvme_mpath_clear_current_path. If head->current_path is the "old", and
the "old" is removing. The "old" is already removed from the list by
list_del_rcu, but head->current_path is not clear to NULL by
nvme_mpath_clear_current_path.
Find path is race with nvme_ns_remove, use the "old" pass to
nvme_round_robin_path to find path.
> 
> Cheers,
> 
> Hannes


From borisp at mellanox.com  Mon Feb  1 05:04:50 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:50 +0200
Subject: [PATCH v3 net-next 02/21] net: Introduce direct data placement tcp
 offload
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-3-borisp@mellanox.com>

This commit introduces direct data placement offload for TCP.
This capability is accompanied by new net_device operations that
configure hardware contexts. There is a context per socket, and a context per DDP
opreation. Additionally, a resynchronization routine is used to assist
hardware handle TCP OOO, and continue the offload.
Furthermore, we let the offloading driver advertise what is the max hw
sectors/segments.

Using this interface, the NIC hardware will scatter TCP payload directly
to the BIO pages according to the command_id.
To maintain the correctness of the network stack, the driver is expected
to construct SKBs that point to the BIO pages.

The SKB passed to the network stack from the driver
represents data as it is on the wire, while it is pointing
directly to data in destination buffers.
As a result, data from page frags should not be copied out to
the linear part. To avoid needless copies, such as when using
skb_condense, we mark the skb->ddp_crc bit. This bit will be
used to indicate both ddp and crc offload (next patch in series).

A follow-up patch will use this interface for DDP in NVMe-TCP.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/netdev_features.h    |   2 +
 include/linux/netdevice.h          |   5 ++
 include/linux/skbuff.h             |   4 +
 include/net/inet_connection_sock.h |   4 +
 include/net/tcp_ddp.h              | 136 +++++++++++++++++++++++++++++
 net/Kconfig                        |   9 ++
 net/core/skbuff.c                  |   8 +-
 net/ethtool/common.c               |   1 +
 net/ipv4/tcp_input.c               |   8 ++
 net/ipv4/tcp_ipv4.c                |   3 +
 net/ipv4/tcp_offload.c             |   3 +
 11 files changed, 182 insertions(+), 1 deletion(-)
 create mode 100644 include/net/tcp_ddp.h

diff --git a/include/linux/netdev_features.h b/include/linux/netdev_features.h
index c06d6aaba9df..7977371d2dd1 100644
--- a/include/linux/netdev_features.h
+++ b/include/linux/netdev_features.h
@@ -85,6 +85,7 @@ enum {
 
 	NETIF_F_HW_MACSEC_BIT,		/* Offload MACsec operations */
 	NETIF_F_GRO_UDP_FWD_BIT,	/* Allow UDP GRO for forwarding */
+	NETIF_F_HW_TCP_DDP_BIT,		/* TCP direct data placement offload */
 
 	/*
 	 * Add your fresh new feature above and remember to update
@@ -159,6 +160,7 @@ enum {
 #define NETIF_F_GSO_FRAGLIST	__NETIF_F(GSO_FRAGLIST)
 #define NETIF_F_HW_MACSEC	__NETIF_F(HW_MACSEC)
 #define NETIF_F_GRO_UDP_FWD	__NETIF_F(GRO_UDP_FWD)
+#define NETIF_F_HW_TCP_DDP	__NETIF_F(HW_TCP_DDP)
 
 /* Finds the next feature with the highest number of the range of start till 0.
  */
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index e9e7ada07ea1..bd28520e30f2 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -941,6 +941,7 @@ struct dev_ifalias {
 
 struct devlink;
 struct tlsdev_ops;
+struct tcp_ddp_dev_ops;
 
 struct netdev_name_node {
 	struct hlist_node hlist;
@@ -1942,6 +1943,10 @@ struct net_device {
 	const struct tlsdev_ops *tlsdev_ops;
 #endif
 
+#ifdef CONFIG_TCP_DDP
+	const struct tcp_ddp_dev_ops *tcp_ddp_ops;
+#endif
+
 	const struct header_ops *header_ops;
 
 	unsigned char		operstate;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 9313b5aaf45b..6d00d62628c8 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -684,6 +684,7 @@ typedef unsigned char *sk_buff_data_t;
  *		CHECKSUM_UNNECESSARY (max 3)
  *	@dst_pending_confirm: need to confirm neighbour
  *	@decrypted: Decrypted SKB
+ *	@ddp_crc: DDP or CRC offloaded
  *	@napi_id: id of the NAPI struct this skb came from
  *	@sender_cpu: (aka @napi_id) source CPU in XPS
  *	@secmark: security marking
@@ -860,6 +861,9 @@ struct sk_buff {
 #ifdef CONFIG_TLS_DEVICE
 	__u8			decrypted:1;
 #endif
+#ifdef CONFIG_TCP_DDP
+	__u8                    ddp_crc:1;
+#endif
 
 #ifdef CONFIG_NET_SCHED
 	__u16			tc_index;	/* traffic control index */
diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
index c11f80f328f1..f9eb28f14d43 100644
--- a/include/net/inet_connection_sock.h
+++ b/include/net/inet_connection_sock.h
@@ -66,6 +66,8 @@ struct inet_connection_sock_af_ops {
  * @icsk_ulp_ops	   Pluggable ULP control hook
  * @icsk_ulp_data	   ULP private data
  * @icsk_clean_acked	   Clean acked data hook
+ * @icsk_ulp_ddp_ops	   Pluggable ULP direct data placement control hook
+ * @icsk_ulp_ddp_data	   ULP direct data placement private data
  * @icsk_listen_portaddr_node	hash to the portaddr listener hashtable
  * @icsk_ca_state:	   Congestion control state
  * @icsk_retransmits:	   Number of unrecovered [RTO] timeouts
@@ -96,6 +98,8 @@ struct inet_connection_sock {
 	const struct tcp_ulp_ops  *icsk_ulp_ops;
 	void __rcu		  *icsk_ulp_data;
 	void (*icsk_clean_acked)(struct sock *sk, u32 acked_seq);
+	const struct tcp_ddp_ulp_ops  *icsk_ulp_ddp_ops;
+	void __rcu		  *icsk_ulp_ddp_data;
 	struct hlist_node         icsk_listen_portaddr_node;
 	unsigned int		  (*icsk_sync_mss)(struct sock *sk, u32 pmtu);
 	__u8			  icsk_ca_state:5,
diff --git a/include/net/tcp_ddp.h b/include/net/tcp_ddp.h
new file mode 100644
index 000000000000..31e5b1a16d0f
--- /dev/null
+++ b/include/net/tcp_ddp.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * tcp_ddp.h
+ *	Author:	Boris Pismenny <borisp at mellanox.com>
+ *	Copyright (C) 2021 Mellanox Technologies.
+ */
+#ifndef _TCP_DDP_H
+#define _TCP_DDP_H
+
+#include <linux/netdevice.h>
+#include <net/inet_connection_sock.h>
+#include <net/sock.h>
+
+/* limits returned by the offload driver, zero means don't care */
+struct tcp_ddp_limits {
+	int	 max_ddp_sgl_len;
+};
+
+enum tcp_ddp_type {
+	TCP_DDP_NVME = 1,
+};
+
+/**
+ * struct tcp_ddp_config - Generic tcp ddp configuration: tcp ddp IO queue
+ * config implementations must use this as the first member.
+ * Add new instances of tcp_ddp_config below (nvme-tcp, etc.).
+ */
+struct tcp_ddp_config {
+	enum tcp_ddp_type    type;
+	unsigned char        buf[];
+};
+
+/**
+ * struct nvme_tcp_ddp_config - nvme tcp ddp configuration for an IO queue
+ *
+ * @pfv:        pdu version (e.g., NVME_TCP_PFV_1_0)
+ * @cpda:       controller pdu data alignmend (dwords, 0's based)
+ * @dgst:       digest types enabled.
+ *              The netdev will offload crc if ddp_crc is supported.
+ * @queue_size: number of nvme-tcp IO queue elements
+ * @queue_id:   queue identifier
+ * @cpu_io:     cpu core running the IO thread for this queue
+ */
+struct nvme_tcp_ddp_config {
+	struct tcp_ddp_config   cfg;
+
+	u16			pfv;
+	u8			cpda;
+	u8			dgst;
+	int			queue_size;
+	int			queue_id;
+	int			io_cpu;
+};
+
+/**
+ * struct tcp_ddp_io - tcp ddp configuration for an IO request.
+ *
+ * @command_id:  identifier on the wire associated with these buffers
+ * @nents:       number of entries in the sg_table
+ * @sg_table:    describing the buffers for this IO request
+ * @first_sgl:   first SGL in sg_table
+ */
+struct tcp_ddp_io {
+	u32			command_id;
+	int			nents;
+	struct sg_table		sg_table;
+	struct scatterlist	first_sgl[SG_CHUNK_SIZE];
+};
+
+/* struct tcp_ddp_dev_ops - operations used by an upper layer protocol to configure ddp offload
+ *
+ * @tcp_ddp_limits:    limit the number of scatter gather entries per IO.
+ *                     the device driver can use this to limit the resources allocated per queue.
+ * @tcp_ddp_sk_add:    add offload for the queue represennted by the socket+config pair.
+ *                     this function is used to configure either copy, crc or both offloads.
+ * @tcp_ddp_sk_del:    remove offload from the socket, and release any device related resources.
+ * @tcp_ddp_setup:     request copy offload for buffers associated with a command_id in tcp_ddp_io.
+ * @tcp_ddp_teardown:  release offload resources association between buffers and command_id in
+ *                     tcp_ddp_io.
+ * @tcp_ddp_resync:    respond to the driver's resync_request. Called only if resync is successful.
+ */
+struct tcp_ddp_dev_ops {
+	int (*tcp_ddp_limits)(struct net_device *netdev,
+			      struct tcp_ddp_limits *limits);
+	int (*tcp_ddp_sk_add)(struct net_device *netdev,
+			      struct sock *sk,
+			      struct tcp_ddp_config *config);
+	void (*tcp_ddp_sk_del)(struct net_device *netdev,
+			       struct sock *sk);
+	int (*tcp_ddp_setup)(struct net_device *netdev,
+			     struct sock *sk,
+			     struct tcp_ddp_io *io);
+	int (*tcp_ddp_teardown)(struct net_device *netdev,
+				struct sock *sk,
+				struct tcp_ddp_io *io,
+				void *ddp_ctx);
+	void (*tcp_ddp_resync)(struct net_device *netdev,
+			       struct sock *sk, u32 seq);
+};
+
+#define TCP_DDP_RESYNC_REQ BIT(0)
+
+/**
+ * struct tcp_ddp_ulp_ops - Interface to register uppper layer Direct Data Placement (DDP) TCP offload
+ */
+struct tcp_ddp_ulp_ops {
+	/* NIC requests ulp to indicate if @seq is the start of a message */
+	bool (*resync_request)(struct sock *sk, u32 seq, u32 flags);
+	/* NIC driver informs the ulp that ddp teardown is done - used for async completions*/
+	void (*ddp_teardown_done)(void *ddp_ctx);
+};
+
+/**
+ * struct tcp_ddp_ctx - Generic tcp ddp context: device driver per queue contexts must
+ * use this as the first member.
+ */
+struct tcp_ddp_ctx {
+	enum tcp_ddp_type    type;
+	unsigned char        buf[];
+};
+
+static inline struct tcp_ddp_ctx *tcp_ddp_get_ctx(const struct sock *sk)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+
+	return (__force struct tcp_ddp_ctx *)icsk->icsk_ulp_ddp_data;
+}
+
+static inline void tcp_ddp_set_ctx(struct sock *sk, void *ctx)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+
+	rcu_assign_pointer(icsk->icsk_ulp_ddp_data, ctx);
+}
+
+#endif //_TCP_DDP_H
diff --git a/net/Kconfig b/net/Kconfig
index f4c32d982af6..3876861cdc90 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -457,6 +457,15 @@ config ETHTOOL_NETLINK
 	  netlink. It provides better extensibility and some new features,
 	  e.g. notification messages.
 
+config TCP_DDP
+	bool "TCP direct data placement offload"
+	default n
+	help
+	  Direct Data Placement (DDP) offload for TCP enables ULP, such as
+	  NVMe-TCP/iSCSI, to request the NIC to place TCP payload data
+	  of a command response directly into kernel pages.
+
+
 endif   # if NET
 
 # Used by archs to tell that they support BPF JIT compiler plus which flavour.
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 2af12f7e170c..ef3f2714994d 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -69,6 +69,7 @@
 #include <net/xfrm.h>
 #include <net/mpls.h>
 #include <net/mptcp.h>
+#include <net/tcp_ddp.h>
 
 #include <linux/uaccess.h>
 #include <trace/events/skb.h>
@@ -6185,9 +6186,14 @@ EXPORT_SYMBOL(pskb_extract);
  */
 void skb_condense(struct sk_buff *skb)
 {
+	bool is_ddp = false;
+
+#ifdef CONFIG_TCP_DDP
+	is_ddp = skb->ddp_crc;
+#endif
 	if (skb->data_len) {
 		if (skb->data_len > skb->end - skb->tail ||
-		    skb_cloned(skb))
+		    skb_cloned(skb) || is_ddp)
 			return;
 
 		/* Nice, we can free page frag(s) right now */
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index 181220101a6e..bbf83cd01106 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -69,6 +69,7 @@ const char netdev_features_strings[NETDEV_FEATURE_COUNT][ETH_GSTRING_LEN] = {
 	[NETIF_F_GRO_FRAGLIST_BIT] =	 "rx-gro-list",
 	[NETIF_F_HW_MACSEC_BIT] =	 "macsec-hw-offload",
 	[NETIF_F_GRO_UDP_FWD_BIT] =	 "rx-udp-gro-forwarding",
+	[NETIF_F_HW_TCP_DDP_BIT] =	 "tcp-ddp-offload",
 };
 
 const char
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index a8f8f9815953..0ae1ffca090d 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -5149,6 +5149,9 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 		memcpy(nskb->cb, skb->cb, sizeof(skb->cb));
 #ifdef CONFIG_TLS_DEVICE
 		nskb->decrypted = skb->decrypted;
+#endif
+#ifdef CONFIG_TCP_DDP
+		nskb->ddp_crc = skb->ddp_crc;
 #endif
 		TCP_SKB_CB(nskb)->seq = TCP_SKB_CB(nskb)->end_seq = start;
 		if (list)
@@ -5182,6 +5185,11 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 #ifdef CONFIG_TLS_DEVICE
 				if (skb->decrypted != nskb->decrypted)
 					goto end;
+#endif
+#ifdef CONFIG_TCP_DDP
+
+				if (skb->ddp_crc != nskb->ddp_crc)
+					goto end;
 #endif
 			}
 		}
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 777306b5bc22..b114fc870513 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1810,6 +1810,9 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb)
 	      TCP_SKB_CB(skb)->tcp_flags) & (TCPHDR_ECE | TCPHDR_CWR)) ||
 #ifdef CONFIG_TLS_DEVICE
 	    tail->decrypted != skb->decrypted ||
+#endif
+#ifdef CONFIG_TCP_DDP
+	    tail->ddp_crc != skb->ddp_crc ||
 #endif
 	    thtail->doff != th->doff ||
 	    memcmp(thtail + 1, th + 1, hdrlen - sizeof(*th)))
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index e09147ac9a99..3ce196375d94 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -262,6 +262,9 @@ struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb)
 #ifdef CONFIG_TLS_DEVICE
 	flush |= p->decrypted ^ skb->decrypted;
 #endif
+#ifdef CONFIG_TCP_DDP
+	flush |= p->ddp_crc ^ skb->ddp_crc;
+#endif
 
 	if (flush || skb_gro_receive(p, skb)) {
 		mss = 1;
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:48 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:48 +0200
Subject: [PATCH v3 net-next  00/21] nvme-tcp receive offloads
Message-ID: <20210201100509.27351-1-borisp@mellanox.com>

Changes since v2:
=========================================
* Use skb->ddp_crc for copy offload to avoid skb_condense
* Default mellanox driver support to no (experimental feature)
* In iov_iter use non-ddp functions for kvec and iovec
* Remove typecasting in nvme-tcp

Changes since v1:
=========================================
* Rework iov_iter copy skip if src==dst to be less intrusive (David Ahern)
* Add tcp-ddp documentation (David Ahern)
* Refactor mellanox driver patches into more patches (Saeed Mahameed)
* Avoid pointer casting (David Ahern)
* Rename nvme-tcp offload flags (Shai Malin)
* Update cover-letter according to the above

Changes since RFC v1:
=========================================
* Split mlx5 driver patches to several commits
* Fix nvme-tcp handling of recovery flows. In particular, move queue offlaod
  init/teardown to the start/stop functions.

Overview
=========================================
This series adds support for nvme-tcp receive offloads
which do not mandate the offload of the network stack to the device.
Instead, these work together with TCP to offload:
1. copy from SKB to the block layer buffers
2. CRC verification for received PDU

The series implements these as a generic offload infrastructure for storage
protocols, which we call TCP Direct Data Placement (TCP_DDP) and TCP DDP CRC,
respectively. We use this infrastructure to implement NVMe-TCP offload for copy
and CRC. Future implementations can reuse the same infrastructure for other
protcols such as iSCSI.

Note:
These offloads are similar in nature to the packet-based NIC TLS offloads,
which are already upstream (see net/tls/tls_device.c).
You can read more about TLS offload here:
https://www.kernel.org/doc/html/latest/networking/tls-offload.html

Initialization and teardown:
=========================================
The offload for IO queues is initialized after the handshake of the
NVMe-TCP protocol is finished by calling `nvme_tcp_offload_socket`
with the tcp socket of the nvme_tcp_queue:
This operation sets all relevant hardware contexts in
hardware. If it fails, then the IO queue proceeds as usually with no offload.
If it succeeds then `nvme_tcp_setup_ddp` and `nvme_tcp_teardown_ddp` may be
called to perform copy offload, and crc offload will be used.
This initialization does not change the normal operation of nvme-tcp in any
way besides adding the option to call the above mentioned NDO operations.

For the admin queue, nvme-tcp does not initialize the offload.
Instead, nvme-tcp calls the driver to configure limits for the controller,
such as max_hw_sectors and max_segments; these must be limited to accomodate
potential HW resource limits, and to improve performance.

If some error occured, and the IO queue must be closed or reconnected, then
offload is teardown and initialized again. Additionally, we handle netdev
down events via the existing error recovery flow.

Copy offload works as follows:
=========================================
The nvme-tcp layer calls the NIC drive to map block layer buffers to ccid using
`nvme_tcp_setup_ddp` before sending the read request. When the repsonse is
received, then the NIC HW will write the PDU payload directly into the
designated buffer, and build an SKB such that it points into the destination
buffer; this SKB represents the entire packet received on the wire, but it
points to the block layer buffers. Once nvme-tcp attempts to copy data from
this SKB to the block layer buffer it can skip the copy by checking in the
copying function (memcpy_to_page):
if (src == dst) -> skip copy
Finally, when the PDU has been processed to completion, the nvme-tcp layer
releases the NIC HW context be calling `nvme_tcp_teardown_ddp` which
asynchronously unmaps the buffers from NIC HW.

As the copy skip change is in a sensative function, we are careful to avoid
changing it. To that end, we create alternative skb copy and hash iterators
that skip copy/hash if (src == dst). Nvme-tcp is the first user for these.

Asynchronous completion:
=========================================
The NIC must release its mapping between command IDs and the target buffers.
This mapping is released when NVMe-TCP calls the NIC
driver (`nvme_tcp_offload_socket`).
As completing IOs is performance criticial, we introduce asynchronous
completions for NVMe-TCP, i.e. NVMe-TCP calls the NIC, which will later
call NVMe-TCP to complete the IO (`nvme_tcp_ddp_teardown_done`).

An alternative approach is to move all the functions related to coping from
SKBs to the block layer buffers inside the nvme-tcp code - about 200 LOC.

CRC offload works as follows:
=========================================
After offload is initialized, we use the SKB's ddp_crc bit to indicate that:
"there was no problem with the verification of all CRC fields in this packet's
payload". The bit is set to zero if there was an error, or if HW skipped
offload for some reason. If *any* SKB in a PDU has (ddp_crc != 1), then software
must compute the CRC, and check it. We perform this check, and
accompanying software fallback at the end of the processing of a received PDU.

SKB changes:
=========================================
The CRC offload requires an additional bit in the SKB, which is useful for
preventing the coalescing of SKB with different crc offload values. This bit
is similar in concept to the "decrypted" bit. 

Performance:
=========================================
The expected performance gain from this offload varies with the block size.
We perform a CPU cycles breakdown of the copy/CRC operations in nvme-tcp
fio random read workloads:
For 4K blocks we see up to 11% improvement for a 100% read fio workload,
while for 128K blocks we see upto 52%. If we run nvme-tcp, and skip these
operations, then we observe a gain of about 1.1x and 2x respectively.

Resynchronization:
=========================================
The resynchronization flow is performed to reset the hardware tracking of
NVMe-TCP PDUs within the TCP stream. The flow consists of a request from
the driver, regarding a possible location of a PDU header. Followed by
a response from the nvme-tcp driver.

This flow is rare, and it should happen only after packet loss or
reordering events that involve nvme-tcp PDU headers.

The patches are organized as follows:
=========================================
Patch 1         the iov_iter change to skip copy if (src == dst).
Patches 2-4     the infrastructure for all TCP DDP.
                and TCP DDP CRC offloads, respectively.
Patch 5         exposes the get_netdev_for_sock function from TLS.
Patch 6         NVMe-TCP changes to call NIC driver on queue init/teardown.
Patches 7       NVMe-TCP changes to call NIC driver on IO operation.
                setup/teardown, and support async completions.
Patches 8       NVMe-TCP changes to support CRC offload on receive.
                Also, this patch moves CRC calculation to the end of PDU
                in case offload requires software fallback.
Patches 9       NVMe-TCP handling of netdev events: stop the offload if
                netdev is going down.
Patches 10-20   implement support for NVMe-TCP copy and CRC offload in
                the mlx5 NIC driver as the first user.
Patches 21      Document TCP DDP offload.

Testing:
=========================================
This series was tested using fio with various configurations of IO sizes,
depths, MTUs, and with both the SPDK and kernel NVMe-TCP targets.
Also, we have used QEMU and gate-level simulation to verify these patches.

Future work:
=========================================
A follow-up series will introduce support for transmit side CRC. Then,
we will work on adding support for TLS in NVMe-TCP and combining the
two offloads.

Ben Ben-Ishay (8):
  net/mlx5e: NVMEoTCP offload initialization
  net/mlx5e: KLM UMR helper macros
  net/mlx5e: NVMEoTCP use KLM UMRs
  net/mlx5e: NVMEoTCP queue init/teardown
  net/mlx5e: NVMEoTCP async ddp invalidation
  net/mlx5e: NVMEoTCP ddp setup and resync
  net/mlx5e: NVMEoTCP, data-path for DDP+CRC offload
  net/mlx5e: NVMEoTCP statistics

Ben Ben-ishay (4):
  net: SKB copy(+hash) iterators for DDP offloads
  nvme-tcp : Recalculate crc in the end of the capsule
  net/mlx5: Header file changes for nvme-tcp offload
  net/mlx5: Add 128B CQE for NVMEoTCP offload

Boris Pismenny (8):
  iov_iter: Introduce new procedures for copy to iter/pages
  net: Introduce direct data placement tcp offload
  net: Introduce crc offload for tcp ddp ulp
  net/tls: expose get_netdev_for_sock
  nvme-tcp: Add DDP offload control path
  nvme-tcp: Add DDP data-path
  net/mlx5e: TCP flow steering for nvme-tcp
  Documentation: add TCP DDP offload documentation

Or Gerlitz (1):
  nvme-tcp: Deal with netdevice DOWN events

 Documentation/networking/index.rst            |    1 +
 Documentation/networking/tcp-ddp-offload.rst  |  296 +++++
 .../net/ethernet/mellanox/mlx5/core/Kconfig   |   10 +
 .../net/ethernet/mellanox/mlx5/core/Makefile  |    2 +
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   31 +-
 .../net/ethernet/mellanox/mlx5/core/en/fs.h   |    4 +-
 .../ethernet/mellanox/mlx5/core/en/params.h   |    1 +
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   13 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.c   |    1 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.h   |    1 +
 .../mellanox/mlx5/core/en_accel/en_accel.h    |    9 +-
 .../mellanox/mlx5/core/en_accel/fs_tcp.c      |   10 +
 .../mellanox/mlx5/core/en_accel/fs_tcp.h      |    2 +-
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 1015 +++++++++++++++++
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    |  120 ++
 .../mlx5/core/en_accel/nvmeotcp_rxtx.c        |  264 +++++
 .../mlx5/core/en_accel/nvmeotcp_rxtx.h        |   43 +
 .../mlx5/core/en_accel/nvmeotcp_utils.h       |   80 ++
 .../net/ethernet/mellanox/mlx5/core/en_main.c |   39 +-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |   66 +-
 .../ethernet/mellanox/mlx5/core/en_stats.c    |   37 +
 .../ethernet/mellanox/mlx5/core/en_stats.h    |   24 +
 .../net/ethernet/mellanox/mlx5/core/en_txrx.c |   17 +
 drivers/net/ethernet/mellanox/mlx5/core/fw.c  |    6 +
 drivers/nvme/host/tcp.c                       |  467 +++++++-
 include/linux/mlx5/device.h                   |   44 +-
 include/linux/mlx5/mlx5_ifc.h                 |  101 +-
 include/linux/mlx5/qp.h                       |    1 +
 include/linux/netdev_features.h               |    4 +
 include/linux/netdevice.h                     |    5 +
 include/linux/skbuff.h                        |    9 +
 include/linux/uio.h                           |   12 +
 include/net/inet_connection_sock.h            |    4 +
 include/net/sock.h                            |   17 +
 include/net/tcp_ddp.h                         |  136 +++
 lib/iov_iter.c                                |   47 +
 net/Kconfig                                   |   17 +
 net/core/datagram.c                           |   44 +
 net/core/skbuff.c                             |    8 +-
 net/ethtool/common.c                          |    2 +
 net/ipv4/tcp_input.c                          |    8 +
 net/ipv4/tcp_ipv4.c                           |    3 +
 net/ipv4/tcp_offload.c                        |    3 +
 net/tls/tls_device.c                          |   20 +-
 44 files changed, 2986 insertions(+), 58 deletions(-)
 create mode 100644 Documentation/networking/tcp-ddp-offload.rst
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
 create mode 100644 include/net/tcp_ddp.h

-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:57 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:57 +0200
Subject: [PATCH v3 net-next  09/21] nvme-tcp: Deal with netdevice DOWN events
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-10-borisp@mellanox.com>

From: Or Gerlitz <ogerlitz at mellanox.com>

For ddp setup/teardown and resync, the offloading logic
uses HW resources at the NIC driver such as SQ and CQ.

These resources are destroyed when the netdevice does down
and hence we must stop using them before the NIC driver
destroys them.

Use netdevice notifier for that matter -- offloaded connections
are stopped before the stack continues to call the NIC driver
close ndo.

We use the existing recovery flow which has the advantage
of resuming the offload once the connection is re-set.

This also buys us proper handling for the UNREGISTER event
b/c our offloading starts in the UP state, and down is always
there between up to unregister.

Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index eb47cf6982d7..5f6eed3a9bc5 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -146,6 +146,7 @@ struct nvme_tcp_ctrl {
 
 static LIST_HEAD(nvme_tcp_ctrl_list);
 static DEFINE_MUTEX(nvme_tcp_ctrl_mutex);
+static struct notifier_block nvme_tcp_netdevice_nb;
 static struct workqueue_struct *nvme_tcp_wq;
 static const struct blk_mq_ops nvme_tcp_mq_ops;
 static const struct blk_mq_ops nvme_tcp_admin_mq_ops;
@@ -2930,6 +2931,27 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
 	return ERR_PTR(ret);
 }
 
+static int nvme_tcp_netdev_event(struct notifier_block *this,
+				 unsigned long event, void *ptr)
+{
+	struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
+	struct nvme_tcp_ctrl *ctrl;
+
+	switch (event) {
+	case NETDEV_GOING_DOWN:
+		mutex_lock(&nvme_tcp_ctrl_mutex);
+		list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list) {
+			if (ndev != ctrl->offloading_netdev)
+				continue;
+			nvme_tcp_error_recovery(&ctrl->ctrl);
+		}
+		mutex_unlock(&nvme_tcp_ctrl_mutex);
+		flush_workqueue(nvme_reset_wq);
+		/* we assume that the going down part of error recovery is over */
+	}
+	return NOTIFY_DONE;
+}
+
 static struct nvmf_transport_ops nvme_tcp_transport = {
 	.name		= "tcp",
 	.module		= THIS_MODULE,
@@ -2944,13 +2966,26 @@ static struct nvmf_transport_ops nvme_tcp_transport = {
 
 static int __init nvme_tcp_init_module(void)
 {
+	int ret;
+
 	nvme_tcp_wq = alloc_workqueue("nvme_tcp_wq",
 			WQ_MEM_RECLAIM | WQ_HIGHPRI, 0);
 	if (!nvme_tcp_wq)
 		return -ENOMEM;
 
+	nvme_tcp_netdevice_nb.notifier_call = nvme_tcp_netdev_event;
+	ret = register_netdevice_notifier(&nvme_tcp_netdevice_nb);
+	if (ret) {
+		pr_err("failed to register netdev notifier\n");
+		goto out_err_reg_notifier;
+	}
+
 	nvmf_register_transport(&nvme_tcp_transport);
 	return 0;
+
+out_err_reg_notifier:
+	destroy_workqueue(nvme_tcp_wq);
+	return ret;
 }
 
 static void __exit nvme_tcp_cleanup_module(void)
@@ -2958,6 +2993,7 @@ static void __exit nvme_tcp_cleanup_module(void)
 	struct nvme_tcp_ctrl *ctrl;
 
 	nvmf_unregister_transport(&nvme_tcp_transport);
+	unregister_netdevice_notifier(&nvme_tcp_netdevice_nb);
 
 	mutex_lock(&nvme_tcp_ctrl_mutex);
 	list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list)
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:52 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:52 +0200
Subject: [PATCH v3 net-next 04/21] net: SKB copy(+hash) iterators for DDP
 offloads
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-5-borisp@mellanox.com>

From: Ben Ben-ishay <benishay at nvidia.com>

This commit introduces new functions to support direct data placement
operation, when using direct data placement the copy of the data from
the SKB to the destination buffer might be unnecessary and thus the
copy should be skipped, those functions take care of it in cases that
the destination buffer is represented by bio_vec.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/skbuff.h |  5 +++++
 net/core/datagram.c    | 44 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 49 insertions(+)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 32c5ab8e2f67..79fdcdd1261b 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -3546,6 +3546,8 @@ __poll_t datagram_poll(struct file *file, struct socket *sock,
 			   struct poll_table_struct *wait);
 int skb_copy_datagram_iter(const struct sk_buff *from, int offset,
 			   struct iov_iter *to, int size);
+int skb_ddp_copy_datagram_iter(const struct sk_buff *from, int offset,
+			       struct iov_iter *to, int size);
 static inline int skb_copy_datagram_msg(const struct sk_buff *from, int offset,
 					struct msghdr *msg, int size)
 {
@@ -3556,6 +3558,9 @@ int skb_copy_and_csum_datagram_msg(struct sk_buff *skb, int hlen,
 int skb_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
 			   struct iov_iter *to, int len,
 			   struct ahash_request *hash);
+int skb_ddp_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
+					struct iov_iter *to, int len,
+					struct ahash_request *hash);
 int skb_copy_datagram_from_iter(struct sk_buff *skb, int offset,
 				 struct iov_iter *from, int len);
 int zerocopy_sg_from_iter(struct sk_buff *skb, struct iov_iter *frm);
diff --git a/net/core/datagram.c b/net/core/datagram.c
index 81809fa735a7..bbc476cadc71 100644
--- a/net/core/datagram.c
+++ b/net/core/datagram.c
@@ -495,6 +495,25 @@ static int __skb_datagram_iter(const struct sk_buff *skb, int offset,
 	return 0;
 }
 
+/**
+ *	skb_ddp_copy_and_hash_datagram_iter - Copies datagrams from skb frags to
+ *	an iterator and update a hash. If the iterator and skb frag point to the
+ *	same page and offset, then the copy is skipped.
+ *	@skb: buffer to copy
+ *	@offset: offset in the buffer to start copying from
+ *	@to: iovec iterator to copy to
+ *	@len: amount of data to copy from buffer to iovec
+ *      @hash: hash request to update
+ */
+int skb_ddp_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
+					struct iov_iter *to, int len,
+					struct ahash_request *hash)
+{
+	return __skb_datagram_iter(skb, offset, to, len, true,
+			ddp_hash_and_copy_to_iter, hash);
+}
+EXPORT_SYMBOL(skb_ddp_copy_and_hash_datagram_iter);
+
 /**
  *	skb_copy_and_hash_datagram_iter - Copy datagram to an iovec iterator
  *          and update a hash.
@@ -513,6 +532,31 @@ int skb_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
 }
 EXPORT_SYMBOL(skb_copy_and_hash_datagram_iter);
 
+static size_t simple_ddp_copy_to_iter(const void *addr, size_t bytes,
+				      void *data __always_unused,
+				      struct iov_iter *i)
+{
+	return ddp_copy_to_iter(addr, bytes, i);
+}
+
+/**
+ *	skb_ddp_copy_datagram_iter - Copies datagrams from skb frags to an
+ *	iterator. If the iterator and skb frag point to the same page and
+ *	offset, then the copy is skipped.
+ *	@skb: buffer to copy
+ *	@offset: offset in the buffer to start copying from
+ *	@to: iovec iterator to copy to
+ *	@len: amount of data to copy from buffer to iovec
+ */
+int skb_ddp_copy_datagram_iter(const struct sk_buff *skb, int offset,
+			       struct iov_iter *to, int len)
+{
+	trace_skb_copy_datagram_iovec(skb, len);
+	return __skb_datagram_iter(skb, offset, to, len, false,
+			simple_ddp_copy_to_iter, NULL);
+}
+EXPORT_SYMBOL(skb_ddp_copy_datagram_iter);
+
 static size_t simple_copy_to_iter(const void *addr, size_t bytes,
 		void *data __always_unused, struct iov_iter *i)
 {
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:05 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:05 +0200
Subject: [PATCH v3 net-next 17/21] net/mlx5e: NVMEoTCP async ddp invalidation
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-18-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

Teardown ddp contexts asynchronously by posting a WQE, and calling back
to nvme-tcp when the corresponding CQE is received.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |  4 ++
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 66 ++++++++++++++++---
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    |  1 +
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  6 ++
 4 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
index 32fa9f1a4a1f..e23d165ba264 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
@@ -36,6 +36,7 @@ enum mlx5e_icosq_wqe_type {
 #endif
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 	MLX5E_ICOSQ_WQE_UMR_NVME_TCP,
+	MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE,
 	MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP,
 #endif
 };
@@ -184,6 +185,9 @@ struct mlx5e_icosq_wqe_info {
 		struct {
 			struct mlx5e_nvmeotcp_queue *queue;
 		} nvmeotcp_q;
+		struct {
+			struct nvmeotcp_queue_entry *entry;
+		} nvmeotcp_qe;
 #endif
 	};
 };
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index 91125022c0f6..f30bfcb43701 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -152,6 +152,7 @@ enum wqe_type {
 	BSF_KLM_UMR = 1,
 	SET_PSV_UMR = 2,
 	BSF_UMR = 3,
+	KLM_INV_UMR = 4,
 };
 
 static void
@@ -208,6 +209,13 @@ build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 				   MLX5E_KLM_UMR_DS_CNT(ALIGN(klm_entries, KLM_ALIGNMENT)));
 	cseg->general_id = cpu_to_be32(id);
 
+	if (!klm_entries) { /* this is invalidate */
+		ucseg->mkey_mask = cpu_to_be64(MLX5_MKEY_MASK_FREE);
+		ucseg->flags = MLX5_UMR_INLINE;
+		mkc->status = MLX5_MKEY_STATUS_FREE;
+		return;
+	}
+
 	if (klm_type == KLM_UMR && !klm_offset) {
 		ucseg->mkey_mask |= cpu_to_be64(MLX5_MKEY_MASK_XLT_OCT_SIZE |
 						MLX5_MKEY_MASK_LEN | MLX5_MKEY_MASK_FREE);
@@ -308,8 +316,8 @@ build_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
 
 static void
 mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
-		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi,
-		       enum wqe_type type)
+		       struct mlx5e_icosq *sq, u32 wqe_bbs,
+		       u16 pi, u16 ccid, enum wqe_type type)
 {
 	struct mlx5e_icosq_wqe_info *wi = &sq->db.wqe_info[pi];
 
@@ -318,12 +326,17 @@ mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
 	case SET_PSV_UMR:
 		wi->wqe_type = MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP;
 		break;
+	case KLM_INV_UMR:
+		wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE;
+		break;
 	default:
 		wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
 		break;
 	}
 
-	if (type == SET_PSV_UMR)
+	if (type == KLM_INV_UMR)
+		wi->nvmeotcp_qe.entry = &nvmeotcp_queue->ccid_table[ccid];
+	else if (type == SET_PSV_UMR)
 		wi->nvmeotcp_q.queue = nvmeotcp_queue;
 }
 
@@ -338,7 +351,7 @@ mlx5e_nvmeotcp_rx_post_static_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS;
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_STATIC_PARAMS_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(NULL, sq, wqe_bbs, pi, BSF_UMR);
+	mlx5e_nvmeotcp_fill_wi(NULL, sq, wqe_bbs, pi, 0, BSF_UMR);
 	build_nvmeotcp_static_params(queue, wqe, resync_seq, queue->zerocopy, queue->crc_rx);
 	sq->pc += wqe_bbs;
 	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
@@ -355,7 +368,7 @@ mlx5e_nvmeotcp_rx_post_progress_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQEBBS;
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_PROGRESS_PARAMS_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, SET_PSV_UMR);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, 0, SET_PSV_UMR);
 	build_nvmeotcp_progress_params(queue, wqe, seq);
 	sq->pc += wqe_bbs;
 	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
@@ -379,7 +392,8 @@ post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, wqe_type);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, ccid,
+			       klm_length ? KLM_UMR : KLM_INV_UMR);
 	build_nvmeotcp_klm_umr(queue, wqe, ccid, cur_klm_entries, *klm_offset,
 			       klm_length, wqe_type);
 	*klm_offset += cur_klm_entries;
@@ -397,8 +411,13 @@ mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	struct mlx5e_icosq *sq = &queue->sq->icosq;
 
 	/* TODO: set stricter wqe_sz; using max for now */
-	wqes = DIV_ROUND_UP(klm_length, queue->max_klms_per_wqe);
-	wqe_sz = MLX5E_KLM_UMR_WQE_SZ(queue->max_klms_per_wqe);
+	if (klm_length == 0) {
+		wqes = 1;
+		wqe_sz = MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS;
+	} else {
+		wqes = DIV_ROUND_UP(klm_length, queue->max_klms_per_wqe);
+		wqe_sz = MLX5E_KLM_UMR_WQE_SZ(queue->max_klms_per_wqe);
+	}
 
 	max_wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
 
@@ -738,6 +757,24 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	return 0;
 }
 
+void mlx5e_nvmeotcp_ddp_inv_done(struct mlx5e_icosq_wqe_info *wi)
+{
+	struct nvmeotcp_queue_entry *q_entry = wi->nvmeotcp_qe.entry;
+	struct mlx5e_nvmeotcp_queue *queue = q_entry->queue;
+	struct mlx5_core_dev *mdev = queue->priv->mdev;
+	struct tcp_ddp_io *ddp = q_entry->ddp;
+	const struct tcp_ddp_ulp_ops *ulp_ops;
+
+	dma_unmap_sg(mdev->device, ddp->sg_table.sgl,
+		     q_entry->sgl_length, DMA_FROM_DEVICE);
+
+	q_entry->sgl_length = 0;
+
+	ulp_ops = inet_csk(queue->sk)->icsk_ulp_ddp_ops;
+	if (ulp_ops && ulp_ops->ddp_teardown_done)
+		ulp_ops->ddp_teardown_done(q_entry->ddp_ctx);
+}
+
 void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi)
 {
 	struct mlx5e_nvmeotcp_queue *queue = wi->nvmeotcp_q.queue;
@@ -754,6 +791,19 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 			    struct tcp_ddp_io *ddp,
 			    void *ddp_ctx)
 {
+	struct mlx5e_nvmeotcp_queue *queue =
+		(struct mlx5e_nvmeotcp_queue *)tcp_ddp_get_ctx(sk);
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct nvmeotcp_queue_entry *q_entry;
+
+	q_entry  = &queue->ccid_table[ddp->command_id];
+	WARN_ON(q_entry->sgl_length == 0);
+
+	q_entry->ddp_ctx = ddp_ctx;
+	q_entry->queue = queue;
+
+	mlx5e_nvmeotcp_post_klm_wqe(queue, KLM_UMR, ddp->command_id, 0);
+
 	return 0;
 }
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
index d0e515502d6d..5a613addfc7c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
@@ -103,6 +103,7 @@ void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv);
 struct mlx5e_nvmeotcp_queue *
 mlx5e_nvmeotcp_get_queue(struct mlx5e_nvmeotcp *nvmeotcp, int id);
 void mlx5e_nvmeotcp_put_queue(struct mlx5e_nvmeotcp_queue *queue);
+void mlx5e_nvmeotcp_ddp_inv_done(struct mlx5e_icosq_wqe_info *wi);
 void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi);
 int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv);
 void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index c08d8cfdaebe..7e044a211aa1 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -630,6 +630,9 @@ void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 			break;
+		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE:
+			mlx5e_nvmeotcp_ddp_inv_done(wi);
+			break;
 		case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
 			mlx5e_nvmeotcp_ctx_comp(wi);
 			break;
@@ -707,6 +710,9 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 				break;
+			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE:
+				mlx5e_nvmeotcp_ddp_inv_done(wi);
+				break;
 			case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
 				mlx5e_nvmeotcp_ctx_comp(wi);
 				break;
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:55 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:55 +0200
Subject: [PATCH v3 net-next  07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-8-borisp@mellanox.com>

Introduce the NVMe-TCP DDP data-path offload.
Using this interface, the NIC hardware will scatter TCP payload directly
to the BIO pages according to the command_id in the PDU.
To maintain the correctness of the network stack, the driver is expected
to construct SKBs that point to the BIO pages.

The data-path interface contains two routines: tcp_ddp_setup/teardown.
The setup provides the mapping from command_id to the request buffers,
while the teardown removes this mapping.

For efficiency, we introduce an asynchronous nvme completion, which is
split between NVMe-TCP and the NIC driver as follows:
NVMe-TCP performs the specific completion, while NIC driver performs the
generic mq_blk completion.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 141 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 131 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ea67caf9d326..5cb46deb56e0 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -57,6 +57,11 @@ struct nvme_tcp_request {
 	size_t			offset;
 	size_t			data_sent;
 	enum nvme_tcp_send_state state;
+
+	bool			offloaded;
+	struct tcp_ddp_io	ddp;
+	__le16			status;
+	union nvme_result	result;
 };
 
 enum nvme_tcp_queue_flags {
@@ -232,10 +237,74 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
 #ifdef CONFIG_TCP_DDP
 
 static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
+static void nvme_tcp_ddp_teardown_done(void *ddp_ctx);
 static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
 	.resync_request		= nvme_tcp_resync_request,
+	.ddp_teardown_done	= nvme_tcp_ddp_teardown_done,
 };
 
+static
+int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
+			  u16 command_id,
+			  struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+	int ret;
+
+	if (unlikely(!netdev)) {
+		pr_info_ratelimited("%s: netdev not found\n", __func__);
+		return -EINVAL;
+	}
+
+	ret = netdev->tcp_ddp_ops->tcp_ddp_teardown(netdev, queue->sock->sk,
+						    &req->ddp, rq);
+	sg_free_table_chained(&req->ddp.sg_table, SG_CHUNK_SIZE);
+	req->offloaded = false;
+	return ret;
+}
+
+static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
+{
+	struct request *rq = ddp_ctx;
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+
+	if (!nvme_try_complete_req(rq, cpu_to_le16(req->status << 1), req->result))
+		nvme_complete_rq(rq);
+}
+
+static
+int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
+		       u16 command_id,
+		       struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+	int ret;
+
+	req->offloaded = false;
+
+	if (unlikely(!netdev)) {
+		pr_info_ratelimited("%s: netdev not found\n", __func__);
+		return -EINVAL;
+	}
+
+	req->ddp.command_id = command_id;
+	req->ddp.sg_table.sgl = req->ddp.first_sgl;
+	ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
+				     req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
+	if (ret)
+		return -ENOMEM;
+	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
+
+	ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
+						 queue->sock->sk,
+						 &req->ddp);
+	if (!ret)
+		req->offloaded = true;
+	return ret;
+}
+
 static
 int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 {
@@ -377,6 +446,25 @@ bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
 
 #else
 
+static
+int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
+		       u16 command_id,
+		       struct request *rq)
+{
+	return -EINVAL;
+}
+
+static
+int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
+			  u16 command_id,
+			  struct request *rq)
+{
+	return -EINVAL;
+}
+
+static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
+{}
+
 static
 int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 {
@@ -665,6 +753,7 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
 static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		struct nvme_completion *cqe)
 {
+	struct nvme_tcp_request *req;
 	struct request *rq;
 
 	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), cqe->command_id);
@@ -676,8 +765,15 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		return -EINVAL;
 	}
 
-	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
-		nvme_complete_rq(rq);
+	req = blk_mq_rq_to_pdu(rq);
+	if (req->offloaded) {
+		req->status = cqe->status;
+		req->result = cqe->result;
+		nvme_tcp_teardown_ddp(queue, cqe->command_id, rq);
+	} else {
+		if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
+			nvme_complete_rq(rq);
+	}
 	queue->nr_cqe++;
 
 	return 0;
@@ -871,9 +967,18 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 static inline void nvme_tcp_end_request(struct request *rq, u16 status)
 {
 	union nvme_result res = {};
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+	struct nvme_tcp_queue *queue = req->queue;
+	struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
 
-	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
-		nvme_complete_rq(rq);
+	if (req->offloaded) {
+		req->status = cpu_to_le16(status << 1);
+		req->result = res;
+		nvme_tcp_teardown_ddp(queue, pdu->command_id, rq);
+	} else {
+		if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
+			nvme_complete_rq(rq);
+	}
 }
 
 static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
@@ -920,12 +1025,22 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 		recv_len = min_t(size_t, recv_len,
 				iov_iter_count(&req->iter));
 
-		if (queue->data_digest)
-			ret = skb_copy_and_hash_datagram_iter(skb, *offset,
-				&req->iter, recv_len, queue->rcv_hash);
-		else
-			ret = skb_copy_datagram_iter(skb, *offset,
-					&req->iter, recv_len);
+		if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
+			if (queue->data_digest)
+				ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset,
+						&req->iter, recv_len, queue->rcv_hash);
+			else
+				ret = skb_ddp_copy_datagram_iter(skb, *offset,
+						&req->iter, recv_len);
+		} else {
+			if (queue->data_digest)
+				ret = skb_copy_and_hash_datagram_iter(skb, *offset,
+						&req->iter, recv_len, queue->rcv_hash);
+			else
+				ret = skb_copy_datagram_iter(skb, *offset,
+						&req->iter, recv_len);
+		}
+
 		if (ret) {
 			dev_err(queue->ctrl->ctrl.device,
 				"queue %d failed to copy request %#x data",
@@ -1149,6 +1264,7 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
 	bool inline_data = nvme_tcp_has_inline_data(req);
 	u8 hdgst = nvme_tcp_hdgst_len(queue);
 	int len = sizeof(*pdu) + hdgst - req->offset;
+	struct request *rq = blk_mq_rq_from_pdu(req);
 	int flags = MSG_DONTWAIT;
 	int ret;
 
@@ -1157,6 +1273,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
 	else
 		flags |= MSG_EOR;
 
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) &&
+	    blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
+		nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);
+
 	if (queue->hdr_digest && !req->offset)
 		nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu));
 
@@ -2464,6 +2584,7 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns,
 	req->data_len = blk_rq_nr_phys_segments(rq) ?
 				blk_rq_payload_bytes(rq) : 0;
 	req->curr_bio = rq->bio;
+	req->offloaded = false;
 
 	if (rq_data_dir(rq) == WRITE &&
 	    req->data_len <= nvme_tcp_inline_data_size(queue))
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:02 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:02 +0200
Subject: [PATCH v3 net-next  14/21] net/mlx5e: KLM UMR helper macros
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-15-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

Add helper macros for posting KLM UMR WQE.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 3003bae2c300..13a0d4d5e9f6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -153,6 +153,24 @@ struct page_pool;
 #define MLX5E_UMR_WQEBBS \
 	(DIV_ROUND_UP(MLX5E_UMR_WQE_INLINE_SZ, MLX5_SEND_WQE_BB))
 
+#define KLM_ALIGNMENT 4
+#define MLX5E_KLM_UMR_WQE_SZ(sgl_len)\
+	(sizeof(struct mlx5e_umr_wqe) +\
+	(sizeof(struct mlx5_klm) * (sgl_len)))
+
+#define MLX5E_KLM_UMR_WQEBBS(sgl_len)\
+	(DIV_ROUND_UP(MLX5E_KLM_UMR_WQE_SZ(sgl_len), MLX5_SEND_WQE_BB))
+
+#define MLX5E_KLM_UMR_DS_CNT(sgl_len)\
+	DIV_ROUND_UP(MLX5E_KLM_UMR_WQE_SZ(sgl_len), MLX5_SEND_WQE_DS)
+
+#define MLX5E_MAX_KLM_ENTRIES_PER_WQE(wqe_size)\
+	(((wqe_size) - sizeof(struct mlx5e_umr_wqe)) / sizeof(struct mlx5_klm))
+
+#define MLX5E_KLM_ENTRIES_PER_WQE(wqe_size)\
+	(MLX5E_MAX_KLM_ENTRIES_PER_WQE(wqe_size) -\
+			(MLX5E_MAX_KLM_ENTRIES_PER_WQE(wqe_size) % KLM_ALIGNMENT))
+
 #define MLX5E_MSG_LEVEL			NETIF_MSG_LINK
 
 #define mlx5e_dbg(mlevel, priv, format, ...)                    \
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:59 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:59 +0200
Subject: [PATCH v3 net-next 11/21] net/mlx5: Add 128B CQE for NVMEoTCP offload
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-12-borisp@mellanox.com>

From: Ben Ben-ishay <benishay at nvidia.com>

Add the NVMEoTCP offload definition and access functions for 128B CQEs.

Signed-off-by: Ben Ben-ishay <benishay at nvidia.com>
Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/mlx5/device.h | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index ab04959188b9..f6548c255290 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -791,7 +791,7 @@ struct mlx5_err_cqe {
 
 struct mlx5_cqe64 {
 	u8		tls_outer_l3_tunneled;
-	u8		rsvd0;
+	u8		nvmetcp;
 	__be16		wqe_id;
 	u8		lro_tcppsh_abort_dupack;
 	u8		lro_min_ttl;
@@ -824,6 +824,19 @@ struct mlx5_cqe64 {
 	u8		op_own;
 };
 
+struct mlx5e_cqe128 {
+	__be16		cclen;
+	__be16		hlen;
+	union {
+		__be32		resync_tcp_sn;
+		__be32		ccoff;
+	};
+	__be16		ccid;
+	__be16		rsvd8;
+	u8		rsvd12[52];
+	struct mlx5_cqe64 cqe64;
+};
+
 struct mlx5_mini_cqe8 {
 	union {
 		__be32 rx_hash_result;
@@ -854,6 +867,27 @@ enum {
 
 #define MLX5_MINI_CQE_ARRAY_SIZE 8
 
+static inline bool cqe_is_nvmeotcp_resync(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 6) & 0x1);
+}
+
+static inline bool cqe_is_nvmeotcp_crcvalid(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 5) & 0x1);
+}
+
+static inline bool cqe_is_nvmeotcp_zc(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 4) & 0x1);
+}
+
+/* check if cqe is zc or crc or resync */
+static inline bool cqe_is_nvmeotcp(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 4) & 0x7);
+}
+
 static inline u8 mlx5_get_cqe_format(struct mlx5_cqe64 *cqe)
 {
 	return (cqe->op_own >> 2) & 0x3;
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:49 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:49 +0200
Subject: [PATCH v3 net-next 01/21] iov_iter: Introduce new procedures for copy
 to iter/pages
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-2-borisp@mellanox.com>

When using direct data placement the NIC writes some of the payload
directly to the destination buffer, and constructs the SKB such that it
points to this data. As a result, the skb_copy datagram_iter call will
attempt to copy data when it is not necessary.

Introduce new procedures for copy to iter/pages in case that the
source of the copy operation might be identical to the destination,
in such cases the copy is skipped only for bio_vec, later commits
uses those functions to introduce new skb copy(+hash) functions.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/uio.h | 12 ++++++++++++
 lib/iov_iter.c      | 47 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)

diff --git a/include/linux/uio.h b/include/linux/uio.h
index 72d88566694e..3c42125a7f24 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -123,6 +123,7 @@ size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes,
 			 struct iov_iter *i);
 
 size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i);
+size_t _ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i);
 size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i);
 bool _copy_from_iter_full(void *addr, size_t bytes, struct iov_iter *i);
 size_t _copy_from_iter_nocache(void *addr, size_t bytes, struct iov_iter *i);
@@ -137,6 +138,15 @@ size_t copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
 		return _copy_to_iter(addr, bytes, i);
 }
 
+static __always_inline __must_check
+size_t ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
+{
+	if (unlikely(!check_copy_size(addr, bytes, true)))
+		return 0;
+	else
+		return _ddp_copy_to_iter(addr, bytes, i);
+}
+
 static __always_inline __must_check
 size_t copy_from_iter(void *addr, size_t bytes, struct iov_iter *i)
 {
@@ -265,6 +275,8 @@ size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, struct io
 bool csum_and_copy_from_iter_full(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i);
 size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
 		struct iov_iter *i);
+size_t ddp_hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
+		struct iov_iter *i);
 
 struct iovec *iovec_from_user(const struct iovec __user *uvector,
 		unsigned long nr_segs, unsigned long fast_segs,
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index a21e6a5792c5..b8af1d3bbec0 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -473,6 +473,16 @@ static void memcpy_from_page(char *to, struct page *page, size_t offset, size_t
 	kunmap_atomic(from);
 }
 
+static void ddp_memcpy_to_page(struct page *page, size_t offset, const char *from, size_t len)
+{
+	char *to = kmap_atomic(page);
+
+	if (to + offset != from)
+		memcpy(to + offset, from, len);
+
+	kunmap_atomic(to);
+}
+
 static void memcpy_to_page(struct page *page, size_t offset, const char *from, size_t len)
 {
 	char *to = kmap_atomic(page);
@@ -625,6 +635,24 @@ static size_t csum_and_copy_to_pipe_iter(const void *addr, size_t bytes,
 	return bytes;
 }
 
+size_t _ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
+{
+	const char *from = addr;
+	if (unlikely(iov_iter_is_pipe(i)))
+		return copy_pipe_to_iter(addr, bytes, i);
+	if (iter_is_iovec(i))
+		might_fault();
+	iterate_and_advance(i, bytes, v,
+		copyout(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
+		ddp_memcpy_to_page(v.bv_page, v.bv_offset,
+				   (from += v.bv_len) - v.bv_len, v.bv_len),
+		memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len)
+		)
+
+	return bytes;
+}
+EXPORT_SYMBOL(_ddp_copy_to_iter);
+
 size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
 {
 	const char *from = addr;
@@ -1566,6 +1594,25 @@ size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *csump,
 }
 EXPORT_SYMBOL(csum_and_copy_to_iter);
 
+size_t ddp_hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
+		struct iov_iter *i)
+{
+#ifdef CONFIG_CRYPTO_HASH
+	struct ahash_request *hash = hashp;
+	struct scatterlist sg;
+	size_t copied;
+
+	copied = ddp_copy_to_iter(addr, bytes, i);
+	sg_init_one(&sg, addr, copied);
+	ahash_request_set_crypt(hash, &sg, NULL, copied);
+	crypto_ahash_update(hash);
+	return copied;
+#else
+	return 0;
+#endif
+}
+EXPORT_SYMBOL(ddp_hash_and_copy_to_iter);
+
 size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
 		struct iov_iter *i)
 {
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:53 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:53 +0200
Subject: [PATCH v3 net-next  05/21] net/tls: expose get_netdev_for_sock
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-6-borisp@mellanox.com>

get_netdev_for_sock is a utility that is used to obtain
the net_device structure from a connected socket.

Later patches will use this for nvme-tcp DDP and DDP CRC offloads.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
---
 include/net/sock.h   | 17 +++++++++++++++++
 net/tls/tls_device.c | 20 ++------------------
 2 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index 129d200bccb4..3f9ae5615daa 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2728,4 +2728,21 @@ void sock_set_sndtimeo(struct sock *sk, s64 secs);
 
 int sock_bind_add(struct sock *sk, struct sockaddr *addr, int addr_len);
 
+/* Assume that the socket is already connected */
+static inline struct net_device *get_netdev_for_sock(struct sock *sk, bool hold)
+{
+	struct dst_entry *dst = sk_dst_get(sk);
+	struct net_device *netdev = NULL;
+
+	if (likely(dst)) {
+		netdev = dst->dev;
+		if (hold)
+			dev_hold(netdev);
+	}
+
+	dst_release(dst);
+
+	return netdev;
+}
+
 #endif	/* _SOCK_H */
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index d9cd229aa111..792c0a477850 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -106,22 +106,6 @@ static void tls_device_queue_ctx_destruction(struct tls_context *ctx)
 	spin_unlock_irqrestore(&tls_device_lock, flags);
 }
 
-/* We assume that the socket is already connected */
-static struct net_device *get_netdev_for_sock(struct sock *sk)
-{
-	struct dst_entry *dst = sk_dst_get(sk);
-	struct net_device *netdev = NULL;
-
-	if (likely(dst)) {
-		netdev = netdev_sk_get_lowest_dev(dst->dev, sk);
-		dev_hold(netdev);
-	}
-
-	dst_release(dst);
-
-	return netdev;
-}
-
 static void destroy_record(struct tls_record_info *record)
 {
 	int i;
@@ -1106,7 +1090,7 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
 	if (skb)
 		TCP_SKB_CB(skb)->eor = 1;
 
-	netdev = get_netdev_for_sock(sk);
+	netdev = get_netdev_for_sock(sk, true);
 	if (!netdev) {
 		pr_err_ratelimited("%s: netdev not found\n", __func__);
 		rc = -EINVAL;
@@ -1182,7 +1166,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
 	if (ctx->crypto_recv.info.version != TLS_1_2_VERSION)
 		return -EOPNOTSUPP;
 
-	netdev = get_netdev_for_sock(sk);
+	netdev = get_netdev_for_sock(sk, true);
 	if (!netdev) {
 		pr_err_ratelimited("%s: netdev not found\n", __func__);
 		return -EINVAL;
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:04 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:04 +0200
Subject: [PATCH v3 net-next  16/21] net/mlx5e: NVMEoTCP queue init/teardown
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-17-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

When nvme-tcp establishes new connections, we allocate a hardware
context to offload operations for this queue:
- Use a separate TIR to identify the queue and maintain the HW context
- Use a separate ICOSQ for maintain the HW context
- Use a separate tag buffer for buffer registration
- Maintain static and progress HW contexts by posting the proper
  WQEs at creation time, or upon resync

Queue teardown will free the corresponding contexts.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   6 +
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 653 +++++++++++++++++-
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    |   4 +
 .../mlx5/core/en_accel/nvmeotcp_utils.h       |  68 ++
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |   7 +
 5 files changed, 714 insertions(+), 24 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
index e997b7230028..32fa9f1a4a1f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
@@ -36,6 +36,7 @@ enum mlx5e_icosq_wqe_type {
 #endif
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 	MLX5E_ICOSQ_WQE_UMR_NVME_TCP,
+	MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP,
 #endif
 };
 
@@ -178,6 +179,11 @@ struct mlx5e_icosq_wqe_info {
 		struct {
 			struct mlx5e_ktls_rx_resync_buf *buf;
 		} tls_get_params;
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+		struct {
+			struct mlx5e_nvmeotcp_queue *queue;
+		} nvmeotcp_q;
 #endif
 	};
 };
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index bbeded08ef01..91125022c0f6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -3,6 +3,7 @@
 
 #include <linux/netdevice.h>
 #include <linux/idr.h>
+#include <linux/nvme-tcp.h>
 #include "en_accel/nvmeotcp.h"
 #include "en_accel/nvmeotcp_utils.h"
 #include "en_accel/fs_tcp.h"
@@ -20,35 +21,180 @@ static const struct rhashtable_params rhash_queues = {
 	.max_size = MAX_NVMEOTCP_QUEUES,
 };
 
+#define MLX5_NVME_TCP_MAX_SEGMENTS 128
+
+static u32 mlx5e_get_max_sgl(struct mlx5_core_dev *mdev)
+{
+	return min_t(u32,
+		     MLX5_NVME_TCP_MAX_SEGMENTS,
+		     1 << MLX5_CAP_GEN(mdev, log_max_klm_list_size));
+}
+
+static void mlx5e_nvmeotcp_destroy_tir(struct mlx5e_priv *priv, int tirn)
+{
+	mlx5_core_destroy_tir(priv->mdev, tirn);
+}
+
+static inline u32
+mlx5e_get_channel_ix_from_io_cpu(struct mlx5e_priv *priv, u32 io_cpu)
+{
+	int num_channels = priv->channels.params.num_channels;
+	u32 channel_ix = io_cpu;
+
+	if (channel_ix >= num_channels)
+		channel_ix = channel_ix % num_channels;
+
+	return channel_ix;
+}
+
+static int mlx5e_nvmeotcp_create_tir(struct mlx5e_priv *priv,
+				     struct sock *sk,
+				     struct nvme_tcp_ddp_config *config,
+				     struct mlx5e_nvmeotcp_queue *queue,
+				     bool zerocopy, bool crc_rx)
+{
+	u32 rqtn = priv->direct_tir[queue->channel_ix].rqt.rqtn;
+	int err, inlen;
+	void *tirc;
+	u32 tirn;
+	u32 *in;
+
+	inlen = MLX5_ST_SZ_BYTES(create_tir_in);
+	in = kvzalloc(inlen, GFP_KERNEL);
+	if (!in)
+		return -ENOMEM;
+	tirc = MLX5_ADDR_OF(create_tir_in, in, ctx);
+	MLX5_SET(tirc, tirc, disp_type, MLX5_TIRC_DISP_TYPE_INDIRECT);
+	MLX5_SET(tirc, tirc, rx_hash_fn, MLX5_RX_HASH_FN_INVERTED_XOR8);
+	MLX5_SET(tirc, tirc, indirect_table, rqtn);
+	MLX5_SET(tirc, tirc, transport_domain, priv->mdev->mlx5e_res.td.tdn);
+	if (zerocopy) {
+		MLX5_SET(tirc, tirc, nvmeotcp_zero_copy_en, 1);
+		MLX5_SET(tirc, tirc, nvmeotcp_tag_buffer_table_id,
+			 queue->tag_buf_table_id);
+	}
+
+	if (crc_rx)
+		MLX5_SET(tirc, tirc, nvmeotcp_crc_en, 1);
+
+	MLX5_SET(tirc, tirc, self_lb_block,
+		 MLX5_TIRC_SELF_LB_BLOCK_BLOCK_UNICAST |
+		 MLX5_TIRC_SELF_LB_BLOCK_BLOCK_MULTICAST);
+	err = mlx5_core_create_tir(priv->mdev, in, &tirn);
+
+	if (!err)
+		queue->tirn = tirn;
+
+	kvfree(in);
+	return err;
+}
+
+static
+int mlx5e_create_nvmeotcp_tag_buf_table(struct mlx5_core_dev *mdev,
+					struct mlx5e_nvmeotcp_queue *queue,
+					u8 log_table_size)
+{
+	u32 in[MLX5_ST_SZ_DW(create_nvmeotcp_tag_buf_table_in)] = {};
+	u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)];
+	u64 general_obj_types;
+	void *obj;
+	int err;
+
+	obj = MLX5_ADDR_OF(create_nvmeotcp_tag_buf_table_in, in,
+			   nvmeotcp_tag_buf_table_obj);
+
+	general_obj_types = MLX5_CAP_GEN_64(mdev, general_obj_types);
+	if (!(general_obj_types &
+	      MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE))
+		return -EINVAL;
+
+	MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+		 MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+	MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
+		 MLX5_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE);
+	MLX5_SET(nvmeotcp_tag_buf_table_obj, obj,
+		 log_tag_buffer_table_size, log_table_size);
+
+	err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+	if (!err)
+		queue->tag_buf_table_id = MLX5_GET(general_obj_out_cmd_hdr,
+						   out, obj_id);
+	return err;
+}
+
+static
+void mlx5_destroy_nvmeotcp_tag_buf_table(struct mlx5_core_dev *mdev, u32 uid)
+{
+	u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {};
+	u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)];
+
+	MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+		 MLX5_CMD_OP_DESTROY_GENERAL_OBJECT);
+	MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
+		 MLX5_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE);
+	MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, uid);
+
+	mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_TIR_PARAMS 0x2
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_STATIC_PARAMS 0x2
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_UMR 0x0
+
+#define STATIC_PARAMS_DS_CNT \
+	DIV_ROUND_UP(MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ, MLX5_SEND_WQE_DS)
+
+#define PROGRESS_PARAMS_DS_CNT \
+	DIV_ROUND_UP(MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ, MLX5_SEND_WQE_DS)
+
+enum wqe_type {
+	KLM_UMR = 0,
+	BSF_KLM_UMR = 1,
+	SET_PSV_UMR = 2,
+	BSF_UMR = 3,
+};
+
 static void
 fill_nvmeotcp_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 		      struct mlx5e_umr_wqe *wqe, u16 ccid, u32 klm_entries,
-		      u16 klm_offset)
+		      u16 klm_offset, enum wqe_type klm_type)
 {
 	struct scatterlist *sgl_mkey;
 	u32 lkey, i;
 
-	lkey = queue->priv->mdev->mlx5e_res.mkey.key;
-	for (i = 0; i < klm_entries; i++) {
-		sgl_mkey = &queue->ccid_table[ccid].sgl[i + klm_offset];
-		wqe->inline_klms[i].bcount = cpu_to_be32(sgl_mkey->length);
-		wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
-		wqe->inline_klms[i].va	   = cpu_to_be64(sgl_mkey->dma_address);
-	}
-
-	for (; i < ALIGN(klm_entries, KLM_ALIGNMENT); i++) {
-		wqe->inline_klms[i].bcount = 0;
-		wqe->inline_klms[i].key    = 0;
-		wqe->inline_klms[i].va     = 0;
+	if (klm_type == BSF_KLM_UMR) {
+		for (i = 0; i < klm_entries; i++) {
+			lkey = queue->ccid_table[i + klm_offset].klm_mkey.key;
+			wqe->inline_klms[i].bcount = cpu_to_be32(1);
+			wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
+			wqe->inline_klms[i].va	   = 0;
+		}
+	} else {
+		lkey = queue->priv->mdev->mlx5e_res.mkey.key;
+		for (i = 0; i < klm_entries; i++) {
+			sgl_mkey = &queue->ccid_table[ccid].sgl[i + klm_offset];
+			wqe->inline_klms[i].bcount = cpu_to_be32(sgl_mkey->length);
+			wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
+			wqe->inline_klms[i].va	   = cpu_to_be64(sgl_mkey->dma_address);
+		}
+
+		for (; i < ALIGN(klm_entries, KLM_ALIGNMENT); i++) {
+			wqe->inline_klms[i].bcount = 0;
+			wqe->inline_klms[i].key    = 0;
+			wqe->inline_klms[i].va     = 0;
+		}
 	}
 }
 
 static void
 build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 		       struct mlx5e_umr_wqe *wqe, u16 ccid, int klm_entries,
-		       u32 klm_offset, u32 len)
+		       u32 klm_offset, u32 len, enum wqe_type klm_type)
 {
-	u32 id = queue->ccid_table[ccid].klm_mkey.key;
+	u32 id = (klm_type == KLM_UMR) ? queue->ccid_table[ccid].klm_mkey.key :
+		(queue->tirn << MLX5_WQE_CTRL_TIR_TIS_INDEX_SHIFT);
+	u8 opc_mod = (klm_type == KLM_UMR) ? MLX5_CTRL_SEGMENT_OPC_MOD_UMR_UMR :
+		MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_STATIC_PARAMS;
 	struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl;
 	struct mlx5_wqe_ctrl_seg      *cseg = &wqe->ctrl;
 	struct mlx5_mkey_seg	       *mkc = &wqe->mkc;
@@ -57,12 +203,12 @@ build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 	u16 pc = queue->sq->icosq.pc;
 
 	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
-					     MLX5_OPCODE_UMR);
+					     MLX5_OPCODE_UMR | (opc_mod) << 24);
 	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
 				   MLX5E_KLM_UMR_DS_CNT(ALIGN(klm_entries, KLM_ALIGNMENT)));
 	cseg->general_id = cpu_to_be32(id);
 
-	if (!klm_offset) {
+	if (klm_type == KLM_UMR && !klm_offset) {
 		ucseg->mkey_mask |= cpu_to_be64(MLX5_MKEY_MASK_XLT_OCT_SIZE |
 						MLX5_MKEY_MASK_LEN | MLX5_MKEY_MASK_FREE);
 		mkc->xlt_oct_size = cpu_to_be32(ALIGN(len, KLM_ALIGNMENT));
@@ -72,21 +218,152 @@ build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 	ucseg->flags = MLX5_UMR_INLINE | MLX5_UMR_TRANSLATION_OFFSET_EN;
 	ucseg->xlt_octowords = cpu_to_be16(ALIGN(klm_entries, KLM_ALIGNMENT));
 	ucseg->xlt_offset = cpu_to_be16(klm_offset);
-	fill_nvmeotcp_klm_wqe(queue, wqe, ccid, klm_entries, klm_offset);
+	fill_nvmeotcp_klm_wqe(queue, wqe, ccid, klm_entries, klm_offset, klm_type);
+}
+
+static void
+fill_nvmeotcp_progress_params(struct mlx5e_nvmeotcp_queue *queue,
+			      struct mlx5_seg_nvmeotcp_progress_params *params,
+			      u32 seq)
+{
+	void *ctx = params->ctx;
+
+	params->tir_num = cpu_to_be32(queue->tirn);
+
+	MLX5_SET(nvmeotcp_progress_params, ctx,
+		 next_pdu_tcp_sn, seq);
+	MLX5_SET(nvmeotcp_progress_params, ctx, pdu_tracker_state,
+		 MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_START);
+}
+
+void
+build_nvmeotcp_progress_params(struct mlx5e_nvmeotcp_queue *queue,
+			       struct mlx5e_set_nvmeotcp_progress_params_wqe *wqe,
+			       u32 seq)
+{
+	struct mlx5_wqe_ctrl_seg *cseg = &wqe->ctrl;
+	u32 sqn = queue->sq->icosq.sqn;
+	u16 pc = queue->sq->icosq.pc;
+	u8 opc_mod;
+
+	memset(wqe, 0, MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ);
+	opc_mod = MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_PROGRESS_PARAMS;
+	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
+					     MLX5_OPCODE_SET_PSV | (opc_mod << 24));
+	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
+				   PROGRESS_PARAMS_DS_CNT);
+	fill_nvmeotcp_progress_params(queue, &wqe->params, seq);
+}
+
+static void
+fill_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
+			    struct mlx5_seg_nvmeotcp_static_params *params,
+			    u32 resync_seq, bool zero_copy_en,
+			    bool ddgst_offload_en)
+{
+	void *ctx = params->ctx;
+
+	MLX5_SET(transport_static_params, ctx, const_1, 1);
+	MLX5_SET(transport_static_params, ctx, const_2, 2);
+	MLX5_SET(transport_static_params, ctx, acc_type,
+		 MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_NVMETCP);
+	MLX5_SET(transport_static_params, ctx, nvme_resync_tcp_sn, resync_seq);
+	MLX5_SET(transport_static_params, ctx, pda, queue->pda);
+	MLX5_SET(transport_static_params, ctx, ddgst_en,
+		 queue->dgst & NVME_TCP_DATA_DIGEST_ENABLE);
+	MLX5_SET(transport_static_params, ctx, ddgst_offload_en, ddgst_offload_en);
+	MLX5_SET(transport_static_params, ctx, hddgst_en,
+		 queue->dgst & NVME_TCP_HDR_DIGEST_ENABLE);
+	MLX5_SET(transport_static_params, ctx, hdgst_offload_en, 0);
+	MLX5_SET(transport_static_params, ctx, ti,
+		 MLX5_TRANSPORT_STATIC_PARAMS_TI_INITIATOR);
+	MLX5_SET(transport_static_params, ctx, const1, 1);
+	MLX5_SET(transport_static_params, ctx, zero_copy_en, zero_copy_en);
+}
+
+void
+build_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
+			     struct mlx5e_set_nvmeotcp_static_params_wqe *wqe,
+			     u32 resync_seq, bool zerocopy, bool crc_rx)
+{
+	u8 opc_mod = MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_STATIC_PARAMS;
+	struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl;
+	struct mlx5_wqe_ctrl_seg      *cseg = &wqe->ctrl;
+	u32 sqn = queue->sq->icosq.sqn;
+	u16 pc = queue->sq->icosq.pc;
+
+	memset(wqe, 0, MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ);
+
+	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
+					     MLX5_OPCODE_UMR | (opc_mod) << 24);
+	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
+				   STATIC_PARAMS_DS_CNT);
+	cseg->imm = cpu_to_be32(queue->tirn << MLX5_WQE_CTRL_TIR_TIS_INDEX_SHIFT);
+
+	ucseg->flags = MLX5_UMR_INLINE;
+	ucseg->bsf_octowords =
+		cpu_to_be16(MLX5E_NVMEOTCP_STATIC_PARAMS_OCTWORD_SIZE);
+	fill_nvmeotcp_static_params(queue, &wqe->params, resync_seq, zerocopy, crc_rx);
 }
 
 static void
 mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
-		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi)
+		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi,
+		       enum wqe_type type)
 {
 	struct mlx5e_icosq_wqe_info *wi = &sq->db.wqe_info[pi];
 
 	wi->num_wqebbs = wqe_bbs;
-	wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
+	switch (type) {
+	case SET_PSV_UMR:
+		wi->wqe_type = MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP;
+		break;
+	default:
+		wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
+		break;
+	}
+
+	if (type == SET_PSV_UMR)
+		wi->nvmeotcp_q.queue = nvmeotcp_queue;
+}
+
+static void
+mlx5e_nvmeotcp_rx_post_static_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
+					 u32 resync_seq)
+{
+	struct mlx5e_set_nvmeotcp_static_params_wqe *wqe;
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+	u16 pi, wqe_bbs;
+
+	wqe_bbs = MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS;
+	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
+	wqe = MLX5E_NVMEOTCP_FETCH_STATIC_PARAMS_WQE(sq, pi);
+	mlx5e_nvmeotcp_fill_wi(NULL, sq, wqe_bbs, pi, BSF_UMR);
+	build_nvmeotcp_static_params(queue, wqe, resync_seq, queue->zerocopy, queue->crc_rx);
+	sq->pc += wqe_bbs;
+	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
+}
+
+static void
+mlx5e_nvmeotcp_rx_post_progress_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
+					   u32 seq)
+{
+	struct mlx5e_set_nvmeotcp_progress_params_wqe *wqe;
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+	u16 pi, wqe_bbs;
+
+	wqe_bbs = MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQEBBS;
+	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
+	wqe = MLX5E_NVMEOTCP_FETCH_PROGRESS_PARAMS_WQE(sq, pi);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, SET_PSV_UMR);
+	build_nvmeotcp_progress_params(queue, wqe, seq);
+	sq->pc += wqe_bbs;
+	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
 }
 
 static void
 post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+	     enum wqe_type wqe_type,
 	     u16 ccid,
 	     u32 klm_length,
 	     u32 *klm_offset)
@@ -102,9 +379,9 @@ post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, wqe_type);
 	build_nvmeotcp_klm_umr(queue, wqe, ccid, cur_klm_entries, *klm_offset,
-			       klm_length);
+			       klm_length, wqe_type);
 	*klm_offset += cur_klm_entries;
 	sq->pc += wqe_bbs;
 	sq->doorbell_cseg = &wqe->ctrl;
@@ -112,6 +389,7 @@ post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 
 static int
 mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+			    enum wqe_type wqe_type,
 			    u16 ccid,
 			    u32 klm_length)
 {
@@ -129,31 +407,327 @@ mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 		return -ENOSPC;
 
 	for (i = 0; i < wqes; i++)
-		post_klm_wqe(queue, ccid, klm_length, &klm_offset);
+		post_klm_wqe(queue, wqe_type, ccid, klm_length, &klm_offset);
 
 	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, sq->doorbell_cseg);
 	return 0;
 }
 
+static int mlx5e_create_nvmeotcp_mkey(struct mlx5_core_dev *mdev,
+				      u8 access_mode,
+				      u32 translation_octword_size,
+				      struct mlx5_core_mkey *mkey)
+{
+	int inlen = MLX5_ST_SZ_BYTES(create_mkey_in);
+	void *mkc;
+	u32 *in;
+	int err;
+
+	in = kvzalloc(inlen, GFP_KERNEL);
+	if (!in)
+		return -ENOMEM;
+
+	mkc = MLX5_ADDR_OF(create_mkey_in, in, memory_key_mkey_entry);
+	MLX5_SET(mkc, mkc, free, 1);
+	MLX5_SET(mkc, mkc, translations_octword_size, translation_octword_size);
+	MLX5_SET(mkc, mkc, umr_en, 1);
+	MLX5_SET(mkc, mkc, lw, 1);
+	MLX5_SET(mkc, mkc, lr, 1);
+	MLX5_SET(mkc, mkc, access_mode_1_0, access_mode);
+
+	MLX5_SET(mkc, mkc, qpn, 0xffffff);
+	MLX5_SET(mkc, mkc, pd, mdev->mlx5e_res.pdn);
+
+	err = mlx5_core_create_mkey(mdev, mkey, in, inlen);
+
+	kvfree(in);
+	return err;
+}
+
 static int
 mlx5e_nvmeotcp_offload_limits(struct net_device *netdev,
 			      struct tcp_ddp_limits *limits)
 {
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+
+	limits->max_ddp_sgl_len = mlx5e_get_max_sgl(mdev);
+	return 0;
+}
+
+static void
+mlx5e_nvmeotcp_destroy_sq(struct mlx5e_nvmeotcp_sq *nvmeotcpsq)
+{
+	mlx5e_deactivate_icosq(&nvmeotcpsq->icosq);
+	mlx5e_close_icosq(&nvmeotcpsq->icosq);
+	mlx5e_close_cq(&nvmeotcpsq->icosq.cq);
+	list_del(&nvmeotcpsq->list);
+	kfree(nvmeotcpsq);
+}
+
+static int
+mlx5e_nvmeotcp_build_icosq(struct mlx5e_nvmeotcp_queue *queue,
+			   struct mlx5e_priv *priv)
+{
+	u16 max_sgl, max_klm_per_wqe, max_umr_per_ccid, sgl_rest, wqebbs_rest;
+	struct mlx5e_channel *c = priv->channels.c[queue->channel_ix];
+	struct mlx5e_sq_param icosq_param = {0};
+	struct dim_cq_moder icocq_moder = {0};
+	struct mlx5e_nvmeotcp_sq *nvmeotcp_sq;
+	struct mlx5e_create_cq_param ccp;
+	struct mlx5e_icosq *icosq;
+	int err = -ENOMEM;
+	u16 log_icosq_sz;
+	u32 max_wqebbs;
+
+	nvmeotcp_sq = kzalloc(sizeof(*nvmeotcp_sq), GFP_KERNEL);
+	if (!nvmeotcp_sq)
+		return err;
+
+	icosq = &nvmeotcp_sq->icosq;
+	max_sgl = mlx5e_get_max_sgl(priv->mdev);
+	max_klm_per_wqe = queue->max_klms_per_wqe;
+	max_umr_per_ccid = max_sgl / max_klm_per_wqe;
+	sgl_rest = max_sgl % max_klm_per_wqe;
+	wqebbs_rest = sgl_rest ? MLX5E_KLM_UMR_WQEBBS(sgl_rest) : 0;
+	max_wqebbs = (MLX5E_KLM_UMR_WQEBBS(max_klm_per_wqe) *
+		     max_umr_per_ccid + wqebbs_rest) * queue->size;
+	log_icosq_sz = order_base_2(max_wqebbs);
+
+	mlx5e_build_icosq_param(priv, log_icosq_sz, &icosq_param);
+	mlx5e_build_create_cq_param(&ccp, c);
+	err = mlx5e_open_cq(priv, icocq_moder, &icosq_param.cqp, &ccp, &icosq->cq);
+	if (err)
+		goto err_nvmeotcp_sq;
+
+	err = mlx5e_open_icosq(c, &priv->channels.params, &icosq_param, icosq);
+	if (err)
+		goto close_cq;
+
+	INIT_LIST_HEAD(&nvmeotcp_sq->list);
+	spin_lock(&c->nvmeotcp_icosq_lock);
+	list_add(&nvmeotcp_sq->list, &c->list_nvmeotcpsq);
+	spin_unlock(&c->nvmeotcp_icosq_lock);
+	queue->sq = nvmeotcp_sq;
+	mlx5e_activate_icosq(icosq);
+	return 0;
+
+close_cq:
+	mlx5e_close_cq(&icosq->cq);
+err_nvmeotcp_sq:
+	kfree(nvmeotcp_sq);
+
+	return err;
+}
+
+static void
+mlx5e_nvmeotcp_destroy_rx(struct mlx5e_nvmeotcp_queue *queue,
+			  struct mlx5_core_dev *mdev, bool zerocopy)
+{
+	int i;
+
+	mlx5e_accel_fs_del_sk(queue->fh);
+	for (i = 0; i < queue->size && zerocopy; i++)
+		mlx5_core_destroy_mkey(mdev, &queue->ccid_table[i].klm_mkey);
+
+	mlx5e_nvmeotcp_destroy_tir(queue->priv, queue->tirn);
+	if (zerocopy) {
+		kfree(queue->ccid_table);
+		mlx5_destroy_nvmeotcp_tag_buf_table(mdev, queue->tag_buf_table_id);
+	}
+
+	mlx5e_nvmeotcp_destroy_sq(queue->sq);
+}
+
+static int
+mlx5e_nvmeotcp_queue_rx_init(struct mlx5e_nvmeotcp_queue *queue,
+			     struct nvme_tcp_ddp_config *config,
+			     struct net_device *netdev,
+			     bool zerocopy, bool crc)
+{
+	u8 log_queue_size = order_base_2(config->queue_size);
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+	struct sock *sk = queue->sk;
+	int err, max_sgls, i;
+
+	if (zerocopy) {
+		if (config->queue_size >
+		    BIT(MLX5_CAP_DEV_NVMEOTCP(mdev, log_max_nvmeotcp_tag_buffer_size))) {
+			return -EINVAL;
+		}
+
+		err = mlx5e_create_nvmeotcp_tag_buf_table(mdev, queue, log_queue_size);
+		if (err)
+			return err;
+	}
+
+	err = mlx5e_nvmeotcp_build_icosq(queue, priv);
+	if (err)
+		goto destroy_tag_buffer_table;
+
+	/* initializes queue->tirn */
+	err = mlx5e_nvmeotcp_create_tir(priv, sk, config, queue, zerocopy, crc);
+	if (err)
+		goto destroy_icosq;
+
+	mlx5e_nvmeotcp_rx_post_static_params_wqe(queue, 0);
+	mlx5e_nvmeotcp_rx_post_progress_params_wqe(queue, tcp_sk(sk)->copied_seq);
+
+	if (zerocopy) {
+		queue->ccid_table = kcalloc(queue->size,
+					    sizeof(struct nvmeotcp_queue_entry),
+					    GFP_KERNEL);
+		if (!queue->ccid_table) {
+			err = -ENOMEM;
+			goto destroy_tir;
+		}
+
+		max_sgls = mlx5e_get_max_sgl(mdev);
+		for (i = 0; i < queue->size; i++) {
+			err = mlx5e_create_nvmeotcp_mkey(mdev,
+							 MLX5_MKC_ACCESS_MODE_KLMS,
+							 max_sgls,
+							 &queue->ccid_table[i].klm_mkey);
+			if (err)
+				goto free_sgl;
+		}
+
+		err = mlx5e_nvmeotcp_post_klm_wqe(queue, BSF_KLM_UMR, 0, queue->size);
+		if (err)
+			goto free_sgl;
+	}
+
+	if (!(WARN_ON(!wait_for_completion_timeout(&queue->done, 0))))
+		queue->fh = mlx5e_accel_fs_add_sk(priv, sk, queue->tirn, queue->id);
+
+	if (IS_ERR_OR_NULL(queue->fh)) {
+		err = -EINVAL;
+		goto free_sgl;
+	}
+
 	return 0;
+
+free_sgl:
+	while ((i--) && zerocopy)
+		mlx5_core_destroy_mkey(mdev, &queue->ccid_table[i].klm_mkey);
+
+	if (zerocopy)
+		kfree(queue->ccid_table);
+destroy_tir:
+	mlx5e_nvmeotcp_destroy_tir(priv, queue->tirn);
+destroy_icosq:
+	mlx5e_nvmeotcp_destroy_sq(queue->sq);
+destroy_tag_buffer_table:
+	if (zerocopy)
+		mlx5_destroy_nvmeotcp_tag_buf_table(mdev, queue->tag_buf_table_id);
+
+	return err;
 }
 
+#define OCTWORD_SHIFT 4
+#define MAX_DS_VALUE 63
 static int
 mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 			  struct sock *sk,
 			  struct tcp_ddp_config *tconfig)
 {
-	return 0;
+	struct nvme_tcp_ddp_config *config = (struct nvme_tcp_ddp_config *)tconfig;
+	bool crc_rx = ((netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX) &&
+		       (config->dgst & NVME_TCP_DATA_DIGEST_ENABLE));
+	bool zerocopy = (netdev->features & NETIF_F_HW_TCP_DDP);
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+	struct mlx5e_nvmeotcp_queue *queue;
+	int max_wqe_sz_cap, queue_id, err;
+
+	if (tconfig->type != TCP_DDP_NVME) {
+		err = -EOPNOTSUPP;
+		goto out;
+	}
+
+	queue = kzalloc(sizeof(*queue), GFP_KERNEL);
+	if (!queue) {
+		err = -ENOMEM;
+		goto out;
+	}
+
+	queue_id = ida_simple_get(&priv->nvmeotcp->queue_ids,
+				  MIN_NVMEOTCP_QUEUES, MAX_NVMEOTCP_QUEUES,
+				  GFP_KERNEL);
+	if (queue_id < 0) {
+		err = -ENOSPC;
+		goto free_queue;
+	}
+
+	queue->crc_rx = crc_rx;
+	queue->zerocopy = zerocopy;
+	queue->tcp_ddp_ctx.type = TCP_DDP_NVME;
+	queue->sk = sk;
+	queue->id = queue_id;
+	queue->dgst = config->dgst;
+	queue->pda = config->cpda;
+	queue->channel_ix = mlx5e_get_channel_ix_from_io_cpu(priv,
+							     config->io_cpu);
+	queue->size = config->queue_size;
+	max_wqe_sz_cap  = min_t(int, MAX_DS_VALUE * MLX5_SEND_WQE_DS,
+				MLX5_CAP_GEN(mdev, max_wqe_sz_sq) << OCTWORD_SHIFT);
+	queue->max_klms_per_wqe = MLX5E_KLM_ENTRIES_PER_WQE(max_wqe_sz_cap);
+	queue->priv = priv;
+	init_completion(&queue->done);
+
+	if (zerocopy || crc_rx) {
+		err = mlx5e_nvmeotcp_queue_rx_init(queue, config, netdev,
+						   zerocopy, crc_rx);
+		if (err)
+			goto remove_queue_id;
+	}
+
+	err = rhashtable_insert_fast(&priv->nvmeotcp->queue_hash, &queue->hash,
+				     rhash_queues);
+	if (err)
+		goto destroy_rx;
+
+	write_lock_bh(&sk->sk_callback_lock);
+	tcp_ddp_set_ctx(sk, queue);
+	write_unlock_bh(&sk->sk_callback_lock);
+	refcount_set(&queue->ref_count, 1);
+	return err;
+
+destroy_rx:
+	if (zerocopy || crc_rx)
+		mlx5e_nvmeotcp_destroy_rx(queue, mdev, zerocopy);
+remove_queue_id:
+	ida_simple_remove(&priv->nvmeotcp->queue_ids, queue_id);
+free_queue:
+	kfree(queue);
+out:
+	return err;
 }
 
 static void
 mlx5e_nvmeotcp_queue_teardown(struct net_device *netdev,
 			      struct sock *sk)
 {
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+	struct mlx5e_nvmeotcp_queue *queue;
+
+	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
+
+	napi_synchronize(&priv->channels.c[queue->channel_ix]->napi);
+
+	WARN_ON(refcount_read(&queue->ref_count) != 1);
+	if (queue->zerocopy | queue->crc_rx)
+		mlx5e_nvmeotcp_destroy_rx(queue, mdev, queue->zerocopy);
+
+	rhashtable_remove_fast(&priv->nvmeotcp->queue_hash, &queue->hash,
+			       rhash_queues);
+	ida_simple_remove(&priv->nvmeotcp->queue_ids, queue->id);
+	write_lock_bh(&sk->sk_callback_lock);
+	tcp_ddp_set_ctx(sk, NULL);
+	write_unlock_bh(&sk->sk_callback_lock);
+	mlx5e_nvmeotcp_put_queue(queue);
 }
 
 static int
@@ -164,6 +738,16 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	return 0;
 }
 
+void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi)
+{
+	struct mlx5e_nvmeotcp_queue *queue = wi->nvmeotcp_q.queue;
+
+	if (unlikely(!queue))
+		return;
+
+	complete(&queue->done);
+}
+
 static int
 mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 			    struct sock *sk,
@@ -188,6 +772,27 @@ static const struct tcp_ddp_dev_ops mlx5e_nvmeotcp_ops = {
 	.tcp_ddp_resync = mlx5e_nvmeotcp_dev_resync,
 };
 
+struct mlx5e_nvmeotcp_queue *
+mlx5e_nvmeotcp_get_queue(struct mlx5e_nvmeotcp *nvmeotcp, int id)
+{
+	struct mlx5e_nvmeotcp_queue *queue;
+
+	rcu_read_lock();
+	queue = rhashtable_lookup_fast(&nvmeotcp->queue_hash,
+				       &id, rhash_queues);
+	if (queue && !IS_ERR(queue))
+		if (!refcount_inc_not_zero(&queue->ref_count))
+			queue = NULL;
+	rcu_read_unlock();
+	return queue;
+}
+
+void mlx5e_nvmeotcp_put_queue(struct mlx5e_nvmeotcp_queue *queue)
+{
+	if (refcount_dec_and_test(&queue->ref_count))
+		kfree(queue);
+}
+
 int set_feature_nvme_tcp(struct net_device *netdev, bool enable)
 {
 	struct mlx5e_priv *priv = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
index 753757fc44a3..d0e515502d6d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
@@ -100,6 +100,10 @@ int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv);
 int set_feature_nvme_tcp(struct net_device *netdev, bool enable);
 int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable);
 void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv);
+struct mlx5e_nvmeotcp_queue *
+mlx5e_nvmeotcp_get_queue(struct mlx5e_nvmeotcp *nvmeotcp, int id);
+void mlx5e_nvmeotcp_put_queue(struct mlx5e_nvmeotcp_queue *queue);
+void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi);
 int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv);
 void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv);
 #else
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
index 329e114d6571..44671e28a9ea 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
@@ -4,9 +4,77 @@
 #define __MLX5E_NVMEOTCP_UTILS_H__
 
 #include "en.h"
+#include "en_accel/nvmeotcp.h"
+
+enum {
+	MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_START     = 0,
+	MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_TRACKING  = 1,
+	MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_SEARCHING = 2,
+};
+
+struct mlx5_seg_nvmeotcp_static_params {
+	u8     ctx[MLX5_ST_SZ_BYTES(transport_static_params)];
+};
+
+struct mlx5_seg_nvmeotcp_progress_params {
+	__be32 tir_num;
+	u8     ctx[MLX5_ST_SZ_BYTES(nvmeotcp_progress_params)];
+};
+
+struct mlx5e_set_nvmeotcp_static_params_wqe {
+	struct mlx5_wqe_ctrl_seg          ctrl;
+	struct mlx5_wqe_umr_ctrl_seg      uctrl;
+	struct mlx5_mkey_seg              mkc;
+	struct mlx5_seg_nvmeotcp_static_params params;
+};
+
+struct mlx5e_set_nvmeotcp_progress_params_wqe {
+	struct mlx5_wqe_ctrl_seg            ctrl;
+	struct mlx5_seg_nvmeotcp_progress_params params;
+};
+
+struct mlx5e_get_psv_wqe {
+	struct mlx5_wqe_ctrl_seg ctrl;
+	struct mlx5_seg_get_psv  psv;
+};
+
+///////////////////////////////////////////
+#define MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ \
+	(sizeof(struct mlx5e_set_nvmeotcp_static_params_wqe))
+
+#define MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ \
+	(sizeof(struct mlx5e_set_nvmeotcp_progress_params_wqe))
+#define MLX5E_NVMEOTCP_STATIC_PARAMS_OCTWORD_SIZE \
+	(MLX5_ST_SZ_BYTES(transport_static_params) / MLX5_SEND_WQE_DS)
+
+#define MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS \
+	(DIV_ROUND_UP(MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ, MLX5_SEND_WQE_BB))
+#define MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQEBBS \
+	(DIV_ROUND_UP(MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ, MLX5_SEND_WQE_BB))
+
+#define MLX5E_NVMEOTCP_FETCH_STATIC_PARAMS_WQE(sq, pi) \
+	((struct mlx5e_set_nvmeotcp_static_params_wqe *)\
+	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_set_nvmeotcp_static_params_wqe)))
+
+#define MLX5E_NVMEOTCP_FETCH_PROGRESS_PARAMS_WQE(sq, pi) \
+	((struct mlx5e_set_nvmeotcp_progress_params_wqe *)\
+	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_set_nvmeotcp_progress_params_wqe)))
 
 #define MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi) \
 	((struct mlx5e_umr_wqe *)\
 	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_umr_wqe)))
 
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_PROGRESS_PARAMS 0x4
+
+void
+build_nvmeotcp_progress_params(struct mlx5e_nvmeotcp_queue *queue,
+			       struct mlx5e_set_nvmeotcp_progress_params_wqe *wqe,
+			       u32 seq);
+
+void
+build_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
+			     struct mlx5e_set_nvmeotcp_static_params_wqe *wqe,
+			     u32 resync_seq,
+			     bool zerocopy, bool crc_rx);
+
 #endif /* __MLX5E_NVMEOTCP_UTILS_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 974e34c956ed..c08d8cfdaebe 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -47,6 +47,7 @@
 #include "fpga/ipsec.h"
 #include "en_accel/ipsec_rxtx.h"
 #include "en_accel/tls_rxtx.h"
+#include "en_accel/nvmeotcp.h"
 #include "lib/clock.h"
 #include "en/xdp.h"
 #include "en/xsk/rx.h"
@@ -629,6 +630,9 @@ void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 			break;
+		case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
+			mlx5e_nvmeotcp_ctx_comp(wi);
+			break;
 #endif
 		}
 	}
@@ -703,6 +707,9 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 				break;
+			case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
+				mlx5e_nvmeotcp_ctx_comp(wi);
+				break;
 #endif
 			default:
 				netdev_WARN_ONCE(cq->netdev,
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:09 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:09 +0200
Subject: [PATCH v3 net-next 21/21] Documentation: add TCP DDP offload
 documentation
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-22-borisp@mellanox.com>

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 Documentation/networking/index.rst           |   1 +
 Documentation/networking/tcp-ddp-offload.rst | 296 +++++++++++++++++++
 2 files changed, 297 insertions(+)
 create mode 100644 Documentation/networking/tcp-ddp-offload.rst

diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index b8a29997d433..99644159a0cc 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -99,6 +99,7 @@ Contents:
    sysfs-tagging
    tc-actions-env-rules
    tcp-thin
+   tcp-ddp-offload
    team
    timestamping
    tipc
diff --git a/Documentation/networking/tcp-ddp-offload.rst b/Documentation/networking/tcp-ddp-offload.rst
new file mode 100644
index 000000000000..1607e8210968
--- /dev/null
+++ b/Documentation/networking/tcp-ddp-offload.rst
@@ -0,0 +1,296 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+=================================
+TCP direct data placement offload
+=================================
+
+Overview
+========
+
+The Linux kernel TCP direct data placement (DDP) offload infrastructure
+provides tagged request-response protocols, such as NVMe-TCP, the ability to
+place response data directly in pre-registered buffers according to header
+tags. DDP is particularly useful for data-intensive pipelined protocols whose
+responses may be reordered.
+
+For example, in NVMe-TCP numerous read requests are sent together and each
+request is tagged using the PDU header CID field. Receiving servers process
+requests as fast as possible and sometimes responses for smaller requests
+bypasses responses to larger requests, i.e., read 4KB bypasses read 1GB.
+Thereafter, clients corrleate responses to requests using PDU header CID tags.
+The processing of each response requires copying data from SKBs to read
+request destination buffers; The offload avoids this copy. The offload is
+oblivious to destination buffers which can reside either in userspace
+(O_DIRECT) or in kernel pagecache.
+
+Request TCP byte-stream:
+
+.. parsed-literal::
+
+ +---------------+-------+---------------+-------+---------------+-------+
+ | PDU hdr CID=1 | Req 1 | PDU hdr CID=2 | Req 2 | PDU hdr CID=3 | Req 3 |
+ +---------------+-------+---------------+-------+---------------+-------+
+
+Response TCP byte-stream:
+
+.. parsed-literal::
+
+ +---------------+--------+---------------+--------+---------------+--------+
+ | PDU hdr CID=2 | Resp 2 | PDU hdr CID=3 | Resp 3 | PDU hdr CID=1 | Resp 1 |
+ +---------------+--------+---------------+--------+---------------+--------+
+
+Offloading requires no new SKB bits. Instead, the driver builds SKB page
+fragments that point destination buffers. Consequently, SKBs represent the
+original data on the wire, which enables *transparent* inter-operation with the
+network stack.  To avoid copies between SKBs and destination buffers, the
+layer-5 protocol (L5P) will check ``if (src == dst)`` for SKB page fragments,
+success indicates that data is already placed there by NIC hardware and copy
+should be skipped.
+
+Offloading does require NIC hardware to track L5P procotol framing, similarly
+to RX TLS offload (see documentation at
+:ref:`Documentation/networking/tls-offload.rst <tls_offload>`).  NIC hardware
+will parse PDU headers extract fields such as operation type, length, ,tag
+identifier, etc. and offload only segments that correspond to tags registered
+with the NIC, see the :ref:`buf_reg` section.
+
+Device configuration
+====================
+
+During driver initialization the device sets the ``NETIF_F_HW_TCP_DDP`` and
+feature and installs its
+:c:type:`struct tcp_ddp_ops <tcp_ddp_ops>`
+pointer in the :c:member:`tcp_ddp_ops` member of the
+:c:type:`struct net_device <net_device>`.
+
+Later, after the L5P completes its handshake offload is installed on the socket.
+If offload installation fails, then the connection is handled by software as if
+offload was not attempted. Offload installation should configure 
+
+To request offload for a socket `sk`, the L5P calls :c:member:`tcp_ddp_sk_add`:
+
+.. code-block:: c
+
+ int (*tcp_ddp_sk_add)(struct net_device *netdev,
+ 		      struct sock *sk,
+ 		      struct tcp_ddp_config *config);
+
+The function return 0 for success. In case of failure, L5P software should
+fallback to normal non-offloaded operation.  The `config` parameter indicates
+the L5P type and any metadata relevant for that protocol. For example, in
+NVMe-TCP the following config is used:
+
+.. code-block:: c
+
+ /**
+  * struct nvme_tcp_ddp_config - nvme tcp ddp configuration for an IO queue
+  *
+  * @pfv:        pdu version (e.g., NVME_TCP_PFV_1_0)
+  * @cpda:       controller pdu data alignmend (dwords, 0's based)
+  * @dgst:       digest types enabled.
+  *              The netdev will offload crc if ddp_crc is supported.
+  * @queue_size: number of nvme-tcp IO queue elements
+  * @queue_id:   queue identifier
+  * @cpu_io:     cpu core running the IO thread for this queue
+  */
+ struct nvme_tcp_ddp_config {
+ 	struct tcp_ddp_config   cfg;
+ 
+ 	u16			pfv;
+ 	u8			cpda;
+ 	u8			dgst;
+ 	int			queue_size;
+ 	int			queue_id;
+ 	int			io_cpu;
+ };
+
+When offload is not needed anymore, e.g., the socket is being released, the L5P
+calls :c:member:`tcp_ddp_sk_del` to release device contexts:
+
+.. code-block:: c
+
+ void (*tcp_ddp_sk_del)(struct net_device *netdev,
+  		        struct sock *sk);
+
+Normal operation
+================
+
+At the very least, the device maintains the following state for each connection:
+
+ * 5-tuple
+ * expected TCP sequence number
+ * mapping between tags and corresponding buffers
+ * current offset within PDU, PDU length, current PDU tag
+
+NICs should not assume any correleation between PDUs and TCP packets.  Assuming
+that TCP packets arrive in-order, offload will place PDU payload directly
+inside corresponding registered buffers. No packets are to be delayed by NIC
+offload. If offload is not possible, than the packet is to be passed as-is to
+software. To perform offload on incoming packets without buffering packets in
+the NIC, the NIC stores some inter-packet state, such as partial PDU headers.
+
+RX data-path
+------------
+
+After the device validates TCP checksums, it can perform DDP offload.  The
+packet is steered to the DDP offload context according to the 5-tuple.
+Thereafter, the expected TCP sequence number is checked against the packet's
+TCP sequence number. If there's a match, then offload is performed: PDU payload
+is DMA written to corresponding destination buffer according to the PDU header
+tag.  The data should be DMAed only once, and the NIC receive ring will only
+store the remaining TCP and PDU headers.
+
+We remark that a single TCP packet may have numerous PDUs embedded inside. NICs
+can choose to offload one or more of these PDUs according to various
+trade-offs. Possibly, offloading such small PDUs is of little value, and it is
+better to leave it to software.
+
+Upon receiving a DDP offloaded packet, the driver reconstructs the original SKB
+using page frags, while pointing to the destination buffers whenever possible.
+This method enables seemless integration with the network stack, which can
+inspect and modify packet fields transperently to the offload.
+
+.. _buf_reg:
+
+Destination buffer registration
+-------------------------------
+
+To register the mapping betwteen tags and destination buffers for a socket
+`sk`, the L5P calls :c:member:`tcp_ddp_setup` of :c:type:`struct tcp_ddp_ops
+<tcp_ddp_ops>`:
+
+.. code-block:: c
+  
+ int (*tcp_ddp_setup)(struct net_device *netdev,
+ 		     struct sock *sk,
+ 		     struct tcp_ddp_io *io);
+
+
+The `io` provides the buffer via scatter-gather list (`sg_table`) and
+corresponding tag (`command_id`):
+
+.. code-block:: c
+ /**
+  * struct tcp_ddp_io - tcp ddp configuration for an IO request.
+  *
+  * @command_id:  identifier on the wire associated with these buffers
+  * @nents:       number of entries in the sg_table
+  * @sg_table:    describing the buffers for this IO request
+  * @first_sgl:   first SGL in sg_table
+  */
+ struct tcp_ddp_io {
+ 	u32			command_id;
+ 	int			nents;
+ 	struct sg_table		sg_table;
+ 	struct scatterlist	first_sgl[SG_CHUNK_SIZE];
+ };
+
+After the buffers have been consumed by the L5P, to release the NIC mapping of
+buffers the L5P calls :c:member:`tcp_ddp_teardown` of :c:type:`struct
+tcp_ddp_ops <tcp_ddp_ops>`: 
+
+.. code-block:: c
+  
+ int (*tcp_ddp_teardown)(struct net_device *netdev,
+ 			struct sock *sk,
+ 			struct tcp_ddp_io *io,
+ 			void *ddp_ctx);
+
+`tcp_ddp_teardown` receives the same `io` context and an additional opaque
+`ddp_ctx` that is used for asynchronous teardown, see the :ref:`async_release`
+section.
+
+.. _async_release:
+
+Asynchronous teardown
+---------------------
+
+To teardown the association between tags and buffers and allow tag reuse NIC HW
+is called by the NIC driver during `tcp_ddp_teardown`. This operation may be
+performed either synchronously or asynchronously. In asynchronous teardown,
+`tcp_ddp_teardown` returns immediately without unmapping NIC HW buffers. Later,
+when the unmapping completes by NIC HW, the NIC driver will call up to L5P
+using :c:member:`ddp_teardown_done` of :c:type:`struct tcp_ddp_ulp_ops`:
+
+.. code-block:: c
+
+ void (*ddp_teardown_done)(void *ddp_ctx);
+
+The `ddp_ctx` parameter passed in `ddp_teardown_done` is the same on provided
+in `tcp_ddp_teardown` and it is used to carry some context about the buffers
+and tags that are released.
+
+Resync handling
+===============
+
+In presence of packet drops or network packet reordering, the device may lose
+synchronization between the TCP stream and the L5P framing, and require a
+resync with the kernel's TCP stack. When the device is out of sync, no offload
+takes place, and packets are passed as-is to software. (resync is very similar
+to TLS offload (see documentation at
+:ref:`Documentation/networking/tls-offload.rst <tls_offload>`)
+
+If only packets with L5P data are lost or reordered, then resynchronization may
+be avoided by NIC HW that keeps tracking PDU headers. If, however, PDU headers
+are reordered, then resynchronization is necessary.
+
+To resynchronize hardware during traffic, we use a handshake between hardware
+and software. The NIC HW searches for a sequence of bytes that identifies L5P
+headers (i.e., magic pattern).  For example, in NVMe-TCP, the PDU operation
+type can be used for this purpose.  Using the PDU header length field, the NIC
+HW will continue to find and match magic patterns in subsequent PDU headers. If
+the pattern is missing in an expected position, then searching for the pattern
+starts anew.
+
+The NIC will not resume offload when the magic pattern is first identified.
+Instead, it will request L5P software to confirm that indeed this is a PDU
+header. To request confirmation the NIC driver calls up to L5P using
+:c:member:`*resync_request` of :c:type:`struct tcp_ddp_ulp_ops`:
+
+.. code-block:: c
+
+  bool (*resync_request)(struct sock *sk, u32 seq, u32 flags);
+
+The `seq` field contains the TCP sequence of the last byte in the PDU header.
+L5P software will respond to this request after observing the packet containing
+TCP sequence `seq` in-order. If the PDU header is indeed there, then L5P
+software calls the NIC driver using the :c:member:`tcp_ddp_resync` function of
+the :c:type:`struct tcp_ddp_ops <tcp_ddp_ops>` inside the :c:type:`struct
+net_device <net_device>` while passing the same `seq` to confirm it is a PDU
+header.
+
+.. code-block:: c
+
+ void (*tcp_ddp_resync)(struct net_device *netdev,
+ 		       struct sock *sk, u32 seq);
+
+Statistics
+==========
+
+Per L5P protocol, the following NIC driver must report statistics for the above
+netdevice operations and packets processed by offload. For example, NVMe-TCP
+offload reports:
+
+ * ``rx_nvmeotcp_queue_init`` - number of NVMe-TCP offload contexts created.
+ * ``rx_nvmeotcp_queue_teardown`` - number of NVMe-TCP offload contexts
+   destroyed.
+ * ``rx_nvmeotcp_ddp_setup`` - number of DDP buffers mapped.
+ * ``rx_nvmeotcp_ddp_setup_fail`` - number of DDP buffers mapping that failed.
+ * ``rx_nvmeotcp_ddp_teardown`` - number of DDP buffers unmapped.
+ * ``rx_nvmeotcp_drop`` - number of packets dropped in the driver due to fatal
+   errors.
+ * ``rx_nvmeotcp_resync`` - number of packets with resync requests.
+ * ``rx_nvmeotcp_offload_packets`` - number of packets that used offload.
+ * ``rx_nvmeotcp_offload_bytes`` - number of bytes placed in DDP buffers.
+
+NIC requirements
+================
+
+NIC hardware should meet the following requirements to provide this offload:
+
+ * Offload must never buffer TCP packets.
+ * Offload must never modify TCP packet headers.
+ * Offload must never reorder TCP packets within a flow.
+ * Offload must never drop TCP packets.
+ * Offload must not depend on any TCP fields beyond the
+   5-tuple and TCP sequence number.
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:01 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:01 +0200
Subject: [PATCH v3 net-next 13/21] net/mlx5e: NVMEoTCP offload initialization
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-14-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

This commit introduce the initialization blocks for NVMEoTCP offload:
- Use 128B CQEs when NVME-TCP offload is enabled.
- Use a dedicated icosq for NVME-TCP work. This list of SQ is unique in the
  sense that it is driven directly by the NVME-TCP layer to submit and
  invalidate ddp requests.
- Query nvmeotcp capabilities

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/Kconfig   |  10 +
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   7 +
 .../ethernet/mellanox/mlx5/core/en/params.h   |   1 +
 .../mellanox/mlx5/core/en_accel/en_accel.h    |   9 +-
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 196 ++++++++++++++++++
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    | 115 ++++++++++
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  39 +++-
 .../net/ethernet/mellanox/mlx5/core/en_txrx.c |  17 ++
 drivers/net/ethernet/mellanox/mlx5/core/fw.c  |   6 +
 10 files changed, 400 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
index c38b791e2406..6d91df52bb9c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
@@ -222,3 +222,13 @@ config MLX5_SF_MANAGER
 	port is managed through devlink.  A subfunction supports RDMA, netdevice
 	and vdpa device. It is similar to a SRIOV VF but it doesn't require
 	SRIOV support.
+
+config MLX5_EN_NVMEOTCP
+	bool "NVMEoTCP accelaration"
+	depends on MLX5_CORE_EN
+	depends on TCP_DDP=y || TCP_DDP_CRC=y
+	default n
+	help
+	Build support for NVMEoTCP accelaration in the NIC.
+	Note: Support for hardware with this capability needs to be selected
+	for this option to become available.
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index 8809dd4de57e..020fe2478fd3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -97,3 +97,5 @@ mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o
 # SF manager
 #
 mlx5_core-$(CONFIG_MLX5_SF_MANAGER) += sf/cmd.o sf/hw_table.o sf/devlink.o
+
+mlx5_core-$(CONFIG_MLX5_EN_NVMEOTCP) += en_accel/fs_tcp.o en_accel/nvmeotcp.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index e0b102958524..3003bae2c300 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -669,6 +669,10 @@ struct mlx5e_channel {
 	struct mlx5e_txqsq         sq[MLX5E_MAX_NUM_TC];
 	struct mlx5e_icosq         icosq;   /* internal control operations */
 	struct mlx5e_txqsq __rcu * __rcu *qos_sqs;
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	struct list_head	   list_nvmeotcpsq;   /* nvmeotcp umrs  */
+	spinlock_t                 nvmeotcp_icosq_lock;
+#endif
 	bool                       xdp;
 	struct napi_struct         napi;
 	struct device             *pdev;
@@ -874,6 +878,9 @@ struct mlx5e_priv {
 #endif
 #ifdef CONFIG_MLX5_EN_TLS
 	struct mlx5e_tls          *tls;
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	struct mlx5e_nvmeotcp      *nvmeotcp;
 #endif
 	struct devlink_health_reporter *tx_reporter;
 	struct devlink_health_reporter *rx_reporter;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
index ea2cfb04b31a..855a41893b3d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
@@ -16,6 +16,7 @@ struct mlx5e_cq_param {
 	struct mlx5_wq_param       wq;
 	u16                        eq_ix;
 	u8                         cq_period_mode;
+	bool                       force_cqe128;
 };
 
 struct mlx5e_rq_param {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h
index 959bb6cd7203..eabba2168dd3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h
@@ -39,6 +39,7 @@
 #include "en_accel/ipsec_rxtx.h"
 #include "en_accel/tls.h"
 #include "en_accel/tls_rxtx.h"
+#include "en_accel/nvmeotcp.h"
 #include "en.h"
 #include "en/txrx.h"
 
@@ -198,11 +199,17 @@ static inline void mlx5e_accel_tx_finish(struct mlx5e_txqsq *sq,
 
 static inline int mlx5e_accel_init_rx(struct mlx5e_priv *priv)
 {
-	return mlx5e_ktls_init_rx(priv);
+	int tls, nvmeotcp;
+
+	tls = mlx5e_ktls_init_rx(priv);
+	nvmeotcp = mlx5e_nvmeotcp_init_rx(priv);
+
+	return tls && nvmeotcp;
 }
 
 static inline void mlx5e_accel_cleanup_rx(struct mlx5e_priv *priv)
 {
+	mlx5e_nvmeotcp_cleanup_rx(priv);
 	mlx5e_ktls_cleanup_rx(priv);
 }
 #endif /* __MLX5E_EN_ACCEL_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
new file mode 100644
index 000000000000..0834deb7a3ac
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2021 Mellanox Technologies. */
+
+#include <linux/netdevice.h>
+#include <linux/idr.h>
+#include "en_accel/nvmeotcp.h"
+#include "en_accel/fs_tcp.h"
+#include "en/txrx.h"
+
+#define MAX_NVMEOTCP_QUEUES	(512)
+#define MIN_NVMEOTCP_QUEUES	(1)
+
+static const struct rhashtable_params rhash_queues = {
+	.key_len = sizeof(int),
+	.key_offset = offsetof(struct mlx5e_nvmeotcp_queue, id),
+	.head_offset = offsetof(struct mlx5e_nvmeotcp_queue, hash),
+	.automatic_shrinking = true,
+	.min_size = 1,
+	.max_size = MAX_NVMEOTCP_QUEUES,
+};
+
+static int
+mlx5e_nvmeotcp_offload_limits(struct net_device *netdev,
+			      struct tcp_ddp_limits *limits)
+{
+	return 0;
+}
+
+static int
+mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
+			  struct sock *sk,
+			  struct tcp_ddp_config *tconfig)
+{
+	return 0;
+}
+
+static void
+mlx5e_nvmeotcp_queue_teardown(struct net_device *netdev,
+			      struct sock *sk)
+{
+}
+
+static int
+mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
+			 struct sock *sk,
+			 struct tcp_ddp_io *ddp)
+{
+	return 0;
+}
+
+static int
+mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
+			    struct sock *sk,
+			    struct tcp_ddp_io *ddp,
+			    void *ddp_ctx)
+{
+	return 0;
+}
+
+static void
+mlx5e_nvmeotcp_dev_resync(struct net_device *netdev,
+			  struct sock *sk, u32 seq)
+{
+}
+
+static const struct tcp_ddp_dev_ops mlx5e_nvmeotcp_ops = {
+	.tcp_ddp_limits = mlx5e_nvmeotcp_offload_limits,
+	.tcp_ddp_sk_add = mlx5e_nvmeotcp_queue_init,
+	.tcp_ddp_sk_del = mlx5e_nvmeotcp_queue_teardown,
+	.tcp_ddp_setup = mlx5e_nvmeotcp_ddp_setup,
+	.tcp_ddp_teardown = mlx5e_nvmeotcp_ddp_teardown,
+	.tcp_ddp_resync = mlx5e_nvmeotcp_dev_resync,
+};
+
+int set_feature_nvme_tcp(struct net_device *netdev, bool enable)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	int err = 0;
+
+	mutex_lock(&priv->state_lock);
+	if (enable)
+		err = mlx5e_accel_fs_tcp_create(priv);
+	else
+		mlx5e_accel_fs_tcp_destroy(priv);
+	mutex_unlock(&priv->state_lock);
+	if (err)
+		return err;
+
+	priv->nvmeotcp->enable = enable;
+	err = mlx5e_safe_reopen_channels(priv);
+	return err;
+}
+
+int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	int err = 0;
+
+	mutex_lock(&priv->state_lock);
+	if (enable)
+		err = mlx5e_accel_fs_tcp_create(priv);
+	else
+		mlx5e_accel_fs_tcp_destroy(priv);
+	mutex_unlock(&priv->state_lock);
+
+	priv->nvmeotcp->crc_rx_enable = enable;
+	err = mlx5e_safe_reopen_channels(priv);
+	if (err)
+		netdev_err(priv->netdev,
+			   "%s failed to reopen channels, err(%d).\n",
+			   __func__, err);
+
+	return err;
+}
+
+void mlx5e_nvmeotcp_build_netdev(struct mlx5e_priv *priv)
+{
+	struct net_device *netdev = priv->netdev;
+
+	if (!MLX5_CAP_GEN(priv->mdev, nvmeotcp))
+		return;
+
+	if (MLX5_CAP_DEV_NVMEOTCP(priv->mdev, zerocopy)) {
+		netdev->features |= NETIF_F_HW_TCP_DDP;
+		netdev->hw_features |= NETIF_F_HW_TCP_DDP;
+	}
+
+	if (MLX5_CAP_DEV_NVMEOTCP(priv->mdev, crc_rx)) {
+		netdev->features |= NETIF_F_HW_TCP_DDP_CRC_RX;
+		netdev->hw_features |= NETIF_F_HW_TCP_DDP_CRC_RX;
+	}
+
+	netdev->tcp_ddp_ops = &mlx5e_nvmeotcp_ops;
+	priv->nvmeotcp->enable = true;
+}
+
+int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv)
+{
+	int ret = 0;
+
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP) {
+		ret = mlx5e_accel_fs_tcp_create(priv);
+		if (ret)
+			return ret;
+	}
+
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+		ret = mlx5e_accel_fs_tcp_create(priv);
+
+	return ret;
+}
+
+void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv)
+{
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP)
+		mlx5e_accel_fs_tcp_destroy(priv);
+
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+		mlx5e_accel_fs_tcp_destroy(priv);
+}
+
+int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv)
+{
+	struct mlx5e_nvmeotcp *nvmeotcp = kzalloc(sizeof(*nvmeotcp), GFP_KERNEL);
+	int ret = 0;
+
+	if (!nvmeotcp)
+		return -ENOMEM;
+
+	ida_init(&nvmeotcp->queue_ids);
+	ret = rhashtable_init(&nvmeotcp->queue_hash, &rhash_queues);
+	if (ret)
+		goto err_ida;
+
+	priv->nvmeotcp = nvmeotcp;
+	goto out;
+
+err_ida:
+	ida_destroy(&nvmeotcp->queue_ids);
+	kfree(nvmeotcp);
+out:
+	return ret;
+}
+
+void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv)
+{
+	struct mlx5e_nvmeotcp *nvmeotcp = priv->nvmeotcp;
+
+	if (!nvmeotcp)
+		return;
+
+	rhashtable_destroy(&nvmeotcp->queue_hash);
+	ida_destroy(&nvmeotcp->queue_ids);
+	kfree(nvmeotcp);
+	priv->nvmeotcp = NULL;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
new file mode 100644
index 000000000000..753757fc44a3
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
@@ -0,0 +1,115 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021 Mellanox Technologies. */
+#ifndef __MLX5E_NVMEOTCP_H__
+#define __MLX5E_NVMEOTCP_H__
+
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+
+#include "net/tcp_ddp.h"
+#include "en.h"
+#include "en/params.h"
+
+struct nvmeotcp_queue_entry {
+	struct mlx5e_nvmeotcp_queue	*queue;
+	u32				sgl_length;
+	struct mlx5_core_mkey		klm_mkey;
+	struct scatterlist		*sgl;
+	u32				ccid_gen;
+	u64				size;
+
+	/* for the ddp invalidate done callback */
+	void				*ddp_ctx;
+	struct tcp_ddp_io		*ddp;
+};
+
+struct mlx5e_nvmeotcp_sq {
+	struct list_head		list;
+	struct mlx5e_icosq		icosq;
+};
+
+/**
+ *	struct mlx5e_nvmeotcp_queue - MLX5 metadata for NVMEoTCP queue
+ *	@fh: Flow handle representing the 5-tuple steering for this flow
+ *	@tirn: Destination TIR number created for NVMEoTCP offload
+ *	@id: Flow tag ID used to identify this queue
+ *	@size: NVMEoTCP queue depth
+ *	@sq: Send queue used for sending control messages
+ *	@ccid_table: Table holding metadata for each CC
+ *	@tag_buf_table_id: Tag buffer table for CCIDs
+ *	@hash: Hash table of queues mapped by @id
+ *	@ref_count: Reference count for this structure
+ *	@ccoff: Offset within the current CC
+ *	@pda: Padding alignment
+ *	@ccid_gen: Generation ID for the CCID, used to avoid conflicts in DDP
+ *	@max_klms_per_wqe: Number of KLMs per DDP operation
+ *	@channel_ix: Channel IX for this nvmeotcp_queue
+ *	@sk: The socket used by the NVMe-TCP queue
+ *	@zerocopy: if this queue is used for zerocopy offload.
+ *	@crc_rx: if this queue is used for CRC Rx offload.
+ *	@ccid: ID of the current CC
+ *	@ccsglidx: Index within the scatter-gather list (SGL) of the current CC
+ *	@ccoff_inner: Current offset within the @ccsglidx element
+ *	@priv: mlx5e netdev priv
+ *	@inv_done: invalidate callback of the nvme tcp driver
+ *	@after_resync_cqe: indicate if resync occurred
+ */
+struct mlx5e_nvmeotcp_queue {
+	struct tcp_ddp_ctx		tcp_ddp_ctx;
+	struct mlx5_flow_handle		*fh;
+	int				tirn;
+	int				id;
+	u32				size;
+	struct mlx5e_nvmeotcp_sq	*sq;
+	struct nvmeotcp_queue_entry	*ccid_table;
+	u32				tag_buf_table_id;
+	struct rhash_head		hash;
+	refcount_t			ref_count;
+	bool				dgst;
+	int				pda;
+	u32				ccid_gen;
+	u32				max_klms_per_wqe;
+	u32				channel_ix;
+	struct sock			*sk;
+	bool				zerocopy;
+	bool				crc_rx;
+
+	/* current ccid fields */
+	off_t				ccoff;
+	int				ccid;
+	int				ccsglidx;
+	int				ccoff_inner;
+
+	/* for ddp invalidate flow */
+	struct mlx5e_priv		*priv;
+
+	/* for flow_steering flow */
+	struct completion		done;
+	/* for MASK HW resync cqe */
+	bool				after_resync_cqe;
+};
+
+struct mlx5e_nvmeotcp {
+	struct ida			queue_ids;
+	struct rhashtable		queue_hash;
+	bool				enable;
+	bool				crc_rx_enable;
+};
+
+void mlx5e_nvmeotcp_build_netdev(struct mlx5e_priv *priv);
+int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv);
+int set_feature_nvme_tcp(struct net_device *netdev, bool enable);
+int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable);
+void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv);
+int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv);
+void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv);
+#else
+
+static inline void mlx5e_nvmeotcp_build_netdev(struct mlx5e_priv *priv) { }
+static inline int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv) { return 0; }
+static inline void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv) { }
+static inline int set_feature_nvme_tcp(struct net_device *netdev, bool enable) { return 0; }
+static inline int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable) { return 0; }
+static inline int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv) { return 0; }
+static inline void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv) { }
+#endif
+#endif /* __MLX5E_NVMEOTCP_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 3ff91247a97c..edda0ff8090b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -47,6 +47,7 @@
 #include "en_accel/ipsec.h"
 #include "en_accel/en_accel.h"
 #include "en_accel/tls.h"
+#include "en_accel/nvmeotcp.h"
 #include "accel/ipsec.h"
 #include "accel/tls.h"
 #include "lib/vxlan.h"
@@ -2022,6 +2023,10 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
 	c->aff_mask = irq_get_effective_affinity_mask(irq);
 	c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix);
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	INIT_LIST_HEAD(&c->list_nvmeotcpsq);
+	spin_lock_init(&c->nvmeotcp_icosq_lock);
+#endif
 	netif_napi_add(netdev, &c->napi, mlx5e_napi_poll, 64);
 
 	err = mlx5e_open_queues(c, params, cparam);
@@ -2259,7 +2264,8 @@ static void mlx5e_build_common_cq_param(struct mlx5e_priv *priv,
 	void *cqc = param->cqc;
 
 	MLX5_SET(cqc, cqc, uar_page, priv->mdev->priv.uar->index);
-	if (MLX5_CAP_GEN(priv->mdev, cqe_128_always) && cache_line_size() >= 128)
+	if (MLX5_CAP_GEN(priv->mdev, cqe_128_always) &&
+	    (cache_line_size() >= 128 || param->force_cqe128))
 		MLX5_SET(cqc, cqc, cqe_sz, CQE_STRIDE_128_PAD);
 }
 
@@ -2273,6 +2279,11 @@ void mlx5e_build_rx_cq_param(struct mlx5e_priv *priv,
 	void *cqc = param->cqc;
 	u8 log_cq_size;
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	/* nvme-tcp offload mandates 128 byte cqes */
+	param->force_cqe128 |= (priv->nvmeotcp->enable || priv->nvmeotcp->crc_rx_enable);
+#endif
+
 	switch (params->rq_wq_type) {
 	case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
 		log_cq_size = mlx5e_mpwqe_get_log_rq_size(params, xsk) +
@@ -4066,6 +4077,10 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features)
 	err |= MLX5E_HANDLE_FEATURE(NETIF_F_NTUPLE, set_feature_arfs);
 #endif
 	err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TLS_RX, mlx5e_ktls_set_feature_rx);
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TCP_DDP, set_feature_nvme_tcp);
+	err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TCP_DDP_CRC_RX, set_feature_nvme_tcp_crc);
+#endif
 
 	if (err) {
 		netdev->features = oper_features;
@@ -4102,6 +4117,23 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
 		features &= ~NETIF_F_RXHASH;
 		if (netdev->features & NETIF_F_RXHASH)
 			netdev_warn(netdev, "Disabling rxhash, not supported when CQE compress is active\n");
+
+		features &= ~NETIF_F_HW_TCP_DDP;
+		if (netdev->features & NETIF_F_HW_TCP_DDP)
+			netdev_warn(netdev, "Disabling tcp-ddp offload, not supported when CQE compress is active\n");
+
+		features &= ~NETIF_F_HW_TCP_DDP_CRC_RX;
+		if (netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+			netdev_warn(netdev, "Disabling tcp-ddp-crc-rx offload, not supported when CQE compression is active\n");
+	}
+
+	if (netdev->features & NETIF_F_LRO) {
+		features &= ~NETIF_F_HW_TCP_DDP;
+		if (netdev->features & NETIF_F_HW_TCP_DDP)
+			netdev_warn(netdev, "Disabling tcp-ddp offload, not supported when LRO is active\n");
+		features &= ~NETIF_F_HW_TCP_DDP_CRC_RX;
+		if (netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+			netdev_warn(netdev, "Disabling tcp-ddp-crc-rx offload, not supported when LRO is active\n");
 	}
 
 	mutex_unlock(&priv->state_lock);
@@ -5180,6 +5212,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 	mlx5e_set_netdev_dev_addr(netdev);
 	mlx5e_ipsec_build_netdev(priv);
 	mlx5e_tls_build_netdev(priv);
+	mlx5e_nvmeotcp_build_netdev(priv);
 }
 
 void mlx5e_create_q_counters(struct mlx5e_priv *priv)
@@ -5244,6 +5277,9 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev,
 	err = mlx5e_tls_init(priv);
 	if (err)
 		mlx5_core_err(mdev, "TLS initialization failed, %d\n", err);
+	err = mlx5e_nvmeotcp_init(priv);
+	if (err)
+		mlx5_core_err(mdev, "NVMEoTCP initialization failed, %d\n", err);
 	mlx5e_build_nic_netdev(netdev);
 	err = mlx5e_devlink_port_register(priv);
 	if (err)
@@ -5257,6 +5293,7 @@ static void mlx5e_nic_cleanup(struct mlx5e_priv *priv)
 {
 	mlx5e_health_destroy_reporters(priv);
 	mlx5e_devlink_port_unregister(priv);
+	mlx5e_nvmeotcp_cleanup(priv);
 	mlx5e_tls_cleanup(priv);
 	mlx5e_ipsec_cleanup(priv);
 	mlx5e_netdev_cleanup(priv->netdev, priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index d54da3797c30..2817615f06b6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -36,6 +36,7 @@
 #include "en/xdp.h"
 #include "en/xsk/rx.h"
 #include "en/xsk/tx.h"
+#include "en_accel/nvmeotcp.h"
 
 static inline bool mlx5e_channel_no_affinity_change(struct mlx5e_channel *c)
 {
@@ -118,6 +119,10 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
 	struct mlx5e_txqsq __rcu **qos_sqs;
 	struct mlx5e_rq *xskrq = &c->xskrq;
 	struct mlx5e_rq *rq = &c->rq;
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	struct mlx5e_nvmeotcp_sq *nvmeotcp_sq;
+	struct list_head *cur;
+#endif
 	bool aff_change = false;
 	bool busy_xsk = false;
 	bool busy = false;
@@ -170,6 +175,12 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
 		 * queueing more WQEs and overflowing the async ICOSQ.
 		 */
 		clear_bit(MLX5E_SQ_STATE_PENDING_XSK_TX, &c->async_icosq.state);
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	list_for_each(cur, &c->list_nvmeotcpsq) {
+		nvmeotcp_sq = list_entry(cur, struct mlx5e_nvmeotcp_sq, list);
+		mlx5e_poll_ico_cq(&nvmeotcp_sq->icosq.cq);
+	}
+#endif
 
 	busy |= INDIRECT_CALL_2(rq->post_wqes,
 				mlx5e_post_rx_mpwqes,
@@ -218,6 +229,12 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
 	mlx5e_cq_arm(&rq->cq);
 	mlx5e_cq_arm(&c->icosq.cq);
 	mlx5e_cq_arm(&c->async_icosq.cq);
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	list_for_each(cur, &c->list_nvmeotcpsq) {
+		nvmeotcp_sq = list_entry(cur, struct mlx5e_nvmeotcp_sq, list);
+		mlx5e_cq_arm(&nvmeotcp_sq->icosq.cq);
+	}
+#endif
 	mlx5e_cq_arm(&c->xdpsq.cq);
 
 	if (xsk_open) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c
index 02558ac2ace6..5e7544ccae91 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c
@@ -256,6 +256,12 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev)
 			return err;
 	}
 
+	if (MLX5_CAP_GEN(dev, nvmeotcp)) {
+		err = mlx5_core_get_caps(dev, MLX5_CAP_DEV_NVMEOTCP);
+		if (err)
+			return err;
+	}
+
 	return 0;
 }
 
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:54 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:54 +0200
Subject: [PATCH v3 net-next  06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-7-borisp@mellanox.com>

This commit introduces direct data placement offload to NVME
TCP. There is a context per queue, which is established after the
handshake using the tcp_ddp_sk_add/del NDOs.

Additionally, a resynchronization routine is used to assist
hardware recovery from TCP OOO, and continue the offload.
Resynchronization operates as follows:

1. TCP OOO causes the NIC HW to stop the offload

2. NIC HW identifies a PDU header at some TCP sequence number,
and asks NVMe-TCP to confirm it.
This request is delivered from the NIC driver to NVMe-TCP by first
finding the socket for the packet that triggered the request, and
then finding the nvme_tcp_queue that is used by this routine.
Finally, the request is recorded in the nvme_tcp_queue.

3. When NVMe-TCP observes the requested TCP sequence, it will compare
it with the PDU header TCP sequence, and report the result to the
NIC driver (tcp_ddp_resync), which will update the HW,
and resume offload when all is successful.

Furthermore, we let the offloading driver advertise what is the max hw
sectors/segments via tcp_ddp_limits.

A follow-up patch introduces the data-path changes required for this
offload.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 200 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 198 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 881d28eb15e9..ea67caf9d326 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -14,6 +14,7 @@
 #include <linux/blk-mq.h>
 #include <crypto/hash.h>
 #include <net/busy_poll.h>
+#include <net/tcp_ddp.h>
 
 #include "nvme.h"
 #include "fabrics.h"
@@ -62,6 +63,7 @@ enum nvme_tcp_queue_flags {
 	NVME_TCP_Q_ALLOCATED	= 0,
 	NVME_TCP_Q_LIVE		= 1,
 	NVME_TCP_Q_POLLING	= 2,
+	NVME_TCP_Q_OFF_DDP	= 3,
 };
 
 enum nvme_tcp_recv_state {
@@ -111,6 +113,8 @@ struct nvme_tcp_queue {
 	void (*state_change)(struct sock *);
 	void (*data_ready)(struct sock *);
 	void (*write_space)(struct sock *);
+
+	atomic64_t  resync_req;
 };
 
 struct nvme_tcp_ctrl {
@@ -129,6 +133,8 @@ struct nvme_tcp_ctrl {
 	struct delayed_work	connect_work;
 	struct nvme_tcp_request async_req;
 	u32			io_queues[HCTX_MAX_TYPES];
+
+	struct net_device       *offloading_netdev;
 };
 
 static LIST_HEAD(nvme_tcp_ctrl_list);
@@ -223,6 +229,183 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
 	return nvme_tcp_pdu_data_left(req) <= len;
 }
 
+#ifdef CONFIG_TCP_DDP
+
+static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
+static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
+	.resync_request		= nvme_tcp_resync_request,
+};
+
+static
+int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
+{
+	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
+	struct nvme_tcp_ddp_config config = {};
+	int ret;
+
+	if (!netdev) {
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
+		return -ENODEV;
+	}
+
+	if (!(netdev->features & NETIF_F_HW_TCP_DDP)) {
+		dev_put(netdev);
+		return -EOPNOTSUPP;
+	}
+
+	config.cfg.type		= TCP_DDP_NVME;
+	config.pfv		= NVME_TCP_PFV_1_0;
+	config.cpda		= 0;
+	config.dgst		= queue->hdr_digest ?
+		NVME_TCP_HDR_DIGEST_ENABLE : 0;
+	config.dgst		|= queue->data_digest ?
+		NVME_TCP_DATA_DIGEST_ENABLE : 0;
+	config.queue_size	= queue->queue_size;
+	config.queue_id		= nvme_tcp_queue_id(queue);
+	config.io_cpu		= queue->io_cpu;
+
+	ret = netdev->tcp_ddp_ops->tcp_ddp_sk_add(netdev,
+						  queue->sock->sk,
+						  &config.cfg);
+	if (ret) {
+		dev_put(netdev);
+		return ret;
+	}
+
+	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = &nvme_tcp_ddp_ulp_ops;
+	if (netdev->features & NETIF_F_HW_TCP_DDP)
+		set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
+
+	return ret;
+}
+
+static
+void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
+{
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+
+	if (!netdev) {
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
+		return;
+	}
+
+	netdev->tcp_ddp_ops->tcp_ddp_sk_del(netdev, queue->sock->sk);
+
+	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = NULL;
+	dev_put(netdev); /* put the queue_init get_netdev_for_sock() */
+}
+
+static
+int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
+{
+	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
+	struct tcp_ddp_limits limits;
+	int ret = 0;
+
+	if (!netdev) {
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
+		return -ENODEV;
+	}
+
+	if (netdev->features & NETIF_F_HW_TCP_DDP &&
+	    netdev->tcp_ddp_ops &&
+	    netdev->tcp_ddp_ops->tcp_ddp_limits)
+		ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
+	else
+		ret = -EOPNOTSUPP;
+
+	if (!ret) {
+		queue->ctrl->offloading_netdev = netdev;
+		dev_dbg_ratelimited(queue->ctrl->ctrl.device,
+				    "netdev %s offload limits: max_ddp_sgl_len %d\n",
+				    netdev->name, limits.max_ddp_sgl_len);
+		queue->ctrl->ctrl.max_segments = limits.max_ddp_sgl_len;
+		queue->ctrl->ctrl.max_hw_sectors =
+			limits.max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
+	} else {
+		queue->ctrl->offloading_netdev = NULL;
+	}
+
+	/* release the device as no offload context is established yet. */
+	dev_put(netdev);
+
+	return ret;
+}
+
+static
+void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
+			      unsigned int pdu_seq)
+{
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+	u64 resync_val;
+	u32 resync_seq;
+
+	resync_val = atomic64_read(&queue->resync_req);
+	/* Lower 32 bit flags. Check validity of the request */
+	if ((resync_val & TCP_DDP_RESYNC_REQ) == 0)
+		return;
+
+	/* Obtain and check requested sequence number: is this PDU header before the request? */
+	resync_seq = resync_val >> 32;
+	if (before(pdu_seq, resync_seq))
+		return;
+
+	if (unlikely(!netdev)) {
+		pr_info_ratelimited("%s: netdev not found\n", __func__);
+		return;
+	}
+
+	/**
+	 * The atomic operation gurarantees that we don't miss any NIC driver
+	 * resync requests submitted after the above checks.
+	 */
+	if (atomic64_cmpxchg(&queue->resync_req, resync_val,
+			     resync_val & ~TCP_DDP_RESYNC_REQ))
+		netdev->tcp_ddp_ops->tcp_ddp_resync(netdev, queue->sock->sk, pdu_seq);
+}
+
+static
+bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
+{
+	struct nvme_tcp_queue *queue = sk->sk_user_data;
+
+	atomic64_set(&queue->resync_req,
+		     (((uint64_t)seq << 32) | flags));
+
+	return true;
+}
+
+#else
+
+static
+int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
+{
+	return -EINVAL;
+}
+
+static
+void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
+{}
+
+static
+int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
+{
+	return -EINVAL;
+}
+
+static
+void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
+			      unsigned int pdu_seq)
+{}
+
+static
+bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
+{
+	return false;
+}
+
+#endif
+
 static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
 		unsigned int dir)
 {
@@ -638,6 +821,11 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 	size_t rcv_len = min_t(size_t, *len, queue->pdu_remaining);
 	int ret;
 
+	u64 pdu_seq = TCP_SKB_CB(skb)->seq + *offset - queue->pdu_offset;
+
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+		nvme_tcp_resync_response(queue, pdu_seq);
+
 	ret = skb_copy_bits(skb, *offset,
 		&pdu[queue->pdu_offset], rcv_len);
 	if (unlikely(ret))
@@ -1532,6 +1720,9 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
 	kernel_sock_shutdown(queue->sock, SHUT_RDWR);
 	nvme_tcp_restore_sock_calls(queue);
 	cancel_work_sync(&queue->io_work);
+
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+		nvme_tcp_unoffload_socket(queue);
 }
 
 static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
@@ -1550,10 +1741,13 @@ static int nvme_tcp_start_queue(struct nvme_ctrl *nctrl, int idx)
 	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
 	int ret;
 
-	if (idx)
+	if (idx) {
 		ret = nvmf_connect_io_queue(nctrl, idx, false);
-	else
+		nvme_tcp_offload_socket(&ctrl->queues[idx]);
+	} else {
 		ret = nvmf_connect_admin_queue(nctrl);
+		nvme_tcp_offload_limits(&ctrl->queues[idx]);
+	}
 
 	if (!ret) {
 		set_bit(NVME_TCP_Q_LIVE, &ctrl->queues[idx].flags);
@@ -1656,6 +1850,8 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
 {
 	int ret;
 
+	to_tcp_ctrl(ctrl)->offloading_netdev = NULL;
+
 	ret = nvme_tcp_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
 	if (ret)
 		return ret;
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:00 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:00 +0200
Subject: [PATCH v3 net-next  12/21] net/mlx5e: TCP flow steering for nvme-tcp
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-13-borisp@mellanox.com>

Both nvme-tcp and tls require tcp flow steering. Compile it for both of
them. Additionally, use reference counting to allocate/free TCP flow
steering.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en/fs.h        |  4 ++--
 .../net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c  | 10 ++++++++++
 .../net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h  |  2 +-
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
index a16297e7e2ac..a7fe3a6358ea 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
@@ -137,7 +137,7 @@ enum {
 	MLX5E_L2_FT_LEVEL,
 	MLX5E_TTC_FT_LEVEL,
 	MLX5E_INNER_TTC_FT_LEVEL,
-#ifdef CONFIG_MLX5_EN_TLS
+#if defined(CONFIG_MLX5_EN_TLS) || defined(CONFIG_MLX5_EN_NVMEOTCP)
 	MLX5E_ACCEL_FS_TCP_FT_LEVEL,
 #endif
 #ifdef CONFIG_MLX5_EN_ARFS
@@ -256,7 +256,7 @@ struct mlx5e_flow_steering {
 #ifdef CONFIG_MLX5_EN_ARFS
 	struct mlx5e_arfs_tables        arfs;
 #endif
-#ifdef CONFIG_MLX5_EN_TLS
+#if defined(CONFIG_MLX5_EN_TLS) || defined(CONFIG_MLX5_EN_NVMEOTCP)
 	struct mlx5e_accel_fs_tcp      *accel_tcp;
 #endif
 };
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c
index e51f60b55daa..21341a92f355 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c
@@ -14,6 +14,7 @@ enum accel_fs_tcp_type {
 struct mlx5e_accel_fs_tcp {
 	struct mlx5e_flow_table tables[ACCEL_FS_TCP_NUM_TYPES];
 	struct mlx5_flow_handle *default_rules[ACCEL_FS_TCP_NUM_TYPES];
+	refcount_t		ref_count;
 };
 
 static enum mlx5e_traffic_types fs_accel2tt(enum accel_fs_tcp_type i)
@@ -337,6 +338,7 @@ static int accel_fs_tcp_enable(struct mlx5e_priv *priv)
 			return err;
 		}
 	}
+	refcount_set(&priv->fs.accel_tcp->ref_count, 1);
 	return 0;
 }
 
@@ -360,6 +362,9 @@ void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv)
 	if (!priv->fs.accel_tcp)
 		return;
 
+	if (!refcount_dec_and_test(&priv->fs.accel_tcp->ref_count))
+		return;
+
 	accel_fs_tcp_disable(priv);
 
 	for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++)
@@ -376,6 +381,11 @@ int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv)
 	if (!MLX5_CAP_FLOWTABLE_NIC_RX(priv->mdev, ft_field_support.outer_ip_version))
 		return -EOPNOTSUPP;
 
+	if (priv->fs.accel_tcp) {
+		refcount_inc(&priv->fs.accel_tcp->ref_count);
+		return 0;
+	}
+
 	priv->fs.accel_tcp = kzalloc(sizeof(*priv->fs.accel_tcp), GFP_KERNEL);
 	if (!priv->fs.accel_tcp)
 		return -ENOMEM;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h
index 589235824543..8aff9298183c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h
@@ -6,7 +6,7 @@
 
 #include "en.h"
 
-#ifdef CONFIG_MLX5_EN_TLS
+#if defined(CONFIG_MLX5_EN_TLS) || defined(CONFIG_MLX5_EN_NVMEOTCP)
 int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv);
 void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv);
 struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_priv *priv,
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:03 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:03 +0200
Subject: [PATCH v3 net-next  15/21] net/mlx5e: NVMEoTCP use KLM UMRs
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-16-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

NVMEoTCP offload uses buffer registration for ddp operation,
every request comprises from SG list that might have elements with size > 4K,
thus the appropriate way to perform buffer registration is with KLM UMRs.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   5 +-
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   3 +
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 116 ++++++++++++++++++
 .../mlx5/core/en_accel/nvmeotcp_utils.h       |  12 ++
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  12 +-
 5 files changed, 145 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 13a0d4d5e9f6..f282c0f6cdab 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -236,7 +236,10 @@ struct mlx5e_umr_wqe {
 	struct mlx5_wqe_ctrl_seg       ctrl;
 	struct mlx5_wqe_umr_ctrl_seg   uctrl;
 	struct mlx5_mkey_seg           mkc;
-	struct mlx5_mtt                inline_mtts[0];
+	union {
+		struct mlx5_mtt        inline_mtts[0];
+		struct mlx5_klm	       inline_klms[0];
+	};
 };
 
 extern const char mlx5e_self_tests[][ETH_GSTRING_LEN];
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
index 4880f2179273..e997b7230028 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
@@ -34,6 +34,9 @@ enum mlx5e_icosq_wqe_type {
 	MLX5E_ICOSQ_WQE_SET_PSV_TLS,
 	MLX5E_ICOSQ_WQE_GET_PSV_TLS,
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	MLX5E_ICOSQ_WQE_UMR_NVME_TCP,
+#endif
 };
 
 /* General */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index 0834deb7a3ac..bbeded08ef01 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -4,6 +4,7 @@
 #include <linux/netdevice.h>
 #include <linux/idr.h>
 #include "en_accel/nvmeotcp.h"
+#include "en_accel/nvmeotcp_utils.h"
 #include "en_accel/fs_tcp.h"
 #include "en/txrx.h"
 
@@ -19,6 +20,121 @@ static const struct rhashtable_params rhash_queues = {
 	.max_size = MAX_NVMEOTCP_QUEUES,
 };
 
+static void
+fill_nvmeotcp_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+		      struct mlx5e_umr_wqe *wqe, u16 ccid, u32 klm_entries,
+		      u16 klm_offset)
+{
+	struct scatterlist *sgl_mkey;
+	u32 lkey, i;
+
+	lkey = queue->priv->mdev->mlx5e_res.mkey.key;
+	for (i = 0; i < klm_entries; i++) {
+		sgl_mkey = &queue->ccid_table[ccid].sgl[i + klm_offset];
+		wqe->inline_klms[i].bcount = cpu_to_be32(sgl_mkey->length);
+		wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
+		wqe->inline_klms[i].va	   = cpu_to_be64(sgl_mkey->dma_address);
+	}
+
+	for (; i < ALIGN(klm_entries, KLM_ALIGNMENT); i++) {
+		wqe->inline_klms[i].bcount = 0;
+		wqe->inline_klms[i].key    = 0;
+		wqe->inline_klms[i].va     = 0;
+	}
+}
+
+static void
+build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
+		       struct mlx5e_umr_wqe *wqe, u16 ccid, int klm_entries,
+		       u32 klm_offset, u32 len)
+{
+	u32 id = queue->ccid_table[ccid].klm_mkey.key;
+	struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl;
+	struct mlx5_wqe_ctrl_seg      *cseg = &wqe->ctrl;
+	struct mlx5_mkey_seg	       *mkc = &wqe->mkc;
+
+	u32 sqn = queue->sq->icosq.sqn;
+	u16 pc = queue->sq->icosq.pc;
+
+	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
+					     MLX5_OPCODE_UMR);
+	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
+				   MLX5E_KLM_UMR_DS_CNT(ALIGN(klm_entries, KLM_ALIGNMENT)));
+	cseg->general_id = cpu_to_be32(id);
+
+	if (!klm_offset) {
+		ucseg->mkey_mask |= cpu_to_be64(MLX5_MKEY_MASK_XLT_OCT_SIZE |
+						MLX5_MKEY_MASK_LEN | MLX5_MKEY_MASK_FREE);
+		mkc->xlt_oct_size = cpu_to_be32(ALIGN(len, KLM_ALIGNMENT));
+		mkc->len = cpu_to_be64(queue->ccid_table[ccid].size);
+	}
+
+	ucseg->flags = MLX5_UMR_INLINE | MLX5_UMR_TRANSLATION_OFFSET_EN;
+	ucseg->xlt_octowords = cpu_to_be16(ALIGN(klm_entries, KLM_ALIGNMENT));
+	ucseg->xlt_offset = cpu_to_be16(klm_offset);
+	fill_nvmeotcp_klm_wqe(queue, wqe, ccid, klm_entries, klm_offset);
+}
+
+static void
+mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
+		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi)
+{
+	struct mlx5e_icosq_wqe_info *wi = &sq->db.wqe_info[pi];
+
+	wi->num_wqebbs = wqe_bbs;
+	wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
+}
+
+static void
+post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+	     u16 ccid,
+	     u32 klm_length,
+	     u32 *klm_offset)
+{
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+	u32 wqe_bbs, cur_klm_entries;
+	struct mlx5e_umr_wqe *wqe;
+	u16 pi, wqe_sz;
+
+	cur_klm_entries = min_t(int, queue->max_klms_per_wqe,
+				klm_length - *klm_offset);
+	wqe_sz = MLX5E_KLM_UMR_WQE_SZ(ALIGN(cur_klm_entries, KLM_ALIGNMENT));
+	wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
+	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
+	wqe = MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi);
+	build_nvmeotcp_klm_umr(queue, wqe, ccid, cur_klm_entries, *klm_offset,
+			       klm_length);
+	*klm_offset += cur_klm_entries;
+	sq->pc += wqe_bbs;
+	sq->doorbell_cseg = &wqe->ctrl;
+}
+
+static int
+mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+			    u16 ccid,
+			    u32 klm_length)
+{
+	u32 klm_offset = 0, wqes, wqe_sz, max_wqe_bbs, i, room;
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+
+	/* TODO: set stricter wqe_sz; using max for now */
+	wqes = DIV_ROUND_UP(klm_length, queue->max_klms_per_wqe);
+	wqe_sz = MLX5E_KLM_UMR_WQE_SZ(queue->max_klms_per_wqe);
+
+	max_wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
+
+	room = mlx5e_stop_room_for_wqe(max_wqe_bbs) * wqes;
+	if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room)))
+		return -ENOSPC;
+
+	for (i = 0; i < wqes; i++)
+		post_klm_wqe(queue, ccid, klm_length, &klm_offset);
+
+	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, sq->doorbell_cseg);
+	return 0;
+}
+
 static int
 mlx5e_nvmeotcp_offload_limits(struct net_device *netdev,
 			      struct tcp_ddp_limits *limits)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
new file mode 100644
index 000000000000..329e114d6571
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021 Mellanox Technologies. */
+#ifndef __MLX5E_NVMEOTCP_UTILS_H__
+#define __MLX5E_NVMEOTCP_UTILS_H__
+
+#include "en.h"
+
+#define MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi) \
+	((struct mlx5e_umr_wqe *)\
+	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_umr_wqe)))
+
+#endif /* __MLX5E_NVMEOTCP_UTILS_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 98b56f495b32..974e34c956ed 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -617,16 +617,20 @@ void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq)
 		ci = mlx5_wq_cyc_ctr2ix(&sq->wq, sqcc);
 		wi = &sq->db.wqe_info[ci];
 		sqcc += wi->num_wqebbs;
-#ifdef CONFIG_MLX5_EN_TLS
 		switch (wi->wqe_type) {
+#ifdef CONFIG_MLX5_EN_TLS
 		case MLX5E_ICOSQ_WQE_SET_PSV_TLS:
 			mlx5e_ktls_handle_ctx_completion(wi);
 			break;
 		case MLX5E_ICOSQ_WQE_GET_PSV_TLS:
 			mlx5e_ktls_handle_get_psv_completion(wi, sq);
 			break;
-		}
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
+			break;
+#endif
+		}
 	}
 	sq->cc = sqcc;
 }
@@ -695,6 +699,10 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)
 			case MLX5E_ICOSQ_WQE_GET_PSV_TLS:
 				mlx5e_ktls_handle_get_psv_completion(wi, sq);
 				break;
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
+				break;
 #endif
 			default:
 				netdev_WARN_ONCE(cq->netdev,
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:51 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:51 +0200
Subject: [PATCH v3 net-next 03/21] net: Introduce crc offload for tcp ddp ulp
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-4-borisp@mellanox.com>

This commit introduces support for CRC offload to direct data placement
ULP on the receive side. Both DDP and CRC share a common API to
initialize the offload for a TCP socket. But otherwise, both can
be executed independently.

On the receive side, CRC offload relies on a new SKB bit that
indicates that no CRC error was encountered while processing this packet.
If all packets of a ULP message have this bit set, then the CRC
verification for the message can be skipped, as hardware already checked
it.

The following patches will set and use this bit to perform NVME-TCP
CRC offload.

A subsequent series, will add NVMe-TCP transmit side CRC support.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
---
 include/linux/netdev_features.h | 2 ++
 include/linux/netdevice.h       | 2 +-
 include/linux/skbuff.h          | 2 +-
 net/Kconfig                     | 8 ++++++++
 net/ethtool/common.c            | 1 +
 net/ipv4/tcp_input.c            | 4 ++--
 net/ipv4/tcp_ipv4.c             | 2 +-
 net/ipv4/tcp_offload.c          | 2 +-
 8 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/include/linux/netdev_features.h b/include/linux/netdev_features.h
index 7977371d2dd1..77fb2cb99b78 100644
--- a/include/linux/netdev_features.h
+++ b/include/linux/netdev_features.h
@@ -86,6 +86,7 @@ enum {
 	NETIF_F_HW_MACSEC_BIT,		/* Offload MACsec operations */
 	NETIF_F_GRO_UDP_FWD_BIT,	/* Allow UDP GRO for forwarding */
 	NETIF_F_HW_TCP_DDP_BIT,		/* TCP direct data placement offload */
+	NETIF_F_HW_TCP_DDP_CRC_RX_BIT,	/* TCP DDP CRC RX offload */
 
 	/*
 	 * Add your fresh new feature above and remember to update
@@ -161,6 +162,7 @@ enum {
 #define NETIF_F_HW_MACSEC	__NETIF_F(HW_MACSEC)
 #define NETIF_F_GRO_UDP_FWD	__NETIF_F(GRO_UDP_FWD)
 #define NETIF_F_HW_TCP_DDP	__NETIF_F(HW_TCP_DDP)
+#define NETIF_F_HW_TCP_DDP_CRC_RX	__NETIF_F(HW_TCP_DDP_CRC_RX)
 
 /* Finds the next feature with the highest number of the range of start till 0.
  */
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index bd28520e30f2..1065ddaa8e6b 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1943,7 +1943,7 @@ struct net_device {
 	const struct tlsdev_ops *tlsdev_ops;
 #endif
 
-#ifdef CONFIG_TCP_DDP
+#if IS_ENABLED(CONFIG_TCP_DDP) || IS_ENABLED(CONFIG_TCP_DDP_CRC)
 	const struct tcp_ddp_dev_ops *tcp_ddp_ops;
 #endif
 
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 6d00d62628c8..32c5ab8e2f67 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -861,7 +861,7 @@ struct sk_buff {
 #ifdef CONFIG_TLS_DEVICE
 	__u8			decrypted:1;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 	__u8                    ddp_crc:1;
 #endif
 
diff --git a/net/Kconfig b/net/Kconfig
index 3876861cdc90..80ed9f038968 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -465,6 +465,14 @@ config TCP_DDP
 	  NVMe-TCP/iSCSI, to request the NIC to place TCP payload data
 	  of a command response directly into kernel pages.
 
+config TCP_DDP_CRC
+	bool "TCP direct data placement CRC offload"
+	default n
+	help
+	  Direct Data Placement (DDP) CRC32C offload for TCP enables ULP, such as
+	  NVMe-TCP/iSCSI, to request the NIC to calculate/verify the data digest
+	  of commands as they go through the NIC. Thus avoiding the costly
+	  per-byte overhead.
 
 endif   # if NET
 
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index bbf83cd01106..14286afb2a69 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -70,6 +70,7 @@ const char netdev_features_strings[NETDEV_FEATURE_COUNT][ETH_GSTRING_LEN] = {
 	[NETIF_F_HW_MACSEC_BIT] =	 "macsec-hw-offload",
 	[NETIF_F_GRO_UDP_FWD_BIT] =	 "rx-udp-gro-forwarding",
 	[NETIF_F_HW_TCP_DDP_BIT] =	 "tcp-ddp-offload",
+	[NETIF_F_HW_TCP_DDP_CRC_RX_BIT] =	 "tcp-ddp-crc-rx-offload",
 };
 
 const char
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 0ae1ffca090d..cd490bd548a3 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -5150,7 +5150,7 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 #ifdef CONFIG_TLS_DEVICE
 		nskb->decrypted = skb->decrypted;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 		nskb->ddp_crc = skb->ddp_crc;
 #endif
 		TCP_SKB_CB(nskb)->seq = TCP_SKB_CB(nskb)->end_seq = start;
@@ -5186,7 +5186,7 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 				if (skb->decrypted != nskb->decrypted)
 					goto end;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 
 				if (skb->ddp_crc != nskb->ddp_crc)
 					goto end;
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index b114fc870513..4c903c1a6c90 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1811,7 +1811,7 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb)
 #ifdef CONFIG_TLS_DEVICE
 	    tail->decrypted != skb->decrypted ||
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 	    tail->ddp_crc != skb->ddp_crc ||
 #endif
 	    thtail->doff != th->doff ||
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index 3ce196375d94..9e43a044e730 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -262,7 +262,7 @@ struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb)
 #ifdef CONFIG_TLS_DEVICE
 	flush |= p->decrypted ^ skb->decrypted;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 	flush |= p->ddp_crc ^ skb->ddp_crc;
 #endif
 
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:58 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:58 +0200
Subject: [PATCH v3 net-next 10/21] net/mlx5: Header file changes for nvme-tcp
 offload
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-11-borisp@mellanox.com>

From: Ben Ben-ishay <benishay at nvidia.com>

Add the necessary infrastructure for NVMEoTCP offload:

- Add nvmeocp_en + nvmeotcp_crc_en bit to the TIR for identify NVMEoTCP offload flow
  And tag_buffer_id that will be used by the connected nvmeotcp_queues
- Add new CQE field that will be used to pass scattered data information to SW
- Add new capability to HCA_CAP that represnts the NVMEoTCP offload ability

Signed-off-by: Ben Ben-ishay <benishay at nvidia.com>
Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/mlx5/device.h   |   8 +++
 include/linux/mlx5/mlx5_ifc.h | 101 +++++++++++++++++++++++++++++++++-
 include/linux/mlx5/qp.h       |   1 +
 3 files changed, 107 insertions(+), 3 deletions(-)

diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index c85d96180b8f..ab04959188b9 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -263,6 +263,7 @@ enum {
 enum {
 	MLX5_MKEY_MASK_LEN		= 1ull << 0,
 	MLX5_MKEY_MASK_PAGE_SIZE	= 1ull << 1,
+	MLX5_MKEY_MASK_XLT_OCT_SIZE     = 1ull << 2,
 	MLX5_MKEY_MASK_START_ADDR	= 1ull << 6,
 	MLX5_MKEY_MASK_PD		= 1ull << 7,
 	MLX5_MKEY_MASK_EN_RINVAL	= 1ull << 8,
@@ -1173,6 +1174,7 @@ enum mlx5_cap_type {
 	MLX5_CAP_VDPA_EMULATION = 0x13,
 	MLX5_CAP_DEV_EVENT = 0x14,
 	MLX5_CAP_IPSEC,
+	MLX5_CAP_DEV_NVMEOTCP = 0x19,
 	/* NUM OF CAP Types */
 	MLX5_CAP_NUM
 };
@@ -1393,6 +1395,12 @@ enum mlx5_qcam_feature_groups {
 #define MLX5_CAP_IPSEC(mdev, cap)\
 	MLX5_GET(ipsec_cap, (mdev)->caps.hca_cur[MLX5_CAP_IPSEC], cap)
 
+#define MLX5_CAP_DEV_NVMEOTCP(mdev, cap)\
+	MLX5_GET(nvmeotcp_cap, mdev->caps.hca_cur[MLX5_CAP_DEV_NVMEOTCP], cap)
+
+#define MLX5_CAP64_NVMEOTCP(mdev, cap)\
+	MLX5_GET64(nvmeotcp_cap, mdev->caps.hca_cur[MLX5_CAP_DEV_NVMEOTCP], cap)
+
 enum {
 	MLX5_CMD_STAT_OK			= 0x0,
 	MLX5_CMD_STAT_INT_ERR			= 0x1,
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index b9f5894f6f8d..ae6edf08a70b 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -1278,7 +1278,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         log_max_srq_sz[0x8];
 	u8         log_max_qp_sz[0x8];
 	u8         event_cap[0x1];
-	u8         reserved_at_91[0x7];
+	u8         reserved_at_91[0x5];
+	u8         nvmeotcp[0x1];
+	u8         reserved_at_97[0x1];
 	u8         prio_tag_required[0x1];
 	u8         reserved_at_99[0x2];
 	u8         log_max_qp[0x5];
@@ -3029,6 +3031,21 @@ struct mlx5_ifc_roce_addr_layout_bits {
 	u8         reserved_at_e0[0x20];
 };
 
+struct mlx5_ifc_nvmeotcp_cap_bits {
+	u8    zerocopy[0x1];
+	u8    crc_rx[0x1];
+	u8    crc_tx[0x1];
+	u8    reserved_at_3[0x15];
+	u8    version[0x8];
+
+	u8    reserved_at_20[0x13];
+	u8    log_max_nvmeotcp_tag_buffer_table[0x5];
+	u8    reserved_at_38[0x3];
+	u8    log_max_nvmeotcp_tag_buffer_size[0x5];
+
+	u8    reserved_at_40[0x7c0];
+};
+
 union mlx5_ifc_hca_cap_union_bits {
 	struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
 	struct mlx5_ifc_odp_cap_bits odp_cap;
@@ -3045,6 +3062,7 @@ union mlx5_ifc_hca_cap_union_bits {
 	struct mlx5_ifc_tls_cap_bits tls_cap;
 	struct mlx5_ifc_device_mem_cap_bits device_mem_cap;
 	struct mlx5_ifc_virtio_emulation_cap_bits virtio_emulation_cap;
+	struct mlx5_ifc_nvmeotcp_cap_bits nvmeotcp_cap;
 	u8         reserved_at_0[0x8000];
 };
 
@@ -3239,7 +3257,9 @@ struct mlx5_ifc_tirc_bits {
 
 	u8         disp_type[0x4];
 	u8         tls_en[0x1];
-	u8         reserved_at_25[0x1b];
+	u8         nvmeotcp_zero_copy_en[0x1];
+	u8         nvmeotcp_crc_en[0x1];
+	u8         reserved_at_27[0x19];
 
 	u8         reserved_at_40[0x40];
 
@@ -3270,7 +3290,8 @@ struct mlx5_ifc_tirc_bits {
 
 	struct mlx5_ifc_rx_hash_field_select_bits rx_hash_field_selector_inner;
 
-	u8         reserved_at_2c0[0x4c0];
+	u8         nvmeotcp_tag_buffer_table_id[0x20];
+	u8         reserved_at_2e0[0x4a0];
 };
 
 enum {
@@ -10745,12 +10766,14 @@ enum {
 	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_ENCRYPTION_KEY = BIT_ULL(0xc),
 	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_IPSEC = BIT_ULL(0x13),
 	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
+	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE = BIT_ULL(0x21),
 };
 
 enum {
 	MLX5_GENERAL_OBJECT_TYPES_ENCRYPTION_KEY = 0xc,
 	MLX5_GENERAL_OBJECT_TYPES_IPSEC = 0x13,
 	MLX5_GENERAL_OBJECT_TYPES_SAMPLER = 0x20,
+	MLX5_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE = 0x21
 };
 
 enum {
@@ -10852,6 +10875,20 @@ struct mlx5_ifc_create_sampler_obj_in_bits {
 	struct mlx5_ifc_sampler_obj_bits sampler_object;
 };
 
+struct mlx5_ifc_nvmeotcp_tag_buf_table_obj_bits {
+	u8    modify_field_select[0x40];
+
+	u8    reserved_at_20[0x20];
+
+	u8    reserved_at_40[0x1b];
+	u8    log_tag_buffer_table_size[0x5];
+};
+
+struct mlx5_ifc_create_nvmeotcp_tag_buf_table_in_bits {
+	struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr;
+	struct mlx5_ifc_nvmeotcp_tag_buf_table_obj_bits nvmeotcp_tag_buf_table_obj;
+};
+
 enum {
 	MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_KEY_SIZE_128 = 0x0,
 	MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_KEY_SIZE_256 = 0x1,
@@ -10862,6 +10899,18 @@ enum {
 	MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_IPSEC = 0x2,
 };
 
+enum {
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_XTS               = 0x0,
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_TLS               = 0x1,
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_NVMETCP           = 0x2,
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_NVMETCP_WITH_TLS  = 0x3,
+};
+
+enum {
+	MLX5_TRANSPORT_STATIC_PARAMS_TI_INITIATOR  = 0x0,
+	MLX5_TRANSPORT_STATIC_PARAMS_TI_TARGET     = 0x1,
+};
+
 struct mlx5_ifc_tls_static_params_bits {
 	u8         const_2[0x2];
 	u8         tls_version[0x4];
@@ -10902,4 +10951,50 @@ enum {
 	MLX5_MTT_PERM_RW	= MLX5_MTT_PERM_READ | MLX5_MTT_PERM_WRITE,
 };
 
+struct mlx5_ifc_nvmeotcp_progress_params_bits {
+	u8    next_pdu_tcp_sn[0x20];
+
+	u8    hw_resync_tcp_sn[0x20];
+
+	u8    pdu_tracker_state[0x2];
+	u8    offloading_state[0x2];
+	u8    reserved_at_64[0xc];
+	u8    cccid_ttag[0x10];
+};
+
+struct mlx5_ifc_transport_static_params_bits {
+	u8    const_2[0x2];
+	u8    tls_version[0x4];
+	u8    const_1[0x2];
+	u8    reserved_at_8[0x14];
+	u8    acc_type[0x4];
+
+	u8    reserved_at_20[0x20];
+
+	u8    initial_record_number[0x40];
+
+	u8    resync_tcp_sn[0x20];
+
+	u8    gcm_iv[0x20];
+
+	u8    implicit_iv[0x40];
+
+	u8    reserved_at_100[0x8];
+	u8    dek_index[0x18];
+
+	u8    reserved_at_120[0x14];
+	u8    const1[0x1];
+	u8    ti[0x1];
+	u8    zero_copy_en[0x1];
+	u8    ddgst_offload_en[0x1];
+	u8    hdgst_offload_en[0x1];
+	u8    ddgst_en[0x1];
+	u8    hddgst_en[0x1];
+	u8    pda[0x5];
+
+	u8    nvme_resync_tcp_sn[0x20];
+
+	u8    reserved_at_160[0xa0];
+};
+
 #endif /* MLX5_IFC_H */
diff --git a/include/linux/mlx5/qp.h b/include/linux/mlx5/qp.h
index d75ef8aa8fac..5fa8b82c9edb 100644
--- a/include/linux/mlx5/qp.h
+++ b/include/linux/mlx5/qp.h
@@ -220,6 +220,7 @@ struct mlx5_wqe_ctrl_seg {
 #define MLX5_WQE_CTRL_OPCODE_MASK 0xff
 #define MLX5_WQE_CTRL_WQE_INDEX_MASK 0x00ffff00
 #define MLX5_WQE_CTRL_WQE_INDEX_SHIFT 8
+#define MLX5_WQE_CTRL_TIR_TIS_INDEX_SHIFT 8
 
 enum {
 	MLX5_ETH_WQE_L3_INNER_CSUM      = 1 << 4,
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:04:56 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:04:56 +0200
Subject: [PATCH v3 net-next 08/21] nvme-tcp : Recalculate crc in the end of
 the capsule
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-9-borisp@mellanox.com>

From: Ben Ben-ishay <benishay at nvidia.com>

crc offload of the nvme capsule. Check if all the skb bits
are on, and if not recalculate the crc in SW and check it.

This patch reworks the receive-side crc calculation to always
run at the end, so as to keep a single flow for both offload
and non-offload. This change simplifies the code, but it may degrade
performance for non-offload crc calculation.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 118 ++++++++++++++++++++++++++++++++--------
 1 file changed, 95 insertions(+), 23 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 5cb46deb56e0..eb47cf6982d7 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -69,6 +69,7 @@ enum nvme_tcp_queue_flags {
 	NVME_TCP_Q_LIVE		= 1,
 	NVME_TCP_Q_POLLING	= 2,
 	NVME_TCP_Q_OFF_DDP	= 3,
+	NVME_TCP_Q_OFF_DDGST_RX = 4,
 };
 
 enum nvme_tcp_recv_state {
@@ -96,6 +97,7 @@ struct nvme_tcp_queue {
 	size_t			data_remaining;
 	size_t			ddgst_remaining;
 	unsigned int		nr_cqe;
+	bool			ddgst_valid;
 
 	/* send state */
 	struct nvme_tcp_request *request;
@@ -234,7 +236,56 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
 	return nvme_tcp_pdu_data_left(req) <= len;
 }
 
-#ifdef CONFIG_TCP_DDP
+static inline bool nvme_tcp_ddp_ddgst_ok(struct nvme_tcp_queue *queue)
+{
+	return queue->ddgst_valid;
+}
+
+static inline void nvme_tcp_ddp_ddgst_update(struct nvme_tcp_queue *queue,
+					     struct sk_buff *skb)
+{
+	if (queue->ddgst_valid)
+#ifdef CONFIG_TCP_DDP_CRC
+		queue->ddgst_valid = skb->ddp_crc;
+#else
+		queue->ddgst_valid = false;
+#endif
+}
+
+static int nvme_tcp_req_map_sg(struct nvme_tcp_request *req, struct request *rq)
+{
+	int ret;
+
+	req->ddp.sg_table.sgl = req->ddp.first_sgl;
+	ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
+				     req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
+	if (ret)
+		return -ENOMEM;
+	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
+	return 0;
+}
+
+static void nvme_tcp_ddp_ddgst_recalc(struct ahash_request *hash,
+				      struct request *rq)
+{
+	struct nvme_tcp_request *req;
+
+	if (!rq)
+		return;
+
+	req = blk_mq_rq_to_pdu(rq);
+
+	if (!req->offloaded && nvme_tcp_req_map_sg(req, rq))
+		return;
+
+	crypto_ahash_init(hash);
+	req->ddp.sg_table.sgl = req->ddp.first_sgl;
+	ahash_request_set_crypt(hash, req->ddp.sg_table.sgl, NULL,
+				le32_to_cpu(req->data_len));
+	crypto_ahash_update(hash);
+}
+
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 
 static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
 static void nvme_tcp_ddp_teardown_done(void *ddp_ctx);
@@ -290,12 +341,9 @@ int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
 	}
 
 	req->ddp.command_id = command_id;
-	req->ddp.sg_table.sgl = req->ddp.first_sgl;
-	ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
-				     req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
+	ret = nvme_tcp_req_map_sg(req, rq);
 	if (ret)
 		return -ENOMEM;
-	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
 
 	ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
 						 queue->sock->sk,
@@ -317,7 +365,7 @@ int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 		return -ENODEV;
 	}
 
-	if (!(netdev->features & NETIF_F_HW_TCP_DDP)) {
+	if (!(netdev->features & (NETIF_F_HW_TCP_DDP | NETIF_F_HW_TCP_DDP_CRC_RX))) {
 		dev_put(netdev);
 		return -EOPNOTSUPP;
 	}
@@ -345,6 +393,9 @@ int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 	if (netdev->features & NETIF_F_HW_TCP_DDP)
 		set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
 
+	if (netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+		set_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags);
+
 	return ret;
 }
 
@@ -376,7 +427,7 @@ int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
 		return -ENODEV;
 	}
 
-	if (netdev->features & NETIF_F_HW_TCP_DDP &&
+	if ((netdev->features & (NETIF_F_HW_TCP_DDP | NETIF_F_HW_TCP_DDP_CRC_RX)) &&
 	    netdev->tcp_ddp_ops &&
 	    netdev->tcp_ddp_ops->tcp_ddp_limits)
 		ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
@@ -739,6 +790,7 @@ static void nvme_tcp_init_recv_ctx(struct nvme_tcp_queue *queue)
 	queue->pdu_offset = 0;
 	queue->data_remaining = -1;
 	queue->ddgst_remaining = 0;
+	queue->ddgst_valid = true;
 }
 
 static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
@@ -919,7 +971,8 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 
 	u64 pdu_seq = TCP_SKB_CB(skb)->seq + *offset - queue->pdu_offset;
 
-	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) ||
+	    test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
 		nvme_tcp_resync_response(queue, pdu_seq);
 
 	ret = skb_copy_bits(skb, *offset,
@@ -988,6 +1041,8 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 	struct nvme_tcp_request *req;
 	struct request *rq;
 
+	if (queue->data_digest && test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
+		nvme_tcp_ddp_ddgst_update(queue, skb);
 	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
 	if (!rq) {
 		dev_err(queue->ctrl->ctrl.device,
@@ -1025,15 +1080,17 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 		recv_len = min_t(size_t, recv_len,
 				iov_iter_count(&req->iter));
 
-		if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
-			if (queue->data_digest)
+		if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) { 
+			if (queue->data_digest &&
+			    !test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
 				ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset,
 						&req->iter, recv_len, queue->rcv_hash);
 			else
 				ret = skb_ddp_copy_datagram_iter(skb, *offset,
 						&req->iter, recv_len);
 		} else {
-			if (queue->data_digest)
+			if (queue->data_digest &&
+			    !test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
 				ret = skb_copy_and_hash_datagram_iter(skb, *offset,
 						&req->iter, recv_len, queue->rcv_hash);
 			else
@@ -1055,7 +1112,6 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 
 	if (!queue->data_remaining) {
 		if (queue->data_digest) {
-			nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
 			queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH;
 		} else {
 			if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
@@ -1076,8 +1132,12 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
 	char *ddgst = (char *)&queue->recv_ddgst;
 	size_t recv_len = min_t(size_t, *len, queue->ddgst_remaining);
 	off_t off = NVME_TCP_DIGEST_LENGTH - queue->ddgst_remaining;
+	bool offload_fail, offload_en;
+	struct request *rq = NULL;
 	int ret;
 
+	if (test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
+		nvme_tcp_ddp_ddgst_update(queue, skb);
 	ret = skb_copy_bits(skb, *offset, &ddgst[off], recv_len);
 	if (unlikely(ret))
 		return ret;
@@ -1088,17 +1148,29 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
 	if (queue->ddgst_remaining)
 		return 0;
 
-	if (queue->recv_ddgst != queue->exp_ddgst) {
-		dev_err(queue->ctrl->ctrl.device,
-			"data digest error: recv %#x expected %#x\n",
-			le32_to_cpu(queue->recv_ddgst),
-			le32_to_cpu(queue->exp_ddgst));
-		return -EIO;
+	offload_fail = !nvme_tcp_ddp_ddgst_ok(queue);
+	offload_en = test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags);
+	if (!offload_en || offload_fail) {
+		if (offload_en && offload_fail) { // software-fallback
+			rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
+					      pdu->command_id);
+			nvme_tcp_ddp_ddgst_recalc(queue->rcv_hash, rq);
+		}
+
+		nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
+		if (queue->recv_ddgst != queue->exp_ddgst) {
+			dev_err(queue->ctrl->ctrl.device,
+				"data digest error: recv %#x expected %#x\n",
+				le32_to_cpu(queue->recv_ddgst),
+				le32_to_cpu(queue->exp_ddgst));
+			return -EIO;
+		}
 	}
 
 	if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
-		struct request *rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
-						pdu->command_id);
+		if (!rq)
+			rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
+					      pdu->command_id);
 
 		nvme_tcp_end_request(rq, NVME_SC_SUCCESS);
 		queue->nr_cqe++;
@@ -1841,8 +1913,10 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
 	nvme_tcp_restore_sock_calls(queue);
 	cancel_work_sync(&queue->io_work);
 
-	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) ||
+	    test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
 		nvme_tcp_unoffload_socket(queue);
+
 }
 
 static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
@@ -1970,8 +2044,6 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
 {
 	int ret;
 
-	to_tcp_ctrl(ctrl)->offloading_netdev = NULL;
-
 	ret = nvme_tcp_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
 	if (ret)
 		return ret;
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:08 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:08 +0200
Subject: [PATCH v3 net-next  20/21] net/mlx5e: NVMEoTCP statistics
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-21-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

NVMEoTCP offload statistics includes both control and data path
statistic: counters for ndo, offloaded packets/bytes, dropped packets
and resync operation.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 23 +++++++++++-
 .../mlx5/core/en_accel/nvmeotcp_rxtx.c        | 16 ++++++++
 .../ethernet/mellanox/mlx5/core/en_stats.c    | 37 +++++++++++++++++++
 .../ethernet/mellanox/mlx5/core/en_stats.h    | 24 ++++++++++++
 4 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index 4cbd86b9ed42..c42eb14ee081 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -659,6 +659,11 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 	struct mlx5_core_dev *mdev = priv->mdev;
 	struct mlx5e_nvmeotcp_queue *queue;
 	int max_wqe_sz_cap, queue_id, err;
+	struct mlx5e_rq_stats *stats;
+	u32 channel_ix;
+
+	channel_ix = mlx5e_get_channel_ix_from_io_cpu(priv, config->io_cpu);
+	stats = &priv->channel_stats[channel_ix].rq;
 
 	if (tconfig->type != TCP_DDP_NVME) {
 		err = -EOPNOTSUPP;
@@ -686,8 +691,7 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 	queue->id = queue_id;
 	queue->dgst = config->dgst;
 	queue->pda = config->cpda;
-	queue->channel_ix = mlx5e_get_channel_ix_from_io_cpu(priv,
-							     config->io_cpu);
+	queue->channel_ix = channel_ix;
 	queue->size = config->queue_size;
 	max_wqe_sz_cap  = min_t(int, MAX_DS_VALUE * MLX5_SEND_WQE_DS,
 				MLX5_CAP_GEN(mdev, max_wqe_sz_sq) << OCTWORD_SHIFT);
@@ -707,6 +711,7 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 	if (err)
 		goto destroy_rx;
 
+	stats->nvmeotcp_queue_init++;
 	write_lock_bh(&sk->sk_callback_lock);
 	tcp_ddp_set_ctx(sk, queue);
 	write_unlock_bh(&sk->sk_callback_lock);
@@ -721,6 +726,7 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 free_queue:
 	kfree(queue);
 out:
+	stats->nvmeotcp_queue_init_fail++;
 	return err;
 }
 
@@ -731,11 +737,15 @@ mlx5e_nvmeotcp_queue_teardown(struct net_device *netdev,
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct mlx5_core_dev *mdev = priv->mdev;
 	struct mlx5e_nvmeotcp_queue *queue;
+	struct mlx5e_rq_stats *stats;
 
 	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
 
 	napi_synchronize(&priv->channels.c[queue->channel_ix]->napi);
 
+	stats = &priv->channel_stats[queue->channel_ix].rq;
+	stats->nvmeotcp_queue_teardown++;
+
 	WARN_ON(refcount_read(&queue->ref_count) != 1);
 	if (queue->zerocopy | queue->crc_rx)
 		mlx5e_nvmeotcp_destroy_rx(queue, mdev, queue->zerocopy);
@@ -757,6 +767,7 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct scatterlist *sg = ddp->sg_table.sgl;
 	struct mlx5e_nvmeotcp_queue *queue;
+	struct mlx5e_rq_stats *stats;
 	struct mlx5_core_dev *mdev;
 	int i, size = 0, count = 0;
 
@@ -778,6 +789,11 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	queue->ccid_table[ddp->command_id].ccid_gen++;
 	queue->ccid_table[ddp->command_id].sgl_length = count;
 
+	stats = &priv->channel_stats[queue->channel_ix].rq;
+	stats->nvmeotcp_ddp_setup++;
+	if (unlikely(mlx5e_nvmeotcp_post_klm_wqe(queue, KLM_UMR, ddp->command_id, count)))
+		stats->nvmeotcp_ddp_setup_fail++;
+
 	return 0;
 }
 
@@ -818,6 +834,7 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 	struct mlx5e_nvmeotcp_queue *queue;
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct nvmeotcp_queue_entry *q_entry;
+	struct mlx5e_rq_stats *stats;
 
 	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
 	q_entry  = &queue->ccid_table[ddp->command_id];
@@ -827,6 +844,8 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 	q_entry->queue = queue;
 
 	mlx5e_nvmeotcp_post_klm_wqe(queue, KLM_UMR, ddp->command_id, 0);
+	stats = &priv->channel_stats[queue->channel_ix].rq;
+	stats->nvmeotcp_ddp_teardown++;
 
 	return 0;
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
index b16fcf051665..158f3798bf0c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
@@ -10,12 +10,16 @@ static void nvmeotcp_update_resync(struct mlx5e_nvmeotcp_queue *queue,
 				   struct mlx5e_cqe128 *cqe128)
 {
 	const struct tcp_ddp_ulp_ops *ulp_ops;
+	struct mlx5e_rq_stats *stats;
 	u32 seq;
 
 	seq = be32_to_cpu(cqe128->resync_tcp_sn);
 	ulp_ops = inet_csk(queue->sk)->icsk_ulp_ddp_ops;
 	if (ulp_ops && ulp_ops->resync_request)
 		ulp_ops->resync_request(queue->sk, seq, TCP_DDP_RESYNC_REQ);
+
+	stats = queue->priv->channels.c[queue->channel_ix]->rq.stats;
+	stats->nvmeotcp_resync++;
 }
 
 static void mlx5e_nvmeotcp_advance_sgl_iter(struct mlx5e_nvmeotcp_queue *queue)
@@ -50,10 +54,13 @@ mlx5_nvmeotcp_add_tail_nonlinear(struct mlx5e_nvmeotcp_queue *queue,
 				 int org_nr_frags, int frag_index)
 {
 	struct mlx5e_priv *priv = queue->priv;
+	struct mlx5e_rq_stats *stats;
 
 	while (org_nr_frags != frag_index) {
 		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
 			dev_kfree_skb_any(skb);
+			stats = priv->channels.c[queue->channel_ix]->rq.stats;
+			stats->nvmeotcp_drop++;
 			return NULL;
 		}
 		skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
@@ -72,9 +79,12 @@ mlx5_nvmeotcp_add_tail(struct mlx5e_nvmeotcp_queue *queue, struct sk_buff *skb,
 		       int offset, int len)
 {
 	struct mlx5e_priv *priv = queue->priv;
+	struct mlx5e_rq_stats *stats;
 
 	if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
 		dev_kfree_skb_any(skb);
+		stats = priv->channels.c[queue->channel_ix]->rq.stats;
+		stats->nvmeotcp_drop++;
 		return NULL;
 	}
 	skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
@@ -135,6 +145,7 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 	skb_frag_t org_frags[MAX_SKB_FRAGS];
 	struct mlx5e_nvmeotcp_queue *queue;
 	struct nvmeotcp_queue_entry *nqe;
+	struct mlx5e_rq_stats *stats;
 	int org_nr_frags, frag_index;
 	struct mlx5e_cqe128 *cqe128;
 	u32 queue_id;
@@ -172,6 +183,8 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 		return skb;
 	}
 
+	stats = priv->channels.c[queue->channel_ix]->rq.stats;
+
 	/* cc ddp from cqe */
 	ccid = be16_to_cpu(cqe128->ccid);
 	ccoff = be32_to_cpu(cqe128->ccoff);
@@ -214,6 +227,7 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 	while (to_copy < cclen) {
 		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
 			dev_kfree_skb_any(skb);
+			stats->nvmeotcp_drop++;
 			mlx5e_nvmeotcp_put_queue(queue);
 			return NULL;
 		}
@@ -243,6 +257,8 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 							       frag_index);
 	}
 
+	stats->nvmeotcp_offload_packets++;
+	stats->nvmeotcp_offload_bytes += cclen;
 	mlx5e_nvmeotcp_put_queue(queue);
 	return skb;
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index 92c5b81427b9..353662f3fc5f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -34,6 +34,7 @@
 #include "en.h"
 #include "en_accel/tls.h"
 #include "en_accel/en_accel.h"
+#include "en_accel/nvmeotcp.h"
 
 static unsigned int stats_grps_num(struct mlx5e_priv *priv)
 {
@@ -189,6 +190,18 @@ static const struct counter_desc sw_stats_desc[] = {
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_resync_res_ok) },
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_resync_res_skip) },
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_err) },
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_queue_init) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_queue_init_fail) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_queue_teardown) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_ddp_setup) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_ddp_setup_fail) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_ddp_teardown) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_drop) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_resync) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_offload_packets) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_offload_bytes) },
 #endif
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, ch_events) },
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, ch_poll) },
@@ -352,6 +365,18 @@ static void mlx5e_stats_grp_sw_update_stats_rq_stats(struct mlx5e_sw_stats *s,
 	s->rx_tls_resync_res_skip     += rq_stats->tls_resync_res_skip;
 	s->rx_tls_err                 += rq_stats->tls_err;
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	s->rx_nvmeotcp_queue_init      += rq_stats->nvmeotcp_queue_init;
+	s->rx_nvmeotcp_queue_init_fail += rq_stats->nvmeotcp_queue_init_fail;
+	s->rx_nvmeotcp_queue_teardown  += rq_stats->nvmeotcp_queue_teardown;
+	s->rx_nvmeotcp_ddp_setup       += rq_stats->nvmeotcp_ddp_setup;
+	s->rx_nvmeotcp_ddp_setup_fail  += rq_stats->nvmeotcp_ddp_setup_fail;
+	s->rx_nvmeotcp_ddp_teardown    += rq_stats->nvmeotcp_ddp_teardown;
+	s->rx_nvmeotcp_drop            += rq_stats->nvmeotcp_drop;
+	s->rx_nvmeotcp_resync          += rq_stats->nvmeotcp_resync;
+	s->rx_nvmeotcp_offload_packets += rq_stats->nvmeotcp_offload_packets;
+	s->rx_nvmeotcp_offload_bytes   += rq_stats->nvmeotcp_offload_bytes;
+#endif
 }
 
 static void mlx5e_stats_grp_sw_update_stats_ch_stats(struct mlx5e_sw_stats *s,
@@ -1632,6 +1657,18 @@ static const struct counter_desc rq_stats_desc[] = {
 	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_resync_res_skip) },
 	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_err) },
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_queue_init) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_queue_init_fail) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_queue_teardown) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_ddp_setup) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_ddp_setup_fail) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_ddp_teardown) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_drop) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_resync) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_offload_packets) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_offload_bytes) },
+#endif
 };
 
 static const struct counter_desc sq_stats_desc[] = {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
index 93c41312fb03..674cee2a884d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
@@ -181,6 +181,18 @@ struct mlx5e_sw_stats {
 	u64 rx_congst_umr;
 	u64 rx_arfs_err;
 	u64 rx_recover;
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	u64 rx_nvmeotcp_queue_init;
+	u64 rx_nvmeotcp_queue_init_fail;
+	u64 rx_nvmeotcp_queue_teardown;
+	u64 rx_nvmeotcp_ddp_setup;
+	u64 rx_nvmeotcp_ddp_setup_fail;
+	u64 rx_nvmeotcp_ddp_teardown;
+	u64 rx_nvmeotcp_drop;
+	u64 rx_nvmeotcp_resync;
+	u64 rx_nvmeotcp_offload_packets;
+	u64 rx_nvmeotcp_offload_bytes;
+#endif
 	u64 ch_events;
 	u64 ch_poll;
 	u64 ch_arm;
@@ -344,6 +356,18 @@ struct mlx5e_rq_stats {
 	u64 tls_resync_res_skip;
 	u64 tls_err;
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	u64 nvmeotcp_queue_init;
+	u64 nvmeotcp_queue_init_fail;
+	u64 nvmeotcp_queue_teardown;
+	u64 nvmeotcp_ddp_setup;
+	u64 nvmeotcp_ddp_setup_fail;
+	u64 nvmeotcp_ddp_teardown;
+	u64 nvmeotcp_drop;
+	u64 nvmeotcp_resync;
+	u64 nvmeotcp_offload_packets;
+	u64 nvmeotcp_offload_bytes;
+#endif
 };
 
 struct mlx5e_sq_stats {
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:06 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:06 +0200
Subject: [PATCH v3 net-next  18/21] net/mlx5e: NVMEoTCP ddp setup and resync
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-19-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

NVMEoTCP offload uses buffer registration for every NVME request to
perform direct data placement, The registration is done via KLM UMR
WQE's.  The driver resync handler advertise the software resync response
via static params WQE.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 33 +++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index f30bfcb43701..4cbd86b9ed42 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -754,6 +754,30 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 			 struct sock *sk,
 			 struct tcp_ddp_io *ddp)
 {
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct scatterlist *sg = ddp->sg_table.sgl;
+	struct mlx5e_nvmeotcp_queue *queue;
+	struct mlx5_core_dev *mdev;
+	int i, size = 0, count = 0;
+
+	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
+
+	mdev = queue->priv->mdev;
+	count = dma_map_sg(mdev->device, ddp->sg_table.sgl, ddp->nents,
+			   DMA_FROM_DEVICE);
+
+	if (WARN_ON(count > mlx5e_get_max_sgl(mdev)))
+		return -ENOSPC;
+
+	for (i = 0; i < count; i++)
+		size += sg[i].length;
+
+	queue->ccid_table[ddp->command_id].size = size;
+	queue->ccid_table[ddp->command_id].ddp = ddp;
+	queue->ccid_table[ddp->command_id].sgl = sg;
+	queue->ccid_table[ddp->command_id].ccid_gen++;
+	queue->ccid_table[ddp->command_id].sgl_length = count;
+
 	return 0;
 }
 
@@ -791,11 +815,11 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 			    struct tcp_ddp_io *ddp,
 			    void *ddp_ctx)
 {
-	struct mlx5e_nvmeotcp_queue *queue =
-		(struct mlx5e_nvmeotcp_queue *)tcp_ddp_get_ctx(sk);
+	struct mlx5e_nvmeotcp_queue *queue;
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct nvmeotcp_queue_entry *q_entry;
 
+	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
 	q_entry  = &queue->ccid_table[ddp->command_id];
 	WARN_ON(q_entry->sgl_length == 0);
 
@@ -811,6 +835,11 @@ static void
 mlx5e_nvmeotcp_dev_resync(struct net_device *netdev,
 			  struct sock *sk, u32 seq)
 {
+	struct mlx5e_nvmeotcp_queue *queue =
+		container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
+
+	queue->after_resync_cqe = 1;
+	mlx5e_nvmeotcp_rx_post_static_params_wqe(queue, seq);
 }
 
 static const struct tcp_ddp_dev_ops mlx5e_nvmeotcp_ops = {
-- 
2.24.1



From borisp at mellanox.com  Mon Feb  1 05:05:07 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Mon,  1 Feb 2021 12:05:07 +0200
Subject: [PATCH v3 net-next  19/21] net/mlx5e: NVMEoTCP,
 data-path for DDP+CRC offload
In-Reply-To: <20210201100509.27351-1-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
Message-ID: <20210201100509.27351-20-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

This patch implements the data-path for direct data placement (DDP)
and CRC offloads. NVMEoTCP DDP constructs an SKB from each CQE, while
pointing at NVME destination buffers. In turn, this enables the offload,
as the NVMe-TCP layer will skip the copy when src == dst.

Additionally, this patch adds support for CRC offload.
HW will report CRC offload only if it has not encountered an error
in the received packet. We pass this indication in skb->ddp_crc
up the stack to NVMe-TCP to skip computing the CRC if all
corresponding SKBs were verified by HW.

This patch also handles context resynchronization requests made by
NIC HW. The resync request is passed to the NVMe-TCP layer
to be handled at a later point in time.

Finally, we also use the skb->ddp_crc bit to avoid skb_condense.
This is critical as every SKB that uses DDP has a hole that fits
perfectly with skb_condense's policy, but filling this hole is
counter-productive as the data there already resides in its
destination buffer.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   1 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.c   |   1 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.h   |   1 +
 .../mlx5/core/en_accel/nvmeotcp_rxtx.c        | 248 ++++++++++++++++++
 .../mlx5/core/en_accel/nvmeotcp_rxtx.h        |  43 +++
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  41 ++-
 7 files changed, 330 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index 020fe2478fd3..deff6b691619 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -98,4 +98,4 @@ mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o
 #
 mlx5_core-$(CONFIG_MLX5_SF_MANAGER) += sf/cmd.o sf/hw_table.o sf/devlink.o
 
-mlx5_core-$(CONFIG_MLX5_EN_NVMEOTCP) += en_accel/fs_tcp.o en_accel/nvmeotcp.o
+mlx5_core-$(CONFIG_MLX5_EN_NVMEOTCP) += en_accel/fs_tcp.o en_accel/nvmeotcp.o en_accel/nvmeotcp_rxtx.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index f282c0f6cdab..6853ffafbf7d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -577,6 +577,7 @@ struct mlx5e_rq;
 typedef void (*mlx5e_fp_handle_rx_cqe)(struct mlx5e_rq*, struct mlx5_cqe64*);
 typedef struct sk_buff *
 (*mlx5e_fp_skb_from_cqe_mpwrq)(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+			       struct mlx5_cqe64 *cqe,
 			       u16 cqe_bcnt, u32 head_offset, u32 page_idx);
 typedef struct sk_buff *
 (*mlx5e_fp_skb_from_cqe)(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
index 8e7b877d8a12..9a6fbd1b1c34 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
@@ -25,6 +25,7 @@ static struct sk_buff *mlx5e_xsk_construct_skb(struct mlx5e_rq *rq, void *data,
 
 struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq,
 						    struct mlx5e_mpw_info *wi,
+						    struct mlx5_cqe64 *cqe,
 						    u16 cqe_bcnt,
 						    u32 head_offset,
 						    u32 page_idx)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
index 7f88ccf67fdd..112c5b3ec165 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
@@ -11,6 +11,7 @@
 
 struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq,
 						    struct mlx5e_mpw_info *wi,
+						    struct mlx5_cqe64 *cqe,
 						    u16 cqe_bcnt,
 						    u32 head_offset,
 						    u32 page_idx);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
new file mode 100644
index 000000000000..b16fcf051665
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
@@ -0,0 +1,248 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2021 Mellanox Technologies. */
+
+#include "en_accel/nvmeotcp_rxtx.h"
+#include "en_accel/nvmeotcp.h"
+#include <linux/mlx5/mlx5_ifc.h>
+
+#define	MLX5E_TC_FLOW_ID_MASK  0x00ffffff
+static void nvmeotcp_update_resync(struct mlx5e_nvmeotcp_queue *queue,
+				   struct mlx5e_cqe128 *cqe128)
+{
+	const struct tcp_ddp_ulp_ops *ulp_ops;
+	u32 seq;
+
+	seq = be32_to_cpu(cqe128->resync_tcp_sn);
+	ulp_ops = inet_csk(queue->sk)->icsk_ulp_ddp_ops;
+	if (ulp_ops && ulp_ops->resync_request)
+		ulp_ops->resync_request(queue->sk, seq, TCP_DDP_RESYNC_REQ);
+}
+
+static void mlx5e_nvmeotcp_advance_sgl_iter(struct mlx5e_nvmeotcp_queue *queue)
+{
+	struct nvmeotcp_queue_entry *nqe = &queue->ccid_table[queue->ccid];
+
+	queue->ccoff += nqe->sgl[queue->ccsglidx].length;
+	queue->ccoff_inner = 0;
+	queue->ccsglidx++;
+}
+
+static inline void
+mlx5e_nvmeotcp_add_skb_frag(struct net_device *netdev, struct sk_buff *skb,
+			    struct mlx5e_nvmeotcp_queue *queue,
+			    struct nvmeotcp_queue_entry *nqe, u32 fragsz)
+{
+	dma_sync_single_for_cpu(&netdev->dev,
+				nqe->sgl[queue->ccsglidx].offset + queue->ccoff_inner,
+				fragsz, DMA_FROM_DEVICE);
+	page_ref_inc(compound_head(sg_page(&nqe->sgl[queue->ccsglidx])));
+	// XXX: consider reducing the truesize, as no new memory is consumed
+	skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+			sg_page(&nqe->sgl[queue->ccsglidx]),
+			nqe->sgl[queue->ccsglidx].offset + queue->ccoff_inner,
+			fragsz,
+			fragsz);
+}
+
+static struct sk_buff*
+mlx5_nvmeotcp_add_tail_nonlinear(struct mlx5e_nvmeotcp_queue *queue,
+				 struct sk_buff *skb, skb_frag_t *org_frags,
+				 int org_nr_frags, int frag_index)
+{
+	struct mlx5e_priv *priv = queue->priv;
+
+	while (org_nr_frags != frag_index) {
+		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
+			dev_kfree_skb_any(skb);
+			return NULL;
+		}
+		skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+				skb_frag_page(&org_frags[frag_index]),
+				skb_frag_off(&org_frags[frag_index]),
+				skb_frag_size(&org_frags[frag_index]),
+				skb_frag_size(&org_frags[frag_index]));
+		page_ref_inc(skb_frag_page(&org_frags[frag_index]));
+		frag_index++;
+	}
+	return skb;
+}
+
+static struct sk_buff*
+mlx5_nvmeotcp_add_tail(struct mlx5e_nvmeotcp_queue *queue, struct sk_buff *skb,
+		       int offset, int len)
+{
+	struct mlx5e_priv *priv = queue->priv;
+
+	if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
+		dev_kfree_skb_any(skb);
+		return NULL;
+	}
+	skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+			virt_to_page(skb->data),
+			offset,
+			len,
+			len);
+	page_ref_inc(virt_to_page(skb->data));
+	return skb;
+}
+
+static void mlx5_nvmeotcp_trim_nonlinear(struct sk_buff *skb,
+					 skb_frag_t *org_frags,
+					 int *frag_index,
+					 int remaining)
+{
+	unsigned int frag_size;
+	int nr_frags;
+
+	/* skip @remaining bytes in frags */
+	*frag_index = 0;
+	while (remaining) {
+		frag_size = skb_frag_size(&skb_shinfo(skb)->frags[*frag_index]);
+		if (frag_size > remaining) {
+			skb_frag_off_add(&skb_shinfo(skb)->frags[*frag_index],
+					 remaining);
+			skb_frag_size_sub(&skb_shinfo(skb)->frags[*frag_index],
+					  remaining);
+			remaining = 0;
+		} else {
+			remaining -= frag_size;
+			skb_frag_unref(skb, *frag_index);
+			*frag_index += 1;
+		}
+	}
+
+	/* save original frags for the tail and unref */
+	nr_frags = skb_shinfo(skb)->nr_frags;
+	memcpy(&org_frags[*frag_index], &skb_shinfo(skb)->frags[*frag_index],
+	       (nr_frags - *frag_index) * sizeof(skb_frag_t));
+	while (--nr_frags >= *frag_index)
+		skb_frag_unref(skb, nr_frags);
+
+	/* remove frags from skb */
+	skb_shinfo(skb)->nr_frags = 0;
+	skb->len -= skb->data_len;
+	skb->truesize -= skb->data_len;
+	skb->data_len = 0;
+}
+
+struct sk_buff*
+mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
+			     struct mlx5_cqe64 *cqe, u32 cqe_bcnt,
+			     bool linear)
+{
+	int ccoff, cclen, hlen, ccid, remaining, fragsz, to_copy = 0;
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	skb_frag_t org_frags[MAX_SKB_FRAGS];
+	struct mlx5e_nvmeotcp_queue *queue;
+	struct nvmeotcp_queue_entry *nqe;
+	int org_nr_frags, frag_index;
+	struct mlx5e_cqe128 *cqe128;
+	u32 queue_id;
+
+	queue_id = (be32_to_cpu(cqe->sop_drop_qpn) & MLX5E_TC_FLOW_ID_MASK);
+	queue = mlx5e_nvmeotcp_get_queue(priv->nvmeotcp, queue_id);
+	if (unlikely(!queue)) {
+		dev_kfree_skb_any(skb);
+		return NULL;
+	}
+
+	cqe128 = container_of(cqe, struct mlx5e_cqe128, cqe64);
+	if (cqe_is_nvmeotcp_resync(cqe)) {
+		nvmeotcp_update_resync(queue, cqe128);
+		mlx5e_nvmeotcp_put_queue(queue);
+		return skb;
+	}
+
+	/* If a resync occurred in the previous cqe,
+	 * the current cqe.crcvalid bit may not be valid,
+	 * so we will treat it as 0
+	 */
+	if (unlikely(queue->after_resync_cqe)) {
+		skb->ddp_crc = 0;
+		queue->after_resync_cqe = 0;
+	} else {
+		if (queue->crc_rx)
+			skb->ddp_crc = cqe_is_nvmeotcp_crcvalid(cqe);
+		else
+			skb->ddp_crc = cqe_is_nvmeotcp_zc(cqe);
+	}
+
+	if (!cqe_is_nvmeotcp_zc(cqe)) {
+		mlx5e_nvmeotcp_put_queue(queue);
+		return skb;
+	}
+
+	/* cc ddp from cqe */
+	ccid = be16_to_cpu(cqe128->ccid);
+	ccoff = be32_to_cpu(cqe128->ccoff);
+	cclen = be16_to_cpu(cqe128->cclen);
+	hlen  = be16_to_cpu(cqe128->hlen);
+
+	/* carve a hole in the skb for DDP data */
+	if (linear) {
+		skb_trim(skb, hlen);
+	} else {
+		org_nr_frags = skb_shinfo(skb)->nr_frags;
+		mlx5_nvmeotcp_trim_nonlinear(skb, org_frags, &frag_index,
+					     cclen);
+	}
+
+	nqe = &queue->ccid_table[ccid];
+
+	/* packet starts new ccid? */
+	if (queue->ccid != ccid || queue->ccid_gen != nqe->ccid_gen) {
+		queue->ccid = ccid;
+		queue->ccoff = 0;
+		queue->ccoff_inner = 0;
+		queue->ccsglidx = 0;
+		queue->ccid_gen = nqe->ccid_gen;
+	}
+
+	/* skip inside cc until the ccoff in the cqe */
+	while (queue->ccoff + queue->ccoff_inner < ccoff) {
+		remaining = nqe->sgl[queue->ccsglidx].length - queue->ccoff_inner;
+		fragsz = min_t(off_t, remaining,
+			       ccoff - (queue->ccoff + queue->ccoff_inner));
+
+		if (fragsz == remaining)
+			mlx5e_nvmeotcp_advance_sgl_iter(queue);
+		else
+			queue->ccoff_inner += fragsz;
+	}
+
+	/* adjust the skb according to the cqe cc */
+	while (to_copy < cclen) {
+		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
+			dev_kfree_skb_any(skb);
+			mlx5e_nvmeotcp_put_queue(queue);
+			return NULL;
+		}
+
+		remaining = nqe->sgl[queue->ccsglidx].length - queue->ccoff_inner;
+		fragsz = min_t(int, remaining, cclen - to_copy);
+
+		mlx5e_nvmeotcp_add_skb_frag(netdev, skb, queue, nqe, fragsz);
+		to_copy += fragsz;
+		if (fragsz == remaining)
+			mlx5e_nvmeotcp_advance_sgl_iter(queue);
+		else
+			queue->ccoff_inner += fragsz;
+	}
+
+	if (cqe_bcnt > hlen + cclen) {
+		remaining = cqe_bcnt - hlen - cclen;
+		if (linear)
+			skb = mlx5_nvmeotcp_add_tail(queue, skb,
+						     offset_in_page(skb->data) +
+								hlen + cclen,
+						     remaining);
+		else
+			skb = mlx5_nvmeotcp_add_tail_nonlinear(queue, skb,
+							       org_frags,
+							       org_nr_frags,
+							       frag_index);
+	}
+
+	mlx5e_nvmeotcp_put_queue(queue);
+	return skb;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h
new file mode 100644
index 000000000000..65456b46c33f
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021 Mellanox Technologies. */
+#ifndef __MLX5E_NVMEOTCP_RXTX_H__
+#define __MLX5E_NVMEOTCP_RXTX_H__
+
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+
+#include <linux/skbuff.h>
+#include "en.h"
+
+struct sk_buff*
+mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
+			     struct mlx5_cqe64 *cqe, u32 cqe_bcnt, bool linear);
+
+static inline int mlx5_nvmeotcp_get_headlen(struct mlx5_cqe64 *cqe, u32 cqe_bcnt)
+{
+	struct mlx5e_cqe128 *cqe128;
+
+	if (!cqe_is_nvmeotcp_zc(cqe) || cqe_is_nvmeotcp_resync(cqe))
+		return cqe_bcnt;
+
+	cqe128 = container_of(cqe, struct mlx5e_cqe128, cqe64);
+	return be16_to_cpu(cqe128->hlen);
+}
+
+#else
+static inline struct sk_buff*
+mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
+			     struct mlx5_cqe64 *cqe, u32 cqe_bcnt, bool linear)
+{ return skb; }
+
+static inline int mlx5_nvmeotcp_get_headlen(struct mlx5_cqe64 *cqe, u32 cqe_bcnt)
+{ return cqe_bcnt; }
+
+#endif /* CONFIG_MLX5_EN_NVMEOTCP */
+
+static inline u16 mlx5e_get_headlen_hint(struct mlx5_cqe64 *cqe, u32 cqe_bcnt)
+{
+	return min_t(u32, MLX5E_RX_MAX_HEAD, mlx5_nvmeotcp_get_headlen(cqe, cqe_bcnt));
+}
+
+
+#endif /* __MLX5E_NVMEOTCP_RXTX_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 7e044a211aa1..b4a3b94e894d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -48,6 +48,7 @@
 #include "en_accel/ipsec_rxtx.h"
 #include "en_accel/tls_rxtx.h"
 #include "en_accel/nvmeotcp.h"
+#include "en_accel/nvmeotcp_rxtx.h"
 #include "lib/clock.h"
 #include "en/xdp.h"
 #include "en/xsk/rx.h"
@@ -57,9 +58,11 @@
 
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				struct mlx5_cqe64 *cqe,
 				u16 cqe_bcnt, u32 head_offset, u32 page_idx);
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				   struct mlx5_cqe64 *cqe,
 				   u16 cqe_bcnt, u32 head_offset, u32 page_idx);
 static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe);
 static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe);
@@ -1185,6 +1188,12 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 	/* queue up for recycling/reuse */
 	page_ref_inc(di->page);
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, true);
+#endif
+
 	return skb;
 }
 
@@ -1193,8 +1202,8 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 			     struct mlx5e_wqe_frag_info *wi, u32 cqe_bcnt)
 {
 	struct mlx5e_rq_frag_info *frag_info = &rq->wqe.info.arr[0];
+	u16 headlen = mlx5e_get_headlen_hint(cqe, cqe_bcnt);
 	struct mlx5e_wqe_frag_info *head_wi = wi;
-	u16 headlen      = min_t(u32, MLX5E_RX_MAX_HEAD, cqe_bcnt);
 	u16 frag_headlen = headlen;
 	u16 byte_cnt     = cqe_bcnt - headlen;
 	struct sk_buff *skb;
@@ -1203,7 +1212,7 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 	 * might spread among multiple pages.
 	 */
 	skb = napi_alloc_skb(rq->cq.napi,
-			     ALIGN(MLX5E_RX_MAX_HEAD, sizeof(long)));
+			     ALIGN(headlen, sizeof(long)));
 	if (unlikely(!skb)) {
 		rq->stats->buff_alloc_err++;
 		return NULL;
@@ -1229,6 +1238,12 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 	skb->tail += headlen;
 	skb->len  += headlen;
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, false);
+#endif
+
 	return skb;
 }
 
@@ -1383,7 +1398,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_rep(struct mlx5e_rq *rq, struct mlx5_cqe64
 	skb = INDIRECT_CALL_2(rq->mpwqe.skb_from_cqe_mpwrq,
 			      mlx5e_skb_from_cqe_mpwrq_linear,
 			      mlx5e_skb_from_cqe_mpwrq_nonlinear,
-			      rq, wi, cqe_bcnt, head_offset, page_idx);
+			      rq, wi, cqe, cqe_bcnt, head_offset, page_idx);
 	if (!skb)
 		goto mpwrq_cqe_out;
 
@@ -1414,17 +1429,18 @@ const struct mlx5e_rx_handlers mlx5e_rx_handlers_rep = {
 
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				   struct mlx5_cqe64 *cqe,
 				   u16 cqe_bcnt, u32 head_offset, u32 page_idx)
 {
-	u16 headlen = min_t(u16, MLX5E_RX_MAX_HEAD, cqe_bcnt);
 	struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx];
+	u16 headlen = mlx5e_get_headlen_hint(cqe, cqe_bcnt);
 	u32 frag_offset    = head_offset + headlen;
 	u32 byte_cnt       = cqe_bcnt - headlen;
 	struct mlx5e_dma_info *head_di = di;
 	struct sk_buff *skb;
 
 	skb = napi_alloc_skb(rq->cq.napi,
-			     ALIGN(MLX5E_RX_MAX_HEAD, sizeof(long)));
+			     ALIGN(headlen, sizeof(long)));
 	if (unlikely(!skb)) {
 		rq->stats->buff_alloc_err++;
 		return NULL;
@@ -1455,11 +1471,18 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w
 	skb->tail += headlen;
 	skb->len  += headlen;
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, false);
+#endif
+
 	return skb;
 }
 
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				struct mlx5_cqe64 *cqe,
 				u16 cqe_bcnt, u32 head_offset, u32 page_idx)
 {
 	struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx];
@@ -1501,6 +1524,12 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
 	/* queue up for recycling/reuse */
 	page_ref_inc(di->page);
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, true);
+#endif
+
 	return skb;
 }
 
@@ -1539,7 +1568,7 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq
 	skb = INDIRECT_CALL_2(rq->mpwqe.skb_from_cqe_mpwrq,
 			      mlx5e_skb_from_cqe_mpwrq_linear,
 			      mlx5e_skb_from_cqe_mpwrq_nonlinear,
-			      rq, wi, cqe_bcnt, head_offset, page_idx);
+			      rq, wi, cqe, cqe_bcnt, head_offset, page_idx);
 	if (!skb)
 		goto mpwrq_cqe_out;
 
-- 
2.24.1



From hare at suse.de  Mon Feb  1 05:45:26 2021
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 1 Feb 2021 11:45:26 +0100
Subject: [PATCH v2] nvme-multipath: Early exit if no path is available
In-Reply-To: <bd37abd5-759d-efe2-fdcd-8b004a41c75a@huawei.com>
References: <20210127103033.15318-1-dwagner@suse.de>
 <db9baae0-547c-7ff4-8b2c-0b95f14be67c@huawei.com>
 <20210128075837.u5u56t23fq5gu6ou@beryllium.lan>
 <69575290-200e-b4a1-4269-c71e4c2cc37b@huawei.com>
 <20210128094004.erwnszjqcxlsi2kd@beryllium.lan>
 <ebb1d098-3ded-e592-4419-e905aabe824f@huawei.com>
 <675d3cf7-1ae8-adc5-b6d0-359fe10f6b23@grimberg.me>
 <59cd053e-46cb-0235-141f-4ce919c93f48@huawei.com>
 <65392653-6b03-9195-f686-5fe4b3290bd2@suse.de>
 <81b22bbf-4dd3-6161-e63a-9699690a4e4f@huawei.com>
 <715dd943-0587-be08-2840-e0948cf0bc62@suse.de>
 <eb131d8f-f009-42e7-105d-58b84060f0dd@huawei.com>
 <ac019690-7f02-d28c-ed58-bfc8c1d48879@suse.de>
 <6ceff3cb-c9e9-7e74-92f0-dd745987c943@huawei.com>
 <114751ac-1f7d-ce5e-12c5-7d6303bdb999@suse.de>
 <aebc6c2d-4711-95ba-daf2-1cd17fc6f0e7@huawei.com>
 <47a1b796-9d91-5947-4bac-dd8f397041a3@suse.de>
 <bd37abd5-759d-efe2-fdcd-8b004a41c75a@huawei.com>
Message-ID: <2752ed93-6bd5-1a13-0e05-b91e2dcc24e1@suse.de>

On 2/1/21 10:40 AM, Chao Leng wrote:
> 
> 
> On 2021/2/1 16:57, Hannes Reinecke wrote:
>> On 2/1/21 9:47 AM, Chao Leng wrote:
>>>
>>>
>>> On 2021/2/1 15:29, Hannes Reinecke wrote:[ .. ]
>>>> Urgh. Please, no. That is well impossible to debug.
>>>> Can you please open-code it to demonstrate where the difference to 
>>>> the current (and my fixed) versions is?
>>>> I'm still not clear where the problem is once we applied both patches.
>>> For example assume the list has three path, and all path is not 
>>> NVME_ANA_OPTIMIZED:
>>> head->next = ns1;
>>> ns1->next = ns2;
>>> ns2->next = head;
>>> old->next = ns2;
>>>
>> And this is where I have issues with.
>> Where does 'old' come from?
>> Clearly it was part of the list at one point; so what happened to it?
> I explained this earlier.
> In nvme_ns_remove, there is a hole between list_del_rcu and
> nvme_mpath_clear_current_path. If head->current_path is the "old", and
> the "old" is removing. The "old" is already removed from the list by
> list_del_rcu, but head->current_path is not clear to NULL by
> nvme_mpath_clear_current_path.
> Find path is race with nvme_ns_remove, use the "old" pass to
> nvme_round_robin_path to find path.

Ah. So this should be better:

@@ -202,10 +202,12 @@ static struct nvme_ns *__nvme_find_path(struct 
nvme_ns_head *head, int node)
  static struct nvme_ns *nvme_next_ns(struct nvme_ns_head *head,
                 struct nvme_ns *ns)
  {
-       ns = list_next_or_null_rcu(&head->list, &ns->siblings, struct 
nvme_ns,
-                       siblings);
-       if (ns)
-               return ns;
+       if (ns && !test_bit(NVME_NS_REMOVING, &ns->flags)) {
+               ns = list_next_or_null_rcu(&head->list, &ns->siblings,
+                                          struct nvme_ns, siblings);
+               if (ns)
+                       return ns;
+       }
         return list_first_or_null_rcu(&head->list, struct nvme_ns, 
siblings);
  }

The 'NVME_NS_REMOVING' bit is set before list_del_rcu() is called, so it 
should guard against the issue you mentioned.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From Jeff.Lien at wdc.com  Mon Feb  1 09:36:12 2021
From: Jeff.Lien at wdc.com (Jeffrey Lien)
Date: Mon, 1 Feb 2021 14:36:12 +0000
Subject: Buffer I/O Errors from Zoned NVME devices
Message-ID: <BYAPR04MB56561E48A54530BC7F6DCF30EAB69@BYAPR04MB5656.namprd04.prod.outlook.com>

Christoph, Keith
We're seeing a lot of these Buffer I/O errors with our zoned nvme devices.  One of the FW developers looked into it and had the following explanation:
All these Reads are from the kernel during enumeration and for LBAs that are in last zone's hole hence expected to return boundary error which is getting logged by kernel.

[65281.936988] Buffer I/O error on dev nvme1n2, logical block 3800039296, async page read
[65281.937165] blk_update_request: I/O error, dev nvme1n2, sector 3800039297 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[65281.937166] Buffer I/O error on dev nvme1n2, logical block 3800039297, async page read
[65281.937335] blk_update_request: I/O error, dev nvme1n2, sector 3800039298 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[65281.937336] Buffer I/O error on dev nvme1n2, logical block 3800039298, async page read
[65281.937498] blk_update_request: I/O error, dev nvme1n2, sector 3800039299 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0

Are you aware of this issue and if so, do you have any recommendations on how to avoid or resolve?  

Thanks

Jeff Lien
eSSD Core SW Tools & Drivers

Western Digital
2900 37th St NW
Building 108-1
Rochester, MN 55901
Email:  Jeff.Lien at wdc.com
Office: +1-507-322-2416



From mlombard at redhat.com  Mon Feb  1 09:47:56 2021
From: mlombard at redhat.com (Maurizio Lombardi)
Date: Mon,  1 Feb 2021 15:47:56 +0100
Subject: [PATCH] nvmetcli documentation: fix typo
Message-ID: <20210201144756.37538-1-mlombard@redhat.com>

Signed-off-by: Maurizio Lombardi <mlombard at redhat.com>
---
 Documentation/nvmetcli.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/nvmetcli.txt b/Documentation/nvmetcli.txt
index 05a0344..7d6ffda 100644
--- a/Documentation/nvmetcli.txt
+++ b/Documentation/nvmetcli.txt
@@ -116,7 +116,7 @@ your devices and all dependent modules are loaded,
 and configfs is mounted on /sys/kernel/config
 using:
 
-	mount -t configs none /sys/kernel/config
+	mount -t configfs none /sys/kernel/config
 
 The following section walks through a configuration example.
 
-- 
2.29.2



From hch at lst.de  Mon Feb  1 12:24:38 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 1 Feb 2021 18:24:38 +0100
Subject: [PATCH 01/10] nvmet: zeroout id-ns buffer for invalid nsid
In-Reply-To: <20210201054138.34324-2-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-2-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201172438.GB12054@lst.de>

On Sun, Jan 31, 2021 at 09:41:29PM -0800, Chaitanya Kulkarni wrote:
> According to spec :-
> "5.15.2.7 Identify Namespace data structure for an Allocated Namespace
> ID (CNS 11h) The Identify Namespace data structure (refer to Figure 245)
> is returned to the host for the namespace specified in the Namespace
> Identifier (NSID) field if it is an allocated NSID. If the specified
> namespace is an unallocated NSID then the controller returns a zero
> filled data structure."
> 
> Move call to nvmet_find_namespace() in nvmet_execute_identify_ns()
> above the id-ns buffer allocation since there is no point in allocating
> a buffer if namespace doesn't exist. Call nvmet_zero_sgl() when call to
> nvmet_find_namespace() fails and add an explicit comment to specify the
> reason for zeroing the buffer which is not common for the NVMe commands.
> Remove the done label and we can directly jump to out label from
> nvmet_find_namespace() error case.
> 
> Fixes: bffcd507780e ("nvmet: set right status on error in id-ns handler")
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> ---
>  drivers/nvme/target/admin-cmd.c | 22 ++++++++++++++--------
>  1 file changed, 14 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
> index 613a4d8feac1..8cc7bb25d10d 100644
> --- a/drivers/nvme/target/admin-cmd.c
> +++ b/drivers/nvme/target/admin-cmd.c
> @@ -476,19 +476,25 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
>  		goto out;
>  	}
>  
> +	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
> +	if (!req->ns) {
> +		status = NVME_SC_INVALID_NS;
> +		/*
> +		 * According to spec : If the specified namespace is
> +		 * an unallocated NSID then the controller returns a zero filled
> +		 * data structure. Also don't override the error status as invalid
> +		 * namespace takes priority over the failed zeroout buffer case.
> +		 */
> +		nvmet_zero_sgl(req, 0, sizeof(*id));
> +		goto out;
> +	}
> +
>  	id = kzalloc(sizeof(*id), GFP_KERNEL);
>  	if (!id) {
>  		status = NVME_SC_INTERNAL;
>  		goto out;
>  	}
>  
> -	/* return an all zeroed buffer if we can't find an active namespace */
> -	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
> -	if (!req->ns) {
> -		status = NVME_SC_INVALID_NS;
> -		goto done;

I think all we need to do is to remove the status assignment here.
While you're patch avoids the memory allocation for this case, it isn't
really the fast path so I'd rather avoid the extra code.


From hch at lst.de  Mon Feb  1 12:27:30 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 1 Feb 2021 18:27:30 +0100
Subject: [PATCH 03/10] nvmet: set error in nvmet_find_namespace()
In-Reply-To: <20210201054138.34324-4-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-4-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201172730.GC12054@lst.de>

This does look like a nice cleanup, but I think we also need to
rename the function now that it does not return the nvmet_ns
but only sets it in the nvmet_req structure.

What about nvmet_req_find_namespace?


From hch at lst.de  Mon Feb  1 12:28:10 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 1 Feb 2021 18:28:10 +0100
Subject: [PATCH 04/10] nvmet: remove nsid param from nvmet_find_namespace()
In-Reply-To: <20210201054138.34324-5-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-5-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201172810.GD12054@lst.de>

On Sun, Jan 31, 2021 at 09:41:32PM -0800, Chaitanya Kulkarni wrote:
> The nvmet_find_namespace() takes nsid parameter which is from NVMe
> commands structures such as get_log_page, identify, rw and common. All
> these commands have same offset for the nsid field.
> 
> Derive nsid from req (req->cmd->common.nsid) and remove the extra
> parameter from the nvmet_find_namespace().

I'd merge this into the previous one as that totally changes the
calling conventions anyway.  Otherwise this looks good to me.


From hch at lst.de  Mon Feb  1 12:29:21 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 1 Feb 2021 18:29:21 +0100
Subject: [PATCH 06/10] nvmet: add helper to report invalid opcode
In-Reply-To: <20210201054138.34324-7-chaitanya.kulkarni@wdc.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-7-chaitanya.kulkarni@wdc.com>
Message-ID: <20210201172921.GE12054@lst.de>

On Sun, Jan 31, 2021 at 09:41:34PM -0800, Chaitanya Kulkarni wrote:
> +static inline u16 nvmet_report_invalid_opcode(struct nvmet_req *req)
> +{
> +	char *backend;
> +
> +	if (req->ns->bdev)
> +		backend = "bdev";
> +	else if (req->ns->file)
> +		backend = "file";
> +	else if (nvmet_req_passthru_ctrl(req))
> +		backend = "passthru";
> +	else
> +		backend = "unknown";
> +
> +	pr_err("unhandled cmd %d on qid %d for %s backend\n",
> +		req->cmd->common.opcode, req->sq->qid, backend);

I think we can just skip reporting the backend, as this doesn't
really add much value.  In fact given that this can be triggered
remotely it should probably be downgraded to a pr_debug.


From hch at lst.de  Mon Feb  1 12:35:48 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 1 Feb 2021 18:35:48 +0100
Subject: [PATCH v3 net-next  01/21] iov_iter: Introduce new procedures
 for copy to iter/pages
In-Reply-To: <20210201100509.27351-2-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-2-borisp@mellanox.com>
Message-ID: <20210201173548.GA12960@lst.de>

On Mon, Feb 01, 2021 at 12:04:49PM +0200, Boris Pismenny wrote:
> +static __always_inline __must_check
> +size_t ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
> +{
> +	if (unlikely(!check_copy_size(addr, bytes, true)))
> +		return 0;
> +	else
> +		return _ddp_copy_to_iter(addr, bytes, i);
> +}

No need for the else after a return, and the normal kernel convention
double underscores for magic internal functions.

But more importantly: does this belong into the generic header without
and comments what the ddp means and when it should be used?

> +static void ddp_memcpy_to_page(struct page *page, size_t offset, const char *from, size_t len)

Overly long line.  But we're also looking into generic helpers for
this kind of things, not sure if they made it to linux-next in the
meantime, but please check.

> +size_t _ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
> +{
> +	const char *from = addr;
> +	if (unlikely(iov_iter_is_pipe(i)))
> +		return copy_pipe_to_iter(addr, bytes, i);
> +	if (iter_is_iovec(i))
> +		might_fault();
> +	iterate_and_advance(i, bytes, v,
> +		copyout(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
> +		ddp_memcpy_to_page(v.bv_page, v.bv_offset,
> +				   (from += v.bv_len) - v.bv_len, v.bv_len),
> +		memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len)
> +		)
> +
> +	return bytes;
> +}

This bloats every kernel build, so please move it into a conditionally
built file.  And please document the whole thing.


From hch at lst.de  Mon Feb  1 12:37:19 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 1 Feb 2021 18:37:19 +0100
Subject: [PATCH v3 net-next  06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <20210201100509.27351-7-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-7-borisp@mellanox.com>
Message-ID: <20210201173719.GB12960@lst.de>

> +static
> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)

Please use the same coding style as the rest of the file, and not some
weirdo version.


From hch at lst.de  Mon Feb  1 12:37:44 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 1 Feb 2021 18:37:44 +0100
Subject: [PATCH v3 net-next  07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <20210201100509.27351-8-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-8-borisp@mellanox.com>
Message-ID: <20210201173744.GC12960@lst.de>

Given how much ddp code there is can you split it into a separate file?


From kbusch at kernel.org  Mon Feb  1 12:53:06 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 1 Feb 2021 09:53:06 -0800
Subject: Buffer I/O Errors from Zoned NVME devices
In-Reply-To: <BYAPR04MB56561E48A54530BC7F6DCF30EAB69@BYAPR04MB5656.namprd04.prod.outlook.com>
References: <BYAPR04MB56561E48A54530BC7F6DCF30EAB69@BYAPR04MB5656.namprd04.prod.outlook.com>
Message-ID: <20210201175306.GA2127465@dhcp-10-100-145-180.wdc.com>

On Mon, Feb 01, 2021 at 02:36:12PM +0000, Jeffrey Lien wrote:
> Christoph, Keith
> We're seeing a lot of these Buffer I/O errors with our zoned nvme devices.  One of the FW developers looked into it and had the following explanation:
> All these Reads are from the kernel during enumeration and for LBAs that are in last zone's hole hence expected to return boundary error which is getting logged by kernel.
> 
> [65281.936988] Buffer I/O error on dev nvme1n2, logical block 3800039296, async page read
> [65281.937165] blk_update_request: I/O error, dev nvme1n2, sector 3800039297 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> [65281.937166] Buffer I/O error on dev nvme1n2, logical block 3800039297, async page read
> [65281.937335] blk_update_request: I/O error, dev nvme1n2, sector 3800039298 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> [65281.937336] Buffer I/O error on dev nvme1n2, logical block 3800039298, async page read
> [65281.937498] blk_update_request: I/O error, dev nvme1n2, sector 3800039299 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> 
> Are you aware of this issue and if so, do you have any recommendations on how to avoid or resolve?  

Is this from the partition scanning? We don't partition zoned devices,
so I think we can skip it. Does the following resolve the issue?

---
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1a3cdc6b1036..fafd02ab3a46 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3925,6 +3925,9 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 		}
 	}
 
+	if (blk_queue_is_zoned(ns->queue))
+		disk->flags |= GENHD_FL_NO_PART_SCAN;
+
 	down_write(&ctrl->namespaces_rwsem);
 	list_add_tail(&ns->list, &ctrl->namespaces);
 	up_write(&ctrl->namespaces_rwsem);
--


From Martin.Belanger at dell.com  Mon Feb  1 12:53:48 2021
From: Martin.Belanger at dell.com (Belanger, Martin)
Date: Mon, 1 Feb 2021 17:53:48 +0000
Subject: How to submit bugs for the nvme kernel modules?
Message-ID: <SJ0PR19MB45442FC3EA568AD0D5FFFE9BF2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>

Hi,

How does one submit a bug against the nvme kernel modules. I found the GitHub for the user space nvme utilities (https://github.com/linux-nvme/), but I assume that's not the right place to file bugs against the kernel modules (or is it?).

Thanks,
Martin

From hch at lst.de  Mon Feb  1 13:02:57 2021
From: hch at lst.de (hch at lst.de)
Date: Mon, 1 Feb 2021 19:02:57 +0100
Subject: Buffer I/O Errors from Zoned NVME devices
In-Reply-To: <20210201175306.GA2127465@dhcp-10-100-145-180.wdc.com>
References: <BYAPR04MB56561E48A54530BC7F6DCF30EAB69@BYAPR04MB5656.namprd04.prod.outlook.com>
 <20210201175306.GA2127465@dhcp-10-100-145-180.wdc.com>
Message-ID: <20210201180257.GA14971@lst.de>

On Mon, Feb 01, 2021 at 09:53:06AM -0800, Keith Busch wrote:
> On Mon, Feb 01, 2021 at 02:36:12PM +0000, Jeffrey Lien wrote:
> > Christoph, Keith
> > We're seeing a lot of these Buffer I/O errors with our zoned nvme devices.  One of the FW developers looked into it and had the following explanation:
> > All these Reads are from the kernel during enumeration and for LBAs that are in last zone's hole hence expected to return boundary error which is getting logged by kernel.
> > 
> > [65281.936988] Buffer I/O error on dev nvme1n2, logical block 3800039296, async page read
> > [65281.937165] blk_update_request: I/O error, dev nvme1n2, sector 3800039297 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> > [65281.937166] Buffer I/O error on dev nvme1n2, logical block 3800039297, async page read
> > [65281.937335] blk_update_request: I/O error, dev nvme1n2, sector 3800039298 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> > [65281.937336] Buffer I/O error on dev nvme1n2, logical block 3800039298, async page read
> > [65281.937498] blk_update_request: I/O error, dev nvme1n2, sector 3800039299 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> > 
> > Are you aware of this issue and if so, do you have any recommendations on how to avoid or resolve?  
> 
> Is this from the partition scanning? We don't partition zoned devices,
> so I think we can skip it. Does the following resolve the issue?

We already have special zoned device handling in the partitioning code.

But NVMe should make sure to never span a zone boundary as we set the
chunk size to avoid that.

What kernel version is this?  Is CONFIG_BLK_DEV_ZONED enabled?


From kbusch at kernel.org  Mon Feb  1 13:06:31 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 1 Feb 2021 10:06:31 -0800
Subject: How to submit bugs for the nvme kernel modules?
In-Reply-To: <SJ0PR19MB45442FC3EA568AD0D5FFFE9BF2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45442FC3EA568AD0D5FFFE9BF2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <20210201180631.GB2127465@dhcp-10-100-145-180.wdc.com>

On Mon, Feb 01, 2021 at 05:53:48PM +0000, Belanger, Martin wrote:
> Hi,
> 
> How does one submit a bug against the nvme kernel modules. I found the
> GitHub for the user space nvme utilities
> (https://github.com/linux-nvme/), but I assume that's not the right
> place to file bugs against the kernel modules (or is it?).

You can send nvme kernel bug reports to this mailing list. Some people
also use https://bugzilla.kernel.org/.

The github link you found is just for user space tools and is not used
for kernel issues.


From jxgao at google.com  Mon Feb  1 13:30:14 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Mon,  1 Feb 2021 10:30:14 -0800
Subject: [PATCH V2 0/3] SWIOTLB: Preserve swiotlb map offset when needed.
Message-ID: <20210201183017.3339130-1-jxgao@google.com>

NVMe driver and other applications may depend on the data offset
to operate correctly. Currently when unaligned data is mapped via
SWIOTLB, the data is mapped as slab aligned with the SWIOTLB. This
patch adds an option to make sure the mapped data preserves its
offset of the orginal addrss.

Without the patch when creating xfs formatted disk on NVMe backends,
with swiotlb=force in kernel boot option, creates the following error:
meta-data=/dev/nvme2n1   isize=512    agcount=4, agsize=131072 blks
         =               sectsz=512   attr=2, projid32bit=1
         =               crc=1        finobt=1, sparse=0, rmapbt=0, refl
ink=0
data     =               bsize=4096   blocks=524288, imaxpct=25
         =               sunit=0      swidth=0 blks
naming   =version 2      bsize=4096   ascii-ci=0 ftype=1
log      =internal log   bsize=4096   blocks=2560, version=2
         =               sectsz=512   sunit=0 blks, lazy-count=1
realtime =none           extsz=4096   blocks=0, rtextents=0
mkfs.xfs: pwrite failed: Input/output error

Jianxiong Gao (3):
  Adding page_offset_mask to device_dma_parameters
  Add swiotlb offset preserving mapping when
    dma_dma_parameters->page_offset_mask is non zero.
  Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

 drivers/nvme/host/pci.c     |  4 ++++
 include/linux/device.h      |  1 +
 include/linux/dma-mapping.h | 17 +++++++++++++++++
 kernel/dma/swiotlb.c        | 16 +++++++++++++++-
 4 files changed, 37 insertions(+), 1 deletion(-)

-- 
2.27.0



From jxgao at google.com  Mon Feb  1 13:30:15 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Mon,  1 Feb 2021 10:30:15 -0800
Subject: [PATCH V2 1/3] Adding page_offset_mask to device_dma_parameters
In-Reply-To: <20210201183017.3339130-1-jxgao@google.com>
References: <20210201183017.3339130-1-jxgao@google.com>
Message-ID: <20210201183017.3339130-2-jxgao@google.com>

Some devices rely on the address offset in a page to function
correctly (NVMe driver as an example). These devices may use
a different page size than the Linux kernel. The address offset
has to be preserved upon mapping, and in order to do so, we
need to record the page_offset_mask first.

Signed-off-by: Jianxiong Gao <jxgao at google.com>
---
 include/linux/device.h      |  1 +
 include/linux/dma-mapping.h | 17 +++++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/include/linux/device.h b/include/linux/device.h
index 1779f90eeb4c..7960bf516dd7 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -291,6 +291,7 @@ struct device_dma_parameters {
 	 * sg limitations.
 	 */
 	unsigned int max_segment_size;
+	unsigned int min_align_mask;
 	unsigned long segment_boundary_mask;
 };
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 2e49996a8f39..27ec3cab8cbd 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -500,6 +500,23 @@ static inline int dma_set_seg_boundary(struct device *dev, unsigned long mask)
 	return -EIO;
 }
 
+static inline unsigned int dma_get_min_align_mask(struct device *dev)
+{
+	if (dev->dma_parms)
+		return dev->dma_parms->min_align_mask;
+	return 0;
+}
+
+static inline int dma_set_min_align_mask(struct device *dev,
+		unsigned int min_align_mask)
+{
+	if (dev->dma_parms) {
+		dev->dma_parms->min_align_mask = min_align_mask;
+		return 0;
+	}
+	return -EIO;
+}
+
 static inline int dma_get_cache_alignment(void)
 {
 #ifdef ARCH_DMA_MINALIGN
-- 
2.27.0



From jxgao at google.com  Mon Feb  1 13:30:16 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Mon,  1 Feb 2021 10:30:16 -0800
Subject: [PATCH V2 2/3] Add swiotlb offset preserving mapping when
 dma_dma_parameters->page_offset_mask is non zero.
In-Reply-To: <20210201183017.3339130-1-jxgao@google.com>
References: <20210201183017.3339130-1-jxgao@google.com>
Message-ID: <20210201183017.3339130-3-jxgao@google.com>

For devices that need to preserve address offset on mapping through
swiotlb, this patch adds offset preserving based on page_offset_mask
and keeps the offset if the mask is non zero. This is needed for
device drivers like NVMe.

Signed-off-by: Jianxiong Gao <jxgao at google.com>
---
 kernel/dma/swiotlb.c | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 7c42df6e6100..eeb640df35f3 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -468,7 +468,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 	dma_addr_t tbl_dma_addr = phys_to_dma_unencrypted(hwdev, io_tlb_start);
 	unsigned long flags;
 	phys_addr_t tlb_addr;
-	unsigned int nslots, stride, index, wrap;
+	unsigned int nslots, stride, index, wrap, min_align_mask, page_offset;
 	int i;
 	unsigned long mask;
 	unsigned long offset_slots;
@@ -500,12 +500,16 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 		    ? ALIGN(mask + 1, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT
 		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
 
+	min_align_mask = dma_get_min_align_mask(hwdev);
+	page_offset = orig_addr & min_align_mask;
+	alloc_size += page_offset;
+
 	/*
 	 * For mappings greater than or equal to a page, we limit the stride
 	 * (and hence alignment) to a page size.
 	 */
 	nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
-	if (alloc_size >= PAGE_SIZE)
+	if ((alloc_size >= PAGE_SIZE) || (min_align_mask > (1 << IO_TLB_SHIFT)))
 		stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
 	else
 		stride = 1;
@@ -583,6 +587,11 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 	 */
 	for (i = 0; i < nslots; i++)
 		io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
+	/*
+	 * When keeping the offset of the original data, we need to advance
+	 * the tlb_addr by the offset of orig_addr.
+	 */
+	tlb_addr += page_offset;
 	if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
 	    (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
 		swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_TO_DEVICE);
@@ -598,7 +607,11 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 			      enum dma_data_direction dir, unsigned long attrs)
 {
 	unsigned long flags;
-	int i, count, nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+	unsigned int num_page_offset_slabs;
+	unsigned int min_align_mask = dma_get_min_align_mask(hwdev);
+	int i, count;
+	int nslots = ALIGN(alloc_size + (tlb_addr & min_align_mask),
+			1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
 	int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
 	phys_addr_t orig_addr = io_tlb_orig_addr[index];
 
@@ -610,6 +623,14 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 	    ((dir == DMA_FROM_DEVICE) || (dir == DMA_BIDIRECTIONAL)))
 		swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_FROM_DEVICE);
 
+	/*
+	 * When dma_get_min_align_mask is used, we may have padded more slabs
+	 * when padding exceeds one slab. We need to move index back to the
+	 * beginning of the padding.
+	 */
+	num_page_offset_slabs = (tlb_addr & min_align_mask) / (1 << IO_TLB_SHIFT);
+	index -= num_page_offset_slabs;
+
 	/*
 	 * Return the buffer to the free list by setting the corresponding
 	 * entries to indicate the number of contiguous entries available.
-- 
2.27.0



From jxgao at google.com  Mon Feb  1 13:30:17 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Mon,  1 Feb 2021 10:30:17 -0800
Subject: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to
 NVMe driver.
In-Reply-To: <20210201183017.3339130-1-jxgao@google.com>
References: <20210201183017.3339130-1-jxgao@google.com>
Message-ID: <20210201183017.3339130-4-jxgao@google.com>

NVMe driver relies on the address offset to function properly.
This patch adds the offset preserve mask to NVMe driver when mapping
via dma_map_sg_attrs and unmapping via nvme_unmap_sg. The mask
depends on the page size defined by CC.MPS register of NVMe
controller.

Signed-off-by: Jianxiong Gao <jxgao at google.com>
---
 drivers/nvme/host/pci.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b2042..30e45f7e0f75 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -580,12 +580,15 @@ static void nvme_free_sgls(struct nvme_dev *dev, struct request *req)
 static void nvme_unmap_sg(struct nvme_dev *dev, struct request *req)
 {
 	struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
-
+	if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1))
+		dev_warn(dev->dev, "dma_set_min_align_mask failed to set offset\n");
 	if (is_pci_p2pdma_page(sg_page(iod->sg)))
 		pci_p2pdma_unmap_sg(dev->dev, iod->sg, iod->nents,
 				    rq_dma_dir(req));
 	else
 		dma_unmap_sg(dev->dev, iod->sg, iod->nents, rq_dma_dir(req));
+	if (dma_set_min_align_mask(dev->dev, 0))
+		dev_warn(dev->dev, "dma_set_min_align_mask failed to reset offset\n");
 }
 
 static void nvme_unmap_data(struct nvme_dev *dev, struct request *req)
@@ -842,7 +845,7 @@ static blk_status_t nvme_map_data(struct nvme_dev *dev, struct request *req,
 {
 	struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
 	blk_status_t ret = BLK_STS_RESOURCE;
-	int nr_mapped;
+	int nr_mapped, offset_ret;
 
 	if (blk_rq_nr_phys_segments(req) == 1) {
 		struct bio_vec bv = req_bvec(req);
@@ -868,12 +871,24 @@ static blk_status_t nvme_map_data(struct nvme_dev *dev, struct request *req,
 	if (!iod->nents)
 		goto out_free_sg;
 
+	offset_ret = dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1);
+	if (offset_ret) {
+		dev_warn(dev->dev, "dma_set_min_align_mask failed to set offset\n");
+		goto out_free_sg;
+	}
+
 	if (is_pci_p2pdma_page(sg_page(iod->sg)))
 		nr_mapped = pci_p2pdma_map_sg_attrs(dev->dev, iod->sg,
 				iod->nents, rq_dma_dir(req), DMA_ATTR_NO_WARN);
 	else
 		nr_mapped = dma_map_sg_attrs(dev->dev, iod->sg, iod->nents,
 					     rq_dma_dir(req), DMA_ATTR_NO_WARN);
+
+	offset_ret = dma_set_min_align_mask(dev->dev, 0);
+	if (offset_ret) {
+		dev_warn(dev->dev, "dma_set_min_align_mask failed to reset offset\n");
+		goto out_free_sg;
+	}
 	if (!nr_mapped)
 		goto out_free_sg;
 
-- 
2.27.0



From andriy.shevchenko at linux.intel.com  Mon Feb  1 13:55:40 2021
From: andriy.shevchenko at linux.intel.com (Andy Shevchenko)
Date: Mon, 1 Feb 2021 20:55:40 +0200
Subject: [PATCH V2 3/3] Adding
 device_dma_parameters->offset_preserve_mask to NVMe driver.
In-Reply-To: <20210201183017.3339130-4-jxgao@google.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
Message-ID: <YBhOrK+kea7PNmXn@smile.fi.intel.com>

On Mon, Feb 01, 2021 at 10:30:17AM -0800, Jianxiong Gao wrote:
> NVMe driver relies on the address offset to function properly.
> This patch adds the offset preserve mask to NVMe driver when mapping
> via dma_map_sg_attrs and unmapping via nvme_unmap_sg. The mask
> depends on the page size defined by CC.MPS register of NVMe
> controller.

...

>  	if (is_pci_p2pdma_page(sg_page(iod->sg)))
>  		nr_mapped = pci_p2pdma_map_sg_attrs(dev->dev, iod->sg,
>  				iod->nents, rq_dma_dir(req), DMA_ATTR_NO_WARN);
>  	else
>  		nr_mapped = dma_map_sg_attrs(dev->dev, iod->sg, iod->nents,
>  					     rq_dma_dir(req), DMA_ATTR_NO_WARN);
> +
> +	offset_ret = dma_set_min_align_mask(dev->dev, 0);
> +	if (offset_ret) {
> +		dev_warn(dev->dev, "dma_set_min_align_mask failed to reset offset\n");
> +		goto out_free_sg;
> +	}

Seems like rebasing effect which makes empty line goes in the middle of some
other group of code lines.

>  	if (!nr_mapped)
>  		goto out_free_sg;

Perhaps it should be here?

-- 
With Best Regards,
Andy Shevchenko




From Martin.Belanger at dell.com  Mon Feb  1 14:04:39 2021
From: Martin.Belanger at dell.com (Belanger, Martin)
Date: Mon, 1 Feb 2021 19:04:39 +0000
Subject: nvme-tcp bricks my computer
Message-ID: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>

I'm running "nvme discover" over a TCP connection. The nvme-tcp module freezes completely and bricks my computer.

Steps:
$ sudo modprobe nvme-tcp
$ sudo nvme discover -t tcp -a [IP address] -s 8009
<System Bricked!>
Only a reboot (Alt-SysRq-B) can recover the system. 

Conditions to reproduce the problem:
The Discovery Controller must support sending Discovery Log Change Notifications. That is, bit 31 of the Identity's OAES field returned by the discovery controller must be set to 1. If OAES[31]=0, then everything is OK.

Systems tested:
1) Ubuntu 20.04, Linux 5.8, nvme 1.13.21
2) Fedora 33, Linux 5.10, nvme 1.11.1

Regards,
Martin

From jxgao at google.com  Mon Feb  1 14:35:52 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Mon, 1 Feb 2021 11:35:52 -0800
Subject: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask
 to NVMe driver.
In-Reply-To: <YBhOrK+kea7PNmXn@smile.fi.intel.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <YBhOrK+kea7PNmXn@smile.fi.intel.com>
Message-ID: <CAMGD6P0qc0FZagmimTCuYsbJXo-WBHjtgoWSRzs63EULybTzNg@mail.gmail.com>

On Mon, Feb 1, 2021 at 10:56 AM Andy Shevchenko
<andriy.shevchenko at linux.intel.com> wrote:
>
> On Mon, Feb 01, 2021 at 10:30:17AM -0800, Jianxiong Gao wrote:
> > NVMe driver relies on the address offset to function properly.
> > This patch adds the offset preserve mask to NVMe driver when mapping
> > via dma_map_sg_attrs and unmapping via nvme_unmap_sg. The mask
> > depends on the page size defined by CC.MPS register of NVMe
> > controller.
>
> ...
>
> >       if (is_pci_p2pdma_page(sg_page(iod->sg)))
> >               nr_mapped = pci_p2pdma_map_sg_attrs(dev->dev, iod->sg,
> >                               iod->nents, rq_dma_dir(req), DMA_ATTR_NO_WARN);
> >       else
> >               nr_mapped = dma_map_sg_attrs(dev->dev, iod->sg, iod->nents,
> >                                            rq_dma_dir(req), DMA_ATTR_NO_WARN);
> > +
> > +     offset_ret = dma_set_min_align_mask(dev->dev, 0);
> > +     if (offset_ret) {
> > +             dev_warn(dev->dev, "dma_set_min_align_mask failed to reset offset\n");
> > +             goto out_free_sg;
> > +     }
>
> Seems like rebasing effect which makes empty line goes in the middle of some
> other group of code lines.
>
> >       if (!nr_mapped)
> >               goto out_free_sg;
>
> Perhaps it should be here?
Yes you are correct, it should be

     else
              nr_mapped = dma_map_sg_attrs(dev->dev, iod->sg, iod->nents,
                                           rq_dma_dir(req), DMA_ATTR_NO_WARN);
      if (!nr_mapped)
              goto out_free_sg;
+
+     offset_ret = dma_set_min_align_mask(dev->dev, 0);
+     if (offset_ret) {
+             dev_warn(dev->dev, "dma_set_min_align_mask failed to
reset offset\n");
+             goto out_free_sg;
+     }

Thanks for pointing it out.
>
> --
> With Best Regards,
> Andy Shevchenko
>
>


-- 
Jianxiong Gao


From kbusch at kernel.org  Mon Feb  1 14:48:44 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon,  1 Feb 2021 11:48:44 -0800
Subject: [PATCH] Update email address
Message-ID: <20210201194844.2127981-1-kbusch@kernel.org>

Redirect my older email addresses that are in the git logs.

Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 .mailmap | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.mailmap b/.mailmap
index b1ab0129c7d6..c0ce7f0951a0 100644
--- a/.mailmap
+++ b/.mailmap
@@ -182,6 +182,8 @@ Kees Cook <keescook at chromium.org> <kees.cook at canonical.com>
 Kees Cook <keescook at chromium.org> <keescook at google.com>
 Kees Cook <keescook at chromium.org> <kees at outflux.net>
 Kees Cook <keescook at chromium.org> <kees at ubuntu.com>
+Keith Busch <kbusch at kernel.org> <keith.busch at intel.com>
+Keith Busch <kbusch at kernel.org> <keith.busch at linux.intel.com>
 Kenneth W Chen <kenneth.w.chen at intel.com>
 Konstantin Khlebnikov <koct9i at gmail.com> <khlebnikov at yandex-team.ru>
 Konstantin Khlebnikov <koct9i at gmail.com> <k.khlebnikov at samsung.com>
-- 
2.25.4



From kbusch at kernel.org  Mon Feb  1 15:57:59 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 1 Feb 2021 12:57:59 -0800
Subject: [PATCH V2 3/3] Adding
 device_dma_parameters->offset_preserve_mask to NVMe driver.
In-Reply-To: <20210201183017.3339130-4-jxgao@google.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
Message-ID: <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>

On Mon, Feb 01, 2021 at 10:30:17AM -0800, Jianxiong Gao wrote:
> @@ -868,12 +871,24 @@ static blk_status_t nvme_map_data(struct nvme_dev *dev, struct request *req,
>  	if (!iod->nents)
>  		goto out_free_sg;
>  
> +	offset_ret = dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1);
> +	if (offset_ret) {
> +		dev_warn(dev->dev, "dma_set_min_align_mask failed to set offset\n");
> +		goto out_free_sg;
> +	}
> +
>  	if (is_pci_p2pdma_page(sg_page(iod->sg)))
>  		nr_mapped = pci_p2pdma_map_sg_attrs(dev->dev, iod->sg,
>  				iod->nents, rq_dma_dir(req), DMA_ATTR_NO_WARN);
>  	else
>  		nr_mapped = dma_map_sg_attrs(dev->dev, iod->sg, iod->nents,
>  					     rq_dma_dir(req), DMA_ATTR_NO_WARN);
> +
> +	offset_ret = dma_set_min_align_mask(dev->dev, 0);
> +	if (offset_ret) {
> +		dev_warn(dev->dev, "dma_set_min_align_mask failed to reset offset\n");
> +		goto out_free_sg;
> +	}
>  	if (!nr_mapped)
>  		goto out_free_sg;

Why is this setting being done and undone on each IO? Wouldn't it be
more efficient to set it once during device initialization?

And more importantly, this isn't thread safe: one CPU may be setting the
device's dma alignment mask to 0 while another CPU is expecting it to be
NVME_CTRL_PAGE_SIZE - 1.


From Damien.LeMoal at wdc.com  Mon Feb  1 16:04:44 2021
From: Damien.LeMoal at wdc.com (Damien Le Moal)
Date: Mon, 1 Feb 2021 21:04:44 +0000
Subject: Buffer I/O Errors from Zoned NVME devices
References: <BYAPR04MB56561E48A54530BC7F6DCF30EAB69@BYAPR04MB5656.namprd04.prod.outlook.com>
 <20210201175306.GA2127465@dhcp-10-100-145-180.wdc.com>
 <20210201180257.GA14971@lst.de>
Message-ID: <BL0PR04MB65142D8FBF2B6BC88E32DCCEE7B69@BL0PR04MB6514.namprd04.prod.outlook.com>

On 2021/02/02 3:03, hch at lst.de wrote:
> On Mon, Feb 01, 2021 at 09:53:06AM -0800, Keith Busch wrote:
>> On Mon, Feb 01, 2021 at 02:36:12PM +0000, Jeffrey Lien wrote:
>>> Christoph, Keith
>>> We're seeing a lot of these Buffer I/O errors with our zoned nvme devices.  One of the FW developers looked into it and had the following explanation:
>>> All these Reads are from the kernel during enumeration and for LBAs that are in last zone's hole hence expected to return boundary error which is getting logged by kernel.
>>>
>>> [65281.936988] Buffer I/O error on dev nvme1n2, logical block 3800039296, async page read
>>> [65281.937165] blk_update_request: I/O error, dev nvme1n2, sector 3800039297 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
>>> [65281.937166] Buffer I/O error on dev nvme1n2, logical block 3800039297, async page read
>>> [65281.937335] blk_update_request: I/O error, dev nvme1n2, sector 3800039298 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
>>> [65281.937336] Buffer I/O error on dev nvme1n2, logical block 3800039298, async page read
>>> [65281.937498] blk_update_request: I/O error, dev nvme1n2, sector 3800039299 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
>>>
>>> Are you aware of this issue and if so, do you have any recommendations on how to avoid or resolve?  
>>
>> Is this from the partition scanning? We don't partition zoned devices,
>> so I think we can skip it. Does the following resolve the issue?
> 
> We already have special zoned device handling in the partitioning code.

Partitions are ignored and warning printed, but the partition table is still
being read...

> 
> But NVMe should make sure to never span a zone boundary as we set the
> chunk size to avoid that.
> 
> What kernel version is this?  Is CONFIG_BLK_DEV_ZONED enabled?

I had a very similar problem doing zonefs tests on Matias machine on a ZNS drive
last week. The problem was the firmware... An upgrade to the latest version
fixed the issue. Not sure what FW rev you are running here, but upgrading might
solve this.

> 


-- 
Damien Le Moal
Western Digital Research


From claus.stovgaard at gmail.com  Mon Feb  1 16:08:22 2021
From: claus.stovgaard at gmail.com (Claus Stovgaard)
Date: Mon,  1 Feb 2021 22:08:22 +0100
Subject: [PATCH] nvme-pci: Mark Phison E16 (Corsair MP600) as IGNORE_DEV_SUBNQN
Message-ID: <20210201210822.6501-1-claus.stovgaard@gmail.com>

Tested both with Corsairs firmware 11.3 and 13.0 for the MP600 and both
have the issue as reported by the kernel.

nvme nvme0: missing or invalid SUBNQN field.

Signed-off-by: Claus Stovgaard <claus.stovgaard at gmail.com>
---
 drivers/nvme/host/pci.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b2042..41b1b0731cbd 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3242,6 +3242,8 @@ static const struct pci_device_id nvme_id_table[] = {
 	{ PCI_DEVICE(0x144d, 0xa822),   /* Samsung PM1725a */
 		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY |
 				NVME_QUIRK_IGNORE_DEV_SUBNQN, },
+	{ PCI_DEVICE(0x1987, 0x5016),	/* Phison E16 (Corsair MP600) */
+		.driver_data = NVME_QUIRK_IGNORE_DEV_SUBNQN, },
 	{ PCI_DEVICE(0x1d1d, 0x1f1f),	/* LighNVM qemu device */
 		.driver_data = NVME_QUIRK_LIGHTNVM, },
 	{ PCI_DEVICE(0x1d1d, 0x2807),	/* CNEX WL */
-- 
2.26.2



From claus.stovgaard at gmail.com  Mon Feb  1 16:22:21 2021
From: claus.stovgaard at gmail.com (Claus Stovgaard)
Date: Mon, 01 Feb 2021 22:22:21 +0100
Subject: Missing or invalid SUBNQN on Corsair MP600 1TB device
In-Reply-To: <20210125175827.GA1732047@dhcp-10-100-145-180.wdc.com>
References: <04fdb6a116bbc4d9eadc3f9434d8b6f2426e3c57.camel@gmail.com>
 <20210125175827.GA1732047@dhcp-10-100-145-180.wdc.com>
Message-ID: <7dce6c69b48d97b467bb558c3d13ec134d44964b.camel@gmail.com>

On Mon, 2021-01-25 at 09:58 -0800, Keith Busch wrote:
> On Mon, Jan 25, 2021 at 05:00:57PM +0100, Claus Stovgaard wrote:
> > So I guess that Corsair MP600 is a candidate for the same quirk.
> > 
> > Considered to add the Corsair to the list of drives with
> > NVME_QUIRK_IGNORE_DEV_SUBNQN, but would just ask here first, if it
> > would be the correct use and understanding.
> 
> Yes, that is the correct quirk to use with devices reporting spec
> revision 1.2.1 compatibility but are not compliant with the subsys
> nqn
> field. The device should continue to work without the quirk, but
> you'll
> get the warning.

Thanks for the reply Keith. I have just created a patch with this quirk
for the Phison E16 (Corsair MP600). Being a simple change, I figured it
was a good start for novice in submitting kernel patches.

Regards
Claus



From jxgao at google.com  Mon Feb  1 16:22:45 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Mon, 1 Feb 2021 13:22:45 -0800
Subject: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask
 to NVMe driver.
In-Reply-To: <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>
 <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
Message-ID: <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>

> Why is this setting being done and undone on each IO? Wouldn't it be
> more efficient to set it once during device initialization?
>
> And more importantly, this isn't thread safe: one CPU may be setting the
> device's dma alignment mask to 0 while another CPU is expecting it to be
> NVME_CTRL_PAGE_SIZE - 1.

 I was having trouble getting the OS booted when setting it once during
 initialization. However when I rebased to the latest rc6 this morning it
 seems to be working with setting the mask on probe. I am still testing out
 and will appreciate any idea what may cause the nvme driver to fail
 with a single mask.
-- 
Jianxiong Gao


From mwilck at suse.com  Mon Feb  1 17:37:13 2021
From: mwilck at suse.com (mwilck at suse.com)
Date: Mon,  1 Feb 2021 23:37:13 +0100
Subject: [PATCH] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
Message-ID: <20210201223713.15243-1-mwilck@suse.com>

From: Martin Wilck <mwilck at suse.com>

While a controller is still connecting, ctrl->subsys is NULL and
thus reading the controller's "subsysnqn" sysfs attribute returns
"(efault)". This happens all the time when user space processes
"add" events for NVMe controller devices.

Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
if ctrl->subsys isn't initialized yet.

Signed-off-by: Martin Wilck <mwilck at suse.com>
---
 drivers/nvme/host/core.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f320273fc672..5d901dd0098b 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3538,8 +3538,16 @@ static ssize_t nvme_sysfs_show_subsysnqn(struct device *dev,
 					 char *buf)
 {
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	char *subsysnqn;
 
-	return snprintf(buf, PAGE_SIZE, "%s\n", ctrl->subsys->subnqn);
+	if (ctrl->subsys)
+		subsysnqn = ctrl->subsys->subnqn;
+	else if (ctrl->opts && ctrl->opts->subsysnqn)
+		subsysnqn = ctrl->opts->subsysnqn;
+	else
+		subsysnqn = "unknown";
+
+	return snprintf(buf, PAGE_SIZE, "%s\n", subsysnqn);
 }
 static DEVICE_ATTR(subsysnqn, S_IRUGO, nvme_sysfs_show_subsysnqn, NULL);
 
-- 
2.29.2



From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 18:56:03 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon, 1 Feb 2021 23:56:03 +0000
Subject: [PATCH] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
References: <20210201223713.15243-1-mwilck@suse.com>
Message-ID: <BYAPR04MB4965FA3C4ADF1E74056D5FC686B69@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 14:41, mwilck at suse.com wrote:
> -	return snprintf(buf, PAGE_SIZE, "%s\n", ctrl->subsys->subnqn);
> +	if (ctrl->subsys)
> +		subsysnqn = ctrl->subsys->subnqn;
> +	else if (ctrl->opts && ctrl->opts->subsysnqn)
> +		subsysnqn = ctrl->opts->subsysnqn;
> +	else
> +		subsysnqn = "unknown";
Just initialize the subsysnqnq to unknown string at the time of declaration.

Also, can you please add a testcase for this bug in blktests to make
sure this
behavior will not repeat  ?


From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 18:59:52 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon, 1 Feb 2021 23:59:52 +0000
Subject: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask
 to NVMe driver.
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>
 <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
 <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
Message-ID: <BYAPR04MB4965AAC6C59013D475093D9486B69@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 13:27, Jianxiong Gao wrote:
>> Why is this setting being done and undone on each IO? Wouldn't it be
>> more efficient to set it once during device initialization?
>>
>> And more importantly, this isn't thread safe: one CPU may be setting the
>> device's dma alignment mask to 0 while another CPU is expecting it to be
>> NVME_CTRL_PAGE_SIZE - 1.
>  I was having trouble getting the OS booted when setting it once during
>  initialization. However when I rebased to the latest rc6 this morning it
>  seems to be working with setting the mask on probe. I am still testing out
>  and will appreciate any idea what may cause the nvme driver to fail
>  with a single mask.
Based on the Keith's comment it needs to be completely avoided in hot path.

Did you get a chance to bisect the problem in the rc6 ? We need to know the
root cause otherwise it might happen again after we add this patch.


From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 19:03:17 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 2 Feb 2021 00:03:17 +0000
Subject: [PATCH] nvmetcli documentation: fix typo
References: <20210201144756.37538-1-mlombard@redhat.com>
Message-ID: <BYAPR04MB49658DDB7011AD1A1F9E6C9886B59@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 06:55, Maurizio Lombardi wrote:
> Signed-off-by: Maurizio Lombardi <mlombard at redhat.com>
Looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>



From enbyamy at gmail.com  Mon Feb  1 19:11:52 2021
From: enbyamy at gmail.com (Amy Parker)
Date: Mon, 1 Feb 2021 16:11:52 -0800
Subject: [PATCH] nvme-pci: Mark Phison E16 (Corsair MP600) as
 IGNORE_DEV_SUBNQN
In-Reply-To: <20210201210822.6501-1-claus.stovgaard@gmail.com>
References: <20210201210822.6501-1-claus.stovgaard@gmail.com>
Message-ID: <CAE1WUT7vHzzVXpx5vKbEKb+63G6RmM0s39EtmUgYDdANkveRYQ@mail.gmail.com>

On Mon, Feb 1, 2021 at 1:11 PM Claus Stovgaard
<claus.stovgaard at gmail.com> wrote:
>
> Tested both with Corsairs firmware 11.3 and 13.0 for the MP600 and both
> have the issue as reported by the kernel.
>
> nvme nvme0: missing or invalid SUBNQN field.
>
> Signed-off-by: Claus Stovgaard <claus.stovgaard at gmail.com>
> ---
>  drivers/nvme/host/pci.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 81e6389b2042..41b1b0731cbd 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -3242,6 +3242,8 @@ static const struct pci_device_id nvme_id_table[] = {
>         { PCI_DEVICE(0x144d, 0xa822),   /* Samsung PM1725a */
>                 .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY |
>                                 NVME_QUIRK_IGNORE_DEV_SUBNQN, },
> +       { PCI_DEVICE(0x1987, 0x5016),   /* Phison E16 (Corsair MP600) */
> +               .driver_data = NVME_QUIRK_IGNORE_DEV_SUBNQN, },
>         { PCI_DEVICE(0x1d1d, 0x1f1f),   /* LighNVM qemu device */
>                 .driver_data = NVME_QUIRK_LIGHTNVM, },
>         { PCI_DEVICE(0x1d1d, 0x2807),   /* CNEX WL */
> --
> 2.26.2
>

Can't test this - don't have an MP600 of my own to test with - but it
should work just fine. If anyone else has an MP600 to throw at this
that'd be great.

Acked-by: Amy Parker <enbyamy at gmail.com>

Best regards,
Amy Parker
(she/her/hers)


From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 19:12:33 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 2 Feb 2021 00:12:33 +0000
Subject: [PATCH 01/10] nvmet: zeroout id-ns buffer for invalid nsid
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-2-chaitanya.kulkarni@wdc.com>
 <20210201172438.GB12054@lst.de>
Message-ID: <BYAPR04MB49653AC1A94F78112729EC6686B59@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 09:24, Christoph Hellwig wrote:
>>  drivers/nvme/target/admin-cmd.c | 22 ++++++++++++++--------
>>  1 file changed, 14 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
>> index 613a4d8feac1..8cc7bb25d10d 100644
>> --- a/drivers/nvme/target/admin-cmd.c
>> +++ b/drivers/nvme/target/admin-cmd.c
>> @@ -476,19 +476,25 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
>>  		goto out;
>>  	}
>>  
>> +	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
>> +	if (!req->ns) {
>> +		status = NVME_SC_INVALID_NS;
>> +		/*
>> +		 * According to spec : If the specified namespace is
>> +		 * an unallocated NSID then the controller returns a zero filled
>> +		 * data structure. Also don't override the error status as invalid
>> +		 * namespace takes priority over the failed zeroout buffer case.
>> +		 */
>> +		nvmet_zero_sgl(req, 0, sizeof(*id));
>> +		goto out;
>> +	}
>> +
>>  	id = kzalloc(sizeof(*id), GFP_KERNEL);
>>  	if (!id) {
>>  		status = NVME_SC_INTERNAL;
>>  		goto out;
>>  	}
>>  
>> -	/* return an all zeroed buffer if we can't find an active namespace */
>> -	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
>> -	if (!req->ns) {
>> -		status = NVME_SC_INVALID_NS;
>> -		goto done;
> I think all we need to do is to remove the status assignment here.
> While you're patch avoids the memory allocation for this case, it isn't
> really the fast path so I'd rather avoid the extra code.
>
Sorry, I didn't understand this comment. If we remove the status assignment
then host will not get the right error. That is the bug we fixed it
initially
with bffcd507780e.

Regarding fast path, if system is under pressure even single memory
allocation
can be costly, especially when host tries to do read id-ns, is there any
reason why we should not consider this scenario ?


From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 19:13:09 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 2 Feb 2021 00:13:09 +0000
Subject: [PATCH 03/10] nvmet: set error in nvmet_find_namespace()
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-4-chaitanya.kulkarni@wdc.com>
 <20210201172730.GC12054@lst.de>
Message-ID: <BYAPR04MB49656D0526749AED6B2C6FA386B59@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 09:27, Christoph Hellwig wrote:
> This does look like a nice cleanup, but I think we also need to
> rename the function now that it does not return the nvmet_ns
> but only sets it in the nvmet_req structure.
>
> What about nvmet_req_find_namespace?
Perfect.


From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 19:13:47 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 2 Feb 2021 00:13:47 +0000
Subject: [PATCH 04/10] nvmet: remove nsid param from nvmet_find_namespace()
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-5-chaitanya.kulkarni@wdc.com>
 <20210201172810.GD12054@lst.de>
Message-ID: <BYAPR04MB4965C6585639860A00269B2286B59@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 09:28, Christoph Hellwig wrote:
>> The nvmet_find_namespace() takes nsid parameter which is from NVMe
>> commands structures such as get_log_page, identify, rw and common. All
>> these commands have same offset for the nsid field.
>>
>> Derive nsid from req (req->cmd->common.nsid) and remove the extra
>> parameter from the nvmet_find_namespace().
> I'd merge this into the previous one as that totally changes the
> calling conventions anyway.  Otherwise this looks good to me.
>
Okay.


From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 19:14:47 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 2 Feb 2021 00:14:47 +0000
Subject: [PATCH 06/10] nvmet: add helper to report invalid opcode
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-7-chaitanya.kulkarni@wdc.com>
 <20210201172921.GE12054@lst.de>
Message-ID: <BYAPR04MB49650B7A725B55D0809DB0CE86B59@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 09:29, Christoph Hellwig wrote:
> I think we can just skip reporting the backend, as this doesn't
> really add much value.  In fact given that this can be triggered
> remotely it should probably be downgraded to a pr_debug.
It is easy to grep the backend given that we have 3 and ZBD is
being worked on. I'll remove it for now with pr_debug.


From jxgao at google.com  Mon Feb  1 19:25:55 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Mon, 1 Feb 2021 16:25:55 -0800
Subject: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask
 to NVMe driver.
In-Reply-To: <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>
 <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
 <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
Message-ID: <CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com>

> Why is this setting being done and undone on each IO? Wouldn't it be
> more efficient to set it once during device initialization?

I agree that setting it once is the right way of doing it.

So I have changed the patch to enable the mask once in nvme_probe.

 drivers/nvme/host/pci.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b2042..4ce78373f98d 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2630,6 +2630,9 @@ static void nvme_reset_work(struct work_struct *work)
         */
        dma_set_max_seg_size(dev->dev, 0xffffffff);

+       if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1))
+               dev_warn(dev->dev, "dma_set_min_align_mask failed to
set offset\n");
+
        mutex_unlock(&dev->shutdown_lock);

        /*

However on boot of the system, the following error happens occasionally.
The error seems related to Journal service. Whenever "Stopping Journal
Service..."
appears, the boot succeeds. Otherwise boot fails with the following message.

----------------------------log start here--------------------------
[  OK  ] Started Journal Service.
[   10.774545] xfs filesystem being remounted at / supports timestamps
until 2038 (0x7fffffff)
[  OK  ] Started Remount Root and Kernel File Systems.
         Starting Flush Journal to Persistent Storage...
         Starting Load/Save Random Seed...
         Starting Create Static [   10.804340] systemd-journald[780]:
Received request to flush runtime journal from PID 1
Device Nodes in /dev...
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started Flush Journal to Persistent Storage.
[  OK  ] Started Create Static Device Nodes in /dev.
         Starting udev Kernel Device Manager...
[  OK  ] Reached target Local File Systems (Pre).
         Starting File System Check on /dev/disk/by-uuid/7281-17FC...
[  OK  ] Started File System Check on /dev/disk/by-uuid/7281-17FC.
         Mounting /boot/efi...
[  OK  ] Mounted /boo[   11.203461] systemd[1]: segfault at 2e0 ip
000055b08607cc24 sp 00007ffe13809090 error 4 in
systemd[55b086000000+140000]
t/efi.
[   11.216088] Code: 02 c7 44 24 10 fe ff ff ff 49 89 e4 89 06 48 8d
6c 24 08 48 8d 5c 24 10 48 c7 44 24 18 00 00 00 00 eb 10 0f 1f 00 48
8b 3c 24 <44> 39 b7 e0 02 00 00 74 3b 49 8b 7d 00 4c 89 e1 48 89 ea 48
89 de
---------------log ends here-----------

> Based on the Keith's comment it needs to be completely avoided in hot path.
>
> Did you get a chance to bisect the problem in the rc6 ? We need to know the
> root cause otherwise it might happen again after we add this patch.

I am now trying to bisect the problem.
I am not sure how the mapping offset could have caused the error.
Any suggestions are appreciated.

--
Jianxiong Gao


From mwilck at suse.com  Mon Feb  1 19:59:51 2021
From: mwilck at suse.com (mwilck at suse.com)
Date: Tue,  2 Feb 2021 01:59:51 +0100
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
Message-ID: <20210202005951.26614-1-mwilck@suse.com>

From: Martin Wilck <mwilck at suse.com>

While a controller is still connecting, ctrl->subsys is NULL and
thus reading the controller's "subsysnqn" sysfs attribute returns
"(efault)". This happens all the time when user space processes
"add" events for NVMe controller devices.

Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
if ctrl->subsys isn't initialized yet.

Signed-off-by: Martin Wilck <mwilck at suse.com>
---
 drivers/nvme/host/core.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f320273fc672..8b0fc4a07b31 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3538,8 +3538,14 @@ static ssize_t nvme_sysfs_show_subsysnqn(struct device *dev,
 					 char *buf)
 {
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	char *subsysnqn = "unknown";
 
-	return snprintf(buf, PAGE_SIZE, "%s\n", ctrl->subsys->subnqn);
+	if (ctrl->subsys)
+		subsysnqn = ctrl->subsys->subnqn;
+	else if (ctrl->opts && ctrl->opts->subsysnqn)
+		subsysnqn = ctrl->opts->subsysnqn;
+
+	return snprintf(buf, PAGE_SIZE, "%s\n", subsysnqn);
 }
 static DEVICE_ATTR(subsysnqn, S_IRUGO, nvme_sysfs_show_subsysnqn, NULL);
 
-- 
2.29.2



From mwilck at suse.com  Mon Feb  1 20:01:46 2021
From: mwilck at suse.com (Martin Wilck)
Date: Tue, 02 Feb 2021 02:01:46 +0100
Subject: [PATCH] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <BYAPR04MB4965FA3C4ADF1E74056D5FC686B69@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210201223713.15243-1-mwilck@suse.com>
 <BYAPR04MB4965FA3C4ADF1E74056D5FC686B69@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <953df866cc397fd8231da9b8d82d97754719f459.camel@suse.com>

On Mon, 2021-02-01 at 23:56 +0000, Chaitanya Kulkarni wrote:
> On 2/1/21 14:41, mwilck at suse.com?wrote:
> > -???????return snprintf(buf, PAGE_SIZE, "%s\n", ctrl->subsys-
> > >subnqn);
> > +???????if (ctrl->subsys)
> > +???????????????subsysnqn = ctrl->subsys->subnqn;
> > +???????else if (ctrl->opts && ctrl->opts->subsysnqn)
> > +???????????????subsysnqn = ctrl->opts->subsysnqn;
> > +???????else
> > +???????????????subsysnqn = "unknown";
> Just initialize the subsysnqnq to unknown string at the time of
> declaration.

Just sent a v2.

> 
> Also, can you please add a testcase for this bug in blktests to make
> sure this
> behavior will not repeat? ?

Yes, but it'll take time because I have zero experience with blktests
and little time the rest of this week. Please kindly review the kernel
patch nonetheless.

Regards
Martin





From lengchao at huawei.com  Mon Feb  1 20:12:30 2021
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 2 Feb 2021 09:12:30 +0800
Subject: [PATCH v2] nvme-multipath: Early exit if no path is available
In-Reply-To: <2752ed93-6bd5-1a13-0e05-b91e2dcc24e1@suse.de>
References: <20210127103033.15318-1-dwagner@suse.de>
 <db9baae0-547c-7ff4-8b2c-0b95f14be67c@huawei.com>
 <20210128075837.u5u56t23fq5gu6ou@beryllium.lan>
 <69575290-200e-b4a1-4269-c71e4c2cc37b@huawei.com>
 <20210128094004.erwnszjqcxlsi2kd@beryllium.lan>
 <ebb1d098-3ded-e592-4419-e905aabe824f@huawei.com>
 <675d3cf7-1ae8-adc5-b6d0-359fe10f6b23@grimberg.me>
 <59cd053e-46cb-0235-141f-4ce919c93f48@huawei.com>
 <65392653-6b03-9195-f686-5fe4b3290bd2@suse.de>
 <81b22bbf-4dd3-6161-e63a-9699690a4e4f@huawei.com>
 <715dd943-0587-be08-2840-e0948cf0bc62@suse.de>
 <eb131d8f-f009-42e7-105d-58b84060f0dd@huawei.com>
 <ac019690-7f02-d28c-ed58-bfc8c1d48879@suse.de>
 <6ceff3cb-c9e9-7e74-92f0-dd745987c943@huawei.com>
 <114751ac-1f7d-ce5e-12c5-7d6303bdb999@suse.de>
 <aebc6c2d-4711-95ba-daf2-1cd17fc6f0e7@huawei.com>
 <47a1b796-9d91-5947-4bac-dd8f397041a3@suse.de>
 <bd37abd5-759d-efe2-fdcd-8b004a41c75a@huawei.com>
 <2752ed93-6bd5-1a13-0e05-b91e2dcc24e1@suse.de>
Message-ID: <6ef6df55-318b-887d-b14f-62cae6419b4a@huawei.com>



On 2021/2/1 18:45, Hannes Reinecke wrote:
> On 2/1/21 10:40 AM, Chao Leng wrote:
>>
>>
>> On 2021/2/1 16:57, Hannes Reinecke wrote:
>>> On 2/1/21 9:47 AM, Chao Leng wrote:
>>>>
>>>>
>>>> On 2021/2/1 15:29, Hannes Reinecke wrote:[ .. ]
>>>>> Urgh. Please, no. That is well impossible to debug.
>>>>> Can you please open-code it to demonstrate where the difference to the current (and my fixed) versions is?
>>>>> I'm still not clear where the problem is once we applied both patches.
>>>> For example assume the list has three path, and all path is not NVME_ANA_OPTIMIZED:
>>>> head->next = ns1;
>>>> ns1->next = ns2;
>>>> ns2->next = head;
>>>> old->next = ns2;
>>>>
>>> And this is where I have issues with.
>>> Where does 'old' come from?
>>> Clearly it was part of the list at one point; so what happened to it?
>> I explained this earlier.
>> In nvme_ns_remove, there is a hole between list_del_rcu and
>> nvme_mpath_clear_current_path. If head->current_path is the "old", and
>> the "old" is removing. The "old" is already removed from the list by
>> list_del_rcu, but head->current_path is not clear to NULL by
>> nvme_mpath_clear_current_path.
>> Find path is race with nvme_ns_remove, use the "old" pass to
>> nvme_round_robin_path to find path.
> 
> Ah. So this should be better:
> 
> @@ -202,10 +202,12 @@ static struct nvme_ns *__nvme_find_path(struct nvme_ns_head *head, int node)
>  ?static struct nvme_ns *nvme_next_ns(struct nvme_ns_head *head,
>  ??????????????? struct nvme_ns *ns)
>  ?{
> -?????? ns = list_next_or_null_rcu(&head->list, &ns->siblings, struct nvme_ns,
> -?????????????????????? siblings);
> -?????? if (ns)
> -?????????????? return ns;
> +?????? if (ns && !test_bit(NVME_NS_REMOVING, &ns->flags)) {
> +?????????????? ns = list_next_or_null_rcu(&head->list, &ns->siblings,
> +????????????????????????????????????????? struct nvme_ns, siblings);
> +?????????????? if (ns)
> +?????????????????????? return ns;
> +?????? }
>  ??????? return list_first_or_null_rcu(&head->list, struct nvme_ns, siblings);
>  ?}
> 
> The 'NVME_NS_REMOVING' bit is set before list_del_rcu() is called, so it should guard against the issue you mentioned.
Looks useless, it is still infinite loop. You can check the workflow for the scenario I mentioned.
> 
> Cheers,
> 
> Hannes


From Chaitanya.Kulkarni at wdc.com  Mon Feb  1 20:34:37 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 2 Feb 2021 01:34:37 +0000
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
References: <20210202005951.26614-1-mwilck@suse.com>
Message-ID: <BYAPR04MB49653DDA94AA1D6983F070FD86B59@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 17:00, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
>
> While a controller is still connecting, ctrl->subsys is NULL and
> thus reading the controller's "subsysnqn" sysfs attribute returns
> "(efault)". This happens all the time when user space processes
> "add" events for NVMe controller devices.
>
> Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
> if ctrl->subsys isn't initialized yet.
>
> Signed-off-by: Martin Wilck <mwilck at suse.com>
Please add the version history from next time, as it helps others
to review the patch easily than the first-reviewer.

Looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>


From jiapeng.chong at linux.alibaba.com  Tue Feb  2 02:06:17 2021
From: jiapeng.chong at linux.alibaba.com (Jiapeng Chong)
Date: Tue,  2 Feb 2021 15:06:17 +0800
Subject: [PATCH] nvme: convert sysfs sprintf/snprintf family to sysfs_emit
Message-ID: <1612249577-66130-1-git-send-email-jiapeng.chong@linux.alibaba.com>

Fix the following coccicheck warning:

./drivers/nvme/host/core.c:3580:8-16: WARNING: use scnprintf or sprintf.
./drivers/nvme/host/core.c:3570:8-16: WARNING: use scnprintf or sprintf.
./drivers/nvme/host/core.c:3560:8-16: WARNING: use scnprintf or sprintf.
./drivers/nvme/host/core.c:3526:8-16: WARNING: use scnprintf or sprintf.
./drivers/nvme/host/core.c:2833:8-16: WARNING: use scnprintf or sprintf.

Reported-by: Abaci Robot<abaci at linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong at linux.alibaba.com>
---
 drivers/nvme/host/core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f13eb4d..5abbab3 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2830,7 +2830,7 @@ static ssize_t nvme_subsys_show_nqn(struct device *dev,
 	struct nvme_subsystem *subsys =
 		container_of(dev, struct nvme_subsystem, dev);
 
-	return snprintf(buf, PAGE_SIZE, "%s\n", subsys->subnqn);
+	return sysfs_emit(buf, "%s\n", subsys->subnqn);
 }
 static SUBSYS_ATTR_RO(subsysnqn, S_IRUGO, nvme_subsys_show_nqn);
 
@@ -3523,7 +3523,7 @@ static ssize_t nvme_sysfs_show_transport(struct device *dev,
 {
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
 
-	return snprintf(buf, PAGE_SIZE, "%s\n", ctrl->ops->name);
+	return sysfs_emit(buf, "%s\n", ctrl->ops->name);
 }
 static DEVICE_ATTR(transport, S_IRUGO, nvme_sysfs_show_transport, NULL);
 
@@ -3557,7 +3557,7 @@ static ssize_t nvme_sysfs_show_subsysnqn(struct device *dev,
 {
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
 
-	return snprintf(buf, PAGE_SIZE, "%s\n", ctrl->subsys->subnqn);
+	return sysfs_emit(buf, "%s\n", ctrl->subsys->subnqn);
 }
 static DEVICE_ATTR(subsysnqn, S_IRUGO, nvme_sysfs_show_subsysnqn, NULL);
 
@@ -3567,7 +3567,7 @@ static ssize_t nvme_sysfs_show_hostnqn(struct device *dev,
 {
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
 
-	return snprintf(buf, PAGE_SIZE, "%s\n", ctrl->opts->host->nqn);
+	return sysfs_emit(buf, "%s\n", ctrl->opts->host->nqn);
 }
 static DEVICE_ATTR(hostnqn, S_IRUGO, nvme_sysfs_show_hostnqn, NULL);
 
@@ -3577,7 +3577,7 @@ static ssize_t nvme_sysfs_show_hostid(struct device *dev,
 {
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
 
-	return snprintf(buf, PAGE_SIZE, "%pU\n", &ctrl->opts->host->id);
+	return sysfs_emit(buf, "%pU\n", &ctrl->opts->host->id);
 }
 static DEVICE_ATTR(hostid, S_IRUGO, nvme_sysfs_show_hostid, NULL);
 
-- 
1.8.3.1



From Chaitanya.Kulkarni at wdc.com  Tue Feb  2 02:56:55 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 2 Feb 2021 07:56:55 +0000
Subject: [PATCH] nvme: convert sysfs sprintf/snprintf family to sysfs_emit
References: <1612249577-66130-1-git-send-email-jiapeng.chong@linux.alibaba.com>
Message-ID: <BYAPR04MB49658C9E558544DAE549D64286B59@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/1/21 23:12, Jiapeng Chong wrote:
> Fix the following coccicheck warning:
>
> ./drivers/nvme/host/core.c:3580:8-16: WARNING: use scnprintf or sprintf.
> ./drivers/nvme/host/core.c:3570:8-16: WARNING: use scnprintf or sprintf.
> ./drivers/nvme/host/core.c:3560:8-16: WARNING: use scnprintf or sprintf.
> ./drivers/nvme/host/core.c:3526:8-16: WARNING: use scnprintf or sprintf.
> ./drivers/nvme/host/core.c:2833:8-16: WARNING: use scnprintf or sprintf.
>
> Reported-by: Abaci Robot<abaci at linux.alibaba.com>
> Signed-off-by: Jiapeng Chong <jiapeng.chong at linux.alibaba.com>
Looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>



From hch at lst.de  Tue Feb  2 02:58:28 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 2 Feb 2021 08:58:28 +0100
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <20210202005951.26614-1-mwilck@suse.com>
References: <20210202005951.26614-1-mwilck@suse.com>
Message-ID: <20210202075828.GA12160@lst.de>

On Tue, Feb 02, 2021 at 01:59:51AM +0100, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> While a controller is still connecting, ctrl->subsys is NULL and
> thus reading the controller's "subsysnqn" sysfs attribute returns
> "(efault)". This happens all the time when user space processes
> "add" events for NVMe controller devices.
> 
> Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
> if ctrl->subsys isn't initialized yet.

This is just papering over symptoms.  We should not be sending
events or display sysfs files until the controlle is live.

In other wwords - I think we should remove the cdev_device_add
from nvme_init_ctrl to nvme_start_ctrl (+ any error handling changes
resulting from that).


From hch at lst.de  Tue Feb  2 04:22:25 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 2 Feb 2021 10:22:25 +0100
Subject: [PATCH] nvmetcli documentation: fix typo
In-Reply-To: <20210201144756.37538-1-mlombard@redhat.com>
References: <20210201144756.37538-1-mlombard@redhat.com>
Message-ID: <20210202092225.GA17125@lst.de>

Thanks,

applied.


From hch at infradead.org  Tue Feb  2 04:33:20 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Tue, 2 Feb 2021 10:33:20 +0100
Subject: [GIT PULL] first round of nvme updates for 5.12
Message-ID: <YBkcYFf7hJfpK+vl@infradead.org>

The following changes since commit e8628013e5ddc7cf78cc2f738ab760e8c0fa8559:

  drbd: Avoid comma separated statements (2021-01-31 08:05:36 -0700)

are available in the Git repository at:

  git://git.infradead.org/nvme.git tags/nvme-5.21-2020-02-02

for you to fetch changes up to 563c81586d0ab2841487a61fb34d6e9cd5efded7:

  nvme-tcp: use cancel tagset helper for tear down (2021-02-02 10:26:13 +0100)

----------------------------------------------------------------
nvme updates for 5.12:

 - failed reconnect fixes (Chao Leng)
 - various tracing improvements (Michal Krakowiak, Johannes Thumshirn)
 - switch the nvmet-fc assoc_list to use RCU protection (Leonid Ravich)
 - resync the status codes with the latest spec (Max Gurtovoy)
 - minor nvme-tcp improvements (Sagi Grimberg)
 - various cleanups (Rikard Falkeborn, Minwoo Im, Chaitanya Kulkarni,
   Israel Rukshin)

----------------------------------------------------------------
Chaitanya Kulkarni (5):
      nvmet: remove extra variable in smart log nsid
      nvmet: remove extra variable in id-desclist
      nvmet: remove extra variable in identify ns
      nvmet: add lba to sect conversion helpers
      nvme-core: get rid of the extra space

Chao Leng (5):
      nvme-core: add cancel tagset helpers
      nvme-rdma: add clean action for failed reconnection
      nvme-tcp: add clean action for failed reconnection
      nvme-rdma: use cancel tagset helper for tear down
      nvme-tcp: use cancel tagset helper for tear down

Israel Rukshin (2):
      nvmet: Use nvmet_is_port_enabled helper for pi_enable
      nvmet: Fix nvmet_is_port_enabled indentation

Johannes Thumshirn (1):
      nvme: add tracing of zns commands

Leonid Ravich (1):
      nvmet-fc: use RCU proctection for assoc_list

Max Gurtovoy (1):
      nvme: update enumerations for status codes

Michal Krakowiak (1):
      nvme: parse format nvm command details when tracing

Minwoo Im (2):
      nvme: support command retry delay for admin command
      nvme: refactor ns->ctrl by request

Rikard Falkeborn (1):
      nvme: constify static attribute_group structs

Sagi Grimberg (3):
      nvme-tcp: fix wrong setting of request iov_iter
      nvme-tcp: get rid of unused helper function
      nvme-tcp: pass multipage bvec to request iov_iter

 drivers/nvme/host/core.c          | 37 +++++++++++++-----
 drivers/nvme/host/fc.c            |  2 +-
 drivers/nvme/host/nvme.h          |  2 +
 drivers/nvme/host/rdma.c          | 30 +++++++++------
 drivers/nvme/host/tcp.c           | 55 +++++++++++++-------------
 drivers/nvme/host/trace.c         | 53 +++++++++++++++++++++++++
 drivers/nvme/target/admin-cmd.c   | 71 ++++++++++++++++------------------
 drivers/nvme/target/configfs.c    |  6 +--
 drivers/nvme/target/fc.c          | 81 ++++++++++++++++++---------------------
 drivers/nvme/target/fcloop.c      |  2 +-
 drivers/nvme/target/io-cmd-bdev.c |  8 ++--
 drivers/nvme/target/nvmet.h       | 10 +++++
 include/linux/nvme.h              | 30 ++++++++++++---
 13 files changed, 243 insertions(+), 144 deletions(-)


From hch at lst.de  Tue Feb  2 04:38:31 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 2 Feb 2021 10:38:31 +0100
Subject: [PATCH 01/10] nvmet: zeroout id-ns buffer for invalid nsid
In-Reply-To: <BYAPR04MB49653AC1A94F78112729EC6686B59@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-2-chaitanya.kulkarni@wdc.com>
 <20210201172438.GB12054@lst.de>
 <BYAPR04MB49653AC1A94F78112729EC6686B59@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210202093831.GG17771@lst.de>

On Tue, Feb 02, 2021 at 12:12:33AM +0000, Chaitanya Kulkarni wrote:
> >> +	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
> >> +	if (!req->ns) {
> >> +		status = NVME_SC_INVALID_NS;
> >> +		/*
> >> +		 * According to spec : If the specified namespace is
> >> +		 * an unallocated NSID then the controller returns a zero filled
> >> +		 * data structure. Also don't override the error status as invalid
> >> +		 * namespace takes priority over the failed zeroout buffer case.
> >> +		 */
> >> +		nvmet_zero_sgl(req, 0, sizeof(*id));
> >> +		goto out;
> >> +	}
> >> +
> >>  	id = kzalloc(sizeof(*id), GFP_KERNEL);
> >>  	if (!id) {
> >>  		status = NVME_SC_INTERNAL;
> >>  		goto out;
> >>  	}
> >>  
> >> -	/* return an all zeroed buffer if we can't find an active namespace */
> >> -	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
> >> -	if (!req->ns) {
> >> -		status = NVME_SC_INVALID_NS;
> >> -		goto done;
> > I think all we need to do is to remove the status assignment here.
> > While you're patch avoids the memory allocation for this case, it isn't
> > really the fast path so I'd rather avoid the extra code.
> >
> Sorry, I didn't understand this comment. If we remove the status assignment
> then host will not get the right error. That is the bug we fixed it
> initially
> with bffcd507780e.

Actually, looking at it, bffcd507780e is wrong.  Identify Namespace
to a namespace that is not active should return a status of 0 and
a zeroed structure.

> Regarding fast path, if system is under pressure even single memory
> allocation
> can be costly, especially when host tries to do read id-ns, is there any
> reason why we should not consider this scenario ?

No sensible host gets there.  I'd rather keep the code simple.


From hch at lst.de  Tue Feb  2 04:39:08 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 2 Feb 2021 10:39:08 +0100
Subject: [PATCH 06/10] nvmet: add helper to report invalid opcode
In-Reply-To: <BYAPR04MB49650B7A725B55D0809DB0CE86B59@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-7-chaitanya.kulkarni@wdc.com>
 <20210201172921.GE12054@lst.de>
 <BYAPR04MB49650B7A725B55D0809DB0CE86B59@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210202093908.GH17771@lst.de>

On Tue, Feb 02, 2021 at 12:14:47AM +0000, Chaitanya Kulkarni wrote:
> On 2/1/21 09:29, Christoph Hellwig wrote:
> > I think we can just skip reporting the backend, as this doesn't
> > really add much value.  In fact given that this can be triggered
> > remotely it should probably be downgraded to a pr_debug.
> It is easy to grep the backend given that we have 3 and ZBD is
> being worked on. I'll remove it for now with pr_debug.

Well, you'd better know what backend your namespaces uses when looking
at the logs.


From hch at lst.de  Tue Feb  2 04:49:21 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 2 Feb 2021 10:49:21 +0100
Subject: [PATCH] nvme-pci: add quirk to make Kingston A2000 SSD avoid
 deepest sleep state
In-Reply-To: <20210129052442.310780-1-linux@leemhuis.info>
References: <20210129052442.310780-1-linux@leemhuis.info>
Message-ID: <20210202094921.GA18895@lst.de>

Thanks,

applied to nvme-5.11.


From hch at lst.de  Tue Feb  2 04:49:35 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 2 Feb 2021 10:49:35 +0100
Subject: [PATCH] Update email address
In-Reply-To: <20210201194844.2127981-1-kbusch@kernel.org>
References: <20210201194844.2127981-1-kbusch@kernel.org>
Message-ID: <20210202094935.GB18895@lst.de>

Thanks,

applied to nvme-5.11.


From hch at lst.de  Tue Feb  2 04:49:45 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 2 Feb 2021 10:49:45 +0100
Subject: [PATCH] nvme-pci: Mark Phison E16 (Corsair MP600) as
 IGNORE_DEV_SUBNQN
In-Reply-To: <20210201210822.6501-1-claus.stovgaard@gmail.com>
References: <20210201210822.6501-1-claus.stovgaard@gmail.com>
Message-ID: <20210202094945.GC18895@lst.de>

Thanks,

applied to nvme-5.11.


From tparkin at katalix.com  Tue Feb  2 05:06:53 2021
From: tparkin at katalix.com (Tom Parkin)
Date: Tue, 2 Feb 2021 10:06:53 +0000
Subject: [PATCH v3 net-next  02/21] net: Introduce direct data placement
 tcp offload
In-Reply-To: <20210201100509.27351-3-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-3-borisp@mellanox.com>
Message-ID: <20210202100652.GA4487@katalix.com>

On  Mon, Feb 01, 2021 at 12:04:50 +0200, Boris Pismenny wrote:
> This commit introduces direct data placement offload for TCP.
> This capability is accompanied by new net_device operations that
> configure hardware contexts. There is a context per socket, and a context per DDP
> opreation. Additionally, a resynchronization routine is used to assist
> hardware handle TCP OOO, and continue the offload.
> Furthermore, we let the offloading driver advertise what is the max hw
> sectors/segments.
> 
> Using this interface, the NIC hardware will scatter TCP payload directly
> to the BIO pages according to the command_id.
> To maintain the correctness of the network stack, the driver is expected
> to construct SKBs that point to the BIO pages.
> 
> The SKB passed to the network stack from the driver
> represents data as it is on the wire, while it is pointing
> directly to data in destination buffers.
> As a result, data from page frags should not be copied out to
> the linear part. To avoid needless copies, such as when using
> skb_condense, we mark the skb->ddp_crc bit. This bit will be
> used to indicate both ddp and crc offload (next patch in series).
> 
> A follow-up patch will use this interface for DDP in NVMe-TCP.
> 
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>  include/linux/netdev_features.h    |   2 +
>  include/linux/netdevice.h          |   5 ++
>  include/linux/skbuff.h             |   4 +
>  include/net/inet_connection_sock.h |   4 +
>  include/net/tcp_ddp.h              | 136 +++++++++++++++++++++++++++++
>  net/Kconfig                        |   9 ++
>  net/core/skbuff.c                  |   8 +-
>  net/ethtool/common.c               |   1 +
>  net/ipv4/tcp_input.c               |   8 ++
>  net/ipv4/tcp_ipv4.c                |   3 +
>  net/ipv4/tcp_offload.c             |   3 +
>  11 files changed, 182 insertions(+), 1 deletion(-)
>  create mode 100644 include/net/tcp_ddp.h
> 
> diff --git a/include/linux/netdev_features.h b/include/linux/netdev_features.h
> index c06d6aaba9df..7977371d2dd1 100644
> --- a/include/linux/netdev_features.h
> +++ b/include/linux/netdev_features.h
> @@ -85,6 +85,7 @@ enum {
>  
>  	NETIF_F_HW_MACSEC_BIT,		/* Offload MACsec operations */
>  	NETIF_F_GRO_UDP_FWD_BIT,	/* Allow UDP GRO for forwarding */
> +	NETIF_F_HW_TCP_DDP_BIT,		/* TCP direct data placement offload */
>  
>  	/*
>  	 * Add your fresh new feature above and remember to update
> @@ -159,6 +160,7 @@ enum {
>  #define NETIF_F_GSO_FRAGLIST	__NETIF_F(GSO_FRAGLIST)
>  #define NETIF_F_HW_MACSEC	__NETIF_F(HW_MACSEC)
>  #define NETIF_F_GRO_UDP_FWD	__NETIF_F(GRO_UDP_FWD)
> +#define NETIF_F_HW_TCP_DDP	__NETIF_F(HW_TCP_DDP)
>  
>  /* Finds the next feature with the highest number of the range of start till 0.
>   */
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index e9e7ada07ea1..bd28520e30f2 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -941,6 +941,7 @@ struct dev_ifalias {
>  
>  struct devlink;
>  struct tlsdev_ops;
> +struct tcp_ddp_dev_ops;
>  
>  struct netdev_name_node {
>  	struct hlist_node hlist;
> @@ -1942,6 +1943,10 @@ struct net_device {
>  	const struct tlsdev_ops *tlsdev_ops;
>  #endif
>  
> +#ifdef CONFIG_TCP_DDP
> +	const struct tcp_ddp_dev_ops *tcp_ddp_ops;
> +#endif
> +
>  	const struct header_ops *header_ops;
>  
>  	unsigned char		operstate;
> diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
> index 9313b5aaf45b..6d00d62628c8 100644
> --- a/include/linux/skbuff.h
> +++ b/include/linux/skbuff.h
> @@ -684,6 +684,7 @@ typedef unsigned char *sk_buff_data_t;
>   *		CHECKSUM_UNNECESSARY (max 3)
>   *	@dst_pending_confirm: need to confirm neighbour
>   *	@decrypted: Decrypted SKB
> + *	@ddp_crc: DDP or CRC offloaded
>   *	@napi_id: id of the NAPI struct this skb came from
>   *	@sender_cpu: (aka @napi_id) source CPU in XPS
>   *	@secmark: security marking
> @@ -860,6 +861,9 @@ struct sk_buff {
>  #ifdef CONFIG_TLS_DEVICE
>  	__u8			decrypted:1;
>  #endif
> +#ifdef CONFIG_TCP_DDP
> +	__u8                    ddp_crc:1;
> +#endif
>  
>  #ifdef CONFIG_NET_SCHED
>  	__u16			tc_index;	/* traffic control index */
> diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
> index c11f80f328f1..f9eb28f14d43 100644
> --- a/include/net/inet_connection_sock.h
> +++ b/include/net/inet_connection_sock.h
> @@ -66,6 +66,8 @@ struct inet_connection_sock_af_ops {
>   * @icsk_ulp_ops	   Pluggable ULP control hook
>   * @icsk_ulp_data	   ULP private data
>   * @icsk_clean_acked	   Clean acked data hook
> + * @icsk_ulp_ddp_ops	   Pluggable ULP direct data placement control hook
> + * @icsk_ulp_ddp_data	   ULP direct data placement private data
>   * @icsk_listen_portaddr_node	hash to the portaddr listener hashtable
>   * @icsk_ca_state:	   Congestion control state
>   * @icsk_retransmits:	   Number of unrecovered [RTO] timeouts
> @@ -96,6 +98,8 @@ struct inet_connection_sock {
>  	const struct tcp_ulp_ops  *icsk_ulp_ops;
>  	void __rcu		  *icsk_ulp_data;
>  	void (*icsk_clean_acked)(struct sock *sk, u32 acked_seq);
> +	const struct tcp_ddp_ulp_ops  *icsk_ulp_ddp_ops;
> +	void __rcu		  *icsk_ulp_ddp_data;
>  	struct hlist_node         icsk_listen_portaddr_node;
>  	unsigned int		  (*icsk_sync_mss)(struct sock *sk, u32 pmtu);
>  	__u8			  icsk_ca_state:5,
> diff --git a/include/net/tcp_ddp.h b/include/net/tcp_ddp.h
> new file mode 100644
> index 000000000000..31e5b1a16d0f
> --- /dev/null
> +++ b/include/net/tcp_ddp.h
> @@ -0,0 +1,136 @@
> +/* SPDX-License-Identifier: GPL-2.0
> + *
> + * tcp_ddp.h
> + *	Author:	Boris Pismenny <borisp at mellanox.com>
> + *	Copyright (C) 2021 Mellanox Technologies.
> + */
> +#ifndef _TCP_DDP_H
> +#define _TCP_DDP_H
> +
> +#include <linux/netdevice.h>
> +#include <net/inet_connection_sock.h>
> +#include <net/sock.h>
> +
> +/* limits returned by the offload driver, zero means don't care */
> +struct tcp_ddp_limits {
> +	int	 max_ddp_sgl_len;
> +};
> +
> +enum tcp_ddp_type {
> +	TCP_DDP_NVME = 1,
> +};
> +
> +/**
> + * struct tcp_ddp_config - Generic tcp ddp configuration: tcp ddp IO queue
> + * config implementations must use this as the first member.
> + * Add new instances of tcp_ddp_config below (nvme-tcp, etc.).
> + */
> +struct tcp_ddp_config {
> +	enum tcp_ddp_type    type;
> +	unsigned char        buf[];
> +};
> +
> +/**
> + * struct nvme_tcp_ddp_config - nvme tcp ddp configuration for an IO queue
> + *
> + * @pfv:        pdu version (e.g., NVME_TCP_PFV_1_0)
> + * @cpda:       controller pdu data alignmend (dwords, 0's based)
> + * @dgst:       digest types enabled.
> + *              The netdev will offload crc if ddp_crc is supported.
> + * @queue_size: number of nvme-tcp IO queue elements
> + * @queue_id:   queue identifier
> + * @cpu_io:     cpu core running the IO thread for this queue
> + */
> +struct nvme_tcp_ddp_config {
> +	struct tcp_ddp_config   cfg;
> +
> +	u16			pfv;
> +	u8			cpda;
> +	u8			dgst;
> +	int			queue_size;
> +	int			queue_id;
> +	int			io_cpu;
> +};
> +
> +/**
> + * struct tcp_ddp_io - tcp ddp configuration for an IO request.
> + *
> + * @command_id:  identifier on the wire associated with these buffers
> + * @nents:       number of entries in the sg_table
> + * @sg_table:    describing the buffers for this IO request
> + * @first_sgl:   first SGL in sg_table
> + */
> +struct tcp_ddp_io {
> +	u32			command_id;
> +	int			nents;
> +	struct sg_table		sg_table;
> +	struct scatterlist	first_sgl[SG_CHUNK_SIZE];
> +};
> +
> +/* struct tcp_ddp_dev_ops - operations used by an upper layer protocol to configure ddp offload
> + *
> + * @tcp_ddp_limits:    limit the number of scatter gather entries per IO.
> + *                     the device driver can use this to limit the resources allocated per queue.
> + * @tcp_ddp_sk_add:    add offload for the queue represennted by the socket+config pair.
> + *                     this function is used to configure either copy, crc or both offloads.
> + * @tcp_ddp_sk_del:    remove offload from the socket, and release any device related resources.
> + * @tcp_ddp_setup:     request copy offload for buffers associated with a command_id in tcp_ddp_io.
> + * @tcp_ddp_teardown:  release offload resources association between buffers and command_id in
> + *                     tcp_ddp_io.
> + * @tcp_ddp_resync:    respond to the driver's resync_request. Called only if resync is successful.
> + */
> +struct tcp_ddp_dev_ops {
> +	int (*tcp_ddp_limits)(struct net_device *netdev,
> +			      struct tcp_ddp_limits *limits);
> +	int (*tcp_ddp_sk_add)(struct net_device *netdev,
> +			      struct sock *sk,
> +			      struct tcp_ddp_config *config);
> +	void (*tcp_ddp_sk_del)(struct net_device *netdev,
> +			       struct sock *sk);
> +	int (*tcp_ddp_setup)(struct net_device *netdev,
> +			     struct sock *sk,
> +			     struct tcp_ddp_io *io);
> +	int (*tcp_ddp_teardown)(struct net_device *netdev,
> +				struct sock *sk,
> +				struct tcp_ddp_io *io,
> +				void *ddp_ctx);
> +	void (*tcp_ddp_resync)(struct net_device *netdev,
> +			       struct sock *sk, u32 seq);
> +};
> +
> +#define TCP_DDP_RESYNC_REQ BIT(0)
> +
> +/**
> + * struct tcp_ddp_ulp_ops - Interface to register uppper layer Direct Data Placement (DDP) TCP offload
> + */

Super trivial (my mail filter just happened to pick up on it), but if
you happen to respin:

s/uppper/upper/

> +struct tcp_ddp_ulp_ops {
> +	/* NIC requests ulp to indicate if @seq is the start of a message */
> +	bool (*resync_request)(struct sock *sk, u32 seq, u32 flags);
> +	/* NIC driver informs the ulp that ddp teardown is done - used for async completions*/
> +	void (*ddp_teardown_done)(void *ddp_ctx);
> +};
> +
> +/**
> + * struct tcp_ddp_ctx - Generic tcp ddp context: device driver per queue contexts must
> + * use this as the first member.
> + */
> +struct tcp_ddp_ctx {
> +	enum tcp_ddp_type    type;
> +	unsigned char        buf[];
> +};
> +
> +static inline struct tcp_ddp_ctx *tcp_ddp_get_ctx(const struct sock *sk)
> +{
> +	struct inet_connection_sock *icsk = inet_csk(sk);
> +
> +	return (__force struct tcp_ddp_ctx *)icsk->icsk_ulp_ddp_data;
> +}
> +
> +static inline void tcp_ddp_set_ctx(struct sock *sk, void *ctx)
> +{
> +	struct inet_connection_sock *icsk = inet_csk(sk);
> +
> +	rcu_assign_pointer(icsk->icsk_ulp_ddp_data, ctx);
> +}
> +
> +#endif //_TCP_DDP_H
> diff --git a/net/Kconfig b/net/Kconfig
> index f4c32d982af6..3876861cdc90 100644
> --- a/net/Kconfig
> +++ b/net/Kconfig
> @@ -457,6 +457,15 @@ config ETHTOOL_NETLINK
>  	  netlink. It provides better extensibility and some new features,
>  	  e.g. notification messages.
>  
> +config TCP_DDP
> +	bool "TCP direct data placement offload"
> +	default n
> +	help
> +	  Direct Data Placement (DDP) offload for TCP enables ULP, such as
> +	  NVMe-TCP/iSCSI, to request the NIC to place TCP payload data
> +	  of a command response directly into kernel pages.
> +
> +
>  endif   # if NET
>  
>  # Used by archs to tell that they support BPF JIT compiler plus which flavour.
> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
> index 2af12f7e170c..ef3f2714994d 100644
> --- a/net/core/skbuff.c
> +++ b/net/core/skbuff.c
> @@ -69,6 +69,7 @@
>  #include <net/xfrm.h>
>  #include <net/mpls.h>
>  #include <net/mptcp.h>
> +#include <net/tcp_ddp.h>
>  
>  #include <linux/uaccess.h>
>  #include <trace/events/skb.h>
> @@ -6185,9 +6186,14 @@ EXPORT_SYMBOL(pskb_extract);
>   */
>  void skb_condense(struct sk_buff *skb)
>  {
> +	bool is_ddp = false;
> +
> +#ifdef CONFIG_TCP_DDP
> +	is_ddp = skb->ddp_crc;
> +#endif
>  	if (skb->data_len) {
>  		if (skb->data_len > skb->end - skb->tail ||
> -		    skb_cloned(skb))
> +		    skb_cloned(skb) || is_ddp)
>  			return;
>  
>  		/* Nice, we can free page frag(s) right now */
> diff --git a/net/ethtool/common.c b/net/ethtool/common.c
> index 181220101a6e..bbf83cd01106 100644
> --- a/net/ethtool/common.c
> +++ b/net/ethtool/common.c
> @@ -69,6 +69,7 @@ const char netdev_features_strings[NETDEV_FEATURE_COUNT][ETH_GSTRING_LEN] = {
>  	[NETIF_F_GRO_FRAGLIST_BIT] =	 "rx-gro-list",
>  	[NETIF_F_HW_MACSEC_BIT] =	 "macsec-hw-offload",
>  	[NETIF_F_GRO_UDP_FWD_BIT] =	 "rx-udp-gro-forwarding",
> +	[NETIF_F_HW_TCP_DDP_BIT] =	 "tcp-ddp-offload",
>  };
>  
>  const char
> diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> index a8f8f9815953..0ae1ffca090d 100644
> --- a/net/ipv4/tcp_input.c
> +++ b/net/ipv4/tcp_input.c
> @@ -5149,6 +5149,9 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
>  		memcpy(nskb->cb, skb->cb, sizeof(skb->cb));
>  #ifdef CONFIG_TLS_DEVICE
>  		nskb->decrypted = skb->decrypted;
> +#endif
> +#ifdef CONFIG_TCP_DDP
> +		nskb->ddp_crc = skb->ddp_crc;
>  #endif
>  		TCP_SKB_CB(nskb)->seq = TCP_SKB_CB(nskb)->end_seq = start;
>  		if (list)
> @@ -5182,6 +5185,11 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
>  #ifdef CONFIG_TLS_DEVICE
>  				if (skb->decrypted != nskb->decrypted)
>  					goto end;
> +#endif
> +#ifdef CONFIG_TCP_DDP
> +
> +				if (skb->ddp_crc != nskb->ddp_crc)
> +					goto end;
>  #endif
>  			}
>  		}
> diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
> index 777306b5bc22..b114fc870513 100644
> --- a/net/ipv4/tcp_ipv4.c
> +++ b/net/ipv4/tcp_ipv4.c
> @@ -1810,6 +1810,9 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb)
>  	      TCP_SKB_CB(skb)->tcp_flags) & (TCPHDR_ECE | TCPHDR_CWR)) ||
>  #ifdef CONFIG_TLS_DEVICE
>  	    tail->decrypted != skb->decrypted ||
> +#endif
> +#ifdef CONFIG_TCP_DDP
> +	    tail->ddp_crc != skb->ddp_crc ||
>  #endif
>  	    thtail->doff != th->doff ||
>  	    memcmp(thtail + 1, th + 1, hdrlen - sizeof(*th)))
> diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
> index e09147ac9a99..3ce196375d94 100644
> --- a/net/ipv4/tcp_offload.c
> +++ b/net/ipv4/tcp_offload.c
> @@ -262,6 +262,9 @@ struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb)
>  #ifdef CONFIG_TLS_DEVICE
>  	flush |= p->decrypted ^ skb->decrypted;
>  #endif
> +#ifdef CONFIG_TCP_DDP
> +	flush |= p->ddp_crc ^ skb->ddp_crc;
> +#endif
>  
>  	if (flush || skb_gro_receive(p, skb)) {
>  		mss = 1;
> -- 
> 2.24.1
> 

-- 
Tom Parkin
Katalix Systems Ltd
https://katalix.com
Catalysts for your Embedded Linux software development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210202/24a6b69e/attachment-0001.sig>

From andriy.shevchenko at linux.intel.com  Tue Feb  2 06:21:09 2021
From: andriy.shevchenko at linux.intel.com (Andy Shevchenko)
Date: Tue, 2 Feb 2021 13:21:09 +0200
Subject: [PATCH V2 3/3] Adding
 device_dma_parameters->offset_preserve_mask to NVMe driver.
In-Reply-To: <CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>
 <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
 <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
 <CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com>
Message-ID: <YBk1pUbQ/JZQ7WZe@smile.fi.intel.com>

On Mon, Feb 01, 2021 at 04:25:55PM -0800, Jianxiong Gao wrote:

> +       if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1))

Side note: we have DMA_BIT_MASK(), please use it.

> +               dev_warn(dev->dev, "dma_set_min_align_mask failed to
> set offset\n");

-- 
With Best Regards,
Andy Shevchenko




From gregkh at linuxfoundation.org  Tue Feb  2 06:53:02 2021
From: gregkh at linuxfoundation.org (Greg KH)
Date: Tue, 2 Feb 2021 12:53:02 +0100
Subject: [PATCH V2 0/3] SWIOTLB: Preserve swiotlb map offset when needed.
In-Reply-To: <20210201183017.3339130-1-jxgao@google.com>
References: <20210201183017.3339130-1-jxgao@google.com>
Message-ID: <YBk9HsrdvcUb50Kz@kroah.com>

On Mon, Feb 01, 2021 at 10:30:14AM -0800, Jianxiong Gao wrote:
> NVMe driver and other applications may depend on the data offset
> to operate correctly. Currently when unaligned data is mapped via
> SWIOTLB, the data is mapped as slab aligned with the SWIOTLB. This
> patch adds an option to make sure the mapped data preserves its
> offset of the orginal addrss.
> 
> Without the patch when creating xfs formatted disk on NVMe backends,
> with swiotlb=force in kernel boot option, creates the following error:
> meta-data=/dev/nvme2n1   isize=512    agcount=4, agsize=131072 blks
>          =               sectsz=512   attr=2, projid32bit=1
>          =               crc=1        finobt=1, sparse=0, rmapbt=0, refl
> ink=0
> data     =               bsize=4096   blocks=524288, imaxpct=25
>          =               sunit=0      swidth=0 blks
> naming   =version 2      bsize=4096   ascii-ci=0 ftype=1
> log      =internal log   bsize=4096   blocks=2560, version=2
>          =               sectsz=512   sunit=0 blks, lazy-count=1
> realtime =none           extsz=4096   blocks=0, rtextents=0
> mkfs.xfs: pwrite failed: Input/output error
> 
> Jianxiong Gao (3):
>   Adding page_offset_mask to device_dma_parameters
>   Add swiotlb offset preserving mapping when
>     dma_dma_parameters->page_offset_mask is non zero.
>   Adding device_dma_parameters->offset_preserve_mask to NVMe driver.
> 
>  drivers/nvme/host/pci.c     |  4 ++++
>  include/linux/device.h      |  1 +
>  include/linux/dma-mapping.h | 17 +++++++++++++++++
>  kernel/dma/swiotlb.c        | 16 +++++++++++++++-
>  4 files changed, 37 insertions(+), 1 deletion(-)
> 
> -- 
> 2.27.0
> 

You forgot to mention somewhere, what changed from v1 to v2 :(


From robin.murphy at arm.com  Tue Feb  2 07:07:52 2021
From: robin.murphy at arm.com (Robin Murphy)
Date: Tue, 2 Feb 2021 12:07:52 +0000
Subject: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask
 to NVMe driver.
In-Reply-To: <YBk1pUbQ/JZQ7WZe@smile.fi.intel.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>
 <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
 <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
 <CAMGD6P0uwVxKuG503ahGTbPcwb+y2wRXSiE_gvzfdUrMfZ5YbA@mail.gmail.com>
 <YBk1pUbQ/JZQ7WZe@smile.fi.intel.com>
Message-ID: <a34328d1-3918-1eca-b632-57be6a40baee@arm.com>

On 2021-02-02 11:21, Andy Shevchenko wrote:
> On Mon, Feb 01, 2021 at 04:25:55PM -0800, Jianxiong Gao wrote:
> 
>> +       if (dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1))
> 
> Side note: we have DMA_BIT_MASK(), please use it.

FWIW I'd actually disagree on that point. Conceptually, this is a very 
different thing from dev->{coherent_}dma_mask. It does not need to 
handle everything up to 2^64-1 correctly without overflow issues, and 
data alignments typically *are* defined in terms of sizes rather than 
numbers of bits.

In fact, it might be neat to just have callers pass a size directly to a 
dma_set_min_align() interface which asserts that it is a power of two 
and stores it as a mask internally.

Robin.

> 
>> +               dev_warn(dev->dev, "dma_set_min_align_mask failed to
>> set offset\n");
> 


From Jeff.Lien at wdc.com  Tue Feb  2 10:06:22 2021
From: Jeff.Lien at wdc.com (Jeffrey Lien)
Date: Tue, 2 Feb 2021 15:06:22 +0000
Subject: Buffer I/O Errors from Zoned NVME devices
In-Reply-To: <BL0PR04MB65142D8FBF2B6BC88E32DCCEE7B69@BL0PR04MB6514.namprd04.prod.outlook.com>
References: <BYAPR04MB56561E48A54530BC7F6DCF30EAB69@BYAPR04MB5656.namprd04.prod.outlook.com>
 <20210201175306.GA2127465@dhcp-10-100-145-180.wdc.com>
 <20210201180257.GA14971@lst.de>
 <BL0PR04MB65142D8FBF2B6BC88E32DCCEE7B69@BL0PR04MB6514.namprd04.prod.outlook.com>
Message-ID: <BYAPR04MB5656F20169A935B9100ADC1DEAB59@BYAPR04MB5656.namprd04.prod.outlook.com>

Keith, Christoph, Damien,
This errors are happening on both the 5.9 and 5.10.7 kernels.  CONFIG_BLK_DEV_ZONED is set to y in the .config file.   

I will try the patch to disable partition scanning that Keith suggested.  I'll also get the latest FW loaded and see if that resolves the issue.  

-----Original Message-----
From: Damien Le Moal <Damien.LeMoal at wdc.com> 
Sent: Monday, February 1, 2021 3:05 PM
To: hch at lst.de; Keith Busch <kbusch at kernel.org>
Cc: Jeffrey Lien <Jeff.Lien at wdc.com>; linux-nvme at lists.infradead.org
Subject: Re: Buffer I/O Errors from Zoned NVME devices

On 2021/02/02 3:03, hch at lst.de wrote:
> On Mon, Feb 01, 2021 at 09:53:06AM -0800, Keith Busch wrote:
>> On Mon, Feb 01, 2021 at 02:36:12PM +0000, Jeffrey Lien wrote:
>>> Christoph, Keith
>>> We're seeing a lot of these Buffer I/O errors with our zoned nvme devices.  One of the FW developers looked into it and had the following explanation:
>>> All these Reads are from the kernel during enumeration and for LBAs that are in last zone's hole hence expected to return boundary error which is getting logged by kernel.
>>>
>>> [65281.936988] Buffer I/O error on dev nvme1n2, logical block 
>>> 3800039296, async page read [65281.937165] blk_update_request: I/O 
>>> error, dev nvme1n2, sector 3800039297 op 0x0:(READ) flags 0x0 
>>> phys_seg 1 prio class 0 [65281.937166] Buffer I/O error on dev 
>>> nvme1n2, logical block 3800039297, async page read [65281.937335] 
>>> blk_update_request: I/O error, dev nvme1n2, sector 3800039298 op 
>>> 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 [65281.937336] Buffer 
>>> I/O error on dev nvme1n2, logical block 3800039298, async page read 
>>> [65281.937498] blk_update_request: I/O error, dev nvme1n2, sector 
>>> 3800039299 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
>>>
>>> Are you aware of this issue and if so, do you have any recommendations on how to avoid or resolve?  
>>
>> Is this from the partition scanning? We don't partition zoned 
>> devices, so I think we can skip it. Does the following resolve the issue?
> 
> We already have special zoned device handling in the partitioning code.

Partitions are ignored and warning printed, but the partition table is still being read...

> 
> But NVMe should make sure to never span a zone boundary as we set the 
> chunk size to avoid that.
> 
> What kernel version is this?  Is CONFIG_BLK_DEV_ZONED enabled?

I had a very similar problem doing zonefs tests on Matias machine on a ZNS drive last week. The problem was the firmware... An upgrade to the latest version fixed the issue. Not sure what FW rev you are running here, but upgrading might solve this.

> 


--
Damien Le Moal
Western Digital Research


From sashal at kernel.org  Tue Feb  2 10:06:15 2021
From: sashal at kernel.org (Sasha Levin)
Date: Tue,  2 Feb 2021 10:06:15 -0500
Subject: [PATCH AUTOSEL 5.10 25/25] nvme-pci: add the DISABLE_WRITE_ZEROES
 quirk for a SPCC device
In-Reply-To: <20210202150615.1864175-1-sashal@kernel.org>
References: <20210202150615.1864175-1-sashal@kernel.org>
Message-ID: <20210202150615.1864175-25-sashal@kernel.org>

From: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>

[ Upstream commit 899199292b14b7c735808a37517de4dd2160c300 ]

This adds a quirk for SPCC 256GB NVMe 1.3 drive which fixes timeouts and
I/O errors due to the fact that the controller does not properly
handle the Write Zeroes command:

[ 2745.659527] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G            E 5.10.6-BET #1
[ 2745.659528] Hardware name: System manufacturer System Product Name/PRIME X570-P, BIOS 3001 12/04/2020
[ 2776.138874] nvme nvme1: I/O 414 QID 3 timeout, aborting
[ 2776.138886] nvme nvme1: I/O 415 QID 3 timeout, aborting
[ 2776.138891] nvme nvme1: I/O 416 QID 3 timeout, aborting
[ 2776.138895] nvme nvme1: I/O 417 QID 3 timeout, aborting
[ 2776.138912] nvme nvme1: Abort status: 0x0
[ 2776.138921] nvme nvme1: I/O 428 QID 3 timeout, aborting
[ 2776.138922] nvme nvme1: Abort status: 0x0
[ 2776.138925] nvme nvme1: Abort status: 0x0
[ 2776.138974] nvme nvme1: Abort status: 0x0
[ 2776.138977] nvme nvme1: Abort status: 0x0
[ 2806.346792] nvme nvme1: I/O 414 QID 3 timeout, reset controller
[ 2806.363566] nvme nvme1: 15/0/0 default/read/poll queues
[ 2836.554298] nvme nvme1: I/O 415 QID 3 timeout, disable controller
[ 2836.672064] blk_update_request: I/O error, dev nvme1n1, sector 16350 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672072] blk_update_request: I/O error, dev nvme1n1, sector 16093 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672074] blk_update_request: I/O error, dev nvme1n1, sector 15836 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672076] blk_update_request: I/O error, dev nvme1n1, sector 15579 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672078] blk_update_request: I/O error, dev nvme1n1, sector 15322 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672080] blk_update_request: I/O error, dev nvme1n1, sector 15065 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672082] blk_update_request: I/O error, dev nvme1n1, sector 14808 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672083] blk_update_request: I/O error, dev nvme1n1, sector 14551 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672085] blk_update_request: I/O error, dev nvme1n1, sector 14294 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672087] blk_update_request: I/O error, dev nvme1n1, sector 14037 op 0x9:(WRITE_ZEROES) flags 0x0 phys_seg 0 prio class 0
[ 2836.672121] nvme nvme1: failed to mark controller live state
[ 2836.672123] nvme nvme1: Removing after probe failure status: -19
[ 2836.689016] Aborting journal on device dm-0-8.
[ 2836.689024] Buffer I/O error on dev dm-0, logical block 25198592, lost sync page write
[ 2836.689027] JBD2: Error -5 detected when updating journal superblock for dm-0-8.

Reported-by: Bradley Chapman <chapman6235 at comcast.net>
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Tested-by: Bradley Chapman <chapman6235 at comcast.net>
Reviewed-by: Keith Busch <kbusch at kernel.org>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/pci.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 77f615568194d..a4b169408bbf0 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3248,6 +3248,8 @@ static const struct pci_device_id nvme_id_table[] = {
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(0x15b7, 0x2001),   /*  Sandisk Skyhawk */
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+	{ PCI_DEVICE(0x1d97, 0x2263),   /* SPCC */
+		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2001),
 		.driver_data = NVME_QUIRK_SINGLE_VECTOR },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2003) },
-- 
2.27.0



From kbusch at kernel.org  Tue Feb  2 10:22:50 2021
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 2 Feb 2021 07:22:50 -0800
Subject: Buffer I/O Errors from Zoned NVME devices
In-Reply-To: <BYAPR04MB5656F20169A935B9100ADC1DEAB59@BYAPR04MB5656.namprd04.prod.outlook.com>
References: <BYAPR04MB56561E48A54530BC7F6DCF30EAB69@BYAPR04MB5656.namprd04.prod.outlook.com>
 <20210201175306.GA2127465@dhcp-10-100-145-180.wdc.com>
 <20210201180257.GA14971@lst.de>
 <BL0PR04MB65142D8FBF2B6BC88E32DCCEE7B69@BL0PR04MB6514.namprd04.prod.outlook.com>
 <BYAPR04MB5656F20169A935B9100ADC1DEAB59@BYAPR04MB5656.namprd04.prod.outlook.com>
Message-ID: <20210202152250.GB2128135@dhcp-10-100-145-180.wdc.com>

On Tue, Feb 02, 2021 at 03:06:22PM +0000, Jeffrey Lien wrote:
> Keith, Christoph, Damien,
> This errors are happening on both the 5.9 and 5.10.7 kernels.  CONFIG_BLK_DEV_ZONED is set to y in the .config file.   
> 
> I will try the patch to disable partition scanning that Keith suggested.  I'll also get the latest FW loaded and see if that resolves the issue.  

After re-reading FW dev's explanation, it sounds like something is off
with the implementation. The spec only allows a "boundary error" if
you're crossing zones, but you said the reads are in the last zone, so
there's no opprotunity to cross to the next zone. 

What did you mean by the "zone's hole"? Does this drive have ZCAP less
than ZSZE and we're reading from unmapped LBAs? If so, I think we are
supposed to be allowed to read these, but we just can't write them.


> -----Original Message-----
> From: Damien Le Moal <Damien.LeMoal at wdc.com> 
> Sent: Monday, February 1, 2021 3:05 PM
> To: hch at lst.de; Keith Busch <kbusch at kernel.org>
> Cc: Jeffrey Lien <Jeff.Lien at wdc.com>; linux-nvme at lists.infradead.org
> Subject: Re: Buffer I/O Errors from Zoned NVME devices
> 
> On 2021/02/02 3:03, hch at lst.de wrote:
> > On Mon, Feb 01, 2021 at 09:53:06AM -0800, Keith Busch wrote:
> >> On Mon, Feb 01, 2021 at 02:36:12PM +0000, Jeffrey Lien wrote:
> >>> Christoph, Keith
> >>> We're seeing a lot of these Buffer I/O errors with our zoned nvme devices.  One of the FW developers looked into it and had the following explanation:
> >>> All these Reads are from the kernel during enumeration and for LBAs that are in last zone's hole hence expected to return boundary error which is getting logged by kernel.
> >>>
> >>> [65281.936988] Buffer I/O error on dev nvme1n2, logical block 
> >>> 3800039296, async page read [65281.937165] blk_update_request: I/O 
> >>> error, dev nvme1n2, sector 3800039297 op 0x0:(READ) flags 0x0 
> >>> phys_seg 1 prio class 0 [65281.937166] Buffer I/O error on dev 
> >>> nvme1n2, logical block 3800039297, async page read [65281.937335] 
> >>> blk_update_request: I/O error, dev nvme1n2, sector 3800039298 op 
> >>> 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 [65281.937336] Buffer 
> >>> I/O error on dev nvme1n2, logical block 3800039298, async page read 
> >>> [65281.937498] blk_update_request: I/O error, dev nvme1n2, sector 
> >>> 3800039299 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
> >>>
> >>> Are you aware of this issue and if so, do you have any recommendations on how to avoid or resolve?  
> >>
> >> Is this from the partition scanning? We don't partition zoned 
> >> devices, so I think we can skip it. Does the following resolve the issue?
> > 
> > We already have special zoned device handling in the partitioning code.
> 
> Partitions are ignored and warning printed, but the partition table is still being read...
> 
> > 
> > But NVMe should make sure to never span a zone boundary as we set the 
> > chunk size to avoid that.
> > 
> > What kernel version is this?  Is CONFIG_BLK_DEV_ZONED enabled?
> 
> I had a very similar problem doing zonefs tests on Matias machine on a ZNS drive last week. The problem was the firmware... An upgrade to the latest version fixed the issue. Not sure what FW rev you are running here, but upgrading might solve this.
> 
> > 
> 
> 
> --
> Damien Le Moal
> Western Digital Research


From axboe at kernel.dk  Tue Feb  2 12:43:28 2021
From: axboe at kernel.dk (Jens Axboe)
Date: Tue, 2 Feb 2021 10:43:28 -0700
Subject: [GIT PULL] first round of nvme updates for 5.12
In-Reply-To: <YBkcYFf7hJfpK+vl@infradead.org>
References: <YBkcYFf7hJfpK+vl@infradead.org>
Message-ID: <fba2a647-1885-f508-0f40-e3968b005cbe@kernel.dk>

On 2/2/21 2:33 AM, Christoph Hellwig wrote:
>   git://git.infradead.org/nvme.git tags/nvme-5.21-2020-02-02

Pulled, thanks.

-- 
Jens Axboe



From gerlitz.or at gmail.com  Tue Feb  2 13:00:51 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Tue, 2 Feb 2021 20:00:51 +0200
Subject: [PATCH v3 net-next 01/21] iov_iter: Introduce new procedures for
 copy to iter/pages
In-Reply-To: <20210201173548.GA12960@lst.de>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-2-borisp@mellanox.com>
 <20210201173548.GA12960@lst.de>
Message-ID: <CAJ3xEMjLKoQe_OB_L+w2wwUGck74Gm6=GPA=CK73QpeFbXr7Bw@mail.gmail.com>

On Mon, Feb 1, 2021 at 7:38 PM Christoph Hellwig <hch at lst.de> wrote:
> On Mon, Feb 01, 2021 at 12:04:49PM +0200, Boris Pismenny wrote:
> > +static __always_inline __must_check
> > +size_t ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
> > +{
> > +     if (unlikely(!check_copy_size(addr, bytes, true)))
> > +             return 0;
> > +     else
> > +             return _ddp_copy_to_iter(addr, bytes, i);
> > +}
>
> No need for the else after a return, and the normal kernel convention
> double underscores for magic internal functions.

ack for the no-else-after-a-return

Re the double underscoring, I was not sure, e.g the non-ddp counterpart
(_copy_to_iter) is single underscored

> But more importantly: does this belong into the generic header without
> and comments what the ddp means and when it should be used?

will look into this, any idea for a more suitable location?

> > +static void ddp_memcpy_to_page(struct page *page, size_t offset, const char *from, size_t len)
>
> Overly long line.  But we're also looking into generic helpers for
> this kind of things, not sure if they made it to linux-next in the
> meantime, but please check.

This is what I found in linux-next - note sure if you were referring to it

commit 11432a3cc061c39475295be533c3674c4f8a6d0b
Author: David Howells <dhowells at redhat.com>

    iov_iter: Add ITER_XARRAY

> > +size_t _ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
> > +{
> > +     const char *from = addr;
> > +     if (unlikely(iov_iter_is_pipe(i)))
> > +             return copy_pipe_to_iter(addr, bytes, i);
> > +     if (iter_is_iovec(i))
> > +             might_fault();
> > +     iterate_and_advance(i, bytes, v,
> > +             copyout(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
> > +             ddp_memcpy_to_page(v.bv_page, v.bv_offset,
> > +                                (from += v.bv_len) - v.bv_len, v.bv_len),
> > +             memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len)
> > +             )
> > +
> > +     return bytes;
> > +}
>
> This bloats every kernel build, so please move it into a conditionally built file.

ack

>  And please document the whole thing.

ok


From gerlitz.or at gmail.com  Tue Feb  2 13:09:50 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Tue, 2 Feb 2021 20:09:50 +0200
Subject: [PATCH v3 net-next 06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <20210201173719.GB12960@lst.de>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-7-borisp@mellanox.com>
 <20210201173719.GB12960@lst.de>
Message-ID: <CAJ3xEMgnWKFkN3EOksLeqRMS+Ku-8oxOqwt1COkYwq5i1UYyJg@mail.gmail.com>

On Mon, Feb 1, 2021 at 7:39 PM Christoph Hellwig <hch at lst.de> wrote:
>> +static
>> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
>
> Please use the same coding style as the rest of the file, and not some
> weirdo version.

ack


From gerlitz.or at gmail.com  Tue Feb  2 13:14:30 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Tue, 2 Feb 2021 20:14:30 +0200
Subject: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <20210201173744.GC12960@lst.de>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-8-borisp@mellanox.com>
 <20210201173744.GC12960@lst.de>
Message-ID: <CAJ3xEMhninJE5zw7=QFL4gBVkH=1tAmQHyq7tKMqcSJ_KkDsWQ@mail.gmail.com>

On Mon, Feb 1, 2021 at 7:40 PM Christoph Hellwig <hch at lst.de> wrote:
> Given how much ddp code there is can you split it into a separate file?

mmm, do we need to check the preferences or get to a consensus among
the maintainers for that one?


From lkp at intel.com  Tue Feb  2 17:52:24 2021
From: lkp at intel.com (kernel test robot)
Date: Wed, 03 Feb 2021 06:52:24 +0800
Subject: [linux-nvme:nvme-5.11] BUILD SUCCESS
 46121fa7c2dc55bcbb729b6a2ab323aa1e8986cf
Message-ID: <6019d7a8.IK+/fneCXftyxpm+%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.11
branch HEAD: 46121fa7c2dc55bcbb729b6a2ab323aa1e8986cf  update the email address for Keith Bush

elapsed time: 727m

configs tested: 167
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
sh                           se7722_defconfig
sh                        edosk7760_defconfig
sh                        dreamcast_defconfig
xtensa                  cadence_csp_defconfig
m68k                          multi_defconfig
c6x                        evmc6457_defconfig
powerpc                      mgcoge_defconfig
mips                        qi_lb60_defconfig
mips                        nlm_xlr_defconfig
powerpc                     kilauea_defconfig
powerpc               mpc834x_itxgp_defconfig
arm                        mvebu_v5_defconfig
xtensa                          iss_defconfig
sh                            hp6xx_defconfig
arm                       spear13xx_defconfig
mips                           ip22_defconfig
arm                         socfpga_defconfig
mips                      malta_kvm_defconfig
powerpc                    klondike_defconfig
powerpc                     ep8248e_defconfig
parisc                generic-32bit_defconfig
riscv                               defconfig
powerpc                  mpc866_ads_defconfig
arm                       imx_v6_v7_defconfig
mips                          malta_defconfig
arm                       multi_v4t_defconfig
c6x                        evmc6474_defconfig
s390                             alldefconfig
arm                          exynos_defconfig
sh                              ul2_defconfig
powerpc                 mpc834x_mds_defconfig
m68k                            q40_defconfig
powerpc                     asp8347_defconfig
powerpc                     stx_gp3_defconfig
arc                          axs103_defconfig
s390                       zfcpdump_defconfig
powerpc                     mpc5200_defconfig
arm                       omap2plus_defconfig
mips                      pistachio_defconfig
powerpc                 mpc85xx_cds_defconfig
nios2                               defconfig
openrisc                    or1ksim_defconfig
nios2                         3c120_defconfig
sh                          rsk7203_defconfig
powerpc                       holly_defconfig
mips                       lemote2f_defconfig
arm                        cerfcube_defconfig
arm                         mv78xx0_defconfig
arm                         cm_x300_defconfig
sh                           se7206_defconfig
powerpc                     pq2fads_defconfig
mips                             allyesconfig
arm                      integrator_defconfig
mips                        bcm63xx_defconfig
powerpc                    sam440ep_defconfig
arm                           omap1_defconfig
mips                       capcella_defconfig
mips                         cobalt_defconfig
arm                        trizeps4_defconfig
arm                            mmp2_defconfig
powerpc                     tqm8555_defconfig
arm                            hisi_defconfig
m68k                            mac_defconfig
m68k                           sun3_defconfig
riscv                            allmodconfig
sh                           se7750_defconfig
sh                          rsk7264_defconfig
um                           x86_64_defconfig
arm                        realview_defconfig
mips                     cu1830-neo_defconfig
arm                             rpc_defconfig
sh                            shmin_defconfig
arm                          pxa910_defconfig
i386                             allyesconfig
arm                            pleb_defconfig
arc                    vdk_hs38_smp_defconfig
arm                          ep93xx_defconfig
sh                           se7343_defconfig
arm                          badge4_defconfig
sparc                               defconfig
mips                       rbtx49xx_defconfig
mips                           ip32_defconfig
mips                      maltasmvp_defconfig
arm                           efm32_defconfig
nios2                         10m50_defconfig
c6x                              alldefconfig
sh                         ecovec24_defconfig
mips                      maltaaprp_defconfig
arm                     davinci_all_defconfig
arm                         axm55xx_defconfig
sparc                       sparc64_defconfig
powerpc                mpc7448_hpc2_defconfig
powerpc                 mpc832x_rdb_defconfig
powerpc                        fsp2_defconfig
arm                        keystone_defconfig
powerpc                      ppc40x_defconfig
sh                            migor_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
sparc                            allyesconfig
i386                               tinyconfig
i386                                defconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a001-20210202
i386                 randconfig-a005-20210202
i386                 randconfig-a003-20210202
i386                 randconfig-a006-20210202
i386                 randconfig-a002-20210202
i386                 randconfig-a004-20210202
x86_64               randconfig-a013-20210202
x86_64               randconfig-a014-20210202
x86_64               randconfig-a015-20210202
x86_64               randconfig-a016-20210202
x86_64               randconfig-a011-20210202
x86_64               randconfig-a012-20210202
i386                 randconfig-a013-20210202
i386                 randconfig-a016-20210202
i386                 randconfig-a014-20210202
i386                 randconfig-a012-20210202
i386                 randconfig-a015-20210202
i386                 randconfig-a011-20210202
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                          rv32_defconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a006-20210202
x86_64               randconfig-a001-20210202
x86_64               randconfig-a005-20210202
x86_64               randconfig-a002-20210202
x86_64               randconfig-a004-20210202
x86_64               randconfig-a003-20210202

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From lkp at intel.com  Tue Feb  2 17:52:19 2021
From: lkp at intel.com (kernel test robot)
Date: Wed, 03 Feb 2021 06:52:19 +0800
Subject: [linux-nvme:nvme-5.12] BUILD SUCCESS
 563c81586d0ab2841487a61fb34d6e9cd5efded7
Message-ID: <6019d7a3.aRcScetGqzreOAfa%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.12
branch HEAD: 563c81586d0ab2841487a61fb34d6e9cd5efded7  nvme-tcp: use cancel tagset helper for tear down

elapsed time: 727m

configs tested: 121
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
powerpc               mpc834x_itxgp_defconfig
arm                        mvebu_v5_defconfig
xtensa                          iss_defconfig
sh                            hp6xx_defconfig
arm                       spear13xx_defconfig
mips                           ip22_defconfig
powerpc                  mpc866_ads_defconfig
arm                       imx_v6_v7_defconfig
mips                          malta_defconfig
arm                       multi_v4t_defconfig
c6x                        evmc6474_defconfig
m68k                          multi_defconfig
s390                             alldefconfig
arm                          exynos_defconfig
sh                              ul2_defconfig
powerpc                 mpc834x_mds_defconfig
s390                       zfcpdump_defconfig
powerpc                     mpc5200_defconfig
arm                       omap2plus_defconfig
mips                      pistachio_defconfig
powerpc                 mpc85xx_cds_defconfig
arm                        cerfcube_defconfig
arm                         mv78xx0_defconfig
powerpc                    sam440ep_defconfig
arm                           omap1_defconfig
mips                       capcella_defconfig
mips                         cobalt_defconfig
arm                        trizeps4_defconfig
sh                           se7750_defconfig
sh                          rsk7264_defconfig
um                           x86_64_defconfig
arm                        realview_defconfig
mips                     cu1830-neo_defconfig
arm                             rpc_defconfig
sh                            shmin_defconfig
arm                          pxa910_defconfig
arm                           efm32_defconfig
nios2                         10m50_defconfig
c6x                              alldefconfig
sh                         ecovec24_defconfig
mips                      maltaaprp_defconfig
arm                     davinci_all_defconfig
arm                         axm55xx_defconfig
sparc                       sparc64_defconfig
powerpc                mpc7448_hpc2_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                               tinyconfig
i386                                defconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a001-20210202
i386                 randconfig-a005-20210202
i386                 randconfig-a003-20210202
i386                 randconfig-a006-20210202
i386                 randconfig-a002-20210202
i386                 randconfig-a004-20210202
x86_64               randconfig-a013-20210202
x86_64               randconfig-a014-20210202
x86_64               randconfig-a015-20210202
x86_64               randconfig-a016-20210202
x86_64               randconfig-a011-20210202
x86_64               randconfig-a012-20210202
i386                 randconfig-a013-20210202
i386                 randconfig-a016-20210202
i386                 randconfig-a014-20210202
i386                 randconfig-a012-20210202
i386                 randconfig-a015-20210202
i386                 randconfig-a011-20210202
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a006-20210202
x86_64               randconfig-a001-20210202
x86_64               randconfig-a005-20210202
x86_64               randconfig-a002-20210202
x86_64               randconfig-a004-20210202
x86_64               randconfig-a003-20210202

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From Chaitanya.Kulkarni at wdc.com  Tue Feb  2 23:31:06 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 3 Feb 2021 04:31:06 +0000
Subject: [PATCH 01/10] nvmet: zeroout id-ns buffer for invalid nsid
References: <20210201054138.34324-1-chaitanya.kulkarni@wdc.com>
 <20210201054138.34324-2-chaitanya.kulkarni@wdc.com>
 <20210201172438.GB12054@lst.de>
 <BYAPR04MB49653AC1A94F78112729EC6686B59@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210202093831.GG17771@lst.de>
Message-ID: <BYAPR04MB496538B136C5484C817FF50C86B49@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/2/21 01:38, Christoph Hellwig wrote:
> On Tue, Feb 02, 2021 at 12:12:33AM +0000, Chaitanya Kulkarni wrote:
>>>> +	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
>>>> +	if (!req->ns) {
>>>> +		status = NVME_SC_INVALID_NS;
>>>> +		/*
>>>> +		 * According to spec : If the specified namespace is
>>>> +		 * an unallocated NSID then the controller returns a zero filled
>>>> +		 * data structure. Also don't override the error status as invalid
>>>> +		 * namespace takes priority over the failed zeroout buffer case.
>>>> +		 */
>>>> +		nvmet_zero_sgl(req, 0, sizeof(*id));
>>>> +		goto out;
>>>> +	}
>>>> +
>>>>  	id = kzalloc(sizeof(*id), GFP_KERNEL);
>>>>  	if (!id) {
>>>>  		status = NVME_SC_INTERNAL;
>>>>  		goto out;
>>>>  	}
>>>>  
>>>> -	/* return an all zeroed buffer if we can't find an active namespace */
>>>> -	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
>>>> -	if (!req->ns) {
>>>> -		status = NVME_SC_INVALID_NS;
>>>> -		goto done;
>>> I think all we need to do is to remove the status assignment here.
>>> While you're patch avoids the memory allocation for this case, it isn't
>>> really the fast path so I'd rather avoid the extra code.
>>>
>> Sorry, I didn't understand this comment. If we remove the status assignment
>> then host will not get the right error. That is the bug we fixed it
>> initially
>> with bffcd507780e.
> Actually, looking at it, bffcd507780e is wrong.  Identify Namespace
> to a namespace that is not active should return a status of 0 and
> a zeroed structure.

I didn't find anything in the spec that says we need to return the
the status of 0 for unallocated [1] NSID.

I've checked with two different PCIe controllers for id-ns with invalid
nsid they both returned NVME_SC_INVALID_NS and so does QEMU :-

PCIe:-

# nvme list | tr -s ' ' ' '
Node SN Model Namespace Usage Format FW Rev
/dev/nvme0n1 191515805865 XXXXXXXXXXX-00SJG0 1 250.06 GB / 250.06 GB 512
B + 0 B 102000XX
/dev/nvme1n1 191912800089 XXXXXXXXXXX-00SJG0 1 2.00 TB / 2.00 TB 512 B +
0 B 102430XX

# nvme id-ns /dev/nvme0n1 -n 100
NVMe Status:INVALID_NS(400b) NSID:100

# nvme id-ns /dev/nvme1n1 -n 100
NVMe Status:INVALID_NS(400b) NSID:100

QEMU:-

# nvme list | tr -s ' ' ' '
Node SN Model Namespace Usage Format FW Rev
/dev/nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0
# nvme id-ns /dev/nvme0 -n 100
NVMe status: INVALID_NS: The namespace or the format of that namespace
is invalid(0x400b)

Without bffcd507780e targetwas returning success and zeroed buffer [2].
Now with bffcd507780etarget is returning NVME_SC_INVALID but the buffer
is not
zeroed (which was there in original patch [3] then we changed to not copy
buffer [4]). Above call to nvmet_zero_sgl() in this patch fixes that.

Please correct me if I missed something.
>> Regarding fast path, if system is under pressure even single memory
>> allocation
>> can be costly, especially when host tries to do read id-ns, is there any
>> reason why we should not consider this scenario ?
> No sensible host gets there.  I'd rather keep the code simple.
>

[1] Unallocated NSIDs do not refer to any namespaces that exist in the
NVM subsystem.
[2] http://lists.infradead.org/pipermail/linux-nvme/2021-January/021954.html
[3] http://lists.infradead.org/pipermail/linux-nvme/2021-January/021954.html
[4] http://lists.infradead.org/pipermail/linux-nvme/2021-January/021979.html


From sagi at grimberg.me  Wed Feb  3 03:05:27 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 00:05:27 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>


> I'm running "nvme discover" over a TCP connection. The nvme-tcp module freezes completely and bricks my computer.
> 
> Steps:
> $ sudo modprobe nvme-tcp
> $ sudo nvme discover -t tcp -a [IP address] -s 8009
> <System Bricked!>
> Only a reboot (Alt-SysRq-B) can recover the system.

Do you have a stack trace to share?

> Conditions to reproduce the problem:
> The Discovery Controller must support sending Discovery Log Change Notifications. That is, bit 31 of the Identity's OAES field returned by the discovery controller must be set to 1. If OAES[31]=0, then everything is OK.

What is the discovery log page returned by the nvme discovery 
controller? Does it include referrals? There was an issue fixed
in nvme-cli with respect to referrals (although nothing that is
related to any oaes changes).

> Systems tested:
> 1) Ubuntu 20.04, Linux 5.8, nvme 1.13.21
> 2) Fedora 33, Linux 5.10, nvme 1.11.1

Are these the default kernels that come with the distribution?
Does this happen with the latest upstream?

I know people are doing this for a long time now and no reports
came in on this phenomenon.. I'm assuming this is not Linux nvmet
target correct?


From sagi at grimberg.me  Wed Feb  3 03:51:45 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 00:51:45 -0800
Subject: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <20210201100509.27351-8-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-8-borisp@mellanox.com>
Message-ID: <9a8041aa-513f-09b9-63f4-3d12db19231c@grimberg.me>



On 2/1/21 2:04 AM, Boris Pismenny wrote:
> Introduce the NVMe-TCP DDP data-path offload.
> Using this interface, the NIC hardware will scatter TCP payload directly
> to the BIO pages according to the command_id in the PDU.
> To maintain the correctness of the network stack, the driver is expected
> to construct SKBs that point to the BIO pages.
> 
> The data-path interface contains two routines: tcp_ddp_setup/teardown.
> The setup provides the mapping from command_id to the request buffers,
> while the teardown removes this mapping.
> 
> For efficiency, we introduce an asynchronous nvme completion, which is
> split between NVMe-TCP and the NIC driver as follows:
> NVMe-TCP performs the specific completion, while NIC driver performs the
> generic mq_blk completion.
> 
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>   drivers/nvme/host/tcp.c | 141 +++++++++++++++++++++++++++++++++++++---
>   1 file changed, 131 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index ea67caf9d326..5cb46deb56e0 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -57,6 +57,11 @@ struct nvme_tcp_request {
>   	size_t			offset;
>   	size_t			data_sent;
>   	enum nvme_tcp_send_state state;
> +
> +	bool			offloaded;
> +	struct tcp_ddp_io	ddp;
> +	__le16			status;
> +	union nvme_result	result;
>   };
>   
>   enum nvme_tcp_queue_flags {
> @@ -232,10 +237,74 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>   #ifdef CONFIG_TCP_DDP
>   
>   static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
> +static void nvme_tcp_ddp_teardown_done(void *ddp_ctx);
>   static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
>   	.resync_request		= nvme_tcp_resync_request,
> +	.ddp_teardown_done	= nvme_tcp_ddp_teardown_done,
>   };
>   
> +static
> +int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> +			  u16 command_id,
> +			  struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	int ret;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
> +		return -EINVAL;
> +	}
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_teardown(netdev, queue->sock->sk,
> +						    &req->ddp, rq);
> +	sg_free_table_chained(&req->ddp.sg_table, SG_CHUNK_SIZE);
> +	req->offloaded = false;

Why is the offloaded = false needed here? you also clear it when
you setup.

> +	return ret;
> +}
> +
> +static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> +{
> +	struct request *rq = ddp_ctx;
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +
> +	if (!nvme_try_complete_req(rq, cpu_to_le16(req->status << 1), req->result))
> +		nvme_complete_rq(rq);

Why is the status shifted here? it was taken from the cqe as is..

> +}
> +
> +static
> +int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> +		       u16 command_id,
> +		       struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	int ret;
> +
> +	req->offloaded = false;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);

dev_info_ratelimited please.

> +		return -EINVAL;
> +	}
> +
> +	req->ddp.command_id = command_id;
> +	req->ddp.sg_table.sgl = req->ddp.first_sgl;
> +	ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
> +				     req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
> +	if (ret)
> +		return -ENOMEM;
> +	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
> +						 queue->sock->sk,
> +						 &req->ddp);
> +	if (!ret)
> +		req->offloaded = true;
> +	return ret;
> +}
> +
>   static
>   int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
>   {
> @@ -377,6 +446,25 @@ bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
>   
>   #else
>   
> +static
> +int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> +		       u16 command_id,
> +		       struct request *rq)
> +{
> +	return -EINVAL;
> +}
> +
> +static
> +int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> +			  u16 command_id,
> +			  struct request *rq)
> +{
> +	return -EINVAL;
> +}
> +
> +static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> +{}
> +
>   static
>   int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
>   {
> @@ -665,6 +753,7 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
>   static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		struct nvme_completion *cqe)
>   {
> +	struct nvme_tcp_request *req;
>   	struct request *rq;
>   
>   	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), cqe->command_id);
> @@ -676,8 +765,15 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
>   		return -EINVAL;
>   	}
>   
> -	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> -		nvme_complete_rq(rq);
> +	req = blk_mq_rq_to_pdu(rq);
> +	if (req->offloaded) {
> +		req->status = cqe->status;
> +		req->result = cqe->result;
> +		nvme_tcp_teardown_ddp(queue, cqe->command_id, rq);
> +	} else {
> +		if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> +			nvme_complete_rq(rq);
> +	}

Maybe move this to nvme_tcp_complete_request as it is called from two
code paths.

>   	queue->nr_cqe++;
>   
>   	return 0;
> @@ -871,9 +967,18 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   static inline void nvme_tcp_end_request(struct request *rq, u16 status)
>   {
>   	union nvme_result res = {};
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct nvme_tcp_queue *queue = req->queue;
> +	struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
>   
> -	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> -		nvme_complete_rq(rq);
> +	if (req->offloaded) {
> +		req->status = cpu_to_le16(status << 1);
> +		req->result = res;
> +		nvme_tcp_teardown_ddp(queue, pdu->command_id, rq);
> +	} else {
> +		if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> +			nvme_complete_rq(rq);
> +	}
>   }
>   
>   static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> @@ -920,12 +1025,22 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   		recv_len = min_t(size_t, recv_len,
>   				iov_iter_count(&req->iter));
>   
> -		if (queue->data_digest)
> -			ret = skb_copy_and_hash_datagram_iter(skb, *offset,
> -				&req->iter, recv_len, queue->rcv_hash);
> -		else
> -			ret = skb_copy_datagram_iter(skb, *offset,
> -					&req->iter, recv_len);
> +		if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
> +			if (queue->data_digest)
> +				ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset,
> +						&req->iter, recv_len, queue->rcv_hash);
> +			else
> +				ret = skb_ddp_copy_datagram_iter(skb, *offset,
> +						&req->iter, recv_len);
> +		} else {
> +			if (queue->data_digest)
> +				ret = skb_copy_and_hash_datagram_iter(skb, *offset,
> +						&req->iter, recv_len, queue->rcv_hash);
> +			else
> +				ret = skb_copy_datagram_iter(skb, *offset,
> +						&req->iter, recv_len);
> +		}
> +

Maybe move this hunk to nvme_tcp_consume_skb or something?

>   		if (ret) {
>   			dev_err(queue->ctrl->ctrl.device,
>   				"queue %d failed to copy request %#x data",
> @@ -1149,6 +1264,7 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>   	bool inline_data = nvme_tcp_has_inline_data(req);
>   	u8 hdgst = nvme_tcp_hdgst_len(queue);
>   	int len = sizeof(*pdu) + hdgst - req->offset;
> +	struct request *rq = blk_mq_rq_from_pdu(req);
>   	int flags = MSG_DONTWAIT;
>   	int ret;
>   
> @@ -1157,6 +1273,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>   	else
>   		flags |= MSG_EOR;
>   
> +	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) &&
> +	    blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
> +		nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);
> +
>   	if (queue->hdr_digest && !req->offset)
>   		nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu));
>   
> @@ -2464,6 +2584,7 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns,
>   	req->data_len = blk_rq_nr_phys_segments(rq) ?
>   				blk_rq_payload_bytes(rq) : 0;
>   	req->curr_bio = rq->bio;
> +	req->offloaded = false;

offloaded is being cleared lots of times, and I'm not clear what are
the lifetime rules here.


From sagi at grimberg.me  Wed Feb  3 03:56:35 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 00:56:35 -0800
Subject: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <CAJ3xEMhninJE5zw7=QFL4gBVkH=1tAmQHyq7tKMqcSJ_KkDsWQ@mail.gmail.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-8-borisp@mellanox.com> <20210201173744.GC12960@lst.de>
 <CAJ3xEMhninJE5zw7=QFL4gBVkH=1tAmQHyq7tKMqcSJ_KkDsWQ@mail.gmail.com>
Message-ID: <80074375-2d37-d9b9-afbe-1f3d1db4a41f@grimberg.me>


>> Given how much ddp code there is can you split it into a separate file?
> 
> mmm, do we need to check the preferences or get to a consensus among
> the maintainers for that one?

Not sure if moving it would be better here. Given that the ddp code is
working directly on nvme-tcp structs we'll need a new shared header
file..

Its possible to do, but I'm not sure the end result will be better..



From sagi at grimberg.me  Wed Feb  3 04:06:53 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 01:06:53 -0800
Subject: [PATCH v3 net-next 08/21] nvme-tcp : Recalculate crc in the end
 of the capsule
In-Reply-To: <20210201100509.27351-9-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-9-borisp@mellanox.com>
Message-ID: <a104a5d1-b4cb-4275-6ced-b80f911b6f47@grimberg.me>



On 2/1/21 2:04 AM, Boris Pismenny wrote:
> From: Ben Ben-ishay <benishay at nvidia.com>
> 
> crc offload of the nvme capsule. Check if all the skb bits
> are on, and if not recalculate the crc in SW and check it.
> 
> This patch reworks the receive-side crc calculation to always
> run at the end, so as to keep a single flow for both offload
> and non-offload. This change simplifies the code, but it may degrade
> performance for non-offload crc calculation.
> 
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>   drivers/nvme/host/tcp.c | 118 ++++++++++++++++++++++++++++++++--------
>   1 file changed, 95 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 5cb46deb56e0..eb47cf6982d7 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -69,6 +69,7 @@ enum nvme_tcp_queue_flags {
>   	NVME_TCP_Q_LIVE		= 1,
>   	NVME_TCP_Q_POLLING	= 2,
>   	NVME_TCP_Q_OFF_DDP	= 3,
> +	NVME_TCP_Q_OFF_DDGST_RX = 4,
>   };
>   
>   enum nvme_tcp_recv_state {
> @@ -96,6 +97,7 @@ struct nvme_tcp_queue {
>   	size_t			data_remaining;
>   	size_t			ddgst_remaining;
>   	unsigned int		nr_cqe;
> +	bool			ddgst_valid;
>   
>   	/* send state */
>   	struct nvme_tcp_request *request;
> @@ -234,7 +236,56 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>   	return nvme_tcp_pdu_data_left(req) <= len;
>   }
>   
> -#ifdef CONFIG_TCP_DDP
> +static inline bool nvme_tcp_ddp_ddgst_ok(struct nvme_tcp_queue *queue)
> +{
> +	return queue->ddgst_valid;
> +}
> +
> +static inline void nvme_tcp_ddp_ddgst_update(struct nvme_tcp_queue *queue,
> +					     struct sk_buff *skb)
> +{
> +	if (queue->ddgst_valid)
> +#ifdef CONFIG_TCP_DDP_CRC
> +		queue->ddgst_valid = skb->ddp_crc;
> +#else
> +		queue->ddgst_valid = false;
> +#endif
> +}
> +
> +static int nvme_tcp_req_map_sg(struct nvme_tcp_request *req, struct request *rq)
> +{
> +	int ret;
> +
> +	req->ddp.sg_table.sgl = req->ddp.first_sgl;
> +	ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
> +				     req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
> +	if (ret)
> +		return -ENOMEM;
> +	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
> +	return 0;
> +}
> +
> +static void nvme_tcp_ddp_ddgst_recalc(struct ahash_request *hash,
> +				      struct request *rq)
> +{
> +	struct nvme_tcp_request *req;
> +
> +	if (!rq)
> +		return;
> +
> +	req = blk_mq_rq_to_pdu(rq);
> +
> +	if (!req->offloaded && nvme_tcp_req_map_sg(req, rq))
> +		return;
> +
> +	crypto_ahash_init(hash);
> +	req->ddp.sg_table.sgl = req->ddp.first_sgl;
> +	ahash_request_set_crypt(hash, req->ddp.sg_table.sgl, NULL,
> +				le32_to_cpu(req->data_len));
> +	crypto_ahash_update(hash);
> +}
> +
> +#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
>   
>   static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
>   static void nvme_tcp_ddp_teardown_done(void *ddp_ctx);
> @@ -290,12 +341,9 @@ int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
>   	}
>   
>   	req->ddp.command_id = command_id;
> -	req->ddp.sg_table.sgl = req->ddp.first_sgl;
> -	ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
> -				     req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
> +	ret = nvme_tcp_req_map_sg(req, rq);

Why didn't you introduce nvme_tcp_req_map_sg in the first place?

>   	if (ret)
>   		return -ENOMEM;
> -	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
>   
>   	ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
>   						 queue->sock->sk,
> @@ -317,7 +365,7 @@ int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
>   		return -ENODEV;
>   	}
>   
> -	if (!(netdev->features & NETIF_F_HW_TCP_DDP)) {
> +	if (!(netdev->features & (NETIF_F_HW_TCP_DDP | NETIF_F_HW_TCP_DDP_CRC_RX))) {
>   		dev_put(netdev);
>   		return -EOPNOTSUPP;
>   	}
> @@ -345,6 +393,9 @@ int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
>   	if (netdev->features & NETIF_F_HW_TCP_DDP)
>   		set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
>   
> +	if (netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
> +		set_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags);
> +
>   	return ret;
>   }
>   
> @@ -376,7 +427,7 @@ int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
>   		return -ENODEV;
>   	}
>   
> -	if (netdev->features & NETIF_F_HW_TCP_DDP &&
> +	if ((netdev->features & (NETIF_F_HW_TCP_DDP | NETIF_F_HW_TCP_DDP_CRC_RX)) &&
>   	    netdev->tcp_ddp_ops &&
>   	    netdev->tcp_ddp_ops->tcp_ddp_limits)
>   		ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
> @@ -739,6 +790,7 @@ static void nvme_tcp_init_recv_ctx(struct nvme_tcp_queue *queue)
>   	queue->pdu_offset = 0;
>   	queue->data_remaining = -1;
>   	queue->ddgst_remaining = 0;
> +	queue->ddgst_valid = true;
>   }
>   
>   static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
> @@ -919,7 +971,8 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   
>   	u64 pdu_seq = TCP_SKB_CB(skb)->seq + *offset - queue->pdu_offset;
>   
> -	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
> +	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) ||
> +	    test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
>   		nvme_tcp_resync_response(queue, pdu_seq);
>   
>   	ret = skb_copy_bits(skb, *offset,
> @@ -988,6 +1041,8 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   	struct nvme_tcp_request *req;
>   	struct request *rq;
>   
> +	if (queue->data_digest && test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
> +		nvme_tcp_ddp_ddgst_update(queue, skb);
>   	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
>   	if (!rq) {
>   		dev_err(queue->ctrl->ctrl.device,
> @@ -1025,15 +1080,17 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   		recv_len = min_t(size_t, recv_len,
>   				iov_iter_count(&req->iter));
>   
> -		if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
> -			if (queue->data_digest)
> +		if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
> +			if (queue->data_digest &&
> +			    !test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
>   				ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset,
>   						&req->iter, recv_len, queue->rcv_hash);
>   			else
>   				ret = skb_ddp_copy_datagram_iter(skb, *offset,
>   						&req->iter, recv_len);
>   		} else {
> -			if (queue->data_digest)
> +			if (queue->data_digest &&
> +			    !test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
>   				ret = skb_copy_and_hash_datagram_iter(skb, *offset,
>   						&req->iter, recv_len, queue->rcv_hash);
>   			else
> @@ -1055,7 +1112,6 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   
>   	if (!queue->data_remaining) {
>   		if (queue->data_digest) {
> -			nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
>   			queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH;
>   		} else {
>   			if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
> @@ -1076,8 +1132,12 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
>   	char *ddgst = (char *)&queue->recv_ddgst;
>   	size_t recv_len = min_t(size_t, *len, queue->ddgst_remaining);
>   	off_t off = NVME_TCP_DIGEST_LENGTH - queue->ddgst_remaining;
> +	bool offload_fail, offload_en;
> +	struct request *rq = NULL;
>   	int ret;
>   
> +	if (test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
> +		nvme_tcp_ddp_ddgst_update(queue, skb);
>   	ret = skb_copy_bits(skb, *offset, &ddgst[off], recv_len);
>   	if (unlikely(ret))
>   		return ret;
> @@ -1088,17 +1148,29 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
>   	if (queue->ddgst_remaining)
>   		return 0;
>   
> -	if (queue->recv_ddgst != queue->exp_ddgst) {
> -		dev_err(queue->ctrl->ctrl.device,
> -			"data digest error: recv %#x expected %#x\n",
> -			le32_to_cpu(queue->recv_ddgst),
> -			le32_to_cpu(queue->exp_ddgst));
> -		return -EIO;
> +	offload_fail = !nvme_tcp_ddp_ddgst_ok(queue);
> +	offload_en = test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags);
> +	if (!offload_en || offload_fail) {
> +		if (offload_en && offload_fail) { // software-fallback
> +			rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
> +					      pdu->command_id);
> +			nvme_tcp_ddp_ddgst_recalc(queue->rcv_hash, rq);
> +		}
> +
> +		nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
> +		if (queue->recv_ddgst != queue->exp_ddgst) {
> +			dev_err(queue->ctrl->ctrl.device,
> +				"data digest error: recv %#x expected %#x\n",
> +				le32_to_cpu(queue->recv_ddgst),
> +				le32_to_cpu(queue->exp_ddgst));
> +			return -EIO;
> +		}

I still dislike this hunk. Can you split the recalc login to its
own ddp function at least? This is just a confusing piece of code.

>   	}
>   
>   	if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
> -		struct request *rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
> -						pdu->command_id);
> +		if (!rq)
> +			rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
> +					      pdu->command_id);

Why is this change needed? Maybe just move this assignment up?

>   
>   		nvme_tcp_end_request(rq, NVME_SC_SUCCESS);
>   		queue->nr_cqe++;
> @@ -1841,8 +1913,10 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
>   	nvme_tcp_restore_sock_calls(queue);
>   	cancel_work_sync(&queue->io_work);
>   
> -	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
> +	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) ||
> +	    test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
>   		nvme_tcp_unoffload_socket(queue);
> +

extra newline

>   }
>   
>   static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
> @@ -1970,8 +2044,6 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
>   {
>   	int ret;
>   
> -	to_tcp_ctrl(ctrl)->offloading_netdev = NULL;
> -

Unclear what is the intent here.

>   	ret = nvme_tcp_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
>   	if (ret)
>   		return ret;
> 


From sagi at grimberg.me  Wed Feb  3 04:09:52 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 01:09:52 -0800
Subject: [PATCH v3 net-next 09/21] nvme-tcp: Deal with netdevice DOWN
 events
In-Reply-To: <20210201100509.27351-10-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-10-borisp@mellanox.com>
Message-ID: <c864eb96-18ea-21dd-3ef3-15ca908c5959@grimberg.me>


> @@ -2930,6 +2931,27 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
>   	return ERR_PTR(ret);
>   }
>   
> +static int nvme_tcp_netdev_event(struct notifier_block *this,
> +				 unsigned long event, void *ptr)
> +{
> +	struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
> +	struct nvme_tcp_ctrl *ctrl;
> +
> +	switch (event) {
> +	case NETDEV_GOING_DOWN:
> +		mutex_lock(&nvme_tcp_ctrl_mutex);
> +		list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list) {
> +			if (ndev != ctrl->offloading_netdev)
> +				continue;
> +			nvme_tcp_error_recovery(&ctrl->ctrl);
> +		}
> +		mutex_unlock(&nvme_tcp_ctrl_mutex);
> +		flush_workqueue(nvme_reset_wq);
> +		/* we assume that the going down part of error recovery is over */

Maybe phrase it as:
/*
  * The associated controllers teardown has completed, ddp contexts
  * were also torn down so we should be safe to continue...
  */


From sagi at grimberg.me  Wed Feb  3 04:17:40 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 01:17:40 -0800
Subject: [PATCH v3 net-next 06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <20210201100509.27351-7-borisp@mellanox.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-7-borisp@mellanox.com>
Message-ID: <701052c3-1d55-ecb1-b14e-65b2e3a7c4e6@grimberg.me>



On 2/1/21 2:04 AM, Boris Pismenny wrote:
> This commit introduces direct data placement offload to NVME
> TCP. There is a context per queue, which is established after the
> handshake using the tcp_ddp_sk_add/del NDOs.
> 
> Additionally, a resynchronization routine is used to assist
> hardware recovery from TCP OOO, and continue the offload.
> Resynchronization operates as follows:
> 
> 1. TCP OOO causes the NIC HW to stop the offload
> 
> 2. NIC HW identifies a PDU header at some TCP sequence number,
> and asks NVMe-TCP to confirm it.
> This request is delivered from the NIC driver to NVMe-TCP by first
> finding the socket for the packet that triggered the request, and
> then finding the nvme_tcp_queue that is used by this routine.
> Finally, the request is recorded in the nvme_tcp_queue.
> 
> 3. When NVMe-TCP observes the requested TCP sequence, it will compare
> it with the PDU header TCP sequence, and report the result to the
> NIC driver (tcp_ddp_resync), which will update the HW,
> and resume offload when all is successful.
> 
> Furthermore, we let the offloading driver advertise what is the max hw
> sectors/segments via tcp_ddp_limits.
> 
> A follow-up patch introduces the data-path changes required for this
> offload.
> 
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>   drivers/nvme/host/tcp.c | 200 +++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 198 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 881d28eb15e9..ea67caf9d326 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -14,6 +14,7 @@
>   #include <linux/blk-mq.h>
>   #include <crypto/hash.h>
>   #include <net/busy_poll.h>
> +#include <net/tcp_ddp.h>
>   
>   #include "nvme.h"
>   #include "fabrics.h"
> @@ -62,6 +63,7 @@ enum nvme_tcp_queue_flags {
>   	NVME_TCP_Q_ALLOCATED	= 0,
>   	NVME_TCP_Q_LIVE		= 1,
>   	NVME_TCP_Q_POLLING	= 2,
> +	NVME_TCP_Q_OFF_DDP	= 3,
>   };
>   
>   enum nvme_tcp_recv_state {
> @@ -111,6 +113,8 @@ struct nvme_tcp_queue {
>   	void (*state_change)(struct sock *);
>   	void (*data_ready)(struct sock *);
>   	void (*write_space)(struct sock *);
> +
> +	atomic64_t  resync_req;
>   };
>   
>   struct nvme_tcp_ctrl {
> @@ -129,6 +133,8 @@ struct nvme_tcp_ctrl {
>   	struct delayed_work	connect_work;
>   	struct nvme_tcp_request async_req;
>   	u32			io_queues[HCTX_MAX_TYPES];
> +
> +	struct net_device       *offloading_netdev;
>   };
>   
>   static LIST_HEAD(nvme_tcp_ctrl_list);
> @@ -223,6 +229,183 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>   	return nvme_tcp_pdu_data_left(req) <= len;
>   }
>   
> +#ifdef CONFIG_TCP_DDP
> +
> +static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
> +static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
> +	.resync_request		= nvme_tcp_resync_request,
> +};
> +
> +static
> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
> +{
> +	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);

Sometimes you use get_netdev_for_sock and sometimes
queue->ctrl->offloading_netdev, is this because of admin vs. io queue?

> +	struct nvme_tcp_ddp_config config = {};
> +	int ret;
> +
> +	if (!netdev) {
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
> +		return -ENODEV;
> +	}
> +
> +	if (!(netdev->features & NETIF_F_HW_TCP_DDP)) {
> +		dev_put(netdev);
> +		return -EOPNOTSUPP;
> +	}
> +
> +	config.cfg.type		= TCP_DDP_NVME;
> +	config.pfv		= NVME_TCP_PFV_1_0;
> +	config.cpda		= 0;
> +	config.dgst		= queue->hdr_digest ?
> +		NVME_TCP_HDR_DIGEST_ENABLE : 0;
> +	config.dgst		|= queue->data_digest ?
> +		NVME_TCP_DATA_DIGEST_ENABLE : 0;
> +	config.queue_size	= queue->queue_size;
> +	config.queue_id		= nvme_tcp_queue_id(queue);
> +	config.io_cpu		= queue->io_cpu;
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_sk_add(netdev,
> +						  queue->sock->sk,
> +						  &config.cfg);
> +	if (ret) {
> +		dev_put(netdev);
> +		return ret;
> +	}
> +
> +	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = &nvme_tcp_ddp_ulp_ops;
> +	if (netdev->features & NETIF_F_HW_TCP_DDP)
> +		set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
> +
> +	return ret;
> +}
> +
> +static
> +void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
> +{
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +
> +	if (!netdev) {
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
> +		return;
> +	}
> +
> +	netdev->tcp_ddp_ops->tcp_ddp_sk_del(netdev, queue->sock->sk);
> +
> +	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = NULL;
> +	dev_put(netdev); /* put the queue_init get_netdev_for_sock() */

Isn't the comment redundant.

> +}
> +
> +static
> +int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
> +{
> +	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
> +	struct tcp_ddp_limits limits;
> +	int ret = 0;
> +
> +	if (!netdev) {
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
> +		return -ENODEV;
> +	}
> +
> +	if (netdev->features & NETIF_F_HW_TCP_DDP &&
> +	    netdev->tcp_ddp_ops &&
> +	    netdev->tcp_ddp_ops->tcp_ddp_limits)
> +		ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
> +	else
> +		ret = -EOPNOTSUPP;
> +
> +	if (!ret) {
> +		queue->ctrl->offloading_netdev = netdev;
> +		dev_dbg_ratelimited(queue->ctrl->ctrl.device,
> +				    "netdev %s offload limits: max_ddp_sgl_len %d\n",
> +				    netdev->name, limits.max_ddp_sgl_len);
> +		queue->ctrl->ctrl.max_segments = limits.max_ddp_sgl_len;
> +		queue->ctrl->ctrl.max_hw_sectors =
> +			limits.max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
> +	} else {
> +		queue->ctrl->offloading_netdev = NULL;

In other error paths in the function this assignment is not needed?

> +	}
> +
> +	/* release the device as no offload context is established yet. */
> +	dev_put(netdev);
> +
> +	return ret;
> +}
> +
> +static
> +void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
> +			      unsigned int pdu_seq)
> +{
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	u64 resync_val;
> +	u32 resync_seq;
> +
> +	resync_val = atomic64_read(&queue->resync_req);
> +	/* Lower 32 bit flags. Check validity of the request */
> +	if ((resync_val & TCP_DDP_RESYNC_REQ) == 0)
> +		return;
> +
> +	/* Obtain and check requested sequence number: is this PDU header before the request? */
> +	resync_seq = resync_val >> 32;
> +	if (before(pdu_seq, resync_seq))
> +		return;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);
> +		return;
> +	}
> +
> +	/**
> +	 * The atomic operation gurarantees that we don't miss any NIC driver
> +	 * resync requests submitted after the above checks.
> +	 */
> +	if (atomic64_cmpxchg(&queue->resync_req, resync_val,
> +			     resync_val & ~TCP_DDP_RESYNC_REQ))
> +		netdev->tcp_ddp_ops->tcp_ddp_resync(netdev, queue->sock->sk, pdu_seq);
> +}
> +
> +static
> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
> +{
> +	struct nvme_tcp_queue *queue = sk->sk_user_data;
> +
> +	atomic64_set(&queue->resync_req,
> +		     (((uint64_t)seq << 32) | flags));
> +
> +	return true;
> +}
> +
> +#else
> +
> +static
> +int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
> +{
> +	return -EINVAL;
> +}
> +
> +static
> +void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
> +{}
> +
> +static
> +int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
> +{
> +	return -EINVAL;
> +}
> +
> +static
> +void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
> +			      unsigned int pdu_seq)
> +{}
> +
> +static
> +bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
> +{
> +	return false;
> +}
> +
> +#endif
> +
>   static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>   		unsigned int dir)
>   {
> @@ -638,6 +821,11 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>   	size_t rcv_len = min_t(size_t, *len, queue->pdu_remaining);
>   	int ret;
>   
> +	u64 pdu_seq = TCP_SKB_CB(skb)->seq + *offset - queue->pdu_offset;
> +
> +	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
> +		nvme_tcp_resync_response(queue, pdu_seq);

Maybe just pass (queue, skb, *offset) and retrieve the pdu_seq in
nvme_tcp_resync_response?

> +
>   	ret = skb_copy_bits(skb, *offset,
>   		&pdu[queue->pdu_offset], rcv_len);
>   	if (unlikely(ret))
> @@ -1532,6 +1720,9 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
>   	kernel_sock_shutdown(queue->sock, SHUT_RDWR);
>   	nvme_tcp_restore_sock_calls(queue);
>   	cancel_work_sync(&queue->io_work);
> +
> +	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
> +		nvme_tcp_unoffload_socket(queue);
>   }
>   
>   static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
> @@ -1550,10 +1741,13 @@ static int nvme_tcp_start_queue(struct nvme_ctrl *nctrl, int idx)
>   	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
>   	int ret;
>   
> -	if (idx)
> +	if (idx) {
>   		ret = nvmf_connect_io_queue(nctrl, idx, false);
> -	else
> +		nvme_tcp_offload_socket(&ctrl->queues[idx]);
> +	} else {
>   		ret = nvmf_connect_admin_queue(nctrl);
> +		nvme_tcp_offload_limits(&ctrl->queues[idx]);
> +	}
>   
>   	if (!ret) {
>   		set_bit(NVME_TCP_Q_LIVE, &ctrl->queues[idx].flags);
> @@ -1656,6 +1850,8 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
>   {
>   	int ret;
>   
> +	to_tcp_ctrl(ctrl)->offloading_netdev = NULL;
> +
>   	ret = nvme_tcp_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
>   	if (ret)
>   		return ret;
> 


From sagi at grimberg.me  Wed Feb  3 04:20:25 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed,  3 Feb 2021 01:20:25 -0800
Subject: [PATCH] nvmet-tcp: Fix out-of-bounds access when receiving multiple
 h2cdata PDUs
Message-ID: <20210203092025.437839-1-sagi@grimberg.me>

When the host sends multiple h2cdata PDUs, we keep track on
the receive progress and calculate the scatterlist index and
offsets.

The issue is that sg_offset should only be kept for the first
iov entry we map in the iovec as this is the difference between
our cursor and the sg entry offset itself.

In addition, the sg index was calculated wrong because we should
not round up when dividing the command byte offset with PAG_SIZE.

Fixes: 872d26a391da ("nvmet-tcp: add NVMe over TCP target driver")
Reported-by: Narayan Ayalasomayajula <Narayan.Ayalasomayajula at wdc.com>
Tested-by: Narayan Ayalasomayajula <Narayan.Ayalasomayajula at wdc.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/target/tcp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index dc1f0f647189..aacf06f0b431 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -305,7 +305,7 @@ static void nvmet_tcp_map_pdu_iovec(struct nvmet_tcp_cmd *cmd)
 	length = cmd->pdu_len;
 	cmd->nr_mapped = DIV_ROUND_UP(length, PAGE_SIZE);
 	offset = cmd->rbytes_done;
-	cmd->sg_idx = DIV_ROUND_UP(offset, PAGE_SIZE);
+	cmd->sg_idx = offset / PAGE_SIZE;
 	sg_offset = offset % PAGE_SIZE;
 	sg = &cmd->req.sg[cmd->sg_idx];
 
@@ -318,6 +318,7 @@ static void nvmet_tcp_map_pdu_iovec(struct nvmet_tcp_cmd *cmd)
 		length -= iov_len;
 		sg = sg_next(sg);
 		iov++;
+		sg_offset = 0;
 	}
 
 	iov_iter_kvec(&cmd->recv_msg.msg_iter, READ, cmd->iov,
-- 
2.27.0



From sironi at amazon.de  Wed Feb  3 04:43:38 2021
From: sironi at amazon.de (Filippo Sironi)
Date: Wed, 3 Feb 2021 10:43:38 +0100
Subject: [PATCH] nvme: Add 48-bit DMA address quirk
Message-ID: <20210203094338.19473-1-sironi@amazon.de>

Certain NVMe controllers don't support 64-bit DMA addresses.  Instead,
they are limited to 48-bit DMA addresses.  Let's add a quirk to use them
properly.

Signed-off-by: Filippo Sironi <sironi at amazon.de>
---
 drivers/nvme/host/nvme.h |  5 +++++
 drivers/nvme/host/pci.c  | 12 +++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 88a6b97247f5..dae747b4ac35 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -144,6 +144,11 @@ enum nvme_quirks {
 	 * NVMe 1.3 compliance.
 	 */
 	NVME_QUIRK_NO_NS_DESC_LIST		= (1 << 15),
+
+	/*
+	 * The controller supports up to 48-bit DMA address.
+	 */
+	NVME_QUIRK_DMA_ADDRESS_BITS_48		= (1 << 16),
 };
 
 /*
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b2042..5716ae16c7a7 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2362,13 +2362,16 @@ static int nvme_pci_enable(struct nvme_dev *dev)
 {
 	int result = -ENOMEM;
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
+	int dma_address_bits = 64;
 
 	if (pci_enable_device_mem(pdev))
 		return result;
 
 	pci_set_master(pdev);
 
-	if (dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(64)))
+	if (dev->ctrl.quirks & NVME_QUIRK_DMA_ADDRESS_BITS_48)
+		dma_address_bits = 48;
+	if (dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(dma_address_bits)))
 		goto disable;
 
 	if (readl(dev->bar + NVME_REG_CSTS) == -1) {
@@ -3259,6 +3262,13 @@ static const struct pci_device_id nvme_id_table[] = {
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(0x1d97, 0x2263),   /* SPCC */
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+	{ .vendor = PCI_VENDOR_ID_AMAZON,
+	  .device = PCI_ANY_ID,
+	  .subvendor = PCI_ANY_ID,
+	  .subdevice = PCI_ANY_ID,
+	  .class = PCI_CLASS_STORAGE_EXPRESS,
+	  .class_mask = 0xffffff,
+	  .driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48 },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2001),
 		.driver_data = NVME_QUIRK_SINGLE_VECTOR },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2003) },
-- 
2.17.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879





From hch at lst.de  Wed Feb  3 04:51:48 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 10:51:48 +0100
Subject: [PATCH] nvme: Add 48-bit DMA address quirk
In-Reply-To: <20210203094338.19473-1-sironi@amazon.de>
References: <20210203094338.19473-1-sironi@amazon.de>
Message-ID: <20210203095148.GA8897@lst.de>

On Wed, Feb 03, 2021 at 10:43:38AM +0100, Filippo Sironi wrote:
> Certain NVMe controllers don't support 64-bit DMA addresses.  Instead,
> they are limited to 48-bit DMA addresses.  Let's add a quirk to use them
> properly.

WTF?  This is such a grave NVMe spec compiance bug that I do not think
we should support this buggy mess in Linux.


From sagi at grimberg.me  Wed Feb  3 05:01:08 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 02:01:08 -0800
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <20210202075828.GA12160@lst.de>
References: <20210202005951.26614-1-mwilck@suse.com>
 <20210202075828.GA12160@lst.de>
Message-ID: <3fbf065a-0c1b-dbf6-7f07-06838a6efcd4@grimberg.me>


>> From: Martin Wilck <mwilck at suse.com>
>>
>> While a controller is still connecting, ctrl->subsys is NULL and
>> thus reading the controller's "subsysnqn" sysfs attribute returns
>> "(efault)". This happens all the time when user space processes
>> "add" events for NVMe controller devices.
>>
>> Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
>> if ctrl->subsys isn't initialized yet.
> 
> This is just papering over symptoms.  We should not be sending
> events or display sysfs files until the controlle is live.
> 
> In other wwords - I think we should remove the cdev_device_add
> from nvme_init_ctrl to nvme_start_ctrl (+ any error handling changes
> resulting from that).

Maybe move it to nvme_init_identify? nvme_start_ctrl is after I/O queues
were created.


From sagi at grimberg.me  Wed Feb  3 05:02:00 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 02:02:00 -0800
Subject: [PATCH] nvme: convert sysfs sprintf/snprintf family to sysfs_emit
In-Reply-To: <1612249577-66130-1-git-send-email-jiapeng.chong@linux.alibaba.com>
References: <1612249577-66130-1-git-send-email-jiapeng.chong@linux.alibaba.com>
Message-ID: <3bc03849-b8ef-3596-5f9d-ff172d74f4e4@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From hch at lst.de  Wed Feb  3 05:02:34 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 11:02:34 +0100
Subject: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <80074375-2d37-d9b9-afbe-1f3d1db4a41f@grimberg.me>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-8-borisp@mellanox.com> <20210201173744.GC12960@lst.de>
 <CAJ3xEMhninJE5zw7=QFL4gBVkH=1tAmQHyq7tKMqcSJ_KkDsWQ@mail.gmail.com>
 <80074375-2d37-d9b9-afbe-1f3d1db4a41f@grimberg.me>
Message-ID: <20210203100234.GA9050@lst.de>

On Wed, Feb 03, 2021 at 12:56:35AM -0800, Sagi Grimberg wrote:
>
>>> Given how much ddp code there is can you split it into a separate file?
>>
>> mmm, do we need to check the preferences or get to a consensus among
>> the maintainers for that one?
>
> Not sure if moving it would be better here. Given that the ddp code is
> working directly on nvme-tcp structs we'll need a new shared header
> file..
>
> Its possible to do, but I'm not sure the end result will be better..

In the end its your code base.  But I hate having all this offload
cruft all over the place.  Just saying no to offloads might be an even
better position, though.


From sagi at grimberg.me  Wed Feb  3 05:21:01 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 02:21:01 -0800
Subject: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <20210203100234.GA9050@lst.de>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-8-borisp@mellanox.com> <20210201173744.GC12960@lst.de>
 <CAJ3xEMhninJE5zw7=QFL4gBVkH=1tAmQHyq7tKMqcSJ_KkDsWQ@mail.gmail.com>
 <80074375-2d37-d9b9-afbe-1f3d1db4a41f@grimberg.me>
 <20210203100234.GA9050@lst.de>
Message-ID: <12247db9-5847-b791-c9e2-ce9993abc4d4@grimberg.me>


>>>> Given how much ddp code there is can you split it into a separate file?
>>>
>>> mmm, do we need to check the preferences or get to a consensus among
>>> the maintainers for that one?
>>
>> Not sure if moving it would be better here. Given that the ddp code is
>> working directly on nvme-tcp structs we'll need a new shared header
>> file..
>>
>> Its possible to do, but I'm not sure the end result will be better..
> 
> In the end its your code base.  But I hate having all this offload
> cruft all over the place.

I know, I think that the folks did a solid job consolidating it
given the complexity. But looking at the code again, it is sprinkled
more than I'd like it to be. I think it can be better with a little
more work.

If we can get to a point where we have all the specific logic
moved to dedicated routines and just a few interceptions on
the main flows we should be ok.

> Just saying no to offloads might be an even better position, though.

:)

I've heard complaints about nvme-tcp taking more cpu cycles than
nvme-rdma (well duh..) so I'm not opposed to having mainstream devices
offering offload capabilities to help out with that, not at all.


From sironi at amazon.de  Wed Feb  3 06:12:31 2021
From: sironi at amazon.de (Filippo Sironi)
Date: Wed, 3 Feb 2021 12:12:31 +0100
Subject: [PATCH] nvme: Add 48-bit DMA address quirk
In-Reply-To: <20210203095148.GA8897@lst.de>
References: <20210203094338.19473-1-sironi@amazon.de>
 <20210203095148.GA8897@lst.de>
Message-ID: <0c38f5eb-41ef-7934-940b-77b6e73c5239@amazon.de>


On 2/3/21 10:51 AM, Christoph Hellwig wrote:
> 
> On Wed, Feb 03, 2021 at 10:43:38AM +0100, Filippo Sironi wrote:
>> Certain NVMe controllers don't support 64-bit DMA addresses.  Instead,
>> they are limited to 48-bit DMA addresses.  Let's add a quirk to use them
>> properly.
> 
> WTF?  This is such a grave NVMe spec compiance bug that I do not think
> we should support this buggy mess in Linux.
> 

I don't disagree on the first part of your sentence, this is a big 
oversight.

On the other hand, those controllers are out there and are in use by a 
lot of customers.  We can keep relying on luck, hoping that customers 
don't run into troubles or we can merge a few lines of code :)



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879



From hch at lst.de  Wed Feb  3 06:15:48 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 12:15:48 +0100
Subject: [PATCH] nvme: Add 48-bit DMA address quirk
In-Reply-To: <0c38f5eb-41ef-7934-940b-77b6e73c5239@amazon.de>
References: <20210203094338.19473-1-sironi@amazon.de>
 <20210203095148.GA8897@lst.de>
 <0c38f5eb-41ef-7934-940b-77b6e73c5239@amazon.de>
Message-ID: <20210203111548.GA11075@lst.de>

On Wed, Feb 03, 2021 at 12:12:31PM +0100, Filippo Sironi wrote:
> I don't disagree on the first part of your sentence, this is a big 
> oversight.

But it is not what your commit log suggests.

> On the other hand, those controllers are out there and are in use by a lot 
> of customers.  We can keep relying on luck, hoping that customers don't run 
> into troubles or we can merge a few lines of code :)

Your patch does not just quirk a few controllers out there, but all
current and future controllers with an Amazon vendor ID.  We could
probably talk about quirking an existing vendor ID or two as long as
this doesn't happen for future hardware.


From sironi at amazon.de  Wed Feb  3 06:22:31 2021
From: sironi at amazon.de (Filippo Sironi)
Date: Wed, 3 Feb 2021 12:22:31 +0100
Subject: [PATCH] nvme: Add 48-bit DMA address quirk
In-Reply-To: <20210203111548.GA11075@lst.de>
References: <20210203094338.19473-1-sironi@amazon.de>
 <20210203095148.GA8897@lst.de>
 <0c38f5eb-41ef-7934-940b-77b6e73c5239@amazon.de>
 <20210203111548.GA11075@lst.de>
Message-ID: <396b45d5-3dd5-8159-34c2-5668510d1d68@amazon.de>


On 2/3/21 12:15 PM, Christoph Hellwig wrote:
> 
> On Wed, Feb 03, 2021 at 12:12:31PM +0100, Filippo Sironi wrote:
>> I don't disagree on the first part of your sentence, this is a big
>> oversight.
> 
> But it is not what your commit log suggests.

I can definitely rephrase the commit.

>> On the other hand, those controllers are out there and are in use by a lot
>> of customers.  We can keep relying on luck, hoping that customers don't run
>> into troubles or we can merge a few lines of code :)
> 
> Your patch does not just quirk a few controllers out there, but all
> current and future controllers with an Amazon vendor ID.  We could
> probably talk about quirking an existing vendor ID or two as long as
> this doesn't happen for future hardware.

I know that the hardware team is working on this but I don't know the 
timelines and there are a few upcoming controllers - of which I don't 
know the device ids yet - that have the same issue.

To avoid issues, it is easier to apply the quirk to all Amazon NVMe 
controllers for now till the new lines of controllers with the fix comes 
out.  At that point, we'll be able to restrict the application to the 
known bad controllers.



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879



From hch at lst.de  Wed Feb  3 06:26:45 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 12:26:45 +0100
Subject: [PATCH] nvme: Add 48-bit DMA address quirk
In-Reply-To: <396b45d5-3dd5-8159-34c2-5668510d1d68@amazon.de>
References: <20210203094338.19473-1-sironi@amazon.de>
 <20210203095148.GA8897@lst.de>
 <0c38f5eb-41ef-7934-940b-77b6e73c5239@amazon.de>
 <20210203111548.GA11075@lst.de>
 <396b45d5-3dd5-8159-34c2-5668510d1d68@amazon.de>
Message-ID: <20210203112645.GA11942@lst.de>

On Wed, Feb 03, 2021 at 12:22:31PM +0100, Filippo Sironi wrote:
> To avoid issues, it is easier to apply the quirk to all Amazon NVMe 
> controllers for now till the new lines of controllers with the fix comes 
> out.  At that point, we'll be able to restrict the application to the known 
> bad controllers.

No, that is simply not acceptable.  For one we've had enough cases where
knowlege about old devices was lost after a decade or so.  And secondly
shipping more of these broken devices should be as painful as possible
for you so that it preferably does not happen at all.


From mwilck at suse.com  Wed Feb  3 07:31:13 2021
From: mwilck at suse.com (Martin Wilck)
Date: Wed, 03 Feb 2021 13:31:13 +0100
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <20210202075828.GA12160@lst.de>
References: <20210202005951.26614-1-mwilck@suse.com>
 <20210202075828.GA12160@lst.de>
Message-ID: <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>

On Tue, 2021-02-02 at 08:58 +0100, Christoph Hellwig wrote:
> On Tue, Feb 02, 2021 at 01:59:51AM +0100, mwilck at suse.com?wrote:
> > From: Martin Wilck <mwilck at suse.com>
> > 
> > While a controller is still connecting, ctrl->subsys is NULL and
> > thus reading the controller's "subsysnqn" sysfs attribute returns
> > "(efault)". This happens all the time when user space processes
> > "add" events for NVMe controller devices.
> > 
> > Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
> > if ctrl->subsys isn't initialized yet.
> 
> This is just papering over symptoms.? We should not be sending
> events or display sysfs files until the controlle is live.

I don't follow.?What's wrong with a "connecting" controller device in
sysfs??

Controllers can loose connectivity any time. What's the difference
between a controller that is reconnecting after a connection loss and a
freshly created controller that's just not live yet??You wouldn't
delete the former from sysfs, I suppose.

>From the user space point of view, it's the same, a controller device
that is not in usable state. I don't see a general problem with that.
Seeing a controller appear in connecting state might actually be useful
for user space.

The subsysnqn attribute is correct for controllers that were once
connected and lost connectivity. My patch was only intended to fix the
"just created and never connected" case, where it is not, nothing more.

Regards
Martin

> 
> In other wwords - I think we should remove the cdev_device_add
> from nvme_init_ctrl to nvme_start_ctrl (+ any error handling changes
> resulting from that).
> 




From hch at lst.de  Wed Feb  3 08:37:12 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 14:37:12 +0100
Subject: [PATCH V2 3/3] Adding
 device_dma_parameters->offset_preserve_mask to NVMe driver.
In-Reply-To: <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>
 <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
 <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
Message-ID: <20210203133712.GA24674@lst.de>

Please try with this extra patch:

---
>From 212764c3c15ce859e6f55d2146f450ea4ca6fdb9 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <hch at lst.de>
Date: Wed, 3 Feb 2021 14:27:13 +0100
Subject: nvme-pci: fix 2nd PRP setup in nvme_setup_prp_simple

Use the dma address instead of the bio_vec offset for the arithmetics
to find the address for the 2nd PRP.

Fixes: dff824b2aadb ("nvme-pci: optimize mapping of small single segment requests")
Reported-by: Jianxiong Gao <jxgao at google.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 30e45f7e0f750c..4ae51735d72f4a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -808,8 +808,7 @@ static blk_status_t nvme_setup_prp_simple(struct nvme_dev *dev,
 		struct bio_vec *bv)
 {
 	struct nvme_iod *iod = blk_mq_rq_to_pdu(req);
-	unsigned int offset = bv->bv_offset & (NVME_CTRL_PAGE_SIZE - 1);
-	unsigned int first_prp_len = NVME_CTRL_PAGE_SIZE - offset;
+	dma_addr_t next_prp;
 
 	iod->first_dma = dma_map_bvec(dev->dev, bv, rq_dma_dir(req), 0);
 	if (dma_mapping_error(dev->dev, iod->first_dma))
@@ -817,8 +816,9 @@ static blk_status_t nvme_setup_prp_simple(struct nvme_dev *dev,
 	iod->dma_len = bv->bv_len;
 
 	cmnd->dptr.prp1 = cpu_to_le64(iod->first_dma);
-	if (bv->bv_len > first_prp_len)
-		cmnd->dptr.prp2 = cpu_to_le64(iod->first_dma + first_prp_len);
+	next_prp = round_down(iod->first_dma + bv->bv_len, NVME_CTRL_PAGE_SIZE);
+	if (next_prp > iod->first_dma)
+		cmnd->dptr.prp2 = cpu_to_le64(next_prp);
 	return BLK_STS_OK;
 }
 
-- 
2.29.2



From hch at lst.de  Wed Feb  3 10:57:32 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 16:57:32 +0100
Subject: [PATCH] nvmet-tcp: Fix out-of-bounds access when receiving
 multiple h2cdata PDUs
In-Reply-To: <20210203092025.437839-1-sagi@grimberg.me>
References: <20210203092025.437839-1-sagi@grimberg.me>
Message-ID: <20210203155732.GA2785@lst.de>

Applied to nvme-5.11.


From hch at lst.de  Wed Feb  3 11:13:44 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 17:13:44 +0100
Subject: [PATCH] nvme: convert sysfs sprintf/snprintf family to sysfs_emit
In-Reply-To: <1612249577-66130-1-git-send-email-jiapeng.chong@linux.alibaba.com>
References: <1612249577-66130-1-git-send-email-jiapeng.chong@linux.alibaba.com>
Message-ID: <20210203161344.GA4116@lst.de>

Thanks,

applied to nvme-5.12.


From hch at lst.de  Wed Feb  3 11:14:55 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 17:14:55 +0100
Subject: [PATCH v5 0/3] avoid double request completion and IO error
In-Reply-To: <20210201034940.18891-1-lengchao@huawei.com>
References: <20210201034940.18891-1-lengchao@huawei.com>
Message-ID: <20210203161455.GB4116@lst.de>

So I think this is conceptually fine, but I still find the API a little
arcane.  What do you think about the following incremental patch?
If that looks good and tests good for you I can apply the series with
the modifications:

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 0befaad788a094..02579f4f776c7d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -355,6 +355,21 @@ void nvme_complete_rq(struct request *req)
 }
 EXPORT_SYMBOL_GPL(nvme_complete_rq);
 
+/*
+ * Called to unwind from ->queue_rq on a failed command submission so that the
+ * multipathing code gets called to potentially failover to another path.
+ * The caller needs to unwind all transport specific resource allocations and
+ * must return propagate the return value.
+ */
+blk_status_t nvme_host_path_error(struct request *req)
+{
+	nvme_req(req)->status = NVME_SC_HOST_PATH_ERROR;
+	blk_mq_set_request_complete(req);
+	nvme_complete_rq(req);
+	return BLK_STS_OK;
+}
+EXPORT_SYMBOL_GPL(nvme_host_path_error);
+
 bool nvme_cancel_request(struct request *req, void *data, bool reserved)
 {
 	dev_dbg_ratelimited(((struct nvme_ctrl *) data)->device,
diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index cedf9b31898673..5dfd806fc2d28c 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -552,11 +552,7 @@ blk_status_t nvmf_fail_nonready_command(struct nvme_ctrl *ctrl,
 	    !test_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags) &&
 	    !blk_noretry_request(rq) && !(rq->cmd_flags & REQ_NVME_MPATH))
 		return BLK_STS_RESOURCE;
-
-	nvme_req(rq)->status = NVME_SC_HOST_PATH_ERROR;
-	blk_mq_set_request_complete(rq);
-	nvme_complete_rq(rq);
-	return BLK_STS_OK;
+	return nvme_host_path_error(rq);
 }
 EXPORT_SYMBOL_GPL(nvmf_fail_nonready_command);
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index a72f0718109100..5819f038104149 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -575,6 +575,7 @@ static inline bool nvme_is_aen_req(u16 qid, __u16 command_id)
 }
 
 void nvme_complete_rq(struct request *req);
+blk_status_t nvme_host_path_error(struct request *req);
 bool nvme_cancel_request(struct request *req, void *data, bool reserved);
 void nvme_cancel_tagset(struct nvme_ctrl *ctrl);
 void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl);
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 6993efb27b39f0..f51af5e4970a2b 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -2091,16 +2091,6 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
 	err = nvme_rdma_post_send(queue, sqe, req->sge, req->num_sge,
 			req->mr ? &req->reg_wr.wr : NULL);
 	if (unlikely(err)) {
-		if (err == -EIO) {
-			/*
-			 * Fail the reqest so upper layer can failover I/O
-			 * if another path is available
-			 */
-			req->status = NVME_SC_HOST_PATH_ERROR;
-			blk_mq_set_request_complete(rq);
-			nvme_rdma_complete_rq(rq);
-			return BLK_STS_OK;
-		}
 		goto err_unmap;
 	}
 
@@ -2109,7 +2099,9 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
 err_unmap:
 	nvme_rdma_unmap_data(queue, rq);
 err:
-	if (err == -ENOMEM || err == -EAGAIN)
+	if (err == -EIO)
+		ret = nvme_host_path_error(rq);
+	else if (err == -ENOMEM || err == -EAGAIN)
 		ret = BLK_STS_RESOURCE;
 	else
 		ret = BLK_STS_IOERR;


From jxgao at google.com  Wed Feb  3 11:47:45 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Wed, 3 Feb 2021 08:47:45 -0800
Subject: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask
 to NVMe driver.
In-Reply-To: <20210203133712.GA24674@lst.de>
References: <20210201183017.3339130-1-jxgao@google.com>
 <20210201183017.3339130-4-jxgao@google.com>
 <20210201205759.GA2128135@dhcp-10-100-145-180.wdc.com>
 <CAMGD6P2Gz9nWELMdsAhwQvXx3PXv2aXet=Tn9Rca61obZawfgw@mail.gmail.com>
 <CAMGD6P1_cs1W8jMt7Sz6broDdnFNPrxbiZW-JZ+GAtg5aoTdOQ@mail.gmail.com>
 <20210203133712.GA24674@lst.de>
Message-ID: <CAMGD6P1Ksn4DJ0w=O20Ou7BOKAW9yzwJgRV57xaqLdVDK4ATaQ@mail.gmail.com>

>
> Please try with this extra patch:
>
I have tried with the extra patch and it still fails to boot.
I have attached dmesg output for the error:

-------------dmesg starts here-----------------
[    6.357755] XFS (nvme0n1p2): Mounting V5 Filesystem
[    6.430092] XFS (nvme0n1p2): Torn write (CRC failure) detected at
log block 0x20d0. Truncating head block from 0x20e0.
[    6.442828] XFS (nvme0n1p2): Starting recovery (logdev: internal)
[    7.272456] XFS (nvme0n1p2): Ending recovery (logdev: internal)
[    7.610250] systemd-journald[434]: Received SIGTERM from PID 1 (systemd).
...
[   10.054121] systemd[755]:
/usr/lib/systemd/system-generators/systemd-rc-local-generator
terminated by signal ABRT.
[   10.726122] audit: type=1400 audit(1612370261.090:4): avc:  denied
{ search } for  pid=783 comm="systemd-sysctl" name="crash"
dev="nvme0n1p2" ino=50789805
scontext=system_u:system_r:systemd_sysctl_t:s0
tcontext=system_u:object_r:kdump_crash_t:s0 tclass=dir permissive=0
[   10.751764] audit: type=1400 audit(1612370261.090:5): avc:  denied
{ search } for  pid=783 comm="systemd-sysctl" name="crash"
dev="nvme0n1p2" ino=50789805
scontext=system_u:system_r:systemd_sysctl_t:s0
tcontext=system_u:object_r:kdump_crash_t:s0 tclass=dir permissive=0
[   12.366607] xfs filesystem being remounted at / supports timestamps
until 2038 (0x7fffffff)
[   12.376379] audit: type=1400 audit(1612370262.740:6): avc:  denied
{ write } for  pid=788 comm="systemd-remount" name="crash"
dev="nvme0n1p2" ino=50789805 scontext=system_u:system_r:init_t:s0
tcontext=system_u:object_r:kdump_crash_t:s0 tclass=dir permissive=0
[   12.413155] systemd-journald[781]: Received request to flush
runtime journal from PID 1
[   12.428917] audit: type=1400 audit(1612370262.793:7): avc:  denied
{ write } for  pid=815 comm="journalctl" name="crash" dev="nvme0n1p2"
ino=50789805 scontext=system_u:system_r:init_t:s0
tcontext=system_u:object_r:kdump_crash_t:s0 tclass=dir permissive=0
[   12.453006] audit: type=1400 audit(1612370262.799:8): avc:  denied
{ write } for  pid=817 comm="systemd-random-" name="crash"
dev="nvme0n1p2" ino=50789805 scontext=system_u:system_r:init_t:s0
tcontext=system_u:object_r:kdump_crash_t:s0 tclass=dir permissive=0
[   13.306030] plymouth[853]: segfault at 0 ip 00007f2eabca8090 sp
00007fffe94d8c08 error 6 in libc-2.28.so[7f2eabbcd000+1b9000]
[   13.318772] Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00
[   78.782418] sed[911]: segfault at 0 ip 00007fc3540da090 sp
00007ffdb4373ff8 error 6 in libc-2.28.so[7fc353fff000+1b9000]
[   78.794007] Code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00
--------------dmesg ends here-----------

-- 
Jianxiong Gao


From hch at lst.de  Wed Feb  3 11:51:49 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 17:51:49 +0100
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>
References: <20210202005951.26614-1-mwilck@suse.com>
 <20210202075828.GA12160@lst.de>
 <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>
Message-ID: <20210203165149.GA6691@lst.de>

On Wed, Feb 03, 2021 at 01:31:13PM +0100, Martin Wilck wrote:
> > This is just papering over symptoms.? We should not be sending
> > events or display sysfs files until the controlle is live.
> 
> I don't follow.?What's wrong with a "connecting" controller device in
> sysfs??
> 
> Controllers can loose connectivity any time. What's the difference
> between a controller that is reconnecting after a connection loss and a
> freshly created controller that's just not live yet??You wouldn't
> delete the former from sysfs, I suppose.

What is wrong is that we should only add the device and thus send the
uevent notifcations when the device is fully set up.


From kbusch at kernel.org  Wed Feb  3 11:52:03 2021
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 3 Feb 2021 08:52:03 -0800
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>
References: <20210202005951.26614-1-mwilck@suse.com>
 <20210202075828.GA12160@lst.de>
 <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>
Message-ID: <20210203165203.GA2182779@dhcp-10-100-145-180.wdc.com>

On Wed, Feb 03, 2021 at 01:31:13PM +0100, Martin Wilck wrote:
> On Tue, 2021-02-02 at 08:58 +0100, Christoph Hellwig wrote:
> > On Tue, Feb 02, 2021 at 01:59:51AM +0100, mwilck at suse.com?wrote:
> > > From: Martin Wilck <mwilck at suse.com>
> > > 
> > > While a controller is still connecting, ctrl->subsys is NULL and
> > > thus reading the controller's "subsysnqn" sysfs attribute returns
> > > "(efault)". This happens all the time when user space processes
> > > "add" events for NVMe controller devices.
> > > 
> > > Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
> > > if ctrl->subsys isn't initialized yet.
> > 
> > This is just papering over symptoms.? We should not be sending
> > events or display sysfs files until the controlle is live.
> 
> I don't follow.?What's wrong with a "connecting" controller device in
> sysfs??
> 
> Controllers can loose connectivity any time. What's the difference
> between a controller that is reconnecting after a connection loss and a
> freshly created controller that's just not live yet??You wouldn't
> delete the former from sysfs, I suppose.
> 
> From the user space point of view, it's the same, a controller device
> that is not in usable state. I don't see a general problem with that.
> Seeing a controller appear in connecting state might actually be useful
> for user space.
> 
> The subsysnqn attribute is correct for controllers that were once
> connected and lost connectivity. My patch was only intended to fix the
> "just created and never connected" case, where it is not, nothing more.

I think Christoph is suggesting just wait until we have all the
properties the driver exports rather than specifically exporting only
during a 'live' state. Something like this (untested) should do the
trick:

---
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1a3cdc6b1036..4c0258d7eb91 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3069,6 +3069,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		if (ret)
 			goto out_free;
 
+		ret = cdev_device_add(&ctrl->cdev, ctrl->device);
+		if (ret)
+			goto out_free;
 		/*
 		 * Check for quirks.  Quirk can depend on firmware version,
 		 * so, in principle, the set of quirks present can change
@@ -4545,9 +4548,6 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	nvme_get_ctrl(ctrl);
 	cdev_init(&ctrl->cdev, &nvme_dev_fops);
 	ctrl->cdev.owner = ops->module;
-	ret = cdev_device_add(&ctrl->cdev, ctrl->device);
-	if (ret)
-		goto out_free_name;
 
 	/*
 	 * Initialize latency tolerance controls.  The sysfs files won't
@@ -4560,9 +4560,6 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	nvme_fault_inject_init(&ctrl->fault_inject, dev_name(ctrl->device));
 
 	return 0;
-out_free_name:
-	nvme_put_ctrl(ctrl);
-	kfree_const(ctrl->device->kobj.name);
 out_release_instance:
 	ida_simple_remove(&nvme_instance_ida, ctrl->instance);
 out:
--


From hch at lst.de  Wed Feb  3 11:56:21 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 3 Feb 2021 17:56:21 +0100
Subject: [PATCH v3 net-next 01/21] iov_iter: Introduce new procedures
 for copy to iter/pages
In-Reply-To: <CAJ3xEMjLKoQe_OB_L+w2wwUGck74Gm6=GPA=CK73QpeFbXr7Bw@mail.gmail.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-2-borisp@mellanox.com> <20210201173548.GA12960@lst.de>
 <CAJ3xEMjLKoQe_OB_L+w2wwUGck74Gm6=GPA=CK73QpeFbXr7Bw@mail.gmail.com>
Message-ID: <20210203165621.GB6691@lst.de>

On Tue, Feb 02, 2021 at 08:00:51PM +0200, Or Gerlitz wrote:
> will look into this, any idea for a more suitable location?

Maybe just a new file under lib/ for now?

> > Overly long line.  But we're also looking into generic helpers for
> > this kind of things, not sure if they made it to linux-next in the
> > meantime, but please check.
> 
> This is what I found in linux-next - note sure if you were referring to it
> 
> commit 11432a3cc061c39475295be533c3674c4f8a6d0b
> Author: David Howells <dhowells at redhat.com>
> 
>     iov_iter: Add ITER_XARRAY

No, that's not it.  Ira, what is the status of the memcpy_{to,from}_page
helpers?


From kbusch at kernel.org  Wed Feb  3 11:57:01 2021
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 3 Feb 2021 08:57:01 -0800
Subject: [PATCH] nvme: Add 48-bit DMA address quirk
In-Reply-To: <396b45d5-3dd5-8159-34c2-5668510d1d68@amazon.de>
References: <20210203094338.19473-1-sironi@amazon.de>
 <20210203095148.GA8897@lst.de>
 <0c38f5eb-41ef-7934-940b-77b6e73c5239@amazon.de>
 <20210203111548.GA11075@lst.de>
 <396b45d5-3dd5-8159-34c2-5668510d1d68@amazon.de>
Message-ID: <20210203165701.GB2182779@dhcp-10-100-145-180.wdc.com>

On Wed, Feb 03, 2021 at 12:22:31PM +0100, Filippo Sironi wrote:
> 
> On 2/3/21 12:15 PM, Christoph Hellwig wrote:
> > 
> > On Wed, Feb 03, 2021 at 12:12:31PM +0100, Filippo Sironi wrote:
> > > I don't disagree on the first part of your sentence, this is a big
> > > oversight.
> > 
> > But it is not what your commit log suggests.
> 
> I can definitely rephrase the commit.
> 
> > > On the other hand, those controllers are out there and are in use by a lot
> > > of customers.  We can keep relying on luck, hoping that customers don't run
> > > into troubles or we can merge a few lines of code :)
> > 
> > Your patch does not just quirk a few controllers out there, but all
> > current and future controllers with an Amazon vendor ID.  We could
> > probably talk about quirking an existing vendor ID or two as long as
> > this doesn't happen for future hardware.
> 
> I know that the hardware team is working on this but I don't know the
> timelines and there are a few upcoming controllers - of which I don't know
> the device ids yet - that have the same issue.
> 
> To avoid issues, it is easier to apply the quirk to all Amazon NVMe
> controllers for now till the new lines of controllers with the fix comes
> out.  At that point, we'll be able to restrict the application to the known
> bad controllers.

Just set the quirk for the known device id's and append more as needed
(which should hopefully never happen). Sure, your future broken devices
may not work with the first kernel that introduced the quirk, but this
is how the quirks should be documented in the code.


From ira.weiny at intel.com  Wed Feb  3 14:34:34 2021
From: ira.weiny at intel.com (Ira Weiny)
Date: Wed, 3 Feb 2021 11:34:34 -0800
Subject: [PATCH v3 net-next 01/21] iov_iter: Introduce new procedures for
 copy to iter/pages
In-Reply-To: <20210203165621.GB6691@lst.de>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-2-borisp@mellanox.com>
 <20210201173548.GA12960@lst.de>
 <CAJ3xEMjLKoQe_OB_L+w2wwUGck74Gm6=GPA=CK73QpeFbXr7Bw@mail.gmail.com>
 <20210203165621.GB6691@lst.de>
Message-ID: <20210203193434.GD3200985@iweiny-DESK2.sc.intel.com>

On Wed, Feb 03, 2021 at 05:56:21PM +0100, Christoph Hellwig wrote:
> On Tue, Feb 02, 2021 at 08:00:51PM +0200, Or Gerlitz wrote:
> > will look into this, any idea for a more suitable location?
> 
> Maybe just a new file under lib/ for now?
> 
> > > Overly long line.  But we're also looking into generic helpers for
> > > this kind of things, not sure if they made it to linux-next in the
> > > meantime, but please check.
> > 
> > This is what I found in linux-next - note sure if you were referring to it
> > 
> > commit 11432a3cc061c39475295be533c3674c4f8a6d0b
> > Author: David Howells <dhowells at redhat.com>
> > 
> >     iov_iter: Add ITER_XARRAY
> 
> No, that's not it.  Ira, what is the status of the memcpy_{to,from}_page
> helpers?

Converting the entire kernel tree in one series has become unwieldy so I've
given up.

But I have a series to convert btrfs which I could release by the end of the
week.  That should be good enough to push the memcpy_*_page() support in.

I'm get it formatted and submitted,
Ira


From sagi at grimberg.me  Wed Feb  3 17:19:44 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 14:19:44 -0800
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <20210203165203.GA2182779@dhcp-10-100-145-180.wdc.com>
References: <20210202005951.26614-1-mwilck@suse.com>
 <20210202075828.GA12160@lst.de>
 <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>
 <20210203165203.GA2182779@dhcp-10-100-145-180.wdc.com>
Message-ID: <8b1fc5ef-4c6b-4fa3-7cb1-b42ed71e7413@grimberg.me>



On 2/3/21 8:52 AM, Keith Busch wrote:
> On Wed, Feb 03, 2021 at 01:31:13PM +0100, Martin Wilck wrote:
>> On Tue, 2021-02-02 at 08:58 +0100, Christoph Hellwig wrote:
>>> On Tue, Feb 02, 2021 at 01:59:51AM +0100, mwilck at suse.com?wrote:
>>>> From: Martin Wilck <mwilck at suse.com>
>>>>
>>>> While a controller is still connecting, ctrl->subsys is NULL and
>>>> thus reading the controller's "subsysnqn" sysfs attribute returns
>>>> "(efault)". This happens all the time when user space processes
>>>> "add" events for NVMe controller devices.
>>>>
>>>> Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
>>>> if ctrl->subsys isn't initialized yet.
>>>
>>> This is just papering over symptoms.? We should not be sending
>>> events or display sysfs files until the controlle is live.
>>
>> I don't follow.?What's wrong with a "connecting" controller device in
>> sysfs?
>>
>> Controllers can loose connectivity any time. What's the difference
>> between a controller that is reconnecting after a connection loss and a
>> freshly created controller that's just not live yet??You wouldn't
>> delete the former from sysfs, I suppose.
>>
>>  From the user space point of view, it's the same, a controller device
>> that is not in usable state. I don't see a general problem with that.
>> Seeing a controller appear in connecting state might actually be useful
>> for user space.
>>
>> The subsysnqn attribute is correct for controllers that were once
>> connected and lost connectivity. My patch was only intended to fix the
>> "just created and never connected" case, where it is not, nothing more.
> 
> I think Christoph is suggesting just wait until we have all the
> properties the driver exports rather than specifically exporting only
> during a 'live' state. Something like this (untested) should do the
> trick:
> 
> ---
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 1a3cdc6b1036..4c0258d7eb91 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -3069,6 +3069,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
>   		if (ret)
>   			goto out_free;
>   
> +		ret = cdev_device_add(&ctrl->cdev, ctrl->device);
> +		if (ret)
> +			goto out_free;

This happens on every reset, we should probably just do this once?


From sagi at grimberg.me  Wed Feb  3 17:22:00 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 14:22:00 -0800
Subject: [PATCH v5 0/3] avoid double request completion and IO error
In-Reply-To: <20210203161455.GB4116@lst.de>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210203161455.GB4116@lst.de>
Message-ID: <ae2ff390-739e-ac2d-a7a8-162a6297653f@grimberg.me>



On 2/3/21 8:14 AM, Christoph Hellwig wrote:
> So I think this is conceptually fine, but I still find the API a little
> arcane.  What do you think about the following incremental patch?
> If that looks good and tests good for you I can apply the series with
> the modifications:
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 0befaad788a094..02579f4f776c7d 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -355,6 +355,21 @@ void nvme_complete_rq(struct request *req)
>   }
>   EXPORT_SYMBOL_GPL(nvme_complete_rq);
>   
> +/*
> + * Called to unwind from ->queue_rq on a failed command submission so that the
> + * multipathing code gets called to potentially failover to another path.
> + * The caller needs to unwind all transport specific resource allocations and
> + * must return propagate the return value.
> + */
> +blk_status_t nvme_host_path_error(struct request *req)
> +{
> +	nvme_req(req)->status = NVME_SC_HOST_PATH_ERROR;
> +	blk_mq_set_request_complete(req);
> +	nvme_complete_rq(req);
> +	return BLK_STS_OK;
> +}
> +EXPORT_SYMBOL_GPL(nvme_host_path_error);
> +
>   bool nvme_cancel_request(struct request *req, void *data, bool reserved)
>   {
>   	dev_dbg_ratelimited(((struct nvme_ctrl *) data)->device,
> diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
> index cedf9b31898673..5dfd806fc2d28c 100644
> --- a/drivers/nvme/host/fabrics.c
> +++ b/drivers/nvme/host/fabrics.c
> @@ -552,11 +552,7 @@ blk_status_t nvmf_fail_nonready_command(struct nvme_ctrl *ctrl,
>   	    !test_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags) &&
>   	    !blk_noretry_request(rq) && !(rq->cmd_flags & REQ_NVME_MPATH))
>   		return BLK_STS_RESOURCE;
> -
> -	nvme_req(rq)->status = NVME_SC_HOST_PATH_ERROR;
> -	blk_mq_set_request_complete(rq);
> -	nvme_complete_rq(rq);
> -	return BLK_STS_OK;
> +	return nvme_host_path_error(rq);
>   }
>   EXPORT_SYMBOL_GPL(nvmf_fail_nonready_command);
>   
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index a72f0718109100..5819f038104149 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -575,6 +575,7 @@ static inline bool nvme_is_aen_req(u16 qid, __u16 command_id)
>   }
>   
>   void nvme_complete_rq(struct request *req);
> +blk_status_t nvme_host_path_error(struct request *req);
>   bool nvme_cancel_request(struct request *req, void *data, bool reserved);
>   void nvme_cancel_tagset(struct nvme_ctrl *ctrl);
>   void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl);
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 6993efb27b39f0..f51af5e4970a2b 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -2091,16 +2091,6 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
>   	err = nvme_rdma_post_send(queue, sqe, req->sge, req->num_sge,
>   			req->mr ? &req->reg_wr.wr : NULL);
>   	if (unlikely(err)) {
> -		if (err == -EIO) {
> -			/*
> -			 * Fail the reqest so upper layer can failover I/O
> -			 * if another path is available
> -			 */
> -			req->status = NVME_SC_HOST_PATH_ERROR;
> -			blk_mq_set_request_complete(rq);
> -			nvme_rdma_complete_rq(rq);
> -			return BLK_STS_OK;
> -		}
>   		goto err_unmap;
>   	}
>   
> @@ -2109,7 +2099,9 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
>   err_unmap:
>   	nvme_rdma_unmap_data(queue, rq);
>   err:
> -	if (err == -ENOMEM || err == -EAGAIN)
> +	if (err == -EIO)
> +		ret = nvme_host_path_error(rq);
> +	else if (err == -ENOMEM || err == -EAGAIN)
>   		ret = BLK_STS_RESOURCE;
>   	else
>   		ret = BLK_STS_IOERR;
> 

This looks good to me.


From sagi at grimberg.me  Wed Feb  3 17:36:53 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 14:36:53 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>


>> I'm running "nvme discover" over a TCP connection. The nvme-tcp module freezes completely and bricks my computer.
>> 
>> Steps:
>> $ sudo modprobe nvme-tcp
>> $ sudo nvme discover -t tcp -a [IP address] -s 8009
>> <System Bricked!>
>> Only a reboot (Alt-SysRq-B) can recover the system.
> 
> Do you have a stack trace to share?
> *<MB> No. I'm not able to collect data since my computer freezes 
> completely. I had a window opened that was showing the syslog at the 
> time the freeze occurred. I took a couple of pictures (attached). *

OK, Are you able to scroll up a bit to see the RIP line in the
stack trace? and if this is a NULL dereference or something else?

>> Conditions to reproduce the problem:
>> The Discovery Controller must support sending Discovery Log Change Notifications. That is, bit 31 of the Identity's OAES field returned by the discovery controller must be set to 1. If OAES[31]=0, then everything is OK.
> 
> What is the discovery log page returned by the nvme discovery
> controller? Does it include referrals? There was an issue fixed
> in nvme-cli with respect to referrals (although nothing that is
> related to any oaes changes).
> *<MB> The driver never makes it to asking for the discovery log page. It 
> freezes as soon as it receives the "Identity" message. *

You mean identify controller? That is strange because not sure it should
be any difference..

>> Systems tested:
>> 1) Ubuntu 20.04, Linux 5.8, nvme 1.13.21
>> 2) Fedora 33, Linux 5.10, nvme 1.11.1
> 
> Are these the default kernels that come with the distribution?
> *<MB> Yes.**
> *
> *On Ubuntu 20.04: *
> *$ uname -rsvpi*
> *Linux 5.8.0-41-generic #46~20.04.1-Ubuntu SMP Mon Jan 18 17:52:23 UTC 
> 2021 x86_64 x86_64
> *
> *
> *
> *On Fedora 33:*
> *$ uname -rsvpi*
> *Linux 5.10.11-200.fc33.x86_64 #1 SMP Wed Jan 27 20:21:22 UTC 2021 
> x86_64 x86_64
> *
> 
> Does this happen with the latest upstream?
> *<MB> I did compile the latest upstream kernel modules, but 
> Ubuntu/Fedora won't let me modprobe them. I'm not a kernel expert. There 
> seems to be some security in place that prevents one from loading a 
> kernel module that did not come with the official release. I tried 
> several things suggested on Google to work around this but could never 
> get the latest kernel modules loaded. By the way, Fedora 33 is pretty 
> close to the latest upstream (i.e. Linux 5.10) and I see the same issue 
> there.*

We definitely didn't get such a bug report on this kernel.

Does this happen if you directly connect to a normal nvme controller?

> *<MB> As I said earlier, everything works fine until I change the 
> Discovery Controller to return **OAES[31]=1 in the Identity message. 
>  From what I see in the nvme-tcp code, this tells the driver to enable 
> AER/AEN. I think that's where the issue is. Since I'm not a kernel 
> expert, I cannot diagnose the problem further than that.*

It appears that without discovery log change events we never submit
async event, so it must be something there.

Can you share your kernel config file?

> I'm assuming this is not Linux nvmet target correct?
> *<MB> I don't know what that means: nvmet? *

What is your target implementation? Is this the nvme target that
is built into Linux?


From sagi at grimberg.me  Wed Feb  3 17:56:08 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 14:56:08 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
Message-ID: <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>

Martin,

I can actually see a RIP line,

Can you run gdb on vmlinux and share what this provides?

l *(nvme_tcp_init_iter+0x60)

This is strange that we even get there, because we only
initialize a request iter if we have data, and this shouldn't
be the case for the async request.


From sagi at grimberg.me  Wed Feb  3 18:00:01 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed,  3 Feb 2021 15:00:01 -0800
Subject: [PATCH] nvmet-tcp: Fix receive data digest calculation for multiple
 h2cdata PDUs
Message-ID: <20210203230001.18797-1-sagi@grimberg.me>

When a host sends multiple h2cdata PDUs for a single command, we
should verify the data digest calculation per PDU and not
per command.

Fixes: 872d26a391da ("nvmet-tcp: add NVMe over TCP target driver")
Reported-by: Narayan Ayalasomayajula <Narayan.Ayalasomayajula at wdc.com>
Tested-by: Narayan Ayalasomayajula <Narayan.Ayalasomayajula at wdc.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/target/tcp.c | 31 ++++++++++++++++++++++++-------
 1 file changed, 24 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index aacf06f0b431..577ce7d403ae 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -379,7 +379,7 @@ static int nvmet_tcp_map_data(struct nvmet_tcp_cmd *cmd)
 	return NVME_SC_INTERNAL;
 }
 
-static void nvmet_tcp_ddgst(struct ahash_request *hash,
+static void nvmet_tcp_send_ddgst(struct ahash_request *hash,
 		struct nvmet_tcp_cmd *cmd)
 {
 	ahash_request_set_crypt(hash, cmd->req.sg,
@@ -387,6 +387,23 @@ static void nvmet_tcp_ddgst(struct ahash_request *hash,
 	crypto_ahash_digest(hash);
 }
 
+static void nvmet_tcp_recv_ddgst(struct ahash_request *hash,
+		struct nvmet_tcp_cmd *cmd)
+{
+	struct scatterlist sg;
+	struct kvec *iov;
+	int i;
+
+	crypto_ahash_init(hash);
+	for (i = 0, iov = cmd->iov; i < cmd->nr_mapped; i++, iov++) {
+		sg_init_one(&sg, iov->iov_base, iov->iov_len);
+		ahash_request_set_crypt(hash, &sg, NULL, iov->iov_len);
+		crypto_ahash_update(hash);
+	}
+	ahash_request_set_crypt(hash, NULL, (void *)&cmd->exp_ddgst, 0);
+	crypto_ahash_final(hash);
+}
+
 static void nvmet_setup_c2h_data_pdu(struct nvmet_tcp_cmd *cmd)
 {
 	struct nvme_tcp_data_pdu *pdu = cmd->data_pdu;
@@ -411,7 +428,7 @@ static void nvmet_setup_c2h_data_pdu(struct nvmet_tcp_cmd *cmd)
 
 	if (queue->data_digest) {
 		pdu->hdr.flags |= NVME_TCP_F_DDGST;
-		nvmet_tcp_ddgst(queue->snd_hash, cmd);
+		nvmet_tcp_send_ddgst(queue->snd_hash, cmd);
 	}
 
 	if (cmd->queue->hdr_digest) {
@@ -1060,7 +1077,7 @@ static void nvmet_tcp_prep_recv_ddgst(struct nvmet_tcp_cmd *cmd)
 {
 	struct nvmet_tcp_queue *queue = cmd->queue;
 
-	nvmet_tcp_ddgst(queue->rcv_hash, cmd);
+	nvmet_tcp_recv_ddgst(queue->rcv_hash, cmd);
 	queue->offset = 0;
 	queue->left = NVME_TCP_DIGEST_LENGTH;
 	queue->rcv_state = NVMET_TCP_RECV_DDGST;
@@ -1081,14 +1098,14 @@ static int nvmet_tcp_try_recv_data(struct nvmet_tcp_queue *queue)
 		cmd->rbytes_done += ret;
 	}
 
+	if (queue->data_digest) {
+		nvmet_tcp_prep_recv_ddgst(cmd);
+		return 0;
+	}
 	nvmet_tcp_unmap_pdu_iovec(cmd);
 
 	if (!(cmd->flags & NVMET_TCP_F_INIT_FAILED) &&
 	    cmd->rbytes_done == cmd->req.transfer_len) {
-		if (queue->data_digest) {
-			nvmet_tcp_prep_recv_ddgst(cmd);
-			return 0;
-		}
 		cmd->req.execute(&cmd->req);
 	}
 
-- 
2.27.0



From kbusch at kernel.org  Wed Feb  3 21:42:20 2021
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 3 Feb 2021 18:42:20 -0800
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <8b1fc5ef-4c6b-4fa3-7cb1-b42ed71e7413@grimberg.me>
References: <20210202005951.26614-1-mwilck@suse.com>
 <20210202075828.GA12160@lst.de>
 <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>
 <20210203165203.GA2182779@dhcp-10-100-145-180.wdc.com>
 <8b1fc5ef-4c6b-4fa3-7cb1-b42ed71e7413@grimberg.me>
Message-ID: <20210204024220.GE2182779@dhcp-10-100-145-180.wdc.com>

On Wed, Feb 03, 2021 at 02:19:44PM -0800, Sagi Grimberg wrote:
> 
> 
> On 2/3/21 8:52 AM, Keith Busch wrote:
> > On Wed, Feb 03, 2021 at 01:31:13PM +0100, Martin Wilck wrote:
> > > On Tue, 2021-02-02 at 08:58 +0100, Christoph Hellwig wrote:
> > > > On Tue, Feb 02, 2021 at 01:59:51AM +0100, mwilck at suse.com?wrote:
> > > > > From: Martin Wilck <mwilck at suse.com>
> > > > > 
> > > > > While a controller is still connecting, ctrl->subsys is NULL and
> > > > > thus reading the controller's "subsysnqn" sysfs attribute returns
> > > > > "(efault)". This happens all the time when user space processes
> > > > > "add" events for NVMe controller devices.
> > > > > 
> > > > > Fix it by returning the nvmf_ctrl_options' subsysnqn attribute
> > > > > if ctrl->subsys isn't initialized yet.
> > > > 
> > > > This is just papering over symptoms.? We should not be sending
> > > > events or display sysfs files until the controlle is live.
> > > 
> > > I don't follow.?What's wrong with a "connecting" controller device in
> > > sysfs?
> > > 
> > > Controllers can loose connectivity any time. What's the difference
> > > between a controller that is reconnecting after a connection loss and a
> > > freshly created controller that's just not live yet??You wouldn't
> > > delete the former from sysfs, I suppose.
> > > 
> > >  From the user space point of view, it's the same, a controller device
> > > that is not in usable state. I don't see a general problem with that.
> > > Seeing a controller appear in connecting state might actually be useful
> > > for user space.
> > > 
> > > The subsysnqn attribute is correct for controllers that were once
> > > connected and lost connectivity. My patch was only intended to fix the
> > > "just created and never connected" case, where it is not, nothing more.
> > 
> > I think Christoph is suggesting just wait until we have all the
> > properties the driver exports rather than specifically exporting only
> > during a 'live' state. Something like this (untested) should do the
> > trick:
> > 
> > ---
> > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > index 1a3cdc6b1036..4c0258d7eb91 100644
> > --- a/drivers/nvme/host/core.c
> > +++ b/drivers/nvme/host/core.c
> > @@ -3069,6 +3069,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
> >   		if (ret)
> >   			goto out_free;
> > +		ret = cdev_device_add(&ctrl->cdev, ctrl->device);
> > +		if (ret)
> > +			goto out_free;
> 
> This happens on every reset, we should probably just do this once?

Right, and while the function is invoked on each reset, the above code
sequence happens only once. You can't immediately tell from looking at
the diff, but it's in the 'if (!ctrl->identified)' code block.


From lengchao at huawei.com  Thu Feb  4 00:56:34 2021
From: lengchao at huawei.com (Chao Leng)
Date: Thu, 4 Feb 2021 13:56:34 +0800
Subject: [PATCH v5 0/3] avoid double request completion and IO error
In-Reply-To: <20210203161455.GB4116@lst.de>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210203161455.GB4116@lst.de>
Message-ID: <7d39e0a4-3765-85b8-aab5-b0ded93f8bec@huawei.com>

This looks good to me.
Thank you very much.

On 2021/2/4 0:14, Christoph Hellwig wrote:
> So I think this is conceptually fine, but I still find the API a little
> arcane.  What do you think about the following incremental patch?
> If that looks good and tests good for you I can apply the series with
> the modifications:
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 0befaad788a094..02579f4f776c7d 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -355,6 +355,21 @@ void nvme_complete_rq(struct request *req)
>   }
>   EXPORT_SYMBOL_GPL(nvme_complete_rq);
>   
> +/*
> + * Called to unwind from ->queue_rq on a failed command submission so that the
> + * multipathing code gets called to potentially failover to another path.
> + * The caller needs to unwind all transport specific resource allocations and
> + * must return propagate the return value.
> + */
> +blk_status_t nvme_host_path_error(struct request *req)
> +{
> +	nvme_req(req)->status = NVME_SC_HOST_PATH_ERROR;
> +	blk_mq_set_request_complete(req);
> +	nvme_complete_rq(req);
> +	return BLK_STS_OK;
> +}
> +EXPORT_SYMBOL_GPL(nvme_host_path_error);
> +
>   bool nvme_cancel_request(struct request *req, void *data, bool reserved)
>   {
>   	dev_dbg_ratelimited(((struct nvme_ctrl *) data)->device,
> diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
> index cedf9b31898673..5dfd806fc2d28c 100644
> --- a/drivers/nvme/host/fabrics.c
> +++ b/drivers/nvme/host/fabrics.c
> @@ -552,11 +552,7 @@ blk_status_t nvmf_fail_nonready_command(struct nvme_ctrl *ctrl,
>   	    !test_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags) &&
>   	    !blk_noretry_request(rq) && !(rq->cmd_flags & REQ_NVME_MPATH))
>   		return BLK_STS_RESOURCE;
> -
> -	nvme_req(rq)->status = NVME_SC_HOST_PATH_ERROR;
> -	blk_mq_set_request_complete(rq);
> -	nvme_complete_rq(rq);
> -	return BLK_STS_OK;
> +	return nvme_host_path_error(rq);
>   }
>   EXPORT_SYMBOL_GPL(nvmf_fail_nonready_command);
>   
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index a72f0718109100..5819f038104149 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -575,6 +575,7 @@ static inline bool nvme_is_aen_req(u16 qid, __u16 command_id)
>   }
>   
>   void nvme_complete_rq(struct request *req);
> +blk_status_t nvme_host_path_error(struct request *req);
>   bool nvme_cancel_request(struct request *req, void *data, bool reserved);
>   void nvme_cancel_tagset(struct nvme_ctrl *ctrl);
>   void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl);
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 6993efb27b39f0..f51af5e4970a2b 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -2091,16 +2091,6 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
>   	err = nvme_rdma_post_send(queue, sqe, req->sge, req->num_sge,
>   			req->mr ? &req->reg_wr.wr : NULL);
>   	if (unlikely(err)) {
> -		if (err == -EIO) {
> -			/*
> -			 * Fail the reqest so upper layer can failover I/O
> -			 * if another path is available
> -			 */
> -			req->status = NVME_SC_HOST_PATH_ERROR;
> -			blk_mq_set_request_complete(rq);
> -			nvme_rdma_complete_rq(rq);
> -			return BLK_STS_OK;
> -		}
>   		goto err_unmap;
>   	}
>   
> @@ -2109,7 +2099,9 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
>   err_unmap:
>   	nvme_rdma_unmap_data(queue, rq);
>   err:
> -	if (err == -ENOMEM || err == -EAGAIN)
> +	if (err == -EIO)
> +		ret = nvme_host_path_error(rq);
> +	else if (err == -ENOMEM || err == -EAGAIN)
>   		ret = BLK_STS_RESOURCE;
>   	else
>   		ret = BLK_STS_IOERR;
> .
> 


From lkp at intel.com  Thu Feb  4 01:37:28 2021
From: lkp at intel.com (kernel test robot)
Date: Thu, 04 Feb 2021 14:37:28 +0800
Subject: [linux-nvme:nvme-5.11] BUILD SUCCESS
 cb8563f5c735a042ea2dd7df1ad55ae06d63ffeb
Message-ID: <601b9628.0pSiZj4XZgWgYYVU%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.11
branch HEAD: cb8563f5c735a042ea2dd7df1ad55ae06d63ffeb  nvmet-tcp: fix out-of-bounds access when receiving multiple h2cdata PDUs

elapsed time: 727m

configs tested: 151
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
s390                       zfcpdump_defconfig
mips                           ip28_defconfig
sh                           se7724_defconfig
c6x                                 defconfig
xtensa                              defconfig
arm                      jornada720_defconfig
mips                           ip27_defconfig
powerpc                     powernv_defconfig
arm                        mvebu_v7_defconfig
ia64                         bigsur_defconfig
mips                            gpr_defconfig
alpha                               defconfig
arm                         cm_x300_defconfig
sh                          urquell_defconfig
arm                        spear6xx_defconfig
sh                ecovec24-romimage_defconfig
c6x                        evmc6472_defconfig
powerpc                       ebony_defconfig
m68k                             allmodconfig
riscv                             allnoconfig
arm                       multi_v4t_defconfig
sh                              ul2_defconfig
powerpc                  mpc866_ads_defconfig
sh                           se7751_defconfig
sh                            migor_defconfig
sh                          landisk_defconfig
arm                       aspeed_g5_defconfig
mips                           xway_defconfig
mips                          ath79_defconfig
arm                           viper_defconfig
arc                      axs103_smp_defconfig
arc                            hsdk_defconfig
arc                        nsim_700_defconfig
powerpc                      ppc6xx_defconfig
powerpc                     tqm8548_defconfig
powerpc                 mpc837x_rdb_defconfig
powerpc                 mpc836x_rdk_defconfig
h8300                     edosk2674_defconfig
sh                          rsk7201_defconfig
xtensa                  nommu_kc705_defconfig
arm                             pxa_defconfig
arm                          moxart_defconfig
powerpc                      katmai_defconfig
powerpc                         wii_defconfig
powerpc                 mpc834x_mds_defconfig
powerpc                          allyesconfig
powerpc                    socrates_defconfig
sh                          sdk7780_defconfig
sh                         microdev_defconfig
arm                          pxa168_defconfig
arc                          axs103_defconfig
arm                            zeus_defconfig
arm                         at91_dt_defconfig
powerpc                      pmac32_defconfig
s390                          debug_defconfig
nds32                             allnoconfig
mips                         bigsur_defconfig
c6x                              allyesconfig
arm                          lpd270_defconfig
powerpc                         ps3_defconfig
arc                    vdk_hs38_smp_defconfig
arm                            xcep_defconfig
arm                        multi_v5_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                               tinyconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a001-20210202
i386                 randconfig-a005-20210202
i386                 randconfig-a003-20210202
i386                 randconfig-a006-20210202
i386                 randconfig-a002-20210202
i386                 randconfig-a004-20210202
x86_64               randconfig-a013-20210202
x86_64               randconfig-a014-20210202
x86_64               randconfig-a015-20210202
x86_64               randconfig-a016-20210202
x86_64               randconfig-a011-20210202
x86_64               randconfig-a012-20210202
i386                 randconfig-a013-20210202
i386                 randconfig-a016-20210202
i386                 randconfig-a014-20210202
i386                 randconfig-a012-20210202
i386                 randconfig-a015-20210202
i386                 randconfig-a011-20210202
i386                 randconfig-a013-20210203
i386                 randconfig-a016-20210203
i386                 randconfig-a014-20210203
i386                 randconfig-a012-20210203
i386                 randconfig-a015-20210203
i386                 randconfig-a011-20210203
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a006-20210204
x86_64               randconfig-a001-20210204
x86_64               randconfig-a005-20210204
x86_64               randconfig-a002-20210204
x86_64               randconfig-a004-20210204
x86_64               randconfig-a003-20210204
x86_64               randconfig-a006-20210202
x86_64               randconfig-a001-20210202
x86_64               randconfig-a005-20210202
x86_64               randconfig-a002-20210202
x86_64               randconfig-a004-20210202
x86_64               randconfig-a003-20210202
x86_64               randconfig-a013-20210203
x86_64               randconfig-a014-20210203
x86_64               randconfig-a015-20210203
x86_64               randconfig-a016-20210203
x86_64               randconfig-a011-20210203
x86_64               randconfig-a012-20210203

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From hare at suse.de  Thu Feb  4 01:58:47 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 07:58:47 +0100
Subject: [PATCH 05/35] monitor: add uevent filters
In-Reply-To: <20210126203324.23610-6-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-6-mwilck@suse.com>
Message-ID: <fa6ae95c-ce41-4af9-5c31-ba8e5fdfa956@suse.de>

On 1/26/21 9:32 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   monitor.c | 4 ++++
>   1 file changed, 4 insertions(+)
> 
> diff --git a/monitor.c b/monitor.c
> index 63e26bc..a6c8905 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -58,6 +58,10 @@ static int create_udev_monitor(struct udev_monitor **pmon)
>   	if (!mon)
>   		return errno ? -errno : -ENOMEM;
>   
> +	/* Add match for NVMe controller devices */
> +	ret = udev_monitor_filter_add_match_subsystem_devtype(mon, "nvme", NULL);
> +	/* Add match for fc_udev_device */
> +	ret = udev_monitor_filter_add_match_subsystem_devtype(mon, "fc", NULL);
>   	/*
>   	 * This fails in unpriviliged mode. Use the same value as udevd.
>   	 * We may able to decrease this buffer size later.
> 
I guess you could merge all these five patches together; no need to keep 
them separate.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:01:00 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:01:00 +0100
Subject: [PATCH 06/35] monitor: Create a log() macro.
In-Reply-To: <20210126203324.23610-7-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-7-mwilck@suse.com>
Message-ID: <580ae616-0b40-122a-670b-57558f0ece58@suse.de>

On 1/26/21 9:32 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> As this is a long running program, we need to make the log output
> configurable. First step: replace fprintf() by log(). The log level
> and printing of time stamps can be configured at run time using
> global variables. These will live in fabrics.c.
> 
> Allow toggling function name printing at build time.
> Printing the function name is useful for development, but perhaps
> not desired for production.
> Put '#define LOG_FUNCNAME' before '#include "log.h"' to switch it on.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   log.h | 44 ++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 44 insertions(+)
>   create mode 100644 log.h
> 
> diff --git a/log.h b/log.h
> new file mode 100644
> index 0000000..2017731
> --- /dev/null
> +++ b/log.h
> @@ -0,0 +1,44 @@
> +#ifndef _LOG_H
> +#define _LOG_H
> +
> +#ifndef MAX_LOGLEVEL
> +#  define MAX_LOGLEVEL LOG_DEBUG
> +#endif
> +#ifndef DEFAULT_LOGLEVEL
> +#  define DEFAULT_LOGLEVEL LOG_NOTICE
> +#endif
> +
> +#ifdef LOG_FUNCNAME
> +#define _func_fmt "%s: "
> +#define _func_arg __func__
> +#else
> +#define _func_fmt "%s"
> +#define _func_arg ""
> +#endif
> +
> +extern int log_level;
> +extern bool log_timestamp;
> +#define _TIME_FMT "[%ld.%06ld] "
> +#define log(lvl, format, ...) \
> +	do {								\
> +		int __lvl = (lvl);					\
> +									\
> +		if (__lvl <= MAX_LOGLEVEL && __lvl <= log_level) {	\
> +			if (log_timestamp) {				\
> +				struct timespec __ts;			\
> +									\
> +				clock_gettime(CLOCK_MONOTONIC, &__ts);	\
> +				fprintf(stderr,				\
> +					_TIME_FMT _func_fmt format,	\
> +					__ts.tv_sec, __ts.tv_nsec / 1000,\
> +					_func_arg,			\
> +					##__VA_ARGS__);			\
> +			} else {					\
> +				fprintf(stderr, _func_fmt format,	\
> +					_func_arg,			\
> +					##__VA_ARGS__);			\
> +			};						\
> +		}							\
> +	} while (0)
> +
> +#endif /* _LOG_H */
> 
Urgh. Long macros are always horrible.
Can't you convert it into a function?

It might also be an idea to move this as the first function, as it's
arguably an extension to existing functionality, and not directly 
related to the monitor.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:02:37 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:02:37 +0100
Subject: [PATCH 07/35] fabrics: use log() macro
In-Reply-To: <20210126203324.23610-8-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-8-mwilck@suse.com>
Message-ID: <60ddb80d-8a40-17e9-f601-4c19663825b9@suse.de>

On 1/26/21 9:32 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> Fabrics functionality will be used by the monitor code, we
> need consistent logging. This patch causes no functional change
> for the fabrics module.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   fabrics.c | 83 ++++++++++++++++++++++++++++++-------------------------
>   1 file changed, 45 insertions(+), 38 deletions(-)
> 
Reviewed-by: Hannes Reinecke <hare at suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:04:52 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:04:52 +0100
Subject: [PATCH 08/35] monitor: add command line options to control logging
In-Reply-To: <20210126203324.23610-9-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-9-mwilck@suse.com>
Message-ID: <37259dc3-16f3-ef0c-fafa-d6a4749c5126@suse.de>

On 1/26/21 9:32 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> Standard syslog loglevels are used.
> Options:
> 
>   -S/--silent   to suppress LOG_NOTICE
>   -v/--verbose  to enable LOG_INFO (overrides -S)
>   -D/--debug    to enable LOG_DEBUG (overrides -S, -v)
>   -C/--clockstamps to enable time stamps on log messages.
> 
> I tried to use short options that don't conflict with options
> from nvme connect-all, because many of those options might be
> useful for the monitor, too.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   monitor.c | 47 +++++++++++++++++++++++++++++++++++++++++------
>   1 file changed, 41 insertions(+), 6 deletions(-)
> ... and if you had moved the logging functionality prior to the monitor 
code, this patch could be merged with the main monitor functionality, too.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:05:51 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:05:51 +0100
Subject: [PATCH 09/35] nvme_get_ctrl_attr(): constify "path" argument
In-Reply-To: <20210126203324.23610-10-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-10-mwilck@suse.com>
Message-ID: <3240d983-2b54-b196-a801-b1c54059e486@suse.de>

On 1/26/21 9:32 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   nvme-topology.c | 2 +-
>   nvme.h          | 2 +-
>   2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/nvme-topology.c b/nvme-topology.c
> index 71371c5..f62446b 100644
> --- a/nvme-topology.c
> +++ b/nvme-topology.c
> @@ -45,7 +45,7 @@ close_fd:
>   	return subsysnqn;
>   }
>   
> -char *nvme_get_ctrl_attr(char *path, const char *attr)
> +char *nvme_get_ctrl_attr(const char *path, const char *attr)
>   {
>   	char *attrpath, *value;
>   	ssize_t ret;
> diff --git a/nvme.h b/nvme.h
> index 3fb1060..382e625 100644
> --- a/nvme.h
> +++ b/nvme.h
> @@ -109,7 +109,7 @@ int scan_subsystems(struct nvme_topology *t, const char *subsysnqn,
>   		    __u32 ns_instance, char *dev_dir);
>   void free_topology(struct nvme_topology *t);
>   char *get_nvme_subsnqn(char *path);
> -char *nvme_get_ctrl_attr(char *path, const char *attr);
> +char *nvme_get_ctrl_attr(const char *path, const char *attr);
>   
>   void *nvme_alloc(size_t len, bool *huge);
>   void nvme_free(void *p, bool huge);
> 
Could be moved prior to the monitor code as it's unrelated to that.

Other than that:

Reviewed-by: Hannes Reinecke <hare at suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:09:21 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:09:21 +0100
Subject: [PATCH 10/35] fabrics: export do_discover(), build_options() and
 config
In-Reply-To: <20210126203324.23610-11-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-11-mwilck@suse.com>
Message-ID: <5572e869-cd25-15ca-e95b-b4304e19d83b@suse.de>

On 1/26/21 9:32 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> These functions will be called by the monitor code.
> Also export BUF_SIZE as the size of argstr.
> 
> Being able to access struct config and the "cfg" variable
> from fabrics.c is essential for the monitor to leverage the existing,
> well tested code as much as possible.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   fabrics.c | 33 ++++-----------------------------
>   fabrics.h | 34 ++++++++++++++++++++++++++++++++++
>   2 files changed, 38 insertions(+), 29 deletions(-)
> 
> diff --git a/fabrics.c b/fabrics.c
> index 555b6b4..c1a4bb5 100644
> --- a/fabrics.c
> +++ b/fabrics.c
> @@ -69,31 +69,7 @@ const char *conarg_traddr = "traddr";
>   const char *conarg_trsvcid = "trsvcid";
>   const char *conarg_host_traddr = "host_traddr";
>   
> -static struct config {
> -	char *nqn;
> -	char *transport;
> -	char *traddr;
> -	char *trsvcid;
> -	char *host_traddr;
> -	char *hostnqn;
> -	char *hostid;
> -	int  nr_io_queues;
> -	int  nr_write_queues;
> -	int  nr_poll_queues;
> -	int  queue_size;
> -	int  keep_alive_tmo;
> -	int  reconnect_delay;
> -	int  ctrl_loss_tmo;
> -	int  tos;
> -	char *raw;
> -	char *device;
> -	int  duplicate_connect;
> -	int  disable_sqflow;
> -	int  hdr_digest;
> -	int  data_digest;
> -	bool persistent;
> -	bool matching_only;
> -} cfg = { .ctrl_loss_tmo = NVMF_DEF_CTRL_LOSS_TMO };
> +struct config cfg = { .ctrl_loss_tmo = NVMF_DEF_CTRL_LOSS_TMO };
>   
>   struct connect_args {
>   	char *subsysnqn;
> @@ -107,7 +83,6 @@ struct connect_args {
>   
>   struct connect_args *tracked_ctrls;
>   
> -#define BUF_SIZE		4096
>   #define PATH_NVME_FABRICS	"/dev/nvme-fabrics"
>   #define PATH_NVMF_DISC		"/etc/nvme/discovery.conf"
>   #define PATH_NVMF_HOSTNQN	"/etc/nvme/hostnqn"
> @@ -226,7 +201,7 @@ static const char *cms_str(__u8 cm)
>   	return arg_str(cms, ARRAY_SIZE(cms), cm);
>   }
>   
> -static int do_discover(char *argstr, bool connect);
> +int do_discover(char *argstr, bool connect);
>   
>   /*
>    * parse strings with connect arguments to find a particular field.
> @@ -855,7 +830,7 @@ add_argument(char **argstr, int *max_len, char *arg_str, char *arg)
>   	return 0;
>   }
>   
> -static int build_options(char *argstr, int max_len, bool discover)
> +int build_options(char *argstr, int max_len, bool discover)
>   {
>   	int len;
>   
> @@ -1264,7 +1239,7 @@ static void nvmf_get_host_identifiers(int ctrl_instance)
>   	cfg.hostid = nvme_get_ctrl_attr(path, "hostid");
>   }
>   
> -static int do_discover(char *argstr, bool connect)
> +int do_discover(char *argstr, bool connect)
>   {
>   	struct nvmf_disc_rsp_page_hdr *log = NULL;
>   	char *dev_name;
> diff --git a/fabrics.h b/fabrics.h
> index f5b8eaf..ce965a3 100644
> --- a/fabrics.h
> +++ b/fabrics.h
> @@ -10,4 +10,38 @@ extern int fabrics_connect(const char *desc, int argc, char **argv);
>   extern int fabrics_disconnect(const char *desc, int argc, char **argv);
>   extern int fabrics_disconnect_all(const char *desc, int argc, char **argv);
>   
> +/* Symbols used by monitor.c */
> +
> +struct config {
> +	char *nqn;
> +	char *transport;
> +	char *traddr;
> +	char *trsvcid;
> +	char *host_traddr;
> +	char *hostnqn;
> +	char *hostid;
> +	int  nr_io_queues;
> +	int  nr_write_queues;
> +	int  nr_poll_queues;
> +	int  queue_size;
> +	int  keep_alive_tmo;
> +	int  reconnect_delay;
> +	int  ctrl_loss_tmo;
> +	int  tos;
> +	char *raw;
> +	char *device;
> +	int  duplicate_connect;
> +	int  disable_sqflow;
> +	int  hdr_digest;
> +	int  data_digest;
> +	bool persistent;
> +	bool matching_only;
> +};
> +extern struct config cfg;
> +
> +#define BUF_SIZE 4096
> +
> +int build_options(char *argstr, int max_len, bool discover);
> +int do_discover(char *argstr, bool connect);
> +
>   #endif
> 

Please name is 'struct config fabrics_cfg'. 'cfg' is used as a generic 
name throughout the code, and using the same name in a header might lead 
to an accidental name clash.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:13:08 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:13:08 +0100
Subject: [PATCH 11/35] monitor: add option -A / --autoconnect
In-Reply-To: <20210126203324.23610-12-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-12-mwilck@suse.com>
Message-ID: <819efd43-0216-7836-9367-ca5bca131efa@suse.de>

On 1/26/21 9:33 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> With this option, the monitor will try to connect to newly discovered
> controllers.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   monitor.c | 5 +++++
>   1 file changed, 5 insertions(+)
> 
> diff --git a/monitor.c b/monitor.c
> index a7afa1c..ecf3be2 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -31,6 +31,10 @@
>   #define LOG_FUNCNAME 1
>   #include "log.h"
>   
> +static struct monitor_config {
> +	bool autoconnect;
> +} mon_cfg;
> +
>   static struct udev *udev;
>   
>   static void close_ptr(int *p)
> @@ -177,6 +181,7 @@ static int monitor_parse_opts(const char *desc, int argc, char **argv)
>   	bool debug = false;
>   	int ret;
>   	OPT_ARGS(opts) = {
> +		OPT_FLAG("autoconnect",    'A', &mon_cfg.autoconnect, "automatically connect newly discovered controllers"),
>   		OPT_FLAG("silent",         'S', &quiet,               "log level: silent"),
>   		OPT_FLAG("verbose",        'v', &verbose,             "log level: verbose"),
>   		OPT_FLAG("debug",          'D', &debug,               "log level: debug"),
> 
This patch is slightly confusing.

It just adds an option, but there is not code evaluating this option. So 
where does it hook in?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:14:16 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:14:16 +0100
Subject: [PATCH 12/35] monitor: add helpers for __attribute__((cleanup))
In-Reply-To: <20210126203324.23610-13-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-13-mwilck@suse.com>
Message-ID: <66d73520-5022-9d6b-a27e-4da02c42ea94@suse.de>

On 1/26/21 9:33 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> __attribute__((cleanup)) is very helpful but ugly. Try to avoid
> defining lots of cleanup functions with these macros.
> 
> Usage: to declare an auto-cleanup variable of type (some_type *),
> write
> 
> CLEANUP_FUNC(some_type)
> 
> void some_func(void)
> {
>          CLEANUP(some_type, varname) = NULL;
> 	...
> }
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   common.h | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
> 
> diff --git a/common.h b/common.h
> index 1c214a4..9386ec5 100644
> --- a/common.h
> +++ b/common.h
> @@ -12,4 +12,16 @@
>   #define __stringify_1(x...) #x
>   #define __stringify(x...)  __stringify_1(x)
>   
> +#define CLEANUP_FUNC(type) \
> +static void __cleanup_ ## type ##_p(type ** __p) \
> +{						 \
> +	if (*__p) {				 \
> +		free(*__p);			 \
> +		*__p = NULL;			 \
> +	}					 \
> +}
> +
> +#define CLEANUP(__t, __v) \
> +	__t *__v __attribute__((cleanup(__cleanup_ ## __t ## _p)))
> +
>   #endif
> 
And this is used where?

Can't you merge it with the patch usig the __cleanup functionality?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:16:25 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:16:25 +0100
Subject: [PATCH 13/35] monitor: disable nvmf-autoconnect udev rules in
 autoconnect mode
In-Reply-To: <20210126203324.23610-14-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-14-mwilck@suse.com>
Message-ID: <ff1b9621-c01b-7884-1e8f-44a07ac845c1@suse.de>

On 1/26/21 9:33 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> If autoconnect is enabled, disable the respective udev rules
> by symlinking /run/udev/rules.d to /dev/null, in order to avoid
> the connections being set up by the monitor and the udev workers
> at the same time. This is probably the preferred mode of operation
> for the monitor.
> 
> Users can override this by copying 70-nvmf-autoconnect.rules
> from /usr/lib/udev/rules.d to /etc/udev/rules.d (/etc/udev/rules.d
> takes precedence over /run/udev/rules.d).
> 
> If the symlink can't be created for some reason, autoconnect will
> be disabled. There is  only one exception: If
> /run/udev/rules.d/70-nvmf-autoconnect.rules already points to
> /dev/null at startup, autoconnect can be left on, but the symlink
> isn't removed on exit.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   monitor.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
>   1 file changed, 73 insertions(+), 2 deletions(-)
> 
> diff --git a/monitor.c b/monitor.c
> index ecf3be2..2a906db 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -17,14 +17,18 @@
>   
>   #include <stddef.h>
>   #include <stdio.h>
> +#include <stdlib.h>
>   #include <unistd.h>
>   #include <errno.h>
>   #include <libudev.h>
>   #include <signal.h>
>   #include <time.h>
> +#include <limits.h>
>   #include <syslog.h>
> +#include <sys/stat.h>
>   #include <sys/epoll.h>
>   
> +#include "common.h"
>   #include "nvme-status.h"
>   #include "util/argconfig.h"
>   #include "monitor.h"
> @@ -33,6 +37,7 @@
>   
>   static struct monitor_config {
>   	bool autoconnect;
> +	bool skip_udev_on_exit;
>   } mon_cfg;
>   
>   static struct udev *udev;
> @@ -45,6 +50,8 @@ static void close_ptr(int *p)
>   	}
>   }
>   
> +CLEANUP_FUNC(char)
> +
>   static void cleanup_monitor(struct udev_monitor **pmon)
>   {
>   	if (*pmon) {
> @@ -174,12 +181,64 @@ static int monitor_main_loop(struct udev_monitor *monitor)
>   	return ret;
>   }
>   
> +static const char autoconnect_rules[] = "/run/udev/rules.d/70-nvmf-autoconnect.rules";
> +
> +static int monitor_disable_udev_rules(void)
> +{
> +	CLEANUP(char, path) = strdup(autoconnect_rules);
> +	char *s1, *s2;
> +	int rc;
> +
> +	if (!path)
> +		return -ENOMEM;
> +
> +	s2 = strrchr(path, '/');
> +	for (s1 = s2 - 1; s1 > path && *s1 != '/'; s1--);
> +
> +	*s2 = *s1 = '\0';
> +	rc = mkdir(path, 0755);
> +	if (rc == 0 || errno == EEXIST) {
> +		*s1 = '/';
> +		rc = mkdir(path, 0755);
> +		if (rc == 0 || errno == EEXIST) {
> +			*s2 = '/';
> +			rc = symlink("/dev/null", path);
> +		}
> +	}
> +	if (rc) {
> +		if (errno == EEXIST) {
> +			char target[PATH_MAX];
> +
> +			if (readlink(path, target, sizeof(target)) != -1 &&
> +			    !strcmp(target, "/dev/null")) {
> +				log(LOG_INFO,
> +				    "symlink %s -> /dev/null exists already\n",
> +				    autoconnect_rules);
> +				return 1;
> +			}
> +		}
> +		log(LOG_ERR, "error creating %s: %m\n", autoconnect_rules);
> +	} else
> +		log(LOG_INFO, "created %s\n", autoconnect_rules);
> +
> +	return rc ? (errno ? -errno : -EIO) : 0;
> +}
> +
> +static void monitor_enable_udev_rules(void)
> +{
> +	if (unlink(autoconnect_rules) == -1 && errno != ENOENT)
> +		log(LOG_ERR, "error removing %s: %m\n", autoconnect_rules);
> +	else
> +		log(LOG_INFO, "removed %s\n", autoconnect_rules);
> +}
> +
>   static int monitor_parse_opts(const char *desc, int argc, char **argv)
>   {
>   	bool quiet = false;
>   	bool verbose = false;
>   	bool debug = false;
> -	int ret;
> +	int ret = 0;
> +
>   	OPT_ARGS(opts) = {
>   		OPT_FLAG("autoconnect",    'A', &mon_cfg.autoconnect, "automatically connect newly discovered controllers"),
>   		OPT_FLAG("silent",         'S', &quiet,               "log level: silent"),
> @@ -198,7 +257,17 @@ static int monitor_parse_opts(const char *desc, int argc, char **argv)
>   		log_level = LOG_INFO;
>   	if (debug)
>   		log_level = LOG_DEBUG;
> -
> +	if (mon_cfg.autoconnect) {
> +		ret = monitor_disable_udev_rules();
> +		if (ret < 0) {
> +			mon_cfg.autoconnect = false;
> +			log(LOG_WARNING, "autoconnect disabled\n");
> +			ret = 0;
> +		} else if (ret > 0) {
> +			mon_cfg.skip_udev_on_exit = true;
> +			ret = 0;
> +		}
> +	}
>   	return ret;
>   }
>   
> @@ -221,6 +290,8 @@ int aen_monitor(const char *desc, int argc, char **argv)
>   		udev_monitor_unref(monitor);
>   	}
>   	udev = udev_unref(udev);
> +	if (mon_cfg.autoconnect && !mon_cfg.skip_udev_on_exit)
> +		monitor_enable_udev_rules();
>   out:
>   	return nvme_status_to_errno(ret, true);
>   }
> 
What on earth ...

No way.
We should _not_ reference individual files or mess with udev rules from 
a program. If a file needs to be modified, modify it.
But do not try to modify a file from a program context, unless the 
program itself has created it.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:20:11 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:20:11 +0100
Subject: [PATCH 19/35] fabrics: use "const char *" in struct config
In-Reply-To: <20210126203324.23610-20-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-20-mwilck@suse.com>
Message-ID: <b875397b-9ecd-3630-6e97-f4852fc60fba@suse.de>

On 1/26/21 9:33 PM, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> This is easily done, and allows passing in some const char* pointers
> that would otherwise need to be strdup()d first.
> 
> constifying cfg->device requires changes in the ctrl_instance()
> function, because basename() can't be used.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>   fabrics.c | 23 ++++++++++++++---------
>   fabrics.h | 17 +++++++++--------
>   2 files changed, 23 insertions(+), 17 deletions(-)
> 
> diff --git a/fabrics.c b/fabrics.c
> index c1a4bb5..e3d2a3a 100644
> --- a/fabrics.c
> +++ b/fabrics.c
> @@ -252,17 +252,22 @@ empty_field:
>   	return strdup("\0");
>   }
>   
> -static int ctrl_instance(char *device)
> +int ctrl_instance(const char *device)
>   {
>   	char d[64];
> +	const char *p;
>   	int ret, instance;
>   
> -	device = basename(device);
> -	ret = sscanf(device, "nvme%d", &instance);
> +	p = strrchr(device, '/');
> +	if (p == NULL)
> +		p = device;
> +	else
> +		p++;
> +	ret = sscanf(p, "nvme%d", &instance);
>   	if (ret <= 0)
>   		return -EINVAL;
>   	if (snprintf(d, sizeof(d), "nvme%d", instance) <= 0 ||
> -	    strcmp(device, d))
> +	    strcmp(p, d))
>   		return -EINVAL;
>   	return instance;
>   }
> @@ -273,7 +278,7 @@ static int ctrl_instance(char *device)
>    * given.
>    * Return true/false based on whether it matches
>    */
> -static bool ctrl_matches_connectargs(char *name, struct connect_args *args)
> +static bool ctrl_matches_connectargs(const char *name, struct connect_args *args)
>   {
>   	struct connect_args cargs;
>   	bool found = false;
> @@ -815,7 +820,7 @@ add_int_argument(char **argstr, int *max_len, char *arg_str, int arg,
>   }
>   
>   static int
> -add_argument(char **argstr, int *max_len, char *arg_str, char *arg)
> +add_argument(char **argstr, int *max_len, char *arg_str, const char *arg)
>   {
>   	int len;
>   
> @@ -1541,7 +1546,7 @@ static int scan_sys_nvme_filter(const struct dirent *d)
>   /*
>    * Returns 1 if disconnect occurred, 0 otherwise.
>    */
> -static int disconnect_subsys(char *nqn, char *ctrl)
> +static int disconnect_subsys(const char *nqn, char *ctrl)
>   {
>   	char *sysfs_nqn_path = NULL, *sysfs_del_path = NULL;
>   	char subsysnqn[NVMF_NQN_SIZE] = {};
> @@ -1579,7 +1584,7 @@ static int disconnect_subsys(char *nqn, char *ctrl)
>   /*
>    * Returns the number of controllers successfully disconnected.
>    */
> -static int disconnect_by_nqn(char *nqn)
> +static int disconnect_by_nqn(const char *nqn)
>   {
>   	struct dirent **devices = NULL;
>   	int i, n, ret = 0;
> @@ -1601,7 +1606,7 @@ static int disconnect_by_nqn(char *nqn)
>   	return ret;
>   }
>   
> -static int disconnect_by_device(char *device)
> +static int disconnect_by_device(const char *device)
>   {
>   	int instance;
>   
> diff --git a/fabrics.h b/fabrics.h
> index ce965a3..1dfbd67 100644
> --- a/fabrics.h
> +++ b/fabrics.h
> @@ -13,13 +13,13 @@ extern int fabrics_disconnect_all(const char *desc, int argc, char **argv);
>   /* Symbols used by monitor.c */
>   
>   struct config {
> -	char *nqn;
> -	char *transport;
> -	char *traddr;
> -	char *trsvcid;
> -	char *host_traddr;
> -	char *hostnqn;
> -	char *hostid;
> +	const char *nqn;
> +	const char *transport;
> +	const char *traddr;
> +	const char *trsvcid;
> +	const char *host_traddr;
> +	const char *hostnqn;
> +	const char *hostid;
>   	int  nr_io_queues;
>   	int  nr_write_queues;
>   	int  nr_poll_queues;
> @@ -29,7 +29,7 @@ struct config {
>   	int  ctrl_loss_tmo;
>   	int  tos;
>   	char *raw;
> -	char *device;
> +	const char *device;
>   	int  duplicate_connect;
>   	int  disable_sqflow;
>   	int  hdr_digest;
> @@ -43,5 +43,6 @@ extern struct config cfg;
>   
>   int build_options(char *argstr, int max_len, bool discover);
>   int do_discover(char *argstr, bool connect);
> +int ctrl_instance(const char *device);
>   
>   #endif
> 
This really is independent on the monitor code, and should be moved to 
the beginning of the patch series.

Other than that:

Reviewed-by: Hannes Reinecke <hare at suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb  4 02:34:22 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:34:22 +0100
Subject: [PATCH 35/35] monitor: add option --keep/-K
In-Reply-To: <c00de29ef02bf39ae231a2816f03c8e920ebdc29.camel@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-36-mwilck@suse.com>
 <90145500-c2c5-023c-6d69-27db6ee600a7@grimberg.me>
 <12827c5f748bd2eff31450c47bbef1cfc7b73459.camel@suse.com>
 <400d4f5e-a578-e89f-ae45-7ed15d089a8d@grimberg.me>
 <b4b9a52011d17ef16cbc86d81ef3d19f7a517d93.camel@suse.com>
 <c127bde8-7cae-677c-4b8d-44cf757d05eb@grimberg.me>
 <2394e23e80f2bdcf0a95eb39e5eefcb00237d52f.camel@suse.com>
 <8cb1830e-b7f3-48ef-1190-0d917aa200bb@grimberg.me>
 <5623aeee6a57889123b8a3441ee6df7b97f76451.camel@suse.com>
 <552cb65a-8f1b-e414-4de8-f53ff469e8a1@grimberg.me>
 <c00de29ef02bf39ae231a2816f03c8e920ebdc29.camel@suse.com>
Message-ID: <864dd77a-650f-9260-9d9d-02ab83b4a18d@suse.de>

On 1/29/21 10:15 PM, Martin Wilck wrote:
> On Fri, 2021-01-29 at 13:11 -0800, Sagi Grimberg wrote:
>>
>>>>> If it does, it will recreate discovery controllers for every
>>>>> host_traddr/traddr/trsvcid tuple it finds. "--keep" semantics
>>>>> are
>>>>> only
>>>>> necessary for addresses on which no regular (non-discovery)
>>>>> connection
>>>>> exists.
>>>>
>>>> Wait, Maybe I'm missing something here, but are you saying that
>>>> for
>>>> every traddr/trsvcid it finds (both nvm and discovery) it will
>>>> attempt
>>>> to connect a discovery controller?
>>>>
>>>> If so, this is absolutely wrong.
>>>
>>> Currently, it tries to do that on startup, if (and only if) the
>>> --startup option is given. My expectation was that the connection
>>> attempts would simply fail if there was no discovery subsystem to
>>> connect to. Anyway, it's not the default behavior, and can be
>>> dropped
>>> completely if it's so bad that we shouldn't ever attempt to do it.
>>
>> IMO it needs to be dropped. I didn't understand this at first because
>> it never even occurred to me that such an assumption can be even
>> made.
>>
>>> If the service is started early on during boot, and event-based
>>> discovery works (i.e. we also have the mDNS part in place), this
>>> won't
>>> be necessary of course.
>>
>> This isn't necessary regardless. At best the discovery controller
>> endpoints should be obtained from discovery.conf or equivalent.
> 
> Understood.
> 
>> I don't even see how does this help in early boot anyways, how
>> do the existing controllers get connected?
> 
> For FC, it works today. You can start the monitor in the initrd, it
> will receive the fc_udev_device events, and autoconnect (just like
> traditional udev/systemd based discovery). For other transports, it'd
> be more work, as we'd need to do mDNS discovery from an initrd
> environment. I wouldn't say it's impossible though.
> 
mDNS isn't related to the initrd, mDNS is for discovery discovery :-)
For everything _but_ FC we have the problem that we don't know where to 
connect to for the initial discovery (ie we don't have a list of 
potential discovery controllers).
Currently we either have to specify the parameters manually, or add them 
to /etc/nvme/discovery.conf.
Which isn't _that_ dynamic.
mDNS will provide a way of promoting changes in any remote discovery 
controllers to the potential hosts, allowing the host to pick up 
information about potential discovery controllers from there.

So the monitor code would need to interact with mDNS to get the 
information, and should use that to establish discovery connections.

As this information is transient I would not tear down discovery 
controller connections on exit, but rather keep them alive.
That way a controller restart could pick up these existing connections 
and would not need to recreate them (which will be tricky as the monitor 
code doesn't have the information which connections to restart).

The alternative would be a callout to mDNS to get the current topology, 
but that doesn't exist yet.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From sagi at grimberg.me  Thu Feb  4 02:43:41 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 23:43:41 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
 <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>
 <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>


> Hi Sagi,
> 
> I was able to capture a bit more info. Again, because my computer 
> freezes (i.e. no keyboard or mouse response) I could only take a picture 
> of the screen (attached). This does say NULL pointer dereference.
> 
> To answer your questions:
> 
> Q)?Does this happen if you directly connect to a normal nvme controller?
> A) I still need to try this. All I know is that it doesn't happen when 
> the controller returns?OAES[31]=0.
> 
> Q)?What is your target implementation? Is this the nvme target thatis 
> built into Linux?
> A) I work for Dell, and I'm on the same team as Douglas Farley and Erik 
> Smith, whom I believe you've had discussions with. Dell is currently 
> developing a Central Discovery Controller (CDC). That's the target I've 
> been testing with. I need to find different targets I can test with but 
> working from home makes it a bit difficult. I will check with my team 
> tomorrow to see if I can connect to a different target.
> 
> Q)?Can you share your kernel config file?
> Q) Can you run gdb on vmlinux and share what this provides?
> A) I'll get that tomorrow (both questions).

For the record, I compiled Fedora 33 kernel and ran it on my VM
and it doesn't happen. I also modified Linux nvme target to
report the same limits as your target and still it doesn't happen.
--
[  162.006049] nvme nvme0: Failed to read smart log (error 24577)
[  162.006060] nvme nvme0: queue_size 128 > ctrl sqsize 32, clamping down
[  162.006063] nvme nvme0: sqsize 32 > ctrl maxcmd 31, clamping down
[  162.006778] nvme nvme0: new ctrl: NQN 
"nqn.2014-08.org.nvmexpress.discovery", addr 192.168.123.1:8009
[  162.008939] nvme nvme0: Removing ctrl: NQN 
"nqn.2014-08.org.nvmexpress.discovery"
--

So something here seems to be specific to your env.


From sagi at grimberg.me  Thu Feb  4 02:44:18 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 3 Feb 2021 23:44:18 -0800
Subject: [PATCH v2] nvme: avoid "(efault)" from nvme_sysfs_show_subsysnqn()
In-Reply-To: <20210204024220.GE2182779@dhcp-10-100-145-180.wdc.com>
References: <20210202005951.26614-1-mwilck@suse.com>
 <20210202075828.GA12160@lst.de>
 <a7f3114e22a07e60abfd98155df91dea3a83b386.camel@suse.com>
 <20210203165203.GA2182779@dhcp-10-100-145-180.wdc.com>
 <8b1fc5ef-4c6b-4fa3-7cb1-b42ed71e7413@grimberg.me>
 <20210204024220.GE2182779@dhcp-10-100-145-180.wdc.com>
Message-ID: <82855b31-6698-6991-35b0-d0496b3de70d@grimberg.me>


>>> I think Christoph is suggesting just wait until we have all the
>>> properties the driver exports rather than specifically exporting only
>>> during a 'live' state. Something like this (untested) should do the
>>> trick:
>>>
>>> ---
>>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>>> index 1a3cdc6b1036..4c0258d7eb91 100644
>>> --- a/drivers/nvme/host/core.c
>>> +++ b/drivers/nvme/host/core.c
>>> @@ -3069,6 +3069,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
>>>    		if (ret)
>>>    			goto out_free;
>>> +		ret = cdev_device_add(&ctrl->cdev, ctrl->device);
>>> +		if (ret)
>>> +			goto out_free;
>>
>> This happens on every reset, we should probably just do this once?
> 
> Right, and while the function is invoked on each reset, the above code
> sequence happens only once. You can't immediately tell from looking at
> the diff, but it's in the 'if (!ctrl->identified)' code block.

Yea, you're right...


From hare at suse.de  Thu Feb  4 02:52:04 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 4 Feb 2021 08:52:04 +0100
Subject: [PATCH 00/35] RFC: add "nvme monitor" subcommand
In-Reply-To: <26138a83eee9362b1769940f6da86387716ab2d8.camel@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
 <60846bb5-c0df-ac23-260b-b53afd48f661@grimberg.me>
 <d8956cf973867ad7cc6c4d968682ec3db3054aa3.camel@suse.com>
 <d8e35845-31d3-a46d-b464-728d751d0b33@grimberg.me>
 <26138a83eee9362b1769940f6da86387716ab2d8.camel@suse.com>
Message-ID: <1d039cca-ce24-f8c8-1d75-9056747e2ffe@suse.de>

On 1/29/21 9:27 PM, Martin Wilck wrote:
> On Fri, 2021-01-29 at 12:08 -0800, Sagi Grimberg wrote:
>>
>>>>>
>>>>> This method for discovery and autodetection has some advantages
>>>>> over the
>>>>> current udev-rule based approach:
>>>>>
>>>>>  ?? * By using the `--persistent` option, users can easily
>>>>> control
>>>>> whether
>>>>>  ???? persistent discovery controllers for discovered transport
>>>>> addresses should
>>>>>  ???? be created and monitored for AEN events. **nvme monitor**
>>>>> watches known
>>>>>  ???? transport addresses, creates discovery controllers as
>>>>> required,
>>>>> and re-uses
>>>>>  ???? existing ones if possible.
>>>>
>>>> What does that mean?
>>>
>>> In general, if the monitor detects a new host_traddr/traddr/trsvcid
>>> tuple, it runs a discovery on it, and keeps the discovery
>>> controller
>>> open if?--persistent was given. On startup, it scans existing
>>> controllers, and if it finds already existing discovery
>>> controllers,
>>> re-uses them. These will not be shut down when the monitor exits.
>>
>> And if it doesn't run with --persistent it deletes them? even if
>> these
>> weren't created by it?
> 
> No, not those. That's what I wanted to express with "These will not be
> shut down when the monitor exits."
> 
> It's not trivial to determine reliably which controller the monitor
> created and which it didn't. The current code assumes that all
> discovery controllers that didn't exist at startup were created by the
> monitor. We can improve the intelligence of the tool in that area, of
> course. Currently it makes this dumb assumption.
> 
>>
>> And, if none exist where does it get new discovery controller
>> details?
>> Today we use discovery.conf for that.
> 
> As I said, this can be done with a separate "nvme connect-all" call,
> which could be run from an ExecStartPre or from a separate service.
> Or we can integrate the functionality in the monitor. I don't have a
> strong opinion either way. I can adapt to what you guys prefer.
> 
>>
>>> This allows users fine-grained control about what discovery
>>> controllers
>>> to operate persistently. Users who want all discovery controllers
>>> to be
>>> persistent just use --persistent. Others can set up those that they
>>> want to have (manually or with a script), and not use --persistent.
>>>
>>> The background is that hosts may not need every detected discovery
>>> controller to be persistent. In multipath scenarios, you may see
>>> more
>>> discovery subsystems than anything else, and not everyone likes
>>> that.
>>> That's a generic issue and unrelated to the monitor, but running
>>> the
>>> monitor with --persistent creates discovery controllers that would
>>> otherwise not be visible.
>>>
>>> Hope this clarifies it.
>>
>> Well, How do people expect to know if stuff change without a
>> persistent
>> discovery controller? Not sure if people may see more discovery
>> controllers this is a real issue given what they are getting from it.
> 
> You're right. At this very early stage, I wanted to give users the
> freedom of choice, and not force dozens of discovery controller
> connections upon everyone. If there's consensus that the discovery
> connections should always be created, fine with me.
> 
> 
I would make the monitor code to always create persistent discovery 
connections.
(Or, at least, _attempt_ to create persistent discovery connections.)

_if_ the user starts the monitor he will have a reason for it; and that 
reason typically is that he _wants_ the system to react on discovery 
changes.
And that is precisely why persistent discovery controllers exist.

Plus there is a provision in the spec allowing the controller to reject 
persistent connections, so if the controller is worried about increased 
resource consumption he can just disallow persistent controller connections.

Similarly, I would keep the created discovery connections after exit.
Currently the simple reason is that we don't store the information from 
the created discovery connections, so if we tear them down on exit we 
won't be able to recreate them once we restart the monitor.

This situation might change with once we have mDNS integration; and, of 
course, for FC figuring out this information is trivial.

>>> I agree. But it's not easy to fix the issue otherwise. In the
>>> customer
>>> problem where we observed it, I worked around it by adding the udev
>>> seqnum to the "instance name" of the systemd service, thus allowing
>>> several "nvme connect-all" processes to run for the same transport
>>> address simultaneously. But I don't think that would scale well;
>>> the monitor can handle it more cleanly.
>>
>> Still changing the entire thing for a corner case...
> 
> It's just one puzzle piece.
> 
And it's actually not a corner case, but a real issue we're seeing in 
our (or rather, our customers) deployments.
AENs are being received at the same time, causing 'nvme discover' to run 
in parallel with the same information.
A very nice exercise in finding all race conditions in the code.
And completely pointless to boot.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hch at lst.de  Thu Feb  4 03:04:42 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 4 Feb 2021 09:04:42 +0100
Subject: [PATCH v5 0/3] avoid double request completion and IO error
In-Reply-To: <7d39e0a4-3765-85b8-aab5-b0ded93f8bec@huawei.com>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210203161455.GB4116@lst.de>
 <7d39e0a4-3765-85b8-aab5-b0ded93f8bec@huawei.com>
Message-ID: <20210204080442.GA30542@lst.de>

On Thu, Feb 04, 2021 at 01:56:34PM +0800, Chao Leng wrote:
> This looks good to me.
> Thank you very much.

Thanks a lot to you!

Please double check there version here:

http://git.infradead.org/nvme.git/shortlog/refs/heads/nvme-5.12


From Damien.LeMoal at wdc.com  Thu Feb  4 03:47:51 2021
From: Damien.LeMoal at wdc.com (Damien Le Moal)
Date: Thu, 4 Feb 2021 08:47:51 +0000
Subject: [PATCH v4 0/8] block: add zone write granularity limit
References: <20210128044733.503606-1-damien.lemoal@wdc.com>
Message-ID: <BL0PR04MB65147501739ABF4F27B72290E7B39@BL0PR04MB6514.namprd04.prod.outlook.com>

On 2021/01/28 13:50, Damien Le Moal wrote:
> The first patch of this series adds missing documentation of the
> zone_append_max_bytes sysfs attribute.
> 
> The following 3 patches are cleanup and preparatory patches for the
> introduction of the zone write granularity limit. The goal of these
> patches is to have all code setting a device queue zoned model to use
> the helper function blk_queue_set_zoned(). The nvme driver, null_blk
> driver and the partition code are modified to do so.
> 
> The fourth patch in this series introduces the zone write granularity
> queue limit to indicate the alignment constraint for write operations
> into sequential zones of zoned block devices. This limit is always set
> by default to the device logical block size. The following patch
> documents this new limit.
> 
> The last 2 patches introduce the blk_queue_clear_zone_settings()
> function and modify the SCSI sd driver to clear the zone related queue
> limits and resources of a host-aware zoned disk that is changed to a
> regular disk due to the presence of partitions.

Hi Jens,

Any comment on this series ?

Martin,

The scsi bits (patch 5 and 8) may need your ack.

Thanks !

> 
> Changes from v3:
> * Added pathces 2, 3, 4, 7 and 8
> * Addressed Christoph's comments on patch 5
> 
> Changes from v2:
> * Added patch 3 for zonefs
> * Addressed Christoph's comments on patch 1 and added the limit
>   initialization for zoned nullblk
> 
> Changes from v1:
> * Fixed typo in patch 2
> 
> Damien Le Moal (8):
>   block: document zone_append_max_bytes attribute
>   nvme: cleanup zone information initialization
>   nullb: use blk_queue_set_zoned() to setup zoned devices
>   block: use blk_queue_set_zoned in add_partition()
>   block: introduce zone_write_granularity limit
>   zonefs: use zone write granularity as block size
>   block: introduce blk_queue_clear_zone_settings()
>   sd_zbc: clear zone resources for non-zoned case
> 
> Damien Le Moal (8):
>   block: document zone_append_max_bytes attribute
>   nvme: cleanup zone information initialization
>   nullb: use blk_queue_set_zoned() to setup zoned devices
>   block: use blk_queue_set_zoned in add_partition()
>   block: introduce zone_write_granularity limit
>   zonefs: use zone write granularity as block size
>   block: introduce blk_queue_clear_zone_settings()
>   sd_zbc: clear zone resources for non-zoned case
> 
>  Documentation/block/queue-sysfs.rst | 13 +++++++++
>  block/blk-settings.c                | 39 +++++++++++++++++++++++++-
>  block/blk-sysfs.c                   |  8 ++++++
>  block/blk-zoned.c                   | 17 ++++++++++++
>  block/blk.h                         |  2 ++
>  block/partitions/core.c             |  2 +-
>  drivers/block/null_blk/zoned.c      |  8 +++---
>  drivers/nvme/host/core.c            | 11 ++++----
>  drivers/nvme/host/zns.c             | 11 ++------
>  drivers/scsi/sd_zbc.c               | 43 ++++++++++++++++++++++++++---
>  fs/zonefs/super.c                   |  9 +++---
>  include/linux/blkdev.h              | 15 ++++++++++
>  12 files changed, 150 insertions(+), 28 deletions(-)
> 


-- 
Damien Le Moal
Western Digital Research


From mwilck at suse.com  Thu Feb  4 04:14:23 2021
From: mwilck at suse.com (Martin Wilck)
Date: Thu, 04 Feb 2021 10:14:23 +0100
Subject: [PATCH 06/35] monitor: Create a log() macro.
In-Reply-To: <580ae616-0b40-122a-670b-57558f0ece58@suse.de>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-7-mwilck@suse.com>
 <580ae616-0b40-122a-670b-57558f0ece58@suse.de>
Message-ID: <cfa6ab414180922ee23befabe17fe561f68a455f.camel@suse.com>

On Thu, 2021-02-04 at 08:01 +0100, Hannes Reinecke wrote:
> On 1/26/21 9:32 PM, mwilck at suse.com?wrote:
> > +???????????????int __lvl =
> > (lvl);??????????????????????????????????????\
> > +??????????????????????????????????????????????????????????????????
> > ?????\
> > +???????????????if (__lvl <= MAX_LOGLEVEL && __lvl <= log_level)
> > {??????\
> > +???????????????????????if (log_timestamp)
> > {????????????????????????????\
> > +???????????????????????????????struct timespec
> > __ts;???????????????????\
> > +??????????????????????????????????????????????????????????????????
> > ?????\
> > +???????????????????????????????clock_gettime(CLOCK_MONOTONIC,
> > &__ts);??\
> > +???????????????????????????????fprintf(stderr,????????????????????
> > ?????\
> > +???????????????????????????????????????_TIME_FMT _func_fmt
> > format,?????\
> > +???????????????????????????????????????__ts.tv_sec, __ts.tv_nsec /
> > 1000,\
> > +???????????????????????????????????????_func_arg,?????????????????
> > ?????\
> > +???????????????????????????????????????##__VA_ARGS__);????????????
> > ?????\
> > +???????????????????????} else
> > {????????????????????????????????????????\
> > +???????????????????????????????fprintf(stderr, _func_fmt
> > format,???????\
> > +???????????????????????????????????????_func_arg,?????????????????
> > ?????\
> > +???????????????????????????????????????##__VA_ARGS__);????????????
> > ?????\
> > +???????????????????????};?????????????????????????????????????????
> > ?????\
> > +???????????????}??????????????????????????????????????????????????
> > ?????\
> > +???????} while (0)
> > +
> > +#endif /* _LOG_H */
> > 
> Urgh. Long macros are always horrible.
> Can't you convert it into a function?

Sure. It won't work without macros, but I can write smaller ones.

> It might also be an idea to move this as the first function, as it's
> arguably an extension to existing functionality, and not directly 
> related to the monitor.

Ok.

Thanks,
Martin




From mwilck at suse.com  Thu Feb  4 04:18:33 2021
From: mwilck at suse.com (Martin Wilck)
Date: Thu, 04 Feb 2021 10:18:33 +0100
Subject: [PATCH 08/35] monitor: add command line options to control logging
In-Reply-To: <37259dc3-16f3-ef0c-fafa-d6a4749c5126@suse.de>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-9-mwilck@suse.com>
 <37259dc3-16f3-ef0c-fafa-d6a4749c5126@suse.de>
Message-ID: <d5bf9ad2b364ba7f07287b62f627e2263fc32458.camel@suse.com>

On Thu, 2021-02-04 at 08:04 +0100, Hannes Reinecke wrote:
> On 1/26/21 9:32 PM, mwilck at suse.com?wrote:
> > From: Martin Wilck <mwilck at suse.com>
> > 
> > Standard syslog loglevels are used.
> > Options:
> > 
> > ? -S/--silent?? to suppress LOG_NOTICE
> > ? -v/--verbose? to enable LOG_INFO (overrides -S)
> > ? -D/--debug??? to enable LOG_DEBUG (overrides -S, -v)
> > ? -C/--clockstamps to enable time stamps on log messages.
> > 
> > I tried to use short options that don't conflict with options
> > from nvme connect-all, because many of those options might be
> > useful for the monitor, too.
> > 
> > Signed-off-by: Martin Wilck <mwilck at suse.com>
> > ---
> > ? monitor.c | 47 +++++++++++++++++++++++++++++++++++++++++------
> > ? 1 file changed, 41 insertions(+), 6 deletions(-)
> > ... and if you had moved the logging functionality prior to the
> > monitor 
> code, this patch could be merged with the main monitor functionality,
> too.

I expected that reviewers preferred smaller chunks. If that's not the
case, fine with me, I'll change i

Thanks,
Martin




From mwilck at suse.com  Thu Feb  4 04:21:12 2021
From: mwilck at suse.com (Martin Wilck)
Date: Thu, 04 Feb 2021 10:21:12 +0100
Subject: [PATCH 10/35] fabrics: export do_discover(), build_options()
 and config
In-Reply-To: <5572e869-cd25-15ca-e95b-b4304e19d83b@suse.de>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-11-mwilck@suse.com>
 <5572e869-cd25-15ca-e95b-b4304e19d83b@suse.de>
Message-ID: <17afdd6d41a9e7294157c11567dee797752a21e3.camel@suse.com>

On Thu, 2021-02-04 at 08:09 +0100, Hannes Reinecke wrote:
> On 1/26/21 9:32 PM, mwilck at suse.com?wrote:
> > ? 
> > +/* Symbols used by monitor.c */
> > +
> > +struct config {
> > +???????char *nqn;
> > +???????char *transport;
> > +???????char *traddr;
> > +???????char *trsvcid;
> > +???????char *host_traddr;
> > +???????char *hostnqn;
> > +???????char *hostid;
> > +???????int? nr_io_queues;
> > +???????int? nr_write_queues;
> > +???????int? nr_poll_queues;
> > +???????int? queue_size;
> > +???????int? keep_alive_tmo;
> > +???????int? reconnect_delay;
> > +???????int? ctrl_loss_tmo;
> > +???????int? tos;
> > +???????char *raw;
> > +???????char *device;
> > +???????int? duplicate_connect;
> > +???????int? disable_sqflow;
> > +???????int? hdr_digest;
> > +???????int? data_digest;
> > +???????bool persistent;
> > +???????bool matching_only;
> > +};
> > +extern struct config cfg;
> > +
> > +#define BUF_SIZE 4096
> > +
> > +int build_options(char *argstr, int max_len, bool discover);
> > +int do_discover(char *argstr, bool connect);
> > +
> > ? #endif
> > 
> 
> Please name is 'struct config fabrics_cfg'. 'cfg' is used as a
> generic 
> name throughout the code, and using the same name in a header might
> lead to an accidental name clash.

Will do (the reason I didn't in the first place was that I wanted to
minimize changes of existing code, but you're right of course).

Thanks,
Martin




From mwilck at suse.com  Thu Feb  4 04:37:40 2021
From: mwilck at suse.com (Martin Wilck)
Date: Thu, 04 Feb 2021 10:37:40 +0100
Subject: [PATCH 13/35] monitor: disable nvmf-autoconnect udev rules in
 autoconnect mode
In-Reply-To: <ff1b9621-c01b-7884-1e8f-44a07ac845c1@suse.de>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-14-mwilck@suse.com>
 <ff1b9621-c01b-7884-1e8f-44a07ac845c1@suse.de>
Message-ID: <bca4cd03f4a9a64d36a746c92eb2afb4d7ce4750.camel@suse.com>

On Thu, 2021-02-04 at 08:16 +0100, Hannes Reinecke wrote:
> On 1/26/21 9:33 PM, mwilck at suse.com?wrote:
> > From: Martin Wilck <mwilck at suse.com>
> > 
> > If autoconnect is enabled, disable the respective udev rules
> > by symlinking /run/udev/rules.d to /dev/null, in order to avoid
> > the connections being set up by the monitor and the udev workers
> > at the same time. This is probably the preferred mode of operation
> > for the monitor.
> > 
> > Users can override this by copying 70-nvmf-autoconnect.rules
> > from /usr/lib/udev/rules.d to /etc/udev/rules.d (/etc/udev/rules.d
> > takes precedence over /run/udev/rules.d).
> > 
> > If the symlink can't be created for some reason, autoconnect will
> > be disabled. There is? only one exception: If
> > /run/udev/rules.d/70-nvmf-autoconnect.rules already points to
> > /dev/null at startup, autoconnect can be left on, but the symlink
> > isn't removed on exit.
> > 
> > Signed-off-by: Martin Wilck <mwilck at suse.com>
> > ---
> > ? monitor.c | 75
> > +++++++++++++++++++++++++++++++++++++++++++++++++++++--
> > ? 1 file changed, 73 insertions(+), 2 deletions(-)
> > 
> > diff --git a/monitor.c b/monitor.c
> > index ecf3be2..2a906db 100644
> > --- a/monitor.c
> > +++ b/monitor.c
> > @@ -17,14 +17,18 @@
> > ? 
> > ? #include <stddef.h>
> > ? #include <stdio.h>
> > +#include <stdlib.h>
> > ? #include <unistd.h>
> > ? #include <errno.h>
> > ? #include <libudev.h>
> > ? #include <signal.h>
> > ? #include <time.h>
> > +#include <limits.h>
> > ? #include <syslog.h>
> > +#include <sys/stat.h>
> > ? #include <sys/epoll.h>
> > ? 
> > +#include "common.h"
> > ? #include "nvme-status.h"
> > ? #include "util/argconfig.h"
> > ? #include "monitor.h"
> > @@ -33,6 +37,7 @@
> > ? 
> > ? static struct monitor_config {
> > ????????bool autoconnect;
> > +???????bool skip_udev_on_exit;
> > ? } mon_cfg;
> > ? 
> > ? static struct udev *udev;
> > @@ -45,6 +50,8 @@ static void close_ptr(int *p)
> > ????????}
> > ? }
> > ? 
> > +CLEANUP_FUNC(char)
> > +
> > ? static void cleanup_monitor(struct udev_monitor **pmon)
> > ? {
> > ????????if (*pmon) {
> > @@ -174,12 +181,64 @@ static int monitor_main_loop(struct
> > udev_monitor *monitor)
> > ????????return ret;
> > ? }
> > ? 
> > +static const char autoconnect_rules[] = "/run/udev/rules.d/70-
> > nvmf-autoconnect.rules";
> > +
> > +static int monitor_disable_udev_rules(void)
> > +{
> > +???????CLEANUP(char, path) = strdup(autoconnect_rules);
> > +???????char *s1, *s2;
> > +???????int rc;
> > +
> > +???????if (!path)
> > +???????????????return -ENOMEM;
> > +
> > +???????s2 = strrchr(path, '/');
> > +???????for (s1 = s2 - 1; s1 > path && *s1 != '/'; s1--);
> > +
> > +???????*s2 = *s1 = '\0';
> > +???????rc = mkdir(path, 0755);
> > +???????if (rc == 0 || errno == EEXIST) {
> > +???????????????*s1 = '/';
> > +???????????????rc = mkdir(path, 0755);
> > +???????????????if (rc == 0 || errno == EEXIST) {
> > +???????????????????????*s2 = '/';
> > +???????????????????????rc = symlink("/dev/null", path);
> > +???????????????}
> > +???????}
> > +???????if (rc) {
> > +???????????????if (errno == EEXIST) {
> > +???????????????????????char target[PATH_MAX];
> > +
> > +???????????????????????if (readlink(path, target, sizeof(target))
> > != -1 &&
> > +?????????????????????????? !strcmp(target, "/dev/null")) {
> > +???????????????????????????????log(LOG_INFO,
> > +?????????????????????????????????? "symlink %s -> /dev/null exists
> > already\n",
> > +?????????????????????????????????? autoconnect_rules);
> > +???????????????????????????????return 1;
> > +???????????????????????}
> > +???????????????}
> > +???????????????log(LOG_ERR, "error creating %s: %m\n",
> > autoconnect_rules);
> > +???????} else
> > +???????????????log(LOG_INFO, "created %s\n", autoconnect_rules);
> > +
> > +???????return rc ? (errno ? -errno : -EIO) : 0;
> > +}
> > +
> > +static void monitor_enable_udev_rules(void)
> > +{
> > +???????if (unlink(autoconnect_rules) == -1 && errno != ENOENT)
> > +???????????????log(LOG_ERR, "error removing %s: %m\n",
> > autoconnect_rules);
> > +???????else
> > +???????????????log(LOG_INFO, "removed %s\n", autoconnect_rules);
> > +}
> > +
> > ? static int monitor_parse_opts(const char *desc, int argc, char
> > **argv)
> > ? {
> > ????????bool quiet = false;
> > ????????bool verbose = false;
> > ????????bool debug = false;
> > -???????int ret;
> > +???????int ret = 0;
> > +
> > ????????OPT_ARGS(opts) = {
> > ????????????????OPT_FLAG("autoconnect",??? 'A',
> > &mon_cfg.autoconnect, "automatically connect newly discovered
> > controllers"),
> > ????????????????OPT_FLAG("silent",???????? 'S',
> > &quiet,?????????????? "log level: silent"),
> > @@ -198,7 +257,17 @@ static int monitor_parse_opts(const char
> > *desc, int argc, char **argv)
> > ????????????????log_level = LOG_INFO;
> > ????????if (debug)
> > ????????????????log_level = LOG_DEBUG;
> > -
> > +???????if (mon_cfg.autoconnect) {
> > +???????????????ret = monitor_disable_udev_rules();
> > +???????????????if (ret < 0) {
> > +???????????????????????mon_cfg.autoconnect = false;
> > +???????????????????????log(LOG_WARNING, "autoconnect disabled\n");
> > +???????????????????????ret = 0;
> > +???????????????} else if (ret > 0) {
> > +???????????????????????mon_cfg.skip_udev_on_exit = true;
> > +???????????????????????ret = 0;
> > +???????????????}
> > +???????}
> > ????????return ret;
> > ? }
> > ? 
> > @@ -221,6 +290,8 @@ int aen_monitor(const char *desc, int argc,
> > char **argv)
> > ????????????????udev_monitor_unref(monitor);
> > ????????}
> > ????????udev = udev_unref(udev);
> > +???????if (mon_cfg.autoconnect && !mon_cfg.skip_udev_on_exit)
> > +???????????????monitor_enable_udev_rules();
> > ? out:
> > ????????return nvme_status_to_errno(ret, true);
> > ? }
> > 
> What on earth ...
> 
> No way.
> We should _not_ reference individual files or mess with udev rules
> from 
> a program. If a file needs to be modified, modify it.
> But do not try to modify a file from a program context, unless the 
> program itself has created it.

Sagi said the same, I'll drop this. I thought it was acceptable because
I'd be masking a rule from the same package, and preferable because
making the user responsible for it would be cumbersome and error-prone
on the part of the user.

Thanks,
Martin

> 
> Cheers,
> 
> Hannes




From mwilck at suse.com  Thu Feb  4 04:41:59 2021
From: mwilck at suse.com (Martin Wilck)
Date: Thu, 04 Feb 2021 10:41:59 +0100
Subject: [PATCH 35/35] monitor: add option --keep/-K
In-Reply-To: <864dd77a-650f-9260-9d9d-02ab83b4a18d@suse.de>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210126203324.23610-36-mwilck@suse.com>
 <90145500-c2c5-023c-6d69-27db6ee600a7@grimberg.me>
 <12827c5f748bd2eff31450c47bbef1cfc7b73459.camel@suse.com>
 <400d4f5e-a578-e89f-ae45-7ed15d089a8d@grimberg.me>
 <b4b9a52011d17ef16cbc86d81ef3d19f7a517d93.camel@suse.com>
 <c127bde8-7cae-677c-4b8d-44cf757d05eb@grimberg.me>
 <2394e23e80f2bdcf0a95eb39e5eefcb00237d52f.camel@suse.com>
 <8cb1830e-b7f3-48ef-1190-0d917aa200bb@grimberg.me>
 <5623aeee6a57889123b8a3441ee6df7b97f76451.camel@suse.com>
 <552cb65a-8f1b-e414-4de8-f53ff469e8a1@grimberg.me>
 <c00de29ef02bf39ae231a2816f03c8e920ebdc29.camel@suse.com>
 <864dd77a-650f-9260-9d9d-02ab83b4a18d@suse.de>
Message-ID: <1c9baa8c4b76f3b808479bc0ede984456bfb6ea1.camel@suse.com>

On Thu, 2021-02-04 at 08:34 +0100, Hannes Reinecke wrote:
> On 1/29/21 10:15 PM, Martin Wilck wrote:
> > On Fri, 2021-01-29 at 13:11 -0800, Sagi Grimberg wrote:
> > > 
> > > > > > If it does, it will recreate discovery controllers for
> > > > > > every
> > > > > > host_traddr/traddr/trsvcid tuple it finds. "--keep"
> > > > > > semantics
> > > > > > are
> > > > > > only
> > > > > > necessary for addresses on which no regular (non-discovery)
> > > > > > connection
> > > > > > exists.
> > > > > 
> > > > > Wait, Maybe I'm missing something here, but are you saying
> > > > > that
> > > > > for
> > > > > every traddr/trsvcid it finds (both nvm and discovery) it
> > > > > will
> > > > > attempt
> > > > > to connect a discovery controller?
> > > > > 
> > > > > If so, this is absolutely wrong.
> > > > 
> > > > Currently, it tries to do that on startup, if (and only if) the
> > > > --startup option is given. My expectation was that the
> > > > connection
> > > > attempts would simply fail if there was no discovery subsystem
> > > > to
> > > > connect to. Anyway, it's not the default behavior, and can be
> > > > dropped
> > > > completely if it's so bad that we shouldn't ever attempt to do
> > > > it.
> > > 
> > > IMO it needs to be dropped. I didn't understand this at first
> > > because
> > > it never even occurred to me that such an assumption can be even
> > > made.
> > > 
> > > > If the service is started early on during boot, and event-based
> > > > discovery works (i.e. we also have the mDNS part in place),
> > > > this
> > > > won't
> > > > be necessary of course.
> > > 
> > > This isn't necessary regardless. At best the discovery controller
> > > endpoints should be obtained from discovery.conf or equivalent.
> > 
> > Understood.
> > 
> > > I don't even see how does this help in early boot anyways, how
> > > do the existing controllers get connected?
> > 
> > For FC, it works today. You can start the monitor in the initrd, it
> > will receive the fc_udev_device events, and autoconnect (just like
> > traditional udev/systemd based discovery). For other transports,
> > it'd
> > be more work, as we'd need to do mDNS discovery from an initrd
> > environment. I wouldn't say it's impossible though.
> > 
> mDNS isn't related to the initrd, mDNS is for discovery discovery :-)

What I had in mind was booting from NVMeoF (non-FC), without prior
local configuration. That requires "discovery discovery" in the initrd,
and thus making mDNS calls in the initrd.

Regards
Martin




From hch at infradead.org  Thu Feb  4 10:07:38 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 4 Feb 2021 16:07:38 +0100
Subject: [GIT PULL] nvme fixes for 5.11
Message-ID: <YBwNukLwQfsXQL9U@infradead.org>

The following changes since commit cd92cdb9c8bcfc27a8f28bcbf7c414a0ea79e5ec:

  null_blk: cleanup zoned mode initialization (2021-01-29 07:49:22 -0700)

are available in the Git repository at:

  git://git.infradead.org/nvme.git nvme-5.11

for you to fetch changes up to cb8563f5c735a042ea2dd7df1ad55ae06d63ffeb:

  nvmet-tcp: fix out-of-bounds access when receiving multiple h2cdata PDUs (2021-02-03 16:57:36 +0100)

----------------------------------------------------------------
Claus Stovgaard (1):
      nvme-pci: ignore the subsysem NQN on Phison E16

Keith Busch (1):
      update the email address for Keith Bush

Sagi Grimberg (1):
      nvmet-tcp: fix out-of-bounds access when receiving multiple h2cdata PDUs

Thorsten Leemhuis (1):
      nvme-pci: avoid the deepest sleep state on Kingston A2000 SSDs

 .mailmap                  | 2 ++
 drivers/nvme/host/pci.c   | 4 ++++
 drivers/nvme/target/tcp.c | 3 ++-
 3 files changed, 8 insertions(+), 1 deletion(-)


From axboe at kernel.dk  Thu Feb  4 10:25:05 2021
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 4 Feb 2021 08:25:05 -0700
Subject: [GIT PULL] nvme fixes for 5.11
In-Reply-To: <YBwNukLwQfsXQL9U@infradead.org>
References: <YBwNukLwQfsXQL9U@infradead.org>
Message-ID: <e407087c-3760-b725-4b02-692eee28041b@kernel.dk>

On 2/4/21 8:07 AM, Christoph Hellwig wrote:
> git://git.infradead.org/nvme.git nvme-5.11

You forgot the signed tag, but I pulled it.

-- 
Jens Axboe



From hch at infradead.org  Thu Feb  4 10:27:04 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 4 Feb 2021 15:27:04 +0000
Subject: [GIT PULL] nvme fixes for 5.11
In-Reply-To: <e407087c-3760-b725-4b02-692eee28041b@kernel.dk>
References: <YBwNukLwQfsXQL9U@infradead.org>
 <e407087c-3760-b725-4b02-692eee28041b@kernel.dk>
Message-ID: <20210204152704.GA249252@infradead.org>

On Thu, Feb 04, 2021 at 08:25:05AM -0700, Jens Axboe wrote:
> On 2/4/21 8:07 AM, Christoph Hellwig wrote:
> > git://git.infradead.org/nvme.git nvme-5.11
> 
> You forgot the signed tag, but I pulled it.

Sorry.  The signed tag would also have been available as
"nvme-5.11-2021-02-04", but I fat-fingered the actual pull request.


From axboe at fb.com  Thu Feb  4 10:27:46 2021
From: axboe at fb.com (Jens Axboe)
Date: Thu, 4 Feb 2021 08:27:46 -0700
Subject: [PATCH v5 1/3] blk-mq: introduce blk_mq_set_request_complete
In-Reply-To: <20210201034940.18891-2-lengchao@huawei.com>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210201034940.18891-2-lengchao@huawei.com>
Message-ID: <3b75b43c-638b-b5ac-eb5e-e37314d25ce1@fb.com>

On 1/31/21 8:49 PM, Chao Leng wrote:
> nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
> directly complete request in queue_rq.
> So add blk_mq_set_request_complete.

This is a bit iffy, but looks ok for the intended use case. We just have
to be careful NOT to add frivolous users of it...

-- 
Jens Axboe



From hch at lst.de  Thu Feb  4 10:30:14 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 4 Feb 2021 16:30:14 +0100
Subject: [PATCH v5 1/3] blk-mq: introduce blk_mq_set_request_complete
In-Reply-To: <3b75b43c-638b-b5ac-eb5e-e37314d25ce1@fb.com>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210201034940.18891-2-lengchao@huawei.com>
 <3b75b43c-638b-b5ac-eb5e-e37314d25ce1@fb.com>
Message-ID: <20210204153014.GA29736@lst.de>

On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
> On 1/31/21 8:49 PM, Chao Leng wrote:
> > nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
> > directly complete request in queue_rq.
> > So add blk_mq_set_request_complete.
> 
> This is a bit iffy, but looks ok for the intended use case. We just have
> to be careful NOT to add frivolous users of it...

Yes, that is my main issue with it.  The current use case looks fine,
but I suspect every time someone else uses it it's probly going to be
as misuse.  I've applied this to nvme-5.12 with the rest of the patches,
it should be on its way to you soon.


From axboe at fb.com  Thu Feb  4 10:32:17 2021
From: axboe at fb.com (Jens Axboe)
Date: Thu, 4 Feb 2021 08:32:17 -0700
Subject: [PATCH v5 1/3] blk-mq: introduce blk_mq_set_request_complete
In-Reply-To: <20210204153014.GA29736@lst.de>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210201034940.18891-2-lengchao@huawei.com>
 <3b75b43c-638b-b5ac-eb5e-e37314d25ce1@fb.com> <20210204153014.GA29736@lst.de>
Message-ID: <70bdb0b1-c7aa-af25-4b28-f8f6675c73ae@fb.com>

On 2/4/21 8:30 AM, Christoph Hellwig wrote:
> On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
>> On 1/31/21 8:49 PM, Chao Leng wrote:
>>> nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
>>> directly complete request in queue_rq.
>>> So add blk_mq_set_request_complete.
>>
>> This is a bit iffy, but looks ok for the intended use case. We just have
>> to be careful NOT to add frivolous users of it...
> 
> Yes, that is my main issue with it.  The current use case looks fine,
> but I suspect every time someone else uses it it's probly going to be
> as misuse.  I've applied this to nvme-5.12 with the rest of the patches,
> it should be on its way to you soon.

Might benefit from a big fat comment on why you probably shouldn't
be using it...

-- 
Jens Axboe



From hch at lst.de  Thu Feb  4 10:36:31 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 4 Feb 2021 16:36:31 +0100
Subject: [PATCH v5 1/3] blk-mq: introduce blk_mq_set_request_complete
In-Reply-To: <70bdb0b1-c7aa-af25-4b28-f8f6675c73ae@fb.com>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210201034940.18891-2-lengchao@huawei.com>
 <3b75b43c-638b-b5ac-eb5e-e37314d25ce1@fb.com> <20210204153014.GA29736@lst.de>
 <70bdb0b1-c7aa-af25-4b28-f8f6675c73ae@fb.com>
Message-ID: <20210204153631.GA30261@lst.de>

On Thu, Feb 04, 2021 at 08:32:17AM -0700, Jens Axboe wrote:
> On 2/4/21 8:30 AM, Christoph Hellwig wrote:
> > On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
> >> On 1/31/21 8:49 PM, Chao Leng wrote:
> >>> nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
> >>> directly complete request in queue_rq.
> >>> So add blk_mq_set_request_complete.
> >>
> >> This is a bit iffy, but looks ok for the intended use case. We just have
> >> to be careful NOT to add frivolous users of it...
> > 
> > Yes, that is my main issue with it.  The current use case looks fine,
> > but I suspect every time someone else uses it it's probly going to be
> > as misuse.  I've applied this to nvme-5.12 with the rest of the patches,
> > it should be on its way to you soon.
> 
> Might benefit from a big fat comment on why you probably shouldn't
> be using it...

I actually reworded the comment a bit, this is the current version:

http://git.infradead.org/nvme.git/commitdiff/bdd2a4a61460a341c1f8462e5caf63195e960623


From axboe at fb.com  Thu Feb  4 10:41:54 2021
From: axboe at fb.com (Jens Axboe)
Date: Thu, 4 Feb 2021 08:41:54 -0700
Subject: [PATCH v5 1/3] blk-mq: introduce blk_mq_set_request_complete
In-Reply-To: <20210204153631.GA30261@lst.de>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210201034940.18891-2-lengchao@huawei.com>
 <3b75b43c-638b-b5ac-eb5e-e37314d25ce1@fb.com> <20210204153014.GA29736@lst.de>
 <70bdb0b1-c7aa-af25-4b28-f8f6675c73ae@fb.com> <20210204153631.GA30261@lst.de>
Message-ID: <4e9b9999-497c-4b74-468a-1fd4504c49ff@fb.com>

On 2/4/21 8:36 AM, Christoph Hellwig wrote:
> On Thu, Feb 04, 2021 at 08:32:17AM -0700, Jens Axboe wrote:
>> On 2/4/21 8:30 AM, Christoph Hellwig wrote:
>>> On Thu, Feb 04, 2021 at 08:27:46AM -0700, Jens Axboe wrote:
>>>> On 1/31/21 8:49 PM, Chao Leng wrote:
>>>>> nvme drivers need to set the state of request to MQ_RQ_COMPLETE when
>>>>> directly complete request in queue_rq.
>>>>> So add blk_mq_set_request_complete.
>>>>
>>>> This is a bit iffy, but looks ok for the intended use case. We just have
>>>> to be careful NOT to add frivolous users of it...
>>>
>>> Yes, that is my main issue with it.  The current use case looks fine,
>>> but I suspect every time someone else uses it it's probly going to be
>>> as misuse.  I've applied this to nvme-5.12 with the rest of the patches,
>>> it should be on its way to you soon.
>>
>> Might benefit from a big fat comment on why you probably shouldn't
>> be using it...
> 
> I actually reworded the comment a bit, this is the current version:
> 
> http://git.infradead.org/nvme.git/commitdiff/bdd2a4a61460a341c1f8462e5caf63195e960623

Alright looks fine, as it clearly states it should be used from
->queue_rq().

-- 
Jens Axboe



From gerlitz.or at gmail.com  Thu Feb  4 13:29:41 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Thu, 4 Feb 2021 20:29:41 +0200
Subject: [PATCH v3 net-next 09/21] nvme-tcp: Deal with netdevice DOWN
 events
In-Reply-To: <c864eb96-18ea-21dd-3ef3-15ca908c5959@grimberg.me>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-10-borisp@mellanox.com>
 <c864eb96-18ea-21dd-3ef3-15ca908c5959@grimberg.me>
Message-ID: <CAJ3xEMjzh=PEqLNwW5RMtofskGQubD3Z5+CR_JhuCUCH50jtGg@mail.gmail.com>

On Wed, Feb 3, 2021 at 11:17 AM Sagi Grimberg <sagi at grimberg.me> wrote:
> > @@ -2930,6 +2931,27 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
> >       return ERR_PTR(ret);
> >   }
> >
> > +static int nvme_tcp_netdev_event(struct notifier_block *this,
> > +                              unsigned long event, void *ptr)
> > +{
> > +     struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
> > +     struct nvme_tcp_ctrl *ctrl;
> > +
> > +     switch (event) {
> > +     case NETDEV_GOING_DOWN:
> > +             mutex_lock(&nvme_tcp_ctrl_mutex);
> > +             list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list) {
> > +                     if (ndev != ctrl->offloading_netdev)
> > +                             continue;
> > +                     nvme_tcp_error_recovery(&ctrl->ctrl);
> > +             }
> > +             mutex_unlock(&nvme_tcp_ctrl_mutex);
> > +             flush_workqueue(nvme_reset_wq);
> > +             /* we assume that the going down part of error recovery is over */
>
> Maybe phrase it as:
> /*
>   * The associated controllers teardown has completed, ddp contexts
>   * were also torn down so we should be safe to continue...
>   */

sure


From gerlitz.or at gmail.com  Thu Feb  4 13:36:31 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Thu, 4 Feb 2021 20:36:31 +0200
Subject: [PATCH v3 net-next 08/21] nvme-tcp : Recalculate crc in the end
 of the capsule
In-Reply-To: <a104a5d1-b4cb-4275-6ced-b80f911b6f47@grimberg.me>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-9-borisp@mellanox.com>
 <a104a5d1-b4cb-4275-6ced-b80f911b6f47@grimberg.me>
Message-ID: <CAJ3xEMhSxrjLfDvfzVRCLb27K4KmRh-KHQaKxvDF_6VDbNN8Jw@mail.gmail.com>

On Wed, Feb 3, 2021 at 11:12 AM Sagi Grimberg <sagi at grimberg.me> wrote:

> > @@ -1841,8 +1913,10 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
> >       nvme_tcp_restore_sock_calls(queue);
> >       cancel_work_sync(&queue->io_work);
> >
> > -     if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
> > +     if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) ||
> > +         test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
> >               nvme_tcp_unoffload_socket(queue);
> > +
>
> extra newline

will remove

> >   }
> >
> >   static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
> > @@ -1970,8 +2044,6 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
> >   {
> >       int ret;
> >
> > -     to_tcp_ctrl(ctrl)->offloading_netdev = NULL;
> > -
>
> Unclear what is the intent here.

yep, unclear indeed.. will look and probably remove

as for your other comment on this patch, will get back to you later on

> >       ret = nvme_tcp_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
> >       if (ret)
> >               return ret;


From gerlitz.or at gmail.com  Thu Feb  4 14:20:21 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Thu, 4 Feb 2021 21:20:21 +0200
Subject: [PATCH v3 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <9a8041aa-513f-09b9-63f4-3d12db19231c@grimberg.me>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-8-borisp@mellanox.com>
 <9a8041aa-513f-09b9-63f4-3d12db19231c@grimberg.me>
Message-ID: <CAJ3xEMh90F8LKn2CCW5pP83moVQ3+O8ON_7q0HmoD8YNyomKyA@mail.gmail.com>

On Wed, Feb 3, 2021 at 10:54 AM Sagi Grimberg <sagi at grimberg.me> wrote:

> > +static
> > +int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> > +                       u16 command_id,
> > +                       struct request *rq)
> > +{
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +     struct net_device *netdev = queue->ctrl->offloading_netdev;
> > +     int ret;
> > +
> > +     if (unlikely(!netdev)) {
> > +             pr_info_ratelimited("%s: netdev not found\n", __func__);
> > +             return -EINVAL;
> > +     }
> > +
> > +     ret = netdev->tcp_ddp_ops->tcp_ddp_teardown(netdev, queue->sock->sk,
> > +                                                 &req->ddp, rq);
> > +     sg_free_table_chained(&req->ddp.sg_table, SG_CHUNK_SIZE);
> > +     req->offloaded = false;
>
> Why is the offloaded = false needed here? you also clear it when you setup.

yep, there two places where we needlessly falsified the offloaded flag
- will remove them

The lifetime rules are - set to false on cmd setup and set to true in
ddp setup if it succeeded

>
> > +     return ret;
> > +}
> > +
> > +static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> > +{
> > +     struct request *rq = ddp_ctx;
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +
> > +     if (!nvme_try_complete_req(rq, cpu_to_le16(req->status << 1), req->result))
> > +             nvme_complete_rq(rq);
>
> Why is the status shifted here? it was taken from the cqe as is..

there are two cases

1. the status is taken from the cqe as is
2. the status is taken from the req with shift left (the success bit
IO read flow)

for #2 we already do the shift left in nvme_tcp_end_request so no need to
repeat it here,  will fix

>
> > +}
> > +
> > +static
> > +int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> > +                    u16 command_id,
> > +                    struct request *rq)
> > +{
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +     struct net_device *netdev = queue->ctrl->offloading_netdev;
> > +     int ret;
> > +
> > +     req->offloaded = false;
> > +
> > +     if (unlikely(!netdev)) {
> > +             pr_info_ratelimited("%s: netdev not found\n", __func__);
>
> dev_info_ratelimited please.

ok

> > +             return -EINVAL;
> > +     }
> > +
> > +     req->ddp.command_id = command_id;
> > +     req->ddp.sg_table.sgl = req->ddp.first_sgl;
> > +     ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
> > +                                  req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
> > +     if (ret)
> > +             return -ENOMEM;
> > +     req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
> > +
> > +     ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
> > +                                              queue->sock->sk,
> > +                                              &req->ddp);
> > +     if (!ret)
> > +             req->offloaded = true;
> > +     return ret;
> > +}
> > +
> >   static
> >   int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
> >   {
> > @@ -377,6 +446,25 @@ bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
> >
> >   #else
> >
> > +static
> > +int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> > +                    u16 command_id,
> > +                    struct request *rq)
> > +{
> > +     return -EINVAL;
> > +}
> > +
> > +static
> > +int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> > +                       u16 command_id,
> > +                       struct request *rq)
> > +{
> > +     return -EINVAL;
> > +}
> > +
> > +static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> > +{}
> > +
> >   static
> >   int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
> >   {
> > @@ -665,6 +753,7 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
> >   static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
> >               struct nvme_completion *cqe)
> >   {
> > +     struct nvme_tcp_request *req;
> >       struct request *rq;
> >
> >       rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), cqe->command_id);
> > @@ -676,8 +765,15 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
> >               return -EINVAL;
> >       }
> >
> > -     if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> > -             nvme_complete_rq(rq);
> > +     req = blk_mq_rq_to_pdu(rq);
> > +     if (req->offloaded) {
> > +             req->status = cqe->status;
> > +             req->result = cqe->result;
> > +             nvme_tcp_teardown_ddp(queue, cqe->command_id, rq);
> > +     } else {
> > +             if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
> > +                     nvme_complete_rq(rq);
> > +     }
>
> Maybe move this to nvme_tcp_complete_request as it is called from two
> code paths.

to make sure, add

void nvme_tcp_complete_request(struct request *rq, u16 status, union
nvme_result *res, u16 status)

and invoke it from the two calls sites?

> >       queue->nr_cqe++;
> >
> >       return 0;
> > @@ -871,9 +967,18 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> >   static inline void nvme_tcp_end_request(struct request *rq, u16 status)
> >   {
> >       union nvme_result res = {};
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +     struct nvme_tcp_queue *queue = req->queue;
> > +     struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
> >
> > -     if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> > -             nvme_complete_rq(rq);
> > +     if (req->offloaded) {
> > +             req->status = cpu_to_le16(status << 1);
> > +             req->result = res;
> > +             nvme_tcp_teardown_ddp(queue, pdu->command_id, rq);
> > +     } else {
> > +             if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> > +                     nvme_complete_rq(rq);
> > +     }
> >   }
> >
> >   static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> > @@ -920,12 +1025,22 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> >               recv_len = min_t(size_t, recv_len,
> >                               iov_iter_count(&req->iter));
> >
> > -             if (queue->data_digest)
> > -                     ret = skb_copy_and_hash_datagram_iter(skb, *offset,
> > -                             &req->iter, recv_len, queue->rcv_hash);
> > -             else
> > -                     ret = skb_copy_datagram_iter(skb, *offset,
> > -                                     &req->iter, recv_len);
> > +             if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
> > +                     if (queue->data_digest)
> > +                             ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset,
> > +                                             &req->iter, recv_len, queue->rcv_hash);
> > +                     else
> > +                             ret = skb_ddp_copy_datagram_iter(skb, *offset,
> > +                                             &req->iter, recv_len);
> > +             } else {
> > +                     if (queue->data_digest)
> > +                             ret = skb_copy_and_hash_datagram_iter(skb, *offset,
> > +                                             &req->iter, recv_len, queue->rcv_hash);
> > +                     else
> > +                             ret = skb_copy_datagram_iter(skb, *offset,
> > +                                             &req->iter, recv_len);
> > +             }
> > +
>
> Maybe move this hunk to nvme_tcp_consume_skb or something?

ok, we can add

int nvme_tcp_consume_skb(queue, skb, offset, iter, recv_len)

and put this hunk there

> > @@ -2464,6 +2584,7 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns,
> >       req->data_len = blk_rq_nr_phys_segments(rq) ?
> >                               blk_rq_payload_bytes(rq) : 0;
> >       req->curr_bio = rq->bio;
> > +     req->offloaded = false;
>
> offloaded is being cleared lots of times, and I'm not clear what are
> the lifetime rules here.

replied above


From hch at lst.de  Thu Feb  4 14:30:27 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:27 +0100
Subject: preserve DMA offsets when using swiotlb 
Message-ID: <20210204193035.2606838-1-hch@lst.de>

Hi all,

this series make NVMe happy when running with swiotlb.  This caters
towards to completely broken NVMe controllers that ignore the
specification (hello to the biggest cloud provider on the planet!),
to crappy SOC that have addressing limitations, or "secure"
virtualization that force bounce buffering to enhance the user
experience.  Or in other words, no one sane should hit it, but
people do.

It is basically a respin of the

    "SWIOTLB: Preserve swiotlb map offset when needed."

series from Jianxiong Gao.  It complete rewrites the swiotlb part so that
the offset really is preserved and not just the offset into the swiotlb
slot, and to do so it grew half a dozen patches to refactor the swiotlb
so that a mere mortal like me could actually understand it.



From hch at lst.de  Thu Feb  4 14:30:28 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:28 +0100
Subject: [PATCH 1/8] driver core: add a min_align_mask field to struct
 device_dma_parameters
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-2-hch@lst.de>

From: Jianxiong Gao <jxgao at google.com>

Some devices rely on the address offset in a page to function
correctly (NVMe driver as an example). These devices may use
a different page size than the Linux kernel. The address offset
has to be preserved upon mapping, and in order to do so, we
need to record the page_offset_mask first.

Signed-off-by: Jianxiong Gao <jxgao at google.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 include/linux/device.h      |  1 +
 include/linux/dma-mapping.h | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/include/linux/device.h b/include/linux/device.h
index 1779f90eeb4cb4..7960bf516dd7fe 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -291,6 +291,7 @@ struct device_dma_parameters {
 	 * sg limitations.
 	 */
 	unsigned int max_segment_size;
+	unsigned int min_align_mask;
 	unsigned long segment_boundary_mask;
 };
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 2e49996a8f391a..9c26225754e719 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -500,6 +500,22 @@ static inline int dma_set_seg_boundary(struct device *dev, unsigned long mask)
 	return -EIO;
 }
 
+static inline unsigned int dma_get_min_align_mask(struct device *dev)
+{
+	if (dev->dma_parms)
+		return dev->dma_parms->min_align_mask;
+	return 0;
+}
+
+static inline int dma_set_min_align_mask(struct device *dev,
+		unsigned int min_align_mask)
+{
+	if (WARN_ON_ONCE(!dev->dma_parms))
+		return -EIO;
+	dev->dma_parms->min_align_mask = min_align_mask;
+	return 0;
+}
+
 static inline int dma_get_cache_alignment(void)
 {
 #ifdef ARCH_DMA_MINALIGN
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:30:29 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:29 +0100
Subject: [PATCH 2/8] swiotlb: add a io_tlb_offset helper
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-3-hch@lst.de>

Replace the very genericly named OFFSET macro with a little inline
helper that hardcodes the alignment to the only value ever passed.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 7c42df6e61001d..838dbad10ab916 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -50,9 +50,6 @@
 #define CREATE_TRACE_POINTS
 #include <trace/events/swiotlb.h>
 
-#define OFFSET(val,align) ((unsigned long)	\
-	                   ( (val) & ( (align) - 1)))
-
 #define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
 
 /*
@@ -192,6 +189,11 @@ void swiotlb_print_info(void)
 	       bytes >> 20);
 }
 
+static inline unsigned long io_tlb_offset(unsigned long val)
+{
+	return val & (IO_TLB_SEGSIZE - 1);
+}
+
 /*
  * Early SWIOTLB allocation may be too early to allow an architecture to
  * perform the desired operations.  This function allows the architecture to
@@ -241,7 +243,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 		      __func__, alloc_size, PAGE_SIZE);
 
 	for (i = 0; i < io_tlb_nslabs; i++) {
-		io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE);
+		io_tlb_list[i] = IO_TLB_SEGSIZE - io_tlb_offset(i);
 		io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
 	}
 	io_tlb_index = 0;
@@ -375,7 +377,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
 		goto cleanup4;
 
 	for (i = 0; i < io_tlb_nslabs; i++) {
-		io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE);
+		io_tlb_list[i] = IO_TLB_SEGSIZE - io_tlb_offset(i);
 		io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
 	}
 	io_tlb_index = 0;
@@ -546,7 +548,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 
 			for (i = index; i < (int) (index + nslots); i++)
 				io_tlb_list[i] = 0;
-			for (i = index - 1; (OFFSET(i, IO_TLB_SEGSIZE) != IO_TLB_SEGSIZE - 1) && io_tlb_list[i]; i--)
+			for (i = index - 1;
+			     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
+			     io_tlb_list[i]; i--)
 				io_tlb_list[i] = ++count;
 			tlb_addr = io_tlb_start + (index << IO_TLB_SHIFT);
 
@@ -632,7 +636,9 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 		 * Step 2: merge the returned slots with the preceding slots,
 		 * if available (non zero)
 		 */
-		for (i = index - 1; (OFFSET(i, IO_TLB_SEGSIZE) != IO_TLB_SEGSIZE -1) && io_tlb_list[i]; i--)
+		for (i = index - 1;
+		     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
+		     io_tlb_list[i]; i--)
 			io_tlb_list[i] = ++count;
 
 		io_tlb_used -= nslots;
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:30:30 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:30 +0100
Subject: [PATCH 3/8] swiotlb: factor out a nr_slots helper
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-4-hch@lst.de>

Factor out a helper to find the number of slots for a given size.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 838dbad10ab916..0c0b81799edbdb 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -194,6 +194,11 @@ static inline unsigned long io_tlb_offset(unsigned long val)
 	return val & (IO_TLB_SEGSIZE - 1);
 }
 
+static unsigned long nr_slots(u64 val)
+{
+	return ALIGN(val, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+}
+
 /*
  * Early SWIOTLB allocation may be too early to allow an architecture to
  * perform the desired operations.  This function allows the architecture to
@@ -493,20 +498,20 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 
 	tbl_dma_addr &= mask;
 
-	offset_slots = ALIGN(tbl_dma_addr, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+	offset_slots = nr_slots(tbl_dma_addr);
 
 	/*
 	 * Carefully handle integer overflow which can occur when mask == ~0UL.
 	 */
 	max_slots = mask + 1
-		    ? ALIGN(mask + 1, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT
+		    ? nr_slots(mask + 1)
 		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
 
 	/*
 	 * For mappings greater than or equal to a page, we limit the stride
 	 * (and hence alignment) to a page size.
 	 */
-	nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+	nslots = nr_slots(alloc_size);
 	if (alloc_size >= PAGE_SIZE)
 		stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
 	else
@@ -602,7 +607,7 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 			      enum dma_data_direction dir, unsigned long attrs)
 {
 	unsigned long flags;
-	int i, count, nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+	int i, count, nslots = nr_slots(alloc_size);
 	int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
 	phys_addr_t orig_addr = io_tlb_orig_addr[index];
 
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:30:31 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:31 +0100
Subject: [PATCH 4/8] swiotlb: clean up swiotlb_tbl_unmap_single
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-5-hch@lst.de>

Remove a layer of pointless indentation, replace a hard to follow
ternary expression with a plain if/else.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 41 +++++++++++++++++++++--------------------
 1 file changed, 21 insertions(+), 20 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 0c0b81799edbdb..79d5b236f25f10 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -626,28 +626,29 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 	 * with slots below and above the pool being returned.
 	 */
 	spin_lock_irqsave(&io_tlb_lock, flags);
-	{
-		count = ((index + nslots) < ALIGN(index + 1, IO_TLB_SEGSIZE) ?
-			 io_tlb_list[index + nslots] : 0);
-		/*
-		 * Step 1: return the slots to the free list, merging the
-		 * slots with superceeding slots
-		 */
-		for (i = index + nslots - 1; i >= index; i--) {
-			io_tlb_list[i] = ++count;
-			io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
-		}
-		/*
-		 * Step 2: merge the returned slots with the preceding slots,
-		 * if available (non zero)
-		 */
-		for (i = index - 1;
-		     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
-		     io_tlb_list[i]; i--)
-			io_tlb_list[i] = ++count;
+	if (index + nslots < ALIGN(index + 1, IO_TLB_SEGSIZE))
+		count = io_tlb_list[index + nslots];
+	else
+		count = 0;
 
-		io_tlb_used -= nslots;
+	/*
+	 * Step 1: return the slots to the free list, merging the slots with
+	 * superceeding slots
+	 */
+	for (i = index + nslots - 1; i >= index; i--) {
+		io_tlb_list[i] = ++count;
+		io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
 	}
+
+	/*
+	 * Step 2: merge the returned slots with the preceding slots, if
+	 * available (non zero)
+	 */
+	for (i = index - 1;
+	     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 && io_tlb_list[i];
+	     i--)
+		io_tlb_list[i] = ++count;
+	io_tlb_used -= nslots;
 	spin_unlock_irqrestore(&io_tlb_lock, flags);
 }
 
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:30:32 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:32 +0100
Subject: [PATCH 5/8] swiotlb: refactor swiotlb_tbl_map_single
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-6-hch@lst.de>

Split out a bunch of a self-contained helpers to make the function easier
to follow.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 177 +++++++++++++++++++++----------------------
 1 file changed, 87 insertions(+), 90 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 79d5b236f25f10..e78615e3be2906 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -468,134 +468,131 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
 	}
 }
 
-phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
-		size_t mapping_size, size_t alloc_size,
-		enum dma_data_direction dir, unsigned long attrs)
+/*
+ * Carefully handle integer overflow which can occur when boundary_mask == ~0UL.
+ */
+static inline unsigned long get_max_slots(unsigned long boundary_mask)
 {
-	dma_addr_t tbl_dma_addr = phys_to_dma_unencrypted(hwdev, io_tlb_start);
-	unsigned long flags;
-	phys_addr_t tlb_addr;
-	unsigned int nslots, stride, index, wrap;
-	int i;
-	unsigned long mask;
-	unsigned long offset_slots;
-	unsigned long max_slots;
-	unsigned long tmp_io_tlb_used;
-
-	if (no_iotlb_memory)
-		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
-
-	if (mem_encrypt_active())
-		pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
-
-	if (mapping_size > alloc_size) {
-		dev_warn_once(hwdev, "Invalid sizes (mapping: %zd bytes, alloc: %zd bytes)",
-			      mapping_size, alloc_size);
-		return (phys_addr_t)DMA_MAPPING_ERROR;
-	}
-
-	mask = dma_get_seg_boundary(hwdev);
+	if (boundary_mask + 1 == ~0UL)
+		return 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
+	return nr_slots(boundary_mask + 1);
+}
 
-	tbl_dma_addr &= mask;
+static unsigned int wrap_index(unsigned int index)
+{
+	if (index >= io_tlb_nslabs)
+		return 0;
+	return index;
+}
 
-	offset_slots = nr_slots(tbl_dma_addr);
+/*
+ * Find a suitable number of IO TLB entries size that will fit this request and
+ * allocate a buffer from that IO TLB pool.
+ */
+static int find_slots(struct device *dev, size_t alloc_size,
+		dma_addr_t tbl_dma_addr)
+{
+	unsigned int max_slots = get_max_slots(dma_get_seg_boundary(dev));
+	unsigned int nslots = nr_slots(alloc_size), stride = 1;
+	unsigned int index, wrap, count = 0, i;
+	unsigned long flags;
 
-	/*
-	 * Carefully handle integer overflow which can occur when mask == ~0UL.
-	 */
-	max_slots = mask + 1
-		    ? nr_slots(mask + 1)
-		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
+	BUG_ON(!nslots);
 
 	/*
 	 * For mappings greater than or equal to a page, we limit the stride
 	 * (and hence alignment) to a page size.
 	 */
-	nslots = nr_slots(alloc_size);
 	if (alloc_size >= PAGE_SIZE)
-		stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
-	else
-		stride = 1;
-
-	BUG_ON(!nslots);
+		stride <<= (PAGE_SHIFT - IO_TLB_SHIFT);
 
-	/*
-	 * Find suitable number of IO TLB entries size that will fit this
-	 * request and allocate a buffer from that IO TLB pool.
-	 */
 	spin_lock_irqsave(&io_tlb_lock, flags);
-
 	if (unlikely(nslots > io_tlb_nslabs - io_tlb_used))
 		goto not_found;
 
-	index = ALIGN(io_tlb_index, stride);
-	if (index >= io_tlb_nslabs)
-		index = 0;
-	wrap = index;
-
+	index = wrap = wrap_index(ALIGN(io_tlb_index, stride));
 	do {
-		while (iommu_is_span_boundary(index, nslots, offset_slots,
-					      max_slots)) {
-			index += stride;
-			if (index >= io_tlb_nslabs)
-				index = 0;
-			if (index == wrap)
-				goto not_found;
-		}
-
 		/*
 		 * If we find a slot that indicates we have 'nslots' number of
 		 * contiguous buffers, we allocate the buffers from that slot
 		 * and mark the entries as '0' indicating unavailable.
 		 */
-		if (io_tlb_list[index] >= nslots) {
-			int count = 0;
-
-			for (i = index; i < (int) (index + nslots); i++)
-				io_tlb_list[i] = 0;
-			for (i = index - 1;
-			     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
-			     io_tlb_list[i]; i--)
-				io_tlb_list[i] = ++count;
-			tlb_addr = io_tlb_start + (index << IO_TLB_SHIFT);
-
-			/*
-			 * Update the indices to avoid searching in the next
-			 * round.
-			 */
-			io_tlb_index = ((index + nslots) < io_tlb_nslabs
-					? (index + nslots) : 0);
-
-			goto found;
+		if (!iommu_is_span_boundary(index, nslots,
+					    nr_slots(tbl_dma_addr),
+					    max_slots)) {
+			if (io_tlb_list[index] >= nslots)
+				goto found;
 		}
-		index += stride;
-		if (index >= io_tlb_nslabs)
-			index = 0;
+		index = wrap_index(index + stride);
 	} while (index != wrap);
 
 not_found:
-	tmp_io_tlb_used = io_tlb_used;
-
 	spin_unlock_irqrestore(&io_tlb_lock, flags);
-	if (!(attrs & DMA_ATTR_NO_WARN) && printk_ratelimit())
-		dev_warn(hwdev, "swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (slots)\n",
-			 alloc_size, io_tlb_nslabs, tmp_io_tlb_used);
-	return (phys_addr_t)DMA_MAPPING_ERROR;
+	return -1;
+
 found:
+	for (i = index; i < index + nslots; i++)
+		io_tlb_list[i] = 0;
+	for (i = index - 1;
+	     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
+	     io_tlb_list[i]; i--)
+		io_tlb_list[i] = ++count;
+
+	/*
+	 * Update the indices to avoid searching in the next round.
+	 */
+	if (index + nslots < io_tlb_nslabs)
+		io_tlb_index = index + nslots;
+	else
+		io_tlb_index = 0;
 	io_tlb_used += nslots;
+
 	spin_unlock_irqrestore(&io_tlb_lock, flags);
+	return index;
+}
+
+phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
+		size_t mapping_size, size_t alloc_size,
+		enum dma_data_direction dir, unsigned long attrs)
+{
+	dma_addr_t tbl_dma_addr = phys_to_dma_unencrypted(dev, io_tlb_start) &
+			dma_get_seg_boundary(dev);
+	unsigned int index, i;
+	phys_addr_t tlb_addr;
+
+	if (no_iotlb_memory)
+		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
+
+	if (mem_encrypt_active())
+		pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
+
+	if (mapping_size > alloc_size) {
+		dev_warn_once(dev, "Invalid sizes (mapping: %zd bytes, alloc: %zd bytes)",
+			      mapping_size, alloc_size);
+		return (phys_addr_t)DMA_MAPPING_ERROR;
+	}
+
+	index = find_slots(dev, alloc_size, tbl_dma_addr);
+	if (index == -1) {
+		if (!(attrs & DMA_ATTR_NO_WARN))
+			dev_warn_ratelimited(dev,
+	"swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (slots)\n",
+				 alloc_size, io_tlb_nslabs, io_tlb_used);
+		return (phys_addr_t)DMA_MAPPING_ERROR;
+	}
 
 	/*
 	 * Save away the mapping from the original address to the DMA address.
 	 * This is needed when we sync the memory.  Then we sync the buffer if
 	 * needed.
 	 */
-	for (i = 0; i < nslots; i++)
-		io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
+	for (i = 0; i < nr_slots(alloc_size); i++)
+		io_tlb_orig_addr[index + i] = orig_addr + (i << IO_TLB_SHIFT);
+
+	tlb_addr = io_tlb_start + (index << IO_TLB_SHIFT);
 	if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
 	    (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
 		swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_TO_DEVICE);
-
 	return tlb_addr;
 }
 
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:30:33 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:33 +0100
Subject: [PATCH 6/8] swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-7-hch@lst.de>

swiotlb_tbl_map_single currently nevers sets a tlb_addr that is not
aligned to the tlb bucket size.  But we're going to add such a case
soon, for which this adjustment would be bogus.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index e78615e3be2906..6a2439826a1ba4 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -658,7 +658,6 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
 
 	if (orig_addr == INVALID_PHYS_ADDR)
 		return;
-	orig_addr += (unsigned long)tlb_addr & ((1 << IO_TLB_SHIFT) - 1);
 
 	switch (target) {
 	case SYNC_FOR_CPU:
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:30:34 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:34 +0100
Subject: [PATCH 7/8] swiotlb: respect min_align_mask
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-8-hch@lst.de>

Respect the min_align_mask in struct device_dma_parameters in swiotlb.

There are two parts to it:
 1) for the lower bits of the alignment inside the io tlb slot, just
    extent the size of the allocation and leave the start of the slot
     empty
 2) for the high bits ensure we find a slot that matches the high bits
    of the alignment to avoid wasting too much memory

Based on an earlier patch from Jianxiong Gao <jxgao at google.com>.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 49 +++++++++++++++++++++++++++++++++++++-------
 1 file changed, 42 insertions(+), 7 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 6a2439826a1ba4..ab3192142b9906 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -468,6 +468,18 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
 	}
 }
 
+/*
+ * Return the offset into a iotlb slot required to keep the device happy.
+ */
+static unsigned int swiotlb_align_offset(struct device *dev, u64 addr)
+{
+	unsigned min_align_mask = dma_get_min_align_mask(dev);
+
+	if (!min_align_mask)
+		return 0;
+	return addr & min_align_mask & ((1 << IO_TLB_SHIFT) - 1);
+}
+
 /*
  * Carefully handle integer overflow which can occur when boundary_mask == ~0UL.
  */
@@ -478,6 +490,16 @@ static inline unsigned long get_max_slots(unsigned long boundary_mask)
 	return nr_slots(boundary_mask + 1);
 }
 
+static inline bool check_alignment(phys_addr_t orig_addr,
+		dma_addr_t tbl_dma_addr, unsigned int index,
+		unsigned int min_align_mask)
+{
+	if (!min_align_mask)
+		return true;
+	return ((tbl_dma_addr + (index << IO_TLB_SHIFT)) & min_align_mask) ==
+		(orig_addr & min_align_mask);
+}
+
 static unsigned int wrap_index(unsigned int index)
 {
 	if (index >= io_tlb_nslabs)
@@ -489,9 +511,11 @@ static unsigned int wrap_index(unsigned int index)
  * Find a suitable number of IO TLB entries size that will fit this request and
  * allocate a buffer from that IO TLB pool.
  */
-static int find_slots(struct device *dev, size_t alloc_size,
-		dma_addr_t tbl_dma_addr)
+static int find_slots(struct device *dev, phys_addr_t orig_addr,
+		size_t alloc_size, dma_addr_t tbl_dma_addr)
 {
+	unsigned int min_align_mask = dma_get_min_align_mask(dev) &
+			~((1 << IO_TLB_SHIFT) - 1);
 	unsigned int max_slots = get_max_slots(dma_get_seg_boundary(dev));
 	unsigned int nslots = nr_slots(alloc_size), stride = 1;
 	unsigned int index, wrap, count = 0, i;
@@ -503,7 +527,9 @@ static int find_slots(struct device *dev, size_t alloc_size,
 	 * For mappings greater than or equal to a page, we limit the stride
 	 * (and hence alignment) to a page size.
 	 */
-	if (alloc_size >= PAGE_SIZE)
+	if (min_align_mask)
+		stride = (min_align_mask + 1) >> IO_TLB_SHIFT;
+	else if (alloc_size >= PAGE_SIZE)
 		stride <<= (PAGE_SHIFT - IO_TLB_SHIFT);
 
 	spin_lock_irqsave(&io_tlb_lock, flags);
@@ -512,6 +538,12 @@ static int find_slots(struct device *dev, size_t alloc_size,
 
 	index = wrap = wrap_index(ALIGN(io_tlb_index, stride));
 	do {
+		if (!check_alignment(orig_addr, tbl_dma_addr, index,
+				     min_align_mask)) {
+			index = wrap_index(index + 1);
+			continue;
+		}
+
 		/*
 		 * If we find a slot that indicates we have 'nslots' number of
 		 * contiguous buffers, we allocate the buffers from that slot
@@ -557,6 +589,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
 {
 	dma_addr_t tbl_dma_addr = phys_to_dma_unencrypted(dev, io_tlb_start) &
 			dma_get_seg_boundary(dev);
+	unsigned int offset = swiotlb_align_offset(dev, orig_addr);
 	unsigned int index, i;
 	phys_addr_t tlb_addr;
 
@@ -572,7 +605,8 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
 		return (phys_addr_t)DMA_MAPPING_ERROR;
 	}
 
-	index = find_slots(dev, alloc_size, tbl_dma_addr);
+	alloc_size += offset;
+	index = find_slots(dev, orig_addr, alloc_size, tbl_dma_addr);
 	if (index == -1) {
 		if (!(attrs & DMA_ATTR_NO_WARN))
 			dev_warn_ratelimited(dev,
@@ -589,7 +623,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
 	for (i = 0; i < nr_slots(alloc_size); i++)
 		io_tlb_orig_addr[index + i] = orig_addr + (i << IO_TLB_SHIFT);
 
-	tlb_addr = io_tlb_start + (index << IO_TLB_SHIFT);
+	tlb_addr = io_tlb_start + (index << IO_TLB_SHIFT) + offset;
 	if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
 	    (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
 		swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_TO_DEVICE);
@@ -604,8 +638,9 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 			      enum dma_data_direction dir, unsigned long attrs)
 {
 	unsigned long flags;
-	int i, count, nslots = nr_slots(alloc_size);
-	int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
+	unsigned int offset = swiotlb_align_offset(hwdev, tlb_addr);
+	int i, count, nslots = nr_slots(alloc_size + offset);
+	int index = (tlb_addr - offset - io_tlb_start) >> IO_TLB_SHIFT;
 	phys_addr_t orig_addr = io_tlb_orig_addr[index];
 
 	/*
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:30:35 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu,  4 Feb 2021 20:30:35 +0100
Subject: [PATCH 8/8] nvme-pci: set min_align_mask
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210204193035.2606838-9-hch@lst.de>

From: Jianxiong Gao <jxgao at google.com>

The PRP addressing scheme requires all PRP entries except for the
first one to have a zero offset into the NVMe controller pages (which
can be different from the Linux PAGE_SIZE).  Use the min_align_mask
device parameter to ensure that swiotlb does not change the address
of the buffer modulo the device page size to ensure that the PRPs
won't be malformed.

Signed-off-by: Jianxiong Gao <jxgao at google.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b204205..5d194b4e814719 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2629,6 +2629,7 @@ static void nvme_reset_work(struct work_struct *work)
 	 * Don't limit the IOMMU merged segment size.
 	 */
 	dma_set_max_seg_size(dev->dev, 0xffffffff);
+	dma_set_min_align_mask(&pdev->dev, NVME_CTRL_PAGE_SIZE - 1);
 
 	mutex_unlock(&dev->shutdown_lock);
 
-- 
2.29.2



From hch at lst.de  Thu Feb  4 14:32:30 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 4 Feb 2021 20:32:30 +0100
Subject: [PATCH 8/8] nvme-pci: set min_align_mask
In-Reply-To: <20210204193035.2606838-9-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-9-hch@lst.de>
Message-ID: <20210204193230.GA14946@lst.de>

> +	dma_set_min_align_mask(&pdev->dev, NVME_CTRL_PAGE_SIZE - 1);

And due to a last minute change from me this doesn't actually compile,
as pdev should be dev.


From gregkh at linuxfoundation.org  Thu Feb  4 14:44:57 2021
From: gregkh at linuxfoundation.org (Greg KH)
Date: Thu, 4 Feb 2021 20:44:57 +0100
Subject: [PATCH 1/8] driver core: add a min_align_mask field to struct
 device_dma_parameters
In-Reply-To: <20210204193035.2606838-2-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-2-hch@lst.de>
Message-ID: <YBxOuScfAXhdcuf/@kroah.com>

On Thu, Feb 04, 2021 at 08:30:28PM +0100, Christoph Hellwig wrote:
> From: Jianxiong Gao <jxgao at google.com>
> 
> Some devices rely on the address offset in a page to function
> correctly (NVMe driver as an example). These devices may use
> a different page size than the Linux kernel. The address offset
> has to be preserved upon mapping, and in order to do so, we
> need to record the page_offset_mask first.
> 
> Signed-off-by: Jianxiong Gao <jxgao at google.com>
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>  include/linux/device.h      |  1 +
>  include/linux/dma-mapping.h | 16 ++++++++++++++++
>  2 files changed, 17 insertions(+)

Acked-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>


From lkp at intel.com  Thu Feb  4 15:18:34 2021
From: lkp at intel.com (kernel test robot)
Date: Fri, 05 Feb 2021 04:18:34 +0800
Subject: [linux-nvme:nvme-5.12] BUILD SUCCESS
 aced5580e793b6d0287587bbb36f11510775b627
Message-ID: <601c569a.lIUZfAYZ2EGdyXCI%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.12
branch HEAD: aced5580e793b6d0287587bbb36f11510775b627  nvme-rdma: handle nvme_rdma_post_send failures better

elapsed time: 723m

configs tested: 123
configs skipped: 3

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
i386                             alldefconfig
sh                          rsk7264_defconfig
arm                           u8500_defconfig
m68k                           sun3_defconfig
mips                        vocore2_defconfig
sh                           se7206_defconfig
arm                        clps711x_defconfig
arm                            hisi_defconfig
sh                           se7343_defconfig
arm                       multi_v4t_defconfig
powerpc                 linkstation_defconfig
powerpc                      cm5200_defconfig
arm                      pxa255-idp_defconfig
openrisc                 simple_smp_defconfig
powerpc                     mpc5200_defconfig
xtensa                         virt_defconfig
powerpc                    gamecube_defconfig
c6x                              alldefconfig
sh                            hp6xx_defconfig
c6x                        evmc6474_defconfig
sh                           se7750_defconfig
arm                        spear6xx_defconfig
arm                            pleb_defconfig
ia64                            zx1_defconfig
sh                  sh7785lcr_32bit_defconfig
powerpc                     redwood_defconfig
s390                             alldefconfig
powerpc                      ppc40x_defconfig
mips                       lemote2f_defconfig
mips                   sb1250_swarm_defconfig
powerpc                         ps3_defconfig
arm                       omap2plus_defconfig
mips                     cu1830-neo_defconfig
arc                        vdk_hs38_defconfig
mips                           ip27_defconfig
powerpc                     rainier_defconfig
parisc                           alldefconfig
x86_64                              defconfig
arm                         cm_x300_defconfig
powerpc                 xes_mpc85xx_defconfig
sh                          rsk7269_defconfig
sh                ecovec24-romimage_defconfig
m68k                          atari_defconfig
arc                            hsdk_defconfig
h8300                    h8300h-sim_defconfig
sh                     sh7710voipgw_defconfig
powerpc                        icon_defconfig
sh                           se7721_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                               tinyconfig
i386                                defconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a001-20210204
i386                 randconfig-a005-20210204
i386                 randconfig-a003-20210204
i386                 randconfig-a006-20210204
i386                 randconfig-a002-20210204
i386                 randconfig-a004-20210204
x86_64               randconfig-a013-20210204
x86_64               randconfig-a014-20210204
x86_64               randconfig-a015-20210204
x86_64               randconfig-a011-20210204
x86_64               randconfig-a016-20210204
x86_64               randconfig-a012-20210204
i386                 randconfig-a013-20210204
i386                 randconfig-a016-20210204
i386                 randconfig-a014-20210204
i386                 randconfig-a012-20210204
i386                 randconfig-a015-20210204
i386                 randconfig-a011-20210204
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a006-20210204
x86_64               randconfig-a001-20210204
x86_64               randconfig-a005-20210204
x86_64               randconfig-a002-20210204
x86_64               randconfig-a004-20210204
x86_64               randconfig-a003-20210204

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From robin.murphy at arm.com  Thu Feb  4 17:09:02 2021
From: robin.murphy at arm.com (Robin Murphy)
Date: Thu, 4 Feb 2021 22:09:02 +0000
Subject: [PATCH 3/8] swiotlb: factor out a nr_slots helper
In-Reply-To: <20210204193035.2606838-4-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-4-hch@lst.de>
Message-ID: <ae995e9d-4476-cea2-a4d7-9e9cbab5aafd@arm.com>

On 2021-02-04 19:30, Christoph Hellwig wrote:
> Factor out a helper to find the number of slots for a given size.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>   kernel/dma/swiotlb.c | 13 +++++++++----
>   1 file changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index 838dbad10ab916..0c0b81799edbdb 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -194,6 +194,11 @@ static inline unsigned long io_tlb_offset(unsigned long val)
>   	return val & (IO_TLB_SEGSIZE - 1);
>   }
>   
> +static unsigned long nr_slots(u64 val)
> +{
> +	return ALIGN(val, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;

Would DIV_ROUND_UP(val, 1 << IOTLB_SHIFT) be even clearer?

Robin.

> +}
> +
>   /*
>    * Early SWIOTLB allocation may be too early to allow an architecture to
>    * perform the desired operations.  This function allows the architecture to
> @@ -493,20 +498,20 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
>   
>   	tbl_dma_addr &= mask;
>   
> -	offset_slots = ALIGN(tbl_dma_addr, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
> +	offset_slots = nr_slots(tbl_dma_addr);
>   
>   	/*
>   	 * Carefully handle integer overflow which can occur when mask == ~0UL.
>   	 */
>   	max_slots = mask + 1
> -		    ? ALIGN(mask + 1, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT
> +		    ? nr_slots(mask + 1)
>   		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
>   
>   	/*
>   	 * For mappings greater than or equal to a page, we limit the stride
>   	 * (and hence alignment) to a page size.
>   	 */
> -	nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
> +	nslots = nr_slots(alloc_size);
>   	if (alloc_size >= PAGE_SIZE)
>   		stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
>   	else
> @@ -602,7 +607,7 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
>   			      enum dma_data_direction dir, unsigned long attrs)
>   {
>   	unsigned long flags;
> -	int i, count, nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
> +	int i, count, nslots = nr_slots(alloc_size);
>   	int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
>   	phys_addr_t orig_addr = io_tlb_orig_addr[index];
>   
> 


From robin.murphy at arm.com  Thu Feb  4 17:12:31 2021
From: robin.murphy at arm.com (Robin Murphy)
Date: Thu, 4 Feb 2021 22:12:31 +0000
Subject: [PATCH 5/8] swiotlb: refactor swiotlb_tbl_map_single
In-Reply-To: <20210204193035.2606838-6-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-6-hch@lst.de>
Message-ID: <f60ddc1b-94c3-32a1-06eb-b2187d56d797@arm.com>

On 2021-02-04 19:30, Christoph Hellwig wrote:
> Split out a bunch of a self-contained helpers to make the function easier
> to follow.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>   kernel/dma/swiotlb.c | 177 +++++++++++++++++++++----------------------
>   1 file changed, 87 insertions(+), 90 deletions(-)
> 
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index 79d5b236f25f10..e78615e3be2906 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -468,134 +468,131 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
>   	}
>   }
>   
> -phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
> -		size_t mapping_size, size_t alloc_size,
> -		enum dma_data_direction dir, unsigned long attrs)
> +/*
> + * Carefully handle integer overflow which can occur when boundary_mask == ~0UL.
> + */
> +static inline unsigned long get_max_slots(unsigned long boundary_mask)
>   {
> -	dma_addr_t tbl_dma_addr = phys_to_dma_unencrypted(hwdev, io_tlb_start);
> -	unsigned long flags;
> -	phys_addr_t tlb_addr;
> -	unsigned int nslots, stride, index, wrap;
> -	int i;
> -	unsigned long mask;
> -	unsigned long offset_slots;
> -	unsigned long max_slots;
> -	unsigned long tmp_io_tlb_used;
> -
> -	if (no_iotlb_memory)
> -		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
> -
> -	if (mem_encrypt_active())
> -		pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
> -
> -	if (mapping_size > alloc_size) {
> -		dev_warn_once(hwdev, "Invalid sizes (mapping: %zd bytes, alloc: %zd bytes)",
> -			      mapping_size, alloc_size);
> -		return (phys_addr_t)DMA_MAPPING_ERROR;
> -	}
> -
> -	mask = dma_get_seg_boundary(hwdev);
> +	if (boundary_mask + 1 == ~0UL)

Either "mask == ~0UL" or "mask + 1 == 0", surely?

> +		return 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
> +	return nr_slots(boundary_mask + 1);
> +}
>   
> -	tbl_dma_addr &= mask;
> +static unsigned int wrap_index(unsigned int index)
> +{
> +	if (index >= io_tlb_nslabs)
> +		return 0;
> +	return index;
> +}
>   
> -	offset_slots = nr_slots(tbl_dma_addr);
> +/*
> + * Find a suitable number of IO TLB entries size that will fit this request and
> + * allocate a buffer from that IO TLB pool.
> + */
> +static int find_slots(struct device *dev, size_t alloc_size,
> +		dma_addr_t tbl_dma_addr)
> +{
> +	unsigned int max_slots = get_max_slots(dma_get_seg_boundary(dev));
> +	unsigned int nslots = nr_slots(alloc_size), stride = 1;
> +	unsigned int index, wrap, count = 0, i;
> +	unsigned long flags;
>   
> -	/*
> -	 * Carefully handle integer overflow which can occur when mask == ~0UL.
> -	 */
> -	max_slots = mask + 1
> -		    ? nr_slots(mask + 1)
> -		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);

...since the condition here is effectively the latter.

Robin.


From robin.murphy at arm.com  Thu Feb  4 18:13:45 2021
From: robin.murphy at arm.com (Robin Murphy)
Date: Thu, 4 Feb 2021 23:13:45 +0000
Subject: [PATCH 7/8] swiotlb: respect min_align_mask
In-Reply-To: <20210204193035.2606838-8-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-8-hch@lst.de>
Message-ID: <2e51481c-1591-034c-3476-1a1f8891506a@arm.com>

On 2021-02-04 19:30, Christoph Hellwig wrote:
> Respect the min_align_mask in struct device_dma_parameters in swiotlb.
> 
> There are two parts to it:
>   1) for the lower bits of the alignment inside the io tlb slot, just
>      extent the size of the allocation and leave the start of the slot
>       empty
>   2) for the high bits ensure we find a slot that matches the high bits
>      of the alignment to avoid wasting too much memory
> 
> Based on an earlier patch from Jianxiong Gao <jxgao at google.com>.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>   kernel/dma/swiotlb.c | 49 +++++++++++++++++++++++++++++++++++++-------
>   1 file changed, 42 insertions(+), 7 deletions(-)
> 
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index 6a2439826a1ba4..ab3192142b9906 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -468,6 +468,18 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
>   	}
>   }
>   
> +/*
> + * Return the offset into a iotlb slot required to keep the device happy.
> + */
> +static unsigned int swiotlb_align_offset(struct device *dev, u64 addr)
> +{
> +	unsigned min_align_mask = dma_get_min_align_mask(dev);
> +
> +	if (!min_align_mask)
> +		return 0;

I doubt that's beneficial - even if the compiler can convert it into a 
csel, it'll then be doing unnecessary work to throw away a 
cheaply-calculated 0 in favour of hard-coded 0 in the one case it matters ;)

> +	return addr & min_align_mask & ((1 << IO_TLB_SHIFT) - 1);

(BTW, for readability throughout, "#define IO_TLB_SIZE (1 << 
IO_TLB_SHIFT)" sure wouldn't go amiss...)

> +}
> +
>   /*
>    * Carefully handle integer overflow which can occur when boundary_mask == ~0UL.
>    */
> @@ -478,6 +490,16 @@ static inline unsigned long get_max_slots(unsigned long boundary_mask)
>   	return nr_slots(boundary_mask + 1);
>   }
>   
> +static inline bool check_alignment(phys_addr_t orig_addr,
> +		dma_addr_t tbl_dma_addr, unsigned int index,
> +		unsigned int min_align_mask)
> +{
> +	if (!min_align_mask)
> +		return true;

Ditto - even the 5 or so operations this might skip is unlikely to 
outweigh a branch on anything that matters, and again csel would be a 
net loss since x & 0 == y & 0 is still the correct answer.

> +	return ((tbl_dma_addr + (index << IO_TLB_SHIFT)) & min_align_mask) ==
> +		(orig_addr & min_align_mask);
> +}
> +
>   static unsigned int wrap_index(unsigned int index)
>   {
>   	if (index >= io_tlb_nslabs)
> @@ -489,9 +511,11 @@ static unsigned int wrap_index(unsigned int index)
>    * Find a suitable number of IO TLB entries size that will fit this request and
>    * allocate a buffer from that IO TLB pool.
>    */
> -static int find_slots(struct device *dev, size_t alloc_size,
> -		dma_addr_t tbl_dma_addr)
> +static int find_slots(struct device *dev, phys_addr_t orig_addr,
> +		size_t alloc_size, dma_addr_t tbl_dma_addr)
>   {
> +	unsigned int min_align_mask = dma_get_min_align_mask(dev) &
> +			~((1 << IO_TLB_SHIFT) - 1);
>   	unsigned int max_slots = get_max_slots(dma_get_seg_boundary(dev));
>   	unsigned int nslots = nr_slots(alloc_size), stride = 1;
>   	unsigned int index, wrap, count = 0, i;
> @@ -503,7 +527,9 @@ static int find_slots(struct device *dev, size_t alloc_size,
>   	 * For mappings greater than or equal to a page, we limit the stride
>   	 * (and hence alignment) to a page size.
>   	 */
> -	if (alloc_size >= PAGE_SIZE)
> +	if (min_align_mask)
> +		stride = (min_align_mask + 1) >> IO_TLB_SHIFT;

So this can't underflow because "min_align_mask" is actually just the 
high-order bits representing the number of iotlb slots needed to meet 
the requirement, right? (It took a good 5 minutes to realise this wasn't 
doing what I initially thought it did...)

In that case, a) could the local var be called something like 
iotlb_align_mask to clarify that it's *not* just a copy of the device's 
min_align_mask, and b) maybe just have an unconditional initialisation 
that works either way:

	stride = (min_align_mask >> IO_TLB_SHIFT) + 1;

In fact with that, I think could just mask orig_addr with ~IO_TLB_SIZE 
in the call to check_alignment() below, or shift everything down by 
IO_TLB_SHIFT in check_alignment() itself, instead of mangling 
min_align_mask at all (I'm assuming we do need to ignore the low-order 
bits of orig_addr at this point).

Robin.


From lengchao at huawei.com  Thu Feb  4 21:02:16 2021
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 5 Feb 2021 10:02:16 +0800
Subject: [PATCH v5 0/3] avoid double request completion and IO error
In-Reply-To: <20210204080442.GA30542@lst.de>
References: <20210201034940.18891-1-lengchao@huawei.com>
 <20210203161455.GB4116@lst.de>
 <7d39e0a4-3765-85b8-aab5-b0ded93f8bec@huawei.com>
 <20210204080442.GA30542@lst.de>
Message-ID: <47992bb8-3f37-ac6a-1544-e2cd30caefae@huawei.com>



On 2021/2/4 16:04, Christoph Hellwig wrote:
> On Thu, Feb 04, 2021 at 01:56:34PM +0800, Chao Leng wrote:
>> This looks good to me.
>> Thank you very much.
> 
> Thanks a lot to you!
> 
> Please double check there version here:
> 
> http://git.infradead.org/nvme.git/shortlog/refs/heads/nvme-5.12
Looks good.
> .
> 


From martin.petersen at oracle.com  Thu Feb  4 21:54:25 2021
From: martin.petersen at oracle.com (Martin K. Petersen)
Date: Thu, 04 Feb 2021 21:54:25 -0500
Subject: [PATCH v4 5/8] block: introduce zone_write_granularity limit
In-Reply-To: <20210128044733.503606-6-damien.lemoal@wdc.com> (Damien Le Moal's
 message of "Thu, 28 Jan 2021 13:47:30 +0900")
References: <20210128044733.503606-1-damien.lemoal@wdc.com>
 <20210128044733.503606-6-damien.lemoal@wdc.com>
Message-ID: <yq1y2g3teb3.fsf@ca-mkp.ca.oracle.com>


Damien,

> Per ZBC and ZAC specifications, host-managed SMR hard-disks mandate that
> all writes into sequential write required zones be aligned to the device
> physical block size. However, NVMe ZNS does not have this constraint and
> allows write operations into sequential zones to be aligned to the
> device logical block size. This inconsistency does not help with
> software portability across device types.
>
> To solve this, introduce the zone_write_granularity queue limit to
> indicate the alignment constraint, in bytes, of write operations into
> zones of a zoned block device. This new limit is exported as a
> read-only sysfs queue attribute and the helper
> blk_queue_zone_write_granularity() introduced for drivers to set this
> limit.
>
> The function blk_queue_set_zoned() is modified to set this new limit to
> the device logical block size by default. NVMe ZNS devices as well as
> zoned nullb devices use this default value as is. The scsi disk driver
> is modified to execute the blk_queue_zone_write_granularity() helper to
> set the zone write granularity of host-managed SMR disks to the disk
> physical block size.
>
> The accessor functions queue_zone_write_granularity() and
> bdev_zone_write_granularity() are also introduced.

Looks fine.

Reviewed-by: Martin K. Petersen <martin.petersen at oracle.com>

-- 
Martin K. Petersen	Oracle Linux Engineering


From martin.petersen at oracle.com  Thu Feb  4 21:56:54 2021
From: martin.petersen at oracle.com (Martin K. Petersen)
Date: Thu, 04 Feb 2021 21:56:54 -0500
Subject: [PATCH v4 8/8] sd_zbc: clear zone resources for non-zoned case
In-Reply-To: <20210128044733.503606-9-damien.lemoal@wdc.com> (Damien Le Moal's
 message of "Thu, 28 Jan 2021 13:47:33 +0900")
References: <20210128044733.503606-1-damien.lemoal@wdc.com>
 <20210128044733.503606-9-damien.lemoal@wdc.com>
Message-ID: <yq1sg6bte6w.fsf@ca-mkp.ca.oracle.com>


Damien,

> For host-aware ZBC disk, setting the device zoned model to
> BLK_ZONED_HA using blk_queue_set_zoned() in
> sd_read_block_characteristics() may result in the block device
> effective zoned model to be "none" (BLK_ZONED_NONE) if partitions are
> present on the device. In this case, sd_zbc_read_zones() should not
> setup the zone related queue limits for the disk so that the device
> limits and configuration is consistent with a regular disk and
> resources not uselessly allocated (e.g. the zone write pointer
> tracking array for zone append emulation).
>
> Furthermore, if the disk zoned model changes at run time due to the
> creation of a partition by the user, the zone related resources can be
> released.
>
> Fix both problems by introducing the function sd_zbc_clear_zone_info()
> to reset the scsi disk zone information and free resources and by
> returning early in sd_zbc_read_zones() for a block device that has a
> zoned model equal to BLK_ZONED_NONE.

Reviewed-by: Martin K. Petersen <martin.petersen at oracle.com>

-- 
Martin K. Petersen	Oracle Linux Engineering


From hare at suse.de  Fri Feb  5 04:29:26 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri,  5 Feb 2021 10:29:26 +0100
Subject: [PATCH] nvmet: per-host namespaces masking
Message-ID: <20210205092926.64161-1-hare@suse.de>

Implement per-host namespaces masking, allowing to expose
different namespaces to different hosts connecting to the
same subsystem.
The original method of adding host nqns to the subsystem
takes precedence over the per-host namespace masking.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/target/admin-cmd.c | 16 +++++++
 drivers/nvme/target/configfs.c  | 82 +++++++++++++++++++++++++++++++++
 drivers/nvme/target/core.c      | 21 ++++++++-
 drivers/nvme/target/nvmet.h     |  4 ++
 4 files changed, 122 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 613a4d8feac1..d63fd33200db 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -489,6 +489,13 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 		goto done;
 	}
 
+	if (!nvmet_ns_host_allowed(req->ns, ctrl->hostnqn)) {
+		status = NVME_SC_INVALID_NS;
+		nvmet_put_namespace(req->ns);
+		req->ns = NULL;
+		goto out;
+	}
+
 	nvmet_ns_revalidate(req->ns);
 
 	/*
@@ -566,6 +573,8 @@ static void nvmet_execute_identify_nslist(struct nvmet_req *req)
 	}
 
 	xa_for_each(&ctrl->subsys->namespaces, idx, ns) {
+		if (!nvmet_ns_host_allowed(ns, ctrl->hostnqn))
+			continue;
 		if (ns->nsid <= min_nsid)
 			continue;
 		list[i++] = cpu_to_le32(ns->nsid);
@@ -613,6 +622,13 @@ static void nvmet_execute_identify_desclist(struct nvmet_req *req)
 		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
 		goto out;
 	}
+	if (!nvmet_ns_host_allowed(req->ns, req->sq->ctrl->hostnqn)) {
+		nvmet_put_namespace(req->ns);
+		req->ns = NULL;
+		req->error_loc = offsetof(struct nvme_identify, nsid);
+		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
+		goto out;
+	}
 
 	if (memchr_inv(&req->ns->uuid, 0, sizeof(req->ns->uuid))) {
 		status = nvmet_copy_ns_identifier(req, NVME_NIDT_UUID,
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 635a7cb45d0b..5c85291eb16a 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -607,6 +607,85 @@ static struct configfs_attribute *nvmet_ns_attrs[] = {
 	NULL,
 };
 
+static int nvmet_ns_allowed_hosts_allow_link(struct config_item *parent,
+		struct config_item *target)
+{
+	struct nvmet_ns *ns = to_nvmet_ns(parent->ci_parent);
+	struct nvmet_host *host;
+	struct nvmet_host_link *link, *p;
+	int ret;
+
+	if (target->ci_type != &nvmet_host_type) {
+		pr_err("can only link hosts into the allowed_hosts directory!\n");
+		return -EINVAL;
+	}
+
+	host = to_host(target);
+	link = kmalloc(sizeof(*link), GFP_KERNEL);
+	if (!link)
+		return -ENOMEM;
+	link->host = host;
+
+	down_write(&nvmet_config_sem);
+	ret = -EINVAL;
+	if (ns->subsys->allow_any_host) {
+		pr_err("can't add hosts when allow_any_host is set!\n");
+		goto out_free_link;
+	}
+
+	ret = -EEXIST;
+	list_for_each_entry(p, &ns->subsys->hosts, entry) {
+		if (!strcmp(nvmet_host_name(p->host), nvmet_host_name(host)))
+			goto out_free_link;
+	}
+	list_for_each_entry(p, &ns->hosts, entry) {
+		if (!strcmp(nvmet_host_name(p->host), nvmet_host_name(host)))
+			goto out_free_link;
+	}
+	list_add_tail(&link->entry, &ns->hosts);
+	nvmet_subsys_disc_changed(ns->subsys, host);
+
+	up_write(&nvmet_config_sem);
+	return 0;
+out_free_link:
+	up_write(&nvmet_config_sem);
+	kfree(link);
+	return ret;
+}
+
+static void nvmet_ns_allowed_hosts_drop_link(struct config_item *parent,
+		struct config_item *target)
+{
+	struct nvmet_ns *ns = to_nvmet_ns(parent->ci_parent);
+	struct nvmet_host *host = to_host(target);
+	struct nvmet_host_link *p;
+
+	down_write(&nvmet_config_sem);
+	list_for_each_entry(p, &ns->hosts, entry) {
+		if (!strcmp(nvmet_host_name(p->host), nvmet_host_name(host)))
+			goto found;
+	}
+	up_write(&nvmet_config_sem);
+	return;
+
+found:
+	list_del(&p->entry);
+	nvmet_subsys_disc_changed(ns->subsys, host);
+
+	up_write(&nvmet_config_sem);
+	kfree(p);
+}
+
+static struct configfs_item_operations nvmet_ns_allowed_hosts_item_ops = {
+	.allow_link		= nvmet_ns_allowed_hosts_allow_link,
+	.drop_link		= nvmet_ns_allowed_hosts_drop_link,
+};
+
+static const struct config_item_type nvmet_ns_allowed_hosts_type = {
+	.ct_item_ops		= &nvmet_ns_allowed_hosts_item_ops,
+	.ct_owner		= THIS_MODULE,
+};
+
 static void nvmet_ns_release(struct config_item *item)
 {
 	struct nvmet_ns *ns = to_nvmet_ns(item);
@@ -647,6 +726,9 @@ static struct config_group *nvmet_ns_make(struct config_group *group,
 	if (!ns)
 		goto out;
 	config_group_init_type_name(&ns->group, name, &nvmet_ns_type);
+	config_group_init_type_name(&ns->allowed_hosts_group,
+				    "allowed_hosts", &nvmet_ns_allowed_hosts_type);
+	configfs_add_default_group(&ns->allowed_hosts_group, &ns->group);
 
 	pr_info("adding nsid %d to subsystem %s\n", nsid, subsys->subsysnqn);
 
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 8ce4d59cc9e7..d62a49f8a6e0 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -673,6 +673,7 @@ struct nvmet_ns *nvmet_ns_alloc(struct nvmet_subsys *subsys, u32 nsid)
 
 	ns->nsid = nsid;
 	ns->subsys = subsys;
+	INIT_LIST_HEAD(&ns->hosts);
 
 	down_write(&nvmet_ana_sem);
 	ns->anagrpid = NVMET_DEFAULT_ANA_GRPID;
@@ -1225,9 +1226,24 @@ u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd)
 	return 0;
 }
 
+bool nvmet_ns_host_allowed(struct nvmet_ns *ns, const char *hostnqn)
+{
+	struct nvmet_host_link *p;
+
+	lockdep_assert_held(&nvmet_config_sem);
+
+	list_for_each_entry(p, &ns->hosts, entry) {
+		if (!strcmp(nvmet_host_name(p->host), hostnqn))
+			return true;
+	}
+	return false;
+}
+
 bool nvmet_host_allowed(struct nvmet_subsys *subsys, const char *hostnqn)
 {
 	struct nvmet_host_link *p;
+	struct nvmet_ns *ns;
+	unsigned long idx;
 
 	lockdep_assert_held(&nvmet_config_sem);
 
@@ -1241,7 +1257,10 @@ bool nvmet_host_allowed(struct nvmet_subsys *subsys, const char *hostnqn)
 		if (!strcmp(nvmet_host_name(p->host), hostnqn))
 			return true;
 	}
-
+	xa_for_each(&subsys->namespaces, idx, ns) {
+		if (nvmet_ns_host_allowed(ns, hostnqn))
+			return true;
+	}
 	return false;
 }
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 8776dd1a0490..7d2e5814cfa3 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -70,6 +70,9 @@ struct nvmet_ns {
 	struct nvmet_subsys	*subsys;
 	const char		*device_path;
 
+	struct list_head	hosts;
+
+	struct config_group	allowed_hosts_group;
 	struct config_group	device_group;
 	struct config_group	group;
 
@@ -518,6 +521,7 @@ extern u32 nvmet_ana_group_enabled[NVMET_MAX_ANAGRPS + 1];
 extern u64 nvmet_ana_chgcnt;
 extern struct rw_semaphore nvmet_ana_sem;
 
+bool nvmet_ns_host_allowed(struct nvmet_ns *ns, const char *hostnqn);
 bool nvmet_host_allowed(struct nvmet_subsys *subsys, const char *hostnqn);
 
 int nvmet_bdev_ns_enable(struct nvmet_ns *ns);
-- 
2.29.2



From hch at lst.de  Fri Feb  5 04:32:37 2021
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 5 Feb 2021 10:32:37 +0100
Subject: [PATCH] nvmet: per-host namespaces masking
In-Reply-To: <20210205092926.64161-1-hare@suse.de>
References: <20210205092926.64161-1-hare@suse.de>
Message-ID: <20210205093237.GA2066@lst.de>

On Fri, Feb 05, 2021 at 10:29:26AM +0100, Hannes Reinecke wrote:
> Implement per-host namespaces masking, allowing to expose
> different namespaces to different hosts connecting to the
> same subsystem.
> The original method of adding host nqns to the subsystem
> takes precedence over the per-host namespace masking.

Why do you want to expose a single subsystem to multiple hosts?  This
creates all kinds of covert channels better avoided.


From hare at suse.de  Fri Feb  5 04:38:41 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 5 Feb 2021 10:38:41 +0100
Subject: [PATCH] nvmet: per-host namespaces masking
In-Reply-To: <20210205093237.GA2066@lst.de>
References: <20210205092926.64161-1-hare@suse.de>
 <20210205093237.GA2066@lst.de>
Message-ID: <e505e360-ec8a-ba06-4a40-bf9d2006a7a0@suse.de>

On 2/5/21 10:32 AM, Christoph Hellwig wrote:
> On Fri, Feb 05, 2021 at 10:29:26AM +0100, Hannes Reinecke wrote:
>> Implement per-host namespaces masking, allowing to expose
>> different namespaces to different hosts connecting to the
>> same subsystem.
>> The original method of adding host nqns to the subsystem
>> takes precedence over the per-host namespace masking.
> 
> Why do you want to expose a single subsystem to multiple hosts?  This
> creates all kinds of covert channels better avoided.
> 
But the alternative is a subsystem per host.
Which leads to a proliferation of subsystems on the target.
And I wasn't quite keen on that.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hch at lst.de  Fri Feb  5 04:40:43 2021
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 5 Feb 2021 10:40:43 +0100
Subject: [PATCH] nvmet: per-host namespaces masking
In-Reply-To: <e505e360-ec8a-ba06-4a40-bf9d2006a7a0@suse.de>
References: <20210205092926.64161-1-hare@suse.de>
 <20210205093237.GA2066@lst.de> <e505e360-ec8a-ba06-4a40-bf9d2006a7a0@suse.de>
Message-ID: <20210205094043.GA2694@lst.de>

On Fri, Feb 05, 2021 at 10:38:41AM +0100, Hannes Reinecke wrote:
> On 2/5/21 10:32 AM, Christoph Hellwig wrote:
>> On Fri, Feb 05, 2021 at 10:29:26AM +0100, Hannes Reinecke wrote:
>>> Implement per-host namespaces masking, allowing to expose
>>> different namespaces to different hosts connecting to the
>>> same subsystem.
>>> The original method of adding host nqns to the subsystem
>>> takes precedence over the per-host namespace masking.
>>
>> Why do you want to expose a single subsystem to multiple hosts?  This
>> creates all kinds of covert channels better avoided.
>>
> But the alternative is a subsystem per host.
> Which leads to a proliferation of subsystems on the target.

Yes, that was the choice we had in mind when writing the code.  The
subsystems are very cheap as they don't require any resources except
for some fairly small memory allocations.


From hch at lst.de  Fri Feb  5 04:45:11 2021
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 5 Feb 2021 10:45:11 +0100
Subject: [PATCH 3/8] swiotlb: factor out a nr_slots helper
In-Reply-To: <ae995e9d-4476-cea2-a4d7-9e9cbab5aafd@arm.com>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-4-hch@lst.de>
 <ae995e9d-4476-cea2-a4d7-9e9cbab5aafd@arm.com>
Message-ID: <20210205094511.GA2801@lst.de>

On Thu, Feb 04, 2021 at 10:09:02PM +0000, Robin Murphy wrote:
>> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
>> index 838dbad10ab916..0c0b81799edbdb 100644
>> --- a/kernel/dma/swiotlb.c
>> +++ b/kernel/dma/swiotlb.c
>> @@ -194,6 +194,11 @@ static inline unsigned long io_tlb_offset(unsigned long val)
>>   	return val & (IO_TLB_SEGSIZE - 1);
>>   }
>>   +static unsigned long nr_slots(u64 val)
>> +{
>> +	return ALIGN(val, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
>
> Would DIV_ROUND_UP(val, 1 << IOTLB_SHIFT) be even clearer?

Not sure it is all that much cleaner, but it does fit a common pattern,
so I'll switch to that.


From hch at lst.de  Fri Feb  5 04:45:49 2021
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 5 Feb 2021 10:45:49 +0100
Subject: [PATCH 5/8] swiotlb: refactor swiotlb_tbl_map_single
In-Reply-To: <f60ddc1b-94c3-32a1-06eb-b2187d56d797@arm.com>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-6-hch@lst.de>
 <f60ddc1b-94c3-32a1-06eb-b2187d56d797@arm.com>
Message-ID: <20210205094549.GB2801@lst.de>

On Thu, Feb 04, 2021 at 10:12:31PM +0000, Robin Murphy wrote:
>> -	mask = dma_get_seg_boundary(hwdev);
>> +	if (boundary_mask + 1 == ~0UL)
>
> Either "mask == ~0UL" or "mask + 1 == 0", surely?

I switched forth and back a few times and ended up with the broken
variant in the middle.  Fixed.


From hare at suse.de  Fri Feb  5 04:56:36 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri,  5 Feb 2021 10:56:36 +0100
Subject: [PATCH] nvmetcli: fixup ana groupid setting for namespaces
Message-ID: <20210205095636.65151-1-hare@suse.de>

The ANA group of a namespace is displayed, but can't be set.
So add a new attribute group 'ana' to the namespace such that
it can be set via 'set ana grpid'.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 nvmet/nvme.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/nvmet/nvme.py b/nvmet/nvme.py
index 83fd75b..d170204 100644
--- a/nvmet/nvme.py
+++ b/nvmet/nvme.py
@@ -567,7 +567,7 @@ class Namespace(CFSNode):
             if nsid < 1 or nsid > self.MAX_NSID:
                 raise CFSError("NSID must be 1 to %d" % self.MAX_NSID)
 
-        self.attr_groups = ['device']
+        self.attr_groups = ['device', 'ana']
         self._subsystem = subsystem
         self._nsid = nsid
         self._path = "%s/namespaces/%d" % (self.subsystem.path, self.nsid)
-- 
2.29.2



From hch at lst.de  Fri Feb  5 05:34:17 2021
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 5 Feb 2021 11:34:17 +0100
Subject: [PATCH 7/8] swiotlb: respect min_align_mask
In-Reply-To: <2e51481c-1591-034c-3476-1a1f8891506a@arm.com>
References: <20210204193035.2606838-1-hch@lst.de>
 <20210204193035.2606838-8-hch@lst.de>
 <2e51481c-1591-034c-3476-1a1f8891506a@arm.com>
Message-ID: <20210205103417.GA6694@lst.de>

On Thu, Feb 04, 2021 at 11:13:45PM +0000, Robin Murphy wrote:
>> + */
>> +static unsigned int swiotlb_align_offset(struct device *dev, u64 addr)
>> +{
>> +	unsigned min_align_mask = dma_get_min_align_mask(dev);
>> +
>> +	if (!min_align_mask)
>> +		return 0;
>
> I doubt that's beneficial - even if the compiler can convert it into a 
> csel, it'll then be doing unnecessary work to throw away a 
> cheaply-calculated 0 in favour of hard-coded 0 in the one case it matters 

True, I'll drop the checks.

> ;)
>
>> +	return addr & min_align_mask & ((1 << IO_TLB_SHIFT) - 1);
>
> (BTW, for readability throughout, "#define IO_TLB_SIZE (1 << IO_TLB_SHIFT)" 
> sure wouldn't go amiss...)

I actually had a patch doing just that, but as it is the only patch
touching swiotlb.h it caused endless rebuilds for me, so I dropped it
as it only had a few uses anyway.  But I've added it back.

>> -	if (alloc_size >= PAGE_SIZE)
>> +	if (min_align_mask)
>> +		stride = (min_align_mask + 1) >> IO_TLB_SHIFT;
>
> So this can't underflow because "min_align_mask" is actually just the 
> high-order bits representing the number of iotlb slots needed to meet the 
> requirement, right? (It took a good 5 minutes to realise this wasn't doing 
> what I initially thought it did...)

Yes.

> In that case, a) could the local var be called something like 
> iotlb_align_mask to clarify that it's *not* just a copy of the device's 
> min_align_mask,

Ok.

> and b) maybe just have an unconditional initialisation that 
> works either way:
>
> 	stride = (min_align_mask >> IO_TLB_SHIFT) + 1;

Sure.

> In fact with that, I think could just mask orig_addr with ~IO_TLB_SIZE in 
> the call to check_alignment() below, or shift everything down by 
> IO_TLB_SHIFT in check_alignment() itself, instead of mangling 
> min_align_mask at all (I'm assuming we do need to ignore the low-order bits 
> of orig_addr at this point).

Yes, we do need to ignore the low bits as they won't ever be set in
tlb_dma_addr.  Not sure the shift helps as we need to mask first.

I ended up killing check_alignment entirely, in favor of a new
slot_addr helper that calculates the address based off the base and index
and which can be used in a few other places as this one.


From hch at lst.de  Fri Feb  5 06:50:33 2021
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 5 Feb 2021 12:50:33 +0100
Subject: preserve DMA offsets when using swiotlb
In-Reply-To: <20210204193035.2606838-1-hch@lst.de>
References: <20210204193035.2606838-1-hch@lst.de>
Message-ID: <20210205115033.GA12623@lst.de>

I've pushed the updated series out to:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/swiotlb-offset

but I'm going to wait until next week before patch bombing everyone
again.


From hare at suse.de  Fri Feb  5 10:47:14 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri,  5 Feb 2021 16:47:14 +0100
Subject: [PATCH] nvme: move NVMF_OPT_CTRL_LOSS_TMO to generic list of allowed
 options
Message-ID: <20210205154714.127995-1-hare@suse.de>

Every single fabrics driver with the exception of loop is setting
NVMF_OPT_CTRL_LOSS_TMO, and recent nvme-cli has begun to set this
in the default options when create a discovery controller.
As it's not listed for loop connections to the loop controller will
fail.
So move it to the generic options to avoid this failure.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/fabrics.c | 1 +
 drivers/nvme/host/fc.c      | 2 +-
 drivers/nvme/host/rdma.c    | 2 +-
 drivers/nvme/host/tcp.c     | 2 +-
 4 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index 72ac00173500..e752acfd85a4 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -1007,6 +1007,7 @@ EXPORT_SYMBOL_GPL(nvmf_free_options);
 				 NVMF_OPT_KATO | NVMF_OPT_HOSTNQN | \
 				 NVMF_OPT_HOST_ID | NVMF_OPT_DUP_CONNECT |\
 				 NVMF_OPT_DISABLE_SQFLOW |\
+				 NVMF_OPT_CTRL_LOSS_TMO |\
 				 NVMF_OPT_FAIL_FAST_TMO)
 
 static struct nvme_ctrl *
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index bcc086fcbe61..2348722ed276 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3721,7 +3721,7 @@ static struct nvmf_transport_ops nvme_fc_transport = {
 	.name		= "fc",
 	.module		= THIS_MODULE,
 	.required_opts	= NVMF_OPT_TRADDR | NVMF_OPT_HOST_TRADDR,
-	.allowed_opts	= NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO,
+	.allowed_opts	= NVMF_OPT_RECONNECT_DELAY,
 	.create_ctrl	= nvme_fc_create_ctrl,
 };
 
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 634ed5f5f778..80f5740ca9c1 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -2410,7 +2410,7 @@ static struct nvmf_transport_ops nvme_rdma_transport = {
 	.module		= THIS_MODULE,
 	.required_opts	= NVMF_OPT_TRADDR,
 	.allowed_opts	= NVMF_OPT_TRSVCID | NVMF_OPT_RECONNECT_DELAY |
-			  NVMF_OPT_HOST_TRADDR | NVMF_OPT_CTRL_LOSS_TMO |
+			  NVMF_OPT_HOST_TRADDR |
 			  NVMF_OPT_NR_WRITE_QUEUES | NVMF_OPT_NR_POLL_QUEUES |
 			  NVMF_OPT_TOS,
 	.create_ctrl	= nvme_rdma_create_ctrl,
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index eee9470bb1bd..94ab52045dfc 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2544,7 +2544,7 @@ static struct nvmf_transport_ops nvme_tcp_transport = {
 	.module		= THIS_MODULE,
 	.required_opts	= NVMF_OPT_TRADDR,
 	.allowed_opts	= NVMF_OPT_TRSVCID | NVMF_OPT_RECONNECT_DELAY |
-			  NVMF_OPT_HOST_TRADDR | NVMF_OPT_CTRL_LOSS_TMO |
+			  NVMF_OPT_HOST_TRADDR |
 			  NVMF_OPT_HDR_DIGEST | NVMF_OPT_DATA_DIGEST |
 			  NVMF_OPT_NR_WRITE_QUEUES | NVMF_OPT_NR_POLL_QUEUES |
 			  NVMF_OPT_TOS,
-- 
2.29.2



From hch at lst.de  Fri Feb  5 11:25:06 2021
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 5 Feb 2021 17:25:06 +0100
Subject: [PATCH] nvme: move NVMF_OPT_CTRL_LOSS_TMO to generic list of
 allowed options
In-Reply-To: <20210205154714.127995-1-hare@suse.de>
References: <20210205154714.127995-1-hare@suse.de>
Message-ID: <20210205162506.GA27415@lst.de>

On Fri, Feb 05, 2021 at 04:47:14PM +0100, Hannes Reinecke wrote:
> Every single fabrics driver with the exception of loop is setting
> NVMF_OPT_CTRL_LOSS_TMO, and recent nvme-cli has begun to set this
> in the default options when create a discovery controller.
> As it's not listed for loop connections to the loop controller will
> fail.
> So move it to the generic options to avoid this failure.

Well, the option doesn't really make sense for loop.  I think nvme-cli
needs to be fixed to not set in this case.


From hare at suse.de  Fri Feb  5 11:35:07 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 5 Feb 2021 17:35:07 +0100
Subject: [PATCH] nvme: move NVMF_OPT_CTRL_LOSS_TMO to generic list of
 allowed options
In-Reply-To: <20210205162506.GA27415@lst.de>
References: <20210205154714.127995-1-hare@suse.de>
 <20210205162506.GA27415@lst.de>
Message-ID: <14420650-7548-3897-3baf-a8c303165842@suse.de>

On 2/5/21 5:25 PM, Christoph Hellwig wrote:
> On Fri, Feb 05, 2021 at 04:47:14PM +0100, Hannes Reinecke wrote:
>> Every single fabrics driver with the exception of loop is setting
>> NVMF_OPT_CTRL_LOSS_TMO, and recent nvme-cli has begun to set this
>> in the default options when create a discovery controller.
>> As it's not listed for loop connections to the loop controller will
>> fail.
>> So move it to the generic options to avoid this failure.
> 
> Well, the option doesn't really make sense for loop.  I think nvme-cli
> needs to be fixed to not set in this case.
> 
Neither does NVMF_OPT_FAIL_FAST_TMO, yet there it is ...

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From sagi at grimberg.me  Fri Feb  5 13:42:37 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 5 Feb 2021 10:42:37 -0800
Subject: [PATCH] nvmet: per-host namespaces masking
In-Reply-To: <20210205092926.64161-1-hare@suse.de>
References: <20210205092926.64161-1-hare@suse.de>
Message-ID: <6be00c89-90cb-1bfb-a3a3-33949bf5dc84@grimberg.me>


> Implement per-host namespaces masking, allowing to expose
> different namespaces to different hosts connecting to the
> same subsystem.
> The original method of adding host nqns to the subsystem
> takes precedence over the per-host namespace masking.

What is preventing a host to go ahead and issue an I/O command
to a ns that exists in the subsystem but is not exposed to
it in the nslist?

I don't think this can be even considered without a protection
against it. And given that you don't want to take the global
semaphore on each and every I/O, I'd say its a bit more involving
than what you have here...


From Chaitanya.Kulkarni at wdc.com  Fri Feb  5 13:47:04 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Fri, 5 Feb 2021 18:47:04 +0000
Subject: [PATCH] nvme: move NVMF_OPT_CTRL_LOSS_TMO to generic list of
 allowed options
In-Reply-To: <14420650-7548-3897-3baf-a8c303165842@suse.de>
References: <20210205154714.127995-1-hare@suse.de>
 <20210205162506.GA27415@lst.de>, <14420650-7548-3897-3baf-a8c303165842@suse.de>
Message-ID: <72432D31-3061-494B-8D86-AD28A6CCEA43@wdc.com>


> On Feb 5, 2021, at 8:45 AM, Hannes Reinecke <hare at suse.de> wrote:
> 
> ?On 2/5/21 5:25 PM, Christoph Hellwig wrote:
>>> On Fri, Feb 05, 2021 at 04:47:14PM +0100, Hannes Reinecke wrote:
>>> Every single fabrics driver with the exception of loop is setting
>>> NVMF_OPT_CTRL_LOSS_TMO, and recent nvme-cli has begun to set this
>>> in the default options when create a discovery controller.
>>> As it's not listed for loop connections to the loop controller will
>>> fail.
>>> So move it to the generic options to avoid this failure.
>> Well, the option doesn't really make sense for loop.  I think nvme-cli
>> needs to be fixed to not set in this case.
> Neither does NVMF_OPT_FAIL_FAST_TMO, yet there it is ...
> 

I have already sent a patch to fix this and we have decided to fix the issue with NVMe-cli. 


> Cheers,
> 
> Hannes
> -- 
> Dr. Hannes Reinecke                Kernel Storage Architect
> hare at suse.de                              +49 911 74053 688
> SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
> HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

From sagi at grimberg.me  Fri Feb  5 13:49:33 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 5 Feb 2021 10:49:33 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
 <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>
 <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>
Message-ID: <2251aa94-0d6c-1405-b1d8-185bcd714d8c@grimberg.me>


>> Hi Sagi,
>>
>> I was able to capture a bit more info. Again, because my computer 
>> freezes (i.e. no keyboard or mouse response) I could only take a 
>> picture of the screen (attached). This does say NULL pointer dereference.
>>
>> To answer your questions:
>>
>> Q)?Does this happen if you directly connect to a normal nvme controller?
>> A) I still need to try this. All I know is that it doesn't happen when 
>> the controller returns?OAES[31]=0.
>>
>> Q)?What is your target implementation? Is this the nvme target thatis 
>> built into Linux?
>> A) I work for Dell, and I'm on the same team as Douglas Farley and 
>> Erik Smith, whom I believe you've had discussions with. Dell is 
>> currently developing a Central Discovery Controller (CDC). That's the 
>> target I've been testing with. I need to find different targets I can 
>> test with but working from home makes it a bit difficult. I will check 
>> with my team tomorrow to see if I can connect to a different target.
>>
>> Q)?Can you share your kernel config file?
>> Q) Can you run gdb on vmlinux and share what this provides?
>> A) I'll get that tomorrow (both questions).
> 
> For the record, I compiled Fedora 33 kernel and ran it on my VM
> and it doesn't happen. I also modified Linux nvme target to
> report the same limits as your target and still it doesn't happen.
> -- 
> [? 162.006049] nvme nvme0: Failed to read smart log (error 24577)
> [? 162.006060] nvme nvme0: queue_size 128 > ctrl sqsize 32, clamping down
> [? 162.006063] nvme nvme0: sqsize 32 > ctrl maxcmd 31, clamping down
> [? 162.006778] nvme nvme0: new ctrl: NQN 
> "nqn.2014-08.org.nvmexpress.discovery", addr 192.168.123.1:8009
> [? 162.008939] nvme nvme0: Removing ctrl: NQN 
> "nqn.2014-08.org.nvmexpress.discovery"
> -- 
> 
> So something here seems to be specific to your env.

Hey Martin, any updates with this?


From sagi at grimberg.me  Fri Feb  5 14:38:59 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 5 Feb 2021 11:38:59 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <SJ0PR19MB45442A0B8E171B55A8AACACEF2B29@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
 <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>
 <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>
 <2251aa94-0d6c-1405-b1d8-185bcd714d8c@grimberg.me>
 <SJ0PR19MB45442A0B8E171B55A8AACACEF2B29@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <8937148e-f667-9a05-3828-58114dca6da3@grimberg.me>


> Hi Sagi,
> 
> I tried with nvmet-tcp and, just like you, I don't see a problem. So now 
> I'm doing packet captures and comparing the differences between nvmet 
> and the new cdc we're working on. I'm also asking the cdc developers for 
> their help. I will let you know when I can clearlyidentify which 
> packets/bytes/bits are causing nvme-tcp to crash.

Thanks Martin, obviously we have an issue because no matter what the
target does, it should not trigger a host crash.


From sagi at grimberg.me  Fri Feb  5 14:47:25 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri,  5 Feb 2021 11:47:25 -0800
Subject: [PATCH] nvmet-tcp: fix potential race of tcp socket closing
 accept_work
Message-ID: <20210205194725.625035-1-sagi@grimberg.me>

When we accept a TCP connection and allocate an nvmet-tcp queue we should
make sure not to fully establish it or reference it as the connection may
be already closing, which triggers queue release work, which does not
fence against queue establishment.

In order to address such a race, we make sure to check the sk_state and
contain the queue reference to be done underneath the sk_callback_lock
such that the queue release work correctly fences against it.

Fixes: 872d26a391da ("nvmet-tcp: add NVMe over TCP target driver")
Reported-by: Elad Grupi <elad.grupi at dell.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/target/tcp.c | 28 ++++++++++++++++++----------
 1 file changed, 18 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index 577ce7d403ae..8b0485ada315 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1485,17 +1485,27 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 	if (inet->rcv_tos > 0)
 		ip_sock_set_tos(sock->sk, inet->rcv_tos);
 
+	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
-	sock->sk->sk_user_data = queue;
-	queue->data_ready = sock->sk->sk_data_ready;
-	sock->sk->sk_data_ready = nvmet_tcp_data_ready;
-	queue->state_change = sock->sk->sk_state_change;
-	sock->sk->sk_state_change = nvmet_tcp_state_change;
-	queue->write_space = sock->sk->sk_write_space;
-	sock->sk->sk_write_space = nvmet_tcp_write_space;
+	if (sock->sk->sk_state != TCP_ESTABLISHED) {
+		/*
+		 * If the socket is already closing, don't even start
+		 * consuming it
+		 */
+		ret = -ENOTCONN;
+	} else {
+		sock->sk->sk_user_data = queue;
+		queue->data_ready = sock->sk->sk_data_ready;
+		sock->sk->sk_data_ready = nvmet_tcp_data_ready;
+		queue->state_change = sock->sk->sk_state_change;
+		sock->sk->sk_state_change = nvmet_tcp_state_change;
+		queue->write_space = sock->sk->sk_write_space;
+		sock->sk->sk_write_space = nvmet_tcp_write_space;
+		queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);
+	}
 	write_unlock_bh(&sock->sk->sk_callback_lock);
 
-	return 0;
+	return ret;
 }
 
 static int nvmet_tcp_alloc_queue(struct nvmet_tcp_port *port,
@@ -1543,8 +1553,6 @@ static int nvmet_tcp_alloc_queue(struct nvmet_tcp_port *port,
 	if (ret)
 		goto out_destroy_sq;
 
-	queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);
-
 	return 0;
 out_destroy_sq:
 	mutex_lock(&nvmet_tcp_queue_mutex);
-- 
2.27.0



From kbusch at kernel.org  Fri Feb  5 14:50:02 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri,  5 Feb 2021 11:50:02 -0800
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
Message-ID: <20210205195002.2657209-1-kbusch@kernel.org>

The bio based drivers only require the request_queue's nr_zones is set,
so set this field in the head if the namespace path is zoned.

Reported-by: Minwoo Im <minwoo.im.dev at gmail.com>
Cc: Damien Le Moal <damien.lemoal at wdc.com>
Signed-off-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/multipath.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 1427c9555cef..a1d476e1ac02 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -677,6 +677,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
 	if (blk_queue_stable_writes(ns->queue) && ns->head->disk)
 		blk_queue_flag_set(QUEUE_FLAG_STABLE_WRITES,
 				   ns->head->disk->queue);
+#ifdef CONFIG_BLK_DEV_ZONED
+	if (blk_queue_is_zoned(ns->queue) && ns->head->disk)
+		ns->head->disk->queue->nr_zones = ns->queue->nr_zones;
+#endif
 }
 
 void nvme_mpath_remove_disk(struct nvme_ns_head *head)
-- 
2.25.4



From yi.zhang at redhat.com  Sat Feb  6 23:50:01 2021
From: yi.zhang at redhat.com (Yi Zhang)
Date: Sun, 7 Feb 2021 12:50:01 +0800
Subject: kernel null pointer at nvme_tcp_init_iter[nvme_tcp] with blktests
 nvme-tcp/012
In-Reply-To: <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
Message-ID: <80b7184e-cdfb-cebc-fe07-a228ce57a9e7@redhat.com>


The issue was introduced after merge the NVMe updates

commit 0fd6456fd1f4c8f3ec5a2df6ed7f34458a180409 (HEAD)
Merge: 44d10e4b2f2c 0d7389718c32
Author: Jens Axboe <axboe at kernel.dk>
Date:?? Tue Feb 2 07:12:06 2021 -0700

 ??? Merge branch 'for-5.12/drivers' into for-next

 ??? * for-5.12/drivers: (22 commits)
 ????? nvme-tcp: use cancel tagset helper for tear down
 ????? nvme-rdma: use cancel tagset helper for tear down
 ????? nvme-tcp: add clean action for failed reconnection
 ????? nvme-rdma: add clean action for failed reconnection
 ????? nvme-core: add cancel tagset helpers
 ????? nvme-core: get rid of the extra space
 ????? nvme: add tracing of zns commands
 ????? nvme: parse format nvm command details when tracing
 ????? nvme: update enumerations for status codes
 ????? nvmet: add lba to sect conversion helpers
 ????? nvmet: remove extra variable in identify ns
 ????? nvmet: remove extra variable in id-desclist
 ????? nvmet: remove extra variable in smart log nsid
 ????? nvme: refactor ns->ctrl by request
 ????? nvme-tcp: pass multipage bvec to request iov_iter
 ????? nvme-tcp: get rid of unused helper function
 ????? nvme-tcp: fix wrong setting of request iov_iter
 ????? nvme: support command retry delay for admin command
 ????? nvme: constify static attribute_group structs
 ????? nvmet-fc: use RCU proctection for assoc_list


On 2/6/21 11:08 AM, Yi Zhang wrote:
> blktests nvme-tcp/012



From Chaitanya.Kulkarni at wdc.com  Sun Feb  7 00:25:03 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 7 Feb 2021 05:25:03 +0000
Subject: kernel null pointer at nvme_tcp_init_iter[nvme_tcp] with blktests
 nvme-tcp/012
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <80b7184e-cdfb-cebc-fe07-a228ce57a9e7@redhat.com>
Message-ID: <BYAPR04MB49654E940FACB778A787B68E86B09@BYAPR04MB4965.namprd04.prod.outlook.com>

Yi,

On 2/6/21 20:51, Yi Zhang wrote:
> The issue was introduced after merge the NVMe updates
>
> commit 0fd6456fd1f4c8f3ec5a2df6ed7f34458a180409 (HEAD)
> Merge: 44d10e4b2f2c 0d7389718c32
> Author: Jens Axboe <axboe at kernel.dk>
> Date:   Tue Feb 2 07:12:06 2021 -0700
>
>      Merge branch 'for-5.12/drivers' into for-next
>
>      * for-5.12/drivers: (22 commits)
>        nvme-tcp: use cancel tagset helper for tear down
>        nvme-rdma: use cancel tagset helper for tear down
>        nvme-tcp: add clean action for failed reconnection
>        nvme-rdma: add clean action for failed reconnection
>        nvme-core: add cancel tagset helpers
>        nvme-core: get rid of the extra space
>        nvme: add tracing of zns commands
>        nvme: parse format nvm command details when tracing
>        nvme: update enumerations for status codes
>        nvmet: add lba to sect conversion helpers
>        nvmet: remove extra variable in identify ns
>        nvmet: remove extra variable in id-desclist
>        nvmet: remove extra variable in smart log nsid
>        nvme: refactor ns->ctrl by request
>        nvme-tcp: pass multipage bvec to request iov_iter
>        nvme-tcp: get rid of unused helper function
>        nvme-tcp: fix wrong setting of request iov_iter
>        nvme: support command retry delay for admin command
>        nvme: constify static attribute_group structs
>        nvmet-fc: use RCU proctection for assoc_list
>
>
> On 2/6/21 11:08 AM, Yi Zhang wrote:
>> blktests nvme-tcp/012
Thanks for reporting, you can further bisect this using following
NVMe tree if is from NVMEe tree :- http://git.infradead.org/nvme.git
Branch :- 5.12

Looking at the commit log it seems like nvme-tcp commits might be
the issue given that you got the problem in nvme_tcp_init_iter()
and just eliminating by the looking at the code
commit 9f99a29e5307 can lead to this behavior.

*I may be completely wrong as I'm not expert in the nvme-tcp host.*




From Chaitanya.Kulkarni at wdc.com  Sun Feb  7 00:48:22 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 7 Feb 2021 05:48:22 +0000
Subject: kernel null pointer at nvme_tcp_init_iter[nvme_tcp] with blktests
 nvme-tcp/012
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <80b7184e-cdfb-cebc-fe07-a228ce57a9e7@redhat.com>
Message-ID: <BYAPR04MB4965A152E9796C95C72C80D186B09@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/6/21 20:51, Yi Zhang wrote:
> The issue was introduced after merge the NVMe updates
>
> commit 0fd6456fd1f4c8f3ec5a2df6ed7f34458a180409 (HEAD)
> Merge: 44d10e4b2f2c 0d7389718c32
> Author: Jens Axboe <axboe at kernel.dk>
> Date:   Tue Feb 2 07:12:06 2021 -0700
>
>      Merge branch 'for-5.12/drivers' into for-next
>
>      * for-5.12/drivers: (22 commits)
>        nvme-tcp: use cancel tagset helper for tear down
>        nvme-rdma: use cancel tagset helper for tear down
>        nvme-tcp: add clean action for failed reconnection
>        nvme-rdma: add clean action for failed reconnection
>        nvme-core: add cancel tagset helpers
>        nvme-core: get rid of the extra space
>        nvme: add tracing of zns commands
>        nvme: parse format nvm command details when tracing
>        nvme: update enumerations for status codes
>        nvmet: add lba to sect conversion helpers
>        nvmet: remove extra variable in identify ns
>        nvmet: remove extra variable in id-desclist
>        nvmet: remove extra variable in smart log nsid
>        nvme: refactor ns->ctrl by request
>        nvme-tcp: pass multipage bvec to request iov_iter
>        nvme-tcp: get rid of unused helper function
>        nvme-tcp: fix wrong setting of request iov_iter
>        nvme: support command retry delay for admin command
>        nvme: constify static attribute_group structs
>        nvmet-fc: use RCU proctection for assoc_list
>
>
> On 2/6/21 11:08 AM, Yi Zhang wrote:
>> blktests nvme-tcp/012
>
Running the test few times I got this once no sign of Oops though  :-


# nvme_trtype=tcp ./check nvme/012
nvme/012 (run mkfs and data verification fio job on NVMeOF block
device-backed ns) [failed]
    runtime  37.624s  ...  42.272s
    something found in dmesg:
    [   69.198819] run blktests nvme/012 at 2021-02-06 21:32:23
    [   69.330277] loop: module loaded
    [   69.333383] loop0: detected capacity change from 2097152 to 0
    [   69.351091] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
    [   69.372439] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
    [   69.396581] nvmet: creating controller 1 for subsystem
blktests-subsystem-1 for NQN
nqn.2014-08.org.nvmexpress:uuid:e4cfc949-8f19-4db2-a232-ab360b79204a.
    [   69.399482] nvme nvme1: creating 64 I/O queues.
    [   69.414721] nvme nvme1: mapped 64/0/0 default/read/poll queues.
    [   69.448193] nvme nvme1: new ctrl: NQN "blktests-subsystem-1",
addr 127.0.0.1:4420
    [   72.381282] XFS (nvme1n1): Mounting V5 Filesystem
    ...
    (See '/root/blktests/results/nodev/nvme/012.dmesg' for the entire
message)
blktests (master) # dmesg  -c
[   38.347661] nvme nvme0: 63/0/0 default/read/poll queues
[   69.198819] run blktests nvme/012 at 2021-02-06 21:32:23
[   69.330277] loop: module loaded
[   69.333383] loop0: detected capacity change from 2097152 to 0
[   69.351091] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[   69.372439] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[   69.396581] nvmet: creating controller 1 for subsystem
blktests-subsystem-1 for NQN
nqn.2014-08.org.nvmexpress:uuid:e4cfc949-8f19-4db2-a232-ab360b79204a.
[   69.399482] nvme nvme1: creating 64 I/O queues.
[   69.414721] nvme nvme1: mapped 64/0/0 default/read/poll queues.
[   69.448193] nvme nvme1: new ctrl: NQN "blktests-subsystem-1", addr
127.0.0.1:4420
[   72.381282] XFS (nvme1n1): Mounting V5 Filesystem
[   72.394603] XFS (nvme1n1): Ending clean mount
[   72.395463] xfs filesystem being mounted at /mnt/blktests supports
timestamps until 2038 (0x7fffffff)

[   73.402838] ======================================================
[   73.403491] WARNING: possible circular locking dependency detected
[   73.404125] 5.11.0-rc6blk+ #166 Tainted: G           OE   
[   73.404690] ------------------------------------------------------
[   73.405332] fio/3671 is trying to acquire lock:
[   73.405803] ffff88881d110cb0 (sk_lock-AF_INET){+.+.}-{0:0}, at:
tcp_sendpage+0x23/0x50
[   73.406627]
but task is already holding lock:
[   73.407246] ffff888128cabcf0 (&xfs_dir_ilock_class/5){+.+.}-{3:3},
at: xfs_ilock+0xbf/0x250 [xfs]
[   73.408233]
which lock already depends on the new lock.

[   73.409096]
the existing dependency chain (in reverse order) is:
[   73.409885]
-> #3 (&xfs_dir_ilock_class/5){+.+.}-{3:3}:
[   73.410594]        lock_acquire+0xd2/0x390
[   73.411045]        down_write_nested+0x47/0x110
[   73.411530]        xfs_ilock+0xbf/0x250 [xfs]
[   73.412068]        xfs_create+0x1d9/0x6b0 [xfs]
[   73.412626]        xfs_generic_create+0x205/0x2c0 [xfs]
[   73.413255]        lookup_open+0x4f6/0x630
[   73.413710]        path_openat+0x298/0xa80
[   73.414158]        do_filp_open+0x93/0x100
[   73.414606]        do_sys_openat2+0x24b/0x310
[   73.415090]        do_sys_open+0x4b/0x80
[   73.415569]        do_syscall_64+0x33/0x40
[   73.416075]        entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   73.416747]
-> #2 (sb_internal){.+.+}-{0:0}:
[   73.417436]        lock_acquire+0xd2/0x390
[   73.417938]        xfs_trans_alloc+0x19e/0x2e0 [xfs]
[   73.418607]        xfs_vn_update_time+0xc8/0x250 [xfs]
[   73.419295]        touch_atime+0x16b/0x200
[   73.419814]        xfs_file_mmap+0xa7/0xb0 [xfs]
[   73.420447]        mmap_region+0x3f6/0x690
[   73.420955]        do_mmap+0x379/0x580
[   73.421415]        vm_mmap_pgoff+0xdf/0x170
[   73.421929]        ksys_mmap_pgoff+0x1dd/0x240
[   73.422477]        do_syscall_64+0x33/0x40
[   73.422980]        entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   73.423672]
-> #1 (&mm->mmap_lock#2){++++}-{3:3}:
[   73.424426]        lock_acquire+0xd2/0x390
[   73.424932]        __might_fault+0x5e/0x80
[   73.425433]        _copy_from_user+0x1e/0xa0
[   73.425966]        do_ip_setsockopt.isra.15+0xbba/0x12f0
[   73.426616]        ip_setsockopt+0x34/0x90
[   73.427120]        __sys_setsockopt+0x8f/0x110
[   73.427681]        __x64_sys_setsockopt+0x20/0x30
[   73.428252]        do_syscall_64+0x33/0x40
[   73.428751]        entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   73.429422]
-> #0 (sk_lock-AF_INET){+.+.}-{0:0}:
[   73.430154]        validate_chain+0x8fa/0xec0
[   73.430681]        __lock_acquire+0x563/0x940
[   73.431212]        lock_acquire+0xd2/0x390
[   73.431719]        lock_sock_nested+0x72/0xa0
[   73.432252]        tcp_sendpage+0x23/0x50
[   73.432740]        inet_sendpage+0x4f/0x80
[   73.433240]        kernel_sendpage+0x57/0xc0
[   73.433760]        nvme_tcp_try_send+0x137/0x7d0 [nvme_tcp]
[   73.434435]        nvme_tcp_queue_rq+0x317/0x330 [nvme_tcp]
[   73.435104]        __blk_mq_try_issue_directly+0x109/0x1b0
[   73.435768]        blk_mq_request_issue_directly+0x4b/0x80
[   73.436433]        blk_mq_try_issue_list_directly+0x62/0xf0
[   73.437104]        blk_mq_sched_insert_requests+0x192/0x2b0
[   73.437774]        blk_mq_flush_plug_list+0x13c/0x260
[   73.438389]        blk_flush_plug_list+0xd7/0x100
[   73.438960]        blk_finish_plug+0x21/0x30
[   73.439485]        _xfs_buf_ioapply+0x2cc/0x3c0 [xfs]
[   73.440171]        __xfs_buf_submit+0x85/0x220 [xfs]
[   73.440833]        xfs_buf_read_map+0x105/0x2a0 [xfs]
[   73.441511]        xfs_trans_read_buf_map+0x2b2/0x610 [xfs]
[   73.442254]        xfs_read_agi+0xb4/0x1d0 [xfs]
[   73.442874]        xfs_ialloc_read_agi+0x48/0x170 [xfs]
[   73.443568]        xfs_dialloc_select_ag+0x94/0x2a0 [xfs]
[   73.444277]        xfs_dir_ialloc+0x72/0x630 [xfs]
[   73.444927]        xfs_create+0x241/0x6b0 [xfs]
[   73.445550]        xfs_generic_create+0x205/0x2c0 [xfs]
[   73.446255]        lookup_open+0x4f6/0x630
[   73.446759]        path_openat+0x298/0xa80
[   73.447268]        do_filp_open+0x93/0x100
[   73.447770]        do_sys_openat2+0x24b/0x310
[   73.448301]        do_sys_open+0x4b/0x80
[   73.448779]        do_syscall_64+0x33/0x40
[   73.449275]        entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   73.449944]
other info that might help us debug this:

[   73.450899] Chain exists of:
  sk_lock-AF_INET --> sb_internal --> &xfs_dir_ilock_class/5

[   73.452222]  Possible unsafe locking scenario:

[   73.452931]        CPU0                    CPU1
[   73.453479]        ----                    ----
[   73.454024]   lock(&xfs_dir_ilock_class/5);
[   73.454528]                                lock(sb_internal);
[   73.455217]                                lock(&xfs_dir_ilock_class/5);
[   73.456027]   lock(sk_lock-AF_INET);
[   73.456463]
 *** DEADLOCK ***

[   73.457175] 6 locks held by fio/3671:
[   73.457618]  #0: ffff8881185a9488 (sb_writers#9){.+.+}-{0:0}, at:
path_openat+0xa61/0xa80
[   73.458604]  #1: ffff888128cabfe0
(&inode->i_sb->s_type->i_mutex_dir_key){++++}-{3:3}, at:
path_openat+0x287/0xa80
[   73.459847]  #2: ffff8881185a96a8 (sb_internal){.+.+}-{0:0}, at:
xfs_create+0x1b4/0x6b0 [xfs]
[   73.460926]  #3: ffff888128cabcf0
(&xfs_dir_ilock_class/5){+.+.}-{3:3}, at: xfs_ilock+0xbf/0x250 [xfs]
[   73.462100]  #4: ffff88881bd54c58 (hctx->srcu){....}-{0:0}, at:
hctx_lock+0x62/0xe0
[   73.463029]  #5: ffff888142ab29d0 (&queue->send_mutex){+.+.}-{3:3},
at: nvme_tcp_queue_rq+0x2fc/0x330 [nvme_tcp]
[   73.464288]
stack backtrace:
[   73.464824] CPU: 16 PID: 3671 Comm: fio Tainted: G           OE    
5.11.0-rc6blk+ #166
[   73.465785] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
[   73.467176] Call Trace:
[   73.467483]  dump_stack+0x8d/0xb5
[   73.467910]  check_noncircular+0x119/0x130
[   73.468413]  ? save_trace+0x3d/0x2c0
[   73.468854]  validate_chain+0x8fa/0xec0
[   73.469329]  __lock_acquire+0x563/0x940
[   73.469800]  lock_acquire+0xd2/0x390
[   73.470240]  ? tcp_sendpage+0x23/0x50
[   73.470689]  lock_sock_nested+0x72/0xa0
[   73.471163]  ? tcp_sendpage+0x23/0x50
[   73.471624]  tcp_sendpage+0x23/0x50
[   73.472059]  inet_sendpage+0x4f/0x80
[   73.472504]  kernel_sendpage+0x57/0xc0
[   73.472965]  ? mark_held_locks+0x4f/0x70
[   73.473445]  nvme_tcp_try_send+0x137/0x7d0 [nvme_tcp]
[   73.474062]  nvme_tcp_queue_rq+0x317/0x330 [nvme_tcp]
[   73.474697]  __blk_mq_try_issue_directly+0x109/0x1b0
[   73.475324]  blk_mq_request_issue_directly+0x4b/0x80
[   73.475953]  blk_mq_try_issue_list_directly+0x62/0xf0
[   73.476587]  blk_mq_sched_insert_requests+0x192/0x2b0
[   73.477225]  blk_mq_flush_plug_list+0x13c/0x260
[   73.477802]  blk_flush_plug_list+0xd7/0x100
[   73.478338]  blk_finish_plug+0x21/0x30
[   73.478818]  _xfs_buf_ioapply+0x2cc/0x3c0 [xfs]
[   73.479457]  __xfs_buf_submit+0x85/0x220 [xfs]
[   73.480091]  xfs_buf_read_map+0x105/0x2a0 [xfs]
[   73.480728]  ? xfs_read_agi+0xb4/0x1d0 [xfs]
[   73.481331]  xfs_trans_read_buf_map+0x2b2/0x610 [xfs]
[   73.482036]  ? xfs_read_agi+0xb4/0x1d0 [xfs]
[   73.482635]  xfs_read_agi+0xb4/0x1d0 [xfs]
[   73.483221]  xfs_ialloc_read_agi+0x48/0x170 [xfs]
[   73.483866]  xfs_dialloc_select_ag+0x94/0x2a0 [xfs]
[   73.484513]  xfs_dir_ialloc+0x72/0x630 [xfs]
[   73.485092]  ? xfs_ilock+0xbf/0x250 [xfs]
[   73.485648]  xfs_create+0x241/0x6b0 [xfs]
[   73.486198]  xfs_generic_create+0x205/0x2c0 [xfs]
[   73.486857]  lookup_open+0x4f6/0x630
[   73.487317]  path_openat+0x298/0xa80
[   73.487786]  ? __lock_acquire+0x581/0x940
[   73.488302]  do_filp_open+0x93/0x100
[   73.488769]  ? do_raw_spin_unlock+0x49/0xc0
[   73.489307]  ? _raw_spin_unlock+0x1f/0x30
[   73.489825]  do_sys_openat2+0x24b/0x310
[   73.490321]  do_sys_open+0x4b/0x80
[   73.490764]  do_syscall_64+0x33/0x40
[   73.491230]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   73.491890] RIP: 0033:0x7f7d6f2eaead
[   73.492353] Code: c5 20 00 00 75 10 b8 02 00 00 00 0f 05 48 3d 01 f0
ff ff 73 31 c3 48 83 ec 08 e8 7e f4 ff ff 48 89 04 24 b8 02 00 00 00 0f
05 <48> 8b 3c 24 48 89 c2 e8 c7 f4 ff ff 48 89 d0 48 83 c4 08 48 3d 01
[   73.494713] RSP: 002b:00007ffc072e8910 EFLAGS: 00000293 ORIG_RAX:
0000000000000002
[   73.495671] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007f7d6f2eaead
[   73.496526] RDX: 00000000000001a4 RSI: 0000000000000041 RDI:
00007f7d6d666690
[   73.497381] RBP: 0000000000000000 R08: 0000000000000041 R09:
0000000000000041
[   73.498232] R10: 00007ffc072e85a0 R11: 0000000000000293 R12:
0000000000000000
[   73.499121] R13: 000000003b600000 R14: 00007f7d47136000 R15:
00007f7d6d666510
[  111.135896] XFS (nvme1n1): Unmounting Filesystem
[  111.308034] nvme nvme1: Removing ctrl: NQN "blktests-subsystem-1"



From Chaitanya.Kulkarni at wdc.com  Sun Feb  7 00:58:00 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 7 Feb 2021 05:58:00 +0000
Subject: kernel null pointer at nvme_tcp_init_iter[nvme_tcp] with blktests
 nvme-tcp/012
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <80b7184e-cdfb-cebc-fe07-a228ce57a9e7@redhat.com>
Message-ID: <BYAPR04MB4965F26763967C4EC93A73A286B09@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/6/21 20:51, Yi Zhang wrote:
> The issue was introduced after merge the NVMe updates
>
> commit 0fd6456fd1f4c8f3ec5a2df6ed7f34458a180409 (HEAD)
> Merge: 44d10e4b2f2c 0d7389718c32
> Author: Jens Axboe <axboe at kernel.dk>
> Date:   Tue Feb 2 07:12:06 2021 -0700
>
>      Merge branch 'for-5.12/drivers' into for-next
>
>      * for-5.12/drivers: (22 commits)
>        nvme-tcp: use cancel tagset helper for tear down
>        nvme-rdma: use cancel tagset helper for tear down
>        nvme-tcp: add clean action for failed reconnection
>        nvme-rdma: add clean action for failed reconnection
>        nvme-core: add cancel tagset helpers
>        nvme-core: get rid of the extra space
>        nvme: add tracing of zns commands
>        nvme: parse format nvm command details when tracing
>        nvme: update enumerations for status codes
>        nvmet: add lba to sect conversion helpers
>        nvmet: remove extra variable in identify ns
>        nvmet: remove extra variable in id-desclist
>        nvmet: remove extra variable in smart log nsid
>        nvme: refactor ns->ctrl by request
>        nvme-tcp: pass multipage bvec to request iov_iter
>        nvme-tcp: get rid of unused helper function
>        nvme-tcp: fix wrong setting of request iov_iter
>        nvme: support command retry delay for admin command
>        nvme: constify static attribute_group structs
>        nvmet-fc: use RCU proctection for assoc_list
>
>
> On 2/6/21 11:08 AM, Yi Zhang wrote:
>> blktests nvme-tcp/012
Can you try following patch and see in your Oops if you get that message
so that
we can at-least eliminate one case ?

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 619b0d8f6e38..13d44d155478 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2271,8 +2271,11 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct
nvme_ns *ns,
        req->data_len = blk_rq_nr_phys_segments(rq) ?
                                blk_rq_payload_bytes(rq) : 0;
        req->curr_bio = rq->bio;
-       if (req->curr_bio)
+       if (req->curr_bio) {
+               if (rq->bio != rq->biotail)
+                       printk(KERN_INFO"%s %d req->bio !=
req->biotail\n", __func__, __LINE__);
                nvme_tcp_init_iter(req, rq_data_dir(rq));
+       }
 
        if (rq_data_dir(rq) == WRITE &&
            req->data_len <= nvme_tcp_inline_data_size(queue))



From Chaitanya.Kulkarni at wdc.com  Sun Feb  7 02:14:32 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 7 Feb 2021 07:14:32 +0000
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
Message-ID: <BYAPR04MB4965E82FFD9D6FC443C8E67686B09@BYAPR04MB4965.namprd04.prod.outlook.com>

Sagi,

On 2/5/21 19:19, Yi Zhang wrote:
> Hello
>
> We found this kernel NULL pointer issue with latest linux-block/for-next and it's 100% reproduced, let me know if you need more info/testing, thanks 
>
> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>
> Reproducer: blktests nvme-tcp/012
>
>
> [  124.458121] run blktests nvme/012 at 2021-02-05 21:53:34 
> [  125.525568] BUG: kernel NULL pointer dereference, address: 0000000000000008 
> [  125.532524] #PF: supervisor read access in kernel mode 
> [  125.537665] #PF: error_code(0x0000) - not-present page 
> [  125.542803] PGD 0 P4D 0  
> [  125.545343] Oops: 0000 [#1] SMP NOPTI 
> [  125.549009] CPU: 15 PID: 12069 Comm: kworker/15:2H Tainted: G S        I       5.11.0-rc6+ #1 
> [  125.557528] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020 
> [  125.565093] Workqueue: kblockd blk_mq_run_work_fn 
> [  125.569797] RIP: 0010:nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp] 
> [  125.575544] Code: 8b 75 68 44 8b 45 28 44 8b 7d 30 49 89 d4 48 c1 e2 04 4c 01 f2 45 89 fb 44 89 c7 85 ff 74 4d 44 89 e0 44 8b 55 10 48 c1 e0 04 <41> 8b 5c 06 08 45 0f b6 ca 89 d8 44 29 d8 39 f8 0f 47 c7 41 83 e9 
> [  125.594290] RSP: 0018:ffffbd084447bd18 EFLAGS: 00010246 
> [  125.599515] RAX: 0000000000000000 RBX: ffffa0bba9f3ce80 RCX: 0000000000000000 
> [  125.606648] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000002000000 
> [  125.613781] RBP: ffffa0ba8ac6fec0 R08: 0000000002000000 R09: 0000000000000000 
> [  125.620914] R10: 0000000002800809 R11: 0000000000000000 R12: 0000000000000000 
> [  125.628045] R13: ffffa0bba9f3cf90 R14: 0000000000000000 R15: 0000000000000000 
> [  125.635178] FS:  0000000000000000(0000) GS:ffffa0c9ff9c0000(0000) knlGS:0000000000000000 
> [  125.643264] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
> [  125.649009] CR2: 0000000000000008 CR3: 00000001c9c6c005 CR4: 00000000007706e0 
> [  125.656142] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 
> [  125.663274] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 
> [  125.670407] PKRU: 55555554 
> [  125.673119] Call Trace: 
> [  125.675575]  nvme_tcp_queue_rq+0xef/0x330 [nvme_tcp] 
> [  125.680537]  blk_mq_dispatch_rq_list+0x11c/0x7c0 
> [  125.685157]  ? blk_mq_flush_busy_ctxs+0xf6/0x110 
> [  125.689775]  __blk_mq_sched_dispatch_requests+0x12b/0x170 
> [  125.695175]  blk_mq_sched_dispatch_requests+0x30/0x60 
> [  125.700227]  __blk_mq_run_hw_queue+0x2b/0x60 
> [  125.704500]  process_one_work+0x1cb/0x360 
> [  125.708513]  ? process_one_work+0x360/0x360 
> [  125.712699]  worker_thread+0x30/0x370 
> [  125.716365]  ? process_one_work+0x360/0x360 
> [  125.720550]  kthread+0x116/0x130 
> [  125.723782]  ? kthread_park+0x80/0x80 
> [  125.727448]  ret_from_fork+0x1f/0x30 
The NVMe TCP does support merging for non-admin queues
(nvme_tcp_alloc_tagset()).

Based on the what is been done for the bvecs in other places in
kernel especially when merging is enabled bio split case seems to be
missing from tcp when building bvec. What I mean is following
completely untested patch :-

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 619b0d8f6e38..dabb2633b28c 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -222,7 +222,9 @@ static void nvme_tcp_init_iter(struct
nvme_tcp_request *req,
         unsigned int dir)
 {
     struct request *rq = blk_mq_rq_from_pdu(req);
-    struct bio_vec *vec;
+    struct req_iterator rq_iter;
+    struct bio_vec *vec, *tvec;
+    struct bio_vec tmp;
     unsigned int size;
     int nr_bvec;
     size_t offset;
@@ -233,17 +235,29 @@ static void nvme_tcp_init_iter(struct
nvme_tcp_request *req,
         size = blk_rq_payload_bytes(rq);
         offset = 0;
     } else {
-        struct bio *bio = req->curr_bio;
-        struct bvec_iter bi;
-        struct bio_vec bv;
-
-        vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
-        nr_bvec = 0;
-        bio_for_each_bvec(bv, bio, bi) {
+        rq_for_each_bvec(tmp, rq, rq_iter)
             nr_bvec++;
+
+        if (rq->bio != rq->biotail) {
+            vec = kmalloc_array(nr_bvec, sizeof(struct bio_vec),
+                    GFP_NOIO);
+            if (!vec)
+                return;
+
+            tvec = vec;
+            rq_for_each_bvec(tmp, rq, rq_iter) {
+                *vec = tmp;
+                vec++;
+            }
+            vec = tvec;
+            offset = 0;
+        } else {
+            struct bio *bio = req->curr_bio;
+
+            vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
+            size = bio->bi_iter.bi_size;
+            offset = bio->bi_iter.bi_bvec_done;
         }
-        size = bio->bi_iter.bi_size;
-        offset = bio->bi_iter.bi_bvec_done;
     }
 
     iov_iter_bvec(&req->iter, dir, vec, nr_bvec, size);
@@ -2271,8 +2285,11 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct
nvme_ns *ns,
     req->data_len = blk_rq_nr_phys_segments(rq) ?
                 blk_rq_payload_bytes(rq) : 0;
     req->curr_bio = rq->bio;
-    if (req->curr_bio)
+    if (req->curr_bio) {
+        if (rq->bio != rq->biotail)
+            printk(KERN_INFO"%s %d req->bio != req->biotail\n",
__func__, __LINE__);
         nvme_tcp_init_iter(req, rq_data_dir(rq));
+    }
 
     if (rq_data_dir(rq) == WRITE &&
         req->data_len <= nvme_tcp_inline_data_size(queue))
-- 
2.22.1


Feel free to ignore this as I might be *completely wrong* here since I don't
have a reproducer, everything is working just fine on my machine.









From borispismenny at gmail.com  Sun Feb  7 09:13:03 2021
From: borispismenny at gmail.com (Boris Pismenny)
Date: Sun, 7 Feb 2021 16:13:03 +0200
Subject: [PATCH v3 net-next 01/21] iov_iter: Introduce new procedures for
 copy to iter/pages
In-Reply-To: <20210203193434.GD3200985@iweiny-DESK2.sc.intel.com>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-2-borisp@mellanox.com> <20210201173548.GA12960@lst.de>
 <CAJ3xEMjLKoQe_OB_L+w2wwUGck74Gm6=GPA=CK73QpeFbXr7Bw@mail.gmail.com>
 <20210203165621.GB6691@lst.de>
 <20210203193434.GD3200985@iweiny-DESK2.sc.intel.com>
Message-ID: <e1eb7f7d-9a2a-f879-6fc9-4b929f2a4239@gmail.com>

On 03/02/2021 21:34, Ira Weiny wrote:
> On Wed, Feb 03, 2021 at 05:56:21PM +0100, Christoph Hellwig wrote:
>> On Tue, Feb 02, 2021 at 08:00:51PM +0200, Or Gerlitz wrote:
>>> will look into this, any idea for a more suitable location?
>>
>> Maybe just a new file under lib/ for now?
>>
>>>> Overly long line.  But we're also looking into generic helpers for
>>>> this kind of things, not sure if they made it to linux-next in the
>>>> meantime, but please check.
>>>
>>> This is what I found in linux-next - note sure if you were referring to it
>>>
>>> commit 11432a3cc061c39475295be533c3674c4f8a6d0b
>>> Author: David Howells <dhowells at redhat.com>
>>>
>>>     iov_iter: Add ITER_XARRAY
>>
>> No, that's not it.  Ira, what is the status of the memcpy_{to,from}_page
>> helpers?
> 
> Converting the entire kernel tree in one series has become unwieldy so I've
> given up.
> 
> But I have a series to convert btrfs which I could release by the end of the
> week.  That should be good enough to push the memcpy_*_page() support in.
> 
> I'm get it formatted and submitted,
> Ira
> 

To conclude this discussion, there's nothing that we need to change here
as the relevant series is still WIP, right?


From borispismenny at gmail.com  Sun Feb  7 09:24:47 2021
From: borispismenny at gmail.com (Boris Pismenny)
Date: Sun, 7 Feb 2021 16:24:47 +0200
Subject: [PATCH v3 net-next 01/21] iov_iter: Introduce new procedures for
 copy to iter/pages
In-Reply-To: <20210203165621.GB6691@lst.de>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-2-borisp@mellanox.com> <20210201173548.GA12960@lst.de>
 <CAJ3xEMjLKoQe_OB_L+w2wwUGck74Gm6=GPA=CK73QpeFbXr7Bw@mail.gmail.com>
 <20210203165621.GB6691@lst.de>
Message-ID: <736bdc2f-8073-6d34-9509-182ed1ab2b4f@gmail.com>

On 03/02/2021 18:56, Christoph Hellwig wrote:
> On Tue, Feb 02, 2021 at 08:00:51PM +0200, Or Gerlitz wrote:
>> will look into this, any idea for a more suitable location?
> 
> Maybe just a new file under lib/ for now?
> 

That doesn't work unless we copy quite a lot of code. There are macros
here (in lib/iov_iter.c) that we rely on, e.g. iterate_and_advance and
friends.

Instead, I propose that we place all of the new code under an ifdef to
reduce the impact on object size if the code is unused. We'll also
improve documentation around this commit.


From hch at lst.de  Sun Feb  7 11:03:18 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:18 +0100
Subject: preserve DMA offsets when using swiotlb v2
Message-ID: <20210207160327.2955490-1-hch@lst.de>

Hi all,

this series make NVMe happy when running with swiotlb.  This caters
towards to completely broken NVMe controllers that ignore the
specification (hello to the biggest cloud provider on the planet!),
to crappy SOC that have addressing limitations, or "secure"
virtualization that force bounce buffering to enhance the user
experience.  Or in other words, no one sane should hit it, but
people do.

It is basically a respin of the

    "SWIOTLB: Preserve swiotlb map offset when needed."

series from Jianxiong Gao.  It complete rewrites the swiotlb part so that
the offset really is preserved and not just the offset into the swiotlb
slot, and to do so it grew half a dozen patches to refactor the swiotlb
so that a mere mortal like me could actually understand it.

Changes since v2:
 - cleanup nr_slots a little more
 - fix a check in get_max_slots
 - add a IO_TLB_SIZE define
 - cleanup find_slots a little more
 - fix the nvme patch to actually compile


From hch at lst.de  Sun Feb  7 11:03:19 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:19 +0100
Subject: [PATCH 1/9] driver core: add a min_align_mask field to struct
 device_dma_parameters
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-2-hch@lst.de>

From: Jianxiong Gao <jxgao at google.com>

Some devices rely on the address offset in a page to function
correctly (NVMe driver as an example). These devices may use
a different page size than the Linux kernel. The address offset
has to be preserved upon mapping, and in order to do so, we
need to record the page_offset_mask first.

Signed-off-by: Jianxiong Gao <jxgao at google.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Acked-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 include/linux/device.h      |  1 +
 include/linux/dma-mapping.h | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/include/linux/device.h b/include/linux/device.h
index 1779f90eeb4cb4..7960bf516dd7fe 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -291,6 +291,7 @@ struct device_dma_parameters {
 	 * sg limitations.
 	 */
 	unsigned int max_segment_size;
+	unsigned int min_align_mask;
 	unsigned long segment_boundary_mask;
 };
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 2e49996a8f391a..9c26225754e719 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -500,6 +500,22 @@ static inline int dma_set_seg_boundary(struct device *dev, unsigned long mask)
 	return -EIO;
 }
 
+static inline unsigned int dma_get_min_align_mask(struct device *dev)
+{
+	if (dev->dma_parms)
+		return dev->dma_parms->min_align_mask;
+	return 0;
+}
+
+static inline int dma_set_min_align_mask(struct device *dev,
+		unsigned int min_align_mask)
+{
+	if (WARN_ON_ONCE(!dev->dma_parms))
+		return -EIO;
+	dev->dma_parms->min_align_mask = min_align_mask;
+	return 0;
+}
+
 static inline int dma_get_cache_alignment(void)
 {
 #ifdef ARCH_DMA_MINALIGN
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:20 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:20 +0100
Subject: [PATCH 2/9] swiotlb: add a IO_TLB_SIZE define
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-3-hch@lst.de>

Add a new IO_TLB_SIZE define instead open coding it using
IO_TLB_SHIFT all over.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 include/linux/swiotlb.h |  1 +
 kernel/dma/swiotlb.c    | 14 +++++++-------
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index d9c9fc9ca5d217..5857a937c63722 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -29,6 +29,7 @@ enum swiotlb_force {
  * controllable.
  */
 #define IO_TLB_SHIFT 11
+#define IO_TLB_SIZE (1 << IO_TLB_SHIFT)
 
 /* default to 64MB */
 #define IO_TLB_DEFAULT_SIZE (64UL<<20)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 7c42df6e61001d..768187d0e17e31 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -171,7 +171,7 @@ void __init swiotlb_adjust_size(unsigned long new_size)
 	 * adjust/expand SWIOTLB size for their use.
 	 */
 	if (!io_tlb_nslabs) {
-		size = ALIGN(new_size, 1 << IO_TLB_SHIFT);
+		size = ALIGN(new_size, IO_TLB_SIZE);
 		io_tlb_nslabs = size >> IO_TLB_SHIFT;
 		io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
 
@@ -491,20 +491,20 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 
 	tbl_dma_addr &= mask;
 
-	offset_slots = ALIGN(tbl_dma_addr, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+	offset_slots = ALIGN(tbl_dma_addr, IO_TLB_SIZE) >> IO_TLB_SHIFT;
 
 	/*
 	 * Carefully handle integer overflow which can occur when mask == ~0UL.
 	 */
 	max_slots = mask + 1
-		    ? ALIGN(mask + 1, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT
+		    ? ALIGN(mask + 1, IO_TLB_SIZE) >> IO_TLB_SHIFT
 		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
 
 	/*
 	 * For mappings greater than or equal to a page, we limit the stride
 	 * (and hence alignment) to a page size.
 	 */
-	nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+	nslots = ALIGN(alloc_size, IO_TLB_SIZE) >> IO_TLB_SHIFT;
 	if (alloc_size >= PAGE_SIZE)
 		stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
 	else
@@ -598,7 +598,7 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 			      enum dma_data_direction dir, unsigned long attrs)
 {
 	unsigned long flags;
-	int i, count, nslots = ALIGN(alloc_size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
+	int i, count, nslots = ALIGN(alloc_size, IO_TLB_SIZE) >> IO_TLB_SHIFT;
 	int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
 	phys_addr_t orig_addr = io_tlb_orig_addr[index];
 
@@ -649,7 +649,7 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
 
 	if (orig_addr == INVALID_PHYS_ADDR)
 		return;
-	orig_addr += (unsigned long)tlb_addr & ((1 << IO_TLB_SHIFT) - 1);
+	orig_addr += (unsigned long)tlb_addr & (IO_TLB_SIZE - 1);
 
 	switch (target) {
 	case SYNC_FOR_CPU:
@@ -707,7 +707,7 @@ dma_addr_t swiotlb_map(struct device *dev, phys_addr_t paddr, size_t size,
 
 size_t swiotlb_max_mapping_size(struct device *dev)
 {
-	return ((size_t)1 << IO_TLB_SHIFT) * IO_TLB_SEGSIZE;
+	return ((size_t)IO_TLB_SIZE) * IO_TLB_SEGSIZE;
 }
 
 bool is_swiotlb_active(void)
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:21 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:21 +0100
Subject: [PATCH 3/9] swiotlb: factor out an io_tlb_offset helper
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-4-hch@lst.de>

Replace the very genericly named OFFSET macro with a little inline
helper that hardcodes the alignment to the only value ever passed.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 768187d0e17e31..7705821dcdbd27 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -50,9 +50,6 @@
 #define CREATE_TRACE_POINTS
 #include <trace/events/swiotlb.h>
 
-#define OFFSET(val,align) ((unsigned long)	\
-	                   ( (val) & ( (align) - 1)))
-
 #define SLABS_PER_PAGE (1 << (PAGE_SHIFT - IO_TLB_SHIFT))
 
 /*
@@ -192,6 +189,11 @@ void swiotlb_print_info(void)
 	       bytes >> 20);
 }
 
+static inline unsigned long io_tlb_offset(unsigned long val)
+{
+	return val & (IO_TLB_SEGSIZE - 1);
+}
+
 /*
  * Early SWIOTLB allocation may be too early to allow an architecture to
  * perform the desired operations.  This function allows the architecture to
@@ -241,7 +243,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long nslabs, int verbose)
 		      __func__, alloc_size, PAGE_SIZE);
 
 	for (i = 0; i < io_tlb_nslabs; i++) {
-		io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE);
+		io_tlb_list[i] = IO_TLB_SEGSIZE - io_tlb_offset(i);
 		io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
 	}
 	io_tlb_index = 0;
@@ -375,7 +377,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
 		goto cleanup4;
 
 	for (i = 0; i < io_tlb_nslabs; i++) {
-		io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE);
+		io_tlb_list[i] = IO_TLB_SEGSIZE - io_tlb_offset(i);
 		io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
 	}
 	io_tlb_index = 0;
@@ -546,7 +548,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 
 			for (i = index; i < (int) (index + nslots); i++)
 				io_tlb_list[i] = 0;
-			for (i = index - 1; (OFFSET(i, IO_TLB_SEGSIZE) != IO_TLB_SEGSIZE - 1) && io_tlb_list[i]; i--)
+			for (i = index - 1;
+			     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
+			     io_tlb_list[i]; i--)
 				io_tlb_list[i] = ++count;
 			tlb_addr = io_tlb_start + (index << IO_TLB_SHIFT);
 
@@ -632,7 +636,9 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 		 * Step 2: merge the returned slots with the preceding slots,
 		 * if available (non zero)
 		 */
-		for (i = index - 1; (OFFSET(i, IO_TLB_SEGSIZE) != IO_TLB_SEGSIZE -1) && io_tlb_list[i]; i--)
+		for (i = index - 1;
+		     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
+		     io_tlb_list[i]; i--)
 			io_tlb_list[i] = ++count;
 
 		io_tlb_used -= nslots;
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:22 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:22 +0100
Subject: [PATCH 4/9] swiotlb: factor out a nr_slots helper
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-5-hch@lst.de>

Factor out a helper to find the number of slots for a given size.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 7705821dcdbd27..9492219b0743ae 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -194,6 +194,11 @@ static inline unsigned long io_tlb_offset(unsigned long val)
 	return val & (IO_TLB_SEGSIZE - 1);
 }
 
+static inline unsigned long nr_slots(u64 val)
+{
+	return DIV_ROUND_UP(val, IO_TLB_SIZE);
+}
+
 /*
  * Early SWIOTLB allocation may be too early to allow an architecture to
  * perform the desired operations.  This function allows the architecture to
@@ -493,20 +498,20 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
 
 	tbl_dma_addr &= mask;
 
-	offset_slots = ALIGN(tbl_dma_addr, IO_TLB_SIZE) >> IO_TLB_SHIFT;
+	offset_slots = nr_slots(tbl_dma_addr);
 
 	/*
 	 * Carefully handle integer overflow which can occur when mask == ~0UL.
 	 */
 	max_slots = mask + 1
-		    ? ALIGN(mask + 1, IO_TLB_SIZE) >> IO_TLB_SHIFT
+		    ? nr_slots(mask + 1)
 		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
 
 	/*
 	 * For mappings greater than or equal to a page, we limit the stride
 	 * (and hence alignment) to a page size.
 	 */
-	nslots = ALIGN(alloc_size, IO_TLB_SIZE) >> IO_TLB_SHIFT;
+	nslots = nr_slots(alloc_size);
 	if (alloc_size >= PAGE_SIZE)
 		stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
 	else
@@ -602,7 +607,7 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 			      enum dma_data_direction dir, unsigned long attrs)
 {
 	unsigned long flags;
-	int i, count, nslots = ALIGN(alloc_size, IO_TLB_SIZE) >> IO_TLB_SHIFT;
+	int i, count, nslots = nr_slots(alloc_size);
 	int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
 	phys_addr_t orig_addr = io_tlb_orig_addr[index];
 
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:23 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:23 +0100
Subject: [PATCH 5/9] swiotlb: clean up swiotlb_tbl_unmap_single
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-6-hch@lst.de>

Remove a layer of pointless indentation, replace a hard to follow
ternary expression with a plain if/else.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 41 +++++++++++++++++++++--------------------
 1 file changed, 21 insertions(+), 20 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 9492219b0743ae..b38b1553c4663a 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -626,28 +626,29 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 	 * with slots below and above the pool being returned.
 	 */
 	spin_lock_irqsave(&io_tlb_lock, flags);
-	{
-		count = ((index + nslots) < ALIGN(index + 1, IO_TLB_SEGSIZE) ?
-			 io_tlb_list[index + nslots] : 0);
-		/*
-		 * Step 1: return the slots to the free list, merging the
-		 * slots with superceeding slots
-		 */
-		for (i = index + nslots - 1; i >= index; i--) {
-			io_tlb_list[i] = ++count;
-			io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
-		}
-		/*
-		 * Step 2: merge the returned slots with the preceding slots,
-		 * if available (non zero)
-		 */
-		for (i = index - 1;
-		     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
-		     io_tlb_list[i]; i--)
-			io_tlb_list[i] = ++count;
+	if (index + nslots < ALIGN(index + 1, IO_TLB_SEGSIZE))
+		count = io_tlb_list[index + nslots];
+	else
+		count = 0;
 
-		io_tlb_used -= nslots;
+	/*
+	 * Step 1: return the slots to the free list, merging the slots with
+	 * superceeding slots
+	 */
+	for (i = index + nslots - 1; i >= index; i--) {
+		io_tlb_list[i] = ++count;
+		io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
 	}
+
+	/*
+	 * Step 2: merge the returned slots with the preceding slots, if
+	 * available (non zero)
+	 */
+	for (i = index - 1;
+	     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 && io_tlb_list[i];
+	     i--)
+		io_tlb_list[i] = ++count;
+	io_tlb_used -= nslots;
 	spin_unlock_irqrestore(&io_tlb_lock, flags);
 }
 
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:24 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:24 +0100
Subject: [PATCH 6/9] swiotlb: refactor swiotlb_tbl_map_single
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-7-hch@lst.de>

Split out a bunch of a self-contained helpers to make the function easier
to follow.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 179 +++++++++++++++++++++----------------------
 1 file changed, 89 insertions(+), 90 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index b38b1553c4663a..381c24ef1ac1d0 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -468,134 +468,133 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
 	}
 }
 
-phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
-		size_t mapping_size, size_t alloc_size,
-		enum dma_data_direction dir, unsigned long attrs)
-{
-	dma_addr_t tbl_dma_addr = phys_to_dma_unencrypted(hwdev, io_tlb_start);
-	unsigned long flags;
-	phys_addr_t tlb_addr;
-	unsigned int nslots, stride, index, wrap;
-	int i;
-	unsigned long mask;
-	unsigned long offset_slots;
-	unsigned long max_slots;
-	unsigned long tmp_io_tlb_used;
-
-	if (no_iotlb_memory)
-		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
-
-	if (mem_encrypt_active())
-		pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
+#define slot_addr(start, idx)	((start) + ((idx) << IO_TLB_SHIFT))
 
-	if (mapping_size > alloc_size) {
-		dev_warn_once(hwdev, "Invalid sizes (mapping: %zd bytes, alloc: %zd bytes)",
-			      mapping_size, alloc_size);
-		return (phys_addr_t)DMA_MAPPING_ERROR;
-	}
-
-	mask = dma_get_seg_boundary(hwdev);
+/*
+ * Carefully handle integer overflow which can occur when boundary_mask == ~0UL.
+ */
+static inline unsigned long get_max_slots(unsigned long boundary_mask)
+{
+	if (boundary_mask == ~0UL)
+		return 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
+	return nr_slots(boundary_mask + 1);
+}
 
-	tbl_dma_addr &= mask;
+static unsigned int wrap_index(unsigned int index)
+{
+	if (index >= io_tlb_nslabs)
+		return 0;
+	return index;
+}
 
-	offset_slots = nr_slots(tbl_dma_addr);
+/*
+ * Find a suitable number of IO TLB entries size that will fit this request and
+ * allocate a buffer from that IO TLB pool.
+ */
+static int find_slots(struct device *dev, size_t alloc_size)
+{
+	unsigned long boundary_mask = dma_get_seg_boundary(dev);
+	dma_addr_t tbl_dma_addr =
+		phys_to_dma_unencrypted(dev, io_tlb_start) & boundary_mask;
+	unsigned int max_slots = get_max_slots(boundary_mask);
+	unsigned int nslots = nr_slots(alloc_size), stride = 1;
+	unsigned int index, wrap, count = 0, i;
+	unsigned long flags;
 
-	/*
-	 * Carefully handle integer overflow which can occur when mask == ~0UL.
-	 */
-	max_slots = mask + 1
-		    ? nr_slots(mask + 1)
-		    : 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
+	BUG_ON(!nslots);
 
 	/*
 	 * For mappings greater than or equal to a page, we limit the stride
 	 * (and hence alignment) to a page size.
 	 */
-	nslots = nr_slots(alloc_size);
 	if (alloc_size >= PAGE_SIZE)
-		stride = (1 << (PAGE_SHIFT - IO_TLB_SHIFT));
-	else
-		stride = 1;
+		stride <<= (PAGE_SHIFT - IO_TLB_SHIFT);
 
-	BUG_ON(!nslots);
-
-	/*
-	 * Find suitable number of IO TLB entries size that will fit this
-	 * request and allocate a buffer from that IO TLB pool.
-	 */
 	spin_lock_irqsave(&io_tlb_lock, flags);
-
 	if (unlikely(nslots > io_tlb_nslabs - io_tlb_used))
 		goto not_found;
 
-	index = ALIGN(io_tlb_index, stride);
-	if (index >= io_tlb_nslabs)
-		index = 0;
-	wrap = index;
-
+	index = wrap = wrap_index(ALIGN(io_tlb_index, stride));
 	do {
-		while (iommu_is_span_boundary(index, nslots, offset_slots,
-					      max_slots)) {
-			index += stride;
-			if (index >= io_tlb_nslabs)
-				index = 0;
-			if (index == wrap)
-				goto not_found;
-		}
-
 		/*
 		 * If we find a slot that indicates we have 'nslots' number of
 		 * contiguous buffers, we allocate the buffers from that slot
 		 * and mark the entries as '0' indicating unavailable.
 		 */
-		if (io_tlb_list[index] >= nslots) {
-			int count = 0;
-
-			for (i = index; i < (int) (index + nslots); i++)
-				io_tlb_list[i] = 0;
-			for (i = index - 1;
-			     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
-			     io_tlb_list[i]; i--)
-				io_tlb_list[i] = ++count;
-			tlb_addr = io_tlb_start + (index << IO_TLB_SHIFT);
-
-			/*
-			 * Update the indices to avoid searching in the next
-			 * round.
-			 */
-			io_tlb_index = ((index + nslots) < io_tlb_nslabs
-					? (index + nslots) : 0);
-
-			goto found;
+		if (!iommu_is_span_boundary(index, nslots,
+					    nr_slots(tbl_dma_addr),
+					    max_slots)) {
+			if (io_tlb_list[index] >= nslots)
+				goto found;
 		}
-		index += stride;
-		if (index >= io_tlb_nslabs)
-			index = 0;
+		index = wrap_index(index + stride);
 	} while (index != wrap);
 
 not_found:
-	tmp_io_tlb_used = io_tlb_used;
-
 	spin_unlock_irqrestore(&io_tlb_lock, flags);
-	if (!(attrs & DMA_ATTR_NO_WARN) && printk_ratelimit())
-		dev_warn(hwdev, "swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (slots)\n",
-			 alloc_size, io_tlb_nslabs, tmp_io_tlb_used);
-	return (phys_addr_t)DMA_MAPPING_ERROR;
+	return -1;
+
 found:
+	for (i = index; i < index + nslots; i++)
+		io_tlb_list[i] = 0;
+	for (i = index - 1;
+	     io_tlb_offset(i) != IO_TLB_SEGSIZE - 1 &&
+	     io_tlb_list[i]; i--)
+		io_tlb_list[i] = ++count;
+
+	/*
+	 * Update the indices to avoid searching in the next round.
+	 */
+	if (index + nslots < io_tlb_nslabs)
+		io_tlb_index = index + nslots;
+	else
+		io_tlb_index = 0;
 	io_tlb_used += nslots;
+
 	spin_unlock_irqrestore(&io_tlb_lock, flags);
+	return index;
+}
+
+phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
+		size_t mapping_size, size_t alloc_size,
+		enum dma_data_direction dir, unsigned long attrs)
+{
+	unsigned int index, i;
+	phys_addr_t tlb_addr;
+
+	if (no_iotlb_memory)
+		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
+
+	if (mem_encrypt_active())
+		pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
+
+	if (mapping_size > alloc_size) {
+		dev_warn_once(dev, "Invalid sizes (mapping: %zd bytes, alloc: %zd bytes)",
+			      mapping_size, alloc_size);
+		return (phys_addr_t)DMA_MAPPING_ERROR;
+	}
+
+	index = find_slots(dev, alloc_size);
+	if (index == -1) {
+		if (!(attrs & DMA_ATTR_NO_WARN))
+			dev_warn_ratelimited(dev,
+	"swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (slots)\n",
+				 alloc_size, io_tlb_nslabs, io_tlb_used);
+		return (phys_addr_t)DMA_MAPPING_ERROR;
+	}
 
 	/*
 	 * Save away the mapping from the original address to the DMA address.
 	 * This is needed when we sync the memory.  Then we sync the buffer if
 	 * needed.
 	 */
-	for (i = 0; i < nslots; i++)
-		io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
+	for (i = 0; i < nr_slots(alloc_size); i++)
+		io_tlb_orig_addr[index + i] = slot_addr(orig_addr, i);
+
+	tlb_addr = slot_addr(io_tlb_start, index);
 	if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
 	    (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
 		swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_TO_DEVICE);
-
 	return tlb_addr;
 }
 
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:25 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:25 +0100
Subject: [PATCH 7/9] swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-8-hch@lst.de>

swiotlb_tbl_map_single currently nevers sets a tlb_addr that is not
aligned to the tlb bucket size.  But we're going to add such a case
soon, for which this adjustment would be bogus.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 381c24ef1ac1d0..8eb84c337fe90c 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -660,7 +660,6 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
 
 	if (orig_addr == INVALID_PHYS_ADDR)
 		return;
-	orig_addr += (unsigned long)tlb_addr & (IO_TLB_SIZE - 1);
 
 	switch (target) {
 	case SYNC_FOR_CPU:
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:26 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:26 +0100
Subject: [PATCH 8/9] swiotlb: respect min_align_mask
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-9-hch@lst.de>

Respect the min_align_mask in struct device_dma_parameters in swiotlb.

There are two parts to it:
 1) for the lower bits of the alignment inside the io tlb slot, just
    extent the size of the allocation and leave the start of the slot
     empty
 2) for the high bits ensure we find a slot that matches the high bits
    of the alignment to avoid wasting too much memory

Based on an earlier patch from Jianxiong Gao <jxgao at google.com>.

Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 kernel/dma/swiotlb.c | 41 +++++++++++++++++++++++++++++++----------
 1 file changed, 31 insertions(+), 10 deletions(-)

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 8eb84c337fe90c..adcc3c87e10078 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -470,6 +470,14 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
 
 #define slot_addr(start, idx)	((start) + ((idx) << IO_TLB_SHIFT))
 
+/*
+ * Return the offset into a iotlb slot required to keep the device happy.
+ */
+static unsigned int swiotlb_align_offset(struct device *dev, u64 addr)
+{
+	return addr & dma_get_min_align_mask(dev) & (IO_TLB_SIZE - 1);
+}
+
 /*
  * Carefully handle integer overflow which can occur when boundary_mask == ~0UL.
  */
@@ -491,24 +499,29 @@ static unsigned int wrap_index(unsigned int index)
  * Find a suitable number of IO TLB entries size that will fit this request and
  * allocate a buffer from that IO TLB pool.
  */
-static int find_slots(struct device *dev, size_t alloc_size)
+static int find_slots(struct device *dev, phys_addr_t orig_addr,
+		size_t alloc_size)
 {
 	unsigned long boundary_mask = dma_get_seg_boundary(dev);
 	dma_addr_t tbl_dma_addr =
 		phys_to_dma_unencrypted(dev, io_tlb_start) & boundary_mask;
 	unsigned int max_slots = get_max_slots(boundary_mask);
-	unsigned int nslots = nr_slots(alloc_size), stride = 1;
+	unsigned int iotlb_align_mask =
+		dma_get_min_align_mask(dev) & ~(IO_TLB_SIZE - 1);
+	unsigned int nslots = nr_slots(alloc_size), stride;
 	unsigned int index, wrap, count = 0, i;
 	unsigned long flags;
 
 	BUG_ON(!nslots);
 
 	/*
-	 * For mappings greater than or equal to a page, we limit the stride
-	 * (and hence alignment) to a page size.
+	 * For mappings with an alignment requirement don't bother looping to
+	 * unaligned slots once we found an aligned one.  For allocations of
+	 * PAGE_SIZE or larger only look for page aligned allocations.
 	 */
+	stride = (iotlb_align_mask >> IO_TLB_SHIFT) + 1;
 	if (alloc_size >= PAGE_SIZE)
-		stride <<= (PAGE_SHIFT - IO_TLB_SHIFT);
+		stride = max(stride, stride << (PAGE_SHIFT - IO_TLB_SHIFT));
 
 	spin_lock_irqsave(&io_tlb_lock, flags);
 	if (unlikely(nslots > io_tlb_nslabs - io_tlb_used))
@@ -516,6 +529,12 @@ static int find_slots(struct device *dev, size_t alloc_size)
 
 	index = wrap = wrap_index(ALIGN(io_tlb_index, stride));
 	do {
+		if ((slot_addr(tbl_dma_addr, index) & iotlb_align_mask) !=
+		    (orig_addr & iotlb_align_mask)) {
+			index = wrap_index(index + 1);
+			continue;
+		}
+
 		/*
 		 * If we find a slot that indicates we have 'nslots' number of
 		 * contiguous buffers, we allocate the buffers from that slot
@@ -559,6 +578,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
 		size_t mapping_size, size_t alloc_size,
 		enum dma_data_direction dir, unsigned long attrs)
 {
+	unsigned int offset = swiotlb_align_offset(dev, orig_addr);
 	unsigned int index, i;
 	phys_addr_t tlb_addr;
 
@@ -574,7 +594,7 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
 		return (phys_addr_t)DMA_MAPPING_ERROR;
 	}
 
-	index = find_slots(dev, alloc_size);
+	index = find_slots(dev, orig_addr, alloc_size + offset);
 	if (index == -1) {
 		if (!(attrs & DMA_ATTR_NO_WARN))
 			dev_warn_ratelimited(dev,
@@ -588,10 +608,10 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
 	 * This is needed when we sync the memory.  Then we sync the buffer if
 	 * needed.
 	 */
-	for (i = 0; i < nr_slots(alloc_size); i++)
+	for (i = 0; i < nr_slots(alloc_size + offset); i++)
 		io_tlb_orig_addr[index + i] = slot_addr(orig_addr, i);
 
-	tlb_addr = slot_addr(io_tlb_start, index);
+	tlb_addr = slot_addr(io_tlb_start, index) + offset;
 	if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
 	    (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
 		swiotlb_bounce(orig_addr, tlb_addr, mapping_size, DMA_TO_DEVICE);
@@ -606,8 +626,9 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
 			      enum dma_data_direction dir, unsigned long attrs)
 {
 	unsigned long flags;
-	int i, count, nslots = nr_slots(alloc_size);
-	int index = (tlb_addr - io_tlb_start) >> IO_TLB_SHIFT;
+	unsigned int offset = swiotlb_align_offset(hwdev, tlb_addr);
+	int i, count, nslots = nr_slots(alloc_size + offset);
+	int index = (tlb_addr - offset - io_tlb_start) >> IO_TLB_SHIFT;
 	phys_addr_t orig_addr = io_tlb_orig_addr[index];
 
 	/*
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:03:27 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:03:27 +0100
Subject: [PATCH 9/9] nvme-pci: set min_align_mask
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210207160327.2955490-10-hch@lst.de>

From: Jianxiong Gao <jxgao at google.com>

The PRP addressing scheme requires all PRP entries except for the
first one to have a zero offset into the NVMe controller pages (which
can be different from the Linux PAGE_SIZE).  Use the min_align_mask
device parameter to ensure that swiotlb does not change the address
of the buffer modulo the device page size to ensure that the PRPs
won't be malformed.

Signed-off-by: Jianxiong Gao <jxgao at google.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/host/pci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 81e6389b204205..83303e4e301924 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2629,6 +2629,7 @@ static void nvme_reset_work(struct work_struct *work)
 	 * Don't limit the IOMMU merged segment size.
 	 */
 	dma_set_max_seg_size(dev->dev, 0xffffffff);
+	dma_set_min_align_mask(dev->dev, NVME_CTRL_PAGE_SIZE - 1);
 
 	mutex_unlock(&dev->shutdown_lock);
 
-- 
2.29.2



From hch at lst.de  Sun Feb  7 11:16:08 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun, 7 Feb 2021 17:16:08 +0100
Subject: [PATCH] nvmet-tcp: Fix receive data digest calculation for
 multiple h2cdata PDUs
In-Reply-To: <20210203230001.18797-1-sagi@grimberg.me>
References: <20210203230001.18797-1-sagi@grimberg.me>
Message-ID: <20210207161608.GA25547@lst.de>

Thanks,

applied to nvme-5.12.


From hch at lst.de  Sun Feb  7 11:16:18 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun, 7 Feb 2021 17:16:18 +0100
Subject: [PATCH] nvmet-tcp: fix potential race of tcp socket closing
 accept_work
In-Reply-To: <20210205194725.625035-1-sagi@grimberg.me>
References: <20210205194725.625035-1-sagi@grimberg.me>
Message-ID: <20210207161618.GB25547@lst.de>

Thanks,

applied to nvme-5.12.


From hch at lst.de  Sun Feb  7 11:16:28 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun, 7 Feb 2021 17:16:28 +0100
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
In-Reply-To: <20210205195002.2657209-1-kbusch@kernel.org>
References: <20210205195002.2657209-1-kbusch@kernel.org>
Message-ID: <20210207161628.GC25547@lst.de>

Thanks,

applied to nvme-5.12.


From hch at lst.de  Sun Feb  7 11:20:28 2021
From: hch at lst.de (Christoph Hellwig)
Date: Sun,  7 Feb 2021 17:20:28 +0100
Subject: [PATCH] nvmet-fc: add a missing __rcu annotation to
 nvmet_fc_tgt_assoc.queues
Message-ID: <20210207162028.2959778-1-hch@lst.de>

Make sparse happy after the recent conversion to RCU lookups.

Fixes: 4e2f02bf77da ("nvmet-fc: use RCU proctection for assoc_list")
Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/target/fc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/fc.c b/drivers/nvme/target/fc.c
index c14c60bfdf85c6..d375745fc4ed3d 100644
--- a/drivers/nvme/target/fc.c
+++ b/drivers/nvme/target/fc.c
@@ -165,7 +165,7 @@ struct nvmet_fc_tgt_assoc {
 	struct nvmet_fc_hostport	*hostport;
 	struct nvmet_fc_ls_iod		*rcv_disconn;
 	struct list_head		a_list;
-	struct nvmet_fc_tgt_queue	*queues[NVMET_NR_QUEUES + 1];
+	struct nvmet_fc_tgt_queue __rcu	*queues[NVMET_NR_QUEUES + 1];
 	struct kref			ref;
 	struct work_struct		del_work;
 	struct rcu_head			rcu;
-- 
2.29.2



From gerlitz.or at gmail.com  Sun Feb  7 11:40:17 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Sun, 7 Feb 2021 18:40:17 +0200
Subject: [PATCH v3 net-next 08/21] nvme-tcp : Recalculate crc in the end
 of the capsule
In-Reply-To: <a104a5d1-b4cb-4275-6ced-b80f911b6f47@grimberg.me>
References: <20210201100509.27351-1-borisp@mellanox.com>
 <20210201100509.27351-9-borisp@mellanox.com>
 <a104a5d1-b4cb-4275-6ced-b80f911b6f47@grimberg.me>
Message-ID: <CAJ3xEMjnCzs7+6iusPdD666KodjdAfZCUn0rhUZa6wd_qL+kWQ@mail.gmail.com>

On Wed, Feb 3, 2021 at 11:12 AM Sagi Grimberg <sagi at grimberg.me> wrote:
> On 2/1/21 2:04 AM, Boris Pismenny wrote:

> > @@ -290,12 +341,9 @@ int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> >       }
> >
> >       req->ddp.command_id = command_id;
> > -     req->ddp.sg_table.sgl = req->ddp.first_sgl;
> > -     ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
> > -                                  req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
> > +     ret = nvme_tcp_req_map_sg(req, rq);
>
> Why didn't you introduce nvme_tcp_req_map_sg in the first place?

OK, will do

> >       if (ret)
> >               return -ENOMEM;
> > -     req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
> >
> >       ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
> >                                                queue->sock->sk,

> > @@ -1088,17 +1148,29 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
> >       if (queue->ddgst_remaining)
> >               return 0;
> >
> > -     if (queue->recv_ddgst != queue->exp_ddgst) {
> > -             dev_err(queue->ctrl->ctrl.device,
> > -                     "data digest error: recv %#x expected %#x\n",
> > -                     le32_to_cpu(queue->recv_ddgst),
> > -                     le32_to_cpu(queue->exp_ddgst));
> > -             return -EIO;
> > +     offload_fail = !nvme_tcp_ddp_ddgst_ok(queue);
> > +     offload_en = test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags);
> > +     if (!offload_en || offload_fail) {
> > +             if (offload_en && offload_fail) { // software-fallback
> > +                     rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
> > +                                           pdu->command_id);
> > +                     nvme_tcp_ddp_ddgst_recalc(queue->rcv_hash, rq);
> > +             }
> > +
> > +             nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
> > +             if (queue->recv_ddgst != queue->exp_ddgst) {
> > +                     dev_err(queue->ctrl->ctrl.device,
> > +                             "data digest error: recv %#x expected %#x\n",
> > +                             le32_to_cpu(queue->recv_ddgst),
> > +                             le32_to_cpu(queue->exp_ddgst));
> > +                     return -EIO;
> > +             }
>
> I still dislike this hunk. Can you split the recalc login to its
> own ddp function at least? This is just a confusing piece of code.

mmm, we added two boolean predicates (offload_en and
offload_failed) plus a comment (software-fallback)
to clarify this piece... thought it can fly

> >       if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
> > -             struct request *rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
> > -                                             pdu->command_id);
> > +             if (!rq)
> > +                     rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
> > +                                           pdu->command_id);
>
> Why is this change needed? Maybe just move this assignment up?

OK will move it up


From smalin at marvell.com  Sun Feb  7 13:13:16 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:16 +0200
Subject: [RFC PATCH v3 03/11] nvme-tcp-offload: Add device scan implementation
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-4-smalin@marvell.com>

From: Dean Balandin <dbalandin at marvell.com>

This patch implements the create_ctrl skeleton, specifcially to
demonstrate how the claim_dev op will be used.

The driver scans the registered devices and calls the claim_dev op on
each of them, to find the first devices that matches the connection
params. Once the correct devices is found (claim_dev returns true), we
raise the refcnt of that device and return that device as the device to
be used for ctrl currently being created.

Signed-off-by: Dean Balandin <dbalandin at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/host/tcp-offload.c | 82 +++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c
index ee3800250e47..49e11638b4fd 100644
--- a/drivers/nvme/host/tcp-offload.c
+++ b/drivers/nvme/host/tcp-offload.c
@@ -13,6 +13,12 @@
 static LIST_HEAD(nvme_tcp_ofld_devices);
 static DECLARE_RWSEM(nvme_tcp_ofld_devices_rwsem);
 
+static inline struct nvme_tcp_ofld_ctrl *
+to_tcp_ofld_ctrl(struct nvme_ctrl *nctrl)
+{
+	return container_of(nctrl, struct nvme_tcp_ofld_ctrl, nctrl);
+}
+
 /**
  * nvme_tcp_ofld_register_dev() - NVMeTCP Offload Library registration
  * function.
@@ -96,6 +102,81 @@ nvme_tcp_ofld_req_done(struct nvme_tcp_ofld_req *req,
 	/* Placeholder - complete request with/without error */
 }
 
+struct nvme_tcp_ofld_dev *
+nvme_tcp_ofld_lookup_dev(struct nvme_tcp_ofld_ctrl *ctrl)
+{
+	struct nvme_tcp_ofld_dev *dev;
+
+	down_read(&nvme_tcp_ofld_devices_rwsem);
+	list_for_each_entry(dev, &nvme_tcp_ofld_devices, entry) {
+		if (dev->ops->claim_dev(dev, &ctrl->conn_params)) {
+			/* Increase driver refcnt */
+			if (!try_module_get(dev->ops->module)) {
+				pr_err("try_module_get failed\n");
+				dev = NULL;
+			}
+
+			goto out;
+		}
+	}
+
+	dev = NULL;
+out:
+	up_read(&nvme_tcp_ofld_devices_rwsem);
+
+	return dev;
+}
+
+static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new)
+{
+	/* Placeholder - validates inputs and creates admin and IO queues */
+
+	return 0;
+}
+
+static struct nvme_ctrl *
+nvme_tcp_ofld_create_ctrl(struct device *ndev, struct nvmf_ctrl_options *opts)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl;
+	struct nvme_tcp_ofld_dev *dev;
+	struct nvme_ctrl *nctrl;
+	int rc = 0;
+
+	ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
+	if (!ctrl)
+		return ERR_PTR(-ENOMEM);
+
+	/* Init nvme_tcp_ofld_ctrl and nvme_ctrl params based on received opts */
+
+	/* Find device that can reach the dest addr */
+	dev = nvme_tcp_ofld_lookup_dev(ctrl);
+	if (!dev) {
+		pr_info("no device found for addr %s:%s.\n",
+			opts->traddr, opts->trsvcid);
+		rc = -EINVAL;
+		goto out_free_ctrl;
+	}
+
+	ctrl->dev = dev;
+
+	/* Init queues */
+
+	/* Call nvme_init_ctrl */
+
+	rc = nvme_tcp_ofld_setup_ctrl(nctrl, true);
+	if (rc)
+		goto out_module_put;
+
+	return nctrl;
+
+out_module_put:
+	module_put(dev->ops->module);
+out_free_ctrl:
+	kfree(ctrl);
+
+	return ERR_PTR(rc);
+}
+
 static struct nvmf_transport_ops nvme_tcp_ofld_transport = {
 	.name		= "tcp_offload",
 	.module		= THIS_MODULE,
@@ -105,6 +186,7 @@ static struct nvmf_transport_ops nvme_tcp_ofld_transport = {
 			  NVMF_OPT_CTRL_LOSS_TMO | NVMF_OPT_RECONNECT_DELAY |
 			  NVMF_OPT_HDR_DIGEST | NVMF_OPT_DATA_DIGEST |
 			  NVMF_OPT_NR_POLL_QUEUES | NVMF_OPT_TOS,
+	.create_ctrl	= nvme_tcp_ofld_create_ctrl,
 };
 
 static int __init nvme_tcp_ofld_init_module(void)
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:13 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:13 +0200
Subject: [RFC PATCH v3 00/11] NVMeTCP Offload ULP and QEDN Device Driver
Message-ID: <20210207181324.11429-1-smalin@marvell.com>


With the goal of enabling a generic infrastructure that allows NVMe/TCP 
offload devices like NICs to seamlessly plug into the NVMe-oF stack, this 
patch series introduces the nvme-tcp-offload ULP host layer, which will 
be a new transport type called "tcp-offload" and will serve as an 
abstraction layer to work with vendor specific nvme-tcp offload drivers.

NVMeTCP offload is a full offload of the NVMeTCP protocol, this includes 
both the TCP level and the NVMeTCP level.

The nvme-tcp-offload transport can co-exist with the existing tcp and 
other transports. The tcp offload was designed so that stack changes are 
kept to a bare minimum: only registering new transports. 
All other APIs, ops etc. are identical to the regular tcp transport.
Representing the TCP offload as a new transport allows clear and manageable
differentiation between the connections which should use the offload path
and those that are not offloaded (even on the same device).


Queue Initialization:
=====================
The nvme-tcp-offload ULP module shall register with the existing 
nvmf_transport_ops (.name = "tcp_offload"), nvme_ctrl_ops and blk_mq_ops.
The nvme-tcp-offload vendor driver shall register to nvme-tcp-offload ULP
with the following ops:
 - claim_dev() - in order to resolve the route to the target according to
                 the net_dev.
 - create_queue() - in order to create offloaded nvme-tcp queue.

The nvme-tcp-offload ULP module shall manage all the controller level
functionalities, call claim_dev and based on the return values shall call
the relevant module create_queue in order to create the admin queue and
the IO queues.


IO-path:
========
The nvme-tcp-offload shall work at the IO-level - the nvme-tcp-offload 
ULP module shall pass the request (the IO) to the nvme-tcp-offload vendor
driver and later, the nvme-tcp-offload vendor driver returns the request
completion (the IO completion).
No additional handling is needed in between; this design will reduce the
CPU utilization as we will describe below.

The nvme-tcp-offload vendor driver shall register to nvme-tcp-offload ULP
with the following IO-path ops:
 - init_req()
 - map_sg() - in order to map the request sg (similar to 
              nvme_rdma_map_data() ).
 - send_req() - in order to pass the request to the handling of the
                offload driver that shall pass it to the vendor specific
				device.
 - poll_queue()

Once the IO completes, the nvme-tcp-offload vendor driver shall call 
command.done() that will invoke the nvme-tcp-offload ULP layer to
complete the request.


TCP events:
===========
The Marvell FastLinQ NIC HW engine handle all the TCP re-transmissions
and OOO events.


Teardown and errors:
====================
In case of NVMeTCP queue error the nvme-tcp-offload vendor driver shall
call the nvme_tcp_ofld_report_queue_err.
The nvme-tcp-offload vendor driver shall register to nvme-tcp-offload ULP
with the following teardown ops:
 - drain_queue()
 - destroy_queue()
 

 The Marvell FastLinQ NIC HW engine:
====================================
The Marvell NIC HW engine is capable of offloading the entire TCP/IP
stack and managing up to 64K connections per PF, already implemented and 
upstream use cases for this include iWARP (by the Marvell qedr driver) 
and iSCSI (by the Marvell qedi driver).
In addition, the Marvell NIC HW engine offloads the NVMeTCP queue layer
and is able to manage the IO level also in case of TCP re-transmissions
and OOO events.
The HW engine enables direct data placement (including the data digest CRC
calculation and validation) and direct data transmission (including data
digest CRC calculation).


The Marvell qedn driver:
========================
The new driver will be added under "drivers/nvme/hw" and will be enabled
by the Kconfig "Marvell NVM Express over Fabrics TCP offload".
As part of the qedn init, the driver will register as a pci device driver 
and will work with the Marvell fastlinQ NIC.
As part of the probe, the driver will register to the nvme_tcp_offload
(ULP) and to the qed module (qed_nvmetcp_ops) - similar to other
"qed_*_ops" which are used by the qede, qedr, qedf and qedi device
drivers.
  
 
The series' patches:
===================
Patch 1-2       Add the nvme-tcp-offload ULP module, including the APIs.
Patches 3-5     nvme-tcp-offload ULP controller level functionalities.
Patch 6         nvme-tcp-offload ULP queue level functionalities.
Patch 7         nvme-tcp-offload ULP IO level functionalities.
Patch 8			nvme-qedn Marvell's NVMeTCP HW offload vendor driver.
Patch 9			net-qed NVMeTCP Offload PF level FW and HW HSI.
Patch 10-11		nvme-qedn probe level functionalities.


Performance:
============
With this implementation on top of the Marvell qedn driver (using the
Marvell FastLinQ NIC), we were able to demonstrate x3 CPU utilization
improvement for 4K queued read/write IOs and up to x20 in case of 512K
read/write IOs.
In addition, we were able to demonstrate latency improvement, and 
specifically 99.99% tail latency improvement of up to x2-5 (depends on
the queue-depth).


Future work:
============
For simplicity, the RFC series does not include the following 
functionalities which will be added based on the comments on patches 1-11.
 - nvme-tcp-offload teardown, IO timeout and async flows.
 - qedn device/queue/IO level.
 
 
Long term future work:
============
 - The nvme-tcp-offload ULP target abstraction layer.
 - The Marvell nvme-tcp-offload "qednt" target driver.


Changes since RFC v1:
=====================
- Fix nvme_tcp_ofld_ops return values.
- Remove NVMF_TRTYPE_TCP_OFFLOAD.
- Add nvme_tcp_ofld_poll() implementation.
- Fix nvme_tcp_ofld_queue_rq() to check map_sg() and send_req() return
  values.


Changes since RFC v2:
=====================
- Add qedn - Marvell's NVMeTCP HW offload vendor driver init and probe
  (patches 8-11).
- Fixes in controller and queue level (patches 3-6).
  

Arie Gershberg (3):
  nvme-fabrics: Move NVMF_ALLOWED_OPTS and NVMF_REQUIRED_OPTS
    definitions
  nvme-tcp-offload: Add controller level implementation
  nvme-tcp-offload: Add controller level error recovery implementation

Dean Balandin (3):
  nvme-tcp-offload: Add device scan implementation
  nvme-tcp-offload: Add queue level implementation
  nvme-tcp-offload: Add IO level implementation

Shai Malin (5):
  nvme-tcp-offload: Add nvme-tcp-offload - NVMeTCP HW offload ULP
  nvme-qedn: Add qedn - Marvell's NVMeTCP HW offload vendor driver
  net-qed: Add NVMeTCP Offload PF Level FW and HW HSI
  nvme-qedn: Add qedn probe
  nvme-qedn: Add IRQ and fast-path resources initializations

 MAINTAINERS                                   |   10 +
 drivers/net/ethernet/qlogic/Kconfig           |    3 +
 drivers/net/ethernet/qlogic/qed/Makefile      |    1 +
 drivers/net/ethernet/qlogic/qed/qed.h         |    3 +
 drivers/net/ethernet/qlogic/qed/qed_hsi.h     |    1 +
 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.c |  269 ++++
 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.h |   48 +
 drivers/net/ethernet/qlogic/qed/qed_sp.h      |    2 +
 drivers/nvme/Kconfig                          |    1 +
 drivers/nvme/Makefile                         |    1 +
 drivers/nvme/host/Kconfig                     |   16 +
 drivers/nvme/host/Makefile                    |    3 +
 drivers/nvme/host/fabrics.c                   |    7 -
 drivers/nvme/host/fabrics.h                   |    7 +
 drivers/nvme/host/tcp-offload.c               | 1123 +++++++++++++++++
 drivers/nvme/host/tcp-offload.h               |  185 +++
 drivers/nvme/hw/Kconfig                       |    9 +
 drivers/nvme/hw/Makefile                      |    3 +
 drivers/nvme/hw/qedn/Makefile                 |    3 +
 drivers/nvme/hw/qedn/qedn.c                   |  652 ++++++++++
 drivers/nvme/hw/qedn/qedn.h                   |   90 ++
 include/linux/qed/common_hsi.h                |    1 +
 include/linux/qed/nvmetcp_common.h            |   47 +
 include/linux/qed/qed_if.h                    |   22 +
 include/linux/qed/qed_nvmetcp_if.h            |   71 ++
 25 files changed, 2571 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.c
 create mode 100644 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.h
 create mode 100644 drivers/nvme/host/tcp-offload.c
 create mode 100644 drivers/nvme/host/tcp-offload.h
 create mode 100644 drivers/nvme/hw/Kconfig
 create mode 100644 drivers/nvme/hw/Makefile
 create mode 100644 drivers/nvme/hw/qedn/Makefile
 create mode 100644 drivers/nvme/hw/qedn/qedn.c
 create mode 100644 drivers/nvme/hw/qedn/qedn.h
 create mode 100644 include/linux/qed/nvmetcp_common.h
 create mode 100644 include/linux/qed/qed_nvmetcp_if.h

-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:15 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:15 +0200
Subject: [RFC PATCH v3 02/11] nvme-fabrics: Move NVMF_ALLOWED_OPTS and
 NVMF_REQUIRED_OPTS definitions
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-3-smalin@marvell.com>

From: Arie Gershberg <agershberg at marvell.com>

Move NVMF_ALLOWED_OPTS and NVMF_REQUIRED_OPTS definitions
to header file, so it can be used by transport modules.

Signed-off-by: Arie Gershberg <agershberg at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/host/fabrics.c | 7 -------
 drivers/nvme/host/fabrics.h | 7 +++++++
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index 72ac00173500..ccc626cbf6d0 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -1002,13 +1002,6 @@ void nvmf_free_options(struct nvmf_ctrl_options *opts)
 }
 EXPORT_SYMBOL_GPL(nvmf_free_options);
 
-#define NVMF_REQUIRED_OPTS	(NVMF_OPT_TRANSPORT | NVMF_OPT_NQN)
-#define NVMF_ALLOWED_OPTS	(NVMF_OPT_QUEUE_SIZE | NVMF_OPT_NR_IO_QUEUES | \
-				 NVMF_OPT_KATO | NVMF_OPT_HOSTNQN | \
-				 NVMF_OPT_HOST_ID | NVMF_OPT_DUP_CONNECT |\
-				 NVMF_OPT_DISABLE_SQFLOW |\
-				 NVMF_OPT_FAIL_FAST_TMO)
-
 static struct nvme_ctrl *
 nvmf_create_ctrl(struct device *dev, const char *buf)
 {
diff --git a/drivers/nvme/host/fabrics.h b/drivers/nvme/host/fabrics.h
index 733010d2eafd..e44955ed6fa9 100644
--- a/drivers/nvme/host/fabrics.h
+++ b/drivers/nvme/host/fabrics.h
@@ -61,6 +61,13 @@ enum {
 	NVMF_OPT_FAIL_FAST_TMO	= 1 << 20,
 };
 
+#define NVMF_REQUIRED_OPTS	(NVMF_OPT_TRANSPORT | NVMF_OPT_NQN)
+#define NVMF_ALLOWED_OPTS	(NVMF_OPT_QUEUE_SIZE | NVMF_OPT_NR_IO_QUEUES | \
+				 NVMF_OPT_KATO | NVMF_OPT_HOSTNQN | \
+				 NVMF_OPT_HOST_ID | NVMF_OPT_DUP_CONNECT |\
+				 NVMF_OPT_DISABLE_SQFLOW |\
+				 NVMF_OPT_FAIL_FAST_TMO)
+
 /**
  * struct nvmf_ctrl_options - Used to hold the options specified
  *			      with the parsing opts enum.
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:17 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:17 +0200
Subject: [RFC PATCH v3 04/11] nvme-tcp-offload: Add controller level
 implementation
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-5-smalin@marvell.com>

From: Arie Gershberg <agershberg at marvell.com>

In this patch we implement controller level functionality including:
- create_ctrl.
- delete_ctrl.
- free_ctrl.

The implementation is similar to other nvme fabrics modules, the main
difference being that the nvme-tcp-offload ULP calls the vendor specific
claim_dev() op with the given TCP/IP parameters to determine which device
will be used for this controller.
Once found, the vendor specific device and controller will be paired and
kept in a controller list managed by the ULP.

Signed-off-by: Arie Gershberg <agershberg at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/host/tcp-offload.c | 458 +++++++++++++++++++++++++++++++-
 drivers/nvme/host/tcp-offload.h |   1 +
 2 files changed, 453 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c
index 49e11638b4fd..da934c6fed9e 100644
--- a/drivers/nvme/host/tcp-offload.c
+++ b/drivers/nvme/host/tcp-offload.c
@@ -12,6 +12,10 @@
 
 static LIST_HEAD(nvme_tcp_ofld_devices);
 static DECLARE_RWSEM(nvme_tcp_ofld_devices_rwsem);
+static LIST_HEAD(nvme_tcp_ofld_ctrl_list);
+static DECLARE_RWSEM(nvme_tcp_ofld_ctrl_rwsem);
+static struct blk_mq_ops nvme_tcp_ofld_admin_mq_ops;
+static struct blk_mq_ops nvme_tcp_ofld_mq_ops;
 
 static inline struct nvme_tcp_ofld_ctrl *
 to_tcp_ofld_ctrl(struct nvme_ctrl *nctrl)
@@ -127,26 +131,423 @@ nvme_tcp_ofld_lookup_dev(struct nvme_tcp_ofld_ctrl *ctrl)
 	return dev;
 }
 
+static struct blk_mq_tag_set *
+nvme_tcp_ofld_alloc_tagset(struct nvme_ctrl *nctrl, bool admin)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	struct blk_mq_tag_set *set;
+	int rc;
+
+	if (admin) {
+		set = &ctrl->admin_tag_set;
+		memset(set, 0, sizeof(*set));
+		set->ops = &nvme_tcp_ofld_admin_mq_ops;
+		set->queue_depth = NVME_AQ_MQ_TAG_DEPTH;
+		set->reserved_tags = 2; /* connect + keep-alive */
+		set->numa_node = nctrl->numa_node;
+		set->flags = BLK_MQ_F_BLOCKING;
+		set->cmd_size = sizeof(struct nvme_tcp_ofld_req);
+		set->driver_data = ctrl;
+		set->nr_hw_queues = 1;
+		set->timeout = NVME_ADMIN_TIMEOUT;
+	} else {
+		set = &ctrl->tag_set;
+		memset(set, 0, sizeof(*set));
+		set->ops = &nvme_tcp_ofld_mq_ops;
+		set->queue_depth = nctrl->sqsize + 1;
+		set->reserved_tags = 1; /* fabric connect */
+		set->numa_node = nctrl->numa_node;
+		set->flags = BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_BLOCKING;
+		set->cmd_size = sizeof(struct nvme_tcp_ofld_req);
+		set->driver_data = ctrl;
+		set->nr_hw_queues = nctrl->queue_count - 1;
+		set->timeout = NVME_IO_TIMEOUT;
+		set->nr_maps = nctrl->opts->nr_poll_queues ?
+							HCTX_MAX_TYPES : 2;
+	}
+
+	rc = blk_mq_alloc_tag_set(set);
+	if (rc)
+		return ERR_PTR(rc);
+
+	return set;
+}
+
+static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl,
+					       bool new)
+{
+	int rc;
+
+	/* Placeholder - alloc_admin_queue */
+	if (new) {
+		nctrl->admin_tagset =
+				nvme_tcp_ofld_alloc_tagset(nctrl, true);
+		if (IS_ERR(nctrl->admin_tagset)) {
+			rc = PTR_ERR(nctrl->admin_tagset);
+			nctrl->admin_tagset = NULL;
+			goto out_free_queue;
+		}
+
+		nctrl->fabrics_q = blk_mq_init_queue(nctrl->admin_tagset);
+		if (IS_ERR(nctrl->fabrics_q)) {
+			rc = PTR_ERR(nctrl->fabrics_q);
+			nctrl->fabrics_q = NULL;
+			goto out_free_tagset;
+		}
+
+		nctrl->admin_q = blk_mq_init_queue(nctrl->admin_tagset);
+		if (IS_ERR(nctrl->admin_q)) {
+			rc = PTR_ERR(nctrl->admin_q);
+			nctrl->admin_q = NULL;
+			goto out_cleanup_fabrics_q;
+		}
+	}
+
+	/* Placeholder - nvme_tcp_ofld_start_queue */
+
+	rc = nvme_enable_ctrl(nctrl);
+	if (rc)
+		goto out_stop_queue;
+
+	blk_mq_unquiesce_queue(nctrl->admin_q);
+
+	rc = nvme_init_identify(nctrl);
+	if (rc)
+		goto out_stop_queue;
+
+	return 0;
+
+out_stop_queue:
+	/* Placeholder - stop offload queue */
+
+out_cleanup_fabrics_q:
+	if (new)
+		blk_cleanup_queue(nctrl->fabrics_q);
+out_free_tagset:
+	if (new)
+		blk_mq_free_tag_set(nctrl->admin_tagset);
+out_free_queue:
+	/* Placeholder - free admin queue */
+
+	return rc;
+}
+
+static int
+nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new)
+{
+	int rc;
+
+	/* Placeholder - alloc_io_queues */
+
+	if (new) {
+		nctrl->tagset = nvme_tcp_ofld_alloc_tagset(nctrl, false);
+		if (IS_ERR(nctrl->tagset)) {
+			rc = PTR_ERR(nctrl->tagset);
+			nctrl->tagset = NULL;
+			goto out_free_io_queues;
+		}
+
+		nctrl->connect_q = blk_mq_init_queue(nctrl->tagset);
+		if (IS_ERR(nctrl->connect_q)) {
+			rc = PTR_ERR(nctrl->connect_q);
+			nctrl->connect_q = NULL;
+			goto out_free_tag_set;
+		}
+	}
+
+	/* Placeholder - start_io_queues */
+
+	if (!new) {
+		nvme_start_queues(nctrl);
+		if (!nvme_wait_freeze_timeout(nctrl, NVME_IO_TIMEOUT)) {
+			/*
+			 * If we timed out waiting for freeze we are likely to
+			 * be stuck.  Fail the controller initialization just
+			 * to be safe.
+			 */
+			rc = -ENODEV;
+			goto out_wait_freeze_timed_out;
+		}
+		blk_mq_update_nr_hw_queues(nctrl->tagset, nctrl->queue_count - 1);
+		nvme_unfreeze(nctrl);
+	}
+
+	return 0;
+
+out_wait_freeze_timed_out:
+	nvme_stop_queues(nctrl);
+
+	/* Placeholder - Stop IO queues */
+
+	if (new)
+		blk_cleanup_queue(nctrl->connect_q);
+out_free_tag_set:
+	if (new)
+		blk_mq_free_tag_set(nctrl->tagset);
+
+out_free_io_queues:
+	/* Placeholder - free_io_queues */
+
+	return rc;
+}
+
 static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new)
 {
-	/* Placeholder - validates inputs and creates admin and IO queues */
+	struct nvmf_ctrl_options *opts = nctrl->opts;
+	int rc;
+
+	rc = nvme_tcp_ofld_configure_admin_queue(nctrl, new);
+	if (rc)
+		goto destroy_admin;
+
+	if (nctrl->icdoff) {
+		dev_err(nctrl->device, "icdoff is not supported!\n");
+		rc = -EINVAL;
+		goto destroy_admin;
+	}
+
+	if (opts->queue_size > nctrl->sqsize + 1)
+		dev_warn(nctrl->device,
+			 "queue_size %zu > ctrl sqsize %u, clamping down\n",
+			 opts->queue_size, nctrl->sqsize + 1);
+
+	if (nctrl->sqsize + 1 > nctrl->maxcmd) {
+		dev_warn(nctrl->device,
+			 "sqsize %u > ctrl maxcmd %u, clamping down\n",
+			 nctrl->sqsize + 1, nctrl->maxcmd);
+		nctrl->sqsize = nctrl->maxcmd - 1;
+	}
+
+	if (nctrl->queue_count > 1) {
+		rc = nvme_tcp_ofld_configure_io_queues(nctrl, new);
+		if (rc)
+			goto destroy_admin;
+	}
+
+	if (!nvme_change_ctrl_state(nctrl, NVME_CTRL_LIVE)) {
+		/*
+		 * state change failure is ok if we started ctrl delete,
+		 * unless we're during creation of a new controller to
+		 * avoid races with teardown flow.
+		 */
+		WARN_ON_ONCE(nctrl->state != NVME_CTRL_DELETING &&
+			     nctrl->state != NVME_CTRL_DELETING_NOIO);
+		WARN_ON_ONCE(new);
+		rc = -EINVAL;
+		goto destroy_io;
+	}
+
+	nvme_start_ctrl(nctrl);
+	return 0;
+
+destroy_io:
+	/* Placeholder - stop and destroy io queues*/
+destroy_admin:
+	/* Placeholder - stop and destroy admin queue*/
+
+	return rc;
+}
+
+static int
+nvme_tcp_ofld_check_dev_opts(struct nvmf_ctrl_options *opts,
+			     struct nvme_tcp_ofld_ops *ofld_ops)
+{
+	unsigned int nvme_tcp_ofld_opt_mask = NVMF_ALLOWED_OPTS |
+			ofld_ops->allowed_opts | ofld_ops->required_opts;
+	if (opts->mask & ~nvme_tcp_ofld_opt_mask) {
+		pr_warn("One or more of the nvmf options isn't supported by %s.\n",
+			ofld_ops->name);
+		return -EINVAL;
+	}
 
 	return 0;
 }
 
+static void nvme_tcp_ofld_free_ctrl(struct nvme_ctrl *nctrl)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	struct nvme_tcp_ofld_dev *dev = ctrl->dev;
+
+	if (list_empty(&ctrl->list))
+		goto free_ctrl;
+
+	down_write(&nvme_tcp_ofld_ctrl_rwsem);
+	list_del(&ctrl->list);
+	up_write(&nvme_tcp_ofld_ctrl_rwsem);
+
+	nvmf_free_options(nctrl->opts);
+free_ctrl:
+	module_put(dev->ops->module);
+	kfree(ctrl->queues);
+	kfree(ctrl);
+}
+
+static void nvme_tcp_ofld_submit_async_event(struct nvme_ctrl *arg)
+{
+	/* Placeholder - submit_async_event */
+}
+
+static void
+nvme_tcp_ofld_teardown_admin_queue(struct nvme_ctrl *ctrl, bool remove)
+{
+	/* Placeholder - teardown_admin_queue */
+}
+
+static void
+nvme_tcp_ofld_teardown_io_queues(struct nvme_ctrl *nctrl, bool remove)
+{
+	/* Placeholder - teardown_io_queues */
+}
+
+static void
+nvme_tcp_ofld_teardown_ctrl(struct nvme_ctrl *nctrl, bool shutdown)
+{
+	/* Placeholder - err_work and connect_work */
+	nvme_tcp_ofld_teardown_io_queues(nctrl, shutdown);
+	blk_mq_quiesce_queue(nctrl->admin_q);
+	if (shutdown)
+		nvme_shutdown_ctrl(nctrl);
+	else
+		nvme_disable_ctrl(nctrl);
+	nvme_tcp_ofld_teardown_admin_queue(nctrl, shutdown);
+}
+
+static void nvme_tcp_ofld_delete_ctrl(struct nvme_ctrl *nctrl)
+{
+	nvme_tcp_ofld_teardown_ctrl(nctrl, true);
+}
+
+static int
+nvme_tcp_ofld_init_request(struct blk_mq_tag_set *set,
+			   struct request *rq,
+			   unsigned int hctx_idx,
+			   unsigned int numa_node)
+{
+	struct nvme_tcp_ofld_req *req = blk_mq_rq_to_pdu(rq);
+	struct nvme_tcp_ofld_ctrl *ctrl = set->driver_data;
+
+	/* Placeholder - init request */
+
+	req->done = nvme_tcp_ofld_req_done;
+	ctrl->dev->ops->init_req(req);
+
+	return 0;
+}
+
+static blk_status_t
+nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx,
+		       const struct blk_mq_queue_data *bd)
+{
+	/* Call nvme_setup_cmd(...) */
+
+	/* Call ops->map_sg(...) */
+
+	return BLK_STS_OK;
+}
+
+static struct blk_mq_ops nvme_tcp_ofld_mq_ops = {
+	.queue_rq	= nvme_tcp_ofld_queue_rq,
+	.init_request	= nvme_tcp_ofld_init_request,
+	/*
+	 * All additional ops will be also implemented and registered similar to
+	 * tcp.c
+	 */
+};
+
+static struct blk_mq_ops nvme_tcp_ofld_admin_mq_ops = {
+	.queue_rq	= nvme_tcp_ofld_queue_rq,
+	.init_request	= nvme_tcp_ofld_init_request,
+	/*
+	 * All additional ops will be also implemented and registered similar to
+	 * tcp.c
+	 */
+};
+
+static const struct nvme_ctrl_ops nvme_tcp_ofld_ctrl_ops = {
+	.name			= "tcp_offload",
+	.module			= THIS_MODULE,
+	.flags			= NVME_F_FABRICS,
+	.reg_read32		= nvmf_reg_read32,
+	.reg_read64		= nvmf_reg_read64,
+	.reg_write32		= nvmf_reg_write32,
+	.free_ctrl		= nvme_tcp_ofld_free_ctrl,
+	.submit_async_event	= nvme_tcp_ofld_submit_async_event,
+	.delete_ctrl		= nvme_tcp_ofld_delete_ctrl,
+	.get_address		= nvmf_get_address,
+};
+
+static bool
+nvme_tcp_ofld_existing_controller(struct nvmf_ctrl_options *opts)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl;
+	bool found = false;
+
+	down_read(&nvme_tcp_ofld_ctrl_rwsem);
+	list_for_each_entry(ctrl, &nvme_tcp_ofld_ctrl_list, list) {
+		found = nvmf_ip_options_match(&ctrl->nctrl, opts);
+		if (found)
+			break;
+	}
+	up_read(&nvme_tcp_ofld_ctrl_rwsem);
+
+	return found;
+}
+
 static struct nvme_ctrl *
 nvme_tcp_ofld_create_ctrl(struct device *ndev, struct nvmf_ctrl_options *opts)
 {
+	struct nvme_tcp_ofld_queue *queue;
 	struct nvme_tcp_ofld_ctrl *ctrl;
 	struct nvme_tcp_ofld_dev *dev;
 	struct nvme_ctrl *nctrl;
-	int rc = 0;
+	int i, rc = 0;
 
 	ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
 	if (!ctrl)
 		return ERR_PTR(-ENOMEM);
 
-	/* Init nvme_tcp_ofld_ctrl and nvme_ctrl params based on received opts */
+	INIT_LIST_HEAD(&ctrl->list);
+	nctrl = &ctrl->nctrl;
+	nctrl->opts = opts;
+	nctrl->queue_count = opts->nr_io_queues + opts->nr_write_queues +
+			     opts->nr_poll_queues + 1;
+	nctrl->sqsize = opts->queue_size - 1;
+	nctrl->kato = opts->kato;
+	if (!(opts->mask & NVMF_OPT_TRSVCID)) {
+		opts->trsvcid =
+			kstrdup(__stringify(NVME_TCP_DISC_PORT), GFP_KERNEL);
+		if (!opts->trsvcid) {
+			rc = -ENOMEM;
+			goto out_free_ctrl;
+		}
+		opts->mask |= NVMF_OPT_TRSVCID;
+	}
+
+	rc = inet_pton_with_scope(&init_net, AF_UNSPEC, opts->traddr,
+				  opts->trsvcid,
+				  &ctrl->conn_params.remote_ip_addr);
+	if (rc) {
+		pr_err("malformed address passed: %s:%s\n",
+		       opts->traddr, opts->trsvcid);
+		goto out_free_ctrl;
+	}
+
+	if (opts->mask & NVMF_OPT_HOST_TRADDR) {
+		rc = inet_pton_with_scope(&init_net, AF_UNSPEC,
+					  opts->host_traddr, NULL,
+					  &ctrl->conn_params.local_ip_addr);
+		if (rc) {
+			pr_err("malformed src address passed: %s\n",
+			       opts->host_traddr);
+			goto out_free_ctrl;
+		}
+	}
+
+	if (!opts->duplicate_connect &&
+	    nvme_tcp_ofld_existing_controller(opts)) {
+		rc = -EALREADY;
+		goto out_free_ctrl;
+	}
 
 	/* Find device that can reach the dest addr */
 	dev = nvme_tcp_ofld_lookup_dev(ctrl);
@@ -157,18 +558,55 @@ nvme_tcp_ofld_create_ctrl(struct device *ndev, struct nvmf_ctrl_options *opts)
 		goto out_free_ctrl;
 	}
 
+	rc = nvme_tcp_ofld_check_dev_opts(opts, dev->ops);
+	if (rc)
+		goto out_module_put;
+
 	ctrl->dev = dev;
 
-	/* Init queues */
+	ctrl->queues = kcalloc(nctrl->queue_count,
+			       sizeof(struct nvme_tcp_ofld_queue),
+			       GFP_KERNEL);
+	if (!ctrl->queues) {
+		rc = -ENOMEM;
+		goto out_module_put;
+	}
+
+	for (i = 0; i < nctrl->queue_count; ++i) {
+		queue = &ctrl->queues[i];
+		queue->ctrl = ctrl;
+		queue->dev = dev;
+		queue->report_err = nvme_tcp_ofld_report_queue_err;
+	}
 
-	/* Call nvme_init_ctrl */
+	rc = nvme_init_ctrl(nctrl, ndev, &nvme_tcp_ofld_ctrl_ops, 0);
+	if (rc)
+		goto out_free_queues;
+
+	if (!nvme_change_ctrl_state(nctrl, NVME_CTRL_CONNECTING)) {
+		WARN_ON_ONCE(1);
+		rc = -EINTR;
+		goto out_uninit_ctrl;
+	}
 
 	rc = nvme_tcp_ofld_setup_ctrl(nctrl, true);
 	if (rc)
-		goto out_module_put;
+		goto out_uninit_ctrl;
+
+	dev_info(nctrl->device, "new ctrl: NQN \"%s\", addr %pISp\n",
+		 opts->subsysnqn, &ctrl->conn_params.remote_ip_addr);
+
+	down_write(&nvme_tcp_ofld_ctrl_rwsem);
+	list_add_tail(&ctrl->list, &nvme_tcp_ofld_ctrl_list);
+	up_write(&nvme_tcp_ofld_ctrl_rwsem);
 
 	return nctrl;
 
+out_uninit_ctrl:
+	nvme_uninit_ctrl(nctrl);
+	nvme_put_ctrl(nctrl);
+out_free_queues:
+	kfree(ctrl->queues);
 out_module_put:
 	module_put(dev->ops->module);
 out_free_ctrl:
@@ -198,7 +636,15 @@ static int __init nvme_tcp_ofld_init_module(void)
 
 static void __exit nvme_tcp_ofld_cleanup_module(void)
 {
+	struct nvme_tcp_ofld_ctrl *ctrl;
+
 	nvmf_unregister_transport(&nvme_tcp_ofld_transport);
+
+	down_write(&nvme_tcp_ofld_ctrl_rwsem);
+	list_for_each_entry(ctrl, &nvme_tcp_ofld_ctrl_list, list)
+		nvme_delete_ctrl(&ctrl->nctrl);
+	up_write(&nvme_tcp_ofld_ctrl_rwsem);
+	flush_workqueue(nvme_delete_wq);
 }
 
 module_init(nvme_tcp_ofld_init_module);
diff --git a/drivers/nvme/host/tcp-offload.h b/drivers/nvme/host/tcp-offload.h
index 468617a58e34..a9bcd4f7d150 100644
--- a/drivers/nvme/host/tcp-offload.h
+++ b/drivers/nvme/host/tcp-offload.h
@@ -76,6 +76,7 @@ struct nvme_tcp_ofld_ctrl_con_params {
 /* Allocated by nvme_tcp_ofld */
 struct nvme_tcp_ofld_ctrl {
 	struct nvme_ctrl nctrl;
+	struct list_head list;
 	struct nvme_tcp_ofld_dev *dev;
 
 	/* admin and IO queues */
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:14 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:14 +0200
Subject: [RFC PATCH v3 01/11] nvme-tcp-offload: Add nvme-tcp-offload - NVMeTCP
 HW offload ULP
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-2-smalin@marvell.com>

This patch will present the structure for the NVMeTCP offload common
layer driver. This module is added under "drivers/nvme/host/" and future
offload drivers which will register to it will be placed under
"drivers/nvme/hw".
This new driver will be enabled by the Kconfig "NVM Express over Fabrics
TCP offload commmon layer".
In order to support the new transport type, for host mode, no change is
needed.

Each new vendor-specific offload driver will register to this ULP during
its probe function, by filling out the nvme_tcp_ofld_dev->ops and
nvme_tcp_ofld_dev->private_data and calling nvme_tcp_ofld_register_dev
with the initialized struct.

The internal implementation:
- tcp-offload.h:
  Includes all common structs and ops to be used and shared by offload
  drivers.

- tcp-offload.c:
  Includes the init function which registers as a NVMf transport just
  like any other transport.

Signed-off-by: Dean Balandin <dbalandin at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/host/Kconfig       |  16 +++
 drivers/nvme/host/Makefile      |   3 +
 drivers/nvme/host/tcp-offload.c | 124 ++++++++++++++++++++++++
 drivers/nvme/host/tcp-offload.h | 167 ++++++++++++++++++++++++++++++++
 4 files changed, 310 insertions(+)
 create mode 100644 drivers/nvme/host/tcp-offload.c
 create mode 100644 drivers/nvme/host/tcp-offload.h

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index a44d49d63968..6e869e94e67f 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -84,3 +84,19 @@ config NVME_TCP
 	  from https://github.com/linux-nvme/nvme-cli.
 
 	  If unsure, say N.
+
+config NVME_TCP_OFFLOAD
+	tristate "NVM Express over Fabrics TCP offload common layer"
+	default m
+	depends on INET
+	depends on BLK_DEV_NVME
+	select NVME_FABRICS
+	help
+	  This provides support for the NVMe over Fabrics protocol using
+	  the TCP offload transport. This allows you to use remote block devices
+	  exported using the NVMe protocol set.
+
+	  To configure a NVMe over Fabrics controller use the nvme-cli tool
+	  from https://github.com/linux-nvme/nvme-cli.
+
+	  If unsure, say N.
diff --git a/drivers/nvme/host/Makefile b/drivers/nvme/host/Makefile
index d7f6a87687b8..0e7ef044cf29 100644
--- a/drivers/nvme/host/Makefile
+++ b/drivers/nvme/host/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_NVME_FABRICS)		+= nvme-fabrics.o
 obj-$(CONFIG_NVME_RDMA)			+= nvme-rdma.o
 obj-$(CONFIG_NVME_FC)			+= nvme-fc.o
 obj-$(CONFIG_NVME_TCP)			+= nvme-tcp.o
+obj-$(CONFIG_NVME_TCP_OFFLOAD)	+= nvme-tcp-offload.o
 
 nvme-core-y				:= core.o
 nvme-core-$(CONFIG_TRACING)		+= trace.o
@@ -26,3 +27,5 @@ nvme-rdma-y				+= rdma.o
 nvme-fc-y				+= fc.o
 
 nvme-tcp-y				+= tcp.o
+
+nvme-tcp-offload-y		+= tcp-offload.o
diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c
new file mode 100644
index 000000000000..ee3800250e47
--- /dev/null
+++ b/drivers/nvme/host/tcp-offload.c
@@ -0,0 +1,124 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+/* Kernel includes */
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+/* Driver includes */
+#include "tcp-offload.h"
+
+static LIST_HEAD(nvme_tcp_ofld_devices);
+static DECLARE_RWSEM(nvme_tcp_ofld_devices_rwsem);
+
+/**
+ * nvme_tcp_ofld_register_dev() - NVMeTCP Offload Library registration
+ * function.
+ * @dev:	NVMeTCP offload device instance to be registered to the
+ *		common tcp offload instance.
+ *
+ * API function that registers the type of vendor specific driver
+ * being implemented to the common NVMe over TCP offload library. Part of
+ * the overall init sequence of starting up an offload driver.
+ */
+int nvme_tcp_ofld_register_dev(struct nvme_tcp_ofld_dev *dev)
+{
+	struct nvme_tcp_ofld_ops *ops = dev->ops;
+
+	if (!ops->claim_dev ||
+	    !ops->create_queue ||
+	    !ops->drain_queue ||
+	    !ops->destroy_queue ||
+	    !ops->poll_queue ||
+	    !ops->init_req ||
+	    !ops->map_sg ||
+	    !ops->send_req)
+		return -EINVAL;
+
+	down_write(&nvme_tcp_ofld_devices_rwsem);
+	list_add_tail(&dev->entry, &nvme_tcp_ofld_devices);
+	up_write(&nvme_tcp_ofld_devices_rwsem);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(nvme_tcp_ofld_register_dev);
+
+/**
+ * nvme_tcp_ofld_unregister_dev() - NVMeTCP Offload Library unregistration
+ * function.
+ * @dev:	NVMeTCP offload device instance to be unregistered from the
+ *		common tcp offload instance.
+ *
+ * API function that unregisters the type of vendor specific driver being
+ * implemented from the common NVMe over TCP offload library.
+ * Part of the overall exit sequence of unloading the implemented driver.
+ */
+void nvme_tcp_ofld_unregister_dev(struct nvme_tcp_ofld_dev *dev)
+{
+	down_write(&nvme_tcp_ofld_devices_rwsem);
+	list_del(&dev->entry);
+	up_write(&nvme_tcp_ofld_devices_rwsem);
+}
+EXPORT_SYMBOL_GPL(nvme_tcp_ofld_unregister_dev);
+
+/**
+ * nvme_tcp_ofld_report_queue_err() - NVMeTCP Offload report error event
+ * callback function. Pointed to by nvme_tcp_ofld_queue->report_err.
+ * @queue:	NVMeTCP offload queue instance on which the error has occurred.
+ *
+ * API function that allows the vendor specific offload driver to reports errors
+ * to the common offload layer, to invoke error recovery.
+ */
+int nvme_tcp_ofld_report_queue_err(struct nvme_tcp_ofld_queue *queue)
+{
+	/* Placeholder - invoke error recovery flow */
+
+	return 0;
+}
+
+/**
+ * nvme_tcp_ofld_req_done() - NVMeTCP Offload request done callback
+ * function. Pointed to by nvme_tcp_ofld_req->done.
+ * @req:	NVMeTCP offload request to complete.
+ * @result:     The nvme_result.
+ * @status:     The completion status.
+ *
+ * API function that allows the vendor specific offload driver to report request
+ * completions to the common offload layer.
+ */
+void
+nvme_tcp_ofld_req_done(struct nvme_tcp_ofld_req *req,
+		       union nvme_result *result,
+		       __le16 status)
+{
+	/* Placeholder - complete request with/without error */
+}
+
+static struct nvmf_transport_ops nvme_tcp_ofld_transport = {
+	.name		= "tcp_offload",
+	.module		= THIS_MODULE,
+	.required_opts	= NVMF_OPT_TRADDR,
+	.allowed_opts	= NVMF_OPT_TRSVCID | NVMF_OPT_DISABLE_SQFLOW |
+			  NVMF_OPT_NR_WRITE_QUEUES | NVMF_OPT_HOST_TRADDR |
+			  NVMF_OPT_CTRL_LOSS_TMO | NVMF_OPT_RECONNECT_DELAY |
+			  NVMF_OPT_HDR_DIGEST | NVMF_OPT_DATA_DIGEST |
+			  NVMF_OPT_NR_POLL_QUEUES | NVMF_OPT_TOS,
+};
+
+static int __init nvme_tcp_ofld_init_module(void)
+{
+	nvmf_register_transport(&nvme_tcp_ofld_transport);
+
+	return 0;
+}
+
+static void __exit nvme_tcp_ofld_cleanup_module(void)
+{
+	nvmf_unregister_transport(&nvme_tcp_ofld_transport);
+}
+
+module_init(nvme_tcp_ofld_init_module);
+module_exit(nvme_tcp_ofld_cleanup_module);
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/nvme/host/tcp-offload.h b/drivers/nvme/host/tcp-offload.h
new file mode 100644
index 000000000000..468617a58e34
--- /dev/null
+++ b/drivers/nvme/host/tcp-offload.h
@@ -0,0 +1,167 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+
+/* Linux includes */
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+#include <linux/types.h>
+#include <linux/nvme-tcp.h>
+
+/* Driver includes */
+#include "nvme.h"
+#include "fabrics.h"
+
+/* Forward declarations */
+struct nvme_tcp_ofld_ops;
+
+/* Representation of a vendor-specific device. This is the struct used to
+ * register to the offload layer by the vendor-specific driver during its probe
+ * function.
+ * Allocated by vendor-specific driver.
+ */
+struct nvme_tcp_ofld_dev {
+	struct list_head entry;
+	struct nvme_tcp_ofld_ops *ops;
+};
+
+/* Per IO struct holding the nvme_request and command
+ * Allocated by blk-mq.
+ */
+struct nvme_tcp_ofld_req {
+	struct nvme_request req;
+	struct nvme_command nvme_cmd;
+	struct nvme_tcp_ofld_queue *queue;
+
+	/* Vendor specific driver context */
+	void *private_data;
+
+	void (*done)(struct nvme_tcp_ofld_req *req,
+		     union nvme_result *result,
+		     __le16 status);
+};
+
+/* Allocated by nvme_tcp_ofld */
+struct nvme_tcp_ofld_queue {
+	/* Offload device associated to this queue */
+	struct nvme_tcp_ofld_dev *dev;
+	struct nvme_tcp_ofld_ctrl *ctrl;
+
+	/* Vendor specific driver context */
+	void *private_data;
+
+	/* Error callback function */
+	int (*report_err)(struct nvme_tcp_ofld_queue *queue);
+};
+
+/* Connectivity (routing) params used for establishing a connection */
+struct nvme_tcp_ofld_ctrl_con_params {
+	/* Input params */
+	struct sockaddr_storage remote_ip_addr;
+
+	/* If NVMF_OPT_HOST_TRADDR is provided it will be set in local_ip_addr
+	 * in nvme_tcp_ofld_create_ctrl().
+	 * If NVMF_OPT_HOST_TRADDR is not provided the local_ip_addr will be
+	 * initialized by claim_dev().
+	 */
+	struct sockaddr_storage local_ip_addr;
+
+	/* Output params */
+	struct sockaddr	remote_mac_addr;
+	struct sockaddr	local_mac_addr;
+	u16 vlan_id;
+};
+
+/* Allocated by nvme_tcp_ofld */
+struct nvme_tcp_ofld_ctrl {
+	struct nvme_ctrl nctrl;
+	struct nvme_tcp_ofld_dev *dev;
+
+	/* admin and IO queues */
+	struct blk_mq_tag_set tag_set;
+	struct blk_mq_tag_set admin_tag_set;
+	struct nvme_tcp_ofld_queue *queues;
+
+	/* Connectivity params */
+	struct nvme_tcp_ofld_ctrl_con_params conn_params;
+
+	/* Vendor specific driver context */
+	void *private_data;
+};
+
+struct nvme_tcp_ofld_ops {
+	const char *name;
+	struct module *module;
+
+	/* For vendor-specific driver to report what opts it supports */
+	int required_opts; /* bitmap using enum nvmf_parsing_opts */
+	int allowed_opts; /* bitmap using enum nvmf_parsing_opts */
+
+	/**
+	 * claim_dev: Return True if addr is reachable via offload device.
+	 * @dev: The offload device to check.
+	 * @conn_params: ptr to routing params to be filled by the lower
+	 *               driver. Input+Output argument.
+	 */
+	int (*claim_dev)(struct nvme_tcp_ofld_dev *dev,
+			 struct nvme_tcp_ofld_ctrl_con_params *conn_params);
+
+	/**
+	 * create_queue: Create offload queue and establish TCP + NVMeTCP
+	 * (icreq+icresp) connection. Return true on successful connection.
+	 * Based on nvme_tcp_alloc_queue.
+	 * @queue: The queue itself - used as input and output.
+	 * @qid: The queue ID associated with the requested queue.
+	 * @q_size: The queue depth.
+	 */
+	int (*create_queue)(struct nvme_tcp_ofld_queue *queue, int qid,
+			    size_t q_size);
+
+	/**
+	 * drain_queue: Drain a given queue - Returning from this function
+	 * ensures that no additional completions will arrive on this queue.
+	 * @queue: The queue to drain.
+	 */
+	void (*drain_queue)(struct nvme_tcp_ofld_queue *queue);
+
+	/**
+	 * destroy_queue: Close the TCP + NVMeTCP connection of a given queue
+	 * and make sure its no longer active (no completions will arrive on the
+	 * queue).
+	 * @queue: The queue to destroy.
+	 */
+	void (*destroy_queue)(struct nvme_tcp_ofld_queue *queue);
+
+	/**
+	 * poll_queue: Poll a given queue for completions.
+	 * @queue: The queue to poll.
+	 */
+	int (*poll_queue)(struct nvme_tcp_ofld_queue *queue);
+
+	/**
+	 * init_req: Initialize vendor-specific params for a new request.
+	 * @req: Ptr to request to be initialized. Input+Output argument.
+	 */
+	int (*init_req)(struct nvme_tcp_ofld_req *req);
+
+	/**
+	 * send_req: Dispatch a request. Returns the execution status.
+	 * @req: Ptr to request to be sent.
+	 */
+	int (*send_req)(struct nvme_tcp_ofld_req *req);
+
+	/**
+	 * map_sg: Map a scatter/gather list to DMA addresses Returns the
+	 * number of SGs entries mapped successfully.
+	 * @dev: The device for which the DMA addresses are to be created.
+	 * @req: The request corresponding to the SGs, to allow vendor-specific
+	 *       driver to initialize additional params if it needs to.
+	 */
+	int (*map_sg)(struct nvme_tcp_ofld_dev *dev,
+		      struct nvme_tcp_ofld_req *req);
+};
+
+/* Exported functions for lower vendor specific offload drivers */
+int nvme_tcp_ofld_register_dev(struct nvme_tcp_ofld_dev *dev);
+void nvme_tcp_ofld_unregister_dev(struct nvme_tcp_ofld_dev *dev);
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:18 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:18 +0200
Subject: [RFC PATCH v3 05/11] nvme-tcp-offload: Add controller level error
 recovery implementation
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-6-smalin@marvell.com>

From: Arie Gershberg <agershberg at marvell.com>

In this patch, we implement controller level error handling and recovery.
Upon an error discovered by the ULP or reset controller initiated by the
nvme-core (using reset_ctrl workqueue), the ULP will initiate a controller
recovery which includes teardown and re-connect of all queues.

Signed-off-by: Arie Gershberg <agershberg at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/host/tcp-offload.c | 121 +++++++++++++++++++++++++++++++-
 drivers/nvme/host/tcp-offload.h |  10 +++
 2 files changed, 130 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c
index da934c6fed9e..e3759c281927 100644
--- a/drivers/nvme/host/tcp-offload.c
+++ b/drivers/nvme/host/tcp-offload.c
@@ -73,6 +73,23 @@ void nvme_tcp_ofld_unregister_dev(struct nvme_tcp_ofld_dev *dev)
 }
 EXPORT_SYMBOL_GPL(nvme_tcp_ofld_unregister_dev);
 
+/**
+ * nvme_tcp_ofld_error_recovery() - NVMeTCP Offload Library error recovery.
+ * function.
+ * @nctrl:	NVMe controller instance to change to resetting.
+ *
+ * API function that change the controller state to resseting.
+ * Part of the overall controller reset sequence.
+ */
+void nvme_tcp_ofld_error_recovery(struct nvme_ctrl *nctrl)
+{
+	if (!nvme_change_ctrl_state(nctrl, NVME_CTRL_RESETTING))
+		return;
+
+	queue_work(nvme_reset_wq, &to_tcp_ofld_ctrl(nctrl)->err_work);
+}
+EXPORT_SYMBOL_GPL(nvme_tcp_ofld_error_recovery);
+
 /**
  * nvme_tcp_ofld_report_queue_err() - NVMeTCP Offload report error event
  * callback function. Pointed to by nvme_tcp_ofld_queue->report_err.
@@ -291,6 +308,27 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new)
 	return rc;
 }
 
+static void nvme_tcp_ofld_reconnect_or_remove(struct nvme_ctrl *nctrl)
+{
+	/* If we are resetting/deleting then do nothing */
+	if (nctrl->state != NVME_CTRL_CONNECTING) {
+		WARN_ON_ONCE(nctrl->state == NVME_CTRL_NEW ||
+			     nctrl->state == NVME_CTRL_LIVE);
+		return;
+	}
+
+	if (nvmf_should_reconnect(nctrl)) {
+		dev_info(nctrl->device, "Reconnecting in %d seconds...\n",
+			 nctrl->opts->reconnect_delay);
+		queue_delayed_work(nvme_wq,
+				   &to_tcp_ofld_ctrl(nctrl)->connect_work,
+				   nctrl->opts->reconnect_delay * HZ);
+	} else {
+		dev_info(nctrl->device, "Removing controller...\n");
+		nvme_delete_ctrl(nctrl);
+	}
+}
+
 static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new)
 {
 	struct nvmf_ctrl_options *opts = nctrl->opts;
@@ -399,10 +437,62 @@ nvme_tcp_ofld_teardown_io_queues(struct nvme_ctrl *nctrl, bool remove)
 	/* Placeholder - teardown_io_queues */
 }
 
+static void nvme_tcp_ofld_reconnect_ctrl_work(struct work_struct *work)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl =
+				container_of(to_delayed_work(work),
+					     struct nvme_tcp_ofld_ctrl,
+					     connect_work);
+	struct nvme_ctrl *nctrl = &ctrl->nctrl;
+
+	++nctrl->nr_reconnects;
+
+	if (nvme_tcp_ofld_setup_ctrl(nctrl, false))
+		goto requeue;
+
+	dev_info(nctrl->device, "Successfully reconnected (%d attempt)\n",
+		 nctrl->nr_reconnects);
+
+	nctrl->nr_reconnects = 0;
+
+	return;
+
+requeue:
+	dev_info(nctrl->device, "Failed reconnect attempt %d\n",
+		 nctrl->nr_reconnects);
+	nvme_tcp_ofld_reconnect_or_remove(nctrl);
+}
+
+static void nvme_tcp_ofld_error_recovery_work(struct work_struct *work)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl =
+		container_of(work, struct nvme_tcp_ofld_ctrl, err_work);
+	struct nvme_ctrl *nctrl = &ctrl->nctrl;
+
+	nvme_stop_keep_alive(nctrl);
+	nvme_tcp_ofld_teardown_io_queues(nctrl, false);
+	/* unquiesce to fail fast pending requests */
+	nvme_start_queues(nctrl);
+	nvme_tcp_ofld_teardown_admin_queue(nctrl, false);
+	blk_mq_unquiesce_queue(nctrl->admin_q);
+
+	if (!nvme_change_ctrl_state(nctrl, NVME_CTRL_CONNECTING)) {
+		/* state change failure is ok if we started nctrl delete */
+		WARN_ON_ONCE(nctrl->state != NVME_CTRL_DELETING &&
+			     nctrl->state != NVME_CTRL_DELETING_NOIO);
+		return;
+	}
+
+	nvme_tcp_ofld_reconnect_or_remove(nctrl);
+}
+
 static void
 nvme_tcp_ofld_teardown_ctrl(struct nvme_ctrl *nctrl, bool shutdown)
 {
-	/* Placeholder - err_work and connect_work */
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+
+	cancel_work_sync(&ctrl->err_work);
+	cancel_delayed_work_sync(&ctrl->connect_work);
 	nvme_tcp_ofld_teardown_io_queues(nctrl, shutdown);
 	blk_mq_quiesce_queue(nctrl->admin_q);
 	if (shutdown)
@@ -417,6 +507,31 @@ static void nvme_tcp_ofld_delete_ctrl(struct nvme_ctrl *nctrl)
 	nvme_tcp_ofld_teardown_ctrl(nctrl, true);
 }
 
+static void nvme_tcp_ofld_reset_ctrl_work(struct work_struct *work)
+{
+	struct nvme_ctrl *nctrl =
+		container_of(work, struct nvme_ctrl, reset_work);
+
+	nvme_stop_ctrl(nctrl);
+	nvme_tcp_ofld_teardown_ctrl(nctrl, false);
+
+	if (!nvme_change_ctrl_state(nctrl, NVME_CTRL_CONNECTING)) {
+		/* state change failure is ok if we started ctrl delete */
+		WARN_ON_ONCE(nctrl->state != NVME_CTRL_DELETING &&
+			     nctrl->state != NVME_CTRL_DELETING_NOIO);
+		return;
+	}
+
+	if (nvme_tcp_ofld_setup_ctrl(nctrl, false))
+		goto out_fail;
+
+	return;
+
+out_fail:
+	++nctrl->nr_reconnects;
+	nvme_tcp_ofld_reconnect_or_remove(nctrl);
+}
+
 static int
 nvme_tcp_ofld_init_request(struct blk_mq_tag_set *set,
 			   struct request *rq,
@@ -513,6 +628,10 @@ nvme_tcp_ofld_create_ctrl(struct device *ndev, struct nvmf_ctrl_options *opts)
 			     opts->nr_poll_queues + 1;
 	nctrl->sqsize = opts->queue_size - 1;
 	nctrl->kato = opts->kato;
+	INIT_DELAYED_WORK(&ctrl->connect_work,
+			  nvme_tcp_ofld_reconnect_ctrl_work);
+	INIT_WORK(&ctrl->err_work, nvme_tcp_ofld_error_recovery_work);
+	INIT_WORK(&nctrl->reset_work, nvme_tcp_ofld_reset_ctrl_work);
 	if (!(opts->mask & NVMF_OPT_TRSVCID)) {
 		opts->trsvcid =
 			kstrdup(__stringify(NVME_TCP_DISC_PORT), GFP_KERNEL);
diff --git a/drivers/nvme/host/tcp-offload.h b/drivers/nvme/host/tcp-offload.h
index a9bcd4f7d150..0fb212f9193a 100644
--- a/drivers/nvme/host/tcp-offload.h
+++ b/drivers/nvme/host/tcp-offload.h
@@ -84,6 +84,15 @@ struct nvme_tcp_ofld_ctrl {
 	struct blk_mq_tag_set admin_tag_set;
 	struct nvme_tcp_ofld_queue *queues;
 
+	struct work_struct err_work;
+	struct delayed_work connect_work;
+
+	/*
+	 * Each entry in the array indicates the number of queues of
+	 * corresponding type.
+	 */
+	u32 queue_type_mapping[HCTX_MAX_TYPES];
+
 	/* Connectivity params */
 	struct nvme_tcp_ofld_ctrl_con_params conn_params;
 
@@ -166,3 +175,4 @@ struct nvme_tcp_ofld_ops {
 /* Exported functions for lower vendor specific offload drivers */
 int nvme_tcp_ofld_register_dev(struct nvme_tcp_ofld_dev *dev);
 void nvme_tcp_ofld_unregister_dev(struct nvme_tcp_ofld_dev *dev);
+void nvme_tcp_ofld_error_recovery(struct nvme_ctrl *nctrl);
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:19 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:19 +0200
Subject: [RFC PATCH v3 06/11] nvme-tcp-offload: Add queue level implementation
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-7-smalin@marvell.com>

From: Dean Balandin <dbalandin at marvell.com>

In this patch we implement queue level functionality.
The implementation is similar to the nvme-tcp module, the main
difference being that we call the vendor specific create_queue op which
creates the TCP connection, and NVMeTPC connection including
icreq+icresp negotiation.
Once create_queue returns successfully, we can move on to the fabrics
connect.

Signed-off-by: Dean Balandin <dbalandin at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/host/tcp-offload.c | 322 ++++++++++++++++++++++++++++++--
 drivers/nvme/host/tcp-offload.h |   7 +
 2 files changed, 310 insertions(+), 19 deletions(-)

diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c
index e3759c281927..f1023f2eae8d 100644
--- a/drivers/nvme/host/tcp-offload.c
+++ b/drivers/nvme/host/tcp-offload.c
@@ -23,6 +23,11 @@ to_tcp_ofld_ctrl(struct nvme_ctrl *nctrl)
 	return container_of(nctrl, struct nvme_tcp_ofld_ctrl, nctrl);
 }
 
+static inline int nvme_tcp_ofld_qid(struct nvme_tcp_ofld_queue *queue)
+{
+	return queue - queue->ctrl->queues;
+}
+
 /**
  * nvme_tcp_ofld_register_dev() - NVMeTCP Offload Library registration
  * function.
@@ -190,12 +195,97 @@ nvme_tcp_ofld_alloc_tagset(struct nvme_ctrl *nctrl, bool admin)
 	return set;
 }
 
+static bool nvme_tcp_ofld_poll_queue(struct nvme_tcp_ofld_queue *queue)
+{
+	/* Placeholder - implement logic to determine if poll queue */
+
+	return false;
+}
+
+static void __nvme_tcp_ofld_stop_queue(struct nvme_tcp_ofld_queue *queue)
+{
+	queue->dev->ops->drain_queue(queue);
+	queue->dev->ops->destroy_queue(queue);
+
+	/* Placeholder - additional cleanup such as cancel_work_sync io_work */
+	clear_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags);
+}
+
+static void nvme_tcp_ofld_stop_queue(struct nvme_ctrl *nctrl, int qid)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid];
+
+	if (!test_and_clear_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags))
+		return;
+
+	__nvme_tcp_ofld_stop_queue(queue);
+}
+
+static void nvme_tcp_ofld_free_queue(struct nvme_ctrl *nctrl, int qid)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid];
+
+	if (!test_and_clear_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags))
+		return;
+
+	/* Placeholder - additional queue cleanup */
+}
+
+static void
+nvme_tcp_ofld_terminate_admin_queue(struct nvme_ctrl *nctrl, bool remove)
+{
+	nvme_tcp_ofld_stop_queue(nctrl, 0);
+	if (remove) {
+		if (nctrl->admin_q && !blk_queue_dead(nctrl->admin_q))
+			blk_cleanup_queue(nctrl->admin_q);
+
+		if (nctrl->fabrics_q)
+			blk_cleanup_queue(nctrl->fabrics_q);
+
+		if (nctrl->admin_tagset)
+			blk_mq_free_tag_set(nctrl->admin_tagset);
+	}
+}
+
+static int nvme_tcp_ofld_start_queue(struct nvme_ctrl *nctrl, int qid)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	struct nvme_tcp_ofld_queue *queue = &ctrl->queues[qid];
+	int rc;
+
+	queue = &ctrl->queues[qid];
+	if (qid)
+		rc = nvmf_connect_io_queue(nctrl, qid,
+					   nvme_tcp_ofld_poll_queue(queue));
+	else
+		rc = nvmf_connect_admin_queue(nctrl);
+
+	if (!rc) {
+		set_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags);
+	} else {
+		if (test_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags))
+			__nvme_tcp_ofld_stop_queue(queue);
+		dev_err(nctrl->device,
+			"failed to connect queue: %d ret=%d\n", qid, rc);
+	}
+
+	return rc;
+}
+
 static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl,
 					       bool new)
 {
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	struct nvme_tcp_ofld_queue *queue = &ctrl->queues[0];
 	int rc;
 
-	/* Placeholder - alloc_admin_queue */
+	rc = ctrl->dev->ops->create_queue(queue, 0, NVME_AQ_DEPTH);
+	if (rc)
+		return rc;
+
+	set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &queue->flags);
 	if (new) {
 		nctrl->admin_tagset =
 				nvme_tcp_ofld_alloc_tagset(nctrl, true);
@@ -220,7 +310,9 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl,
 		}
 	}
 
-	/* Placeholder - nvme_tcp_ofld_start_queue */
+	rc = nvme_tcp_ofld_start_queue(nctrl, 0);
+	if (rc)
+		goto out_cleanup_queue;
 
 	rc = nvme_enable_ctrl(nctrl);
 	if (rc)
@@ -235,8 +327,10 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl,
 	return 0;
 
 out_stop_queue:
-	/* Placeholder - stop offload queue */
-
+	nvme_tcp_ofld_stop_queue(nctrl, 0);
+out_cleanup_queue:
+	if (new)
+		blk_cleanup_queue(nctrl->admin_q);
 out_cleanup_fabrics_q:
 	if (new)
 		blk_cleanup_queue(nctrl->fabrics_q);
@@ -244,7 +338,123 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl,
 	if (new)
 		blk_mq_free_tag_set(nctrl->admin_tagset);
 out_free_queue:
-	/* Placeholder - free admin queue */
+	nvme_tcp_ofld_free_queue(nctrl, 0);
+
+	return rc;
+}
+
+static unsigned int nvme_tcp_ofld_nr_io_queues(struct nvme_ctrl *nctrl)
+{
+	unsigned int nr_io_queues;
+
+	nr_io_queues = min(nctrl->opts->nr_io_queues, num_online_cpus());
+	nr_io_queues += min(nctrl->opts->nr_write_queues, num_online_cpus());
+	nr_io_queues += min(nctrl->opts->nr_poll_queues, num_online_cpus());
+
+	return nr_io_queues;
+}
+
+static void
+nvme_tcp_ofld_set_io_queues(struct nvme_ctrl *nctrl, unsigned int nr_io_queues)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	struct nvmf_ctrl_options *opts = nctrl->opts;
+
+	if (opts->nr_write_queues && opts->nr_io_queues < nr_io_queues) {
+		/*
+		 * separate read/write queues
+		 * hand out dedicated default queues only after we have
+		 * sufficient read queues.
+		 */
+		ctrl->io_queues[HCTX_TYPE_READ] = opts->nr_io_queues;
+		nr_io_queues -= ctrl->io_queues[HCTX_TYPE_READ];
+		ctrl->io_queues[HCTX_TYPE_DEFAULT] =
+			min(opts->nr_write_queues, nr_io_queues);
+		nr_io_queues -= ctrl->io_queues[HCTX_TYPE_DEFAULT];
+	} else {
+		/*
+		 * shared read/write queues
+		 * either no write queues were requested, or we don't have
+		 * sufficient queue count to have dedicated default queues.
+		 */
+		ctrl->io_queues[HCTX_TYPE_DEFAULT] =
+			min(opts->nr_io_queues, nr_io_queues);
+		nr_io_queues -= ctrl->io_queues[HCTX_TYPE_DEFAULT];
+	}
+
+	if (opts->nr_poll_queues && nr_io_queues) {
+		/* map dedicated poll queues only if we have queues left */
+		ctrl->io_queues[HCTX_TYPE_POLL] =
+			min(opts->nr_poll_queues, nr_io_queues);
+	}
+}
+
+static void
+nvme_tcp_ofld_terminate_io_queues(struct nvme_ctrl *nctrl, int start_from)
+{
+	int i;
+
+	/* admin-q will be ignored because of the loop condition */
+	for (i = start_from; i >= 1; i--)
+		nvme_tcp_ofld_stop_queue(nctrl, i);
+}
+
+static int nvme_tcp_ofld_create_io_queues(struct nvme_ctrl *nctrl)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = to_tcp_ofld_ctrl(nctrl);
+	int i, rc;
+
+	for (i = 1; i < nctrl->queue_count; i++) {
+		rc = ctrl->dev->ops->create_queue(&ctrl->queues[i],
+						  i, nctrl->sqsize + 1);
+		if (rc)
+			goto out_free_queues;
+
+		set_bit(NVME_TCP_OFLD_Q_ALLOCATED, &ctrl->queues[i].flags);
+	}
+
+	return 0;
+
+out_free_queues:
+	nvme_tcp_ofld_terminate_io_queues(nctrl, --i);
+
+	return rc;
+}
+
+static int nvme_tcp_ofld_alloc_io_queues(struct nvme_ctrl *nctrl)
+{
+	unsigned int nr_io_queues;
+	int rc;
+
+	nr_io_queues = nvme_tcp_ofld_nr_io_queues(nctrl);
+	rc = nvme_set_queue_count(nctrl, &nr_io_queues);
+	if (rc)
+		return rc;
+
+	nctrl->queue_count = nr_io_queues + 1;
+	if (nctrl->queue_count < 2)
+		return 0;
+
+	dev_info(nctrl->device, "creating %d I/O queues.\n", nr_io_queues);
+	nvme_tcp_ofld_set_io_queues(nctrl, nr_io_queues);
+
+	return nvme_tcp_ofld_create_io_queues(nctrl);
+}
+
+static int nvme_tcp_ofld_start_io_queues(struct nvme_ctrl *nctrl)
+{
+	int i, rc = 0;
+
+	for (i = 1; i < nctrl->queue_count; i++) {
+		rc = nvme_tcp_ofld_start_queue(nctrl, i);
+		if (rc)
+			goto terminate_queues;
+	}
+
+	return 0;
+
+terminate_queues:
+	nvme_tcp_ofld_terminate_io_queues(nctrl, --i);
 
 	return rc;
 }
@@ -252,9 +462,10 @@ static int nvme_tcp_ofld_configure_admin_queue(struct nvme_ctrl *nctrl,
 static int
 nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new)
 {
-	int rc;
+	int rc = nvme_tcp_ofld_alloc_io_queues(nctrl);
 
-	/* Placeholder - alloc_io_queues */
+	if (rc)
+		return rc;
 
 	if (new) {
 		nctrl->tagset = nvme_tcp_ofld_alloc_tagset(nctrl, false);
@@ -272,7 +483,9 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new)
 		}
 	}
 
-	/* Placeholder - start_io_queues */
+	rc = nvme_tcp_ofld_start_io_queues(nctrl);
+	if (rc)
+		goto out_cleanup_connect_q;
 
 	if (!new) {
 		nvme_start_queues(nctrl);
@@ -296,6 +509,7 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new)
 
 	/* Placeholder - Stop IO queues */
 
+out_cleanup_connect_q:
 	if (new)
 		blk_cleanup_queue(nctrl->connect_q);
 out_free_tag_set:
@@ -303,7 +517,7 @@ nvme_tcp_ofld_configure_io_queues(struct nvme_ctrl *nctrl, bool new)
 		blk_mq_free_tag_set(nctrl->tagset);
 
 out_free_io_queues:
-	/* Placeholder - free_io_queues */
+	nvme_tcp_ofld_terminate_io_queues(nctrl, nctrl->queue_count);
 
 	return rc;
 }
@@ -379,9 +593,9 @@ static int nvme_tcp_ofld_setup_ctrl(struct nvme_ctrl *nctrl, bool new)
 	return 0;
 
 destroy_io:
-	/* Placeholder - stop and destroy io queues*/
+	nvme_tcp_ofld_terminate_io_queues(nctrl, nctrl->queue_count);
 destroy_admin:
-	/* Placeholder - stop and destroy admin queue*/
+	nvme_tcp_ofld_terminate_admin_queue(nctrl, new);
 
 	return rc;
 }
@@ -560,22 +774,92 @@ nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx,
 	return BLK_STS_OK;
 }
 
+static void
+nvme_tcp_ofld_exit_request(struct blk_mq_tag_set *set,
+			   struct request *rq, unsigned int hctx_idx)
+{
+	/* Placeholder */
+}
+
+static int
+nvme_tcp_ofld_init_hctx(struct blk_mq_hw_ctx *hctx, void *data,
+			unsigned int hctx_idx)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = data;
+	struct nvme_tcp_ofld_queue *queue = &ctrl->queues[hctx_idx + 1];
+
+	hctx->driver_data = queue;
+	return 0;
+}
+
+static int nvme_tcp_ofld_map_queues(struct blk_mq_tag_set *set)
+{
+	struct nvme_tcp_ofld_ctrl *ctrl = set->driver_data;
+	struct nvmf_ctrl_options *opts = ctrl->nctrl.opts;
+
+	if (opts->nr_write_queues && ctrl->queue_type_mapping[HCTX_TYPE_READ]) {
+		/* separate read/write queues */
+		set->map[HCTX_TYPE_DEFAULT].nr_queues =
+			ctrl->queue_type_mapping[HCTX_TYPE_DEFAULT];
+		set->map[HCTX_TYPE_DEFAULT].queue_offset = 0;
+		set->map[HCTX_TYPE_READ].nr_queues =
+			ctrl->queue_type_mapping[HCTX_TYPE_READ];
+		set->map[HCTX_TYPE_READ].queue_offset =
+			ctrl->queue_type_mapping[HCTX_TYPE_DEFAULT];
+	} else {
+		/* shared read/write queues */
+		set->map[HCTX_TYPE_DEFAULT].nr_queues =
+			ctrl->queue_type_mapping[HCTX_TYPE_DEFAULT];
+		set->map[HCTX_TYPE_DEFAULT].queue_offset = 0;
+		set->map[HCTX_TYPE_READ].nr_queues =
+			ctrl->queue_type_mapping[HCTX_TYPE_DEFAULT];
+		set->map[HCTX_TYPE_READ].queue_offset = 0;
+	}
+	blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]);
+	blk_mq_map_queues(&set->map[HCTX_TYPE_READ]);
+
+	if (opts->nr_poll_queues && ctrl->queue_type_mapping[HCTX_TYPE_POLL]) {
+		/* map dedicated poll queues only if we have queues left */
+		set->map[HCTX_TYPE_POLL].nr_queues =
+				ctrl->queue_type_mapping[HCTX_TYPE_POLL];
+		set->map[HCTX_TYPE_POLL].queue_offset =
+			ctrl->queue_type_mapping[HCTX_TYPE_DEFAULT] +
+			ctrl->queue_type_mapping[HCTX_TYPE_READ];
+		blk_mq_map_queues(&set->map[HCTX_TYPE_POLL]);
+	}
+
+	dev_info(ctrl->nctrl.device,
+		 "mapped %d/%d/%d default/read/poll queues.\n",
+		 ctrl->queue_type_mapping[HCTX_TYPE_DEFAULT],
+		 ctrl->queue_type_mapping[HCTX_TYPE_READ],
+		 ctrl->queue_type_mapping[HCTX_TYPE_POLL]);
+
+	return 0;
+}
+
+static int nvme_tcp_ofld_poll(struct blk_mq_hw_ctx *hctx)
+{
+	/* Placeholder - Implement polling mechanism */
+
+	return 0;
+}
+
 static struct blk_mq_ops nvme_tcp_ofld_mq_ops = {
 	.queue_rq	= nvme_tcp_ofld_queue_rq,
 	.init_request	= nvme_tcp_ofld_init_request,
-	/*
-	 * All additional ops will be also implemented and registered similar to
-	 * tcp.c
-	 */
+	.complete	= nvme_complete_rq,
+	.exit_request	= nvme_tcp_ofld_exit_request,
+	.init_hctx	= nvme_tcp_ofld_init_hctx,
+	.map_queues	= nvme_tcp_ofld_map_queues,
+	.poll		= nvme_tcp_ofld_poll,
 };
 
 static struct blk_mq_ops nvme_tcp_ofld_admin_mq_ops = {
 	.queue_rq	= nvme_tcp_ofld_queue_rq,
 	.init_request	= nvme_tcp_ofld_init_request,
-	/*
-	 * All additional ops will be also implemented and registered similar to
-	 * tcp.c
-	 */
+	.complete	= nvme_complete_rq,
+	.exit_request	= nvme_tcp_ofld_exit_request,
+	.init_hctx	= nvme_tcp_ofld_init_hctx,
 };
 
 static const struct nvme_ctrl_ops nvme_tcp_ofld_ctrl_ops = {
diff --git a/drivers/nvme/host/tcp-offload.h b/drivers/nvme/host/tcp-offload.h
index 0fb212f9193a..65c33ad97da2 100644
--- a/drivers/nvme/host/tcp-offload.h
+++ b/drivers/nvme/host/tcp-offload.h
@@ -42,11 +42,17 @@ struct nvme_tcp_ofld_req {
 		     __le16 status);
 };
 
+enum nvme_tcp_ofld_queue_flags {
+	NVME_TCP_OFLD_Q_ALLOCATED = 0,
+	NVME_TCP_OFLD_Q_LIVE = 1,
+};
+
 /* Allocated by nvme_tcp_ofld */
 struct nvme_tcp_ofld_queue {
 	/* Offload device associated to this queue */
 	struct nvme_tcp_ofld_dev *dev;
 	struct nvme_tcp_ofld_ctrl *ctrl;
+	unsigned long flags;
 
 	/* Vendor specific driver context */
 	void *private_data;
@@ -92,6 +98,7 @@ struct nvme_tcp_ofld_ctrl {
 	 * corresponding type.
 	 */
 	u32 queue_type_mapping[HCTX_MAX_TYPES];
+	u32 io_queues[HCTX_MAX_TYPES];
 
 	/* Connectivity params */
 	struct nvme_tcp_ofld_ctrl_con_params conn_params;
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:20 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:20 +0200
Subject: [RFC PATCH v3 07/11] nvme-tcp-offload: Add IO level implementation
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-8-smalin@marvell.com>

From: Dean Balandin <dbalandin at marvell.com>

In this patch, we present the IO level functionality.
The nvme-tcp-offload shall work on the IO-level, meaning the
nvme-tcp-offload ULP module shall pass the request to the nvme-tcp-offload
vendor driver and shall expect for the request compilation.
No additional handling is needed in between, this design will reduce the
CPU utilization as we will describe below.

The nvme-tcp-offload vendor driver shall register to nvme-tcp-offload ULP
with the following IO-path ops:
 - init_req
 - map_sg - in order to map the request sg (similar to nvme_rdma_map_data)
 - send_req - in order to pass the request to the handling of the offload
   driver that shall pass it to the vendor specific device
 - poll_queue

The vendor driver will manage the context from which the request will be
executed and the request aggregations.
Once the IO completed, the nvme-tcp-offload vendor driver shall call
command.done() that shall invoke the nvme-tcp-offload ULP layer for
completing the request.

Signed-off-by: Dean Balandin <dbalandin at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/host/tcp-offload.c | 82 ++++++++++++++++++++++++++++++---
 1 file changed, 75 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/tcp-offload.c b/drivers/nvme/host/tcp-offload.c
index f1023f2eae8d..bf64551c7a91 100644
--- a/drivers/nvme/host/tcp-offload.c
+++ b/drivers/nvme/host/tcp-offload.c
@@ -113,6 +113,7 @@ int nvme_tcp_ofld_report_queue_err(struct nvme_tcp_ofld_queue *queue)
 /**
  * nvme_tcp_ofld_req_done() - NVMeTCP Offload request done callback
  * function. Pointed to by nvme_tcp_ofld_req->done.
+ * Handles both NVME_TCP_F_DATA_SUCCESS flag and NVMe CQE.
  * @req:	NVMeTCP offload request to complete.
  * @result:     The nvme_result.
  * @status:     The completion status.
@@ -125,7 +126,10 @@ nvme_tcp_ofld_req_done(struct nvme_tcp_ofld_req *req,
 		       union nvme_result *result,
 		       __le16 status)
 {
-	/* Placeholder - complete request with/without error */
+	struct request *rq = blk_mq_rq_from_pdu(req);
+
+	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), *result))
+		nvme_complete_rq(rq);
 }
 
 struct nvme_tcp_ofld_dev *
@@ -754,9 +758,10 @@ nvme_tcp_ofld_init_request(struct blk_mq_tag_set *set,
 {
 	struct nvme_tcp_ofld_req *req = blk_mq_rq_to_pdu(rq);
 	struct nvme_tcp_ofld_ctrl *ctrl = set->driver_data;
+	int qid = (set == &ctrl->tag_set) ? hctx_idx + 1 : 0;
 
-	/* Placeholder - init request */
-
+	req->queue = &ctrl->queues[qid];
+	nvme_req(rq)->ctrl = &ctrl->nctrl;
 	req->done = nvme_tcp_ofld_req_done;
 	ctrl->dev->ops->init_req(req);
 
@@ -767,9 +772,32 @@ static blk_status_t
 nvme_tcp_ofld_queue_rq(struct blk_mq_hw_ctx *hctx,
 		       const struct blk_mq_queue_data *bd)
 {
-	/* Call nvme_setup_cmd(...) */
+	struct nvme_tcp_ofld_req *req = blk_mq_rq_to_pdu(bd->rq);
+	struct nvme_tcp_ofld_queue *queue = hctx->driver_data;
+	struct nvme_tcp_ofld_ctrl *ctrl = queue->ctrl;
+	struct nvme_ns *ns = hctx->queue->queuedata;
+	struct nvme_tcp_ofld_dev *dev = queue->dev;
+	struct nvme_tcp_ofld_ops *ops = dev->ops;
+	struct request *rq = bd->rq;
+	bool queue_ready;
+	int rc;
+
+	queue_ready = test_bit(NVME_TCP_OFLD_Q_LIVE, &queue->flags);
+	if (!nvmf_check_ready(&ctrl->nctrl, rq, queue_ready))
+		return nvmf_fail_nonready_command(&ctrl->nctrl, rq);
+
+	rc = nvme_setup_cmd(ns, rq, &req->nvme_cmd);
+	if (unlikely(rc))
+		return rc;
+
+	blk_mq_start_request(rq);
+	rc = ops->map_sg(dev, req);
+	if (unlikely(rc))
+		return rc;
 
-	/* Call ops->map_sg(...) */
+	rc = ops->send_req(req);
+	if (unlikely(rc))
+		return rc;
 
 	return BLK_STS_OK;
 }
@@ -839,9 +867,47 @@ static int nvme_tcp_ofld_map_queues(struct blk_mq_tag_set *set)
 
 static int nvme_tcp_ofld_poll(struct blk_mq_hw_ctx *hctx)
 {
-	/* Placeholder - Implement polling mechanism */
+	struct nvme_tcp_ofld_queue *queue = hctx->driver_data;
+	struct nvme_tcp_ofld_dev *dev = queue->dev;
+	struct nvme_tcp_ofld_ops *ops = dev->ops;
 
-	return 0;
+	return ops->poll_queue(queue);
+}
+
+static enum blk_eh_timer_return
+nvme_tcp_ofld_timeout(struct request *rq, bool reserved)
+{
+	struct nvme_tcp_ofld_req *req = blk_mq_rq_to_pdu(rq);
+	struct nvme_tcp_ofld_ctrl *ctrl = req->queue->ctrl;
+
+	/* Restart the timer if a controller reset is already scheduled. Any
+	 * timed out request would be handled before entering the connecting
+	 * state.
+	 */
+	if (ctrl->nctrl.state == NVME_CTRL_RESETTING)
+		return BLK_EH_RESET_TIMER;
+
+	dev_warn(ctrl->nctrl.device,
+		 "queue %d: timeout request %#x type %d\n",
+		 nvme_tcp_ofld_qid(req->queue), rq->tag,
+		 req->nvme_cmd.common.opcode);
+
+	if (ctrl->nctrl.state != NVME_CTRL_LIVE) {
+		/*
+		 * Teardown immediately if controller times out while starting
+		 * or we are already started error recovery. all outstanding
+		 * requests are completed on shutdown, so we return BLK_EH_DONE.
+		 */
+		flush_work(&ctrl->err_work);
+		nvme_tcp_ofld_teardown_io_queues(&ctrl->nctrl, false);
+		nvme_tcp_ofld_teardown_admin_queue(&ctrl->nctrl, false);
+		return BLK_EH_DONE;
+	}
+
+	dev_warn(ctrl->nctrl.device, "starting error recovery\n");
+	nvme_tcp_ofld_error_recovery(&ctrl->nctrl);
+
+	return BLK_EH_RESET_TIMER;
 }
 
 static struct blk_mq_ops nvme_tcp_ofld_mq_ops = {
@@ -851,6 +917,7 @@ static struct blk_mq_ops nvme_tcp_ofld_mq_ops = {
 	.exit_request	= nvme_tcp_ofld_exit_request,
 	.init_hctx	= nvme_tcp_ofld_init_hctx,
 	.map_queues	= nvme_tcp_ofld_map_queues,
+	.timeout	= nvme_tcp_ofld_timeout,
 	.poll		= nvme_tcp_ofld_poll,
 };
 
@@ -860,6 +927,7 @@ static struct blk_mq_ops nvme_tcp_ofld_admin_mq_ops = {
 	.complete	= nvme_complete_rq,
 	.exit_request	= nvme_tcp_ofld_exit_request,
 	.init_hctx	= nvme_tcp_ofld_init_hctx,
+	.timeout	= nvme_tcp_ofld_timeout,
 };
 
 static const struct nvme_ctrl_ops nvme_tcp_ofld_ctrl_ops = {
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:21 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:21 +0200
Subject: [RFC PATCH v3 08/11] nvme-qedn: Add qedn - Marvell's NVMeTCP HW
 offload vendor driver
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-9-smalin@marvell.com>

This patch will present the skeleton of the qedn driver.
The new driver will be added under "drivers/nvme/hw/qedn" and will be
enabled by the Kconfig "Marvell NVM Express over Fabrics TCP offload".

The internal implementation:
- qedn.h:
  Includes all common structs to be used by the qedn vendor driver.

- qedn.c
  Includes the qedn_init and qedn_cleanup implementation.
  As part of the qedn init, the driver will register as a pci device and
  will work with the Marvell fastlinQ NICs.
  As part of the probe, the driver will register to the nvme_tcp_offload
  (ULP).

Signed-off-by: Arie Gershberg <agershberg at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Ariel Elior <aelior at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 MAINTAINERS                   |  10 ++
 drivers/nvme/Kconfig          |   1 +
 drivers/nvme/Makefile         |   1 +
 drivers/nvme/hw/Kconfig       |   8 ++
 drivers/nvme/hw/Makefile      |   3 +
 drivers/nvme/hw/qedn/Makefile |   3 +
 drivers/nvme/hw/qedn/qedn.c   | 191 ++++++++++++++++++++++++++++++++++
 drivers/nvme/hw/qedn/qedn.h   |  14 +++
 8 files changed, 231 insertions(+)
 create mode 100644 drivers/nvme/hw/Kconfig
 create mode 100644 drivers/nvme/hw/Makefile
 create mode 100644 drivers/nvme/hw/qedn/Makefile
 create mode 100644 drivers/nvme/hw/qedn/qedn.c
 create mode 100644 drivers/nvme/hw/qedn/qedn.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 546aa66428c9..707623d92008 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14572,6 +14572,16 @@ S:	Supported
 F:	drivers/infiniband/hw/qedr/
 F:	include/uapi/rdma/qedr-abi.h
 
+QLOGIC QL4xxx NVME-TCP-OFFLOAD DRIVER
+M:	Shai Malin <smalin at marvell.com>
+M:	Ariel Elior <aelior at marvell.com>
+L:	linux-nvme at lists.infradead.org
+S:	Supported
+W:	http://git.infradead.org/nvme.git
+T:	git://git.infradead.org/nvme.git
+F:	drivers/nvme/hw/qedn/
+F:	include/linux/qed/
+
 QLOGIC QLA1280 SCSI DRIVER
 M:	Michael Reed <mdr at sgi.com>
 L:	linux-scsi at vger.kernel.org
diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
index 87ae409a32b9..827c2c9f0ad1 100644
--- a/drivers/nvme/Kconfig
+++ b/drivers/nvme/Kconfig
@@ -3,5 +3,6 @@ menu "NVME Support"
 
 source "drivers/nvme/host/Kconfig"
 source "drivers/nvme/target/Kconfig"
+source "drivers/nvme/hw/Kconfig"
 
 endmenu
diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
index fb42c44609a8..14c569040ef2 100644
--- a/drivers/nvme/Makefile
+++ b/drivers/nvme/Makefile
@@ -2,3 +2,4 @@
 
 obj-y		+= host/
 obj-y		+= target/
+obj-y		+= hw/
\ No newline at end of file
diff --git a/drivers/nvme/hw/Kconfig b/drivers/nvme/hw/Kconfig
new file mode 100644
index 000000000000..374f1f9dbd3d
--- /dev/null
+++ b/drivers/nvme/hw/Kconfig
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NVME_QEDN
+	tristate "Marvell NVM Express over Fabrics TCP offload"
+	depends on NVME_TCP_OFFLOAD
+	help
+	  This enables the Marvell NVMe TCP offload support (qedn).
+
+	  If unsure, say N.
diff --git a/drivers/nvme/hw/Makefile b/drivers/nvme/hw/Makefile
new file mode 100644
index 000000000000..7780ea5ab520
--- /dev/null
+++ b/drivers/nvme/hw/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+obj-$(CONFIG_NVME_QEDN)		+= qedn/
diff --git a/drivers/nvme/hw/qedn/Makefile b/drivers/nvme/hw/qedn/Makefile
new file mode 100644
index 000000000000..5f1bd2cd36da
--- /dev/null
+++ b/drivers/nvme/hw/qedn/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+obj-$(CONFIG_NVME_QEDN) := qedn.o
diff --git a/drivers/nvme/hw/qedn/qedn.c b/drivers/nvme/hw/qedn/qedn.c
new file mode 100644
index 000000000000..1fe33335989c
--- /dev/null
+++ b/drivers/nvme/hw/qedn/qedn.c
@@ -0,0 +1,191 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+ /* Kernel includes */
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+/* Driver includes */
+#include "qedn.h"
+
+#define CHIP_NUM_AHP_NVMETCP 0x8194
+
+static struct pci_device_id qedn_pci_tbl[] = {
+	{ PCI_VDEVICE(QLOGIC, CHIP_NUM_AHP_NVMETCP), 0 },
+	{0, 0},
+};
+
+static int
+qedn_claim_dev(struct nvme_tcp_ofld_dev *dev,
+	       struct nvme_tcp_ofld_ctrl_con_params *conn_params)
+{
+	/* Placeholder - qedn_claim_dev */
+
+	return 0;
+}
+
+static int qedn_create_queue(struct nvme_tcp_ofld_queue *queue, int qid,
+			     size_t q_size)
+{
+	/* Placeholder - qedn_create_queue */
+
+	return 0;
+}
+
+static void qedn_drain_queue(struct nvme_tcp_ofld_queue *queue)
+{
+	/* Placeholder - qedn_drain_queue */
+}
+
+static void qedn_destroy_queue(struct nvme_tcp_ofld_queue *queue)
+{
+	/* Placeholder - qedn_destroy_queue */
+}
+
+static int qedn_poll_queue(struct nvme_tcp_ofld_queue *queue)
+{
+	/* Placeholder - qedn_poll_queue */
+
+	return 0;
+}
+
+static int qedn_init_req(struct nvme_tcp_ofld_req *req)
+{
+	/* Placeholder - qedn_init_req */
+
+	return 0;
+}
+
+static int qedn_map_sg(struct nvme_tcp_ofld_dev *dev,
+		       struct nvme_tcp_ofld_req *req)
+{
+	/* Placeholder - qedn_map_sg */
+
+	return 0;
+}
+
+static int qedn_send_req(struct nvme_tcp_ofld_req *req)
+{
+	/* Placeholder - qedn_map_sg */
+
+	return 0;
+}
+
+static struct nvme_tcp_ofld_ops qedn_ofld_ops = {
+	.name = "qedn",
+	.module = THIS_MODULE,
+	.required_opts = NVMF_OPT_TRADDR,
+	.allowed_opts = NVMF_OPT_TRSVCID | NVMF_OPT_DISABLE_SQFLOW |
+			NVMF_OPT_NR_WRITE_QUEUES | NVMF_OPT_HOST_TRADDR |
+			NVMF_OPT_CTRL_LOSS_TMO | NVMF_OPT_RECONNECT_DELAY |
+			NVMF_OPT_HDR_DIGEST | NVMF_OPT_DATA_DIGEST |
+			NVMF_OPT_NR_POLL_QUEUES | NVMF_OPT_TOS,
+	.claim_dev = qedn_claim_dev,
+	.create_queue = qedn_create_queue,
+	.drain_queue = qedn_drain_queue,
+	.destroy_queue = qedn_destroy_queue,
+	.poll_queue = qedn_poll_queue,
+	.init_req = qedn_init_req,
+	.map_sg = qedn_map_sg,
+	.send_req = qedn_send_req,
+};
+
+static void __qedn_remove(struct pci_dev *pdev)
+{
+	struct qedn_ctx *qedn = pci_get_drvdata(pdev);
+
+	pr_notice("Starting qedn_remove\n");
+	nvme_tcp_ofld_unregister_dev(&qedn->qedn_ofld_dev);
+	kfree(qedn);
+	pr_notice("Ending qedn_remove successfully\n");
+}
+
+static void qedn_remove(struct pci_dev *pdev)
+{
+	__qedn_remove(pdev);
+}
+
+static void qedn_shutdown(struct pci_dev *pdev)
+{
+	__qedn_remove(pdev);
+}
+
+static struct qedn_ctx *qedn_alloc_ctx(struct pci_dev *pdev)
+{
+	struct qedn_ctx *qedn = NULL;
+
+	qedn = kzalloc(sizeof(*qedn), GFP_KERNEL);
+	if (!qedn)
+		return NULL;
+
+	qedn->pdev = pdev;
+	pci_set_drvdata(pdev, qedn);
+
+	return qedn;
+}
+
+static int __qedn_probe(struct pci_dev *pdev)
+{
+	struct qedn_ctx *qedn;
+	int rc;
+
+	pr_notice("Starting qedn probe\n");
+
+	qedn = qedn_alloc_ctx(pdev);
+	if (!qedn)
+		return -ENODEV;
+
+	qedn->qedn_ofld_dev.ops = &qedn_ofld_ops;
+	INIT_LIST_HEAD(&qedn->qedn_ofld_dev.entry);
+	rc = nvme_tcp_ofld_register_dev(&qedn->qedn_ofld_dev);
+	if (rc)
+		goto release_qedn;
+
+	return 0;
+release_qedn:
+	kfree(qedn);
+
+	return rc;
+}
+
+static int qedn_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+	return __qedn_probe(pdev);
+}
+
+static struct pci_driver qedn_pci_driver = {
+	.name     = QEDN_MODULE_NAME,
+	.id_table = qedn_pci_tbl,
+	.probe    = qedn_probe,
+	.remove   = qedn_remove,
+	.shutdown = qedn_shutdown,
+};
+
+static int __init qedn_init(void)
+{
+	int rc;
+
+	rc = pci_register_driver(&qedn_pci_driver);
+	if (rc) {
+		pr_err("Failed to register pci driver\n");
+		return -EINVAL;
+	}
+
+	pr_notice("driver loaded successfully\n");
+
+	return 0;
+}
+
+static void __exit qedn_cleanup(void)
+{
+	pci_unregister_driver(&qedn_pci_driver);
+	pr_notice("Unloading qedn ended\n");
+}
+
+module_init(qedn_init);
+module_exit(qedn_cleanup);
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/nvme/hw/qedn/qedn.h b/drivers/nvme/hw/qedn/qedn.h
new file mode 100644
index 000000000000..401fde08000e
--- /dev/null
+++ b/drivers/nvme/hw/qedn/qedn.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+
+/* Driver includes */
+#include "../../host/tcp-offload.h"
+
+#define QEDN_MODULE_NAME "qedn"
+
+struct qedn_ctx {
+	struct pci_dev *pdev;
+	struct nvme_tcp_ofld_dev qedn_ofld_dev;
+};
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:22 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:22 +0200
Subject: [RFC PATCH v3 09/11] net-qed: Add NVMeTCP Offload PF Level FW and HW
 HSI
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-10-smalin@marvell.com>

This patch introduces the NVMeTCP device and PF level HSI and HSI
functionality in order to initialize and interact with the HW device.

This patch is based on the qede, qedr, qedi, qedf drivers HSI.

Signed-off-by: Dean Balandin <dbalandin at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/net/ethernet/qlogic/Kconfig           |   3 +
 drivers/net/ethernet/qlogic/qed/Makefile      |   1 +
 drivers/net/ethernet/qlogic/qed/qed.h         |   3 +
 drivers/net/ethernet/qlogic/qed/qed_hsi.h     |   1 +
 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.c | 269 ++++++++++++++++++
 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.h |  48 ++++
 drivers/net/ethernet/qlogic/qed/qed_sp.h      |   2 +
 include/linux/qed/common_hsi.h                |   1 +
 include/linux/qed/nvmetcp_common.h            |  47 +++
 include/linux/qed/qed_if.h                    |  22 ++
 include/linux/qed/qed_nvmetcp_if.h            |  71 +++++
 11 files changed, 468 insertions(+)
 create mode 100644 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.c
 create mode 100644 drivers/net/ethernet/qlogic/qed/qed_nvmetcp.h
 create mode 100644 include/linux/qed/nvmetcp_common.h
 create mode 100644 include/linux/qed/qed_nvmetcp_if.h

diff --git a/drivers/net/ethernet/qlogic/Kconfig b/drivers/net/ethernet/qlogic/Kconfig
index 4366c7a8de95..1ddd82d83e3f 100644
--- a/drivers/net/ethernet/qlogic/Kconfig
+++ b/drivers/net/ethernet/qlogic/Kconfig
@@ -109,6 +109,9 @@ config QED_RDMA
 config QED_ISCSI
 	bool
 
+config QED_NVMETCP
+	bool
+
 config QED_FCOE
 	bool
 
diff --git a/drivers/net/ethernet/qlogic/qed/Makefile b/drivers/net/ethernet/qlogic/qed/Makefile
index 8251755ec18c..c8f4d40a6285 100644
--- a/drivers/net/ethernet/qlogic/qed/Makefile
+++ b/drivers/net/ethernet/qlogic/qed/Makefile
@@ -23,6 +23,7 @@ qed-y :=			\
 	qed_sp_commands.o	\
 	qed_spq.o
 
+qed-$(CONFIG_QED_NVMETCP) += qed_nvmetcp.o
 qed-$(CONFIG_QED_FCOE) += qed_fcoe.o
 qed-$(CONFIG_QED_ISCSI) += qed_iscsi.o
 qed-$(CONFIG_QED_LL2) += qed_ll2.o
diff --git a/drivers/net/ethernet/qlogic/qed/qed.h b/drivers/net/ethernet/qlogic/qed/qed.h
index a20cb8a0c377..91d4635009ab 100644
--- a/drivers/net/ethernet/qlogic/qed/qed.h
+++ b/drivers/net/ethernet/qlogic/qed/qed.h
@@ -240,6 +240,7 @@ enum QED_FEATURE {
 	QED_VF,
 	QED_RDMA_CNQ,
 	QED_ISCSI_CQ,
+	QED_NVMETCP_CQ = QED_ISCSI_CQ,
 	QED_FCOE_CQ,
 	QED_VF_L2_QUE,
 	QED_MAX_FEATURES,
@@ -592,6 +593,7 @@ struct qed_hwfn {
 	struct qed_ooo_info		*p_ooo_info;
 	struct qed_rdma_info		*p_rdma_info;
 	struct qed_iscsi_info		*p_iscsi_info;
+	struct qed_nvmetcp_info		*p_nvmetcp_info;
 	struct qed_fcoe_info		*p_fcoe_info;
 	struct qed_pf_params		pf_params;
 
@@ -828,6 +830,7 @@ struct qed_dev {
 		struct qed_eth_cb_ops		*eth;
 		struct qed_fcoe_cb_ops		*fcoe;
 		struct qed_iscsi_cb_ops		*iscsi;
+		struct qed_nvmetcp_cb_ops	*nvmetcp;
 	} protocol_ops;
 	void				*ops_cookie;
 
diff --git a/drivers/net/ethernet/qlogic/qed/qed_hsi.h b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
index 559df9f4d656..24472f6a83c2 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_hsi.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
@@ -20,6 +20,7 @@
 #include <linux/qed/fcoe_common.h>
 #include <linux/qed/eth_common.h>
 #include <linux/qed/iscsi_common.h>
+#include <linux/qed/nvmetcp_common.h>
 #include <linux/qed/iwarp_common.h>
 #include <linux/qed/rdma_common.h>
 #include <linux/qed/roce_common.h>
diff --git a/drivers/net/ethernet/qlogic/qed/qed_nvmetcp.c b/drivers/net/ethernet/qlogic/qed/qed_nvmetcp.c
new file mode 100644
index 000000000000..720b32a35820
--- /dev/null
+++ b/drivers/net/ethernet/qlogic/qed/qed_nvmetcp.c
@@ -0,0 +1,269 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+
+#include <linux/types.h>
+#include <asm/byteorder.h>
+#include <asm/param.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/etherdevice.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/log2.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/stddef.h>
+#include <linux/string.h>
+#include <linux/workqueue.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/qed/qed_nvmetcp_if.h>
+#include "qed.h"
+#include "qed_cxt.h"
+#include "qed_dev_api.h"
+#include "qed_hsi.h"
+#include "qed_hw.h"
+#include "qed_int.h"
+#include "qed_nvmetcp.h"
+#include "qed_ll2.h"
+#include "qed_mcp.h"
+#include "qed_sp.h"
+#include "qed_reg_addr.h"
+
+static int qed_nvmetcp_async_event(struct qed_hwfn *p_hwfn, u8 fw_event_code,
+				   u16 echo, union event_ring_data *data,
+				   u8 fw_return_code)
+{
+	if (p_hwfn->p_nvmetcp_info->event_cb) {
+		struct qed_nvmetcp_info *p_nvmetcp = p_hwfn->p_nvmetcp_info;
+
+		return p_nvmetcp->event_cb(p_nvmetcp->event_context,
+					 fw_event_code, data);
+	} else {
+		DP_NOTICE(p_hwfn, "nvmetcp async completion is not set\n");
+		return -EINVAL;
+	}
+}
+
+static int qed_sp_nvmetcp_func_start(struct qed_hwfn *p_hwfn,
+				     enum spq_mode comp_mode,
+				     struct qed_spq_comp_cb *p_comp_addr,
+				     void *event_context,
+				     nvmetcp_event_cb_t async_event_cb)
+{
+	struct nvmetcp_init_ramrod_params *p_ramrod = NULL;
+	struct scsi_init_func_queues *p_queue = NULL;
+	struct qed_nvmetcp_pf_params *p_params = NULL;
+	struct nvmetcp_spe_func_init *p_init = NULL;
+	struct qed_sp_init_data init_data = {};
+	struct qed_spq_entry *p_ent = NULL;
+	int rc = 0;
+	u16 val;
+	u8 i;
+
+	/* Get SPQ entry */
+	init_data.cid = qed_spq_get_cid(p_hwfn);
+	init_data.opaque_fid = p_hwfn->hw_info.opaque_fid;
+	init_data.comp_mode = comp_mode;
+	init_data.p_comp_data = p_comp_addr;
+
+	rc = qed_sp_init_request(p_hwfn, &p_ent,
+				 NVMETCP_RAMROD_CMD_ID_INIT_FUNC,
+				 PROTOCOLID_NVMETCP, &init_data);
+	if (rc)
+		return rc;
+
+	p_ramrod = &p_ent->ramrod.nvmetcp_init;
+	p_init = &p_ramrod->nvmetcp_init_spe;
+	p_params = &p_hwfn->pf_params.nvmetcp_pf_params;
+	p_queue = &p_init->q_params;
+
+	p_init->num_sq_pages_in_ring = p_params->num_sq_pages_in_ring;
+	p_init->num_r2tq_pages_in_ring = p_params->num_r2tq_pages_in_ring;
+	p_init->num_uhq_pages_in_ring = p_params->num_uhq_pages_in_ring;
+	p_init->ll2_rx_queue_id = RESC_START(p_hwfn, QED_LL2_RAM_QUEUE) +
+					p_params->ll2_ooo_queue_id;
+
+	p_init->func_params.log_page_size = ilog2(PAGE_SIZE);
+	p_init->func_params.num_tasks = cpu_to_le16(p_params->num_tasks);
+
+	DMA_REGPAIR_LE(p_queue->glbl_q_params_addr,
+		       p_params->glbl_q_params_addr);
+
+	p_queue->cq_num_entries = cpu_to_le16(QED_NVMETCP_FW_CQ_SIZE);
+	p_queue->num_queues = p_params->num_queues;
+	p_queue->queue_relative_offset = (u8)RESC_START(p_hwfn, QED_CMDQS_CQS);
+	p_queue->cq_sb_pi = p_params->gl_rq_pi;
+
+	for (i = 0; i < p_params->num_queues; i++) {
+		val = qed_get_igu_sb_id(p_hwfn, i);
+		p_queue->cq_cmdq_sb_num_arr[i] = cpu_to_le16(val);
+	}
+
+	/* p_ramrod->tcp_init.min_rto = cpu_to_le16(p_params->min_rto); */
+	p_ramrod->tcp_init.two_msl_timer = cpu_to_le32(QED_TCP_TWO_MSL_TIMER);
+	p_ramrod->tcp_init.tx_sws_timer = cpu_to_le16(QED_TCP_SWS_TIMER);
+	p_init->half_way_close_timeout = cpu_to_le16(QED_TCP_HALF_WAY_CLOSE_TIMEOUT);
+	p_ramrod->tcp_init.max_fin_rt = QED_TCP_MAX_FIN_RT;
+
+	p_hwfn->p_nvmetcp_info->event_context = event_context;
+	p_hwfn->p_nvmetcp_info->event_cb = async_event_cb;
+
+	qed_spq_register_async_cb(p_hwfn, PROTOCOLID_NVMETCP,
+				  qed_nvmetcp_async_event);
+
+	return qed_spq_post(p_hwfn, p_ent, NULL);
+}
+
+static int qed_sp_nvmetcp_func_stop(struct qed_hwfn *p_hwfn,
+				    enum spq_mode comp_mode,
+				    struct qed_spq_comp_cb *p_comp_addr)
+{
+	struct qed_spq_entry *p_ent = NULL;
+	struct qed_sp_init_data init_data;
+	int rc = 0;
+
+	/* Get SPQ entry */
+	memset(&init_data, 0, sizeof(init_data));
+	init_data.cid = qed_spq_get_cid(p_hwfn);
+	init_data.opaque_fid = p_hwfn->hw_info.opaque_fid;
+	init_data.comp_mode = comp_mode;
+	init_data.p_comp_data = p_comp_addr;
+
+	rc = qed_sp_init_request(p_hwfn, &p_ent,
+				 NVMETCP_RAMROD_CMD_ID_DESTROY_FUNC,
+				 PROTOCOLID_NVMETCP, &init_data);
+	if (rc)
+		return rc;
+
+	rc = qed_spq_post(p_hwfn, p_ent, NULL);
+
+	qed_spq_unregister_async_cb(p_hwfn, PROTOCOLID_NVMETCP);
+	return rc;
+}
+
+static int qed_fill_nvmetcp_dev_info(struct qed_dev *cdev,
+				     struct qed_dev_nvmetcp_info *info)
+{
+	struct qed_hwfn *hwfn = QED_AFFIN_HWFN(cdev);
+
+	int rc;
+
+	memset(info, 0, sizeof(*info));
+	rc = qed_fill_dev_info(cdev, &info->common);
+
+	info->num_cqs = FEAT_NUM(hwfn, QED_NVMETCP_CQ);
+
+	return rc;
+}
+
+static void qed_register_nvmetcp_ops(struct qed_dev *cdev,
+				     struct qed_nvmetcp_cb_ops *ops,
+				     void *cookie)
+{
+	cdev->protocol_ops.nvmetcp = ops;
+	cdev->ops_cookie = cookie;
+}
+
+static int qed_nvmetcp_stop(struct qed_dev *cdev)
+{
+	int rc;
+
+	if (!(cdev->flags & QED_FLAG_STORAGE_STARTED)) {
+		DP_NOTICE(cdev, "nvmetcp already stopped\n");
+		return 0;
+	}
+
+	if (!hash_empty(cdev->connections)) {
+		DP_NOTICE(cdev,
+			  "Can't stop nvmetcp - not all connections were returned\n");
+		return -EINVAL;
+	}
+
+	/* Stop the nvmetcp */
+	rc = qed_sp_nvmetcp_func_stop(QED_AFFIN_HWFN(cdev), QED_SPQ_MODE_EBLOCK,
+				      NULL);
+	cdev->flags &= ~QED_FLAG_STORAGE_STARTED;
+
+	return rc;
+}
+
+static int qed_nvmetcp_start(struct qed_dev *cdev,
+			     struct qed_nvmetcp_tid *tasks,
+			     void *event_context,
+			     nvmetcp_event_cb_t async_event_cb)
+{
+	struct qed_tid_mem *tid_info;
+	int rc;
+
+	if (cdev->flags & QED_FLAG_STORAGE_STARTED) {
+		DP_NOTICE(cdev, "nvmetcp already started;\n");
+		return 0;
+	}
+
+	rc = qed_sp_nvmetcp_func_start(QED_AFFIN_HWFN(cdev),
+				       QED_SPQ_MODE_EBLOCK,
+				       NULL,
+				       event_context,
+				       async_event_cb);
+	if (rc) {
+		DP_NOTICE(cdev, "Failed to start nvmetcp\n");
+		return rc;
+	}
+
+	cdev->flags |= QED_FLAG_STORAGE_STARTED;
+	hash_init(cdev->connections);
+
+	if (!tasks)
+		return 0;
+
+	tid_info = kzalloc(sizeof(*tid_info), GFP_KERNEL);
+
+	if (!tid_info) {
+		qed_nvmetcp_stop(cdev);
+		return -ENOMEM;
+	}
+
+	rc = qed_cxt_get_tid_mem_info(QED_AFFIN_HWFN(cdev), tid_info);
+	if (rc) {
+		DP_NOTICE(cdev, "Failed to gather task information\n");
+		qed_nvmetcp_stop(cdev);
+		kfree(tid_info);
+		return rc;
+	}
+
+	/* Fill task information */
+	tasks->size = tid_info->tid_size;
+	tasks->num_tids_per_block = tid_info->num_tids_per_block;
+	memcpy(tasks->blocks, tid_info->blocks,
+	       MAX_TID_BLOCKS_NVMETCP * sizeof(u8 *));
+
+	kfree(tid_info);
+
+	return 0;
+}
+
+static const struct qed_nvmetcp_ops qed_nvmetcp_ops_pass = {
+	.common = &qed_common_ops_pass,
+	.ll2 = &qed_ll2_ops_pass,
+	.fill_dev_info = &qed_fill_nvmetcp_dev_info,
+	.register_ops = &qed_register_nvmetcp_ops,
+	.start = &qed_nvmetcp_start,
+	.stop = &qed_nvmetcp_stop,
+
+	/* Placeholder - Connection level ops */
+};
+
+const struct qed_nvmetcp_ops *qed_get_nvmetcp_ops(void)
+{
+	return &qed_nvmetcp_ops_pass;
+}
+EXPORT_SYMBOL(qed_get_nvmetcp_ops);
+
+void qed_put_nvmetcp_ops(void)
+{
+}
+EXPORT_SYMBOL(qed_put_nvmetcp_ops);
diff --git a/drivers/net/ethernet/qlogic/qed/qed_nvmetcp.h b/drivers/net/ethernet/qlogic/qed/qed_nvmetcp.h
new file mode 100644
index 000000000000..cb10d526a241
--- /dev/null
+++ b/drivers/net/ethernet/qlogic/qed/qed_nvmetcp.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+
+#ifndef _QED_NVMETCP_H
+#define _QED_NVMETCP_H
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/qed/tcp_common.h>
+#include <linux/qed/qed_nvmetcp_if.h>
+#include <linux/qed/qed_chain.h>
+#include "qed.h"
+#include "qed_hsi.h"
+#include "qed_mcp.h"
+#include "qed_sp.h"
+
+#define QED_NVMETCP_FW_CQ_SIZE	(4 * 1024)
+
+/* tcp parameters */
+#define QED_TCP_TWO_MSL_TIMER 4000
+#define QED_TCP_HALF_WAY_CLOSE_TIMEOUT 10
+#define QED_TCP_MAX_FIN_RT 2
+#define QED_TCP_SWS_TIMER 5000
+
+struct qed_nvmetcp_info {
+	spinlock_t lock; /* Connection resources. */
+	struct list_head free_list;
+	u16 max_num_outstanding_tasks;
+	void *event_context;
+	nvmetcp_event_cb_t event_cb;
+};
+
+#if IS_ENABLED(CONFIG_QED_NVMETCP)
+void qed_nvmetcp_setup(struct qed_hwfn *p_hwfn);
+
+void qed_nvmetcp_free(struct qed_hwfn *p_hwfn);
+
+#else /* IS_ENABLED(CONFIG_QED_NVMETCP) */
+static inline void qed_nvmetcp_setup(struct qed_hwfn *p_hwfn) {}
+
+static inline void qed_nvmetcp_free(struct qed_hwfn *p_hwfn) {}
+
+#endif /* IS_ENABLED(CONFIG_QED_NVMETCP) */
+
+#endif
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp.h b/drivers/net/ethernet/qlogic/qed/qed_sp.h
index 993f1357b6fc..525159e747a5 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp.h
@@ -100,6 +100,8 @@ union ramrod_data {
 	struct iscsi_spe_conn_mac_update iscsi_conn_mac_update;
 	struct iscsi_spe_conn_termination iscsi_conn_terminate;
 
+	struct nvmetcp_init_ramrod_params nvmetcp_init;
+
 	struct vf_start_ramrod_data vf_start;
 	struct vf_stop_ramrod_data vf_stop;
 };
diff --git a/include/linux/qed/common_hsi.h b/include/linux/qed/common_hsi.h
index 977807e1be53..59c5e5866607 100644
--- a/include/linux/qed/common_hsi.h
+++ b/include/linux/qed/common_hsi.h
@@ -703,6 +703,7 @@ enum mf_mode {
 /* Per-protocol connection types */
 enum protocol_type {
 	PROTOCOLID_ISCSI,
+	PROTOCOLID_NVMETCP = PROTOCOLID_ISCSI,
 	PROTOCOLID_FCOE,
 	PROTOCOLID_ROCE,
 	PROTOCOLID_CORE,
diff --git a/include/linux/qed/nvmetcp_common.h b/include/linux/qed/nvmetcp_common.h
new file mode 100644
index 000000000000..0b097125efce
--- /dev/null
+++ b/include/linux/qed/nvmetcp_common.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+
+#ifndef __NVMETCP_COMMON__
+#define __NVMETCP_COMMON__
+
+#include "tcp_common.h"
+
+/*  * NVMeTCP firmware function init parameters */
+struct nvmetcp_spe_func_init {
+	__le16 reserved;
+	__le16 half_way_close_timeout;
+	u8 num_sq_pages_in_ring;
+	u8 num_r2tq_pages_in_ring;
+	u8 num_uhq_pages_in_ring;
+	u8 ll2_rx_queue_id;
+	u8 flags;
+#define NVMETCP_SPE_FUNC_INIT_COUNTERS_EN_MASK   0x1
+#define NVMETCP_SPE_FUNC_INIT_COUNTERS_EN_SHIFT  0
+#define NVMETCP_SPE_FUNC_INIT_NVMETCP_MODE_MASK  0x1
+#define NVMETCP_SPE_FUNC_INIT_NVMETCP_MODE_SHIFT 1
+#define NVMETCP_SPE_FUNC_INIT_RESERVED0_MASK     0x3F
+#define NVMETCP_SPE_FUNC_INIT_RESERVED0_SHIFT    2
+	u8 debug_flags;
+	__le16 reserved1;
+	__le32 reserved2;
+	struct scsi_init_func_params func_params;
+	struct scsi_init_func_queues q_params;
+};
+
+/* NVMeTCP init params passed by driver to FW in NVMeTCP init ramrod. */
+struct nvmetcp_init_ramrod_params {
+	struct nvmetcp_spe_func_init nvmetcp_init_spe;
+	struct tcp_init_params tcp_init;
+};
+
+/* NVMeTCP Ramrod Command IDs */
+enum nvmetcp_ramrod_cmd_id {
+	NVMETCP_RAMROD_CMD_ID_UNUSED = 0,
+	NVMETCP_RAMROD_CMD_ID_INIT_FUNC = 1,
+	NVMETCP_RAMROD_CMD_ID_DESTROY_FUNC = 2,
+	MAX_NVMETCP_RAMROD_CMD_ID
+};
+
+#endif /* __NVMETCP_COMMON__ */
diff --git a/include/linux/qed/qed_if.h b/include/linux/qed/qed_if.h
index 68d17a4fbf20..524f57821ba2 100644
--- a/include/linux/qed/qed_if.h
+++ b/include/linux/qed/qed_if.h
@@ -542,6 +542,26 @@ struct qed_iscsi_pf_params {
 	u8 bdq_pbl_num_entries[3];
 };
 
+struct qed_nvmetcp_pf_params {
+	u64 glbl_q_params_addr;
+	u16 cq_num_entries;
+
+	u16 num_cons;
+	u16 num_tasks;
+
+	u8 num_sq_pages_in_ring;
+	u8 num_r2tq_pages_in_ring;
+	u8 num_uhq_pages_in_ring;
+
+	u8 num_queues;
+	u8 gl_rq_pi;
+	u8 gl_cmd_pi;
+	u8 debug_mode;
+	u8 ll2_ooo_queue_id;
+
+	u16 min_rto;
+};
+
 struct qed_rdma_pf_params {
 	/* Supplied to QED during resource allocation (may affect the ILT and
 	 * the doorbell BAR).
@@ -560,6 +580,7 @@ struct qed_pf_params {
 	struct qed_eth_pf_params eth_pf_params;
 	struct qed_fcoe_pf_params fcoe_pf_params;
 	struct qed_iscsi_pf_params iscsi_pf_params;
+	struct qed_nvmetcp_pf_params nvmetcp_pf_params;
 	struct qed_rdma_pf_params rdma_pf_params;
 };
 
@@ -662,6 +683,7 @@ enum qed_sb_type {
 enum qed_protocol {
 	QED_PROTOCOL_ETH,
 	QED_PROTOCOL_ISCSI,
+	QED_PROTOCOL_NVMETCP = QED_PROTOCOL_ISCSI,
 	QED_PROTOCOL_FCOE,
 };
 
diff --git a/include/linux/qed/qed_nvmetcp_if.h b/include/linux/qed/qed_nvmetcp_if.h
new file mode 100644
index 000000000000..4e0f4c02d347
--- /dev/null
+++ b/include/linux/qed/qed_nvmetcp_if.h
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/*
+ * Copyright 2021 Marvell. All rights reserved.
+ */
+
+#ifndef _QED_NVMETCP_IF_H
+#define _QED_NVMETCP_IF_H
+#include <linux/types.h>
+#include <linux/qed/qed_if.h>
+
+typedef int (*nvmetcp_event_cb_t) (void *context,
+				   u8 fw_event_code, void *fw_handle);
+
+struct qed_dev_nvmetcp_info {
+	struct qed_dev_info common;
+
+	u8 num_cqs;
+};
+
+#define MAX_TID_BLOCKS_NVMETCP (512)
+struct qed_nvmetcp_tid {
+	u32 size;		/* In bytes per task */
+	u32 num_tids_per_block;
+	u8 *blocks[MAX_TID_BLOCKS_NVMETCP];
+};
+
+struct qed_nvmetcp_cb_ops {
+	struct qed_common_cb_ops common;
+};
+
+/**
+ * struct qed_nvmetcp_ops - qed NVMeTCP operations.
+ * @common:		common operations pointer
+ * @ll2:		light L2 operations pointer
+ * @fill_dev_info:	fills NVMeTCP specific information
+ *			@param cdev
+ *			@param info
+ *			@return 0 on success, otherwise error value.
+ * @register_ops:	register nvmetcp operations
+ *			@param cdev
+ *			@param ops - specified using qed_nvmetcp_cb_ops
+ *			@param cookie - driver private
+ * @start:		nvmetcp in FW
+ *			@param cdev
+ *			@param tasks - qed will fill information about tasks
+ *			return 0 on success, otherwise error value.
+ * @stop:		nvmetcp in FW
+ *			@param cdev
+ *			return 0 on success, otherwise error value.
+ */
+struct qed_nvmetcp_ops {
+	const struct qed_common_ops *common;
+
+	const struct qed_ll2_ops *ll2;
+
+	int (*fill_dev_info)(struct qed_dev *cdev,
+			     struct qed_dev_nvmetcp_info *info);
+
+	void (*register_ops)(struct qed_dev *cdev,
+			     struct qed_nvmetcp_cb_ops *ops, void *cookie);
+
+	int (*start)(struct qed_dev *cdev,
+		     struct qed_nvmetcp_tid *tasks,
+		     void *event_context, nvmetcp_event_cb_t async_event_cb);
+
+	int (*stop)(struct qed_dev *cdev);
+};
+
+const struct qed_nvmetcp_ops *qed_get_nvmetcp_ops(void);
+void qed_put_nvmetcp_ops(void);
+#endif
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:23 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:23 +0200
Subject: [RFC PATCH v3 10/11] nvme-qedn: Add qedn probe
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-11-smalin@marvell.com>

As part of the qedn init, the driver will register as a pci device and
will work with the Marvell fastlinQ NICs.
The HW ops- qed_nvmetcp_ops are similar to other "qed_*_ops" which are
used by the qede, qedr, qedf and qedi device drivers.

Struct qedn_ctx is per pci physical function (PF) container for
PF-specific attributes and resources.

Signed-off-by: Dean Balandin <dbalandin at marvell.com>
Signed-off-by: Prabhakar Kushwaha <pkushwaha at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/hw/Kconfig     |   1 +
 drivers/nvme/hw/qedn/qedn.c | 193 +++++++++++++++++++++++++++++++++++-
 drivers/nvme/hw/qedn/qedn.h |  49 +++++++++
 3 files changed, 238 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/hw/Kconfig b/drivers/nvme/hw/Kconfig
index 374f1f9dbd3d..91b1bd6f07d8 100644
--- a/drivers/nvme/hw/Kconfig
+++ b/drivers/nvme/hw/Kconfig
@@ -2,6 +2,7 @@
 config NVME_QEDN
 	tristate "Marvell NVM Express over Fabrics TCP offload"
 	depends on NVME_TCP_OFFLOAD
+	select QED_NVMETCP
 	help
 	  This enables the Marvell NVMe TCP offload support (qedn).
 
diff --git a/drivers/nvme/hw/qedn/qedn.c b/drivers/nvme/hw/qedn/qedn.c
index 1fe33335989c..11b31d086d27 100644
--- a/drivers/nvme/hw/qedn/qedn.c
+++ b/drivers/nvme/hw/qedn/qedn.c
@@ -14,6 +14,10 @@
 
 #define CHIP_NUM_AHP_NVMETCP 0x8194
 
+const struct qed_nvmetcp_ops *qed_ops;
+
+/* Global context instance */
+struct qedn_global qedn_glb;
 static struct pci_device_id qedn_pci_tbl[] = {
 	{ PCI_VDEVICE(QLOGIC, CHIP_NUM_AHP_NVMETCP), 0 },
 	{0, 0},
@@ -94,12 +98,132 @@ static struct nvme_tcp_ofld_ops qedn_ofld_ops = {
 	.send_req = qedn_send_req,
 };
 
+/* Initialize qedn fields, such as locks, lists, atomics, workqueues , hashes */
+static inline void qedn_init_pf_struct(struct qedn_ctx *qedn)
+{
+	/* Placeholder */
+}
+
+static inline void
+qedn_init_core_probe_params(struct qed_probe_params *probe_params)
+{
+	memset(probe_params, 0, sizeof(*probe_params));
+	probe_params->protocol = QED_PROTOCOL_NVMETCP;
+	probe_params->is_vf = false;
+	probe_params->recov_in_prog = 0;
+}
+
+static inline int qedn_core_probe(struct qedn_ctx *qedn)
+{
+	struct qed_probe_params probe_params;
+	int rc = 0;
+
+	qedn_init_core_probe_params(&probe_params);
+	pr_info("Starting QED probe\n");
+	qedn->cdev = qed_ops->common->probe(qedn->pdev, &probe_params);
+	if (!qedn->cdev) {
+		rc = -ENODEV;
+		pr_err("QED probe failed\n");
+	}
+
+	return rc;
+}
+
+static void qedn_add_pf_to_gl_list(struct qedn_ctx *qedn)
+{
+	mutex_lock(&qedn_glb.glb_mutex);
+	list_add_tail(&qedn->gl_pf_entry, &qedn_glb.qedn_pf_list);
+	mutex_unlock(&qedn_glb.glb_mutex);
+}
+
+static void qedn_remove_pf_from_gl_list(struct qedn_ctx *qedn)
+{
+	mutex_lock(&qedn_glb.glb_mutex);
+	list_del_init(&qedn->gl_pf_entry);
+	mutex_unlock(&qedn_glb.glb_mutex);
+}
+
+static int qedn_set_nvmetcp_pf_param(struct qedn_ctx *qedn)
+{
+	struct qed_nvmetcp_pf_params *pf_params;
+
+	pf_params = &qedn->pf_params.nvmetcp_pf_params;
+	memset(pf_params, 0, sizeof(*pf_params));
+	qedn->num_fw_cqs = min_t(u8, qedn->dev_info.num_cqs, num_online_cpus());
+
+	pf_params->num_cons = QEDN_MAX_CONNS_PER_PF;
+	pf_params->num_tasks = QEDN_MAX_TASKS_PER_PF;
+
+	/* Placeholder - Initialize function level queues */
+
+	/* Placeholder - Initialize TCP params */
+
+	/* Queues */
+	pf_params->num_sq_pages_in_ring = QEDN_NVMETCP_NUM_FW_SQ_PAGES * 2;
+	pf_params->num_r2tq_pages_in_ring = QEDN_NVMETCP_NUM_FW_SQ_PAGES;
+	pf_params->num_uhq_pages_in_ring = QEDN_NVMETCP_NUM_FW_SQ_PAGES;
+	pf_params->num_queues = qedn->num_fw_cqs;
+	pf_params->cq_num_entries = QEDN_FW_CQ_SIZE;
+
+	/* the CQ SB pi */
+	pf_params->gl_rq_pi = QEDN_PROTO_CQ_PROD_IDX;
+
+	return 0;
+}
+
+static inline int qedn_slowpath_start(struct qedn_ctx *qedn)
+{
+	struct qed_slowpath_params sp_params = {};
+	int rc = 0;
+
+	/* Start the Slowpath-process */
+	sp_params.int_mode = QED_INT_MODE_MSIX;
+	sp_params.drv_major = QEDN_MAJOR_VERSION;
+	sp_params.drv_minor = QEDN_MINOR_VERSION;
+	sp_params.drv_rev = QEDN_REVISION_VERSION;
+	sp_params.drv_eng = QEDN_ENGINEERING_VERSION;
+	strlcpy(sp_params.name, "qedn NVMeTCP", QED_DRV_VER_STR_SIZE);
+	rc = qed_ops->common->slowpath_start(qedn->cdev, &sp_params);
+	if (rc)
+		pr_err("Cannot start slowpath\n");
+
+	return rc;
+}
+
 static void __qedn_remove(struct pci_dev *pdev)
 {
 	struct qedn_ctx *qedn = pci_get_drvdata(pdev);
+	int rc;
+
+	pr_notice("qedn remove started: abs PF id=%u\n",
+		  qedn->dev_info.common.abs_pf_id);
+
+	if (test_and_set_bit(QEDN_STATE_MODULE_REMOVE_ONGOING, &qedn->state)) {
+		pr_err("Remove already ongoing\n");
+
+		return;
+	}
+
+	if (test_and_clear_bit(QEDN_STATE_REGISTERED_OFFLOAD_DEV, &qedn->state))
+		nvme_tcp_ofld_unregister_dev(&qedn->qedn_ofld_dev);
+
+	if (test_and_set_bit(QEDN_STATE_MFW_STATE, &qedn->state)) {
+		rc = qed_ops->common->update_drv_state(qedn->cdev, false);
+		if (rc)
+			pr_err("Failed to send drv state to MFW\n");
+	}
+
+	if (test_and_clear_bit(QEDN_STATE_GL_PF_LIST_ADDED, &qedn->state))
+		qedn_remove_pf_from_gl_list(qedn);
+	else
+		pr_err("Failed to remove from global PF list\n");
+
+	if (test_and_clear_bit(QEDN_STATE_CORE_OPEN, &qedn->state))
+		qed_ops->common->slowpath_stop(qedn->cdev);
+
+	if (test_and_clear_bit(QEDN_STATE_CORE_PROBED, &qedn->state))
+		qed_ops->common->remove(qedn->cdev);
 
-	pr_notice("Starting qedn_remove\n");
-	nvme_tcp_ofld_unregister_dev(&qedn->qedn_ofld_dev);
 	kfree(qedn);
 	pr_notice("Ending qedn_remove successfully\n");
 }
@@ -139,15 +263,56 @@ static int __qedn_probe(struct pci_dev *pdev)
 	if (!qedn)
 		return -ENODEV;
 
+	qedn_init_pf_struct(qedn);
+
+	/* QED probe */
+	rc = qedn_core_probe(qedn);
+	if (rc)
+		goto exit_probe_and_release_mem;
+
+	set_bit(QEDN_STATE_CORE_PROBED, &qedn->state);
+
+	/* For global number of FW CQs */
+	rc = qed_ops->fill_dev_info(qedn->cdev, &qedn->dev_info);
+	if (rc) {
+		pr_err("fill_dev_info failed\n");
+		goto exit_probe_and_release_mem;
+	}
+
+	qedn_add_pf_to_gl_list(qedn);
+	set_bit(QEDN_STATE_GL_PF_LIST_ADDED, &qedn->state);
+
+	rc = qedn_set_nvmetcp_pf_param(qedn);
+	if (rc)
+		goto exit_probe_and_release_mem;
+
+	qed_ops->common->update_pf_params(qedn->cdev, &qedn->pf_params);
+	rc = qedn_slowpath_start(qedn);
+	if (rc)
+		goto exit_probe_and_release_mem;
+
+	set_bit(QEDN_STATE_CORE_OPEN, &qedn->state);
+
+	rc = qed_ops->common->update_drv_state(qedn->cdev, true);
+	if (rc) {
+		pr_err("Failed to send drv state to MFW\n");
+		goto exit_probe_and_release_mem;
+	}
+
+	set_bit(QEDN_STATE_MFW_STATE, &qedn->state);
+
 	qedn->qedn_ofld_dev.ops = &qedn_ofld_ops;
 	INIT_LIST_HEAD(&qedn->qedn_ofld_dev.entry);
 	rc = nvme_tcp_ofld_register_dev(&qedn->qedn_ofld_dev);
 	if (rc)
-		goto release_qedn;
+		goto exit_probe_and_release_mem;
+
+	set_bit(QEDN_STATE_REGISTERED_OFFLOAD_DEV, &qedn->state);
 
 	return 0;
-release_qedn:
-	kfree(qedn);
+exit_probe_and_release_mem:
+	__qedn_remove(pdev);
+	pr_err("probe ended with error\n");
 
 	return rc;
 }
@@ -165,13 +330,30 @@ static struct pci_driver qedn_pci_driver = {
 	.shutdown = qedn_shutdown,
 };
 
+static inline void qedn_init_global_contxt(void)
+{
+	INIT_LIST_HEAD(&qedn_glb.qedn_pf_list);
+	INIT_LIST_HEAD(&qedn_glb.ctrl_list);
+	mutex_init(&qedn_glb.glb_mutex);
+}
+
 static int __init qedn_init(void)
 {
 	int rc;
 
+	qedn_init_global_contxt();
+
+	qed_ops = qed_get_nvmetcp_ops();
+	if (!qed_ops) {
+		pr_err("Failed to get QED NVMeTCP ops\n");
+
+		return -EINVAL;
+	}
+
 	rc = pci_register_driver(&qedn_pci_driver);
 	if (rc) {
 		pr_err("Failed to register pci driver\n");
+
 		return -EINVAL;
 	}
 
@@ -183,6 +365,7 @@ static int __init qedn_init(void)
 static void __exit qedn_cleanup(void)
 {
 	pci_unregister_driver(&qedn_pci_driver);
+	qed_put_nvmetcp_ops();
 	pr_notice("Unloading qedn ended\n");
 }
 
diff --git a/drivers/nvme/hw/qedn/qedn.h b/drivers/nvme/hw/qedn/qedn.h
index 401fde08000e..634e1217639a 100644
--- a/drivers/nvme/hw/qedn/qedn.h
+++ b/drivers/nvme/hw/qedn/qedn.h
@@ -3,12 +3,61 @@
  * Copyright 2021 Marvell. All rights reserved.
  */
 
+#include <linux/qed/qed_if.h>
+#include <linux/qed/qed_nvmetcp_if.h>
+
 /* Driver includes */
 #include "../../host/tcp-offload.h"
 
+#define QEDN_MAJOR_VERSION		8
+#define QEDN_MINOR_VERSION		62
+#define QEDN_REVISION_VERSION		10
+#define QEDN_ENGINEERING_VERSION	0
+#define DRV_MODULE_VERSION __stringify(QEDE_MAJOR_VERSION) "."	\
+		__stringify(QEDE_MINOR_VERSION) "."		\
+		__stringify(QEDE_REVISION_VERSION) "."		\
+		__stringify(QEDE_ENGINEERING_VERSION)
+
 #define QEDN_MODULE_NAME "qedn"
 
+#define QEDN_MAX_TASKS_PER_PF (32 * 1024)
+#define QEDN_MAX_CONNS_PER_PF (8 * 1024)
+#define QEDN_FW_CQ_SIZE (4 * 1024)
+#define QEDN_PROTO_CQ_PROD_IDX	0
+#define QEDN_NVMETCP_NUM_FW_SQ_PAGES 4
+
+enum qedn_state {
+	QEDN_STATE_CORE_PROBED = 0,
+	QEDN_STATE_CORE_OPEN,
+	QEDN_STATE_GL_PF_LIST_ADDED,
+	QEDN_STATE_MFW_STATE,
+	QEDN_STATE_REGISTERED_OFFLOAD_DEV,
+	QEDN_STATE_MODULE_REMOVE_ONGOING,
+};
+
 struct qedn_ctx {
 	struct pci_dev *pdev;
+	struct qed_dev *cdev;
+	struct qed_dev_nvmetcp_info dev_info;
 	struct nvme_tcp_ofld_dev qedn_ofld_dev;
+	struct qed_pf_params pf_params;
+
+	/* Global PF list entry */
+	struct list_head gl_pf_entry;
+
+	/* Accessed with atomic bit ops , used with enum qedn_state */
+	unsigned long state;
+
+	/* Fast path queues */
+	u8 num_fw_cqs;
+};
+
+struct qedn_global {
+	struct list_head qedn_pf_list;
+
+	/* Host mode */
+	struct list_head ctrl_list;
+
+	/* Mutex for accessing the global struct */
+	struct mutex glb_mutex;
 };
-- 
2.22.0



From smalin at marvell.com  Sun Feb  7 13:13:24 2021
From: smalin at marvell.com (Shai Malin)
Date: Sun, 7 Feb 2021 20:13:24 +0200
Subject: [RFC PATCH v3 11/11] nvme-qedn: Add IRQ and fast-path resources
 initializations
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210207181324.11429-12-smalin@marvell.com>

This patch will present the adding of qedn_fp_queue - this is a per cpu
core element which handles all of the connections on that cpu core.
The qedn_fp_queue will handle a group of connections (NVMeoF QPs) which
are handled on the same cpu core, and will only use the same FW-driver
resources with no need to be related to the same NVMeoF controller.

The per qedn_fq_queue resources are the FW CQ and FW status block:
- The FW CQ will be used for the FW to notify the driver that the
  the exchange has ended and the FW will pass the incoming NVMeoF CQE
  (if exist) to the driver.
- FW status block - which is used for the FW to notify the driver with
  the producer update of the FW CQE chain.

The FW fast-path queues are based on qed_chain.h

Signed-off-by: Prabhakar Kushwaha <pkushwaha at marvell.com>
Signed-off-by: Michal Kalderon <mkalderon at marvell.com>
Signed-off-by: Shai Malin <smalin at marvell.com>
---
 drivers/nvme/hw/qedn/qedn.c | 284 +++++++++++++++++++++++++++++++++++-
 drivers/nvme/hw/qedn/qedn.h |  27 ++++
 2 files changed, 308 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/hw/qedn/qedn.c b/drivers/nvme/hw/qedn/qedn.c
index 11b31d086d27..64277dc28901 100644
--- a/drivers/nvme/hw/qedn/qedn.c
+++ b/drivers/nvme/hw/qedn/qedn.c
@@ -98,6 +98,103 @@ static struct nvme_tcp_ofld_ops qedn_ofld_ops = {
 	.send_req = qedn_send_req,
 };
 
+/* Fastpath IRQ handler */
+static irqreturn_t qedn_irq_handler(int irq, void *dev_id)
+{
+	/* Placeholder */
+
+	return IRQ_HANDLED;
+}
+
+static void qedn_sync_free_irqs(struct qedn_ctx *qedn)
+{
+	u16 vector_idx;
+	int i;
+
+	for (i = 0; i < qedn->num_fw_cqs; i++) {
+		vector_idx = i * qedn->dev_info.common.num_hwfns +
+			     qed_ops->common->get_affin_hwfn_idx(qedn->cdev);
+		synchronize_irq(qedn->int_info.msix[vector_idx].vector);
+		irq_set_affinity_hint(qedn->int_info.msix[vector_idx].vector,
+				      NULL);
+		free_irq(qedn->int_info.msix[vector_idx].vector,
+			 &qedn->fp_q_arr[i]);
+	}
+
+	qedn->int_info.used_cnt = 0;
+	qed_ops->common->set_fp_int(qedn->cdev, 0);
+}
+
+static int qedn_request_msix_irq(struct qedn_ctx *qedn)
+{
+	struct pci_dev *pdev = qedn->pdev;
+	struct qedn_fp_queue *fp_q = NULL;
+	int i, rc, cpu;
+	u16 vector_idx;
+	u32 vector;
+
+	cpu = cpumask_first(cpu_online_mask);
+	for (i = 0; i < qedn->num_fw_cqs; i++) {
+		fp_q = &qedn->fp_q_arr[i];
+		vector_idx = i * qedn->dev_info.common.num_hwfns +
+			     qed_ops->common->get_affin_hwfn_idx(qedn->cdev);
+		vector = qedn->int_info.msix[vector_idx].vector;
+		sprintf(fp_q->irqname, "qedn_queue_%x.%x.%x_%d",
+			pdev->bus->number, PCI_SLOT(pdev->devfn),
+			PCI_FUNC(pdev->devfn), i);
+		rc = request_irq(vector, qedn_irq_handler, QEDN_IRQ_NO_FLAGS,
+				 fp_q->irqname, fp_q);
+		if (rc) {
+			pr_err("request_irq failed.\n");
+			qedn_sync_free_irqs(qedn);
+
+			return rc;
+		}
+
+		fp_q->cpu = cpu;
+		qedn->int_info.used_cnt++;
+		rc = irq_set_affinity_hint(vector, get_cpu_mask(cpu));
+		cpu = cpumask_next_wrap(cpu, cpu_online_mask, -1, false);
+	}
+
+	return 0;
+}
+
+static int qedn_setup_irq(struct qedn_ctx *qedn)
+{
+	int rc = 0;
+	u8 rval;
+
+	rval = qed_ops->common->set_fp_int(qedn->cdev, qedn->num_fw_cqs);
+	if (rval < qedn->num_fw_cqs) {
+		qedn->num_fw_cqs = rval;
+		if (rval == 0) {
+			pr_err("set_fp_int return 0 IRQs\n");
+
+			return -ENODEV;
+		}
+	}
+
+	rc = qed_ops->common->get_fp_int(qedn->cdev, &qedn->int_info);
+	if (rc) {
+		pr_err("get_fp_int failed\n");
+		goto exit_setup_int;
+	}
+
+	if (qedn->int_info.msix_cnt) {
+		rc = qedn_request_msix_irq(qedn);
+		goto exit_setup_int;
+	} else {
+		pr_err("msix_cnt = 0\n");
+		rc = -EINVAL;
+		goto exit_setup_int;
+	}
+
+exit_setup_int:
+
+	return rc;
+}
+
 /* Initialize qedn fields, such as locks, lists, atomics, workqueues , hashes */
 static inline void qedn_init_pf_struct(struct qedn_ctx *qedn)
 {
@@ -143,9 +240,151 @@ static void qedn_remove_pf_from_gl_list(struct qedn_ctx *qedn)
 	mutex_unlock(&qedn_glb.glb_mutex);
 }
 
+static void qedn_free_function_queues(struct qedn_ctx *qedn)
+{
+	struct qed_sb_info *sb_info = NULL;
+	struct qedn_fp_queue *fp_q;
+	int i;
+
+	/* Free workqueues */
+
+	/* Free the fast path queues*/
+	for (i = 0; i < qedn->num_fw_cqs; i++) {
+		fp_q = &qedn->fp_q_arr[i];
+
+		/* Free SB */
+		sb_info = fp_q->sb_info;
+		if (sb_info->sb_virt) {
+			qed_ops->common->sb_release(qedn->cdev, sb_info,
+						    fp_q->sb_id,
+						    QED_SB_TYPE_STORAGE);
+			dma_free_coherent(&qedn->pdev->dev,
+					  sizeof(*sb_info->sb_virt),
+					  (void *)sb_info->sb_virt,
+					  sb_info->sb_phys);
+			kfree(sb_info);
+			fp_q->sb_info = NULL;
+		}
+
+		qed_ops->common->chain_free(qedn->cdev, &fp_q->cq_chain);
+	}
+
+	if (qedn->fw_cq_array_virt)
+		dma_free_coherent(&qedn->pdev->dev,
+				  qedn->num_fw_cqs * sizeof(u64),
+				  qedn->fw_cq_array_virt,
+				  qedn->fw_cq_array_phy);
+	kfree(qedn->fp_q_arr);
+	qedn->fp_q_arr = NULL;
+}
+
+static int qedn_alloc_and_init_sb(struct qedn_ctx *qedn,
+				  struct qed_sb_info *sb_info, u16 sb_id)
+{
+	int rc = 0;
+
+	sb_info->sb_virt = dma_alloc_coherent(&qedn->pdev->dev,
+					      sizeof(struct status_block_e4),
+					      &sb_info->sb_phys, GFP_KERNEL);
+	if (!sb_info->sb_virt) {
+		pr_err("Status block allocation failed\n");
+
+		return -ENOMEM;
+	}
+
+	rc = qed_ops->common->sb_init(qedn->cdev, sb_info, sb_info->sb_virt,
+				      sb_info->sb_phys, sb_id,
+				      QED_SB_TYPE_STORAGE);
+	if (rc) {
+		pr_err("Status block initialization failed\n");
+
+		return rc;
+	}
+
+	return 0;
+}
+
+static int qedn_alloc_function_queues(struct qedn_ctx *qedn)
+{
+	struct qed_chain_init_params chain_params = {};
+	struct status_block_e4 *sb = NULL;  /* To change to status_block_e4 */
+	struct qedn_fp_queue *fp_q = NULL;
+	int rc = 0;
+	int i;
+
+	/* Place holder - IO-path workqueues */
+
+	qedn->fp_q_arr = kcalloc(qedn->num_fw_cqs,
+				 sizeof(struct qedn_fp_queue), GFP_KERNEL);
+	if (!qedn->fp_q_arr)
+		return -ENOMEM;
+
+	qedn->fw_cq_array_virt = dma_alloc_coherent(&qedn->pdev->dev,
+						    qedn->num_fw_cqs * sizeof(u64),
+						    &qedn->fw_cq_array_phy,
+						    GFP_KERNEL);
+	if (!qedn->fw_cq_array_virt) {
+		rc = -ENOMEM;
+		goto mem_alloc_failure;
+	}
+
+	/* placeholder - create task pools */
+
+	for (i = 0; i < qedn->num_fw_cqs; i++) {
+		fp_q = &qedn->fp_q_arr[i];
+		mutex_init(&fp_q->cq_mutex);
+
+		/* FW CQ */
+		chain_params.intended_use = QED_CHAIN_USE_TO_CONSUME,
+		chain_params.mode = QED_CHAIN_MODE_PBL,
+		chain_params.cnt_type = QED_CHAIN_CNT_TYPE_U16,
+		chain_params.num_elems = QEDN_FW_CQ_SIZE;
+		chain_params.elem_size = 64; /*Placeholder - sizeof(struct nvmetcp_fw_cqe)*/
+
+		rc = qed_ops->common->chain_alloc(qedn->cdev,
+						  &fp_q->cq_chain,
+						  &chain_params);
+		if (rc) {
+			pr_err("CQ chain pci_alloc_consistent fail\n");
+			goto mem_alloc_failure;
+		}
+
+		qedn->fw_cq_array_virt[i] = qed_chain_get_pbl_phys(&fp_q->cq_chain);
+
+		/* SB */
+		fp_q->sb_info = kzalloc(sizeof(*fp_q->sb_info), GFP_KERNEL);
+		if (!fp_q->sb_info)
+			goto mem_alloc_failure;
+
+		fp_q->sb_id = i;
+		rc = qedn_alloc_and_init_sb(qedn, fp_q->sb_info, fp_q->sb_id);
+		if (rc) {
+			pr_err("SB allocation and initialization failed.\n");
+			goto mem_alloc_failure;
+		}
+
+		sb = fp_q->sb_info->sb_virt;
+		fp_q->cq_prod = (u16 *)&sb->pi_array[QEDN_PROTO_CQ_PROD_IDX];
+		fp_q->qedn = qedn;
+
+		/* Placeholder - Init IO-path workqueue */
+
+		/* Placeholder - Init IO-path resources */
+	}
+
+	return 0;
+
+mem_alloc_failure:
+	pr_err("Function allocation failed\n");
+	qedn_free_function_queues(qedn);
+
+	return rc;
+}
+
 static int qedn_set_nvmetcp_pf_param(struct qedn_ctx *qedn)
 {
 	struct qed_nvmetcp_pf_params *pf_params;
+	int rc;
 
 	pf_params = &qedn->pf_params.nvmetcp_pf_params;
 	memset(pf_params, 0, sizeof(*pf_params));
@@ -154,9 +393,13 @@ static int qedn_set_nvmetcp_pf_param(struct qedn_ctx *qedn)
 	pf_params->num_cons = QEDN_MAX_CONNS_PER_PF;
 	pf_params->num_tasks = QEDN_MAX_TASKS_PER_PF;
 
-	/* Placeholder - Initialize function level queues */
+	rc = qedn_alloc_function_queues(qedn);
+	if (rc) {
+		pr_err("Global queue allocation failed.\n");
+		goto err_alloc_mem;
+	}
 
-	/* Placeholder - Initialize TCP params */
+	set_bit(QEDN_STATE_FP_WORK_THREAD_SET, &qedn->state);
 
 	/* Queues */
 	pf_params->num_sq_pages_in_ring = QEDN_NVMETCP_NUM_FW_SQ_PAGES * 2;
@@ -164,11 +407,14 @@ static int qedn_set_nvmetcp_pf_param(struct qedn_ctx *qedn)
 	pf_params->num_uhq_pages_in_ring = QEDN_NVMETCP_NUM_FW_SQ_PAGES;
 	pf_params->num_queues = qedn->num_fw_cqs;
 	pf_params->cq_num_entries = QEDN_FW_CQ_SIZE;
+	pf_params->glbl_q_params_addr = qedn->fw_cq_array_phy;
 
 	/* the CQ SB pi */
 	pf_params->gl_rq_pi = QEDN_PROTO_CQ_PROD_IDX;
 
-	return 0;
+err_alloc_mem:
+
+	return rc;
 }
 
 static inline int qedn_slowpath_start(struct qedn_ctx *qedn)
@@ -218,9 +464,22 @@ static void __qedn_remove(struct pci_dev *pdev)
 	else
 		pr_err("Failed to remove from global PF list\n");
 
+	if (test_and_clear_bit(QEDN_STATE_IRQ_SET, &qedn->state))
+		qedn_sync_free_irqs(qedn);
+
+	if (test_and_clear_bit(QEDN_STATE_NVMETCP_OPEN, &qedn->state))
+		qed_ops->stop(qedn->cdev);
+
+	rc = qed_ops->common->update_drv_state(qedn->cdev, false);
+	if (rc)
+		pr_err("Failed to send drv state to MFW\n");
+
 	if (test_and_clear_bit(QEDN_STATE_CORE_OPEN, &qedn->state))
 		qed_ops->common->slowpath_stop(qedn->cdev);
 
+	if (test_and_clear_bit(QEDN_STATE_FP_WORK_THREAD_SET, &qedn->state))
+		qedn_free_function_queues(qedn);
+
 	if (test_and_clear_bit(QEDN_STATE_CORE_PROBED, &qedn->state))
 		qed_ops->common->remove(qedn->cdev);
 
@@ -301,6 +560,25 @@ static int __qedn_probe(struct pci_dev *pdev)
 
 	set_bit(QEDN_STATE_MFW_STATE, &qedn->state);
 
+	rc = qedn_setup_irq(qedn);
+	if (rc)
+		goto exit_probe_and_release_mem;
+
+	set_bit(QEDN_STATE_IRQ_SET, &qedn->state);
+
+	/* NVMeTCP start HW PF */
+	rc = qed_ops->start(qedn->cdev,
+			    NULL /* Placeholder for FW IO-path resources */,
+			    qedn,
+			    NULL /* Placeholder for FW Event callback */);
+	if (rc) {
+		rc = -ENODEV;
+		pr_err("Cannot start NVMeTCP Function\n");
+		goto exit_probe_and_release_mem;
+	}
+
+	set_bit(QEDN_STATE_NVMETCP_OPEN, &qedn->state);
+
 	qedn->qedn_ofld_dev.ops = &qedn_ofld_ops;
 	INIT_LIST_HEAD(&qedn->qedn_ofld_dev.entry);
 	rc = nvme_tcp_ofld_register_dev(&qedn->qedn_ofld_dev);
diff --git a/drivers/nvme/hw/qedn/qedn.h b/drivers/nvme/hw/qedn/qedn.h
index 634e1217639a..17cebf741314 100644
--- a/drivers/nvme/hw/qedn/qedn.h
+++ b/drivers/nvme/hw/qedn/qedn.h
@@ -5,6 +5,7 @@
 
 #include <linux/qed/qed_if.h>
 #include <linux/qed/qed_nvmetcp_if.h>
+#include <linux/qed/qed_chain.h>
 
 /* Driver includes */
 #include "../../host/tcp-offload.h"
@@ -26,18 +27,41 @@
 #define QEDN_PROTO_CQ_PROD_IDX	0
 #define QEDN_NVMETCP_NUM_FW_SQ_PAGES 4
 
+#define QEDN_PAGE_SIZE	4096 /* FW page size - Configurable */
+#define QEDN_MAX_GLOBAL_QUEUES_PAIRS 16
+#define QEDN_IRQ_NAME_LEN 24
+#define QEDN_IRQ_NO_FLAGS 0
+
+/* TCP defines */
+#define QEDN_TCP_RTO_DEFAULT 280
+
 enum qedn_state {
 	QEDN_STATE_CORE_PROBED = 0,
 	QEDN_STATE_CORE_OPEN,
 	QEDN_STATE_GL_PF_LIST_ADDED,
 	QEDN_STATE_MFW_STATE,
+	QEDN_STATE_NVMETCP_OPEN,
+	QEDN_STATE_IRQ_SET,
+	QEDN_STATE_FP_WORK_THREAD_SET,
 	QEDN_STATE_REGISTERED_OFFLOAD_DEV,
 	QEDN_STATE_MODULE_REMOVE_ONGOING,
 };
 
+struct qedn_fp_queue {
+	struct qedn_ctx	*qedn;
+	struct qed_chain cq_chain;
+	struct qed_sb_info *sb_info;
+	u16 sb_id;
+	u16 *cq_prod;
+	unsigned int cpu;
+	char irqname[QEDN_IRQ_NAME_LEN];
+	struct mutex cq_mutex; /* cq handler mutex */
+};
+
 struct qedn_ctx {
 	struct pci_dev *pdev;
 	struct qed_dev *cdev;
+	struct qed_int_info int_info;
 	struct qed_dev_nvmetcp_info dev_info;
 	struct nvme_tcp_ofld_dev qedn_ofld_dev;
 	struct qed_pf_params pf_params;
@@ -50,6 +74,9 @@ struct qedn_ctx {
 
 	/* Fast path queues */
 	u8 num_fw_cqs;
+	struct qedn_fp_queue *fp_q_arr;
+	u64 *fw_cq_array_virt;
+	dma_addr_t fw_cq_array_phy;	/* Physical address of fw_cq_array_virt */
 };
 
 struct qedn_global {
-- 
2.22.0



From Chaitanya.Kulkarni at wdc.com  Sun Feb  7 13:58:22 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 7 Feb 2021 18:58:22 +0000
Subject: [PATCH] nvmet-fc: add a missing __rcu annotation to
 nvmet_fc_tgt_assoc.queues
References: <20210207162028.2959778-1-hch@lst.de>
Message-ID: <BYAPR04MB49655AB622A135A6FBB6B97B86B09@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/7/21 08:20, Christoph Hellwig wrote:
> Make sparse happy after the recent conversion to RCU lookups.
>
> Fixes: 4e2f02bf77da ("nvmet-fc: use RCU proctection for assoc_list")
> Signed-off-by: Christoph Hellwig <hch at lst.de>
Looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>



From Damien.LeMoal at wdc.com  Sun Feb  7 18:06:36 2021
From: Damien.LeMoal at wdc.com (Damien Le Moal)
Date: Sun, 7 Feb 2021 23:06:36 +0000
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
References: <20210205195002.2657209-1-kbusch@kernel.org>
Message-ID: <BL0PR04MB65147CDC426FC6345B2048D7E7B09@BL0PR04MB6514.namprd04.prod.outlook.com>

On 2021/02/06 4:50, Keith Busch wrote:
> The bio based drivers only require the request_queue's nr_zones is set,
> so set this field in the head if the namespace path is zoned.
> 
> Reported-by: Minwoo Im <minwoo.im.dev at gmail.com>
> Cc: Damien Le Moal <damien.lemoal at wdc.com>
> Signed-off-by: Keith Busch <kbusch at kernel.org>

Shouldn't this have a Fixes tag ?

> ---
>  drivers/nvme/host/multipath.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> index 1427c9555cef..a1d476e1ac02 100644
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -677,6 +677,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
>  	if (blk_queue_stable_writes(ns->queue) && ns->head->disk)
>  		blk_queue_flag_set(QUEUE_FLAG_STABLE_WRITES,
>  				   ns->head->disk->queue);
> +#ifdef CONFIG_BLK_DEV_ZONED
> +	if (blk_queue_is_zoned(ns->queue) && ns->head->disk)
> +		ns->head->disk->queue->nr_zones = ns->queue->nr_zones;
> +#endif

Is the zone model set already ? I have not checked...

>  }
>  
>  void nvme_mpath_remove_disk(struct nvme_ns_head *head)
> 


-- 
Damien Le Moal
Western Digital Research


From mskmchanouk at gmail.com  Sun Feb  7 19:52:07 2021
From: mskmchanouk at gmail.com (Ms. Kim Chan)
Date: Mon, 8 Feb 2021 07:52:07 +0700
Subject: Investment research about your country
Message-ID: <mailman.8.1612745533.938.linux-nvme@lists.infradead.org>

My name is Ms. Kim Chan Ouk, A Thai citizen in Asian Continent but currently
residing in abroad, I trust this email meets you in good health, I just got
your contact today during a research about your country and I will say that
I foresee success of my future investment ventures in your country. I want
to invest in your country and I would like you to assist me in my investment
my investment Please do reply me as soon as possible so
we can discuss more.

I look forward to hear from you soon
mskmchanouk at gmail.com


From kbusch at kernel.org  Sun Feb  7 22:28:51 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 8 Feb 2021 12:28:51 +0900
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
In-Reply-To: <BL0PR04MB65147CDC426FC6345B2048D7E7B09@BL0PR04MB6514.namprd04.prod.outlook.com>
References: <20210205195002.2657209-1-kbusch@kernel.org>
 <BL0PR04MB65147CDC426FC6345B2048D7E7B09@BL0PR04MB6514.namprd04.prod.outlook.com>
Message-ID: <20210208032851.GA14062@redsun51.ssa.fujisawa.hgst.com>

On Sun, Feb 07, 2021 at 11:06:36PM +0000, Damien Le Moal wrote:
> On 2021/02/06 4:50, Keith Busch wrote:
> > The bio based drivers only require the request_queue's nr_zones is set,
> > so set this field in the head if the namespace path is zoned.
> > 
> > Reported-by: Minwoo Im <minwoo.im.dev at gmail.com>
> > Cc: Damien Le Moal <damien.lemoal at wdc.com>
> > Signed-off-by: Keith Busch <kbusch at kernel.org>
> 
> Shouldn't this have a Fixes tag ?

Probably yes, since this omission does go back to the initial inclusion:

Fixes: 240e6ee272c07 ("nvme: support for zoned namespaces")
 
> > ---
> >  drivers/nvme/host/multipath.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> > index 1427c9555cef..a1d476e1ac02 100644
> > --- a/drivers/nvme/host/multipath.c
> > +++ b/drivers/nvme/host/multipath.c
> > @@ -677,6 +677,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
> >  	if (blk_queue_stable_writes(ns->queue) && ns->head->disk)
> >  		blk_queue_flag_set(QUEUE_FLAG_STABLE_WRITES,
> >  				   ns->head->disk->queue);
> > +#ifdef CONFIG_BLK_DEV_ZONED
> > +	if (blk_queue_is_zoned(ns->queue) && ns->head->disk)
> > +		ns->head->disk->queue->nr_zones = ns->queue->nr_zones;
> > +#endif
> 
> Is the zone model set already ? I have not checked...

That's a good question, and we should be safe with the zone namespace's
zone settings set at this point: nvme_mpath_add_disk() is invoked after
nvme_update_ns_info().


From Damien.LeMoal at wdc.com  Sun Feb  7 22:51:18 2021
From: Damien.LeMoal at wdc.com (Damien Le Moal)
Date: Mon, 8 Feb 2021 03:51:18 +0000
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
References: <20210205195002.2657209-1-kbusch@kernel.org>
 <BL0PR04MB65147CDC426FC6345B2048D7E7B09@BL0PR04MB6514.namprd04.prod.outlook.com>
 <20210208032851.GA14062@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <BL0PR04MB651449A902E7FAD397E2169EE78F9@BL0PR04MB6514.namprd04.prod.outlook.com>

On 2021/02/08 12:28, Keith Busch wrote:
> On Sun, Feb 07, 2021 at 11:06:36PM +0000, Damien Le Moal wrote:
>> On 2021/02/06 4:50, Keith Busch wrote:
>>> The bio based drivers only require the request_queue's nr_zones is set,
>>> so set this field in the head if the namespace path is zoned.
>>>
>>> Reported-by: Minwoo Im <minwoo.im.dev at gmail.com>
>>> Cc: Damien Le Moal <damien.lemoal at wdc.com>
>>> Signed-off-by: Keith Busch <kbusch at kernel.org>
>>
>> Shouldn't this have a Fixes tag ?
> 
> Probably yes, since this omission does go back to the initial inclusion:
> 
> Fixes: 240e6ee272c07 ("nvme: support for zoned namespaces")
>  
>>> ---
>>>  drivers/nvme/host/multipath.c | 4 ++++
>>>  1 file changed, 4 insertions(+)
>>>
>>> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
>>> index 1427c9555cef..a1d476e1ac02 100644
>>> --- a/drivers/nvme/host/multipath.c
>>> +++ b/drivers/nvme/host/multipath.c
>>> @@ -677,6 +677,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
>>>  	if (blk_queue_stable_writes(ns->queue) && ns->head->disk)
>>>  		blk_queue_flag_set(QUEUE_FLAG_STABLE_WRITES,
>>>  				   ns->head->disk->queue);
>>> +#ifdef CONFIG_BLK_DEV_ZONED
>>> +	if (blk_queue_is_zoned(ns->queue) && ns->head->disk)
>>> +		ns->head->disk->queue->nr_zones = ns->queue->nr_zones;
>>> +#endif
>>
>> Is the zone model set already ? I have not checked...
> 
> That's a good question, and we should be safe with the zone namespace's
> zone settings set at this point: nvme_mpath_add_disk() is invoked after
> nvme_update_ns_info().

Hmmm... I see the zone model being set for ns->queue, but nothing for
ns->head->disk->queue... Shoudln't nvme_mpath_add_disk() also set the zoned
model for ns->head->disk->queue ?


-- 
Damien Le Moal
Western Digital Research


From yi.zhang at redhat.com  Mon Feb  8 01:48:11 2021
From: yi.zhang at redhat.com (Yi Zhang)
Date: Mon, 8 Feb 2021 14:48:11 +0800
Subject: kernel null pointer at nvme_tcp_init_iter[nvme_tcp] with blktests
 nvme-tcp/012
In-Reply-To: <BYAPR04MB49654E940FACB778A787B68E86B09@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <80b7184e-cdfb-cebc-fe07-a228ce57a9e7@redhat.com>
 <BYAPR04MB49654E940FACB778A787B68E86B09@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <79af2e42-f1f1-bb51-2e6c-b0d8a7294bcc@redhat.com>



On 2/7/21 1:25 PM, Chaitanya Kulkarni wrote:
> Yi,
>
> On 2/6/21 20:51, Yi Zhang wrote:
>> The issue was introduced after merge the NVMe updates
>>
>> commit 0fd6456fd1f4c8f3ec5a2df6ed7f34458a180409 (HEAD)
>> Merge: 44d10e4b2f2c 0d7389718c32
>> Author: Jens Axboe <axboe at kernel.dk>
>> Date:   Tue Feb 2 07:12:06 2021 -0700
>>
>>       Merge branch 'for-5.12/drivers' into for-next
>>
>>       * for-5.12/drivers: (22 commits)
>>         nvme-tcp: use cancel tagset helper for tear down
>>         nvme-rdma: use cancel tagset helper for tear down
>>         nvme-tcp: add clean action for failed reconnection
>>         nvme-rdma: add clean action for failed reconnection
>>         nvme-core: add cancel tagset helpers
>>         nvme-core: get rid of the extra space
>>         nvme: add tracing of zns commands
>>         nvme: parse format nvm command details when tracing
>>         nvme: update enumerations for status codes
>>         nvmet: add lba to sect conversion helpers
>>         nvmet: remove extra variable in identify ns
>>         nvmet: remove extra variable in id-desclist
>>         nvmet: remove extra variable in smart log nsid
>>         nvme: refactor ns->ctrl by request
>>         nvme-tcp: pass multipage bvec to request iov_iter
>>         nvme-tcp: get rid of unused helper function
>>         nvme-tcp: fix wrong setting of request iov_iter
>>         nvme: support command retry delay for admin command
>>         nvme: constify static attribute_group structs
>>         nvmet-fc: use RCU proctection for assoc_list
>>
>>
>> On 2/6/21 11:08 AM, Yi Zhang wrote:
>>> blktests nvme-tcp/012
> Thanks for reporting, you can further bisect this using following
> NVMe tree if is from NVMEe tree :- http://git.infradead.org/nvme.git
> Branch :- 5.12
I've tried that branch, but encountered another NULL pointer.

[? 224.290720] run blktests nvme/012 at 2021-02-08 01:46:25
[? 224.515479] BUG: kernel NULL pointer dereference, address: 
0000000000000368
[? 224.522442] #PF: supervisor read access in kernel mode
[? 224.527590] #PF: error_code(0x0000) - not-present page
[? 224.532737] PGD 0 P4D 0
[? 224.535276] Oops: 0000 [#1] SMP PTI
[? 224.538770] CPU: 0 PID: 2403 Comm: multipath Tainted: G S I?????? 
5.11.0-rc5+ #1
[? 224.546776] Hardware name: Dell Inc. PowerEdge R640/08HT8T, BIOS 
2.10.0 11/12/2020
[? 224.554340] RIP: 0010:bio_associate_blkg_from_css+0xbd/0x2c0
[? 224.560009] Code: 03 75 79 65 48 ff 00 e8 b1 94 cc ff e8 ac 94 cc ff 
49 89 5c 24 48 48 83 c4 10 5b 5d 41 5c 41 5d 41 5e 41 5f c3 49 8b 44 24 
08 <48> 8b 808
[? 224.578761] RSP: 0018:ffffad748ec5bd78 EFLAGS: 00010246
[? 224.583988] RAX: 0000000000000000 RBX: ffffa0e96452b3c0 RCX: 
0000000000000000
[? 224.591120] RDX: ffffa11077088000 RSI: ffffffffa06edb20 RDI: 
ffffa0e96452b3c0
[? 224.598253] RBP: 0000000000000000 R08: 0000000000000000 R09: 
0000000000000002
[? 224.605385] R10: 0000000000000022 R11: 0000000000000c10 R12: 
ffffa0e96452b3c0
[? 224.612516] R13: ffffffffa06edb20 R14: 0000000000001000 R15: 
0000000000000000
[? 224.619649] FS:? 00007fb6647feb00(0000) GS:ffffa107f0000000(0000) 
knlGS:0000000000000000
[? 224.627734] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[? 224.633479] CR2: 0000000000000368 CR3: 0000002884b54002 CR4: 
00000000007706f0
[? 224.640611] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
[? 224.647751] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
0000000000000400
[? 224.654884] PKRU: 55555554
[? 224.657595] Call Trace:
[? 224.660043]? bio_associate_blkg+0x20/0x70
[? 224.664055]? nvme_submit_user_cmd+0xd7/0x340 [nvme_core]
[? 224.669375]? nvme_user_cmd.isra.82+0x120/0x190 [nvme_core]
[? 224.674859]? nvme_dev_ioctl+0x13b/0x150 [nvme_core]
[? 224.679738]? __x64_sys_ioctl+0x84/0xc0
[? 224.683500]? do_syscall_64+0x33/0x40
[? 224.687088]? entry_SYSCALL_64_after_hwframe+0x44/0xa9
[? 224.692149] RIP: 0033:0x7fb66338161b
[? 224.695726] Code: 0f 1e fa 48 8b 05 6d b8 2c 00 64 c7 00 26 00 00 00 
48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 
05 <48> 3d 018
[? 224.714469] RSP: 002b:00007ffeae59aba8 EFLAGS: 00000246 ORIG_RAX: 
0000000000000010
[? 224.722036] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 
00007fb66338161b
[? 224.729168] RDX: 00007ffeae59abb0 RSI: 00000000c0484e41 RDI: 
0000000000000003
[? 224.736300] RBP: 00007ffeae59ac10 R08: 0000559af0bb3d20 R09: 
00007fb66340e580
[? 224.743431] R10: 0000000000000000 R11: 0000000000000246 R12: 
00007ffeae59bcb0
[? 224.750563] R13: 0000559af0bb3d20 R14: 0000000000000003 R15: 
0000000000000000
[? 224.757697] Modules linked in: loop nvme_tcp nvme_fabrics nvme_core 
nvmet_tcp nvmet rfkill sunrpc dm_multipath intel_rapl_msr 
intel_rapl_common isst_if_cod
[? 224.833264] CR2: 0000000000000368
[? 224.836604] ---[ end trace fa77e7cc26f8cfc1 ]---
[? 224.883915] RIP: 0010:bio_associate_blkg_from_css+0xbd/0x2c0
[? 224.889584] Code: 03 75 79 65 48 ff 00 e8 b1 94 cc ff e8 ac 94 cc ff 
49 89 5c 24 48 48 83 c4 10 5b 5d 41 5c 41 5d 41 5e 41 5f c3 49 8b 44 24 
08 <48> 8b 808
[? 224.908329] RSP: 0018:ffffad748ec5bd78 EFLAGS: 00010246
[? 224.913554] RAX: 0000000000000000 RBX: ffffa0e96452b3c0 RCX: 
0000000000000000
[? 224.920686] RDX: ffffa11077088000 RSI: ffffffffa06edb20 RDI: 
ffffa0e96452b3c0
[? 224.927818] RBP: 0000000000000000 R08: 0000000000000000 R09: 
0000000000000002
[? 224.934948] R10: 0000000000000022 R11: 0000000000000c10 R12: 
ffffa0e96452b3c0
[? 224.942082] R13: ffffffffa06edb20 R14: 0000000000001000 R15: 
0000000000000000
[? 224.949213] FS:? 00007fb6647feb00(0000) GS:ffffa107f0000000(0000) 
knlGS:0000000000000000
[? 224.957301] CS:? 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[? 224.963044] CR2: 0000000000000368 CR3: 0000002884b54002 CR4: 
00000000007706f0
[? 224.970177] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
[? 224.977309] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
0000000000000400
[? 224.984441] PKRU: 55555554
[? 224.987155] Kernel panic - not syncing: Fatal exception
[? 225.425279] Kernel Offset: 0x1d400000 from 0xffffffff81000000 
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[? 225.478268] ---[ end Kernel panic - not syncing: Fatal exception ]---

> Looking at the commit log it seems like nvme-tcp commits might be
> the issue given that you got the problem in nvme_tcp_init_iter()
> and just eliminating by the looking at the code
> commit 9f99a29e5307 can lead to this behavior.
>
> *I may be completely wrong as I'm not expert in the nvme-tcp host.*
>
>
>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
>



From lkp at intel.com  Mon Feb  8 03:29:01 2021
From: lkp at intel.com (kernel test robot)
Date: Mon, 08 Feb 2021 16:29:01 +0800
Subject: [linux-nvme:nvme-5.12] BUILD SUCCESS
 51964ee257c859c06c62887f8709c82f9dafd67e
Message-ID: <6020f64d.9CMxJyseLFpJKb3I%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.12
branch HEAD: 51964ee257c859c06c62887f8709c82f9dafd67e  nvme-multipath: set nr_zones for zoned namespaces

elapsed time: 966m

configs tested: 92
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
powerpc64                           defconfig
powerpc                      ppc44x_defconfig
i386                             alldefconfig
arm                          gemini_defconfig
arm                      pxa255-idp_defconfig
arm                         vf610m4_defconfig
mips                           ip32_defconfig
s390                             allyesconfig
arc                        nsim_700_defconfig
parisc                generic-32bit_defconfig
arm                      integrator_defconfig
um                           x86_64_defconfig
arm                           h3600_defconfig
sparc                               defconfig
arm                             mxs_defconfig
arm                       cns3420vb_defconfig
microblaze                          defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allmodconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
i386                               tinyconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a001-20210208
i386                 randconfig-a005-20210208
i386                 randconfig-a003-20210208
i386                 randconfig-a002-20210208
i386                 randconfig-a006-20210208
i386                 randconfig-a004-20210208
x86_64               randconfig-a013-20210208
x86_64               randconfig-a014-20210208
x86_64               randconfig-a015-20210208
x86_64               randconfig-a012-20210208
x86_64               randconfig-a016-20210208
x86_64               randconfig-a011-20210208
i386                 randconfig-a013-20210207
i386                 randconfig-a016-20210207
i386                 randconfig-a014-20210207
i386                 randconfig-a012-20210207
i386                 randconfig-a015-20210207
i386                 randconfig-a011-20210207
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a006-20210208
x86_64               randconfig-a001-20210208
x86_64               randconfig-a005-20210208
x86_64               randconfig-a004-20210208
x86_64               randconfig-a002-20210208
x86_64               randconfig-a003-20210208

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From sagi at grimberg.me  Mon Feb  8 04:46:17 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 01:46:17 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
Message-ID: <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>


> Hello
> 
> We found this kernel NULL pointer issue with latest linux-block/for-next and it's 100% reproduced, let me know if you need more info/testing, thanks
> 
> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
> 
> Reproducer: blktests nvme-tcp/012

Thanks for reporting Ming, I've tried to reproduce this on my VM
but did not succeed. Given that you have it 100% reproducible,
can you try to revert commit:

0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter

Also, would it be possible to share your config?


From sagi at grimberg.me  Mon Feb  8 04:53:36 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 01:53:36 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <BYAPR04MB4965E82FFD9D6FC443C8E67686B09@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <BYAPR04MB4965E82FFD9D6FC443C8E67686B09@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <36f95b6a-bd50-eae8-ccd5-9a714c942c12@grimberg.me>



On 2/6/21 11:14 PM, Chaitanya Kulkarni wrote:
> Sagi,
> 
> On 2/5/21 19:19, Yi Zhang wrote:
>> Hello
>>
>> We found this kernel NULL pointer issue with latest linux-block/for-next and it's 100% reproduced, let me know if you need more info/testing, thanks
>>
>> Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>
>> Reproducer: blktests nvme-tcp/012
>>
>>
>> [  124.458121] run blktests nvme/012 at 2021-02-05 21:53:34
>> [  125.525568] BUG: kernel NULL pointer dereference, address: 0000000000000008
>> [  125.532524] #PF: supervisor read access in kernel mode
>> [  125.537665] #PF: error_code(0x0000) - not-present page
>> [  125.542803] PGD 0 P4D 0
>> [  125.545343] Oops: 0000 [#1] SMP NOPTI
>> [  125.549009] CPU: 15 PID: 12069 Comm: kworker/15:2H Tainted: G S        I       5.11.0-rc6+ #1
>> [  125.557528] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
>> [  125.565093] Workqueue: kblockd blk_mq_run_work_fn
>> [  125.569797] RIP: 0010:nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
>> [  125.575544] Code: 8b 75 68 44 8b 45 28 44 8b 7d 30 49 89 d4 48 c1 e2 04 4c 01 f2 45 89 fb 44 89 c7 85 ff 74 4d 44 89 e0 44 8b 55 10 48 c1 e0 04 <41> 8b 5c 06 08 45 0f b6 ca 89 d8 44 29 d8 39 f8 0f 47 c7 41 83 e9
>> [  125.594290] RSP: 0018:ffffbd084447bd18 EFLAGS: 00010246
>> [  125.599515] RAX: 0000000000000000 RBX: ffffa0bba9f3ce80 RCX: 0000000000000000
>> [  125.606648] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000002000000
>> [  125.613781] RBP: ffffa0ba8ac6fec0 R08: 0000000002000000 R09: 0000000000000000
>> [  125.620914] R10: 0000000002800809 R11: 0000000000000000 R12: 0000000000000000
>> [  125.628045] R13: ffffa0bba9f3cf90 R14: 0000000000000000 R15: 0000000000000000
>> [  125.635178] FS:  0000000000000000(0000) GS:ffffa0c9ff9c0000(0000) knlGS:0000000000000000
>> [  125.643264] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [  125.649009] CR2: 0000000000000008 CR3: 00000001c9c6c005 CR4: 00000000007706e0
>> [  125.656142] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [  125.663274] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> [  125.670407] PKRU: 55555554
>> [  125.673119] Call Trace:
>> [  125.675575]  nvme_tcp_queue_rq+0xef/0x330 [nvme_tcp]
>> [  125.680537]  blk_mq_dispatch_rq_list+0x11c/0x7c0
>> [  125.685157]  ? blk_mq_flush_busy_ctxs+0xf6/0x110
>> [  125.689775]  __blk_mq_sched_dispatch_requests+0x12b/0x170
>> [  125.695175]  blk_mq_sched_dispatch_requests+0x30/0x60
>> [  125.700227]  __blk_mq_run_hw_queue+0x2b/0x60
>> [  125.704500]  process_one_work+0x1cb/0x360
>> [  125.708513]  ? process_one_work+0x360/0x360
>> [  125.712699]  worker_thread+0x30/0x370
>> [  125.716365]  ? process_one_work+0x360/0x360
>> [  125.720550]  kthread+0x116/0x130
>> [  125.723782]  ? kthread_park+0x80/0x80
>> [  125.727448]  ret_from_fork+0x1f/0x30
> The NVMe TCP does support merging for non-admin queues
> (nvme_tcp_alloc_tagset()).
> 
> Based on the what is been done for the bvecs in other places in
> kernel especially when merging is enabled bio split case seems to be
> missing from tcp when building bvec. What I mean is following
> completely untested patch :-
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 619b0d8f6e38..dabb2633b28c 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -222,7 +222,9 @@ static void nvme_tcp_init_iter(struct
> nvme_tcp_request *req,
>           unsigned int dir)
>   {
>       struct request *rq = blk_mq_rq_from_pdu(req);
> -    struct bio_vec *vec;
> +    struct req_iterator rq_iter;
> +    struct bio_vec *vec, *tvec;
> +    struct bio_vec tmp;
>       unsigned int size;
>       int nr_bvec;
>       size_t offset;
> @@ -233,17 +235,29 @@ static void nvme_tcp_init_iter(struct
> nvme_tcp_request *req,
>           size = blk_rq_payload_bytes(rq);
>           offset = 0;
>       } else {
> -        struct bio *bio = req->curr_bio;
> -        struct bvec_iter bi;
> -        struct bio_vec bv;
> -
> -        vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
> -        nr_bvec = 0;
> -        bio_for_each_bvec(bv, bio, bi) {
> +        rq_for_each_bvec(tmp, rq, rq_iter)
>               nr_bvec++;
> +
> +        if (rq->bio != rq->biotail) {
> +            vec = kmalloc_array(nr_bvec, sizeof(struct bio_vec),
> +                    GFP_NOIO);
> +            if (!vec)
> +                return;
> +
> +            tvec = vec;
> +            rq_for_each_bvec(tmp, rq, rq_iter) {
> +                *vec = tmp;
> +                vec++;
> +            }
> +            vec = tvec;
> +            offset = 0;
> +        } else {
> +            struct bio *bio = req->curr_bio;
> +
> +            vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
> +            size = bio->bi_iter.bi_size;
> +            offset = bio->bi_iter.bi_bvec_done;
>           }
> -        size = bio->bi_iter.bi_size;
> -        offset = bio->bi_iter.bi_bvec_done;
>       }
>   
>       iov_iter_bvec(&req->iter, dir, vec, nr_bvec, size);
> @@ -2271,8 +2285,11 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct
> nvme_ns *ns,
>       req->data_len = blk_rq_nr_phys_segments(rq) ?
>                   blk_rq_payload_bytes(rq) : 0;
>       req->curr_bio = rq->bio;
> -    if (req->curr_bio)
> +    if (req->curr_bio) {
> +        if (rq->bio != rq->biotail)
> +            printk(KERN_INFO"%s %d req->bio != req->biotail\n",
> __func__, __LINE__);
>           nvme_tcp_init_iter(req, rq_data_dir(rq));
> +    }
>   
>       if (rq_data_dir(rq) == WRITE &&
>           req->data_len <= nvme_tcp_inline_data_size(queue))
> 

That cannot work, nvme-tcp basically will initialize the iter based
on the request bios, as it iterates it may continue if the request
spans more bios. We could change that, but this would mean we need to
allocate in the fast path which is something I'd prefer to avoid.
This is a regression so it must be coming from one of the latest
patches, so that should be addressed.


From yi.zhang at redhat.com  Mon Feb  8 08:28:43 2021
From: yi.zhang at redhat.com (Yi Zhang)
Date: Mon, 8 Feb 2021 21:28:43 +0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
Message-ID: <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>

Hi Sagi

On 2/8/21 5:46 PM, Sagi Grimberg wrote:
>
>> Hello
>>
>> We found this kernel NULL pointer issue with latest 
>> linux-block/for-next and it's 100% reproduced, let me know if you 
>> need more info/testing, thanks
>>
>> Kernel repo: 
>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>
>> Reproducer: blktests nvme-tcp/012
>
> Thanks for reporting Ming, I've tried to reproduce this on my VM
> but did not succeed. Given that you have it 100% reproducible,
> can you try to revert commit:
>
> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>

Revert this commit fixed the issue and I've attached the config. :)


> Also, would it be possible to share your config?
>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
>

-------------- next part --------------
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86 5.11.0-rc6 Kernel Configuration
#
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=80401
CONFIG_LD_VERSION=230000000
CONFIG_CLANG_VERSION=0
CONFIG_LLD_VERSION=0
CONFIG_CC_CAN_LINK=y
CONFIG_CC_HAS_ASM_GOTO=y
CONFIG_CC_HAS_ASM_INLINE=y
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_TABLE_SORT=y
CONFIG_THREAD_INFO_IN_TASK=y

#
# General setup
#
CONFIG_INIT_ENV_ARG_LIMIT=32
# CONFIG_COMPILE_TEST is not set
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_BUILD_SALT=""
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_HAVE_KERNEL_ZSTD=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
# CONFIG_KERNEL_ZSTD is not set
CONFIG_DEFAULT_INIT=""
CONFIG_DEFAULT_HOSTNAME="(none)"
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
# CONFIG_WATCH_QUEUE is not set
CONFIG_CROSS_MEMORY_ATTACH=y
# CONFIG_USELIB is not set
CONFIG_AUDIT=y
CONFIG_HAVE_ARCH_AUDITSYSCALL=y
CONFIG_AUDITSYSCALL=y

#
# IRQ subsystem
#
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_GENERIC_IRQ_MIGRATION=y
CONFIG_GENERIC_IRQ_INJECTION=y
CONFIG_HARDIRQS_SW_RESEND=y
CONFIG_IRQ_DOMAIN=y
CONFIG_IRQ_DOMAIN_HIERARCHY=y
CONFIG_GENERIC_MSI_IRQ=y
CONFIG_GENERIC_MSI_IRQ_DOMAIN=y
CONFIG_IRQ_MSI_IOMMU=y
CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR=y
CONFIG_GENERIC_IRQ_RESERVATION_MODE=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
# CONFIG_GENERIC_IRQ_DEBUGFS is not set
# end of IRQ subsystem

CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_ARCH_CLOCKSOURCE_INIT=y
CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_HAVE_POSIX_CPU_TIMERS_TASK_WORK=y
CONFIG_POSIX_CPU_TIMERS_TASK_WORK=y

#
# Timers subsystem
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
# CONFIG_NO_HZ_IDLE is not set
CONFIG_NO_HZ_FULL=y
CONFIG_CONTEXT_TRACKING=y
# CONFIG_CONTEXT_TRACKING_FORCE is not set
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
# end of Timers subsystem

# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set

#
# CPU/Task time and stats accounting
#
CONFIG_VIRT_CPU_ACCOUNTING=y
CONFIG_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_SCHED_AVG_IRQ=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_BSD_PROCESS_ACCT_V3=y
CONFIG_TASKSTATS=y
CONFIG_TASK_DELAY_ACCT=y
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_PSI=y
CONFIG_PSI_DEFAULT_DISABLED=y
# end of CPU/Task time and stats accounting

CONFIG_CPU_ISOLATION=y

#
# RCU Subsystem
#
CONFIG_TREE_RCU=y
# CONFIG_RCU_EXPERT is not set
CONFIG_SRCU=y
CONFIG_TREE_SRCU=y
CONFIG_TASKS_RCU_GENERIC=y
CONFIG_TASKS_RUDE_RCU=y
CONFIG_TASKS_TRACE_RCU=y
CONFIG_RCU_STALL_COMMON=y
CONFIG_RCU_NEED_SEGCBLIST=y
CONFIG_RCU_NOCB_CPU=y
# end of RCU Subsystem

CONFIG_BUILD_BIN2C=y
# CONFIG_IKCONFIG is not set
CONFIG_IKHEADERS=m
CONFIG_LOG_BUF_SHIFT=20
CONFIG_LOG_CPU_MAX_BUF_SHIFT=12
CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y

#
# Scheduler features
#
# CONFIG_UCLAMP_TASK is not set
# end of Scheduler features

CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH=y
CONFIG_CC_HAS_INT128=y
CONFIG_ARCH_SUPPORTS_INT128=y
CONFIG_NUMA_BALANCING=y
CONFIG_NUMA_BALANCING_DEFAULT_ENABLED=y
CONFIG_CGROUPS=y
CONFIG_PAGE_COUNTER=y
CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
CONFIG_MEMCG_KMEM=y
CONFIG_BLK_CGROUP=y
CONFIG_CGROUP_WRITEBACK=y
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_CFS_BANDWIDTH=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_CGROUP_PIDS=y
CONFIG_CGROUP_RDMA=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_HUGETLB=y
CONFIG_CPUSETS=y
CONFIG_PROC_PID_CPUSET=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_BPF=y
# CONFIG_CGROUP_DEBUG is not set
CONFIG_SOCK_CGROUP_DATA=y
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_TIME_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_CHECKPOINT_RESTORE=y
CONFIG_SCHED_AUTOGROUP=y
# CONFIG_SYSFS_DEPRECATED is not set
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
CONFIG_RD_ZSTD=y
# CONFIG_BOOT_CONFIG is not set
CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_LD_ORPHAN_WARN=y
CONFIG_SYSCTL=y
CONFIG_HAVE_UID16=y
CONFIG_SYSCTL_EXCEPTION_TRACE=y
CONFIG_HAVE_PCSPKR_PLATFORM=y
CONFIG_BPF=y
# CONFIG_EXPERT is not set
CONFIG_UID16=y
CONFIG_MULTIUSER=y
CONFIG_SGETMASK_SYSCALL=y
CONFIG_SYSFS_SYSCALL=y
CONFIG_FHANDLE=y
CONFIG_POSIX_TIMERS=y
CONFIG_PRINTK=y
CONFIG_PRINTK_NMI=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_FUTEX_PI=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_IO_URING=y
CONFIG_ADVISE_SYSCALLS=y
CONFIG_HAVE_ARCH_USERFAULTFD_WP=y
CONFIG_MEMBARRIER=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_ABSOLUTE_PERCPU=y
CONFIG_KALLSYMS_BASE_RELATIVE=y
# CONFIG_BPF_LSM is not set
CONFIG_BPF_SYSCALL=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_BPF_JIT_DEFAULT_ON=y
# CONFIG_BPF_PRELOAD is not set
CONFIG_USERFAULTFD=y
CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE=y
CONFIG_RSEQ=y
# CONFIG_EMBEDDED is not set
CONFIG_HAVE_PERF_EVENTS=y

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
# end of Kernel Performance Events And Counters

CONFIG_VM_EVENT_COUNTERS=y
CONFIG_SLUB_DEBUG=y
# CONFIG_COMPAT_BRK is not set
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLAB_MERGE_DEFAULT is not set
CONFIG_SLAB_FREELIST_RANDOM=y
# CONFIG_SLAB_FREELIST_HARDENED is not set
CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
CONFIG_SLUB_CPU_PARTIAL=y
CONFIG_SYSTEM_DATA_VERIFICATION=y
CONFIG_PROFILING=y
CONFIG_TRACEPOINTS=y
# end of General setup

CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_MMU=y
CONFIG_ARCH_MMAP_RND_BITS_MIN=28
CONFIG_ARCH_MMAP_RND_BITS_MAX=32
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=8
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=16
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_ARCH_HAS_FILTER_PGPROT=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
CONFIG_ZONE_DMA32=y
CONFIG_AUDIT_ARCH=y
CONFIG_HAVE_INTEL_TXT=y
CONFIG_X86_64_SMP=y
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_DYNAMIC_PHYSICAL_MASK=y
CONFIG_PGTABLE_LEVELS=5
CONFIG_CC_HAS_SANE_STACKPROTECTOR=y

#
# Processor type and features
#
CONFIG_ZONE_DMA=y
CONFIG_SMP=y
CONFIG_X86_FEATURE_NAMES=y
CONFIG_X86_X2APIC=y
CONFIG_X86_MPPARSE=y
# CONFIG_GOLDFISH is not set
CONFIG_RETPOLINE=y
CONFIG_X86_CPU_RESCTRL=y
CONFIG_X86_EXTENDED_PLATFORM=y
# CONFIG_X86_NUMACHIP is not set
# CONFIG_X86_VSMP is not set
CONFIG_X86_UV=y
# CONFIG_X86_GOLDFISH is not set
# CONFIG_X86_INTEL_MID is not set
CONFIG_X86_INTEL_LPSS=y
CONFIG_X86_AMD_PLATFORM_DEVICE=y
CONFIG_IOSF_MBI=y
# CONFIG_IOSF_MBI_DEBUG is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_DEBUG is not set
CONFIG_PARAVIRT_SPINLOCKS=y
CONFIG_X86_HV_CALLBACK_VECTOR=y
CONFIG_XEN=y
# CONFIG_XEN_PV is not set
CONFIG_XEN_PVHVM=y
CONFIG_XEN_PVHVM_SMP=y
CONFIG_XEN_PVHVM_GUEST=y
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_DEBUG_FS is not set
# CONFIG_XEN_PVH is not set
CONFIG_KVM_GUEST=y
CONFIG_ARCH_CPUIDLE_HALTPOLL=y
# CONFIG_PVH is not set
CONFIG_PARAVIRT_TIME_ACCOUNTING=y
CONFIG_PARAVIRT_CLOCK=y
# CONFIG_JAILHOUSE_GUEST is not set
# CONFIG_ACRN_GUEST is not set
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
CONFIG_GENERIC_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_IA32_FEAT_CTL=y
CONFIG_X86_VMX_FEATURE_NAMES=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_HYGON=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_CPU_SUP_ZHAOXIN=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
# CONFIG_GART_IOMMU is not set
CONFIG_MAXSMP=y
CONFIG_NR_CPUS_RANGE_BEGIN=8192
CONFIG_NR_CPUS_RANGE_END=8192
CONFIG_NR_CPUS_DEFAULT=8192
CONFIG_NR_CPUS=8192
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
CONFIG_SCHED_MC_PRIO=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
CONFIG_X86_MCE=y
CONFIG_X86_MCELOG_LEGACY=y
CONFIG_X86_MCE_INTEL=y
CONFIG_X86_MCE_AMD=y
CONFIG_X86_MCE_THRESHOLD=y
CONFIG_X86_MCE_INJECT=m
CONFIG_X86_THERMAL_VECTOR=y

#
# Performance monitoring
#
CONFIG_PERF_EVENTS_INTEL_UNCORE=m
CONFIG_PERF_EVENTS_INTEL_RAPL=m
CONFIG_PERF_EVENTS_INTEL_CSTATE=m
CONFIG_PERF_EVENTS_AMD_POWER=m
# end of Performance monitoring

CONFIG_X86_16BIT=y
CONFIG_X86_ESPFIX64=y
CONFIG_X86_VSYSCALL_EMULATION=y
CONFIG_X86_IOPL_IOPERM=y
CONFIG_I8K=m
CONFIG_MICROCODE=y
CONFIG_MICROCODE_INTEL=y
CONFIG_MICROCODE_AMD=y
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_X86_5LEVEL=y
CONFIG_X86_DIRECT_GBPAGES=y
# CONFIG_X86_CPA_STATISTICS is not set
CONFIG_AMD_MEM_ENCRYPT=y
# CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT is not set
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
CONFIG_NUMA_EMU=y
CONFIG_NODES_SHIFT=10
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
# CONFIG_ARCH_MEMORY_PROBE is not set
CONFIG_ARCH_PROC_KCORE_TEXT=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_X86_PMEM_LEGACY_DEVICE=y
CONFIG_X86_PMEM_LEGACY=m
CONFIG_X86_CHECK_BIOS_CORRUPTION=y
# CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK is not set
CONFIG_X86_RESERVE_LOW=64
CONFIG_MTRR=y
CONFIG_MTRR_SANITIZER=y
CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1
CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=1
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
CONFIG_ARCH_RANDOM=y
CONFIG_X86_SMAP=y
CONFIG_X86_UMIP=y
CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS=y
CONFIG_X86_INTEL_TSX_MODE_OFF=y
# CONFIG_X86_INTEL_TSX_MODE_ON is not set
# CONFIG_X86_INTEL_TSX_MODE_AUTO is not set
# CONFIG_X86_SGX is not set
CONFIG_EFI=y
CONFIG_EFI_STUB=y
CONFIG_EFI_MIXED=y
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
CONFIG_KEXEC_FILE=y
CONFIG_ARCH_HAS_KEXEC_PURGATORY=y
CONFIG_KEXEC_SIG=y
# CONFIG_KEXEC_SIG_FORCE is not set
CONFIG_KEXEC_BZIMAGE_VERIFY_SIG=y
CONFIG_CRASH_DUMP=y
CONFIG_KEXEC_JUMP=y
CONFIG_PHYSICAL_START=0x1000000
CONFIG_RELOCATABLE=y
CONFIG_RANDOMIZE_BASE=y
CONFIG_X86_NEED_RELOCS=y
CONFIG_PHYSICAL_ALIGN=0x200000
CONFIG_DYNAMIC_MEMORY_LAYOUT=y
CONFIG_RANDOMIZE_MEMORY=y
CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING=0xa
CONFIG_HOTPLUG_CPU=y
CONFIG_BOOTPARAM_HOTPLUG_CPU0=y
# CONFIG_DEBUG_HOTPLUG_CPU0 is not set
# CONFIG_COMPAT_VDSO is not set
CONFIG_LEGACY_VSYSCALL_EMULATE=y
# CONFIG_LEGACY_VSYSCALL_XONLY is not set
# CONFIG_LEGACY_VSYSCALL_NONE is not set
# CONFIG_CMDLINE_BOOL is not set
CONFIG_MODIFY_LDT_SYSCALL=y
CONFIG_HAVE_LIVEPATCH=y
CONFIG_LIVEPATCH=y
# end of Processor type and features

CONFIG_ARCH_HAS_ADD_PAGES=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION=y
CONFIG_ARCH_ENABLE_THP_MIGRATION=y

#
# Power management and ACPI options
#
CONFIG_ARCH_HIBERNATION_HEADER=y
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
CONFIG_HIBERNATE_CALLBACKS=y
CONFIG_HIBERNATION=y
CONFIG_HIBERNATION_SNAPSHOT_DEV=y
CONFIG_PM_STD_PARTITION=""
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
# CONFIG_PM_AUTOSLEEP is not set
# CONFIG_PM_WAKELOCKS is not set
CONFIG_PM=y
CONFIG_PM_DEBUG=y
# CONFIG_PM_ADVANCED_DEBUG is not set
# CONFIG_PM_TEST_SUSPEND is not set
CONFIG_PM_SLEEP_DEBUG=y
# CONFIG_PM_TRACE_RTC is not set
CONFIG_PM_CLK=y
CONFIG_PM_GENERIC_DOMAINS=y
# CONFIG_WQ_POWER_EFFICIENT_DEFAULT is not set
CONFIG_PM_GENERIC_DOMAINS_SLEEP=y
# CONFIG_ENERGY_MODEL is not set
CONFIG_ARCH_SUPPORTS_ACPI=y
CONFIG_ACPI=y
CONFIG_ACPI_LEGACY_TABLES_LOOKUP=y
CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC=y
CONFIG_ACPI_SYSTEM_POWER_STATES_SUPPORT=y
# CONFIG_ACPI_DEBUGGER is not set
CONFIG_ACPI_SPCR_TABLE=y
CONFIG_ACPI_LPIT=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_REV_OVERRIDE_POSSIBLE=y
CONFIG_ACPI_EC_DEBUGFS=m
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=y
CONFIG_ACPI_TAD=m
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_CPU_FREQ_PSS=y
CONFIG_ACPI_PROCESSOR_CSTATE=y
CONFIG_ACPI_PROCESSOR_IDLE=y
CONFIG_ACPI_CPPC_LIB=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_IPMI=m
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PROCESSOR_AGGREGATOR=m
CONFIG_ACPI_THERMAL=y
CONFIG_ARCH_HAS_ACPI_TABLE_UPGRADE=y
CONFIG_ACPI_TABLE_UPGRADE=y
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_PCI_SLOT=y
CONFIG_ACPI_CONTAINER=y
CONFIG_ACPI_HOTPLUG_MEMORY=y
CONFIG_ACPI_HOTPLUG_IOAPIC=y
CONFIG_ACPI_SBS=m
CONFIG_ACPI_HED=y
# CONFIG_ACPI_CUSTOM_METHOD is not set
CONFIG_ACPI_BGRT=y
CONFIG_ACPI_NFIT=m
# CONFIG_NFIT_SECURITY_DEBUG is not set
CONFIG_ACPI_NUMA=y
CONFIG_ACPI_HMAT=y
CONFIG_HAVE_ACPI_APEI=y
CONFIG_HAVE_ACPI_APEI_NMI=y
CONFIG_ACPI_APEI=y
CONFIG_ACPI_APEI_GHES=y
CONFIG_ACPI_APEI_PCIEAER=y
CONFIG_ACPI_APEI_MEMORY_FAILURE=y
CONFIG_ACPI_APEI_EINJ=m
# CONFIG_ACPI_APEI_ERST_DEBUG is not set
# CONFIG_ACPI_DPTF is not set
CONFIG_ACPI_WATCHDOG=y
CONFIG_ACPI_EXTLOG=m
CONFIG_ACPI_ADXL=y
# CONFIG_ACPI_CONFIGFS is not set
CONFIG_PMIC_OPREGION=y
CONFIG_X86_PM_TIMER=y
CONFIG_SFI=y

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_ATTR_SET=y
CONFIG_CPU_FREQ_GOV_COMMON=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y

#
# CPU frequency scaling drivers
#
CONFIG_X86_INTEL_PSTATE=y
# CONFIG_X86_PCC_CPUFREQ is not set
CONFIG_X86_ACPI_CPUFREQ=m
CONFIG_X86_ACPI_CPUFREQ_CPB=y
CONFIG_X86_POWERNOW_K8=m
CONFIG_X86_AMD_FREQ_SENSITIVITY=m
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
CONFIG_X86_P4_CLOCKMOD=m

#
# shared options
#
CONFIG_X86_SPEEDSTEP_LIB=m
# end of CPU Frequency scaling

#
# CPU Idle
#
CONFIG_CPU_IDLE=y
# CONFIG_CPU_IDLE_GOV_LADDER is not set
CONFIG_CPU_IDLE_GOV_MENU=y
# CONFIG_CPU_IDLE_GOV_TEO is not set
CONFIG_CPU_IDLE_GOV_HALTPOLL=y
CONFIG_HALTPOLL_CPUIDLE=m
# end of CPU Idle

CONFIG_INTEL_IDLE=y
# end of Power management and ACPI options

#
# Bus options (PCI etc.)
#
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_XEN=y
CONFIG_MMCONF_FAM10H=y
CONFIG_ISA_DMA_API=y
CONFIG_AMD_NB=y
# CONFIG_X86_SYSFB is not set
# end of Bus options (PCI etc.)

#
# Binary Emulations
#
CONFIG_IA32_EMULATION=y
# CONFIG_X86_X32 is not set
CONFIG_COMPAT_32=y
CONFIG_COMPAT=y
CONFIG_COMPAT_FOR_U64_ALIGNMENT=y
CONFIG_SYSVIPC_COMPAT=y
# end of Binary Emulations

#
# Firmware Drivers
#
CONFIG_EDD=m
# CONFIG_EDD_OFF is not set
CONFIG_FIRMWARE_MEMMAP=y
CONFIG_DMIID=y
CONFIG_DMI_SYSFS=y
CONFIG_DMI_SCAN_MACHINE_NON_EFI_FALLBACK=y
CONFIG_ISCSI_IBFT_FIND=y
CONFIG_ISCSI_IBFT=m
CONFIG_FW_CFG_SYSFS=y
# CONFIG_FW_CFG_SYSFS_CMDLINE is not set
# CONFIG_GOOGLE_FIRMWARE is not set

#
# EFI (Extensible Firmware Interface) Support
#
CONFIG_EFI_VARS=y
CONFIG_EFI_ESRT=y
CONFIG_EFI_VARS_PSTORE=y
CONFIG_EFI_VARS_PSTORE_DEFAULT_DISABLE=y
CONFIG_EFI_RUNTIME_MAP=y
# CONFIG_EFI_FAKE_MEMMAP is not set
CONFIG_EFI_SOFT_RESERVE=y
CONFIG_EFI_RUNTIME_WRAPPERS=y
CONFIG_EFI_GENERIC_STUB_INITRD_CMDLINE_LOADER=y
# CONFIG_EFI_BOOTLOADER_CONTROL is not set
# CONFIG_EFI_CAPSULE_LOADER is not set
# CONFIG_EFI_TEST is not set
CONFIG_APPLE_PROPERTIES=y
# CONFIG_RESET_ATTACK_MITIGATION is not set
CONFIG_EFI_RCI2_TABLE=y
# CONFIG_EFI_DISABLE_PCI_DMA is not set
# end of EFI (Extensible Firmware Interface) Support

CONFIG_UEFI_CPER=y
CONFIG_UEFI_CPER_X86=y
CONFIG_EFI_DEV_PATH_PARSER=y
CONFIG_EFI_EARLYCON=y
CONFIG_EFI_CUSTOM_SSDT_OVERLAYS=y

#
# Tegra firmware driver
#
# end of Tegra firmware driver
# end of Firmware Drivers

CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_COMPAT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_HAVE_KVM_NO_POLL=y
CONFIG_KVM_XFER_TO_GUEST_WORK=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
CONFIG_KVM_AMD=m
CONFIG_KVM_AMD_SEV=y
CONFIG_KVM_MMU_AUDIT=y
CONFIG_AS_AVX512=y
CONFIG_AS_SHA1_NI=y
CONFIG_AS_SHA256_NI=y
CONFIG_AS_TPAUSE=y

#
# General architecture-dependent options
#
CONFIG_CRASH_CORE=y
CONFIG_KEXEC_CORE=y
CONFIG_HOTPLUG_SMT=y
CONFIG_GENERIC_ENTRY=y
CONFIG_OPROFILE=m
CONFIG_OPROFILE_EVENT_MULTIPLEX=y
CONFIG_HAVE_OPROFILE=y
CONFIG_OPROFILE_NMI_TIMER=y
CONFIG_KPROBES=y
CONFIG_JUMP_LABEL=y
# CONFIG_STATIC_KEYS_SELFTEST is not set
# CONFIG_STATIC_CALL_SELFTEST is not set
CONFIG_OPTPROBES=y
CONFIG_KPROBES_ON_FTRACE=y
CONFIG_UPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_ARCH_USE_BUILTIN_BSWAP=y
CONFIG_KRETPROBES=y
CONFIG_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_HAVE_FUNCTION_ERROR_INJECTION=y
CONFIG_HAVE_NMI=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_CONTIGUOUS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_ARCH_HAS_FORTIFY_SOURCE=y
CONFIG_ARCH_HAS_SET_MEMORY=y
CONFIG_ARCH_HAS_SET_DIRECT_MAP=y
CONFIG_HAVE_ARCH_THREAD_STRUCT_WHITELIST=y
CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT=y
CONFIG_HAVE_ASM_MODVERSIONS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_RSEQ=y
CONFIG_HAVE_FUNCTION_ARG_ACCESS_API=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_HARDLOCKUP_DETECTOR_PERF=y
CONFIG_HAVE_PERF_REGS=y
CONFIG_HAVE_PERF_USER_STACK_DUMP=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE=y
CONFIG_MMU_GATHER_TABLE_FREE=y
CONFIG_MMU_GATHER_RCU_TABLE_FREE=y
CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y
CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y
CONFIG_HAVE_CMPXCHG_LOCAL=y
CONFIG_HAVE_CMPXCHG_DOUBLE=y
CONFIG_ARCH_WANT_COMPAT_IPC_PARSE_VERSION=y
CONFIG_ARCH_WANT_OLD_COMPAT_IPC=y
CONFIG_HAVE_ARCH_SECCOMP=y
CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
CONFIG_SECCOMP=y
CONFIG_SECCOMP_FILTER=y
# CONFIG_SECCOMP_CACHE_DEBUG is not set
CONFIG_HAVE_ARCH_STACKLEAK=y
CONFIG_HAVE_STACKPROTECTOR=y
CONFIG_STACKPROTECTOR=y
CONFIG_STACKPROTECTOR_STRONG=y
CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES=y
CONFIG_HAVE_CONTEXT_TRACKING=y
CONFIG_HAVE_CONTEXT_TRACKING_OFFSTACK=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_MOVE_PUD=y
CONFIG_HAVE_MOVE_PMD=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD=y
CONFIG_HAVE_ARCH_HUGE_VMAP=y
CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
CONFIG_HAVE_ARCH_SOFT_DIRTY=y
CONFIG_HAVE_MOD_ARCH_SPECIFIC=y
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
CONFIG_HAVE_ARCH_MMAP_RND_BITS=y
CONFIG_HAVE_EXIT_THREAD=y
CONFIG_ARCH_MMAP_RND_BITS=28
CONFIG_HAVE_ARCH_MMAP_RND_COMPAT_BITS=y
CONFIG_ARCH_MMAP_RND_COMPAT_BITS=8
CONFIG_HAVE_ARCH_COMPAT_MMAP_BASES=y
CONFIG_HAVE_STACK_VALIDATION=y
CONFIG_HAVE_RELIABLE_STACKTRACE=y
CONFIG_OLD_SIGSUSPEND3=y
CONFIG_COMPAT_OLD_SIGACTION=y
CONFIG_COMPAT_32BIT_TIME=y
CONFIG_HAVE_ARCH_VMAP_STACK=y
CONFIG_VMAP_STACK=y
CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
CONFIG_STRICT_KERNEL_RWX=y
CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
CONFIG_STRICT_MODULE_RWX=y
CONFIG_HAVE_ARCH_PREL32_RELOCATIONS=y
CONFIG_ARCH_USE_MEMREMAP_PROT=y
# CONFIG_LOCK_EVENT_COUNTS is not set
CONFIG_ARCH_HAS_MEM_ENCRYPT=y
CONFIG_HAVE_STATIC_CALL=y
CONFIG_HAVE_STATIC_CALL_INLINE=y
CONFIG_ARCH_WANT_LD_ORPHAN_WARN=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y

#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y
# end of GCOV-based kernel profiling

CONFIG_HAVE_GCC_PLUGINS=y
# end of General architecture-dependent options

CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
CONFIG_MODULE_FORCE_LOAD=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_MODVERSIONS=y
CONFIG_ASM_MODVERSIONS=y
CONFIG_MODULE_SRCVERSION_ALL=y
CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set
CONFIG_MODULE_SIG_ALL=y
# CONFIG_MODULE_SIG_SHA1 is not set
# CONFIG_MODULE_SIG_SHA224 is not set
CONFIG_MODULE_SIG_SHA256=y
# CONFIG_MODULE_SIG_SHA384 is not set
# CONFIG_MODULE_SIG_SHA512 is not set
CONFIG_MODULE_SIG_HASH="sha256"
# CONFIG_MODULE_COMPRESS is not set
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set
# CONFIG_UNUSED_SYMBOLS is not set
# CONFIG_TRIM_UNUSED_KSYMS is not set
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_BLOCK=y
CONFIG_BLK_SCSI_REQUEST=y
CONFIG_BLK_CGROUP_RWSTAT=y
CONFIG_BLK_DEV_BSG=y
CONFIG_BLK_DEV_BSGLIB=y
CONFIG_BLK_DEV_INTEGRITY=y
CONFIG_BLK_DEV_INTEGRITY_T10=m
# CONFIG_BLK_DEV_ZONED is not set
CONFIG_BLK_DEV_THROTTLING=y
# CONFIG_BLK_DEV_THROTTLING_LOW is not set
# CONFIG_BLK_CMDLINE_PARSER is not set
CONFIG_BLK_WBT=y
CONFIG_BLK_CGROUP_IOLATENCY=y
# CONFIG_BLK_CGROUP_IOCOST is not set
CONFIG_BLK_WBT_MQ=y
CONFIG_BLK_DEBUG_FS=y
# CONFIG_BLK_SED_OPAL is not set
# CONFIG_BLK_INLINE_ENCRYPTION is not set

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_AIX_PARTITION is not set
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
# CONFIG_ATARI_PARTITION is not set
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
# CONFIG_LDM_PARTITION is not set
CONFIG_SGI_PARTITION=y
# CONFIG_ULTRIX_PARTITION is not set
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
# CONFIG_CMDLINE_PARTITION is not set
# end of Partition Types

CONFIG_BLOCK_COMPAT=y
CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_BLK_MQ_RDMA=y
CONFIG_BLK_PM=y

#
# IO Schedulers
#
CONFIG_MQ_IOSCHED_DEADLINE=y
CONFIG_MQ_IOSCHED_KYBER=y
CONFIG_IOSCHED_BFQ=y
CONFIG_BFQ_GROUP_IOSCHED=y
# CONFIG_BFQ_CGROUP_DEBUG is not set
# end of IO Schedulers

CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_PADATA=y
CONFIG_ASN1=y
CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
CONFIG_INLINE_READ_UNLOCK=y
CONFIG_INLINE_READ_UNLOCK_IRQ=y
CONFIG_INLINE_WRITE_UNLOCK=y
CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
CONFIG_MUTEX_SPIN_ON_OWNER=y
CONFIG_RWSEM_SPIN_ON_OWNER=y
CONFIG_LOCK_SPIN_ON_OWNER=y
CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
CONFIG_QUEUED_SPINLOCKS=y
CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
CONFIG_QUEUED_RWLOCKS=y
CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE=y
CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE=y
CONFIG_ARCH_HAS_SYSCALL_WRAPPER=y
CONFIG_FREEZER=y

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
CONFIG_ELFCORE=y
CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BINFMT_MISC=m
CONFIG_COREDUMP=y
# end of Executable file formats

#
# Memory Management options
#
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_NEED_MULTIPLE_NODES=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_HAVE_FAST_GUP=y
CONFIG_NUMA_KEEP_MEMINFO=y
CONFIG_MEMORY_ISOLATION=y
CONFIG_HAVE_BOOTMEM_INFO_NODE=y
CONFIG_MEMORY_HOTPLUG=y
CONFIG_MEMORY_HOTPLUG_SPARSE=y
# CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE is not set
CONFIG_MEMORY_HOTREMOVE=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_MEMORY_BALLOON=y
CONFIG_BALLOON_COMPACTION=y
CONFIG_COMPACTION=y
CONFIG_PAGE_REPORTING=y
CONFIG_MIGRATION=y
CONFIG_CONTIG_ALLOC=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MMU_NOTIFIER=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
CONFIG_MEMORY_FAILURE=y
CONFIG_HWPOISON_INJECT=m
CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y
# CONFIG_TRANSPARENT_HUGEPAGE_MADVISE is not set
CONFIG_ARCH_WANTS_THP_SWAP=y
CONFIG_THP_SWAP=y
CONFIG_CLEANCACHE=y
CONFIG_FRONTSWAP=y
# CONFIG_CMA is not set
CONFIG_MEM_SOFT_DIRTY=y
CONFIG_ZSWAP=y
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_DEFLATE is not set
CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZO=y
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_842 is not set
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZ4 is not set
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZ4HC is not set
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_ZSTD is not set
CONFIG_ZSWAP_COMPRESSOR_DEFAULT="lzo"
CONFIG_ZSWAP_ZPOOL_DEFAULT_ZBUD=y
# CONFIG_ZSWAP_ZPOOL_DEFAULT_Z3FOLD is not set
# CONFIG_ZSWAP_ZPOOL_DEFAULT_ZSMALLOC is not set
CONFIG_ZSWAP_ZPOOL_DEFAULT="zbud"
# CONFIG_ZSWAP_DEFAULT_ON is not set
CONFIG_ZPOOL=y
CONFIG_ZBUD=y
# CONFIG_Z3FOLD is not set
CONFIG_ZSMALLOC=y
CONFIG_ZSMALLOC_STAT=y
CONFIG_GENERIC_EARLY_IOREMAP=y
CONFIG_DEFERRED_STRUCT_PAGE_INIT=y
CONFIG_IDLE_PAGE_TRACKING=y
CONFIG_ARCH_HAS_PTE_DEVMAP=y
CONFIG_ZONE_DEVICE=y
CONFIG_DEV_PAGEMAP_OPS=y
CONFIG_HMM_MIRROR=y
CONFIG_DEVICE_PRIVATE=y
CONFIG_VMAP_PFN=y
CONFIG_FRAME_VECTOR=y
CONFIG_ARCH_USES_HIGH_VMA_FLAGS=y
CONFIG_ARCH_HAS_PKEYS=y
# CONFIG_PERCPU_STATS is not set
# CONFIG_GUP_TEST is not set
# CONFIG_READ_ONLY_THP_FOR_FS is not set
CONFIG_ARCH_HAS_PTE_SPECIAL=y
CONFIG_MAPPING_DIRTY_HELPERS=y
# end of Memory Management options

CONFIG_NET=y
CONFIG_NET_INGRESS=y
CONFIG_NET_EGRESS=y
CONFIG_NET_REDIRECT=y
CONFIG_SKB_EXTENSIONS=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_DIAG=m
CONFIG_UNIX=y
CONFIG_UNIX_SCM=y
CONFIG_UNIX_DIAG=m
CONFIG_TLS=m
CONFIG_TLS_DEVICE=y
# CONFIG_TLS_TOE is not set
CONFIG_XFRM=y
CONFIG_XFRM_OFFLOAD=y
CONFIG_XFRM_ALGO=y
CONFIG_XFRM_USER=y
# CONFIG_XFRM_USER_COMPAT is not set
CONFIG_XFRM_INTERFACE=m
CONFIG_XFRM_SUB_POLICY=y
CONFIG_XFRM_MIGRATE=y
CONFIG_XFRM_STATISTICS=y
CONFIG_XFRM_AH=m
CONFIG_XFRM_ESP=m
CONFIG_XFRM_IPCOMP=m
CONFIG_NET_KEY=m
CONFIG_NET_KEY_MIGRATE=y
CONFIG_XFRM_ESPINTCP=y
# CONFIG_SMC is not set
CONFIG_XDP_SOCKETS=y
CONFIG_XDP_SOCKETS_DIAG=m
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_FIB_TRIE_STATS=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
CONFIG_IP_ROUTE_CLASSID=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IP_TUNNEL=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE_COMMON=y
CONFIG_IP_MROUTE=y
CONFIG_IP_MROUTE_MULTIPLE_TABLES=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_NET_UDP_TUNNEL=m
# CONFIG_NET_FOU is not set
# CONFIG_NET_FOU_IP_TUNNELS is not set
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_ESP_OFFLOAD=m
CONFIG_INET_ESPINTCP=y
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_DIAG=m
CONFIG_INET_TCP_DIAG=m
CONFIG_INET_UDP_DIAG=m
CONFIG_INET_RAW_DIAG=m
# CONFIG_INET_DIAG_DESTROY is not set
CONFIG_TCP_CONG_ADVANCED=y
CONFIG_TCP_CONG_BIC=m
CONFIG_TCP_CONG_CUBIC=y
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
CONFIG_TCP_CONG_HSTCP=m
CONFIG_TCP_CONG_HYBLA=m
CONFIG_TCP_CONG_VEGAS=m
CONFIG_TCP_CONG_NV=m
CONFIG_TCP_CONG_SCALABLE=m
CONFIG_TCP_CONG_LP=m
CONFIG_TCP_CONG_VENO=m
CONFIG_TCP_CONG_YEAH=m
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_TCP_CONG_DCTCP=m
# CONFIG_TCP_CONG_CDG is not set
CONFIG_TCP_CONG_BBR=m
CONFIG_DEFAULT_CUBIC=y
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="cubic"
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_ESP_OFFLOAD=m
CONFIG_INET6_ESPINTCP=y
CONFIG_INET6_IPCOMP=m
CONFIG_IPV6_MIP6=m
# CONFIG_IPV6_ILA is not set
CONFIG_INET6_XFRM_TUNNEL=m
CONFIG_INET6_TUNNEL=m
CONFIG_IPV6_VTI=m
CONFIG_IPV6_SIT=m
CONFIG_IPV6_SIT_6RD=y
CONFIG_IPV6_NDISC_NODETYPE=y
CONFIG_IPV6_TUNNEL=m
CONFIG_IPV6_GRE=m
CONFIG_IPV6_MULTIPLE_TABLES=y
# CONFIG_IPV6_SUBTREES is not set
CONFIG_IPV6_MROUTE=y
CONFIG_IPV6_MROUTE_MULTIPLE_TABLES=y
CONFIG_IPV6_PIMSM_V2=y
# CONFIG_IPV6_SEG6_LWTUNNEL is not set
# CONFIG_IPV6_SEG6_HMAC is not set
# CONFIG_IPV6_RPL_LWTUNNEL is not set
CONFIG_NETLABEL=y
CONFIG_MPTCP=y
CONFIG_INET_MPTCP_DIAG=m
CONFIG_MPTCP_IPV6=y
CONFIG_MPTCP_KUNIT_TESTS=m
CONFIG_NETWORK_SECMARK=y
CONFIG_NET_PTP_CLASSIFY=y
CONFIG_NETWORK_PHY_TIMESTAMPING=y
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=m

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_INGRESS=y
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_FAMILY_BRIDGE=y
CONFIG_NETFILTER_FAMILY_ARP=y
# CONFIG_NETFILTER_NETLINK_ACCT is not set
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NETFILTER_NETLINK_OSF=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_LOG_COMMON=m
CONFIG_NF_LOG_NETDEV=m
CONFIG_NETFILTER_CONNCOUNT=m
CONFIG_NF_CONNTRACK_MARK=y
CONFIG_NF_CONNTRACK_SECMARK=y
CONFIG_NF_CONNTRACK_ZONES=y
CONFIG_NF_CONNTRACK_PROCFS=y
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CONNTRACK_TIMEOUT=y
CONFIG_NF_CONNTRACK_TIMESTAMP=y
CONFIG_NF_CONNTRACK_LABELS=y
CONFIG_NF_CT_PROTO_DCCP=y
CONFIG_NF_CT_PROTO_GRE=y
CONFIG_NF_CT_PROTO_SCTP=y
CONFIG_NF_CT_PROTO_UDPLITE=y
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NF_CONNTRACK_IRC=m
CONFIG_NF_CONNTRACK_BROADCAST=m
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NF_CONNTRACK_SNMP=m
CONFIG_NF_CONNTRACK_PPTP=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
CONFIG_NF_CT_NETLINK=m
CONFIG_NF_CT_NETLINK_TIMEOUT=m
CONFIG_NF_CT_NETLINK_HELPER=m
CONFIG_NETFILTER_NETLINK_GLUE_CT=y
CONFIG_NF_NAT=m
CONFIG_NF_NAT_AMANDA=m
CONFIG_NF_NAT_FTP=m
CONFIG_NF_NAT_IRC=m
CONFIG_NF_NAT_SIP=m
CONFIG_NF_NAT_TFTP=m
CONFIG_NF_NAT_REDIRECT=y
CONFIG_NF_NAT_MASQUERADE=y
CONFIG_NETFILTER_SYNPROXY=m
CONFIG_NF_TABLES=m
CONFIG_NF_TABLES_INET=y
CONFIG_NF_TABLES_NETDEV=y
CONFIG_NFT_NUMGEN=m
CONFIG_NFT_CT=m
CONFIG_NFT_FLOW_OFFLOAD=m
CONFIG_NFT_COUNTER=m
CONFIG_NFT_CONNLIMIT=m
CONFIG_NFT_LOG=m
CONFIG_NFT_LIMIT=m
CONFIG_NFT_MASQ=m
CONFIG_NFT_REDIR=m
CONFIG_NFT_NAT=m
# CONFIG_NFT_TUNNEL is not set
CONFIG_NFT_OBJREF=m
CONFIG_NFT_QUEUE=m
CONFIG_NFT_QUOTA=m
CONFIG_NFT_REJECT=m
CONFIG_NFT_REJECT_INET=m
CONFIG_NFT_COMPAT=m
CONFIG_NFT_HASH=m
CONFIG_NFT_FIB=m
CONFIG_NFT_FIB_INET=m
CONFIG_NFT_XFRM=m
CONFIG_NFT_SOCKET=m
# CONFIG_NFT_OSF is not set
CONFIG_NFT_TPROXY=m
# CONFIG_NFT_SYNPROXY is not set
CONFIG_NF_DUP_NETDEV=m
CONFIG_NFT_DUP_NETDEV=m
CONFIG_NFT_FWD_NETDEV=m
CONFIG_NFT_FIB_NETDEV=m
# CONFIG_NFT_REJECT_NETDEV is not set
CONFIG_NF_FLOW_TABLE_INET=m
CONFIG_NF_FLOW_TABLE=m
CONFIG_NETFILTER_XTABLES=y

#
# Xtables combined modules
#
CONFIG_NETFILTER_XT_MARK=m
CONFIG_NETFILTER_XT_CONNMARK=m
CONFIG_NETFILTER_XT_SET=m

#
# Xtables targets
#
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=m
CONFIG_NETFILTER_XT_TARGET_CT=m
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_HL=m
CONFIG_NETFILTER_XT_TARGET_HMARK=m
CONFIG_NETFILTER_XT_TARGET_IDLETIMER=m
# CONFIG_NETFILTER_XT_TARGET_LED is not set
CONFIG_NETFILTER_XT_TARGET_LOG=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_NAT=m
CONFIG_NETFILTER_XT_TARGET_NETMAP=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_RATEEST=m
CONFIG_NETFILTER_XT_TARGET_REDIRECT=m
CONFIG_NETFILTER_XT_TARGET_MASQUERADE=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_SECMARK=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m

#
# Xtables matches
#
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
CONFIG_NETFILTER_XT_MATCH_BPF=m
CONFIG_NETFILTER_XT_MATCH_CGROUP=m
CONFIG_NETFILTER_XT_MATCH_CLUSTER=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNLABEL=m
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_CPU=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DEVGROUP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ECN=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_HL=m
# CONFIG_NETFILTER_XT_MATCH_IPCOMP is not set
CONFIG_NETFILTER_XT_MATCH_IPRANGE=m
CONFIG_NETFILTER_XT_MATCH_IPVS=m
# CONFIG_NETFILTER_XT_MATCH_L2TP is not set
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
# CONFIG_NETFILTER_XT_MATCH_NFACCT is not set
CONFIG_NETFILTER_XT_MATCH_OSF=m
CONFIG_NETFILTER_XT_MATCH_OWNER=m
CONFIG_NETFILTER_XT_MATCH_POLICY=m
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
# CONFIG_NETFILTER_XT_MATCH_TIME is not set
CONFIG_NETFILTER_XT_MATCH_U32=m
# end of Core Netfilter Configuration

CONFIG_IP_SET=m
CONFIG_IP_SET_MAX=256
CONFIG_IP_SET_BITMAP_IP=m
CONFIG_IP_SET_BITMAP_IPMAC=m
CONFIG_IP_SET_BITMAP_PORT=m
CONFIG_IP_SET_HASH_IP=m
CONFIG_IP_SET_HASH_IPMARK=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
CONFIG_IP_SET_HASH_IPMAC=m
CONFIG_IP_SET_HASH_MAC=m
CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_IP_VS=m
CONFIG_IP_VS_IPV6=y
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12

#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_AH_ESP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y
CONFIG_IP_VS_PROTO_SCTP=y

#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_FO=m
CONFIG_IP_VS_OVF=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
# CONFIG_IP_VS_MH is not set
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m

#
# IPVS SH scheduler
#
CONFIG_IP_VS_SH_TAB_BITS=8

#
# IPVS MH scheduler
#
CONFIG_IP_VS_MH_TAB_INDEX=12

#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
CONFIG_IP_VS_NFCT=y
CONFIG_IP_VS_PE_SIP=m

#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=m
CONFIG_NF_SOCKET_IPV4=m
CONFIG_NF_TPROXY_IPV4=m
CONFIG_NF_TABLES_IPV4=y
CONFIG_NFT_REJECT_IPV4=m
CONFIG_NFT_DUP_IPV4=m
CONFIG_NFT_FIB_IPV4=m
CONFIG_NF_TABLES_ARP=y
# CONFIG_NF_FLOW_TABLE_IPV4 is not set
CONFIG_NF_DUP_IPV4=m
CONFIG_NF_LOG_ARP=m
CONFIG_NF_LOG_IPV4=m
CONFIG_NF_REJECT_IPV4=m
CONFIG_NF_NAT_SNMP_BASIC=m
CONFIG_NF_NAT_PPTP=m
CONFIG_NF_NAT_H323=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_RPFILTER=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_MANGLE=m
# CONFIG_IP_NF_TARGET_CLUSTERIP is not set
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_SECURITY=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
# end of IP: Netfilter Configuration

#
# IPv6: Netfilter Configuration
#
CONFIG_NF_SOCKET_IPV6=m
CONFIG_NF_TPROXY_IPV6=m
CONFIG_NF_TABLES_IPV6=y
CONFIG_NFT_REJECT_IPV6=m
CONFIG_NFT_DUP_IPV6=m
CONFIG_NFT_FIB_IPV6=m
# CONFIG_NF_FLOW_TABLE_IPV6 is not set
CONFIG_NF_DUP_IPV6=m
CONFIG_NF_REJECT_IPV6=m
CONFIG_NF_LOG_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_MH=m
CONFIG_IP6_NF_MATCH_RPFILTER=m
CONFIG_IP6_NF_MATCH_RT=m
# CONFIG_IP6_NF_MATCH_SRH is not set
# CONFIG_IP6_NF_TARGET_HL is not set
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_IP6_NF_SECURITY=m
CONFIG_IP6_NF_NAT=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
# end of IPv6: Netfilter Configuration

CONFIG_NF_DEFRAG_IPV6=m
CONFIG_NF_TABLES_BRIDGE=m
# CONFIG_NFT_BRIDGE_META is not set
CONFIG_NFT_BRIDGE_REJECT=m
CONFIG_NF_LOG_BRIDGE=m
# CONFIG_NF_CONNTRACK_BRIDGE is not set
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_IP6=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_BRIDGE_EBT_NFLOG=m
# CONFIG_BPFILTER is not set
# CONFIG_IP_DCCP is not set
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_DEFAULT_COOKIE_HMAC_MD5 is not set
CONFIG_SCTP_DEFAULT_COOKIE_HMAC_SHA1=y
# CONFIG_SCTP_DEFAULT_COOKIE_HMAC_NONE is not set
CONFIG_SCTP_COOKIE_HMAC_MD5=y
CONFIG_SCTP_COOKIE_HMAC_SHA1=y
CONFIG_INET_SCTP_DIAG=m
# CONFIG_RDS is not set
CONFIG_TIPC=m
CONFIG_TIPC_MEDIA_IB=y
CONFIG_TIPC_MEDIA_UDP=y
CONFIG_TIPC_CRYPTO=y
CONFIG_TIPC_DIAG=m
CONFIG_ATM=m
CONFIG_ATM_CLIP=m
# CONFIG_ATM_CLIP_NO_ICMP is not set
CONFIG_ATM_LANE=m
# CONFIG_ATM_MPOA is not set
CONFIG_ATM_BR2684=m
# CONFIG_ATM_BR2684_IPFILTER is not set
CONFIG_L2TP=m
CONFIG_L2TP_DEBUGFS=m
CONFIG_L2TP_V3=y
CONFIG_L2TP_IP=m
CONFIG_L2TP_ETH=m
CONFIG_STP=m
CONFIG_GARP=m
CONFIG_MRP=m
CONFIG_BRIDGE=m
CONFIG_BRIDGE_IGMP_SNOOPING=y
CONFIG_BRIDGE_VLAN_FILTERING=y
# CONFIG_BRIDGE_MRP is not set
# CONFIG_BRIDGE_CFM is not set
CONFIG_HAVE_NET_DSA=y
# CONFIG_NET_DSA is not set
CONFIG_VLAN_8021Q=m
CONFIG_VLAN_8021Q_GVRP=y
CONFIG_VLAN_8021Q_MVRP=y
# CONFIG_DECNET is not set
CONFIG_LLC=m
# CONFIG_LLC2 is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_PHONET is not set
CONFIG_6LOWPAN=m
# CONFIG_6LOWPAN_DEBUGFS is not set
# CONFIG_6LOWPAN_NHC is not set
CONFIG_IEEE802154=m
# CONFIG_IEEE802154_NL802154_EXPERIMENTAL is not set
CONFIG_IEEE802154_SOCKET=m
CONFIG_IEEE802154_6LOWPAN=m
CONFIG_MAC802154=m
CONFIG_NET_SCHED=y

#
# Queueing/Scheduling
#
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_ATM=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_MULTIQ=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFB=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_CBS=m
# CONFIG_NET_SCH_ETF is not set
# CONFIG_NET_SCH_TAPRIO is not set
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_DRR=m
CONFIG_NET_SCH_MQPRIO=m
# CONFIG_NET_SCH_SKBPRIO is not set
CONFIG_NET_SCH_CHOKE=m
CONFIG_NET_SCH_QFQ=m
CONFIG_NET_SCH_CODEL=m
CONFIG_NET_SCH_FQ_CODEL=y
# CONFIG_NET_SCH_CAKE is not set
CONFIG_NET_SCH_FQ=m
CONFIG_NET_SCH_HHF=m
CONFIG_NET_SCH_PIE=m
# CONFIG_NET_SCH_FQ_PIE is not set
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_SCH_PLUG=m
CONFIG_NET_SCH_ETS=m
CONFIG_NET_SCH_DEFAULT=y
# CONFIG_DEFAULT_FQ is not set
# CONFIG_DEFAULT_CODEL is not set
CONFIG_DEFAULT_FQ_CODEL=y
# CONFIG_DEFAULT_SFQ is not set
# CONFIG_DEFAULT_PFIFO_FAST is not set
CONFIG_DEFAULT_NET_SCH="fq_codel"

#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_CLS_U32_PERF=y
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_FLOW=m
CONFIG_NET_CLS_CGROUP=y
CONFIG_NET_CLS_BPF=m
CONFIG_NET_CLS_FLOWER=m
CONFIG_NET_CLS_MATCHALL=m
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_STACK=32
CONFIG_NET_EMATCH_CMP=m
CONFIG_NET_EMATCH_NBYTE=m
CONFIG_NET_EMATCH_U32=m
CONFIG_NET_EMATCH_META=m
CONFIG_NET_EMATCH_TEXT=m
# CONFIG_NET_EMATCH_CANID is not set
CONFIG_NET_EMATCH_IPSET=m
# CONFIG_NET_EMATCH_IPT is not set
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=m
CONFIG_NET_ACT_GACT=m
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_SAMPLE=m
# CONFIG_NET_ACT_IPT is not set
CONFIG_NET_ACT_NAT=m
CONFIG_NET_ACT_PEDIT=m
CONFIG_NET_ACT_SIMP=m
CONFIG_NET_ACT_SKBEDIT=m
CONFIG_NET_ACT_CSUM=m
CONFIG_NET_ACT_MPLS=m
CONFIG_NET_ACT_VLAN=m
CONFIG_NET_ACT_BPF=m
# CONFIG_NET_ACT_CONNMARK is not set
# CONFIG_NET_ACT_CTINFO is not set
CONFIG_NET_ACT_SKBMOD=m
# CONFIG_NET_ACT_IFE is not set
CONFIG_NET_ACT_TUNNEL_KEY=m
CONFIG_NET_ACT_CT=m
# CONFIG_NET_ACT_GATE is not set
CONFIG_NET_TC_SKB_EXT=y
CONFIG_NET_SCH_FIFO=y
CONFIG_DCB=y
CONFIG_DNS_RESOLVER=m
# CONFIG_BATMAN_ADV is not set
CONFIG_OPENVSWITCH=m
CONFIG_OPENVSWITCH_GRE=m
CONFIG_OPENVSWITCH_VXLAN=m
CONFIG_OPENVSWITCH_GENEVE=m
CONFIG_VSOCKETS=m
CONFIG_VSOCKETS_DIAG=m
CONFIG_VSOCKETS_LOOPBACK=m
CONFIG_VMWARE_VMCI_VSOCKETS=m
CONFIG_VIRTIO_VSOCKETS=m
CONFIG_VIRTIO_VSOCKETS_COMMON=m
CONFIG_HYPERV_VSOCKETS=m
CONFIG_NETLINK_DIAG=m
CONFIG_MPLS=y
CONFIG_NET_MPLS_GSO=y
CONFIG_MPLS_ROUTING=m
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=y
# CONFIG_HSR is not set
CONFIG_NET_SWITCHDEV=y
CONFIG_NET_L3_MASTER_DEV=y
# CONFIG_QRTR is not set
# CONFIG_NET_NCSI is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_CGROUP_NET_CLASSID=y
CONFIG_NET_RX_BUSY_POLL=y
CONFIG_BQL=y
CONFIG_BPF_JIT=y
CONFIG_BPF_STREAM_PARSER=y
CONFIG_NET_FLOW_LIMIT=y

#
# Network testing
#
CONFIG_NET_PKTGEN=m
CONFIG_NET_DROP_MONITOR=y
# end of Network testing
# end of Networking options

# CONFIG_HAMRADIO is not set
CONFIG_CAN=m
CONFIG_CAN_RAW=m
CONFIG_CAN_BCM=m
CONFIG_CAN_GW=m
# CONFIG_CAN_J1939 is not set
# CONFIG_CAN_ISOTP is not set

#
# CAN Device Drivers
#
CONFIG_CAN_VCAN=m
# CONFIG_CAN_VXCAN is not set
CONFIG_CAN_SLCAN=m
CONFIG_CAN_DEV=m
CONFIG_CAN_CALC_BITTIMING=y
# CONFIG_CAN_KVASER_PCIEFD is not set
CONFIG_CAN_C_CAN=m
CONFIG_CAN_C_CAN_PLATFORM=m
CONFIG_CAN_C_CAN_PCI=m
CONFIG_CAN_CC770=m
# CONFIG_CAN_CC770_ISA is not set
CONFIG_CAN_CC770_PLATFORM=m
# CONFIG_CAN_IFI_CANFD is not set
# CONFIG_CAN_M_CAN is not set
# CONFIG_CAN_PEAK_PCIEFD is not set
CONFIG_CAN_SJA1000=m
CONFIG_CAN_EMS_PCI=m
# CONFIG_CAN_F81601 is not set
CONFIG_CAN_KVASER_PCI=m
CONFIG_CAN_PEAK_PCI=m
CONFIG_CAN_PEAK_PCIEC=y
CONFIG_CAN_PLX_PCI=m
# CONFIG_CAN_SJA1000_ISA is not set
CONFIG_CAN_SJA1000_PLATFORM=m
CONFIG_CAN_SOFTING=m

#
# CAN SPI interfaces
#
# CONFIG_CAN_HI311X is not set
# CONFIG_CAN_MCP251X is not set
# CONFIG_CAN_MCP251XFD is not set
# end of CAN SPI interfaces

#
# CAN USB interfaces
#
CONFIG_CAN_8DEV_USB=m
CONFIG_CAN_EMS_USB=m
CONFIG_CAN_ESD_USB2=m
# CONFIG_CAN_GS_USB is not set
CONFIG_CAN_KVASER_USB=m
# CONFIG_CAN_MCBA_USB is not set
CONFIG_CAN_PEAK_USB=m
CONFIG_CAN_UCAN=m
# end of CAN USB interfaces

# CONFIG_CAN_DEBUG_DEVICES is not set
# end of CAN Device Drivers

CONFIG_BT=m
CONFIG_BT_BREDR=y
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_CMTP=m
CONFIG_BT_HIDP=m
CONFIG_BT_HS=y
CONFIG_BT_LE=y
# CONFIG_BT_6LOWPAN is not set
# CONFIG_BT_LEDS is not set
# CONFIG_BT_MSFTEXT is not set
CONFIG_BT_DEBUGFS=y
# CONFIG_BT_SELFTEST is not set

#
# Bluetooth device drivers
#
CONFIG_BT_INTEL=m
CONFIG_BT_BCM=m
CONFIG_BT_RTL=m
CONFIG_BT_HCIBTUSB=m
CONFIG_BT_HCIBTUSB_AUTOSUSPEND=y
CONFIG_BT_HCIBTUSB_BCM=y
# CONFIG_BT_HCIBTUSB_MTK is not set
CONFIG_BT_HCIBTUSB_RTL=y
CONFIG_BT_HCIBTSDIO=m
CONFIG_BT_HCIUART=m
CONFIG_BT_HCIUART_H4=y
CONFIG_BT_HCIUART_BCSP=y
CONFIG_BT_HCIUART_ATH3K=y
# CONFIG_BT_HCIUART_INTEL is not set
# CONFIG_BT_HCIUART_AG6XX is not set
CONFIG_BT_HCIBCM203X=m
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIVHCI=m
CONFIG_BT_MRVL=m
CONFIG_BT_MRVL_SDIO=m
CONFIG_BT_ATH3K=m
# CONFIG_BT_MTKSDIO is not set
# end of Bluetooth device drivers

# CONFIG_AF_RXRPC is not set
# CONFIG_AF_KCM is not set
CONFIG_STREAM_PARSER=y
CONFIG_FIB_RULES=y
CONFIG_WIRELESS=y
CONFIG_CFG80211=m
# CONFIG_NL80211_TESTMODE is not set
# CONFIG_CFG80211_DEVELOPER_WARNINGS is not set
CONFIG_CFG80211_REQUIRE_SIGNED_REGDB=y
CONFIG_CFG80211_USE_KERNEL_REGDB_KEYS=y
CONFIG_CFG80211_DEFAULT_PS=y
# CONFIG_CFG80211_DEBUGFS is not set
CONFIG_CFG80211_CRDA_SUPPORT=y
# CONFIG_CFG80211_WEXT is not set
CONFIG_MAC80211=m
CONFIG_MAC80211_HAS_RC=y
CONFIG_MAC80211_RC_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT="minstrel_ht"
# CONFIG_MAC80211_MESH is not set
CONFIG_MAC80211_LEDS=y
CONFIG_MAC80211_DEBUGFS=y
# CONFIG_MAC80211_MESSAGE_TRACING is not set
# CONFIG_MAC80211_DEBUG_MENU is not set
CONFIG_MAC80211_STA_HASH_MAX_SIZE=0
CONFIG_RFKILL=m
CONFIG_RFKILL_LEDS=y
CONFIG_RFKILL_INPUT=y
# CONFIG_RFKILL_GPIO is not set
# CONFIG_NET_9P is not set
# CONFIG_CAIF is not set
CONFIG_CEPH_LIB=m
# CONFIG_CEPH_LIB_PRETTYDEBUG is not set
CONFIG_CEPH_LIB_USE_DNS_RESOLVER=y
# CONFIG_NFC is not set
CONFIG_PSAMPLE=m
# CONFIG_NET_IFE is not set
CONFIG_LWTUNNEL=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_DST_CACHE=y
CONFIG_GRO_CELLS=y
CONFIG_SOCK_VALIDATE_XMIT=y
CONFIG_NET_SOCK_MSG=y
CONFIG_NET_DEVLINK=y
CONFIG_PAGE_POOL=y
CONFIG_FAILOVER=m
CONFIG_ETHTOOL_NETLINK=y
CONFIG_HAVE_EBPF_JIT=y

#
# Device Drivers
#
CONFIG_HAVE_EISA=y
# CONFIG_EISA is not set
CONFIG_HAVE_PCI=y
CONFIG_PCI=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_PCIEAER=y
CONFIG_PCIEAER_INJECT=m
CONFIG_PCIE_ECRC=y
CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_DEFAULT=y
# CONFIG_PCIEASPM_POWERSAVE is not set
# CONFIG_PCIEASPM_POWER_SUPERSAVE is not set
# CONFIG_PCIEASPM_PERFORMANCE is not set
CONFIG_PCIE_PME=y
CONFIG_PCIE_DPC=y
# CONFIG_PCIE_PTM is not set
# CONFIG_PCIE_BW is not set
CONFIG_PCIE_EDR=y
CONFIG_PCI_MSI=y
CONFIG_PCI_MSI_IRQ_DOMAIN=y
CONFIG_PCI_QUIRKS=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_REALLOC_ENABLE_AUTO is not set
CONFIG_PCI_STUB=y
CONFIG_PCI_PF_STUB=m
# CONFIG_XEN_PCIDEV_FRONTEND is not set
CONFIG_PCI_ATS=y
CONFIG_PCI_LOCKLESS_CONFIG=y
CONFIG_PCI_IOV=y
CONFIG_PCI_PRI=y
CONFIG_PCI_PASID=y
# CONFIG_PCI_P2PDMA is not set
CONFIG_PCI_LABEL=y
CONFIG_PCI_HYPERV=m
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
# CONFIG_HOTPLUG_PCI_CPCI is not set
CONFIG_HOTPLUG_PCI_SHPC=y

#
# PCI controller drivers
#
CONFIG_VMD=y
CONFIG_PCI_HYPERV_INTERFACE=m

#
# DesignWare PCI Core Support
#
# CONFIG_PCIE_DW_PLAT_HOST is not set
# CONFIG_PCI_MESON is not set
# end of DesignWare PCI Core Support

#
# Mobiveil PCIe Core Support
#
# end of Mobiveil PCIe Core Support

#
# Cadence PCIe controllers support
#
# end of Cadence PCIe controllers support
# end of PCI controller drivers

#
# PCI Endpoint
#
# CONFIG_PCI_ENDPOINT is not set
# end of PCI Endpoint

#
# PCI switch controller drivers
#
# CONFIG_PCI_SW_SWITCHTEC is not set
# end of PCI switch controller drivers

CONFIG_PCCARD=y
# CONFIG_PCMCIA is not set
CONFIG_CARDBUS=y

#
# PC-card bridges
#
CONFIG_YENTA=m
CONFIG_YENTA_O2=y
CONFIG_YENTA_RICOH=y
CONFIG_YENTA_TI=y
CONFIG_YENTA_ENE_TUNE=y
CONFIG_YENTA_TOSHIBA=y
# CONFIG_RAPIDIO is not set

#
# Generic Driver Options
#
CONFIG_AUXILIARY_BUS=y
# CONFIG_UEVENT_HELPER is not set
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y

#
# Firmware loader
#
CONFIG_FW_LOADER=y
CONFIG_FW_LOADER_PAGED_BUF=y
CONFIG_EXTRA_FIRMWARE=""
CONFIG_FW_LOADER_USER_HELPER=y
# CONFIG_FW_LOADER_USER_HELPER_FALLBACK is not set
# CONFIG_FW_LOADER_COMPRESS is not set
CONFIG_FW_CACHE=y
# end of Firmware loader

CONFIG_WANT_DEV_COREDUMP=y
CONFIG_ALLOW_DEV_COREDUMP=y
CONFIG_DEV_COREDUMP=y
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
CONFIG_HMEM_REPORTING=y
# CONFIG_TEST_ASYNC_DRIVER_PROBE is not set
CONFIG_SYS_HYPERVISOR=y
CONFIG_GENERIC_CPU_AUTOPROBE=y
CONFIG_GENERIC_CPU_VULNERABILITIES=y
CONFIG_REGMAP=y
CONFIG_REGMAP_I2C=y
CONFIG_REGMAP_SPI=m
CONFIG_REGMAP_SOUNDWIRE=m
CONFIG_DMA_SHARED_BUFFER=y
# CONFIG_DMA_FENCE_TRACE is not set
# end of Generic Driver Options

#
# Bus devices
#
# CONFIG_MHI_BUS is not set
# end of Bus devices

CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y
# CONFIG_GNSS is not set
CONFIG_MTD=m
# CONFIG_MTD_TESTS is not set

#
# Partition parsers
#
# CONFIG_MTD_AR7_PARTS is not set
# CONFIG_MTD_CMDLINE_PARTS is not set
# CONFIG_MTD_REDBOOT_PARTS is not set
# end of Partition parsers

#
# User Modules And Translation Layers
#
CONFIG_MTD_BLKDEVS=m
CONFIG_MTD_BLOCK=m
# CONFIG_MTD_BLOCK_RO is not set
# CONFIG_FTL is not set
# CONFIG_NFTL is not set
# CONFIG_INFTL is not set
# CONFIG_RFD_FTL is not set
# CONFIG_SSFDC is not set
# CONFIG_SM_FTL is not set
# CONFIG_MTD_OOPS is not set
# CONFIG_MTD_SWAP is not set
# CONFIG_MTD_PARTITIONED_MASTER is not set

#
# RAM/ROM/Flash chip drivers
#
# CONFIG_MTD_CFI is not set
# CONFIG_MTD_JEDECPROBE is not set
CONFIG_MTD_MAP_BANK_WIDTH_1=y
CONFIG_MTD_MAP_BANK_WIDTH_2=y
CONFIG_MTD_MAP_BANK_WIDTH_4=y
CONFIG_MTD_CFI_I1=y
CONFIG_MTD_CFI_I2=y
# CONFIG_MTD_RAM is not set
# CONFIG_MTD_ROM is not set
# CONFIG_MTD_ABSENT is not set
# end of RAM/ROM/Flash chip drivers

#
# Mapping drivers for chip access
#
# CONFIG_MTD_COMPLEX_MAPPINGS is not set
# CONFIG_MTD_INTEL_VR_NOR is not set
# CONFIG_MTD_PLATRAM is not set
# end of Mapping drivers for chip access

#
# Self-contained MTD device drivers
#
# CONFIG_MTD_PMC551 is not set
# CONFIG_MTD_DATAFLASH is not set
# CONFIG_MTD_MCHP23K256 is not set
# CONFIG_MTD_SST25L is not set
# CONFIG_MTD_SLRAM is not set
# CONFIG_MTD_PHRAM is not set
# CONFIG_MTD_MTDRAM is not set
# CONFIG_MTD_BLOCK2MTD is not set

#
# Disk-On-Chip Device Drivers
#
# CONFIG_MTD_DOCG3 is not set
# end of Self-contained MTD device drivers

#
# NAND
#
# CONFIG_MTD_ONENAND is not set
# CONFIG_MTD_RAW_NAND is not set
# CONFIG_MTD_SPI_NAND is not set

#
# ECC engine support
#
# CONFIG_MTD_NAND_ECC_SW_HAMMING is not set
# CONFIG_MTD_NAND_ECC_SW_BCH is not set
# end of ECC engine support
# end of NAND

#
# LPDDR & LPDDR2 PCM memory drivers
#
# CONFIG_MTD_LPDDR is not set
# end of LPDDR & LPDDR2 PCM memory drivers

# CONFIG_MTD_SPI_NOR is not set
CONFIG_MTD_UBI=m
CONFIG_MTD_UBI_WL_THRESHOLD=4096
CONFIG_MTD_UBI_BEB_LIMIT=20
# CONFIG_MTD_UBI_FASTMAP is not set
# CONFIG_MTD_UBI_GLUEBI is not set
# CONFIG_MTD_UBI_BLOCK is not set
# CONFIG_MTD_HYPERBUS is not set
# CONFIG_OF is not set
CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_SERIAL=m
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not set
# CONFIG_PARPORT_AX88796 is not set
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
# CONFIG_PNP_DEBUG_MESSAGES is not set

#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_NULL_BLK=m
# CONFIG_BLK_DEV_FD is not set
CONFIG_CDROM=m
# CONFIG_PARIDE is not set
# CONFIG_BLK_DEV_PCIESSD_MTIP32XX is not set
CONFIG_ZRAM=m
CONFIG_ZRAM_DEF_COMP_LZORLE=y
# CONFIG_ZRAM_DEF_COMP_LZO is not set
CONFIG_ZRAM_DEF_COMP="lzo-rle"
CONFIG_ZRAM_WRITEBACK=y
# CONFIG_ZRAM_MEMORY_TRACKING is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_LOOP_MIN_COUNT=0
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_DRBD is not set
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_RAM=m
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=16384
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
# CONFIG_ATA_OVER_ETH is not set
CONFIG_XEN_BLKDEV_FRONTEND=m
CONFIG_VIRTIO_BLK=m
CONFIG_BLK_DEV_RBD=m
# CONFIG_BLK_DEV_RSXX is not set

#
# NVME Support
#
CONFIG_NVME_CORE=m
CONFIG_BLK_DEV_NVME=m
CONFIG_NVME_MULTIPATH=y
# CONFIG_NVME_HWMON is not set
CONFIG_NVME_FABRICS=m
CONFIG_NVME_RDMA=m
CONFIG_NVME_FC=m
CONFIG_NVME_TCP=m
CONFIG_NVME_TARGET=m
# CONFIG_NVME_TARGET_PASSTHRU is not set
CONFIG_NVME_TARGET_LOOP=m
CONFIG_NVME_TARGET_RDMA=m
CONFIG_NVME_TARGET_FC=m
CONFIG_NVME_TARGET_FCLOOP=m
CONFIG_NVME_TARGET_TCP=m
# end of NVME Support

#
# Misc devices
#
CONFIG_SENSORS_LIS3LV02D=m
# CONFIG_AD525X_DPOT is not set
# CONFIG_DUMMY_IRQ is not set
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
CONFIG_TIFM_CORE=m
CONFIG_TIFM_7XX1=m
# CONFIG_ICS932S401 is not set
CONFIG_ENCLOSURE_SERVICES=m
CONFIG_SGI_XP=m
CONFIG_HP_ILO=m
CONFIG_SGI_GRU=m
# CONFIG_SGI_GRU_DEBUG is not set
CONFIG_APDS9802ALS=m
CONFIG_ISL29003=m
CONFIG_ISL29020=m
CONFIG_SENSORS_TSL2550=m
CONFIG_SENSORS_BH1770=m
CONFIG_SENSORS_APDS990X=m
# CONFIG_HMC6352 is not set
# CONFIG_DS1682 is not set
CONFIG_VMWARE_BALLOON=m
# CONFIG_LATTICE_ECP3_CONFIG is not set
# CONFIG_SRAM is not set
# CONFIG_PCI_ENDPOINT_TEST is not set
# CONFIG_XILINX_SDFEC is not set
CONFIG_MISC_RTSX=m
CONFIG_PVPANIC=y
# CONFIG_C2PORT is not set

#
# EEPROM support
#
# CONFIG_EEPROM_AT24 is not set
# CONFIG_EEPROM_AT25 is not set
CONFIG_EEPROM_LEGACY=m
CONFIG_EEPROM_MAX6875=m
CONFIG_EEPROM_93CX6=m
# CONFIG_EEPROM_93XX46 is not set
# CONFIG_EEPROM_IDT_89HPESX is not set
# CONFIG_EEPROM_EE1004 is not set
# end of EEPROM support

CONFIG_CB710_CORE=m
# CONFIG_CB710_DEBUG is not set
CONFIG_CB710_DEBUG_ASSUMPTIONS=y

#
# Texas Instruments shared transport line discipline
#
# CONFIG_TI_ST is not set
# end of Texas Instruments shared transport line discipline

CONFIG_SENSORS_LIS3_I2C=m
CONFIG_ALTERA_STAPL=m
CONFIG_INTEL_MEI=m
CONFIG_INTEL_MEI_ME=m
# CONFIG_INTEL_MEI_TXE is not set
# CONFIG_INTEL_MEI_HDCP is not set
CONFIG_VMWARE_VMCI=m
# CONFIG_GENWQE is not set
# CONFIG_ECHO is not set
# CONFIG_MISC_ALCOR_PCI is not set
CONFIG_MISC_RTSX_PCI=m
CONFIG_MISC_RTSX_USB=m
# CONFIG_HABANA_AI is not set
# CONFIG_UACCE is not set
# end of Misc devices

CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set

#
# SCSI device support
#
CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
CONFIG_CHR_DEV_ST=m
CONFIG_BLK_DEV_SR=m
CONFIG_CHR_DEV_SG=m
CONFIG_CHR_DEV_SCH=m
CONFIG_SCSI_ENCLOSURE=m
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y

#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
CONFIG_SCSI_ISCSI_ATTRS=m
CONFIG_SCSI_SAS_ATTRS=m
CONFIG_SCSI_SAS_LIBSAS=m
CONFIG_SCSI_SAS_ATA=y
CONFIG_SCSI_SAS_HOST_SMP=y
CONFIG_SCSI_SRP_ATTRS=m
# end of SCSI Transports

CONFIG_SCSI_LOWLEVEL=y
CONFIG_ISCSI_TCP=m
CONFIG_ISCSI_BOOT_SYSFS=m
# CONFIG_SCSI_CXGB3_ISCSI is not set
CONFIG_SCSI_CXGB4_ISCSI=m
CONFIG_SCSI_BNX2_ISCSI=m
CONFIG_SCSI_BNX2X_FCOE=m
CONFIG_BE2ISCSI=m
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
CONFIG_SCSI_HPSA=m
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_3W_SAS is not set
# CONFIG_SCSI_ACARD is not set
CONFIG_SCSI_AACRAID=m
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_AIC94XX is not set
# CONFIG_SCSI_MVSAS is not set
# CONFIG_SCSI_MVUMI is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_ARCMSR is not set
# CONFIG_SCSI_ESAS2R is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
CONFIG_MEGARAID_SAS=m
CONFIG_SCSI_MPT3SAS=m
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT3SAS_MAX_SGE=128
CONFIG_SCSI_MPT2SAS=m
CONFIG_SCSI_SMARTPQI=m
# CONFIG_SCSI_UFSHCD is not set
# CONFIG_SCSI_HPTIOP is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_MYRB is not set
# CONFIG_SCSI_MYRS is not set
CONFIG_VMWARE_PVSCSI=m
# CONFIG_XEN_SCSI_FRONTEND is not set
CONFIG_HYPERV_STORAGE=m
CONFIG_LIBFC=m
CONFIG_LIBFCOE=m
# CONFIG_FCOE is not set
CONFIG_FCOE_FNIC=m
# CONFIG_SCSI_SNIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_FDOMAIN_PCI is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_ISCI is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_IMM is not set
# CONFIG_SCSI_STEX is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
CONFIG_SCSI_QLA_FC=m
# CONFIG_TCM_QLA2XXX is not set
CONFIG_SCSI_QLA_ISCSI=m
CONFIG_QEDI=m
CONFIG_QEDF=m
CONFIG_SCSI_LPFC=m
# CONFIG_SCSI_LPFC_DEBUG_FS is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_AM53C974 is not set
# CONFIG_SCSI_WD719X is not set
CONFIG_SCSI_DEBUG=m
# CONFIG_SCSI_PMCRAID is not set
# CONFIG_SCSI_PM8001 is not set
# CONFIG_SCSI_BFA_FC is not set
CONFIG_SCSI_VIRTIO=m
CONFIG_SCSI_CHELSIO_FCOE=m
CONFIG_SCSI_DH=y
CONFIG_SCSI_DH_RDAC=y
CONFIG_SCSI_DH_HP_SW=y
CONFIG_SCSI_DH_EMC=y
CONFIG_SCSI_DH_ALUA=y
# end of SCSI device support

CONFIG_ATA=m
CONFIG_SATA_HOST=y
CONFIG_PATA_TIMINGS=y
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_FORCE=y
CONFIG_ATA_ACPI=y
# CONFIG_SATA_ZPODD is not set
CONFIG_SATA_PMP=y

#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=m
CONFIG_SATA_MOBILE_LPM_POLICY=0
CONFIG_SATA_AHCI_PLATFORM=m
# CONFIG_SATA_INIC162X is not set
# CONFIG_SATA_ACARD_AHCI is not set
# CONFIG_SATA_SIL24 is not set
CONFIG_ATA_SFF=y

#
# SFF controllers with custom DMA interface
#
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_SX4 is not set
CONFIG_ATA_BMDMA=y

#
# SATA SFF controllers with BMDMA
#
CONFIG_ATA_PIIX=m
# CONFIG_SATA_DWC is not set
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_SVW is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set

#
# PATA SFF controllers with BMDMA
#
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
# CONFIG_PATA_ATIIXP is not set
# CONFIG_PATA_ATP867X is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RDC is not set
# CONFIG_PATA_SCH is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_TOSHIBA is not set
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set

#
# PIO-only SFF controllers
#
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_RZ1000 is not set

#
# Generic fallback / legacy drivers
#
# CONFIG_PATA_ACPI is not set
CONFIG_ATA_GENERIC=m
# CONFIG_PATA_LEGACY is not set
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=m
# CONFIG_MD_MULTIPATH is not set
CONFIG_MD_FAULTY=m
CONFIG_MD_CLUSTER=m
# CONFIG_BCACHE is not set
CONFIG_BLK_DEV_DM_BUILTIN=y
CONFIG_BLK_DEV_DM=m
CONFIG_DM_DEBUG=y
CONFIG_DM_BUFIO=m
# CONFIG_DM_DEBUG_BLOCK_MANAGER_LOCKING is not set
CONFIG_DM_BIO_PRISON=m
CONFIG_DM_PERSISTENT_DATA=m
# CONFIG_DM_UNSTRIPED is not set
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_THIN_PROVISIONING=m
CONFIG_DM_CACHE=m
CONFIG_DM_CACHE_SMQ=m
CONFIG_DM_WRITECACHE=m
# CONFIG_DM_EBS is not set
CONFIG_DM_ERA=m
# CONFIG_DM_CLONE is not set
CONFIG_DM_MIRROR=m
CONFIG_DM_LOG_USERSPACE=m
CONFIG_DM_RAID=m
CONFIG_DM_ZERO=m
CONFIG_DM_MULTIPATH=m
CONFIG_DM_MULTIPATH_QL=m
CONFIG_DM_MULTIPATH_ST=m
# CONFIG_DM_MULTIPATH_HST is not set
# CONFIG_DM_MULTIPATH_IOA is not set
CONFIG_DM_DELAY=m
# CONFIG_DM_DUST is not set
CONFIG_DM_UEVENT=y
CONFIG_DM_FLAKEY=m
CONFIG_DM_VERITY=m
# CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG is not set
# CONFIG_DM_VERITY_FEC is not set
CONFIG_DM_SWITCH=m
CONFIG_DM_LOG_WRITES=m
CONFIG_DM_INTEGRITY=m
CONFIG_TARGET_CORE=m
CONFIG_TCM_IBLOCK=m
CONFIG_TCM_FILEIO=m
CONFIG_TCM_PSCSI=m
CONFIG_TCM_USER2=m
CONFIG_LOOPBACK_TARGET=m
# CONFIG_TCM_FC is not set
CONFIG_ISCSI_TARGET=m
CONFIG_ISCSI_TARGET_CXGB4=m
# CONFIG_SBP_TARGET is not set
CONFIG_FUSION=y
CONFIG_FUSION_SPI=m
# CONFIG_FUSION_FC is not set
CONFIG_FUSION_SAS=m
CONFIG_FUSION_MAX_SGE=128
# CONFIG_FUSION_CTL is not set
CONFIG_FUSION_LOGGING=y

#
# IEEE 1394 (FireWire) support
#
CONFIG_FIREWIRE=m
CONFIG_FIREWIRE_OHCI=m
CONFIG_FIREWIRE_SBP2=m
CONFIG_FIREWIRE_NET=m
# CONFIG_FIREWIRE_NOSY is not set
# end of IEEE 1394 (FireWire) support

CONFIG_MACINTOSH_DRIVERS=y
CONFIG_MAC_EMUMOUSEBTN=y
CONFIG_NETDEVICES=y
CONFIG_MII=m
CONFIG_NET_CORE=y
CONFIG_BONDING=m
CONFIG_DUMMY=m
# CONFIG_WIREGUARD is not set
# CONFIG_EQUALIZER is not set
CONFIG_NET_FC=y
CONFIG_IFB=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_MACVLAN=m
CONFIG_MACVTAP=m
CONFIG_IPVLAN_L3S=y
CONFIG_IPVLAN=m
CONFIG_IPVTAP=m
CONFIG_VXLAN=m
CONFIG_GENEVE=m
CONFIG_BAREUDP=m
# CONFIG_GTP is not set
CONFIG_MACSEC=m
CONFIG_NETCONSOLE=m
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_NETPOLL=y
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_NTB_NETDEV is not set
CONFIG_TUN=m
CONFIG_TAP=m
# CONFIG_TUN_VNET_CROSS_LE is not set
CONFIG_VETH=m
CONFIG_VIRTIO_NET=m
CONFIG_NLMON=m
CONFIG_NET_VRF=m
CONFIG_VSOCKMON=m
# CONFIG_ARCNET is not set
# CONFIG_ATM_DRIVERS is not set

#
# Distributed Switch Architecture drivers
#
# end of Distributed Switch Architecture drivers

CONFIG_ETHERNET=y
CONFIG_MDIO=m
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_NET_VENDOR_ADAPTEC is not set
# CONFIG_NET_VENDOR_AGERE is not set
# CONFIG_NET_VENDOR_ALACRITECH is not set
# CONFIG_NET_VENDOR_ALTEON is not set
# CONFIG_ALTERA_TSE is not set
CONFIG_NET_VENDOR_AMAZON=y
CONFIG_ENA_ETHERNET=m
CONFIG_NET_VENDOR_AMD=y
# CONFIG_AMD8111_ETH is not set
# CONFIG_PCNET32 is not set
CONFIG_AMD_XGBE=m
# CONFIG_AMD_XGBE_DCB is not set
CONFIG_AMD_XGBE_HAVE_ECC=y
CONFIG_NET_VENDOR_AQUANTIA=y
CONFIG_AQTION=m
# CONFIG_NET_VENDOR_ARC is not set
CONFIG_NET_VENDOR_ATHEROS=y
CONFIG_ATL2=m
CONFIG_ATL1=m
CONFIG_ATL1E=m
CONFIG_ATL1C=m
CONFIG_ALX=m
# CONFIG_NET_VENDOR_AURORA is not set
CONFIG_NET_VENDOR_BROADCOM=y
# CONFIG_B44 is not set
# CONFIG_BCMGENET is not set
CONFIG_BNX2=m
CONFIG_CNIC=m
CONFIG_TIGON3=m
CONFIG_TIGON3_HWMON=y
CONFIG_BNX2X=m
CONFIG_BNX2X_SRIOV=y
# CONFIG_SYSTEMPORT is not set
CONFIG_BNXT=m
CONFIG_BNXT_SRIOV=y
CONFIG_BNXT_FLOWER_OFFLOAD=y
CONFIG_BNXT_DCB=y
CONFIG_BNXT_HWMON=y
CONFIG_NET_VENDOR_BROCADE=y
# CONFIG_BNA is not set
CONFIG_NET_VENDOR_CADENCE=y
# CONFIG_MACB is not set
CONFIG_NET_VENDOR_CAVIUM=y
# CONFIG_THUNDER_NIC_PF is not set
# CONFIG_THUNDER_NIC_VF is not set
# CONFIG_THUNDER_NIC_BGX is not set
# CONFIG_THUNDER_NIC_RGX is not set
CONFIG_CAVIUM_PTP=y
CONFIG_LIQUIDIO=m
CONFIG_LIQUIDIO_VF=m
CONFIG_NET_VENDOR_CHELSIO=y
# CONFIG_CHELSIO_T1 is not set
# CONFIG_CHELSIO_T3 is not set
CONFIG_CHELSIO_T4=m
# CONFIG_CHELSIO_T4_DCB is not set
CONFIG_CHELSIO_T4VF=m
CONFIG_CHELSIO_LIB=m
CONFIG_CHELSIO_INLINE_CRYPTO=y
CONFIG_CHELSIO_IPSEC_INLINE=m
CONFIG_CHELSIO_TLS_DEVICE=m
CONFIG_NET_VENDOR_CISCO=y
CONFIG_ENIC=m
# CONFIG_NET_VENDOR_CORTINA is not set
# CONFIG_CX_ECAT is not set
CONFIG_DNET=m
CONFIG_NET_VENDOR_DEC=y
# CONFIG_NET_TULIP is not set
CONFIG_NET_VENDOR_DLINK=y
CONFIG_DL2K=m
# CONFIG_SUNDANCE is not set
CONFIG_NET_VENDOR_EMULEX=y
CONFIG_BE2NET=m
CONFIG_BE2NET_HWMON=y
# CONFIG_BE2NET_BE2 is not set
# CONFIG_BE2NET_BE3 is not set
CONFIG_BE2NET_LANCER=y
CONFIG_BE2NET_SKYHAWK=y
# CONFIG_NET_VENDOR_EZCHIP is not set
CONFIG_NET_VENDOR_GOOGLE=y
CONFIG_GVE=m
CONFIG_NET_VENDOR_HUAWEI=y
CONFIG_HINIC=m
# CONFIG_NET_VENDOR_I825XX is not set
CONFIG_NET_VENDOR_INTEL=y
# CONFIG_E100 is not set
CONFIG_E1000=m
CONFIG_E1000E=m
CONFIG_E1000E_HWTS=y
CONFIG_IGB=m
CONFIG_IGB_HWMON=y
CONFIG_IGB_DCA=y
CONFIG_IGBVF=m
# CONFIG_IXGB is not set
CONFIG_IXGBE=m
CONFIG_IXGBE_HWMON=y
CONFIG_IXGBE_DCA=y
CONFIG_IXGBE_DCB=y
CONFIG_IXGBE_IPSEC=y
CONFIG_IXGBEVF=m
CONFIG_IXGBEVF_IPSEC=y
CONFIG_I40E=m
CONFIG_I40E_DCB=y
CONFIG_IAVF=m
CONFIG_I40EVF=m
CONFIG_ICE=m
CONFIG_FM10K=m
CONFIG_IGC=m
# CONFIG_JME is not set
# CONFIG_NET_VENDOR_MARVELL is not set
CONFIG_NET_VENDOR_MELLANOX=y
CONFIG_MLX4_EN=m
CONFIG_MLX4_EN_DCB=y
CONFIG_MLX4_CORE=m
CONFIG_MLX4_DEBUG=y
# CONFIG_MLX4_CORE_GEN2 is not set
CONFIG_MLX5_CORE=m
CONFIG_MLX5_ACCEL=y
CONFIG_MLX5_FPGA=y
CONFIG_MLX5_CORE_EN=y
CONFIG_MLX5_EN_ARFS=y
CONFIG_MLX5_EN_RXNFC=y
CONFIG_MLX5_MPFS=y
CONFIG_MLX5_ESWITCH=y
CONFIG_MLX5_CLS_ACT=y
CONFIG_MLX5_TC_CT=y
CONFIG_MLX5_CORE_EN_DCB=y
CONFIG_MLX5_CORE_IPOIB=y
CONFIG_MLX5_FPGA_IPSEC=y
CONFIG_MLX5_IPSEC=y
CONFIG_MLX5_EN_IPSEC=y
# CONFIG_MLX5_FPGA_TLS is not set
CONFIG_MLX5_TLS=y
CONFIG_MLX5_EN_TLS=y
CONFIG_MLX5_SW_STEERING=y
CONFIG_MLXSW_CORE=m
CONFIG_MLXSW_CORE_HWMON=y
CONFIG_MLXSW_CORE_THERMAL=y
CONFIG_MLXSW_PCI=m
CONFIG_MLXSW_I2C=m
CONFIG_MLXSW_SWITCHIB=m
CONFIG_MLXSW_SWITCHX2=m
CONFIG_MLXSW_SPECTRUM=m
CONFIG_MLXSW_SPECTRUM_DCB=y
CONFIG_MLXSW_MINIMAL=m
CONFIG_MLXFW=m
# CONFIG_NET_VENDOR_MICREL is not set
# CONFIG_NET_VENDOR_MICROCHIP is not set
# CONFIG_NET_VENDOR_MICROSEMI is not set
CONFIG_NET_VENDOR_MYRI=y
CONFIG_MYRI10GE=m
CONFIG_MYRI10GE_DCA=y
# CONFIG_FEALNX is not set
# CONFIG_NET_VENDOR_NATSEMI is not set
CONFIG_NET_VENDOR_NETERION=y
# CONFIG_S2IO is not set
# CONFIG_VXGE is not set
CONFIG_NET_VENDOR_NETRONOME=y
CONFIG_NFP=m
CONFIG_NFP_APP_FLOWER=y
CONFIG_NFP_APP_ABM_NIC=y
# CONFIG_NFP_DEBUG is not set
# CONFIG_NET_VENDOR_NI is not set
# CONFIG_NET_VENDOR_NVIDIA is not set
CONFIG_NET_VENDOR_OKI=y
CONFIG_ETHOC=m
CONFIG_NET_VENDOR_PACKET_ENGINES=y
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
CONFIG_NET_VENDOR_PENSANDO=y
CONFIG_IONIC=m
CONFIG_NET_VENDOR_QLOGIC=y
CONFIG_QLA3XXX=m
# CONFIG_QLCNIC is not set
CONFIG_NETXEN_NIC=m
CONFIG_QED=m
CONFIG_QED_LL2=y
CONFIG_QED_SRIOV=y
CONFIG_QEDE=m
CONFIG_QED_RDMA=y
CONFIG_QED_ISCSI=y
CONFIG_QED_FCOE=y
CONFIG_QED_OOO=y
# CONFIG_NET_VENDOR_QUALCOMM is not set
# CONFIG_NET_VENDOR_RDC is not set
CONFIG_NET_VENDOR_REALTEK=y
# CONFIG_ATP is not set
CONFIG_8139CP=m
CONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
# CONFIG_8139TOO_TUNE_TWISTER is not set
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_R8169=m
# CONFIG_NET_VENDOR_RENESAS is not set
CONFIG_NET_VENDOR_ROCKER=y
CONFIG_ROCKER=m
# CONFIG_NET_VENDOR_SAMSUNG is not set
# CONFIG_NET_VENDOR_SEEQ is not set
CONFIG_NET_VENDOR_SOLARFLARE=y
CONFIG_SFC=m
CONFIG_SFC_MTD=y
CONFIG_SFC_MCDI_MON=y
CONFIG_SFC_SRIOV=y
CONFIG_SFC_MCDI_LOGGING=y
# CONFIG_SFC_FALCON is not set
# CONFIG_NET_VENDOR_SILAN is not set
# CONFIG_NET_VENDOR_SIS is not set
# CONFIG_NET_VENDOR_SMSC is not set
# CONFIG_NET_VENDOR_SOCIONEXT is not set
# CONFIG_NET_VENDOR_STMICRO is not set
# CONFIG_NET_VENDOR_SUN is not set
# CONFIG_NET_VENDOR_SYNOPSYS is not set
# CONFIG_NET_VENDOR_TEHUTI is not set
# CONFIG_NET_VENDOR_TI is not set
# CONFIG_NET_VENDOR_VIA is not set
# CONFIG_NET_VENDOR_WIZNET is not set
CONFIG_NET_VENDOR_XILINX=y
# CONFIG_XILINX_AXI_EMAC is not set
# CONFIG_XILINX_LL_TEMAC is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_NET_SB1000 is not set
CONFIG_PHYLIB=y
CONFIG_SWPHY=y
CONFIG_LED_TRIGGER_PHY=y
CONFIG_FIXED_PHY=y

#
# MII PHY device drivers
#
CONFIG_AMD_PHY=m
# CONFIG_ADIN_PHY is not set
CONFIG_AQUANTIA_PHY=m
# CONFIG_AX88796B_PHY is not set
CONFIG_BROADCOM_PHY=m
# CONFIG_BCM54140_PHY is not set
CONFIG_BCM7XXX_PHY=m
# CONFIG_BCM84881_PHY is not set
CONFIG_BCM87XX_PHY=m
CONFIG_BCM_NET_PHYLIB=m
CONFIG_CICADA_PHY=m
CONFIG_CORTINA_PHY=m
CONFIG_DAVICOM_PHY=m
CONFIG_ICPLUS_PHY=m
CONFIG_LXT_PHY=m
CONFIG_INTEL_XWAY_PHY=m
CONFIG_LSI_ET1011C_PHY=m
CONFIG_MARVELL_PHY=m
CONFIG_MARVELL_10G_PHY=m
CONFIG_MICREL_PHY=m
CONFIG_MICROCHIP_PHY=m
CONFIG_MICROCHIP_T1_PHY=m
CONFIG_MICROSEMI_PHY=m
CONFIG_NATIONAL_PHY=m
# CONFIG_NXP_TJA11XX_PHY is not set
CONFIG_QSEMI_PHY=m
CONFIG_REALTEK_PHY=m
CONFIG_RENESAS_PHY=m
CONFIG_ROCKCHIP_PHY=m
CONFIG_SMSC_PHY=m
CONFIG_STE10XP=m
CONFIG_TERANETICS_PHY=m
CONFIG_DP83822_PHY=m
CONFIG_DP83TC811_PHY=m
CONFIG_DP83848_PHY=m
CONFIG_DP83867_PHY=m
# CONFIG_DP83869_PHY is not set
CONFIG_VITESSE_PHY=m
CONFIG_XILINX_GMII2RGMII=m
CONFIG_MICREL_KS8995MA=m
CONFIG_MDIO_DEVICE=y
CONFIG_MDIO_BUS=y
CONFIG_MDIO_DEVRES=y
CONFIG_MDIO_BITBANG=m
CONFIG_MDIO_BCM_UNIMAC=m
CONFIG_MDIO_CAVIUM=m
# CONFIG_MDIO_GPIO is not set
# CONFIG_MDIO_MVUSB is not set
CONFIG_MDIO_MSCC_MIIM=m
CONFIG_MDIO_THUNDER=m

#
# MDIO Multiplexers
#

#
# PCS device drivers
#
# CONFIG_PCS_XPCS is not set
# end of PCS device drivers

# CONFIG_PLIP is not set
CONFIG_PPP=m
CONFIG_PPP_BSDCOMP=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_FILTER=y
CONFIG_PPP_MPPE=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPPOATM=m
CONFIG_PPPOE=m
CONFIG_PPTP=m
CONFIG_PPPOL2TP=m
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_SLIP=m
CONFIG_SLHC=m
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLIP_SMART=y
# CONFIG_SLIP_MODE_SLIP6 is not set
CONFIG_USB_NET_DRIVERS=y
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_RTL8152=m
CONFIG_USB_LAN78XX=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_AX88179_178A=m
CONFIG_USB_NET_CDCETHER=m
CONFIG_USB_NET_CDC_EEM=m
CONFIG_USB_NET_CDC_NCM=m
CONFIG_USB_NET_HUAWEI_CDC_NCM=m
CONFIG_USB_NET_CDC_MBIM=m
CONFIG_USB_NET_DM9601=m
# CONFIG_USB_NET_SR9700 is not set
# CONFIG_USB_NET_SR9800 is not set
CONFIG_USB_NET_SMSC75XX=m
CONFIG_USB_NET_SMSC95XX=m
CONFIG_USB_NET_GL620A=m
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
CONFIG_USB_NET_MCS7830=m
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET_ENABLE=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_KC2190=y
CONFIG_USB_NET_ZAURUS=m
CONFIG_USB_NET_CX82310_ETH=m
CONFIG_USB_NET_KALMIA=m
CONFIG_USB_NET_QMI_WWAN=m
CONFIG_USB_HSO=m
CONFIG_USB_NET_INT51X1=m
CONFIG_USB_IPHETH=m
CONFIG_USB_SIERRA_NET=m
CONFIG_USB_VL600=m
CONFIG_USB_NET_CH9200=m
# CONFIG_USB_NET_AQC111 is not set
# CONFIG_USB_RTL8153_ECM is not set
CONFIG_WLAN=y
# CONFIG_WLAN_VENDOR_ADMTEK is not set
CONFIG_ATH_COMMON=m
CONFIG_WLAN_VENDOR_ATH=y
# CONFIG_ATH_DEBUG is not set
# CONFIG_ATH5K is not set
# CONFIG_ATH5K_PCI is not set
CONFIG_ATH9K_HW=m
CONFIG_ATH9K_COMMON=m
CONFIG_ATH9K_COMMON_DEBUG=y
CONFIG_ATH9K_BTCOEX_SUPPORT=y
CONFIG_ATH9K=m
CONFIG_ATH9K_PCI=y
CONFIG_ATH9K_AHB=y
CONFIG_ATH9K_DEBUGFS=y
# CONFIG_ATH9K_STATION_STATISTICS is not set
# CONFIG_ATH9K_DYNACK is not set
CONFIG_ATH9K_WOW=y
CONFIG_ATH9K_RFKILL=y
# CONFIG_ATH9K_CHANNEL_CONTEXT is not set
CONFIG_ATH9K_PCOEM=y
# CONFIG_ATH9K_PCI_NO_EEPROM is not set
CONFIG_ATH9K_HTC=m
# CONFIG_ATH9K_HTC_DEBUGFS is not set
# CONFIG_ATH9K_HWRNG is not set
# CONFIG_ATH9K_COMMON_SPECTRAL is not set
# CONFIG_CARL9170 is not set
# CONFIG_ATH6KL is not set
# CONFIG_AR5523 is not set
# CONFIG_WIL6210 is not set
CONFIG_ATH10K=m
CONFIG_ATH10K_CE=y
CONFIG_ATH10K_PCI=m
# CONFIG_ATH10K_SDIO is not set
# CONFIG_ATH10K_USB is not set
# CONFIG_ATH10K_DEBUG is not set
CONFIG_ATH10K_DEBUGFS=y
# CONFIG_ATH10K_SPECTRAL is not set
# CONFIG_ATH10K_TRACING is not set
# CONFIG_WCN36XX is not set
# CONFIG_ATH11K is not set
# CONFIG_WLAN_VENDOR_ATMEL is not set
CONFIG_WLAN_VENDOR_BROADCOM=y
# CONFIG_B43 is not set
# CONFIG_B43LEGACY is not set
CONFIG_BRCMUTIL=m
CONFIG_BRCMSMAC=m
CONFIG_BRCMFMAC=m
CONFIG_BRCMFMAC_PROTO_BCDC=y
CONFIG_BRCMFMAC_PROTO_MSGBUF=y
CONFIG_BRCMFMAC_SDIO=y
CONFIG_BRCMFMAC_USB=y
CONFIG_BRCMFMAC_PCIE=y
# CONFIG_BRCM_TRACING is not set
# CONFIG_BRCMDBG is not set
# CONFIG_WLAN_VENDOR_CISCO is not set
CONFIG_WLAN_VENDOR_INTEL=y
# CONFIG_IPW2100 is not set
# CONFIG_IPW2200 is not set
# CONFIG_IWL4965 is not set
# CONFIG_IWL3945 is not set
CONFIG_IWLWIFI=m
CONFIG_IWLWIFI_LEDS=y
CONFIG_IWLDVM=m
CONFIG_IWLMVM=m
CONFIG_IWLWIFI_OPMODE_MODULAR=y
# CONFIG_IWLWIFI_BCAST_FILTERING is not set

#
# Debugging Options
#
# CONFIG_IWLWIFI_DEBUG is not set
CONFIG_IWLWIFI_DEBUGFS=y
# CONFIG_IWLWIFI_DEVICE_TRACING is not set
# end of Debugging Options

# CONFIG_WLAN_VENDOR_INTERSIL is not set
CONFIG_WLAN_VENDOR_MARVELL=y
# CONFIG_LIBERTAS is not set
# CONFIG_LIBERTAS_THINFIRM is not set
CONFIG_MWIFIEX=m
CONFIG_MWIFIEX_SDIO=m
CONFIG_MWIFIEX_PCIE=m
CONFIG_MWIFIEX_USB=m
# CONFIG_MWL8K is not set
CONFIG_WLAN_VENDOR_MEDIATEK=y
CONFIG_MT7601U=m
CONFIG_MT76_CORE=m
CONFIG_MT76_LEDS=y
CONFIG_MT76_USB=m
CONFIG_MT76x02_LIB=m
CONFIG_MT76x02_USB=m
CONFIG_MT76x0_COMMON=m
CONFIG_MT76x0U=m
# CONFIG_MT76x0E is not set
CONFIG_MT76x2_COMMON=m
# CONFIG_MT76x2E is not set
CONFIG_MT76x2U=m
# CONFIG_MT7603E is not set
# CONFIG_MT7615E is not set
# CONFIG_MT7663U is not set
# CONFIG_MT7663S is not set
# CONFIG_MT7915E is not set
CONFIG_WLAN_VENDOR_MICROCHIP=y
# CONFIG_WILC1000_SDIO is not set
# CONFIG_WILC1000_SPI is not set
CONFIG_WLAN_VENDOR_RALINK=y
CONFIG_RT2X00=m
# CONFIG_RT2400PCI is not set
# CONFIG_RT2500PCI is not set
# CONFIG_RT61PCI is not set
CONFIG_RT2800PCI=m
CONFIG_RT2800PCI_RT33XX=y
CONFIG_RT2800PCI_RT35XX=y
CONFIG_RT2800PCI_RT53XX=y
CONFIG_RT2800PCI_RT3290=y
# CONFIG_RT2500USB is not set
# CONFIG_RT73USB is not set
CONFIG_RT2800USB=m
CONFIG_RT2800USB_RT33XX=y
CONFIG_RT2800USB_RT35XX=y
CONFIG_RT2800USB_RT3573=y
CONFIG_RT2800USB_RT53XX=y
CONFIG_RT2800USB_RT55XX=y
CONFIG_RT2800USB_UNKNOWN=y
CONFIG_RT2800_LIB=m
CONFIG_RT2800_LIB_MMIO=m
CONFIG_RT2X00_LIB_MMIO=m
CONFIG_RT2X00_LIB_PCI=m
CONFIG_RT2X00_LIB_USB=m
CONFIG_RT2X00_LIB=m
CONFIG_RT2X00_LIB_FIRMWARE=y
CONFIG_RT2X00_LIB_CRYPTO=y
CONFIG_RT2X00_LIB_LEDS=y
CONFIG_RT2X00_LIB_DEBUGFS=y
# CONFIG_RT2X00_DEBUG is not set
CONFIG_WLAN_VENDOR_REALTEK=y
# CONFIG_RTL8180 is not set
# CONFIG_RTL8187 is not set
CONFIG_RTL_CARDS=m
CONFIG_RTL8192CE=m
CONFIG_RTL8192SE=m
CONFIG_RTL8192DE=m
CONFIG_RTL8723AE=m
CONFIG_RTL8723BE=m
CONFIG_RTL8188EE=m
CONFIG_RTL8192EE=m
CONFIG_RTL8821AE=m
CONFIG_RTL8192CU=m
CONFIG_RTLWIFI=m
CONFIG_RTLWIFI_PCI=m
CONFIG_RTLWIFI_USB=m
# CONFIG_RTLWIFI_DEBUG is not set
CONFIG_RTL8192C_COMMON=m
CONFIG_RTL8723_COMMON=m
CONFIG_RTLBTCOEXIST=m
CONFIG_RTL8XXXU=m
# CONFIG_RTL8XXXU_UNTESTED is not set
CONFIG_RTW88=m
CONFIG_RTW88_CORE=m
CONFIG_RTW88_PCI=m
CONFIG_RTW88_8822B=m
CONFIG_RTW88_8822C=m
CONFIG_RTW88_8822BE=m
CONFIG_RTW88_8822CE=m
# CONFIG_RTW88_8723DE is not set
# CONFIG_RTW88_8821CE is not set
# CONFIG_RTW88_DEBUG is not set
# CONFIG_RTW88_DEBUGFS is not set
# CONFIG_WLAN_VENDOR_RSI is not set
# CONFIG_WLAN_VENDOR_ST is not set
# CONFIG_WLAN_VENDOR_TI is not set
# CONFIG_WLAN_VENDOR_ZYDAS is not set
CONFIG_WLAN_VENDOR_QUANTENNA=y
# CONFIG_QTNFMAC_PCIE is not set
CONFIG_MAC80211_HWSIM=m
# CONFIG_USB_NET_RNDIS_WLAN is not set
# CONFIG_VIRT_WIFI is not set
CONFIG_WAN=y
# CONFIG_LANMEDIA is not set
CONFIG_HDLC=m
CONFIG_HDLC_RAW=m
# CONFIG_HDLC_RAW_ETH is not set
CONFIG_HDLC_CISCO=m
CONFIG_HDLC_FR=m
CONFIG_HDLC_PPP=m

#
# X.25/LAPB support is disabled
#
# CONFIG_PCI200SYN is not set
# CONFIG_WANXL is not set
# CONFIG_PC300TOO is not set
# CONFIG_FARSYNC is not set
# CONFIG_SBNI is not set
CONFIG_IEEE802154_DRIVERS=m
CONFIG_IEEE802154_FAKELB=m
# CONFIG_IEEE802154_AT86RF230 is not set
# CONFIG_IEEE802154_MRF24J40 is not set
# CONFIG_IEEE802154_CC2520 is not set
# CONFIG_IEEE802154_ATUSB is not set
# CONFIG_IEEE802154_ADF7242 is not set
# CONFIG_IEEE802154_CA8210 is not set
# CONFIG_IEEE802154_MCR20A is not set
# CONFIG_IEEE802154_HWSIM is not set
CONFIG_XEN_NETDEV_FRONTEND=m
CONFIG_VMXNET3=m
CONFIG_FUJITSU_ES=m
CONFIG_USB4_NET=m
CONFIG_HYPERV_NET=m
CONFIG_NETDEVSIM=m
CONFIG_NET_FAILOVER=m
CONFIG_ISDN=y
CONFIG_ISDN_CAPI=y
CONFIG_CAPI_TRACE=y
CONFIG_ISDN_CAPI_MIDDLEWARE=y
CONFIG_MISDN=m
CONFIG_MISDN_DSP=m
CONFIG_MISDN_L1OIP=m

#
# mISDN hardware drivers
#
CONFIG_MISDN_HFCPCI=m
CONFIG_MISDN_HFCMULTI=m
CONFIG_MISDN_HFCUSB=m
CONFIG_MISDN_AVMFRITZ=m
CONFIG_MISDN_SPEEDFAX=m
CONFIG_MISDN_INFINEON=m
CONFIG_MISDN_W6692=m
CONFIG_MISDN_NETJET=m
CONFIG_MISDN_HDLC=m
CONFIG_MISDN_IPAC=m
CONFIG_MISDN_ISAR=m
# CONFIG_NVM is not set

#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_LEDS=y
CONFIG_INPUT_FF_MEMLESS=m
CONFIG_INPUT_SPARSEKMAP=m
# CONFIG_INPUT_MATRIXKMAP is not set

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADC is not set
# CONFIG_KEYBOARD_ADP5588 is not set
# CONFIG_KEYBOARD_ADP5589 is not set
# CONFIG_KEYBOARD_APPLESPI is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT1050 is not set
# CONFIG_KEYBOARD_QT1070 is not set
# CONFIG_KEYBOARD_QT2160 is not set
# CONFIG_KEYBOARD_DLINK_DIR685 is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_GPIO is not set
# CONFIG_KEYBOARD_GPIO_POLLED is not set
# CONFIG_KEYBOARD_TCA6416 is not set
# CONFIG_KEYBOARD_TCA8418 is not set
# CONFIG_KEYBOARD_MATRIX is not set
# CONFIG_KEYBOARD_LM8323 is not set
# CONFIG_KEYBOARD_LM8333 is not set
# CONFIG_KEYBOARD_MAX7359 is not set
# CONFIG_KEYBOARD_MCS is not set
# CONFIG_KEYBOARD_MPR121 is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
# CONFIG_KEYBOARD_SAMSUNG is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_TM2_TOUCHKEY is not set
# CONFIG_KEYBOARD_XTKBD is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_BYD=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_SYNAPTICS_SMBUS=y
CONFIG_MOUSE_PS2_CYPRESS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
CONFIG_MOUSE_PS2_ELANTECH=y
CONFIG_MOUSE_PS2_ELANTECH_SMBUS=y
CONFIG_MOUSE_PS2_SENTELIC=y
# CONFIG_MOUSE_PS2_TOUCHKIT is not set
CONFIG_MOUSE_PS2_FOCALTECH=y
CONFIG_MOUSE_PS2_VMMOUSE=y
CONFIG_MOUSE_PS2_SMBUS=y
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_APPLETOUCH=m
CONFIG_MOUSE_BCM5974=m
CONFIG_MOUSE_CYAPA=m
CONFIG_MOUSE_ELAN_I2C=m
CONFIG_MOUSE_ELAN_I2C_I2C=y
CONFIG_MOUSE_ELAN_I2C_SMBUS=y
CONFIG_MOUSE_VSXXXAA=m
# CONFIG_MOUSE_GPIO is not set
CONFIG_MOUSE_SYNAPTICS_I2C=m
CONFIG_MOUSE_SYNAPTICS_USB=m
# CONFIG_INPUT_JOYSTICK is not set
CONFIG_INPUT_TABLET=y
CONFIG_TABLET_USB_ACECAD=m
CONFIG_TABLET_USB_AIPTEK=m
# CONFIG_TABLET_USB_HANWANG is not set
CONFIG_TABLET_USB_KBTAB=m
# CONFIG_TABLET_USB_PEGASUS is not set
CONFIG_TABLET_SERIAL_WACOM4=m
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_PROPERTIES=y
# CONFIG_TOUCHSCREEN_ADS7846 is not set
# CONFIG_TOUCHSCREEN_AD7877 is not set
# CONFIG_TOUCHSCREEN_AD7879 is not set
# CONFIG_TOUCHSCREEN_ADC is not set
# CONFIG_TOUCHSCREEN_ATMEL_MXT is not set
# CONFIG_TOUCHSCREEN_AUO_PIXCIR is not set
# CONFIG_TOUCHSCREEN_BU21013 is not set
# CONFIG_TOUCHSCREEN_BU21029 is not set
# CONFIG_TOUCHSCREEN_CHIPONE_ICN8505 is not set
# CONFIG_TOUCHSCREEN_CY8CTMA140 is not set
# CONFIG_TOUCHSCREEN_CY8CTMG110 is not set
# CONFIG_TOUCHSCREEN_CYTTSP_CORE is not set
# CONFIG_TOUCHSCREEN_CYTTSP4_CORE is not set
# CONFIG_TOUCHSCREEN_DYNAPRO is not set
# CONFIG_TOUCHSCREEN_HAMPSHIRE is not set
# CONFIG_TOUCHSCREEN_EETI is not set
# CONFIG_TOUCHSCREEN_EGALAX_SERIAL is not set
# CONFIG_TOUCHSCREEN_EXC3000 is not set
# CONFIG_TOUCHSCREEN_FUJITSU is not set
# CONFIG_TOUCHSCREEN_GOODIX is not set
# CONFIG_TOUCHSCREEN_HIDEEP is not set
# CONFIG_TOUCHSCREEN_ILI210X is not set
# CONFIG_TOUCHSCREEN_S6SY761 is not set
# CONFIG_TOUCHSCREEN_GUNZE is not set
# CONFIG_TOUCHSCREEN_EKTF2127 is not set
# CONFIG_TOUCHSCREEN_ELAN is not set
CONFIG_TOUCHSCREEN_ELO=m
CONFIG_TOUCHSCREEN_WACOM_W8001=m
CONFIG_TOUCHSCREEN_WACOM_I2C=m
# CONFIG_TOUCHSCREEN_MAX11801 is not set
# CONFIG_TOUCHSCREEN_MCS5000 is not set
# CONFIG_TOUCHSCREEN_MMS114 is not set
# CONFIG_TOUCHSCREEN_MELFAS_MIP4 is not set
# CONFIG_TOUCHSCREEN_MTOUCH is not set
# CONFIG_TOUCHSCREEN_INEXIO is not set
# CONFIG_TOUCHSCREEN_MK712 is not set
# CONFIG_TOUCHSCREEN_PENMOUNT is not set
# CONFIG_TOUCHSCREEN_EDT_FT5X06 is not set
# CONFIG_TOUCHSCREEN_TOUCHRIGHT is not set
# CONFIG_TOUCHSCREEN_TOUCHWIN is not set
# CONFIG_TOUCHSCREEN_PIXCIR is not set
# CONFIG_TOUCHSCREEN_WDT87XX_I2C is not set
# CONFIG_TOUCHSCREEN_WM97XX is not set
# CONFIG_TOUCHSCREEN_USB_COMPOSITE is not set
# CONFIG_TOUCHSCREEN_TOUCHIT213 is not set
# CONFIG_TOUCHSCREEN_TSC_SERIO is not set
# CONFIG_TOUCHSCREEN_TSC2004 is not set
# CONFIG_TOUCHSCREEN_TSC2005 is not set
# CONFIG_TOUCHSCREEN_TSC2007 is not set
# CONFIG_TOUCHSCREEN_RM_TS is not set
# CONFIG_TOUCHSCREEN_SILEAD is not set
# CONFIG_TOUCHSCREEN_SIS_I2C is not set
# CONFIG_TOUCHSCREEN_ST1232 is not set
# CONFIG_TOUCHSCREEN_STMFTS is not set
# CONFIG_TOUCHSCREEN_SUR40 is not set
# CONFIG_TOUCHSCREEN_SURFACE3_SPI is not set
# CONFIG_TOUCHSCREEN_SX8654 is not set
# CONFIG_TOUCHSCREEN_TPS6507X is not set
# CONFIG_TOUCHSCREEN_ZET6223 is not set
# CONFIG_TOUCHSCREEN_ZFORCE is not set
# CONFIG_TOUCHSCREEN_ROHM_BU21023 is not set
# CONFIG_TOUCHSCREEN_IQS5XX is not set
# CONFIG_TOUCHSCREEN_ZINITIX is not set
CONFIG_INPUT_MISC=y
# CONFIG_INPUT_AD714X is not set
# CONFIG_INPUT_BMA150 is not set
# CONFIG_INPUT_E3X0_BUTTON is not set
CONFIG_INPUT_PCSPKR=m
# CONFIG_INPUT_MMA8450 is not set
CONFIG_INPUT_APANEL=m
# CONFIG_INPUT_GPIO_BEEPER is not set
# CONFIG_INPUT_GPIO_DECODER is not set
# CONFIG_INPUT_GPIO_VIBRA is not set
CONFIG_INPUT_ATLAS_BTNS=m
CONFIG_INPUT_ATI_REMOTE2=m
CONFIG_INPUT_KEYSPAN_REMOTE=m
# CONFIG_INPUT_KXTJ9 is not set
CONFIG_INPUT_POWERMATE=m
CONFIG_INPUT_YEALINK=m
CONFIG_INPUT_CM109=m
CONFIG_INPUT_UINPUT=m
# CONFIG_INPUT_PCF8574 is not set
# CONFIG_INPUT_PWM_BEEPER is not set
# CONFIG_INPUT_PWM_VIBRA is not set
CONFIG_INPUT_GPIO_ROTARY_ENCODER=m
# CONFIG_INPUT_DA7280_HAPTICS is not set
# CONFIG_INPUT_ADXL34X is not set
# CONFIG_INPUT_IMS_PCU is not set
# CONFIG_INPUT_IQS269A is not set
# CONFIG_INPUT_CMA3000 is not set
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=m
# CONFIG_INPUT_IDEAPAD_SLIDEBAR is not set
# CONFIG_INPUT_DRV260X_HAPTICS is not set
# CONFIG_INPUT_DRV2665_HAPTICS is not set
# CONFIG_INPUT_DRV2667_HAPTICS is not set
CONFIG_RMI4_CORE=m
CONFIG_RMI4_I2C=m
CONFIG_RMI4_SPI=m
CONFIG_RMI4_SMB=m
CONFIG_RMI4_F03=y
CONFIG_RMI4_F03_SERIO=m
CONFIG_RMI4_2D_SENSOR=y
CONFIG_RMI4_F11=y
CONFIG_RMI4_F12=y
CONFIG_RMI4_F30=y
CONFIG_RMI4_F34=y
# CONFIG_RMI4_F3A is not set
# CONFIG_RMI4_F54 is not set
CONFIG_RMI4_F55=y

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
CONFIG_SERIO_ALTERA_PS2=m
# CONFIG_SERIO_PS2MULT is not set
CONFIG_SERIO_ARC_PS2=m
CONFIG_HYPERV_KEYBOARD=m
# CONFIG_SERIO_GPIO_PS2 is not set
# CONFIG_USERIO is not set
# CONFIG_GAMEPORT is not set
# end of Hardware I/O ports
# end of Input device support

#
# Character devices
#
CONFIG_TTY=y
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_VT_CONSOLE_SLEEP=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set
CONFIG_LDISC_AUTOLOAD=y

#
# Serial drivers
#
CONFIG_SERIAL_EARLYCON=y
CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set
CONFIG_SERIAL_8250_PNP=y
# CONFIG_SERIAL_8250_16550A_VARIANTS is not set
# CONFIG_SERIAL_8250_FINTEK is not set
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_DMA=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_EXAR=y
CONFIG_SERIAL_8250_NR_UARTS=64
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
CONFIG_SERIAL_8250_RSA=y
CONFIG_SERIAL_8250_DWLIB=y
CONFIG_SERIAL_8250_DW=y
# CONFIG_SERIAL_8250_RT288X is not set
CONFIG_SERIAL_8250_LPSS=y
CONFIG_SERIAL_8250_MID=y

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_KGDB_NMI is not set
# CONFIG_SERIAL_MAX3100 is not set
# CONFIG_SERIAL_MAX310X is not set
# CONFIG_SERIAL_UARTLITE is not set
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_CONSOLE_POLL=y
CONFIG_SERIAL_JSM=m
# CONFIG_SERIAL_LANTIQ is not set
# CONFIG_SERIAL_SCCNXP is not set
# CONFIG_SERIAL_SC16IS7XX is not set
# CONFIG_SERIAL_BCM63XX is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
# CONFIG_SERIAL_IFX6X60 is not set
CONFIG_SERIAL_ARC=m
CONFIG_SERIAL_ARC_NR_PORTS=1
# CONFIG_SERIAL_RP2 is not set
# CONFIG_SERIAL_FSL_LPUART is not set
# CONFIG_SERIAL_FSL_LINFLEXUART is not set
# CONFIG_SERIAL_SPRD is not set
# end of Serial drivers

CONFIG_SERIAL_MCTRL_GPIO=y
CONFIG_SERIAL_NONSTANDARD=y
# CONFIG_ROCKETPORT is not set
CONFIG_CYCLADES=m
# CONFIG_CYZ_INTR is not set
# CONFIG_MOXA_INTELLIO is not set
# CONFIG_MOXA_SMARTIO is not set
CONFIG_SYNCLINK_GT=m
# CONFIG_ISI is not set
CONFIG_N_HDLC=m
CONFIG_N_GSM=m
CONFIG_NOZOMI=m
# CONFIG_NULL_TTY is not set
# CONFIG_TRACE_SINK is not set
CONFIG_HVC_DRIVER=y
CONFIG_HVC_IRQ=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
# CONFIG_SERIAL_DEV_BUS is not set
CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
CONFIG_PPDEV=m
CONFIG_VIRTIO_CONSOLE=m
CONFIG_IPMI_HANDLER=m
CONFIG_IPMI_DMI_DECODE=y
CONFIG_IPMI_PLAT_DATA=y
CONFIG_IPMI_PANIC_EVENT=y
CONFIG_IPMI_PANIC_STRING=y
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_SSIF=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_TIMERIOMEM=m
CONFIG_HW_RANDOM_INTEL=m
CONFIG_HW_RANDOM_AMD=m
# CONFIG_HW_RANDOM_BA431 is not set
CONFIG_HW_RANDOM_VIA=m
CONFIG_HW_RANDOM_VIRTIO=y
# CONFIG_HW_RANDOM_XIPHERA is not set
# CONFIG_APPLICOM is not set
# CONFIG_MWAVE is not set
CONFIG_DEVMEM=y
# CONFIG_DEVKMEM is not set
CONFIG_NVRAM=y
CONFIG_RAW_DRIVER=y
CONFIG_MAX_RAW_DEVS=8192
CONFIG_DEVPORT=y
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
# CONFIG_HPET_MMAP_DEFAULT is not set
CONFIG_HANGCHECK_TIMER=m
CONFIG_UV_MMTIMER=m
CONFIG_TCG_TPM=y
CONFIG_HW_RANDOM_TPM=y
CONFIG_TCG_TIS_CORE=y
CONFIG_TCG_TIS=y
# CONFIG_TCG_TIS_SPI is not set
CONFIG_TCG_TIS_I2C_ATMEL=m
CONFIG_TCG_TIS_I2C_INFINEON=m
CONFIG_TCG_TIS_I2C_NUVOTON=m
CONFIG_TCG_NSC=m
CONFIG_TCG_ATMEL=m
CONFIG_TCG_INFINEON=m
# CONFIG_TCG_XEN is not set
CONFIG_TCG_CRB=y
# CONFIG_TCG_VTPM_PROXY is not set
CONFIG_TCG_TIS_ST33ZP24=m
CONFIG_TCG_TIS_ST33ZP24_I2C=m
# CONFIG_TCG_TIS_ST33ZP24_SPI is not set
CONFIG_TELCLOCK=m
# CONFIG_XILLYBUS is not set
# end of Character devices

CONFIG_RANDOM_TRUST_CPU=y
# CONFIG_RANDOM_TRUST_BOOTLOADER is not set

#
# I2C support
#
CONFIG_I2C=y
CONFIG_ACPI_I2C_OPREGION=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=m
CONFIG_I2C_MUX=m

#
# Multiplexer I2C Chip support
#
# CONFIG_I2C_MUX_GPIO is not set
# CONFIG_I2C_MUX_LTC4306 is not set
# CONFIG_I2C_MUX_PCA9541 is not set
# CONFIG_I2C_MUX_PCA954x is not set
# CONFIG_I2C_MUX_REG is not set
CONFIG_I2C_MUX_MLXCPLD=m
# end of Multiplexer I2C Chip support

CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_SMBUS=m
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCA=m

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD756_S4882=m
CONFIG_I2C_AMD8111=m
# CONFIG_I2C_AMD_MP2 is not set
CONFIG_I2C_I801=m
CONFIG_I2C_ISCH=m
CONFIG_I2C_ISMT=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_NFORCE2_S4985=m
# CONFIG_I2C_NVIDIA_GPU is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m

#
# ACPI drivers
#
CONFIG_I2C_SCMI=m

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_CBUS_GPIO is not set
CONFIG_I2C_DESIGNWARE_CORE=m
# CONFIG_I2C_DESIGNWARE_SLAVE is not set
CONFIG_I2C_DESIGNWARE_PLATFORM=m
CONFIG_I2C_DESIGNWARE_BAYTRAIL=y
# CONFIG_I2C_DESIGNWARE_PCI is not set
# CONFIG_I2C_EMEV2 is not set
# CONFIG_I2C_GPIO is not set
# CONFIG_I2C_OCORES is not set
CONFIG_I2C_PCA_PLATFORM=m
CONFIG_I2C_SIMTEC=m
# CONFIG_I2C_XILINX is not set

#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_DIOLAN_U2C=m
CONFIG_I2C_PARPORT=m
# CONFIG_I2C_ROBOTFUZZ_OSIF is not set
# CONFIG_I2C_TAOS_EVM is not set
CONFIG_I2C_TINY_USB=m
CONFIG_I2C_VIPERBOARD=m

#
# Other I2C/SMBus bus drivers
#
CONFIG_I2C_MLXCPLD=m
# end of I2C Hardware Bus support

CONFIG_I2C_STUB=m
# CONFIG_I2C_SLAVE is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# end of I2C support

# CONFIG_I3C is not set
CONFIG_SPI=y
# CONFIG_SPI_DEBUG is not set
CONFIG_SPI_MASTER=y
# CONFIG_SPI_MEM is not set

#
# SPI Master Controller Drivers
#
# CONFIG_SPI_ALTERA is not set
# CONFIG_SPI_AXI_SPI_ENGINE is not set
# CONFIG_SPI_BITBANG is not set
# CONFIG_SPI_BUTTERFLY is not set
# CONFIG_SPI_CADENCE is not set
# CONFIG_SPI_DESIGNWARE is not set
# CONFIG_SPI_NXP_FLEXSPI is not set
# CONFIG_SPI_GPIO is not set
# CONFIG_SPI_LM70_LLP is not set
# CONFIG_SPI_LANTIQ_SSC is not set
# CONFIG_SPI_OC_TINY is not set
CONFIG_SPI_PXA2XX=m
CONFIG_SPI_PXA2XX_PCI=m
# CONFIG_SPI_ROCKCHIP is not set
# CONFIG_SPI_SC18IS602 is not set
# CONFIG_SPI_SIFIVE is not set
# CONFIG_SPI_MXIC is not set
# CONFIG_SPI_XCOMM is not set
# CONFIG_SPI_XILINX is not set
# CONFIG_SPI_ZYNQMP_GQSPI is not set
# CONFIG_SPI_AMD is not set

#
# SPI Multiplexer support
#
# CONFIG_SPI_MUX is not set

#
# SPI Protocol Masters
#
# CONFIG_SPI_SPIDEV is not set
# CONFIG_SPI_LOOPBACK_TEST is not set
# CONFIG_SPI_TLE62X0 is not set
# CONFIG_SPI_SLAVE is not set
CONFIG_SPI_DYNAMIC=y
# CONFIG_SPMI is not set
# CONFIG_HSI is not set
CONFIG_PPS=y
# CONFIG_PPS_DEBUG is not set

#
# PPS clients support
#
# CONFIG_PPS_CLIENT_KTIMER is not set
CONFIG_PPS_CLIENT_LDISC=m
CONFIG_PPS_CLIENT_PARPORT=m
CONFIG_PPS_CLIENT_GPIO=m

#
# PPS generators support
#

#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK=y
CONFIG_DP83640_PHY=m
# CONFIG_PTP_1588_CLOCK_INES is not set
CONFIG_PTP_1588_CLOCK_KVM=m
# CONFIG_PTP_1588_CLOCK_IDT82P33 is not set
# CONFIG_PTP_1588_CLOCK_IDTCM is not set
CONFIG_PTP_1588_CLOCK_VMW=m
# CONFIG_PTP_1588_CLOCK_OCP is not set
# end of PTP clock support

CONFIG_PINCTRL=y
CONFIG_PINMUX=y
CONFIG_PINCONF=y
CONFIG_GENERIC_PINCONF=y
# CONFIG_DEBUG_PINCTRL is not set
CONFIG_PINCTRL_AMD=m
# CONFIG_PINCTRL_MCP23S08 is not set
# CONFIG_PINCTRL_SX150X is not set
CONFIG_PINCTRL_BAYTRAIL=y
# CONFIG_PINCTRL_CHERRYVIEW is not set
# CONFIG_PINCTRL_LYNXPOINT is not set
CONFIG_PINCTRL_INTEL=y
# CONFIG_PINCTRL_ALDERLAKE is not set
CONFIG_PINCTRL_BROXTON=m
CONFIG_PINCTRL_CANNONLAKE=m
CONFIG_PINCTRL_CEDARFORK=m
CONFIG_PINCTRL_DENVERTON=m
# CONFIG_PINCTRL_ELKHARTLAKE is not set
# CONFIG_PINCTRL_EMMITSBURG is not set
CONFIG_PINCTRL_GEMINILAKE=m
CONFIG_PINCTRL_ICELAKE=m
# CONFIG_PINCTRL_JASPERLAKE is not set
# CONFIG_PINCTRL_LAKEFIELD is not set
CONFIG_PINCTRL_LEWISBURG=m
CONFIG_PINCTRL_SUNRISEPOINT=m
CONFIG_PINCTRL_TIGERLAKE=m

#
# Renesas pinctrl drivers
#
# end of Renesas pinctrl drivers

CONFIG_GPIOLIB=y
CONFIG_GPIOLIB_FASTPATH_LIMIT=512
CONFIG_GPIO_ACPI=y
CONFIG_GPIOLIB_IRQCHIP=y
# CONFIG_DEBUG_GPIO is not set
CONFIG_GPIO_CDEV=y
CONFIG_GPIO_CDEV_V1=y
CONFIG_GPIO_GENERIC=m

#
# Memory mapped GPIO drivers
#
CONFIG_GPIO_AMDPT=m
# CONFIG_GPIO_DWAPB is not set
# CONFIG_GPIO_EXAR is not set
# CONFIG_GPIO_GENERIC_PLATFORM is not set
CONFIG_GPIO_ICH=m
# CONFIG_GPIO_MB86S7X is not set
# CONFIG_GPIO_VX855 is not set
# CONFIG_GPIO_XILINX is not set
# CONFIG_GPIO_AMD_FCH is not set
# end of Memory mapped GPIO drivers

#
# Port-mapped I/O GPIO drivers
#
# CONFIG_GPIO_F7188X is not set
# CONFIG_GPIO_IT87 is not set
# CONFIG_GPIO_SCH is not set
# CONFIG_GPIO_SCH311X is not set
# CONFIG_GPIO_WINBOND is not set
# CONFIG_GPIO_WS16C48 is not set
# end of Port-mapped I/O GPIO drivers

#
# I2C GPIO expanders
#
# CONFIG_GPIO_ADP5588 is not set
# CONFIG_GPIO_MAX7300 is not set
# CONFIG_GPIO_MAX732X is not set
# CONFIG_GPIO_PCA953X is not set
# CONFIG_GPIO_PCA9570 is not set
# CONFIG_GPIO_PCF857X is not set
# CONFIG_GPIO_TPIC2810 is not set
# end of I2C GPIO expanders

#
# MFD GPIO expanders
#
# end of MFD GPIO expanders

#
# PCI GPIO expanders
#
# CONFIG_GPIO_AMD8111 is not set
# CONFIG_GPIO_ML_IOH is not set
# CONFIG_GPIO_PCI_IDIO_16 is not set
# CONFIG_GPIO_PCIE_IDIO_24 is not set
# CONFIG_GPIO_RDC321X is not set
# end of PCI GPIO expanders

#
# SPI GPIO expanders
#
# CONFIG_GPIO_MAX3191X is not set
# CONFIG_GPIO_MAX7301 is not set
# CONFIG_GPIO_MC33880 is not set
# CONFIG_GPIO_PISOSR is not set
# CONFIG_GPIO_XRA1403 is not set
# end of SPI GPIO expanders

#
# USB GPIO expanders
#
CONFIG_GPIO_VIPERBOARD=m
# end of USB GPIO expanders

#
# Virtual GPIO drivers
#
# CONFIG_GPIO_AGGREGATOR is not set
# CONFIG_GPIO_MOCKUP is not set
# end of Virtual GPIO drivers

# CONFIG_W1 is not set
CONFIG_POWER_RESET=y
# CONFIG_POWER_RESET_RESTART is not set
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
CONFIG_POWER_SUPPLY_HWMON=y
# CONFIG_PDA_POWER is not set
# CONFIG_GENERIC_ADC_BATTERY is not set
# CONFIG_TEST_POWER is not set
# CONFIG_CHARGER_ADP5061 is not set
# CONFIG_BATTERY_CW2015 is not set
# CONFIG_BATTERY_DS2780 is not set
# CONFIG_BATTERY_DS2781 is not set
# CONFIG_BATTERY_DS2782 is not set
# CONFIG_BATTERY_SBS is not set
# CONFIG_CHARGER_SBS is not set
# CONFIG_MANAGER_SBS is not set
# CONFIG_BATTERY_BQ27XXX is not set
# CONFIG_BATTERY_MAX17040 is not set
# CONFIG_BATTERY_MAX17042 is not set
# CONFIG_CHARGER_MAX8903 is not set
# CONFIG_CHARGER_LP8727 is not set
# CONFIG_CHARGER_GPIO is not set
# CONFIG_CHARGER_LT3651 is not set
# CONFIG_CHARGER_BQ2415X is not set
# CONFIG_CHARGER_BQ24257 is not set
# CONFIG_CHARGER_BQ24735 is not set
# CONFIG_CHARGER_BQ2515X is not set
# CONFIG_CHARGER_BQ25890 is not set
# CONFIG_CHARGER_BQ25980 is not set
CONFIG_CHARGER_SMB347=m
# CONFIG_BATTERY_GAUGE_LTC2941 is not set
# CONFIG_CHARGER_RT9455 is not set
# CONFIG_CHARGER_BD99954 is not set
CONFIG_HWMON=y
CONFIG_HWMON_VID=m
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Native drivers
#
CONFIG_SENSORS_ABITUGURU=m
CONFIG_SENSORS_ABITUGURU3=m
# CONFIG_SENSORS_AD7314 is not set
CONFIG_SENSORS_AD7414=m
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1026=m
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
# CONFIG_SENSORS_ADM1177 is not set
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7X10=m
# CONFIG_SENSORS_ADT7310 is not set
CONFIG_SENSORS_ADT7410=m
CONFIG_SENSORS_ADT7411=m
CONFIG_SENSORS_ADT7462=m
CONFIG_SENSORS_ADT7470=m
CONFIG_SENSORS_ADT7475=m
# CONFIG_SENSORS_AS370 is not set
CONFIG_SENSORS_ASC7621=m
# CONFIG_SENSORS_AXI_FAN_CONTROL is not set
CONFIG_SENSORS_K8TEMP=m
CONFIG_SENSORS_K10TEMP=m
CONFIG_SENSORS_FAM15H_POWER=m
CONFIG_SENSORS_AMD_ENERGY=m
CONFIG_SENSORS_APPLESMC=m
CONFIG_SENSORS_ASB100=m
# CONFIG_SENSORS_ASPEED is not set
CONFIG_SENSORS_ATXP1=m
# CONFIG_SENSORS_CORSAIR_CPRO is not set
# CONFIG_SENSORS_CORSAIR_PSU is not set
# CONFIG_SENSORS_DRIVETEMP is not set
CONFIG_SENSORS_DS620=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_DELL_SMM=m
CONFIG_SENSORS_I5K_AMB=m
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_FSCHMD=m
# CONFIG_SENSORS_FTSTEUTATES is not set
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_GL520SM=m
CONFIG_SENSORS_G760A=m
# CONFIG_SENSORS_G762 is not set
# CONFIG_SENSORS_HIH6130 is not set
CONFIG_SENSORS_IBMAEM=m
CONFIG_SENSORS_IBMPEX=m
# CONFIG_SENSORS_IIO_HWMON is not set
CONFIG_SENSORS_I5500=m
CONFIG_SENSORS_CORETEMP=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_JC42=m
# CONFIG_SENSORS_POWR1220 is not set
CONFIG_SENSORS_LINEAGE=m
# CONFIG_SENSORS_LTC2945 is not set
# CONFIG_SENSORS_LTC2947_I2C is not set
# CONFIG_SENSORS_LTC2947_SPI is not set
# CONFIG_SENSORS_LTC2990 is not set
# CONFIG_SENSORS_LTC2992 is not set
CONFIG_SENSORS_LTC4151=m
CONFIG_SENSORS_LTC4215=m
# CONFIG_SENSORS_LTC4222 is not set
CONFIG_SENSORS_LTC4245=m
# CONFIG_SENSORS_LTC4260 is not set
CONFIG_SENSORS_LTC4261=m
# CONFIG_SENSORS_MAX1111 is not set
# CONFIG_SENSORS_MAX127 is not set
CONFIG_SENSORS_MAX16065=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_MAX1668=m
CONFIG_SENSORS_MAX197=m
# CONFIG_SENSORS_MAX31722 is not set
# CONFIG_SENSORS_MAX31730 is not set
# CONFIG_SENSORS_MAX6621 is not set
CONFIG_SENSORS_MAX6639=m
CONFIG_SENSORS_MAX6642=m
CONFIG_SENSORS_MAX6650=m
CONFIG_SENSORS_MAX6697=m
# CONFIG_SENSORS_MAX31790 is not set
CONFIG_SENSORS_MCP3021=m
CONFIG_SENSORS_MLXREG_FAN=m
# CONFIG_SENSORS_TC654 is not set
# CONFIG_SENSORS_MR75203 is not set
# CONFIG_SENSORS_ADCXX is not set
CONFIG_SENSORS_LM63=m
# CONFIG_SENSORS_LM70 is not set
CONFIG_SENSORS_LM73=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM87=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_LM92=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_LM95234=m
CONFIG_SENSORS_LM95241=m
CONFIG_SENSORS_LM95245=m
CONFIG_SENSORS_PC87360=m
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_NTC_THERMISTOR=m
# CONFIG_SENSORS_NCT6683 is not set
CONFIG_SENSORS_NCT6775=m
# CONFIG_SENSORS_NCT7802 is not set
# CONFIG_SENSORS_NCT7904 is not set
# CONFIG_SENSORS_NPCM7XX is not set
CONFIG_SENSORS_PCF8591=m
CONFIG_PMBUS=m
CONFIG_SENSORS_PMBUS=m
# CONFIG_SENSORS_ADM1266 is not set
CONFIG_SENSORS_ADM1275=m
# CONFIG_SENSORS_BEL_PFE is not set
# CONFIG_SENSORS_IBM_CFFPS is not set
# CONFIG_SENSORS_INSPUR_IPSPS is not set
# CONFIG_SENSORS_IR35221 is not set
# CONFIG_SENSORS_IR38064 is not set
# CONFIG_SENSORS_IRPS5401 is not set
# CONFIG_SENSORS_ISL68137 is not set
CONFIG_SENSORS_LM25066=m
CONFIG_SENSORS_LTC2978=m
# CONFIG_SENSORS_LTC3815 is not set
CONFIG_SENSORS_MAX16064=m
# CONFIG_SENSORS_MAX16601 is not set
# CONFIG_SENSORS_MAX20730 is not set
# CONFIG_SENSORS_MAX20751 is not set
# CONFIG_SENSORS_MAX31785 is not set
CONFIG_SENSORS_MAX34440=m
CONFIG_SENSORS_MAX8688=m
# CONFIG_SENSORS_MP2975 is not set
# CONFIG_SENSORS_PM6764TR is not set
# CONFIG_SENSORS_PXE1610 is not set
# CONFIG_SENSORS_Q54SJ108A2 is not set
# CONFIG_SENSORS_TPS40422 is not set
# CONFIG_SENSORS_TPS53679 is not set
CONFIG_SENSORS_UCD9000=m
CONFIG_SENSORS_UCD9200=m
# CONFIG_SENSORS_XDPE122 is not set
CONFIG_SENSORS_ZL6100=m
# CONFIG_SENSORS_SBTSI is not set
CONFIG_SENSORS_SHT15=m
CONFIG_SENSORS_SHT21=m
# CONFIG_SENSORS_SHT3x is not set
# CONFIG_SENSORS_SHTC1 is not set
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_DME1737=m
CONFIG_SENSORS_EMC1403=m
# CONFIG_SENSORS_EMC2103 is not set
CONFIG_SENSORS_EMC6W201=m
CONFIG_SENSORS_SMSC47M1=m
CONFIG_SENSORS_SMSC47M192=m
CONFIG_SENSORS_SMSC47B397=m
CONFIG_SENSORS_SCH56XX_COMMON=m
CONFIG_SENSORS_SCH5627=m
CONFIG_SENSORS_SCH5636=m
# CONFIG_SENSORS_STTS751 is not set
# CONFIG_SENSORS_SMM665 is not set
# CONFIG_SENSORS_ADC128D818 is not set
CONFIG_SENSORS_ADS7828=m
# CONFIG_SENSORS_ADS7871 is not set
CONFIG_SENSORS_AMC6821=m
CONFIG_SENSORS_INA209=m
CONFIG_SENSORS_INA2XX=m
# CONFIG_SENSORS_INA3221 is not set
# CONFIG_SENSORS_TC74 is not set
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_TMP102=m
# CONFIG_SENSORS_TMP103 is not set
# CONFIG_SENSORS_TMP108 is not set
CONFIG_SENSORS_TMP401=m
CONFIG_SENSORS_TMP421=m
# CONFIG_SENSORS_TMP513 is not set
CONFIG_SENSORS_VIA_CPUTEMP=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_VT1211=m
CONFIG_SENSORS_VT8231=m
# CONFIG_SENSORS_W83773G is not set
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83791D=m
CONFIG_SENSORS_W83792D=m
CONFIG_SENSORS_W83793=m
CONFIG_SENSORS_W83795=m
# CONFIG_SENSORS_W83795_FANCTRL is not set
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
# CONFIG_SENSORS_XGENE is not set

#
# ACPI drivers
#
CONFIG_SENSORS_ACPI_POWER=m
CONFIG_SENSORS_ATK0110=m
CONFIG_THERMAL=y
# CONFIG_THERMAL_NETLINK is not set
# CONFIG_THERMAL_STATISTICS is not set
CONFIG_THERMAL_EMERGENCY_POWEROFF_DELAY_MS=0
CONFIG_THERMAL_HWMON=y
CONFIG_THERMAL_WRITABLE_TRIPS=y
CONFIG_THERMAL_DEFAULT_GOV_STEP_WISE=y
# CONFIG_THERMAL_DEFAULT_GOV_FAIR_SHARE is not set
# CONFIG_THERMAL_DEFAULT_GOV_USER_SPACE is not set
CONFIG_THERMAL_GOV_FAIR_SHARE=y
CONFIG_THERMAL_GOV_STEP_WISE=y
CONFIG_THERMAL_GOV_BANG_BANG=y
CONFIG_THERMAL_GOV_USER_SPACE=y
# CONFIG_THERMAL_EMULATION is not set

#
# Intel thermal drivers
#
CONFIG_INTEL_POWERCLAMP=m
CONFIG_X86_PKG_TEMP_THERMAL=m
CONFIG_INTEL_SOC_DTS_IOSF_CORE=m
# CONFIG_INTEL_SOC_DTS_THERMAL is not set

#
# ACPI INT340X thermal drivers
#
CONFIG_INT340X_THERMAL=m
CONFIG_ACPI_THERMAL_REL=m
# CONFIG_INT3406_THERMAL is not set
CONFIG_PROC_THERMAL_MMIO_RAPL=m
# end of ACPI INT340X thermal drivers

CONFIG_INTEL_PCH_THERMAL=m
# end of Intel thermal drivers

# CONFIG_GENERIC_ADC_THERMAL is not set
CONFIG_WATCHDOG=y
CONFIG_WATCHDOG_CORE=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
CONFIG_WATCHDOG_HANDLE_BOOT_ENABLED=y
CONFIG_WATCHDOG_OPEN_TIMEOUT=0
CONFIG_WATCHDOG_SYSFS=y

#
# Watchdog Pretimeout Governors
#
# CONFIG_WATCHDOG_PRETIMEOUT_GOV is not set

#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
CONFIG_WDAT_WDT=m
# CONFIG_XILINX_WATCHDOG is not set
# CONFIG_ZIIRAVE_WATCHDOG is not set
CONFIG_MLX_WDT=m
# CONFIG_CADENCE_WATCHDOG is not set
# CONFIG_DW_WATCHDOG is not set
# CONFIG_MAX63XX_WATCHDOG is not set
# CONFIG_ACQUIRE_WDT is not set
# CONFIG_ADVANTECH_WDT is not set
CONFIG_ALIM1535_WDT=m
CONFIG_ALIM7101_WDT=m
# CONFIG_EBC_C384_WDT is not set
CONFIG_F71808E_WDT=m
CONFIG_SP5100_TCO=m
CONFIG_SBC_FITPC2_WATCHDOG=m
# CONFIG_EUROTECH_WDT is not set
CONFIG_IB700_WDT=m
CONFIG_IBMASR=m
# CONFIG_WAFER_WDT is not set
CONFIG_I6300ESB_WDT=m
CONFIG_IE6XX_WDT=m
CONFIG_ITCO_WDT=m
CONFIG_ITCO_VENDOR_SUPPORT=y
CONFIG_IT8712F_WDT=m
CONFIG_IT87_WDT=m
CONFIG_HP_WATCHDOG=m
CONFIG_HPWDT_NMI_DECODING=y
# CONFIG_SC1200_WDT is not set
# CONFIG_PC87413_WDT is not set
CONFIG_NV_TCO=m
# CONFIG_60XX_WDT is not set
# CONFIG_CPU5_WDT is not set
CONFIG_SMSC_SCH311X_WDT=m
# CONFIG_SMSC37B787_WDT is not set
# CONFIG_TQMX86_WDT is not set
CONFIG_VIA_WDT=m
CONFIG_W83627HF_WDT=m
CONFIG_W83877F_WDT=m
CONFIG_W83977F_WDT=m
CONFIG_MACHZ_WDT=m
# CONFIG_SBC_EPX_C3_WATCHDOG is not set
CONFIG_INTEL_MEI_WDT=m
# CONFIG_NI903X_WDT is not set
# CONFIG_NIC7018_WDT is not set
# CONFIG_MEN_A21_WDT is not set
CONFIG_XEN_WDT=m

#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m

#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_SSB_POSSIBLE=y
# CONFIG_SSB is not set
CONFIG_BCMA_POSSIBLE=y
CONFIG_BCMA=m
CONFIG_BCMA_HOST_PCI_POSSIBLE=y
CONFIG_BCMA_HOST_PCI=y
# CONFIG_BCMA_HOST_SOC is not set
CONFIG_BCMA_DRIVER_PCI=y
CONFIG_BCMA_DRIVER_GMAC_CMN=y
CONFIG_BCMA_DRIVER_GPIO=y
# CONFIG_BCMA_DEBUG is not set

#
# Multifunction device drivers
#
CONFIG_MFD_CORE=y
# CONFIG_MFD_AS3711 is not set
# CONFIG_PMIC_ADP5520 is not set
# CONFIG_MFD_AAT2870_CORE is not set
# CONFIG_MFD_BCM590XX is not set
# CONFIG_MFD_BD9571MWV is not set
# CONFIG_MFD_AXP20X_I2C is not set
# CONFIG_MFD_MADERA is not set
# CONFIG_PMIC_DA903X is not set
# CONFIG_MFD_DA9052_SPI is not set
# CONFIG_MFD_DA9052_I2C is not set
# CONFIG_MFD_DA9055 is not set
# CONFIG_MFD_DA9062 is not set
# CONFIG_MFD_DA9063 is not set
# CONFIG_MFD_DA9150 is not set
# CONFIG_MFD_DLN2 is not set
# CONFIG_MFD_MC13XXX_SPI is not set
# CONFIG_MFD_MC13XXX_I2C is not set
# CONFIG_MFD_MP2629 is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_HTC_I2CPLD is not set
# CONFIG_MFD_INTEL_QUARK_I2C_GPIO is not set
CONFIG_LPC_ICH=m
CONFIG_LPC_SCH=m
# CONFIG_INTEL_SOC_PMIC_CHTDC_TI is not set
CONFIG_MFD_INTEL_LPSS=m
CONFIG_MFD_INTEL_LPSS_ACPI=m
CONFIG_MFD_INTEL_LPSS_PCI=m
# CONFIG_MFD_INTEL_PMC_BXT is not set
# CONFIG_MFD_INTEL_PMT is not set
# CONFIG_MFD_IQS62X is not set
# CONFIG_MFD_JANZ_CMODIO is not set
# CONFIG_MFD_KEMPLD is not set
# CONFIG_MFD_88PM800 is not set
# CONFIG_MFD_88PM805 is not set
# CONFIG_MFD_88PM860X is not set
# CONFIG_MFD_MAX14577 is not set
# CONFIG_MFD_MAX77693 is not set
# CONFIG_MFD_MAX77843 is not set
# CONFIG_MFD_MAX8907 is not set
# CONFIG_MFD_MAX8925 is not set
# CONFIG_MFD_MAX8997 is not set
# CONFIG_MFD_MAX8998 is not set
# CONFIG_MFD_MT6360 is not set
# CONFIG_MFD_MT6397 is not set
# CONFIG_MFD_MENF21BMC is not set
# CONFIG_EZX_PCAP is not set
CONFIG_MFD_VIPERBOARD=m
# CONFIG_MFD_RETU is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_UCB1400_CORE is not set
# CONFIG_MFD_RDC321X is not set
# CONFIG_MFD_RT5033 is not set
# CONFIG_MFD_RC5T583 is not set
# CONFIG_MFD_SEC_CORE is not set
# CONFIG_MFD_SI476X_CORE is not set
CONFIG_MFD_SM501=m
CONFIG_MFD_SM501_GPIO=y
# CONFIG_MFD_SKY81452 is not set
# CONFIG_ABX500_CORE is not set
# CONFIG_MFD_SYSCON is not set
# CONFIG_MFD_TI_AM335X_TSCADC is not set
# CONFIG_MFD_LP3943 is not set
# CONFIG_MFD_LP8788 is not set
# CONFIG_MFD_TI_LMU is not set
# CONFIG_MFD_PALMAS is not set
# CONFIG_TPS6105X is not set
# CONFIG_TPS65010 is not set
# CONFIG_TPS6507X is not set
# CONFIG_MFD_TPS65086 is not set
# CONFIG_MFD_TPS65090 is not set
# CONFIG_MFD_TI_LP873X is not set
# CONFIG_MFD_TPS6586X is not set
# CONFIG_MFD_TPS65910 is not set
# CONFIG_MFD_TPS65912_I2C is not set
# CONFIG_MFD_TPS65912_SPI is not set
# CONFIG_MFD_TPS80031 is not set
# CONFIG_TWL4030_CORE is not set
# CONFIG_TWL6040_CORE is not set
# CONFIG_MFD_WL1273_CORE is not set
# CONFIG_MFD_LM3533 is not set
# CONFIG_MFD_TQMX86 is not set
CONFIG_MFD_VX855=m
# CONFIG_MFD_ARIZONA_I2C is not set
# CONFIG_MFD_ARIZONA_SPI is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM831X_I2C is not set
# CONFIG_MFD_WM831X_SPI is not set
# CONFIG_MFD_WM8350_I2C is not set
# CONFIG_MFD_WM8994 is not set
# CONFIG_MFD_INTEL_M10_BMC is not set
# end of Multifunction device drivers

# CONFIG_REGULATOR is not set
CONFIG_RC_CORE=m
CONFIG_RC_MAP=m
CONFIG_LIRC=y
CONFIG_RC_DECODERS=y
CONFIG_IR_NEC_DECODER=m
CONFIG_IR_RC5_DECODER=m
CONFIG_IR_RC6_DECODER=m
CONFIG_IR_JVC_DECODER=m
CONFIG_IR_SONY_DECODER=m
CONFIG_IR_SANYO_DECODER=m
# CONFIG_IR_SHARP_DECODER is not set
CONFIG_IR_MCE_KBD_DECODER=m
# CONFIG_IR_XMP_DECODER is not set
CONFIG_IR_IMON_DECODER=m
# CONFIG_IR_RCMM_DECODER is not set
CONFIG_RC_DEVICES=y
CONFIG_RC_ATI_REMOTE=m
CONFIG_IR_ENE=m
CONFIG_IR_IMON=m
CONFIG_IR_IMON_RAW=m
CONFIG_IR_MCEUSB=m
CONFIG_IR_ITE_CIR=m
CONFIG_IR_FINTEK=m
CONFIG_IR_NUVOTON=m
CONFIG_IR_REDRAT3=m
CONFIG_IR_STREAMZAP=m
CONFIG_IR_WINBOND_CIR=m
# CONFIG_IR_IGORPLUGUSB is not set
CONFIG_IR_IGUANA=m
CONFIG_IR_TTUSBIR=m
# CONFIG_RC_LOOPBACK is not set
CONFIG_IR_SERIAL=m
CONFIG_IR_SERIAL_TRANSMITTER=y
CONFIG_IR_SIR=m
# CONFIG_RC_XBOX_DVD is not set
# CONFIG_IR_TOY is not set
CONFIG_CEC_CORE=m
# CONFIG_MEDIA_CEC_RC is not set
CONFIG_MEDIA_CEC_SUPPORT=y
# CONFIG_CEC_CH7322 is not set
# CONFIG_CEC_SECO is not set
CONFIG_USB_PULSE8_CEC=m
CONFIG_USB_RAINSHADOW_CEC=m
CONFIG_MEDIA_SUPPORT=m
CONFIG_MEDIA_SUPPORT_FILTER=y
CONFIG_MEDIA_SUBDRV_AUTOSELECT=y

#
# Media device types
#
CONFIG_MEDIA_CAMERA_SUPPORT=y
CONFIG_MEDIA_ANALOG_TV_SUPPORT=y
CONFIG_MEDIA_DIGITAL_TV_SUPPORT=y
CONFIG_MEDIA_RADIO_SUPPORT=y
# CONFIG_MEDIA_SDR_SUPPORT is not set
# CONFIG_MEDIA_PLATFORM_SUPPORT is not set
# CONFIG_MEDIA_TEST_SUPPORT is not set
# end of Media device types

CONFIG_VIDEO_DEV=m
CONFIG_MEDIA_CONTROLLER=y
CONFIG_DVB_CORE=m

#
# Video4Linux options
#
CONFIG_VIDEO_V4L2=m
CONFIG_VIDEO_V4L2_I2C=y
# CONFIG_VIDEO_V4L2_SUBDEV_API is not set
# CONFIG_VIDEO_ADV_DEBUG is not set
# CONFIG_VIDEO_FIXED_MINOR_RANGES is not set
CONFIG_VIDEO_TUNER=m
CONFIG_VIDEOBUF_GEN=m
CONFIG_VIDEOBUF_DMA_SG=m
CONFIG_VIDEOBUF_VMALLOC=m
# end of Video4Linux options

#
# Media controller options
#
CONFIG_MEDIA_CONTROLLER_DVB=y
# end of Media controller options

#
# Digital TV options
#
# CONFIG_DVB_MMAP is not set
CONFIG_DVB_NET=y
CONFIG_DVB_MAX_ADAPTERS=8
CONFIG_DVB_DYNAMIC_MINORS=y
# CONFIG_DVB_DEMUX_SECTION_LOSS_LOG is not set
# CONFIG_DVB_ULE_DEBUG is not set
# end of Digital TV options

#
# Media drivers
#

#
# Drivers filtered as selected at 'Filter media drivers'
#
CONFIG_TTPCI_EEPROM=m
CONFIG_MEDIA_USB_SUPPORT=y

#
# Webcam devices
#
CONFIG_USB_VIDEO_CLASS=m
CONFIG_USB_VIDEO_CLASS_INPUT_EVDEV=y
CONFIG_USB_GSPCA=m
CONFIG_USB_M5602=m
CONFIG_USB_STV06XX=m
CONFIG_USB_GL860=m
CONFIG_USB_GSPCA_BENQ=m
CONFIG_USB_GSPCA_CONEX=m
CONFIG_USB_GSPCA_CPIA1=m
# CONFIG_USB_GSPCA_DTCS033 is not set
CONFIG_USB_GSPCA_ETOMS=m
CONFIG_USB_GSPCA_FINEPIX=m
CONFIG_USB_GSPCA_JEILINJ=m
CONFIG_USB_GSPCA_JL2005BCD=m
# CONFIG_USB_GSPCA_KINECT is not set
CONFIG_USB_GSPCA_KONICA=m
CONFIG_USB_GSPCA_MARS=m
CONFIG_USB_GSPCA_MR97310A=m
CONFIG_USB_GSPCA_NW80X=m
CONFIG_USB_GSPCA_OV519=m
CONFIG_USB_GSPCA_OV534=m
CONFIG_USB_GSPCA_OV534_9=m
CONFIG_USB_GSPCA_PAC207=m
CONFIG_USB_GSPCA_PAC7302=m
CONFIG_USB_GSPCA_PAC7311=m
CONFIG_USB_GSPCA_SE401=m
CONFIG_USB_GSPCA_SN9C2028=m
CONFIG_USB_GSPCA_SN9C20X=m
CONFIG_USB_GSPCA_SONIXB=m
CONFIG_USB_GSPCA_SONIXJ=m
CONFIG_USB_GSPCA_SPCA500=m
CONFIG_USB_GSPCA_SPCA501=m
CONFIG_USB_GSPCA_SPCA505=m
CONFIG_USB_GSPCA_SPCA506=m
CONFIG_USB_GSPCA_SPCA508=m
CONFIG_USB_GSPCA_SPCA561=m
CONFIG_USB_GSPCA_SPCA1528=m
CONFIG_USB_GSPCA_SQ905=m
CONFIG_USB_GSPCA_SQ905C=m
CONFIG_USB_GSPCA_SQ930X=m
CONFIG_USB_GSPCA_STK014=m
# CONFIG_USB_GSPCA_STK1135 is not set
CONFIG_USB_GSPCA_STV0680=m
CONFIG_USB_GSPCA_SUNPLUS=m
CONFIG_USB_GSPCA_T613=m
CONFIG_USB_GSPCA_TOPRO=m
# CONFIG_USB_GSPCA_TOUPTEK is not set
CONFIG_USB_GSPCA_TV8532=m
CONFIG_USB_GSPCA_VC032X=m
CONFIG_USB_GSPCA_VICAM=m
CONFIG_USB_GSPCA_XIRLINK_CIT=m
CONFIG_USB_GSPCA_ZC3XX=m
CONFIG_USB_PWC=m
# CONFIG_USB_PWC_DEBUG is not set
CONFIG_USB_PWC_INPUT_EVDEV=y
# CONFIG_VIDEO_CPIA2 is not set
CONFIG_USB_ZR364XX=m
CONFIG_USB_STKWEBCAM=m
CONFIG_USB_S2255=m
# CONFIG_VIDEO_USBTV is not set

#
# Analog TV USB devices
#
CONFIG_VIDEO_PVRUSB2=m
CONFIG_VIDEO_PVRUSB2_SYSFS=y
CONFIG_VIDEO_PVRUSB2_DVB=y
# CONFIG_VIDEO_PVRUSB2_DEBUGIFC is not set
CONFIG_VIDEO_HDPVR=m
# CONFIG_VIDEO_STK1160_COMMON is not set
# CONFIG_VIDEO_GO7007 is not set

#
# Analog/digital TV USB devices
#
CONFIG_VIDEO_AU0828=m
CONFIG_VIDEO_AU0828_V4L2=y
# CONFIG_VIDEO_AU0828_RC is not set
CONFIG_VIDEO_CX231XX=m
CONFIG_VIDEO_CX231XX_RC=y
CONFIG_VIDEO_CX231XX_ALSA=m
CONFIG_VIDEO_CX231XX_DVB=m
CONFIG_VIDEO_TM6000=m
CONFIG_VIDEO_TM6000_ALSA=m
CONFIG_VIDEO_TM6000_DVB=m

#
# Digital TV USB devices
#
CONFIG_DVB_USB=m
# CONFIG_DVB_USB_DEBUG is not set
CONFIG_DVB_USB_DIB3000MC=m
CONFIG_DVB_USB_A800=m
CONFIG_DVB_USB_DIBUSB_MB=m
# CONFIG_DVB_USB_DIBUSB_MB_FAULTY is not set
CONFIG_DVB_USB_DIBUSB_MC=m
CONFIG_DVB_USB_DIB0700=m
CONFIG_DVB_USB_UMT_010=m
CONFIG_DVB_USB_CXUSB=m
# CONFIG_DVB_USB_CXUSB_ANALOG is not set
CONFIG_DVB_USB_M920X=m
CONFIG_DVB_USB_DIGITV=m
CONFIG_DVB_USB_VP7045=m
CONFIG_DVB_USB_VP702X=m
CONFIG_DVB_USB_GP8PSK=m
CONFIG_DVB_USB_NOVA_T_USB2=m
CONFIG_DVB_USB_TTUSB2=m
CONFIG_DVB_USB_DTT200U=m
CONFIG_DVB_USB_OPERA1=m
CONFIG_DVB_USB_AF9005=m
CONFIG_DVB_USB_AF9005_REMOTE=m
CONFIG_DVB_USB_PCTV452E=m
CONFIG_DVB_USB_DW2102=m
CONFIG_DVB_USB_CINERGY_T2=m
CONFIG_DVB_USB_DTV5100=m
CONFIG_DVB_USB_AZ6027=m
CONFIG_DVB_USB_TECHNISAT_USB2=m
CONFIG_DVB_USB_V2=m
CONFIG_DVB_USB_AF9015=m
CONFIG_DVB_USB_AF9035=m
CONFIG_DVB_USB_ANYSEE=m
CONFIG_DVB_USB_AU6610=m
CONFIG_DVB_USB_AZ6007=m
CONFIG_DVB_USB_CE6230=m
CONFIG_DVB_USB_EC168=m
CONFIG_DVB_USB_GL861=m
CONFIG_DVB_USB_LME2510=m
CONFIG_DVB_USB_MXL111SF=m
CONFIG_DVB_USB_RTL28XXU=m
# CONFIG_DVB_USB_DVBSKY is not set
# CONFIG_DVB_USB_ZD1301 is not set
CONFIG_DVB_TTUSB_BUDGET=m
CONFIG_DVB_TTUSB_DEC=m
CONFIG_SMS_USB_DRV=m
CONFIG_DVB_B2C2_FLEXCOP_USB=m
# CONFIG_DVB_B2C2_FLEXCOP_USB_DEBUG is not set
# CONFIG_DVB_AS102 is not set

#
# Webcam, TV (analog/digital) USB devices
#
CONFIG_VIDEO_EM28XX=m
# CONFIG_VIDEO_EM28XX_V4L2 is not set
CONFIG_VIDEO_EM28XX_ALSA=m
CONFIG_VIDEO_EM28XX_DVB=m
CONFIG_VIDEO_EM28XX_RC=m
CONFIG_MEDIA_PCI_SUPPORT=y

#
# Media capture support
#
# CONFIG_VIDEO_MEYE is not set
# CONFIG_VIDEO_SOLO6X10 is not set
# CONFIG_VIDEO_TW5864 is not set
# CONFIG_VIDEO_TW68 is not set
# CONFIG_VIDEO_TW686X is not set

#
# Media capture/analog TV support
#
CONFIG_VIDEO_IVTV=m
# CONFIG_VIDEO_IVTV_DEPRECATED_IOCTLS is not set
# CONFIG_VIDEO_IVTV_ALSA is not set
CONFIG_VIDEO_FB_IVTV=m
# CONFIG_VIDEO_FB_IVTV_FORCE_PAT is not set
# CONFIG_VIDEO_HEXIUM_GEMINI is not set
# CONFIG_VIDEO_HEXIUM_ORION is not set
# CONFIG_VIDEO_MXB is not set
# CONFIG_VIDEO_DT3155 is not set

#
# Media capture/analog/hybrid TV support
#
CONFIG_VIDEO_CX18=m
CONFIG_VIDEO_CX18_ALSA=m
CONFIG_VIDEO_CX23885=m
CONFIG_MEDIA_ALTERA_CI=m
# CONFIG_VIDEO_CX25821 is not set
CONFIG_VIDEO_CX88=m
CONFIG_VIDEO_CX88_ALSA=m
CONFIG_VIDEO_CX88_BLACKBIRD=m
CONFIG_VIDEO_CX88_DVB=m
# CONFIG_VIDEO_CX88_ENABLE_VP3054 is not set
CONFIG_VIDEO_CX88_MPEG=m
CONFIG_VIDEO_BT848=m
CONFIG_DVB_BT8XX=m
CONFIG_VIDEO_SAA7134=m
CONFIG_VIDEO_SAA7134_ALSA=m
CONFIG_VIDEO_SAA7134_RC=y
CONFIG_VIDEO_SAA7134_DVB=m
CONFIG_VIDEO_SAA7164=m

#
# Media digital TV PCI Adapters
#
CONFIG_DVB_AV7110_IR=y
CONFIG_DVB_AV7110=m
CONFIG_DVB_AV7110_OSD=y
CONFIG_DVB_BUDGET_CORE=m
CONFIG_DVB_BUDGET=m
CONFIG_DVB_BUDGET_CI=m
CONFIG_DVB_BUDGET_AV=m
CONFIG_DVB_BUDGET_PATCH=m
CONFIG_DVB_B2C2_FLEXCOP_PCI=m
# CONFIG_DVB_B2C2_FLEXCOP_PCI_DEBUG is not set
CONFIG_DVB_PLUTO2=m
CONFIG_DVB_DM1105=m
CONFIG_DVB_PT1=m
# CONFIG_DVB_PT3 is not set
CONFIG_MANTIS_CORE=m
CONFIG_DVB_MANTIS=m
CONFIG_DVB_HOPPER=m
CONFIG_DVB_NGENE=m
CONFIG_DVB_DDBRIDGE=m
# CONFIG_DVB_DDBRIDGE_MSIENABLE is not set
# CONFIG_DVB_SMIPCIE is not set
# CONFIG_DVB_NETUP_UNIDVB is not set
# CONFIG_VIDEO_IPU3_CIO2 is not set
CONFIG_RADIO_ADAPTERS=y
CONFIG_RADIO_TEA575X=m
# CONFIG_RADIO_SI470X is not set
# CONFIG_RADIO_SI4713 is not set
# CONFIG_USB_MR800 is not set
# CONFIG_USB_DSBR is not set
# CONFIG_RADIO_MAXIRADIO is not set
# CONFIG_RADIO_SHARK is not set
# CONFIG_RADIO_SHARK2 is not set
# CONFIG_USB_KEENE is not set
# CONFIG_USB_RAREMONO is not set
# CONFIG_USB_MA901 is not set
# CONFIG_RADIO_TEA5764 is not set
# CONFIG_RADIO_SAA7706H is not set
# CONFIG_RADIO_TEF6862 is not set
# CONFIG_RADIO_WL1273 is not set
CONFIG_MEDIA_COMMON_OPTIONS=y

#
# common driver options
#
CONFIG_VIDEO_CX2341X=m
CONFIG_VIDEO_TVEEPROM=m
CONFIG_CYPRESS_FIRMWARE=m
CONFIG_VIDEOBUF2_CORE=m
CONFIG_VIDEOBUF2_V4L2=m
CONFIG_VIDEOBUF2_MEMOPS=m
CONFIG_VIDEOBUF2_VMALLOC=m
CONFIG_VIDEOBUF2_DMA_SG=m
CONFIG_VIDEOBUF2_DVB=m
CONFIG_DVB_B2C2_FLEXCOP=m
CONFIG_VIDEO_SAA7146=m
CONFIG_VIDEO_SAA7146_VV=m
CONFIG_SMS_SIANO_MDTV=m
CONFIG_SMS_SIANO_RC=y

#
# FireWire (IEEE 1394) Adapters
#
CONFIG_DVB_FIREDTV=m
CONFIG_DVB_FIREDTV_INPUT=y
# end of Media drivers

CONFIG_MEDIA_HIDE_ANCILLARY_SUBDRV=y

#
# Media ancillary drivers
#
CONFIG_MEDIA_ATTACH=y

#
# IR I2C driver auto-selected by 'Autoselect ancillary drivers'
#
CONFIG_VIDEO_IR_I2C=m

#
# audio, video and radio I2C drivers auto-selected by 'Autoselect ancillary drivers'
#
CONFIG_VIDEO_TVAUDIO=m
CONFIG_VIDEO_TDA7432=m
CONFIG_VIDEO_MSP3400=m
CONFIG_VIDEO_CS3308=m
CONFIG_VIDEO_CS5345=m
CONFIG_VIDEO_CS53L32A=m
CONFIG_VIDEO_WM8775=m
CONFIG_VIDEO_WM8739=m
CONFIG_VIDEO_VP27SMPX=m
CONFIG_VIDEO_SAA6588=m
CONFIG_VIDEO_SAA711X=m

#
# Video and audio decoders
#
CONFIG_VIDEO_SAA717X=m
CONFIG_VIDEO_CX25840=m
CONFIG_VIDEO_SAA7127=m
CONFIG_VIDEO_UPD64031A=m
CONFIG_VIDEO_UPD64083=m
CONFIG_VIDEO_SAA6752HS=m
CONFIG_VIDEO_M52790=m

#
# Camera sensor devices
#
# CONFIG_VIDEO_HI556 is not set
# CONFIG_VIDEO_IMX214 is not set
# CONFIG_VIDEO_IMX219 is not set
# CONFIG_VIDEO_IMX258 is not set
# CONFIG_VIDEO_IMX274 is not set
# CONFIG_VIDEO_IMX290 is not set
# CONFIG_VIDEO_IMX319 is not set
# CONFIG_VIDEO_IMX355 is not set
# CONFIG_VIDEO_OV02A10 is not set
# CONFIG_VIDEO_OV2640 is not set
# CONFIG_VIDEO_OV2659 is not set
# CONFIG_VIDEO_OV2680 is not set
# CONFIG_VIDEO_OV2685 is not set
# CONFIG_VIDEO_OV2740 is not set
# CONFIG_VIDEO_OV5647 is not set
# CONFIG_VIDEO_OV6650 is not set
# CONFIG_VIDEO_OV5670 is not set
# CONFIG_VIDEO_OV5675 is not set
# CONFIG_VIDEO_OV5695 is not set
# CONFIG_VIDEO_OV7251 is not set
# CONFIG_VIDEO_OV772X is not set
# CONFIG_VIDEO_OV7640 is not set
# CONFIG_VIDEO_OV7670 is not set
# CONFIG_VIDEO_OV7740 is not set
# CONFIG_VIDEO_OV8856 is not set
# CONFIG_VIDEO_OV9640 is not set
# CONFIG_VIDEO_OV9650 is not set
# CONFIG_VIDEO_OV9734 is not set
# CONFIG_VIDEO_OV13858 is not set
# CONFIG_VIDEO_VS6624 is not set
# CONFIG_VIDEO_MT9M001 is not set
# CONFIG_VIDEO_MT9M032 is not set
# CONFIG_VIDEO_MT9M111 is not set
# CONFIG_VIDEO_MT9P031 is not set
# CONFIG_VIDEO_MT9T001 is not set
# CONFIG_VIDEO_MT9T112 is not set
# CONFIG_VIDEO_MT9V011 is not set
# CONFIG_VIDEO_MT9V032 is not set
# CONFIG_VIDEO_MT9V111 is not set
# CONFIG_VIDEO_SR030PC30 is not set
# CONFIG_VIDEO_NOON010PC30 is not set
# CONFIG_VIDEO_M5MOLS is not set
# CONFIG_VIDEO_RDACM20 is not set
# CONFIG_VIDEO_RJ54N1 is not set
# CONFIG_VIDEO_S5K6AA is not set
# CONFIG_VIDEO_S5K6A3 is not set
# CONFIG_VIDEO_S5K4ECGX is not set
# CONFIG_VIDEO_S5K5BAF is not set
# CONFIG_VIDEO_CCS is not set
# CONFIG_VIDEO_ET8EK8 is not set
# CONFIG_VIDEO_S5C73M3 is not set
# end of Camera sensor devices

#
# Lens drivers
#
# CONFIG_VIDEO_AD5820 is not set
# CONFIG_VIDEO_AK7375 is not set
# CONFIG_VIDEO_DW9714 is not set
# CONFIG_VIDEO_DW9768 is not set
# CONFIG_VIDEO_DW9807_VCM is not set
# end of Lens drivers

#
# Flash devices
#
# CONFIG_VIDEO_ADP1653 is not set
# CONFIG_VIDEO_LM3560 is not set
# CONFIG_VIDEO_LM3646 is not set
# end of Flash devices

#
# SPI I2C drivers auto-selected by 'Autoselect ancillary drivers'
#

#
# Media SPI Adapters
#
# CONFIG_CXD2880_SPI_DRV is not set
# end of Media SPI Adapters

CONFIG_MEDIA_TUNER=m

#
# Tuner drivers auto-selected by 'Autoselect ancillary drivers'
#
CONFIG_MEDIA_TUNER_SIMPLE=m
CONFIG_MEDIA_TUNER_TDA18250=m
CONFIG_MEDIA_TUNER_TDA8290=m
CONFIG_MEDIA_TUNER_TDA827X=m
CONFIG_MEDIA_TUNER_TDA18271=m
CONFIG_MEDIA_TUNER_TDA9887=m
CONFIG_MEDIA_TUNER_TEA5761=m
CONFIG_MEDIA_TUNER_TEA5767=m
CONFIG_MEDIA_TUNER_MT20XX=m
CONFIG_MEDIA_TUNER_MT2060=m
CONFIG_MEDIA_TUNER_MT2063=m
CONFIG_MEDIA_TUNER_MT2266=m
CONFIG_MEDIA_TUNER_MT2131=m
CONFIG_MEDIA_TUNER_QT1010=m
CONFIG_MEDIA_TUNER_XC2028=m
CONFIG_MEDIA_TUNER_XC5000=m
CONFIG_MEDIA_TUNER_XC4000=m
CONFIG_MEDIA_TUNER_MXL5005S=m
CONFIG_MEDIA_TUNER_MXL5007T=m
CONFIG_MEDIA_TUNER_MC44S803=m
CONFIG_MEDIA_TUNER_MAX2165=m
CONFIG_MEDIA_TUNER_TDA18218=m
CONFIG_MEDIA_TUNER_FC0011=m
CONFIG_MEDIA_TUNER_FC0012=m
CONFIG_MEDIA_TUNER_FC0013=m
CONFIG_MEDIA_TUNER_TDA18212=m
CONFIG_MEDIA_TUNER_E4000=m
CONFIG_MEDIA_TUNER_FC2580=m
CONFIG_MEDIA_TUNER_M88RS6000T=m
CONFIG_MEDIA_TUNER_TUA9001=m
CONFIG_MEDIA_TUNER_SI2157=m
CONFIG_MEDIA_TUNER_IT913X=m
CONFIG_MEDIA_TUNER_R820T=m
CONFIG_MEDIA_TUNER_QM1D1C0042=m
CONFIG_MEDIA_TUNER_QM1D1B0004=m

#
# DVB Frontend drivers auto-selected by 'Autoselect ancillary drivers'
#

#
# Multistandard (satellite) frontends
#
CONFIG_DVB_STB0899=m
CONFIG_DVB_STB6100=m
CONFIG_DVB_STV090x=m
CONFIG_DVB_STV0910=m
CONFIG_DVB_STV6110x=m
CONFIG_DVB_STV6111=m
CONFIG_DVB_MXL5XX=m
CONFIG_DVB_M88DS3103=m

#
# Multistandard (cable + terrestrial) frontends
#
CONFIG_DVB_DRXK=m
CONFIG_DVB_TDA18271C2DD=m
CONFIG_DVB_SI2165=m
CONFIG_DVB_MN88472=m
CONFIG_DVB_MN88473=m

#
# DVB-S (satellite) frontends
#
CONFIG_DVB_CX24110=m
CONFIG_DVB_CX24123=m
CONFIG_DVB_MT312=m
CONFIG_DVB_ZL10036=m
CONFIG_DVB_ZL10039=m
CONFIG_DVB_S5H1420=m
CONFIG_DVB_STV0288=m
CONFIG_DVB_STB6000=m
CONFIG_DVB_STV0299=m
CONFIG_DVB_STV6110=m
CONFIG_DVB_STV0900=m
CONFIG_DVB_TDA8083=m
CONFIG_DVB_TDA10086=m
CONFIG_DVB_TDA8261=m
CONFIG_DVB_VES1X93=m
CONFIG_DVB_TUNER_ITD1000=m
CONFIG_DVB_TUNER_CX24113=m
CONFIG_DVB_TDA826X=m
CONFIG_DVB_TUA6100=m
CONFIG_DVB_CX24116=m
CONFIG_DVB_CX24117=m
CONFIG_DVB_CX24120=m
CONFIG_DVB_SI21XX=m
CONFIG_DVB_TS2020=m
CONFIG_DVB_DS3000=m
CONFIG_DVB_MB86A16=m
CONFIG_DVB_TDA10071=m

#
# DVB-T (terrestrial) frontends
#
CONFIG_DVB_SP8870=m
CONFIG_DVB_SP887X=m
CONFIG_DVB_CX22700=m
CONFIG_DVB_CX22702=m
CONFIG_DVB_DRXD=m
CONFIG_DVB_L64781=m
CONFIG_DVB_TDA1004X=m
CONFIG_DVB_NXT6000=m
CONFIG_DVB_MT352=m
CONFIG_DVB_ZL10353=m
CONFIG_DVB_DIB3000MB=m
CONFIG_DVB_DIB3000MC=m
CONFIG_DVB_DIB7000M=m
CONFIG_DVB_DIB7000P=m
CONFIG_DVB_TDA10048=m
CONFIG_DVB_AF9013=m
CONFIG_DVB_EC100=m
CONFIG_DVB_STV0367=m
CONFIG_DVB_CXD2820R=m
CONFIG_DVB_CXD2841ER=m
CONFIG_DVB_RTL2830=m
CONFIG_DVB_RTL2832=m
CONFIG_DVB_SI2168=m
CONFIG_DVB_GP8PSK_FE=m

#
# DVB-C (cable) frontends
#
CONFIG_DVB_VES1820=m
CONFIG_DVB_TDA10021=m
CONFIG_DVB_TDA10023=m
CONFIG_DVB_STV0297=m

#
# ATSC (North American/Korean Terrestrial/Cable DTV) frontends
#
CONFIG_DVB_NXT200X=m
CONFIG_DVB_OR51211=m
CONFIG_DVB_OR51132=m
CONFIG_DVB_BCM3510=m
CONFIG_DVB_LGDT330X=m
CONFIG_DVB_LGDT3305=m
CONFIG_DVB_LGDT3306A=m
CONFIG_DVB_LG2160=m
CONFIG_DVB_S5H1409=m
CONFIG_DVB_AU8522=m
CONFIG_DVB_AU8522_DTV=m
CONFIG_DVB_AU8522_V4L=m
CONFIG_DVB_S5H1411=m

#
# ISDB-T (terrestrial) frontends
#
CONFIG_DVB_S921=m
CONFIG_DVB_DIB8000=m
CONFIG_DVB_MB86A20S=m

#
# ISDB-S (satellite) & ISDB-T (terrestrial) frontends
#
CONFIG_DVB_TC90522=m

#
# Digital terrestrial only tuners/PLL
#
CONFIG_DVB_PLL=m
CONFIG_DVB_TUNER_DIB0070=m
CONFIG_DVB_TUNER_DIB0090=m

#
# SEC control devices for DVB-S
#
CONFIG_DVB_DRX39XYJ=m
CONFIG_DVB_LNBH25=m
CONFIG_DVB_LNBP21=m
CONFIG_DVB_LNBP22=m
CONFIG_DVB_ISL6405=m
CONFIG_DVB_ISL6421=m
CONFIG_DVB_ISL6423=m
CONFIG_DVB_A8293=m
CONFIG_DVB_LGS8GXX=m
CONFIG_DVB_ATBM8830=m
CONFIG_DVB_TDA665x=m
CONFIG_DVB_IX2505V=m
CONFIG_DVB_M88RS2000=m
CONFIG_DVB_AF9033=m

#
# Common Interface (EN50221) controller drivers
#
CONFIG_DVB_CXD2099=m
# end of Media ancillary drivers

#
# Graphics support
#
# CONFIG_AGP is not set
CONFIG_INTEL_GTT=m
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=64
CONFIG_VGA_SWITCHEROO=y
CONFIG_DRM=m
CONFIG_DRM_MIPI_DSI=y
CONFIG_DRM_DP_AUX_CHARDEV=y
# CONFIG_DRM_DEBUG_SELFTEST is not set
CONFIG_DRM_KMS_HELPER=m
CONFIG_DRM_KMS_FB_HELPER=y
CONFIG_DRM_FBDEV_EMULATION=y
CONFIG_DRM_FBDEV_OVERALLOC=100
CONFIG_DRM_LOAD_EDID_FIRMWARE=y
# CONFIG_DRM_DP_CEC is not set
CONFIG_DRM_TTM=m
CONFIG_DRM_VRAM_HELPER=m
CONFIG_DRM_TTM_HELPER=m
CONFIG_DRM_GEM_SHMEM_HELPER=y
CONFIG_DRM_SCHED=m

#
# I2C encoder or helper chips
#
CONFIG_DRM_I2C_CH7006=m
CONFIG_DRM_I2C_SIL164=m
# CONFIG_DRM_I2C_NXP_TDA998X is not set
# CONFIG_DRM_I2C_NXP_TDA9950 is not set
# end of I2C encoder or helper chips

#
# ARM devices
#
# end of ARM devices

CONFIG_DRM_RADEON=m
CONFIG_DRM_RADEON_USERPTR=y
CONFIG_DRM_AMDGPU=m
# CONFIG_DRM_AMDGPU_SI is not set
# CONFIG_DRM_AMDGPU_CIK is not set
# CONFIG_DRM_AMDGPU_USERPTR is not set
# CONFIG_DRM_AMDGPU_GART_DEBUGFS is not set

#
# ACP (Audio CoProcessor) Configuration
#
CONFIG_DRM_AMD_ACP=y
# end of ACP (Audio CoProcessor) Configuration

#
# Display Engine Configuration
#
CONFIG_DRM_AMD_DC=y
CONFIG_DRM_AMD_DC_DCN=y
# CONFIG_DRM_AMD_DC_HDCP is not set
# CONFIG_DRM_AMD_DC_SI is not set
# CONFIG_DEBUG_KERNEL_DC is not set
# end of Display Engine Configuration

CONFIG_HSA_AMD=y
CONFIG_DRM_NOUVEAU=m
# CONFIG_NOUVEAU_LEGACY_CTX_SUPPORT is not set
CONFIG_NOUVEAU_DEBUG=5
CONFIG_NOUVEAU_DEBUG_DEFAULT=3
# CONFIG_NOUVEAU_DEBUG_MMU is not set
# CONFIG_NOUVEAU_DEBUG_PUSH is not set
CONFIG_DRM_NOUVEAU_BACKLIGHT=y
CONFIG_DRM_I915=m
CONFIG_DRM_I915_FORCE_PROBE=""
CONFIG_DRM_I915_CAPTURE_ERROR=y
CONFIG_DRM_I915_COMPRESS_ERROR=y
CONFIG_DRM_I915_USERPTR=y
CONFIG_DRM_I915_GVT=y
CONFIG_DRM_I915_GVT_KVMGT=m
CONFIG_DRM_I915_FENCE_TIMEOUT=10000
CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND=250
CONFIG_DRM_I915_HEARTBEAT_INTERVAL=2500
CONFIG_DRM_I915_PREEMPT_TIMEOUT=640
CONFIG_DRM_I915_MAX_REQUEST_BUSYWAIT=8000
CONFIG_DRM_I915_STOP_TIMEOUT=100
CONFIG_DRM_I915_TIMESLICE_DURATION=1
# CONFIG_DRM_VGEM is not set
CONFIG_DRM_VKMS=m
CONFIG_DRM_VMWGFX=m
CONFIG_DRM_VMWGFX_FBCON=y
CONFIG_DRM_GMA500=m
CONFIG_DRM_GMA600=y
CONFIG_DRM_GMA3600=y
CONFIG_DRM_UDL=m
CONFIG_DRM_AST=m
CONFIG_DRM_MGAG200=m
CONFIG_DRM_QXL=m
CONFIG_DRM_BOCHS=m
CONFIG_DRM_VIRTIO_GPU=m
CONFIG_DRM_PANEL=y

#
# Display Panels
#
# CONFIG_DRM_PANEL_RASPBERRYPI_TOUCHSCREEN is not set
# end of Display Panels

CONFIG_DRM_BRIDGE=y
CONFIG_DRM_PANEL_BRIDGE=y

#
# Display Interface Bridges
#
# CONFIG_DRM_ANALOGIX_ANX78XX is not set
# end of Display Interface Bridges

# CONFIG_DRM_ETNAVIV is not set
CONFIG_DRM_CIRRUS_QEMU=m
# CONFIG_DRM_GM12U320 is not set
# CONFIG_TINYDRM_HX8357D is not set
# CONFIG_TINYDRM_ILI9225 is not set
# CONFIG_TINYDRM_ILI9341 is not set
# CONFIG_TINYDRM_ILI9486 is not set
# CONFIG_TINYDRM_MI0283QT is not set
# CONFIG_TINYDRM_REPAPER is not set
# CONFIG_TINYDRM_ST7586 is not set
# CONFIG_TINYDRM_ST7735R is not set
# CONFIG_DRM_XEN is not set
# CONFIG_DRM_VBOXVIDEO is not set
# CONFIG_DRM_LEGACY is not set
CONFIG_DRM_PANEL_ORIENTATION_QUIRKS=y

#
# Frame buffer Devices
#
CONFIG_FB_CMDLINE=y
CONFIG_FB_NOTIFY=y
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_BOOT_VESA_SUPPORT=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
CONFIG_FB_SYS_FILLRECT=m
CONFIG_FB_SYS_COPYAREA=m
CONFIG_FB_SYS_IMAGEBLIT=m
# CONFIG_FB_FOREIGN_ENDIAN is not set
CONFIG_FB_SYS_FOPS=m
CONFIG_FB_DEFERRED_IO=y
# CONFIG_FB_MODE_HELPERS is not set
CONFIG_FB_TILEBLITTING=y

#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_UVESA is not set
CONFIG_FB_VESA=y
CONFIG_FB_EFI=y
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_OPENCORES is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_I740 is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_SM501 is not set
# CONFIG_FB_SMSCUFX is not set
# CONFIG_FB_UDL is not set
# CONFIG_FB_IBM_GXT4500 is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_XEN_FBDEV_FRONTEND is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
CONFIG_FB_HYPERV=m
# CONFIG_FB_SIMPLE is not set
# CONFIG_FB_SM712 is not set
# end of Frame buffer Devices

#
# Backlight & LCD device support
#
CONFIG_LCD_CLASS_DEVICE=m
# CONFIG_LCD_L4F00242T03 is not set
# CONFIG_LCD_LMS283GF05 is not set
# CONFIG_LCD_LTV350QV is not set
# CONFIG_LCD_ILI922X is not set
# CONFIG_LCD_ILI9320 is not set
# CONFIG_LCD_TDO24M is not set
# CONFIG_LCD_VGG2432A4 is not set
CONFIG_LCD_PLATFORM=m
# CONFIG_LCD_AMS369FG06 is not set
# CONFIG_LCD_LMS501KF03 is not set
# CONFIG_LCD_HX8357 is not set
# CONFIG_LCD_OTM3225A is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
# CONFIG_BACKLIGHT_KTD253 is not set
# CONFIG_BACKLIGHT_PWM is not set
CONFIG_BACKLIGHT_APPLE=m
# CONFIG_BACKLIGHT_QCOM_WLED is not set
# CONFIG_BACKLIGHT_SAHARA is not set
# CONFIG_BACKLIGHT_ADP8860 is not set
# CONFIG_BACKLIGHT_ADP8870 is not set
# CONFIG_BACKLIGHT_LM3630A is not set
# CONFIG_BACKLIGHT_LM3639 is not set
CONFIG_BACKLIGHT_LP855X=m
# CONFIG_BACKLIGHT_GPIO is not set
# CONFIG_BACKLIGHT_LV5207LP is not set
# CONFIG_BACKLIGHT_BD6107 is not set
# CONFIG_BACKLIGHT_ARCXCNN is not set
# end of Backlight & LCD device support

CONFIG_HDMI=y

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_DUMMY_CONSOLE_COLUMNS=80
CONFIG_DUMMY_CONSOLE_ROWS=25
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
# CONFIG_FRAMEBUFFER_CONSOLE_DEFERRED_TAKEOVER is not set
# end of Console display driver support

CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not set
CONFIG_LOGO_LINUX_CLUT224=y
# end of Graphics support

CONFIG_SOUND=m
CONFIG_SOUND_OSS_CORE=y
CONFIG_SOUND_OSS_CORE_PRECLAIM=y
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_PCM_ELD=y
CONFIG_SND_HWDEP=m
CONFIG_SND_SEQ_DEVICE=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_COMPRESS_OFFLOAD=m
CONFIG_SND_JACK=y
CONFIG_SND_JACK_INPUT_DEV=y
CONFIG_SND_OSSEMUL=y
# CONFIG_SND_MIXER_OSS is not set
# CONFIG_SND_PCM_OSS is not set
CONFIG_SND_PCM_TIMER=y
CONFIG_SND_HRTIMER=m
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_MAX_CARDS=32
# CONFIG_SND_SUPPORT_OLD_API is not set
CONFIG_SND_PROC_FS=y
CONFIG_SND_VERBOSE_PROCFS=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_VMASTER=y
CONFIG_SND_DMA_SGBUF=y
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_SEQUENCER_OSS=m
CONFIG_SND_SEQ_HRTIMER_DEFAULT=y
CONFIG_SND_SEQ_MIDI_EVENT=m
CONFIG_SND_SEQ_MIDI=m
CONFIG_SND_SEQ_MIDI_EMUL=m
CONFIG_SND_SEQ_VIRMIDI=m
CONFIG_SND_MPU401_UART=m
CONFIG_SND_OPL3_LIB=m
CONFIG_SND_OPL3_LIB_SEQ=m
CONFIG_SND_VX_LIB=m
CONFIG_SND_AC97_CODEC=m
CONFIG_SND_DRIVERS=y
CONFIG_SND_PCSP=m
CONFIG_SND_DUMMY=m
CONFIG_SND_ALOOP=m
CONFIG_SND_VIRMIDI=m
CONFIG_SND_MTPAV=m
# CONFIG_SND_MTS64 is not set
# CONFIG_SND_SERIAL_U16550 is not set
CONFIG_SND_MPU401=m
# CONFIG_SND_PORTMAN2X4 is not set
CONFIG_SND_AC97_POWER_SAVE=y
CONFIG_SND_AC97_POWER_SAVE_DEFAULT=5
CONFIG_SND_PCI=y
CONFIG_SND_AD1889=m
# CONFIG_SND_ALS300 is not set
# CONFIG_SND_ALS4000 is not set
CONFIG_SND_ALI5451=m
CONFIG_SND_ASIHPI=m
CONFIG_SND_ATIIXP=m
CONFIG_SND_ATIIXP_MODEM=m
CONFIG_SND_AU8810=m
CONFIG_SND_AU8820=m
CONFIG_SND_AU8830=m
# CONFIG_SND_AW2 is not set
# CONFIG_SND_AZT3328 is not set
CONFIG_SND_BT87X=m
# CONFIG_SND_BT87X_OVERCLOCK is not set
CONFIG_SND_CA0106=m
CONFIG_SND_CMIPCI=m
CONFIG_SND_OXYGEN_LIB=m
CONFIG_SND_OXYGEN=m
# CONFIG_SND_CS4281 is not set
CONFIG_SND_CS46XX=m
CONFIG_SND_CS46XX_NEW_DSP=y
CONFIG_SND_CTXFI=m
CONFIG_SND_DARLA20=m
CONFIG_SND_GINA20=m
CONFIG_SND_LAYLA20=m
CONFIG_SND_DARLA24=m
CONFIG_SND_GINA24=m
CONFIG_SND_LAYLA24=m
CONFIG_SND_MONA=m
CONFIG_SND_MIA=m
CONFIG_SND_ECHO3G=m
CONFIG_SND_INDIGO=m
CONFIG_SND_INDIGOIO=m
CONFIG_SND_INDIGODJ=m
CONFIG_SND_INDIGOIOX=m
CONFIG_SND_INDIGODJX=m
CONFIG_SND_EMU10K1=m
CONFIG_SND_EMU10K1_SEQ=m
CONFIG_SND_EMU10K1X=m
CONFIG_SND_ENS1370=m
CONFIG_SND_ENS1371=m
# CONFIG_SND_ES1938 is not set
CONFIG_SND_ES1968=m
CONFIG_SND_ES1968_INPUT=y
CONFIG_SND_ES1968_RADIO=y
# CONFIG_SND_FM801 is not set
CONFIG_SND_HDSP=m
CONFIG_SND_HDSPM=m
CONFIG_SND_ICE1712=m
CONFIG_SND_ICE1724=m
CONFIG_SND_INTEL8X0=m
CONFIG_SND_INTEL8X0M=m
CONFIG_SND_KORG1212=m
CONFIG_SND_LOLA=m
CONFIG_SND_LX6464ES=m
CONFIG_SND_MAESTRO3=m
CONFIG_SND_MAESTRO3_INPUT=y
CONFIG_SND_MIXART=m
# CONFIG_SND_NM256 is not set
CONFIG_SND_PCXHR=m
# CONFIG_SND_RIPTIDE is not set
CONFIG_SND_RME32=m
CONFIG_SND_RME96=m
CONFIG_SND_RME9652=m
# CONFIG_SND_SONICVIBES is not set
CONFIG_SND_TRIDENT=m
CONFIG_SND_VIA82XX=m
CONFIG_SND_VIA82XX_MODEM=m
CONFIG_SND_VIRTUOSO=m
CONFIG_SND_VX222=m
# CONFIG_SND_YMFPCI is not set

#
# HD-Audio
#
CONFIG_SND_HDA=m
CONFIG_SND_HDA_GENERIC_LEDS=y
CONFIG_SND_HDA_INTEL=m
CONFIG_SND_HDA_HWDEP=y
CONFIG_SND_HDA_RECONFIG=y
CONFIG_SND_HDA_INPUT_BEEP=y
CONFIG_SND_HDA_INPUT_BEEP_MODE=0
CONFIG_SND_HDA_PATCH_LOADER=y
CONFIG_SND_HDA_CODEC_REALTEK=m
CONFIG_SND_HDA_CODEC_ANALOG=m
CONFIG_SND_HDA_CODEC_SIGMATEL=m
CONFIG_SND_HDA_CODEC_VIA=m
CONFIG_SND_HDA_CODEC_HDMI=m
CONFIG_SND_HDA_CODEC_CIRRUS=m
CONFIG_SND_HDA_CODEC_CONEXANT=m
CONFIG_SND_HDA_CODEC_CA0110=m
CONFIG_SND_HDA_CODEC_CA0132=m
CONFIG_SND_HDA_CODEC_CA0132_DSP=y
CONFIG_SND_HDA_CODEC_CMEDIA=m
CONFIG_SND_HDA_CODEC_SI3054=m
CONFIG_SND_HDA_GENERIC=m
CONFIG_SND_HDA_POWER_SAVE_DEFAULT=0
# CONFIG_SND_HDA_INTEL_HDMI_SILENT_STREAM is not set
# end of HD-Audio

CONFIG_SND_HDA_CORE=m
CONFIG_SND_HDA_DSP_LOADER=y
CONFIG_SND_HDA_COMPONENT=y
CONFIG_SND_HDA_I915=y
CONFIG_SND_HDA_EXT_CORE=m
CONFIG_SND_HDA_PREALLOC_SIZE=512
CONFIG_SND_INTEL_NHLT=y
CONFIG_SND_INTEL_DSP_CONFIG=m
# CONFIG_SND_SPI is not set
CONFIG_SND_USB=y
CONFIG_SND_USB_AUDIO=m
CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER=y
CONFIG_SND_USB_UA101=m
CONFIG_SND_USB_USX2Y=m
CONFIG_SND_USB_CAIAQ=m
CONFIG_SND_USB_CAIAQ_INPUT=y
CONFIG_SND_USB_US122L=m
CONFIG_SND_USB_6FIRE=m
CONFIG_SND_USB_HIFACE=m
CONFIG_SND_BCD2000=m
CONFIG_SND_USB_LINE6=m
CONFIG_SND_USB_POD=m
CONFIG_SND_USB_PODHD=m
CONFIG_SND_USB_TONEPORT=m
CONFIG_SND_USB_VARIAX=m
CONFIG_SND_FIREWIRE=y
CONFIG_SND_FIREWIRE_LIB=m
CONFIG_SND_DICE=m
CONFIG_SND_OXFW=m
CONFIG_SND_ISIGHT=m
CONFIG_SND_FIREWORKS=m
CONFIG_SND_BEBOB=m
CONFIG_SND_FIREWIRE_DIGI00X=m
CONFIG_SND_FIREWIRE_TASCAM=m
CONFIG_SND_FIREWIRE_MOTU=m
CONFIG_SND_FIREFACE=m
CONFIG_SND_SOC=m
CONFIG_SND_SOC_COMPRESS=y
CONFIG_SND_SOC_TOPOLOGY=y
CONFIG_SND_SOC_ACPI=m
# CONFIG_SND_SOC_ADI is not set
# CONFIG_SND_SOC_AMD_ACP is not set
# CONFIG_SND_SOC_AMD_ACP3x is not set
CONFIG_SND_SOC_AMD_RENOIR=m
CONFIG_SND_SOC_AMD_RENOIR_MACH=m
# CONFIG_SND_ATMEL_SOC is not set
# CONFIG_SND_BCM63XX_I2S_WHISTLER is not set
# CONFIG_SND_DESIGNWARE_I2S is not set

#
# SoC Audio for Freescale CPUs
#

#
# Common SoC Audio options for Freescale CPUs:
#
# CONFIG_SND_SOC_FSL_ASRC is not set
# CONFIG_SND_SOC_FSL_SAI is not set
# CONFIG_SND_SOC_FSL_AUDMIX is not set
# CONFIG_SND_SOC_FSL_SSI is not set
# CONFIG_SND_SOC_FSL_SPDIF is not set
# CONFIG_SND_SOC_FSL_ESAI is not set
# CONFIG_SND_SOC_FSL_MICFIL is not set
# CONFIG_SND_SOC_FSL_XCVR is not set
# CONFIG_SND_SOC_IMX_AUDMUX is not set
# end of SoC Audio for Freescale CPUs

# CONFIG_SND_I2S_HI6210_I2S is not set
# CONFIG_SND_SOC_IMG is not set
CONFIG_SND_SOC_INTEL_SST_TOPLEVEL=y
CONFIG_SND_SOC_INTEL_SST=m
# CONFIG_SND_SOC_INTEL_CATPT is not set
CONFIG_SND_SST_ATOM_HIFI2_PLATFORM=m
# CONFIG_SND_SST_ATOM_HIFI2_PLATFORM_PCI is not set
CONFIG_SND_SST_ATOM_HIFI2_PLATFORM_ACPI=m
CONFIG_SND_SOC_INTEL_SKYLAKE=m
CONFIG_SND_SOC_INTEL_SKL=m
CONFIG_SND_SOC_INTEL_APL=m
CONFIG_SND_SOC_INTEL_KBL=m
CONFIG_SND_SOC_INTEL_GLK=m
CONFIG_SND_SOC_INTEL_CNL=m
CONFIG_SND_SOC_INTEL_CFL=m
CONFIG_SND_SOC_INTEL_CML_H=m
CONFIG_SND_SOC_INTEL_CML_LP=m
CONFIG_SND_SOC_INTEL_SKYLAKE_FAMILY=m
CONFIG_SND_SOC_INTEL_SKYLAKE_SSP_CLK=m
# CONFIG_SND_SOC_INTEL_SKYLAKE_HDAUDIO_CODEC is not set
CONFIG_SND_SOC_INTEL_SKYLAKE_COMMON=m
CONFIG_SND_SOC_ACPI_INTEL_MATCH=m
CONFIG_SND_SOC_INTEL_MACH=y
CONFIG_SND_SOC_INTEL_USER_FRIENDLY_LONG_NAMES=y
CONFIG_SND_SOC_INTEL_BYTCR_RT5640_MACH=m
CONFIG_SND_SOC_INTEL_BYTCR_RT5651_MACH=m
CONFIG_SND_SOC_INTEL_CHT_BSW_RT5672_MACH=m
CONFIG_SND_SOC_INTEL_CHT_BSW_RT5645_MACH=m
CONFIG_SND_SOC_INTEL_CHT_BSW_MAX98090_TI_MACH=m
CONFIG_SND_SOC_INTEL_CHT_BSW_NAU8824_MACH=m
CONFIG_SND_SOC_INTEL_BYT_CHT_CX2072X_MACH=m
CONFIG_SND_SOC_INTEL_BYT_CHT_DA7213_MACH=m
CONFIG_SND_SOC_INTEL_BYT_CHT_ES8316_MACH=m
CONFIG_SND_SOC_INTEL_BYT_CHT_NOCODEC_MACH=m
CONFIG_SND_SOC_INTEL_SKL_RT286_MACH=m
CONFIG_SND_SOC_INTEL_SKL_NAU88L25_SSM4567_MACH=m
CONFIG_SND_SOC_INTEL_SKL_NAU88L25_MAX98357A_MACH=m
CONFIG_SND_SOC_INTEL_DA7219_MAX98357A_GENERIC=m
CONFIG_SND_SOC_INTEL_BXT_DA7219_MAX98357A_COMMON=m
CONFIG_SND_SOC_INTEL_BXT_DA7219_MAX98357A_MACH=m
CONFIG_SND_SOC_INTEL_BXT_RT298_MACH=m
# CONFIG_SND_SOC_INTEL_SOF_WM8804_MACH is not set
CONFIG_SND_SOC_INTEL_KBL_RT5663_MAX98927_MACH=m
CONFIG_SND_SOC_INTEL_KBL_RT5663_RT5514_MAX98927_MACH=m
CONFIG_SND_SOC_INTEL_KBL_DA7219_MAX98357A_MACH=m
CONFIG_SND_SOC_INTEL_KBL_DA7219_MAX98927_MACH=m
CONFIG_SND_SOC_INTEL_KBL_RT5660_MACH=m
CONFIG_SND_SOC_INTEL_GLK_DA7219_MAX98357A_MACH=m
CONFIG_SND_SOC_INTEL_GLK_RT5682_MAX98357A_MACH=m
CONFIG_SND_SOC_INTEL_SKL_HDA_DSP_GENERIC_MACH=m
CONFIG_SND_SOC_INTEL_SOF_RT5682_MACH=m
CONFIG_SND_SOC_INTEL_SOF_PCM512x_MACH=m
CONFIG_SND_SOC_INTEL_CML_LP_DA7219_MAX98357A_MACH=m
CONFIG_SND_SOC_INTEL_SOF_CML_RT1011_RT5682_MACH=m
CONFIG_SND_SOC_INTEL_SOF_DA7219_MAX98373_MACH=m
# CONFIG_SND_SOC_INTEL_EHL_RT5660_MACH is not set
CONFIG_SND_SOC_INTEL_SOUNDWIRE_SOF_MACH=m
# CONFIG_SND_SOC_MTK_BTCVSD is not set
CONFIG_SND_SOC_SOF_TOPLEVEL=y
CONFIG_SND_SOC_SOF_PCI=m
CONFIG_SND_SOC_SOF_ACPI=m
# CONFIG_SND_SOC_SOF_DEBUG_PROBES is not set
CONFIG_SND_SOC_SOF=m
CONFIG_SND_SOC_SOF_PROBE_WORK_QUEUE=y
CONFIG_SND_SOC_SOF_INTEL_TOPLEVEL=y
CONFIG_SND_SOC_SOF_INTEL_ACPI=m
CONFIG_SND_SOC_SOF_INTEL_PCI=m
CONFIG_SND_SOC_SOF_INTEL_HIFI_EP_IPC=m
CONFIG_SND_SOC_SOF_INTEL_ATOM_HIFI_EP=m
CONFIG_SND_SOC_SOF_INTEL_COMMON=m
# CONFIG_SND_SOC_SOF_BAYTRAIL_SUPPORT is not set
# CONFIG_SND_SOC_SOF_BROADWELL_SUPPORT is not set
CONFIG_SND_SOC_SOF_MERRIFIELD_SUPPORT=y
CONFIG_SND_SOC_SOF_MERRIFIELD=m
CONFIG_SND_SOC_SOF_APOLLOLAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_APOLLOLAKE=m
CONFIG_SND_SOC_SOF_GEMINILAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_GEMINILAKE=m
CONFIG_SND_SOC_SOF_CANNONLAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_CANNONLAKE=m
CONFIG_SND_SOC_SOF_COFFEELAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_COFFEELAKE=m
CONFIG_SND_SOC_SOF_ICELAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_ICELAKE=m
CONFIG_SND_SOC_SOF_COMETLAKE=m
CONFIG_SND_SOC_SOF_COMETLAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_COMETLAKE_LP_SUPPORT=y
CONFIG_SND_SOC_SOF_TIGERLAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_TIGERLAKE=m
CONFIG_SND_SOC_SOF_ELKHARTLAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_ELKHARTLAKE=m
CONFIG_SND_SOC_SOF_JASPERLAKE_SUPPORT=y
CONFIG_SND_SOC_SOF_JASPERLAKE=m
# CONFIG_SND_SOC_SOF_ALDERLAKE_SUPPORT is not set
CONFIG_SND_SOC_SOF_HDA_COMMON=m
CONFIG_SND_SOC_SOF_HDA_LINK=y
CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC=y
# CONFIG_SND_SOC_SOF_HDA_ALWAYS_ENABLE_DMI_L1 is not set
CONFIG_SND_SOC_SOF_HDA_LINK_BASELINE=m
CONFIG_SND_SOC_SOF_HDA=m
CONFIG_SND_SOC_SOF_INTEL_SOUNDWIRE_LINK=y
CONFIG_SND_SOC_SOF_INTEL_SOUNDWIRE_LINK_BASELINE=m
CONFIG_SND_SOC_SOF_INTEL_SOUNDWIRE=m
CONFIG_SND_SOC_SOF_XTENSA=m

#
# STMicroelectronics STM32 SOC audio support
#
# end of STMicroelectronics STM32 SOC audio support

# CONFIG_SND_SOC_XILINX_I2S is not set
# CONFIG_SND_SOC_XILINX_AUDIO_FORMATTER is not set
# CONFIG_SND_SOC_XILINX_SPDIF is not set
# CONFIG_SND_SOC_XTFPGA_I2S is not set
# CONFIG_ZX_TDM is not set
CONFIG_SND_SOC_I2C_AND_SPI=m

#
# CODEC drivers
#
# CONFIG_SND_SOC_AC97_CODEC is not set
# CONFIG_SND_SOC_ADAU1372_I2C is not set
# CONFIG_SND_SOC_ADAU1372_SPI is not set
# CONFIG_SND_SOC_ADAU1701 is not set
# CONFIG_SND_SOC_ADAU1761_I2C is not set
# CONFIG_SND_SOC_ADAU1761_SPI is not set
# CONFIG_SND_SOC_ADAU7002 is not set
# CONFIG_SND_SOC_ADAU7118_HW is not set
# CONFIG_SND_SOC_ADAU7118_I2C is not set
# CONFIG_SND_SOC_AK4104 is not set
# CONFIG_SND_SOC_AK4118 is not set
# CONFIG_SND_SOC_AK4458 is not set
# CONFIG_SND_SOC_AK4554 is not set
# CONFIG_SND_SOC_AK4613 is not set
# CONFIG_SND_SOC_AK4642 is not set
# CONFIG_SND_SOC_AK5386 is not set
# CONFIG_SND_SOC_AK5558 is not set
# CONFIG_SND_SOC_ALC5623 is not set
# CONFIG_SND_SOC_BD28623 is not set
# CONFIG_SND_SOC_BT_SCO is not set
# CONFIG_SND_SOC_CS35L32 is not set
# CONFIG_SND_SOC_CS35L33 is not set
# CONFIG_SND_SOC_CS35L34 is not set
# CONFIG_SND_SOC_CS35L35 is not set
# CONFIG_SND_SOC_CS35L36 is not set
# CONFIG_SND_SOC_CS42L42 is not set
# CONFIG_SND_SOC_CS42L51_I2C is not set
# CONFIG_SND_SOC_CS42L52 is not set
# CONFIG_SND_SOC_CS42L56 is not set
# CONFIG_SND_SOC_CS42L73 is not set
# CONFIG_SND_SOC_CS4234 is not set
# CONFIG_SND_SOC_CS4265 is not set
# CONFIG_SND_SOC_CS4270 is not set
# CONFIG_SND_SOC_CS4271_I2C is not set
# CONFIG_SND_SOC_CS4271_SPI is not set
# CONFIG_SND_SOC_CS42XX8_I2C is not set
# CONFIG_SND_SOC_CS43130 is not set
# CONFIG_SND_SOC_CS4341 is not set
# CONFIG_SND_SOC_CS4349 is not set
# CONFIG_SND_SOC_CS53L30 is not set
CONFIG_SND_SOC_CX2072X=m
CONFIG_SND_SOC_DA7213=m
CONFIG_SND_SOC_DA7219=m
CONFIG_SND_SOC_DMIC=m
# CONFIG_SND_SOC_ES7134 is not set
# CONFIG_SND_SOC_ES7241 is not set
CONFIG_SND_SOC_ES8316=m
# CONFIG_SND_SOC_ES8328_I2C is not set
# CONFIG_SND_SOC_ES8328_SPI is not set
# CONFIG_SND_SOC_GTM601 is not set
CONFIG_SND_SOC_HDAC_HDMI=m
CONFIG_SND_SOC_HDAC_HDA=m
# CONFIG_SND_SOC_INNO_RK3036 is not set
# CONFIG_SND_SOC_MAX98088 is not set
CONFIG_SND_SOC_MAX98090=m
CONFIG_SND_SOC_MAX98357A=m
# CONFIG_SND_SOC_MAX98504 is not set
# CONFIG_SND_SOC_MAX9867 is not set
CONFIG_SND_SOC_MAX98927=m
CONFIG_SND_SOC_MAX98373=m
CONFIG_SND_SOC_MAX98373_I2C=m
CONFIG_SND_SOC_MAX98373_SDW=m
CONFIG_SND_SOC_MAX98390=m
# CONFIG_SND_SOC_MAX9860 is not set
# CONFIG_SND_SOC_MSM8916_WCD_DIGITAL is not set
# CONFIG_SND_SOC_PCM1681 is not set
# CONFIG_SND_SOC_PCM1789_I2C is not set
# CONFIG_SND_SOC_PCM179X_I2C is not set
# CONFIG_SND_SOC_PCM179X_SPI is not set
# CONFIG_SND_SOC_PCM186X_I2C is not set
# CONFIG_SND_SOC_PCM186X_SPI is not set
# CONFIG_SND_SOC_PCM3060_I2C is not set
# CONFIG_SND_SOC_PCM3060_SPI is not set
# CONFIG_SND_SOC_PCM3168A_I2C is not set
# CONFIG_SND_SOC_PCM3168A_SPI is not set
# CONFIG_SND_SOC_PCM5102A is not set
CONFIG_SND_SOC_PCM512x=m
CONFIG_SND_SOC_PCM512x_I2C=m
# CONFIG_SND_SOC_PCM512x_SPI is not set
# CONFIG_SND_SOC_RK3328 is not set
CONFIG_SND_SOC_RL6231=m
CONFIG_SND_SOC_RL6347A=m
CONFIG_SND_SOC_RT286=m
CONFIG_SND_SOC_RT298=m
CONFIG_SND_SOC_RT1011=m
CONFIG_SND_SOC_RT1015=m
CONFIG_SND_SOC_RT1308=m
CONFIG_SND_SOC_RT1308_SDW=m
CONFIG_SND_SOC_RT5514=m
CONFIG_SND_SOC_RT5514_SPI=m
# CONFIG_SND_SOC_RT5616 is not set
# CONFIG_SND_SOC_RT5631 is not set
CONFIG_SND_SOC_RT5640=m
CONFIG_SND_SOC_RT5645=m
CONFIG_SND_SOC_RT5651=m
CONFIG_SND_SOC_RT5660=m
CONFIG_SND_SOC_RT5663=m
CONFIG_SND_SOC_RT5670=m
CONFIG_SND_SOC_RT5682=m
CONFIG_SND_SOC_RT5682_I2C=m
CONFIG_SND_SOC_RT5682_SDW=m
CONFIG_SND_SOC_RT700=m
CONFIG_SND_SOC_RT700_SDW=m
CONFIG_SND_SOC_RT711=m
CONFIG_SND_SOC_RT711_SDW=m
CONFIG_SND_SOC_RT715=m
CONFIG_SND_SOC_RT715_SDW=m
# CONFIG_SND_SOC_SGTL5000 is not set
# CONFIG_SND_SOC_SIMPLE_AMPLIFIER is not set
# CONFIG_SND_SOC_SIMPLE_MUX is not set
# CONFIG_SND_SOC_SIRF_AUDIO_CODEC is not set
# CONFIG_SND_SOC_SPDIF is not set
# CONFIG_SND_SOC_SSM2305 is not set
# CONFIG_SND_SOC_SSM2602_SPI is not set
# CONFIG_SND_SOC_SSM2602_I2C is not set
CONFIG_SND_SOC_SSM4567=m
# CONFIG_SND_SOC_STA32X is not set
# CONFIG_SND_SOC_STA350 is not set
# CONFIG_SND_SOC_STI_SAS is not set
# CONFIG_SND_SOC_TAS2552 is not set
# CONFIG_SND_SOC_TAS2562 is not set
# CONFIG_SND_SOC_TAS2764 is not set
# CONFIG_SND_SOC_TAS2770 is not set
# CONFIG_SND_SOC_TAS5086 is not set
# CONFIG_SND_SOC_TAS571X is not set
# CONFIG_SND_SOC_TAS5720 is not set
# CONFIG_SND_SOC_TAS6424 is not set
# CONFIG_SND_SOC_TDA7419 is not set
# CONFIG_SND_SOC_TFA9879 is not set
# CONFIG_SND_SOC_TLV320AIC23_I2C is not set
# CONFIG_SND_SOC_TLV320AIC23_SPI is not set
# CONFIG_SND_SOC_TLV320AIC31XX is not set
# CONFIG_SND_SOC_TLV320AIC32X4_I2C is not set
# CONFIG_SND_SOC_TLV320AIC32X4_SPI is not set
# CONFIG_SND_SOC_TLV320AIC3X is not set
# CONFIG_SND_SOC_TLV320ADCX140 is not set
CONFIG_SND_SOC_TS3A227E=m
# CONFIG_SND_SOC_TSCS42XX is not set
# CONFIG_SND_SOC_TSCS454 is not set
# CONFIG_SND_SOC_UDA1334 is not set
# CONFIG_SND_SOC_WM8510 is not set
# CONFIG_SND_SOC_WM8523 is not set
# CONFIG_SND_SOC_WM8524 is not set
# CONFIG_SND_SOC_WM8580 is not set
# CONFIG_SND_SOC_WM8711 is not set
# CONFIG_SND_SOC_WM8728 is not set
# CONFIG_SND_SOC_WM8731 is not set
# CONFIG_SND_SOC_WM8737 is not set
# CONFIG_SND_SOC_WM8741 is not set
# CONFIG_SND_SOC_WM8750 is not set
# CONFIG_SND_SOC_WM8753 is not set
# CONFIG_SND_SOC_WM8770 is not set
# CONFIG_SND_SOC_WM8776 is not set
# CONFIG_SND_SOC_WM8782 is not set
# CONFIG_SND_SOC_WM8804_I2C is not set
# CONFIG_SND_SOC_WM8804_SPI is not set
# CONFIG_SND_SOC_WM8903 is not set
# CONFIG_SND_SOC_WM8904 is not set
# CONFIG_SND_SOC_WM8960 is not set
# CONFIG_SND_SOC_WM8962 is not set
# CONFIG_SND_SOC_WM8974 is not set
# CONFIG_SND_SOC_WM8978 is not set
# CONFIG_SND_SOC_WM8985 is not set
# CONFIG_SND_SOC_WSA881X is not set
# CONFIG_SND_SOC_ZL38060 is not set
# CONFIG_SND_SOC_ZX_AUD96P22 is not set
# CONFIG_SND_SOC_MAX9759 is not set
# CONFIG_SND_SOC_MT6351 is not set
# CONFIG_SND_SOC_MT6358 is not set
# CONFIG_SND_SOC_MT6660 is not set
# CONFIG_SND_SOC_NAU8315 is not set
# CONFIG_SND_SOC_NAU8540 is not set
# CONFIG_SND_SOC_NAU8810 is not set
# CONFIG_SND_SOC_NAU8822 is not set
CONFIG_SND_SOC_NAU8824=m
CONFIG_SND_SOC_NAU8825=m
# CONFIG_SND_SOC_TPA6130A2 is not set
# CONFIG_SND_SOC_LPASS_WSA_MACRO is not set
# CONFIG_SND_SOC_LPASS_VA_MACRO is not set
# end of CODEC drivers

# CONFIG_SND_SIMPLE_CARD is not set
CONFIG_SND_X86=y
CONFIG_HDMI_LPE_AUDIO=m
CONFIG_SND_SYNTH_EMUX=m
CONFIG_SND_XEN_FRONTEND=m
CONFIG_AC97_BUS=m

#
# HID support
#
CONFIG_HID=y
CONFIG_HID_BATTERY_STRENGTH=y
CONFIG_HIDRAW=y
CONFIG_UHID=m
CONFIG_HID_GENERIC=y

#
# Special HID drivers
#
CONFIG_HID_A4TECH=m
# CONFIG_HID_ACCUTOUCH is not set
CONFIG_HID_ACRUX=m
# CONFIG_HID_ACRUX_FF is not set
CONFIG_HID_APPLE=m
CONFIG_HID_APPLEIR=m
CONFIG_HID_ASUS=m
CONFIG_HID_AUREAL=m
CONFIG_HID_BELKIN=m
CONFIG_HID_BETOP_FF=m
# CONFIG_HID_BIGBEN_FF is not set
CONFIG_HID_CHERRY=m
CONFIG_HID_CHICONY=m
CONFIG_HID_CORSAIR=m
# CONFIG_HID_COUGAR is not set
# CONFIG_HID_MACALLY is not set
CONFIG_HID_PRODIKEYS=m
CONFIG_HID_CMEDIA=m
# CONFIG_HID_CP2112 is not set
# CONFIG_HID_CREATIVE_SB0540 is not set
CONFIG_HID_CYPRESS=m
CONFIG_HID_DRAGONRISE=m
# CONFIG_DRAGONRISE_FF is not set
# CONFIG_HID_EMS_FF is not set
CONFIG_HID_ELAN=m
CONFIG_HID_ELECOM=m
CONFIG_HID_ELO=m
CONFIG_HID_EZKEY=m
CONFIG_HID_GEMBIRD=m
CONFIG_HID_GFRM=m
# CONFIG_HID_GLORIOUS is not set
CONFIG_HID_HOLTEK=m
# CONFIG_HOLTEK_FF is not set
# CONFIG_HID_VIVALDI is not set
CONFIG_HID_GT683R=m
CONFIG_HID_KEYTOUCH=m
CONFIG_HID_KYE=m
CONFIG_HID_UCLOGIC=m
CONFIG_HID_WALTOP=m
# CONFIG_HID_VIEWSONIC is not set
CONFIG_HID_GYRATION=m
CONFIG_HID_ICADE=m
CONFIG_HID_ITE=m
CONFIG_HID_JABRA=m
CONFIG_HID_TWINHAN=m
CONFIG_HID_KENSINGTON=m
CONFIG_HID_LCPOWER=m
CONFIG_HID_LED=m
CONFIG_HID_LENOVO=m
CONFIG_HID_LOGITECH=m
CONFIG_HID_LOGITECH_DJ=m
CONFIG_HID_LOGITECH_HIDPP=m
# CONFIG_LOGITECH_FF is not set
# CONFIG_LOGIRUMBLEPAD2_FF is not set
# CONFIG_LOGIG940_FF is not set
# CONFIG_LOGIWHEELS_FF is not set
CONFIG_HID_MAGICMOUSE=y
# CONFIG_HID_MALTRON is not set
# CONFIG_HID_MAYFLASH is not set
# CONFIG_HID_REDRAGON is not set
CONFIG_HID_MICROSOFT=m
CONFIG_HID_MONTEREY=m
CONFIG_HID_MULTITOUCH=m
CONFIG_HID_NTI=m
CONFIG_HID_NTRIG=y
CONFIG_HID_ORTEK=m
CONFIG_HID_PANTHERLORD=m
# CONFIG_PANTHERLORD_FF is not set
CONFIG_HID_PENMOUNT=m
CONFIG_HID_PETALYNX=m
CONFIG_HID_PICOLCD=m
CONFIG_HID_PICOLCD_FB=y
CONFIG_HID_PICOLCD_BACKLIGHT=y
CONFIG_HID_PICOLCD_LCD=y
CONFIG_HID_PICOLCD_LEDS=y
CONFIG_HID_PICOLCD_CIR=y
CONFIG_HID_PLANTRONICS=m
CONFIG_HID_PRIMAX=m
# CONFIG_HID_RETRODE is not set
CONFIG_HID_ROCCAT=m
CONFIG_HID_SAITEK=m
CONFIG_HID_SAMSUNG=m
CONFIG_HID_SONY=m
CONFIG_SONY_FF=y
CONFIG_HID_SPEEDLINK=m
# CONFIG_HID_STEAM is not set
CONFIG_HID_STEELSERIES=m
CONFIG_HID_SUNPLUS=m
CONFIG_HID_RMI=m
CONFIG_HID_GREENASIA=m
# CONFIG_GREENASIA_FF is not set
CONFIG_HID_HYPERV_MOUSE=m
CONFIG_HID_SMARTJOYPLUS=m
# CONFIG_SMARTJOYPLUS_FF is not set
CONFIG_HID_TIVO=m
CONFIG_HID_TOPSEED=m
CONFIG_HID_THINGM=m
CONFIG_HID_THRUSTMASTER=m
# CONFIG_THRUSTMASTER_FF is not set
# CONFIG_HID_UDRAW_PS3 is not set
# CONFIG_HID_U2FZERO is not set
CONFIG_HID_WACOM=m
CONFIG_HID_WIIMOTE=m
CONFIG_HID_XINMO=m
CONFIG_HID_ZEROPLUS=m
# CONFIG_ZEROPLUS_FF is not set
CONFIG_HID_ZYDACRON=m
CONFIG_HID_SENSOR_HUB=y
CONFIG_HID_SENSOR_CUSTOM_SENSOR=m
CONFIG_HID_ALPS=m
# CONFIG_HID_MCP2221 is not set
# end of Special HID drivers

#
# USB HID support
#
CONFIG_USB_HID=y
CONFIG_HID_PID=y
CONFIG_USB_HIDDEV=y
# end of USB HID support

#
# I2C HID support
#
CONFIG_I2C_HID=m
# end of I2C HID support

#
# Intel ISH HID support
#
CONFIG_INTEL_ISH_HID=m
# CONFIG_INTEL_ISH_FIRMWARE_DOWNLOADER is not set
# end of Intel ISH HID support

#
# AMD SFH HID Support
#
# CONFIG_AMD_SFH_HID is not set
# end of AMD SFH HID Support
# end of HID support

CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
CONFIG_USB_LED_TRIG=y
# CONFIG_USB_ULPI_BUS is not set
CONFIG_USB_CONN_GPIO=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
CONFIG_USB_PCI=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y

#
# Miscellaneous USB options
#
CONFIG_USB_DEFAULT_PERSIST=y
CONFIG_USB_FEW_INIT_RETRIES=y
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_OTG is not set
# CONFIG_USB_OTG_PRODUCTLIST is not set
CONFIG_USB_LEDS_TRIGGER_USBPORT=m
CONFIG_USB_AUTOSUSPEND_DELAY=2
CONFIG_USB_MON=y

#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_DBGCAP=y
CONFIG_USB_XHCI_PCI=y
# CONFIG_USB_XHCI_PCI_RENESAS is not set
CONFIG_USB_XHCI_PLATFORM=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_EHCI_PCI=y
# CONFIG_USB_EHCI_FSL is not set
# CONFIG_USB_EHCI_HCD_PLATFORM is not set
# CONFIG_USB_OXU210HP_HCD is not set
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_FOTG210_HCD is not set
# CONFIG_USB_MAX3421_HCD is not set
CONFIG_USB_OHCI_HCD=y
CONFIG_USB_OHCI_HCD_PCI=y
# CONFIG_USB_OHCI_HCD_PLATFORM is not set
CONFIG_USB_UHCI_HCD=y
# CONFIG_USB_U132_HCD is not set
# CONFIG_USB_SL811_HCD is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_HCD_BCMA is not set
# CONFIG_USB_HCD_TEST_MODE is not set

#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
CONFIG_USB_WDM=m
CONFIG_USB_TMC=m

#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#

#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_REALTEK=m
CONFIG_REALTEK_AUTOPM=y
CONFIG_USB_STORAGE_DATAFAB=m
CONFIG_USB_STORAGE_FREECOM=m
CONFIG_USB_STORAGE_ISD200=m
CONFIG_USB_STORAGE_USBAT=m
CONFIG_USB_STORAGE_SDDR09=m
CONFIG_USB_STORAGE_SDDR55=m
CONFIG_USB_STORAGE_JUMPSHOT=m
CONFIG_USB_STORAGE_ALAUDA=m
CONFIG_USB_STORAGE_ONETOUCH=m
CONFIG_USB_STORAGE_KARMA=m
CONFIG_USB_STORAGE_CYPRESS_ATACB=m
CONFIG_USB_STORAGE_ENE_UB6250=m
CONFIG_USB_UAS=m

#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
# CONFIG_USBIP_CORE is not set
# CONFIG_USB_CDNS3 is not set
# CONFIG_USB_MUSB_HDRC is not set
# CONFIG_USB_DWC3 is not set
# CONFIG_USB_DWC2 is not set
# CONFIG_USB_CHIPIDEA is not set
# CONFIG_USB_ISP1760 is not set

#
# USB port drivers
#
CONFIG_USB_USS720=m
CONFIG_USB_SERIAL=y
CONFIG_USB_SERIAL_CONSOLE=y
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_SIMPLE is not set
CONFIG_USB_SERIAL_AIRCABLE=m
CONFIG_USB_SERIAL_ARK3116=m
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_CH341=m
CONFIG_USB_SERIAL_WHITEHEAT=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
CONFIG_USB_SERIAL_CP210X=m
CONFIG_USB_SERIAL_CYPRESS_M8=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
# CONFIG_USB_SERIAL_F81232 is not set
CONFIG_USB_SERIAL_F8153X=m
CONFIG_USB_SERIAL_GARMIN=m
CONFIG_USB_SERIAL_IPW=m
CONFIG_USB_SERIAL_IUU=m
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
# CONFIG_USB_SERIAL_METRO is not set
CONFIG_USB_SERIAL_MOS7720=m
CONFIG_USB_SERIAL_MOS7715_PARPORT=y
CONFIG_USB_SERIAL_MOS7840=m
CONFIG_USB_SERIAL_MXUPORT=m
CONFIG_USB_SERIAL_NAVMAN=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_OTI6858=m
CONFIG_USB_SERIAL_QCAUX=m
CONFIG_USB_SERIAL_QUALCOMM=m
CONFIG_USB_SERIAL_SPCP8X5=m
CONFIG_USB_SERIAL_SAFE=m
CONFIG_USB_SERIAL_SAFE_PADDED=y
CONFIG_USB_SERIAL_SIERRAWIRELESS=m
CONFIG_USB_SERIAL_SYMBOL=m
CONFIG_USB_SERIAL_TI=m
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_WWAN=m
CONFIG_USB_SERIAL_OPTION=m
CONFIG_USB_SERIAL_OMNINET=m
CONFIG_USB_SERIAL_OPTICON=m
CONFIG_USB_SERIAL_XSENS_MT=m
# CONFIG_USB_SERIAL_WISHBONE is not set
CONFIG_USB_SERIAL_SSU100=m
CONFIG_USB_SERIAL_QT2=m
CONFIG_USB_SERIAL_UPD78F0730=m
CONFIG_USB_SERIAL_DEBUG=m

#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
CONFIG_USB_ADUTUX=m
CONFIG_USB_SEVSEG=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
CONFIG_USB_IDMOUSE=m
CONFIG_USB_FTDI_ELAN=m
CONFIG_USB_APPLEDISPLAY=m
CONFIG_APPLE_MFI_FASTCHARGE=m
CONFIG_USB_SISUSBVGA=m
CONFIG_USB_LD=m
# CONFIG_USB_TRANCEVIBRATOR is not set
CONFIG_USB_IOWARRIOR=m
# CONFIG_USB_TEST is not set
# CONFIG_USB_EHSET_TEST_FIXTURE is not set
CONFIG_USB_ISIGHTFW=m
# CONFIG_USB_YUREX is not set
CONFIG_USB_EZUSB_FX2=m
# CONFIG_USB_HUB_USB251XB is not set
CONFIG_USB_HSIC_USB3503=m
# CONFIG_USB_HSIC_USB4604 is not set
# CONFIG_USB_LINK_LAYER_TEST is not set
# CONFIG_USB_CHAOSKEY is not set
CONFIG_USB_ATM=m
CONFIG_USB_SPEEDTOUCH=m
CONFIG_USB_CXACRU=m
CONFIG_USB_UEAGLEATM=m
CONFIG_USB_XUSBATM=m

#
# USB Physical Layer drivers
#
# CONFIG_NOP_USB_XCEIV is not set
# CONFIG_USB_GPIO_VBUS is not set
# CONFIG_USB_ISP1301 is not set
# end of USB Physical Layer drivers

# CONFIG_USB_GADGET is not set
CONFIG_TYPEC=y
CONFIG_TYPEC_TCPM=y
CONFIG_TYPEC_TCPCI=y
CONFIG_TYPEC_RT1711H=y
# CONFIG_TYPEC_TCPCI_MAXIM is not set
CONFIG_TYPEC_FUSB302=m
CONFIG_TYPEC_UCSI=y
# CONFIG_UCSI_CCG is not set
CONFIG_UCSI_ACPI=y
CONFIG_TYPEC_HD3SS3220=m
CONFIG_TYPEC_TPS6598X=m
CONFIG_TYPEC_STUSB160X=m

#
# USB Type-C Multiplexer/DeMultiplexer Switch support
#
CONFIG_TYPEC_MUX_PI3USB30532=m
# end of USB Type-C Multiplexer/DeMultiplexer Switch support

#
# USB Type-C Alternate Mode drivers
#
CONFIG_TYPEC_DP_ALTMODE=y
# CONFIG_TYPEC_NVIDIA_ALTMODE is not set
# end of USB Type-C Alternate Mode drivers

CONFIG_USB_ROLE_SWITCH=y
CONFIG_USB_ROLES_INTEL_XHCI=y
CONFIG_MMC=m
CONFIG_MMC_BLOCK=m
CONFIG_MMC_BLOCK_MINORS=8
CONFIG_SDIO_UART=m
# CONFIG_MMC_TEST is not set

#
# MMC/SD/SDIO Host Controller Drivers
#
# CONFIG_MMC_DEBUG is not set
CONFIG_MMC_SDHCI=m
CONFIG_MMC_SDHCI_IO_ACCESSORS=y
CONFIG_MMC_SDHCI_PCI=m
CONFIG_MMC_RICOH_MMC=y
CONFIG_MMC_SDHCI_ACPI=m
CONFIG_MMC_SDHCI_PLTFM=m
# CONFIG_MMC_SDHCI_F_SDH30 is not set
# CONFIG_MMC_WBSD is not set
CONFIG_MMC_TIFM_SD=m
# CONFIG_MMC_SPI is not set
CONFIG_MMC_CB710=m
CONFIG_MMC_VIA_SDMMC=m
CONFIG_MMC_VUB300=m
CONFIG_MMC_USHC=m
# CONFIG_MMC_USDHI6ROL0 is not set
CONFIG_MMC_REALTEK_PCI=m
CONFIG_MMC_REALTEK_USB=m
CONFIG_MMC_CQHCI=m
# CONFIG_MMC_HSQ is not set
# CONFIG_MMC_TOSHIBA_PCI is not set
# CONFIG_MMC_MTK is not set
# CONFIG_MMC_SDHCI_XENON is not set
CONFIG_MEMSTICK=m
# CONFIG_MEMSTICK_DEBUG is not set

#
# MemoryStick drivers
#
# CONFIG_MEMSTICK_UNSAFE_RESUME is not set
CONFIG_MSPRO_BLOCK=m
# CONFIG_MS_BLOCK is not set

#
# MemoryStick Host Controller Drivers
#
CONFIG_MEMSTICK_TIFM_MS=m
CONFIG_MEMSTICK_JMICRON_38X=m
CONFIG_MEMSTICK_R592=m
CONFIG_MEMSTICK_REALTEK_PCI=m
CONFIG_MEMSTICK_REALTEK_USB=m
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y
# CONFIG_LEDS_CLASS_FLASH is not set
# CONFIG_LEDS_CLASS_MULTICOLOR is not set
# CONFIG_LEDS_BRIGHTNESS_HW_CHANGED is not set

#
# LED drivers
#
# CONFIG_LEDS_APU is not set
CONFIG_LEDS_LM3530=m
# CONFIG_LEDS_LM3532 is not set
# CONFIG_LEDS_LM3642 is not set
# CONFIG_LEDS_PCA9532 is not set
# CONFIG_LEDS_GPIO is not set
CONFIG_LEDS_LP3944=m
# CONFIG_LEDS_LP3952 is not set
# CONFIG_LEDS_LP50XX is not set
CONFIG_LEDS_CLEVO_MAIL=m
# CONFIG_LEDS_PCA955X is not set
# CONFIG_LEDS_PCA963X is not set
# CONFIG_LEDS_DAC124S085 is not set
# CONFIG_LEDS_PWM is not set
# CONFIG_LEDS_BD2802 is not set
CONFIG_LEDS_INTEL_SS4200=m
# CONFIG_LEDS_TCA6507 is not set
# CONFIG_LEDS_TLC591XX is not set
# CONFIG_LEDS_LM355x is not set

#
# LED driver for blink(1) USB RGB LED is under Special HID drivers (HID_THINGM)
#
CONFIG_LEDS_BLINKM=m
CONFIG_LEDS_MLXCPLD=m
CONFIG_LEDS_MLXREG=m
# CONFIG_LEDS_USER is not set
# CONFIG_LEDS_NIC78BX is not set
# CONFIG_LEDS_TI_LMU_COMMON is not set

#
# Flash and Torch LED drivers
#

#
# LED Triggers
#
CONFIG_LEDS_TRIGGERS=y
CONFIG_LEDS_TRIGGER_TIMER=m
CONFIG_LEDS_TRIGGER_ONESHOT=m
CONFIG_LEDS_TRIGGER_DISK=y
# CONFIG_LEDS_TRIGGER_MTD is not set
CONFIG_LEDS_TRIGGER_HEARTBEAT=m
CONFIG_LEDS_TRIGGER_BACKLIGHT=m
# CONFIG_LEDS_TRIGGER_CPU is not set
# CONFIG_LEDS_TRIGGER_ACTIVITY is not set
CONFIG_LEDS_TRIGGER_GPIO=m
CONFIG_LEDS_TRIGGER_DEFAULT_ON=m

#
# iptables trigger is under Netfilter config (LED target)
#
CONFIG_LEDS_TRIGGER_TRANSIENT=m
CONFIG_LEDS_TRIGGER_CAMERA=m
# CONFIG_LEDS_TRIGGER_PANIC is not set
# CONFIG_LEDS_TRIGGER_NETDEV is not set
# CONFIG_LEDS_TRIGGER_PATTERN is not set
CONFIG_LEDS_TRIGGER_AUDIO=m
# CONFIG_ACCESSIBILITY is not set
CONFIG_INFINIBAND=m
CONFIG_INFINIBAND_USER_MAD=m
CONFIG_INFINIBAND_USER_ACCESS=m
CONFIG_INFINIBAND_USER_MEM=y
CONFIG_INFINIBAND_ON_DEMAND_PAGING=y
CONFIG_INFINIBAND_ADDR_TRANS=y
CONFIG_INFINIBAND_ADDR_TRANS_CONFIGFS=y
CONFIG_INFINIBAND_VIRT_DMA=y
# CONFIG_INFINIBAND_MTHCA is not set
# CONFIG_INFINIBAND_QIB is not set
CONFIG_INFINIBAND_CXGB4=m
CONFIG_INFINIBAND_EFA=m
CONFIG_INFINIBAND_I40IW=m
CONFIG_MLX4_INFINIBAND=m
CONFIG_MLX5_INFINIBAND=m
# CONFIG_INFINIBAND_OCRDMA is not set
CONFIG_INFINIBAND_VMWARE_PVRDMA=m
CONFIG_INFINIBAND_USNIC=m
CONFIG_INFINIBAND_BNXT_RE=m
CONFIG_INFINIBAND_HFI1=m
# CONFIG_HFI1_DEBUG_SDMA_ORDER is not set
# CONFIG_SDMA_VERBOSITY is not set
CONFIG_INFINIBAND_QEDR=m
CONFIG_INFINIBAND_RDMAVT=m
CONFIG_RDMA_RXE=m
CONFIG_RDMA_SIW=m
CONFIG_INFINIBAND_IPOIB=m
CONFIG_INFINIBAND_IPOIB_CM=y
CONFIG_INFINIBAND_IPOIB_DEBUG=y
# CONFIG_INFINIBAND_IPOIB_DEBUG_DATA is not set
CONFIG_INFINIBAND_SRP=m
CONFIG_INFINIBAND_SRPT=m
CONFIG_INFINIBAND_ISER=m
CONFIG_INFINIBAND_ISERT=m
# CONFIG_INFINIBAND_RTRS_CLIENT is not set
# CONFIG_INFINIBAND_RTRS_SERVER is not set
CONFIG_INFINIBAND_OPA_VNIC=m
CONFIG_EDAC_ATOMIC_SCRUB=y
CONFIG_EDAC_SUPPORT=y
CONFIG_EDAC=y
CONFIG_EDAC_LEGACY_SYSFS=y
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_DECODE_MCE=m
CONFIG_EDAC_GHES=y
CONFIG_EDAC_AMD64=m
# CONFIG_EDAC_AMD64_ERROR_INJECTION is not set
CONFIG_EDAC_E752X=m
CONFIG_EDAC_I82975X=m
CONFIG_EDAC_I3000=m
CONFIG_EDAC_I3200=m
CONFIG_EDAC_IE31200=m
CONFIG_EDAC_X38=m
CONFIG_EDAC_I5400=m
CONFIG_EDAC_I7CORE=m
CONFIG_EDAC_I5000=m
CONFIG_EDAC_I5100=m
CONFIG_EDAC_I7300=m
CONFIG_EDAC_SBRIDGE=m
CONFIG_EDAC_SKX=m
CONFIG_EDAC_I10NM=m
CONFIG_EDAC_PND2=m
# CONFIG_EDAC_IGEN6 is not set
CONFIG_RTC_LIB=y
CONFIG_RTC_MC146818_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
# CONFIG_RTC_SYSTOHC is not set
# CONFIG_RTC_DEBUG is not set
CONFIG_RTC_NVMEM=y

#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set

#
# I2C RTC drivers
#
# CONFIG_RTC_DRV_ABB5ZES3 is not set
# CONFIG_RTC_DRV_ABEOZ9 is not set
# CONFIG_RTC_DRV_ABX80X is not set
CONFIG_RTC_DRV_DS1307=m
# CONFIG_RTC_DRV_DS1307_CENTURY is not set
CONFIG_RTC_DRV_DS1374=m
# CONFIG_RTC_DRV_DS1374_WDT is not set
CONFIG_RTC_DRV_DS1672=m
CONFIG_RTC_DRV_MAX6900=m
CONFIG_RTC_DRV_RS5C372=m
CONFIG_RTC_DRV_ISL1208=m
CONFIG_RTC_DRV_ISL12022=m
CONFIG_RTC_DRV_X1205=m
CONFIG_RTC_DRV_PCF8523=m
# CONFIG_RTC_DRV_PCF85063 is not set
# CONFIG_RTC_DRV_PCF85363 is not set
CONFIG_RTC_DRV_PCF8563=m
CONFIG_RTC_DRV_PCF8583=m
CONFIG_RTC_DRV_M41T80=m
CONFIG_RTC_DRV_M41T80_WDT=y
CONFIG_RTC_DRV_BQ32K=m
# CONFIG_RTC_DRV_S35390A is not set
CONFIG_RTC_DRV_FM3130=m
# CONFIG_RTC_DRV_RX8010 is not set
CONFIG_RTC_DRV_RX8581=m
CONFIG_RTC_DRV_RX8025=m
CONFIG_RTC_DRV_EM3027=m
# CONFIG_RTC_DRV_RV3028 is not set
# CONFIG_RTC_DRV_RV3032 is not set
# CONFIG_RTC_DRV_RV8803 is not set
# CONFIG_RTC_DRV_SD3078 is not set

#
# SPI RTC drivers
#
# CONFIG_RTC_DRV_M41T93 is not set
# CONFIG_RTC_DRV_M41T94 is not set
# CONFIG_RTC_DRV_DS1302 is not set
# CONFIG_RTC_DRV_DS1305 is not set
# CONFIG_RTC_DRV_DS1343 is not set
# CONFIG_RTC_DRV_DS1347 is not set
# CONFIG_RTC_DRV_DS1390 is not set
# CONFIG_RTC_DRV_MAX6916 is not set
# CONFIG_RTC_DRV_R9701 is not set
CONFIG_RTC_DRV_RX4581=m
# CONFIG_RTC_DRV_RS5C348 is not set
# CONFIG_RTC_DRV_MAX6902 is not set
# CONFIG_RTC_DRV_PCF2123 is not set
# CONFIG_RTC_DRV_MCP795 is not set
CONFIG_RTC_I2C_AND_SPI=y

#
# SPI and I2C RTC drivers
#
CONFIG_RTC_DRV_DS3232=m
CONFIG_RTC_DRV_DS3232_HWMON=y
# CONFIG_RTC_DRV_PCF2127 is not set
CONFIG_RTC_DRV_RV3029C2=m
# CONFIG_RTC_DRV_RV3029_HWMON is not set
# CONFIG_RTC_DRV_RX6110 is not set

#
# Platform RTC drivers
#
CONFIG_RTC_DRV_CMOS=y
CONFIG_RTC_DRV_DS1286=m
CONFIG_RTC_DRV_DS1511=m
CONFIG_RTC_DRV_DS1553=m
# CONFIG_RTC_DRV_DS1685_FAMILY is not set
CONFIG_RTC_DRV_DS1742=m
CONFIG_RTC_DRV_DS2404=m
CONFIG_RTC_DRV_STK17TA8=m
# CONFIG_RTC_DRV_M48T86 is not set
CONFIG_RTC_DRV_M48T35=m
CONFIG_RTC_DRV_M48T59=m
CONFIG_RTC_DRV_MSM6242=m
CONFIG_RTC_DRV_BQ4802=m
CONFIG_RTC_DRV_RP5C01=m
CONFIG_RTC_DRV_V3020=m

#
# on-CPU RTC drivers
#
# CONFIG_RTC_DRV_FTRTC010 is not set

#
# HID Sensor RTC drivers
#
# CONFIG_RTC_DRV_HID_SENSOR_TIME is not set
CONFIG_DMADEVICES=y
# CONFIG_DMADEVICES_DEBUG is not set

#
# DMA Devices
#
CONFIG_DMA_ENGINE=y
CONFIG_DMA_VIRTUAL_CHANNELS=y
CONFIG_DMA_ACPI=y
# CONFIG_ALTERA_MSGDMA is not set
CONFIG_INTEL_IDMA64=m
# CONFIG_INTEL_IDXD is not set
CONFIG_INTEL_IOATDMA=m
# CONFIG_PLX_DMA is not set
# CONFIG_XILINX_ZYNQMP_DPDMA is not set
# CONFIG_QCOM_HIDMA_MGMT is not set
# CONFIG_QCOM_HIDMA is not set
CONFIG_DW_DMAC_CORE=y
CONFIG_DW_DMAC=m
CONFIG_DW_DMAC_PCI=y
# CONFIG_DW_EDMA is not set
# CONFIG_DW_EDMA_PCIE is not set
CONFIG_HSU_DMA=y
# CONFIG_SF_PDMA is not set

#
# DMA Clients
#
CONFIG_ASYNC_TX_DMA=y
CONFIG_DMATEST=m
CONFIG_DMA_ENGINE_RAID=y

#
# DMABUF options
#
CONFIG_SYNC_FILE=y
# CONFIG_SW_SYNC is not set
# CONFIG_UDMABUF is not set
# CONFIG_DMABUF_MOVE_NOTIFY is not set
# CONFIG_DMABUF_SELFTESTS is not set
# CONFIG_DMABUF_HEAPS is not set
# end of DMABUF options

CONFIG_DCA=m
# CONFIG_AUXDISPLAY is not set
# CONFIG_PANEL is not set
CONFIG_UIO=m
CONFIG_UIO_CIF=m
CONFIG_UIO_PDRV_GENIRQ=m
# CONFIG_UIO_DMEM_GENIRQ is not set
CONFIG_UIO_AEC=m
CONFIG_UIO_SERCOS3=m
CONFIG_UIO_PCI_GENERIC=m
# CONFIG_UIO_NETX is not set
# CONFIG_UIO_PRUSS is not set
# CONFIG_UIO_MF624 is not set
CONFIG_UIO_HV_GENERIC=m
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO_VIRQFD=m
CONFIG_VFIO=m
CONFIG_VFIO_NOIOMMU=y
CONFIG_VFIO_PCI=m
# CONFIG_VFIO_PCI_VGA is not set
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
# CONFIG_VFIO_PCI_IGD is not set
CONFIG_VFIO_MDEV=m
CONFIG_VFIO_MDEV_DEVICE=m
CONFIG_IRQ_BYPASS_MANAGER=m
CONFIG_VIRT_DRIVERS=y
# CONFIG_VBOXGUEST is not set
CONFIG_NITRO_ENCLAVES=m
CONFIG_VIRTIO=y
CONFIG_VIRTIO_MENU=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_PCI_LEGACY=y
CONFIG_VIRTIO_VDPA=m
# CONFIG_VIRTIO_PMEM is not set
CONFIG_VIRTIO_BALLOON=m
CONFIG_VIRTIO_MEM=m
CONFIG_VIRTIO_INPUT=m
# CONFIG_VIRTIO_MMIO is not set
CONFIG_VIRTIO_DMA_SHARED_BUFFER=m
CONFIG_VDPA=m
CONFIG_VDPA_SIM=m
# CONFIG_VDPA_SIM_NET is not set
CONFIG_IFCVF=m
# CONFIG_MLX5_VDPA_NET is not set
CONFIG_VHOST_IOTLB=m
CONFIG_VHOST_RING=m
CONFIG_VHOST=m
CONFIG_VHOST_MENU=y
CONFIG_VHOST_NET=m
# CONFIG_VHOST_SCSI is not set
CONFIG_VHOST_VSOCK=m
CONFIG_VHOST_VDPA=m
# CONFIG_VHOST_CROSS_ENDIAN_LEGACY is not set

#
# Microsoft Hyper-V guest support
#
CONFIG_HYPERV=m
CONFIG_HYPERV_TIMER=y
CONFIG_HYPERV_UTILS=m
CONFIG_HYPERV_BALLOON=m
# end of Microsoft Hyper-V guest support

#
# Xen driver support
#
# CONFIG_XEN_BALLOON is not set
CONFIG_XEN_DEV_EVTCHN=m
# CONFIG_XEN_BACKEND is not set
CONFIG_XENFS=m
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
# CONFIG_XEN_GNTDEV is not set
# CONFIG_XEN_GRANT_DEV_ALLOC is not set
# CONFIG_XEN_GRANT_DMA_ALLOC is not set
CONFIG_SWIOTLB_XEN=y
# CONFIG_XEN_PVCALLS_FRONTEND is not set
CONFIG_XEN_PRIVCMD=m
CONFIG_XEN_EFI=y
CONFIG_XEN_AUTO_XLATE=y
CONFIG_XEN_ACPI=y
CONFIG_XEN_FRONT_PGDIR_SHBUF=m
# CONFIG_XEN_UNPOPULATED_ALLOC is not set
# end of Xen driver support

# CONFIG_GREYBUS is not set
# CONFIG_STAGING is not set
CONFIG_X86_PLATFORM_DEVICES=y
CONFIG_ACPI_WMI=m
CONFIG_WMI_BMOF=m
# CONFIG_ALIENWARE_WMI is not set
# CONFIG_HUAWEI_WMI is not set
# CONFIG_UV_SYSFS is not set
# CONFIG_INTEL_WMI_SBL_FW_UPDATE is not set
CONFIG_INTEL_WMI_THUNDERBOLT=m
CONFIG_MXM_WMI=m
# CONFIG_PEAQ_WMI is not set
# CONFIG_XIAOMI_WMI is not set
CONFIG_ACERHDF=m
# CONFIG_ACER_WIRELESS is not set
CONFIG_ACER_WMI=m
# CONFIG_AMD_PMC is not set
CONFIG_APPLE_GMUX=m
CONFIG_ASUS_LAPTOP=m
# CONFIG_ASUS_WIRELESS is not set
CONFIG_ASUS_WMI=m
CONFIG_ASUS_NB_WMI=m
CONFIG_EEEPC_LAPTOP=m
CONFIG_EEEPC_WMI=m
CONFIG_DCDBAS=m
CONFIG_DELL_SMBIOS=m
CONFIG_DELL_SMBIOS_WMI=y
# CONFIG_DELL_SMBIOS_SMM is not set
CONFIG_DELL_LAPTOP=m
CONFIG_DELL_RBTN=m
CONFIG_DELL_RBU=m
CONFIG_DELL_SMO8800=m
CONFIG_DELL_WMI=m
# CONFIG_DELL_WMI_SYSMAN is not set
CONFIG_DELL_WMI_DESCRIPTOR=m
CONFIG_DELL_WMI_AIO=m
CONFIG_DELL_WMI_LED=m
CONFIG_AMILO_RFKILL=m
CONFIG_FUJITSU_LAPTOP=m
CONFIG_FUJITSU_TABLET=m
# CONFIG_GPD_POCKET_FAN is not set
CONFIG_HP_ACCEL=m
CONFIG_HP_WIRELESS=m
CONFIG_HP_WMI=m
# CONFIG_IBM_RTL is not set
CONFIG_IDEAPAD_LAPTOP=m
CONFIG_SENSORS_HDAPS=m
CONFIG_THINKPAD_ACPI=m
CONFIG_THINKPAD_ACPI_ALSA_SUPPORT=y
# CONFIG_THINKPAD_ACPI_DEBUGFACILITIES is not set
# CONFIG_THINKPAD_ACPI_DEBUG is not set
# CONFIG_THINKPAD_ACPI_UNSAFE_LEDS is not set
CONFIG_THINKPAD_ACPI_VIDEO=y
CONFIG_THINKPAD_ACPI_HOTKEY_POLL=y
# CONFIG_INTEL_ATOMISP2_PM is not set
CONFIG_INTEL_HID_EVENT=m
# CONFIG_INTEL_INT0002_VGPIO is not set
# CONFIG_INTEL_MENLOW is not set
CONFIG_INTEL_OAKTRAIL=m
CONFIG_INTEL_VBTN=m
CONFIG_MSI_LAPTOP=m
CONFIG_MSI_WMI=m
# CONFIG_PCENGINES_APU2 is not set
CONFIG_SAMSUNG_LAPTOP=m
CONFIG_SAMSUNG_Q10=m
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_TOSHIBA_BT_RFKILL=m
# CONFIG_TOSHIBA_HAPS is not set
# CONFIG_TOSHIBA_WMI is not set
CONFIG_ACPI_CMPC=m
CONFIG_COMPAL_LAPTOP=m
# CONFIG_LG_LAPTOP is not set
CONFIG_PANASONIC_LAPTOP=m
CONFIG_SONY_LAPTOP=m
CONFIG_SONYPI_COMPAT=y
# CONFIG_SYSTEM76_ACPI is not set
CONFIG_TOPSTAR_LAPTOP=m
# CONFIG_I2C_MULTI_INSTANTIATE is not set
CONFIG_MLX_PLATFORM=m
CONFIG_INTEL_IPS=m
CONFIG_INTEL_RST=m
# CONFIG_INTEL_SMARTCONNECT is not set

#
# Intel Speed Select Technology interface support
#
CONFIG_INTEL_SPEED_SELECT_INTERFACE=m
# end of Intel Speed Select Technology interface support

CONFIG_INTEL_TURBO_MAX_3=y
# CONFIG_INTEL_UNCORE_FREQ_CONTROL is not set
CONFIG_INTEL_PMC_CORE=m
# CONFIG_INTEL_PMT_CLASS is not set
# CONFIG_INTEL_PMT_TELEMETRY is not set
# CONFIG_INTEL_PMT_CRASHLOG is not set
# CONFIG_INTEL_PUNIT_IPC is not set
# CONFIG_INTEL_SCU_PCI is not set
# CONFIG_INTEL_SCU_PLATFORM is not set
CONFIG_PMC_ATOM=y
# CONFIG_CHROME_PLATFORMS is not set
CONFIG_MELLANOX_PLATFORM=y
CONFIG_MLXREG_HOTPLUG=m
CONFIG_MLXREG_IO=m
CONFIG_SURFACE_PLATFORMS=y
# CONFIG_SURFACE3_WMI is not set
# CONFIG_SURFACE_3_POWER_OPREGION is not set
# CONFIG_SURFACE_GPE is not set
# CONFIG_SURFACE_PRO3_BUTTON is not set
CONFIG_HAVE_CLK=y
CONFIG_CLKDEV_LOOKUP=y
CONFIG_HAVE_CLK_PREPARE=y
CONFIG_COMMON_CLK=y
# CONFIG_COMMON_CLK_MAX9485 is not set
# CONFIG_COMMON_CLK_SI5341 is not set
# CONFIG_COMMON_CLK_SI5351 is not set
# CONFIG_COMMON_CLK_SI544 is not set
# CONFIG_COMMON_CLK_CDCE706 is not set
# CONFIG_COMMON_CLK_CS2000_CP is not set
# CONFIG_COMMON_CLK_PWM is not set
CONFIG_HWSPINLOCK=y

#
# Clock Source drivers
#
CONFIG_CLKEVT_I8253=y
CONFIG_I8253_LOCK=y
CONFIG_CLKBLD_I8253=y
# end of Clock Source drivers

CONFIG_MAILBOX=y
CONFIG_PCC=y
# CONFIG_ALTERA_MBOX is not set
CONFIG_IOMMU_IOVA=y
CONFIG_IOASID=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y

#
# Generic IOMMU Pagetable Support
#
# end of Generic IOMMU Pagetable Support

# CONFIG_IOMMU_DEBUGFS is not set
CONFIG_IOMMU_DEFAULT_PASSTHROUGH=y
CONFIG_IOMMU_DMA=y
CONFIG_AMD_IOMMU=y
CONFIG_AMD_IOMMU_V2=m
CONFIG_DMAR_TABLE=y
CONFIG_INTEL_IOMMU=y
# CONFIG_INTEL_IOMMU_SVM is not set
# CONFIG_INTEL_IOMMU_DEFAULT_ON is not set
CONFIG_INTEL_IOMMU_FLOPPY_WA=y
# CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON is not set
CONFIG_IRQ_REMAP=y
CONFIG_HYPERV_IOMMU=y

#
# Remoteproc drivers
#
# CONFIG_REMOTEPROC is not set
# end of Remoteproc drivers

#
# Rpmsg drivers
#
# CONFIG_RPMSG_QCOM_GLINK_RPM is not set
# CONFIG_RPMSG_VIRTIO is not set
# end of Rpmsg drivers

CONFIG_SOUNDWIRE=m

#
# SoundWire Devices
#
CONFIG_SOUNDWIRE_CADENCE=m
CONFIG_SOUNDWIRE_INTEL=m
# CONFIG_SOUNDWIRE_QCOM is not set
CONFIG_SOUNDWIRE_GENERIC_ALLOCATION=m

#
# SOC (System On Chip) specific Drivers
#

#
# Amlogic SoC drivers
#
# end of Amlogic SoC drivers

#
# Broadcom SoC drivers
#
# end of Broadcom SoC drivers

#
# NXP/Freescale QorIQ SoC drivers
#
# end of NXP/Freescale QorIQ SoC drivers

#
# i.MX SoC drivers
#
# end of i.MX SoC drivers

#
# Enable LiteX SoC Builder specific drivers
#
# end of Enable LiteX SoC Builder specific drivers

#
# Qualcomm SoC drivers
#
# end of Qualcomm SoC drivers

# CONFIG_SOC_TI is not set

#
# Xilinx SoC drivers
#
# CONFIG_XILINX_VCU is not set
# end of Xilinx SoC drivers
# end of SOC (System On Chip) specific Drivers

# CONFIG_PM_DEVFREQ is not set
# CONFIG_EXTCON is not set
# CONFIG_MEMORY is not set
CONFIG_IIO=m
CONFIG_IIO_BUFFER=y
# CONFIG_IIO_BUFFER_CB is not set
# CONFIG_IIO_BUFFER_DMA is not set
# CONFIG_IIO_BUFFER_DMAENGINE is not set
# CONFIG_IIO_BUFFER_HW_CONSUMER is not set
CONFIG_IIO_KFIFO_BUF=m
CONFIG_IIO_TRIGGERED_BUFFER=m
# CONFIG_IIO_CONFIGFS is not set
CONFIG_IIO_TRIGGER=y
CONFIG_IIO_CONSUMERS_PER_TRIGGER=2
# CONFIG_IIO_SW_DEVICE is not set
# CONFIG_IIO_SW_TRIGGER is not set
# CONFIG_IIO_TRIGGERED_EVENT is not set

#
# Accelerometers
#
# CONFIG_ADIS16201 is not set
# CONFIG_ADIS16209 is not set
# CONFIG_ADXL345_I2C is not set
# CONFIG_ADXL345_SPI is not set
# CONFIG_ADXL372_SPI is not set
# CONFIG_ADXL372_I2C is not set
# CONFIG_BMA180 is not set
# CONFIG_BMA220 is not set
# CONFIG_BMA400 is not set
# CONFIG_BMC150_ACCEL is not set
# CONFIG_DA280 is not set
# CONFIG_DA311 is not set
# CONFIG_DMARD09 is not set
# CONFIG_DMARD10 is not set
CONFIG_HID_SENSOR_ACCEL_3D=m
# CONFIG_IIO_ST_ACCEL_3AXIS is not set
# CONFIG_KXSD9 is not set
# CONFIG_KXCJK1013 is not set
# CONFIG_MC3230 is not set
# CONFIG_MMA7455_I2C is not set
# CONFIG_MMA7455_SPI is not set
# CONFIG_MMA7660 is not set
# CONFIG_MMA8452 is not set
# CONFIG_MMA9551 is not set
# CONFIG_MMA9553 is not set
# CONFIG_MXC4005 is not set
# CONFIG_MXC6255 is not set
# CONFIG_SCA3000 is not set
# CONFIG_STK8312 is not set
# CONFIG_STK8BA50 is not set
# end of Accelerometers

#
# Analog to digital converters
#
# CONFIG_AD7091R5 is not set
# CONFIG_AD7124 is not set
# CONFIG_AD7192 is not set
# CONFIG_AD7266 is not set
# CONFIG_AD7291 is not set
# CONFIG_AD7292 is not set
# CONFIG_AD7298 is not set
# CONFIG_AD7476 is not set
# CONFIG_AD7606_IFACE_PARALLEL is not set
# CONFIG_AD7606_IFACE_SPI is not set
# CONFIG_AD7766 is not set
# CONFIG_AD7768_1 is not set
# CONFIG_AD7780 is not set
# CONFIG_AD7791 is not set
# CONFIG_AD7793 is not set
# CONFIG_AD7887 is not set
# CONFIG_AD7923 is not set
# CONFIG_AD7949 is not set
# CONFIG_AD799X is not set
# CONFIG_AD9467 is not set
# CONFIG_ADI_AXI_ADC is not set
# CONFIG_HI8435 is not set
# CONFIG_HX711 is not set
# CONFIG_INA2XX_ADC is not set
# CONFIG_LTC2471 is not set
# CONFIG_LTC2485 is not set
# CONFIG_LTC2496 is not set
# CONFIG_LTC2497 is not set
# CONFIG_MAX1027 is not set
# CONFIG_MAX11100 is not set
# CONFIG_MAX1118 is not set
# CONFIG_MAX1241 is not set
# CONFIG_MAX1363 is not set
# CONFIG_MAX9611 is not set
# CONFIG_MCP320X is not set
# CONFIG_MCP3422 is not set
# CONFIG_MCP3911 is not set
# CONFIG_NAU7802 is not set
# CONFIG_TI_ADC081C is not set
# CONFIG_TI_ADC0832 is not set
# CONFIG_TI_ADC084S021 is not set
# CONFIG_TI_ADC12138 is not set
# CONFIG_TI_ADC108S102 is not set
# CONFIG_TI_ADC128S052 is not set
# CONFIG_TI_ADC161S626 is not set
# CONFIG_TI_ADS1015 is not set
# CONFIG_TI_ADS7950 is not set
# CONFIG_TI_TLC4541 is not set
# CONFIG_VIPERBOARD_ADC is not set
# CONFIG_XILINX_XADC is not set
# end of Analog to digital converters

#
# Analog Front Ends
#
# end of Analog Front Ends

#
# Amplifiers
#
# CONFIG_AD8366 is not set
# CONFIG_HMC425 is not set
# end of Amplifiers

#
# Chemical Sensors
#
# CONFIG_ATLAS_PH_SENSOR is not set
# CONFIG_ATLAS_EZO_SENSOR is not set
# CONFIG_BME680 is not set
# CONFIG_CCS811 is not set
# CONFIG_IAQCORE is not set
# CONFIG_SCD30_CORE is not set
# CONFIG_SENSIRION_SGP30 is not set
# CONFIG_SPS30 is not set
# CONFIG_VZ89X is not set
# end of Chemical Sensors

#
# Hid Sensor IIO Common
#
CONFIG_HID_SENSOR_IIO_COMMON=m
CONFIG_HID_SENSOR_IIO_TRIGGER=m
# end of Hid Sensor IIO Common

#
# SSP Sensor Common
#
# CONFIG_IIO_SSP_SENSORHUB is not set
# end of SSP Sensor Common

#
# Digital to analog converters
#
# CONFIG_AD5064 is not set
# CONFIG_AD5360 is not set
# CONFIG_AD5380 is not set
# CONFIG_AD5421 is not set
# CONFIG_AD5446 is not set
# CONFIG_AD5449 is not set
# CONFIG_AD5592R is not set
# CONFIG_AD5593R is not set
# CONFIG_AD5504 is not set
# CONFIG_AD5624R_SPI is not set
# CONFIG_AD5686_SPI is not set
# CONFIG_AD5696_I2C is not set
# CONFIG_AD5755 is not set
# CONFIG_AD5758 is not set
# CONFIG_AD5761 is not set
# CONFIG_AD5764 is not set
# CONFIG_AD5770R is not set
# CONFIG_AD5791 is not set
# CONFIG_AD7303 is not set
# CONFIG_AD8801 is not set
# CONFIG_DS4424 is not set
# CONFIG_LTC1660 is not set
# CONFIG_LTC2632 is not set
# CONFIG_M62332 is not set
# CONFIG_MAX517 is not set
# CONFIG_MCP4725 is not set
# CONFIG_MCP4922 is not set
# CONFIG_TI_DAC082S085 is not set
# CONFIG_TI_DAC5571 is not set
# CONFIG_TI_DAC7311 is not set
# CONFIG_TI_DAC7612 is not set
# end of Digital to analog converters

#
# IIO dummy driver
#
# end of IIO dummy driver

#
# Frequency Synthesizers DDS/PLL
#

#
# Clock Generator/Distribution
#
# CONFIG_AD9523 is not set
# end of Clock Generator/Distribution

#
# Phase-Locked Loop (PLL) frequency synthesizers
#
# CONFIG_ADF4350 is not set
# CONFIG_ADF4371 is not set
# end of Phase-Locked Loop (PLL) frequency synthesizers
# end of Frequency Synthesizers DDS/PLL

#
# Digital gyroscope sensors
#
# CONFIG_ADIS16080 is not set
# CONFIG_ADIS16130 is not set
# CONFIG_ADIS16136 is not set
# CONFIG_ADIS16260 is not set
# CONFIG_ADXRS290 is not set
# CONFIG_ADXRS450 is not set
# CONFIG_BMG160 is not set
# CONFIG_FXAS21002C is not set
CONFIG_HID_SENSOR_GYRO_3D=m
# CONFIG_MPU3050_I2C is not set
# CONFIG_IIO_ST_GYRO_3AXIS is not set
# CONFIG_ITG3200 is not set
# end of Digital gyroscope sensors

#
# Health Sensors
#

#
# Heart Rate Monitors
#
# CONFIG_AFE4403 is not set
# CONFIG_AFE4404 is not set
# CONFIG_MAX30100 is not set
# CONFIG_MAX30102 is not set
# end of Heart Rate Monitors
# end of Health Sensors

#
# Humidity sensors
#
# CONFIG_AM2315 is not set
# CONFIG_DHT11 is not set
# CONFIG_HDC100X is not set
# CONFIG_HDC2010 is not set
CONFIG_HID_SENSOR_HUMIDITY=m
# CONFIG_HTS221 is not set
# CONFIG_HTU21 is not set
# CONFIG_SI7005 is not set
# CONFIG_SI7020 is not set
# end of Humidity sensors

#
# Inertial measurement units
#
# CONFIG_ADIS16400 is not set
# CONFIG_ADIS16460 is not set
# CONFIG_ADIS16475 is not set
# CONFIG_ADIS16480 is not set
# CONFIG_BMI160_I2C is not set
# CONFIG_BMI160_SPI is not set
# CONFIG_FXOS8700_I2C is not set
# CONFIG_FXOS8700_SPI is not set
# CONFIG_KMX61 is not set
# CONFIG_INV_ICM42600_I2C is not set
# CONFIG_INV_ICM42600_SPI is not set
# CONFIG_INV_MPU6050_I2C is not set
# CONFIG_INV_MPU6050_SPI is not set
# CONFIG_IIO_ST_LSM6DSX is not set
# end of Inertial measurement units

#
# Light sensors
#
# CONFIG_ACPI_ALS is not set
# CONFIG_ADJD_S311 is not set
# CONFIG_ADUX1020 is not set
# CONFIG_AL3010 is not set
# CONFIG_AL3320A is not set
# CONFIG_APDS9300 is not set
# CONFIG_APDS9960 is not set
# CONFIG_AS73211 is not set
# CONFIG_BH1750 is not set
# CONFIG_BH1780 is not set
# CONFIG_CM32181 is not set
# CONFIG_CM3232 is not set
# CONFIG_CM3323 is not set
# CONFIG_CM36651 is not set
# CONFIG_GP2AP002 is not set
# CONFIG_GP2AP020A00F is not set
# CONFIG_SENSORS_ISL29018 is not set
# CONFIG_SENSORS_ISL29028 is not set
# CONFIG_ISL29125 is not set
CONFIG_HID_SENSOR_ALS=m
CONFIG_HID_SENSOR_PROX=m
# CONFIG_JSA1212 is not set
# CONFIG_RPR0521 is not set
# CONFIG_LTR501 is not set
# CONFIG_LV0104CS is not set
# CONFIG_MAX44000 is not set
# CONFIG_MAX44009 is not set
# CONFIG_NOA1305 is not set
# CONFIG_OPT3001 is not set
# CONFIG_PA12203001 is not set
# CONFIG_SI1133 is not set
# CONFIG_SI1145 is not set
# CONFIG_STK3310 is not set
# CONFIG_ST_UVIS25 is not set
# CONFIG_TCS3414 is not set
# CONFIG_TCS3472 is not set
# CONFIG_SENSORS_TSL2563 is not set
# CONFIG_TSL2583 is not set
# CONFIG_TSL2772 is not set
# CONFIG_TSL4531 is not set
# CONFIG_US5182D is not set
# CONFIG_VCNL4000 is not set
# CONFIG_VCNL4035 is not set
# CONFIG_VEML6030 is not set
# CONFIG_VEML6070 is not set
# CONFIG_VL6180 is not set
# CONFIG_ZOPT2201 is not set
# end of Light sensors

#
# Magnetometer sensors
#
# CONFIG_AK8975 is not set
# CONFIG_AK09911 is not set
# CONFIG_BMC150_MAGN_I2C is not set
# CONFIG_BMC150_MAGN_SPI is not set
# CONFIG_MAG3110 is not set
CONFIG_HID_SENSOR_MAGNETOMETER_3D=m
# CONFIG_MMC35240 is not set
# CONFIG_IIO_ST_MAGN_3AXIS is not set
# CONFIG_SENSORS_HMC5843_I2C is not set
# CONFIG_SENSORS_HMC5843_SPI is not set
# CONFIG_SENSORS_RM3100_I2C is not set
# CONFIG_SENSORS_RM3100_SPI is not set
# end of Magnetometer sensors

#
# Multiplexers
#
# end of Multiplexers

#
# Inclinometer sensors
#
CONFIG_HID_SENSOR_INCLINOMETER_3D=m
CONFIG_HID_SENSOR_DEVICE_ROTATION=m
# end of Inclinometer sensors

#
# Triggers - standalone
#
# CONFIG_IIO_INTERRUPT_TRIGGER is not set
# CONFIG_IIO_SYSFS_TRIGGER is not set
# end of Triggers - standalone

#
# Linear and angular position sensors
#
# end of Linear and angular position sensors

#
# Digital potentiometers
#
# CONFIG_AD5272 is not set
# CONFIG_DS1803 is not set
# CONFIG_MAX5432 is not set
# CONFIG_MAX5481 is not set
# CONFIG_MAX5487 is not set
# CONFIG_MCP4018 is not set
# CONFIG_MCP4131 is not set
# CONFIG_MCP4531 is not set
# CONFIG_MCP41010 is not set
# CONFIG_TPL0102 is not set
# end of Digital potentiometers

#
# Digital potentiostats
#
# CONFIG_LMP91000 is not set
# end of Digital potentiostats

#
# Pressure sensors
#
# CONFIG_ABP060MG is not set
# CONFIG_BMP280 is not set
# CONFIG_DLHL60D is not set
# CONFIG_DPS310 is not set
CONFIG_HID_SENSOR_PRESS=m
# CONFIG_HP03 is not set
# CONFIG_ICP10100 is not set
# CONFIG_MPL115_I2C is not set
# CONFIG_MPL115_SPI is not set
# CONFIG_MPL3115 is not set
# CONFIG_MS5611 is not set
# CONFIG_MS5637 is not set
# CONFIG_IIO_ST_PRESS is not set
# CONFIG_T5403 is not set
# CONFIG_HP206C is not set
# CONFIG_ZPA2326 is not set
# end of Pressure sensors

#
# Lightning sensors
#
# CONFIG_AS3935 is not set
# end of Lightning sensors

#
# Proximity and distance sensors
#
# CONFIG_ISL29501 is not set
# CONFIG_LIDAR_LITE_V2 is not set
# CONFIG_MB1232 is not set
# CONFIG_PING is not set
# CONFIG_RFD77402 is not set
# CONFIG_SRF04 is not set
# CONFIG_SX9310 is not set
# CONFIG_SX9500 is not set
# CONFIG_SRF08 is not set
# CONFIG_VCNL3020 is not set
# CONFIG_VL53L0X_I2C is not set
# end of Proximity and distance sensors

#
# Resolver to digital converters
#
# CONFIG_AD2S90 is not set
# CONFIG_AD2S1200 is not set
# end of Resolver to digital converters

#
# Temperature sensors
#
# CONFIG_LTC2983 is not set
# CONFIG_MAXIM_THERMOCOUPLE is not set
CONFIG_HID_SENSOR_TEMP=m
# CONFIG_MLX90614 is not set
# CONFIG_MLX90632 is not set
# CONFIG_TMP006 is not set
# CONFIG_TMP007 is not set
# CONFIG_TSYS01 is not set
# CONFIG_TSYS02D is not set
# CONFIG_MAX31856 is not set
# end of Temperature sensors

CONFIG_NTB=m
# CONFIG_NTB_MSI is not set
CONFIG_NTB_AMD=m
# CONFIG_NTB_IDT is not set
# CONFIG_NTB_INTEL is not set
# CONFIG_NTB_SWITCHTEC is not set
CONFIG_NTB_PINGPONG=m
CONFIG_NTB_TOOL=m
CONFIG_NTB_PERF=m
CONFIG_NTB_TRANSPORT=m
# CONFIG_VME_BUS is not set
CONFIG_PWM=y
CONFIG_PWM_SYSFS=y
# CONFIG_PWM_DEBUG is not set
CONFIG_PWM_DWC=m
CONFIG_PWM_LPSS=m
CONFIG_PWM_LPSS_PCI=m
CONFIG_PWM_LPSS_PLATFORM=m
# CONFIG_PWM_PCA9685 is not set

#
# IRQ chip support
#
# end of IRQ chip support

# CONFIG_IPACK_BUS is not set
# CONFIG_RESET_CONTROLLER is not set

#
# PHY Subsystem
#
# CONFIG_GENERIC_PHY is not set
# CONFIG_USB_LGM_PHY is not set
# CONFIG_BCM_KONA_USB2_PHY is not set
# CONFIG_PHY_PXA_28NM_HSIC is not set
# CONFIG_PHY_PXA_28NM_USB2 is not set
# CONFIG_PHY_CPCAP_USB is not set
# CONFIG_PHY_INTEL_LGM_EMMC is not set
# end of PHY Subsystem

CONFIG_POWERCAP=y
CONFIG_INTEL_RAPL_CORE=m
CONFIG_INTEL_RAPL=m
# CONFIG_IDLE_INJECT is not set
# CONFIG_MCB is not set

#
# Performance monitor support
#
# end of Performance monitor support

CONFIG_RAS=y
# CONFIG_RAS_CEC is not set
CONFIG_USB4=y
# CONFIG_USB4_DEBUGFS_WRITE is not set
# CONFIG_USB4_DMA_TEST is not set

#
# Android
#
# CONFIG_ANDROID is not set
# end of Android

CONFIG_LIBNVDIMM=m
CONFIG_BLK_DEV_PMEM=m
CONFIG_ND_BLK=m
CONFIG_ND_CLAIM=y
CONFIG_ND_BTT=m
CONFIG_BTT=y
CONFIG_ND_PFN=m
CONFIG_NVDIMM_PFN=y
CONFIG_NVDIMM_DAX=y
CONFIG_NVDIMM_KEYS=y
CONFIG_DAX_DRIVER=y
CONFIG_DAX=y
CONFIG_DEV_DAX=m
CONFIG_DEV_DAX_PMEM=m
CONFIG_DEV_DAX_HMEM=m
CONFIG_DEV_DAX_HMEM_DEVICES=y
CONFIG_DEV_DAX_KMEM=m
CONFIG_DEV_DAX_PMEM_COMPAT=m
CONFIG_NVMEM=y
CONFIG_NVMEM_SYSFS=y

#
# HW tracing support
#
CONFIG_STM=m
# CONFIG_STM_PROTO_BASIC is not set
# CONFIG_STM_PROTO_SYS_T is not set
CONFIG_STM_DUMMY=m
CONFIG_STM_SOURCE_CONSOLE=m
CONFIG_STM_SOURCE_HEARTBEAT=m
CONFIG_STM_SOURCE_FTRACE=m
CONFIG_INTEL_TH=m
CONFIG_INTEL_TH_PCI=m
CONFIG_INTEL_TH_ACPI=m
CONFIG_INTEL_TH_GTH=m
CONFIG_INTEL_TH_STH=m
CONFIG_INTEL_TH_MSU=m
CONFIG_INTEL_TH_PTI=m
# CONFIG_INTEL_TH_DEBUG is not set
# end of HW tracing support

# CONFIG_FPGA is not set
# CONFIG_TEE is not set
# CONFIG_UNISYS_VISORBUS is not set
# CONFIG_SIOX is not set
# CONFIG_SLIMBUS is not set
# CONFIG_INTERCONNECT is not set
# CONFIG_COUNTER is not set
# CONFIG_MOST is not set
# end of Device Drivers

#
# File systems
#
CONFIG_DCACHE_WORD_ACCESS=y
# CONFIG_VALIDATE_FS_PARSER is not set
CONFIG_FS_IOMAP=y
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
CONFIG_EXT4_FS=m
CONFIG_EXT4_USE_FOR_EXT2=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_DEBUG is not set
CONFIG_EXT4_KUNIT_TESTS=m
CONFIG_JBD2=m
# CONFIG_JBD2_DEBUG is not set
CONFIG_FS_MBCACHE=m
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_XFS_FS=m
CONFIG_XFS_SUPPORT_V4=y
CONFIG_XFS_QUOTA=y
CONFIG_XFS_POSIX_ACL=y
# CONFIG_XFS_RT is not set
# CONFIG_XFS_ONLINE_SCRUB is not set
# CONFIG_XFS_WARN is not set
# CONFIG_XFS_DEBUG is not set
CONFIG_GFS2_FS=m
CONFIG_GFS2_FS_LOCKING_DLM=y
# CONFIG_OCFS2_FS is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
# CONFIG_F2FS_FS is not set
CONFIG_FS_DAX=y
CONFIG_FS_DAX_PMD=y
CONFIG_FS_POSIX_ACL=y
CONFIG_EXPORTFS=y
CONFIG_EXPORTFS_BLOCK_OPS=y
CONFIG_FILE_LOCKING=y
# CONFIG_MANDATORY_FILE_LOCKING is not set
# CONFIG_FS_ENCRYPTION is not set
# CONFIG_FS_VERITY is not set
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_FANOTIFY=y
CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
CONFIG_PRINT_QUOTA_WARNING=y
# CONFIG_QUOTA_DEBUG is not set
CONFIG_QUOTA_TREE=y
# CONFIG_QFMT_V1 is not set
CONFIG_QFMT_V2=y
CONFIG_QUOTACTL=y
CONFIG_AUTOFS4_FS=y
CONFIG_AUTOFS_FS=y
CONFIG_FUSE_FS=m
CONFIG_CUSE=m
CONFIG_VIRTIO_FS=m
CONFIG_FUSE_DAX=y
CONFIG_OVERLAY_FS=m
# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
# CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW is not set
# CONFIG_OVERLAY_FS_INDEX is not set
# CONFIG_OVERLAY_FS_XINO_AUTO is not set
# CONFIG_OVERLAY_FS_METACOPY is not set

#
# Caches
#
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
# CONFIG_FSCACHE_HISTOGRAM is not set
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=m
# CONFIG_CACHEFILES_DEBUG is not set
# CONFIG_CACHEFILES_HISTOGRAM is not set
# end of Caches

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=m
# end of CD-ROM/DVD Filesystems

#
# DOS/FAT/EXFAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="ascii"
# CONFIG_FAT_DEFAULT_UTF8 is not set
# CONFIG_EXFAT_FS is not set
# CONFIG_NTFS_FS is not set
# end of DOS/FAT/EXFAT/NT Filesystems

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_VMCORE=y
CONFIG_PROC_VMCORE_DEVICE_DUMP=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_PROC_CHILDREN=y
CONFIG_PROC_PID_ARCH_STATUS=y
CONFIG_PROC_CPU_RESCTRL=y
CONFIG_KERNFS=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_TMPFS_XATTR=y
# CONFIG_TMPFS_INODE64 is not set
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_MEMFD_CREATE=y
CONFIG_ARCH_HAS_GIGANTIC_PAGE=y
CONFIG_CONFIGFS_FS=y
CONFIG_EFIVAR_FS=y
# end of Pseudo filesystems

CONFIG_MISC_FILESYSTEMS=y
# CONFIG_ORANGEFS_FS is not set
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_ECRYPT_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_JFFS2_FS is not set
# CONFIG_UBIFS_FS is not set
CONFIG_CRAMFS=m
CONFIG_CRAMFS_BLOCKDEV=y
# CONFIG_CRAMFS_MTD is not set
CONFIG_SQUASHFS=m
# CONFIG_SQUASHFS_FILE_CACHE is not set
CONFIG_SQUASHFS_FILE_DIRECT=y
# CONFIG_SQUASHFS_DECOMP_SINGLE is not set
# CONFIG_SQUASHFS_DECOMP_MULTI is not set
CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU=y
CONFIG_SQUASHFS_XATTR=y
CONFIG_SQUASHFS_ZLIB=y
# CONFIG_SQUASHFS_LZ4 is not set
CONFIG_SQUASHFS_LZO=y
CONFIG_SQUASHFS_XZ=y
# CONFIG_SQUASHFS_ZSTD is not set
# CONFIG_SQUASHFS_4K_DEVBLK_SIZE is not set
# CONFIG_SQUASHFS_EMBEDDED is not set
CONFIG_SQUASHFS_FRAGMENT_CACHE_SIZE=3
# CONFIG_VXFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_OMFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_QNX6FS_FS is not set
# CONFIG_ROMFS_FS is not set
CONFIG_PSTORE=y
CONFIG_PSTORE_DEFAULT_KMSG_BYTES=10240
CONFIG_PSTORE_DEFLATE_COMPRESS=y
# CONFIG_PSTORE_LZO_COMPRESS is not set
# CONFIG_PSTORE_LZ4_COMPRESS is not set
# CONFIG_PSTORE_LZ4HC_COMPRESS is not set
# CONFIG_PSTORE_842_COMPRESS is not set
# CONFIG_PSTORE_ZSTD_COMPRESS is not set
CONFIG_PSTORE_COMPRESS=y
CONFIG_PSTORE_DEFLATE_COMPRESS_DEFAULT=y
CONFIG_PSTORE_COMPRESS_DEFAULT="deflate"
# CONFIG_PSTORE_CONSOLE is not set
# CONFIG_PSTORE_PMSG is not set
# CONFIG_PSTORE_FTRACE is not set
CONFIG_PSTORE_RAM=m
# CONFIG_PSTORE_BLK is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
# CONFIG_EROFS_FS is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=m
# CONFIG_NFS_V2 is not set
CONFIG_NFS_V3=m
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=m
# CONFIG_NFS_SWAP is not set
CONFIG_NFS_V4_1=y
CONFIG_NFS_V4_2=y
CONFIG_PNFS_FILE_LAYOUT=m
CONFIG_PNFS_BLOCK=m
CONFIG_PNFS_FLEXFILE_LAYOUT=m
CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="kernel.org"
# CONFIG_NFS_V4_1_MIGRATION is not set
CONFIG_NFS_V4_SECURITY_LABEL=y
CONFIG_NFS_FSCACHE=y
# CONFIG_NFS_USE_LEGACY_DNS is not set
CONFIG_NFS_USE_KERNEL_DNS=y
CONFIG_NFS_DEBUG=y
CONFIG_NFS_DISABLE_UDP_SUPPORT=y
# CONFIG_NFS_V4_2_READ_PLUS is not set
CONFIG_NFSD=m
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_PNFS=y
# CONFIG_NFSD_BLOCKLAYOUT is not set
CONFIG_NFSD_SCSILAYOUT=y
# CONFIG_NFSD_FLEXFILELAYOUT is not set
# CONFIG_NFSD_V4_2_INTER_SSC is not set
CONFIG_NFSD_V4_SECURITY_LABEL=y
CONFIG_GRACE_PERIOD=m
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_NFS_ACL_SUPPORT=m
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
CONFIG_SUNRPC_BACKCHANNEL=y
CONFIG_RPCSEC_GSS_KRB5=m
# CONFIG_SUNRPC_DISABLE_INSECURE_ENCTYPES is not set
CONFIG_SUNRPC_DEBUG=y
CONFIG_SUNRPC_XPRT_RDMA=m
CONFIG_CEPH_FS=m
# CONFIG_CEPH_FSCACHE is not set
CONFIG_CEPH_FS_POSIX_ACL=y
CONFIG_CEPH_FS_SECURITY_LABEL=y
CONFIG_CIFS=m
# CONFIG_CIFS_STATS2 is not set
CONFIG_CIFS_ALLOW_INSECURE_LEGACY=y
CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_UPCALL=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_CIFS_DEBUG=y
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_DEBUG_DUMP_KEYS is not set
CONFIG_CIFS_DFS_UPCALL=y
# CONFIG_CIFS_SWN_UPCALL is not set
CONFIG_CIFS_SMB_DIRECT=y
# CONFIG_CIFS_FSCACHE is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_MAC_ROMAN=m
CONFIG_NLS_MAC_CELTIC=m
CONFIG_NLS_MAC_CENTEURO=m
CONFIG_NLS_MAC_CROATIAN=m
CONFIG_NLS_MAC_CYRILLIC=m
CONFIG_NLS_MAC_GAELIC=m
CONFIG_NLS_MAC_GREEK=m
CONFIG_NLS_MAC_ICELAND=m
CONFIG_NLS_MAC_INUIT=m
CONFIG_NLS_MAC_ROMANIAN=m
CONFIG_NLS_MAC_TURKISH=m
CONFIG_NLS_UTF8=m
CONFIG_DLM=m
CONFIG_DLM_DEBUG=y
# CONFIG_UNICODE is not set
CONFIG_IO_WQ=y
# end of File systems

#
# Security options
#
CONFIG_KEYS=y
# CONFIG_KEYS_REQUEST_CACHE is not set
CONFIG_PERSISTENT_KEYRINGS=y
CONFIG_TRUSTED_KEYS=y
CONFIG_ENCRYPTED_KEYS=y
# CONFIG_KEY_DH_OPERATIONS is not set
# CONFIG_SECURITY_DMESG_RESTRICT is not set
CONFIG_SECURITY=y
CONFIG_SECURITY_WRITABLE_HOOKS=y
CONFIG_SECURITYFS=y
CONFIG_SECURITY_NETWORK=y
CONFIG_PAGE_TABLE_ISOLATION=y
CONFIG_SECURITY_INFINIBAND=y
CONFIG_SECURITY_NETWORK_XFRM=y
# CONFIG_SECURITY_PATH is not set
CONFIG_INTEL_TXT=y
CONFIG_LSM_MMAP_MIN_ADDR=65535
CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
CONFIG_HARDENED_USERCOPY=y
CONFIG_HARDENED_USERCOPY_FALLBACK=y
CONFIG_FORTIFY_SOURCE=y
# CONFIG_STATIC_USERMODEHELPER is not set
CONFIG_SECURITY_SELINUX=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_DISABLE=y
CONFIG_SECURITY_SELINUX_DEVELOP=y
CONFIG_SECURITY_SELINUX_AVC_STATS=y
CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
CONFIG_SECURITY_SELINUX_SIDTAB_HASH_BITS=9
CONFIG_SECURITY_SELINUX_SID2STR_CACHE_SIZE=256
# CONFIG_SECURITY_SMACK is not set
# CONFIG_SECURITY_TOMOYO is not set
# CONFIG_SECURITY_APPARMOR is not set
# CONFIG_SECURITY_LOADPIN is not set
CONFIG_SECURITY_YAMA=y
# CONFIG_SECURITY_SAFESETID is not set
# CONFIG_SECURITY_LOCKDOWN_LSM is not set
CONFIG_INTEGRITY=y
CONFIG_INTEGRITY_SIGNATURE=y
CONFIG_INTEGRITY_ASYMMETRIC_KEYS=y
CONFIG_INTEGRITY_TRUSTED_KEYRING=y
CONFIG_INTEGRITY_PLATFORM_KEYRING=y
CONFIG_LOAD_UEFI_KEYS=y
CONFIG_INTEGRITY_AUDIT=y
CONFIG_IMA=y
CONFIG_IMA_MEASURE_PCR_IDX=10
CONFIG_IMA_LSM_RULES=y
# CONFIG_IMA_TEMPLATE is not set
# CONFIG_IMA_NG_TEMPLATE is not set
CONFIG_IMA_SIG_TEMPLATE=y
CONFIG_IMA_DEFAULT_TEMPLATE="ima-sig"
# CONFIG_IMA_DEFAULT_HASH_SHA1 is not set
CONFIG_IMA_DEFAULT_HASH_SHA256=y
CONFIG_IMA_DEFAULT_HASH="sha256"
# CONFIG_IMA_WRITE_POLICY is not set
CONFIG_IMA_READ_POLICY=y
CONFIG_IMA_APPRAISE=y
CONFIG_IMA_ARCH_POLICY=y
# CONFIG_IMA_APPRAISE_BUILD_POLICY is not set
CONFIG_IMA_APPRAISE_BOOTPARAM=y
CONFIG_IMA_APPRAISE_MODSIG=y
CONFIG_IMA_TRUSTED_KEYRING=y
# CONFIG_IMA_BLACKLIST_KEYRING is not set
# CONFIG_IMA_LOAD_X509 is not set
CONFIG_IMA_MEASURE_ASYMMETRIC_KEYS=y
CONFIG_IMA_QUEUE_EARLY_BOOT_KEYS=y
CONFIG_IMA_SECURE_AND_OR_TRUSTED_BOOT=y
CONFIG_EVM=y
CONFIG_EVM_ATTR_FSUUID=y
# CONFIG_EVM_ADD_XATTRS is not set
# CONFIG_EVM_LOAD_X509 is not set
CONFIG_DEFAULT_SECURITY_SELINUX=y
# CONFIG_DEFAULT_SECURITY_DAC is not set
CONFIG_LSM="lockdown,yama,loadpin,safesetid,integrity,selinux,smack,tomoyo,apparmor,bpf"

#
# Kernel hardening options
#

#
# Memory initialization
#
CONFIG_INIT_STACK_NONE=y
# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
# end of Memory initialization
# end of Kernel hardening options
# end of Security options

CONFIG_XOR_BLOCKS=m
CONFIG_ASYNC_CORE=m
CONFIG_ASYNC_MEMCPY=m
CONFIG_ASYNC_XOR=m
CONFIG_ASYNC_PQ=m
CONFIG_ASYNC_RAID6_RECOV=m
CONFIG_CRYPTO=y

#
# Crypto core or helper
#
CONFIG_CRYPTO_FIPS=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_SKCIPHER=y
CONFIG_CRYPTO_SKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG=y
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_RNG_DEFAULT=y
CONFIG_CRYPTO_AKCIPHER2=y
CONFIG_CRYPTO_AKCIPHER=y
CONFIG_CRYPTO_KPP2=y
CONFIG_CRYPTO_KPP=m
CONFIG_CRYPTO_ACOMP2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
CONFIG_CRYPTO_USER=m
# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
# CONFIG_CRYPTO_MANAGER_EXTRA_TESTS is not set
CONFIG_CRYPTO_GF128MUL=y
CONFIG_CRYPTO_NULL=y
CONFIG_CRYPTO_NULL2=y
CONFIG_CRYPTO_PCRYPT=m
CONFIG_CRYPTO_CRYPTD=y
CONFIG_CRYPTO_AUTHENC=y
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_SIMD=y
CONFIG_CRYPTO_GLUE_HELPER_X86=y

#
# Public-key cryptography
#
CONFIG_CRYPTO_RSA=y
CONFIG_CRYPTO_DH=m
CONFIG_CRYPTO_ECC=m
CONFIG_CRYPTO_ECDH=m
# CONFIG_CRYPTO_ECRDSA is not set
# CONFIG_CRYPTO_SM2 is not set
# CONFIG_CRYPTO_CURVE25519 is not set
# CONFIG_CRYPTO_CURVE25519_X86 is not set

#
# Authenticated Encryption with Associated Data
#
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=y
CONFIG_CRYPTO_CHACHA20POLY1305=m
# CONFIG_CRYPTO_AEGIS128 is not set
# CONFIG_CRYPTO_AEGIS128_AESNI_SSE2 is not set
CONFIG_CRYPTO_SEQIV=y
CONFIG_CRYPTO_ECHAINIV=m

#
# Block modes
#
CONFIG_CRYPTO_CBC=y
CONFIG_CRYPTO_CFB=y
CONFIG_CRYPTO_CTR=y
CONFIG_CRYPTO_CTS=y
CONFIG_CRYPTO_ECB=y
CONFIG_CRYPTO_LRW=m
# CONFIG_CRYPTO_OFB is not set
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_XTS=m
# CONFIG_CRYPTO_KEYWRAP is not set
# CONFIG_CRYPTO_NHPOLY1305_SSE2 is not set
# CONFIG_CRYPTO_NHPOLY1305_AVX2 is not set
# CONFIG_CRYPTO_ADIANTUM is not set
CONFIG_CRYPTO_ESSIV=y

#
# Hash modes
#
CONFIG_CRYPTO_CMAC=m
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_VMAC=m

#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
CONFIG_CRYPTO_CRC32C_INTEL=m
CONFIG_CRYPTO_CRC32=m
CONFIG_CRYPTO_CRC32_PCLMUL=m
# CONFIG_CRYPTO_XXHASH is not set
# CONFIG_CRYPTO_BLAKE2B is not set
# CONFIG_CRYPTO_BLAKE2S is not set
# CONFIG_CRYPTO_BLAKE2S_X86 is not set
CONFIG_CRYPTO_CRCT10DIF=y
CONFIG_CRYPTO_CRCT10DIF_PCLMUL=m
CONFIG_CRYPTO_GHASH=y
CONFIG_CRYPTO_POLY1305=m
CONFIG_CRYPTO_POLY1305_X86_64=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD128=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_RMD256=m
CONFIG_CRYPTO_RMD320=m
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA1_SSSE3=y
CONFIG_CRYPTO_SHA256_SSSE3=y
CONFIG_CRYPTO_SHA512_SSSE3=m
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_SHA3=m
# CONFIG_CRYPTO_SM3 is not set
# CONFIG_CRYPTO_STREEBOG is not set
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL=m

#
# Ciphers
#
CONFIG_CRYPTO_AES=y
# CONFIG_CRYPTO_AES_TI is not set
CONFIG_CRYPTO_AES_NI_INTEL=y
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_BLOWFISH_COMMON=m
CONFIG_CRYPTO_BLOWFISH_X86_64=m
CONFIG_CRYPTO_CAMELLIA=m
CONFIG_CRYPTO_CAMELLIA_X86_64=m
CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64=m
CONFIG_CRYPTO_CAMELLIA_AESNI_AVX2_X86_64=m
CONFIG_CRYPTO_CAST_COMMON=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST5_AVX_X86_64=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_CAST6_AVX_X86_64=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_DES3_EDE_X86_64=m
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_SALSA20=m
CONFIG_CRYPTO_CHACHA20=m
CONFIG_CRYPTO_CHACHA20_X86_64=m
CONFIG_CRYPTO_SEED=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_SERPENT_SSE2_X86_64=m
CONFIG_CRYPTO_SERPENT_AVX_X86_64=m
CONFIG_CRYPTO_SERPENT_AVX2_X86_64=m
# CONFIG_CRYPTO_SM4 is not set
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_TWOFISH_COMMON=m
CONFIG_CRYPTO_TWOFISH_X86_64=m
CONFIG_CRYPTO_TWOFISH_X86_64_3WAY=m
CONFIG_CRYPTO_TWOFISH_AVX_X86_64=m

#
# Compression
#
CONFIG_CRYPTO_DEFLATE=y
CONFIG_CRYPTO_LZO=y
# CONFIG_CRYPTO_842 is not set
# CONFIG_CRYPTO_LZ4 is not set
# CONFIG_CRYPTO_LZ4HC is not set
# CONFIG_CRYPTO_ZSTD is not set

#
# Random Number Generation
#
CONFIG_CRYPTO_ANSI_CPRNG=m
CONFIG_CRYPTO_DRBG_MENU=y
CONFIG_CRYPTO_DRBG_HMAC=y
CONFIG_CRYPTO_DRBG_HASH=y
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_DRBG=y
CONFIG_CRYPTO_JITTERENTROPY=y
CONFIG_CRYPTO_USER_API=y
CONFIG_CRYPTO_USER_API_HASH=y
CONFIG_CRYPTO_USER_API_SKCIPHER=y
CONFIG_CRYPTO_USER_API_RNG=y
# CONFIG_CRYPTO_USER_API_RNG_CAVP is not set
CONFIG_CRYPTO_USER_API_AEAD=y
CONFIG_CRYPTO_USER_API_ENABLE_OBSOLETE=y
# CONFIG_CRYPTO_STATS is not set
CONFIG_CRYPTO_HASH_INFO=y

#
# Crypto library routines
#
CONFIG_CRYPTO_LIB_AES=y
CONFIG_CRYPTO_LIB_ARC4=m
# CONFIG_CRYPTO_LIB_BLAKE2S is not set
CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA=m
CONFIG_CRYPTO_LIB_CHACHA_GENERIC=m
# CONFIG_CRYPTO_LIB_CHACHA is not set
# CONFIG_CRYPTO_LIB_CURVE25519 is not set
CONFIG_CRYPTO_LIB_DES=m
CONFIG_CRYPTO_LIB_POLY1305_RSIZE=11
CONFIG_CRYPTO_ARCH_HAVE_LIB_POLY1305=m
CONFIG_CRYPTO_LIB_POLY1305_GENERIC=m
# CONFIG_CRYPTO_LIB_POLY1305 is not set
# CONFIG_CRYPTO_LIB_CHACHA20POLY1305 is not set
CONFIG_CRYPTO_LIB_SHA256=y
CONFIG_CRYPTO_HW=y
CONFIG_CRYPTO_DEV_PADLOCK=m
CONFIG_CRYPTO_DEV_PADLOCK_AES=m
CONFIG_CRYPTO_DEV_PADLOCK_SHA=m
# CONFIG_CRYPTO_DEV_ATMEL_ECC is not set
# CONFIG_CRYPTO_DEV_ATMEL_SHA204A is not set
CONFIG_CRYPTO_DEV_CCP=y
CONFIG_CRYPTO_DEV_CCP_DD=m
CONFIG_CRYPTO_DEV_SP_CCP=y
CONFIG_CRYPTO_DEV_CCP_CRYPTO=m
CONFIG_CRYPTO_DEV_SP_PSP=y
# CONFIG_CRYPTO_DEV_CCP_DEBUGFS is not set
CONFIG_CRYPTO_DEV_QAT=m
CONFIG_CRYPTO_DEV_QAT_DH895xCC=m
CONFIG_CRYPTO_DEV_QAT_C3XXX=m
CONFIG_CRYPTO_DEV_QAT_C62X=m
# CONFIG_CRYPTO_DEV_QAT_4XXX is not set
CONFIG_CRYPTO_DEV_QAT_DH895xCCVF=m
CONFIG_CRYPTO_DEV_QAT_C3XXXVF=m
CONFIG_CRYPTO_DEV_QAT_C62XVF=m
CONFIG_CRYPTO_DEV_NITROX=m
CONFIG_CRYPTO_DEV_NITROX_CNN55XX=m
CONFIG_CRYPTO_DEV_CHELSIO=m
# CONFIG_CRYPTO_DEV_VIRTIO is not set
# CONFIG_CRYPTO_DEV_SAFEXCEL is not set
# CONFIG_CRYPTO_DEV_AMLOGIC_GXL is not set
CONFIG_ASYMMETRIC_KEY_TYPE=y
CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=y
# CONFIG_ASYMMETRIC_TPM_KEY_SUBTYPE is not set
CONFIG_X509_CERTIFICATE_PARSER=y
# CONFIG_PKCS8_PRIVATE_KEY_PARSER is not set
CONFIG_PKCS7_MESSAGE_PARSER=y
# CONFIG_PKCS7_TEST_KEY is not set
CONFIG_SIGNED_PE_FILE_VERIFICATION=y

#
# Certificates for signature checking
#
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS=""
# CONFIG_SYSTEM_EXTRA_CERTIFICATE is not set
# CONFIG_SECONDARY_TRUSTED_KEYRING is not set
CONFIG_SYSTEM_BLACKLIST_KEYRING=y
CONFIG_SYSTEM_BLACKLIST_HASH_LIST=""
# end of Certificates for signature checking

CONFIG_BINARY_PRINTF=y

#
# Library routines
#
CONFIG_RAID6_PQ=m
CONFIG_RAID6_PQ_BENCHMARK=y
# CONFIG_PACKING is not set
CONFIG_BITREVERSE=y
CONFIG_GENERIC_STRNCPY_FROM_USER=y
CONFIG_GENERIC_STRNLEN_USER=y
CONFIG_GENERIC_NET_UTILS=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_CORDIC=m
# CONFIG_PRIME_NUMBERS is not set
CONFIG_RATIONAL=y
CONFIG_GENERIC_PCI_IOMAP=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_USE_CMPXCHG_LOCKREF=y
CONFIG_ARCH_HAS_FAST_MULTIPLIER=y
CONFIG_ARCH_USE_SYM_ANNOTATIONS=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=m
CONFIG_CRC32=y
# CONFIG_CRC32_SELFTEST is not set
CONFIG_CRC32_SLICEBY8=y
# CONFIG_CRC32_SLICEBY4 is not set
# CONFIG_CRC32_SARWATE is not set
# CONFIG_CRC32_BIT is not set
# CONFIG_CRC64 is not set
# CONFIG_CRC4 is not set
CONFIG_CRC7=m
CONFIG_LIBCRC32C=m
CONFIG_CRC8=m
CONFIG_XXHASH=y
# CONFIG_RANDOM32_SELFTEST is not set
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_LZ4_DECOMPRESS=y
CONFIG_ZSTD_DECOMPRESS=y
CONFIG_XZ_DEC=y
CONFIG_XZ_DEC_X86=y
CONFIG_XZ_DEC_POWERPC=y
CONFIG_XZ_DEC_IA64=y
CONFIG_XZ_DEC_ARM=y
CONFIG_XZ_DEC_ARMTHUMB=y
CONFIG_XZ_DEC_SPARC=y
CONFIG_XZ_DEC_BCJ=y
# CONFIG_XZ_DEC_TEST is not set
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_XZ=y
CONFIG_DECOMPRESS_LZO=y
CONFIG_DECOMPRESS_LZ4=y
CONFIG_DECOMPRESS_ZSTD=y
CONFIG_GENERIC_ALLOCATOR=y
CONFIG_REED_SOLOMON=m
CONFIG_REED_SOLOMON_ENC8=y
CONFIG_REED_SOLOMON_DEC8=y
CONFIG_TEXTSEARCH=y
CONFIG_TEXTSEARCH_KMP=m
CONFIG_TEXTSEARCH_BM=m
CONFIG_TEXTSEARCH_FSM=m
CONFIG_BTREE=y
CONFIG_INTERVAL_TREE=y
CONFIG_XARRAY_MULTI=y
CONFIG_ASSOCIATIVE_ARRAY=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT_MAP=y
CONFIG_HAS_DMA=y
CONFIG_DMA_OPS=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_ARCH_HAS_FORCE_DMA_UNENCRYPTED=y
CONFIG_SWIOTLB=y
CONFIG_DMA_COHERENT_POOL=y
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_DMA_MAP_BENCHMARK is not set
CONFIG_SGL_ALLOC=y
CONFIG_CHECK_SIGNATURE=y
CONFIG_CPUMASK_OFFSTACK=y
CONFIG_CPU_RMAP=y
CONFIG_DQL=y
CONFIG_GLOB=y
# CONFIG_GLOB_SELFTEST is not set
CONFIG_NLATTR=y
CONFIG_CLZ_TAB=y
CONFIG_IRQ_POLL=y
CONFIG_MPILIB=y
CONFIG_SIGNATURE=y
CONFIG_DIMLIB=y
CONFIG_OID_REGISTRY=y
CONFIG_UCS2_STRING=y
CONFIG_HAVE_GENERIC_VDSO=y
CONFIG_GENERIC_GETTIMEOFDAY=y
CONFIG_GENERIC_VDSO_TIME_NS=y
CONFIG_FONT_SUPPORT=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
CONFIG_SG_POOL=y
CONFIG_ARCH_HAS_PMEM_API=y
CONFIG_MEMREGION=y
CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE=y
CONFIG_ARCH_HAS_COPY_MC=y
CONFIG_ARCH_STACKWALK=y
CONFIG_STACKDEPOT=y
CONFIG_SBITMAP=y
CONFIG_PARMAN=m
CONFIG_OBJAGG=m
# CONFIG_STRING_SELFTEST is not set
# end of Library routines

CONFIG_PLDMFW=y

#
# Kernel hacking
#

#
# printk and dmesg options
#
CONFIG_PRINTK_TIME=y
# CONFIG_PRINTK_CALLER is not set
CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
CONFIG_CONSOLE_LOGLEVEL_QUIET=4
CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
CONFIG_BOOT_PRINTK_DELAY=y
CONFIG_DYNAMIC_DEBUG=y
CONFIG_DYNAMIC_DEBUG_CORE=y
CONFIG_SYMBOLIC_ERRNAME=y
CONFIG_DEBUG_BUGVERBOSE=y
# end of printk and dmesg options

#
# Compile-time checks and compiler options
#
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
# CONFIG_DEBUG_INFO_COMPRESSED is not set
# CONFIG_DEBUG_INFO_SPLIT is not set
CONFIG_DEBUG_INFO_DWARF4=y
# CONFIG_DEBUG_INFO_BTF is not set
CONFIG_PAHOLE_HAS_SPLIT_BTF=y
# CONFIG_GDB_SCRIPTS is not set
CONFIG_FRAME_WARN=2048
CONFIG_STRIP_ASM_SYMS=y
# CONFIG_READABLE_ASM is not set
# CONFIG_HEADERS_INSTALL is not set
CONFIG_DEBUG_SECTION_MISMATCH=y
CONFIG_SECTION_MISMATCH_WARN_ONLY=y
CONFIG_STACK_VALIDATION=y
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
# end of Compile-time checks and compiler options

#
# Generic Kernel Debugging Instruments
#
CONFIG_MAGIC_SYSRQ=y
CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x1
CONFIG_MAGIC_SYSRQ_SERIAL=y
CONFIG_MAGIC_SYSRQ_SERIAL_SEQUENCE=""
CONFIG_DEBUG_FS=y
CONFIG_DEBUG_FS_ALLOW_ALL=y
# CONFIG_DEBUG_FS_DISALLOW_MOUNT is not set
# CONFIG_DEBUG_FS_ALLOW_NONE is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_KGDB=y
CONFIG_KGDB_HONOUR_BLOCKLIST=y
CONFIG_KGDB_SERIAL_CONSOLE=y
CONFIG_KGDB_TESTS=y
# CONFIG_KGDB_TESTS_ON_BOOT is not set
CONFIG_KGDB_LOW_LEVEL_TRAP=y
CONFIG_KGDB_KDB=y
CONFIG_KDB_DEFAULT_ENABLE=0x0
CONFIG_KDB_KEYBOARD=y
CONFIG_KDB_CONTINUE_CATASTROPHIC=0
CONFIG_ARCH_HAS_EARLY_DEBUG=y
CONFIG_ARCH_HAS_UBSAN_SANITIZE_ALL=y
# CONFIG_UBSAN is not set
CONFIG_HAVE_ARCH_KCSAN=y
# end of Generic Kernel Debugging Instruments

CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_MISC=y

#
# Memory Debugging
#
CONFIG_PAGE_EXTENSION=y
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_PAGE_OWNER=y
# CONFIG_PAGE_POISONING is not set
# CONFIG_DEBUG_PAGE_REF is not set
# CONFIG_DEBUG_RODATA_TEST is not set
CONFIG_ARCH_HAS_DEBUG_WX=y
# CONFIG_DEBUG_WX is not set
CONFIG_GENERIC_PTDUMP=y
# CONFIG_PTDUMP_DEBUGFS is not set
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_SLUB_STATS is not set
CONFIG_HAVE_DEBUG_KMEMLEAK=y
# CONFIG_DEBUG_KMEMLEAK is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_SCHED_STACK_END_CHECK is not set
CONFIG_ARCH_HAS_DEBUG_VM_PGTABLE=y
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_VM_PGTABLE is not set
CONFIG_ARCH_HAS_DEBUG_VIRTUAL=y
# CONFIG_DEBUG_VIRTUAL is not set
CONFIG_DEBUG_MEMORY_INIT=y
# CONFIG_DEBUG_PER_CPU_MAPS is not set
CONFIG_HAVE_ARCH_KASAN=y
CONFIG_HAVE_ARCH_KASAN_VMALLOC=y
CONFIG_CC_HAS_KASAN_GENERIC=y
CONFIG_CC_HAS_WORKING_NOSANITIZE_ADDRESS=y
# CONFIG_KASAN is not set
# end of Memory Debugging

CONFIG_DEBUG_SHIRQ=y

#
# Debug Oops, Lockups and Hangs
#
CONFIG_PANIC_ON_OOPS=y
CONFIG_PANIC_ON_OOPS_VALUE=1
CONFIG_PANIC_TIMEOUT=0
CONFIG_LOCKUP_DETECTOR=y
CONFIG_SOFTLOCKUP_DETECTOR=y
# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
CONFIG_HARDLOCKUP_DETECTOR_PERF=y
CONFIG_HARDLOCKUP_CHECK_TIMESTAMP=y
CONFIG_HARDLOCKUP_DETECTOR=y
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC=y
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE=1
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120
# CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not set
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0
# CONFIG_WQ_WATCHDOG is not set
# CONFIG_TEST_LOCKUP is not set
# end of Debug Oops, Lockups and Hangs

#
# Scheduler Debugging
#
CONFIG_SCHED_DEBUG=y
CONFIG_SCHED_INFO=y
CONFIG_SCHEDSTATS=y
# end of Scheduler Debugging

# CONFIG_DEBUG_TIMEKEEPING is not set

#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_LOCK_DEBUGGING_SUPPORT=y
# CONFIG_PROVE_LOCKING is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_MUTEXES is not set
# CONFIG_DEBUG_WW_MUTEX_SLOWPATH is not set
# CONFIG_DEBUG_RWSEMS is not set
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_DEBUG_ATOMIC_SLEEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_LOCK_TORTURE_TEST=m
# CONFIG_WW_MUTEX_SELFTEST is not set
# CONFIG_SCF_TORTURE_TEST is not set
# CONFIG_CSD_LOCK_WAIT_DEBUG is not set
# end of Lock Debugging (spinlocks, mutexes, etc...)

CONFIG_STACKTRACE=y
# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
# CONFIG_DEBUG_KOBJECT is not set

#
# Debug kernel data structures
#
CONFIG_DEBUG_LIST=y
# CONFIG_DEBUG_PLIST is not set
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
CONFIG_BUG_ON_DATA_CORRUPTION=y
# end of Debug kernel data structures

# CONFIG_DEBUG_CREDENTIALS is not set

#
# RCU Debugging
#
CONFIG_TORTURE_TEST=m
# CONFIG_RCU_SCALE_TEST is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_REF_SCALE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=60
# CONFIG_RCU_TRACE is not set
# CONFIG_RCU_EQS_DEBUG is not set
# end of RCU Debugging

# CONFIG_DEBUG_WQ_FORCE_RR_CPU is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_CPU_HOTPLUG_STATE_CONTROL is not set
# CONFIG_LATENCYTOP is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_FENTRY=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACER_MAX_TRACE=y
CONFIG_TRACE_CLOCK=y
CONFIG_RING_BUFFER=y
CONFIG_EVENT_TRACING=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_RING_BUFFER_ALLOW_SWAP=y
CONFIG_TRACING=y
CONFIG_GENERIC_TRACER=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
# CONFIG_BOOTTIME_TRACING is not set
CONFIG_FUNCTION_TRACER=y
CONFIG_FUNCTION_GRAPH_TRACER=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_FUNCTION_PROFILER=y
CONFIG_STACK_TRACER=y
# CONFIG_IRQSOFF_TRACER is not set
CONFIG_SCHED_TRACER=y
CONFIG_HWLAT_TRACER=y
# CONFIG_MMIOTRACE is not set
CONFIG_FTRACE_SYSCALLS=y
CONFIG_TRACER_SNAPSHOT=y
# CONFIG_TRACER_SNAPSHOT_PER_CPU_SWAP is not set
CONFIG_BRANCH_PROFILE_NONE=y
# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_KPROBE_EVENTS=y
# CONFIG_KPROBE_EVENTS_ON_NOTRACE is not set
CONFIG_UPROBE_EVENTS=y
CONFIG_BPF_EVENTS=y
CONFIG_DYNAMIC_EVENTS=y
CONFIG_PROBE_EVENTS=y
# CONFIG_BPF_KPROBE_OVERRIDE is not set
CONFIG_FTRACE_MCOUNT_RECORD=y
CONFIG_TRACING_MAP=y
CONFIG_SYNTH_EVENTS=y
CONFIG_HIST_TRIGGERS=y
# CONFIG_TRACE_EVENT_INJECT is not set
# CONFIG_TRACEPOINT_BENCHMARK is not set
CONFIG_RING_BUFFER_BENCHMARK=m
# CONFIG_TRACE_EVAL_MAP_FILE is not set
# CONFIG_FTRACE_RECORD_RECURSION is not set
# CONFIG_FTRACE_STARTUP_TEST is not set
# CONFIG_RING_BUFFER_STARTUP_TEST is not set
# CONFIG_RING_BUFFER_VALIDATE_TIME_DELTAS is not set
# CONFIG_PREEMPTIRQ_DELAY_TEST is not set
# CONFIG_SYNTH_EVENT_GEN_TEST is not set
# CONFIG_KPROBE_EVENT_GEN_TEST is not set
# CONFIG_HIST_TRIGGERS_DEBUG is not set
CONFIG_PROVIDE_OHCI1394_DMA_INIT=y
# CONFIG_SAMPLES is not set
CONFIG_ARCH_HAS_DEVMEM_IS_ALLOWED=y
CONFIG_STRICT_DEVMEM=y
# CONFIG_IO_STRICT_DEVMEM is not set

#
# x86 Debugging
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_NMI_SUPPORT=y
CONFIG_EARLY_PRINTK_USB=y
# CONFIG_X86_VERBOSE_BOOTUP is not set
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_PRINTK_DBGP=y
CONFIG_EARLY_PRINTK_USB_XDBC=y
# CONFIG_EFI_PGT_DUMP is not set
# CONFIG_DEBUG_TLBFLUSH is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_X86_DECODER_SELFTEST=y
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEBUG_BOOT_PARAMS=y
# CONFIG_CPA_DEBUG is not set
# CONFIG_DEBUG_ENTRY is not set
# CONFIG_DEBUG_NMI_SELFTEST is not set
# CONFIG_X86_DEBUG_FPU is not set
# CONFIG_PUNIT_ATOM_DEBUG is not set
CONFIG_UNWINDER_ORC=y
# CONFIG_UNWINDER_FRAME_POINTER is not set
# end of x86 Debugging

#
# Kernel Testing and Coverage
#
CONFIG_KUNIT=m
CONFIG_KUNIT_DEBUGFS=y
CONFIG_KUNIT_TEST=m
CONFIG_KUNIT_EXAMPLE_TEST=m
CONFIG_KUNIT_ALL_TESTS=m
# CONFIG_NOTIFIER_ERROR_INJECTION is not set
CONFIG_FUNCTION_ERROR_INJECTION=y
# CONFIG_FAULT_INJECTION is not set
CONFIG_ARCH_HAS_KCOV=y
CONFIG_CC_HAS_SANCOV_TRACE_PC=y
# CONFIG_KCOV is not set
CONFIG_RUNTIME_TESTING_MENU=y
# CONFIG_LKDTM is not set
# CONFIG_TEST_LIST_SORT is not set
# CONFIG_TEST_MIN_HEAP is not set
# CONFIG_TEST_SORT is not set
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_RBTREE_TEST is not set
# CONFIG_REED_SOLOMON_TEST is not set
# CONFIG_INTERVAL_TREE_TEST is not set
# CONFIG_PERCPU_TEST is not set
CONFIG_ATOMIC64_SELFTEST=y
CONFIG_ASYNC_RAID6_TEST=m
# CONFIG_TEST_HEXDUMP is not set
# CONFIG_TEST_STRING_HELPERS is not set
# CONFIG_TEST_STRSCPY is not set
CONFIG_TEST_KSTRTOX=y
# CONFIG_TEST_PRINTF is not set
# CONFIG_TEST_BITMAP is not set
# CONFIG_TEST_UUID is not set
# CONFIG_TEST_XARRAY is not set
# CONFIG_TEST_OVERFLOW is not set
# CONFIG_TEST_RHASHTABLE is not set
# CONFIG_TEST_HASH is not set
# CONFIG_TEST_IDA is not set
# CONFIG_TEST_PARMAN is not set
# CONFIG_TEST_LKM is not set
# CONFIG_TEST_BITOPS is not set
# CONFIG_TEST_VMALLOC is not set
# CONFIG_TEST_USER_COPY is not set
CONFIG_TEST_BPF=m
# CONFIG_TEST_BLACKHOLE_DEV is not set
# CONFIG_FIND_BIT_BENCHMARK is not set
# CONFIG_TEST_FIRMWARE is not set
# CONFIG_TEST_SYSCTL is not set
# CONFIG_BITFIELD_KUNIT is not set
# CONFIG_RESOURCE_KUNIT_TEST is not set
CONFIG_SYSCTL_KUNIT_TEST=m
CONFIG_LIST_KUNIT_TEST=m
# CONFIG_LINEAR_RANGES_TEST is not set
# CONFIG_CMDLINE_KUNIT_TEST is not set
# CONFIG_BITS_TEST is not set
# CONFIG_TEST_UDELAY is not set
# CONFIG_TEST_STATIC_KEYS is not set
# CONFIG_TEST_KMOD is not set
# CONFIG_TEST_MEMCAT_P is not set
CONFIG_TEST_LIVEPATCH=m
# CONFIG_TEST_OBJAGG is not set
# CONFIG_TEST_STACKINIT is not set
# CONFIG_TEST_MEMINIT is not set
# CONFIG_TEST_HMM is not set
# CONFIG_TEST_FREE_PAGES is not set
# CONFIG_TEST_FPU is not set
# CONFIG_MEMTEST is not set
# CONFIG_HYPERV_TESTING is not set
# end of Kernel Testing and Coverage
# end of Kernel hacking

From minwoo.im.dev at gmail.com  Mon Feb  8 09:57:47 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Mon, 8 Feb 2021 23:57:47 +0900
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
In-Reply-To: <BL0PR04MB651449A902E7FAD397E2169EE78F9@BL0PR04MB6514.namprd04.prod.outlook.com>
References: <20210205195002.2657209-1-kbusch@kernel.org>
 <BL0PR04MB65147CDC426FC6345B2048D7E7B09@BL0PR04MB6514.namprd04.prod.outlook.com>
 <20210208032851.GA14062@redsun51.ssa.fujisawa.hgst.com>
 <BL0PR04MB651449A902E7FAD397E2169EE78F9@BL0PR04MB6514.namprd04.prod.outlook.com>
Message-ID: <20210208145747.GA8941@localhost.localdomain>

> Hmmm... I see the zone model being set for ns->queue, but nothing for
> ns->head->disk->queue... Shoudln't nvme_mpath_add_disk() also set the zoned
> model for ns->head->disk->queue ?

I think zone model shows properly with the simple multi-path example:

	cat /sys/block/nvme0n1/queue/zoned:		host-managed
	cat /sys/block/nvme0c0n1/queue/zoned:		host-managed
	cat /sys/block/nvme0n1/queue/nr_zones:		0
	cat /sys/block/nvme0c0n1/queue/nr_zones:	2048

But, there are also little more differences between the path namespace
and head namespace:

	cat /sys/block/nvme0n1/queue/nr_requests:	128
	cat /sys/block/nvme0c0n1/queue/nr_requests:	1023
	cat /sys/block/nvme0n1/queue/iostats:		0
	cat /sys/block/nvme0c0n1/queue/iostats:		1
	cat /sys/block/nvme0n1/queue/rq_affinity:	0
	cat /sys/block/nvme0c0n1/queue/rq_affinity:	1

+ Question with Damien's one:  Shouldn't we sync all the differences
between the two request_queues?


From hch at lst.de  Mon Feb  8 10:05:09 2021
From: hch at lst.de (hch at lst.de)
Date: Mon, 8 Feb 2021 16:05:09 +0100
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
In-Reply-To: <20210208145747.GA8941@localhost.localdomain>
References: <20210205195002.2657209-1-kbusch@kernel.org>
 <BL0PR04MB65147CDC426FC6345B2048D7E7B09@BL0PR04MB6514.namprd04.prod.outlook.com>
 <20210208032851.GA14062@redsun51.ssa.fujisawa.hgst.com>
 <BL0PR04MB651449A902E7FAD397E2169EE78F9@BL0PR04MB6514.namprd04.prod.outlook.com>
 <20210208145747.GA8941@localhost.localdomain>
Message-ID: <20210208150509.GA12734@lst.de>

On Mon, Feb 08, 2021 at 11:57:47PM +0900, Minwoo Im wrote:
> But, there are also little more differences between the path namespace
> and head namespace:
> 
> 	cat /sys/block/nvme0n1/queue/nr_requests:	128
> 	cat /sys/block/nvme0c0n1/queue/nr_requests:	1023

nr_request is a noop for bio-based drivers.

> 	cat /sys/block/nvme0n1/queue/iostats:		0
> 	cat /sys/block/nvme0c0n1/queue/iostats:		1

iostats for bio based drivers require extra code, and we don't really
need those.

> 	cat /sys/block/nvme0n1/queue/rq_affinity:	0
> 	cat /sys/block/nvme0c0n1/queue/rq_affinity:	1

rq_affinity is a noop for bio based drivers.


From kbusch at kernel.org  Mon Feb  8 10:14:14 2021
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 9 Feb 2021 00:14:14 +0900
Subject: [PATCH] nvme-multipath: set nr_zones for zoned namespaces
In-Reply-To: <BL0PR04MB651449A902E7FAD397E2169EE78F9@BL0PR04MB6514.namprd04.prod.outlook.com>
References: <20210205195002.2657209-1-kbusch@kernel.org>
 <BL0PR04MB65147CDC426FC6345B2048D7E7B09@BL0PR04MB6514.namprd04.prod.outlook.com>
 <20210208032851.GA14062@redsun51.ssa.fujisawa.hgst.com>
 <BL0PR04MB651449A902E7FAD397E2169EE78F9@BL0PR04MB6514.namprd04.prod.outlook.com>
Message-ID: <20210208151414.GA16360@redsun51.ssa.fujisawa.hgst.com>

On Mon, Feb 08, 2021 at 03:51:18AM +0000, Damien Le Moal wrote:
> 
> Hmmm... I see the zone model being set for ns->queue, but nothing for
> ns->head->disk->queue... Shoudln't nvme_mpath_add_disk() also set the zoned
> model for ns->head->disk->queue ?

That gets set via nvme_update_ns_info()->blk_stack_limits().


From Martin.Belanger at dell.com  Mon Feb  8 11:32:16 2021
From: Martin.Belanger at dell.com (Belanger, Martin)
Date: Mon, 8 Feb 2021 16:32:16 +0000
Subject: Clarification regarding "nvme discover" and setting IOSQES/IOCQES
Message-ID: <SJ0PR19MB4544CE3903DEEA63D0C3929FF28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>

I'm using "nvme discover" to retrieve discovery log pages. And I'm using nvmet-tcp as the target.

>From what I understand, "nvme discover" is supposed to set up a connection to a "Discovery Controller (DC)". During the exchange between the Host and the Target, there is a time when the Host must set CC.EN=1 to "enable" the DC. I'm using Wireshark to analyse the packets sent by the Host to the DC. I can see that when it's time to set CC.EN=1, the Host also sets two additional parameters: IOCQES=4 and IOSQES=6. 

In the NVME Base Specification, Figure 78 - Controller Configuration, these fields are defined as parameters to configure an "I/O Controller (IOC)" and not to configure a "Discovery Controller (DC)". In fact, it says that "If the controller does not support I/O queues, then this field shall be read-only with a value of 0h". 

The way I understand it, DCs do not support I/O queues. So why set IOCQES and IOSQES to non-zero values?

Thanks,
Martin

From hch at lst.de  Mon Feb  8 12:17:43 2021
From: hch at lst.de (Christoph Hellwig)
Date: Mon, 8 Feb 2021 18:17:43 +0100
Subject: [PATCH v4 0/8] block: add zone write granularity limit
In-Reply-To: <BL0PR04MB65147501739ABF4F27B72290E7B39@BL0PR04MB6514.namprd04.prod.outlook.com>
References: <20210128044733.503606-1-damien.lemoal@wdc.com>
 <BL0PR04MB65147501739ABF4F27B72290E7B39@BL0PR04MB6514.namprd04.prod.outlook.com>
Message-ID: <20210208171743.GA15614@lst.de>

On Thu, Feb 04, 2021 at 08:47:51AM +0000, Damien Le Moal wrote:
> Any comment on this series ?
> 
> Martin,
> 
> The scsi bits (patch 5 and 8) may need your ack.

ping?  Jens, having this series really helps with some of the
ongoing zoned device work.


From sagi at grimberg.me  Mon Feb  8 12:54:05 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 09:54:05 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
Message-ID: <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>


> Hi Sagi
> 
> On 2/8/21 5:46 PM, Sagi Grimberg wrote:
>>
>>> Hello
>>>
>>> We found this kernel NULL pointer issue with latest 
>>> linux-block/for-next and it's 100% reproduced, let me know if you 
>>> need more info/testing, thanks
>>>
>>> Kernel repo: 
>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>>
>>> Reproducer: blktests nvme-tcp/012
>>
>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>> but did not succeed. Given that you have it 100% reproducible,
>> can you try to revert commit:
>>
>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>
> 
> Revert this commit fixed the issue and I've attached the config. :)

Good to know,

I see some differences that I should probably change to hit this:
--
@@ -254,14 +256,15 @@ CONFIG_PERF_EVENTS=y
  # end of Kernel Performance Events And Counters

  CONFIG_VM_EVENT_COUNTERS=y
+CONFIG_SLUB_DEBUG=y
  # CONFIG_COMPAT_BRK is not set
-CONFIG_SLAB=y
-# CONFIG_SLUB is not set
-# CONFIG_SLOB is not set
-CONFIG_SLAB_MERGE_DEFAULT=y
-# CONFIG_SLAB_FREELIST_RANDOM is not set
+# CONFIG_SLAB is not set
+CONFIG_SLUB=y
+# CONFIG_SLAB_MERGE_DEFAULT is not set
+CONFIG_SLAB_FREELIST_RANDOM=y
  # CONFIG_SLAB_FREELIST_HARDENED is not set
-# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
+CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
+CONFIG_SLUB_CPU_PARTIAL=y
  CONFIG_SYSTEM_DATA_VERIFICATION=y
  CONFIG_PROFILING=y
  CONFIG_TRACEPOINTS=y
@@ -299,7 +302,8 @@ CONFIG_HAVE_INTEL_TXT=y
  CONFIG_X86_64_SMP=y
  CONFIG_ARCH_SUPPORTS_UPROBES=y
  CONFIG_FIX_EARLYCON_MEM=y
-CONFIG_PGTABLE_LEVELS=4
+CONFIG_DYNAMIC_PHYSICAL_MASK=y
+CONFIG_PGTABLE_LEVELS=5
  CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
--

Probably CONFIG_SLUB and CONFIG_SLUB_DEBUG should be used.


From sashal at kernel.org  Mon Feb  8 12:57:49 2021
From: sashal at kernel.org (Sasha Levin)
Date: Mon,  8 Feb 2021 12:57:49 -0500
Subject: [PATCH AUTOSEL 5.10 19/36] nvme-pci: ignore the subsysem NQN on
 Phison E16
In-Reply-To: <20210208175806.2091668-1-sashal@kernel.org>
References: <20210208175806.2091668-1-sashal@kernel.org>
Message-ID: <20210208175806.2091668-19-sashal@kernel.org>

From: Claus Stovgaard <claus.stovgaard at gmail.com>

[ Upstream commit c9e95c39280530200cdd0bbd2670e6334a81970b ]

Tested both with Corsairs firmware 11.3 and 13.0 for the Corsairs MP600
and both have the issue as reported by the kernel.

nvme nvme0: missing or invalid SUBNQN field.

Signed-off-by: Claus Stovgaard <claus.stovgaard at gmail.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/pci.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index a3486c1c27f0c..59c4e124f5482 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3247,6 +3247,8 @@ static const struct pci_device_id nvme_id_table[] = {
 	{ PCI_DEVICE(0x144d, 0xa822),   /* Samsung PM1725a */
 		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY |
 				NVME_QUIRK_IGNORE_DEV_SUBNQN, },
+	{ PCI_DEVICE(0x1987, 0x5016),	/* Phison E16 */
+		.driver_data = NVME_QUIRK_IGNORE_DEV_SUBNQN, },
 	{ PCI_DEVICE(0x1d1d, 0x1f1f),	/* LighNVM qemu device */
 		.driver_data = NVME_QUIRK_LIGHTNVM, },
 	{ PCI_DEVICE(0x1d1d, 0x2807),	/* CNEX WL */
-- 
2.27.0



From sashal at kernel.org  Mon Feb  8 12:58:48 2021
From: sashal at kernel.org (Sasha Levin)
Date: Mon,  8 Feb 2021 12:58:48 -0500
Subject: [PATCH AUTOSEL 5.4 09/19] nvme-pci: ignore the subsysem NQN on Phison
 E16
In-Reply-To: <20210208175858.2092008-1-sashal@kernel.org>
References: <20210208175858.2092008-1-sashal@kernel.org>
Message-ID: <20210208175858.2092008-9-sashal@kernel.org>

From: Claus Stovgaard <claus.stovgaard at gmail.com>

[ Upstream commit c9e95c39280530200cdd0bbd2670e6334a81970b ]

Tested both with Corsairs firmware 11.3 and 13.0 for the Corsairs MP600
and both have the issue as reported by the kernel.

nvme nvme0: missing or invalid SUBNQN field.

Signed-off-by: Claus Stovgaard <claus.stovgaard at gmail.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/pci.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index ef93bd3ed339c..511992b86399d 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3147,6 +3147,8 @@ static const struct pci_device_id nvme_id_table[] = {
 	{ PCI_DEVICE(0x144d, 0xa822),   /* Samsung PM1725a */
 		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY |
 				NVME_QUIRK_IGNORE_DEV_SUBNQN, },
+	{ PCI_DEVICE(0x1987, 0x5016),	/* Phison E16 */
+		.driver_data = NVME_QUIRK_IGNORE_DEV_SUBNQN, },
 	{ PCI_DEVICE(0x1d1d, 0x1f1f),	/* LighNVM qemu device */
 		.driver_data = NVME_QUIRK_LIGHTNVM, },
 	{ PCI_DEVICE(0x1d1d, 0x2807),	/* CNEX WL */
-- 
2.27.0



From sagi at grimberg.me  Mon Feb  8 13:16:45 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 10:16:45 -0800
Subject: Clarification regarding "nvme discover" and setting IOSQES/IOCQES
In-Reply-To: <SJ0PR19MB4544CE3903DEEA63D0C3929FF28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB4544CE3903DEEA63D0C3929FF28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <4608296a-6c45-a252-8b49-88cf25cd7365@grimberg.me>


> I'm using "nvme discover" to retrieve discovery log pages. And I'm using nvmet-tcp as the target.
> 
>  From what I understand, "nvme discover" is supposed to set up a connection to a "Discovery Controller (DC)". During the exchange between the Host and the Target, there is a time when the Host must set CC.EN=1 to "enable" the DC. I'm using Wireshark to analyse the packets sent by the Host to the DC. I can see that when it's time to set CC.EN=1, the Host also sets two additional parameters: IOCQES=4 and IOSQES=6.
> 
> In the NVME Base Specification, Figure 78 - Controller Configuration, these fields are defined as parameters to configure an "I/O Controller (IOC)" and not to configure a "Discovery Controller (DC)". In fact, it says that "If the controller does not support I/O queues, then this field shall be read-only with a value of 0h".
> 
> The way I understand it, DCs do not support I/O queues. So why set IOCQES and IOSQES to non-zero values?

Its probably an oversight, and the fact that the existing discovery
controllers simply allowed it.


From sagi at grimberg.me  Mon Feb  8 13:42:28 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 10:42:28 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
Message-ID: <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>


>> Hi Sagi
>>
>> On 2/8/21 5:46 PM, Sagi Grimberg wrote:
>>>
>>>> Hello
>>>>
>>>> We found this kernel NULL pointer issue with latest 
>>>> linux-block/for-next and it's 100% reproduced, let me know if you 
>>>> need more info/testing, thanks
>>>>
>>>> Kernel repo: 
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>>>
>>>> Reproducer: blktests nvme-tcp/012
>>>
>>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>>> but did not succeed. Given that you have it 100% reproducible,
>>> can you try to revert commit:
>>>
>>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>>
>>
>> Revert this commit fixed the issue and I've attached the config. :)
> 
> Good to know,
> 
> I see some differences that I should probably change to hit this:
> -- 
> @@ -254,14 +256,15 @@ CONFIG_PERF_EVENTS=y
>  ?# end of Kernel Performance Events And Counters
> 
>  ?CONFIG_VM_EVENT_COUNTERS=y
> +CONFIG_SLUB_DEBUG=y
>  ?# CONFIG_COMPAT_BRK is not set
> -CONFIG_SLAB=y
> -# CONFIG_SLUB is not set
> -# CONFIG_SLOB is not set
> -CONFIG_SLAB_MERGE_DEFAULT=y
> -# CONFIG_SLAB_FREELIST_RANDOM is not set
> +# CONFIG_SLAB is not set
> +CONFIG_SLUB=y
> +# CONFIG_SLAB_MERGE_DEFAULT is not set
> +CONFIG_SLAB_FREELIST_RANDOM=y
>  ?# CONFIG_SLAB_FREELIST_HARDENED is not set
> -# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
> +CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
> +CONFIG_SLUB_CPU_PARTIAL=y
>  ?CONFIG_SYSTEM_DATA_VERIFICATION=y
>  ?CONFIG_PROFILING=y
>  ?CONFIG_TRACEPOINTS=y
> @@ -299,7 +302,8 @@ CONFIG_HAVE_INTEL_TXT=y
>  ?CONFIG_X86_64_SMP=y
>  ?CONFIG_ARCH_SUPPORTS_UPROBES=y
>  ?CONFIG_FIX_EARLYCON_MEM=y
> -CONFIG_PGTABLE_LEVELS=4
> +CONFIG_DYNAMIC_PHYSICAL_MASK=y
> +CONFIG_PGTABLE_LEVELS=5
>  ?CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
> -- 
> 
> Probably CONFIG_SLUB and CONFIG_SLUB_DEBUG should be used.

Used your profile and this still does not happen :(


From sagi at grimberg.me  Mon Feb  8 13:59:43 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 10:59:43 -0800
Subject: Clarification regarding "nvme discover" and setting IOSQES/IOCQES
In-Reply-To: <SJ0PR19MB454412891E589E03EF8DB948F28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB4544CE3903DEEA63D0C3929FF28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <4608296a-6c45-a252-8b49-88cf25cd7365@grimberg.me>
 <SJ0PR19MB454412891E589E03EF8DB948F28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <c02ae0ac-ce0b-7081-9759-8c3751510ca7@grimberg.me>


> Hi Sagi,
> 
> Yes. The Discovery Controller currently allows it. The problem, however, 
> is that the DC seems to be expecting these values to be non-zero.
> 
> I tried setting IOCQES=0 and IOSQES=0, which the DC allows (i.e. Prop 
> Set return status=0). However, when I follow this by a "Property Get - 
> Controller Status", the Status now shows "Controller Fatal Status (CFS)" 
> (see Base specs - Figure 79).

Yes, the oversight extends to the target that expects it (shared code
with I/O controllers).

Does this fix your issue?
--
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 546a10407385..a7578c601727 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2443,6 +2443,11 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl)
  }
  EXPORT_SYMBOL_GPL(nvme_disable_ctrl);

+static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
+{
+       return ctrl->opts && ctrl->opts->discovery_nqn;
+}
+
  int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
  {
         unsigned dev_page_min;
@@ -2468,7 +2473,8 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
                 ctrl->ctrl_config = NVME_CC_CSS_NVM;
         ctrl->ctrl_config |= (NVME_CTRL_PAGE_SHIFT - 12) << 
NVME_CC_MPS_SHIFT;
         ctrl->ctrl_config |= NVME_CC_AMS_RR | NVME_CC_SHN_NONE;
-       ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
+       if (!nvme_discovery_ctrl(ctrl))
+               ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
         ctrl->ctrl_config |= NVME_CC_ENABLE;

         ret = ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config);
@@ -2888,11 +2894,6 @@ static const struct attribute_group 
*nvme_subsys_attrs_groups[] = {
         NULL,
  };

-static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
-{
-       return ctrl->opts && ctrl->opts->discovery_nqn;
-}
-
  static bool nvme_validate_cntlid(struct nvme_subsystem *subsys,
                 struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
  {
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 8ce4d59cc9e7..baa26a65d83d 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1107,9 +1107,14 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
  {
         lockdep_assert_held(&ctrl->lock);

-       if (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
-           nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES ||
-           nvmet_cc_mps(ctrl->cc) != 0 ||
+       if (!ctrl->subsys->type == NVME_NQN_DISC &&
+           (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
+            nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES)) {
+               ctrl->csts = NVME_CSTS_CFS;
+               return;
+       }
+
+       if (nvmet_cc_mps(ctrl->cc) != 0 ||
             nvmet_cc_ams(ctrl->cc) != 0 ||
             nvmet_cc_css(ctrl->cc) != 0) {
                 ctrl->csts = NVME_CSTS_CFS;
--


From kbusch at kernel.org  Mon Feb  8 14:04:58 2021
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 9 Feb 2021 04:04:58 +0900
Subject: Clarification regarding "nvme discover" and setting IOSQES/IOCQES
In-Reply-To: <c02ae0ac-ce0b-7081-9759-8c3751510ca7@grimberg.me>
References: <SJ0PR19MB4544CE3903DEEA63D0C3929FF28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <4608296a-6c45-a252-8b49-88cf25cd7365@grimberg.me>
 <SJ0PR19MB454412891E589E03EF8DB948F28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <c02ae0ac-ce0b-7081-9759-8c3751510ca7@grimberg.me>
Message-ID: <20210208190458.GB17042@redsun51.ssa.fujisawa.hgst.com>

On Mon, Feb 08, 2021 at 10:59:43AM -0800, Sagi Grimberg wrote:
> 
> > Hi Sagi,
> > 
> > Yes. The Discovery Controller currently allows it. The problem, however,
> > is that the DC seems to be expecting these values to be non-zero.
> > 
> > I tried setting IOCQES=0 and IOSQES=0, which the DC allows (i.e. Prop
> > Set return status=0). However, when I follow this by a "Property Get -
> > Controller Status", the Status now shows "Controller Fatal Status (CFS)"
> > (see Base specs - Figure 79).
> 
> Yes, the oversight extends to the target that expects it (shared code
> with I/O controllers).
> 
> Does this fix your issue?

Seems like the Identify Controller SQES and CQES fields ought to be
updated too, but the spec isn't clear on this. Feels like ECN
material...

> --
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 546a10407385..a7578c601727 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2443,6 +2443,11 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl)
>  }
>  EXPORT_SYMBOL_GPL(nvme_disable_ctrl);
> 
> +static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
> +{
> +       return ctrl->opts && ctrl->opts->discovery_nqn;
> +}
> +
>  int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
>  {
>         unsigned dev_page_min;
> @@ -2468,7 +2473,8 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
>                 ctrl->ctrl_config = NVME_CC_CSS_NVM;
>         ctrl->ctrl_config |= (NVME_CTRL_PAGE_SHIFT - 12) <<
> NVME_CC_MPS_SHIFT;
>         ctrl->ctrl_config |= NVME_CC_AMS_RR | NVME_CC_SHN_NONE;
> -       ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
> +       if (!nvme_discovery_ctrl(ctrl))
> +               ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
>         ctrl->ctrl_config |= NVME_CC_ENABLE;
> 
>         ret = ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config);
> @@ -2888,11 +2894,6 @@ static const struct attribute_group
> *nvme_subsys_attrs_groups[] = {
>         NULL,
>  };
> 
> -static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
> -{
> -       return ctrl->opts && ctrl->opts->discovery_nqn;
> -}
> -
>  static bool nvme_validate_cntlid(struct nvme_subsystem *subsys,
>                 struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
>  {
> diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
> index 8ce4d59cc9e7..baa26a65d83d 100644
> --- a/drivers/nvme/target/core.c
> +++ b/drivers/nvme/target/core.c
> @@ -1107,9 +1107,14 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
>  {
>         lockdep_assert_held(&ctrl->lock);
> 
> -       if (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
> -           nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES ||
> -           nvmet_cc_mps(ctrl->cc) != 0 ||
> +       if (!ctrl->subsys->type == NVME_NQN_DISC &&
> +           (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
> +            nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES)) {
> +               ctrl->csts = NVME_CSTS_CFS;
> +               return;
> +       }
> +
> +       if (nvmet_cc_mps(ctrl->cc) != 0 ||
>             nvmet_cc_ams(ctrl->cc) != 0 ||
>             nvmet_cc_css(ctrl->cc) != 0) {
>                 ctrl->csts = NVME_CSTS_CFS;
> --
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme


From sagi at grimberg.me  Mon Feb  8 18:01:28 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 15:01:28 -0800
Subject: Clarification regarding "nvme discover" and setting IOSQES/IOCQES
In-Reply-To: <20210208190458.GB17042@redsun51.ssa.fujisawa.hgst.com>
References: <SJ0PR19MB4544CE3903DEEA63D0C3929FF28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <4608296a-6c45-a252-8b49-88cf25cd7365@grimberg.me>
 <SJ0PR19MB454412891E589E03EF8DB948F28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <c02ae0ac-ce0b-7081-9759-8c3751510ca7@grimberg.me>
 <20210208190458.GB17042@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <cbf91356-e2b4-692b-38d8-22ec00359b8e@grimberg.me>


>>> Hi Sagi,
>>>
>>> Yes. The Discovery Controller currently allows it. The problem, however,
>>> is that the DC seems to be expecting these values to be non-zero.
>>>
>>> I tried setting IOCQES=0 and IOSQES=0, which the DC allows (i.e. Prop
>>> Set return status=0). However, when I follow this by a "Property Get -
>>> Controller Status", the Status now shows "Controller Fatal Status (CFS)"
>>> (see Base specs - Figure 79).
>>
>> Yes, the oversight extends to the target that expects it (shared code
>> with I/O controllers).
>>
>> Does this fix your issue?
> 
> Seems like the Identify Controller SQES and CQES fields ought to be
> updated too, but the spec isn't clear on this. Feels like ECN
> material...

Yes, we should start with this patch though right?


From james.smart at broadcom.com  Mon Feb  8 18:37:04 2021
From: james.smart at broadcom.com (James Smart)
Date: Mon, 8 Feb 2021 15:37:04 -0800
Subject: [PATCH] nvmet-fc: add a missing __rcu annotation to
 nvmet_fc_tgt_assoc.queues
In-Reply-To: <20210207162028.2959778-1-hch@lst.de>
References: <20210207162028.2959778-1-hch@lst.de>
Message-ID: <d120cf63-3f73-d64d-e338-36f75530c397@broadcom.com>



On 2/7/2021 8:20 AM, Christoph Hellwig wrote:
> Make sparse happy after the recent conversion to RCU lookups.
>
> Fixes: 4e2f02bf77da ("nvmet-fc: use RCU proctection for assoc_list")
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
>
Reviewed-by: James Smart <james.smart at broadcom.com>

Thanks

-- james



-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4163 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210208/4c427985/attachment.p7s>

From sagi at grimberg.me  Mon Feb  8 18:44:16 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 15:44:16 -0800
Subject: [PATCH] nvmet-fc: add a missing __rcu annotation to
 nvmet_fc_tgt_assoc.queues
In-Reply-To: <20210207162028.2959778-1-hch@lst.de>
References: <20210207162028.2959778-1-hch@lst.de>
Message-ID: <9c19e150-add6-a6c6-0b2d-4f0758d75826@grimberg.me>

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>


From Clay.Mayers at kioxia.com  Mon Feb  8 19:53:17 2021
From: Clay.Mayers at kioxia.com (Clay Mayers)
Date: Tue, 9 Feb 2021 00:53:17 +0000
Subject: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO
References: <20210105224939.1336-2-clay.mayers@kioxia.com>
 <20210125195844.1390581-1-clay.mayers@kioxia.com>
 <BYAPR04MB49650B6342C736B5268F833686BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b5c179e3eb904800931712d6b92cfbeb@kioxia.com>
 <BYAPR04MB49652EA8F4CF06D41C708E2386BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com>

> From: Clay Mayers
> Sent: Tuesday, January 26, 2021 1:14 PM
> To: linux-nvme at lists.infradead.org
> Cc: Keith Busch <kbusch at kernel.org>; Jens Axboe <axboe at fb.com>;
> 'Chaitanya Kulkarni' <Chaitanya.Kulkarni at wdc.com>;
> Christoph Hellwig <hch at lst.de>; Sagi Grimberg <sagi at grimberg.me>
> Subject: RE: [PATCH V2 0/2] nvme: Support for fused
> NVME_IOCTL_SUBMIT_IO
>
Is there any other feedback on V2?

My main concern I have about my implementation is how fused requests
are tunneled through the mq request layer.  The 1st request is marked as
started but it won't be in the device until the 2nd command is queued.  As
Keith pointed out, a device reset can split the two so care must be taken to
correctly handle this case.  Despite this, I thought this was a better approach
than modifying mq requests to be fused.  Especially given Christoph's 
concern of cost vs value.  This is the lightest touch I could come up with.

Further consideration of this patch may need a more compelling use case.
I've worked on a proprietary storage systems that relied on fused NVMeOF
support so it seems compelling to me.  There's a comment in target/core.c
that there is "no support for fused commands yet" implying it's been
considered.  Is pci only support for fused too soon or too little?  What would
make it more compelling?







From kbusch at kernel.org  Mon Feb  8 22:12:52 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 8 Feb 2021 20:12:52 -0700
Subject: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO
In-Reply-To: <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com>
References: <20210105224939.1336-2-clay.mayers@kioxia.com>
 <20210125195844.1390581-1-clay.mayers@kioxia.com>
 <BYAPR04MB49650B6342C736B5268F833686BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b5c179e3eb904800931712d6b92cfbeb@kioxia.com>
 <BYAPR04MB49652EA8F4CF06D41C708E2386BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com>
Message-ID: <20210209031252.GA97526@C02WT3WMHTD6>

On Tue, Feb 09, 2021 at 12:53:17AM +0000, Clay Mayers wrote:
> Is there any other feedback on V2?
> 
> My main concern I have about my implementation is how fused requests
> are tunneled through the mq request layer.  The 1st request is marked as
> started but it won't be in the device until the 2nd command is queued.  As
> Keith pointed out, a device reset can split the two so care must be taken to
> correctly handle this case.  Despite this, I thought this was a better approach
> than modifying mq requests to be fused.  Especially given Christoph's 
> concern of cost vs value.  This is the lightest touch I could come up with.
> 
> Further consideration of this patch may need a more compelling use case.
> I've worked on a proprietary storage systems that relied on fused NVMeOF
> support so it seems compelling to me.  There's a comment in target/core.c
> that there is "no support for fused commands yet" implying it's been
> considered.  Is pci only support for fused too soon or too little?  What would
> make it more compelling?

The complications it introduces to the IO path and error handling for an
archaic feature has me on the "Nak" side. NVMeOF was introduced well after the
spec define Reservations, and the kernel has supported that capability for many
years. I'm not aware of any other use case for fused commands, so it appears to
be dead weight in the spec.


From kbusch at kernel.org  Mon Feb  8 22:28:37 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 8 Feb 2021 20:28:37 -0700
Subject: Clarification regarding "nvme discover" and setting IOSQES/IOCQES
In-Reply-To: <cbf91356-e2b4-692b-38d8-22ec00359b8e@grimberg.me>
References: <SJ0PR19MB4544CE3903DEEA63D0C3929FF28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <4608296a-6c45-a252-8b49-88cf25cd7365@grimberg.me>
 <SJ0PR19MB454412891E589E03EF8DB948F28F9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <c02ae0ac-ce0b-7081-9759-8c3751510ca7@grimberg.me>
 <20210208190458.GB17042@redsun51.ssa.fujisawa.hgst.com>
 <cbf91356-e2b4-692b-38d8-22ec00359b8e@grimberg.me>
Message-ID: <20210209032837.GA99682@C02WT3WMHTD6>

On Mon, Feb 08, 2021 at 03:01:28PM -0800, Sagi Grimberg wrote:
> 
> > > > Hi Sagi,
> > > > 
> > > > Yes. The Discovery Controller currently allows it. The problem, however,
> > > > is that the DC seems to be expecting these values to be non-zero.
> > > > 
> > > > I tried setting IOCQES=0 and IOSQES=0, which the DC allows (i.e. Prop
> > > > Set return status=0). However, when I follow this by a "Property Get -
> > > > Controller Status", the Status now shows "Controller Fatal Status (CFS)"
> > > > (see Base specs - Figure 79).
> > > 
> > > Yes, the oversight extends to the target that expects it (shared code
> > > with I/O controllers).
> > > 
> > > Does this fix your issue?
> > 
> > Seems like the Identify Controller SQES and CQES fields ought to be
> > updated too, but the spec isn't clear on this. Feels like ECN
> > material...
> 
> Yes, we should start with this patch though right?

Yep, your patch looks good.

I was just thinking a generic implementation could align the CC settings with
the ID_CTRL values rather than checking for a discovery controller, but I
noticed the spec didn't provide us that opportunity. That was actually a
misguided thought anyway since we don't even have ID_CTRL data on the initial
CC setting! :)


From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:32 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:32 -0800
Subject: [PATCH V2 00/11] nvmet: fixes and some cleanups
Message-ID: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>

Hi,

This patch-series fixes behavior for id-ns that is against the spec and
does cleanup for nvmet_find_namespace() to get rid of the duplicate code
and remove the inconsistent error behavior of the host, Also this adds
an helper for unhandled commands to have uniform error message
reporting for the bdev, file and passthru backend. Patch 9/10 fix
compilation warnings. Last patch removes an unnecessary else at the tail
of the nvmet_parse_io_cmd().

This is based on nvme-5.12. All the blktests are seem to pass on this
series.                

Regarding the comment on V1 for bffcd507780e being wrong see [1], we can
continue the discussion.

-ck   

[1] http://lists.infradead.org/pipermail/linux-nvme/2021-February/022637.html

Changes from V1:-

1, In nvmet_execute_identify_ns() zeroout buffer for unallocated nsid.
2. Adjust the patch #2 to reflect above behavior.
3. Merge patch #3 and #4 from V1 to change the nvmet_find_namespace()
   prototype to only accept request.
4. Rename nvmet_find_namespace()->nvmet_req_find_namespace().
5. Remove the backened string print from nvmet_report_invalid_opcode() &
   replace pr_err() with pr_debug().
6. Add a new patch that creates a helper nvmet_req_subsys() to replace
   a chain of the structures all over code to get the subsys from
   nvmet_req.
7. Add a patch to remove the unnecessay else at the tail of the
   nvmet_parse_io_cmd().

Chaitanya Kulkarni (11):
  nvmet: zeroout id-ns buffer for invalid nsid
  nvmet: return uniform error for invalid ns
  nvmet: make nvmet_find_namespace() req based
  nvmet: remove extra variable in id-ns handler
  nvmet: add helper to report invalid opcode
  nvmet: use invalid cmd opcode helper
  nvmet: use invalid cmd opcode helper
  nvmet: use min of device_path and disk len
  nvme-loop: rename variable to get rid of the warn
  nvmet: add nvmet_req_subsys() helper
  nvmet: remove else at the end of the function

 drivers/nvme/target/admin-cmd.c   | 69 +++++++++++++++----------------
 drivers/nvme/target/core.c        | 28 +++++++------
 drivers/nvme/target/io-cmd-bdev.c |  5 +--
 drivers/nvme/target/io-cmd-file.c |  5 +--
 drivers/nvme/target/loop.c        |  2 +-
 drivers/nvme/target/nvmet.h       | 18 +++++++-
 drivers/nvme/target/passthru.c    |  6 +--
 drivers/nvme/target/trace.h       |  4 +-
 8 files changed, 73 insertions(+), 64 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:33 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:33 -0800
Subject: [PATCH V2 01/11] nvmet: zeroout id-ns buffer for invalid nsid
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-2-chaitanya.kulkarni@wdc.com>

According to spec :-
"5.15.2.7 Identify Namespace data structure for an Allocated Namespace
ID (CNS 11h) The Identify Namespace data structure (refer to Figure 245)
is returned to the host for the namespace specified in the Namespace
Identifier (NSID) field if it is an allocated NSID. If the specified
namespace is an unallocated NSID then the controller returns a zero
filled data structure."

Call nvmet_zero_sgl() when call to nvmet_find_namespace() fails and add
an explicit comment to specify the reason for zeroing the buffer which
is not common for the NVMe commands.

Fixes: bffcd507780e ("nvmet: set right status on error in id-ns handler")
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 613a4d8feac1..fa9a83541d51 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -482,10 +482,16 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 		goto out;
 	}
 
-	/* return an all zeroed buffer if we can't find an active namespace */
 	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
 	if (!req->ns) {
 		status = NVME_SC_INVALID_NS;
+		/*
+		 * According to spec : If the specified namespace is
+		 * an unallocated NSID then the controller returns a zero filled
+		 * data structure. Also don't override the error status as invalid
+		 * namespace takes priority over the failed zeroout buffer case.
+		 */
+		nvmet_zero_sgl(req, 0, sizeof(*id));
 		goto done;
 	}
 
@@ -539,10 +545,11 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 
 	if (req->ns->readonly)
 		id->nsattr |= (1 << 0);
-done:
+
 	if (!status)
 		status = nvmet_copy_to_sgl(req, 0, id, sizeof(*id));
 
+done:
 	kfree(id);
 out:
 	nvmet_req_complete(req, status);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:34 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:34 -0800
Subject: [PATCH V2 02/11] nvmet: return uniform error for invalid ns
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-3-chaitanya.kulkarni@wdc.com>

For nvmet_find_namespace() error case we have inconsistent error code
mapping in the function nvmet_get_smart_log_nsid() and
nvmet_set_feat_write_protect().

There is no point in retrying for the invalid namesapce from the host
side. Set the error code to the NVME_SC_INVALID_NS | NVME_SC_DNR which
matches what we have in nvmet_execute_identify_desclist().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index fa9a83541d51..d6a3ecf66262 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -82,7 +82,7 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req *req,
 		pr_err("Could not find namespace id : %d\n",
 				le32_to_cpu(req->cmd->get_log_page.nsid));
 		req->error_loc = offsetof(struct nvme_rw_command, nsid);
-		return NVME_SC_INVALID_NS;
+		return NVME_SC_INVALID_NS | NVME_SC_DNR;
 	}
 
 	/* we don't have the right data for file backed ns */
@@ -704,7 +704,7 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->rw.nsid);
 	if (unlikely(!req->ns)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return status;
+		return status = NVME_SC_INVALID_NS | NVME_SC_DNR;
 	}
 
 	mutex_lock(&subsys->lock);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:35 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:35 -0800
Subject: [PATCH V2 03/11] nvmet: make nvmet_find_namespace() req based
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-4-chaitanya.kulkarni@wdc.com>

The six callers of nvmet_find_namespace() duplicate the error log page
update and status setting code for each call on failure.

All callers are nvmet requests based functions, so we can pass req
to the nvmet_find_namesapce() & derive ctrl from req, that'll allow us
to update the error log page in nvmet_find_namespace(). Now that we
pass the request we can also get rid of the local variable in
nvmet_find_namespace() and use the req->ns and return the error code.

Replace the ctrl parameter with nvmet_req for nvmet_find_namespace(),
centralize the error log page update for non allocated namesapces, and
return uniform error for non-allocated namespace.

The nvmet_find_namespace() takes nsid parameter which is from NVMe
commands structures such as get_log_page, identify, rw and common. All
these commands have same offset for the nsid field.

Derive nsid from req->cmd->common.nsid) & remove the extra parameter
from the nvmet_find_namespace().

Lastly now we associate the ns to the req parameter that we pass to the
nvmet_find_namespace(), rename the nvmet_find_namespace() to
nvmet_req_find_ns() i.e. :-

Old:-
struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl,
				      __le32 nsid)
New :-
u16 nvmet_req_find_ns(struct nvmet_req *req);

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 47 +++++++++++++--------------------
 drivers/nvme/target/core.c      | 24 +++++++++--------
 drivers/nvme/target/nvmet.h     |  2 +-
 3 files changed, 33 insertions(+), 40 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index d6a3ecf66262..787e41d48a92 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -75,15 +75,11 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req *req,
 		struct nvme_smart_log *slog)
 {
 	u64 host_reads, host_writes, data_units_read, data_units_written;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl,
-				       req->cmd->get_log_page.nsid);
-	if (!req->ns) {
-		pr_err("Could not find namespace id : %d\n",
-				le32_to_cpu(req->cmd->get_log_page.nsid));
-		req->error_loc = offsetof(struct nvme_rw_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	status = nvmet_req_find_ns(req);
+	if (status)
+		return status;
 
 	/* we don't have the right data for file backed ns */
 	if (!req->ns->bdev)
@@ -468,7 +464,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 {
 	struct nvmet_ctrl *ctrl = req->sq->ctrl;
 	struct nvme_id_ns *id;
-	u16 status = 0;
+	u16 status;
 
 	if (le32_to_cpu(req->cmd->identify.nsid) == NVME_NSID_ALL) {
 		req->error_loc = offsetof(struct nvme_identify, nsid);
@@ -482,8 +478,8 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 		goto out;
 	}
 
-	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
-	if (!req->ns) {
+	status = nvmet_req_find_ns(req);
+	if (status) {
 		status = NVME_SC_INVALID_NS;
 		/*
 		 * According to spec : If the specified namespace is
@@ -611,15 +607,12 @@ static u16 nvmet_copy_ns_identifier(struct nvmet_req *req, u8 type, u8 len,
 
 static void nvmet_execute_identify_desclist(struct nvmet_req *req)
 {
-	u16 status = 0;
 	off_t off = 0;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->identify.nsid);
-	if (!req->ns) {
-		req->error_loc = offsetof(struct nvme_identify, nsid);
-		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
+	status = nvmet_req_find_ns(req);
+	if (status)
 		goto out;
-	}
 
 	if (memchr_inv(&req->ns->uuid, 0, sizeof(req->ns->uuid))) {
 		status = nvmet_copy_ns_identifier(req, NVME_NIDT_UUID,
@@ -699,13 +692,11 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 {
 	u32 write_protect = le32_to_cpu(req->cmd->common.cdw11);
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
-	u16 status = NVME_SC_FEATURE_NOT_CHANGEABLE;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->rw.nsid);
-	if (unlikely(!req->ns)) {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return status = NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	status = nvmet_req_find_ns(req);
+	if (status)
+		return status;
 
 	mutex_lock(&subsys->lock);
 	switch (write_protect) {
@@ -720,6 +711,7 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 		status = 0;
 		break;
 	default:
+		status = NVME_SC_FEATURE_NOT_CHANGEABLE;
 		break;
 	}
 
@@ -806,11 +798,10 @@ static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
 	u32 result;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->common.nsid);
-	if (!req->ns)  {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	result = nvmet_req_find_ns(req);
+	if (result)
+		return result;
+
 	mutex_lock(&subsys->lock);
 	if (req->ns->readonly == true)
 		result = NVME_NS_WRITE_PROTECT;
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 8ce4d59cc9e7..95b58d4b1af2 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -417,15 +417,18 @@ void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl)
 	cancel_delayed_work_sync(&ctrl->ka_work);
 }
 
-struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl, __le32 nsid)
+u16 nvmet_req_find_ns(struct nvmet_req *req)
 {
-	struct nvmet_ns *ns;
+	u32 nsid = le32_to_cpu(req->cmd->common.nsid);
 
-	ns = xa_load(&ctrl->subsys->namespaces, le32_to_cpu(nsid));
-	if (ns)
-		percpu_ref_get(&ns->ref);
+	req->ns = xa_load(&req->sq->ctrl->subsys->namespaces, nsid);
+	if (unlikely(!req->ns)) {
+		req->error_loc = offsetof(struct nvme_common_command, nsid);
+		return NVME_SC_INVALID_NS | NVME_SC_DNR;
+	}
 
-	return ns;
+	percpu_ref_get(&req->ns->ref);
+	return NVME_SC_SUCCESS;
 }
 
 static void nvmet_destroy_namespace(struct percpu_ref *ref)
@@ -862,11 +865,10 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 	if (nvmet_req_passthru_ctrl(req))
 		return nvmet_parse_passthru_io_cmd(req);
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, cmd->rw.nsid);
-	if (unlikely(!req->ns)) {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	ret = nvmet_req_find_ns(req);
+	if (unlikely(ret))
+		return ret;
+
 	ret = nvmet_check_ana_state(req->port, req->ns);
 	if (unlikely(ret)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 8776dd1a0490..954b3d8451f5 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -443,7 +443,7 @@ struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
 void nvmet_subsys_put(struct nvmet_subsys *subsys);
 void nvmet_subsys_del_ctrls(struct nvmet_subsys *subsys);
 
-struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl, __le32 nsid);
+u16 nvmet_req_find_ns(struct nvmet_req *req);
 void nvmet_put_namespace(struct nvmet_ns *ns);
 int nvmet_ns_enable(struct nvmet_ns *ns);
 void nvmet_ns_disable(struct nvmet_ns *ns);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:36 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:36 -0800
Subject: [PATCH V2 04/11] nvmet: remove extra variable in id-ns handler
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-5-chaitanya.kulkarni@wdc.com>

In nvmet_execute_identify_ns() local variable ctrl is accessed only in
one place, remove that and directly use it from nvmet_req->sq->ctrl.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 787e41d48a92..d551571b538c 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -462,7 +462,6 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 
 static void nvmet_execute_identify_ns(struct nvmet_req *req)
 {
-	struct nvmet_ctrl *ctrl = req->sq->ctrl;
 	struct nvme_id_ns *id;
 	u16 status;
 
@@ -529,7 +528,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 
 	id->lbaf[0].ds = req->ns->blksize_shift;
 
-	if (ctrl->pi_support && nvmet_ns_has_pi(req->ns)) {
+	if (req->sq->ctrl->pi_support && nvmet_ns_has_pi(req->ns)) {
 		id->dpc = NVME_NS_DPC_PI_FIRST | NVME_NS_DPC_PI_LAST |
 			  NVME_NS_DPC_PI_TYPE1 | NVME_NS_DPC_PI_TYPE2 |
 			  NVME_NS_DPC_PI_TYPE3;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:37 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:37 -0800
Subject: [PATCH V2 05/11] nvmet: add helper to report invalid opcode
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-6-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Add an helper and use it in block device backend to keep the code
and error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-bdev.c | 5 +----
 drivers/nvme/target/nvmet.h       | 9 +++++++++
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c
index 23095bdfce06..105ef2b125cf 100644
--- a/drivers/nvme/target/io-cmd-bdev.c
+++ b/drivers/nvme/target/io-cmd-bdev.c
@@ -449,9 +449,6 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req)
 		req->execute = nvmet_bdev_execute_write_zeroes;
 		return 0;
 	default:
-		pr_err("unhandled cmd %d on qid %d\n", cmd->common.opcode,
-		       req->sq->qid);
-		req->error_loc = offsetof(struct nvme_common_command, opcode);
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 954b3d8451f5..6b5f1b60cf50 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -613,4 +613,13 @@ static inline sector_t nvmet_lba_to_sect(struct nvmet_ns *ns, __le64 lba)
 	return le64_to_cpu(lba) << (ns->blksize_shift - SECTOR_SHIFT);
 }
 
+static inline u16 nvmet_report_invalid_opcode(struct nvmet_req *req)
+{
+	pr_debug("unhandled cmd %d on qid %d\n", req->cmd->common.opcode,
+		 req->sq->qid);
+
+	req->error_loc = offsetof(struct nvme_common_command, opcode);
+	return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+}
+
 #endif /* _NVMET_H */
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:38 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:38 -0800
Subject: [PATCH V2 06/11] nvmet: use invalid cmd opcode helper
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-7-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Use the previously introduced helper in file backend to reduce the
duplicate code and make the error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-file.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c
index 0abbefd9925e..715d4376c997 100644
--- a/drivers/nvme/target/io-cmd-file.c
+++ b/drivers/nvme/target/io-cmd-file.c
@@ -400,9 +400,6 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req)
 		req->execute = nvmet_file_execute_write_zeroes;
 		return 0;
 	default:
-		pr_err("unhandled cmd for file ns %d on qid %d\n",
-				cmd->common.opcode, req->sq->qid);
-		req->error_loc = offsetof(struct nvme_common_command, opcode);
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:39 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:39 -0800
Subject: [PATCH V2 07/11] nvmet: use invalid cmd opcode helper
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-8-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Use the previously introduced helper in the passthru backend to make the
error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index cbc88acdd233..3b22f4a868f4 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -494,7 +494,7 @@ u16 nvmet_parse_passthru_admin_cmd(struct nvmet_req *req)
 		return nvmet_setup_passthru_command(req);
 	default:
 		/* Reject commands not in the allowlist above */
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:40 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:40 -0800
Subject: [PATCH V2 08/11] nvmet: use min of device_path and disk len
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-9-chaitanya.kulkarni@wdc.com>

In function __assign_req_name() instead of using the DEVICE_NAME_LEN in
strncpy() use min of DISK_NAME_LEN and strlen(req->ns->device_path).

This is needed to turn off the following warnings:-

In file included from drivers/nvme/target/core.c:14:
In function ?__assign_req_name?,
    inlined from ?trace_event_raw_event_nvmet_req_init? at drivers/nvme/target/./trace.h:58:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?perf_trace_nvmet_req_complete? at drivers/nvme/target/./trace.h:100:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?perf_trace_nvmet_req_init? at drivers/nvme/target/./trace.h:58:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?trace_event_raw_event_nvmet_req_complete? at drivers/nvme/target/./trace.h:100:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/trace.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/trace.h b/drivers/nvme/target/trace.h
index c14e3249a14d..958dca7bcfc7 100644
--- a/drivers/nvme/target/trace.h
+++ b/drivers/nvme/target/trace.h
@@ -48,8 +48,10 @@ static inline struct nvmet_ctrl *nvmet_req_to_ctrl(struct nvmet_req *req)
 
 static inline void __assign_req_name(char *name, struct nvmet_req *req)
 {
+	size_t len = min_t(size_t, DISK_NAME_LEN, strlen(req->ns->device_path));
+
 	if (req->ns)
-		strncpy(name, req->ns->device_path, DISK_NAME_LEN);
+		strncpy(name, req->ns->device_path, len);
 	else
 		memset(name, 0, DISK_NAME_LEN);
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:42 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:42 -0800
Subject: [PATCH V2 10/11] nvmet: add nvmet_req_subsys() helper
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-11-chaitanya.kulkarni@wdc.com>

Just like what we have to get the passthru ctrl from the req, add an
helper to get the subsystem associated with the nvmet_req() instead
of open coding the chain of structures.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 10 +++++-----
 drivers/nvme/target/core.c      |  2 +-
 drivers/nvme/target/nvmet.h     |  7 ++++++-
 drivers/nvme/target/passthru.c  |  4 ++--
 4 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index d551571b538c..8f8382021c31 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -690,7 +690,7 @@ static u16 nvmet_write_protect_flush_sync(struct nvmet_req *req)
 static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 {
 	u32 write_protect = le32_to_cpu(req->cmd->common.cdw11);
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u16 status;
 
 	status = nvmet_req_find_ns(req);
@@ -750,7 +750,7 @@ u16 nvmet_set_feat_async_event(struct nvmet_req *req, u32 mask)
 
 void nvmet_execute_set_features(struct nvmet_req *req)
 {
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u32 cdw10 = le32_to_cpu(req->cmd->common.cdw10);
 	u32 cdw11 = le32_to_cpu(req->cmd->common.cdw11);
 	u16 status = 0;
@@ -794,7 +794,7 @@ void nvmet_execute_set_features(struct nvmet_req *req)
 
 static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 {
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u32 result;
 
 	result = nvmet_req_find_ns(req);
@@ -824,7 +824,7 @@ void nvmet_get_feat_async_event(struct nvmet_req *req)
 
 void nvmet_execute_get_features(struct nvmet_req *req)
 {
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u32 cdw10 = le32_to_cpu(req->cmd->common.cdw10);
 	u16 status = 0;
 
@@ -931,7 +931,7 @@ u16 nvmet_parse_admin_cmd(struct nvmet_req *req)
 
 	if (nvme_is_fabrics(cmd))
 		return nvmet_parse_fabrics_cmd(req);
-	if (req->sq->ctrl->subsys->type == NVME_NQN_DISC)
+	if (nvmet_req_subsys(req)->type == NVME_NQN_DISC)
 		return nvmet_parse_discovery_cmd(req);
 
 	ret = nvmet_check_ctrl_status(req, cmd);
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 95b58d4b1af2..32221f98f1cc 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -421,7 +421,7 @@ u16 nvmet_req_find_ns(struct nvmet_req *req)
 {
 	u32 nsid = le32_to_cpu(req->cmd->common.nsid);
 
-	req->ns = xa_load(&req->sq->ctrl->subsys->namespaces, nsid);
+	req->ns = xa_load(&nvmet_req_subsys(req)->namespaces, nsid);
 	if (unlikely(!req->ns)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
 		return NVME_SC_INVALID_NS | NVME_SC_DNR;
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 6b5f1b60cf50..50535563e8c5 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -551,6 +551,11 @@ static inline u32 nvmet_dsm_len(struct nvmet_req *req)
 		sizeof(struct nvme_dsm_range);
 }
 
+static inline struct nvmet_subsys *nvmet_req_subsys(struct nvmet_req *req)
+{
+	return req->sq->ctrl->subsys;
+}
+
 #ifdef CONFIG_NVME_TARGET_PASSTHRU
 void nvmet_passthru_subsys_free(struct nvmet_subsys *subsys);
 int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys);
@@ -585,7 +590,7 @@ static inline struct nvme_ctrl *nvmet_passthru_ctrl(struct nvmet_subsys *subsys)
 static inline struct nvme_ctrl *
 nvmet_req_passthru_ctrl(struct nvmet_req *req)
 {
-	return nvmet_passthru_ctrl(req->sq->ctrl->subsys);
+	return nvmet_passthru_ctrl(nvmet_req_subsys(req));
 }
 
 u16 errno_to_nvme_status(struct nvmet_req *req, int errno);
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 3b22f4a868f4..f50c7b2bf21c 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -239,9 +239,9 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		}
 
 		q = ns->queue;
-		timeout = req->sq->ctrl->subsys->io_timeout;
+		timeout = nvmet_req_subsys(req)->io_timeout;
 	} else {
-		timeout = req->sq->ctrl->subsys->admin_timeout;
+		timeout = nvmet_req_subsys(req)->admin_timeout;
 	}
 
 	rq = nvme_alloc_request(q, req->cmd, 0);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:41 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:41 -0800
Subject: [PATCH V2 09/11] nvme-loop: rename variable to get rid of the warn
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-10-chaitanya.kulkarni@wdc.com>

Rename the nvme_loop_init_request() numa_node parameter to
hw_queue_numa_node so that we can get rid of the following warning :-

drivers/nvme/target/loop.c: In function ?nvme_loop_init_request?:
drivers/nvme/target/loop.c:205:16: warning: declaration of ?numa_node? shadows a global declaration [-Wshadow]
   unsigned int numa_node)

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/loop.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..88d0656ad4a5 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -202,7 +202,7 @@ static int nvme_loop_init_iod(struct nvme_loop_ctrl *ctrl,
 
 static int nvme_loop_init_request(struct blk_mq_tag_set *set,
 		struct request *req, unsigned int hctx_idx,
-		unsigned int numa_node)
+		unsigned int hw_queue_numa_node)
 {
 	struct nvme_loop_ctrl *ctrl = set->driver_data;
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Mon Feb  8 23:08:43 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon,  8 Feb 2021 20:08:43 -0800
Subject: [PATCH V2 11/11] nvmet: remove else at the end of the function
In-Reply-To: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209040843.45572-12-chaitanya.kulkarni@wdc.com>

The function nvmet_parse_io_cmd() returns value from
nvmet_file_parse_io_cmd() or nvmet_bdev_parse_io_cmd() based on which
backend is set for the request. Remove the else and just return the
value from nvmet_bdev_parse_io_cmd().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 32221f98f1cc..1d5516272530 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -882,8 +882,8 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 
 	if (req->ns->file)
 		return nvmet_file_parse_io_cmd(req);
-	else
-		return nvmet_bdev_parse_io_cmd(req);
+
+	return nvmet_bdev_parse_io_cmd(req);
 }
 
 bool nvmet_req_init(struct nvmet_req *req, struct nvmet_cq *cq,
-- 
2.22.1



From ming.lei at redhat.com  Mon Feb  8 23:21:03 2021
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 9 Feb 2021 12:21:03 +0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
Message-ID: <20210209042103.GB63798@T590>

On Mon, Feb 08, 2021 at 10:42:28AM -0800, Sagi Grimberg wrote:
> 
> > > Hi Sagi
> > > 
> > > On 2/8/21 5:46 PM, Sagi Grimberg wrote:
> > > > 
> > > > > Hello
> > > > > 
> > > > > We found this kernel NULL pointer issue with latest
> > > > > linux-block/for-next and it's 100% reproduced, let me know
> > > > > if you need more info/testing, thanks
> > > > > 
> > > > > Kernel repo:
> > > > > https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
> > > > > Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
> > > > > 
> > > > > Reproducer: blktests nvme-tcp/012
> > > > 
> > > > Thanks for reporting Ming, I've tried to reproduce this on my VM
> > > > but did not succeed. Given that you have it 100% reproducible,
> > > > can you try to revert commit:
> > > > 
> > > > 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
> > > > 
> > > 
> > > Revert this commit fixed the issue and I've attached the config. :)
> > 
> > Good to know,
> > 
> > I see some differences that I should probably change to hit this:
> > -- 
> > @@ -254,14 +256,15 @@ CONFIG_PERF_EVENTS=y
> >  ?# end of Kernel Performance Events And Counters
> > 
> >  ?CONFIG_VM_EVENT_COUNTERS=y
> > +CONFIG_SLUB_DEBUG=y
> >  ?# CONFIG_COMPAT_BRK is not set
> > -CONFIG_SLAB=y
> > -# CONFIG_SLUB is not set
> > -# CONFIG_SLOB is not set
> > -CONFIG_SLAB_MERGE_DEFAULT=y
> > -# CONFIG_SLAB_FREELIST_RANDOM is not set
> > +# CONFIG_SLAB is not set
> > +CONFIG_SLUB=y
> > +# CONFIG_SLAB_MERGE_DEFAULT is not set
> > +CONFIG_SLAB_FREELIST_RANDOM=y
> >  ?# CONFIG_SLAB_FREELIST_HARDENED is not set
> > -# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
> > +CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
> > +CONFIG_SLUB_CPU_PARTIAL=y
> >  ?CONFIG_SYSTEM_DATA_VERIFICATION=y
> >  ?CONFIG_PROFILING=y
> >  ?CONFIG_TRACEPOINTS=y
> > @@ -299,7 +302,8 @@ CONFIG_HAVE_INTEL_TXT=y
> >  ?CONFIG_X86_64_SMP=y
> >  ?CONFIG_ARCH_SUPPORTS_UPROBES=y
> >  ?CONFIG_FIX_EARLYCON_MEM=y
> > -CONFIG_PGTABLE_LEVELS=4
> > +CONFIG_DYNAMIC_PHYSICAL_MASK=y
> > +CONFIG_PGTABLE_LEVELS=5
> >  ?CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
> > -- 
> > 
> > Probably CONFIG_SLUB and CONFIG_SLUB_DEBUG should be used.
> 
> Used your profile and this still does not happen :(

One obvious error is that nr_segments is computed wrong.

Yi, can you try the following patch?

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 881d28eb15e9..a393d99b74e1 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
 		offset = 0;
 	} else {
 		struct bio *bio = req->curr_bio;
+		struct bio_vec bv;
+		struct bvec_iter iter;
+
+		nsegs = 0;
+		bio_for_each_bvec(bv, bio, iter)
+			nsegs++;
 
 		vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
-		nsegs = bio_segments(bio);
 		size = bio->bi_iter.bi_size;
 		offset = bio->bi_iter.bi_bvec_done;
 	}

-- 
Ming



From rvs at apache.org  Mon Feb  8 23:24:40 2021
From: rvs at apache.org (Roman Shaposhnik)
Date: Mon, 8 Feb 2021 20:24:40 -0800
Subject: vhost/virtio fabric
Message-ID: <CA+ULb+v-2jigmQA=Z2WmWP0n3wCbEs_s94oCoNbjJMw14ATCjw@mail.gmail.com>

Hi!

I'm looking at using NVMe protocol in a KVM virtualization context
along the lines of:
    https://patchwork.kernel.org/project/qemu-devel/patch/1516003315-17878-2-git-send-email-changpeng.liu at intel.com/
but with an eye towards using NVMe-OF so that I can terminate
in the kernel target itself, as opposed to a userland process.

Effectively, I'm looking for vhost/virtio fabric similar to:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/vhost/scsi.c?h=v5.10.14

This search made me come across an old set of slides from
Christoph Hellwig that did, indeed, mention that at some point
vhost/virtio fabric was considered.

Please let me know if there's any work that has ever been done
in the direction (even if in a form of unmerged patches) or whether
if I need this functionality I'd have to implement it from scratch (probably
aping a great deal of drivers/vhost/scsi.c).

Thanks,
Roman.


From Chaitanya.Kulkarni at wdc.com  Mon Feb  8 23:41:21 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 9 Feb 2021 04:41:21 +0000
Subject: vhost/virtio fabric
References: <CA+ULb+v-2jigmQA=Z2WmWP0n3wCbEs_s94oCoNbjJMw14ATCjw@mail.gmail.com>
Message-ID: <BYAPR04MB4965EC7FB74B38612065CDF5868E9@BYAPR04MB4965.namprd04.prod.outlook.com>

Roman,

On 2/8/21 20:29, Roman Shaposhnik wrote:
> This search made me come across an old set of slides from
> Christoph Hellwig that did, indeed, mention that at some point
> vhost/virtio fabric was considered.
>
> Please let me know if there's any work that has ever been done
> in the direction (even if in a form of unmerged patches) or whether
> if I need this functionality I'd have to implement it from scratch (probably
> aping a great deal of drivers/vhost/scsi.c).

At some point of time I've looked into this because that the next logical
step in the ecosystem development, but I'm not sure if this will be
acceptable
or not or do we need to have change the spec. If there is a strong use
case I'd
like to definitely work on or contribute towards the development of this
feature.

I'm interested in knowing what everyone think about this one ...


From gtiwari at redhat.com  Mon Feb  8 23:59:02 2021
From: gtiwari at redhat.com (Gopal Tiwari)
Date: Tue,  9 Feb 2021 10:29:02 +0530
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes
 command
Message-ID: <20210209045902.72345-1-gtiwari@redhat.com>

For SK Hynix PE8000 WriteZerocommands exceed typical I/O MDTS
(Maximum Data Transfer Size) supported by the hardware. Which
intern fail and generate following erros :

nvme nvme0: I/O 0 QID 96 timeout, aborting
nvme nvme0: Abort status: 0x371
nvme nvme0: Shutdown timeout set to 15 seconds
nvme nvme0: 128/0/0 default/read/poll queues
nvme nvme0: I/O 0 QID 96 timeout, disable controller
nvme nvme0: I/O 0 QID 0 timeout, disable controller
nvme nvme0: Device shutdown incomplete; abort shutdown
nvme nvme0: failed to mark controller live state
nvme nvme0: Removing after probe failure status: -19

Firmware unable to complete any outstanding commands
due to lack of resources, therefore placing NVMe drive
in a condition that is unable to further process WriteZerocommands.

fixing by disabling Write zeroes.

Signed-off-by: Gopal Tiwari <gtiwari at redhat.com>
---
 drivers/nvme/host/pci.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 6bad4d4dcdf0..aca67728700e 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3257,6 +3257,8 @@ static const struct pci_device_id nvme_id_table[] = {
 				NVME_QUIRK_IGNORE_DEV_SUBNQN, },
 	{ PCI_DEVICE(0x1c5c, 0x1504),   /* SK Hynix PC400 */
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+	{ PCI_DEVICE(0x1c5c, 0x2839),  /* SK Hynix PE8000 U.3 NVMe storage */
+		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(0x15b7, 0x2001),   /*  Sandisk Skyhawk */
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(0x1d97, 0x2263),   /* SPCC */
-- 
2.26.2



From Chaitanya.Kulkarni at wdc.com  Tue Feb  9 00:38:03 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 9 Feb 2021 05:38:03 +0000
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes
 command
References: <20210209045902.72345-1-gtiwari@redhat.com>
Message-ID: <BYAPR04MB4965000F7E15116893D6D1BD868E9@BYAPR04MB4965.namprd04.prod.outlook.com>

Gopal,

On 2/8/21 21:06, Gopal Tiwari wrote:
> For SK Hynix PE8000 WriteZerocommands exceed typical I/O MDTS
> (Maximum Data Transfer Size) supported by the hardware. Which
> intern fail and generate following erros :
>
> nvme nvme0: I/O 0 QID 96 timeout, aborting
> nvme nvme0: Abort status: 0x371
> nvme nvme0: Shutdown timeout set to 15 seconds
> nvme nvme0: 128/0/0 default/read/poll queues
> nvme nvme0: I/O 0 QID 96 timeout, disable controller
> nvme nvme0: I/O 0 QID 0 timeout, disable controller
> nvme nvme0: Device shutdown incomplete; abort shutdown
> nvme nvme0: failed to mark controller live state
> nvme nvme0: Removing after probe failure status: -19
>
> Firmware unable to complete any outstanding commands
> due to lack of resources, therefore placing NVMe drive
> in a condition that is unable to further process WriteZerocommands.
>
> fixing by disabling Write zeroes.

Please consider following commit log, can be used at the time applying the
patch:-

For SK Hynix PE8000 NVMe Write Zeroes Command exceed typical I/O MDTS
(Maximum Data Transfer Size) supported by the hardware. That intern
fail and generate following errors :

nvme nvme0: I/O 0 QID 96 timeout, aborting
nvme nvme0: Abort status: 0x371
nvme nvme0: Shutdown timeout set to 15 seconds
nvme nvme0: 128/0/0 default/read/poll queues
nvme nvme0: I/O 0 QID 96 timeout, disable controller
nvme nvme0: I/O 0 QID 0 timeout, disable controller
nvme nvme0: Device shutdown incomplete; abort shutdown
nvme nvme0: failed to mark controller live state
nvme nvme0: Removing after probe failure status: -19

Firmware is unable to complete any outstanding commands due to lack of
resources, therefore placing NVMe drive in a condition that is unable to
further process write-zeores
command.                                                                 

Add NVME_QUIRK_DISABLE_WRITE_ZEROES quirk for SK Hynix PE8000 that fixes
the problem.


With the modified commit log, looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>



From gtiwari at redhat.com  Tue Feb  9 00:59:12 2021
From: gtiwari at redhat.com (Gopal Tiwari)
Date: Tue, 9 Feb 2021 00:59:12 -0500 (EST)
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write
 Zeroes command
In-Reply-To: <BYAPR04MB4965000F7E15116893D6D1BD868E9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <BYAPR04MB4965000F7E15116893D6D1BD868E9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <1111178699.30528215.1612850352735.JavaMail.zimbra@redhat.com>

Thanks Chaitanya, Sure. 

Regards,
Gopal 


----- Original Message -----
From: "Chaitanya Kulkarni" <Chaitanya.Kulkarni at wdc.com>
To: "Gopal Tiwari" <gtiwari at redhat.com>, hch at lst.de, kbusch at kernel.org, linux-nvme at lists.infradead.org
Sent: Tuesday, February 9, 2021 11:08:03 AM
Subject: Re: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes command

Gopal,

On 2/8/21 21:06, Gopal Tiwari wrote:
> For SK Hynix PE8000 WriteZerocommands exceed typical I/O MDTS
> (Maximum Data Transfer Size) supported by the hardware. Which
> intern fail and generate following erros :
>
> nvme nvme0: I/O 0 QID 96 timeout, aborting
> nvme nvme0: Abort status: 0x371
> nvme nvme0: Shutdown timeout set to 15 seconds
> nvme nvme0: 128/0/0 default/read/poll queues
> nvme nvme0: I/O 0 QID 96 timeout, disable controller
> nvme nvme0: I/O 0 QID 0 timeout, disable controller
> nvme nvme0: Device shutdown incomplete; abort shutdown
> nvme nvme0: failed to mark controller live state
> nvme nvme0: Removing after probe failure status: -19
>
> Firmware unable to complete any outstanding commands
> due to lack of resources, therefore placing NVMe drive
> in a condition that is unable to further process WriteZerocommands.
>
> fixing by disabling Write zeroes.

Please consider following commit log, can be used at the time applying the
patch:-

For SK Hynix PE8000 NVMe Write Zeroes Command exceed typical I/O MDTS
(Maximum Data Transfer Size) supported by the hardware. That intern
fail and generate following errors :

nvme nvme0: I/O 0 QID 96 timeout, aborting
nvme nvme0: Abort status: 0x371
nvme nvme0: Shutdown timeout set to 15 seconds
nvme nvme0: 128/0/0 default/read/poll queues
nvme nvme0: I/O 0 QID 96 timeout, disable controller
nvme nvme0: I/O 0 QID 0 timeout, disable controller
nvme nvme0: Device shutdown incomplete; abort shutdown
nvme nvme0: failed to mark controller live state
nvme nvme0: Removing after probe failure status: -19

Firmware is unable to complete any outstanding commands due to lack of
resources, therefore placing NVMe drive in a condition that is unable to
further process write-zeores
command.                                                                 

Add NVME_QUIRK_DISABLE_WRITE_ZEROES quirk for SK Hynix PE8000 that fixes
the problem.


With the modified commit log, looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme



From sagi at grimberg.me  Tue Feb  9 02:21:53 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 8 Feb 2021 23:21:53 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <20210209042103.GB63798@T590>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
 <20210209042103.GB63798@T590>
Message-ID: <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>



On 2/8/21 8:21 PM, Ming Lei wrote:
> On Mon, Feb 08, 2021 at 10:42:28AM -0800, Sagi Grimberg wrote:
>>
>>>> Hi Sagi
>>>>
>>>> On 2/8/21 5:46 PM, Sagi Grimberg wrote:
>>>>>
>>>>>> Hello
>>>>>>
>>>>>> We found this kernel NULL pointer issue with latest
>>>>>> linux-block/for-next and it's 100% reproduced, let me know
>>>>>> if you need more info/testing, thanks
>>>>>>
>>>>>> Kernel repo:
>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>>>>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>>>>>
>>>>>> Reproducer: blktests nvme-tcp/012
>>>>>
>>>>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>>>>> but did not succeed. Given that you have it 100% reproducible,
>>>>> can you try to revert commit:
>>>>>
>>>>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>>>>
>>>>
>>>> Revert this commit fixed the issue and I've attached the config. :)
>>>
>>> Good to know,
>>>
>>> I see some differences that I should probably change to hit this:
>>> -- 
>>> @@ -254,14 +256,15 @@ CONFIG_PERF_EVENTS=y
>>>   ?# end of Kernel Performance Events And Counters
>>>
>>>   ?CONFIG_VM_EVENT_COUNTERS=y
>>> +CONFIG_SLUB_DEBUG=y
>>>   ?# CONFIG_COMPAT_BRK is not set
>>> -CONFIG_SLAB=y
>>> -# CONFIG_SLUB is not set
>>> -# CONFIG_SLOB is not set
>>> -CONFIG_SLAB_MERGE_DEFAULT=y
>>> -# CONFIG_SLAB_FREELIST_RANDOM is not set
>>> +# CONFIG_SLAB is not set
>>> +CONFIG_SLUB=y
>>> +# CONFIG_SLAB_MERGE_DEFAULT is not set
>>> +CONFIG_SLAB_FREELIST_RANDOM=y
>>>   ?# CONFIG_SLAB_FREELIST_HARDENED is not set
>>> -# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
>>> +CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
>>> +CONFIG_SLUB_CPU_PARTIAL=y
>>>   ?CONFIG_SYSTEM_DATA_VERIFICATION=y
>>>   ?CONFIG_PROFILING=y
>>>   ?CONFIG_TRACEPOINTS=y
>>> @@ -299,7 +302,8 @@ CONFIG_HAVE_INTEL_TXT=y
>>>   ?CONFIG_X86_64_SMP=y
>>>   ?CONFIG_ARCH_SUPPORTS_UPROBES=y
>>>   ?CONFIG_FIX_EARLYCON_MEM=y
>>> -CONFIG_PGTABLE_LEVELS=4
>>> +CONFIG_DYNAMIC_PHYSICAL_MASK=y
>>> +CONFIG_PGTABLE_LEVELS=5
>>>   ?CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
>>> -- 
>>>
>>> Probably CONFIG_SLUB and CONFIG_SLUB_DEBUG should be used.
>>
>> Used your profile and this still does not happen :(
> 
> One obvious error is that nr_segments is computed wrong.
> 
> Yi, can you try the following patch?
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 881d28eb15e9..a393d99b74e1 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>   		offset = 0;
>   	} else {
>   		struct bio *bio = req->curr_bio;
> +		struct bio_vec bv;
> +		struct bvec_iter iter;
> +
> +		nsegs = 0;
> +		bio_for_each_bvec(bv, bio, iter)
> +			nsegs++;
>   
>   		vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
> -		nsegs = bio_segments(bio);

This was exactly the patch that caused the issue.


From ming.lei at redhat.com  Tue Feb  9 02:50:01 2021
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 9 Feb 2021 15:50:01 +0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
 <20210209042103.GB63798@T590>
 <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>
Message-ID: <20210209075001.GA94287@T590>

On Mon, Feb 08, 2021 at 11:21:53PM -0800, Sagi Grimberg wrote:
> 
> 
> On 2/8/21 8:21 PM, Ming Lei wrote:
> > On Mon, Feb 08, 2021 at 10:42:28AM -0800, Sagi Grimberg wrote:
> > > 
> > > > > Hi Sagi
> > > > > 
> > > > > On 2/8/21 5:46 PM, Sagi Grimberg wrote:
> > > > > > 
> > > > > > > Hello
> > > > > > > 
> > > > > > > We found this kernel NULL pointer issue with latest
> > > > > > > linux-block/for-next and it's 100% reproduced, let me know
> > > > > > > if you need more info/testing, thanks
> > > > > > > 
> > > > > > > Kernel repo:
> > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
> > > > > > > Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
> > > > > > > 
> > > > > > > Reproducer: blktests nvme-tcp/012
> > > > > > 
> > > > > > Thanks for reporting Ming, I've tried to reproduce this on my VM
> > > > > > but did not succeed. Given that you have it 100% reproducible,
> > > > > > can you try to revert commit:
> > > > > > 
> > > > > > 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
> > > > > > 
> > > > > 
> > > > > Revert this commit fixed the issue and I've attached the config. :)
> > > > 
> > > > Good to know,
> > > > 
> > > > I see some differences that I should probably change to hit this:
> > > > -- 
> > > > @@ -254,14 +256,15 @@ CONFIG_PERF_EVENTS=y
> > > >   ?# end of Kernel Performance Events And Counters
> > > > 
> > > >   ?CONFIG_VM_EVENT_COUNTERS=y
> > > > +CONFIG_SLUB_DEBUG=y
> > > >   ?# CONFIG_COMPAT_BRK is not set
> > > > -CONFIG_SLAB=y
> > > > -# CONFIG_SLUB is not set
> > > > -# CONFIG_SLOB is not set
> > > > -CONFIG_SLAB_MERGE_DEFAULT=y
> > > > -# CONFIG_SLAB_FREELIST_RANDOM is not set
> > > > +# CONFIG_SLAB is not set
> > > > +CONFIG_SLUB=y
> > > > +# CONFIG_SLAB_MERGE_DEFAULT is not set
> > > > +CONFIG_SLAB_FREELIST_RANDOM=y
> > > >   ?# CONFIG_SLAB_FREELIST_HARDENED is not set
> > > > -# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
> > > > +CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
> > > > +CONFIG_SLUB_CPU_PARTIAL=y
> > > >   ?CONFIG_SYSTEM_DATA_VERIFICATION=y
> > > >   ?CONFIG_PROFILING=y
> > > >   ?CONFIG_TRACEPOINTS=y
> > > > @@ -299,7 +302,8 @@ CONFIG_HAVE_INTEL_TXT=y
> > > >   ?CONFIG_X86_64_SMP=y
> > > >   ?CONFIG_ARCH_SUPPORTS_UPROBES=y
> > > >   ?CONFIG_FIX_EARLYCON_MEM=y
> > > > -CONFIG_PGTABLE_LEVELS=4
> > > > +CONFIG_DYNAMIC_PHYSICAL_MASK=y
> > > > +CONFIG_PGTABLE_LEVELS=5
> > > >   ?CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
> > > > -- 
> > > > 
> > > > Probably CONFIG_SLUB and CONFIG_SLUB_DEBUG should be used.
> > > 
> > > Used your profile and this still does not happen :(
> > 
> > One obvious error is that nr_segments is computed wrong.
> > 
> > Yi, can you try the following patch?
> > 
> > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > index 881d28eb15e9..a393d99b74e1 100644
> > --- a/drivers/nvme/host/tcp.c
> > +++ b/drivers/nvme/host/tcp.c
> > @@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
> >   		offset = 0;
> >   	} else {
> >   		struct bio *bio = req->curr_bio;
> > +		struct bio_vec bv;
> > +		struct bvec_iter iter;
> > +
> > +		nsegs = 0;
> > +		bio_for_each_bvec(bv, bio, iter)
> > +			nsegs++;
> >   		vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
> > -		nsegs = bio_segments(bio);
> 
> This was exactly the patch that caused the issue.

What was the issue you are talking about? Any link or commit hash?

nvme-tcp builds iov_iter(BVEC) from __bvec_iter_bvec(), the segment
number has to be the actual bvec number. But bio_segment() just returns
number of the single-page segment, which is wrong for iov_iter.

Please see the same usage in lo_rw_aio().

-- 
Ming



From hch at lst.de  Tue Feb  9 02:54:34 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2021 08:54:34 +0100
Subject: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO
In-Reply-To: <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com>
References: <20210105224939.1336-2-clay.mayers@kioxia.com>
 <20210125195844.1390581-1-clay.mayers@kioxia.com>
 <BYAPR04MB49650B6342C736B5268F833686BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b5c179e3eb904800931712d6b92cfbeb@kioxia.com>
 <BYAPR04MB49652EA8F4CF06D41C708E2386BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com>
Message-ID: <20210209075434.GA31373@lst.de>

On Tue, Feb 09, 2021 at 12:53:17AM +0000, Clay Mayers wrote:
> > From: Clay Mayers
> > Sent: Tuesday, January 26, 2021 1:14 PM
> > To: linux-nvme at lists.infradead.org
> > Cc: Keith Busch <kbusch at kernel.org>; Jens Axboe <axboe at fb.com>;
> > 'Chaitanya Kulkarni' <Chaitanya.Kulkarni at wdc.com>;
> > Christoph Hellwig <hch at lst.de>; Sagi Grimberg <sagi at grimberg.me>
> > Subject: RE: [PATCH V2 0/2] nvme: Support for fused
> > NVME_IOCTL_SUBMIT_IO
> >
> Is there any other feedback on V2?

My feedback is that we should not add code to the kernel driver and
our fast path for fused command unless we have a mainstream in-kernel
users, that is not at all for the foreseeable future.


From hch at lst.de  Tue Feb  9 02:57:55 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2021 08:57:55 +0100
Subject: [PATCH V2 01/11] nvmet: zeroout id-ns buffer for invalid nsid
In-Reply-To: <20210209040843.45572-2-chaitanya.kulkarni@wdc.com>
References: <20210209040843.45572-1-chaitanya.kulkarni@wdc.com>
 <20210209040843.45572-2-chaitanya.kulkarni@wdc.com>
Message-ID: <20210209075755.GA31428@lst.de>

NAK.  Please don't add dead code for a slow path, and please fix the
actual bug in that we need to return 0 here and not an error.


From Chaitanya.Kulkarni at wdc.com  Tue Feb  9 03:34:58 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 9 Feb 2021 08:34:58 +0000
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <20210209075001.GA94287@T590>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
 <20210209042103.GB63798@T590>
 <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>,
 <20210209075001.GA94287@T590>
Message-ID: <65F086AF-DDD7-46B7-9134-3E6B577E75A6@wdc.com>


> On Feb 8, 2021, at 11:50 PM, Ming Lei <ming.lei at redhat.com> wrote:
> 
> ?On Mon, Feb 08, 2021 at 11:21:53PM -0800, Sagi Grimberg wrote:
>> 
>> 
>>> On 2/8/21 8:21 PM, Ming Lei wrote:
>>> On Mon, Feb 08, 2021 at 10:42:28AM -0800, Sagi Grimberg wrote:
>>>> 
>>>>>> Hi Sagi
>>>>>> 
>>>>>> On 2/8/21 5:46 PM, Sagi Grimberg wrote:
>>>>>>> 
>>>>>>>> Hello
>>>>>>>> 
>>>>>>>> We found this kernel NULL pointer issue with latest
>>>>>>>> linux-block/for-next and it's 100% reproduced, let me know
>>>>>>>> if you need more info/testing, thanks
>>>>>>>> 
>>>>>>>> Kernel repo:
>>>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>>>>>>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>>>>>>> 
>>>>>>>> Reproducer: blktests nvme-tcp/012
>>>>>>> 
>>>>>>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>>>>>>> but did not succeed. Given that you have it 100% reproducible,
>>>>>>> can you try to revert commit:
>>>>>>> 
>>>>>>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>>>>>> 
>>>>>> 
>>>>>> Revert this commit fixed the issue and I've attached the config. :)
>>>>> 
>>>>> Good to know,
>>>>> 
>>>>> I see some differences that I should probably change to hit this:
>>>>> -- 
>>>>> @@ -254,14 +256,15 @@ CONFIG_PERF_EVENTS=y
>>>>>   # end of Kernel Performance Events And Counters
>>>>> 
>>>>>   CONFIG_VM_EVENT_COUNTERS=y
>>>>> +CONFIG_SLUB_DEBUG=y
>>>>>   # CONFIG_COMPAT_BRK is not set
>>>>> -CONFIG_SLAB=y
>>>>> -# CONFIG_SLUB is not set
>>>>> -# CONFIG_SLOB is not set
>>>>> -CONFIG_SLAB_MERGE_DEFAULT=y
>>>>> -# CONFIG_SLAB_FREELIST_RANDOM is not set
>>>>> +# CONFIG_SLAB is not set
>>>>> +CONFIG_SLUB=y
>>>>> +# CONFIG_SLAB_MERGE_DEFAULT is not set
>>>>> +CONFIG_SLAB_FREELIST_RANDOM=y
>>>>>   # CONFIG_SLAB_FREELIST_HARDENED is not set
>>>>> -# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
>>>>> +CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
>>>>> +CONFIG_SLUB_CPU_PARTIAL=y
>>>>>   CONFIG_SYSTEM_DATA_VERIFICATION=y
>>>>>   CONFIG_PROFILING=y
>>>>>   CONFIG_TRACEPOINTS=y
>>>>> @@ -299,7 +302,8 @@ CONFIG_HAVE_INTEL_TXT=y
>>>>>   CONFIG_X86_64_SMP=y
>>>>>   CONFIG_ARCH_SUPPORTS_UPROBES=y
>>>>>   CONFIG_FIX_EARLYCON_MEM=y
>>>>> -CONFIG_PGTABLE_LEVELS=4
>>>>> +CONFIG_DYNAMIC_PHYSICAL_MASK=y
>>>>> +CONFIG_PGTABLE_LEVELS=5
>>>>>   CONFIG_CC_HAS_SANE_STACKPROTECTOR=y
>>>>> -- 
>>>>> 
>>>>> Probably CONFIG_SLUB and CONFIG_SLUB_DEBUG should be used.
>>>> 
>>>> Used your profile and this still does not happen :(
>>> 
>>> One obvious error is that nr_segments is computed wrong.
>>> 
>>> Yi, can you try the following patch?
>>> 
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index 881d28eb15e9..a393d99b74e1 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>>>          offset = 0;
>>>      } else {
>>>          struct bio *bio = req->curr_bio;
>>> +        struct bio_vec bv;
>>> +        struct bvec_iter iter;
>>> +
>>> +        nsegs = 0;
>>> +        bio_for_each_bvec(bv, bio, iter)
>>> +            nsegs++;
>>>          vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
>>> -        nsegs = bio_segments(bio);
>> 
>> This was exactly the patch that caused the issue.
> 
> What was the issue you are talking about? Any link or commit hash?
> 
> nvme-tcp builds iov_iter(BVEC) from __bvec_iter_bvec(), the segment
> number has to be the actual bvec number. But bio_segment() just returns
> number of the single-page segment, which is wrong for iov_iter.
> 
> Please see the same usage in lo_rw_aio().
> 
That what I have suggested but I've also suggested the memory allocation part which Sagi explained why it is better to avoid. 

In my opinion we should at least try bvec calculation in lo_aio_rw() and see the problem can be fixed or not, unless reverting the commit it right approach for some reason. 
> -- 
> Ming
> 

From hch at lst.de  Tue Feb  9 03:41:56 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2021 09:41:56 +0100
Subject: preserve DMA offsets when using swiotlb v2
In-Reply-To: <20210207160327.2955490-1-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
Message-ID: <20210209084156.GA32320@lst.de>

Sorry for being a little pushy, any chance we could get this reviewed
in time for the 5.12 merge window?

On Sun, Feb 07, 2021 at 05:03:18PM +0100, Christoph Hellwig wrote:
> Hi all,
> 
> this series make NVMe happy when running with swiotlb.  This caters
> towards to completely broken NVMe controllers that ignore the
> specification (hello to the biggest cloud provider on the planet!),
> to crappy SOC that have addressing limitations, or "secure"
> virtualization that force bounce buffering to enhance the user
> experience.  Or in other words, no one sane should hit it, but
> people do.
> 
> It is basically a respin of the
> 
>     "SWIOTLB: Preserve swiotlb map offset when needed."
> 
> series from Jianxiong Gao.  It complete rewrites the swiotlb part so that
> the offset really is preserved and not just the offset into the swiotlb
> slot, and to do so it grew half a dozen patches to refactor the swiotlb
> so that a mere mortal like me could actually understand it.
> 
> Changes since v2:
>  - cleanup nr_slots a little more
>  - fix a check in get_max_slots
>  - add a IO_TLB_SIZE define
>  - cleanup find_slots a little more
>  - fix the nvme patch to actually compile
> _______________________________________________
> iommu mailing list
> iommu at lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu
---end quoted text---


From gregkh at linuxfoundation.org  Tue Feb  9 03:46:59 2021
From: gregkh at linuxfoundation.org (Greg KH)
Date: Tue, 9 Feb 2021 09:46:59 +0100
Subject: preserve DMA offsets when using swiotlb v2
In-Reply-To: <20210209084156.GA32320@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
 <20210209084156.GA32320@lst.de>
Message-ID: <YCJMAxPi1HlVedfL@kroah.com>

On Tue, Feb 09, 2021 at 09:41:56AM +0100, Christoph Hellwig wrote:
> Sorry for being a little pushy, any chance we could get this reviewed
> in time for the 5.12 merge window?

No objection from me, my ack is already on the patch that you need it
for :)

thanks,

greg k-h


From sagi at grimberg.me  Tue Feb  9 05:07:15 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 9 Feb 2021 02:07:15 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <20210209075001.GA94287@T590>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
 <20210209042103.GB63798@T590>
 <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>
 <20210209075001.GA94287@T590>
Message-ID: <d5e33680-5196-2873-332f-19191c60fd3b@grimberg.me>


>>>
>>> One obvious error is that nr_segments is computed wrong.
>>>
>>> Yi, can you try the following patch?
>>>
>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>> index 881d28eb15e9..a393d99b74e1 100644
>>> --- a/drivers/nvme/host/tcp.c
>>> +++ b/drivers/nvme/host/tcp.c
>>> @@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>>>    		offset = 0;
>>>    	} else {
>>>    		struct bio *bio = req->curr_bio;
>>> +		struct bio_vec bv;
>>> +		struct bvec_iter iter;
>>> +
>>> +		nsegs = 0;
>>> +		bio_for_each_bvec(bv, bio, iter)
>>> +			nsegs++;
>>>    		vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
>>> -		nsegs = bio_segments(bio);
>>
>> This was exactly the patch that caused the issue.
> 
> What was the issue you are talking about? Any link or commit hash?

The commit that caused the crash is:
0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter

> 
> nvme-tcp builds iov_iter(BVEC) from __bvec_iter_bvec(), the segment
> number has to be the actual bvec number. But bio_segment() just returns
> number of the single-page segment, which is wrong for iov_iter.

That is what I thought, but its causing a crash, and was fine with
bio_segments. So I'm trying to understand why is that.

> Please see the same usage in lo_rw_aio().

nvme-tcp works on the bio basis to avoid bvec allocation
in the data path. Hence the iterator is fed directly by
the bio bvec and will re-initialize on every bio that
is spanned by the request.


From sagi at grimberg.me  Tue Feb  9 05:09:58 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 9 Feb 2021 02:09:58 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <65F086AF-DDD7-46B7-9134-3E6B577E75A6@wdc.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
 <20210209042103.GB63798@T590>
 <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>
 <20210209075001.GA94287@T590> <65F086AF-DDD7-46B7-9134-3E6B577E75A6@wdc.com>
Message-ID: <92b6d55e-1a29-c5f2-f60c-0d1ac904dd4f@grimberg.me>


>>>> Yi, can you try the following patch?
>>>>
>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>> index 881d28eb15e9..a393d99b74e1 100644
>>>> --- a/drivers/nvme/host/tcp.c
>>>> +++ b/drivers/nvme/host/tcp.c
>>>> @@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>>>>           offset = 0;
>>>>       } else {
>>>>           struct bio *bio = req->curr_bio;
>>>> +        struct bio_vec bv;
>>>> +        struct bvec_iter iter;
>>>> +
>>>> +        nsegs = 0;
>>>> +        bio_for_each_bvec(bv, bio, iter)
>>>> +            nsegs++;
>>>>           vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
>>>> -        nsegs = bio_segments(bio);
>>>
>>> This was exactly the patch that caused the issue.
>>
>> What was the issue you are talking about? Any link or commit hash?
>>
>> nvme-tcp builds iov_iter(BVEC) from __bvec_iter_bvec(), the segment
>> number has to be the actual bvec number. But bio_segment() just returns
>> number of the single-page segment, which is wrong for iov_iter.
>>
>> Please see the same usage in lo_rw_aio().
>>
> That what I have suggested but I've also suggested the memory allocation part which Sagi explained why it is better to avoid.
> 
> In my opinion we should at least try bvec calculation in lo_aio_rw() and see the problem can be fixed or not, unless reverting the commit it right approach for some reason.

I'm trying to understand what this is, but I'm failing to reproduce
this. I may ask Yi to add some debug code for this.


From sagi at grimberg.me  Tue Feb  9 05:12:34 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue,  9 Feb 2021 02:12:34 -0800
Subject: [PATCH 1/2] nvmet: don't check iosqes,
 iocqes for discovery controllers
Message-ID: <20210209101235.1539917-1-sagi@grimberg.me>

>From the base spec, Figure 78:
"Controller Configuration, these fields are defined as parameters to
configure an "I/O Controller (IOC)" and not to configure a "Discovery
Controller (DC)".
...
"If the controller does not support I/O queues, then this field shall
be read-only with a value of 0h"

Just have this check for I/O controllers.

Fixes: a07b4970f464 ("nvmet: add a generic NVMe target")
Reported-by: Belanger, Martin <Martin.Belanger at dell.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/target/core.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 8ce4d59cc9e7..baa26a65d83d 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1107,9 +1107,14 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
 {
 	lockdep_assert_held(&ctrl->lock);
 
-	if (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
-	    nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES ||
-	    nvmet_cc_mps(ctrl->cc) != 0 ||
+	if (!ctrl->subsys->type == NVME_NQN_DISC &&
+	    (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
+	     nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES)) {
+		ctrl->csts = NVME_CSTS_CFS;
+		return;
+	}
+
+	if (nvmet_cc_mps(ctrl->cc) != 0 ||
 	    nvmet_cc_ams(ctrl->cc) != 0 ||
 	    nvmet_cc_css(ctrl->cc) != 0) {
 		ctrl->csts = NVME_CSTS_CFS;
-- 
2.27.0



From sagi at grimberg.me  Tue Feb  9 05:12:35 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue,  9 Feb 2021 02:12:35 -0800
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery controllers
In-Reply-To: <20210209101235.1539917-1-sagi@grimberg.me>
References: <20210209101235.1539917-1-sagi@grimberg.me>
Message-ID: <20210209101235.1539917-2-sagi@grimberg.me>

>From the base spec, Figure 78:
"Controller Configuration, these fields are defined as parameters to
configure an "I/O Controller (IOC)" and not to configure a "Discovery
Controller (DC)".
...
"If the controller does not support I/O queues, then this field shall
be read-only with a value of 0h"

Set these only for I/O controllers.

Reported-by: Belanger, Martin <Martin.Belanger at dell.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/core.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 546a10407385..a7578c601727 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2443,6 +2443,11 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl)
 }
 EXPORT_SYMBOL_GPL(nvme_disable_ctrl);
 
+static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
+{
+	return ctrl->opts && ctrl->opts->discovery_nqn;
+}
+
 int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
 {
 	unsigned dev_page_min;
@@ -2468,7 +2473,8 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
 		ctrl->ctrl_config = NVME_CC_CSS_NVM;
 	ctrl->ctrl_config |= (NVME_CTRL_PAGE_SHIFT - 12) << NVME_CC_MPS_SHIFT;
 	ctrl->ctrl_config |= NVME_CC_AMS_RR | NVME_CC_SHN_NONE;
-	ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
+	if (!nvme_discovery_ctrl(ctrl))
+		ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
 	ctrl->ctrl_config |= NVME_CC_ENABLE;
 
 	ret = ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config);
@@ -2888,11 +2894,6 @@ static const struct attribute_group *nvme_subsys_attrs_groups[] = {
 	NULL,
 };
 
-static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
-{
-	return ctrl->opts && ctrl->opts->discovery_nqn;
-}
-
 static bool nvme_validate_cntlid(struct nvme_subsystem *subsys,
 		struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
 {
-- 
2.27.0



From sagi at grimberg.me  Tue Feb  9 05:25:56 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 9 Feb 2021 02:25:56 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
Message-ID: <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>


> Hi Sagi
> 
> On 2/8/21 5:46 PM, Sagi Grimberg wrote:
>>
>>> Hello
>>>
>>> We found this kernel NULL pointer issue with latest 
>>> linux-block/for-next and it's 100% reproduced, let me know if you 
>>> need more info/testing, thanks
>>>
>>> Kernel repo: 
>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>>
>>> Reproducer: blktests nvme-tcp/012
>>
>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>> but did not succeed. Given that you have it 100% reproducible,
>> can you try to revert commit:
>>
>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>
> 
> Revert this commit fixed the issue and I've attached the config. :)

Hey Ming,

Instead of revert, does this patch makes the issue go away?
--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 619b0d8f6e38..69f59d2c5799 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2271,7 +2271,7 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct 
nvme_ns *ns,
         req->data_len = blk_rq_nr_phys_segments(rq) ?
                                 blk_rq_payload_bytes(rq) : 0;
         req->curr_bio = rq->bio;
-       if (req->curr_bio)
+       if (req->curr_bio && req->data_len)
                 nvme_tcp_init_iter(req, rq_data_dir(rq));

         if (rq_data_dir(rq) == WRITE &&
--


From ming.lei at redhat.com  Tue Feb  9 05:33:00 2021
From: ming.lei at redhat.com (Ming Lei)
Date: Tue, 9 Feb 2021 18:33:00 +0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <d5e33680-5196-2873-332f-19191c60fd3b@grimberg.me>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
 <20210209042103.GB63798@T590>
 <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>
 <20210209075001.GA94287@T590>
 <d5e33680-5196-2873-332f-19191c60fd3b@grimberg.me>
Message-ID: <20210209103300.GA101814@T590>

On Tue, Feb 09, 2021 at 02:07:15AM -0800, Sagi Grimberg wrote:
> 
> > > > 
> > > > One obvious error is that nr_segments is computed wrong.
> > > > 
> > > > Yi, can you try the following patch?
> > > > 
> > > > diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> > > > index 881d28eb15e9..a393d99b74e1 100644
> > > > --- a/drivers/nvme/host/tcp.c
> > > > +++ b/drivers/nvme/host/tcp.c
> > > > @@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
> > > >    		offset = 0;
> > > >    	} else {
> > > >    		struct bio *bio = req->curr_bio;
> > > > +		struct bio_vec bv;
> > > > +		struct bvec_iter iter;
> > > > +
> > > > +		nsegs = 0;
> > > > +		bio_for_each_bvec(bv, bio, iter)
> > > > +			nsegs++;
> > > >    		vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
> > > > -		nsegs = bio_segments(bio);
> > > 
> > > This was exactly the patch that caused the issue.
> > 
> > What was the issue you are talking about? Any link or commit hash?
> 
> The commit that caused the crash is:
> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter

Not found this commit in linus tree, :-(

> 
> > 
> > nvme-tcp builds iov_iter(BVEC) from __bvec_iter_bvec(), the segment
> > number has to be the actual bvec number. But bio_segment() just returns
> > number of the single-page segment, which is wrong for iov_iter.
> 
> That is what I thought, but its causing a crash, and was fine with
> bio_segments. So I'm trying to understand why is that.

I tested this patch, and it works just fine.

> 
> > Please see the same usage in lo_rw_aio().
> 
> nvme-tcp works on the bio basis to avoid bvec allocation
> in the data path. Hence the iterator is fed directly by
> the bio bvec and will re-initialize on every bio that
> is spanned by the request.

Yeah, I know that. What I meant is that rq_for_each_bvec() is used
to figure out bvec number in loop, which may feed the bio bvec
directly to fs via iov_iter too, just similar with nvme-tcp.

The difference is that loop will switch to allocate a new bvec
table and copy bios's bvec to the new table in case of bios merge.

-- 
Ming



From sagi at grimberg.me  Tue Feb  9 05:36:41 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 9 Feb 2021 02:36:41 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <20210209103300.GA101814@T590>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <af1d7e9d-0170-82f6-30e1-01f045d73fc7@grimberg.me>
 <6147d452-a12e-c76c-22f1-5d9e7cb6b01d@grimberg.me>
 <20210209042103.GB63798@T590>
 <1ea82025-44b8-ac3a-2039-35cb8d36dac2@grimberg.me>
 <20210209075001.GA94287@T590>
 <d5e33680-5196-2873-332f-19191c60fd3b@grimberg.me>
 <20210209103300.GA101814@T590>
Message-ID: <7caeb8fa-94cc-89aa-94ec-da084491cc04@grimberg.me>



On 2/9/21 2:33 AM, Ming Lei wrote:
> On Tue, Feb 09, 2021 at 02:07:15AM -0800, Sagi Grimberg wrote:
>>
>>>>>
>>>>> One obvious error is that nr_segments is computed wrong.
>>>>>
>>>>> Yi, can you try the following patch?
>>>>>
>>>>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>>>>> index 881d28eb15e9..a393d99b74e1 100644
>>>>> --- a/drivers/nvme/host/tcp.c
>>>>> +++ b/drivers/nvme/host/tcp.c
>>>>> @@ -239,9 +239,14 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
>>>>>     		offset = 0;
>>>>>     	} else {
>>>>>     		struct bio *bio = req->curr_bio;
>>>>> +		struct bio_vec bv;
>>>>> +		struct bvec_iter iter;
>>>>> +
>>>>> +		nsegs = 0;
>>>>> +		bio_for_each_bvec(bv, bio, iter)
>>>>> +			nsegs++;
>>>>>     		vec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
>>>>> -		nsegs = bio_segments(bio);
>>>>
>>>> This was exactly the patch that caused the issue.
>>>
>>> What was the issue you are talking about? Any link or commit hash?
>>
>> The commit that caused the crash is:
>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
> 
> Not found this commit in linus tree, :-(

The original report is on:
Kernel repo: 
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next

>>> nvme-tcp builds iov_iter(BVEC) from __bvec_iter_bvec(), the segment
>>> number has to be the actual bvec number. But bio_segment() just returns
>>> number of the single-page segment, which is wrong for iov_iter.
>>
>> That is what I thought, but its causing a crash, and was fine with
>> bio_segments. So I'm trying to understand why is that.
> 
> I tested this patch, and it works just fine.

Me too, but Yi hits this crash, I even recompiled with his
config, but still no luck.

>>> Please see the same usage in lo_rw_aio().
>>
>> nvme-tcp works on the bio basis to avoid bvec allocation
>> in the data path. Hence the iterator is fed directly by
>> the bio bvec and will re-initialize on every bio that
>> is spanned by the request.
> 
> Yeah, I know that. What I meant is that rq_for_each_bvec() is used
> to figure out bvec number in loop, which may feed the bio bvec
> directly to fs via iov_iter too, just similar with nvme-tcp.
> 
> The difference is that loop will switch to allocate a new bvec
> table and copy bios's bvec to the new table in case of bios merge.

So in nvme-tcp used bio_for_each_bvec which seems appropriate, just
need to understand what is causing this.


From dwagner at suse.de  Tue Feb  9 07:37:36 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Tue, 9 Feb 2021 13:37:36 +0100
Subject: [PATCHv2] nvme/hwmon: rework to avoid devm allocation
In-Reply-To: <20210127163555.GA22717@lst.de>
References: <20210119064318.124415-1-hare@suse.de>
 <20210119145632.GD54031@roeck-us.net>
 <1093bedb-9b6d-b365-85d0-de8ef7eceb95@suse.de>
 <efddf566-a5a1-cda0-5fec-5e8a1d6bffb3@roeck-us.net>
 <20210127163555.GA22717@lst.de>
Message-ID: <20210209123736.pkfb2aokt47n3jw5@beryllium.lan>

On Wed, Jan 27, 2021 at 05:35:55PM +0100, Christoph Hellwig wrote:
> On Tue, Jan 19, 2021 at 09:54:30PM -0800, Guenter Roeck wrote:
> > It needs lm-sensors version 3.5.0 or later. Older versions
> > don't support a device hierarchy.
>
> Hannes, did you get a chance to test this with lm-sensors?

I gave it a go. All looks good.

Tested-by: Daniel Wagner <dwagner at suse.de>


  dolin:~/:[0]# sensors --version
  sensors version 3.5.0 with libsensors version 3.5.0

Baseline:

  dolin:~/:[0]# sensors
  coretemp-isa-0000
  Adapter: ISA adapter
  Package id 0:  +36.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 0:        +28.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 1:        +29.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 2:        +29.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 3:        +28.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 4:        +29.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 8:        +27.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 9:        +28.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 10:       +28.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 11:       +28.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 12:       +28.0?C  (high = +67.0?C, crit = +77.0?C)

  nvme-virtual-0
  Adapter: Virtual device
  Composite:   -273.1?C

  nvme-virtual-0
  Adapter: Virtual device
  Composite:   -273.1?C
  [...]


With the patch:

  dolin:~/:[0]# sensors
  coretemp-isa-0000
  Adapter: ISA adapter
  Package id 0:  +38.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 0:        +29.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 1:        +31.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 2:        +29.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 3:        +30.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 4:        +29.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 8:        +30.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 9:        +28.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 10:       +30.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 11:       +30.0?C  (high = +67.0?C, crit = +77.0?C)
  Core 12:       +28.0?C  (high = +67.0?C, crit = +77.0?C)

  nvme-virtual-0
  Adapter: Virtual device
  Composite:   -273.1?C

  nvme-virtual-0
  Adapter: Virtual device
  Composite:   -273.1?C
  [...]


  dolin:~/:[0]# find /sys -name hwmon*
  /sys/kernel/debug/tracing/events/hwmon
  /sys/kernel/debug/tracing/events/hwmon/hwmon_attr_show
  /sys/kernel/debug/tracing/events/hwmon/hwmon_attr_store
  /sys/kernel/debug/tracing/events/hwmon/hwmon_attr_show_string
  /sys/class/hwmon
  /sys/class/hwmon/hwmon16
  /sys/class/hwmon/hwmon14
  /sys/class/hwmon/hwmon8
  /sys/class/hwmon/hwmon12
  /sys/class/hwmon/hwmon6
  /sys/class/hwmon/hwmon20
  /sys/class/hwmon/hwmon10
  /sys/class/hwmon/hwmon4
  /sys/class/hwmon/hwmon19
  /sys/class/hwmon/hwmon2
  /sys/class/hwmon/hwmon17
  /sys/class/hwmon/hwmon0
  /sys/class/hwmon/hwmon15
  /sys/class/hwmon/hwmon9
  /sys/class/hwmon/hwmon13
  /sys/class/hwmon/hwmon7
  /sys/class/hwmon/hwmon11
  /sys/class/hwmon/hwmon5
  /sys/class/hwmon/hwmon3
  /sys/class/hwmon/hwmon18
  /sys/class/hwmon/hwmon1
  /sys/devices/platform/coretemp.3/hwmon
  /sys/devices/platform/coretemp.3/hwmon/hwmon19
  /sys/devices/platform/coretemp.1/hwmon
  /sys/devices/platform/coretemp.1/hwmon/hwmon17
  /sys/devices/platform/coretemp.2/hwmon
  /sys/devices/platform/coretemp.2/hwmon/hwmon18
  /sys/devices/platform/coretemp.0/hwmon
  /sys/devices/platform/coretemp.0/hwmon/hwmon16
  /sys/devices/pci0000:00/0000:00:02.2/0000:03:00.0/hwmon
  /sys/devices/pci0000:00/0000:00:02.2/0000:03:00.0/hwmon/hwmon20
  /sys/devices/virtual/nvme-fabrics/ctl/nvme23/hwmon11
  /sys/devices/virtual/nvme-fabrics/ctl/nvme7/hwmon1
  /sys/devices/virtual/nvme-fabrics/ctl/nvme21/hwmon9
  /sys/devices/virtual/nvme-fabrics/ctl/nvme18/hwmon6
  /sys/devices/virtual/nvme-fabrics/ctl/nvme26/hwmon14
  /sys/devices/virtual/nvme-fabrics/ctl/nvme16/hwmon4
  /sys/devices/virtual/nvme-fabrics/ctl/nvme24/hwmon12
  /sys/devices/virtual/nvme-fabrics/ctl/nvme14/hwmon2
  /sys/devices/virtual/nvme-fabrics/ctl/nvme22/hwmon10
  /sys/devices/virtual/nvme-fabrics/ctl/nvme6/hwmon0
  /sys/devices/virtual/nvme-fabrics/ctl/nvme20/hwmon8
  /sys/devices/virtual/nvme-fabrics/ctl/nvme19/hwmon7
  /sys/devices/virtual/nvme-fabrics/ctl/nvme27/hwmon15
  /sys/devices/virtual/nvme-fabrics/ctl/nvme17/hwmon5
  /sys/devices/virtual/nvme-fabrics/ctl/nvme25/hwmon13
  /sys/devices/virtual/nvme-fabrics/ctl/nvme15/hwmon3
  


From yi.zhang at redhat.com  Tue Feb  9 07:57:55 2021
From: yi.zhang at redhat.com (Yi Zhang)
Date: Tue, 9 Feb 2021 20:57:55 +0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
Message-ID: <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>



On 2/9/21 6:25 PM, Sagi Grimberg wrote:
>
>> Hi Sagi
>>
>> On 2/8/21 5:46 PM, Sagi Grimberg wrote:
>>>
>>>> Hello
>>>>
>>>> We found this kernel NULL pointer issue with latest 
>>>> linux-block/for-next and it's 100% reproduced, let me know if you 
>>>> need more info/testing, thanks
>>>>
>>>> Kernel repo: 
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
>>>> Commit: 11f8b6fd0db9 - Merge branch 'for-5.12/io_uring' into for-next
>>>>
>>>> Reproducer: blktests nvme-tcp/012
>>>
>>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>>> but did not succeed. Given that you have it 100% reproducible,
>>> can you try to revert commit:
>>>
>>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>>
>>
>> Revert this commit fixed the issue and I've attached the config. :)
>
> Hey Ming,
>
> Instead of revert, does this patch makes the issue go away?
Hi Sagi

Below patch fixed the issue, let me know if you need more testing. :)

Thanks
Yi

> -- 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 619b0d8f6e38..69f59d2c5799 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2271,7 +2271,7 @@ static blk_status_t 
> nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns,
> ??????? req->data_len = blk_rq_nr_phys_segments(rq) ?
> ??????????????????????????????? blk_rq_payload_bytes(rq) : 0;
> ??????? req->curr_bio = rq->bio;
> -?????? if (req->curr_bio)
> +?????? if (req->curr_bio && req->data_len)
> ??????????????? nvme_tcp_init_iter(req, rq_data_dir(rq));
>
> ??????? if (rq_data_dir(rq) == WRITE &&
> -- 
>



From mgurtovoy at nvidia.com  Tue Feb  9 10:07:05 2021
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Tue, 9 Feb 2021 15:07:05 +0000
Subject: [PATCH 2/2] nvme: resync header file with common nvme-cli tool
In-Reply-To: <20210127174717.GF28180@lst.de>
References: <20210121090948.8027-1-mgurtovoy@nvidia.com>
 <20210121090948.8027-4-mgurtovoy@nvidia.com> <20210127174717.GF28180@lst.de>
Message-ID: <BYAPR12MB349445BDAFCBD9C9FBC32701DE8E9@BYAPR12MB3494.namprd12.prod.outlook.com>

is this respin still needed ?
sorry for the late answer..

-----Original Message-----
From: Christoph Hellwig <hch at lst.de> 
Sent: Wednesday, January 27, 2021 7:47 PM
To: Max Gurtovoy <mgurtovoy at nvidia.com>
Cc: linux-nvme at lists.infradead.org; sagi at grimberg.me; kbusch at kernel.org; hch at lst.de; chaitanya.kulkarni at wdc.com
Subject: Re: [PATCH 2/2] nvme: resync header file with common nvme-cli tool

Can you respin this against the nvme-5.12 branch?  As-is git-am is not happy with it.



From bvanassche at acm.org  Tue Feb  9 10:24:12 2021
From: bvanassche at acm.org (Bart Van Assche)
Date: Tue, 9 Feb 2021 07:24:12 -0800
Subject: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO
In-Reply-To: <20210209031252.GA97526@C02WT3WMHTD6>
References: <20210105224939.1336-2-clay.mayers@kioxia.com>
 <20210125195844.1390581-1-clay.mayers@kioxia.com>
 <BYAPR04MB49650B6342C736B5268F833686BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b5c179e3eb904800931712d6b92cfbeb@kioxia.com>
 <BYAPR04MB49652EA8F4CF06D41C708E2386BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com>
 <20210209031252.GA97526@C02WT3WMHTD6>
Message-ID: <d2bc1899-3660-c0af-e5c9-dca7af8d2c0d@acm.org>

On 2/8/21 7:12 PM, Keith Busch wrote:
> On Tue, Feb 09, 2021 at 12:53:17AM +0000, Clay Mayers wrote:
>> Is there any other feedback on V2?
>>
>> My main concern I have about my implementation is how fused requests
>> are tunneled through the mq request layer.  The 1st request is marked as
>> started but it won't be in the device until the 2nd command is queued.  As
>> Keith pointed out, a device reset can split the two so care must be taken to
>> correctly handle this case.  Despite this, I thought this was a better approach
>> than modifying mq requests to be fused.  Especially given Christoph's
>> concern of cost vs value.  This is the lightest touch I could come up with.
>>
>> Further consideration of this patch may need a more compelling use case.
>> I've worked on a proprietary storage systems that relied on fused NVMeOF
>> support so it seems compelling to me.  There's a comment in target/core.c
>> that there is "no support for fused commands yet" implying it's been
>> considered.  Is pci only support for fused too soon or too little?  What would
>> make it more compelling?
> 
> The complications it introduces to the IO path and error handling for an
> archaic feature has me on the "Nak" side. NVMeOF was introduced well after the
> spec define Reservations, and the kernel has supported that capability for many
> years. I'm not aware of any other use case for fused commands, so it appears to
> be dead weight in the spec.

Hi Keith,

Do you agree that NVMe persistent reservation commands apply to an NVMe 
namespace in its entirety while fused compare-and-write commands allow 
to implement locking for subsets of the LBA range of a namespace? In 
other words, I think there is a valid use case for fused commands.

Thanks,

Bart.


From hch at lst.de  Tue Feb  9 10:35:03 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 9 Feb 2021 16:35:03 +0100
Subject: [PATCH 2/2] nvme: resync header file with common nvme-cli tool
In-Reply-To: <BYAPR12MB349445BDAFCBD9C9FBC32701DE8E9@BYAPR12MB3494.namprd12.prod.outlook.com>
References: <20210121090948.8027-1-mgurtovoy@nvidia.com>
 <20210121090948.8027-4-mgurtovoy@nvidia.com> <20210127174717.GF28180@lst.de>
 <BYAPR12MB349445BDAFCBD9C9FBC32701DE8E9@BYAPR12MB3494.namprd12.prod.outlook.com>
Message-ID: <20210209153503.GA8607@lst.de>

On Tue, Feb 09, 2021 at 03:07:05PM +0000, Max Gurtovoy wrote:
> is this respin still needed ?

Yes.


From Clay.Mayers at kioxia.com  Tue Feb  9 10:38:53 2021
From: Clay.Mayers at kioxia.com (Clay Mayers)
Date: Tue, 9 Feb 2021 15:38:53 +0000
Subject: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO
In-Reply-To: <20210209031252.GA97526@C02WT3WMHTD6>
References: <20210105224939.1336-2-clay.mayers@kioxia.com>
 <20210125195844.1390581-1-clay.mayers@kioxia.com>
 <BYAPR04MB49650B6342C736B5268F833686BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b5c179e3eb904800931712d6b92cfbeb@kioxia.com>
 <BYAPR04MB49652EA8F4CF06D41C708E2386BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com>
 <20210209031252.GA97526@C02WT3WMHTD6>
Message-ID: <16731baa6d8f4729a10edeea57267c41@kioxia.com>

> From: Keith Busch <kbusch at kernel.org>
> Sent: Monday, February 8, 2021 7:13 PM
> To: Clay Mayers <Clay.Mayers at kioxia.com>
> Cc: linux-nvme at lists.infradead.org; Jens Axboe <axboe at fb.com>; Chaitanya
> Kulkarni <Chaitanya.Kulkarni at wdc.com>; Christoph Hellwig <hch at lst.de>;
> Sagi Grimberg <sagi at grimberg.me>
> Subject: Re: [PATCH V2 0/2] nvme: Support for fused
> NVME_IOCTL_SUBMIT_IO
> 
> On Tue, Feb 09, 2021 at 12:53:17AM +0000, Clay Mayers wrote:
> > Is there any other feedback on V2?
> >
> > My main concern I have about my implementation is how fused requests
> > are tunneled through the mq request layer.  The 1st request is marked
> > as started but it won't be in the device until the 2nd command is
> > queued.  As Keith pointed out, a device reset can split the two so
> > care must be taken to correctly handle this case.  Despite this, I
> > thought this was a better approach than modifying mq requests to be
> > fused.  Especially given Christoph's concern of cost vs value.  This is the
> lightest touch I could come up with.
> >
> > Further consideration of this patch may need a more compelling use case.
> > I've worked on a proprietary storage systems that relied on fused
> > NVMeOF support so it seems compelling to me.  There's a comment in
> > target/core.c that there is "no support for fused commands yet"
> > implying it's been considered.  Is pci only support for fused too soon
> > or too little?  What would make it more compelling?
> 
> The complications it introduces to the IO path and error handling for an
> archaic feature has me on the "Nak" side. NVMeOF was introduced well after
> the spec define Reservations, and the kernel has supported that capability
> for many years. I'm not aware of any other use case for fused commands, so
> it appears to be dead weight in the spec.

Thanks for sharing your perspective.  I can see the point that reservations are
good enough for many.  For VmWare, moving from reservations to cmp/write
on SCSI led to an increase in performance (couldn't find an actual # but heard
30% anecdotally). That's a specialized use case (storage for VMs) on a much
slower medium that may not translate to more general storage systems or
much faster storage.



From Clay.Mayers at kioxia.com  Tue Feb  9 10:53:38 2021
From: Clay.Mayers at kioxia.com (Clay Mayers)
Date: Tue, 9 Feb 2021 15:53:38 +0000
Subject: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO
In-Reply-To: <20210209075434.GA31373@lst.de>
References: <20210105224939.1336-2-clay.mayers@kioxia.com>
 <20210125195844.1390581-1-clay.mayers@kioxia.com>
 <BYAPR04MB49650B6342C736B5268F833686BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b5c179e3eb904800931712d6b92cfbeb@kioxia.com>
 <BYAPR04MB49652EA8F4CF06D41C708E2386BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <b2a0593cd1fc4f239bbd44f3f43d4bcd@kioxia.com> <20210209075434.GA31373@lst.de>
Message-ID: <94d6c39917d04b88ae0099473df9f4d0@kioxia.com>

> From: Christoph Hellwig <hch at lst.de>
> Sent: Monday, February 8, 2021 11:55 PM
> To: Clay Mayers <Clay.Mayers at kioxia.com>
> Cc: linux-nvme at lists.infradead.org; Keith Busch <kbusch at kernel.org>; Jens
> Axboe <axboe at fb.com>; Chaitanya Kulkarni
> <Chaitanya.Kulkarni at wdc.com>; Christoph Hellwig <hch at lst.de>; Sagi
> Grimberg <sagi at grimberg.me>
> Subject: Re: [PATCH V2 0/2] nvme: Support for fused
> NVME_IOCTL_SUBMIT_IO
> 
> On Tue, Feb 09, 2021 at 12:53:17AM +0000, Clay Mayers wrote:
> > > From: Clay Mayers
> > > Sent: Tuesday, January 26, 2021 1:14 PM
> > > To: linux-nvme at lists.infradead.org
> > > Cc: Keith Busch <kbusch at kernel.org>; Jens Axboe <axboe at fb.com>;
> > > 'Chaitanya Kulkarni' <Chaitanya.Kulkarni at wdc.com>; Christoph Hellwig
> > > <hch at lst.de>; Sagi Grimberg <sagi at grimberg.me>
> > > Subject: RE: [PATCH V2 0/2] nvme: Support for fused
> > > NVME_IOCTL_SUBMIT_IO
> > >
> > Is there any other feedback on V2?
> 
> My feedback is that we should not add code to the kernel driver and our fast
> path for fused command unless we have a mainstream in-kernel users, that
> is not at all for the foreseeable future.

Thanks - that makes sense.  I know the patch isn't enough to be very useful on
its own.  I shared it at this stage because of my concern over the implementation.
Thanks to the feedback from the list, it is better than it was.


From sagi at grimberg.me  Tue Feb  9 13:01:52 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Tue, 9 Feb 2021 10:01:52 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
 <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>
Message-ID: <6ae16841-5f51-617a-aab7-666b7eed299c@grimberg.me>


>>>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>>>> but did not succeed. Given that you have it 100% reproducible,
>>>> can you try to revert commit:
>>>>
>>>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>>>
>>>
>>> Revert this commit fixed the issue and I've attached the config. :)
>>
>> Hey Ming,
>>
>> Instead of revert, does this patch makes the issue go away?
> Hi Sagi
> 
> Below patch fixed the issue, let me know if you need more testing. :)

Thanks Yi,

I'll submit a proper patch, but can you run this change
to see what command has a bio but without any data?
--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 619b0d8f6e38..311f1b78a9d4 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2271,8 +2271,13 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct 
nvme_ns *ns,
         req->data_len = blk_rq_nr_phys_segments(rq) ?
                                 blk_rq_payload_bytes(rq) : 0;
         req->curr_bio = rq->bio;
-       if (req->curr_bio)
+       if (req->curr_bio) {
+               if (!req->data_len) {
+                       pr_err("rq %d opcode %d\n", rq->tag, 
pdu->cmd.common.opcode);
+                       return BLK_STS_IOERR;
+               }
                 nvme_tcp_init_iter(req, rq_data_dir(rq));
+       }

         if (rq_data_dir(rq) == WRITE &&
             req->data_len <= nvme_tcp_inline_data_size(queue))
--


From jxgao at google.com  Tue Feb  9 14:49:40 2021
From: jxgao at google.com (Jianxiong Gao)
Date: Tue, 9 Feb 2021 11:49:40 -0800
Subject: preserve DMA offsets when using swiotlb v2
In-Reply-To: <20210209084156.GA32320@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
 <20210209084156.GA32320@lst.de>
Message-ID: <CAMGD6P2QiQba=49LMOsQNZ=U9+hmAjxYLwroZGAG_AXoatUYpg@mail.gmail.com>

> Sorry for being a little pushy, any chance we could get this reviewed
> in time for the 5.12 merge window?
>
Tested and it looks good. Thanks for clearing it up!

Gao


From lkp at intel.com  Tue Feb  9 15:08:15 2021
From: lkp at intel.com (kernel test robot)
Date: Wed, 10 Feb 2021 04:08:15 +0800
Subject: [linux-nvme:nvme-5.12] BUILD SUCCESS
 bef623436fdc53642f45a48f54f38e0f78d62e9c
Message-ID: <6022ebaf.uJ9qwtiYMmBKshbj%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.12
branch HEAD: bef623436fdc53642f45a48f54f38e0f78d62e9c  nvmet-fc: add a missing __rcu annotation to nvmet_fc_tgt_assoc.queues

elapsed time: 722m

configs tested: 128
configs skipped: 3

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
m68k                       m5208evb_defconfig
sh                        edosk7760_defconfig
sparc                            alldefconfig
arm                         palmz72_defconfig
h8300                    h8300h-sim_defconfig
powerpc                      ep88xc_defconfig
arm                              zx_defconfig
x86_64                           alldefconfig
mips                          rb532_defconfig
powerpc                 mpc836x_mds_defconfig
powerpc                   motionpro_defconfig
mips                     cu1000-neo_defconfig
csky                             alldefconfig
sh                             sh03_defconfig
mips                           rs90_defconfig
sparc64                             defconfig
sh                             shx3_defconfig
powerpc                     ksi8560_defconfig
powerpc                         ps3_defconfig
sh                        apsh4ad0a_defconfig
sh                        sh7785lcr_defconfig
mips                        vocore2_defconfig
mips                     cu1830-neo_defconfig
arm                        mini2440_defconfig
arm                          ep93xx_defconfig
arm                           sunxi_defconfig
powerpc                      makalu_defconfig
powerpc                    amigaone_defconfig
mips                  decstation_64_defconfig
powerpc                        cell_defconfig
m68k                          multi_defconfig
arc                          axs103_defconfig
um                           x86_64_defconfig
m68k                       m5249evb_defconfig
arm                        realview_defconfig
csky                                defconfig
powerpc                         wii_defconfig
riscv                    nommu_k210_defconfig
mips                          rm200_defconfig
arm                       aspeed_g5_defconfig
powerpc                      bamboo_defconfig
powerpc                 mpc85xx_cds_defconfig
m68k                       m5475evb_defconfig
sh                         microdev_defconfig
powerpc                       ppc64_defconfig
arm                          badge4_defconfig
arc                            hsdk_defconfig
arm                           omap1_defconfig
arm                         s5pv210_defconfig
powerpc                 mpc8540_ads_defconfig
m68k                            mac_defconfig
sh                        dreamcast_defconfig
mips                 decstation_r4k_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
s390                             allmodconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                               tinyconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
x86_64               randconfig-a006-20210209
x86_64               randconfig-a001-20210209
x86_64               randconfig-a005-20210209
x86_64               randconfig-a004-20210209
x86_64               randconfig-a002-20210209
x86_64               randconfig-a003-20210209
i386                 randconfig-a001-20210209
i386                 randconfig-a005-20210209
i386                 randconfig-a003-20210209
i386                 randconfig-a002-20210209
i386                 randconfig-a006-20210209
i386                 randconfig-a004-20210209
i386                 randconfig-a016-20210209
i386                 randconfig-a013-20210209
i386                 randconfig-a012-20210209
i386                 randconfig-a014-20210209
i386                 randconfig-a011-20210209
i386                 randconfig-a015-20210209
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a013-20210209
x86_64               randconfig-a014-20210209
x86_64               randconfig-a015-20210209
x86_64               randconfig-a012-20210209
x86_64               randconfig-a016-20210209
x86_64               randconfig-a011-20210209

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From sironi at amazon.de  Tue Feb  9 19:39:42 2021
From: sironi at amazon.de (Filippo Sironi)
Date: Wed, 10 Feb 2021 01:39:42 +0100
Subject: [PATCH v2] nvme: Add 48-bit DMA address quirk for Amazon NVMe
 controllers
In-Reply-To: <20210203094338.19473-1-sironi@amazon.de>
References: <20210203094338.19473-1-sironi@amazon.de>
Message-ID: <20210210003942.25700-1-sironi@amazon.de>

Amazon NVMe controllers do not support 64-bit DMA addresses; they are
limited to 48-bit DMA addresses.  Let's add a quirk to ensure that we
make use of 48-bit DMA addresses to avoid misbehavior.

This affects all Amazon NVMe controllers that expose EBS volumes
(0x0061, 0x0065, 0x8061) and local instance storage (0xcd00, 0xcd01,
0xcd02).

Signed-off-by: Filippo Sironi <sironi at amazon.de>
---
 drivers/nvme/host/nvme.h |  5 +++++
 drivers/nvme/host/pci.c  | 17 ++++++++++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 88a6b97247f5..dae747b4ac35 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -144,6 +144,11 @@ enum nvme_quirks {
 	 * NVMe 1.3 compliance.
 	 */
 	NVME_QUIRK_NO_NS_DESC_LIST		= (1 << 15),
+
+	/*
+	 * The controller supports up to 48-bit DMA address.
+	 */
+	NVME_QUIRK_DMA_ADDRESS_BITS_48		= (1 << 16),
 };
 
 /*
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 6bad4d4dcdf0..e7001f5ed6e4 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2362,13 +2362,16 @@ static int nvme_pci_enable(struct nvme_dev *dev)
 {
 	int result = -ENOMEM;
 	struct pci_dev *pdev = to_pci_dev(dev->dev);
+	int dma_address_bits = 64;
 
 	if (pci_enable_device_mem(pdev))
 		return result;
 
 	pci_set_master(pdev);
 
-	if (dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(64)))
+	if (dev->ctrl.quirks & NVME_QUIRK_DMA_ADDRESS_BITS_48)
+		dma_address_bits = 48;
+	if (dma_set_mask_and_coherent(dev->dev, DMA_BIT_MASK(dma_address_bits)))
 		goto disable;
 
 	if (readl(dev->bar + NVME_REG_CSTS) == -1) {
@@ -3263,6 +3266,18 @@ static const struct pci_device_id nvme_id_table[] = {
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(0x2646, 0x2263),   /* KINGSTON A2000 NVMe SSD  */
 		.driver_data = NVME_QUIRK_NO_DEEPEST_PS, },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0x0061),
+		.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0x0065),
+		.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0x8061),
+		.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0xcd00),
+		.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0xcd01),
+		.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
+	{ PCI_DEVICE(PCI_VENDOR_ID_AMAZON, 0xcd02),
+		.driver_data = NVME_QUIRK_DMA_ADDRESS_BITS_48, },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2001),
 		.driver_data = NVME_QUIRK_SINGLE_VECTOR },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2003) },
-- 
2.17.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879





From yi.zhang at redhat.com  Tue Feb  9 21:51:56 2021
From: yi.zhang at redhat.com (Yi Zhang)
Date: Wed, 10 Feb 2021 10:51:56 +0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <6ae16841-5f51-617a-aab7-666b7eed299c@grimberg.me>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
 <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>
 <6ae16841-5f51-617a-aab7-666b7eed299c@grimberg.me>
Message-ID: <1a520912-ac7c-1a3c-c432-b382a5da6177@redhat.com>



On 2/10/21 2:01 AM, Sagi Grimberg wrote:
>
>>>>> Thanks for reporting Ming, I've tried to reproduce this on my VM
>>>>> but did not succeed. Given that you have it 100% reproducible,
>>>>> can you try to revert commit:
>>>>>
>>>>> 0dc9edaf80ea nvme-tcp: pass multipage bvec to request iov_iter
>>>>>
>>>>
>>>> Revert this commit fixed the issue and I've attached the config. :)
>>>
>>> Hey Ming,
>>>
>>> Instead of revert, does this patch makes the issue go away?
>> Hi Sagi
>>
>> Below patch fixed the issue, let me know if you need more testing. :)
>
> Thanks Yi,
>
So it's nvme_admin_abort_cmd here

[?? 74.017450] run blktests nvme/012 at 2021-02-09 21:41:55
[?? 74.111311] loop: module loaded
[?? 74.125717] loop0: detected capacity change from 2097152 to 0
[?? 74.141026] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[?? 74.149395] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[?? 74.158298] nvmet: creating controller 1 for subsystem 
blktests-subsystem-1 for NQN 
nqn.2014-08.org.nvmexpress:uuid:41131d88-02ca-4ccc-87b3-6ca3f28b13a4.
[?? 74.158742] nvme nvme0: creating 48 I/O queues.
[?? 74.163391] nvme nvme0: mapped 48/0/0 default/read/poll queues.
[?? 74.184623] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
127.0.0.1:4420
[?? 75.235059] nvme_tcp: rq 38 opcode 8
[?? 75.238653] blk_update_request: I/O error, dev nvme0c0n1, sector 
1048624 op 0x9:(WRITE_ZEROES) flags 0x2800800 phys_seg 0 prio class 0
[?? 75.380179] XFS (nvme0n1): Mounting V5 Filesystem
[?? 75.387457] XFS (nvme0n1): Ending clean mount
[?? 75.388555] xfs filesystem being mounted at /mnt/blktests supports 
timestamps until 2038 (0x7fffffff)
[?? 91.035659] XFS (nvme0n1): Unmounting Filesystem
[?? 91.043334] nvme nvme0: Removing ctrl: NQN "blktests-subsystem-1"


> I'll submit a proper patch, but can you run this change
> to see what command has a bio but without any data?
> -- 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 619b0d8f6e38..311f1b78a9d4 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2271,8 +2271,13 @@ static blk_status_t 
> nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns,
> ??????? req->data_len = blk_rq_nr_phys_segments(rq) ?
> ??????????????????????????????? blk_rq_payload_bytes(rq) : 0;
> ??????? req->curr_bio = rq->bio;
> -?????? if (req->curr_bio)
> +?????? if (req->curr_bio) {
> +?????????????? if (!req->data_len) {
> +?????????????????????? pr_err("rq %d opcode %d\n", rq->tag, 
> pdu->cmd.common.opcode);
> +?????????????????????? return BLK_STS_IOERR;
> +?????????????? }
> ??????????????? nvme_tcp_init_iter(req, rq_data_dir(rq));
> +?????? }
>
> ??????? if (rq_data_dir(rq) == WRITE &&
> ??????????? req->data_len <= nvme_tcp_inline_data_size(queue))
> -- 
>



From kbusch at kernel.org  Tue Feb  9 22:38:02 2021
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 10 Feb 2021 12:38:02 +0900
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <1a520912-ac7c-1a3c-c432-b382a5da6177@redhat.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
 <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>
 <6ae16841-5f51-617a-aab7-666b7eed299c@grimberg.me>
 <1a520912-ac7c-1a3c-c432-b382a5da6177@redhat.com>
Message-ID: <20210210033802.GA23363@redsun51.ssa.fujisawa.hgst.com>

On Wed, Feb 10, 2021 at 10:51:56AM +0800, Yi Zhang wrote:
> On 2/10/21 2:01 AM, Sagi Grimberg wrote:
> So it's nvme_admin_abort_cmd here

The opcode 8 would be abort if this were an admin command, but we can't
tell that from the nvme print. The subsequent blk_update_request log
indicates this is the IO command for a Write Zeroes operation.
 
> [?? 75.235059] nvme_tcp: rq 38 opcode 8
> [?? 75.238653] blk_update_request: I/O error, dev nvme0c0n1, sector 1048624 op 0x9:(WRITE_ZEROES) flags 0x2800800 phys_seg 0 prio class 0


From Chaitanya.Kulkarni at wdc.com  Wed Feb 10 00:03:58 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 10 Feb 2021 05:03:58 +0000
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
 <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>
 <6ae16841-5f51-617a-aab7-666b7eed299c@grimberg.me>
 <1a520912-ac7c-1a3c-c432-b382a5da6177@redhat.com>
Message-ID: <BYAPR04MB49652279710E99EBB6F8AC67868D9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/9/21 18:52, Yi Zhang wrote:
> So it's nvme_admin_abort_cmd here
>
> [   74.017450] run blktests nvme/012 at 2021-02-09 21:41:55
> [   74.111311] loop: module loaded
> [   74.125717] loop0: detected capacity change from 2097152 to 0
> [   74.141026] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
> [   74.149395] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
> [   74.158298] nvmet: creating controller 1 for subsystem 
> blktests-subsystem-1 for NQN 
> nqn.2014-08.org.nvmexpress:uuid:41131d88-02ca-4ccc-87b3-6ca3f28b13a4.
> [   74.158742] nvme nvme0: creating 48 I/O queues.
> [   74.163391] nvme nvme0: mapped 48/0/0 default/read/poll queues.
> [   74.184623] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr 
> 127.0.0.1:4420
> [   75.235059] nvme_tcp: rq 38 opcode 8
> [   75.238653] blk_update_request: I/O error, dev nvme0c0n1, sector 
> 1048624 op 0x9:(WRITE_ZEROES) flags 0x2800800 phys_seg 0 prio class 0
> [   75.380179] XFS (nvme0n1): Mounting V5 Filesystem
> [   75.387457] XFS (nvme0n1): Ending clean mount
> [   75.388555] xfs filesystem being mounted at /mnt/blktests supports 
> timestamps until 2038 (0x7fffffff)
> [   91.035659] XFS (nvme0n1): Unmounting Filesystem
> [   91.043334] nvme nvme0: Removing ctrl: NQN "blktests-subsystem-1"

But write-zeores is also data less command and should not fail.


From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:51 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:51 -0800
Subject: [PATCH V3 00/11] nvmet: fixes and some cleanups
Message-ID: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>

Hi,

This patch-series fixes commit bffcd507780e
("nvmet: set right status on error in id-ns handler") and does cleanup
for nvmet_find_namespace() to get rid of the duplicate code and remove
the inconsistent error behavior of the host, Also this adds an helper
for unhandled commands to have uniform error message reporting for the
bdev, file and passthru backend. Patch 9/10 fix compilation warnings.
Last patch removes an unnecessary else at the tail of the
nvmet_parse_io_cmd().

This is based on nvme-5.12. All the blktests are seem to pass on this
series.                

Below is the testlog for latest change.

-ck   

Changes from V2:-

1. Set the status to 0 when nvmet_req_find_ns() returns an error
   for nvmet_execute_identify_ns().

Changes from V1:-

1, In nvmet_execute_identify_ns() zeroout buffer for unallocated nsid.
2. Adjust the patch #2 to reflect above behavior.
3. Merge patch #3 and #4 from V1 to change the nvmet_find_namespace()
   prototype to only accept request.
4. Rename nvmet_find_namespace()->nvmet_req_find_namespace().
5. Remove the backened string print from nvmet_report_invalid_opcode() &
   replace pr_err() with pr_debug().
6. Add a new patch that creates a helper nvmet_req_subsys() to replace
   a chain of the structures all over code to get the subsys from
   nvmet_req.
7. Add a patch to remove the unnecessay else at the tail of the
   nvmet_parse_io_cmd().

Chaitanya Kulkarni (11):
  nvmet: set status to 0 in case for invalid nsid
  nvmet: return uniform error for invalid ns
  nvmet: make nvmet_find_namespace() req based
  nvmet: remove extra variable in id-ns handler
  nvmet: add helper to report invalid opcode
  nvmet: use invalid cmd opcode helper
  nvmet: use invalid cmd opcode helper
  nvmet: use min of device_path and disk len
  nvme-loop: rename variable to get rid of the warn
  nvmet: add nvmet_req_subsys() helper
  nvmet: remove else at the end of the function

 drivers/nvme/target/admin-cmd.c   | 59 +++++++++++++------------------
 drivers/nvme/target/core.c        | 28 ++++++++-------
 drivers/nvme/target/io-cmd-bdev.c |  5 +--
 drivers/nvme/target/io-cmd-file.c |  5 +--
 drivers/nvme/target/loop.c        |  2 +-
 drivers/nvme/target/nvmet.h       | 18 ++++++++--
 drivers/nvme/target/passthru.c    |  6 ++--
 drivers/nvme/target/trace.h       |  4 ++-
 8 files changed, 64 insertions(+), 63 deletions(-)


# gitlog -11 
2bd1fdd47f27 (HEAD -> nvme-5.12) nvmet: remove else at the end of the function
f3ba4d069b9a nvmet: add nvmet_req_subsys() helper
f72b99a93c10 nvme-loop: rename variable to get rid of the warn
b8caa57f886b nvmet: use min of device_path and disk len
736b56163c13 nvmet: use invalid cmd opcode helper
7ddd60b3d6cf nvmet: use invalid cmd opcode helper
1f83a1024596 nvmet: add helper to report invalid opcode
a3904c9ca583 nvmet: remove extra variable in id-ns handler
c6d99eca6ec9 nvmet: make nvmet_find_namespace() req based
cfe473bf95a2 nvmet: return uniform error for invalid ns
87c302ac8edc nvmet: set status to 0 in case for invalid nsid

# Create 1 subsys with 1 ns
++ FILE=/dev/nvme0n1
++ NN=1
++ NQN=testnqn
++ let NR_DEVICES=NN+1
++ modprobe -r null_blk
++ modprobe null_blk nr_devices=0
++ modprobe nvme
++ modprobe nvme-fabrics
++ modprobe nvmet
++ modprobe nvme-loop
++ dmesg -c
++ sleep 2
++ tree /sys/kernel/config
/sys/kernel/config
??? nullb
??? ??? features
??? nvmet
    ??? hosts
    ??? ports
    ??? subsystems

5 directories, 1 file
++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn
++ mkdir /sys/kernel/config/nvmet/ports/1/
++ echo -n loop
++ echo -n 1
++ ln -s /sys/kernel/config/nvmet/subsystems/testnqn /sys/kernel/config/nvmet/ports/1/subsystems/
++ sleep 1
++ echo transport=loop,nqn=testnqn
+++ shuf -i 1-1 -n 1
++ for i in '`shuf -i  1-$NN -n $NN`'
++ mkdir config/nullb/nullb1
++ echo 4096
++ echo 20971520
++ echo 1
+++ cat config/nullb/nullb1/index
++ IDX=0
++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1
++ echo ' ####### /dev/nullb0'
 ####### /dev/nullb0
++ echo -n /dev/nullb0
++ cat /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1/device_path
/dev/nullb0
++ echo 1
++ dmesg -c
[24762.913991] nvmet: creating controller 1 for subsystem testnqn for NQN nqn.2014-08.org.nvmexpress:uuid:6a4a7451-6274-4d7f-93d3-49724cc1147e.
[24762.914802] nvme nvme1: Please enable CONFIG_NVME_MULTIPATH for full support of multi-port devices.
[24762.915034] nvme nvme1: creating 64 I/O queues.
[24762.932598] nvme nvme1: new ctrl: "testnqn"
[24762.966156] nvmet: adding nsid 1 to subsystem testnqn
[24762.972384] nvme nvme1: rescanning namespaces.
++ sleep 1
++ mount
++ column -t
++ grep nvme
++ '[' 1 ']'
+++ wc -l
+++ ls -l /dev/nvme1 /dev/nvme1n1
++ cnt=2
++ echo 2
2
++ '[' 2 -gt 1 ']'
++ break
++ dmesg -c
# 
# nvme list 
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev  
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     foo                  QEMU NVMe Ctrl                           1           1.07  GB /   1.07  GB    512   B +  0 B   1.0     
/dev/nvme1n1     fc13d75abd55eafa     Linux                                    1          21.99  TB /  21.99  TB      4 KiB +  0 B   5.11.0-r
# nvme id-ns /dev/nvme1 -n 1
NVME Identify Namespace 1:
nsze    : 0x140000000
ncap    : 0x140000000
nuse    : 0x140000000
nsfeat  : 0x12
nlbaf   : 0
flbas   : 0
mc      : 0
dpc     : 0
dps     : 0
nmic    : 0x1
rescap  : 0
fpi     : 0
dlfeat  : 0
nawun   : 0
nawupf  : 0
nacwu   : 0
nabsn   : 0
nabo    : 0
nabspf  : 0
noiob   : 0
nvmcap  : 0
npwg    : 0
npwa    : 0
npdg    : 0
npda    : 0
nows    : 0
mssrl   : 0
mcl     : 0
msrc    : 0
nsattr	: 0
nvmsetid: 0
anagrpid: 1
endgid  : 0
nguid   : 00000000000000000000000000000000
eui64   : 0000000000000000
lbaf  0 : ms:0   lbads:12 rp:0 (in use)
# Make sure we get the status 0 for identify namesapce command
# for unallocated nsid
# nvme id-ns /dev/nvme1 -n 2
NVME Identify Namespace 2:
nsze    : 0
ncap    : 0
nuse    : 0
nsfeat  : 0
nlbaf   : 0
flbas   : 0
mc      : 0
dpc     : 0
dps     : 0
nmic    : 0
rescap  : 0
fpi     : 0
dlfeat  : 0
nawun   : 0
nawupf  : 0
nacwu   : 0
nabsn   : 0
nabo    : 0
nabspf  : 0
noiob   : 0
nvmcap  : 0
mssrl   : 0
mcl     : 0
msrc    : 0
nsattr	: 0
nvmsetid: 0
anagrpid: 0
endgid  : 0
nguid   : 00000000000000000000000000000000
eui64   : 0000000000000000
lbaf  0 : ms:0   lbads:0  rp:0 (in use)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:52 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:52 -0800
Subject: [PATCH V3 01/11] nvmet: set status to 0 in case for invalid nsid
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-2-chaitanya.kulkarni@wdc.com>

For unallocated namespace in nvmet_execute_identify_ns() don't set the
status to NVME_SC_INVALID_NS, set it to zero.

Fixes: bffcd507780e ("nvmet: set right status on error in id-ns handler")
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 613a4d8feac1..5070ea5cf260 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -485,7 +485,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 	/* return an all zeroed buffer if we can't find an active namespace */
 	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
 	if (!req->ns) {
-		status = NVME_SC_INVALID_NS;
+		status = 0;
 		goto done;
 	}
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:53 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:53 -0800
Subject: [PATCH V3 02/11] nvmet: return uniform error for invalid ns
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-3-chaitanya.kulkarni@wdc.com>

For nvmet_find_namespace() error case we have inconsistent error code
mapping in the function nvmet_get_smart_log_nsid() and
nvmet_set_feat_write_protect().

There is no point in retrying for the invalid namesapce from the host
side. Set the error code to the NVME_SC_INVALID_NS | NVME_SC_DNR which
matches what we have in nvmet_execute_identify_desclist().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 5070ea5cf260..e938064254a5 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -82,7 +82,7 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req *req,
 		pr_err("Could not find namespace id : %d\n",
 				le32_to_cpu(req->cmd->get_log_page.nsid));
 		req->error_loc = offsetof(struct nvme_rw_command, nsid);
-		return NVME_SC_INVALID_NS;
+		return NVME_SC_INVALID_NS | NVME_SC_DNR;
 	}
 
 	/* we don't have the right data for file backed ns */
@@ -697,7 +697,7 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->rw.nsid);
 	if (unlikely(!req->ns)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return status;
+		return status = NVME_SC_INVALID_NS | NVME_SC_DNR;
 	}
 
 	mutex_lock(&subsys->lock);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:54 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:54 -0800
Subject: [PATCH V3 03/11] nvmet: make nvmet_find_namespace() req based
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-4-chaitanya.kulkarni@wdc.com>

The six callers of nvmet_find_namespace() duplicate the error log page
update and status setting code for each call on failure.

All callers are nvmet requests based functions, so we can pass req
to the nvmet_find_namesapce() & derive ctrl from req, that'll allow us
to update the error log page in nvmet_find_namespace(). Now that we
pass the request we can also get rid of the local variable in
nvmet_find_namespace() and use the req->ns and return the error code.

Replace the ctrl parameter with nvmet_req for nvmet_find_namespace(),
centralize the error log page update for non allocated namesapces, and
return uniform error for non-allocated namespace.

The nvmet_find_namespace() takes nsid parameter which is from NVMe
commands structures such as get_log_page, identify, rw and common. All
these commands have same offset for the nsid field.

Derive nsid from req->cmd->common.nsid) & remove the extra parameter
from the nvmet_find_namespace().

Lastly now we associate the ns to the req parameter that we pass to the
nvmet_find_namespace(), rename the nvmet_find_namespace() to
nvmet_req_find_ns() i.e. :-

Old:-
struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl,
				      __le32 nsid)
New :-
u16 nvmet_req_find_ns(struct nvmet_req *req);

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 46 +++++++++++++--------------------
 drivers/nvme/target/core.c      | 24 +++++++++--------
 drivers/nvme/target/nvmet.h     |  2 +-
 3 files changed, 32 insertions(+), 40 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index e938064254a5..f32533480e66 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -75,15 +75,11 @@ static u16 nvmet_get_smart_log_nsid(struct nvmet_req *req,
 		struct nvme_smart_log *slog)
 {
 	u64 host_reads, host_writes, data_units_read, data_units_written;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl,
-				       req->cmd->get_log_page.nsid);
-	if (!req->ns) {
-		pr_err("Could not find namespace id : %d\n",
-				le32_to_cpu(req->cmd->get_log_page.nsid));
-		req->error_loc = offsetof(struct nvme_rw_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	status = nvmet_req_find_ns(req);
+	if (status)
+		return status;
 
 	/* we don't have the right data for file backed ns */
 	if (!req->ns->bdev)
@@ -468,7 +464,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 {
 	struct nvmet_ctrl *ctrl = req->sq->ctrl;
 	struct nvme_id_ns *id;
-	u16 status = 0;
+	u16 status;
 
 	if (le32_to_cpu(req->cmd->identify.nsid) == NVME_NSID_ALL) {
 		req->error_loc = offsetof(struct nvme_identify, nsid);
@@ -483,8 +479,8 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 	}
 
 	/* return an all zeroed buffer if we can't find an active namespace */
-	req->ns = nvmet_find_namespace(ctrl, req->cmd->identify.nsid);
-	if (!req->ns) {
+	status = nvmet_req_find_ns(req);
+	if (status) {
 		status = 0;
 		goto done;
 	}
@@ -604,15 +600,12 @@ static u16 nvmet_copy_ns_identifier(struct nvmet_req *req, u8 type, u8 len,
 
 static void nvmet_execute_identify_desclist(struct nvmet_req *req)
 {
-	u16 status = 0;
 	off_t off = 0;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->identify.nsid);
-	if (!req->ns) {
-		req->error_loc = offsetof(struct nvme_identify, nsid);
-		status = NVME_SC_INVALID_NS | NVME_SC_DNR;
+	status = nvmet_req_find_ns(req);
+	if (status)
 		goto out;
-	}
 
 	if (memchr_inv(&req->ns->uuid, 0, sizeof(req->ns->uuid))) {
 		status = nvmet_copy_ns_identifier(req, NVME_NIDT_UUID,
@@ -692,13 +685,11 @@ static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 {
 	u32 write_protect = le32_to_cpu(req->cmd->common.cdw11);
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
-	u16 status = NVME_SC_FEATURE_NOT_CHANGEABLE;
+	u16 status;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->rw.nsid);
-	if (unlikely(!req->ns)) {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return status = NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	status = nvmet_req_find_ns(req);
+	if (status)
+		return status;
 
 	mutex_lock(&subsys->lock);
 	switch (write_protect) {
@@ -799,11 +790,10 @@ static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
 	u32 result;
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, req->cmd->common.nsid);
-	if (!req->ns)  {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	result = nvmet_req_find_ns(req);
+	if (result)
+		return result;
+
 	mutex_lock(&subsys->lock);
 	if (req->ns->readonly == true)
 		result = NVME_NS_WRITE_PROTECT;
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 8ce4d59cc9e7..95b58d4b1af2 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -417,15 +417,18 @@ void nvmet_stop_keep_alive_timer(struct nvmet_ctrl *ctrl)
 	cancel_delayed_work_sync(&ctrl->ka_work);
 }
 
-struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl, __le32 nsid)
+u16 nvmet_req_find_ns(struct nvmet_req *req)
 {
-	struct nvmet_ns *ns;
+	u32 nsid = le32_to_cpu(req->cmd->common.nsid);
 
-	ns = xa_load(&ctrl->subsys->namespaces, le32_to_cpu(nsid));
-	if (ns)
-		percpu_ref_get(&ns->ref);
+	req->ns = xa_load(&req->sq->ctrl->subsys->namespaces, nsid);
+	if (unlikely(!req->ns)) {
+		req->error_loc = offsetof(struct nvme_common_command, nsid);
+		return NVME_SC_INVALID_NS | NVME_SC_DNR;
+	}
 
-	return ns;
+	percpu_ref_get(&req->ns->ref);
+	return NVME_SC_SUCCESS;
 }
 
 static void nvmet_destroy_namespace(struct percpu_ref *ref)
@@ -862,11 +865,10 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 	if (nvmet_req_passthru_ctrl(req))
 		return nvmet_parse_passthru_io_cmd(req);
 
-	req->ns = nvmet_find_namespace(req->sq->ctrl, cmd->rw.nsid);
-	if (unlikely(!req->ns)) {
-		req->error_loc = offsetof(struct nvme_common_command, nsid);
-		return NVME_SC_INVALID_NS | NVME_SC_DNR;
-	}
+	ret = nvmet_req_find_ns(req);
+	if (unlikely(ret))
+		return ret;
+
 	ret = nvmet_check_ana_state(req->port, req->ns);
 	if (unlikely(ret)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 8776dd1a0490..954b3d8451f5 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -443,7 +443,7 @@ struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
 void nvmet_subsys_put(struct nvmet_subsys *subsys);
 void nvmet_subsys_del_ctrls(struct nvmet_subsys *subsys);
 
-struct nvmet_ns *nvmet_find_namespace(struct nvmet_ctrl *ctrl, __le32 nsid);
+u16 nvmet_req_find_ns(struct nvmet_req *req);
 void nvmet_put_namespace(struct nvmet_ns *ns);
 int nvmet_ns_enable(struct nvmet_ns *ns);
 void nvmet_ns_disable(struct nvmet_ns *ns);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:55 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:55 -0800
Subject: [PATCH V3 04/11] nvmet: remove extra variable in id-ns handler
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-5-chaitanya.kulkarni@wdc.com>

In nvmet_execute_identify_ns() local variable ctrl is accessed only in
one place, remove that and directly use it from nvmet_req->sq->ctrl.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index f32533480e66..552da813da18 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -462,7 +462,6 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 
 static void nvmet_execute_identify_ns(struct nvmet_req *req)
 {
-	struct nvmet_ctrl *ctrl = req->sq->ctrl;
 	struct nvme_id_ns *id;
 	u16 status;
 
@@ -523,7 +522,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 
 	id->lbaf[0].ds = req->ns->blksize_shift;
 
-	if (ctrl->pi_support && nvmet_ns_has_pi(req->ns)) {
+	if (req->sq->ctrl->pi_support && nvmet_ns_has_pi(req->ns)) {
 		id->dpc = NVME_NS_DPC_PI_FIRST | NVME_NS_DPC_PI_LAST |
 			  NVME_NS_DPC_PI_TYPE1 | NVME_NS_DPC_PI_TYPE2 |
 			  NVME_NS_DPC_PI_TYPE3;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:56 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:56 -0800
Subject: [PATCH V3 05/11] nvmet: add helper to report invalid opcode
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-6-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Add an helper and use it in block device backend to keep the code
and error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-bdev.c | 5 +----
 drivers/nvme/target/nvmet.h       | 9 +++++++++
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c
index 23095bdfce06..105ef2b125cf 100644
--- a/drivers/nvme/target/io-cmd-bdev.c
+++ b/drivers/nvme/target/io-cmd-bdev.c
@@ -449,9 +449,6 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req)
 		req->execute = nvmet_bdev_execute_write_zeroes;
 		return 0;
 	default:
-		pr_err("unhandled cmd %d on qid %d\n", cmd->common.opcode,
-		       req->sq->qid);
-		req->error_loc = offsetof(struct nvme_common_command, opcode);
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 954b3d8451f5..6b5f1b60cf50 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -613,4 +613,13 @@ static inline sector_t nvmet_lba_to_sect(struct nvmet_ns *ns, __le64 lba)
 	return le64_to_cpu(lba) << (ns->blksize_shift - SECTOR_SHIFT);
 }
 
+static inline u16 nvmet_report_invalid_opcode(struct nvmet_req *req)
+{
+	pr_debug("unhandled cmd %d on qid %d\n", req->cmd->common.opcode,
+		 req->sq->qid);
+
+	req->error_loc = offsetof(struct nvme_common_command, opcode);
+	return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+}
+
 #endif /* _NVMET_H */
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:57 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:57 -0800
Subject: [PATCH V3 06/11] nvmet: use invalid cmd opcode helper
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-7-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Use the previously introduced helper in file backend to reduce the
duplicate code and make the error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/io-cmd-file.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c
index 0abbefd9925e..715d4376c997 100644
--- a/drivers/nvme/target/io-cmd-file.c
+++ b/drivers/nvme/target/io-cmd-file.c
@@ -400,9 +400,6 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req)
 		req->execute = nvmet_file_execute_write_zeroes;
 		return 0;
 	default:
-		pr_err("unhandled cmd for file ns %d on qid %d\n",
-				cmd->common.opcode, req->sq->qid);
-		req->error_loc = offsetof(struct nvme_common_command, opcode);
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:58 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:58 -0800
Subject: [PATCH V3 07/11] nvmet: use invalid cmd opcode helper
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-8-chaitanya.kulkarni@wdc.com>

In the NVMeOF block device backend, file backend, and passthru backend
we reject and report the commands if opcode is not handled.

Use the previously introduced helper in the passthru backend to make the
error message uniform.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/passthru.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index cbc88acdd233..3b22f4a868f4 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -494,7 +494,7 @@ u16 nvmet_parse_passthru_admin_cmd(struct nvmet_req *req)
 		return nvmet_setup_passthru_command(req);
 	default:
 		/* Reject commands not in the allowlist above */
-		return NVME_SC_INVALID_OPCODE | NVME_SC_DNR;
+		return nvmet_report_invalid_opcode(req);
 	}
 }
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:47:59 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:47:59 -0800
Subject: [PATCH V3 08/11] nvmet: use min of device_path and disk len
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-9-chaitanya.kulkarni@wdc.com>

In function __assign_req_name() instead of using the DEVICE_NAME_LEN in
strncpy() use min of DISK_NAME_LEN and strlen(req->ns->device_path).

This is needed to turn off the following warnings:-

In file included from drivers/nvme/target/core.c:14:
In function ?__assign_req_name?,
    inlined from ?trace_event_raw_event_nvmet_req_init? at drivers/nvme/target/./trace.h:58:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?perf_trace_nvmet_req_complete? at drivers/nvme/target/./trace.h:100:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?perf_trace_nvmet_req_init? at drivers/nvme/target/./trace.h:58:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ?__assign_req_name?,
    inlined from ?trace_event_raw_event_nvmet_req_complete? at drivers/nvme/target/./trace.h:100:1:
drivers/nvme/target/trace.h:52:3: warning: ?strncpy? specified bound 32 equals destination size [-Wstringop-truncation]
   strncpy(name, req->ns->device_path, DISK_NAME_LEN);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/trace.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/trace.h b/drivers/nvme/target/trace.h
index c14e3249a14d..958dca7bcfc7 100644
--- a/drivers/nvme/target/trace.h
+++ b/drivers/nvme/target/trace.h
@@ -48,8 +48,10 @@ static inline struct nvmet_ctrl *nvmet_req_to_ctrl(struct nvmet_req *req)
 
 static inline void __assign_req_name(char *name, struct nvmet_req *req)
 {
+	size_t len = min_t(size_t, DISK_NAME_LEN, strlen(req->ns->device_path));
+
 	if (req->ns)
-		strncpy(name, req->ns->device_path, DISK_NAME_LEN);
+		strncpy(name, req->ns->device_path, len);
 	else
 		memset(name, 0, DISK_NAME_LEN);
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:48:00 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:48:00 -0800
Subject: [PATCH V3 09/11] nvme-loop: rename variable to get rid of the warn
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-10-chaitanya.kulkarni@wdc.com>

Rename the nvme_loop_init_request() numa_node parameter to
hw_queue_numa_node so that we can get rid of the following warning :-

drivers/nvme/target/loop.c: In function ?nvme_loop_init_request?:
drivers/nvme/target/loop.c:205:16: warning: declaration of ?numa_node? shadows a global declaration [-Wshadow]
   unsigned int numa_node)

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/loop.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..88d0656ad4a5 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -202,7 +202,7 @@ static int nvme_loop_init_iod(struct nvme_loop_ctrl *ctrl,
 
 static int nvme_loop_init_request(struct blk_mq_tag_set *set,
 		struct request *req, unsigned int hctx_idx,
-		unsigned int numa_node)
+		unsigned int hw_queue_numa_node)
 {
 	struct nvme_loop_ctrl *ctrl = set->driver_data;
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:48:01 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:48:01 -0800
Subject: [PATCH V3 10/11] nvmet: add nvmet_req_subsys() helper
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-11-chaitanya.kulkarni@wdc.com>

Just like what we have to get the passthru ctrl from the req, add an
helper to get the subsystem associated with the nvmet_req() instead
of open coding the chain of structures.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 10 +++++-----
 drivers/nvme/target/core.c      |  2 +-
 drivers/nvme/target/nvmet.h     |  7 ++++++-
 drivers/nvme/target/passthru.c  |  4 ++--
 4 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 552da813da18..bc6a774f2124 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -683,7 +683,7 @@ static u16 nvmet_write_protect_flush_sync(struct nvmet_req *req)
 static u16 nvmet_set_feat_write_protect(struct nvmet_req *req)
 {
 	u32 write_protect = le32_to_cpu(req->cmd->common.cdw11);
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u16 status;
 
 	status = nvmet_req_find_ns(req);
@@ -742,7 +742,7 @@ u16 nvmet_set_feat_async_event(struct nvmet_req *req, u32 mask)
 
 void nvmet_execute_set_features(struct nvmet_req *req)
 {
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u32 cdw10 = le32_to_cpu(req->cmd->common.cdw10);
 	u32 cdw11 = le32_to_cpu(req->cmd->common.cdw11);
 	u16 status = 0;
@@ -786,7 +786,7 @@ void nvmet_execute_set_features(struct nvmet_req *req)
 
 static u16 nvmet_get_feat_write_protect(struct nvmet_req *req)
 {
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u32 result;
 
 	result = nvmet_req_find_ns(req);
@@ -816,7 +816,7 @@ void nvmet_get_feat_async_event(struct nvmet_req *req)
 
 void nvmet_execute_get_features(struct nvmet_req *req)
 {
-	struct nvmet_subsys *subsys = req->sq->ctrl->subsys;
+	struct nvmet_subsys *subsys = nvmet_req_subsys(req);
 	u32 cdw10 = le32_to_cpu(req->cmd->common.cdw10);
 	u16 status = 0;
 
@@ -923,7 +923,7 @@ u16 nvmet_parse_admin_cmd(struct nvmet_req *req)
 
 	if (nvme_is_fabrics(cmd))
 		return nvmet_parse_fabrics_cmd(req);
-	if (req->sq->ctrl->subsys->type == NVME_NQN_DISC)
+	if (nvmet_req_subsys(req)->type == NVME_NQN_DISC)
 		return nvmet_parse_discovery_cmd(req);
 
 	ret = nvmet_check_ctrl_status(req, cmd);
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 95b58d4b1af2..32221f98f1cc 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -421,7 +421,7 @@ u16 nvmet_req_find_ns(struct nvmet_req *req)
 {
 	u32 nsid = le32_to_cpu(req->cmd->common.nsid);
 
-	req->ns = xa_load(&req->sq->ctrl->subsys->namespaces, nsid);
+	req->ns = xa_load(&nvmet_req_subsys(req)->namespaces, nsid);
 	if (unlikely(!req->ns)) {
 		req->error_loc = offsetof(struct nvme_common_command, nsid);
 		return NVME_SC_INVALID_NS | NVME_SC_DNR;
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 6b5f1b60cf50..50535563e8c5 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -551,6 +551,11 @@ static inline u32 nvmet_dsm_len(struct nvmet_req *req)
 		sizeof(struct nvme_dsm_range);
 }
 
+static inline struct nvmet_subsys *nvmet_req_subsys(struct nvmet_req *req)
+{
+	return req->sq->ctrl->subsys;
+}
+
 #ifdef CONFIG_NVME_TARGET_PASSTHRU
 void nvmet_passthru_subsys_free(struct nvmet_subsys *subsys);
 int nvmet_passthru_ctrl_enable(struct nvmet_subsys *subsys);
@@ -585,7 +590,7 @@ static inline struct nvme_ctrl *nvmet_passthru_ctrl(struct nvmet_subsys *subsys)
 static inline struct nvme_ctrl *
 nvmet_req_passthru_ctrl(struct nvmet_req *req)
 {
-	return nvmet_passthru_ctrl(req->sq->ctrl->subsys);
+	return nvmet_passthru_ctrl(nvmet_req_subsys(req));
 }
 
 u16 errno_to_nvme_status(struct nvmet_req *req, int errno);
diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c
index 3b22f4a868f4..f50c7b2bf21c 100644
--- a/drivers/nvme/target/passthru.c
+++ b/drivers/nvme/target/passthru.c
@@ -239,9 +239,9 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req)
 		}
 
 		q = ns->queue;
-		timeout = req->sq->ctrl->subsys->io_timeout;
+		timeout = nvmet_req_subsys(req)->io_timeout;
 	} else {
-		timeout = req->sq->ctrl->subsys->admin_timeout;
+		timeout = nvmet_req_subsys(req)->admin_timeout;
 	}
 
 	rq = nvme_alloc_request(q, req->cmd, 0);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 10 00:48:02 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue,  9 Feb 2021 21:48:02 -0800
Subject: [PATCH V3 11/11] nvmet: remove else at the end of the function
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210054802.34269-12-chaitanya.kulkarni@wdc.com>

The function nvmet_parse_io_cmd() returns value from
nvmet_file_parse_io_cmd() or nvmet_bdev_parse_io_cmd() based on which
backend is set for the request. Remove the else and just return the
value from nvmet_bdev_parse_io_cmd().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 32221f98f1cc..1d5516272530 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -882,8 +882,8 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 
 	if (req->ns->file)
 		return nvmet_file_parse_io_cmd(req);
-	else
-		return nvmet_bdev_parse_io_cmd(req);
+
+	return nvmet_bdev_parse_io_cmd(req);
 }
 
 bool nvmet_req_init(struct nvmet_req *req, struct nvmet_cq *cq,
-- 
2.22.1



From hch at lst.de  Wed Feb 10 02:37:59 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 08:37:59 +0100
Subject: [PATCH v2] nvme: Add 48-bit DMA address quirk for Amazon NVMe
 controllers
In-Reply-To: <20210210003942.25700-1-sironi@amazon.de>
References: <20210203094338.19473-1-sironi@amazon.de>
 <20210210003942.25700-1-sironi@amazon.de>
Message-ID: <20210210073759.GA23269@lst.de>

On Wed, Feb 10, 2021 at 01:39:42AM +0100, Filippo Sironi wrote:
> Amazon NVMe controllers do not support 64-bit DMA addresses; they are
> limited to 48-bit DMA addresses.  Let's add a quirk to ensure that we
> make use of 48-bit DMA addresses to avoid misbehavior.

This should probably say some, and mention that they do not follow
the spec.  But I can fix this up when applying the patch.


From hch at lst.de  Wed Feb 10 03:11:34 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 09:11:34 +0100
Subject: [PATCHv2] nvme/hwmon: rework to avoid devm allocation
In-Reply-To: <20210119064318.124415-1-hare@suse.de>
References: <20210119064318.124415-1-hare@suse.de>
Message-ID: <20210210081134.GA24040@lst.de>

Thanks,

applied to nvme-5.12.


From hch at lst.de  Wed Feb 10 03:11:55 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 09:11:55 +0100
Subject: [PATCH V3 00/11] nvmet: fixes and some cleanups
In-Reply-To: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210081155.GB24040@lst.de>

I've applied this to nvme-5.12 with one exception and a few tweaks.
More in individual replies.


From hch at lst.de  Wed Feb 10 03:12:21 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 09:12:21 +0100
Subject: [PATCH V3 05/11] nvmet: add helper to report invalid opcode
In-Reply-To: <20210210054802.34269-6-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
 <20210210054802.34269-6-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210081221.GC24040@lst.de>

No real need to have this slow path function inline, so I've moved it
out of line.


From hch at lst.de  Wed Feb 10 03:13:06 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 09:13:06 +0100
Subject: [PATCH V3 08/11] nvmet: use min of device_path and disk len
In-Reply-To: <20210210054802.34269-9-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
 <20210210054802.34269-9-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210081306.GD24040@lst.de>

>  static inline void __assign_req_name(char *name, struct nvmet_req *req)
>  {
> +	size_t len = min_t(size_t, DISK_NAME_LEN, strlen(req->ns->device_path));
> +
>  	if (req->ns)
> -		strncpy(name, req->ns->device_path, DISK_NAME_LEN);
> +		strncpy(name, req->ns->device_path, len);
>  	else
>  		memset(name, 0, DISK_NAME_LEN);

This could dereference a NULL req->ns.  I've changed the patch
to eliminate the local variable and to return early for the NULL
req->ns case.


From hch at lst.de  Wed Feb 10 03:13:35 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 09:13:35 +0100
Subject: [PATCH V3 09/11] nvme-loop: rename variable to get rid of the warn
In-Reply-To: <20210210054802.34269-10-chaitanya.kulkarni@wdc.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
 <20210210054802.34269-10-chaitanya.kulkarni@wdc.com>
Message-ID: <20210210081335.GE24040@lst.de>

numa_node is a perfectly fine name for a local variable.  Please find
the global one and rename it instead.


From Chaitanya.Kulkarni at wdc.com  Wed Feb 10 03:18:16 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 10 Feb 2021 08:18:16 +0000
Subject: [PATCH V3 09/11] nvme-loop: rename variable to get rid of the warn
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
 <20210210054802.34269-10-chaitanya.kulkarni@wdc.com>
 <20210210081335.GE24040@lst.de>
Message-ID: <BYAPR04MB496597E9F3AE6D3D82306C0E868D9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/10/21 00:13, Christoph Hellwig wrote:
> numa_node is a perfectly fine name for a local variable.  Please find
> the global one and rename it instead.
>
I know, the warning is really annoying breaks the automated script for
clean compilation.


From hch at lst.de  Wed Feb 10 03:23:39 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 09:23:39 +0100
Subject: [PATCH V3 09/11] nvme-loop: rename variable to get rid of the warn
In-Reply-To: <BYAPR04MB496597E9F3AE6D3D82306C0E868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
 <20210210054802.34269-10-chaitanya.kulkarni@wdc.com>
 <20210210081335.GE24040@lst.de>
 <BYAPR04MB496597E9F3AE6D3D82306C0E868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210210082339.GA24330@lst.de>

On Wed, Feb 10, 2021 at 08:18:16AM +0000, Chaitanya Kulkarni wrote:
> On 2/10/21 00:13, Christoph Hellwig wrote:
> > numa_node is a perfectly fine name for a local variable.  Please find
> > the global one and rename it instead.
> >
> I know, the warning is really annoying breaks the automated script for
> clean compilation.

So fix the global variable, which is indeed very badly named for global
variable.


From Chaitanya.Kulkarni at wdc.com  Wed Feb 10 03:52:15 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 10 Feb 2021 08:52:15 +0000
Subject: [PATCH V3 00/11] nvmet: fixes and some cleanups
References: <20210210054802.34269-1-chaitanya.kulkarni@wdc.com>
 <20210210081155.GB24040@lst.de>
Message-ID: <BYAPR04MB4965C6B3B16E9919691B6B4D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/10/21 00:12, Christoph Hellwig wrote:
> I've applied this to nvme-5.12 with one exception and a few tweaks.
> More in individual replies.
>
I've pulled the changes from nvme-5.12 and ran blocktests. It seems to
work fine.

# gitlog -12
26bb85e126d0 (HEAD -> nvme-5.12) Merge branch 'nvme-5.12' of
git://git.infradead.org/nvme into nvme-5.12
523bb6d79f0e (origin/nvme-5.12) nvme-hwmon: rework to avoid devm allocation
4da2cba48943 nvmet: remove else at the end of the function
7684ff0eac09 nvmet: add nvmet_req_subsys() helper
d93178da21dd nvmet: use min of device_path and disk len
ec3dc8c50add nvmet: use invalid cmd opcode helper
ff887268ac4e nvmet: use invalid cmd opcode helper
022b300c9227 nvmet: add helper to report invalid opcode
1be5d3ef5976 nvmet: remove extra variable in id-ns handler
214bffc29d79 nvmet: make nvmet_find_namespace() req based
617e7b18cc7e nvmet: return uniform error for invalid ns
275f63fa4ddb nvmet: set status to 0 in case for invalid nsid
#

# ./check tests/nvme
nvme/002 (create many subsystems and test discovery)         [passed]
    runtime  19.845s  ...  28.439s
nvme/003 (test if we're sending keep-alives to a discovery controller)
[passed]
    runtime    ...  10.131s
nvme/004 (test nvme and nvmet UUID NS descriptors)           [passed]
    runtime  1.595s  ...  1.704s
nvme/005 (reset local loopback target)                       [not run]
    nvme_core module does not have parameter multipath
nvme/006 (create an NVMeOF target with a block device-backed ns) [passed]
    runtime  0.078s  ...  0.112s
nvme/007 (create an NVMeOF target with a file-backed ns)     [passed]
    runtime  0.073s  ...  0.073s
nvme/008 (create an NVMeOF host with a block device-backed ns) [passed]
    runtime  1.619s  ...  1.707s
nvme/009 (create an NVMeOF host with a file-backed ns)       [passed]
    runtime    ...  1.663s
nvme/010 (run data verification fio job on NVMeOF block device-backed
ns) [passed]
    runtime  24.800s  ...  26.836s
nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  262.535s  ...  284.324s
nvme/012 (run mkfs and data verification fio job on NVMeOF block
device-backed ns) [passed]
    runtime  46.843s  ...  45.088s
nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed
ns) [passed]
    runtime  280.317s  ...  324.476s
nvme/014 (flush a NVMeOF block device-backed ns)             [passed]
    runtime  21.985s  ...  21.937s
nvme/015 (unit test for NVMe flush for file backed ns)       [passed]
    runtime  22.981s  ...  22.106s
nvme/016 (create/delete many NVMeOF block device-backed ns and test
discovery) [passed]
    runtime  15.777s  ...  16.286s
nvme/017 (create/delete many file-ns and test discovery)     [passed]
    runtime  15.995s  ...  16.244s
nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed]
    runtime  1.650s  ...  1.664s
nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed]
    runtime  1.677s  ...  1.726s
nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed]
    runtime  1.660s  ...  1.668s
nvme/021 (test NVMe list command on NVMeOF file-backed ns)   [passed]
    runtime  1.652s  ...  1.669s
nvme/022 (test NVMe reset command on NVMeOF file-backed ns)  [passed]
    runtime  2.087s  ...  2.098s
nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed]
    runtime  1.697s  ...  1.735s
nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed]
    runtime  1.794s  ...  1.682s
nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed]
    runtime  1.658s  ...  1.655s
nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed]
    runtime  1.651s  ...  1.694s
nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed]
    runtime  1.654s  ...  1.763s
nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed]
    runtime  1.659s  ...  1.654s
nvme/029 (test userspace IO via nvme-cli read/write interface) [passed]
    runtime  2.007s  ...  2.051s
nvme/030 (ensure the discovery generation counter is updated
appropriately) [passed]
    runtime  0.321s  ...  0.316s
nvme/031 (test deletion of NVMeOF controllers immediately after setup)
[passed]
    runtime  5.528s  ...  5.587s
nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed]
    runtime  0.046s  ...  0.044s


From sironi at amazon.de  Wed Feb 10 04:13:17 2021
From: sironi at amazon.de (Filippo Sironi)
Date: Wed, 10 Feb 2021 10:13:17 +0100
Subject: [PATCH v2] nvme: Add 48-bit DMA address quirk for Amazon NVMe
 controllers
In-Reply-To: <20210210073759.GA23269@lst.de>
References: <20210203094338.19473-1-sironi@amazon.de>
 <20210210003942.25700-1-sironi@amazon.de> <20210210073759.GA23269@lst.de>
Message-ID: <8a375a50-1aad-96c5-2743-f91120fbc6f9@amazon.de>

On 2/10/21 8:37 AM, Christoph Hellwig wrote:
> 
> On Wed, Feb 10, 2021 at 01:39:42AM +0100, Filippo Sironi wrote:
>> Amazon NVMe controllers do not support 64-bit DMA addresses; they are
>> limited to 48-bit DMA addresses.  Let's add a quirk to ensure that we
>> make use of 48-bit DMA addresses to avoid misbehavior.
> 
> This should probably say some, and mention that they do not follow
> the spec.  But I can fix this up when applying the patch.
> 

Thanks!

Filippo



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879



From hch at lst.de  Wed Feb 10 05:10:37 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 11:10:37 +0100
Subject: [PATCH 1/2] nvmet: don't check iosqes,iocqes for discovery
 controllers
In-Reply-To: <20210209101235.1539917-1-sagi@grimberg.me>
References: <20210209101235.1539917-1-sagi@grimberg.me>
Message-ID: <20210210101037.GA26576@lst.de>

On Tue, Feb 09, 2021 at 02:12:34AM -0800, Sagi Grimberg wrote:
> +	if (!ctrl->subsys->type == NVME_NQN_DISC &&

I think the normal way to write this would be a !=.

> +	    (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
> +	     nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES)) {

By the spec we'd need to check these are 0, but of course we'd break
the existing Linux host.  Maybe throw in a comment?


From hch at lst.de  Wed Feb 10 05:11:26 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 11:11:26 +0100
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery
 controllers
In-Reply-To: <20210209101235.1539917-2-sagi@grimberg.me>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210209101235.1539917-2-sagi@grimberg.me>
Message-ID: <20210210101126.GA26624@lst.de>

So if we change this we break interaction with all old Linux (and maybe
other?) targets.


From hch at infradead.org  Wed Feb 10 05:12:29 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 10 Feb 2021 11:12:29 +0100
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write
 Zeroes command
In-Reply-To: <20210209045902.72345-1-gtiwari@redhat.com>
References: <20210209045902.72345-1-gtiwari@redhat.com>
Message-ID: <YCOxjZhBTBchOq0n@infradead.org>

On Tue, Feb 09, 2021 at 10:29:02AM +0530, Gopal Tiwari wrote:
> For SK Hynix PE8000 WriteZerocommands exceed typical I/O MDTS
> (Maximum Data Transfer Size) supported by the hardware. Which
> intern fail and generate following erros :

Well, if it works under MDTS please add a new quirk that follows
MDTS for Write Zeroes.  In fact I wonder if we should just do
that by default to be on the safe side.


From gtiwari at redhat.com  Wed Feb 10 06:17:23 2021
From: gtiwari at redhat.com (Gopal Tiwari)
Date: Wed, 10 Feb 2021 06:17:23 -0500 (EST)
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write
 Zeroes command
In-Reply-To: <YCOxjZhBTBchOq0n@infradead.org>
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <YCOxjZhBTBchOq0n@infradead.org>
Message-ID: <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>

Hi Christoph, 

Thanks for the response,

----- Original Message -----
From: "Christoph Hellwig" <hch at infradead.org>
To: "Gopal Tiwari" <gtiwari at redhat.com>
Cc: kbusch at kernel.org, hch at lst.de, linux-nvme at lists.infradead.org
Sent: Wednesday, February 10, 2021 3:42:29 PM
Subject: Re: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes command

On Tue, Feb 09, 2021 at 10:29:02AM +0530, Gopal Tiwari wrote:
>> For SK Hynix PE8000 WriteZerocommands exceed typical I/O MDTS
>> (Maximum Data Transfer Size) supported by the hardware. Which
>> intern fail and generate following erros :

>Well, if it works under MDTS please add a new quirk that follows
MDTS for Write Zeroes.

This was the feedback from the hardware guys, We've not tried, as the safe option would be to disable 
write zero's as spec says 

I could see from the identify controller details 

  [0:0] : 0	Single Port

mdts      : 7 <<---------- 
cntlid    : 0
ver       : 0x10300

"This field does not apply to commands that do not transfer
data between host-accessible memory and the controller (e.g., the Verify command,
the Write Uncorrectable command, and the Write Zeroes command); there is no
maximum data transfer size for those commands. "

>  In fact I wonder if we should just do that by default to be on the safe side.
Can you please help me in understanding this ..  

Thanks & regards,
Gopal Tiwari 

_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme



From minwoo.im.dev at gmail.com  Wed Feb 10 07:10:06 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Wed, 10 Feb 2021 21:10:06 +0900
Subject: [PATCHv2] nvme/hwmon: rework to avoid devm allocation
In-Reply-To: <20210119064318.124415-1-hare@suse.de>
References: <20210119064318.124415-1-hare@suse.de>
Message-ID: <20210210121006.GA2325@localhost.localdomain>

On 21-01-19 07:43:18, Hannes Reinecke wrote:
> The original design to use device-managed resource allocation
> doesn't really work as the NVMe controller has a vastly different
> lifetime than the hwmon sysfs attributes, causing warning about
> duplicate sysfs entries upon reconnection.
> This patch reworks the hwmon allocation to avoid device-managed
> resource allocation, and uses the NVMe controller as parent for
> the sysfs attributes.
> 
> Cc: Guenter Roeck <linux at roeck-us.net>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> Tested-by: Daniel Wagner <dwagner at suse.de>
> ---
>  drivers/nvme/host/core.c  |  1 +
>  drivers/nvme/host/hwmon.c | 31 +++++++++++++++++++++----------
>  drivers/nvme/host/nvme.h  |  6 ++++++
>  3 files changed, 28 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index fff49e544fdf..3c6c77e44bf7 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -4419,6 +4419,7 @@ EXPORT_SYMBOL_GPL(nvme_start_ctrl);
>  
>  void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
>  {
> +	nvme_hwmon_exit(ctrl);
>  	nvme_fault_inject_fini(&ctrl->fault_inject);
>  	dev_pm_qos_hide_latency_tolerance(ctrl->device);
>  	cdev_device_del(&ctrl->cdev, ctrl->device);
> diff --git a/drivers/nvme/host/hwmon.c b/drivers/nvme/host/hwmon.c
> index 552dbc04567b..8f9e96986780 100644
> --- a/drivers/nvme/host/hwmon.c
> +++ b/drivers/nvme/host/hwmon.c
> @@ -223,12 +223,12 @@ static const struct hwmon_chip_info nvme_hwmon_chip_info = {
>  
>  int nvme_hwmon_init(struct nvme_ctrl *ctrl)
>  {
> -	struct device *dev = ctrl->dev;
> +	struct device *dev = ctrl->device;
>  	struct nvme_hwmon_data *data;
>  	struct device *hwmon;
>  	int err;
>  
> -	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
> +	data = kzalloc(sizeof(*data), GFP_KERNEL);
>  	if (!data)
>  		return 0;
>  
> @@ -237,19 +237,30 @@ int nvme_hwmon_init(struct nvme_ctrl *ctrl)
>  
>  	err = nvme_hwmon_get_smart_log(data);
>  	if (err) {
> -		dev_warn(ctrl->device,
> -			"Failed to read smart log (error %d)\n", err);
> -		devm_kfree(dev, data);
> +		dev_warn(dev, "Failed to read smart log (error %d)\n", err);
> +		kfree(data);
>  		return err;
>  	}
>  
> -	hwmon = devm_hwmon_device_register_with_info(dev, "nvme", data,
> -						     &nvme_hwmon_chip_info,
> -						     NULL);
> +	hwmon = hwmon_device_register_with_info(dev, "nvme",
> +						data, &nvme_hwmon_chip_info,
> +						NULL);
>  	if (IS_ERR(hwmon)) {
>  		dev_warn(dev, "Failed to instantiate hwmon device\n");
> -		devm_kfree(dev, data);
> +		kfree(data);
>  	}
> -
> +	ctrl->hwmon_device = hwmon;
>  	return 0;
>  }
> +
> +void nvme_hwmon_exit(struct nvme_ctrl *ctrl)
> +{
> +	if (ctrl->hwmon_device) {
> +		struct nvme_hwmon_data *data =
> +			dev_get_drvdata(ctrl->hwmon_device);
> +
> +		hwmon_device_unregister(ctrl->hwmon_device);
> +		ctrl->hwmon_device = NULL;
> +		kfree(data);
> +	}
> +}
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 88a6b97247f5..f51b942bb4f5 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -246,6 +246,9 @@ struct nvme_ctrl {
>  	struct rw_semaphore namespaces_rwsem;
>  	struct device ctrl_device;
>  	struct device *device;	/* char device */
> +#ifdef CONFIG_NVME_HWMON
> +	struct device *hwmon_device;
> +#endif
>  	struct cdev cdev;
>  	struct work_struct reset_work;
>  	struct work_struct delete_work;
> @@ -809,11 +812,14 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
>  
>  #ifdef CONFIG_NVME_HWMON
>  int nvme_hwmon_init(struct nvme_ctrl *ctrl);
> +void nvme_hwmon_exit(struct nvme_ctrl *ctrl);
>  #else
>  static inline int nvme_hwmon_init(struct nvme_ctrl *ctrl)
>  {
>  	return 0;
>  }
> +
> +static inline int nvme_hwmon_exit(struct nvme_ctrl *ctrl) {}

Currently, nvme-5.12 has build errors:

rivers/nvme/target/../host/nvme.h: In function ?nvme_hwmon_exit?:
drivers/nvme/target/../host/nvme.h:825:42: error: no return statement in function returning non-void [-Werror=return-type]
 static inline int nvme_hwmon_exit(struct nvme_ctrl *ctrl) {}
                                          ^~~~~~~~~


From hch at lst.de  Wed Feb 10 08:11:51 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 14:11:51 +0100
Subject: [PATCHv2] nvme/hwmon: rework to avoid devm allocation
In-Reply-To: <20210210121006.GA2325@localhost.localdomain>
References: <20210119064318.124415-1-hare@suse.de>
 <20210210121006.GA2325@localhost.localdomain>
Message-ID: <20210210131151.GA29904@lst.de>

I've fixed the !CONFIG_NVME_HWMON path up, thanks.


From hch at lst.de  Wed Feb 10 08:15:57 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 10 Feb 2021 14:15:57 +0100
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write
 Zeroes command
In-Reply-To: <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <YCOxjZhBTBchOq0n@infradead.org>
 <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
Message-ID: <20210210131557.GB29904@lst.de>

MDTS vs Write Zeroes is a common misconception.  We had that discussion
in the NVMe working group, and while the text should pretty clear that
MDTS only applies to data transfers many implementators did not understand
that, which is why we added the clarification you quoted.

Also the next NVMe spec will allow devices to advertise an explicit
Write Zeroes limit, take a look at TP4040 from the "NVM Express 1.4 Ratified
TPs" at https://nvmexpress.org/developers/nvme-specification/.

So I think limiting to MDTS unless the new WZSL is set might be a fail
safe option, even if it is more pessimistic than what the spec says.

I'd also love to retest most Write Zeroes quirks with that in place.


From axboe at kernel.dk  Wed Feb 10 09:45:12 2021
From: axboe at kernel.dk (Jens Axboe)
Date: Wed, 10 Feb 2021 07:45:12 -0700
Subject: [PATCH v4 0/8] block: add zone write granularity limit
In-Reply-To: <20210128044733.503606-1-damien.lemoal@wdc.com>
References: <20210128044733.503606-1-damien.lemoal@wdc.com>
Message-ID: <cfe15751-e07e-101f-49d8-a2f184e30618@kernel.dk>

On 1/27/21 9:47 PM, Damien Le Moal wrote:
> The first patch of this series adds missing documentation of the
> zone_append_max_bytes sysfs attribute.
> 
> The following 3 patches are cleanup and preparatory patches for the
> introduction of the zone write granularity limit. The goal of these
> patches is to have all code setting a device queue zoned model to use
> the helper function blk_queue_set_zoned(). The nvme driver, null_blk
> driver and the partition code are modified to do so.
> 
> The fourth patch in this series introduces the zone write granularity
> queue limit to indicate the alignment constraint for write operations
> into sequential zones of zoned block devices. This limit is always set
> by default to the device logical block size. The following patch
> documents this new limit.
> 
> The last 2 patches introduce the blk_queue_clear_zone_settings()
> function and modify the SCSI sd driver to clear the zone related queue
> limits and resources of a host-aware zoned disk that is changed to a
> regular disk due to the presence of partitions.

Applied, thanks.

-- 
Jens Axboe



From Jeff.Lien at wdc.com  Wed Feb 10 13:21:17 2021
From: Jeff.Lien at wdc.com (Jeffrey Lien)
Date: Wed, 10 Feb 2021 18:21:17 +0000
Subject: NVMe CLI Commit - Input expects device name without /dev
Message-ID: <BYAPR04MB5656896D86A830305A2AFE31EA8D9@BYAPR04MB5656.namprd04.prod.outlook.com>

Eric,
The subject commit (ID dcfe0156360deaefa26b1d00053a791eefe397d1) is causing the following errors with the Linux 5.9 kernel:
Failed to open /sys/class/nvme-subsystem/nvme-subsys0/nvme0/hostnqn: No such file or directory
Failed to open /sys/class/nvme-subsystem/nvme-subsys0/nvme0/hostid: No such file or directory
Failed to open /sys/class/nvme-subsystem/nvme-subsys1/nvme1/hostnqn: No such file or directory
Failed to open /sys/class/nvme-subsystem/nvme-subsys1/nvme1/hostid: No such file or directory
Failed to open /sys/class/nvme-subsystem/nvme-subsys2/nvme2/hostnqn: No such file or directory
Failed to open /sys/class/nvme-subsystem/nvme-subsys2/nvme2/hostid: No such file or directory 

I'm seeing them on the  format and nvme list command.  They don't seem to impact function but do generate questions.  Do know of a way to resolve?

Thanks

Jeff Lien



From ericcurtin17 at gmail.com  Wed Feb 10 13:35:56 2021
From: ericcurtin17 at gmail.com (Eric Curtin)
Date: Wed, 10 Feb 2021 18:35:56 +0000
Subject: NVMe CLI Commit - Input expects device name without /dev
In-Reply-To: <BYAPR04MB5656896D86A830305A2AFE31EA8D9@BYAPR04MB5656.namprd04.prod.outlook.com>
References: <BYAPR04MB5656896D86A830305A2AFE31EA8D9@BYAPR04MB5656.namprd04.prod.outlook.com>
Message-ID: <CANpvso5nv0LO6XwAVtj8Lc=4HbxUjjNnTmj9T7ap5F=3mccKoA@mail.gmail.com>

Sorry yes, it made sense to print this error for this case:

https://github.com/linux-nvme/nvme-cli/issues/879

But not for others, I'll remove that fprintf line. Here's the fix:

https://github.com/linux-nvme/nvme-cli/pull/905

Apologies for any inconvenience caused.


On Wed, 10 Feb 2021 at 18:21, Jeffrey Lien <Jeff.Lien at wdc.com> wrote:
>
> Eric,
> The subject commit (ID dcfe0156360deaefa26b1d00053a791eefe397d1) is causing the following errors with the Linux 5.9 kernel:
> Failed to open /sys/class/nvme-subsystem/nvme-subsys0/nvme0/hostnqn: No such file or directory
> Failed to open /sys/class/nvme-subsystem/nvme-subsys0/nvme0/hostid: No such file or directory
> Failed to open /sys/class/nvme-subsystem/nvme-subsys1/nvme1/hostnqn: No such file or directory
> Failed to open /sys/class/nvme-subsystem/nvme-subsys1/nvme1/hostid: No such file or directory
> Failed to open /sys/class/nvme-subsystem/nvme-subsys2/nvme2/hostnqn: No such file or directory
> Failed to open /sys/class/nvme-subsystem/nvme-subsys2/nvme2/hostid: No such file or directory
>
> I'm seeing them on the  format and nvme list command.  They don't seem to impact function but do generate questions.  Do know of a way to resolve?
>
> Thanks
>
> Jeff Lien
>


From sagi at grimberg.me  Wed Feb 10 17:04:00 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 10 Feb 2021 14:04:00 -0800
Subject: [PATCH] nvme-tcp: fix crash triggered with a dataless request
 submission
Message-ID: <20210210220400.1730263-1-sagi@grimberg.me>

write-zeros has a bio, but does not have any data buffers associated
with it. Hence should not initialize the request iter for it (which
attempts to reference the bi_io_vec (and crash).
--
 run blktests nvme/012 at 2021-02-05 21:53:34
 BUG: kernel NULL pointer dereference, address: 0000000000000008
 #PF: supervisor read access in kernel mode
 #PF: error_code(0x0000) - not-present page
 PGD 0 P4D 0
 Oops: 0000 [#1] SMP NOPTI
 CPU: 15 PID: 12069 Comm: kworker/15:2H Tainted: G S        I       5.11.0-rc6+ #1
 Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
 Workqueue: kblockd blk_mq_run_work_fn
 RIP: 0010:nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
 RSP: 0018:ffffbd084447bd18 EFLAGS: 00010246
 RAX: 0000000000000000 RBX: ffffa0bba9f3ce80 RCX: 0000000000000000
 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000002000000
 RBP: ffffa0ba8ac6fec0 R08: 0000000002000000 R09: 0000000000000000
 R10: 0000000002800809 R11: 0000000000000000 R12: 0000000000000000
 R13: ffffa0bba9f3cf90 R14: 0000000000000000 R15: 0000000000000000
 FS:  0000000000000000(0000) GS:ffffa0c9ff9c0000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 0000000000000008 CR3: 00000001c9c6c005 CR4: 00000000007706e0
 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
 PKRU: 55555554
 Call Trace:
  nvme_tcp_queue_rq+0xef/0x330 [nvme_tcp]
  blk_mq_dispatch_rq_list+0x11c/0x7c0
  ? blk_mq_flush_busy_ctxs+0xf6/0x110
  __blk_mq_sched_dispatch_requests+0x12b/0x170
  blk_mq_sched_dispatch_requests+0x30/0x60
  __blk_mq_run_hw_queue+0x2b/0x60
  process_one_work+0x1cb/0x360
  ? process_one_work+0x360/0x360
  worker_thread+0x30/0x370
  ? process_one_work+0x360/0x360
  kthread+0x116/0x130
  ? kthread_park+0x80/0x80
  ret_from_fork+0x1f/0x30
--

Fixes: cb9b870fba3e ("nvme-tcp: fix wrong setting of request iov_iter")
Reported-by: Yi Zhang <yi.zhang at redhat.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/host/tcp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 619b0d8f6e38..69f59d2c5799 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2271,7 +2271,7 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns,
 	req->data_len = blk_rq_nr_phys_segments(rq) ?
 				blk_rq_payload_bytes(rq) : 0;
 	req->curr_bio = rq->bio;
-	if (req->curr_bio)
+	if (req->curr_bio && req->data_len)
 		nvme_tcp_init_iter(req, rq_data_dir(rq));
 
 	if (rq_data_dir(rq) == WRITE &&
-- 
2.27.0



From sagi at grimberg.me  Wed Feb 10 17:06:31 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 10 Feb 2021 14:06:31 -0800
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
In-Reply-To: <BYAPR04MB49652279710E99EBB6F8AC67868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
 <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>
 <6ae16841-5f51-617a-aab7-666b7eed299c@grimberg.me>
 <1a520912-ac7c-1a3c-c432-b382a5da6177@redhat.com>
 <BYAPR04MB49652279710E99EBB6F8AC67868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <502e8d85-90cf-9533-0f83-364bee2fd34b@grimberg.me>


>> So it's nvme_admin_abort_cmd here
>>
>> [   74.017450] run blktests nvme/012 at 2021-02-09 21:41:55
>> [   74.111311] loop: module loaded
>> [   74.125717] loop0: detected capacity change from 2097152 to 0
>> [   74.141026] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
>> [   74.149395] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
>> [   74.158298] nvmet: creating controller 1 for subsystem
>> blktests-subsystem-1 for NQN
>> nqn.2014-08.org.nvmexpress:uuid:41131d88-02ca-4ccc-87b3-6ca3f28b13a4.
>> [   74.158742] nvme nvme0: creating 48 I/O queues.
>> [   74.163391] nvme nvme0: mapped 48/0/0 default/read/poll queues.
>> [   74.184623] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr
>> 127.0.0.1:4420
>> [   75.235059] nvme_tcp: rq 38 opcode 8
>> [   75.238653] blk_update_request: I/O error, dev nvme0c0n1, sector
>> 1048624 op 0x9:(WRITE_ZEROES) flags 0x2800800 phys_seg 0 prio class 0
>> [   75.380179] XFS (nvme0n1): Mounting V5 Filesystem
>> [   75.387457] XFS (nvme0n1): Ending clean mount
>> [   75.388555] xfs filesystem being mounted at /mnt/blktests supports
>> timestamps until 2038 (0x7fffffff)
>> [   91.035659] XFS (nvme0n1): Unmounting Filesystem
>> [   91.043334] nvme nvme0: Removing ctrl: NQN "blktests-subsystem-1"
> 
> But write-zeores is also data less command and should not fail.

And it has a bio, which means that nvme-tcp tries to init an iter
for it when it shouldn't. So the actual offending commit is:
cb9b870fba3e, which cleaned up how the iter is initialized but 
introduced this issue.


From Chaitanya.Kulkarni at wdc.com  Wed Feb 10 17:11:15 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 10 Feb 2021 22:11:15 +0000
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes
 command
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <YCOxjZhBTBchOq0n@infradead.org>
 <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
 <20210210131557.GB29904@lst.de>
Message-ID: <BYAPR04MB49650A271CD1C1B3EFA72F3D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/10/21 5:20 AM, Christoph Hellwig wrote:
> MDTS vs Write Zeroes is a common misconception.  We had that discussion
> in the NVMe working group, and while the text should pretty clear that
> MDTS only applies to data transfers many implementators did not understand
> that, which is why we added the clarification you quoted.
>
> Also the next NVMe spec will allow devices to advertise an explicit
> Write Zeroes limit, take a look at TP4040 from the "NVM Express 1.4 Ratified
> TPs" at https://nvmexpress.org/developers/nvme-specification/.
>
> So I think limiting to MDTS unless the new WZSL is set might be a fail
> safe option, even if it is more pessimistic than what the spec says.
>
> I'd also love to retest most Write Zeroes quirks with that in place.

Do you prefer some variant of the following patch (totally untested)? OR
something else ?

Also, I think we should gradually roll out the MDTS replacement and
still keep the
write-zeroes quirk unless you are okay to just replace at one go.

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..d5ca27608881 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1167,7 +1167,8 @@ static int nvme_submit_user_cmd(struct
request_queue *q,
                if (ret)
                        goto out;
                bio = req->bio;
-               bio_set_dev(bio, bdev);
+               if(bdev)
+                       bio_set_dev(bio, bdev);
                if (bdev && meta_buffer && meta_len) {
                        meta = nvme_add_user_metadata(bio, meta_buffer,
meta_len,
                                        meta_seed, write);
@@ -1967,6 +1968,7 @@ static void nvme_config_write_zeroes(struct
gendisk *disk, struct nvme_ns *ns)
        u64 max_blocks;
 
        if (!(ns->ctrl->oncs & NVME_CTRL_ONCS_WRITE_ZEROES) ||
+           !(ns->ctrl->quirks & NVME_QUIRK_USE_MDTS_WRITE_ZEROES) ||
            (ns->ctrl->quirks & NVME_QUIRK_DISABLE_WRITE_ZEROES))
                return;
        /*
@@ -1979,10 +1981,15 @@ static void nvme_config_write_zeroes(struct
gendisk *disk, struct nvme_ns *ns)
         * configured based on the controller's MDTS field in the
         * nvme_init_identify() if available.
         */
-       if (ns->ctrl->max_hw_sectors == UINT_MAX)
-               max_blocks = (u64)USHRT_MAX + 1;
-       else
-               max_blocks = ns->ctrl->max_hw_sectors + 1;
+
+       if (!(ns->ctrl->quirks & NVME_QUIRK_USE_MDTS_WRITE_ZEROES)) {
+               if (ns->ctrl->max_hw_sectors == UINT_MAX)
+                       max_blocks = (u64)USHRT_MAX + 1;
+               else
+                       max_blocks = ns->ctrl->max_hw_sectors + 1;
+       } else {
+               max_blocks = ns->ctrl->max_mdts_hw_sectors;
+       }
 
        blk_queue_max_write_zeroes_sectors(disk->queue,
                                           nvme_lba_to_sect(ns,
max_blocks));
@@ -3141,6 +3148,11 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
        ctrl->max_hw_sectors =
                min_not_zero(ctrl->max_hw_sectors, max_hw_sectors);
 
+       /* explicitely set the mdts value to 0 */
+       ctrl->max_mdts_hw_sectors = 0;
+       if (id->mdts)
+               ctrl->max_mdts_hw_sectors =  1 << (id->mdts + page_shift
- 9);
+
        nvme_set_queue_limits(ctrl, ctrl->admin_q);
        ctrl->sgls = le32_to_cpu(id->sgls);
        ctrl->kas = le16_to_cpu(id->kas);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 0a73e78f09c8..7b60b2196d23 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -144,6 +144,11 @@ enum nvme_quirks {
         * NVMe 1.3 compliance.
         */
        NVME_QUIRK_NO_NS_DESC_LIST              = (1 << 15),
+       /*
+        * The controller doesn't allow the write zeores command of size
+        * more than reported by the MDTS
+        */
+       NVME_QUIRK_USE_MDTS_WRITE_ZEROES        = (1 << 16)
 };
 
 /*
@@ -268,6 +273,7 @@ struct nvme_ctrl {
 
        u64 cap;
        u32 max_hw_sectors;
+       u32 max_mdts_hw_sectors;
        u32 max_segments;
        u32 max_integrity_segments;
 #ifdef CONFIG_BLK_DEV_ZONED





From Chaitanya.Kulkarni at wdc.com  Wed Feb 10 17:15:36 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 10 Feb 2021 22:15:36 +0000
Subject: kernel null pointer at nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
References: <cki.F3E139361A.EN5MUSJKK9@redhat.com>
 <630237787.11660686.1612580898410.JavaMail.zimbra@redhat.com>
 <e1d08160-ca49-91e2-dafc-3ee80516842d@grimberg.me>
 <5848858e-239d-acb2-fa24-c371a3360557@redhat.com>
 <a88fd4c8-8d5c-6934-39bc-5c864e3ed84f@grimberg.me>
 <aec13f12-640c-77d5-bbdd-b4a3e18f1bf2@redhat.com>
 <6ae16841-5f51-617a-aab7-666b7eed299c@grimberg.me>
 <1a520912-ac7c-1a3c-c432-b382a5da6177@redhat.com>
 <BYAPR04MB49652279710E99EBB6F8AC67868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <502e8d85-90cf-9533-0f83-364bee2fd34b@grimberg.me>
Message-ID: <BYAPR04MB496552DEB31A02A06A3DEE0A868D9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/10/21 2:06 PM, Sagi Grimberg wrote:
>>> [   75.235059] nvme_tcp: rq 38 opcode 8
>>> [   75.238653] blk_update_request: I/O error, dev nvme0c0n1, sector
>>> 1048624 op 0x9:(WRITE_ZEROES) flags 0x2800800 phys_seg 0 prio class 0
>>> [   75.380179] XFS (nvme0n1): Mounting V5 Filesystem
>>> [   75.387457] XFS (nvme0n1): Ending clean mount
>>> [   75.388555] xfs filesystem being mounted at /mnt/blktests supports
>>> timestamps until 2038 (0x7fffffff)
>>> [   91.035659] XFS (nvme0n1): Unmounting Filesystem
>>> [   91.043334] nvme nvme0: Removing ctrl: NQN "blktests-subsystem-1"
>> But write-zeores is also data less command and should not fail.
> And it has a bio, which means that nvme-tcp tries to init an iter
> for it when it shouldn't. So the actual offending commit is:
> cb9b870fba3e, which cleaned up how the iter is initialized but 
> introduced this issue.
>
Looking at cb9b870fba3e, that should work, but really surprised that it
never
got triggered even once in my testing since the issue is reported.


From Chaitanya.Kulkarni at wdc.com  Wed Feb 10 17:16:39 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 10 Feb 2021 22:16:39 +0000
Subject: [PATCH] nvme-tcp: fix crash triggered with a dataless request
 submission
References: <20210210220400.1730263-1-sagi@grimberg.me>
Message-ID: <BYAPR04MB49650F2CB6A009986E0DC517868D9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/10/21 2:10 PM, Sagi Grimberg wrote:
> write-zeros has a bio, but does not have any data buffers associated
> with it. Hence should not initialize the request iter for it (which
> attempts to reference the bi_io_vec (and crash).
> --
>  run blktests nvme/012 at 2021-02-05 21:53:34
>  BUG: kernel NULL pointer dereference, address: 0000000000000008
>  #PF: supervisor read access in kernel mode
>  #PF: error_code(0x0000) - not-present page
>  PGD 0 P4D 0
>  Oops: 0000 [#1] SMP NOPTI
>  CPU: 15 PID: 12069 Comm: kworker/15:2H Tainted: G S        I       5.11.0-rc6+ #1
>  Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
>  Workqueue: kblockd blk_mq_run_work_fn
>  RIP: 0010:nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
>  RSP: 0018:ffffbd084447bd18 EFLAGS: 00010246
>  RAX: 0000000000000000 RBX: ffffa0bba9f3ce80 RCX: 0000000000000000
>  RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000002000000
>  RBP: ffffa0ba8ac6fec0 R08: 0000000002000000 R09: 0000000000000000
>  R10: 0000000002800809 R11: 0000000000000000 R12: 0000000000000000
>  R13: ffffa0bba9f3cf90 R14: 0000000000000000 R15: 0000000000000000
>  FS:  0000000000000000(0000) GS:ffffa0c9ff9c0000(0000) knlGS:0000000000000000
>  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>  CR2: 0000000000000008 CR3: 00000001c9c6c005 CR4: 00000000007706e0
>  DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>  DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>  PKRU: 55555554
>  Call Trace:
>   nvme_tcp_queue_rq+0xef/0x330 [nvme_tcp]
>   blk_mq_dispatch_rq_list+0x11c/0x7c0
>   ? blk_mq_flush_busy_ctxs+0xf6/0x110
>   __blk_mq_sched_dispatch_requests+0x12b/0x170
>   blk_mq_sched_dispatch_requests+0x30/0x60
>   __blk_mq_run_hw_queue+0x2b/0x60
>   process_one_work+0x1cb/0x360
>   ? process_one_work+0x360/0x360
>   worker_thread+0x30/0x370
>   ? process_one_work+0x360/0x360
>   kthread+0x116/0x130
>   ? kthread_park+0x80/0x80
>   ret_from_fork+0x1f/0x30
> --
>
> Fixes: cb9b870fba3e ("nvme-tcp: fix wrong setting of request iov_iter")
> Reported-by: Yi Zhang <yi.zhang at redhat.com>
> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
Looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>


From sagi at grimberg.me  Wed Feb 10 17:33:40 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 10 Feb 2021 14:33:40 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <SJ0PR19MB4544A653D3A84A19A28D3984F28E9@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
 <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>
 <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>
 <2251aa94-0d6c-1405-b1d8-185bcd714d8c@grimberg.me>
 <SJ0PR19MB45442A0B8E171B55A8AACACEF2B29@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <8937148e-f667-9a05-3828-58114dca6da3@grimberg.me>
 <SJ0PR19MB4544A653D3A84A19A28D3984F28E9@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <1a430561-498c-70dd-f14b-8fab10673f65@grimberg.me>


> Hi Sagi,
> 
> I was finally?able to get back to the crash issue.
> 
> Using Wireshark, I compared the PDUs from the nvmet to our home-brewed 
> Central Discovery Controller (CDC). I did not see any major differences 
> in the data itself. However, there is one significant difference in the 
> way that the nvmet and CDC indicate that a command has completed 
> successfully.
> 
> The NVMe-oF spec describes two ways that a Controller can indicate to 
> the Host that a command has completed successfully. One way is to send a 
> Response Capsule with the "status" set to 0. Another way is to set the 
> SUCCESS bit to 1 in the last C2HData PDU. This approach eliminates the 
> need for a Response Capsule PDU. Ref. NVMe-oF specs, section 7.4.5.2, 
> 5th paragraph.
> 
> The CDC sets the last C2HData's SUCCESS bit to 1 instead of sending a 
> Response Capsule. nvmet sets SUCCESS=0 and sends a separate Response 
> Capsule. Could this be the cause of the crash?

Don't think so, but did the host ask for this in the connect?
The host should explicitly ask for it when connecting. With nvme-cli
when connecting one should specify disable_sqflow (this optimization
is only possible if sq flow control is explicitly disabled).

In any event, even if the controller is misbehaving, it shouldn't crash
the host. I'll look into that.


From sagi at grimberg.me  Wed Feb 10 17:34:54 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 10 Feb 2021 14:34:54 -0800
Subject: [PATCH 1/2] nvmet: don't check iosqes,iocqes for discovery
 controllers
In-Reply-To: <20210210101037.GA26576@lst.de>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210210101037.GA26576@lst.de>
Message-ID: <bf05392c-8c92-d6c2-d9f9-a22ac23bc338@grimberg.me>


>> +	if (!ctrl->subsys->type == NVME_NQN_DISC &&
> 
> I think the normal way to write this would be a !=.

:)

>> +	    (nvmet_cc_iosqes(ctrl->cc) != NVME_NVM_IOSQES ||
>> +	     nvmet_cc_iocqes(ctrl->cc) != NVME_NVM_IOCQES)) {
> 
> By the spec we'd need to check these are 0, but of course we'd break
> the existing Linux host.  Maybe throw in a comment?

Sure, I'll add it.


From sagi at grimberg.me  Wed Feb 10 17:37:09 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 10 Feb 2021 14:37:09 -0800
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery
 controllers
In-Reply-To: <20210210101126.GA26624@lst.de>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210209101235.1539917-2-sagi@grimberg.me> <20210210101126.GA26624@lst.de>
Message-ID: <3702938a-cfd5-b725-bb75-7c9072a42fde@grimberg.me>

> So if we change this we break interaction with all old Linux (and maybe
> other?) targets.

Well yes.. Do you think that we should do a retry in case it fails for
backward compatibility?


From Chaitanya.Kulkarni at wdc.com  Wed Feb 10 17:42:42 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 10 Feb 2021 22:42:42 +0000
Subject: [PATCH V9 0/9] nvmet: add ZBD backend support
References: <20210112042623.6316-1-chaitanya.kulkarni@wdc.com>
Message-ID: <BYAPR04MB4965A4A8B3BD9E070F492EDF868D9@BYAPR04MB4965.namprd04.prod.outlook.com>

Christoph/Damien,

On 1/11/21 8:26 PM, Chaitanya Kulkarni wrote:
> Hi,
>
> NVMeOF Host is capable of handling the NVMe Protocol based Zoned Block
> Devices (ZBD) in the Zoned Namespaces (ZNS) mode with the passthru
> backend. There is no support for a generic block device backend to
> handle the ZBD devices which are not NVMe protocol compliant.
>
> This adds support to export the ZBDs (which are not NVMe drives) to host
> the from target via NVMeOF using the host side ZNS interface.
>
> The patch series is generated in bottom-top manner where, it first adds
> prep patch and ZNS command-specific handlers on the top of genblk and 
> updates the data structures, then one by one it wires up the admin cmds
> in the order host calls them in namespace initializing sequence. Once
> everything is ready, it wires-up the I/O command handlers. See below for
> patch-series overview.
>
> All the testcases are passing for the ZoneFS where ZBD exported with
> NVMeOF backed by null_blk ZBD and null_blk ZBD without NVMeOF. Adding
> test result below.
>
> Note: This patch-series is based on the earlier posted patch series :-
>
> [PATCH V2 0/4] nvmet: admin-cmd related cleanups and a fix
> http://lists.infradead.org/pipermail/linux-nvme/2021-January/021729.html
>
> -ck
>
> Changes from V8:-
>
> 1. Rebase and retest on latest nvme-5.11.
> 2. Export ctrl->cap csi support only if CONFIG_BLK_DEV_ZONE is set.
> 3. Add a fix to admin ns-desc list handler for handling default csi.
>
I can see that Damien's granularity series is in the linux-block tree, I'm
planning to send v10 of this series given that it also has a block layer
patch
[1] should I use the linux-block/for-next or linux-nvme/nvme-5.12 ?


[1]  [PATCH V9 1/9] block: export bio_add_hw_pages()


From yi.zhang at redhat.com  Wed Feb 10 23:04:52 2021
From: yi.zhang at redhat.com (Yi Zhang)
Date: Thu, 11 Feb 2021 12:04:52 +0800
Subject: [PATCH] nvme-tcp: fix crash triggered with a dataless request
 submission
In-Reply-To: <BYAPR04MB49650F2CB6A009986E0DC517868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210210220400.1730263-1-sagi@grimberg.me>
 <BYAPR04MB49650F2CB6A009986E0DC517868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <3d3efa8d-5e77-6cdf-88b4-a225fab36ffc@redhat.com>



On 2/11/21 6:16 AM, Chaitanya Kulkarni wrote:
> On 2/10/21 2:10 PM, Sagi Grimberg wrote:
>> write-zeros has a bio, but does not have any data buffers associated
>> with it. Hence should not initialize the request iter for it (which
>> attempts to reference the bi_io_vec (and crash).
>> --
>>   run blktests nvme/012 at 2021-02-05 21:53:34
>>   BUG: kernel NULL pointer dereference, address: 0000000000000008
>>   #PF: supervisor read access in kernel mode
>>   #PF: error_code(0x0000) - not-present page
>>   PGD 0 P4D 0
>>   Oops: 0000 [#1] SMP NOPTI
>>   CPU: 15 PID: 12069 Comm: kworker/15:2H Tainted: G S        I       5.11.0-rc6+ #1
>>   Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
>>   Workqueue: kblockd blk_mq_run_work_fn
>>   RIP: 0010:nvme_tcp_init_iter+0x7d/0xd0 [nvme_tcp]
>>   RSP: 0018:ffffbd084447bd18 EFLAGS: 00010246
>>   RAX: 0000000000000000 RBX: ffffa0bba9f3ce80 RCX: 0000000000000000
>>   RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000002000000
>>   RBP: ffffa0ba8ac6fec0 R08: 0000000002000000 R09: 0000000000000000
>>   R10: 0000000002800809 R11: 0000000000000000 R12: 0000000000000000
>>   R13: ffffa0bba9f3cf90 R14: 0000000000000000 R15: 0000000000000000
>>   FS:  0000000000000000(0000) GS:ffffa0c9ff9c0000(0000) knlGS:0000000000000000
>>   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>   CR2: 0000000000000008 CR3: 00000001c9c6c005 CR4: 00000000007706e0
>>   DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>   DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>   PKRU: 55555554
>>   Call Trace:
>>    nvme_tcp_queue_rq+0xef/0x330 [nvme_tcp]
>>    blk_mq_dispatch_rq_list+0x11c/0x7c0
>>    ? blk_mq_flush_busy_ctxs+0xf6/0x110
>>    __blk_mq_sched_dispatch_requests+0x12b/0x170
>>    blk_mq_sched_dispatch_requests+0x30/0x60
>>    __blk_mq_run_hw_queue+0x2b/0x60
>>    process_one_work+0x1cb/0x360
>>    ? process_one_work+0x360/0x360
>>    worker_thread+0x30/0x370
>>    ? process_one_work+0x360/0x360
>>    kthread+0x116/0x130
>>    ? kthread_park+0x80/0x80
>>    ret_from_fork+0x1f/0x30
>> --
>>
>> Fixes: cb9b870fba3e ("nvme-tcp: fix wrong setting of request iov_iter")
>> Reported-by: Yi Zhang <yi.zhang at redhat.com>
>> Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
> Looks good.
>
> Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Verified with latest linux-block/for-next.

Tested-by: Yi Zhang <yi.zhang at redhat.com>

> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
>



From kbusch at kernel.org  Wed Feb 10 23:25:59 2021
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 11 Feb 2021 13:25:59 +0900
Subject: [PATCH] nvme-tcp: fix crash triggered with a dataless request
 submission
In-Reply-To: <20210210220400.1730263-1-sagi@grimberg.me>
References: <20210210220400.1730263-1-sagi@grimberg.me>
Message-ID: <20210211042559.GB28207@redsun51.ssa.fujisawa.hgst.com>

Looks good.

Reviewed-by: Keith Busch <kbusch at kernel.org>


From lkp at intel.com  Thu Feb 11 01:18:27 2021
From: lkp at intel.com (kernel test robot)
Date: Thu, 11 Feb 2021 14:18:27 +0800
Subject: [linux-nvme:nvme-5.12] BUILD SUCCESS
 4bdf260362b3be529d170b04662638fd6dc52241
Message-ID: <6024cc33.FXXaZx+tbdfIMKLQ%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.12
branch HEAD: 4bdf260362b3be529d170b04662638fd6dc52241  nvme: add 48-bit DMA address quirk for Amazon NVMe controllers

elapsed time: 825m

configs tested: 98
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
openrisc                         alldefconfig
arm                         bcm2835_defconfig
powerpc                    klondike_defconfig
c6x                         dsk6455_defconfig
mips                          malta_defconfig
powerpc                        cell_defconfig
mips                         tb0219_defconfig
m68k                            q40_defconfig
mips                        qi_lb60_defconfig
arm                           sunxi_defconfig
arm                    vt8500_v6_v7_defconfig
arm                        realview_defconfig
arm                          badge4_defconfig
riscv                            alldefconfig
arm                        vexpress_defconfig
m68k                            mac_defconfig
sh                  sh7785lcr_32bit_defconfig
m68k                             alldefconfig
m68k                         apollo_defconfig
m68k                       bvme6000_defconfig
mips                           ci20_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
s390                             allmodconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                               tinyconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a001-20210209
i386                 randconfig-a005-20210209
i386                 randconfig-a003-20210209
i386                 randconfig-a002-20210209
i386                 randconfig-a006-20210209
i386                 randconfig-a004-20210209
i386                 randconfig-a016-20210209
i386                 randconfig-a013-20210209
i386                 randconfig-a012-20210209
i386                 randconfig-a014-20210209
i386                 randconfig-a011-20210209
i386                 randconfig-a015-20210209
x86_64               randconfig-a006-20210209
x86_64               randconfig-a001-20210209
x86_64               randconfig-a005-20210209
x86_64               randconfig-a004-20210209
x86_64               randconfig-a002-20210209
x86_64               randconfig-a003-20210209
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a013-20210209
x86_64               randconfig-a014-20210209
x86_64               randconfig-a015-20210209
x86_64               randconfig-a012-20210209
x86_64               randconfig-a016-20210209
x86_64               randconfig-a011-20210209

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From hch at lst.de  Thu Feb 11 02:02:50 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 11 Feb 2021 08:02:50 +0100
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery
 controllers
In-Reply-To: <3702938a-cfd5-b725-bb75-7c9072a42fde@grimberg.me>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210209101235.1539917-2-sagi@grimberg.me> <20210210101126.GA26624@lst.de>
 <3702938a-cfd5-b725-bb75-7c9072a42fde@grimberg.me>
Message-ID: <20210211070250.GA13708@lst.de>

On Wed, Feb 10, 2021 at 02:37:09PM -0800, Sagi Grimberg wrote:
>> So if we change this we break interaction with all old Linux (and maybe
>> other?) targets.
>
> Well yes.. Do you think that we should do a retry in case it fails for
> backward compatibility?

I think we have to.


From hch at lst.de  Thu Feb 11 02:06:43 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 11 Feb 2021 08:06:43 +0100
Subject: [PATCH] nvme-tcp: fix crash triggered with a dataless request
 submission
In-Reply-To: <20210210220400.1730263-1-sagi@grimberg.me>
References: <20210210220400.1730263-1-sagi@grimberg.me>
Message-ID: <20210211070643.GA13837@lst.de>

Applied to nvme-5.12.


From hch at lst.de  Thu Feb 11 02:07:48 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 11 Feb 2021 08:07:48 +0100
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write
 Zeroes command
In-Reply-To: <BYAPR04MB49650A271CD1C1B3EFA72F3D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <YCOxjZhBTBchOq0n@infradead.org>
 <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
 <20210210131557.GB29904@lst.de>
 <BYAPR04MB49650A271CD1C1B3EFA72F3D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210211070748.GA13852@lst.de>

On Wed, Feb 10, 2021 at 10:11:15PM +0000, Chaitanya Kulkarni wrote:
> On 2/10/21 5:20 AM, Christoph Hellwig wrote:
> > MDTS vs Write Zeroes is a common misconception.  We had that discussion
> > in the NVMe working group, and while the text should pretty clear that
> > MDTS only applies to data transfers many implementators did not understand
> > that, which is why we added the clarification you quoted.
> >
> > Also the next NVMe spec will allow devices to advertise an explicit
> > Write Zeroes limit, take a look at TP4040 from the "NVM Express 1.4 Ratified
> > TPs" at https://nvmexpress.org/developers/nvme-specification/.
> >
> > So I think limiting to MDTS unless the new WZSL is set might be a fail
> > safe option, even if it is more pessimistic than what the spec says.
> >
> > I'd also love to retest most Write Zeroes quirks with that in place.
> 
> Do you prefer some variant of the following patch (totally untested)? OR
> something else ?

Somwhat.  As said I suspect defaulting to MDTS with a big fat comment
might make most sense unless the new WZSL is set.


From hch at lst.de  Thu Feb 11 02:20:53 2021
From: hch at lst.de (hch at lst.de)
Date: Thu, 11 Feb 2021 08:20:53 +0100
Subject: [PATCH V9 0/9] nvmet: add ZBD backend support
In-Reply-To: <BYAPR04MB4965A4A8B3BD9E070F492EDF868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210112042623.6316-1-chaitanya.kulkarni@wdc.com>
 <BYAPR04MB4965A4A8B3BD9E070F492EDF868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210211072053.GA14044@lst.de>

On Wed, Feb 10, 2021 at 10:42:42PM +0000, Chaitanya Kulkarni wrote:
> I can see that Damien's granularity series is in the linux-block tree, I'm
> planning to send v10 of this series given that it also has a block layer
> patch
> [1] should I use the linux-block/for-next or linux-nvme/nvme-5.12 ?

I'd just wait for -rc1.


From sagi at grimberg.me  Thu Feb 11 05:28:32 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 11 Feb 2021 02:28:32 -0800
Subject: nvme-5.12 is crashing with fabrics discover
Message-ID: <f4f56a23-c458-5a4c-457d-d9478aea854a@grimberg.me>

Hey,

Started seeing nvme discovery crashing since commit:
a7c7f7b2b641 ("nvme: use bio_set_dev to assign ->bi_bdev")

Not yet sure what's going on, but reverting this commit
makes the issue go away.

Trace:
--
[   86.901818] nvme nvme0: new ctrl: NQN 
"nqn.2014-08.org.nvmexpress.discovery", addr 192.168.123.1:8009
[   86.902223] BUG: kernel NULL pointer dereference, address: 
0000000000000368
[   86.903345] #PF: supervisor read access in kernel mode
[   86.904949] #PF: error_code(0x0000) - not-present page
[   86.906202] PGD 0 P4D 0
[   86.907020] Oops: 0000 [#1] SMP PTI
[   86.908025] CPU: 3 PID: 1470 Comm: nvme Tainted: G           OE 
5.11.0-rc5+ #35
[   86.909333] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 
1.14.0-1 04/01/2014
[   86.910924] RIP: 0010:bio_associate_blkg_from_css+0x45/0x140
[   86.912854] Code: 38 f6 c2 03 0f 85 de 00 00 00 65 48 ff 0a e8 12 52 
cb ff 4d 85 e4 74 4d 49 83 bc 24 c0 00 00 00 00 74 42 48 8b 45 08 4c 89 
e7 <48> 8b 80 68 03 00 00 48 8b 70 50 e8 0b fe ff ff 48 89 c3 48 85 c0
[   86.915683] RSP: 0018:ffffbe6c409d3da0 EFLAGS: 00010282
[   86.917316] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 
0000000000000010
[   86.919128] RDX: ffff9e82f3b60000 RSI: ffff9e82f0fdea00 RDI: 
ffff9e82f0fdea00
[   86.919135] RBP: ffff9e82c2769000 R08: 0000000000000001 R09: 
00000000409d3d00
[   86.919137] R10: ffff9e82c2769000 R11: ffff9e82c2769078 R12: 
ffff9e82f0fdea00
[   86.919139] R13: 0000000000000000 R14: ffff9e82c2769000 R15: 
0000000000000000
[   86.919143] FS:  00007fbecaea0780(0000) GS:ffff9e833dd80000(0000) 
knlGS:0000000000000000
[   86.919146] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   86.919149] CR2: 0000000000000368 CR3: 00000000034f6004 CR4: 
0000000000370ee0
[   86.919162] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
[   86.919164] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
0000000000000400
[   86.919169] Call Trace:
[   86.919243]  bio_associate_blkg+0x20/0x70
[   86.923268]  nvme_submit_user_cmd+0xd9/0x260 [nvme_core]
[   86.923391]  nvme_user_cmd+0x11f/0x190 [nvme_core]
--


From sagi at grimberg.me  Thu Feb 11 05:43:27 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 11 Feb 2021 02:43:27 -0800
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery
 controllers
In-Reply-To: <20210211070250.GA13708@lst.de>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210209101235.1539917-2-sagi@grimberg.me> <20210210101126.GA26624@lst.de>
 <3702938a-cfd5-b725-bb75-7c9072a42fde@grimberg.me>
 <20210211070250.GA13708@lst.de>
Message-ID: <f279bd6a-d4e3-52b6-740b-66f90434cf69@grimberg.me>


>>> So if we change this we break interaction with all old Linux (and maybe
>>> other?) targets.
>>
>> Well yes.. Do you think that we should do a retry in case it fails for
>> backward compatibility?
> 
> I think we have to.

The annoying thing here is that this issue will manifest in the host
waiting on nvme_wait_ready for a long 7.5 seconds to understand that
its maybe incompatible and re-attempt (nvme_enable_ctrl writes to cap
and then polls CSTS)...

What I have now looks like:
--
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 02579f4f776c..ee65c89a991c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2457,8 +2457,14 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl)
  }
  EXPORT_SYMBOL_GPL(nvme_disable_ctrl);

+static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
+{
+       return ctrl->opts && ctrl->opts->discovery_nqn;
+}
+
  int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
  {
+       bool compat_disc_ctrl_config = false;
         unsigned dev_page_min;
         int ret;

@@ -2482,13 +2488,33 @@ int nvme_enable_ctrl(struct nvme_ctrl *ctrl)
                 ctrl->ctrl_config = NVME_CC_CSS_NVM;
         ctrl->ctrl_config |= (NVME_CTRL_PAGE_SHIFT - 12) << 
NVME_CC_MPS_SHIFT;
         ctrl->ctrl_config |= NVME_CC_AMS_RR | NVME_CC_SHN_NONE;
-       ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
+again:
+       if (!nvme_discovery_ctrl(ctrl) || compat_disc_ctrl_config)
+               ctrl->ctrl_config |= NVME_CC_IOSQES | NVME_CC_IOCQES;
         ctrl->ctrl_config |= NVME_CC_ENABLE;

         ret = ctrl->ops->reg_write32(ctrl, NVME_REG_CC, ctrl->ctrl_config);
         if (ret)
                 return ret;
-       return nvme_wait_ready(ctrl, ctrl->cap, true);
+
+       ret = nvme_wait_ready(ctrl, ctrl->cap, true);
+       if (ret) {
+               if (!compat_disc_ctrl_config) {
+                       pr_err("retrying...\n");
+                       /*
+                        * Backward compatibility work-around: some targets
+                        * (i.e. older Linux targets) may incorrectly verify
+                        * iosqes,iocqes are non-zero for discovery
+                        * controllers. So in order not to break them, we
+                        * attempt once again with the incorrect settings.
+                        */
+                       if (nvme_disable_ctrl(ctrl))
+                               return ret;
+                       compat_disc_ctrl_config = true;
+                       goto again;
+               }
+       }
+       return ret;
  }
  EXPORT_SYMBOL_GPL(nvme_enable_ctrl);

@@ -2902,11 +2928,6 @@ static const struct attribute_group 
*nvme_subsys_attrs_groups[] = {
         NULL,
  };

-static inline bool nvme_discovery_ctrl(struct nvme_ctrl *ctrl)
-{
-       return ctrl->opts && ctrl->opts->discovery_nqn;
-}
-
  static bool nvme_validate_cntlid(struct nvme_subsystem *subsys,
                 struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
  {
--


From mgurtovoy at nvidia.com  Thu Feb 11 08:01:56 2021
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Thu, 11 Feb 2021 13:01:56 +0000
Subject: [PATCH 2/2] nvme: resync header file with common nvme-cli tool
In-Reply-To: <20210209153503.GA8607@lst.de>
References: <20210121090948.8027-1-mgurtovoy@nvidia.com>
 <20210121090948.8027-4-mgurtovoy@nvidia.com> <20210127174717.GF28180@lst.de>
 <BYAPR12MB349445BDAFCBD9C9FBC32701DE8E9@BYAPR12MB3494.namprd12.prod.outlook.com>
 <20210209153503.GA8607@lst.de>
Message-ID: <BYAPR12MB34947F0C352E4035BAC258CFDE8C9@BYAPR12MB3494.namprd12.prod.outlook.com>

see attached.

Btw, the nvme-5.12 is not so stable.
I didn't debug it but I couldn't establish rdma/tcp/loop connections.
So the attached was just compiled and not tested.

-----Original Message-----
From: Christoph Hellwig <hch at lst.de> 
Sent: Tuesday, February 9, 2021 5:35 PM
To: Max Gurtovoy <mgurtovoy at nvidia.com>
Cc: Christoph Hellwig <hch at lst.de>; linux-nvme at lists.infradead.org; sagi at grimberg.me; kbusch at kernel.org; chaitanya.kulkarni at wdc.com
Subject: Re: [PATCH 2/2] nvme: resync header file with common nvme-cli tool

On Tue, Feb 09, 2021 at 03:07:05PM +0000, Max Gurtovoy wrote:
> is this respin still needed ?

Yes.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-nvme-resync-header-file-with-common-nvme-cli-tool.patch
Type: application/octet-stream
Size: 5814 bytes
Desc: 0001-nvme-resync-header-file-with-common-nvme-cli-tool.patch
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210211/8270ede8/attachment.obj>

From Chaitanya.Kulkarni at wdc.com  Thu Feb 11 13:11:19 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 11 Feb 2021 18:11:19 +0000
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes
 command
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <YCOxjZhBTBchOq0n@infradead.org>
 <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
 <20210210131557.GB29904@lst.de>
 <BYAPR04MB49650A271CD1C1B3EFA72F3D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210211070748.GA13852@lst.de>
Message-ID: <BYAPR04MB4965C95C3F826D7222EB3176868C9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/10/21 11:07 PM, Christoph Hellwig wrote:
>>> I'd also love to retest most Write Zeroes quirks with that in place.
>> Do you prefer some variant of the following patch (totally untested)? OR
>> something else ?
> Somwhat.  As said I suspect defaulting to MDTS with a big fat comment
> might make most sense unless the new WZSL is set.
>
Okay, will send out patche(s) soon.


From hch at infradead.org  Thu Feb 11 13:14:21 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 11 Feb 2021 18:14:21 +0000
Subject: vhost/virtio fabric
In-Reply-To: <CA+ULb+v-2jigmQA=Z2WmWP0n3wCbEs_s94oCoNbjJMw14ATCjw@mail.gmail.com>
References: <CA+ULb+v-2jigmQA=Z2WmWP0n3wCbEs_s94oCoNbjJMw14ATCjw@mail.gmail.com>
Message-ID: <20210211181421.GA2522386@infradead.org>

On Mon, Feb 08, 2021 at 08:24:40PM -0800, Roman Shaposhnik wrote:
> Please let me know if there's any work that has ever been done
> in the direction (even if in a form of unmerged patches) or whether
> if I need this functionality I'd have to implement it from scratch (probably
> aping a great deal of drivers/vhost/scsi.c).

The old vhost-nvme patches from Ming and Hannes should be out on the
list.  From the kernel POV we're fine with it, but the Qemu people did
not like it very much.


From hch at infradead.org  Thu Feb 11 13:15:26 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 11 Feb 2021 19:15:26 +0100
Subject: [GIT PULL] second round of nvme updates for 5.12
Message-ID: <YCV0PmcdMDOeQp5Q@infradead.org>

Note that we have some issues with the previous updates.  We're working
fast on fixes for those, but in the meantime I think it is good to get
these changes out to you and into linux-next.


The following changes since commit 6751c1e3cff3aa763c760c08862627069a37b50e:

  bcache: Avoid comma separated statements (2021-02-10 08:06:00 -0700)

are available in the Git repository at:

  git://git.infradead.org/nvme.git tags/nvme-5.12-2021-02-11

for you to fetch changes up to e11e5116171dedeaf63735931e72ad5de0f30ed5:

  nvme-tcp: fix crash triggered with a dataless request submission (2021-02-11 08:04:51 +0100)

----------------------------------------------------------------
nvme updates for 5.12:

 - fix multipath handling of ->queue_rq errors (Chao Leng)
 - nvmet cleanups (Chaitanya Kulkarni)
 - add a quirk for buggy Amazon controller (Filippo Sironi)
 - avoid devm allocations in nvme-hwmon that don't interact well with
   fabrics (Hannes Reinecke)
 - sysfs cleanups (Jiapeng Chong)
 - fix nr_zones for multipath (Keith Busch)
 - nvme-tcp crash fix for no-data commands (Sagi Grimberg)
 - nvmet-tcp fixes (Sagi Grimberg)
 - add a missing __rcu annotation (me)

----------------------------------------------------------------
Chaitanya Kulkarni (10):
      nvmet: set status to 0 in case for invalid nsid
      nvmet: return uniform error for invalid ns
      nvmet: make nvmet_find_namespace() req based
      nvmet: remove extra variable in id-ns handler
      nvmet: add helper to report invalid opcode
      nvmet: use invalid cmd opcode helper
      nvmet: use invalid cmd opcode helper
      nvmet: use min of device_path and disk len
      nvmet: add nvmet_req_subsys() helper
      nvmet: remove else at the end of the function

Chao Leng (4):
      blk-mq: introduce blk_mq_set_request_complete
      nvme: introduce a nvme_host_path_error helper
      nvme-fabrics: avoid double completions in nvmf_fail_nonready_command
      nvme-rdma: handle nvme_rdma_post_send failures better

Christoph Hellwig (1):
      nvmet-fc: add a missing __rcu annotation to nvmet_fc_tgt_assoc.queues

Filippo Sironi (1):
      nvme: add 48-bit DMA address quirk for Amazon NVMe controllers

Hannes Reinecke (1):
      nvme-hwmon: rework to avoid devm allocation

Jiapeng Chong (1):
      nvme: convert sysfs sprintf/snprintf family to sysfs_emit

Keith Busch (1):
      nvme-multipath: set nr_zones for zoned namespaces

Sagi Grimberg (3):
      nvmet-tcp: fix receive data digest calculation for multiple h2cdata PDUs
      nvmet-tcp: fix potential race of tcp socket closing accept_work
      nvme-tcp: fix crash triggered with a dataless request submission

 drivers/nvme/host/core.c          | 26 +++++++++++++----
 drivers/nvme/host/fabrics.c       |  6 +---
 drivers/nvme/host/hwmon.c         | 31 +++++++++++++-------
 drivers/nvme/host/multipath.c     |  4 +++
 drivers/nvme/host/nvme.h          | 15 ++++++++++
 drivers/nvme/host/pci.c           | 21 +++++++++++++-
 drivers/nvme/host/rdma.c          |  4 ++-
 drivers/nvme/host/tcp.c           |  2 +-
 drivers/nvme/target/admin-cmd.c   | 59 ++++++++++++++++-----------------------
 drivers/nvme/target/core.c        | 37 +++++++++++++++---------
 drivers/nvme/target/fc.c          |  2 +-
 drivers/nvme/target/io-cmd-bdev.c |  5 +---
 drivers/nvme/target/io-cmd-file.c |  5 +---
 drivers/nvme/target/nvmet.h       | 10 +++++--
 drivers/nvme/target/passthru.c    |  6 ++--
 drivers/nvme/target/tcp.c         | 59 ++++++++++++++++++++++++++++-----------
 drivers/nvme/target/trace.h       |  9 ++++--
 include/linux/blk-mq.h            | 12 ++++++++
 18 files changed, 208 insertions(+), 105 deletions(-)


From Chaitanya.Kulkarni at wdc.com  Thu Feb 11 13:17:56 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 11 Feb 2021 18:17:56 +0000
Subject: nvme-5.12 is crashing with fabrics discover
References: <f4f56a23-c458-5a4c-457d-d9478aea854a@grimberg.me>
Message-ID: <BYAPR04MB4965CD9C122653C1009F0500868C9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/11/21 2:36 AM, Sagi Grimberg wrote:
> Hey,
>
> Started seeing nvme discovery crashing since commit:
> a7c7f7b2b641 ("nvme: use bio_set_dev to assign ->bi_bdev")
>
> Not yet sure what's going on, but reverting this commit
> makes the issue go away.
>
> Trace:
> --
> [   86.901818] nvme nvme0: new ctrl: NQN 
> "nqn.2014-08.org.nvmexpress.discovery", addr 192.168.123.1:8009
> [   86.902223] BUG: kernel NULL pointer dereference, address: 
> 0000000000000368
> [   86.903345] #PF: supervisor read access in kernel mode
> [   86.904949] #PF: error_code(0x0000) - not-present page
> [   86.906202] PGD 0 P4D 0
> [   86.907020] Oops: 0000 [#1] SMP PTI
> [   86.908025] CPU: 3 PID: 1470 Comm: nvme Tainted: G           OE 
> 5.11.0-rc5+ #35
> [   86.909333] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 
> 1.14.0-1 04/01/2014
> [   86.910924] RIP: 0010:bio_associate_blkg_from_css+0x45/0x140
> [   86.912854] Code: 38 f6 c2 03 0f 85 de 00 00 00 65 48 ff 0a e8 12 52 
> cb ff 4d 85 e4 74 4d 49 83 bc 24 c0 00 00 00 00 74 42 48 8b 45 08 4c 89 
> e7 <48> 8b 80 68 03 00 00 48 8b 70 50 e8 0b fe ff ff 48 89 c3 48 85 c0
> [   86.915683] RSP: 0018:ffffbe6c409d3da0 EFLAGS: 00010282
> [   86.917316] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 
> 0000000000000010
> [   86.919128] RDX: ffff9e82f3b60000 RSI: ffff9e82f0fdea00 RDI: 
> ffff9e82f0fdea00
> [   86.919135] RBP: ffff9e82c2769000 R08: 0000000000000001 R09: 
> 00000000409d3d00
> [   86.919137] R10: ffff9e82c2769000 R11: ffff9e82c2769078 R12: 
> ffff9e82f0fdea00
> [   86.919139] R13: 0000000000000000 R14: ffff9e82c2769000 R15: 
> 0000000000000000
> [   86.919143] FS:  00007fbecaea0780(0000) GS:ffff9e833dd80000(0000) 
> knlGS:0000000000000000
> [   86.919146] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   86.919149] CR2: 0000000000000368 CR3: 00000000034f6004 CR4: 
> 0000000000370ee0
> [   86.919162] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> [   86.919164] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
> 0000000000000400
> [   86.919169] Call Trace:
> [   86.919243]  bio_associate_blkg+0x20/0x70
> [   86.923268]  nvme_submit_user_cmd+0xd9/0x260 [nvme_core]
> [   86.923391]  nvme_user_cmd+0x11f/0x190 [nvme_core]
> --
>
>
I think fix for this is present in the linux-block/for-next tree, can
you see if that fixes
your problem ?

commit 59c157433fbc6a7f63f3d708ca2966d0f56bcb7c
Author: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Date:   Tue Jan 26 21:37:38 2021 -0800

    nvme-core: check bdev value for NULL
   





From hch at lst.de  Thu Feb 11 13:22:27 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 11 Feb 2021 19:22:27 +0100
Subject: nvme-5.12 is crashing with fabrics discover
In-Reply-To: <f4f56a23-c458-5a4c-457d-d9478aea854a@grimberg.me>
References: <f4f56a23-c458-5a4c-457d-d9478aea854a@grimberg.me>
Message-ID: <20210211182227.GA25786@lst.de>

On Thu, Feb 11, 2021 at 02:28:32AM -0800, Sagi Grimberg wrote:
> Hey,
>
> Started seeing nvme discovery crashing since commit:
> a7c7f7b2b641 ("nvme: use bio_set_dev to assign ->bi_bdev")
>
> Not yet sure what's going on, but reverting this commit
> makes the issue go away.

The patch below should fix it.  This came up before and I though I had
already queued up a patch from Keith for it, but it looks like it got lost:

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d387..4de6a3a135759d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1167,7 +1167,8 @@ static int nvme_submit_user_cmd(struct request_queue *q,
 		if (ret)
 			goto out;
 		bio = req->bio;
-		bio_set_dev(bio, bdev);
+		if (bdev)
+			bio_set_dev(bio, bdev);
 		if (bdev && meta_buffer && meta_len) {
 			meta = nvme_add_user_metadata(bio, meta_buffer, meta_len,
 					meta_seed, write);


From hch at lst.de  Thu Feb 11 13:23:57 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 11 Feb 2021 19:23:57 +0100
Subject: nvme-5.12 is crashing with fabrics discover
In-Reply-To: <BYAPR04MB4965CD9C122653C1009F0500868C9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <f4f56a23-c458-5a4c-457d-d9478aea854a@grimberg.me>
 <BYAPR04MB4965CD9C122653C1009F0500868C9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210211182357.GA25973@lst.de>

On Thu, Feb 11, 2021 at 06:17:56PM +0000, Chaitanya Kulkarni wrote:
> I think fix for this is present in the linux-block/for-next tree, can
> you see if that fixes
> your problem ?

Sorry that I attributed it to Keith in my last mail, this was you.  Looks
like it only made it into Jens' for-5.12/block tree, but not
for-5.12/drivers on which nvme-5.12 is based.


From hch at lst.de  Thu Feb 11 13:31:24 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 11 Feb 2021 19:31:24 +0100
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery
 controllers
In-Reply-To: <f279bd6a-d4e3-52b6-740b-66f90434cf69@grimberg.me>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210209101235.1539917-2-sagi@grimberg.me> <20210210101126.GA26624@lst.de>
 <3702938a-cfd5-b725-bb75-7c9072a42fde@grimberg.me>
 <20210211070250.GA13708@lst.de>
 <f279bd6a-d4e3-52b6-740b-66f90434cf69@grimberg.me>
Message-ID: <20210211183124.GA26045@lst.de>

On Thu, Feb 11, 2021 at 02:43:27AM -0800, Sagi Grimberg wrote:
> The annoying thing here is that this issue will manifest in the host
> waiting on nvme_wait_ready for a long 7.5 seconds to understand that
> its maybe incompatible and re-attempt (nvme_enable_ctrl writes to cap
> and then polls CSTS)...
>
> What I have now looks like:

In terms of what to do about the retry this is what we'll have to
do.  That being said I think for existing controllers we should just
stick to the existing wrong value, and only set the correct one
for versions > 1.3.  This isn't exactly correct, but probably causes
the least harm.  It will need an extensive comment, and maybe we need
to add an option to override it if controllers show up that do not
like the wrong value.


From axboe at kernel.dk  Thu Feb 11 13:31:58 2021
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 11 Feb 2021 11:31:58 -0700
Subject: [GIT PULL] second round of nvme updates for 5.12
In-Reply-To: <YCV0PmcdMDOeQp5Q@infradead.org>
References: <YCV0PmcdMDOeQp5Q@infradead.org>
Message-ID: <d76200cd-1ab2-8ac5-31b6-388db44bef3b@kernel.dk>

On 2/11/21 11:15 AM, Christoph Hellwig wrote:
> Note that we have some issues with the previous updates.  We're working
> fast on fixes for those, but in the meantime I think it is good to get
> these changes out to you and into linux-next.

Pulled - it throws a trivial reject in the quirks list, fwiw.

-- 
Jens Axboe



From hch at infradead.org  Thu Feb 11 13:33:16 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Thu, 11 Feb 2021 18:33:16 +0000
Subject: [GIT PULL] second round of nvme updates for 5.12
In-Reply-To: <d76200cd-1ab2-8ac5-31b6-388db44bef3b@kernel.dk>
References: <YCV0PmcdMDOeQp5Q@infradead.org>
 <d76200cd-1ab2-8ac5-31b6-388db44bef3b@kernel.dk>
Message-ID: <20210211183316.GA2527859@infradead.org>

On Thu, Feb 11, 2021 at 11:31:58AM -0700, Jens Axboe wrote:
> On 2/11/21 11:15 AM, Christoph Hellwig wrote:
> > Note that we have some issues with the previous updates.  We're working
> > fast on fixes for those, but in the meantime I think it is good to get
> > these changes out to you and into linux-next.
> 
> Pulled - it throws a trivial reject in the quirks list, fwiw.

Yes, people are producing buggy controllers way to quickly.. :(


From Chaitanya.Kulkarni at wdc.com  Thu Feb 11 13:33:23 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 11 Feb 2021 18:33:23 +0000
Subject: nvme-5.12 is crashing with fabrics discover
References: <f4f56a23-c458-5a4c-457d-d9478aea854a@grimberg.me>
 <BYAPR04MB4965CD9C122653C1009F0500868C9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210211182357.GA25973@lst.de>
Message-ID: <BYAPR04MB496523687FF7402E1692AA61868C9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/11/21 10:24 AM, Christoph Hellwig wrote:
> On Thu, Feb 11, 2021 at 06:17:56PM +0000, Chaitanya Kulkarni wrote:
>> I think fix for this is present in the linux-block/for-next tree, can
>> you see if that fixes
>> your problem ?
> Sorry that I attributed it to Keith in my last mail, this was you.  Looks
> like it only made it into Jens' for-5.12/block tree, but not
> for-5.12/drivers on which nvme-5.12 is based.
>

Saw that, I replied to Sagi's email with that commit, not sure if that got
delivered or not.


From axboe at kernel.dk  Thu Feb 11 13:34:23 2021
From: axboe at kernel.dk (Jens Axboe)
Date: Thu, 11 Feb 2021 11:34:23 -0700
Subject: [GIT PULL] second round of nvme updates for 5.12
In-Reply-To: <20210211183316.GA2527859@infradead.org>
References: <YCV0PmcdMDOeQp5Q@infradead.org>
 <d76200cd-1ab2-8ac5-31b6-388db44bef3b@kernel.dk>
 <20210211183316.GA2527859@infradead.org>
Message-ID: <06daa8d8-3ec9-8372-f912-65e844b87e31@kernel.dk>

On 2/11/21 11:33 AM, Christoph Hellwig wrote:
> On Thu, Feb 11, 2021 at 11:31:58AM -0700, Jens Axboe wrote:
>> On 2/11/21 11:15 AM, Christoph Hellwig wrote:
>>> Note that we have some issues with the previous updates.  We're working
>>> fast on fixes for those, but in the meantime I think it is good to get
>>> these changes out to you and into linux-next.
>>
>> Pulled - it throws a trivial reject in the quirks list, fwiw.
> 
> Yes, people are producing buggy controllers way to quickly.. :(

We'll do whitelists instead next time, less churn ;-)

-- 
Jens Axboe



From lkp at intel.com  Thu Feb 11 14:41:45 2021
From: lkp at intel.com (kernel test robot)
Date: Fri, 12 Feb 2021 03:41:45 +0800
Subject: [linux-nvme:nvme-5.12] BUILD SUCCESS
 e11e5116171dedeaf63735931e72ad5de0f30ed5
Message-ID: <60258879.Kj7xgZYkmPFku+rA%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.12
branch HEAD: e11e5116171dedeaf63735931e72ad5de0f30ed5  nvme-tcp: fix crash triggered with a dataless request submission

elapsed time: 727m

configs tested: 109
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
sh                                  defconfig
powerpc                     powernv_defconfig
arm                         palmz72_defconfig
mips                       bmips_be_defconfig
mips                        bcm47xx_defconfig
openrisc                    or1ksim_defconfig
mips                         tb0219_defconfig
m68k                            q40_defconfig
mips                        qi_lb60_defconfig
arm                           sunxi_defconfig
m68k                          amiga_defconfig
arm                       imx_v6_v7_defconfig
powerpc                     tqm8555_defconfig
sh                  sh7785lcr_32bit_defconfig
mips                        bcm63xx_defconfig
sh                           se7721_defconfig
sh                         microdev_defconfig
powerpc                         ps3_defconfig
powerpc                      walnut_defconfig
powerpc                    mvme5100_defconfig
mips                     loongson1c_defconfig
sh                           se7750_defconfig
sh                        sh7763rdp_defconfig
m68k                       m5275evb_defconfig
sparc64                          alldefconfig
powerpc                 mpc836x_mds_defconfig
arm                             pxa_defconfig
powerpc                 mpc832x_rdb_defconfig
sh                           se7751_defconfig
powerpc                     sequoia_defconfig
ia64                         bigsur_defconfig
ia64                      gensparse_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
nios2                               defconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
nios2                            allyesconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
sh                               allmodconfig
parisc                              defconfig
s390                             allyesconfig
s390                             allmodconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                               tinyconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
x86_64               randconfig-a006-20210209
x86_64               randconfig-a001-20210209
x86_64               randconfig-a005-20210209
x86_64               randconfig-a004-20210209
x86_64               randconfig-a002-20210209
x86_64               randconfig-a003-20210209
i386                 randconfig-a001-20210209
i386                 randconfig-a005-20210209
i386                 randconfig-a003-20210209
i386                 randconfig-a002-20210209
i386                 randconfig-a006-20210209
i386                 randconfig-a004-20210209
i386                 randconfig-a016-20210209
i386                 randconfig-a013-20210209
i386                 randconfig-a012-20210209
i386                 randconfig-a014-20210209
i386                 randconfig-a011-20210209
i386                 randconfig-a015-20210209
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                             allnoconfig
riscv                               defconfig
riscv                          rv32_defconfig
riscv                            allmodconfig
x86_64                                   rhel
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a013-20210209
x86_64               randconfig-a014-20210209
x86_64               randconfig-a015-20210209
x86_64               randconfig-a012-20210209
x86_64               randconfig-a016-20210209
x86_64               randconfig-a011-20210209

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From sagi at grimberg.me  Thu Feb 11 14:44:44 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Thu, 11 Feb 2021 11:44:44 -0800
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery
 controllers
In-Reply-To: <20210211183124.GA26045@lst.de>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210209101235.1539917-2-sagi@grimberg.me> <20210210101126.GA26624@lst.de>
 <3702938a-cfd5-b725-bb75-7c9072a42fde@grimberg.me>
 <20210211070250.GA13708@lst.de>
 <f279bd6a-d4e3-52b6-740b-66f90434cf69@grimberg.me>
 <20210211183124.GA26045@lst.de>
Message-ID: <a5ea738a-44c4-e953-3271-a0b97668000e@grimberg.me>


>> The annoying thing here is that this issue will manifest in the host
>> waiting on nvme_wait_ready for a long 7.5 seconds to understand that
>> its maybe incompatible and re-attempt (nvme_enable_ctrl writes to cap
>> and then polls CSTS)...
>>
>> What I have now looks like:
> 
> In terms of what to do about the retry this is what we'll have to
> do.  That being said I think for existing controllers we should just
> stick to the existing wrong value, and only set the correct one
> for versions > 1.3.  This isn't exactly correct, but probably causes
> the least harm.  It will need an extensive comment, and maybe we need
> to add an option to override it if controllers show up that do not
> like the wrong value.
So you suggest that we keep the retry w.a. but continue to send the
wrong value in 1.3 or lower?

I personally think that this would be extremely confusing to users, if
they do trip on this, but I agree that they are less likely to hit
it because existing controllers obviously accept wrong values...

As for option to override. I really hate it.. What is the least exposure
we can get for this but still allow the user to control this?

Wish we had a good and reliable quirk mechanism for fabrics...


From borisp at mellanox.com  Thu Feb 11 16:10:23 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:23 +0200
Subject: [PATCH v4 net-next  00/21] nvme-tcp receive offloads
Message-ID: <20210211211044.32701-1-borisp@mellanox.com>

Changes since v3:
=========================================
* Use DDP_TCP ifdefs in iov_iter and skb iterators to minimize impact
when compiled out (Christoph)
* Simplify netdev references and reduce the use of
get_netdev_for_sock (Sagi)
* Avoid "static" in it's own line, move it one line down (Christoph)
* Pass (queue, skb, *offset) and retrieve the pdu_seq in
nvme_tcp_resync_response (Sagi)
* Add missing assignment of offloading_netdev to null in offload_limits
error case (Sagi)
* Set req->offloaded = false once -- the lifetime rules are:
set to false on cmd_setup / set to true when ddp setup succeeds (Sagi)
* Replace pr_info_ratelimited with dev_info_ratelimited (Sagi)
* Add nvme_tcp_complete_request and invoke it from two similar call
sites (Sagi)
* Introduce nvme_tcp_req_map_sg earlier in the series (Sagi)
* Add nvme_tcp_consume_skb and put into it a hunk from
nvme_tcp_recv_data to handle copy with and without offload

Changes since v2:
=========================================
* Use skb->ddp_crc for copy offload to avoid skb_condense
* Default mellanox driver support to no (experimental feature)
* In iov_iter use non-ddp functions for kvec and iovec
* Remove typecasting in nvme-tcp

Changes since v1:
=========================================
* Rework iov_iter copy skip if src==dst to be less intrusive (David Ahern)
* Add tcp-ddp documentation (David Ahern)
* Refactor mellanox driver patches into more patches (Saeed Mahameed)
* Avoid pointer casting (David Ahern)
* Rename nvme-tcp offload flags (Shai Malin)
* Update cover-letter according to the above

Changes since RFC v1:
=========================================
* Split mlx5 driver patches to several commits
* Fix nvme-tcp handling of recovery flows. In particular, move queue offlaod
  init/teardown to the start/stop functions.

Overview
=========================================
This series adds support for nvme-tcp receive offloads
which do not mandate the offload of the network stack to the device.
Instead, these work together with TCP to offload:
1. copy from SKB to the block layer buffers
2. CRC verification for received PDU

The series implements these as a generic offload infrastructure for storage
protocols, which we call TCP Direct Data Placement (TCP_DDP) and TCP DDP CRC,
respectively. We use this infrastructure to implement NVMe-TCP offload for copy
and CRC. Future implementations can reuse the same infrastructure for other
protcols such as iSCSI.

Note:
These offloads are similar in nature to the packet-based NIC TLS offloads,
which are already upstream (see net/tls/tls_device.c).
You can read more about TLS offload here:
https://www.kernel.org/doc/html/latest/networking/tls-offload.html

Initialization and teardown:
=========================================
The offload for IO queues is initialized after the handshake of the
NVMe-TCP protocol is finished by calling `nvme_tcp_offload_socket`
with the tcp socket of the nvme_tcp_queue:
This operation sets all relevant hardware contexts in
hardware. If it fails, then the IO queue proceeds as usually with no offload.
If it succeeds then `nvme_tcp_setup_ddp` and `nvme_tcp_teardown_ddp` may be
called to perform copy offload, and crc offload will be used.
This initialization does not change the normal operation of nvme-tcp in any
way besides adding the option to call the above mentioned NDO operations.

For the admin queue, nvme-tcp does not initialize the offload.
Instead, nvme-tcp calls the driver to configure limits for the controller,
such as max_hw_sectors and max_segments; these must be limited to accomodate
potential HW resource limits, and to improve performance.

If some error occured, and the IO queue must be closed or reconnected, then
offload is teardown and initialized again. Additionally, we handle netdev
down events via the existing error recovery flow.

Copy offload works as follows:
=========================================
The nvme-tcp layer calls the NIC drive to map block layer buffers to ccid using
`nvme_tcp_setup_ddp` before sending the read request. When the repsonse is
received, then the NIC HW will write the PDU payload directly into the
designated buffer, and build an SKB such that it points into the destination
buffer; this SKB represents the entire packet received on the wire, but it
points to the block layer buffers. Once nvme-tcp attempts to copy data from
this SKB to the block layer buffer it can skip the copy by checking in the
copying function (memcpy_to_page):
if (src == dst) -> skip copy
Finally, when the PDU has been processed to completion, the nvme-tcp layer
releases the NIC HW context be calling `nvme_tcp_teardown_ddp` which
asynchronously unmaps the buffers from NIC HW.

As the copy skip change is in a sensative function, we are careful to avoid
changing it. To that end, we create alternative skb copy and hash iterators
that skip copy/hash if (src == dst). Nvme-tcp is the first user for these.

Asynchronous completion:
=========================================
The NIC must release its mapping between command IDs and the target buffers.
This mapping is released when NVMe-TCP calls the NIC
driver (`nvme_tcp_offload_socket`).
As completing IOs is performance criticial, we introduce asynchronous
completions for NVMe-TCP, i.e. NVMe-TCP calls the NIC, which will later
call NVMe-TCP to complete the IO (`nvme_tcp_ddp_teardown_done`).

An alternative approach is to move all the functions related to coping from
SKBs to the block layer buffers inside the nvme-tcp code - about 200 LOC.

CRC offload works as follows:
=========================================
After offload is initialized, we use the SKB's ddp_crc bit to indicate that:
"there was no problem with the verification of all CRC fields in this packet's
payload". The bit is set to zero if there was an error, or if HW skipped
offload for some reason. If *any* SKB in a PDU has (ddp_crc != 1), then software
must compute the CRC, and check it. We perform this check, and
accompanying software fallback at the end of the processing of a received PDU.

SKB changes:
=========================================
The CRC offload requires an additional bit in the SKB, which is useful for
preventing the coalescing of SKB with different crc offload values. This bit
is similar in concept to the "decrypted" bit. 

Performance:
=========================================
The expected performance gain from this offload varies with the block size.
We perform a CPU cycles breakdown of the copy/CRC operations in nvme-tcp
fio random read workloads:
For 4K blocks we see up to 11% improvement for a 100% read fio workload,
while for 128K blocks we see upto 52%. If we run nvme-tcp, and skip these
operations, then we observe a gain of about 1.1x and 2x respectively.

Resynchronization:
=========================================
The resynchronization flow is performed to reset the hardware tracking of
NVMe-TCP PDUs within the TCP stream. The flow consists of a request from
the driver, regarding a possible location of a PDU header. Followed by
a response from the nvme-tcp driver.

This flow is rare, and it should happen only after packet loss or
reordering events that involve nvme-tcp PDU headers.

The patches are organized as follows:
=========================================
Patch 1         the iov_iter change to skip copy if (src == dst).
Patches 2-4     the infrastructure for all TCP DDP.
                and TCP DDP CRC offloads, respectively.
Patch 5         exposes the get_netdev_for_sock function from TLS.
Patch 6         NVMe-TCP changes to call NIC driver on queue init/teardown.
Patches 7       NVMe-TCP changes to call NIC driver on IO operation.
                setup/teardown, and support async completions.
Patches 8       NVMe-TCP changes to support CRC offload on receive.
                Also, this patch moves CRC calculation to the end of PDU
                in case offload requires software fallback.
Patches 9       NVMe-TCP handling of netdev events: stop the offload if
                netdev is going down.
Patches 10-20   implement support for NVMe-TCP copy and CRC offload in
                the mlx5 NIC driver as the first user.
Patches 21      Document TCP DDP offload.

Testing:
=========================================
This series was tested using fio with various configurations of IO sizes,
depths, MTUs, and with both the SPDK and kernel NVMe-TCP targets.
Also, we have used QEMU and gate-level simulation to verify these patches.

Future work:
=========================================
A follow-up series will introduce support for transmit side CRC. Then,
we will work on adding support for TLS in NVMe-TCP and combining the
two offloads.

Ben Ben-Ishay (8):
  net/mlx5e: NVMEoTCP offload initialization
  net/mlx5e: KLM UMR helper macros
  net/mlx5e: NVMEoTCP use KLM UMRs
  net/mlx5e: NVMEoTCP queue init/teardown
  net/mlx5e: NVMEoTCP async ddp invalidation
  net/mlx5e: NVMEoTCP ddp setup and resync
  net/mlx5e: NVMEoTCP, data-path for DDP+CRC offload
  net/mlx5e: NVMEoTCP statistics

Ben Ben-ishay (2):
  net/mlx5: Header file changes for nvme-tcp offload
  net/mlx5: Add 128B CQE for NVMEoTCP offload

Boris Pismenny (9):
  net: Introduce direct data placement tcp offload
  net: Introduce crc offload for tcp ddp ulp
  iov_iter: DDP copy to iter/pages
  net: skb copy(+hash) iterators for DDP offloads
  net/tls: expose get_netdev_for_sock
  nvme-tcp: Add DDP offload control path
  nvme-tcp: Add DDP data-path
  net/mlx5e: TCP flow steering for nvme-tcp
  Documentation: add TCP DDP offload documentation

Or Gerlitz (1):
  nvme-tcp: Deal with netdevice DOWN events

Yoray Zack (1):
  nvme-tcp: RX CRC offload

 Documentation/networking/index.rst            |    1 +
 Documentation/networking/tcp-ddp-offload.rst  |  296 +++++
 .../net/ethernet/mellanox/mlx5/core/Kconfig   |   10 +
 .../net/ethernet/mellanox/mlx5/core/Makefile  |    2 +
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   31 +-
 .../net/ethernet/mellanox/mlx5/core/en/fs.h   |    4 +-
 .../ethernet/mellanox/mlx5/core/en/params.h   |    1 +
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   13 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.c   |    1 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.h   |    1 +
 .../mellanox/mlx5/core/en_accel/en_accel.h    |    9 +-
 .../mellanox/mlx5/core/en_accel/fs_tcp.c      |   10 +
 .../mellanox/mlx5/core/en_accel/fs_tcp.h      |    2 +-
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 1015 +++++++++++++++++
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    |  120 ++
 .../mlx5/core/en_accel/nvmeotcp_rxtx.c        |  264 +++++
 .../mlx5/core/en_accel/nvmeotcp_rxtx.h        |   43 +
 .../mlx5/core/en_accel/nvmeotcp_utils.h       |   80 ++
 .../net/ethernet/mellanox/mlx5/core/en_main.c |   40 +-
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |   66 +-
 .../ethernet/mellanox/mlx5/core/en_stats.c    |   37 +
 .../ethernet/mellanox/mlx5/core/en_stats.h    |   24 +
 .../net/ethernet/mellanox/mlx5/core/en_txrx.c |   17 +
 drivers/net/ethernet/mellanox/mlx5/core/fw.c  |    6 +
 drivers/nvme/host/tcp.c                       |  448 +++++++-
 include/linux/mlx5/device.h                   |   44 +-
 include/linux/mlx5/mlx5_ifc.h                 |  101 +-
 include/linux/mlx5/qp.h                       |    1 +
 include/linux/netdev_features.h               |    4 +
 include/linux/netdevice.h                     |    5 +
 include/linux/skbuff.h                        |   13 +
 include/linux/uio.h                           |   17 +
 include/net/inet_connection_sock.h            |    4 +
 include/net/sock.h                            |   17 +
 include/net/tcp_ddp.h                         |  136 +++
 lib/iov_iter.c                                |   53 +
 net/Kconfig                                   |   17 +
 net/core/datagram.c                           |   48 +
 net/core/skbuff.c                             |    8 +-
 net/ethtool/common.c                          |    2 +
 net/ipv4/tcp_input.c                          |    8 +
 net/ipv4/tcp_ipv4.c                           |    3 +
 net/ipv4/tcp_offload.c                        |    3 +
 net/tls/tls_device.c                          |   20 +-
 44 files changed, 2985 insertions(+), 60 deletions(-)
 create mode 100644 Documentation/networking/tcp-ddp-offload.rst
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
 create mode 100644 include/net/tcp_ddp.h

-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:24 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:24 +0200
Subject: [PATCH v4 net-next 01/21] net: Introduce direct data placement tcp
 offload
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-2-borisp@mellanox.com>

This commit introduces direct data placement offload for TCP.
This capability is accompanied by new net_device operations that
configure hardware contexts. There is a context per socket, and a context per DDP
opreation. Additionally, a resynchronization routine is used to assist
hardware handle TCP OOO, and continue the offload.
Furthermore, we let the offloading driver advertise what is the max hw
sectors/segments.

Using this interface, the NIC hardware will scatter TCP payload directly
to the BIO pages according to the command_id.
To maintain the correctness of the network stack, the driver is expected
to construct SKBs that point to the BIO pages.

The SKB passed to the network stack from the driver
represents data as it is on the wire, while it is pointing
directly to data in destination buffers.
As a result, data from page frags should not be copied out to
the linear part. To avoid needless copies, such as when using
skb_condense, we mark the skb->ddp_crc bit. This bit will be
used to indicate both ddp and crc offload (next patch in series).

A follow-up patch will use this interface for DDP in NVMe-TCP.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/netdev_features.h    |   2 +
 include/linux/netdevice.h          |   5 ++
 include/linux/skbuff.h             |   4 +
 include/net/inet_connection_sock.h |   4 +
 include/net/tcp_ddp.h              | 136 +++++++++++++++++++++++++++++
 net/Kconfig                        |   9 ++
 net/core/skbuff.c                  |   8 +-
 net/ethtool/common.c               |   1 +
 net/ipv4/tcp_input.c               |   8 ++
 net/ipv4/tcp_ipv4.c                |   3 +
 net/ipv4/tcp_offload.c             |   3 +
 11 files changed, 182 insertions(+), 1 deletion(-)
 create mode 100644 include/net/tcp_ddp.h

diff --git a/include/linux/netdev_features.h b/include/linux/netdev_features.h
index c06d6aaba9df..7977371d2dd1 100644
--- a/include/linux/netdev_features.h
+++ b/include/linux/netdev_features.h
@@ -85,6 +85,7 @@ enum {
 
 	NETIF_F_HW_MACSEC_BIT,		/* Offload MACsec operations */
 	NETIF_F_GRO_UDP_FWD_BIT,	/* Allow UDP GRO for forwarding */
+	NETIF_F_HW_TCP_DDP_BIT,		/* TCP direct data placement offload */
 
 	/*
 	 * Add your fresh new feature above and remember to update
@@ -159,6 +160,7 @@ enum {
 #define NETIF_F_GSO_FRAGLIST	__NETIF_F(GSO_FRAGLIST)
 #define NETIF_F_HW_MACSEC	__NETIF_F(HW_MACSEC)
 #define NETIF_F_GRO_UDP_FWD	__NETIF_F(GRO_UDP_FWD)
+#define NETIF_F_HW_TCP_DDP	__NETIF_F(HW_TCP_DDP)
 
 /* Finds the next feature with the highest number of the range of start till 0.
  */
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index e9e7ada07ea1..bd28520e30f2 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -941,6 +941,7 @@ struct dev_ifalias {
 
 struct devlink;
 struct tlsdev_ops;
+struct tcp_ddp_dev_ops;
 
 struct netdev_name_node {
 	struct hlist_node hlist;
@@ -1942,6 +1943,10 @@ struct net_device {
 	const struct tlsdev_ops *tlsdev_ops;
 #endif
 
+#ifdef CONFIG_TCP_DDP
+	const struct tcp_ddp_dev_ops *tcp_ddp_ops;
+#endif
+
 	const struct header_ops *header_ops;
 
 	unsigned char		operstate;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 0a4e91a2f873..c7c88b2d0d47 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -684,6 +684,7 @@ typedef unsigned char *sk_buff_data_t;
  *		CHECKSUM_UNNECESSARY (max 3)
  *	@dst_pending_confirm: need to confirm neighbour
  *	@decrypted: Decrypted SKB
+ *	@ddp_crc: DDP or CRC offloaded
  *	@napi_id: id of the NAPI struct this skb came from
  *	@sender_cpu: (aka @napi_id) source CPU in XPS
  *	@secmark: security marking
@@ -860,6 +861,9 @@ struct sk_buff {
 #ifdef CONFIG_TLS_DEVICE
 	__u8			decrypted:1;
 #endif
+#ifdef CONFIG_TCP_DDP
+	__u8                    ddp_crc:1;
+#endif
 
 #ifdef CONFIG_NET_SCHED
 	__u16			tc_index;	/* traffic control index */
diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
index 10a625760de9..cb29417e8fd1 100644
--- a/include/net/inet_connection_sock.h
+++ b/include/net/inet_connection_sock.h
@@ -66,6 +66,8 @@ struct inet_connection_sock_af_ops {
  * @icsk_ulp_ops	   Pluggable ULP control hook
  * @icsk_ulp_data	   ULP private data
  * @icsk_clean_acked	   Clean acked data hook
+ * @icsk_ulp_ddp_ops	   Pluggable ULP direct data placement control hook
+ * @icsk_ulp_ddp_data	   ULP direct data placement private data
  * @icsk_listen_portaddr_node	hash to the portaddr listener hashtable
  * @icsk_ca_state:	   Congestion control state
  * @icsk_retransmits:	   Number of unrecovered [RTO] timeouts
@@ -96,6 +98,8 @@ struct inet_connection_sock {
 	const struct tcp_ulp_ops  *icsk_ulp_ops;
 	void __rcu		  *icsk_ulp_data;
 	void (*icsk_clean_acked)(struct sock *sk, u32 acked_seq);
+	const struct tcp_ddp_ulp_ops  *icsk_ulp_ddp_ops;
+	void __rcu		  *icsk_ulp_ddp_data;
 	struct hlist_node         icsk_listen_portaddr_node;
 	unsigned int		  (*icsk_sync_mss)(struct sock *sk, u32 pmtu);
 	__u8			  icsk_ca_state:5,
diff --git a/include/net/tcp_ddp.h b/include/net/tcp_ddp.h
new file mode 100644
index 000000000000..31e5b1a16d0f
--- /dev/null
+++ b/include/net/tcp_ddp.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * tcp_ddp.h
+ *	Author:	Boris Pismenny <borisp at mellanox.com>
+ *	Copyright (C) 2021 Mellanox Technologies.
+ */
+#ifndef _TCP_DDP_H
+#define _TCP_DDP_H
+
+#include <linux/netdevice.h>
+#include <net/inet_connection_sock.h>
+#include <net/sock.h>
+
+/* limits returned by the offload driver, zero means don't care */
+struct tcp_ddp_limits {
+	int	 max_ddp_sgl_len;
+};
+
+enum tcp_ddp_type {
+	TCP_DDP_NVME = 1,
+};
+
+/**
+ * struct tcp_ddp_config - Generic tcp ddp configuration: tcp ddp IO queue
+ * config implementations must use this as the first member.
+ * Add new instances of tcp_ddp_config below (nvme-tcp, etc.).
+ */
+struct tcp_ddp_config {
+	enum tcp_ddp_type    type;
+	unsigned char        buf[];
+};
+
+/**
+ * struct nvme_tcp_ddp_config - nvme tcp ddp configuration for an IO queue
+ *
+ * @pfv:        pdu version (e.g., NVME_TCP_PFV_1_0)
+ * @cpda:       controller pdu data alignmend (dwords, 0's based)
+ * @dgst:       digest types enabled.
+ *              The netdev will offload crc if ddp_crc is supported.
+ * @queue_size: number of nvme-tcp IO queue elements
+ * @queue_id:   queue identifier
+ * @cpu_io:     cpu core running the IO thread for this queue
+ */
+struct nvme_tcp_ddp_config {
+	struct tcp_ddp_config   cfg;
+
+	u16			pfv;
+	u8			cpda;
+	u8			dgst;
+	int			queue_size;
+	int			queue_id;
+	int			io_cpu;
+};
+
+/**
+ * struct tcp_ddp_io - tcp ddp configuration for an IO request.
+ *
+ * @command_id:  identifier on the wire associated with these buffers
+ * @nents:       number of entries in the sg_table
+ * @sg_table:    describing the buffers for this IO request
+ * @first_sgl:   first SGL in sg_table
+ */
+struct tcp_ddp_io {
+	u32			command_id;
+	int			nents;
+	struct sg_table		sg_table;
+	struct scatterlist	first_sgl[SG_CHUNK_SIZE];
+};
+
+/* struct tcp_ddp_dev_ops - operations used by an upper layer protocol to configure ddp offload
+ *
+ * @tcp_ddp_limits:    limit the number of scatter gather entries per IO.
+ *                     the device driver can use this to limit the resources allocated per queue.
+ * @tcp_ddp_sk_add:    add offload for the queue represennted by the socket+config pair.
+ *                     this function is used to configure either copy, crc or both offloads.
+ * @tcp_ddp_sk_del:    remove offload from the socket, and release any device related resources.
+ * @tcp_ddp_setup:     request copy offload for buffers associated with a command_id in tcp_ddp_io.
+ * @tcp_ddp_teardown:  release offload resources association between buffers and command_id in
+ *                     tcp_ddp_io.
+ * @tcp_ddp_resync:    respond to the driver's resync_request. Called only if resync is successful.
+ */
+struct tcp_ddp_dev_ops {
+	int (*tcp_ddp_limits)(struct net_device *netdev,
+			      struct tcp_ddp_limits *limits);
+	int (*tcp_ddp_sk_add)(struct net_device *netdev,
+			      struct sock *sk,
+			      struct tcp_ddp_config *config);
+	void (*tcp_ddp_sk_del)(struct net_device *netdev,
+			       struct sock *sk);
+	int (*tcp_ddp_setup)(struct net_device *netdev,
+			     struct sock *sk,
+			     struct tcp_ddp_io *io);
+	int (*tcp_ddp_teardown)(struct net_device *netdev,
+				struct sock *sk,
+				struct tcp_ddp_io *io,
+				void *ddp_ctx);
+	void (*tcp_ddp_resync)(struct net_device *netdev,
+			       struct sock *sk, u32 seq);
+};
+
+#define TCP_DDP_RESYNC_REQ BIT(0)
+
+/**
+ * struct tcp_ddp_ulp_ops - Interface to register uppper layer Direct Data Placement (DDP) TCP offload
+ */
+struct tcp_ddp_ulp_ops {
+	/* NIC requests ulp to indicate if @seq is the start of a message */
+	bool (*resync_request)(struct sock *sk, u32 seq, u32 flags);
+	/* NIC driver informs the ulp that ddp teardown is done - used for async completions*/
+	void (*ddp_teardown_done)(void *ddp_ctx);
+};
+
+/**
+ * struct tcp_ddp_ctx - Generic tcp ddp context: device driver per queue contexts must
+ * use this as the first member.
+ */
+struct tcp_ddp_ctx {
+	enum tcp_ddp_type    type;
+	unsigned char        buf[];
+};
+
+static inline struct tcp_ddp_ctx *tcp_ddp_get_ctx(const struct sock *sk)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+
+	return (__force struct tcp_ddp_ctx *)icsk->icsk_ulp_ddp_data;
+}
+
+static inline void tcp_ddp_set_ctx(struct sock *sk, void *ctx)
+{
+	struct inet_connection_sock *icsk = inet_csk(sk);
+
+	rcu_assign_pointer(icsk->icsk_ulp_ddp_data, ctx);
+}
+
+#endif //_TCP_DDP_H
diff --git a/net/Kconfig b/net/Kconfig
index f4c32d982af6..3876861cdc90 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -457,6 +457,15 @@ config ETHTOOL_NETLINK
 	  netlink. It provides better extensibility and some new features,
 	  e.g. notification messages.
 
+config TCP_DDP
+	bool "TCP direct data placement offload"
+	default n
+	help
+	  Direct Data Placement (DDP) offload for TCP enables ULP, such as
+	  NVMe-TCP/iSCSI, to request the NIC to place TCP payload data
+	  of a command response directly into kernel pages.
+
+
 endif   # if NET
 
 # Used by archs to tell that they support BPF JIT compiler plus which flavour.
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index d380c7b5a12d..78472520eb82 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -69,6 +69,7 @@
 #include <net/xfrm.h>
 #include <net/mpls.h>
 #include <net/mptcp.h>
+#include <net/tcp_ddp.h>
 
 #include <linux/uaccess.h>
 #include <trace/events/skb.h>
@@ -6191,9 +6192,14 @@ EXPORT_SYMBOL(pskb_extract);
  */
 void skb_condense(struct sk_buff *skb)
 {
+	bool is_ddp = false;
+
+#ifdef CONFIG_TCP_DDP
+	is_ddp = skb->ddp_crc;
+#endif
 	if (skb->data_len) {
 		if (skb->data_len > skb->end - skb->tail ||
-		    skb_cloned(skb))
+		    skb_cloned(skb) || is_ddp)
 			return;
 
 		/* Nice, we can free page frag(s) right now */
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index 835b9bba3e7e..2878a5613e72 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -69,6 +69,7 @@ const char netdev_features_strings[NETDEV_FEATURE_COUNT][ETH_GSTRING_LEN] = {
 	[NETIF_F_GRO_FRAGLIST_BIT] =	 "rx-gro-list",
 	[NETIF_F_HW_MACSEC_BIT] =	 "macsec-hw-offload",
 	[NETIF_F_GRO_UDP_FWD_BIT] =	 "rx-udp-gro-forwarding",
+	[NETIF_F_HW_TCP_DDP_BIT] =	 "tcp-ddp-offload",
 };
 
 const char
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index a8f8f9815953..0ae1ffca090d 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -5149,6 +5149,9 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 		memcpy(nskb->cb, skb->cb, sizeof(skb->cb));
 #ifdef CONFIG_TLS_DEVICE
 		nskb->decrypted = skb->decrypted;
+#endif
+#ifdef CONFIG_TCP_DDP
+		nskb->ddp_crc = skb->ddp_crc;
 #endif
 		TCP_SKB_CB(nskb)->seq = TCP_SKB_CB(nskb)->end_seq = start;
 		if (list)
@@ -5182,6 +5185,11 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 #ifdef CONFIG_TLS_DEVICE
 				if (skb->decrypted != nskb->decrypted)
 					goto end;
+#endif
+#ifdef CONFIG_TCP_DDP
+
+				if (skb->ddp_crc != nskb->ddp_crc)
+					goto end;
 #endif
 			}
 		}
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 611039207d30..676bc1584356 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1813,6 +1813,9 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb)
 	      TCP_SKB_CB(skb)->tcp_flags) & (TCPHDR_ECE | TCPHDR_CWR)) ||
 #ifdef CONFIG_TLS_DEVICE
 	    tail->decrypted != skb->decrypted ||
+#endif
+#ifdef CONFIG_TCP_DDP
+	    tail->ddp_crc != skb->ddp_crc ||
 #endif
 	    thtail->doff != th->doff ||
 	    memcmp(thtail + 1, th + 1, hdrlen - sizeof(*th)))
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index e09147ac9a99..3ce196375d94 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -262,6 +262,9 @@ struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb)
 #ifdef CONFIG_TLS_DEVICE
 	flush |= p->decrypted ^ skb->decrypted;
 #endif
+#ifdef CONFIG_TCP_DDP
+	flush |= p->ddp_crc ^ skb->ddp_crc;
+#endif
 
 	if (flush || skb_gro_receive(p, skb)) {
 		mss = 1;
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:25 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:25 +0200
Subject: [PATCH v4 net-next 02/21] net: Introduce crc offload for tcp ddp ulp
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-3-borisp@mellanox.com>

This commit introduces support for CRC offload to direct data placement
ULP on the receive side. Both DDP and CRC share a common API to
initialize the offload for a TCP socket. But otherwise, both can
be executed independently.

On the receive side, CRC offload relies on a new SKB bit that
indicates that no CRC error was encountered while processing this packet.
If all packets of a ULP message have this bit set, then the CRC
verification for the message can be skipped, as hardware already checked
it.

The following patches will set and use this bit to perform NVME-TCP
CRC offload.

A subsequent series, will add NVMe-TCP transmit side CRC support.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
---
 include/linux/netdev_features.h | 2 ++
 include/linux/netdevice.h       | 2 +-
 include/linux/skbuff.h          | 2 +-
 net/Kconfig                     | 8 ++++++++
 net/ethtool/common.c            | 1 +
 net/ipv4/tcp_input.c            | 4 ++--
 net/ipv4/tcp_ipv4.c             | 2 +-
 net/ipv4/tcp_offload.c          | 2 +-
 8 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/include/linux/netdev_features.h b/include/linux/netdev_features.h
index 7977371d2dd1..77fb2cb99b78 100644
--- a/include/linux/netdev_features.h
+++ b/include/linux/netdev_features.h
@@ -86,6 +86,7 @@ enum {
 	NETIF_F_HW_MACSEC_BIT,		/* Offload MACsec operations */
 	NETIF_F_GRO_UDP_FWD_BIT,	/* Allow UDP GRO for forwarding */
 	NETIF_F_HW_TCP_DDP_BIT,		/* TCP direct data placement offload */
+	NETIF_F_HW_TCP_DDP_CRC_RX_BIT,	/* TCP DDP CRC RX offload */
 
 	/*
 	 * Add your fresh new feature above and remember to update
@@ -161,6 +162,7 @@ enum {
 #define NETIF_F_HW_MACSEC	__NETIF_F(HW_MACSEC)
 #define NETIF_F_GRO_UDP_FWD	__NETIF_F(GRO_UDP_FWD)
 #define NETIF_F_HW_TCP_DDP	__NETIF_F(HW_TCP_DDP)
+#define NETIF_F_HW_TCP_DDP_CRC_RX	__NETIF_F(HW_TCP_DDP_CRC_RX)
 
 /* Finds the next feature with the highest number of the range of start till 0.
  */
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index bd28520e30f2..1065ddaa8e6b 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1943,7 +1943,7 @@ struct net_device {
 	const struct tlsdev_ops *tlsdev_ops;
 #endif
 
-#ifdef CONFIG_TCP_DDP
+#if IS_ENABLED(CONFIG_TCP_DDP) || IS_ENABLED(CONFIG_TCP_DDP_CRC)
 	const struct tcp_ddp_dev_ops *tcp_ddp_ops;
 #endif
 
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index c7c88b2d0d47..0d1be25574cc 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -861,7 +861,7 @@ struct sk_buff {
 #ifdef CONFIG_TLS_DEVICE
 	__u8			decrypted:1;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 	__u8                    ddp_crc:1;
 #endif
 
diff --git a/net/Kconfig b/net/Kconfig
index 3876861cdc90..80ed9f038968 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -465,6 +465,14 @@ config TCP_DDP
 	  NVMe-TCP/iSCSI, to request the NIC to place TCP payload data
 	  of a command response directly into kernel pages.
 
+config TCP_DDP_CRC
+	bool "TCP direct data placement CRC offload"
+	default n
+	help
+	  Direct Data Placement (DDP) CRC32C offload for TCP enables ULP, such as
+	  NVMe-TCP/iSCSI, to request the NIC to calculate/verify the data digest
+	  of commands as they go through the NIC. Thus avoiding the costly
+	  per-byte overhead.
 
 endif   # if NET
 
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index 2878a5613e72..7ac78b0a90b0 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -70,6 +70,7 @@ const char netdev_features_strings[NETDEV_FEATURE_COUNT][ETH_GSTRING_LEN] = {
 	[NETIF_F_HW_MACSEC_BIT] =	 "macsec-hw-offload",
 	[NETIF_F_GRO_UDP_FWD_BIT] =	 "rx-udp-gro-forwarding",
 	[NETIF_F_HW_TCP_DDP_BIT] =	 "tcp-ddp-offload",
+	[NETIF_F_HW_TCP_DDP_CRC_RX_BIT] =	 "tcp-ddp-crc-rx-offload",
 };
 
 const char
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 0ae1ffca090d..cd490bd548a3 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -5150,7 +5150,7 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 #ifdef CONFIG_TLS_DEVICE
 		nskb->decrypted = skb->decrypted;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 		nskb->ddp_crc = skb->ddp_crc;
 #endif
 		TCP_SKB_CB(nskb)->seq = TCP_SKB_CB(nskb)->end_seq = start;
@@ -5186,7 +5186,7 @@ tcp_collapse(struct sock *sk, struct sk_buff_head *list, struct rb_root *root,
 				if (skb->decrypted != nskb->decrypted)
 					goto end;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 
 				if (skb->ddp_crc != nskb->ddp_crc)
 					goto end;
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 676bc1584356..8750646f8621 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1814,7 +1814,7 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb)
 #ifdef CONFIG_TLS_DEVICE
 	    tail->decrypted != skb->decrypted ||
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 	    tail->ddp_crc != skb->ddp_crc ||
 #endif
 	    thtail->doff != th->doff ||
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index 3ce196375d94..9e43a044e730 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -262,7 +262,7 @@ struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb)
 #ifdef CONFIG_TLS_DEVICE
 	flush |= p->decrypted ^ skb->decrypted;
 #endif
-#ifdef CONFIG_TCP_DDP
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 	flush |= p->ddp_crc ^ skb->ddp_crc;
 #endif
 
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:26 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:26 +0200
Subject: [PATCH v4 net-next  03/21] iov_iter: DDP copy to iter/pages
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-4-borisp@mellanox.com>

When using direct data placement (DDP) the NIC writes some of the payload
directly to the destination buffer, and constructs SKBs such that they
point to this data. To skip copies when SKB data already resides in the
destination we use the newly introduced routines in this commit, which
check if (src == dst), and skip the copy when that's true.

As the current user for these routines is in the block layer (nvme-tcp),
then we only apply the change for bio_vec. Other routines use the normal
methods for copying.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/uio.h | 17 +++++++++++++++
 lib/iov_iter.c      | 53 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+)

diff --git a/include/linux/uio.h b/include/linux/uio.h
index 72d88566694e..8438e6d5ac25 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -123,6 +123,9 @@ size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes,
 			 struct iov_iter *i);
 
 size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i);
+#ifdef CONFIG_TCP_DDP
+size_t _ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i);
+#endif
 size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i);
 bool _copy_from_iter_full(void *addr, size_t bytes, struct iov_iter *i);
 size_t _copy_from_iter_nocache(void *addr, size_t bytes, struct iov_iter *i);
@@ -137,6 +140,16 @@ size_t copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
 		return _copy_to_iter(addr, bytes, i);
 }
 
+#ifdef CONFIG_TCP_DDP
+static __always_inline __must_check
+size_t ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
+{
+	if (unlikely(!check_copy_size(addr, bytes, true)))
+		return 0;
+	return _ddp_copy_to_iter(addr, bytes, i);
+}
+#endif
+
 static __always_inline __must_check
 size_t copy_from_iter(void *addr, size_t bytes, struct iov_iter *i)
 {
@@ -265,6 +278,10 @@ size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, struct io
 bool csum_and_copy_from_iter_full(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i);
 size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
 		struct iov_iter *i);
+#ifdef CONFIG_TCP_DDP
+size_t ddp_hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
+		struct iov_iter *i);
+#endif
 
 struct iovec *iovec_from_user(const struct iovec __user *uvector,
 		unsigned long nr_segs, unsigned long fast_segs,
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index a21e6a5792c5..c5990275e583 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -473,6 +473,18 @@ static void memcpy_from_page(char *to, struct page *page, size_t offset, size_t
 	kunmap_atomic(from);
 }
 
+#ifdef CONFIG_TCP_DDP
+static void ddp_memcpy_to_page(struct page *page, size_t offset, const char *from, size_t len)
+{
+	char *to = kmap_atomic(page);
+
+	if (to + offset != from)
+		memcpy(to + offset, from, len);
+
+	kunmap_atomic(to);
+}
+#endif
+
 static void memcpy_to_page(struct page *page, size_t offset, const char *from, size_t len)
 {
 	char *to = kmap_atomic(page);
@@ -625,6 +637,26 @@ static size_t csum_and_copy_to_pipe_iter(const void *addr, size_t bytes,
 	return bytes;
 }
 
+#ifdef CONFIG_TCP_DDP
+size_t _ddp_copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
+{
+	const char *from = addr;
+	if (unlikely(iov_iter_is_pipe(i)))
+		return copy_pipe_to_iter(addr, bytes, i);
+	if (iter_is_iovec(i))
+		might_fault();
+	iterate_and_advance(i, bytes, v,
+		copyout(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len),
+		ddp_memcpy_to_page(v.bv_page, v.bv_offset,
+				   (from += v.bv_len) - v.bv_len, v.bv_len),
+		memcpy(v.iov_base, (from += v.iov_len) - v.iov_len, v.iov_len)
+		)
+
+	return bytes;
+}
+EXPORT_SYMBOL(_ddp_copy_to_iter);
+#endif
+
 size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i)
 {
 	const char *from = addr;
@@ -1566,6 +1598,27 @@ size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *csump,
 }
 EXPORT_SYMBOL(csum_and_copy_to_iter);
 
+#ifdef CONFIG_TCP_DDP
+size_t ddp_hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
+		struct iov_iter *i)
+{
+#ifdef CONFIG_CRYPTO_HASH
+	struct ahash_request *hash = hashp;
+	struct scatterlist sg;
+	size_t copied;
+
+	copied = ddp_copy_to_iter(addr, bytes, i);
+	sg_init_one(&sg, addr, copied);
+	ahash_request_set_crypt(hash, &sg, NULL, copied);
+	crypto_ahash_update(hash);
+	return copied;
+#else
+	return 0;
+#endif
+}
+EXPORT_SYMBOL(ddp_hash_and_copy_to_iter);
+#endif
+
 size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp,
 		struct iov_iter *i)
 {
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:27 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:27 +0200
Subject: [PATCH v4 net-next 04/21] net: skb copy(+hash) iterators for DDP
 offloads
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-5-borisp@mellanox.com>

This commit introduces new functions to support direct data placement
(DDP) NIC offloads that avoid copying data from SKBs.

Later patches will use this for nvme-tcp DDP offload.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/skbuff.h |  9 ++++++++
 net/core/datagram.c    | 48 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 57 insertions(+)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 0d1be25574cc..0470eb3c9976 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -3594,6 +3594,10 @@ __poll_t datagram_poll(struct file *file, struct socket *sock,
 			   struct poll_table_struct *wait);
 int skb_copy_datagram_iter(const struct sk_buff *from, int offset,
 			   struct iov_iter *to, int size);
+#ifdef CONFIG_TCP_DDP
+int skb_ddp_copy_datagram_iter(const struct sk_buff *from, int offset,
+			       struct iov_iter *to, int size);
+#endif
 static inline int skb_copy_datagram_msg(const struct sk_buff *from, int offset,
 					struct msghdr *msg, int size)
 {
@@ -3604,6 +3608,11 @@ int skb_copy_and_csum_datagram_msg(struct sk_buff *skb, int hlen,
 int skb_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
 			   struct iov_iter *to, int len,
 			   struct ahash_request *hash);
+#ifdef CONFIG_TCP_DDP
+int skb_ddp_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
+					struct iov_iter *to, int len,
+					struct ahash_request *hash);
+#endif
 int skb_copy_datagram_from_iter(struct sk_buff *skb, int offset,
 				 struct iov_iter *from, int len);
 int zerocopy_sg_from_iter(struct sk_buff *skb, struct iov_iter *frm);
diff --git a/net/core/datagram.c b/net/core/datagram.c
index 81809fa735a7..f386d4d82b1b 100644
--- a/net/core/datagram.c
+++ b/net/core/datagram.c
@@ -495,6 +495,27 @@ static int __skb_datagram_iter(const struct sk_buff *skb, int offset,
 	return 0;
 }
 
+#ifdef CONFIG_TCP_DDP
+/**
+ *	skb_ddp_copy_and_hash_datagram_iter - Copies datagrams from skb frags to
+ *	an iterator and update a hash. If the iterator and skb frag point to the
+ *	same page and offset, then the copy is skipped.
+ *	@skb: buffer to copy
+ *	@offset: offset in the buffer to start copying from
+ *	@to: iovec iterator to copy to
+ *	@len: amount of data to copy from buffer to iovec
+ *      @hash: hash request to update
+ */
+int skb_ddp_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
+					struct iov_iter *to, int len,
+					struct ahash_request *hash)
+{
+	return __skb_datagram_iter(skb, offset, to, len, true,
+			ddp_hash_and_copy_to_iter, hash);
+}
+EXPORT_SYMBOL(skb_ddp_copy_and_hash_datagram_iter);
+#endif
+
 /**
  *	skb_copy_and_hash_datagram_iter - Copy datagram to an iovec iterator
  *          and update a hash.
@@ -513,6 +534,33 @@ int skb_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset,
 }
 EXPORT_SYMBOL(skb_copy_and_hash_datagram_iter);
 
+#ifdef CONFIG_TCP_DDP
+static size_t simple_ddp_copy_to_iter(const void *addr, size_t bytes,
+				      void *data __always_unused,
+				      struct iov_iter *i)
+{
+	return ddp_copy_to_iter(addr, bytes, i);
+}
+
+/**
+ *	skb_ddp_copy_datagram_iter - Copies datagrams from skb frags to an
+ *	iterator. If the iterator and skb frag point to the same page and
+ *	offset, then the copy is skipped.
+ *	@skb: buffer to copy
+ *	@offset: offset in the buffer to start copying from
+ *	@to: iovec iterator to copy to
+ *	@len: amount of data to copy from buffer to iovec
+ */
+int skb_ddp_copy_datagram_iter(const struct sk_buff *skb, int offset,
+			       struct iov_iter *to, int len)
+{
+	trace_skb_copy_datagram_iovec(skb, len);
+	return __skb_datagram_iter(skb, offset, to, len, false,
+			simple_ddp_copy_to_iter, NULL);
+}
+EXPORT_SYMBOL(skb_ddp_copy_datagram_iter);
+#endif
+
 static size_t simple_copy_to_iter(const void *addr, size_t bytes,
 		void *data __always_unused, struct iov_iter *i)
 {
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:28 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:28 +0200
Subject: [PATCH v4 net-next  05/21] net/tls: expose get_netdev_for_sock
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-6-borisp@mellanox.com>

get_netdev_for_sock is a utility that is used to obtain
the net_device structure from a connected socket.

Later patches will use this for nvme-tcp DDP and DDP CRC offloads.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Reviewed-by: Sagi Grimberg <sagi at grimberg.me>
---
 include/net/sock.h   | 17 +++++++++++++++++
 net/tls/tls_device.c | 20 ++------------------
 2 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index 690e496a0e79..f6d70f689bd1 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2732,4 +2732,21 @@ void sock_set_sndtimeo(struct sock *sk, s64 secs);
 
 int sock_bind_add(struct sock *sk, struct sockaddr *addr, int addr_len);
 
+/* Assume that the socket is already connected */
+static inline struct net_device *get_netdev_for_sock(struct sock *sk, bool hold)
+{
+	struct dst_entry *dst = sk_dst_get(sk);
+	struct net_device *netdev = NULL;
+
+	if (likely(dst)) {
+		netdev = dst->dev;
+		if (hold)
+			dev_hold(netdev);
+	}
+
+	dst_release(dst);
+
+	return netdev;
+}
+
 #endif	/* _SOCK_H */
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index d9cd229aa111..792c0a477850 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -106,22 +106,6 @@ static void tls_device_queue_ctx_destruction(struct tls_context *ctx)
 	spin_unlock_irqrestore(&tls_device_lock, flags);
 }
 
-/* We assume that the socket is already connected */
-static struct net_device *get_netdev_for_sock(struct sock *sk)
-{
-	struct dst_entry *dst = sk_dst_get(sk);
-	struct net_device *netdev = NULL;
-
-	if (likely(dst)) {
-		netdev = netdev_sk_get_lowest_dev(dst->dev, sk);
-		dev_hold(netdev);
-	}
-
-	dst_release(dst);
-
-	return netdev;
-}
-
 static void destroy_record(struct tls_record_info *record)
 {
 	int i;
@@ -1106,7 +1090,7 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
 	if (skb)
 		TCP_SKB_CB(skb)->eor = 1;
 
-	netdev = get_netdev_for_sock(sk);
+	netdev = get_netdev_for_sock(sk, true);
 	if (!netdev) {
 		pr_err_ratelimited("%s: netdev not found\n", __func__);
 		rc = -EINVAL;
@@ -1182,7 +1166,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
 	if (ctx->crypto_recv.info.version != TLS_1_2_VERSION)
 		return -EOPNOTSUPP;
 
-	netdev = get_netdev_for_sock(sk);
+	netdev = get_netdev_for_sock(sk, true);
 	if (!netdev) {
 		pr_err_ratelimited("%s: netdev not found\n", __func__);
 		return -EINVAL;
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:29 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:29 +0200
Subject: [PATCH v4 net-next  06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-7-borisp@mellanox.com>

This commit introduces direct data placement offload to NVME
TCP. There is a context per queue, which is established after the
handshake using the tcp_ddp_sk_add/del NDOs.

Additionally, a resynchronization routine is used to assist
hardware recovery from TCP OOO, and continue the offload.
Resynchronization operates as follows:

1. TCP OOO causes the NIC HW to stop the offload

2. NIC HW identifies a PDU header at some TCP sequence number,
and asks NVMe-TCP to confirm it.
This request is delivered from the NIC driver to NVMe-TCP by first
finding the socket for the packet that triggered the request, and
then finding the nvme_tcp_queue that is used by this routine.
Finally, the request is recorded in the nvme_tcp_queue.

3. When NVMe-TCP observes the requested TCP sequence, it will compare
it with the PDU header TCP sequence, and report the result to the
NIC driver (tcp_ddp_resync), which will update the HW,
and resume offload when all is successful.

Furthermore, we let the offloading driver advertise what is the max hw
sectors/segments via tcp_ddp_limits.

A follow-up patch introduces the data-path changes required for this
offload.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 177 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 175 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 881d28eb15e9..36de4391ba76 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -14,6 +14,7 @@
 #include <linux/blk-mq.h>
 #include <crypto/hash.h>
 #include <net/busy_poll.h>
+#include <net/tcp_ddp.h>
 
 #include "nvme.h"
 #include "fabrics.h"
@@ -62,6 +63,7 @@ enum nvme_tcp_queue_flags {
 	NVME_TCP_Q_ALLOCATED	= 0,
 	NVME_TCP_Q_LIVE		= 1,
 	NVME_TCP_Q_POLLING	= 2,
+	NVME_TCP_Q_OFF_DDP	= 3,
 };
 
 enum nvme_tcp_recv_state {
@@ -111,6 +113,8 @@ struct nvme_tcp_queue {
 	void (*state_change)(struct sock *);
 	void (*data_ready)(struct sock *);
 	void (*write_space)(struct sock *);
+
+	atomic64_t  resync_req;
 };
 
 struct nvme_tcp_ctrl {
@@ -129,6 +133,8 @@ struct nvme_tcp_ctrl {
 	struct delayed_work	connect_work;
 	struct nvme_tcp_request async_req;
 	u32			io_queues[HCTX_MAX_TYPES];
+
+	struct net_device       *offloading_netdev;
 };
 
 static LIST_HEAD(nvme_tcp_ctrl_list);
@@ -223,6 +229,164 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
 	return nvme_tcp_pdu_data_left(req) <= len;
 }
 
+#ifdef CONFIG_TCP_DDP
+
+static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
+static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
+	.resync_request		= nvme_tcp_resync_request,
+};
+
+static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
+{
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+	struct nvme_tcp_ddp_config config = {};
+	int ret;
+
+	if (!(netdev->features & NETIF_F_HW_TCP_DDP))
+		return -EOPNOTSUPP;
+
+	config.cfg.type		= TCP_DDP_NVME;
+	config.pfv		= NVME_TCP_PFV_1_0;
+	config.cpda		= 0;
+	config.dgst		= queue->hdr_digest ?
+		NVME_TCP_HDR_DIGEST_ENABLE : 0;
+	config.dgst		|= queue->data_digest ?
+		NVME_TCP_DATA_DIGEST_ENABLE : 0;
+	config.queue_size	= queue->queue_size;
+	config.queue_id		= nvme_tcp_queue_id(queue);
+	config.io_cpu		= queue->io_cpu;
+
+	dev_hold(netdev); /* put by unoffload_socket */
+	ret = netdev->tcp_ddp_ops->tcp_ddp_sk_add(netdev,
+						  queue->sock->sk,
+						  &config.cfg);
+	if (ret) {
+		dev_put(netdev);
+		return ret;
+	}
+
+	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = &nvme_tcp_ddp_ulp_ops;
+	if (netdev->features & NETIF_F_HW_TCP_DDP)
+		set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
+
+	return ret;
+}
+
+static void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
+{
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+
+	if (!netdev) {
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
+		return;
+	}
+
+	netdev->tcp_ddp_ops->tcp_ddp_sk_del(netdev, queue->sock->sk);
+
+	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = NULL;
+	dev_put(netdev); /* held by offload_socket */
+}
+
+static int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
+{
+	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
+	struct tcp_ddp_limits limits;
+	int ret = 0;
+
+	if (!netdev) {
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
+		queue->ctrl->offloading_netdev = NULL;
+		return -ENODEV;
+	}
+
+	if (netdev->features & NETIF_F_HW_TCP_DDP &&
+	    netdev->tcp_ddp_ops &&
+	    netdev->tcp_ddp_ops->tcp_ddp_limits)
+		ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
+	else
+		ret = -EOPNOTSUPP;
+
+	if (!ret) {
+		queue->ctrl->offloading_netdev = netdev;
+		dev_dbg_ratelimited(queue->ctrl->ctrl.device,
+				    "netdev %s offload limits: max_ddp_sgl_len %d\n",
+				    netdev->name, limits.max_ddp_sgl_len);
+		queue->ctrl->ctrl.max_segments = limits.max_ddp_sgl_len;
+		queue->ctrl->ctrl.max_hw_sectors =
+			limits.max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
+	} else {
+		queue->ctrl->offloading_netdev = NULL;
+	}
+
+	/* release the device as no offload context is established yet. */
+	dev_put(netdev);
+
+	return ret;
+}
+
+static void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
+				     struct sk_buff *skb, unsigned int offset)
+{
+	u64 pdu_seq = TCP_SKB_CB(skb)->seq + offset - queue->pdu_offset;
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+	u64 resync_val;
+	u32 resync_seq;
+
+	resync_val = atomic64_read(&queue->resync_req);
+	/* Lower 32 bit flags. Check validity of the request */
+	if ((resync_val & TCP_DDP_RESYNC_REQ) == 0)
+		return;
+
+	/* Obtain and check requested sequence number: is this PDU header before the request? */
+	resync_seq = resync_val >> 32;
+	if (before(pdu_seq, resync_seq))
+		return;
+
+	if (unlikely(!netdev)) {
+		pr_info_ratelimited("%s: netdev not found\n", __func__);
+		return;
+	}
+
+	/**
+	 * The atomic operation gurarantees that we don't miss any NIC driver
+	 * resync requests submitted after the above checks.
+	 */
+	if (atomic64_cmpxchg(&queue->resync_req, resync_val,
+			     resync_val & ~TCP_DDP_RESYNC_REQ))
+		netdev->tcp_ddp_ops->tcp_ddp_resync(netdev, queue->sock->sk, pdu_seq);
+}
+
+static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
+{
+	struct nvme_tcp_queue *queue = sk->sk_user_data;
+
+	atomic64_set(&queue->resync_req,
+		     (((uint64_t)seq << 32) | flags));
+
+	return true;
+}
+
+#else
+
+static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
+{
+	return -EINVAL;
+}
+
+static void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
+{}
+
+static int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
+{
+	return -EINVAL;
+}
+
+static void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
+				     struct sk_buff *skb, unsigned int offset)
+{}
+
+#endif
+
 static void nvme_tcp_init_iter(struct nvme_tcp_request *req,
 		unsigned int dir)
 {
@@ -638,6 +802,9 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 	size_t rcv_len = min_t(size_t, *len, queue->pdu_remaining);
 	int ret;
 
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+		nvme_tcp_resync_response(queue, skb, *offset);
+
 	ret = skb_copy_bits(skb, *offset,
 		&pdu[queue->pdu_offset], rcv_len);
 	if (unlikely(ret))
@@ -1532,6 +1699,9 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
 	kernel_sock_shutdown(queue->sock, SHUT_RDWR);
 	nvme_tcp_restore_sock_calls(queue);
 	cancel_work_sync(&queue->io_work);
+
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+		nvme_tcp_unoffload_socket(queue);
 }
 
 static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
@@ -1550,10 +1720,13 @@ static int nvme_tcp_start_queue(struct nvme_ctrl *nctrl, int idx)
 	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
 	int ret;
 
-	if (idx)
+	if (idx) {
 		ret = nvmf_connect_io_queue(nctrl, idx, false);
-	else
+		nvme_tcp_offload_socket(&ctrl->queues[idx]);
+	} else {
 		ret = nvmf_connect_admin_queue(nctrl);
+		nvme_tcp_offload_limits(&ctrl->queues[idx]);
+	}
 
 	if (!ret) {
 		set_bit(NVME_TCP_Q_LIVE, &ctrl->queues[idx].flags);
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:30 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:30 +0200
Subject: [PATCH v4 net-next  07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-8-borisp@mellanox.com>

Introduce the NVMe-TCP DDP data-path offload.
Using this interface, the NIC hardware will scatter TCP payload directly
to the BIO pages according to the command_id in the PDU.
To maintain the correctness of the network stack, the driver is expected
to construct SKBs that point to the BIO pages.

The data-path interface contains two routines: tcp_ddp_setup/teardown.
The setup provides the mapping from command_id to the request buffers,
while the teardown removes this mapping.

For efficiency, we introduce an asynchronous nvme completion, which is
split between NVMe-TCP and the NIC driver as follows:
NVMe-TCP performs the specific completion, while NIC driver performs the
generic mq_blk completion.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 158 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 146 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 36de4391ba76..188e26ab7116 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -57,6 +57,11 @@ struct nvme_tcp_request {
 	size_t			offset;
 	size_t			data_sent;
 	enum nvme_tcp_send_state state;
+
+	bool			offloaded;
+	struct tcp_ddp_io	ddp;
+	__le16			status;
+	union nvme_result	result;
 };
 
 enum nvme_tcp_queue_flags {
@@ -229,13 +234,82 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
 	return nvme_tcp_pdu_data_left(req) <= len;
 }
 
+static int nvme_tcp_req_map_sg(struct nvme_tcp_request *req, struct request *rq)
+{
+	int ret;
+
+	req->ddp.sg_table.sgl = req->ddp.first_sgl;
+	ret = sg_alloc_table_chained(&req->ddp.sg_table, blk_rq_nr_phys_segments(rq),
+				     req->ddp.sg_table.sgl, SG_CHUNK_SIZE);
+	if (ret)
+		return -ENOMEM;
+	req->ddp.nents = blk_rq_map_sg(rq->q, rq, req->ddp.sg_table.sgl);
+	return 0;
+}
+
 #ifdef CONFIG_TCP_DDP
 
 static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
+static void nvme_tcp_ddp_teardown_done(void *ddp_ctx);
 static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
 	.resync_request		= nvme_tcp_resync_request,
+	.ddp_teardown_done	= nvme_tcp_ddp_teardown_done,
 };
 
+static int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
+				 u16 command_id,
+				 struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+	int ret;
+
+	if (unlikely(!netdev)) {
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
+		return -EINVAL;
+	}
+
+	ret = netdev->tcp_ddp_ops->tcp_ddp_teardown(netdev, queue->sock->sk,
+						    &req->ddp, rq);
+	sg_free_table_chained(&req->ddp.sg_table, SG_CHUNK_SIZE);
+	return ret;
+}
+
+static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
+{
+	struct request *rq = ddp_ctx;
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+
+	if (!nvme_try_complete_req(rq, req->status, req->result))
+		nvme_complete_rq(rq);
+}
+
+static int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
+			      u16 command_id,
+			      struct request *rq)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+	struct net_device *netdev = queue->ctrl->offloading_netdev;
+	int ret;
+
+	if (unlikely(!netdev)) {
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
+		return -EINVAL;
+	}
+
+	req->ddp.command_id = command_id;
+	ret = nvme_tcp_req_map_sg(req, rq);
+	if (ret)
+		return -ENOMEM;
+
+	ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
+						 queue->sock->sk,
+						 &req->ddp);
+	if (!ret)
+		req->offloaded = true;
+	return ret;
+}
+
 static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 {
 	struct net_device *netdev = queue->ctrl->offloading_netdev;
@@ -343,7 +417,7 @@ static void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
 		return;
 
 	if (unlikely(!netdev)) {
-		pr_info_ratelimited("%s: netdev not found\n", __func__);
+		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
 		return;
 	}
 
@@ -368,6 +442,20 @@ static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags)
 
 #else
 
+static int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
+			      u16 command_id,
+			      struct request *rq)
+{
+	return -EINVAL;
+}
+
+static int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
+				 u16 command_id,
+				 struct request *rq)
+{
+	return -EINVAL;
+}
+
 static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 {
 	return -EINVAL;
@@ -643,6 +731,24 @@ static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
 	queue_work(nvme_reset_wq, &to_tcp_ctrl(ctrl)->err_work);
 }
 
+static void nvme_tcp_complete_request(struct request *rq,
+				      __le16 status,
+				      union nvme_result result,
+				      __u16 command_id)
+{
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+	struct nvme_tcp_queue *queue = req->queue;
+
+	if (req->offloaded) {
+		req->status = status;
+		req->result = result;
+		nvme_tcp_teardown_ddp(queue, command_id, rq);
+	} else {
+		if (!nvme_try_complete_req(rq, status, result))
+			nvme_complete_rq(rq);
+	}
+}
+
 static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		struct nvme_completion *cqe)
 {
@@ -657,10 +763,8 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 		return -EINVAL;
 	}
 
-	if (!nvme_try_complete_req(rq, cqe->status, cqe->result))
-		nvme_complete_rq(rq);
+	nvme_tcp_complete_request(rq, cqe->status, cqe->result, cqe->command_id);
 	queue->nr_cqe++;
-
 	return 0;
 }
 
@@ -849,10 +953,39 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 
 static inline void nvme_tcp_end_request(struct request *rq, u16 status)
 {
+	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
+	struct nvme_tcp_queue *queue = req->queue;
+	struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
 	union nvme_result res = {};
 
-	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
-		nvme_complete_rq(rq);
+	nvme_tcp_complete_request(rq, cpu_to_le16(status << 1), res, pdu->command_id);
+}
+
+
+static int nvme_tcp_consume_skb(struct nvme_tcp_queue *queue, struct sk_buff *skb,
+				unsigned int *offset, struct iov_iter *iter, int recv_len)
+{
+	int ret;
+
+#ifdef CONFIG_TCP_DDP
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
+		if (queue->data_digest)
+			ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset, iter, recv_len,
+					queue->rcv_hash);
+		else
+			ret = skb_ddp_copy_datagram_iter(skb, *offset, iter, recv_len);
+	} else {
+#endif
+		if (queue->data_digest)
+			ret = skb_copy_and_hash_datagram_iter(skb, *offset, iter, recv_len,
+					queue->rcv_hash);
+		else
+			ret = skb_copy_datagram_iter(skb, *offset, iter, recv_len);
+#ifdef CONFIG_TCP_DDP
+	}
+#endif
+
+	return ret;
 }
 
 static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
@@ -899,12 +1032,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 		recv_len = min_t(size_t, recv_len,
 				iov_iter_count(&req->iter));
 
-		if (queue->data_digest)
-			ret = skb_copy_and_hash_datagram_iter(skb, *offset,
-				&req->iter, recv_len, queue->rcv_hash);
-		else
-			ret = skb_copy_datagram_iter(skb, *offset,
-					&req->iter, recv_len);
+		ret = nvme_tcp_consume_skb(queue, skb, offset, &req->iter, recv_len);
 		if (ret) {
 			dev_err(queue->ctrl->ctrl.device,
 				"queue %d failed to copy request %#x data",
@@ -1128,6 +1256,7 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
 	bool inline_data = nvme_tcp_has_inline_data(req);
 	u8 hdgst = nvme_tcp_hdgst_len(queue);
 	int len = sizeof(*pdu) + hdgst - req->offset;
+	struct request *rq = blk_mq_rq_from_pdu(req);
 	int flags = MSG_DONTWAIT;
 	int ret;
 
@@ -1136,6 +1265,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
 	else
 		flags |= MSG_EOR;
 
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) &&
+	    blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
+		nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);
+
 	if (queue->hdr_digest && !req->offset)
 		nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu));
 
@@ -2441,6 +2574,7 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns,
 	req->data_len = blk_rq_nr_phys_segments(rq) ?
 				blk_rq_payload_bytes(rq) : 0;
 	req->curr_bio = rq->bio;
+	req->offloaded = false;
 
 	if (rq_data_dir(rq) == WRITE &&
 	    req->data_len <= nvme_tcp_inline_data_size(queue))
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:31 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:31 +0200
Subject: [PATCH v4 net-next  08/21] nvme-tcp: RX CRC offload
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-9-borisp@mellanox.com>

From: Yoray Zack <yorayz at mellanox.com>

Enable rx side of crc offload when supported.

At the end of the capsule, check if all the skb bits are
on, and if not recalculate the crc in SW and check it.

We reworked the receive-side crc calculation to always run
at the end, so as to keep a single flow for both offload and
non-offload. This change simplifies the code, but it may
degrade performance for non-offload crc calculation.

Signed-off-by: Yoray Zack <yorayz at mellanox.com>
Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 86 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 71 insertions(+), 15 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 188e26ab7116..09cb9b2e2c2d 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -69,6 +69,7 @@ enum nvme_tcp_queue_flags {
 	NVME_TCP_Q_LIVE		= 1,
 	NVME_TCP_Q_POLLING	= 2,
 	NVME_TCP_Q_OFF_DDP	= 3,
+	NVME_TCP_Q_OFF_DDGST_RX = 4,
 };
 
 enum nvme_tcp_recv_state {
@@ -96,6 +97,7 @@ struct nvme_tcp_queue {
 	size_t			data_remaining;
 	size_t			ddgst_remaining;
 	unsigned int		nr_cqe;
+	bool			ddgst_valid;
 
 	/* send state */
 	struct nvme_tcp_request *request;
@@ -234,6 +236,22 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
 	return nvme_tcp_pdu_data_left(req) <= len;
 }
 
+static inline bool nvme_tcp_ddp_ddgst_ok(struct nvme_tcp_queue *queue)
+{
+	return queue->ddgst_valid;
+}
+
+static inline void nvme_tcp_ddp_ddgst_update(struct nvme_tcp_queue *queue,
+					     struct sk_buff *skb)
+{
+	if (queue->ddgst_valid)
+#ifdef CONFIG_TCP_DDP_CRC
+		queue->ddgst_valid = skb->ddp_crc;
+#else
+		queue->ddgst_valid = false;
+#endif
+}
+
 static int nvme_tcp_req_map_sg(struct nvme_tcp_request *req, struct request *rq)
 {
 	int ret;
@@ -247,7 +265,27 @@ static int nvme_tcp_req_map_sg(struct nvme_tcp_request *req, struct request *rq)
 	return 0;
 }
 
-#ifdef CONFIG_TCP_DDP
+static void nvme_tcp_ddp_ddgst_recalc(struct ahash_request *hash,
+				      struct request *rq)
+{
+	struct nvme_tcp_request *req;
+
+	if (!rq)
+		return;
+
+	req = blk_mq_rq_to_pdu(rq);
+
+	if (!req->offloaded && nvme_tcp_req_map_sg(req, rq))
+		return;
+
+	crypto_ahash_init(hash);
+	req->ddp.sg_table.sgl = req->ddp.first_sgl;
+	ahash_request_set_crypt(hash, req->ddp.sg_table.sgl, NULL,
+				le32_to_cpu(req->data_len));
+	crypto_ahash_update(hash);
+}
+
+#if defined(CONFIG_TCP_DDP) || defined(CONFIG_TCP_DDP_CRC)
 
 static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
 static void nvme_tcp_ddp_teardown_done(void *ddp_ctx);
@@ -316,7 +354,7 @@ static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 	struct nvme_tcp_ddp_config config = {};
 	int ret;
 
-	if (!(netdev->features & NETIF_F_HW_TCP_DDP))
+	if (!(netdev->features & (NETIF_F_HW_TCP_DDP | NETIF_F_HW_TCP_DDP_CRC_RX)))
 		return -EOPNOTSUPP;
 
 	config.cfg.type		= TCP_DDP_NVME;
@@ -343,6 +381,9 @@ static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
 	if (netdev->features & NETIF_F_HW_TCP_DDP)
 		set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
 
+	if (netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+		set_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags);
+
 	return ret;
 }
 
@@ -373,7 +414,7 @@ static int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
 		return -ENODEV;
 	}
 
-	if (netdev->features & NETIF_F_HW_TCP_DDP &&
+	if ((netdev->features & (NETIF_F_HW_TCP_DDP | NETIF_F_HW_TCP_DDP_CRC_RX)) &&
 	    netdev->tcp_ddp_ops &&
 	    netdev->tcp_ddp_ops->tcp_ddp_limits)
 		ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
@@ -720,6 +761,7 @@ static void nvme_tcp_init_recv_ctx(struct nvme_tcp_queue *queue)
 	queue->pdu_offset = 0;
 	queue->data_remaining = -1;
 	queue->ddgst_remaining = 0;
+	queue->ddgst_valid = true;
 }
 
 static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
@@ -906,7 +948,8 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 	size_t rcv_len = min_t(size_t, *len, queue->pdu_remaining);
 	int ret;
 
-	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) ||
+	    test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
 		nvme_tcp_resync_response(queue, skb, *offset);
 
 	ret = skb_copy_bits(skb, *offset,
@@ -995,6 +1038,8 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 	struct nvme_tcp_request *req;
 	struct request *rq;
 
+	if (queue->data_digest && test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
+		nvme_tcp_ddp_ddgst_update(queue, skb);
 	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
 	if (!rq) {
 		dev_err(queue->ctrl->ctrl.device,
@@ -1047,7 +1092,6 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 
 	if (!queue->data_remaining) {
 		if (queue->data_digest) {
-			nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
 			queue->ddgst_remaining = NVME_TCP_DIGEST_LENGTH;
 		} else {
 			if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
@@ -1068,8 +1112,12 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
 	char *ddgst = (char *)&queue->recv_ddgst;
 	size_t recv_len = min_t(size_t, *len, queue->ddgst_remaining);
 	off_t off = NVME_TCP_DIGEST_LENGTH - queue->ddgst_remaining;
+	bool offload_fail, offload_en;
+	struct request *rq = NULL;
 	int ret;
 
+	if (test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
+		nvme_tcp_ddp_ddgst_update(queue, skb);
 	ret = skb_copy_bits(skb, *offset, &ddgst[off], recv_len);
 	if (unlikely(ret))
 		return ret;
@@ -1080,18 +1128,25 @@ static int nvme_tcp_recv_ddgst(struct nvme_tcp_queue *queue,
 	if (queue->ddgst_remaining)
 		return 0;
 
-	if (queue->recv_ddgst != queue->exp_ddgst) {
-		dev_err(queue->ctrl->ctrl.device,
-			"data digest error: recv %#x expected %#x\n",
-			le32_to_cpu(queue->recv_ddgst),
-			le32_to_cpu(queue->exp_ddgst));
-		return -EIO;
+	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
+
+	offload_fail = !nvme_tcp_ddp_ddgst_ok(queue);
+	offload_en = test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags);
+	if (!offload_en || offload_fail) {
+		if (offload_en && offload_fail)  // software-fallback
+			nvme_tcp_ddp_ddgst_recalc(queue->rcv_hash, rq);
+
+		nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst);
+		if (queue->recv_ddgst != queue->exp_ddgst) {
+			dev_err(queue->ctrl->ctrl.device,
+				"data digest error: recv %#x expected %#x\n",
+				le32_to_cpu(queue->recv_ddgst),
+				le32_to_cpu(queue->exp_ddgst));
+			return -EIO;
+		}
 	}
 
 	if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) {
-		struct request *rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue),
-						pdu->command_id);
-
 		nvme_tcp_end_request(rq, NVME_SC_SUCCESS);
 		queue->nr_cqe++;
 	}
@@ -1833,7 +1888,8 @@ static void __nvme_tcp_stop_queue(struct nvme_tcp_queue *queue)
 	nvme_tcp_restore_sock_calls(queue);
 	cancel_work_sync(&queue->io_work);
 
-	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags))
+	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) ||
+	    test_bit(NVME_TCP_Q_OFF_DDGST_RX, &queue->flags))
 		nvme_tcp_unoffload_socket(queue);
 }
 
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:32 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:32 +0200
Subject: [PATCH v4 net-next  09/21] nvme-tcp: Deal with netdevice DOWN events
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-10-borisp@mellanox.com>

From: Or Gerlitz <ogerlitz at mellanox.com>

For ddp setup/teardown and resync, the offloading logic
uses HW resources at the NIC driver such as SQ and CQ.

These resources are destroyed when the netdevice does down
and hence we must stop using them before the NIC driver
destroys them.

Use netdevice notifier for that matter -- offloaded connections
are stopped before the stack continues to call the NIC driver
close ndo.

We use the existing recovery flow which has the advantage
of resuming the offload once the connection is re-set.

This also buys us proper handling for the UNREGISTER event
b/c our offloading starts in the UP state, and down is always
there between up to unregister.

Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/nvme/host/tcp.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 09cb9b2e2c2d..8ea5c8fe4a5e 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -146,6 +146,7 @@ struct nvme_tcp_ctrl {
 
 static LIST_HEAD(nvme_tcp_ctrl_list);
 static DEFINE_MUTEX(nvme_tcp_ctrl_mutex);
+static struct notifier_block nvme_tcp_netdevice_nb;
 static struct workqueue_struct *nvme_tcp_wq;
 static const struct blk_mq_ops nvme_tcp_mq_ops;
 static const struct blk_mq_ops nvme_tcp_admin_mq_ops;
@@ -2904,6 +2905,30 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
 	return ERR_PTR(ret);
 }
 
+static int nvme_tcp_netdev_event(struct notifier_block *this,
+				 unsigned long event, void *ptr)
+{
+	struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
+	struct nvme_tcp_ctrl *ctrl;
+
+	switch (event) {
+	case NETDEV_GOING_DOWN:
+		mutex_lock(&nvme_tcp_ctrl_mutex);
+		list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list) {
+			if (ndev != ctrl->offloading_netdev)
+				continue;
+			nvme_tcp_error_recovery(&ctrl->ctrl);
+		}
+		mutex_unlock(&nvme_tcp_ctrl_mutex);
+		flush_workqueue(nvme_reset_wq);
+		/*
+		 * The associated controllers teardown has completed, ddp contexts
+		 * were also torn down so we should be safe to continue...
+		 */
+	}
+	return NOTIFY_DONE;
+}
+
 static struct nvmf_transport_ops nvme_tcp_transport = {
 	.name		= "tcp",
 	.module		= THIS_MODULE,
@@ -2918,13 +2943,26 @@ static struct nvmf_transport_ops nvme_tcp_transport = {
 
 static int __init nvme_tcp_init_module(void)
 {
+	int ret;
+
 	nvme_tcp_wq = alloc_workqueue("nvme_tcp_wq",
 			WQ_MEM_RECLAIM | WQ_HIGHPRI, 0);
 	if (!nvme_tcp_wq)
 		return -ENOMEM;
 
+	nvme_tcp_netdevice_nb.notifier_call = nvme_tcp_netdev_event;
+	ret = register_netdevice_notifier(&nvme_tcp_netdevice_nb);
+	if (ret) {
+		pr_err("failed to register netdev notifier\n");
+		goto out_err_reg_notifier;
+	}
+
 	nvmf_register_transport(&nvme_tcp_transport);
 	return 0;
+
+out_err_reg_notifier:
+	destroy_workqueue(nvme_tcp_wq);
+	return ret;
 }
 
 static void __exit nvme_tcp_cleanup_module(void)
@@ -2932,6 +2970,7 @@ static void __exit nvme_tcp_cleanup_module(void)
 	struct nvme_tcp_ctrl *ctrl;
 
 	nvmf_unregister_transport(&nvme_tcp_transport);
+	unregister_netdevice_notifier(&nvme_tcp_netdevice_nb);
 
 	mutex_lock(&nvme_tcp_ctrl_mutex);
 	list_for_each_entry(ctrl, &nvme_tcp_ctrl_list, list)
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:33 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:33 +0200
Subject: [PATCH v4 net-next 10/21] net/mlx5: Header file changes for nvme-tcp
 offload
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-11-borisp@mellanox.com>

From: Ben Ben-ishay <benishay at nvidia.com>

Add the necessary infrastructure for NVMEoTCP offload:

- Add nvmeocp_en + nvmeotcp_crc_en bit to the TIR for identify NVMEoTCP offload flow
  And tag_buffer_id that will be used by the connected nvmeotcp_queues
- Add new CQE field that will be used to pass scattered data information to SW
- Add new capability to HCA_CAP that represnts the NVMEoTCP offload ability

Signed-off-by: Ben Ben-ishay <benishay at nvidia.com>
Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/mlx5/device.h   |   8 +++
 include/linux/mlx5/mlx5_ifc.h | 101 +++++++++++++++++++++++++++++++++-
 include/linux/mlx5/qp.h       |   1 +
 3 files changed, 107 insertions(+), 3 deletions(-)

diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index c85d96180b8f..ab04959188b9 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -263,6 +263,7 @@ enum {
 enum {
 	MLX5_MKEY_MASK_LEN		= 1ull << 0,
 	MLX5_MKEY_MASK_PAGE_SIZE	= 1ull << 1,
+	MLX5_MKEY_MASK_XLT_OCT_SIZE     = 1ull << 2,
 	MLX5_MKEY_MASK_START_ADDR	= 1ull << 6,
 	MLX5_MKEY_MASK_PD		= 1ull << 7,
 	MLX5_MKEY_MASK_EN_RINVAL	= 1ull << 8,
@@ -1173,6 +1174,7 @@ enum mlx5_cap_type {
 	MLX5_CAP_VDPA_EMULATION = 0x13,
 	MLX5_CAP_DEV_EVENT = 0x14,
 	MLX5_CAP_IPSEC,
+	MLX5_CAP_DEV_NVMEOTCP = 0x19,
 	/* NUM OF CAP Types */
 	MLX5_CAP_NUM
 };
@@ -1393,6 +1395,12 @@ enum mlx5_qcam_feature_groups {
 #define MLX5_CAP_IPSEC(mdev, cap)\
 	MLX5_GET(ipsec_cap, (mdev)->caps.hca_cur[MLX5_CAP_IPSEC], cap)
 
+#define MLX5_CAP_DEV_NVMEOTCP(mdev, cap)\
+	MLX5_GET(nvmeotcp_cap, mdev->caps.hca_cur[MLX5_CAP_DEV_NVMEOTCP], cap)
+
+#define MLX5_CAP64_NVMEOTCP(mdev, cap)\
+	MLX5_GET64(nvmeotcp_cap, mdev->caps.hca_cur[MLX5_CAP_DEV_NVMEOTCP], cap)
+
 enum {
 	MLX5_CMD_STAT_OK			= 0x0,
 	MLX5_CMD_STAT_INT_ERR			= 0x1,
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index b9f5894f6f8d..ae6edf08a70b 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -1278,7 +1278,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         log_max_srq_sz[0x8];
 	u8         log_max_qp_sz[0x8];
 	u8         event_cap[0x1];
-	u8         reserved_at_91[0x7];
+	u8         reserved_at_91[0x5];
+	u8         nvmeotcp[0x1];
+	u8         reserved_at_97[0x1];
 	u8         prio_tag_required[0x1];
 	u8         reserved_at_99[0x2];
 	u8         log_max_qp[0x5];
@@ -3029,6 +3031,21 @@ struct mlx5_ifc_roce_addr_layout_bits {
 	u8         reserved_at_e0[0x20];
 };
 
+struct mlx5_ifc_nvmeotcp_cap_bits {
+	u8    zerocopy[0x1];
+	u8    crc_rx[0x1];
+	u8    crc_tx[0x1];
+	u8    reserved_at_3[0x15];
+	u8    version[0x8];
+
+	u8    reserved_at_20[0x13];
+	u8    log_max_nvmeotcp_tag_buffer_table[0x5];
+	u8    reserved_at_38[0x3];
+	u8    log_max_nvmeotcp_tag_buffer_size[0x5];
+
+	u8    reserved_at_40[0x7c0];
+};
+
 union mlx5_ifc_hca_cap_union_bits {
 	struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap;
 	struct mlx5_ifc_odp_cap_bits odp_cap;
@@ -3045,6 +3062,7 @@ union mlx5_ifc_hca_cap_union_bits {
 	struct mlx5_ifc_tls_cap_bits tls_cap;
 	struct mlx5_ifc_device_mem_cap_bits device_mem_cap;
 	struct mlx5_ifc_virtio_emulation_cap_bits virtio_emulation_cap;
+	struct mlx5_ifc_nvmeotcp_cap_bits nvmeotcp_cap;
 	u8         reserved_at_0[0x8000];
 };
 
@@ -3239,7 +3257,9 @@ struct mlx5_ifc_tirc_bits {
 
 	u8         disp_type[0x4];
 	u8         tls_en[0x1];
-	u8         reserved_at_25[0x1b];
+	u8         nvmeotcp_zero_copy_en[0x1];
+	u8         nvmeotcp_crc_en[0x1];
+	u8         reserved_at_27[0x19];
 
 	u8         reserved_at_40[0x40];
 
@@ -3270,7 +3290,8 @@ struct mlx5_ifc_tirc_bits {
 
 	struct mlx5_ifc_rx_hash_field_select_bits rx_hash_field_selector_inner;
 
-	u8         reserved_at_2c0[0x4c0];
+	u8         nvmeotcp_tag_buffer_table_id[0x20];
+	u8         reserved_at_2e0[0x4a0];
 };
 
 enum {
@@ -10745,12 +10766,14 @@ enum {
 	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_ENCRYPTION_KEY = BIT_ULL(0xc),
 	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_IPSEC = BIT_ULL(0x13),
 	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_SAMPLER = BIT_ULL(0x20),
+	MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE = BIT_ULL(0x21),
 };
 
 enum {
 	MLX5_GENERAL_OBJECT_TYPES_ENCRYPTION_KEY = 0xc,
 	MLX5_GENERAL_OBJECT_TYPES_IPSEC = 0x13,
 	MLX5_GENERAL_OBJECT_TYPES_SAMPLER = 0x20,
+	MLX5_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE = 0x21
 };
 
 enum {
@@ -10852,6 +10875,20 @@ struct mlx5_ifc_create_sampler_obj_in_bits {
 	struct mlx5_ifc_sampler_obj_bits sampler_object;
 };
 
+struct mlx5_ifc_nvmeotcp_tag_buf_table_obj_bits {
+	u8    modify_field_select[0x40];
+
+	u8    reserved_at_20[0x20];
+
+	u8    reserved_at_40[0x1b];
+	u8    log_tag_buffer_table_size[0x5];
+};
+
+struct mlx5_ifc_create_nvmeotcp_tag_buf_table_in_bits {
+	struct mlx5_ifc_general_obj_in_cmd_hdr_bits general_obj_in_cmd_hdr;
+	struct mlx5_ifc_nvmeotcp_tag_buf_table_obj_bits nvmeotcp_tag_buf_table_obj;
+};
+
 enum {
 	MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_KEY_SIZE_128 = 0x0,
 	MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_KEY_SIZE_256 = 0x1,
@@ -10862,6 +10899,18 @@ enum {
 	MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_IPSEC = 0x2,
 };
 
+enum {
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_XTS               = 0x0,
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_TLS               = 0x1,
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_NVMETCP           = 0x2,
+	MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_NVMETCP_WITH_TLS  = 0x3,
+};
+
+enum {
+	MLX5_TRANSPORT_STATIC_PARAMS_TI_INITIATOR  = 0x0,
+	MLX5_TRANSPORT_STATIC_PARAMS_TI_TARGET     = 0x1,
+};
+
 struct mlx5_ifc_tls_static_params_bits {
 	u8         const_2[0x2];
 	u8         tls_version[0x4];
@@ -10902,4 +10951,50 @@ enum {
 	MLX5_MTT_PERM_RW	= MLX5_MTT_PERM_READ | MLX5_MTT_PERM_WRITE,
 };
 
+struct mlx5_ifc_nvmeotcp_progress_params_bits {
+	u8    next_pdu_tcp_sn[0x20];
+
+	u8    hw_resync_tcp_sn[0x20];
+
+	u8    pdu_tracker_state[0x2];
+	u8    offloading_state[0x2];
+	u8    reserved_at_64[0xc];
+	u8    cccid_ttag[0x10];
+};
+
+struct mlx5_ifc_transport_static_params_bits {
+	u8    const_2[0x2];
+	u8    tls_version[0x4];
+	u8    const_1[0x2];
+	u8    reserved_at_8[0x14];
+	u8    acc_type[0x4];
+
+	u8    reserved_at_20[0x20];
+
+	u8    initial_record_number[0x40];
+
+	u8    resync_tcp_sn[0x20];
+
+	u8    gcm_iv[0x20];
+
+	u8    implicit_iv[0x40];
+
+	u8    reserved_at_100[0x8];
+	u8    dek_index[0x18];
+
+	u8    reserved_at_120[0x14];
+	u8    const1[0x1];
+	u8    ti[0x1];
+	u8    zero_copy_en[0x1];
+	u8    ddgst_offload_en[0x1];
+	u8    hdgst_offload_en[0x1];
+	u8    ddgst_en[0x1];
+	u8    hddgst_en[0x1];
+	u8    pda[0x5];
+
+	u8    nvme_resync_tcp_sn[0x20];
+
+	u8    reserved_at_160[0xa0];
+};
+
 #endif /* MLX5_IFC_H */
diff --git a/include/linux/mlx5/qp.h b/include/linux/mlx5/qp.h
index d75ef8aa8fac..5fa8b82c9edb 100644
--- a/include/linux/mlx5/qp.h
+++ b/include/linux/mlx5/qp.h
@@ -220,6 +220,7 @@ struct mlx5_wqe_ctrl_seg {
 #define MLX5_WQE_CTRL_OPCODE_MASK 0xff
 #define MLX5_WQE_CTRL_WQE_INDEX_MASK 0x00ffff00
 #define MLX5_WQE_CTRL_WQE_INDEX_SHIFT 8
+#define MLX5_WQE_CTRL_TIR_TIS_INDEX_SHIFT 8
 
 enum {
 	MLX5_ETH_WQE_L3_INNER_CSUM      = 1 << 4,
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:34 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:34 +0200
Subject: [PATCH v4 net-next 11/21] net/mlx5: Add 128B CQE for NVMEoTCP offload
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-12-borisp@mellanox.com>

From: Ben Ben-ishay <benishay at nvidia.com>

Add the NVMEoTCP offload definition and access functions for 128B CQEs.

Signed-off-by: Ben Ben-ishay <benishay at nvidia.com>
Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 include/linux/mlx5/device.h | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index ab04959188b9..f6548c255290 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -791,7 +791,7 @@ struct mlx5_err_cqe {
 
 struct mlx5_cqe64 {
 	u8		tls_outer_l3_tunneled;
-	u8		rsvd0;
+	u8		nvmetcp;
 	__be16		wqe_id;
 	u8		lro_tcppsh_abort_dupack;
 	u8		lro_min_ttl;
@@ -824,6 +824,19 @@ struct mlx5_cqe64 {
 	u8		op_own;
 };
 
+struct mlx5e_cqe128 {
+	__be16		cclen;
+	__be16		hlen;
+	union {
+		__be32		resync_tcp_sn;
+		__be32		ccoff;
+	};
+	__be16		ccid;
+	__be16		rsvd8;
+	u8		rsvd12[52];
+	struct mlx5_cqe64 cqe64;
+};
+
 struct mlx5_mini_cqe8 {
 	union {
 		__be32 rx_hash_result;
@@ -854,6 +867,27 @@ enum {
 
 #define MLX5_MINI_CQE_ARRAY_SIZE 8
 
+static inline bool cqe_is_nvmeotcp_resync(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 6) & 0x1);
+}
+
+static inline bool cqe_is_nvmeotcp_crcvalid(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 5) & 0x1);
+}
+
+static inline bool cqe_is_nvmeotcp_zc(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 4) & 0x1);
+}
+
+/* check if cqe is zc or crc or resync */
+static inline bool cqe_is_nvmeotcp(struct mlx5_cqe64 *cqe)
+{
+	return ((cqe->nvmetcp >> 4) & 0x7);
+}
+
 static inline u8 mlx5_get_cqe_format(struct mlx5_cqe64 *cqe)
 {
 	return (cqe->op_own >> 2) & 0x3;
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:35 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:35 +0200
Subject: [PATCH v4 net-next  12/21] net/mlx5e: TCP flow steering for nvme-tcp
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-13-borisp@mellanox.com>

Both nvme-tcp and tls require tcp flow steering. Compile it for both of
them. Additionally, use reference counting to allocate/free TCP flow
steering.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en/fs.h        |  4 ++--
 .../net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c  | 10 ++++++++++
 .../net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h  |  2 +-
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
index a16297e7e2ac..a7fe3a6358ea 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
@@ -137,7 +137,7 @@ enum {
 	MLX5E_L2_FT_LEVEL,
 	MLX5E_TTC_FT_LEVEL,
 	MLX5E_INNER_TTC_FT_LEVEL,
-#ifdef CONFIG_MLX5_EN_TLS
+#if defined(CONFIG_MLX5_EN_TLS) || defined(CONFIG_MLX5_EN_NVMEOTCP)
 	MLX5E_ACCEL_FS_TCP_FT_LEVEL,
 #endif
 #ifdef CONFIG_MLX5_EN_ARFS
@@ -256,7 +256,7 @@ struct mlx5e_flow_steering {
 #ifdef CONFIG_MLX5_EN_ARFS
 	struct mlx5e_arfs_tables        arfs;
 #endif
-#ifdef CONFIG_MLX5_EN_TLS
+#if defined(CONFIG_MLX5_EN_TLS) || defined(CONFIG_MLX5_EN_NVMEOTCP)
 	struct mlx5e_accel_fs_tcp      *accel_tcp;
 #endif
 };
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c
index e51f60b55daa..21341a92f355 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c
@@ -14,6 +14,7 @@ enum accel_fs_tcp_type {
 struct mlx5e_accel_fs_tcp {
 	struct mlx5e_flow_table tables[ACCEL_FS_TCP_NUM_TYPES];
 	struct mlx5_flow_handle *default_rules[ACCEL_FS_TCP_NUM_TYPES];
+	refcount_t		ref_count;
 };
 
 static enum mlx5e_traffic_types fs_accel2tt(enum accel_fs_tcp_type i)
@@ -337,6 +338,7 @@ static int accel_fs_tcp_enable(struct mlx5e_priv *priv)
 			return err;
 		}
 	}
+	refcount_set(&priv->fs.accel_tcp->ref_count, 1);
 	return 0;
 }
 
@@ -360,6 +362,9 @@ void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv)
 	if (!priv->fs.accel_tcp)
 		return;
 
+	if (!refcount_dec_and_test(&priv->fs.accel_tcp->ref_count))
+		return;
+
 	accel_fs_tcp_disable(priv);
 
 	for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++)
@@ -376,6 +381,11 @@ int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv)
 	if (!MLX5_CAP_FLOWTABLE_NIC_RX(priv->mdev, ft_field_support.outer_ip_version))
 		return -EOPNOTSUPP;
 
+	if (priv->fs.accel_tcp) {
+		refcount_inc(&priv->fs.accel_tcp->ref_count);
+		return 0;
+	}
+
 	priv->fs.accel_tcp = kzalloc(sizeof(*priv->fs.accel_tcp), GFP_KERNEL);
 	if (!priv->fs.accel_tcp)
 		return -ENOMEM;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h
index 589235824543..8aff9298183c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h
@@ -6,7 +6,7 @@
 
 #include "en.h"
 
-#ifdef CONFIG_MLX5_EN_TLS
+#if defined(CONFIG_MLX5_EN_TLS) || defined(CONFIG_MLX5_EN_NVMEOTCP)
 int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv);
 void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv);
 struct mlx5_flow_handle *mlx5e_accel_fs_add_sk(struct mlx5e_priv *priv,
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:37 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:37 +0200
Subject: [PATCH v4 net-next  14/21] net/mlx5e: KLM UMR helper macros
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-15-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

Add helper macros for posting KLM UMR WQE.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 98be350849b2..1982e0a58be7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -153,6 +153,24 @@ struct page_pool;
 #define MLX5E_UMR_WQEBBS \
 	(DIV_ROUND_UP(MLX5E_UMR_WQE_INLINE_SZ, MLX5_SEND_WQE_BB))
 
+#define KLM_ALIGNMENT 4
+#define MLX5E_KLM_UMR_WQE_SZ(sgl_len)\
+	(sizeof(struct mlx5e_umr_wqe) +\
+	(sizeof(struct mlx5_klm) * (sgl_len)))
+
+#define MLX5E_KLM_UMR_WQEBBS(sgl_len)\
+	(DIV_ROUND_UP(MLX5E_KLM_UMR_WQE_SZ(sgl_len), MLX5_SEND_WQE_BB))
+
+#define MLX5E_KLM_UMR_DS_CNT(sgl_len)\
+	DIV_ROUND_UP(MLX5E_KLM_UMR_WQE_SZ(sgl_len), MLX5_SEND_WQE_DS)
+
+#define MLX5E_MAX_KLM_ENTRIES_PER_WQE(wqe_size)\
+	(((wqe_size) - sizeof(struct mlx5e_umr_wqe)) / sizeof(struct mlx5_klm))
+
+#define MLX5E_KLM_ENTRIES_PER_WQE(wqe_size)\
+	(MLX5E_MAX_KLM_ENTRIES_PER_WQE(wqe_size) -\
+			(MLX5E_MAX_KLM_ENTRIES_PER_WQE(wqe_size) % KLM_ALIGNMENT))
+
 #define MLX5E_MSG_LEVEL			NETIF_MSG_LINK
 
 #define mlx5e_dbg(mlevel, priv, format, ...)                    \
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:36 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:36 +0200
Subject: [PATCH v4 net-next 13/21] net/mlx5e: NVMEoTCP offload initialization
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-14-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

This commit introduce the initialization blocks for NVMEoTCP offload:
- Use 128B CQEs when NVME-TCP offload is enabled.
- Use a dedicated icosq for NVME-TCP work. This list of SQ is unique in the
  sense that it is driven directly by the NVME-TCP layer to submit and
  invalidate ddp requests.
- Query nvmeotcp capabilities

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/Kconfig   |  10 +
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   7 +
 .../ethernet/mellanox/mlx5/core/en/params.h   |   1 +
 .../mellanox/mlx5/core/en_accel/en_accel.h    |   9 +-
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 196 ++++++++++++++++++
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    | 115 ++++++++++
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  40 +++-
 .../net/ethernet/mellanox/mlx5/core/en_txrx.c |  17 ++
 drivers/net/ethernet/mellanox/mlx5/core/fw.c  |   6 +
 10 files changed, 401 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
index c38b791e2406..6d91df52bb9c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
@@ -222,3 +222,13 @@ config MLX5_SF_MANAGER
 	port is managed through devlink.  A subfunction supports RDMA, netdevice
 	and vdpa device. It is similar to a SRIOV VF but it doesn't require
 	SRIOV support.
+
+config MLX5_EN_NVMEOTCP
+	bool "NVMEoTCP accelaration"
+	depends on MLX5_CORE_EN
+	depends on TCP_DDP=y || TCP_DDP_CRC=y
+	default n
+	help
+	Build support for NVMEoTCP accelaration in the NIC.
+	Note: Support for hardware with this capability needs to be selected
+	for this option to become available.
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index 8cb2625472c3..ac9978b0eb5d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -98,3 +98,5 @@ mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o
 # SF manager
 #
 mlx5_core-$(CONFIG_MLX5_SF_MANAGER) += sf/cmd.o sf/hw_table.o sf/devlink.o
+
+mlx5_core-$(CONFIG_MLX5_EN_NVMEOTCP) += en_accel/fs_tcp.o en_accel/nvmeotcp.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index a8e31cdd4a4e..98be350849b2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -669,6 +669,10 @@ struct mlx5e_channel {
 	struct mlx5e_txqsq         sq[MLX5E_MAX_NUM_TC];
 	struct mlx5e_icosq         icosq;   /* internal control operations */
 	struct mlx5e_txqsq __rcu * __rcu *qos_sqs;
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	struct list_head	   list_nvmeotcpsq;   /* nvmeotcp umrs  */
+	spinlock_t                 nvmeotcp_icosq_lock;
+#endif
 	bool                       xdp;
 	struct napi_struct         napi;
 	struct device             *pdev;
@@ -874,6 +878,9 @@ struct mlx5e_priv {
 #endif
 #ifdef CONFIG_MLX5_EN_TLS
 	struct mlx5e_tls          *tls;
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	struct mlx5e_nvmeotcp      *nvmeotcp;
 #endif
 	struct devlink_health_reporter *tx_reporter;
 	struct devlink_health_reporter *rx_reporter;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
index ea2cfb04b31a..855a41893b3d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
@@ -16,6 +16,7 @@ struct mlx5e_cq_param {
 	struct mlx5_wq_param       wq;
 	u16                        eq_ix;
 	u8                         cq_period_mode;
+	bool                       force_cqe128;
 };
 
 struct mlx5e_rq_param {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h
index 959bb6cd7203..eabba2168dd3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h
@@ -39,6 +39,7 @@
 #include "en_accel/ipsec_rxtx.h"
 #include "en_accel/tls.h"
 #include "en_accel/tls_rxtx.h"
+#include "en_accel/nvmeotcp.h"
 #include "en.h"
 #include "en/txrx.h"
 
@@ -198,11 +199,17 @@ static inline void mlx5e_accel_tx_finish(struct mlx5e_txqsq *sq,
 
 static inline int mlx5e_accel_init_rx(struct mlx5e_priv *priv)
 {
-	return mlx5e_ktls_init_rx(priv);
+	int tls, nvmeotcp;
+
+	tls = mlx5e_ktls_init_rx(priv);
+	nvmeotcp = mlx5e_nvmeotcp_init_rx(priv);
+
+	return tls && nvmeotcp;
 }
 
 static inline void mlx5e_accel_cleanup_rx(struct mlx5e_priv *priv)
 {
+	mlx5e_nvmeotcp_cleanup_rx(priv);
 	mlx5e_ktls_cleanup_rx(priv);
 }
 #endif /* __MLX5E_EN_ACCEL_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
new file mode 100644
index 000000000000..37a86ca83913
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2021 Mellanox Technologies. */
+
+#include <linux/netdevice.h>
+#include <linux/idr.h>
+#include "en_accel/nvmeotcp.h"
+#include "en_accel/fs_tcp.h"
+#include "en/txrx.h"
+
+#define MAX_NVMEOTCP_QUEUES	(512)
+#define MIN_NVMEOTCP_QUEUES	(1)
+
+static const struct rhashtable_params rhash_queues = {
+	.key_len = sizeof(int),
+	.key_offset = offsetof(struct mlx5e_nvmeotcp_queue, id),
+	.head_offset = offsetof(struct mlx5e_nvmeotcp_queue, hash),
+	.automatic_shrinking = true,
+	.min_size = 1,
+	.max_size = MAX_NVMEOTCP_QUEUES,
+};
+
+static int
+mlx5e_nvmeotcp_offload_limits(struct net_device *netdev,
+			      struct tcp_ddp_limits *limits)
+{
+	return 0;
+}
+
+static int
+mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
+			  struct sock *sk,
+			  struct tcp_ddp_config *tconfig)
+{
+	return 0;
+}
+
+static void
+mlx5e_nvmeotcp_queue_teardown(struct net_device *netdev,
+			      struct sock *sk)
+{
+}
+
+static int
+mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
+			 struct sock *sk,
+			 struct tcp_ddp_io *ddp)
+{
+	return 0;
+}
+
+static int
+mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
+			    struct sock *sk,
+			    struct tcp_ddp_io *ddp,
+			    void *ddp_ctx)
+{
+	return 0;
+}
+
+static void
+mlx5e_nvmeotcp_dev_resync(struct net_device *netdev,
+			  struct sock *sk, u32 seq)
+{
+}
+
+static const struct tcp_ddp_dev_ops mlx5e_nvmeotcp_ops = {
+	.tcp_ddp_limits = mlx5e_nvmeotcp_offload_limits,
+	.tcp_ddp_sk_add = mlx5e_nvmeotcp_queue_init,
+	.tcp_ddp_sk_del = mlx5e_nvmeotcp_queue_teardown,
+	.tcp_ddp_setup = mlx5e_nvmeotcp_ddp_setup,
+	.tcp_ddp_teardown = mlx5e_nvmeotcp_ddp_teardown,
+	.tcp_ddp_resync = mlx5e_nvmeotcp_dev_resync,
+};
+
+int set_feature_nvme_tcp(struct net_device *netdev, bool enable)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	int err = 0;
+
+	mutex_lock(&priv->state_lock);
+	if (enable)
+		err = mlx5e_accel_fs_tcp_create(priv);
+	else
+		mlx5e_accel_fs_tcp_destroy(priv);
+	mutex_unlock(&priv->state_lock);
+	if (err)
+		return err;
+
+	priv->nvmeotcp->enable = enable;
+	err = mlx5e_safe_reopen_channels(priv);
+	return err;
+}
+
+int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable)
+{
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	int err = 0;
+
+	mutex_lock(&priv->state_lock);
+	if (enable)
+		err = mlx5e_accel_fs_tcp_create(priv);
+	else
+		mlx5e_accel_fs_tcp_destroy(priv);
+	mutex_unlock(&priv->state_lock);
+
+	priv->nvmeotcp->crc_rx_enable = enable;
+	err = mlx5e_safe_reopen_channels(priv);
+	if (err)
+		netdev_err(priv->netdev,
+			   "%s failed to reopen channels, err(%d).\n",
+			   __func__, err);
+
+	return err;
+}
+
+void mlx5e_nvmeotcp_build_netdev(struct mlx5e_priv *priv)
+{
+	struct net_device *netdev = priv->netdev;
+
+	if (!MLX5_CAP_GEN(priv->mdev, nvmeotcp))
+		return;
+
+	if (MLX5_CAP_DEV_NVMEOTCP(priv->mdev, zerocopy)) {
+		netdev->features |= NETIF_F_HW_TCP_DDP;
+		netdev->hw_features |= NETIF_F_HW_TCP_DDP;
+	}
+
+	if (MLX5_CAP_DEV_NVMEOTCP(priv->mdev, crc_rx)) {
+		netdev->features |= NETIF_F_HW_TCP_DDP_CRC_RX;
+		netdev->hw_features |= NETIF_F_HW_TCP_DDP_CRC_RX;
+	}
+
+	netdev->tcp_ddp_ops = &mlx5e_nvmeotcp_ops;
+}
+
+int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv)
+{
+	int ret = 0;
+
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP) {
+		ret = mlx5e_accel_fs_tcp_create(priv);
+		if (ret)
+			return ret;
+	}
+
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+		ret = mlx5e_accel_fs_tcp_create(priv);
+
+	return ret;
+}
+
+void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv)
+{
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP)
+		mlx5e_accel_fs_tcp_destroy(priv);
+
+	if (priv->netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+		mlx5e_accel_fs_tcp_destroy(priv);
+}
+
+int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv)
+{
+	struct mlx5e_nvmeotcp *nvmeotcp = kzalloc(sizeof(*nvmeotcp), GFP_KERNEL);
+	int ret = 0;
+
+	if (!nvmeotcp)
+		return -ENOMEM;
+
+	ida_init(&nvmeotcp->queue_ids);
+	ret = rhashtable_init(&nvmeotcp->queue_hash, &rhash_queues);
+	if (ret)
+		goto err_ida;
+
+	priv->nvmeotcp = nvmeotcp;
+	priv->nvmeotcp->enable = true;
+	goto out;
+
+err_ida:
+	ida_destroy(&nvmeotcp->queue_ids);
+	kfree(nvmeotcp);
+out:
+	return ret;
+}
+
+void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv)
+{
+	struct mlx5e_nvmeotcp *nvmeotcp = priv->nvmeotcp;
+
+	if (!nvmeotcp)
+		return;
+
+	rhashtable_destroy(&nvmeotcp->queue_hash);
+	ida_destroy(&nvmeotcp->queue_ids);
+	kfree(nvmeotcp);
+	priv->nvmeotcp = NULL;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
new file mode 100644
index 000000000000..753757fc44a3
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
@@ -0,0 +1,115 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021 Mellanox Technologies. */
+#ifndef __MLX5E_NVMEOTCP_H__
+#define __MLX5E_NVMEOTCP_H__
+
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+
+#include "net/tcp_ddp.h"
+#include "en.h"
+#include "en/params.h"
+
+struct nvmeotcp_queue_entry {
+	struct mlx5e_nvmeotcp_queue	*queue;
+	u32				sgl_length;
+	struct mlx5_core_mkey		klm_mkey;
+	struct scatterlist		*sgl;
+	u32				ccid_gen;
+	u64				size;
+
+	/* for the ddp invalidate done callback */
+	void				*ddp_ctx;
+	struct tcp_ddp_io		*ddp;
+};
+
+struct mlx5e_nvmeotcp_sq {
+	struct list_head		list;
+	struct mlx5e_icosq		icosq;
+};
+
+/**
+ *	struct mlx5e_nvmeotcp_queue - MLX5 metadata for NVMEoTCP queue
+ *	@fh: Flow handle representing the 5-tuple steering for this flow
+ *	@tirn: Destination TIR number created for NVMEoTCP offload
+ *	@id: Flow tag ID used to identify this queue
+ *	@size: NVMEoTCP queue depth
+ *	@sq: Send queue used for sending control messages
+ *	@ccid_table: Table holding metadata for each CC
+ *	@tag_buf_table_id: Tag buffer table for CCIDs
+ *	@hash: Hash table of queues mapped by @id
+ *	@ref_count: Reference count for this structure
+ *	@ccoff: Offset within the current CC
+ *	@pda: Padding alignment
+ *	@ccid_gen: Generation ID for the CCID, used to avoid conflicts in DDP
+ *	@max_klms_per_wqe: Number of KLMs per DDP operation
+ *	@channel_ix: Channel IX for this nvmeotcp_queue
+ *	@sk: The socket used by the NVMe-TCP queue
+ *	@zerocopy: if this queue is used for zerocopy offload.
+ *	@crc_rx: if this queue is used for CRC Rx offload.
+ *	@ccid: ID of the current CC
+ *	@ccsglidx: Index within the scatter-gather list (SGL) of the current CC
+ *	@ccoff_inner: Current offset within the @ccsglidx element
+ *	@priv: mlx5e netdev priv
+ *	@inv_done: invalidate callback of the nvme tcp driver
+ *	@after_resync_cqe: indicate if resync occurred
+ */
+struct mlx5e_nvmeotcp_queue {
+	struct tcp_ddp_ctx		tcp_ddp_ctx;
+	struct mlx5_flow_handle		*fh;
+	int				tirn;
+	int				id;
+	u32				size;
+	struct mlx5e_nvmeotcp_sq	*sq;
+	struct nvmeotcp_queue_entry	*ccid_table;
+	u32				tag_buf_table_id;
+	struct rhash_head		hash;
+	refcount_t			ref_count;
+	bool				dgst;
+	int				pda;
+	u32				ccid_gen;
+	u32				max_klms_per_wqe;
+	u32				channel_ix;
+	struct sock			*sk;
+	bool				zerocopy;
+	bool				crc_rx;
+
+	/* current ccid fields */
+	off_t				ccoff;
+	int				ccid;
+	int				ccsglidx;
+	int				ccoff_inner;
+
+	/* for ddp invalidate flow */
+	struct mlx5e_priv		*priv;
+
+	/* for flow_steering flow */
+	struct completion		done;
+	/* for MASK HW resync cqe */
+	bool				after_resync_cqe;
+};
+
+struct mlx5e_nvmeotcp {
+	struct ida			queue_ids;
+	struct rhashtable		queue_hash;
+	bool				enable;
+	bool				crc_rx_enable;
+};
+
+void mlx5e_nvmeotcp_build_netdev(struct mlx5e_priv *priv);
+int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv);
+int set_feature_nvme_tcp(struct net_device *netdev, bool enable);
+int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable);
+void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv);
+int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv);
+void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv);
+#else
+
+static inline void mlx5e_nvmeotcp_build_netdev(struct mlx5e_priv *priv) { }
+static inline int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv) { return 0; }
+static inline void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv) { }
+static inline int set_feature_nvme_tcp(struct net_device *netdev, bool enable) { return 0; }
+static inline int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable) { return 0; }
+static inline int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv) { return 0; }
+static inline void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv) { }
+#endif
+#endif /* __MLX5E_NVMEOTCP_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index d3534b657b98..e816d79dbb83 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -47,6 +47,7 @@
 #include "en_accel/ipsec.h"
 #include "en_accel/en_accel.h"
 #include "en_accel/tls.h"
+#include "en_accel/nvmeotcp.h"
 #include "accel/ipsec.h"
 #include "accel/tls.h"
 #include "lib/vxlan.h"
@@ -2022,6 +2023,10 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
 	c->aff_mask = irq_get_effective_affinity_mask(irq);
 	c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix);
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	INIT_LIST_HEAD(&c->list_nvmeotcpsq);
+	spin_lock_init(&c->nvmeotcp_icosq_lock);
+#endif
 	netif_napi_add(netdev, &c->napi, mlx5e_napi_poll, 64);
 
 	err = mlx5e_open_queues(c, params, cparam);
@@ -2259,7 +2264,8 @@ static void mlx5e_build_common_cq_param(struct mlx5e_priv *priv,
 	void *cqc = param->cqc;
 
 	MLX5_SET(cqc, cqc, uar_page, priv->mdev->priv.uar->index);
-	if (MLX5_CAP_GEN(priv->mdev, cqe_128_always) && cache_line_size() >= 128)
+	if (MLX5_CAP_GEN(priv->mdev, cqe_128_always) &&
+	    (cache_line_size() >= 128 || param->force_cqe128))
 		MLX5_SET(cqc, cqc, cqe_sz, CQE_STRIDE_128_PAD);
 }
 
@@ -2273,6 +2279,11 @@ void mlx5e_build_rx_cq_param(struct mlx5e_priv *priv,
 	void *cqc = param->cqc;
 	u8 log_cq_size;
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	/* nvme-tcp offload mandates 128 byte cqes */
+	param->force_cqe128 |= (priv->nvmeotcp->enable || priv->nvmeotcp->crc_rx_enable);
+#endif
+
 	switch (params->rq_wq_type) {
 	case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
 		log_cq_size = mlx5e_mpwqe_get_log_rq_size(params, xsk) +
@@ -4064,6 +4075,10 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features)
 	err |= MLX5E_HANDLE_FEATURE(NETIF_F_NTUPLE, set_feature_arfs);
 #endif
 	err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TLS_RX, mlx5e_ktls_set_feature_rx);
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TCP_DDP, set_feature_nvme_tcp);
+	err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TCP_DDP_CRC_RX, set_feature_nvme_tcp_crc);
+#endif
 
 	if (err) {
 		netdev->features = oper_features;
@@ -4101,6 +4116,23 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
 		features &= ~NETIF_F_RXHASH;
 		if (netdev->features & NETIF_F_RXHASH)
 			netdev_warn(netdev, "Disabling rxhash, not supported when CQE compress is active\n");
+
+		features &= ~NETIF_F_HW_TCP_DDP;
+		if (netdev->features & NETIF_F_HW_TCP_DDP)
+			netdev_warn(netdev, "Disabling tcp-ddp offload, not supported when CQE compress is active\n");
+
+		features &= ~NETIF_F_HW_TCP_DDP_CRC_RX;
+		if (netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+			netdev_warn(netdev, "Disabling tcp-ddp-crc-rx offload, not supported when CQE compression is active\n");
+	}
+
+	if (netdev->features & NETIF_F_LRO) {
+		features &= ~NETIF_F_HW_TCP_DDP;
+		if (netdev->features & NETIF_F_HW_TCP_DDP)
+			netdev_warn(netdev, "Disabling tcp-ddp offload, not supported when LRO is active\n");
+		features &= ~NETIF_F_HW_TCP_DDP_CRC_RX;
+		if (netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX)
+			netdev_warn(netdev, "Disabling tcp-ddp-crc-rx offload, not supported when LRO is active\n");
 	}
 
 	mutex_unlock(&priv->state_lock);
@@ -5176,6 +5208,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 	mlx5e_set_netdev_dev_addr(netdev);
 	mlx5e_ipsec_build_netdev(priv);
 	mlx5e_tls_build_netdev(priv);
+	mlx5e_nvmeotcp_build_netdev(priv);
 }
 
 void mlx5e_create_q_counters(struct mlx5e_priv *priv)
@@ -5235,6 +5268,10 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev,
 	if (err)
 		mlx5_core_err(mdev, "TLS initialization failed, %d\n", err);
 
+	err = mlx5e_nvmeotcp_init(priv);
+	if (err)
+		mlx5_core_err(mdev, "NVMEoTCP initialization failed, %d\n", err);
+
 	err = mlx5e_devlink_port_register(priv);
 	if (err)
 		mlx5_core_err(mdev, "mlx5e_devlink_port_register failed, %d\n", err);
@@ -5248,6 +5285,7 @@ static void mlx5e_nic_cleanup(struct mlx5e_priv *priv)
 {
 	mlx5e_health_destroy_reporters(priv);
 	mlx5e_devlink_port_unregister(priv);
+	mlx5e_nvmeotcp_cleanup(priv);
 	mlx5e_tls_cleanup(priv);
 	mlx5e_ipsec_cleanup(priv);
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index d54da3797c30..2817615f06b6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -36,6 +36,7 @@
 #include "en/xdp.h"
 #include "en/xsk/rx.h"
 #include "en/xsk/tx.h"
+#include "en_accel/nvmeotcp.h"
 
 static inline bool mlx5e_channel_no_affinity_change(struct mlx5e_channel *c)
 {
@@ -118,6 +119,10 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
 	struct mlx5e_txqsq __rcu **qos_sqs;
 	struct mlx5e_rq *xskrq = &c->xskrq;
 	struct mlx5e_rq *rq = &c->rq;
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	struct mlx5e_nvmeotcp_sq *nvmeotcp_sq;
+	struct list_head *cur;
+#endif
 	bool aff_change = false;
 	bool busy_xsk = false;
 	bool busy = false;
@@ -170,6 +175,12 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
 		 * queueing more WQEs and overflowing the async ICOSQ.
 		 */
 		clear_bit(MLX5E_SQ_STATE_PENDING_XSK_TX, &c->async_icosq.state);
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	list_for_each(cur, &c->list_nvmeotcpsq) {
+		nvmeotcp_sq = list_entry(cur, struct mlx5e_nvmeotcp_sq, list);
+		mlx5e_poll_ico_cq(&nvmeotcp_sq->icosq.cq);
+	}
+#endif
 
 	busy |= INDIRECT_CALL_2(rq->post_wqes,
 				mlx5e_post_rx_mpwqes,
@@ -218,6 +229,12 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
 	mlx5e_cq_arm(&rq->cq);
 	mlx5e_cq_arm(&c->icosq.cq);
 	mlx5e_cq_arm(&c->async_icosq.cq);
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	list_for_each(cur, &c->list_nvmeotcpsq) {
+		nvmeotcp_sq = list_entry(cur, struct mlx5e_nvmeotcp_sq, list);
+		mlx5e_cq_arm(&nvmeotcp_sq->icosq.cq);
+	}
+#endif
 	mlx5e_cq_arm(&c->xdpsq.cq);
 
 	if (xsk_open) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c
index 02558ac2ace6..5e7544ccae91 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c
@@ -256,6 +256,12 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev)
 			return err;
 	}
 
+	if (MLX5_CAP_GEN(dev, nvmeotcp)) {
+		err = mlx5_core_get_caps(dev, MLX5_CAP_DEV_NVMEOTCP);
+		if (err)
+			return err;
+	}
+
 	return 0;
 }
 
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:38 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:38 +0200
Subject: [PATCH v4 net-next  15/21] net/mlx5e: NVMEoTCP use KLM UMRs
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-16-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

NVMEoTCP offload uses buffer registration for ddp operation,
every request comprises from SG list that might have elements with size > 4K,
thus the appropriate way to perform buffer registration is with KLM UMRs.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   5 +-
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   3 +
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 116 ++++++++++++++++++
 .../mlx5/core/en_accel/nvmeotcp_utils.h       |  12 ++
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  12 +-
 5 files changed, 145 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 1982e0a58be7..734b54e3090a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -236,7 +236,10 @@ struct mlx5e_umr_wqe {
 	struct mlx5_wqe_ctrl_seg       ctrl;
 	struct mlx5_wqe_umr_ctrl_seg   uctrl;
 	struct mlx5_mkey_seg           mkc;
-	struct mlx5_mtt                inline_mtts[0];
+	union {
+		struct mlx5_mtt        inline_mtts[0];
+		struct mlx5_klm	       inline_klms[0];
+	};
 };
 
 extern const char mlx5e_self_tests[][ETH_GSTRING_LEN];
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
index 4880f2179273..e997b7230028 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
@@ -34,6 +34,9 @@ enum mlx5e_icosq_wqe_type {
 	MLX5E_ICOSQ_WQE_SET_PSV_TLS,
 	MLX5E_ICOSQ_WQE_GET_PSV_TLS,
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	MLX5E_ICOSQ_WQE_UMR_NVME_TCP,
+#endif
 };
 
 /* General */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index 37a86ca83913..b1c3cb11c96b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -4,6 +4,7 @@
 #include <linux/netdevice.h>
 #include <linux/idr.h>
 #include "en_accel/nvmeotcp.h"
+#include "en_accel/nvmeotcp_utils.h"
 #include "en_accel/fs_tcp.h"
 #include "en/txrx.h"
 
@@ -19,6 +20,121 @@ static const struct rhashtable_params rhash_queues = {
 	.max_size = MAX_NVMEOTCP_QUEUES,
 };
 
+static void
+fill_nvmeotcp_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+		      struct mlx5e_umr_wqe *wqe, u16 ccid, u32 klm_entries,
+		      u16 klm_offset)
+{
+	struct scatterlist *sgl_mkey;
+	u32 lkey, i;
+
+	lkey = queue->priv->mdev->mlx5e_res.mkey.key;
+	for (i = 0; i < klm_entries; i++) {
+		sgl_mkey = &queue->ccid_table[ccid].sgl[i + klm_offset];
+		wqe->inline_klms[i].bcount = cpu_to_be32(sgl_mkey->length);
+		wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
+		wqe->inline_klms[i].va	   = cpu_to_be64(sgl_mkey->dma_address);
+	}
+
+	for (; i < ALIGN(klm_entries, KLM_ALIGNMENT); i++) {
+		wqe->inline_klms[i].bcount = 0;
+		wqe->inline_klms[i].key    = 0;
+		wqe->inline_klms[i].va     = 0;
+	}
+}
+
+static void
+build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
+		       struct mlx5e_umr_wqe *wqe, u16 ccid, int klm_entries,
+		       u32 klm_offset, u32 len)
+{
+	u32 id = queue->ccid_table[ccid].klm_mkey.key;
+	struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl;
+	struct mlx5_wqe_ctrl_seg      *cseg = &wqe->ctrl;
+	struct mlx5_mkey_seg	       *mkc = &wqe->mkc;
+
+	u32 sqn = queue->sq->icosq.sqn;
+	u16 pc = queue->sq->icosq.pc;
+
+	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
+					     MLX5_OPCODE_UMR);
+	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
+				   MLX5E_KLM_UMR_DS_CNT(ALIGN(klm_entries, KLM_ALIGNMENT)));
+	cseg->general_id = cpu_to_be32(id);
+
+	if (!klm_offset) {
+		ucseg->mkey_mask |= cpu_to_be64(MLX5_MKEY_MASK_XLT_OCT_SIZE |
+						MLX5_MKEY_MASK_LEN | MLX5_MKEY_MASK_FREE);
+		mkc->xlt_oct_size = cpu_to_be32(ALIGN(len, KLM_ALIGNMENT));
+		mkc->len = cpu_to_be64(queue->ccid_table[ccid].size);
+	}
+
+	ucseg->flags = MLX5_UMR_INLINE | MLX5_UMR_TRANSLATION_OFFSET_EN;
+	ucseg->xlt_octowords = cpu_to_be16(ALIGN(klm_entries, KLM_ALIGNMENT));
+	ucseg->xlt_offset = cpu_to_be16(klm_offset);
+	fill_nvmeotcp_klm_wqe(queue, wqe, ccid, klm_entries, klm_offset);
+}
+
+static void
+mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
+		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi)
+{
+	struct mlx5e_icosq_wqe_info *wi = &sq->db.wqe_info[pi];
+
+	wi->num_wqebbs = wqe_bbs;
+	wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
+}
+
+static void
+post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+	     u16 ccid,
+	     u32 klm_length,
+	     u32 *klm_offset)
+{
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+	u32 wqe_bbs, cur_klm_entries;
+	struct mlx5e_umr_wqe *wqe;
+	u16 pi, wqe_sz;
+
+	cur_klm_entries = min_t(int, queue->max_klms_per_wqe,
+				klm_length - *klm_offset);
+	wqe_sz = MLX5E_KLM_UMR_WQE_SZ(ALIGN(cur_klm_entries, KLM_ALIGNMENT));
+	wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
+	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
+	wqe = MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi);
+	build_nvmeotcp_klm_umr(queue, wqe, ccid, cur_klm_entries, *klm_offset,
+			       klm_length);
+	*klm_offset += cur_klm_entries;
+	sq->pc += wqe_bbs;
+	sq->doorbell_cseg = &wqe->ctrl;
+}
+
+static int
+mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+			    u16 ccid,
+			    u32 klm_length)
+{
+	u32 klm_offset = 0, wqes, wqe_sz, max_wqe_bbs, i, room;
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+
+	/* TODO: set stricter wqe_sz; using max for now */
+	wqes = DIV_ROUND_UP(klm_length, queue->max_klms_per_wqe);
+	wqe_sz = MLX5E_KLM_UMR_WQE_SZ(queue->max_klms_per_wqe);
+
+	max_wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
+
+	room = mlx5e_stop_room_for_wqe(max_wqe_bbs) * wqes;
+	if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room)))
+		return -ENOSPC;
+
+	for (i = 0; i < wqes; i++)
+		post_klm_wqe(queue, ccid, klm_length, &klm_offset);
+
+	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, sq->doorbell_cseg);
+	return 0;
+}
+
 static int
 mlx5e_nvmeotcp_offload_limits(struct net_device *netdev,
 			      struct tcp_ddp_limits *limits)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
new file mode 100644
index 000000000000..329e114d6571
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021 Mellanox Technologies. */
+#ifndef __MLX5E_NVMEOTCP_UTILS_H__
+#define __MLX5E_NVMEOTCP_UTILS_H__
+
+#include "en.h"
+
+#define MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi) \
+	((struct mlx5e_umr_wqe *)\
+	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_umr_wqe)))
+
+#endif /* __MLX5E_NVMEOTCP_UTILS_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 4de5a97ceac6..df6dc111f6df 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -612,16 +612,20 @@ void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq)
 		ci = mlx5_wq_cyc_ctr2ix(&sq->wq, sqcc);
 		wi = &sq->db.wqe_info[ci];
 		sqcc += wi->num_wqebbs;
-#ifdef CONFIG_MLX5_EN_TLS
 		switch (wi->wqe_type) {
+#ifdef CONFIG_MLX5_EN_TLS
 		case MLX5E_ICOSQ_WQE_SET_PSV_TLS:
 			mlx5e_ktls_handle_ctx_completion(wi);
 			break;
 		case MLX5E_ICOSQ_WQE_GET_PSV_TLS:
 			mlx5e_ktls_handle_get_psv_completion(wi, sq);
 			break;
-		}
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
+			break;
+#endif
+		}
 	}
 	sq->cc = sqcc;
 }
@@ -690,6 +694,10 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)
 			case MLX5E_ICOSQ_WQE_GET_PSV_TLS:
 				mlx5e_ktls_handle_get_psv_completion(wi, sq);
 				break;
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
+				break;
 #endif
 			default:
 				netdev_WARN_ONCE(cq->netdev,
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:40 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:40 +0200
Subject: [PATCH v4 net-next 17/21] net/mlx5e: NVMEoTCP async ddp invalidation
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-18-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

Teardown ddp contexts asynchronously by posting a WQE, and calling back
to nvme-tcp when the corresponding CQE is received.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |  4 ++
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 66 ++++++++++++++++---
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    |  1 +
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  6 ++
 4 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
index 32fa9f1a4a1f..e23d165ba264 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
@@ -36,6 +36,7 @@ enum mlx5e_icosq_wqe_type {
 #endif
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 	MLX5E_ICOSQ_WQE_UMR_NVME_TCP,
+	MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE,
 	MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP,
 #endif
 };
@@ -184,6 +185,9 @@ struct mlx5e_icosq_wqe_info {
 		struct {
 			struct mlx5e_nvmeotcp_queue *queue;
 		} nvmeotcp_q;
+		struct {
+			struct nvmeotcp_queue_entry *entry;
+		} nvmeotcp_qe;
 #endif
 	};
 };
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index fcc1f3c92a06..f79e26419a89 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -152,6 +152,7 @@ enum wqe_type {
 	BSF_KLM_UMR = 1,
 	SET_PSV_UMR = 2,
 	BSF_UMR = 3,
+	KLM_INV_UMR = 4,
 };
 
 static void
@@ -208,6 +209,13 @@ build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 				   MLX5E_KLM_UMR_DS_CNT(ALIGN(klm_entries, KLM_ALIGNMENT)));
 	cseg->general_id = cpu_to_be32(id);
 
+	if (!klm_entries) { /* this is invalidate */
+		ucseg->mkey_mask = cpu_to_be64(MLX5_MKEY_MASK_FREE);
+		ucseg->flags = MLX5_UMR_INLINE;
+		mkc->status = MLX5_MKEY_STATUS_FREE;
+		return;
+	}
+
 	if (klm_type == KLM_UMR && !klm_offset) {
 		ucseg->mkey_mask |= cpu_to_be64(MLX5_MKEY_MASK_XLT_OCT_SIZE |
 						MLX5_MKEY_MASK_LEN | MLX5_MKEY_MASK_FREE);
@@ -308,8 +316,8 @@ build_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
 
 static void
 mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
-		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi,
-		       enum wqe_type type)
+		       struct mlx5e_icosq *sq, u32 wqe_bbs,
+		       u16 pi, u16 ccid, enum wqe_type type)
 {
 	struct mlx5e_icosq_wqe_info *wi = &sq->db.wqe_info[pi];
 
@@ -318,12 +326,17 @@ mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
 	case SET_PSV_UMR:
 		wi->wqe_type = MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP;
 		break;
+	case KLM_INV_UMR:
+		wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE;
+		break;
 	default:
 		wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
 		break;
 	}
 
-	if (type == SET_PSV_UMR)
+	if (type == KLM_INV_UMR)
+		wi->nvmeotcp_qe.entry = &nvmeotcp_queue->ccid_table[ccid];
+	else if (type == SET_PSV_UMR)
 		wi->nvmeotcp_q.queue = nvmeotcp_queue;
 }
 
@@ -338,7 +351,7 @@ mlx5e_nvmeotcp_rx_post_static_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS;
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_STATIC_PARAMS_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(NULL, sq, wqe_bbs, pi, BSF_UMR);
+	mlx5e_nvmeotcp_fill_wi(NULL, sq, wqe_bbs, pi, 0, BSF_UMR);
 	build_nvmeotcp_static_params(queue, wqe, resync_seq, queue->zerocopy, queue->crc_rx);
 	sq->pc += wqe_bbs;
 	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
@@ -355,7 +368,7 @@ mlx5e_nvmeotcp_rx_post_progress_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQEBBS;
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_PROGRESS_PARAMS_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, SET_PSV_UMR);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, 0, SET_PSV_UMR);
 	build_nvmeotcp_progress_params(queue, wqe, seq);
 	sq->pc += wqe_bbs;
 	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
@@ -379,7 +392,8 @@ post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, wqe_type);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, ccid,
+			       klm_length ? KLM_UMR : KLM_INV_UMR);
 	build_nvmeotcp_klm_umr(queue, wqe, ccid, cur_klm_entries, *klm_offset,
 			       klm_length, wqe_type);
 	*klm_offset += cur_klm_entries;
@@ -397,8 +411,13 @@ mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	struct mlx5e_icosq *sq = &queue->sq->icosq;
 
 	/* TODO: set stricter wqe_sz; using max for now */
-	wqes = DIV_ROUND_UP(klm_length, queue->max_klms_per_wqe);
-	wqe_sz = MLX5E_KLM_UMR_WQE_SZ(queue->max_klms_per_wqe);
+	if (klm_length == 0) {
+		wqes = 1;
+		wqe_sz = MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS;
+	} else {
+		wqes = DIV_ROUND_UP(klm_length, queue->max_klms_per_wqe);
+		wqe_sz = MLX5E_KLM_UMR_WQE_SZ(queue->max_klms_per_wqe);
+	}
 
 	max_wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
 
@@ -738,6 +757,24 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	return 0;
 }
 
+void mlx5e_nvmeotcp_ddp_inv_done(struct mlx5e_icosq_wqe_info *wi)
+{
+	struct nvmeotcp_queue_entry *q_entry = wi->nvmeotcp_qe.entry;
+	struct mlx5e_nvmeotcp_queue *queue = q_entry->queue;
+	struct mlx5_core_dev *mdev = queue->priv->mdev;
+	struct tcp_ddp_io *ddp = q_entry->ddp;
+	const struct tcp_ddp_ulp_ops *ulp_ops;
+
+	dma_unmap_sg(mdev->device, ddp->sg_table.sgl,
+		     q_entry->sgl_length, DMA_FROM_DEVICE);
+
+	q_entry->sgl_length = 0;
+
+	ulp_ops = inet_csk(queue->sk)->icsk_ulp_ddp_ops;
+	if (ulp_ops && ulp_ops->ddp_teardown_done)
+		ulp_ops->ddp_teardown_done(q_entry->ddp_ctx);
+}
+
 void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi)
 {
 	struct mlx5e_nvmeotcp_queue *queue = wi->nvmeotcp_q.queue;
@@ -754,6 +791,19 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 			    struct tcp_ddp_io *ddp,
 			    void *ddp_ctx)
 {
+	struct mlx5e_nvmeotcp_queue *queue =
+		(struct mlx5e_nvmeotcp_queue *)tcp_ddp_get_ctx(sk);
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct nvmeotcp_queue_entry *q_entry;
+
+	q_entry  = &queue->ccid_table[ddp->command_id];
+	WARN_ON(q_entry->sgl_length == 0);
+
+	q_entry->ddp_ctx = ddp_ctx;
+	q_entry->queue = queue;
+
+	mlx5e_nvmeotcp_post_klm_wqe(queue, KLM_UMR, ddp->command_id, 0);
+
 	return 0;
 }
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
index d0e515502d6d..5a613addfc7c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
@@ -103,6 +103,7 @@ void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv);
 struct mlx5e_nvmeotcp_queue *
 mlx5e_nvmeotcp_get_queue(struct mlx5e_nvmeotcp *nvmeotcp, int id);
 void mlx5e_nvmeotcp_put_queue(struct mlx5e_nvmeotcp_queue *queue);
+void mlx5e_nvmeotcp_ddp_inv_done(struct mlx5e_icosq_wqe_info *wi);
 void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi);
 int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv);
 void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 9f7b7239239f..2fe20d901bbf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -625,6 +625,9 @@ void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 			break;
+		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE:
+			mlx5e_nvmeotcp_ddp_inv_done(wi);
+			break;
 		case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
 			mlx5e_nvmeotcp_ctx_comp(wi);
 			break;
@@ -702,6 +705,9 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 				break;
+			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP_INVALIDATE:
+				mlx5e_nvmeotcp_ddp_inv_done(wi);
+				break;
 			case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
 				mlx5e_nvmeotcp_ctx_comp(wi);
 				break;
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:39 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:39 +0200
Subject: [PATCH v4 net-next  16/21] net/mlx5e: NVMEoTCP queue init/teardown
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-17-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

When nvme-tcp establishes new connections, we allocate a hardware
context to offload operations for this queue:
- Use a separate TIR to identify the queue and maintain the HW context
- Use a separate ICOSQ for maintain the HW context
- Use a separate tag buffer for buffer registration
- Maintain static and progress HW contexts by posting the proper
  WQEs at creation time, or upon resync

Queue teardown will free the corresponding contexts.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   6 +
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 653 +++++++++++++++++-
 .../mellanox/mlx5/core/en_accel/nvmeotcp.h    |   4 +
 .../mlx5/core/en_accel/nvmeotcp_utils.h       |  68 ++
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |   7 +
 5 files changed, 714 insertions(+), 24 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
index e997b7230028..32fa9f1a4a1f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
@@ -36,6 +36,7 @@ enum mlx5e_icosq_wqe_type {
 #endif
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 	MLX5E_ICOSQ_WQE_UMR_NVME_TCP,
+	MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP,
 #endif
 };
 
@@ -178,6 +179,11 @@ struct mlx5e_icosq_wqe_info {
 		struct {
 			struct mlx5e_ktls_rx_resync_buf *buf;
 		} tls_get_params;
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+		struct {
+			struct mlx5e_nvmeotcp_queue *queue;
+		} nvmeotcp_q;
 #endif
 	};
 };
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index b1c3cb11c96b..fcc1f3c92a06 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -3,6 +3,7 @@
 
 #include <linux/netdevice.h>
 #include <linux/idr.h>
+#include <linux/nvme-tcp.h>
 #include "en_accel/nvmeotcp.h"
 #include "en_accel/nvmeotcp_utils.h"
 #include "en_accel/fs_tcp.h"
@@ -20,35 +21,180 @@ static const struct rhashtable_params rhash_queues = {
 	.max_size = MAX_NVMEOTCP_QUEUES,
 };
 
+#define MLX5_NVME_TCP_MAX_SEGMENTS 128
+
+static u32 mlx5e_get_max_sgl(struct mlx5_core_dev *mdev)
+{
+	return min_t(u32,
+		     MLX5_NVME_TCP_MAX_SEGMENTS,
+		     1 << MLX5_CAP_GEN(mdev, log_max_klm_list_size));
+}
+
+static void mlx5e_nvmeotcp_destroy_tir(struct mlx5e_priv *priv, int tirn)
+{
+	mlx5_core_destroy_tir(priv->mdev, tirn);
+}
+
+static inline u32
+mlx5e_get_channel_ix_from_io_cpu(struct mlx5e_priv *priv, u32 io_cpu)
+{
+	int num_channels = priv->channels.params.num_channels;
+	u32 channel_ix = io_cpu;
+
+	if (channel_ix >= num_channels)
+		channel_ix = channel_ix % num_channels;
+
+	return channel_ix;
+}
+
+static int mlx5e_nvmeotcp_create_tir(struct mlx5e_priv *priv,
+				     struct sock *sk,
+				     struct nvme_tcp_ddp_config *config,
+				     struct mlx5e_nvmeotcp_queue *queue,
+				     bool zerocopy, bool crc_rx)
+{
+	u32 rqtn = priv->direct_tir[queue->channel_ix].rqt.rqtn;
+	int err, inlen;
+	void *tirc;
+	u32 tirn;
+	u32 *in;
+
+	inlen = MLX5_ST_SZ_BYTES(create_tir_in);
+	in = kvzalloc(inlen, GFP_KERNEL);
+	if (!in)
+		return -ENOMEM;
+	tirc = MLX5_ADDR_OF(create_tir_in, in, ctx);
+	MLX5_SET(tirc, tirc, disp_type, MLX5_TIRC_DISP_TYPE_INDIRECT);
+	MLX5_SET(tirc, tirc, rx_hash_fn, MLX5_RX_HASH_FN_INVERTED_XOR8);
+	MLX5_SET(tirc, tirc, indirect_table, rqtn);
+	MLX5_SET(tirc, tirc, transport_domain, priv->mdev->mlx5e_res.td.tdn);
+	if (zerocopy) {
+		MLX5_SET(tirc, tirc, nvmeotcp_zero_copy_en, 1);
+		MLX5_SET(tirc, tirc, nvmeotcp_tag_buffer_table_id,
+			 queue->tag_buf_table_id);
+	}
+
+	if (crc_rx)
+		MLX5_SET(tirc, tirc, nvmeotcp_crc_en, 1);
+
+	MLX5_SET(tirc, tirc, self_lb_block,
+		 MLX5_TIRC_SELF_LB_BLOCK_BLOCK_UNICAST |
+		 MLX5_TIRC_SELF_LB_BLOCK_BLOCK_MULTICAST);
+	err = mlx5_core_create_tir(priv->mdev, in, &tirn);
+
+	if (!err)
+		queue->tirn = tirn;
+
+	kvfree(in);
+	return err;
+}
+
+static
+int mlx5e_create_nvmeotcp_tag_buf_table(struct mlx5_core_dev *mdev,
+					struct mlx5e_nvmeotcp_queue *queue,
+					u8 log_table_size)
+{
+	u32 in[MLX5_ST_SZ_DW(create_nvmeotcp_tag_buf_table_in)] = {};
+	u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)];
+	u64 general_obj_types;
+	void *obj;
+	int err;
+
+	obj = MLX5_ADDR_OF(create_nvmeotcp_tag_buf_table_in, in,
+			   nvmeotcp_tag_buf_table_obj);
+
+	general_obj_types = MLX5_CAP_GEN_64(mdev, general_obj_types);
+	if (!(general_obj_types &
+	      MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE))
+		return -EINVAL;
+
+	MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+		 MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+	MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
+		 MLX5_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE);
+	MLX5_SET(nvmeotcp_tag_buf_table_obj, obj,
+		 log_tag_buffer_table_size, log_table_size);
+
+	err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+	if (!err)
+		queue->tag_buf_table_id = MLX5_GET(general_obj_out_cmd_hdr,
+						   out, obj_id);
+	return err;
+}
+
+static
+void mlx5_destroy_nvmeotcp_tag_buf_table(struct mlx5_core_dev *mdev, u32 uid)
+{
+	u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {};
+	u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)];
+
+	MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+		 MLX5_CMD_OP_DESTROY_GENERAL_OBJECT);
+	MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
+		 MLX5_GENERAL_OBJECT_TYPES_NVMEOTCP_TAG_BUFFER_TABLE);
+	MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, uid);
+
+	mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_TIR_PARAMS 0x2
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_STATIC_PARAMS 0x2
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_UMR 0x0
+
+#define STATIC_PARAMS_DS_CNT \
+	DIV_ROUND_UP(MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ, MLX5_SEND_WQE_DS)
+
+#define PROGRESS_PARAMS_DS_CNT \
+	DIV_ROUND_UP(MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ, MLX5_SEND_WQE_DS)
+
+enum wqe_type {
+	KLM_UMR = 0,
+	BSF_KLM_UMR = 1,
+	SET_PSV_UMR = 2,
+	BSF_UMR = 3,
+};
+
 static void
 fill_nvmeotcp_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 		      struct mlx5e_umr_wqe *wqe, u16 ccid, u32 klm_entries,
-		      u16 klm_offset)
+		      u16 klm_offset, enum wqe_type klm_type)
 {
 	struct scatterlist *sgl_mkey;
 	u32 lkey, i;
 
-	lkey = queue->priv->mdev->mlx5e_res.mkey.key;
-	for (i = 0; i < klm_entries; i++) {
-		sgl_mkey = &queue->ccid_table[ccid].sgl[i + klm_offset];
-		wqe->inline_klms[i].bcount = cpu_to_be32(sgl_mkey->length);
-		wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
-		wqe->inline_klms[i].va	   = cpu_to_be64(sgl_mkey->dma_address);
-	}
-
-	for (; i < ALIGN(klm_entries, KLM_ALIGNMENT); i++) {
-		wqe->inline_klms[i].bcount = 0;
-		wqe->inline_klms[i].key    = 0;
-		wqe->inline_klms[i].va     = 0;
+	if (klm_type == BSF_KLM_UMR) {
+		for (i = 0; i < klm_entries; i++) {
+			lkey = queue->ccid_table[i + klm_offset].klm_mkey.key;
+			wqe->inline_klms[i].bcount = cpu_to_be32(1);
+			wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
+			wqe->inline_klms[i].va	   = 0;
+		}
+	} else {
+		lkey = queue->priv->mdev->mlx5e_res.mkey.key;
+		for (i = 0; i < klm_entries; i++) {
+			sgl_mkey = &queue->ccid_table[ccid].sgl[i + klm_offset];
+			wqe->inline_klms[i].bcount = cpu_to_be32(sgl_mkey->length);
+			wqe->inline_klms[i].key	   = cpu_to_be32(lkey);
+			wqe->inline_klms[i].va	   = cpu_to_be64(sgl_mkey->dma_address);
+		}
+
+		for (; i < ALIGN(klm_entries, KLM_ALIGNMENT); i++) {
+			wqe->inline_klms[i].bcount = 0;
+			wqe->inline_klms[i].key    = 0;
+			wqe->inline_klms[i].va     = 0;
+		}
 	}
 }
 
 static void
 build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 		       struct mlx5e_umr_wqe *wqe, u16 ccid, int klm_entries,
-		       u32 klm_offset, u32 len)
+		       u32 klm_offset, u32 len, enum wqe_type klm_type)
 {
-	u32 id = queue->ccid_table[ccid].klm_mkey.key;
+	u32 id = (klm_type == KLM_UMR) ? queue->ccid_table[ccid].klm_mkey.key :
+		(queue->tirn << MLX5_WQE_CTRL_TIR_TIS_INDEX_SHIFT);
+	u8 opc_mod = (klm_type == KLM_UMR) ? MLX5_CTRL_SEGMENT_OPC_MOD_UMR_UMR :
+		MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_STATIC_PARAMS;
 	struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl;
 	struct mlx5_wqe_ctrl_seg      *cseg = &wqe->ctrl;
 	struct mlx5_mkey_seg	       *mkc = &wqe->mkc;
@@ -57,12 +203,12 @@ build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 	u16 pc = queue->sq->icosq.pc;
 
 	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
-					     MLX5_OPCODE_UMR);
+					     MLX5_OPCODE_UMR | (opc_mod) << 24);
 	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
 				   MLX5E_KLM_UMR_DS_CNT(ALIGN(klm_entries, KLM_ALIGNMENT)));
 	cseg->general_id = cpu_to_be32(id);
 
-	if (!klm_offset) {
+	if (klm_type == KLM_UMR && !klm_offset) {
 		ucseg->mkey_mask |= cpu_to_be64(MLX5_MKEY_MASK_XLT_OCT_SIZE |
 						MLX5_MKEY_MASK_LEN | MLX5_MKEY_MASK_FREE);
 		mkc->xlt_oct_size = cpu_to_be32(ALIGN(len, KLM_ALIGNMENT));
@@ -72,21 +218,152 @@ build_nvmeotcp_klm_umr(struct mlx5e_nvmeotcp_queue *queue,
 	ucseg->flags = MLX5_UMR_INLINE | MLX5_UMR_TRANSLATION_OFFSET_EN;
 	ucseg->xlt_octowords = cpu_to_be16(ALIGN(klm_entries, KLM_ALIGNMENT));
 	ucseg->xlt_offset = cpu_to_be16(klm_offset);
-	fill_nvmeotcp_klm_wqe(queue, wqe, ccid, klm_entries, klm_offset);
+	fill_nvmeotcp_klm_wqe(queue, wqe, ccid, klm_entries, klm_offset, klm_type);
+}
+
+static void
+fill_nvmeotcp_progress_params(struct mlx5e_nvmeotcp_queue *queue,
+			      struct mlx5_seg_nvmeotcp_progress_params *params,
+			      u32 seq)
+{
+	void *ctx = params->ctx;
+
+	params->tir_num = cpu_to_be32(queue->tirn);
+
+	MLX5_SET(nvmeotcp_progress_params, ctx,
+		 next_pdu_tcp_sn, seq);
+	MLX5_SET(nvmeotcp_progress_params, ctx, pdu_tracker_state,
+		 MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_START);
+}
+
+void
+build_nvmeotcp_progress_params(struct mlx5e_nvmeotcp_queue *queue,
+			       struct mlx5e_set_nvmeotcp_progress_params_wqe *wqe,
+			       u32 seq)
+{
+	struct mlx5_wqe_ctrl_seg *cseg = &wqe->ctrl;
+	u32 sqn = queue->sq->icosq.sqn;
+	u16 pc = queue->sq->icosq.pc;
+	u8 opc_mod;
+
+	memset(wqe, 0, MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ);
+	opc_mod = MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_PROGRESS_PARAMS;
+	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
+					     MLX5_OPCODE_SET_PSV | (opc_mod << 24));
+	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
+				   PROGRESS_PARAMS_DS_CNT);
+	fill_nvmeotcp_progress_params(queue, &wqe->params, seq);
+}
+
+static void
+fill_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
+			    struct mlx5_seg_nvmeotcp_static_params *params,
+			    u32 resync_seq, bool zero_copy_en,
+			    bool ddgst_offload_en)
+{
+	void *ctx = params->ctx;
+
+	MLX5_SET(transport_static_params, ctx, const_1, 1);
+	MLX5_SET(transport_static_params, ctx, const_2, 2);
+	MLX5_SET(transport_static_params, ctx, acc_type,
+		 MLX5_TRANSPORT_STATIC_PARAMS_ACC_TYPE_NVMETCP);
+	MLX5_SET(transport_static_params, ctx, nvme_resync_tcp_sn, resync_seq);
+	MLX5_SET(transport_static_params, ctx, pda, queue->pda);
+	MLX5_SET(transport_static_params, ctx, ddgst_en,
+		 queue->dgst & NVME_TCP_DATA_DIGEST_ENABLE);
+	MLX5_SET(transport_static_params, ctx, ddgst_offload_en, ddgst_offload_en);
+	MLX5_SET(transport_static_params, ctx, hddgst_en,
+		 queue->dgst & NVME_TCP_HDR_DIGEST_ENABLE);
+	MLX5_SET(transport_static_params, ctx, hdgst_offload_en, 0);
+	MLX5_SET(transport_static_params, ctx, ti,
+		 MLX5_TRANSPORT_STATIC_PARAMS_TI_INITIATOR);
+	MLX5_SET(transport_static_params, ctx, const1, 1);
+	MLX5_SET(transport_static_params, ctx, zero_copy_en, zero_copy_en);
+}
+
+void
+build_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
+			     struct mlx5e_set_nvmeotcp_static_params_wqe *wqe,
+			     u32 resync_seq, bool zerocopy, bool crc_rx)
+{
+	u8 opc_mod = MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_STATIC_PARAMS;
+	struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl;
+	struct mlx5_wqe_ctrl_seg      *cseg = &wqe->ctrl;
+	u32 sqn = queue->sq->icosq.sqn;
+	u16 pc = queue->sq->icosq.pc;
+
+	memset(wqe, 0, MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ);
+
+	cseg->opmod_idx_opcode = cpu_to_be32((pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) |
+					     MLX5_OPCODE_UMR | (opc_mod) << 24);
+	cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
+				   STATIC_PARAMS_DS_CNT);
+	cseg->imm = cpu_to_be32(queue->tirn << MLX5_WQE_CTRL_TIR_TIS_INDEX_SHIFT);
+
+	ucseg->flags = MLX5_UMR_INLINE;
+	ucseg->bsf_octowords =
+		cpu_to_be16(MLX5E_NVMEOTCP_STATIC_PARAMS_OCTWORD_SIZE);
+	fill_nvmeotcp_static_params(queue, &wqe->params, resync_seq, zerocopy, crc_rx);
 }
 
 static void
 mlx5e_nvmeotcp_fill_wi(struct mlx5e_nvmeotcp_queue *nvmeotcp_queue,
-		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi)
+		       struct mlx5e_icosq *sq, u32 wqe_bbs, u16 pi,
+		       enum wqe_type type)
 {
 	struct mlx5e_icosq_wqe_info *wi = &sq->db.wqe_info[pi];
 
 	wi->num_wqebbs = wqe_bbs;
-	wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
+	switch (type) {
+	case SET_PSV_UMR:
+		wi->wqe_type = MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP;
+		break;
+	default:
+		wi->wqe_type = MLX5E_ICOSQ_WQE_UMR_NVME_TCP;
+		break;
+	}
+
+	if (type == SET_PSV_UMR)
+		wi->nvmeotcp_q.queue = nvmeotcp_queue;
+}
+
+static void
+mlx5e_nvmeotcp_rx_post_static_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
+					 u32 resync_seq)
+{
+	struct mlx5e_set_nvmeotcp_static_params_wqe *wqe;
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+	u16 pi, wqe_bbs;
+
+	wqe_bbs = MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS;
+	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
+	wqe = MLX5E_NVMEOTCP_FETCH_STATIC_PARAMS_WQE(sq, pi);
+	mlx5e_nvmeotcp_fill_wi(NULL, sq, wqe_bbs, pi, BSF_UMR);
+	build_nvmeotcp_static_params(queue, wqe, resync_seq, queue->zerocopy, queue->crc_rx);
+	sq->pc += wqe_bbs;
+	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
+}
+
+static void
+mlx5e_nvmeotcp_rx_post_progress_params_wqe(struct mlx5e_nvmeotcp_queue *queue,
+					   u32 seq)
+{
+	struct mlx5e_set_nvmeotcp_progress_params_wqe *wqe;
+	struct mlx5e_icosq *sq = &queue->sq->icosq;
+	u16 pi, wqe_bbs;
+
+	wqe_bbs = MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQEBBS;
+	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
+	wqe = MLX5E_NVMEOTCP_FETCH_PROGRESS_PARAMS_WQE(sq, pi);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, SET_PSV_UMR);
+	build_nvmeotcp_progress_params(queue, wqe, seq);
+	sq->pc += wqe_bbs;
+	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, &wqe->ctrl);
 }
 
 static void
 post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+	     enum wqe_type wqe_type,
 	     u16 ccid,
 	     u32 klm_length,
 	     u32 *klm_offset)
@@ -102,9 +379,9 @@ post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 	wqe_bbs = DIV_ROUND_UP(wqe_sz, MLX5_SEND_WQE_BB);
 	pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs);
 	wqe = MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi);
-	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi);
+	mlx5e_nvmeotcp_fill_wi(queue, sq, wqe_bbs, pi, wqe_type);
 	build_nvmeotcp_klm_umr(queue, wqe, ccid, cur_klm_entries, *klm_offset,
-			       klm_length);
+			       klm_length, wqe_type);
 	*klm_offset += cur_klm_entries;
 	sq->pc += wqe_bbs;
 	sq->doorbell_cseg = &wqe->ctrl;
@@ -112,6 +389,7 @@ post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 
 static int
 mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
+			    enum wqe_type wqe_type,
 			    u16 ccid,
 			    u32 klm_length)
 {
@@ -129,31 +407,327 @@ mlx5e_nvmeotcp_post_klm_wqe(struct mlx5e_nvmeotcp_queue *queue,
 		return -ENOSPC;
 
 	for (i = 0; i < wqes; i++)
-		post_klm_wqe(queue, ccid, klm_length, &klm_offset);
+		post_klm_wqe(queue, wqe_type, ccid, klm_length, &klm_offset);
 
 	mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, sq->doorbell_cseg);
 	return 0;
 }
 
+static int mlx5e_create_nvmeotcp_mkey(struct mlx5_core_dev *mdev,
+				      u8 access_mode,
+				      u32 translation_octword_size,
+				      struct mlx5_core_mkey *mkey)
+{
+	int inlen = MLX5_ST_SZ_BYTES(create_mkey_in);
+	void *mkc;
+	u32 *in;
+	int err;
+
+	in = kvzalloc(inlen, GFP_KERNEL);
+	if (!in)
+		return -ENOMEM;
+
+	mkc = MLX5_ADDR_OF(create_mkey_in, in, memory_key_mkey_entry);
+	MLX5_SET(mkc, mkc, free, 1);
+	MLX5_SET(mkc, mkc, translations_octword_size, translation_octword_size);
+	MLX5_SET(mkc, mkc, umr_en, 1);
+	MLX5_SET(mkc, mkc, lw, 1);
+	MLX5_SET(mkc, mkc, lr, 1);
+	MLX5_SET(mkc, mkc, access_mode_1_0, access_mode);
+
+	MLX5_SET(mkc, mkc, qpn, 0xffffff);
+	MLX5_SET(mkc, mkc, pd, mdev->mlx5e_res.pdn);
+
+	err = mlx5_core_create_mkey(mdev, mkey, in, inlen);
+
+	kvfree(in);
+	return err;
+}
+
 static int
 mlx5e_nvmeotcp_offload_limits(struct net_device *netdev,
 			      struct tcp_ddp_limits *limits)
 {
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+
+	limits->max_ddp_sgl_len = mlx5e_get_max_sgl(mdev);
+	return 0;
+}
+
+static void
+mlx5e_nvmeotcp_destroy_sq(struct mlx5e_nvmeotcp_sq *nvmeotcpsq)
+{
+	mlx5e_deactivate_icosq(&nvmeotcpsq->icosq);
+	mlx5e_close_icosq(&nvmeotcpsq->icosq);
+	mlx5e_close_cq(&nvmeotcpsq->icosq.cq);
+	list_del(&nvmeotcpsq->list);
+	kfree(nvmeotcpsq);
+}
+
+static int
+mlx5e_nvmeotcp_build_icosq(struct mlx5e_nvmeotcp_queue *queue,
+			   struct mlx5e_priv *priv)
+{
+	u16 max_sgl, max_klm_per_wqe, max_umr_per_ccid, sgl_rest, wqebbs_rest;
+	struct mlx5e_channel *c = priv->channels.c[queue->channel_ix];
+	struct mlx5e_sq_param icosq_param = {0};
+	struct dim_cq_moder icocq_moder = {0};
+	struct mlx5e_nvmeotcp_sq *nvmeotcp_sq;
+	struct mlx5e_create_cq_param ccp;
+	struct mlx5e_icosq *icosq;
+	int err = -ENOMEM;
+	u16 log_icosq_sz;
+	u32 max_wqebbs;
+
+	nvmeotcp_sq = kzalloc(sizeof(*nvmeotcp_sq), GFP_KERNEL);
+	if (!nvmeotcp_sq)
+		return err;
+
+	icosq = &nvmeotcp_sq->icosq;
+	max_sgl = mlx5e_get_max_sgl(priv->mdev);
+	max_klm_per_wqe = queue->max_klms_per_wqe;
+	max_umr_per_ccid = max_sgl / max_klm_per_wqe;
+	sgl_rest = max_sgl % max_klm_per_wqe;
+	wqebbs_rest = sgl_rest ? MLX5E_KLM_UMR_WQEBBS(sgl_rest) : 0;
+	max_wqebbs = (MLX5E_KLM_UMR_WQEBBS(max_klm_per_wqe) *
+		     max_umr_per_ccid + wqebbs_rest) * queue->size;
+	log_icosq_sz = order_base_2(max_wqebbs);
+
+	mlx5e_build_icosq_param(priv, log_icosq_sz, &icosq_param);
+	mlx5e_build_create_cq_param(&ccp, c);
+	err = mlx5e_open_cq(priv, icocq_moder, &icosq_param.cqp, &ccp, &icosq->cq);
+	if (err)
+		goto err_nvmeotcp_sq;
+
+	err = mlx5e_open_icosq(c, &priv->channels.params, &icosq_param, icosq);
+	if (err)
+		goto close_cq;
+
+	INIT_LIST_HEAD(&nvmeotcp_sq->list);
+	spin_lock(&c->nvmeotcp_icosq_lock);
+	list_add(&nvmeotcp_sq->list, &c->list_nvmeotcpsq);
+	spin_unlock(&c->nvmeotcp_icosq_lock);
+	queue->sq = nvmeotcp_sq;
+	mlx5e_activate_icosq(icosq);
+	return 0;
+
+close_cq:
+	mlx5e_close_cq(&icosq->cq);
+err_nvmeotcp_sq:
+	kfree(nvmeotcp_sq);
+
+	return err;
+}
+
+static void
+mlx5e_nvmeotcp_destroy_rx(struct mlx5e_nvmeotcp_queue *queue,
+			  struct mlx5_core_dev *mdev, bool zerocopy)
+{
+	int i;
+
+	mlx5e_accel_fs_del_sk(queue->fh);
+	for (i = 0; i < queue->size && zerocopy; i++)
+		mlx5_core_destroy_mkey(mdev, &queue->ccid_table[i].klm_mkey);
+
+	mlx5e_nvmeotcp_destroy_tir(queue->priv, queue->tirn);
+	if (zerocopy) {
+		kfree(queue->ccid_table);
+		mlx5_destroy_nvmeotcp_tag_buf_table(mdev, queue->tag_buf_table_id);
+	}
+
+	mlx5e_nvmeotcp_destroy_sq(queue->sq);
+}
+
+static int
+mlx5e_nvmeotcp_queue_rx_init(struct mlx5e_nvmeotcp_queue *queue,
+			     struct nvme_tcp_ddp_config *config,
+			     struct net_device *netdev,
+			     bool zerocopy, bool crc)
+{
+	u8 log_queue_size = order_base_2(config->queue_size);
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+	struct sock *sk = queue->sk;
+	int err, max_sgls, i;
+
+	if (zerocopy) {
+		if (config->queue_size >
+		    BIT(MLX5_CAP_DEV_NVMEOTCP(mdev, log_max_nvmeotcp_tag_buffer_size))) {
+			return -EINVAL;
+		}
+
+		err = mlx5e_create_nvmeotcp_tag_buf_table(mdev, queue, log_queue_size);
+		if (err)
+			return err;
+	}
+
+	err = mlx5e_nvmeotcp_build_icosq(queue, priv);
+	if (err)
+		goto destroy_tag_buffer_table;
+
+	/* initializes queue->tirn */
+	err = mlx5e_nvmeotcp_create_tir(priv, sk, config, queue, zerocopy, crc);
+	if (err)
+		goto destroy_icosq;
+
+	mlx5e_nvmeotcp_rx_post_static_params_wqe(queue, 0);
+	mlx5e_nvmeotcp_rx_post_progress_params_wqe(queue, tcp_sk(sk)->copied_seq);
+
+	if (zerocopy) {
+		queue->ccid_table = kcalloc(queue->size,
+					    sizeof(struct nvmeotcp_queue_entry),
+					    GFP_KERNEL);
+		if (!queue->ccid_table) {
+			err = -ENOMEM;
+			goto destroy_tir;
+		}
+
+		max_sgls = mlx5e_get_max_sgl(mdev);
+		for (i = 0; i < queue->size; i++) {
+			err = mlx5e_create_nvmeotcp_mkey(mdev,
+							 MLX5_MKC_ACCESS_MODE_KLMS,
+							 max_sgls,
+							 &queue->ccid_table[i].klm_mkey);
+			if (err)
+				goto free_sgl;
+		}
+
+		err = mlx5e_nvmeotcp_post_klm_wqe(queue, BSF_KLM_UMR, 0, queue->size);
+		if (err)
+			goto free_sgl;
+	}
+
+	if (!(WARN_ON(!wait_for_completion_timeout(&queue->done, 0))))
+		queue->fh = mlx5e_accel_fs_add_sk(priv, sk, queue->tirn, queue->id);
+
+	if (IS_ERR_OR_NULL(queue->fh)) {
+		err = -EINVAL;
+		goto free_sgl;
+	}
+
 	return 0;
+
+free_sgl:
+	while ((i--) && zerocopy)
+		mlx5_core_destroy_mkey(mdev, &queue->ccid_table[i].klm_mkey);
+
+	if (zerocopy)
+		kfree(queue->ccid_table);
+destroy_tir:
+	mlx5e_nvmeotcp_destroy_tir(priv, queue->tirn);
+destroy_icosq:
+	mlx5e_nvmeotcp_destroy_sq(queue->sq);
+destroy_tag_buffer_table:
+	if (zerocopy)
+		mlx5_destroy_nvmeotcp_tag_buf_table(mdev, queue->tag_buf_table_id);
+
+	return err;
 }
 
+#define OCTWORD_SHIFT 4
+#define MAX_DS_VALUE 63
 static int
 mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 			  struct sock *sk,
 			  struct tcp_ddp_config *tconfig)
 {
-	return 0;
+	struct nvme_tcp_ddp_config *config = (struct nvme_tcp_ddp_config *)tconfig;
+	bool crc_rx = ((netdev->features & NETIF_F_HW_TCP_DDP_CRC_RX) &&
+		       (config->dgst & NVME_TCP_DATA_DIGEST_ENABLE));
+	bool zerocopy = (netdev->features & NETIF_F_HW_TCP_DDP);
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+	struct mlx5e_nvmeotcp_queue *queue;
+	int max_wqe_sz_cap, queue_id, err;
+
+	if (tconfig->type != TCP_DDP_NVME) {
+		err = -EOPNOTSUPP;
+		goto out;
+	}
+
+	queue = kzalloc(sizeof(*queue), GFP_KERNEL);
+	if (!queue) {
+		err = -ENOMEM;
+		goto out;
+	}
+
+	queue_id = ida_simple_get(&priv->nvmeotcp->queue_ids,
+				  MIN_NVMEOTCP_QUEUES, MAX_NVMEOTCP_QUEUES,
+				  GFP_KERNEL);
+	if (queue_id < 0) {
+		err = -ENOSPC;
+		goto free_queue;
+	}
+
+	queue->crc_rx = crc_rx;
+	queue->zerocopy = zerocopy;
+	queue->tcp_ddp_ctx.type = TCP_DDP_NVME;
+	queue->sk = sk;
+	queue->id = queue_id;
+	queue->dgst = config->dgst;
+	queue->pda = config->cpda;
+	queue->channel_ix = mlx5e_get_channel_ix_from_io_cpu(priv,
+							     config->io_cpu);
+	queue->size = config->queue_size;
+	max_wqe_sz_cap  = min_t(int, MAX_DS_VALUE * MLX5_SEND_WQE_DS,
+				MLX5_CAP_GEN(mdev, max_wqe_sz_sq) << OCTWORD_SHIFT);
+	queue->max_klms_per_wqe = MLX5E_KLM_ENTRIES_PER_WQE(max_wqe_sz_cap);
+	queue->priv = priv;
+	init_completion(&queue->done);
+
+	if (zerocopy || crc_rx) {
+		err = mlx5e_nvmeotcp_queue_rx_init(queue, config, netdev,
+						   zerocopy, crc_rx);
+		if (err)
+			goto remove_queue_id;
+	}
+
+	err = rhashtable_insert_fast(&priv->nvmeotcp->queue_hash, &queue->hash,
+				     rhash_queues);
+	if (err)
+		goto destroy_rx;
+
+	write_lock_bh(&sk->sk_callback_lock);
+	tcp_ddp_set_ctx(sk, queue);
+	write_unlock_bh(&sk->sk_callback_lock);
+	refcount_set(&queue->ref_count, 1);
+	return err;
+
+destroy_rx:
+	if (zerocopy || crc_rx)
+		mlx5e_nvmeotcp_destroy_rx(queue, mdev, zerocopy);
+remove_queue_id:
+	ida_simple_remove(&priv->nvmeotcp->queue_ids, queue_id);
+free_queue:
+	kfree(queue);
+out:
+	return err;
 }
 
 static void
 mlx5e_nvmeotcp_queue_teardown(struct net_device *netdev,
 			      struct sock *sk)
 {
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct mlx5_core_dev *mdev = priv->mdev;
+	struct mlx5e_nvmeotcp_queue *queue;
+
+	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
+
+	napi_synchronize(&priv->channels.c[queue->channel_ix]->napi);
+
+	WARN_ON(refcount_read(&queue->ref_count) != 1);
+	if (queue->zerocopy | queue->crc_rx)
+		mlx5e_nvmeotcp_destroy_rx(queue, mdev, queue->zerocopy);
+
+	rhashtable_remove_fast(&priv->nvmeotcp->queue_hash, &queue->hash,
+			       rhash_queues);
+	ida_simple_remove(&priv->nvmeotcp->queue_ids, queue->id);
+	write_lock_bh(&sk->sk_callback_lock);
+	tcp_ddp_set_ctx(sk, NULL);
+	write_unlock_bh(&sk->sk_callback_lock);
+	mlx5e_nvmeotcp_put_queue(queue);
 }
 
 static int
@@ -164,6 +738,16 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	return 0;
 }
 
+void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi)
+{
+	struct mlx5e_nvmeotcp_queue *queue = wi->nvmeotcp_q.queue;
+
+	if (unlikely(!queue))
+		return;
+
+	complete(&queue->done);
+}
+
 static int
 mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 			    struct sock *sk,
@@ -188,6 +772,27 @@ static const struct tcp_ddp_dev_ops mlx5e_nvmeotcp_ops = {
 	.tcp_ddp_resync = mlx5e_nvmeotcp_dev_resync,
 };
 
+struct mlx5e_nvmeotcp_queue *
+mlx5e_nvmeotcp_get_queue(struct mlx5e_nvmeotcp *nvmeotcp, int id)
+{
+	struct mlx5e_nvmeotcp_queue *queue;
+
+	rcu_read_lock();
+	queue = rhashtable_lookup_fast(&nvmeotcp->queue_hash,
+				       &id, rhash_queues);
+	if (queue && !IS_ERR(queue))
+		if (!refcount_inc_not_zero(&queue->ref_count))
+			queue = NULL;
+	rcu_read_unlock();
+	return queue;
+}
+
+void mlx5e_nvmeotcp_put_queue(struct mlx5e_nvmeotcp_queue *queue)
+{
+	if (refcount_dec_and_test(&queue->ref_count))
+		kfree(queue);
+}
+
 int set_feature_nvme_tcp(struct net_device *netdev, bool enable)
 {
 	struct mlx5e_priv *priv = netdev_priv(netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
index 753757fc44a3..d0e515502d6d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.h
@@ -100,6 +100,10 @@ int mlx5e_nvmeotcp_init(struct mlx5e_priv *priv);
 int set_feature_nvme_tcp(struct net_device *netdev, bool enable);
 int set_feature_nvme_tcp_crc(struct net_device *netdev, bool enable);
 void mlx5e_nvmeotcp_cleanup(struct mlx5e_priv *priv);
+struct mlx5e_nvmeotcp_queue *
+mlx5e_nvmeotcp_get_queue(struct mlx5e_nvmeotcp *nvmeotcp, int id);
+void mlx5e_nvmeotcp_put_queue(struct mlx5e_nvmeotcp_queue *queue);
+void mlx5e_nvmeotcp_ctx_comp(struct mlx5e_icosq_wqe_info *wi);
 int mlx5e_nvmeotcp_init_rx(struct mlx5e_priv *priv);
 void mlx5e_nvmeotcp_cleanup_rx(struct mlx5e_priv *priv);
 #else
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
index 329e114d6571..44671e28a9ea 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_utils.h
@@ -4,9 +4,77 @@
 #define __MLX5E_NVMEOTCP_UTILS_H__
 
 #include "en.h"
+#include "en_accel/nvmeotcp.h"
+
+enum {
+	MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_START     = 0,
+	MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_TRACKING  = 1,
+	MLX5E_NVMEOTCP_PROGRESS_PARAMS_PDU_TRACKER_STATE_SEARCHING = 2,
+};
+
+struct mlx5_seg_nvmeotcp_static_params {
+	u8     ctx[MLX5_ST_SZ_BYTES(transport_static_params)];
+};
+
+struct mlx5_seg_nvmeotcp_progress_params {
+	__be32 tir_num;
+	u8     ctx[MLX5_ST_SZ_BYTES(nvmeotcp_progress_params)];
+};
+
+struct mlx5e_set_nvmeotcp_static_params_wqe {
+	struct mlx5_wqe_ctrl_seg          ctrl;
+	struct mlx5_wqe_umr_ctrl_seg      uctrl;
+	struct mlx5_mkey_seg              mkc;
+	struct mlx5_seg_nvmeotcp_static_params params;
+};
+
+struct mlx5e_set_nvmeotcp_progress_params_wqe {
+	struct mlx5_wqe_ctrl_seg            ctrl;
+	struct mlx5_seg_nvmeotcp_progress_params params;
+};
+
+struct mlx5e_get_psv_wqe {
+	struct mlx5_wqe_ctrl_seg ctrl;
+	struct mlx5_seg_get_psv  psv;
+};
+
+///////////////////////////////////////////
+#define MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ \
+	(sizeof(struct mlx5e_set_nvmeotcp_static_params_wqe))
+
+#define MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ \
+	(sizeof(struct mlx5e_set_nvmeotcp_progress_params_wqe))
+#define MLX5E_NVMEOTCP_STATIC_PARAMS_OCTWORD_SIZE \
+	(MLX5_ST_SZ_BYTES(transport_static_params) / MLX5_SEND_WQE_DS)
+
+#define MLX5E_NVMEOTCP_STATIC_PARAMS_WQEBBS \
+	(DIV_ROUND_UP(MLX5E_NVMEOTCP_STATIC_PARAMS_WQE_SZ, MLX5_SEND_WQE_BB))
+#define MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQEBBS \
+	(DIV_ROUND_UP(MLX5E_NVMEOTCP_PROGRESS_PARAMS_WQE_SZ, MLX5_SEND_WQE_BB))
+
+#define MLX5E_NVMEOTCP_FETCH_STATIC_PARAMS_WQE(sq, pi) \
+	((struct mlx5e_set_nvmeotcp_static_params_wqe *)\
+	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_set_nvmeotcp_static_params_wqe)))
+
+#define MLX5E_NVMEOTCP_FETCH_PROGRESS_PARAMS_WQE(sq, pi) \
+	((struct mlx5e_set_nvmeotcp_progress_params_wqe *)\
+	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_set_nvmeotcp_progress_params_wqe)))
 
 #define MLX5E_NVMEOTCP_FETCH_KLM_WQE(sq, pi) \
 	((struct mlx5e_umr_wqe *)\
 	 mlx5e_fetch_wqe(&(sq)->wq, pi, sizeof(struct mlx5e_umr_wqe)))
 
+#define MLX5_CTRL_SEGMENT_OPC_MOD_UMR_NVMEOTCP_TIR_PROGRESS_PARAMS 0x4
+
+void
+build_nvmeotcp_progress_params(struct mlx5e_nvmeotcp_queue *queue,
+			       struct mlx5e_set_nvmeotcp_progress_params_wqe *wqe,
+			       u32 seq);
+
+void
+build_nvmeotcp_static_params(struct mlx5e_nvmeotcp_queue *queue,
+			     struct mlx5e_set_nvmeotcp_static_params_wqe *wqe,
+			     u32 resync_seq,
+			     bool zerocopy, bool crc_rx);
+
 #endif /* __MLX5E_NVMEOTCP_UTILS_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index df6dc111f6df..9f7b7239239f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -47,6 +47,7 @@
 #include "fpga/ipsec.h"
 #include "en_accel/ipsec_rxtx.h"
 #include "en_accel/tls_rxtx.h"
+#include "en_accel/nvmeotcp.h"
 #include "lib/clock.h"
 #include "en/xdp.h"
 #include "en/xsk/rx.h"
@@ -624,6 +625,9 @@ void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 		case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 			break;
+		case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
+			mlx5e_nvmeotcp_ctx_comp(wi);
+			break;
 #endif
 		}
 	}
@@ -698,6 +702,9 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)
 #ifdef CONFIG_MLX5_EN_NVMEOTCP
 			case MLX5E_ICOSQ_WQE_UMR_NVME_TCP:
 				break;
+			case MLX5E_ICOSQ_WQE_SET_PSV_NVME_TCP:
+				mlx5e_nvmeotcp_ctx_comp(wi);
+				break;
 #endif
 			default:
 				netdev_WARN_ONCE(cq->netdev,
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:41 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:41 +0200
Subject: [PATCH v4 net-next  18/21] net/mlx5e: NVMEoTCP ddp setup and resync
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-19-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

NVMEoTCP offload uses buffer registration for every NVME request to
perform direct data placement, The registration is done via KLM UMR
WQE's.  The driver resync handler advertise the software resync response
via static params WQE.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 33 +++++++++++++++++--
 1 file changed, 31 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index f79e26419a89..6fa35f3a8e21 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -754,6 +754,30 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 			 struct sock *sk,
 			 struct tcp_ddp_io *ddp)
 {
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	struct scatterlist *sg = ddp->sg_table.sgl;
+	struct mlx5e_nvmeotcp_queue *queue;
+	struct mlx5_core_dev *mdev;
+	int i, size = 0, count = 0;
+
+	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
+
+	mdev = queue->priv->mdev;
+	count = dma_map_sg(mdev->device, ddp->sg_table.sgl, ddp->nents,
+			   DMA_FROM_DEVICE);
+
+	if (WARN_ON(count > mlx5e_get_max_sgl(mdev)))
+		return -ENOSPC;
+
+	for (i = 0; i < count; i++)
+		size += sg[i].length;
+
+	queue->ccid_table[ddp->command_id].size = size;
+	queue->ccid_table[ddp->command_id].ddp = ddp;
+	queue->ccid_table[ddp->command_id].sgl = sg;
+	queue->ccid_table[ddp->command_id].ccid_gen++;
+	queue->ccid_table[ddp->command_id].sgl_length = count;
+
 	return 0;
 }
 
@@ -791,11 +815,11 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 			    struct tcp_ddp_io *ddp,
 			    void *ddp_ctx)
 {
-	struct mlx5e_nvmeotcp_queue *queue =
-		(struct mlx5e_nvmeotcp_queue *)tcp_ddp_get_ctx(sk);
+	struct mlx5e_nvmeotcp_queue *queue;
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct nvmeotcp_queue_entry *q_entry;
 
+	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
 	q_entry  = &queue->ccid_table[ddp->command_id];
 	WARN_ON(q_entry->sgl_length == 0);
 
@@ -811,6 +835,11 @@ static void
 mlx5e_nvmeotcp_dev_resync(struct net_device *netdev,
 			  struct sock *sk, u32 seq)
 {
+	struct mlx5e_nvmeotcp_queue *queue =
+		container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
+
+	queue->after_resync_cqe = 1;
+	mlx5e_nvmeotcp_rx_post_static_params_wqe(queue, seq);
 }
 
 static const struct tcp_ddp_dev_ops mlx5e_nvmeotcp_ops = {
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:42 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:42 +0200
Subject: [PATCH v4 net-next  19/21] net/mlx5e: NVMEoTCP,
 data-path for DDP+CRC offload
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-20-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

This patch implements the data-path for direct data placement (DDP)
and CRC offloads. NVMEoTCP DDP constructs an SKB from each CQE, while
pointing at NVME destination buffers. In turn, this enables the offload,
as the NVMe-TCP layer will skip the copy when src == dst.

Additionally, this patch adds support for CRC offload.
HW will report CRC offload only if it has not encountered an error
in the received packet. We pass this indication in skb->ddp_crc
up the stack to NVMe-TCP to skip computing the CRC if all
corresponding SKBs were verified by HW.

This patch also handles context resynchronization requests made by
NIC HW. The resync request is passed to the NVMe-TCP layer
to be handled at a later point in time.

Finally, we also use the skb->ddp_crc bit to avoid skb_condense.
This is critical as every SKB that uses DDP has a hole that fits
perfectly with skb_condense's policy, but filling this hole is
counter-productive as the data there already resides in its
destination buffer.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |   1 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.c   |   1 +
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.h   |   1 +
 .../mlx5/core/en_accel/nvmeotcp_rxtx.c        | 248 ++++++++++++++++++
 .../mlx5/core/en_accel/nvmeotcp_rxtx.h        |  43 +++
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  41 ++-
 7 files changed, 330 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index ac9978b0eb5d..ec6a75536bff 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -99,4 +99,4 @@ mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o
 #
 mlx5_core-$(CONFIG_MLX5_SF_MANAGER) += sf/cmd.o sf/hw_table.o sf/devlink.o
 
-mlx5_core-$(CONFIG_MLX5_EN_NVMEOTCP) += en_accel/fs_tcp.o en_accel/nvmeotcp.o
+mlx5_core-$(CONFIG_MLX5_EN_NVMEOTCP) += en_accel/fs_tcp.o en_accel/nvmeotcp.o en_accel/nvmeotcp_rxtx.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 734b54e3090a..0675092807cf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -577,6 +577,7 @@ struct mlx5e_rq;
 typedef void (*mlx5e_fp_handle_rx_cqe)(struct mlx5e_rq*, struct mlx5_cqe64*);
 typedef struct sk_buff *
 (*mlx5e_fp_skb_from_cqe_mpwrq)(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+			       struct mlx5_cqe64 *cqe,
 			       u16 cqe_bcnt, u32 head_offset, u32 page_idx);
 typedef struct sk_buff *
 (*mlx5e_fp_skb_from_cqe)(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
index 8e7b877d8a12..9a6fbd1b1c34 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
@@ -25,6 +25,7 @@ static struct sk_buff *mlx5e_xsk_construct_skb(struct mlx5e_rq *rq, void *data,
 
 struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq,
 						    struct mlx5e_mpw_info *wi,
+						    struct mlx5_cqe64 *cqe,
 						    u16 cqe_bcnt,
 						    u32 head_offset,
 						    u32 page_idx)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
index 7f88ccf67fdd..112c5b3ec165 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
@@ -11,6 +11,7 @@
 
 struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq,
 						    struct mlx5e_mpw_info *wi,
+						    struct mlx5_cqe64 *cqe,
 						    u16 cqe_bcnt,
 						    u32 head_offset,
 						    u32 page_idx);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
new file mode 100644
index 000000000000..b16fcf051665
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
@@ -0,0 +1,248 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2021 Mellanox Technologies. */
+
+#include "en_accel/nvmeotcp_rxtx.h"
+#include "en_accel/nvmeotcp.h"
+#include <linux/mlx5/mlx5_ifc.h>
+
+#define	MLX5E_TC_FLOW_ID_MASK  0x00ffffff
+static void nvmeotcp_update_resync(struct mlx5e_nvmeotcp_queue *queue,
+				   struct mlx5e_cqe128 *cqe128)
+{
+	const struct tcp_ddp_ulp_ops *ulp_ops;
+	u32 seq;
+
+	seq = be32_to_cpu(cqe128->resync_tcp_sn);
+	ulp_ops = inet_csk(queue->sk)->icsk_ulp_ddp_ops;
+	if (ulp_ops && ulp_ops->resync_request)
+		ulp_ops->resync_request(queue->sk, seq, TCP_DDP_RESYNC_REQ);
+}
+
+static void mlx5e_nvmeotcp_advance_sgl_iter(struct mlx5e_nvmeotcp_queue *queue)
+{
+	struct nvmeotcp_queue_entry *nqe = &queue->ccid_table[queue->ccid];
+
+	queue->ccoff += nqe->sgl[queue->ccsglidx].length;
+	queue->ccoff_inner = 0;
+	queue->ccsglidx++;
+}
+
+static inline void
+mlx5e_nvmeotcp_add_skb_frag(struct net_device *netdev, struct sk_buff *skb,
+			    struct mlx5e_nvmeotcp_queue *queue,
+			    struct nvmeotcp_queue_entry *nqe, u32 fragsz)
+{
+	dma_sync_single_for_cpu(&netdev->dev,
+				nqe->sgl[queue->ccsglidx].offset + queue->ccoff_inner,
+				fragsz, DMA_FROM_DEVICE);
+	page_ref_inc(compound_head(sg_page(&nqe->sgl[queue->ccsglidx])));
+	// XXX: consider reducing the truesize, as no new memory is consumed
+	skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+			sg_page(&nqe->sgl[queue->ccsglidx]),
+			nqe->sgl[queue->ccsglidx].offset + queue->ccoff_inner,
+			fragsz,
+			fragsz);
+}
+
+static struct sk_buff*
+mlx5_nvmeotcp_add_tail_nonlinear(struct mlx5e_nvmeotcp_queue *queue,
+				 struct sk_buff *skb, skb_frag_t *org_frags,
+				 int org_nr_frags, int frag_index)
+{
+	struct mlx5e_priv *priv = queue->priv;
+
+	while (org_nr_frags != frag_index) {
+		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
+			dev_kfree_skb_any(skb);
+			return NULL;
+		}
+		skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+				skb_frag_page(&org_frags[frag_index]),
+				skb_frag_off(&org_frags[frag_index]),
+				skb_frag_size(&org_frags[frag_index]),
+				skb_frag_size(&org_frags[frag_index]));
+		page_ref_inc(skb_frag_page(&org_frags[frag_index]));
+		frag_index++;
+	}
+	return skb;
+}
+
+static struct sk_buff*
+mlx5_nvmeotcp_add_tail(struct mlx5e_nvmeotcp_queue *queue, struct sk_buff *skb,
+		       int offset, int len)
+{
+	struct mlx5e_priv *priv = queue->priv;
+
+	if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
+		dev_kfree_skb_any(skb);
+		return NULL;
+	}
+	skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+			virt_to_page(skb->data),
+			offset,
+			len,
+			len);
+	page_ref_inc(virt_to_page(skb->data));
+	return skb;
+}
+
+static void mlx5_nvmeotcp_trim_nonlinear(struct sk_buff *skb,
+					 skb_frag_t *org_frags,
+					 int *frag_index,
+					 int remaining)
+{
+	unsigned int frag_size;
+	int nr_frags;
+
+	/* skip @remaining bytes in frags */
+	*frag_index = 0;
+	while (remaining) {
+		frag_size = skb_frag_size(&skb_shinfo(skb)->frags[*frag_index]);
+		if (frag_size > remaining) {
+			skb_frag_off_add(&skb_shinfo(skb)->frags[*frag_index],
+					 remaining);
+			skb_frag_size_sub(&skb_shinfo(skb)->frags[*frag_index],
+					  remaining);
+			remaining = 0;
+		} else {
+			remaining -= frag_size;
+			skb_frag_unref(skb, *frag_index);
+			*frag_index += 1;
+		}
+	}
+
+	/* save original frags for the tail and unref */
+	nr_frags = skb_shinfo(skb)->nr_frags;
+	memcpy(&org_frags[*frag_index], &skb_shinfo(skb)->frags[*frag_index],
+	       (nr_frags - *frag_index) * sizeof(skb_frag_t));
+	while (--nr_frags >= *frag_index)
+		skb_frag_unref(skb, nr_frags);
+
+	/* remove frags from skb */
+	skb_shinfo(skb)->nr_frags = 0;
+	skb->len -= skb->data_len;
+	skb->truesize -= skb->data_len;
+	skb->data_len = 0;
+}
+
+struct sk_buff*
+mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
+			     struct mlx5_cqe64 *cqe, u32 cqe_bcnt,
+			     bool linear)
+{
+	int ccoff, cclen, hlen, ccid, remaining, fragsz, to_copy = 0;
+	struct mlx5e_priv *priv = netdev_priv(netdev);
+	skb_frag_t org_frags[MAX_SKB_FRAGS];
+	struct mlx5e_nvmeotcp_queue *queue;
+	struct nvmeotcp_queue_entry *nqe;
+	int org_nr_frags, frag_index;
+	struct mlx5e_cqe128 *cqe128;
+	u32 queue_id;
+
+	queue_id = (be32_to_cpu(cqe->sop_drop_qpn) & MLX5E_TC_FLOW_ID_MASK);
+	queue = mlx5e_nvmeotcp_get_queue(priv->nvmeotcp, queue_id);
+	if (unlikely(!queue)) {
+		dev_kfree_skb_any(skb);
+		return NULL;
+	}
+
+	cqe128 = container_of(cqe, struct mlx5e_cqe128, cqe64);
+	if (cqe_is_nvmeotcp_resync(cqe)) {
+		nvmeotcp_update_resync(queue, cqe128);
+		mlx5e_nvmeotcp_put_queue(queue);
+		return skb;
+	}
+
+	/* If a resync occurred in the previous cqe,
+	 * the current cqe.crcvalid bit may not be valid,
+	 * so we will treat it as 0
+	 */
+	if (unlikely(queue->after_resync_cqe)) {
+		skb->ddp_crc = 0;
+		queue->after_resync_cqe = 0;
+	} else {
+		if (queue->crc_rx)
+			skb->ddp_crc = cqe_is_nvmeotcp_crcvalid(cqe);
+		else
+			skb->ddp_crc = cqe_is_nvmeotcp_zc(cqe);
+	}
+
+	if (!cqe_is_nvmeotcp_zc(cqe)) {
+		mlx5e_nvmeotcp_put_queue(queue);
+		return skb;
+	}
+
+	/* cc ddp from cqe */
+	ccid = be16_to_cpu(cqe128->ccid);
+	ccoff = be32_to_cpu(cqe128->ccoff);
+	cclen = be16_to_cpu(cqe128->cclen);
+	hlen  = be16_to_cpu(cqe128->hlen);
+
+	/* carve a hole in the skb for DDP data */
+	if (linear) {
+		skb_trim(skb, hlen);
+	} else {
+		org_nr_frags = skb_shinfo(skb)->nr_frags;
+		mlx5_nvmeotcp_trim_nonlinear(skb, org_frags, &frag_index,
+					     cclen);
+	}
+
+	nqe = &queue->ccid_table[ccid];
+
+	/* packet starts new ccid? */
+	if (queue->ccid != ccid || queue->ccid_gen != nqe->ccid_gen) {
+		queue->ccid = ccid;
+		queue->ccoff = 0;
+		queue->ccoff_inner = 0;
+		queue->ccsglidx = 0;
+		queue->ccid_gen = nqe->ccid_gen;
+	}
+
+	/* skip inside cc until the ccoff in the cqe */
+	while (queue->ccoff + queue->ccoff_inner < ccoff) {
+		remaining = nqe->sgl[queue->ccsglidx].length - queue->ccoff_inner;
+		fragsz = min_t(off_t, remaining,
+			       ccoff - (queue->ccoff + queue->ccoff_inner));
+
+		if (fragsz == remaining)
+			mlx5e_nvmeotcp_advance_sgl_iter(queue);
+		else
+			queue->ccoff_inner += fragsz;
+	}
+
+	/* adjust the skb according to the cqe cc */
+	while (to_copy < cclen) {
+		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
+			dev_kfree_skb_any(skb);
+			mlx5e_nvmeotcp_put_queue(queue);
+			return NULL;
+		}
+
+		remaining = nqe->sgl[queue->ccsglidx].length - queue->ccoff_inner;
+		fragsz = min_t(int, remaining, cclen - to_copy);
+
+		mlx5e_nvmeotcp_add_skb_frag(netdev, skb, queue, nqe, fragsz);
+		to_copy += fragsz;
+		if (fragsz == remaining)
+			mlx5e_nvmeotcp_advance_sgl_iter(queue);
+		else
+			queue->ccoff_inner += fragsz;
+	}
+
+	if (cqe_bcnt > hlen + cclen) {
+		remaining = cqe_bcnt - hlen - cclen;
+		if (linear)
+			skb = mlx5_nvmeotcp_add_tail(queue, skb,
+						     offset_in_page(skb->data) +
+								hlen + cclen,
+						     remaining);
+		else
+			skb = mlx5_nvmeotcp_add_tail_nonlinear(queue, skb,
+							       org_frags,
+							       org_nr_frags,
+							       frag_index);
+	}
+
+	mlx5e_nvmeotcp_put_queue(queue);
+	return skb;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h
new file mode 100644
index 000000000000..65456b46c33f
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021 Mellanox Technologies. */
+#ifndef __MLX5E_NVMEOTCP_RXTX_H__
+#define __MLX5E_NVMEOTCP_RXTX_H__
+
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+
+#include <linux/skbuff.h>
+#include "en.h"
+
+struct sk_buff*
+mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
+			     struct mlx5_cqe64 *cqe, u32 cqe_bcnt, bool linear);
+
+static inline int mlx5_nvmeotcp_get_headlen(struct mlx5_cqe64 *cqe, u32 cqe_bcnt)
+{
+	struct mlx5e_cqe128 *cqe128;
+
+	if (!cqe_is_nvmeotcp_zc(cqe) || cqe_is_nvmeotcp_resync(cqe))
+		return cqe_bcnt;
+
+	cqe128 = container_of(cqe, struct mlx5e_cqe128, cqe64);
+	return be16_to_cpu(cqe128->hlen);
+}
+
+#else
+static inline struct sk_buff*
+mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
+			     struct mlx5_cqe64 *cqe, u32 cqe_bcnt, bool linear)
+{ return skb; }
+
+static inline int mlx5_nvmeotcp_get_headlen(struct mlx5_cqe64 *cqe, u32 cqe_bcnt)
+{ return cqe_bcnt; }
+
+#endif /* CONFIG_MLX5_EN_NVMEOTCP */
+
+static inline u16 mlx5e_get_headlen_hint(struct mlx5_cqe64 *cqe, u32 cqe_bcnt)
+{
+	return min_t(u32, MLX5E_RX_MAX_HEAD, mlx5_nvmeotcp_get_headlen(cqe, cqe_bcnt));
+}
+
+
+#endif /* __MLX5E_NVMEOTCP_RXTX_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 2fe20d901bbf..e834ce9a8274 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -48,6 +48,7 @@
 #include "en_accel/ipsec_rxtx.h"
 #include "en_accel/tls_rxtx.h"
 #include "en_accel/nvmeotcp.h"
+#include "en_accel/nvmeotcp_rxtx.h"
 #include "lib/clock.h"
 #include "en/xdp.h"
 #include "en/xsk/rx.h"
@@ -57,9 +58,11 @@
 
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				struct mlx5_cqe64 *cqe,
 				u16 cqe_bcnt, u32 head_offset, u32 page_idx);
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				   struct mlx5_cqe64 *cqe,
 				   u16 cqe_bcnt, u32 head_offset, u32 page_idx);
 static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe);
 static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe);
@@ -1180,6 +1183,12 @@ mlx5e_skb_from_cqe_linear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 	/* queue up for recycling/reuse */
 	page_ref_inc(di->page);
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, true);
+#endif
+
 	return skb;
 }
 
@@ -1188,8 +1197,8 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 			     struct mlx5e_wqe_frag_info *wi, u32 cqe_bcnt)
 {
 	struct mlx5e_rq_frag_info *frag_info = &rq->wqe.info.arr[0];
+	u16 headlen = mlx5e_get_headlen_hint(cqe, cqe_bcnt);
 	struct mlx5e_wqe_frag_info *head_wi = wi;
-	u16 headlen      = min_t(u32, MLX5E_RX_MAX_HEAD, cqe_bcnt);
 	u16 frag_headlen = headlen;
 	u16 byte_cnt     = cqe_bcnt - headlen;
 	struct sk_buff *skb;
@@ -1198,7 +1207,7 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 	 * might spread among multiple pages.
 	 */
 	skb = napi_alloc_skb(rq->cq.napi,
-			     ALIGN(MLX5E_RX_MAX_HEAD, sizeof(long)));
+			     ALIGN(headlen, sizeof(long)));
 	if (unlikely(!skb)) {
 		rq->stats->buff_alloc_err++;
 		return NULL;
@@ -1224,6 +1233,12 @@ mlx5e_skb_from_cqe_nonlinear(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe,
 	skb->tail += headlen;
 	skb->len  += headlen;
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, false);
+#endif
+
 	return skb;
 }
 
@@ -1382,7 +1397,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_rep(struct mlx5e_rq *rq, struct mlx5_cqe64
 	skb = INDIRECT_CALL_2(rq->mpwqe.skb_from_cqe_mpwrq,
 			      mlx5e_skb_from_cqe_mpwrq_linear,
 			      mlx5e_skb_from_cqe_mpwrq_nonlinear,
-			      rq, wi, cqe_bcnt, head_offset, page_idx);
+			      rq, wi, cqe, cqe_bcnt, head_offset, page_idx);
 	if (!skb)
 		goto mpwrq_cqe_out;
 
@@ -1415,17 +1430,18 @@ const struct mlx5e_rx_handlers mlx5e_rx_handlers_rep = {
 
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				   struct mlx5_cqe64 *cqe,
 				   u16 cqe_bcnt, u32 head_offset, u32 page_idx)
 {
-	u16 headlen = min_t(u16, MLX5E_RX_MAX_HEAD, cqe_bcnt);
 	struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx];
+	u16 headlen = mlx5e_get_headlen_hint(cqe, cqe_bcnt);
 	u32 frag_offset    = head_offset + headlen;
 	u32 byte_cnt       = cqe_bcnt - headlen;
 	struct mlx5e_dma_info *head_di = di;
 	struct sk_buff *skb;
 
 	skb = napi_alloc_skb(rq->cq.napi,
-			     ALIGN(MLX5E_RX_MAX_HEAD, sizeof(long)));
+			     ALIGN(headlen, sizeof(long)));
 	if (unlikely(!skb)) {
 		rq->stats->buff_alloc_err++;
 		return NULL;
@@ -1456,11 +1472,18 @@ mlx5e_skb_from_cqe_mpwrq_nonlinear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *w
 	skb->tail += headlen;
 	skb->len  += headlen;
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, false);
+#endif
+
 	return skb;
 }
 
 static struct sk_buff *
 mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
+				struct mlx5_cqe64 *cqe,
 				u16 cqe_bcnt, u32 head_offset, u32 page_idx)
 {
 	struct mlx5e_dma_info *di = &wi->umr.dma_info[page_idx];
@@ -1502,6 +1525,12 @@ mlx5e_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi,
 	/* queue up for recycling/reuse */
 	page_ref_inc(di->page);
 
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	if (cqe_is_nvmeotcp(cqe))
+		skb = mlx5e_nvmeotcp_handle_rx_skb(rq->netdev, skb, cqe,
+						   cqe_bcnt, true);
+#endif
+
 	return skb;
 }
 
@@ -1540,7 +1569,7 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq
 	skb = INDIRECT_CALL_2(rq->mpwqe.skb_from_cqe_mpwrq,
 			      mlx5e_skb_from_cqe_mpwrq_linear,
 			      mlx5e_skb_from_cqe_mpwrq_nonlinear,
-			      rq, wi, cqe_bcnt, head_offset, page_idx);
+			      rq, wi, cqe, cqe_bcnt, head_offset, page_idx);
 	if (!skb)
 		goto mpwrq_cqe_out;
 
-- 
2.24.1



From borisp at mellanox.com  Thu Feb 11 16:10:43 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Thu, 11 Feb 2021 23:10:43 +0200
Subject: [PATCH v4 net-next  20/21] net/mlx5e: NVMEoTCP statistics
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <20210211211044.32701-21-borisp@mellanox.com>

From: Ben Ben-Ishay <benishay at nvidia.com>

NVMEoTCP offload statistics includes both control and data path
statistic: counters for ndo, offloaded packets/bytes, dropped packets
and resync operation.

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at nvidia.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 .../mellanox/mlx5/core/en_accel/nvmeotcp.c    | 23 +++++++++++-
 .../mlx5/core/en_accel/nvmeotcp_rxtx.c        | 16 ++++++++
 .../ethernet/mellanox/mlx5/core/en_stats.c    | 37 +++++++++++++++++++
 .../ethernet/mellanox/mlx5/core/en_stats.h    | 24 ++++++++++++
 4 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
index 6fa35f3a8e21..e807e91a459c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp.c
@@ -659,6 +659,11 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 	struct mlx5_core_dev *mdev = priv->mdev;
 	struct mlx5e_nvmeotcp_queue *queue;
 	int max_wqe_sz_cap, queue_id, err;
+	struct mlx5e_rq_stats *stats;
+	u32 channel_ix;
+
+	channel_ix = mlx5e_get_channel_ix_from_io_cpu(priv, config->io_cpu);
+	stats = &priv->channel_stats[channel_ix].rq;
 
 	if (tconfig->type != TCP_DDP_NVME) {
 		err = -EOPNOTSUPP;
@@ -686,8 +691,7 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 	queue->id = queue_id;
 	queue->dgst = config->dgst;
 	queue->pda = config->cpda;
-	queue->channel_ix = mlx5e_get_channel_ix_from_io_cpu(priv,
-							     config->io_cpu);
+	queue->channel_ix = channel_ix;
 	queue->size = config->queue_size;
 	max_wqe_sz_cap  = min_t(int, MAX_DS_VALUE * MLX5_SEND_WQE_DS,
 				MLX5_CAP_GEN(mdev, max_wqe_sz_sq) << OCTWORD_SHIFT);
@@ -707,6 +711,7 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 	if (err)
 		goto destroy_rx;
 
+	stats->nvmeotcp_queue_init++;
 	write_lock_bh(&sk->sk_callback_lock);
 	tcp_ddp_set_ctx(sk, queue);
 	write_unlock_bh(&sk->sk_callback_lock);
@@ -721,6 +726,7 @@ mlx5e_nvmeotcp_queue_init(struct net_device *netdev,
 free_queue:
 	kfree(queue);
 out:
+	stats->nvmeotcp_queue_init_fail++;
 	return err;
 }
 
@@ -731,11 +737,15 @@ mlx5e_nvmeotcp_queue_teardown(struct net_device *netdev,
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct mlx5_core_dev *mdev = priv->mdev;
 	struct mlx5e_nvmeotcp_queue *queue;
+	struct mlx5e_rq_stats *stats;
 
 	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
 
 	napi_synchronize(&priv->channels.c[queue->channel_ix]->napi);
 
+	stats = &priv->channel_stats[queue->channel_ix].rq;
+	stats->nvmeotcp_queue_teardown++;
+
 	WARN_ON(refcount_read(&queue->ref_count) != 1);
 	if (queue->zerocopy | queue->crc_rx)
 		mlx5e_nvmeotcp_destroy_rx(queue, mdev, queue->zerocopy);
@@ -757,6 +767,7 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct scatterlist *sg = ddp->sg_table.sgl;
 	struct mlx5e_nvmeotcp_queue *queue;
+	struct mlx5e_rq_stats *stats;
 	struct mlx5_core_dev *mdev;
 	int i, size = 0, count = 0;
 
@@ -778,6 +789,11 @@ mlx5e_nvmeotcp_ddp_setup(struct net_device *netdev,
 	queue->ccid_table[ddp->command_id].ccid_gen++;
 	queue->ccid_table[ddp->command_id].sgl_length = count;
 
+	stats = &priv->channel_stats[queue->channel_ix].rq;
+	stats->nvmeotcp_ddp_setup++;
+	if (unlikely(mlx5e_nvmeotcp_post_klm_wqe(queue, KLM_UMR, ddp->command_id, count)))
+		stats->nvmeotcp_ddp_setup_fail++;
+
 	return 0;
 }
 
@@ -818,6 +834,7 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 	struct mlx5e_nvmeotcp_queue *queue;
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	struct nvmeotcp_queue_entry *q_entry;
+	struct mlx5e_rq_stats *stats;
 
 	queue = container_of(tcp_ddp_get_ctx(sk), struct mlx5e_nvmeotcp_queue, tcp_ddp_ctx);
 	q_entry  = &queue->ccid_table[ddp->command_id];
@@ -827,6 +844,8 @@ mlx5e_nvmeotcp_ddp_teardown(struct net_device *netdev,
 	q_entry->queue = queue;
 
 	mlx5e_nvmeotcp_post_klm_wqe(queue, KLM_UMR, ddp->command_id, 0);
+	stats = &priv->channel_stats[queue->channel_ix].rq;
+	stats->nvmeotcp_ddp_teardown++;
 
 	return 0;
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
index b16fcf051665..158f3798bf0c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/nvmeotcp_rxtx.c
@@ -10,12 +10,16 @@ static void nvmeotcp_update_resync(struct mlx5e_nvmeotcp_queue *queue,
 				   struct mlx5e_cqe128 *cqe128)
 {
 	const struct tcp_ddp_ulp_ops *ulp_ops;
+	struct mlx5e_rq_stats *stats;
 	u32 seq;
 
 	seq = be32_to_cpu(cqe128->resync_tcp_sn);
 	ulp_ops = inet_csk(queue->sk)->icsk_ulp_ddp_ops;
 	if (ulp_ops && ulp_ops->resync_request)
 		ulp_ops->resync_request(queue->sk, seq, TCP_DDP_RESYNC_REQ);
+
+	stats = queue->priv->channels.c[queue->channel_ix]->rq.stats;
+	stats->nvmeotcp_resync++;
 }
 
 static void mlx5e_nvmeotcp_advance_sgl_iter(struct mlx5e_nvmeotcp_queue *queue)
@@ -50,10 +54,13 @@ mlx5_nvmeotcp_add_tail_nonlinear(struct mlx5e_nvmeotcp_queue *queue,
 				 int org_nr_frags, int frag_index)
 {
 	struct mlx5e_priv *priv = queue->priv;
+	struct mlx5e_rq_stats *stats;
 
 	while (org_nr_frags != frag_index) {
 		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
 			dev_kfree_skb_any(skb);
+			stats = priv->channels.c[queue->channel_ix]->rq.stats;
+			stats->nvmeotcp_drop++;
 			return NULL;
 		}
 		skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
@@ -72,9 +79,12 @@ mlx5_nvmeotcp_add_tail(struct mlx5e_nvmeotcp_queue *queue, struct sk_buff *skb,
 		       int offset, int len)
 {
 	struct mlx5e_priv *priv = queue->priv;
+	struct mlx5e_rq_stats *stats;
 
 	if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
 		dev_kfree_skb_any(skb);
+		stats = priv->channels.c[queue->channel_ix]->rq.stats;
+		stats->nvmeotcp_drop++;
 		return NULL;
 	}
 	skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
@@ -135,6 +145,7 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 	skb_frag_t org_frags[MAX_SKB_FRAGS];
 	struct mlx5e_nvmeotcp_queue *queue;
 	struct nvmeotcp_queue_entry *nqe;
+	struct mlx5e_rq_stats *stats;
 	int org_nr_frags, frag_index;
 	struct mlx5e_cqe128 *cqe128;
 	u32 queue_id;
@@ -172,6 +183,8 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 		return skb;
 	}
 
+	stats = priv->channels.c[queue->channel_ix]->rq.stats;
+
 	/* cc ddp from cqe */
 	ccid = be16_to_cpu(cqe128->ccid);
 	ccoff = be32_to_cpu(cqe128->ccoff);
@@ -214,6 +227,7 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 	while (to_copy < cclen) {
 		if (skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS) {
 			dev_kfree_skb_any(skb);
+			stats->nvmeotcp_drop++;
 			mlx5e_nvmeotcp_put_queue(queue);
 			return NULL;
 		}
@@ -243,6 +257,8 @@ mlx5e_nvmeotcp_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
 							       frag_index);
 	}
 
+	stats->nvmeotcp_offload_packets++;
+	stats->nvmeotcp_offload_bytes += cclen;
 	mlx5e_nvmeotcp_put_queue(queue);
 	return skb;
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index 92c5b81427b9..353662f3fc5f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -34,6 +34,7 @@
 #include "en.h"
 #include "en_accel/tls.h"
 #include "en_accel/en_accel.h"
+#include "en_accel/nvmeotcp.h"
 
 static unsigned int stats_grps_num(struct mlx5e_priv *priv)
 {
@@ -189,6 +190,18 @@ static const struct counter_desc sw_stats_desc[] = {
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_resync_res_ok) },
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_resync_res_skip) },
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_tls_err) },
+#endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_queue_init) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_queue_init_fail) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_queue_teardown) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_ddp_setup) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_ddp_setup_fail) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_ddp_teardown) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_drop) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_resync) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_offload_packets) },
+	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_nvmeotcp_offload_bytes) },
 #endif
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, ch_events) },
 	{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, ch_poll) },
@@ -352,6 +365,18 @@ static void mlx5e_stats_grp_sw_update_stats_rq_stats(struct mlx5e_sw_stats *s,
 	s->rx_tls_resync_res_skip     += rq_stats->tls_resync_res_skip;
 	s->rx_tls_err                 += rq_stats->tls_err;
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	s->rx_nvmeotcp_queue_init      += rq_stats->nvmeotcp_queue_init;
+	s->rx_nvmeotcp_queue_init_fail += rq_stats->nvmeotcp_queue_init_fail;
+	s->rx_nvmeotcp_queue_teardown  += rq_stats->nvmeotcp_queue_teardown;
+	s->rx_nvmeotcp_ddp_setup       += rq_stats->nvmeotcp_ddp_setup;
+	s->rx_nvmeotcp_ddp_setup_fail  += rq_stats->nvmeotcp_ddp_setup_fail;
+	s->rx_nvmeotcp_ddp_teardown    += rq_stats->nvmeotcp_ddp_teardown;
+	s->rx_nvmeotcp_drop            += rq_stats->nvmeotcp_drop;
+	s->rx_nvmeotcp_resync          += rq_stats->nvmeotcp_resync;
+	s->rx_nvmeotcp_offload_packets += rq_stats->nvmeotcp_offload_packets;
+	s->rx_nvmeotcp_offload_bytes   += rq_stats->nvmeotcp_offload_bytes;
+#endif
 }
 
 static void mlx5e_stats_grp_sw_update_stats_ch_stats(struct mlx5e_sw_stats *s,
@@ -1632,6 +1657,18 @@ static const struct counter_desc rq_stats_desc[] = {
 	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_resync_res_skip) },
 	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, tls_err) },
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_queue_init) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_queue_init_fail) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_queue_teardown) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_ddp_setup) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_ddp_setup_fail) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_ddp_teardown) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_drop) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_resync) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_offload_packets) },
+	{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, nvmeotcp_offload_bytes) },
+#endif
 };
 
 static const struct counter_desc sq_stats_desc[] = {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
index 93c41312fb03..674cee2a884d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
@@ -181,6 +181,18 @@ struct mlx5e_sw_stats {
 	u64 rx_congst_umr;
 	u64 rx_arfs_err;
 	u64 rx_recover;
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	u64 rx_nvmeotcp_queue_init;
+	u64 rx_nvmeotcp_queue_init_fail;
+	u64 rx_nvmeotcp_queue_teardown;
+	u64 rx_nvmeotcp_ddp_setup;
+	u64 rx_nvmeotcp_ddp_setup_fail;
+	u64 rx_nvmeotcp_ddp_teardown;
+	u64 rx_nvmeotcp_drop;
+	u64 rx_nvmeotcp_resync;
+	u64 rx_nvmeotcp_offload_packets;
+	u64 rx_nvmeotcp_offload_bytes;
+#endif
 	u64 ch_events;
 	u64 ch_poll;
 	u64 ch_arm;
@@ -344,6 +356,18 @@ struct mlx5e_rq_stats {
 	u64 tls_resync_res_skip;
 	u64 tls_err;
 #endif
+#ifdef CONFIG_MLX5_EN_NVMEOTCP
+	u64 nvmeotcp_queue_init;
+	u64 nvmeotcp_queue_init_fail;
+	u64 nvmeotcp_queue_teardown;
+	u64 nvmeotcp_ddp_setup;
+	u64 nvmeotcp_ddp_setup_fail;
+	u64 nvmeotcp_ddp_teardown;
+	u64 nvmeotcp_drop;
+	u64 nvmeotcp_resync;
+	u64 nvmeotcp_offload_packets;
+	u64 nvmeotcp_offload_bytes;
+#endif
 };
 
 struct mlx5e_sq_stats {
-- 
2.24.1



From rdunlap at infradead.org  Thu Feb 11 16:32:02 2021
From: rdunlap at infradead.org (Randy Dunlap)
Date: Thu, 11 Feb 2021 13:32:02 -0800
Subject: [PATCH v4 net-next 00/21] nvme-tcp receive offloads
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <76c93e83-be63-f7ed-a096-1408e9525331@infradead.org>

On 2/11/21 1:10 PM, Boris Pismenny wrote:
> Changes since v3:
> =========================================
> * Use DDP_TCP ifdefs in iov_iter and skb iterators to minimize impact
> when compiled out (Christoph)
> * Simplify netdev references and reduce the use of
> get_netdev_for_sock (Sagi)
> * Avoid "static" in it's own line, move it one line down (Christoph)
> * Pass (queue, skb, *offset) and retrieve the pdu_seq in
> nvme_tcp_resync_response (Sagi)
> * Add missing assignment of offloading_netdev to null in offload_limits
> error case (Sagi)
> * Set req->offloaded = false once -- the lifetime rules are:
> set to false on cmd_setup / set to true when ddp setup succeeds (Sagi)
> * Replace pr_info_ratelimited with dev_info_ratelimited (Sagi)
> * Add nvme_tcp_complete_request and invoke it from two similar call
> sites (Sagi)
> * Introduce nvme_tcp_req_map_sg earlier in the series (Sagi)
> * Add nvme_tcp_consume_skb and put into it a hunk from
> nvme_tcp_recv_data to handle copy with and without offload

Hi,
Did vger.kernel.org eat patch 21/21?

and does that patch contain the Documentation updates?

thanks.
-- 
~Randy



From mark.wunderlich at intel.com  Thu Feb 11 18:42:55 2021
From: mark.wunderlich at intel.com (Wunderlich, Mark)
Date: Thu, 11 Feb 2021 23:42:55 +0000
Subject: [PATCH] nvmet-tcp: enable optional queue idle period tracking
Message-ID: <BYAPR11MB286909CAA3A8721B46A02AFFE58C9@BYAPR11MB2869.namprd11.prod.outlook.com>

nvmet-tcp: enable optional queue idle period tracking

Add 'queue idle period' option used by io_work() to support
network devices enabled with advanced interrupt moderation
supporting a relaxed interrupt model. It was discovered that
such a NIC used on the target was unable to support initiator
connection establishment, caused by the existing io_work()
flow that immediately exits and does not re-queue itself at
the first loop with no activity.

With this new option a queue is assigned a period of time
that no activity must occur in order to become 'idle'.  Until
the queue is idle the work item is requeued.

The new module option is defined as changeable making it
flexible for testing purposes.

The pre-existing legacy behavior is preserved when no module option
for queue idle period is specified.

Signed-off-by: Mark Wunderlich <mark.wunderlich at intel.com>
---
Testing was performed with a NIC using standard HW interrupt mode, with
and without the new module option enabled.  No measurable performance
drop was seen when the patch was applied and the new option specified
or not.  A side effect of a standard NIC using the new option
will reduce the context switch rate.  We measured a drop from roughly
90K to less than 300 (for 32 active connections).

For a NIC using a passive advanced interrupt moderation policy, it was
then successfully able to achieve and maintain active connections with
the target.
---
 drivers/nvme/target/tcp.c |   65 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 64 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index dc1f0f647189..f5e6e2806841 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -29,6 +29,16 @@ static int so_priority;
 module_param(so_priority, int, 0644);
 MODULE_PARM_DESC(so_priority, "nvmet tcp socket optimize priority");
 
+/*
+ * Define a time period (in msecs) that io_work() shall sample an activated
+ * queue before determining it to be idle.  This optional module behavior
+ * can enable NIC solutions that support socket optimized packet processing
+ * using advanced interrupt moderation techniques.
+ */
+static int queue_idle_period;
+module_param(queue_idle_period, int, 0644);
+MODULE_PARM_DESC(queue_idle_period, "nvmet tcp io_work queue idle time period");
+
 #define NVMET_TCP_RECV_BUDGET		8
 #define NVMET_TCP_SEND_BUDGET		8
 #define NVMET_TCP_IO_WORK_BUDGET	64
@@ -96,6 +106,7 @@ struct nvmet_tcp_queue {
 	struct work_struct	io_work;
 	struct nvmet_cq		nvme_cq;
 	struct nvmet_sq		nvme_sq;
+	unsigned long           idle_period;
 
 	/* send state */
 	struct nvmet_tcp_cmd	*cmds;
@@ -1198,12 +1209,42 @@ static void nvmet_tcp_schedule_release_queue(struct nvmet_tcp_queue *queue)
 	spin_unlock(&queue->state_lock);
 }
 
+/*
+ * This worker function will process all send and recv packet
+ * activity for a queue. It will loop on the queue for up to a
+ * given maximum operation budget, or until there is no activity
+ * during a single loop iteration.
+ *
+ * Two exit modes are possible.
+ *
+ * The default 'pending' mode where the worker will re-queue
+ * itself, after exiting the work loop, only if any send or recv
+ * activity was recorded during the last pass within the loop.
+ *
+ * A optional 'idle period' mode where in addition to re-queueing
+ * itself because of activity it also tracks if a queue has not reached an
+ * assigned 'idle' period of time. The worker consumes from the assigned
+ * time period, across many potential invocations, until it is expired.
+ * A queue with activity always being awarded a fresh time
+ * period for processing.
+ */
 static void nvmet_tcp_io_work(struct work_struct *w)
 {
 	struct nvmet_tcp_queue *queue =
 		container_of(w, struct nvmet_tcp_queue, io_work);
 	bool pending;
 	int ret, ops = 0;
+	long period_remaining;
+	unsigned long idle_deadline;
+
+	/* Setup use of optional tracking for idle time period */
+	if (queue_idle_period) {
+		/* Assign the queue an idle period if not already set */
+		if (!queue->idle_period)
+			queue->idle_period =
+					usecs_to_jiffies(queue_idle_period);
+		idle_deadline = jiffies + queue->idle_period;
+	}
 
 	do {
 		pending = false;
@@ -1222,8 +1263,30 @@ static void nvmet_tcp_io_work(struct work_struct *w)
 
 	} while (pending && ops < NVMET_TCP_IO_WORK_BUDGET);
 
+	/* If optional deadline mode active, determine if queue has reached its
+	 * idle process deadline limit.  Remaining deadline is calculated. Any ops
+	 * activity awards the queue a new deadline period.
+	 */
+	if (queue_idle_period) {
+		/*
+		 * Clear to award active non-idle queue new period, or
+		 * reset for future queue activity after exit when idle reached.
+		 */
+		queue->idle_period = 0;
+		if (ops > 0) {
+			pending = true;
+		} else if (!time_after(jiffies, idle_deadline)) {
+			period_remaining = (long)(idle_deadline - jiffies);
+			if (period_remaining > 0) {
+				pending = true;
+				queue->idle_period = period_remaining;
+			}
+		}
+	}
+
 	/*
-	 * We exahusted our budget, requeue our selves
+	 * We requeue ourself when pending indicates there was activity
+	 * recorded, or queue has not reached optional idle time period.
 	 */
 	if (pending)
 		queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);


From borisp at mellanox.com  Fri Feb 12 00:20:31 2021
From: borisp at mellanox.com (Boris Pismenny)
Date: Fri, 12 Feb 2021 07:20:31 +0200
Subject: [PATCH v4 net-next 21/21] Documentation: add TCP DDP offload
 documentation
Message-ID: <20210212052031.18123-1-borisp@mellanox.com>

Signed-off-by: Boris Pismenny <borisp at mellanox.com>
Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
Signed-off-by: Yoray Zack <yorayz at mellanox.com>
---
 Documentation/networking/index.rst           |   1 +
 Documentation/networking/tcp-ddp-offload.rst | 296 +++++++++++++++++++
 2 files changed, 297 insertions(+)
 create mode 100644 Documentation/networking/tcp-ddp-offload.rst

diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index b8a29997d433..99644159a0cc 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -99,6 +99,7 @@ Contents:
    sysfs-tagging
    tc-actions-env-rules
    tcp-thin
+   tcp-ddp-offload
    team
    timestamping
    tipc
diff --git a/Documentation/networking/tcp-ddp-offload.rst b/Documentation/networking/tcp-ddp-offload.rst
new file mode 100644
index 000000000000..109810e447bf
--- /dev/null
+++ b/Documentation/networking/tcp-ddp-offload.rst
@@ -0,0 +1,296 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+=================================
+TCP direct data placement offload
+=================================
+
+Overview
+========
+
+The Linux kernel TCP direct data placement (DDP) offload infrastructure
+provides tagged request-response protocols, such as NVMe-TCP, the ability to
+place response data directly in pre-registered buffers according to header
+tags. DDP is particularly useful for data-intensive pipelined protocols whose
+responses may be reordered.
+
+For example, in NVMe-TCP numerous read requests are sent together and each
+request is tagged using the PDU header CID field. Receiving servers process
+requests as fast as possible and sometimes responses for smaller requests
+bypasses responses to larger requests, i.e., read 4KB bypasses read 1GB.
+Thereafter, clients corrleate responses to requests using PDU header CID tags.
+The processing of each response requires copying data from SKBs to read
+request destination buffers; The offload avoids this copy. The offload is
+oblivious to destination buffers which can reside either in userspace
+(O_DIRECT) or in kernel pagecache.
+
+Request TCP byte-stream:
+
+.. parsed-literal::
+
+ +---------------+-------+---------------+-------+---------------+-------+
+ | PDU hdr CID=1 | Req 1 | PDU hdr CID=2 | Req 2 | PDU hdr CID=3 | Req 3 |
+ +---------------+-------+---------------+-------+---------------+-------+
+
+Response TCP byte-stream:
+
+.. parsed-literal::
+
+ +---------------+--------+---------------+--------+---------------+--------+
+ | PDU hdr CID=2 | Resp 2 | PDU hdr CID=3 | Resp 3 | PDU hdr CID=1 | Resp 1 |
+ +---------------+--------+---------------+--------+---------------+--------+
+
+Offloading requires no new SKB bits. Instead, the driver builds SKB page
+fragments that point destination buffers. Consequently, SKBs represent the
+original data on the wire, which enables *transparent* inter-operation with the
+network stack.  To avoid copies between SKBs and destination buffers, the
+layer-5 protocol (L5P) will check ``if (src == dst)`` for SKB page fragments,
+success indicates that data is already placed there by NIC hardware and copy
+should be skipped.
+
+Offloading does require NIC hardware to track L5P procotol framing, similarly
+to RX TLS offload (see documentation at
+:ref:`Documentation/networking/tls-offload.rst <tls_offload>`).  NIC hardware
+will parse PDU headers extract fields such as operation type, length, ,tag
+identifier, etc. and offload only segments that correspond to tags registered
+with the NIC, see the :ref:`buf_reg` section.
+
+Device configuration
+====================
+
+During driver initialization the device sets the ``NETIF_F_HW_TCP_DDP`` and
+feature and installs its
+:c:type:`struct tcp_ddp_ops <tcp_ddp_ops>`
+pointer in the :c:member:`tcp_ddp_ops` member of the
+:c:type:`struct net_device <net_device>`.
+
+Later, after the L5P completes its handshake offload is installed on the socket.
+If offload installation fails, then the connection is handled by software as if
+offload was not attempted. Offload installation should configure
+
+To request offload for a socket `sk`, the L5P calls :c:member:`tcp_ddp_sk_add`:
+
+.. code-block:: c
+
+ int (*tcp_ddp_sk_add)(struct net_device *netdev,
+		      struct sock *sk,
+		      struct tcp_ddp_config *config);
+
+The function return 0 for success. In case of failure, L5P software should
+fallback to normal non-offloaded operation.  The `config` parameter indicates
+the L5P type and any metadata relevant for that protocol. For example, in
+NVMe-TCP the following config is used:
+
+.. code-block:: c
+
+ /**
+  * struct nvme_tcp_ddp_config - nvme tcp ddp configuration for an IO queue
+  *
+  * @pfv:        pdu version (e.g., NVME_TCP_PFV_1_0)
+  * @cpda:       controller pdu data alignmend (dwords, 0's based)
+  * @dgst:       digest types enabled.
+  *              The netdev will offload crc if ddp_crc is supported.
+  * @queue_size: number of nvme-tcp IO queue elements
+  * @queue_id:   queue identifier
+  * @cpu_io:     cpu core running the IO thread for this queue
+  */
+ struct nvme_tcp_ddp_config {
+	struct tcp_ddp_config   cfg;
+
+	u16			pfv;
+	u8			cpda;
+	u8			dgst;
+	int			queue_size;
+	int			queue_id;
+	int			io_cpu;
+ };
+
+When offload is not needed anymore, e.g., the socket is being released, the L5P
+calls :c:member:`tcp_ddp_sk_del` to release device contexts:
+
+.. code-block:: c
+
+ void (*tcp_ddp_sk_del)(struct net_device *netdev,
+		        struct sock *sk);
+
+Normal operation
+================
+
+At the very least, the device maintains the following state for each connection:
+
+ * 5-tuple
+ * expected TCP sequence number
+ * mapping between tags and corresponding buffers
+ * current offset within PDU, PDU length, current PDU tag
+
+NICs should not assume any correleation between PDUs and TCP packets.  Assuming
+that TCP packets arrive in-order, offload will place PDU payload directly
+inside corresponding registered buffers. No packets are to be delayed by NIC
+offload. If offload is not possible, than the packet is to be passed as-is to
+software. To perform offload on incoming packets without buffering packets in
+the NIC, the NIC stores some inter-packet state, such as partial PDU headers.
+
+RX data-path
+------------
+
+After the device validates TCP checksums, it can perform DDP offload.  The
+packet is steered to the DDP offload context according to the 5-tuple.
+Thereafter, the expected TCP sequence number is checked against the packet's
+TCP sequence number. If there's a match, then offload is performed: PDU payload
+is DMA written to corresponding destination buffer according to the PDU header
+tag.  The data should be DMAed only once, and the NIC receive ring will only
+store the remaining TCP and PDU headers.
+
+We remark that a single TCP packet may have numerous PDUs embedded inside. NICs
+can choose to offload one or more of these PDUs according to various
+trade-offs. Possibly, offloading such small PDUs is of little value, and it is
+better to leave it to software.
+
+Upon receiving a DDP offloaded packet, the driver reconstructs the original SKB
+using page frags, while pointing to the destination buffers whenever possible.
+This method enables seemless integration with the network stack, which can
+inspect and modify packet fields transperently to the offload.
+
+.. _buf_reg:
+
+Destination buffer registration
+-------------------------------
+
+To register the mapping betwteen tags and destination buffers for a socket
+`sk`, the L5P calls :c:member:`tcp_ddp_setup` of :c:type:`struct tcp_ddp_ops
+<tcp_ddp_ops>`:
+
+.. code-block:: c
+
+ int (*tcp_ddp_setup)(struct net_device *netdev,
+		     struct sock *sk,
+		     struct tcp_ddp_io *io);
+
+
+The `io` provides the buffer via scatter-gather list (`sg_table`) and
+corresponding tag (`command_id`):
+
+.. code-block:: c
+ /**
+  * struct tcp_ddp_io - tcp ddp configuration for an IO request.
+  *
+  * @command_id:  identifier on the wire associated with these buffers
+  * @nents:       number of entries in the sg_table
+  * @sg_table:    describing the buffers for this IO request
+  * @first_sgl:   first SGL in sg_table
+  */
+ struct tcp_ddp_io {
+	u32			command_id;
+	int			nents;
+	struct sg_table		sg_table;
+	struct scatterlist	first_sgl[SG_CHUNK_SIZE];
+ };
+
+After the buffers have been consumed by the L5P, to release the NIC mapping of
+buffers the L5P calls :c:member:`tcp_ddp_teardown` of :c:type:`struct
+tcp_ddp_ops <tcp_ddp_ops>`:
+
+.. code-block:: c
+
+ int (*tcp_ddp_teardown)(struct net_device *netdev,
+			struct sock *sk,
+			struct tcp_ddp_io *io,
+			void *ddp_ctx);
+
+`tcp_ddp_teardown` receives the same `io` context and an additional opaque
+`ddp_ctx` that is used for asynchronous teardown, see the :ref:`async_release`
+section.
+
+.. _async_release:
+
+Asynchronous teardown
+---------------------
+
+To teardown the association between tags and buffers and allow tag reuse NIC HW
+is called by the NIC driver during `tcp_ddp_teardown`. This operation may be
+performed either synchronously or asynchronously. In asynchronous teardown,
+`tcp_ddp_teardown` returns immediately without unmapping NIC HW buffers. Later,
+when the unmapping completes by NIC HW, the NIC driver will call up to L5P
+using :c:member:`ddp_teardown_done` of :c:type:`struct tcp_ddp_ulp_ops`:
+
+.. code-block:: c
+
+ void (*ddp_teardown_done)(void *ddp_ctx);
+
+The `ddp_ctx` parameter passed in `ddp_teardown_done` is the same on provided
+in `tcp_ddp_teardown` and it is used to carry some context about the buffers
+and tags that are released.
+
+Resync handling
+===============
+
+In presence of packet drops or network packet reordering, the device may lose
+synchronization between the TCP stream and the L5P framing, and require a
+resync with the kernel's TCP stack. When the device is out of sync, no offload
+takes place, and packets are passed as-is to software. (resync is very similar
+to TLS offload (see documentation at
+:ref:`Documentation/networking/tls-offload.rst <tls_offload>`)
+
+If only packets with L5P data are lost or reordered, then resynchronization may
+be avoided by NIC HW that keeps tracking PDU headers. If, however, PDU headers
+are reordered, then resynchronization is necessary.
+
+To resynchronize hardware during traffic, we use a handshake between hardware
+and software. The NIC HW searches for a sequence of bytes that identifies L5P
+headers (i.e., magic pattern).  For example, in NVMe-TCP, the PDU operation
+type can be used for this purpose.  Using the PDU header length field, the NIC
+HW will continue to find and match magic patterns in subsequent PDU headers. If
+the pattern is missing in an expected position, then searching for the pattern
+starts anew.
+
+The NIC will not resume offload when the magic pattern is first identified.
+Instead, it will request L5P software to confirm that indeed this is a PDU
+header. To request confirmation the NIC driver calls up to L5P using
+:c:member:`*resync_request` of :c:type:`struct tcp_ddp_ulp_ops`:
+
+.. code-block:: c
+
+  bool (*resync_request)(struct sock *sk, u32 seq, u32 flags);
+
+The `seq` field contains the TCP sequence of the last byte in the PDU header.
+L5P software will respond to this request after observing the packet containing
+TCP sequence `seq` in-order. If the PDU header is indeed there, then L5P
+software calls the NIC driver using the :c:member:`tcp_ddp_resync` function of
+the :c:type:`struct tcp_ddp_ops <tcp_ddp_ops>` inside the :c:type:`struct
+net_device <net_device>` while passing the same `seq` to confirm it is a PDU
+header.
+
+.. code-block:: c
+
+ void (*tcp_ddp_resync)(struct net_device *netdev,
+		       struct sock *sk, u32 seq);
+
+Statistics
+==========
+
+Per L5P protocol, the following NIC driver must report statistics for the above
+netdevice operations and packets processed by offload. For example, NVMe-TCP
+offload reports:
+
+ * ``rx_nvmeotcp_queue_init`` - number of NVMe-TCP offload contexts created.
+ * ``rx_nvmeotcp_queue_teardown`` - number of NVMe-TCP offload contexts
+   destroyed.
+ * ``rx_nvmeotcp_ddp_setup`` - number of DDP buffers mapped.
+ * ``rx_nvmeotcp_ddp_setup_fail`` - number of DDP buffers mapping that failed.
+ * ``rx_nvmeotcp_ddp_teardown`` - number of DDP buffers unmapped.
+ * ``rx_nvmeotcp_drop`` - number of packets dropped in the driver due to fatal
+   errors.
+ * ``rx_nvmeotcp_resync`` - number of packets with resync requests.
+ * ``rx_nvmeotcp_offload_packets`` - number of packets that used offload.
+ * ``rx_nvmeotcp_offload_bytes`` - number of bytes placed in DDP buffers.
+
+NIC requirements
+================
+
+NIC hardware should meet the following requirements to provide this offload:
+
+ * Offload must never buffer TCP packets.
+ * Offload must never modify TCP packet headers.
+ * Offload must never reorder TCP packets within a flow.
+ * Offload must never drop TCP packets.
+ * Offload must not depend on any TCP fields beyond the
+   5-tuple and TCP sequence number.
-- 
2.24.1



From borispismenny at gmail.com  Fri Feb 12 00:22:29 2021
From: borispismenny at gmail.com (Boris Pismenny)
Date: Fri, 12 Feb 2021 07:22:29 +0200
Subject: [PATCH v4 net-next 00/21] nvme-tcp receive offloads
In-Reply-To: <76c93e83-be63-f7ed-a096-1408e9525331@infradead.org>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <76c93e83-be63-f7ed-a096-1408e9525331@infradead.org>
Message-ID: <153bbd21-4704-b28c-d418-8c091cef7348@gmail.com>

On 11/02/2021 23:32, Randy Dunlap wrote:
> 
> Hi,
> Did vger.kernel.org eat patch 21/21?
> 
> and does that patch contain the Documentation updates?
> 
> thanks.
> 

It seems the error was on my end, thanks for raising this, I've resent
that patch now.


From chaitanya.kulkarni at wdc.com  Fri Feb 12 02:13:03 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 11 Feb 2021 23:13:03 -0800
Subject: [PATCH 0/2] nvme: resync nvme-cli and align values
Message-ID: <20210212071305.39901-1-chaitanya.kulkarni@wdc.com>

Hi,

Resync the nvme-cli headers and align the enum values which matches the
rest of the enum definitions.

I've tested this patch with backporting the require commit that needed
nvme repo with nvme-5.12 branch to be in working condition.

-ck

Chaitanya Kulkarni (1):
  nvme: aligned all enum values

Max Gurtovoy (1):
  nvme: resync header file with common nvme-cli tool

 include/linux/nvme.h | 173 ++++++++++++++++++++++++++-----------------
 1 file changed, 104 insertions(+), 69 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Fri Feb 12 02:13:04 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 11 Feb 2021 23:13:04 -0800
Subject: [PATCH 1/2] nvme: resync header file with common nvme-cli tool
In-Reply-To: <20210212071305.39901-1-chaitanya.kulkarni@wdc.com>
References: <20210212071305.39901-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210212071305.39901-2-chaitanya.kulkarni@wdc.com>

From: Max Gurtovoy <mgurtovoy at nvidia.com>

Import constant definitions that were added to nvme-cli and were not
added to Linux. This is the first step to align nvme.h files from Linux
kernel include/linux/nvme.h and nvme-cli linux/nvme.h.

Signed-off-by: Max Gurtovoy <mgurtovoy at nvidia.com>
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
[ Shorten the comments for PMR and CMB which are common shortforms ]
---
 include/linux/nvme.h | 73 ++++++++++++++++++++++++++++++++------------
 1 file changed, 54 insertions(+), 19 deletions(-)

diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index b08787cd0881..3f162e34a2c0 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -113,21 +113,16 @@ enum {
 	NVME_REG_CMBSZ	= 0x003c,	/* Controller Memory Buffer Size */
 	NVME_REG_BPINFO	= 0x0040,	/* Boot Partition Information */
 	NVME_REG_BPRSEL	= 0x0044,	/* Boot Partition Read Select */
-	NVME_REG_BPMBL	= 0x0048,	/* Boot Partition Memory Buffer
-					 * Location
-					 */
-	NVME_REG_CMBMSC = 0x0050,	/* Controller Memory Buffer Memory
-					 * Space Control
-					 */
+	NVME_REG_BPMBL	= 0x0048,	/* Boot Partition Memory Buffer Location */
+	NVME_REG_CMBMSC	= 0x0050,	/* CMB Memory Space Control */
+	NVME_REG_CMBSTS	= 0x0058,	/* CMB Status */
+
 	NVME_REG_PMRCAP	= 0x0e00,	/* Persistent Memory Capabilities */
-	NVME_REG_PMRCTL	= 0x0e04,	/* Persistent Memory Region Control */
-	NVME_REG_PMRSTS	= 0x0e08,	/* Persistent Memory Region Status */
-	NVME_REG_PMREBS	= 0x0e0c,	/* Persistent Memory Region Elasticity
-					 * Buffer Size
-					 */
-	NVME_REG_PMRSWTP = 0x0e10,	/* Persistent Memory Region Sustained
-					 * Write Throughput
-					 */
+	NVME_REG_PMRCTL	= 0x0e04,	/* PMR Control */
+	NVME_REG_PMRSTS	= 0x0e08,	/* PMR Status */
+	NVME_REG_PMREBS	= 0x0e0c,	/* PMR Elasticity Buffer Size */
+	NVME_REG_PMRSWTP = 0x0e10,	/* PMR Sustained Write Throughput */
+	NVME_REG_PMRMSC = 0x0e14,	/* PMR Controller Memory Space Control */
 	NVME_REG_DBS	= 0x1000,	/* SQ 0 Tail Doorbell */
 };
 
@@ -142,6 +137,14 @@ enum {
 
 #define NVME_CMB_BIR(cmbloc)	((cmbloc) & 0x7)
 #define NVME_CMB_OFST(cmbloc)	(((cmbloc) >> 12) & 0xfffff)
+#define NVME_CMB_SZ(cmbsz)	(((cmbsz) >> 12) & 0xfffff)
+#define NVME_CMB_SZU(cmbsz)	(((cmbsz) >> 8) & 0xf)
+
+#define NVME_CMB_WDS(cmbsz)	((cmbsz) & 0x10)
+#define NVME_CMB_RDS(cmbsz)	((cmbsz) & 0x8)
+#define NVME_CMB_LISTS(cmbsz)	((cmbsz) & 0x4)
+#define NVME_CMB_CQS(cmbsz)	((cmbsz) & 0x2)
+#define NVME_CMB_SQS(cmbsz)	((cmbsz) & 0x1)
 
 enum {
 	NVME_CMBSZ_SQS		= 1 << 0,
@@ -244,7 +247,10 @@ struct nvme_id_ctrl {
 	__le32			rtd3e;
 	__le32			oaes;
 	__le32			ctratt;
-	__u8			rsvd100[28];
+	__le16			rrls;
+	__u8			rsvd102[9];
+	__u8			cntrltype;
+	char			fguid[16];
 	__le16			crdt1;
 	__le16			crdt2;
 	__le16			crdt3;
@@ -276,12 +282,14 @@ struct nvme_id_ctrl {
 	__le32			sanicap;
 	__le32			hmminds;
 	__le16			hmmaxd;
-	__u8			rsvd338[4];
+	__le16			nsetidmax;
+	__le16			endgidmax;
 	__u8			anatt;
 	__u8			anacap;
 	__le32			anagrpmax;
 	__le32			nanagrpid;
-	__u8			rsvd352[160];
+	__le32			pels;
+	__u8			rsvd356[156];
 	__u8			sqes;
 	__u8			cqes;
 	__le16			maxcmd;
@@ -295,7 +303,7 @@ struct nvme_id_ctrl {
 	__u8			nvscc;
 	__u8			nwpc;
 	__le16			acwu;
-	__u8			rsvd534[2];
+	__le16			ocfs;
 	__le32			sgls;
 	__le32			mnan;
 	__u8			rsvd544[224];
@@ -368,7 +376,10 @@ struct nvme_id_ns {
 	__le16			npdg;
 	__le16			npda;
 	__le16			nows;
-	__u8			rsvd74[18];
+	__le16			mssrl;
+	__le32			mcl;
+	__u8			msrc;
+	__u8			rsvd81[11];
 	__le32			anagrpid;
 	__u8			rsvd96[3];
 	__u8			nsattr;
@@ -410,8 +421,10 @@ enum {
 	NVME_ID_CNS_CTRL		= 0x01,
 	NVME_ID_CNS_NS_ACTIVE_LIST	= 0x02,
 	NVME_ID_CNS_NS_DESC_LIST	= 0x03,
+	NVME_ID_CNS_NVMSET_LIST		= 0x04,
 	NVME_ID_CNS_CS_NS		= 0x05,
 	NVME_ID_CNS_CS_CTRL		= 0x06,
+	NVME_ID_CNS_CS_NS_ACTIVE_LIST	= 0x07,
 	NVME_ID_CNS_NS_PRESENT_LIST	= 0x10,
 	NVME_ID_CNS_NS_PRESENT		= 0x11,
 	NVME_ID_CNS_CTRL_NS_LIST	= 0x12,
@@ -419,6 +432,10 @@ enum {
 	NVME_ID_CNS_SCNDRY_CTRL_LIST	= 0x15,
 	NVME_ID_CNS_NS_GRANULARITY	= 0x16,
 	NVME_ID_CNS_UUID_LIST		= 0x17,
+	NVME_ID_CNS_CSI_NS_PRESENT_LIST	= 0x1a,
+	NVME_ID_CNS_CSI_NS_PRESENT	= 0x1b,
+	NVME_ID_CNS_CSI			= 0x1c,
+
 };
 
 enum {
@@ -679,6 +696,7 @@ enum nvme_opcode {
 	nvme_cmd_resv_report	= 0x0e,
 	nvme_cmd_resv_acquire	= 0x11,
 	nvme_cmd_resv_release	= 0x15,
+	nvme_cmd_copy		= 0x19,
 	nvme_cmd_zone_mgmt_send	= 0x79,
 	nvme_cmd_zone_mgmt_recv	= 0x7a,
 	nvme_cmd_zone_append	= 0x7d,
@@ -1052,6 +1070,7 @@ enum {
 	NVME_FEAT_PLM_WINDOW	= 0x14,
 	NVME_FEAT_HOST_BEHAVIOR	= 0x16,
 	NVME_FEAT_SANITIZE	= 0x17,
+	NVME_FEAT_IOCS_PROFILE	= 0x19,
 	NVME_FEAT_SW_PROGRESS	= 0x80,
 	NVME_FEAT_HOST_ID	= 0x81,
 	NVME_FEAT_RESV_MASK	= 0x82,
@@ -1068,9 +1087,14 @@ enum {
 	NVME_LOG_TELEMETRY_HOST = 0x07,
 	NVME_LOG_TELEMETRY_CTRL = 0x08,
 	NVME_LOG_ENDURANCE_GROUP = 0x09,
+	NVME_LOG_PRELAT_PER_NVMSET	= 0x0a,
+	NVME_LOG_PRELAT_EVENT_AGG	= 0x0b,
 	NVME_LOG_ANA		= 0x0c,
+	NVME_LOG_PERSISTENT_EVENT   = 0x0d,
 	NVME_LOG_DISC		= 0x70,
 	NVME_LOG_RESERVATION	= 0x80,
+	NVME_LOG_SANITIZE	= 0x81,
+	NVME_LOG_ZONE_CHANGED_LIST = 0xbf,
 	NVME_FWACT_REPL		= (0 << 3),
 	NVME_FWACT_REPL_ACTV	= (1 << 3),
 	NVME_FWACT_ACTV		= (2 << 3),
@@ -1310,6 +1334,9 @@ struct nvmf_disc_rsp_page_entry {
 			__u16	pkey;
 			__u8	resv10[246];
 		} rdma;
+		struct tcp {
+			__u8	sectype;
+		} tcp;
 	} tsas;
 };
 
@@ -1542,6 +1569,14 @@ enum {
 	NVME_SC_ANA_GROUP_ID_INVALID	= 0x124,
 	NVME_SC_ANA_ATTACH_FAILED	= 0x125,
 
+	/*
+	 * Command Set Specific - Namespace Types commands:
+	 */
+	NVME_SC_IOCS_NOT_SUPPORTED	= 0x129,
+	NVME_SC_IOCS_NOT_ENABLED	= 0x12a,
+	NVME_SC_IOCS_COMBINATION_REJ	= 0x12b,
+	NVME_SC_INVALID_IOCS		= 0x12c,
+
 	/*
 	 * I/O Command Set Specific - NVM commands:
 	 */
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Fri Feb 12 02:13:05 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 11 Feb 2021 23:13:05 -0800
Subject: [PATCH 2/2] nvme: aligned all enum values
In-Reply-To: <20210212071305.39901-1-chaitanya.kulkarni@wdc.com>
References: <20210212071305.39901-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210212071305.39901-3-chaitanya.kulkarni@wdc.com>

Add extra tabs and align the values that matches the rest of the
enum definitions.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 include/linux/nvme.h | 108 +++++++++++++++++++++----------------------
 1 file changed, 54 insertions(+), 54 deletions(-)

diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index 3f162e34a2c0..c0d7148f893a 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -1042,62 +1042,62 @@ enum nvme_admin_opcode {
 		nvme_admin_opcode_name(nvme_admin_get_lba_status))
 
 enum {
-	NVME_QUEUE_PHYS_CONTIG	= (1 << 0),
-	NVME_CQ_IRQ_ENABLED	= (1 << 1),
-	NVME_SQ_PRIO_URGENT	= (0 << 1),
-	NVME_SQ_PRIO_HIGH	= (1 << 1),
-	NVME_SQ_PRIO_MEDIUM	= (2 << 1),
-	NVME_SQ_PRIO_LOW	= (3 << 1),
-	NVME_FEAT_ARBITRATION	= 0x01,
-	NVME_FEAT_POWER_MGMT	= 0x02,
-	NVME_FEAT_LBA_RANGE	= 0x03,
-	NVME_FEAT_TEMP_THRESH	= 0x04,
-	NVME_FEAT_ERR_RECOVERY	= 0x05,
-	NVME_FEAT_VOLATILE_WC	= 0x06,
-	NVME_FEAT_NUM_QUEUES	= 0x07,
-	NVME_FEAT_IRQ_COALESCE	= 0x08,
-	NVME_FEAT_IRQ_CONFIG	= 0x09,
-	NVME_FEAT_WRITE_ATOMIC	= 0x0a,
-	NVME_FEAT_ASYNC_EVENT	= 0x0b,
-	NVME_FEAT_AUTO_PST	= 0x0c,
-	NVME_FEAT_HOST_MEM_BUF	= 0x0d,
-	NVME_FEAT_TIMESTAMP	= 0x0e,
-	NVME_FEAT_KATO		= 0x0f,
-	NVME_FEAT_HCTM		= 0x10,
-	NVME_FEAT_NOPSC		= 0x11,
-	NVME_FEAT_RRL		= 0x12,
-	NVME_FEAT_PLM_CONFIG	= 0x13,
-	NVME_FEAT_PLM_WINDOW	= 0x14,
-	NVME_FEAT_HOST_BEHAVIOR	= 0x16,
-	NVME_FEAT_SANITIZE	= 0x17,
-	NVME_FEAT_IOCS_PROFILE	= 0x19,
-	NVME_FEAT_SW_PROGRESS	= 0x80,
-	NVME_FEAT_HOST_ID	= 0x81,
-	NVME_FEAT_RESV_MASK	= 0x82,
-	NVME_FEAT_RESV_PERSIST	= 0x83,
-	NVME_FEAT_WRITE_PROTECT	= 0x84,
-	NVME_FEAT_VENDOR_START	= 0xC0,
-	NVME_FEAT_VENDOR_END	= 0xFF,
-	NVME_LOG_ERROR		= 0x01,
-	NVME_LOG_SMART		= 0x02,
-	NVME_LOG_FW_SLOT	= 0x03,
-	NVME_LOG_CHANGED_NS	= 0x04,
-	NVME_LOG_CMD_EFFECTS	= 0x05,
-	NVME_LOG_DEVICE_SELF_TEST = 0x06,
-	NVME_LOG_TELEMETRY_HOST = 0x07,
-	NVME_LOG_TELEMETRY_CTRL = 0x08,
-	NVME_LOG_ENDURANCE_GROUP = 0x09,
+	NVME_QUEUE_PHYS_CONTIG		= (1 << 0),
+	NVME_CQ_IRQ_ENABLED		= (1 << 1),
+	NVME_SQ_PRIO_URGENT		= (0 << 1),
+	NVME_SQ_PRIO_HIGH		= (1 << 1),
+	NVME_SQ_PRIO_MEDIUM		= (2 << 1),
+	NVME_SQ_PRIO_LOW		= (3 << 1),
+	NVME_FEAT_ARBITRATION		= 0x01,
+	NVME_FEAT_POWER_MGMT		= 0x02,
+	NVME_FEAT_LBA_RANGE		= 0x03,
+	NVME_FEAT_TEMP_THRESH		= 0x04,
+	NVME_FEAT_ERR_RECOVERY		= 0x05,
+	NVME_FEAT_VOLATILE_WC		= 0x06,
+	NVME_FEAT_NUM_QUEUES		= 0x07,
+	NVME_FEAT_IRQ_COALESCE		= 0x08,
+	NVME_FEAT_IRQ_CONFIG		= 0x09,
+	NVME_FEAT_WRITE_ATOMIC		= 0x0a,
+	NVME_FEAT_ASYNC_EVENT		= 0x0b,
+	NVME_FEAT_AUTO_PST		= 0x0c,
+	NVME_FEAT_HOST_MEM_BUF		= 0x0d,
+	NVME_FEAT_TIMESTAMP		= 0x0e,
+	NVME_FEAT_KATO			= 0x0f,
+	NVME_FEAT_HCTM			= 0x10,
+	NVME_FEAT_NOPSC			= 0x11,
+	NVME_FEAT_RRL			= 0x12,
+	NVME_FEAT_PLM_CONFIG		= 0x13,
+	NVME_FEAT_PLM_WINDOW		= 0x14,
+	NVME_FEAT_HOST_BEHAVIOR		= 0x16,
+	NVME_FEAT_SANITIZE		= 0x17,
+	NVME_FEAT_IOCS_PROFILE		= 0x19,
+	NVME_FEAT_SW_PROGRESS		= 0x80,
+	NVME_FEAT_HOST_ID		= 0x81,
+	NVME_FEAT_RESV_MASK		= 0x82,
+	NVME_FEAT_RESV_PERSIST		= 0x83,
+	NVME_FEAT_WRITE_PROTECT		= 0x84,
+	NVME_FEAT_VENDOR_START		= 0xC0,
+	NVME_FEAT_VENDOR_END		= 0xFF,
+	NVME_LOG_ERROR			= 0x01,
+	NVME_LOG_SMART			= 0x02,
+	NVME_LOG_FW_SLOT		= 0x03,
+	NVME_LOG_CHANGED_NS		= 0x04,
+	NVME_LOG_CMD_EFFECTS		= 0x05,
+	NVME_LOG_DEVICE_SELF_TEST	= 0x06,
+	NVME_LOG_TELEMETRY_HOST		= 0x07,
+	NVME_LOG_TELEMETRY_CTRL		= 0x08,
+	NVME_LOG_ENDURANCE_GROUP	= 0x09,
 	NVME_LOG_PRELAT_PER_NVMSET	= 0x0a,
 	NVME_LOG_PRELAT_EVENT_AGG	= 0x0b,
-	NVME_LOG_ANA		= 0x0c,
-	NVME_LOG_PERSISTENT_EVENT   = 0x0d,
-	NVME_LOG_DISC		= 0x70,
-	NVME_LOG_RESERVATION	= 0x80,
-	NVME_LOG_SANITIZE	= 0x81,
-	NVME_LOG_ZONE_CHANGED_LIST = 0xbf,
-	NVME_FWACT_REPL		= (0 << 3),
-	NVME_FWACT_REPL_ACTV	= (1 << 3),
-	NVME_FWACT_ACTV		= (2 << 3),
+	NVME_LOG_ANA			= 0x0c,
+	NVME_LOG_PERSISTENT_EVENT	= 0x0d,
+	NVME_LOG_DISC			= 0x70,
+	NVME_LOG_RESERVATION		= 0x80,
+	NVME_LOG_SANITIZE		= 0x81,
+	NVME_LOG_ZONE_CHANGED_LIST	= 0xbf,
+	NVME_FWACT_REPL			= (0 << 3),
+	NVME_FWACT_REPL_ACTV		= (1 << 3),
+	NVME_FWACT_ACTV			= (2 << 3),
 };
 
 /* NVMe Namespace Write Protect State */
-- 
2.22.1



From Elad.Grupi at dell.com  Fri Feb 12 02:55:14 2021
From: Elad.Grupi at dell.com (Grupi, Elad)
Date: Fri, 12 Feb 2021 07:55:14 +0000
Subject: [PATCH] nvmet-tcp: fix potential race of tcp socket closing
 accept_work
In-Reply-To: <20210205194725.625035-1-sagi@grimberg.me>
References: <20210205194725.625035-1-sagi@grimberg.me>
Message-ID: <DM6PR19MB4011C3BCABE1663D864ECB67EF8B9@DM6PR19MB4011.namprd19.prod.outlook.com>

Sagi,

I think there is another race in that solution.

Please consider following flow:

Thread 1 taking sk_callback_lock and goes into the else block (state is TCP_ESTABLISHED), going to line:
	sock->sk->sk_user_data = queue;

at that point, thread 2 invokes sk_state_change, but this is still the original state_change callback because we didn't set yet to nvmet_tcp_state_change.

Thread 1 continues and set the callback to nvmet_tcp_state_change, but the sk_state_change was already called and nvmet_tcp_state_change will be never called.

In that case, there is a leak, and the queue will never be removed.

Elad

-----Original Message-----
From: Sagi Grimberg <sagi at grimberg.me> 
Sent: Friday, 5 February 2021 21:47
To: linux-nvme at lists.infradead.org; Christoph Hellwig; Keith Busch; Chaitanya Kulkarni
Cc: Grupi, Elad
Subject: [PATCH] nvmet-tcp: fix potential race of tcp socket closing accept_work


[EXTERNAL EMAIL] 

When we accept a TCP connection and allocate an nvmet-tcp queue we should make sure not to fully establish it or reference it as the connection may be already closing, which triggers queue release work, which does not fence against queue establishment.

In order to address such a race, we make sure to check the sk_state and contain the queue reference to be done underneath the sk_callback_lock such that the queue release work correctly fences against it.

Fixes: 872d26a391da ("nvmet-tcp: add NVMe over TCP target driver")
Reported-by: Elad Grupi <elad.grupi at dell.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/target/tcp.c | 28 ++++++++++++++++++----------
 1 file changed, 18 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index 577ce7d403ae..8b0485ada315 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1485,17 +1485,27 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 	if (inet->rcv_tos > 0)
 		ip_sock_set_tos(sock->sk, inet->rcv_tos);
 
+	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
-	sock->sk->sk_user_data = queue;
-	queue->data_ready = sock->sk->sk_data_ready;
-	sock->sk->sk_data_ready = nvmet_tcp_data_ready;
-	queue->state_change = sock->sk->sk_state_change;
-	sock->sk->sk_state_change = nvmet_tcp_state_change;
-	queue->write_space = sock->sk->sk_write_space;
-	sock->sk->sk_write_space = nvmet_tcp_write_space;
+	if (sock->sk->sk_state != TCP_ESTABLISHED) {
+		/*
+		 * If the socket is already closing, don't even start
+		 * consuming it
+		 */
+		ret = -ENOTCONN;
+	} else {
+		sock->sk->sk_user_data = queue;
+		queue->data_ready = sock->sk->sk_data_ready;
+		sock->sk->sk_data_ready = nvmet_tcp_data_ready;
+		queue->state_change = sock->sk->sk_state_change;
+		sock->sk->sk_state_change = nvmet_tcp_state_change;
+		queue->write_space = sock->sk->sk_write_space;
+		sock->sk->sk_write_space = nvmet_tcp_write_space;
+		queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);
+	}
 	write_unlock_bh(&sock->sk->sk_callback_lock);
 
-	return 0;
+	return ret;
 }
 
 static int nvmet_tcp_alloc_queue(struct nvmet_tcp_port *port, @@ -1543,8 +1553,6 @@ static int nvmet_tcp_alloc_queue(struct nvmet_tcp_port *port,
 	if (ret)
 		goto out_destroy_sq;
 
-	queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);
-
 	return 0;
 out_destroy_sq:
 	mutex_lock(&nvmet_tcp_queue_mutex);
--
2.27.0


From dwagner at suse.de  Fri Feb 12 04:30:15 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 12 Feb 2021 10:30:15 +0100
Subject: [PATCH] nvme/hwmon: Return error code when registration fails
Message-ID: <20210212093015.2846-1-dwagner@suse.de>

The hwmon pointer wont be NULL if the registration fails. Though the
exit code path will assign it to ctrl->hwmon_device. Later
nvme_hwmon_exit() will try to free the invalid pointer. Avoid this by
returning the error code from hwmon_device_register_with_info().

Fixes: ec420cdcfab4 ("nvme/hwmon: rework to avoid devm allocation")
Cc: Hannes Reinecke <hare at suse.de>
Signed-off-by: Daniel Wagner <dwagner at suse.de>
---

This patch is against linux-block/for-next.

 drivers/nvme/host/hwmon.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/hwmon.c b/drivers/nvme/host/hwmon.c
index 8f9e96986780..0a586d712920 100644
--- a/drivers/nvme/host/hwmon.c
+++ b/drivers/nvme/host/hwmon.c
@@ -248,6 +248,7 @@ int nvme_hwmon_init(struct nvme_ctrl *ctrl)
 	if (IS_ERR(hwmon)) {
 		dev_warn(dev, "Failed to instantiate hwmon device\n");
 		kfree(data);
+		return PTR_ERR(hwmon);
 	}
 	ctrl->hwmon_device = hwmon;
 	return 0;
-- 
2.29.2



From hare at suse.de  Fri Feb 12 07:46:59 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 12 Feb 2021 13:46:59 +0100
Subject: [PATCH] nvme/hwmon: Return error code when registration fails
In-Reply-To: <20210212093015.2846-1-dwagner@suse.de>
References: <20210212093015.2846-1-dwagner@suse.de>
Message-ID: <a58de23b-40c2-4534-cae0-36b1e42f2d45@suse.de>

On 2/12/21 10:30 AM, Daniel Wagner wrote:
> The hwmon pointer wont be NULL if the registration fails. Though the
> exit code path will assign it to ctrl->hwmon_device. Later
> nvme_hwmon_exit() will try to free the invalid pointer. Avoid this by
> returning the error code from hwmon_device_register_with_info().
> 
> Fixes: ec420cdcfab4 ("nvme/hwmon: rework to avoid devm allocation")
> Cc: Hannes Reinecke <hare at suse.de>
> Signed-off-by: Daniel Wagner <dwagner at suse.de>
> ---
> 
> This patch is against linux-block/for-next.
> 
>   drivers/nvme/host/hwmon.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/nvme/host/hwmon.c b/drivers/nvme/host/hwmon.c
> index 8f9e96986780..0a586d712920 100644
> --- a/drivers/nvme/host/hwmon.c
> +++ b/drivers/nvme/host/hwmon.c
> @@ -248,6 +248,7 @@ int nvme_hwmon_init(struct nvme_ctrl *ctrl)
>   	if (IS_ERR(hwmon)) {
>   		dev_warn(dev, "Failed to instantiate hwmon device\n");
>   		kfree(data);
> +		return PTR_ERR(hwmon);
>   	}
>   	ctrl->hwmon_device = hwmon;
>   	return 0;
> 
Reviewed-by: Hannes Reinecke <hare at suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From nikolay at nvidia.com  Fri Feb 12 09:06:32 2021
From: nikolay at nvidia.com (Nikolay Aleksandrov)
Date: Fri, 12 Feb 2021 16:06:32 +0200
Subject: [PATCH v4 net-next 21/21] Documentation: add TCP DDP offload
 documentation
In-Reply-To: <20210212052031.18123-1-borisp@mellanox.com>
References: <20210212052031.18123-1-borisp@mellanox.com>
Message-ID: <a18aaeec-049e-72c1-f981-3e18381b0f49@nvidia.com>

On 12/02/2021 07:20, Boris Pismenny wrote:
> Signed-off-by: Boris Pismenny <borisp at mellanox.com>
> Signed-off-by: Ben Ben-Ishay <benishay at mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz at mellanox.com>
> Signed-off-by: Yoray Zack <yorayz at mellanox.com>
> ---
>  Documentation/networking/index.rst           |   1 +
>  Documentation/networking/tcp-ddp-offload.rst | 296 +++++++++++++++++++
>  2 files changed, 297 insertions(+)
>  create mode 100644 Documentation/networking/tcp-ddp-offload.rst
> 

Hi Boris,
I got interested and read through the doc, there are a few typos below.

> diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
> index b8a29997d433..99644159a0cc 100644
> --- a/Documentation/networking/index.rst
> +++ b/Documentation/networking/index.rst
> @@ -99,6 +99,7 @@ Contents:
>     sysfs-tagging
>     tc-actions-env-rules
>     tcp-thin
> +   tcp-ddp-offload
>     team
>     timestamping
>     tipc
> diff --git a/Documentation/networking/tcp-ddp-offload.rst b/Documentation/networking/tcp-ddp-offload.rst
> new file mode 100644
> index 000000000000..109810e447bf
> --- /dev/null
> +++ b/Documentation/networking/tcp-ddp-offload.rst
> @@ -0,0 +1,296 @@
> +.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +
> +=================================
> +TCP direct data placement offload
> +=================================
> +
> +Overview
> +========
> +
> +The Linux kernel TCP direct data placement (DDP) offload infrastructure
> +provides tagged request-response protocols, such as NVMe-TCP, the ability to
> +place response data directly in pre-registered buffers according to header
> +tags. DDP is particularly useful for data-intensive pipelined protocols whose
> +responses may be reordered.
> +
> +For example, in NVMe-TCP numerous read requests are sent together and each
> +request is tagged using the PDU header CID field. Receiving servers process
> +requests as fast as possible and sometimes responses for smaller requests
> +bypasses responses to larger requests, i.e., read 4KB bypasses read 1GB.
> +Thereafter, clients corrleate responses to requests using PDU header CID tags.

s/corrleate/correlate/

> +The processing of each response requires copying data from SKBs to read
> +request destination buffers; The offload avoids this copy. The offload is
> +oblivious to destination buffers which can reside either in userspace
> +(O_DIRECT) or in kernel pagecache.
> +
> +Request TCP byte-stream:
> +
> +.. parsed-literal::
> +
> + +---------------+-------+---------------+-------+---------------+-------+
> + | PDU hdr CID=1 | Req 1 | PDU hdr CID=2 | Req 2 | PDU hdr CID=3 | Req 3 |
> + +---------------+-------+---------------+-------+---------------+-------+
> +
> +Response TCP byte-stream:
> +
> +.. parsed-literal::
> +
> + +---------------+--------+---------------+--------+---------------+--------+
> + | PDU hdr CID=2 | Resp 2 | PDU hdr CID=3 | Resp 3 | PDU hdr CID=1 | Resp 1 |
> + +---------------+--------+---------------+--------+---------------+--------+
> +
> +Offloading requires no new SKB bits. Instead, the driver builds SKB page
> +fragments that point destination buffers. Consequently, SKBs represent the
> +original data on the wire, which enables *transparent* inter-operation with the
> +network stack.  To avoid copies between SKBs and destination buffers, the
> +layer-5 protocol (L5P) will check ``if (src == dst)`` for SKB page fragments,
> +success indicates that data is already placed there by NIC hardware and copy
> +should be skipped.
> +
> +Offloading does require NIC hardware to track L5P procotol framing, similarly

s/procotol/protocol/

> +to RX TLS offload (see documentation at
> +:ref:`Documentation/networking/tls-offload.rst <tls_offload>`).  NIC hardware
> +will parse PDU headers extract fields such as operation type, length, ,tag
> +identifier, etc. and offload only segments that correspond to tags registered
> +with the NIC, see the :ref:`buf_reg` section.
> +
> +Device configuration
> +====================
> +
> +During driver initialization the device sets the ``NETIF_F_HW_TCP_DDP`` and
> +feature and installs its
> +:c:type:`struct tcp_ddp_ops <tcp_ddp_ops>`
> +pointer in the :c:member:`tcp_ddp_ops` member of the
> +:c:type:`struct net_device <net_device>`.
> +
> +Later, after the L5P completes its handshake offload is installed on the socket.
> +If offload installation fails, then the connection is handled by software as if
> +offload was not attempted. Offload installation should configure
> +
> +To request offload for a socket `sk`, the L5P calls :c:member:`tcp_ddp_sk_add`:
> +
> +.. code-block:: c
> +
> + int (*tcp_ddp_sk_add)(struct net_device *netdev,
> +		      struct sock *sk,
> +		      struct tcp_ddp_config *config);
> +
> +The function return 0 for success. In case of failure, L5P software should
> +fallback to normal non-offloaded operation.  The `config` parameter indicates
> +the L5P type and any metadata relevant for that protocol. For example, in
> +NVMe-TCP the following config is used:
> +
> +.. code-block:: c
> +
> + /**
> +  * struct nvme_tcp_ddp_config - nvme tcp ddp configuration for an IO queue
> +  *
> +  * @pfv:        pdu version (e.g., NVME_TCP_PFV_1_0)
> +  * @cpda:       controller pdu data alignmend (dwords, 0's based)

s/alignmend/alignment/

> +  * @dgst:       digest types enabled.
> +  *              The netdev will offload crc if ddp_crc is supported.
> +  * @queue_size: number of nvme-tcp IO queue elements
> +  * @queue_id:   queue identifier
> +  * @cpu_io:     cpu core running the IO thread for this queue
> +  */
> + struct nvme_tcp_ddp_config {
> +	struct tcp_ddp_config   cfg;
> +
> +	u16			pfv;
> +	u8			cpda;
> +	u8			dgst;
> +	int			queue_size;
> +	int			queue_id;
> +	int			io_cpu;
> + };
> +
> +When offload is not needed anymore, e.g., the socket is being released, the L5P
> +calls :c:member:`tcp_ddp_sk_del` to release device contexts:
> +
> +.. code-block:: c
> +
> + void (*tcp_ddp_sk_del)(struct net_device *netdev,
> +		        struct sock *sk);
> +
> +Normal operation
> +================
> +
> +At the very least, the device maintains the following state for each connection:
> +
> + * 5-tuple
> + * expected TCP sequence number
> + * mapping between tags and corresponding buffers
> + * current offset within PDU, PDU length, current PDU tag
> +
> +NICs should not assume any correleation between PDUs and TCP packets.  Assuming

s/correleation/correlation/

> +that TCP packets arrive in-order, offload will place PDU payload directly
> +inside corresponding registered buffers. No packets are to be delayed by NIC
> +offload. If offload is not possible, than the packet is to be passed as-is to
> +software. To perform offload on incoming packets without buffering packets in
> +the NIC, the NIC stores some inter-packet state, such as partial PDU headers.
> +
> +RX data-path
> +------------
> +
> +After the device validates TCP checksums, it can perform DDP offload.  The
> +packet is steered to the DDP offload context according to the 5-tuple.
> +Thereafter, the expected TCP sequence number is checked against the packet's
> +TCP sequence number. If there's a match, then offload is performed: PDU payload
> +is DMA written to corresponding destination buffer according to the PDU header
> +tag.  The data should be DMAed only once, and the NIC receive ring will only
> +store the remaining TCP and PDU headers.
> +
> +We remark that a single TCP packet may have numerous PDUs embedded inside. NICs
> +can choose to offload one or more of these PDUs according to various
> +trade-offs. Possibly, offloading such small PDUs is of little value, and it is
> +better to leave it to software.
> +
> +Upon receiving a DDP offloaded packet, the driver reconstructs the original SKB
> +using page frags, while pointing to the destination buffers whenever possible.
> +This method enables seemless integration with the network stack, which can

s/seemless/seamless/

> +inspect and modify packet fields transperently to the offload.

s/transperently/transparently/

> +
> +.. _buf_reg:
> +
> +Destination buffer registration
> +-------------------------------
> +
> +To register the mapping betwteen tags and destination buffers for a socket

s/betwteen/between/

> +`sk`, the L5P calls :c:member:`tcp_ddp_setup` of :c:type:`struct tcp_ddp_ops
> +<tcp_ddp_ops>`:
> +
> +.. code-block:: c
> +
> + int (*tcp_ddp_setup)(struct net_device *netdev,
> +		     struct sock *sk,
> +		     struct tcp_ddp_io *io);
> +
> +
> +The `io` provides the buffer via scatter-gather list (`sg_table`) and
> +corresponding tag (`command_id`):
> +
> +.. code-block:: c
> + /**
> +  * struct tcp_ddp_io - tcp ddp configuration for an IO request.
> +  *
> +  * @command_id:  identifier on the wire associated with these buffers
> +  * @nents:       number of entries in the sg_table
> +  * @sg_table:    describing the buffers for this IO request
> +  * @first_sgl:   first SGL in sg_table
> +  */
> + struct tcp_ddp_io {
> +	u32			command_id;
> +	int			nents;
> +	struct sg_table		sg_table;
> +	struct scatterlist	first_sgl[SG_CHUNK_SIZE];
> + };
> +
> +After the buffers have been consumed by the L5P, to release the NIC mapping of
> +buffers the L5P calls :c:member:`tcp_ddp_teardown` of :c:type:`struct
> +tcp_ddp_ops <tcp_ddp_ops>`:
> +
> +.. code-block:: c
> +
> + int (*tcp_ddp_teardown)(struct net_device *netdev,
> +			struct sock *sk,
> +			struct tcp_ddp_io *io,
> +			void *ddp_ctx);
> +
> +`tcp_ddp_teardown` receives the same `io` context and an additional opaque
> +`ddp_ctx` that is used for asynchronous teardown, see the :ref:`async_release`
> +section.
> +
> +.. _async_release:
> +
> +Asynchronous teardown
> +---------------------
> +
> +To teardown the association between tags and buffers and allow tag reuse NIC HW
> +is called by the NIC driver during `tcp_ddp_teardown`. This operation may be
> +performed either synchronously or asynchronously. In asynchronous teardown,
> +`tcp_ddp_teardown` returns immediately without unmapping NIC HW buffers. Later,
> +when the unmapping completes by NIC HW, the NIC driver will call up to L5P
> +using :c:member:`ddp_teardown_done` of :c:type:`struct tcp_ddp_ulp_ops`:
> +
> +.. code-block:: c
> +
> + void (*ddp_teardown_done)(void *ddp_ctx);
> +
> +The `ddp_ctx` parameter passed in `ddp_teardown_done` is the same on provided
> +in `tcp_ddp_teardown` and it is used to carry some context about the buffers
> +and tags that are released.
> +
> +Resync handling
> +===============
> +
> +In presence of packet drops or network packet reordering, the device may lose
> +synchronization between the TCP stream and the L5P framing, and require a
> +resync with the kernel's TCP stack. When the device is out of sync, no offload
> +takes place, and packets are passed as-is to software. (resync is very similar
> +to TLS offload (see documentation at
> +:ref:`Documentation/networking/tls-offload.rst <tls_offload>`)
> +
> +If only packets with L5P data are lost or reordered, then resynchronization may
> +be avoided by NIC HW that keeps tracking PDU headers. If, however, PDU headers
> +are reordered, then resynchronization is necessary.
> +
> +To resynchronize hardware during traffic, we use a handshake between hardware
> +and software. The NIC HW searches for a sequence of bytes that identifies L5P
> +headers (i.e., magic pattern).  For example, in NVMe-TCP, the PDU operation
> +type can be used for this purpose.  Using the PDU header length field, the NIC
> +HW will continue to find and match magic patterns in subsequent PDU headers. If
> +the pattern is missing in an expected position, then searching for the pattern
> +starts anew.
> +
> +The NIC will not resume offload when the magic pattern is first identified.
> +Instead, it will request L5P software to confirm that indeed this is a PDU
> +header. To request confirmation the NIC driver calls up to L5P using
> +:c:member:`*resync_request` of :c:type:`struct tcp_ddp_ulp_ops`:
> +
> +.. code-block:: c
> +
> +  bool (*resync_request)(struct sock *sk, u32 seq, u32 flags);
> +
> +The `seq` field contains the TCP sequence of the last byte in the PDU header.
> +L5P software will respond to this request after observing the packet containing
> +TCP sequence `seq` in-order. If the PDU header is indeed there, then L5P
> +software calls the NIC driver using the :c:member:`tcp_ddp_resync` function of
> +the :c:type:`struct tcp_ddp_ops <tcp_ddp_ops>` inside the :c:type:`struct
> +net_device <net_device>` while passing the same `seq` to confirm it is a PDU
> +header.
> +
> +.. code-block:: c
> +
> + void (*tcp_ddp_resync)(struct net_device *netdev,
> +		       struct sock *sk, u32 seq);
> +
> +Statistics
> +==========
> +
> +Per L5P protocol, the following NIC driver must report statistics for the above
> +netdevice operations and packets processed by offload. For example, NVMe-TCP
> +offload reports:
> +
> + * ``rx_nvmeotcp_queue_init`` - number of NVMe-TCP offload contexts created.
> + * ``rx_nvmeotcp_queue_teardown`` - number of NVMe-TCP offload contexts
> +   destroyed.
> + * ``rx_nvmeotcp_ddp_setup`` - number of DDP buffers mapped.
> + * ``rx_nvmeotcp_ddp_setup_fail`` - number of DDP buffers mapping that failed.
> + * ``rx_nvmeotcp_ddp_teardown`` - number of DDP buffers unmapped.
> + * ``rx_nvmeotcp_drop`` - number of packets dropped in the driver due to fatal
> +   errors.
> + * ``rx_nvmeotcp_resync`` - number of packets with resync requests.
> + * ``rx_nvmeotcp_offload_packets`` - number of packets that used offload.
> + * ``rx_nvmeotcp_offload_bytes`` - number of bytes placed in DDP buffers.
> +
> +NIC requirements
> +================
> +
> +NIC hardware should meet the following requirements to provide this offload:
> +
> + * Offload must never buffer TCP packets.
> + * Offload must never modify TCP packet headers.
> + * Offload must never reorder TCP packets within a flow.
> + * Offload must never drop TCP packets.
> + * Offload must not depend on any TCP fields beyond the
> +   5-tuple and TCP sequence number.
> 

Cheers,
 Nik



From snitzer at redhat.com  Fri Feb 12 10:45:32 2021
From: snitzer at redhat.com (Mike Snitzer)
Date: Fri, 12 Feb 2021 10:45:32 -0500
Subject: [PATCH 12/78] dm: use set_capacity_and_notify
In-Reply-To: <20201116145809.410558-13-hch@lst.de>
References: <20201116145809.410558-1-hch@lst.de>
 <20201116145809.410558-13-hch@lst.de>
Message-ID: <CAMM=eLfD0_Am3--X+PsKPTfc9qzejxpMNjYwEh=WtjSa-iSncg@mail.gmail.com>

On Mon, Nov 16, 2020 at 10:05 AM Christoph Hellwig <hch at lst.de> wrote:
>
> Use set_capacity_and_notify to set the size of both the disk and block
> device.  This also gets the uevent notifications for the resize for free.
>
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> Reviewed-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/md/dm.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> index c18fc25485186d..62ad44925e73ec 100644
> --- a/drivers/md/dm.c
> +++ b/drivers/md/dm.c
> @@ -1971,8 +1971,7 @@ static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
>         if (size != dm_get_size(md))
>                 memset(&md->geometry, 0, sizeof(md->geometry));
>
> -       set_capacity(md->disk, size);
> -       bd_set_nr_sectors(md->bdev, size);
> +       set_capacity_and_notify(md->disk, size);
>
>         dm_table_event_callback(t, event_callback, md);
>

Not yet pinned down _why_ DM is calling set_capacity_and_notify() with
a size of 0 but, when running various DM regression tests, I'm seeing
a lot of noise like:

[  689.240037] dm-2: detected capacity change from 2097152 to 0

Is this pr_info really useful?  Should it be moved to below: if
(!capacity || !size) so that it only prints if a uevent is sent?

Mike


From hare at suse.de  Fri Feb 12 10:52:27 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 12 Feb 2021 16:52:27 +0100
Subject: [PATCH 1/3] nvmetcli: add 'merge' parameter to set_config()
In-Reply-To: <20210212155229.98816-1-hare@suse.de>
References: <20210212155229.98816-1-hare@suse.de>
Message-ID: <20210212155229.98816-2-hare@suse.de>

Add a parameter 'merge' to set_config() to allow for merge a given
configuration with the existing one.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 nvmet/nvme.py | 8 +++++---
 nvmetcli      | 4 ++--
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/nvmet/nvme.py b/nvmet/nvme.py
index 83fd75b..22aef5a 100644
--- a/nvmet/nvme.py
+++ b/nvmet/nvme.py
@@ -331,17 +331,19 @@ class Root(CFSNode):
         for h in self.hosts:
             h.delete()
 
-    def restore(self, config, clear_existing=False, abort_on_error=False):
+    def restore(self, config, clear_existing=False, abort_on_error=False, merge=False):
         '''
         Takes a dict generated by dump() and reconfigures the target to match.
         Returns list of non-fatal errors that were encountered.
         Will refuse to restore over an existing configuration unless
-        clear_existing is True.
+        clear_existing or merge is True. If clear_existing is True the
+        existing configuration will be cleared. If merge is True the
+        given configuration will be attempted to be merged.
         '''
         if clear_existing:
             self.clear_existing()
         else:
-            if any(self.subsystems):
+            if any(self.subsystems) and not merge:
                 raise CFSError("subsystems present, not restoring")
 
         errors = []
diff --git a/nvmetcli b/nvmetcli
index 8ee8590..f2b59bb 100755
--- a/nvmetcli
+++ b/nvmetcli
@@ -104,11 +104,11 @@ class UIRootNode(UINode):
         UIPortsNode(self)
         UIHostsNode(self)
 
-    def ui_command_restoreconfig(self, savefile=None, clear_existing=False):
+    def ui_command_restoreconfig(self, savefile=None, clear_existing=False, merge=False):
         '''
         Restores configuration from a file.
         '''
-        errors = self.cfnode.restore_from_file(savefile, clear_existing)
+        errors = self.cfnode.restore_from_file(savefile, clear_existing, merge)
         self.refresh()
 
         if errors:
-- 
2.29.2



From hare at suse.de  Fri Feb 12 10:52:26 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 12 Feb 2021 16:52:26 +0100
Subject: [RFC PATCH 0/3] nvmetcli: remote configuation
Message-ID: <20210212155229.98816-1-hare@suse.de>

Hi all,

I've discovered that SPDK has an JSON RPC interface which allows for
remote configuration. And it turns out that the SPDK configuration
itself is pretty similar to the in-kernel NVMe-oF target (surprise,
surprise).
And to not let SPDK have an upper edge here's a simple JSON-RPC proxy
for nvmet, together with a JSON-RPC client 'nvmetadm'.
This allows a full remote configuration of an NVMe-oF target:

# nvmetadm nvmf_create_subsystem --nqn=nqn.nvmf-test1
# nvmetadm nvmf_subsystem_add_port --nqn=nqn.nvmf-test1 \
  --traddr=<target-ip> --trtrype=tcp --adrfam=ipv6
# nvmetadm bdev_create_file --file-name=Testfile1 --size=1G
# nvmetadm nvmf_subsystem_add_ns --nqn=nqn.nvmf-test1 \
  --bdev=Testfile1
# nvmetadm nvmf_subsystem_add_host --nqn=nqn.nvmf-test1 \
  --host=<hostnqn>
# nvme discover --transport=tcp --traddr=<target-ip> \
  --trsvcid=4420

With this it should be possible to convert blktests nvme suite to use
this interface, which will allow blktest to run across arbitrary
transports instead of just nvme-loop.

As usual, comments and reviews are welcome.

Hannes Reinecke (3):
  nvmetcli: add 'merge' parameter to set_config()
  nvmetproxy: add a JSON-RPC proxy daemon
  nvmetadm: add JSON-RPC client for remote configuration

 Documentation/Makefile       |  24 +-
 Documentation/nvmetproxy.txt | 111 ++++++++
 nvmet/__init__.py            |   4 +-
 nvmet/bdev.py                |  72 +++++
 nvmet/nvme.py                |   8 +-
 nvmet/rpc.py                 | 495 +++++++++++++++++++++++++++++++++++
 nvmetadm                     | 259 ++++++++++++++++++
 nvmetcli                     |   4 +-
 nvmetproxy                   | 197 ++++++++++++++
 setup.py                     |   2 +-
 10 files changed, 1157 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/nvmetproxy.txt
 create mode 100644 nvmet/bdev.py
 create mode 100644 nvmet/rpc.py
 create mode 100755 nvmetadm
 create mode 100755 nvmetproxy

-- 
2.29.2



From hare at suse.de  Fri Feb 12 10:52:29 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 12 Feb 2021 16:52:29 +0100
Subject: [PATCH 3/3] nvmetadm: add JSON-RPC client for remote configuration
In-Reply-To: <20210212155229.98816-1-hare@suse.de>
References: <20210212155229.98816-1-hare@suse.de>
Message-ID: <20210212155229.98816-4-hare@suse.de>

Add a JSON-RPC client for remote configuration of an NVMe-over-Fabrics
target.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 nvmetadm | 259 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 setup.py |   2 +-
 2 files changed, 260 insertions(+), 1 deletion(-)
 create mode 100755 nvmetadm

diff --git a/nvmetadm b/nvmetadm
new file mode 100755
index 0000000..74479ff
--- /dev/null
+++ b/nvmetadm
@@ -0,0 +1,259 @@
+#!/usr/bin/python3
+
+import sys
+import argparse
+import requests
+import json
+
+class rpc_client():
+    def bdev_file_create(self, args):
+        self.parser.add_argument('--filename', required=True)
+        self.parser.add_argument('--size', required=True)
+        self.parser.add_argument('--pool')
+        a = self.parser.parse_args(args)
+        s = a.size.lower().rfind('g')
+        if s != -1:
+            size = int(a.size.lower()[:-s]) * 1024 * 1024 * 1024
+        else:
+            s = a.size.lower().rfind('m')
+            if s != -1:
+                size = int(a.size.lower()[:-s]) * 1024 * 1024
+            else:
+                s = a.size.lower().rfind('k')
+                if s != -1:
+                    size = int(a.size.lower()[:-s]) * 1024
+                else:
+                    size = int(a.size)
+        params = {'file_name': a.filename, 'size': size}
+        if a.pool:
+            params['pool'] = a.pool
+        return params
+
+    def bdev_file_delete(self, args):
+        self.parser.add_argument('--filename', required=True)
+        self.parser.add_argument('--pool')
+        a = self.parser.parse_args(args)
+        params = {'filename': a.filename }
+        if a.pool:
+            params['pool'] = a.pool
+        return params
+
+    def nvmf_set_config(self, args):
+        self.parser.add_argument('--cfg-file', required=True)
+        a = self.parser.parse_args(args)
+        try:
+            cfg = open(a.cfg_file, "r")
+        except OSError:
+            sys.exit(1)
+        params = json.load(cfg)
+        cfg.close()
+        return params
+
+    def nvmf_create_transport(self, args):
+        self.parser.add_argument('--trtype', required=True)
+        a = self.parser.parse_args(args)
+        if not a.trtype:
+            return None
+        return {"trtype": a.trtype}
+
+    def nvmf_create_subsystem(self, args):
+        self.parser.add_argument('--nqn')
+        a = self.parser.parse_args(args)
+        params = None
+        if not a.nqn:
+            return None
+        return {"nqn": a.nqn}
+
+    def nvmf_delete_subsystem(self, args):
+        self.parser.add_argument('--nqn', required=True)
+        a = self.parser.parse_args(args)
+        return {"nqn": a.nqn}
+
+    def nvmf_subsystem_add_ns(self, args):
+        self.parser.add_argument('--nqn', required=True)
+        self.parser.add_argument('--bdev-name', required=True)
+        self.parser.add_argument('--nsid')
+        self.parser.add_argument('--nguid')
+        self.parser.add_argument('--eui64')
+        self.parser.add_argument('--uuid')
+        a = self.parser.parse_args(args)
+        ns = { 'bdev_name': a.bdev_name }
+        if a.nsid:
+            ns['nsid'] = a.nsid
+        if a.nguid:
+            ns['nguid'] = a.nguid
+        if a.eui64:
+            ns['eui64'] = a.eui64
+        if a.uuid:
+            ns['uuid'] = a.uuid
+        return {"nqn": a.nqn, 'namespace': ns}
+
+    def nvmf_subsystem_remove_ns(self, args):
+        self.parser.add_argument('--nqn', required=True)
+        self.parser.add_argument('--nsid', required=True, type=int)
+        a = self.parser.parse_args(args)
+        return {'nqn': a.nqn, 'nsid': a.nsid }
+
+    def nvmf_subsystem_port(self, args):
+        self.parser.add_argument('--nqn', required=True)
+        self.parser.add_argument('--portid', type=int)
+        self.parser.add_argument('--trtype', required=True)
+        self.parser.add_argument('--traddr', required=True)
+        self.parser.add_argument('--adrfam')
+        self.parser.add_argument('--trscvid')
+        self.parser.add_argument('--host-traddr')
+        a = self.parser.parse_args(args)
+        port = { 'trtype': a.trtype, 'traddr': a.traddr }
+        if a.portid:
+            port['portid'] = a.portid
+        if a.adrfam:
+            port['adrfam'] = a.adrfam
+        if a.trscvid:
+            port['trsvcid'] = a.trscvid
+        if a.host_traddr:
+            port['host_traddr'] = a.host_traddr
+        return {"nqn": a.nqn, 'port': port}
+
+    def nvmf_subsystem_host(self, args):
+        self.parser.add_argument('--nqn', required=True)
+        self.parser.add_argument('--host', required=True)
+        a = self.parser.parse_args(args)
+        return {'nqn': a.nqn, 'host': a.host }
+
+    def nvmf_port_ana(self, args):
+        self.parser.add_argument('--portid', required=True)
+        self.parser.add_argument('--grpid')
+        self.parser.add_argument('--ana_state')
+        a = self.parser.parse_args(args)
+        params = { 'portid': a.portid }
+        if a.grpid:
+            params['grpid'] = a.grpid
+        if a.ana_state:
+            params['ana_state'] = a.ana_state
+        return params
+
+    def method_no_param(self, args = None):
+        a = self.parser.parse_args(args)
+        return
+
+    _rpc_methods = dict(bdev_file_list_pools=method_no_param,
+                        bdev_file_create=bdev_file_create,
+                        bdev_file_delete=bdev_file_delete,
+                        nvmf_get_config=method_no_param,
+                        nvmf_set_config=nvmf_set_config,
+                        nvmf_get_interfaces=method_no_param,
+                        nvmf_create_transport=nvmf_create_transport,
+                        nvmf_get_transports=method_no_param,
+                        nvmf_create_subsystem=nvmf_create_subsystem,
+                        nvmf_delete_subsystem=nvmf_delete_subsystem,
+                        nvmf_subsystem_add_ns= nvmf_subsystem_add_ns,
+                        nvmf_subsystem_remove_ns=nvmf_subsystem_remove_ns,
+                        nvmf_subsystem_add_port=nvmf_subsystem_port,
+                        nvmf_subsystem_remove_port=nvmf_subsystem_port,
+                        nvmf_subsystem_add_host=nvmf_subsystem_host,
+                        nvmf_subsystem_remove_host=nvmf_subsystem_host,
+                        nvmf_port_add_ana=nvmf_port_ana,
+                        nvmf_port_set_ana=nvmf_port_ana,
+                        nvmf_port_remove_ana=nvmf_port_ana,
+                        nvmf_get_subsystems=method_no_param)
+
+    def __init__(self, method):
+        if method not in self._rpc_methods:
+            print("Invalid rpc method '%s'; must be one of" % method)
+            print(", ".join([x for x in list(self._rpc_methods)]))
+            sys.exit(1)
+                
+        self.parser = argparse.ArgumentParser(prog="nvmetadm %s" % method)
+        self._method = self._rpc_methods[method]
+
+    def _error(self, err):
+        data = None
+        if 'code' not in err:
+            code = -32000
+            message = "Server error"
+            data = []
+            data.append("Invalid JSON RPC response")
+        else:
+            code = err['code']
+        if 'message' in err:
+            message = err['message']
+        else:
+            if code == -32700:
+                message = "Parse error"
+            elif code == -32600:
+                message = "Invalid Request"
+            elif code == -32601:
+                message = "Method not found"
+            elif code == -32602:
+                message = "Invalid params"
+            elif code == -32603:
+                message = "Internal error"
+            elif code <= -32000 and code > -32100:
+                message = "Server error"
+            else:
+                data = []
+                data.append("Invalid JSON RPC error code %d" & code)
+                code = -32001
+                message = "Server error"
+        if 'data' in err:
+            data = err['data']
+        return (code, message, data)
+
+    def call(self, args):
+        return self._method(self, args)
+
+    def response(self, response):
+        if 'error' in response:
+            (code, message, data) = self._error(response['error'])
+            if data:
+                sys.exit("JSON RPC error %d: %s\n%s" % (code, message, json.dumps(data)))
+            else:
+                sys.exit("JSON RPC error %d: %s" % (code, message))
+        elif 'result' not in response:
+            sys.exit("JSON RPC error: Not a valid response\n%s", response)
+        elif response['result']:
+            return response['result']
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('-H', '--host', help='Hostname to connect to',
+                        required=True)
+    parser.add_argument('-p', '--port', type=int, default=4260,
+                        help='Port number for the connection')
+    parser.add_argument('-U', '--username')
+    parser.add_argument('-P', '--password')
+    parser.add_argument('-u', '--uri', default='nvmet',
+                        help='URI of the JSON-RPC namespace')
+    (a, args) = parser.parse_known_args()
+    if a.username:
+        if not a.password:
+            sys.exit("Need to specify both username and password")
+        auth = (a.username, a.password)
+    elif not a.password:
+        auth = None
+    else:
+        sys.exit("Need to specify both username and password")
+    url = "http://%s:%d/%s" % (a.host, a.port, a.uri)
+
+    if not args or not len(args):
+        sys.exit("No method given")
+
+    method = args.pop(0)
+    payload = {
+        "method": method,
+        "jsonrpc": "2.0",
+        "id": 0,
+    }
+    rpc = rpc_client(method)
+    params = rpc.call(args)
+    if params:
+        payload['params'] = params
+    response = requests.post(url, auth=auth,json=payload)
+    response.raise_for_status()
+    msg = rpc.response(response.json())
+    if msg:
+        print("%s" % json.dumps(msg, indent=2))
+
+
+if __name__ == "__main__":
+    main()
diff --git a/setup.py b/setup.py
index f15e5b0..998ff78 100755
--- a/setup.py
+++ b/setup.py
@@ -27,5 +27,5 @@ setup(
     maintainer_email = 'hch at lst.de',
     test_suite='nose2.collector.collector',
     packages = ['nvmet'],
-    scripts=['nvmetcli', 'nvmetproxy']
+    scripts=['nvmetcli', 'nvmetproxy', 'nvmetadm']
     )
-- 
2.29.2



From hare at suse.de  Fri Feb 12 10:52:28 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 12 Feb 2021 16:52:28 +0100
Subject: [PATCH 2/3] nvmetproxy: add a JSON-RPC proxy daemon
In-Reply-To: <20210212155229.98816-1-hare@suse.de>
References: <20210212155229.98816-1-hare@suse.de>
Message-ID: <20210212155229.98816-3-hare@suse.de>

Add a JSON-RPC proxy daemon to allow for remote configuration
of the NVMe-over-Fabrics target.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 Documentation/Makefile       |  24 +-
 Documentation/nvmetproxy.txt | 111 ++++++++
 nvmet/__init__.py            |   4 +-
 nvmet/bdev.py                |  72 +++++
 nvmet/rpc.py                 | 495 +++++++++++++++++++++++++++++++++++
 nvmetproxy                   | 197 ++++++++++++++
 setup.py                     |   2 +-
 7 files changed, 891 insertions(+), 14 deletions(-)
 create mode 100644 Documentation/nvmetproxy.txt
 create mode 100644 nvmet/bdev.py
 create mode 100644 nvmet/rpc.py
 create mode 100755 nvmetproxy

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 8e0281c..2778429 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -1,31 +1,31 @@
 PKGNAME = nvmetcli
-MANPAGE = ${PKGNAME}.8
-HTMLFILE = ${PKGNAME}.html
-XMLFILE = ${PKGNAME}.xml
 INSTALL ?= install
 PREFIX := /usr
 
 ASCIIDOC = asciidoc
 XMLTO = xmlto --skip-validation
 
-DOCDATA = ${XMLFILE} ${HTMLFILE}
- 
-${MANPAGE}: ${DOCDATA}
+DOCFILES = ${PKGNAME}.txt nvmetproxy.txt
+MANPAGES = ${DOCFILES:.txt=.8}
+HTMLFILES = ${DOCFILES:.txt=.html}
+XMLFILES = ${DOCFILES:.txt=.xml}
+
+%.8: %.xml
 	${XMLTO} man $<
- 
+
 %.xml: %.txt
 	${ASCIIDOC} -b docbook -d manpage -o $@ $<
- 
+
 %.html: %.txt
 	${ASCIIDOC} -a toc -o $@ $<
- 
+
 installdoc: man8
 
-man8:
-	${INSTALL} -m 644 ${MANPAGE} ${PREFIX}/share/man/man8
+man8: ${MANPAGES}
+	${INSTALL} -m 644 $< ${PREFIX}/share/man/man8
 
 uninstalldoc:
 	-rm -f ${PREFIX}/share/man/man8/${MANPAGE}
 
 clean:
-	-rm -f ${MANPAGE} ${HTMLFILE} ${XMLFILE}
+	-rm -f ${MANPAGES} ${HTMLFILES} ${XMLFILES}
diff --git a/Documentation/nvmetproxy.txt b/Documentation/nvmetproxy.txt
new file mode 100644
index 0000000..b273f5b
--- /dev/null
+++ b/Documentation/nvmetproxy.txt
@@ -0,0 +1,111 @@
+nvmetproxy(8)
+===========
+
+NAME
+----
+nvmetproxy - JSON-RPC proxy server to configure NVMe-over-Fabrics Target.
+
+USAGE
+------
+[verse]
+'nvmetproxy'
+	[--socket=<socket> | -s <socket>]
+	[--host=<hostname> | -H <hostname>]
+	[--port=<port>     | -p <port>]
+	[--user=<username> | -U <username> ]
+	[--password=<password> | -P <password> ]
+	[--cert=<certfile>     | -c <certfile> ]
+	[--url=<url>           | -u <url>      ]
+
+DESCRIPTION
+-----------
+*nvmetproxy* is JSON-RPC proxy for viewing, editing, saving,
+and starting a Linux kernel NVMe Target, used for an NVMe-over-Fabrics
+network configuration.  It allows an administrator to export
+a storage resource (such as block devices, files, and volumes)
+to a local NVMe block device or expose them to remote systems
+based on the NVMe-over-Fabrics specification from http://www.nvmexpress.org.
+
+The JSON-RPC commands are compatible with the JSON-RPC commands
+the from SPDK.
+
+Connection to the daemon can be via a socket (as specified --socket)
+or via HTTP if the --host option is present.
+
+OPTIONS
+-------
+-s <socket>::
+--socket=<socket>::
+	Socket to listen on, default is /var/tmp/nvmet.sock
+
+-H <hostname>::
+--host=<hostname>::
+	Hostname to listen for HTTP requests. If specified the HTTP
+	server will started and the --socket option is ignored.
+
+-p <port>::
+--port=<port>::
+	Port number to listen for HTTP requests. Default is 4260
+
+-U <username>::
+--user=<username>::
+	Username to use for HTTP Basic Authentication. If not present
+	the HTTP server will not require authentication.
+
+-P <password>::
+--password=<password>::
+	Password to use for HTTP Basic Authentication. Must be specified
+	together with --user
+
+-c <cert>::
+--cert=<cert>::
+	SSL Server certificate to use for HTTP server. If specified the
+	HTTP server will only accept encrypted connections (https).
+
+-u <url>::
+--url=<url>::
+	URL directory to use for the configuration. Defaults to '/nvmet'
+
+
+JSON-RPC commands
+-----------------
+The proxy is compatible with the command set from the JSON-RPC calls from SPDK.
+Currently the following methods are implemented:
+
+	bdev_file_list_pools
+	bdev_file_create
+	bdev_file_delete
+	bdev_file_snapshot
+	bdev_file_clone
+	nvmf_get_interfaces
+	nvmf_create_transport
+	nvmf_get_transports
+	nvmf_create_subsystem
+	nvmf_delete_subsystem
+	nvmf_subsystem_add_ns
+	nvmf_subsystem_remove_ns
+	nvmf_subsystem_add_port
+	nvmf_subsystem_remove_port
+	nvmf_subsystem_add_host
+	nvmf_subsystem_remove_host
+	nvmf_get_subsystems
+	nvmf_port_add_ana
+	nvmf_port_set_ana
+	nvmf_port_remove_ana
+	nvmf_get_config
+	nvmf_set_config
+
+AUTHORS
+-------
+nvmetproxy was written by mailto:hare at suse.de[Hannes Reinecke]
+
+REPORTING BUGS & DEVELOPMENT
+-----------------------------
+Please send patches and bug reports to linux-nvme at lists.infradead.org
+for review and acceptance.
+
+LICENSE
+-------
+nvmetproxy is licensed under the *Apache License, Version 2.0*. Software
+distributed under this license is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either expressed or implied.
diff --git a/nvmet/__init__.py b/nvmet/__init__.py
index cf172bd..730393b 100644
--- a/nvmet/__init__.py
+++ b/nvmet/__init__.py
@@ -1,2 +1,4 @@
-from .nvme import Root, Subsystem, Namespace, Port, Host, Referral, ANAGroup,\
+from .bdev import BackingFile
+from .rpc import JsonRPC
+from .nvme import CFSError, Root, Subsystem, Namespace, Port, Host, Referral, ANAGroup,\
     DEFAULT_SAVE_FILE
diff --git a/nvmet/bdev.py b/nvmet/bdev.py
new file mode 100644
index 0000000..41735f5
--- /dev/null
+++ b/nvmet/bdev.py
@@ -0,0 +1,72 @@
+#!/usr/bin/python
+
+'''
+BackingFile JSON-RPC handling functions
+
+Copyright (c) 2021 by Hannes Reinecke, SUSE Linux LLC
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may
+not use this file except in compliance with the License. You may obtain
+a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations
+under the License.
+'''
+
+from __future__ import print_function
+
+import os
+
+class BackingFile:
+    def __init__(self, pools, name, pool = None, mode = 'lookup'):
+        self.name = name
+        if mode != 'lookup' and not pool:
+            for pool in pools:
+                break
+        if pool and pool not in pools:
+            raise NameError("Invalid pool '%s'" % pool)
+        if mode == 'lookup':
+            if not pool:
+                for pool in pools:
+                    self.prefix = pools[pool]
+                    path = self.prefix + '/' + name
+                    if os.path.exists(path):
+                        self.file_path = path
+                        break
+                if not self.file_path:
+                    raise NameError("backing file '%s' not found" % name)
+            else:
+                self.prefix = pools[pool]
+                path = self.prefix + '/' + name
+                if not os.path.exists(path):
+                    raise NameError("%s: backing file '%s' not found" % (pool, name))
+                self.file_path = path
+        elif mode == 'create':
+            self.prefix = pools[pool]
+            path = self.prefix + '/' + name
+            if os.path.exists(path):
+                raise NameError("%s: backing file '%s' already exists" % (pool, path))
+            try:
+                fd = open(path, 'x')
+            except:
+                raise NameError("%s: Cannot create backing file '%s'" % (pool, name))
+            fd.close()
+            self.file_path = path
+
+    def delete(self):
+        try:
+            os.remove(self.file_path)
+        except FileNotFoundError:
+            return RuntimeError("%s: cannot delete backing file '%s'" % (pool, name))
+
+    def set_size(self, file_size):
+        self.file_size = file_size
+        try:
+            os.truncate(self.file_path, self.file_size)
+        except:
+            raise RuntimeError("Failed to truncate %s" % self.file_path)
diff --git a/nvmet/rpc.py b/nvmet/rpc.py
new file mode 100644
index 0000000..4ec7d75
--- /dev/null
+++ b/nvmet/rpc.py
@@ -0,0 +1,495 @@
+'''
+Implements JSON-RPC handlers for the NVMe target configfs hierarchy
+
+Copyright (c) 2021 Hannes Reinecke, SUSE LLC
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may
+not use this file except in compliance with the License. You may obtain
+a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations
+under the License.
+'''
+
+import os
+import stat
+import json
+import nvmet as nvme
+import netifaces
+from pathlib import Path
+
+class JsonRPC:
+    def _get_subsystems(self, params = None):
+        return [s.dump() for s in self.cfg.subsystems]
+
+    def _get_transports(self, params = None):
+        km = kmod.Kmod()
+        ml = []
+        for m in km.loaded():
+            if m.name.startswith("nvmet_"):
+                ml.append(m.name[len("nvmet_"):])
+            elif m.name == "nvme_loop":
+                ml.append(m.name[len("nvme_"):])
+        return ml
+
+    def _create_transport(self, params):
+        if not params or 'trtype' not in params:
+            raise NameError("Parameter 'trtype' missing")
+        trtype = params['trtype']
+        if trtype.lower() == 'loop':
+            prefix = "nvme_"
+        else:
+            prefix = "nvmet_"
+        try:
+            self.cfg._modprobe(prefix + trtype.lower())
+        except:
+            raise NameError("Module %s%s not found" % (prefix, trtype.lower()))
+
+    def _create_subsystem(self, params):
+        if not params or 'nqn' not in params:
+            nqn = None
+        else:
+            nqn = params['nqn']
+        try:
+            subsys = nvme.Subsystem(nqn, mode='create')
+        except:
+            raise RuntimeError("Failed to create subsystem %s" % nqn)
+        if not params:
+            return subsys.nqn
+        if 'model_number' in params:
+            model = params['model_number']
+            if (len(model) > 40):
+                subsys.delete()
+                raise NameError("Model number longer than 40 characters")
+            try:
+                subsys.set_attr("attr", "model", model)
+            except:
+                subsys.delete()
+                raise RuntimeError("Failed to set model %s" % model)
+        if 'serial_number' in params:
+            serial = params['serial_number']
+            if len(serial) > 20:
+                subsys.delete()
+                raise NameError("Serial number longer than 20 characters")
+            try:
+                subsys.set_attr("attr", "serial", serial)
+            except:
+                subsys.delete()
+                raise RuntimeError("Failed to set serial %s" % serial)
+        if 'allow_any_host' in params:
+            subsys.set_attr("attr", "allow_any_host", "1")
+        return subsys.nqn
+
+    def _delete_subsystem(self, params):
+        if not params and 'nqn' not in params:
+            raise NameError("Parameter 'nqn' missing")
+        nqn = params['nqn']
+        try:
+            subsys = nvme.Subsystem(nqn, mode='lookup');
+        except nvme.CFSError:
+            raise NameError("Subsystem %s not found" % nqn)
+        try:
+            subsys.delete()
+        except:
+            raise RuntimeError("Failed to delete subsystem %s" % nqn)
+
+    def _add_ns(self, params):
+        if not params or 'nqn' not in params:
+            raise NameError("Parameter 'nqn' missing")
+        nqn = params['nqn']
+        if 'namespace' not in params:
+            raise NameError("Parameter 'namespace' missing")
+        ns_params = params['namespace']
+        if 'bdev_name' not in ns_params:
+            raise NameError("Parameter 'namespace:bdev_name' missing")
+        bdev_name = ns_params['bdev_name']
+        try:
+            bdev = nvme.BackingFile(self.pools, bdev_name)
+        except:
+            raise NameError("bdev %s not found" % bdev_name)
+        try:
+            subsys = nvme.Subsystem(nqn, mode='lookup')
+        except nvme.CFSError:
+            raise NameError("Subsystem %s not found" % nqn)
+
+        if 'nsid' in ns_params:
+            nsid = ns_params['nsid']
+        else:
+            nsid = None
+
+        try:
+            ns = nvme.Namespace(subsys, nsid, mode='create')
+        except:
+            raise NameError("Namespace %s already exists" % nsid)
+        nsid = ns.nsid
+        if 'uuid' in ns_params:
+            try:
+                ns.set_attr("device", "uuid", ns_params['uuid'])
+            except:
+                ns.delete()
+                raise RuntimeError("Failed to set uuid %s on ns %s" % (ns_params['uuid'], nsid))
+        if 'nguid' in ns_params:
+            try:
+                ns.set_attr("device", "nguid", ns_params['nguid'])
+            except:
+                ns.delete()
+                raise RuntimeError("Failed to set nguid %s on ns %s" % (ns_params['nguid'], nsid))
+        try:
+            ns.set_attr("device", "path", bdev.file_path)
+        except:
+            ns.delete()
+            raise RuntimeError("Failed to set path on ns %s" % nsid)
+        try:
+            ns.set_enable(1)
+        except:
+            raise RuntimeError("Failed to enable ns %s" % nsid)
+        return nsid
+
+    def _remove_ns(self, params):
+        if not params and 'nqn' not in params:
+            raise NameError("Parameter 'nqn' missing")
+        nqn = params['nqn']
+        if 'nsid' not in params:
+            raise NameError("Parameter 'nsid' missing")
+        nsid = params['nsid']
+        try:
+            subsys = nvme.Subsystem(nqn, mode='lookup')
+        except nvme.CFSError:
+            raise NameError("Subsystem %s not found" % nqn)
+        try:
+            ns = nvme.Namespace(subsys, nsid, mode='lookup')
+        except nvme.CFSError:
+            raise NameError("Namespace %d not found" % nsid)
+        ns.delete()
+
+    def _add_port(self, params):
+        if not params or 'nqn' not in params:
+            raise NameError("Parameter 'nqn' missing")
+        if 'listen_address' in params:
+            port_params = params['listen_address']
+        elif 'port' in params:
+            port_params = params['port']
+        else:
+            raise NameError("Parameter 'listen_address' missing")
+        if 'portid' not in port_params:
+            ids = [port.portid for port in self.cfg.ports]
+            if len(ids):
+                portid = max(ids)
+            else:
+                portid = 0
+            portid = portid + 1
+        else:
+            portid = int(port_params['portid'])
+        try:
+            port = nvme.Port(portid, mode='create')
+        except:
+            raise RuntimeError("Port %d already exists" % portid)
+        for p in ('trtype', 'adrfam', 'traddr', 'trsvcid'):
+            if p not in port_params:
+                if p != 'trsvcid':
+                    port.delete()
+                    raise NameError("Invalid listen_address parameter %s" % p)
+                else:
+                    v = 4420
+            else:
+                v = port_params[p]
+            if p == 'adrfam':
+                v = port_params[p].lower()
+            try:
+                port.set_attr("addr", p, v)
+            except:
+                port.delete()
+                raise RuntimeError("Failed to set %s to %s" % (p, v))
+        nqn = params['nqn']
+        try:
+            port.add_subsystem(nqn)
+        except:
+            port.delete()
+            raise NameError("subsystem %s not found" % nqn)
+        return port.portid
+
+    def _remove_port(self, params):
+        if not params or 'nqn' not in params:
+            raise NameError("Parameter 'nqn' missing")
+        if 'listen_address' in params:
+            port_params = params['listen_address']
+        elif 'port' in params:
+            port_params = params['port']
+        else:
+            raise NameError("Parameter 'listen_address' missing")
+        nqn = params['nqn']
+        for port in self.cfg.ports:
+            for p in ('trtype', 'adrfam', 'traddr', 'trsvcid'):
+                if p not in port_params:
+                    continue
+                if port.get_attr("addr", p) != port_params[p]:
+                    continue
+            for s in port.subsystems:
+                if s != nqn:
+                    continue
+                port.remove_subsystem(nqn)
+                if not len(port.subsystems):
+                    port.delete()
+
+    def _add_host(self, params):
+        if not params or 'nqn' not in params:
+            raise NameError("Parameter 'nqn' missing")
+        if 'host' not in params:
+            raise NameError("Parameter 'host' missing")
+        nqn = params['nqn']
+        try:
+            subsys = nvme.Subsystem(nqn, mode='lookup')
+        except nvme.CFSError:
+            raise NameError("Subsystem %s not found" % nqn)
+        host = nvme.Host(params['host'])
+        try:
+            subsys.add_allowed_host(host.nqn)
+        except nvme.CFSError:
+            raise RuntimeError("Could not add host %s to subsys %s" % (host, nqn))
+
+    def _remove_host(self, params):
+        if not params or 'nqn' not in params:
+            raise NameError("Parameter 'nqn' missing")
+        if 'host' not in params:
+            raise NameError("Parameter 'host' missing")
+        nqn = params['nqn']
+        try:
+            subsys = nvme.Subsystem(nqn, mode='lookup')
+        except nvme.CFSError:
+            raise NameError("Subsystem %s not found" % nqn)
+        try:
+            host = nvme.Host(params['host'], mode='lookup')
+        except nvme.CFSError:
+            raise NameError("Host %s not found" % params['host'])
+        try:
+            subsys.remove_allowed_host(host.nqn)
+        except nvme.CFSError:
+            raise RuntimeError("Failed to remove host %s from subsys %s" %
+                               (nqn,  host.nqn))
+        found = 0
+        for subsys in self.cfg.subsystems:
+            for h in subsys.allowed_hosts:
+                if h.nqn == host.nqn:
+                    found = found + 1
+        if found == 0:
+            host.delete()
+
+    def _add_ana(self, params):
+        if not params or 'portid' not in params:
+            raise NameError("Parameter 'portid' missing")
+        portid = params['portid']
+        if 'grpid' in params:
+            grpid = params['grpid']
+        else:
+            grpid = None
+        try:
+            port = nvme.Port(portid, mode='lookup')
+        except:
+            raise RuntimeError("Port %s not present" % portid)
+        try:
+            a = nvme.ANAGroup(port, grpid)
+        except nvme.CFSError:
+            raise RuntimeError("Port %s ANA Group %s already present" %
+                               (portid, grpid))
+        if 'ana_state' in params:
+            try:
+                a.set_attr("ana", "state", params['ana_state'])
+            except nvme.CFSError:
+                raise RuntimeError("Failed to set ANA state on group %s to %s"
+                                   % (grpid, params['ana_state']))
+        return a.get_attr("ana", "state")
+
+    def _set_ana(self, params):
+        if not params or 'portid' not in params:
+            raise NameError("Parameter 'portid' missing")
+        portid = params['portid']
+        if 'grpid' not in params:
+            raise NameError("Parameter 'grpid' missing")
+        grpid = params['grpid']
+        if 'ana_state' not in params:
+            raise NameError("Parameter 'ana_state' missing")
+        try:
+            port = nvme.Port(portid, mode='lookup')
+        except:
+            raise RuntimeError("Port %s not found" % portid)
+        for ana in port.ana_groups:
+            if ana.grpid == int(grpid):
+                try:
+                    ana.set_attr("ana", "state", params['ana_state'])
+                    return
+                except nvme.CFSError:
+                    raise RuntimeError("Failed to set ANA state to %s" % params['ana_state'])
+        raise RuntimeError("ANA group %s not found" % grpid)
+
+    def _remove_ana(self, params):
+        if not params or 'portid' not in params:
+            raise NameError("Parameter 'portid' missing")
+        if 'grpid' not in params:
+            raise NameError("Parameter 'grpid' missing")
+        grpid = params['grpid']
+        try:
+            port = nvme.Port(params['portid'], mode='lookup')
+        except:
+            raise RuntimeError("Port %s not found" % params['portid'])
+        grpids = [n.grpid for n in port.ana_groups]
+        if int(grpid) not in grpids:
+            raise RuntimeError("ANA group %s not found" % grpid)
+        for ana in port.ana_groups:
+            if ana.grpid == grpid:
+                ana.delete()
+
+    def _get_config(self, params):
+        return self.cfg.dump()
+
+    def _set_config(self, params):
+        if not params:
+            raise RuntimeError("Invalid configuration")
+        try:
+            self.cfg.restore(params, merge=True)
+        except nvme.CFSError:
+            raise RuntimeError("Failed to apply configuration")
+
+    def _get_interfaces(self, params):
+        ifnames = {}
+        for i in netifaces.interfaces():
+            if i == 'lo':
+                continue;
+            iflist = {}
+            ifaddrs = netifaces.ifaddresses(i)
+            try:
+                a = ifaddrs[netifaces.AF_INET]
+            except:
+                pass
+            else:
+                addrlist = []
+                for n in a:
+                    addrlist.append(n['addr'])
+                iflist['ipv4'] = addrlist
+            try:
+                a = ifaddrs[netifaces.AF_INET6]
+            except:
+                pass
+            else:
+                addrlist = []
+                for n in a:
+                    addrlist.append(n['addr'])
+                iflist['ipv6'] = addrlist
+            ifnames[i] = iflist
+        return ifnames
+
+    def _create_file(self, params):
+        if not params or 'file_name' not in params:
+            raise NameError("parameter 'file_name' missing")
+        file_name = params['file_name']
+        if 'size' not in params:
+            raise NameError("parameter 'size' missing")
+        file_size = params['size']
+        if 'pool' in params:
+            file_pool = params['pool']
+        else:
+            file_pool = None
+        bfile = nvme.BackingFile(self.pools, file_name, file_pool,
+                                mode='create')
+        bfile.set_size(int(file_size))
+        return bfile.name
+
+    def _delete_file(self, params):
+        if not params or 'file_name' not in params:
+            raise NameError("parameter 'file_name' missing")
+        file_name = params['file_name']
+        bfile = nvme.BackingFile(self.pools, file_name)
+        bfile.delete()
+
+    def _snapshot_file(self, params):
+        if not params or 'file_name' not in params:
+            raise NameError("parameter 'file_name' missing")
+        file_name = params['file_name']
+        snap = params['snapshot_name']
+        bfile = nvme.BackingFile(self.pools, file_name)
+        return bfile.snapshot(snap)
+
+    def _clone_file(self, params):
+        if not params or 'snapshot_name' not in params:
+            raise NameError("parameter 'snapshot_name' missing")
+        snap = params['snapshot_name']
+        clone = params['clone_name']
+        bfile = nvme.BackingFile(self.pools, snap)
+        return bfile.snapshot(clone)
+
+    def _get_pools(self, params):
+        r = []
+        for p in self.pools:
+            path = Path(self.pools[p])
+            filelist = []
+            for f in path.iterdir():
+                if f.is_file():
+                    filelist.append(f.name)
+            st = os.statvfs(self.pools[p])
+            a = dict(name=p, cluster_size=st.f_frsize,
+                     free_clusters=st.f_bavail,
+                     total_data_clusters=st.f_blocks,
+                     files=filelist)
+            r.append(a)
+        return r
+
+    _rpc_methods = dict(bdev_file_list_pools=_get_pools,
+                        bdev_file_create=_create_file,
+                        bdev_file_delete=_delete_file,
+                        bdev_file_snapshot=_snapshot_file,
+                        bdev_file_clone=_clone_file,
+                        nvmf_get_interfaces=_get_interfaces,
+                        nvmf_create_transport=_create_transport,
+                        nvmf_get_transports=_get_transports,
+                        nvmf_create_subsystem=_create_subsystem,
+                        nvmf_delete_subsystem=_delete_subsystem,
+                        nvmf_subsystem_add_ns=_add_ns,
+                        nvmf_subsystem_remove_ns=_remove_ns,
+                        nvmf_subsystem_add_port=_add_port,
+                        nvmf_subsystem_remove_port=_remove_port,
+                        nvmf_subsystem_add_host=_add_host,
+                        nvmf_subsystem_remove_host=_remove_host,
+                        nvmf_get_subsystems=_get_subsystems,
+                        nvmf_port_add_ana=_add_ana,
+                        nvmf_port_set_ana=_set_ana,
+                        nvmf_port_remove_ana=_remove_ana,
+                        nvmf_get_config=_get_config,
+                        nvmf_set_config=_set_config)
+
+    def __init__(self, pools = None):
+        self.cfg = nvme.Root()
+        self.pools = pools
+
+    def rpc_call(self, req):
+        print ("%s" % req)
+        if 'method' in req:
+            method = req['method']
+        else:
+            method = None
+        if 'params' in req:
+            params = req['params']
+        else:
+            params = None
+        resp = dict(jsonrpc='2.0')
+        if 'id' in req:
+            resp['id'] = req['id']
+        if method not in self._rpc_methods:
+            error = dict(code=-32601,message='Method not found')
+            resp['error'] = error;
+        else:
+            try:
+                result = self._rpc_methods[method](self, params)
+                resp['result'] = result
+            except NameError as n:
+                error = dict(code=-32602, message='Invalid params', data=n.args)
+                resp['error'] = error
+            except RuntimeError as err:
+                error = dict(code=-32000, message=err.args)
+                resp['error'] = error
+            print ("%s" % json.dumps(resp))
+        return json.dumps(resp)
+
diff --git a/nvmetproxy b/nvmetproxy
new file mode 100755
index 0000000..9c644d5
--- /dev/null
+++ b/nvmetproxy
@@ -0,0 +1,197 @@
+#!/usr/bin/python
+
+'''
+Simple json-rpc proxy daemon, based on the SPDK JSON RPC methods
+
+Copyright (c) 2021 by Hannes Reinecke, SUSE Linux LLC
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may
+not use this file except in compliance with the License. You may obtain
+a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations
+under the License.
+'''
+
+from __future__ import print_function
+
+import os
+import sys
+import json
+import socket
+import argparse
+import base64
+import uuid as UUID
+import nvmet as nvme
+try:
+    from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
+except ImportError:
+    from http.server import HTTPServer
+    from http.server import BaseHTTPRequestHandler
+
+nvmet_sock = '/var/tmp/nvmet.sock'
+nvmet_url = '/nvmet'
+
+class SocketHandler:
+    def __init__(self, sockname):
+        self.sockname = sockname
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        try:
+            self.sock.bind(self.sockname)
+        except OSError:
+            print("cannot bind to %s" % self.sockname)
+            return None
+        self.sock.listen(10)
+
+    def __enter__(self):
+        return self
+
+    def rpc_server(self):
+        while True:
+            buf = ''
+            closed = False
+            response = None
+            (client, address) = self.sock.accept()
+            while not closed:
+                newdata = client.recv(1024)
+                if (newdata == b''):
+                    closed = True
+                buf += newdata.decode('ascii')
+                try:
+                    req = json.loads(buf)
+                except ValueError:
+                    continue
+                rpc = nvme.JsonRPC(self.pool)
+                response = rpc.rpc_call(req)
+                client.sendall(response.encode('ascii'))
+                client.close()
+                closed = True
+
+    def rpc_shutdown(self):
+        self.sock.close()
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        os.remove(self.sockname)
+
+class ServerHandler(BaseHTTPRequestHandler):
+    key = ""
+    pool = None
+
+    def do_HEAD(self):
+        self.send_response(200)
+        self.send_header('Content-type', 'application/json')
+        self.end_headers()
+
+    def do_AUTHHEAD(self):
+        self.send_response(401)
+        self.send_header('WWW-Authenticate', 'text/html')
+        self.send_header('Content-type', 'text/html')
+        self.end_headers()
+
+    def do_INTERNALERROR(self):
+        self.send_error(500, message='Internal Server Error',
+                        explain='Failed to parse JSON RPC request')
+
+    def do_NOTFOUND(self):
+        self.send_error(404)
+
+    def do_GET(self):
+        self.send_error(501, message="Unsupported method ('%s')" % self.command)
+
+    def do_PUT(self):
+        self.send_error(501, message="Unsupported method ('%s')" % self.command)
+
+    def do_POST(self):
+        if self.path != self.directory:
+            self.do_NOTFOUND()
+            return
+        if self.key and self.headers['Authorization'] != 'Basic ' + self.key:
+            self.do_AUTHHEAD()
+        else:
+            data_string = self.rfile.read(int(self.headers['Content-Length']))
+            try:
+                req = json.loads(data_string)
+            except ValueError:
+                self.do_INTERNALERROR()
+                return
+            rpc = nvme.JsonRPC(self.pool)
+            response = rpc.rpc_call(req)
+            self.do_HEAD()
+            self.wfile.write(bytes(response.encode(encoding='ascii')))
+
+def main():
+    pool = []
+
+    parser = argparse.ArgumentParser(description='JSON RPC proxy for nvmet')
+    parser.add_argument('-s', '--socket', dest='sock', default=nvmet_sock,
+                        help="Socket to listen on, default is " + nvmet_sock)
+    parser.add_argument('-H', '--host', dest='host', help='Host address')
+    parser.add_argument('-p', '--port', dest='port', type=int, default=4260,
+                        help='Port number')
+    parser.add_argument('-U', '--user', dest='user',
+                        help='user name for authentication')
+    parser.add_argument('-P', '--password', dest='password',
+                        help='password for authentication')
+    parser.add_argument('-c', '--cert', dest='cert',
+                        help='SSL certificate')
+    parser.add_argument('-u', '--url', dest='url', default=nvmet_url,
+                        help="URL path to serve, default is " + nvmet_url)
+    parser.add_argument('-d', '--filepool', dest='filepool', action='append',
+                        help="Directory for backing files")
+    args = parser.parse_args()
+    if args.user and not args.password:
+        sys.exit("No password specified for username %s" % args.user)
+    if args.password and not args.user:
+        sys.exit("No username specified")
+    if args.user:
+        if not args.host:
+            sys.exit("Username and password are only valid for HTTP server")
+        key = base64.b64encode(bytes('%s:%s' % (args.user, args.password), 'utf-8')).decode('ascii')
+        ServerHandler.key = key
+
+    if args.url:
+        ServerHandler.directory = args.url
+
+    if args.filepool:
+        for i in args.filepool:
+            if not os.path.isdir(args.filepool[i]):
+                sys.exit("'%s' is not a directory" % args.filepool[i])
+            pool["pool%d" % i] = args.filepool[i]
+
+    ServerHandler.pool = pool
+    SocketHandler.pool = pool
+
+    if os.geteuid() != 0:
+       print("%s: must run as root." % sys.argv[0], file=sys.stderr)
+       sys.exit(-1)
+
+
+    if args.host:
+        with HTTPServer((args.host, args.port), ServerHandler) as httpd:
+            try:
+                if args.cert is not None:
+                    http.socket = ssl.wrap_socket(httpd.socket,
+                                                  certfile=args.cert,
+                                                  server_side=True)
+                print("Started JSON RPC http proxy server on %s:%d" % (args.host, args.port))
+                httpd.serve_forever()
+            except KeyboardInterrupt:
+                print("Shutting down server")
+                httpd.socket.close()
+    else:
+        with SocketHandler(args.sock) as s:
+            try:
+                print("Started JSON RPC proxy server on %s" % args.sock)
+                s.rpc_server()
+            except KeyboardInterrupt:
+                print('Shutting down server')
+                s.rpc_shutdown()
+
+if __name__ == '__main__':
+    main()
diff --git a/setup.py b/setup.py
index 1956d95..f15e5b0 100755
--- a/setup.py
+++ b/setup.py
@@ -27,5 +27,5 @@ setup(
     maintainer_email = 'hch at lst.de',
     test_suite='nose2.collector.collector',
     packages = ['nvmet'],
-    scripts=['nvmetcli']
+    scripts=['nvmetcli', 'nvmetproxy']
     )
-- 
2.29.2



From cleech at redhat.com  Fri Feb 12 13:06:39 2021
From: cleech at redhat.com (Chris Leech)
Date: Fri, 12 Feb 2021 10:06:39 -0800
Subject: [RFC PATCH v3 00/11] NVMeTCP Offload ULP and QEDN Device Driver
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <20210212180639.GA511742@localhost>

On Sun, Feb 07, 2021 at 08:13:13PM +0200, Shai Malin wrote:
> Queue Initialization:
> =====================
> The nvme-tcp-offload ULP module shall register with the existing 
> nvmf_transport_ops (.name = "tcp_offload"), nvme_ctrl_ops and blk_mq_ops.
> The nvme-tcp-offload vendor driver shall register to nvme-tcp-offload ULP
> with the following ops:
>  - claim_dev() - in order to resolve the route to the target according to
>                  the net_dev.
>  - create_queue() - in order to create offloaded nvme-tcp queue.
> 
> The nvme-tcp-offload ULP module shall manage all the controller level
> functionalities, call claim_dev and based on the return values shall call
> the relevant module create_queue in order to create the admin queue and
> the IO queues.

Hi Shai,

How well does this claim_dev approach work with multipathing?  Is it
expected that providing HOST_TRADDR is sufficient control over which
offload device will be used with multiple valid paths to the controller?

- Chris



From dwagner at suse.de  Fri Feb 12 13:17:38 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 12 Feb 2021 19:17:38 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing it
Message-ID: <20210212181738.79274-1-dwagner@suse.de>

blk_mq_tag_to_rq() will always return a request if the command_id is
in the valid range. Check if the request has been started. If we
blindly process the request we might double complete a request which
can be fatal.

Signed-off-by: Daniel Wagner <dwagner at suse.de>
---

This patch is against nvme-5.12.

There is one blk_mq_tag_to_rq() in nvme_tcp_recv_ddgst() which I
didn't update as I am not sure if it's also needed.

py-crash> bt
#0  0xffffffffa76a33de in arch_atomic_try_cmpxchg (new=<optimized out>, old=<optimized out>, v=<optimized out>)
    at ../arch/x86/include/asm/atomic.h:200
#1  atomic_try_cmpxchg (new=<optimized out>, old=<optimized out>, v=<optimized out>) at ../include/asm-generic/atomic-instrumented.h:695
#2  queued_spin_lock (lock=<optimized out>) at ../include/asm-generic/qspinlock.h:78
#3  do_raw_spin_lock_flags (flags=<optimized out>, lock=<optimized out>) at ../include/linux/spinlock.h:193
#4  __raw_spin_lock_irqsave (lock=<optimized out>) at ../include/linux/spinlock_api_smp.h:119
#5  _raw_spin_lock_irqsave (lock=0x8 <__UNIQUE_ID_license257+8>) at ../kernel/locking/spinlock.c:159
#6  0xffffffffa6eea418 in complete (x=0x0 <__UNIQUE_ID_license257>) at ../kernel/sched/completion.c:32
#7  0xffffffffa721f99c in blk_mq_force_complete_rq (rq=0x8 <__UNIQUE_ID_license257+8>) at ../block/blk-mq.c:634
#8  0xffffffffa721fa0a in blk_mq_complete_request (rq=<optimized out>) at ../block/blk-mq.c:672
#9  0xffffffffc0b092ef in nvme_end_request (result=..., status=<optimized out>, req=<optimized out>) at ../drivers/nvme/host/nvme.h:477
#10 nvme_tcp_process_nvme_cqe (cqe=<optimized out>, queue=<optimized out>) at ../drivers/nvme/host/tcp.c:485
        rq = 0xffff948b840d0000
        hdr = <optimized out>
        ret = 0
        queue = 0xffff949501dd8110
        result = 0
#11 nvme_tcp_handle_comp (pdu=<optimized out>, queue=<optimized out>) at ../drivers/nvme/host/tcp.c:542
#12 nvme_tcp_recv_pdu (len=<optimized out>, offset=<optimized out>, skb=<optimized out>, queue=<optimized out>) at ../drivers/nvme/host/tcp.c:660
#13 nvme_tcp_recv_skb (desc=<optimized out>, skb=<optimized out>, offset=24, len=0) at ../drivers/nvme/host/tcp.c:805
#14 0xffffffffa7598af5 in tcp_read_sock
    (sk=0x8 <__UNIQUE_ID_license257+8>, desc=0xa <__UNIQUE_ID_license257+10>, recv_actor=0x1 <__UNIQUE_ID_license257+1>) at ../net/ipv4/tcp.c:1645
#15 0xffffffffc0b075b8 in nvme_tcp_try_recv (queue=0xffff949501dd8110) at ../drivers/nvme/host/tcp.c:1102
#16 0xffffffffc0b08fc7 in nvme_tcp_io_work (w=0xffff949501dd8118) at ../drivers/nvme/host/tcp.c:1126
#17 0xffffffffa6eba4e4 in process_one_work (worker=0xffff948d1b633ec0, work=0xffff949501dd8118) at ../kernel/workqueue.c:2273
#18 0xffffffffa6eba6fd in worker_thread (__worker=0xffff948d1b633ec0) at ../kernel/workqueue.c:2419
#19 0xffffffffa6ec0a3d in kthread (_create=0xffff948d1b618ec0) at ../kernel/kthread.c:268
#20 0xffffffffa7800215 in ret_from_fork () at ../arch/x86/entry/entry_64.S:351

py-crash>  p /x ((struct request*)0xffff948b840d0000)->state
$2 = 0x2

 drivers/nvme/host/tcp.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 69f59d2c5799..4bec705ce8e6 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -485,7 +485,7 @@ static int nvme_tcp_process_nvme_cqe(struct nvme_tcp_queue *queue,
 	struct request *rq;
 
 	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), cqe->command_id);
-	if (!rq) {
+	if (!rq || !blk_mq_request_started(rq)) {
 		dev_err(queue->ctrl->ctrl.device,
 			"queue %d tag 0x%x not found\n",
 			nvme_tcp_queue_id(queue), cqe->command_id);
@@ -506,7 +506,7 @@ static int nvme_tcp_handle_c2h_data(struct nvme_tcp_queue *queue,
 	struct request *rq;
 
 	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
-	if (!rq) {
+	if (!rq || !blk_mq_request_started(rq)) {
 		dev_err(queue->ctrl->ctrl.device,
 			"queue %d tag %#x not found\n",
 			nvme_tcp_queue_id(queue), pdu->command_id);
@@ -610,7 +610,7 @@ static int nvme_tcp_handle_r2t(struct nvme_tcp_queue *queue,
 	int ret;
 
 	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
-	if (!rq) {
+	if (!rq || !blk_mq_request_started(rq)) {
 		dev_err(queue->ctrl->ctrl.device,
 			"queue %d tag %#x not found\n",
 			nvme_tcp_queue_id(queue), pdu->command_id);
@@ -696,7 +696,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
 	struct request *rq;
 
 	rq = blk_mq_tag_to_rq(nvme_tcp_tagset(queue), pdu->command_id);
-	if (!rq) {
+	if (!rq || !blk_mq_request_started(rq)) {
 		dev_err(queue->ctrl->ctrl.device,
 			"queue %d tag %#x not found\n",
 			nvme_tcp_queue_id(queue), pdu->command_id);
-- 
2.29.2



From sagi at grimberg.me  Fri Feb 12 15:58:27 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 12 Feb 2021 12:58:27 -0800
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <20210212181738.79274-1-dwagner@suse.de>
References: <20210212181738.79274-1-dwagner@suse.de>
Message-ID: <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>

> blk_mq_tag_to_rq() will always return a request if the command_id is
> in the valid range. Check if the request has been started. If we
> blindly process the request we might double complete a request which
> can be fatal.

How did you get to this one? did the controller send a completion for
a completed/bogus request?


From kbusch at kernel.org  Fri Feb 12 16:09:29 2021
From: kbusch at kernel.org (Keith Busch)
Date: Sat, 13 Feb 2021 06:09:29 +0900
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
Message-ID: <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>

On Fri, Feb 12, 2021 at 12:58:27PM -0800, Sagi Grimberg wrote:
> > blk_mq_tag_to_rq() will always return a request if the command_id is
> > in the valid range. Check if the request has been started. If we
> > blindly process the request we might double complete a request which
> > can be fatal.
> 
> How did you get to this one? did the controller send a completion for
> a completed/bogus request?

If that is the case, then that must mean it's possible the driver could
have started the command id just before the bogus completion check. Data
iorruption, right?


From sagi at grimberg.me  Fri Feb 12 16:49:32 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 12 Feb 2021 13:49:32 -0800
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>


>>> blk_mq_tag_to_rq() will always return a request if the command_id is
>>> in the valid range. Check if the request has been started. If we
>>> blindly process the request we might double complete a request which
>>> can be fatal.
>>
>> How did you get to this one? did the controller send a completion for
>> a completed/bogus request?
> 
> If that is the case, then that must mean it's possible the driver could
> have started the command id just before the bogus completion check. Data
> iorruption, right?

Yes, which is why I don't think this check is very useful..


From sagi at grimberg.me  Fri Feb 12 17:22:14 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Fri, 12 Feb 2021 14:22:14 -0800
Subject: [PATCH] nvmet-tcp: fix potential race of tcp socket closing
 accept_work
In-Reply-To: <DM6PR19MB4011C3BCABE1663D864ECB67EF8B9@DM6PR19MB4011.namprd19.prod.outlook.com>
References: <20210205194725.625035-1-sagi@grimberg.me>
 <DM6PR19MB4011C3BCABE1663D864ECB67EF8B9@DM6PR19MB4011.namprd19.prod.outlook.com>
Message-ID: <d26978a7-b9a4-8925-1a34-c5e194eb342b@grimberg.me>


> Sagi,
> 
> I think there is another race in that solution.
> 
> Please consider following flow:
> 
> Thread 1 taking sk_callback_lock and goes into the else block (state is TCP_ESTABLISHED), going to line:
> 	sock->sk->sk_user_data = queue;
> 
> at that point, thread 2 invokes sk_state_change, but this is still the original state_change callback because we didn't set yet to nvmet_tcp_state_change.
> 
> Thread 1 continues and set the callback to nvmet_tcp_state_change, but the sk_state_change was already called and nvmet_tcp_state_change will be never called.
> 
> In that case, there is a leak, and the queue will never be removed.

Are you saying that this is needed?
--
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index ac2d9ed23cea..d82df6cca801 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1486,6 +1486,7 @@ static int nvmet_tcp_set_queue_sock(struct 
nvmet_tcp_queue *queue)

         ret = 0;
         write_lock_bh(&sock->sk->sk_callback_lock);
+       lock_sock(sock->sk);
         if (sock->sk->sk_state != TCP_ESTABLISHED) {
                 /*
                  * If the socket is already closing, don't even start
@@ -1502,6 +1503,7 @@ static int nvmet_tcp_set_queue_sock(struct 
nvmet_tcp_queue *queue)
                 sock->sk->sk_write_space = nvmet_tcp_write_space;
                 queue_work_on(queue_cpu(queue), nvmet_tcp_wq, 
&queue->io_work);
         }
+       release_sock(sock->sk);
         write_unlock_bh(&sock->sk->sk_callback_lock);

         return ret;
--


From hare at suse.de  Sat Feb 13 03:42:46 2021
From: hare at suse.de (Hannes Reinecke)
Date: Sat, 13 Feb 2021 09:42:46 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <20210212181738.79274-1-dwagner@suse.de>
References: <20210212181738.79274-1-dwagner@suse.de>
Message-ID: <3f183393-cc1e-3d03-3074-40c18f3cb9cc@suse.de>

On 2/12/21 7:17 PM, Daniel Wagner wrote:
> blk_mq_tag_to_rq() will always return a request if the command_id is
> in the valid range. Check if the request has been started. If we
> blindly process the request we might double complete a request which
> can be fatal.
> 
> Signed-off-by: Daniel Wagner <dwagner at suse.de>
> ---
> 
> This patch is against nvme-5.12.
> 
> There is one blk_mq_tag_to_rq() in nvme_tcp_recv_ddgst() which I
> didn't update as I am not sure if it's also needed.
> 
I guess it is; this patch is essentially a protection against invalid 
frames, and as such affects all places.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Sat Feb 13 03:46:41 2021
From: hare at suse.de (Hannes Reinecke)
Date: Sat, 13 Feb 2021 09:46:41 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
Message-ID: <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>

On 2/12/21 10:49 PM, Sagi Grimberg wrote:
> 
>>>> blk_mq_tag_to_rq() will always return a request if the command_id is
>>>> in the valid range. Check if the request has been started. If we
>>>> blindly process the request we might double complete a request which
>>>> can be fatal.
>>>
>>> How did you get to this one? did the controller send a completion for
>>> a completed/bogus request?
>>
>> If that is the case, then that must mean it's possible the driver could
>> have started the command id just before the bogus completion check. Data
>> iorruption, right?
> 
> Yes, which is why I don't think this check is very useful..

I actually view that as a valid protection against spoofed frames.
Without it it's easy to crash the machine by injecting fake completions 
with random command ids.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From malin1024 at gmail.com  Sat Feb 13 11:47:02 2021
From: malin1024 at gmail.com (Shai Malin)
Date: Sat, 13 Feb 2021 18:47:02 +0200
Subject: [RFC PATCH v3 00/11] NVMeTCP Offload ULP and QEDN Device Driver
In-Reply-To: <20210212180639.GA511742@localhost>
References: <20210207181324.11429-1-smalin@marvell.com>
 <20210212180639.GA511742@localhost>
Message-ID: <CAKKgK4yof3MsvUFAB-yjSBKtf-UhrTGb7vghOUv=ttYG29v1OQ@mail.gmail.com>

On Fri, 12 Feb 2021 at 20:06, Chris Leech wrote:
>
> On Sun, Feb 07, 2021 at 08:13:13PM +0200, Shai Malin wrote:
> > Queue Initialization:
> > =====================
> > The nvme-tcp-offload ULP module shall register with the existing
> > nvmf_transport_ops (.name = "tcp_offload"), nvme_ctrl_ops and blk_mq_ops.
> > The nvme-tcp-offload vendor driver shall register to nvme-tcp-offload ULP
> > with the following ops:
> >  - claim_dev() - in order to resolve the route to the target according to
> >                  the net_dev.
> >  - create_queue() - in order to create offloaded nvme-tcp queue.
> >
> > The nvme-tcp-offload ULP module shall manage all the controller level
> > functionalities, call claim_dev and based on the return values shall call
> > the relevant module create_queue in order to create the admin queue and
> > the IO queues.
>
> Hi Shai,
>
> How well does this claim_dev approach work with multipathing?  Is it
> expected that providing HOST_TRADDR is sufficient control over which
> offload device will be used with multiple valid paths to the controller?
>
> - Chris
>

Hi Chris,

The nvme-tcp-offload multipath behaves the same as the non-offloaded
nvme-tcp. The HOST_TRADDR is sufficient to control which offload device
will be used with multiple valid paths.

- Shai


From gerlitz.or at gmail.com  Sun Feb 14 02:30:50 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Sun, 14 Feb 2021 09:30:50 +0200
Subject: [PATCH v4 net-next 21/21] Documentation: add TCP DDP offload
 documentation
In-Reply-To: <a18aaeec-049e-72c1-f981-3e18381b0f49@nvidia.com>
References: <20210212052031.18123-1-borisp@mellanox.com>
 <a18aaeec-049e-72c1-f981-3e18381b0f49@nvidia.com>
Message-ID: <CAJ3xEMi0574vgmHMt8LCe36RhH0XJg=LoTU2Jdx+W7+FWMmkBQ@mail.gmail.com>

On Fri, Feb 12, 2021 at 4:11 PM Nikolay Aleksandrov <nikolay at nvidia.com> wrote:

> I got interested and read through the doc, there are a few typos below.

thanks for spotting these, we will fix them


From minwoo.im.dev at gmail.com  Sun Feb 14 06:01:24 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sun, 14 Feb 2021 20:01:24 +0900
Subject: [PATCH 0/2] nvme: support I/O passthrough for multipath
Message-ID: <20210214110126.24023-1-minwoo.im.dev@gmail.com>

Hello,

This series is to support I/O passthrough for multipath I/O.

Currently driver exposes block device for the head namespace.  Also, the
path will be decided by the driver with the I/O policy defined to the
actual hidden device based on ANA state.  This means that userspace is not
allowed to I/O passthrough to a specified namespace target which is hidden
by the driver.

This series is to support passthrough to the namespace attached to a
controller regardless to ANA state (e.g., inaccessible) and decided path
by the driver.

Please review.

Thanks,


From minwoo.im.dev at gmail.com  Sun Feb 14 06:01:25 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sun, 14 Feb 2021 20:01:25 +0900
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
Message-ID: <20210214110126.24023-2-minwoo.im.dev@gmail.com>

We don't allow NVME_IOCTL_IO_CMD ioctl in case that a controller has
multiple namespaces attached.  Also, I/O request to the controller
character device has not been recommended and deprecated because we have
block device to I/O with where the multipath consideration is taken.

Once kernel decided a path to I/O for a namespace based on the I/O
policy of a NVMe subsystem, userspace is not allowed to choose a path to
I/O.  If a path is broken(inaccessible state in ANA), then it will not
try to I/O to that path.

This patch introduced NVME_IOCTL_MPATH_IO command for controller
device(e.g., /dev/nvme0) to support multipath I/O passthrough for
userspace.  Regardless driver's path decision, userspace can target a
namespace to I/O.  In this case, `cmd.nsid` will be used to find out the
namespace instance target which is hidden(e.g., nvmeXcYnZ).

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 drivers/nvme/host/core.c        | 31 +++++++++++++++++++++++++++++++
 include/uapi/linux/nvme_ioctl.h |  1 +
 2 files changed, 32 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..fed9fc41b021 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1707,6 +1707,35 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 	return status;
 }
 
+#ifdef CONFIG_NVME_MULTIPATH
+static int nvme_dev_mpath_cmd(struct nvme_ctrl *ctrl,
+		struct nvme_passthru_cmd __user *ucmd)
+{
+	struct nvme_passthru_cmd cmd;
+	struct nvme_ns *ns;
+
+	if (!capable(CAP_SYS_ADMIN))
+		return -EACCES;
+	if (copy_from_user(&cmd, ucmd, sizeof(cmd)))
+		return -EFAULT;
+
+	ns = nvme_find_get_ns(ctrl, cmd.nsid);
+	if (unlikely(!ns))
+		return -EINVAL;
+	if (WARN_ON_ONCE(!(ns->disk->flags & GENHD_FL_HIDDEN)))
+		return -EINVAL;
+
+	return nvme_user_cmd(ctrl, ns, ucmd);
+}
+#else
+static int nvme_dev_mpath_cmd(struct nvme_ctrl *ctrl,
+		struct nvme_passthru_cmd __user *ucmd)
+{
+	dev_warn(ctrl->device, "CONFIG_NVME_MULTIPATH should be enabled\n");
+	return -EINVAL;
+}
+#endif
+
 /*
  * Issue ioctl requests on the first available path.  Note that unlike normal
  * block layer requests we will not retry failed request on another controller.
@@ -3329,6 +3358,8 @@ static long nvme_dev_ioctl(struct file *file, unsigned int cmd,
 		return nvme_user_cmd64(ctrl, NULL, argp);
 	case NVME_IOCTL_IO_CMD:
 		return nvme_dev_user_cmd(ctrl, argp);
+	case NVME_IOCTL_MPATH_IO:
+		return nvme_dev_mpath_cmd(ctrl, argp);
 	case NVME_IOCTL_RESET:
 		dev_warn(ctrl->device, "resetting controller\n");
 		return nvme_reset_ctrl_sync(ctrl);
diff --git a/include/uapi/linux/nvme_ioctl.h b/include/uapi/linux/nvme_ioctl.h
index d99b5a772698..38bd330e5416 100644
--- a/include/uapi/linux/nvme_ioctl.h
+++ b/include/uapi/linux/nvme_ioctl.h
@@ -78,5 +78,6 @@ struct nvme_passthru_cmd64 {
 #define NVME_IOCTL_RESCAN	_IO('N', 0x46)
 #define NVME_IOCTL_ADMIN64_CMD	_IOWR('N', 0x47, struct nvme_passthru_cmd64)
 #define NVME_IOCTL_IO64_CMD	_IOWR('N', 0x48, struct nvme_passthru_cmd64)
+#define NVME_IOCTL_MPATH_IO	_IOWR('N', 0x49, struct nvme_passthru_cmd)
 
 #endif /* _UAPI_LINUX_NVME_IOCTL_H */
-- 
2.17.1



From minwoo.im.dev at gmail.com  Sun Feb 14 06:01:26 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sun, 14 Feb 2021 20:01:26 +0900
Subject: [nvme-cli PATCH] nvme: add [--mpath|-M] option to io-passthru
In-Reply-To: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
Message-ID: <20210214110126.24023-3-minwoo.im.dev@gmail.com>

`io-passthru` command is to submit a I/O command to controller with a
namespace specified.  Kernel driver does not allow userspace to I/O
passthrough in case that controller has multiple namespaces attached.
NVME_IOCTL_IO_CMD will return -EINVAL error from the kernel driver.

This patch added an option `--mpath|-M` option to the `io-passthru`
command to support specifying device namespace id to I/O.  This option
will make `--namespace-id` option to be used to find out the namespae
instance to the kernel side even it's a hidden blkdev (e.g.,
/dev/nvmeXcYnZ).

This feature can be used to debug or test the broken path for ANA case
by specifying the target namespace of a controller to I/O.

  nvme io-passthru /dev/nvme0 --namespace-id=123 --opcode=0x2 --data-len=512 --read --mpath

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 linux/nvme_ioctl.h |  1 +
 nvme.c             | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/linux/nvme_ioctl.h b/linux/nvme_ioctl.h
index d569414211d1..578faaee62b6 100644
--- a/linux/nvme_ioctl.h
+++ b/linux/nvme_ioctl.h
@@ -87,5 +87,6 @@ struct nvme_passthru_cmd64 {
 #define NVME_IOCTL_RESCAN	_IO('N', 0x46)
 #define NVME_IOCTL_ADMIN64_CMD  _IOWR('N', 0x47, struct nvme_passthru_cmd64)
 #define NVME_IOCTL_IO64_CMD _IOWR('N', 0x48, struct nvme_passthru_cmd64)
+#define NVME_IOCTL_MPATH_IO	_IOWR('N', 0x49, struct nvme_passthru_cmd)
 
 #endif /* _UAPI_LINUX_NVME_IOCTL_H */
diff --git a/nvme.c b/nvme.c
index c79edbda17ed..9dea66edcabe 100644
--- a/nvme.c
+++ b/nvme.c
@@ -5109,6 +5109,7 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 		int   read;
 		int   write;
 		__u8  prefill;
+		bool  mpath;
 	};
 
 	struct config cfg = {
@@ -5129,6 +5130,7 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 		.cdw15        = 0,
 		.input_file   = "",
 		.prefill      = 0,
+		.mpath        = false,
 	};
 
 	const char *opcode = "opcode (required)";
@@ -5153,6 +5155,7 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 	const char *re = "set dataflow direction to receive";
 	const char *wr = "set dataflow direction to send";
 	const char *prefill = "prefill buffers with known byte-value, default 0";
+	const char *mpath = "multipath I/O (valid only if io-passthru)";
 
 	OPT_ARGS(opts) = {
 		OPT_BYTE("opcode",       'o', &cfg.opcode,       opcode),
@@ -5177,6 +5180,7 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 		OPT_FLAG("dry-run",      'd', &cfg.dry_run,      dry),
 		OPT_FLAG("read",         'r', &cfg.read,         re),
 		OPT_FLAG("write",        'w', &cfg.write,        wr),
+		OPT_FLAG("mpath",        'M', &cfg.mpath,        mpath),
 		OPT_END()
 	};
 
@@ -5184,6 +5188,22 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 	if (fd < 0)
 		goto ret;
 
+	if (cfg.mpath) {
+		if (!cfg.namespace_id) {
+			fprintf(stderr, "mpath should be with --namespace-id\n");
+			err = -EINVAL;
+			goto close_fd;
+		}
+
+		if (!is_chardev()) {
+			fprintf(stderr, "mpath should be with chardev (e.g., /dev/nvme0)\n");
+			err = -EINVAL;
+			goto close_fd;
+		}
+
+		ioctl_cmd = NVME_IOCTL_MPATH_IO;
+	}
+
 	if (strlen(cfg.input_file)){
 		wfd = open(cfg.input_file, O_RDONLY,
 			   S_IRUSR | S_IRGRP | S_IROTH);
-- 
2.17.1



From hkallweit1 at gmail.com  Sun Feb 14 11:24:55 2021
From: hkallweit1 at gmail.com (Heiner Kallweit)
Date: Sun, 14 Feb 2021 17:24:55 +0100
Subject: [PATCH] nvme: use lighter smp barriers in nvme_irq
Message-ID: <1c90fd6a-d991-b347-479c-2c8ae5a4c51f@gmail.com>

Based on the description we should be fine using the less heavy-weight
smp barriers here. On x86 this would be compiler barriers only.

Signed-off-by: Heiner Kallweit <hkallweit1 at gmail.com>
---
 drivers/nvme/host/pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 7b6632c00..59fee4a60 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1066,10 +1066,10 @@ static irqreturn_t nvme_irq(int irq, void *data)
 	 * The rmb/wmb pair ensures we see all updates from a previous run of
 	 * the irq handler, even if that was on another CPU.
 	 */
-	rmb();
+	smp_rmb();
 	if (nvme_process_cq(nvmeq))
 		ret = IRQ_HANDLED;
-	wmb();
+	smp_wmb();
 
 	return ret;
 }
-- 
2.30.1



From dsahern at gmail.com  Sun Feb 14 13:16:30 2021
From: dsahern at gmail.com (David Ahern)
Date: Sun, 14 Feb 2021 11:16:30 -0700
Subject: [PATCH v4 net-next 06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <20210211211044.32701-7-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-7-borisp@mellanox.com>
Message-ID: <2dd10b2f-df00-e21c-7886-93f41a987040@gmail.com>

On 2/11/21 2:10 PM, Boris Pismenny wrote:
> @@ -223,6 +229,164 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>  	return nvme_tcp_pdu_data_left(req) <= len;
>  }
>  
> +#ifdef CONFIG_TCP_DDP
> +
> +static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
> +static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
> +	.resync_request		= nvme_tcp_resync_request,
> +};
> +
> +static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
> +{
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	struct nvme_tcp_ddp_config config = {};
> +	int ret;
> +
> +	if (!(netdev->features & NETIF_F_HW_TCP_DDP))

If nvme_tcp_offload_limits does not find a dst_entry on the socket then
offloading_netdev may not NULL at this point.

> +		return -EOPNOTSUPP;
> +
> +	config.cfg.type		= TCP_DDP_NVME;
> +	config.pfv		= NVME_TCP_PFV_1_0;
> +	config.cpda		= 0;
> +	config.dgst		= queue->hdr_digest ?
> +		NVME_TCP_HDR_DIGEST_ENABLE : 0;
> +	config.dgst		|= queue->data_digest ?
> +		NVME_TCP_DATA_DIGEST_ENABLE : 0;
> +	config.queue_size	= queue->queue_size;
> +	config.queue_id		= nvme_tcp_queue_id(queue);
> +	config.io_cpu		= queue->io_cpu;
> +
> +	dev_hold(netdev); /* put by unoffload_socket */
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_sk_add(netdev,
> +						  queue->sock->sk,
> +						  &config.cfg);
> +	if (ret) {
> +		dev_put(netdev);
> +		return ret;
> +	}
> +
> +	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = &nvme_tcp_ddp_ulp_ops;
> +	if (netdev->features & NETIF_F_HW_TCP_DDP)
> +		set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
> +
> +	return ret;
> +}
> +
> +static void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
> +{
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +
> +	if (!netdev) {
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");

you are already logged in nvme_tcp_offload_limits that
get_netdev_for_sock returned NULL; no need to do it again.

> +		return;
> +	}
> +
> +	netdev->tcp_ddp_ops->tcp_ddp_sk_del(netdev, queue->sock->sk);
> +
> +	inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = NULL;
> +	dev_put(netdev); /* held by offload_socket */
> +}
> +
> +static int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
> +{
> +	struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
> +	struct tcp_ddp_limits limits;
> +	int ret = 0;
> +
> +	if (!netdev) {
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");

This should be more informative.

> +		queue->ctrl->offloading_netdev = NULL;
> +		return -ENODEV;
> +	}
> +
> +	if (netdev->features & NETIF_F_HW_TCP_DDP &&
> +	    netdev->tcp_ddp_ops &&
> +	    netdev->tcp_ddp_ops->tcp_ddp_limits)
> +		ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
> +	else
> +		ret = -EOPNOTSUPP;
> +
> +	if (!ret) {
> +		queue->ctrl->offloading_netdev = netdev;

you save a reference to the netdev here, but then release the refcnt
below. That device could be deleted between this point in time and the
initialization of all queues.


> +		dev_dbg_ratelimited(queue->ctrl->ctrl.device,
> +				    "netdev %s offload limits: max_ddp_sgl_len %d\n",
> +				    netdev->name, limits.max_ddp_sgl_len);
> +		queue->ctrl->ctrl.max_segments = limits.max_ddp_sgl_len;
> +		queue->ctrl->ctrl.max_hw_sectors =
> +			limits.max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
> +	} else {
> +		queue->ctrl->offloading_netdev = NULL;
> +	}
> +
> +	/* release the device as no offload context is established yet. */
> +	dev_put(netdev);
> +
> +	return ret;
> +}
> +
> +static void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
> +				     struct sk_buff *skb, unsigned int offset)
> +{
> +	u64 pdu_seq = TCP_SKB_CB(skb)->seq + offset - queue->pdu_offset;
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	u64 resync_val;
> +	u32 resync_seq;
> +
> +	resync_val = atomic64_read(&queue->resync_req);
> +	/* Lower 32 bit flags. Check validity of the request */
> +	if ((resync_val & TCP_DDP_RESYNC_REQ) == 0)
> +		return;
> +
> +	/* Obtain and check requested sequence number: is this PDU header before the request? */
> +	resync_seq = resync_val >> 32;
> +	if (before(pdu_seq, resync_seq))
> +		return;
> +
> +	if (unlikely(!netdev)) {
> +		pr_info_ratelimited("%s: netdev not found\n", __func__);

can't happen right? you get here because NVME_TCP_Q_OFF_DDP is set and
it is only set if offloading_netdev is set and the device supports offload.

> +		return;
> +	}
> +
> +	/**
> +	 * The atomic operation gurarantees that we don't miss any NIC driver
> +	 * resync requests submitted after the above checks.
> +	 */
> +	if (atomic64_cmpxchg(&queue->resync_req, resync_val,
> +			     resync_val & ~TCP_DDP_RESYNC_REQ))
> +		netdev->tcp_ddp_ops->tcp_ddp_resync(netdev, queue->sock->sk, pdu_seq);
> +}
> +



From dsahern at gmail.com  Sun Feb 14 13:27:15 2021
From: dsahern at gmail.com (David Ahern)
Date: Sun, 14 Feb 2021 11:27:15 -0700
Subject: [PATCH v4 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <20210211211044.32701-8-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-8-borisp@mellanox.com>
Message-ID: <cfd61c5a-c5fd-e0d9-fb60-be93f1c98735@gmail.com>

On 2/11/21 2:10 PM, Boris Pismenny wrote:
>  
> +static int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> +				 u16 command_id,
> +				 struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	int ret;
> +
> +	if (unlikely(!netdev)) {
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");

again, unnecessary. you only get here because the rquest is marked
offloaded and that only happens if the netdev exists and supports DDP.

> +		return -EINVAL;
> +	}
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_teardown(netdev, queue->sock->sk,
> +						    &req->ddp, rq);
> +	sg_free_table_chained(&req->ddp.sg_table, SG_CHUNK_SIZE);
> +	return ret;
> +}
> +
> +static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> +{
> +	struct request *rq = ddp_ctx;
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +
> +	if (!nvme_try_complete_req(rq, req->status, req->result))
> +		nvme_complete_rq(rq);
> +}
> +
> +static int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> +			      u16 command_id,
> +			      struct request *rq)
> +{
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct net_device *netdev = queue->ctrl->offloading_netdev;
> +	int ret;
> +
> +	if (unlikely(!netdev)) {
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");

similarly here. you can't get here if netdev is null.

> +		return -EINVAL;
> +	}
> +
> +	req->ddp.command_id = command_id;
> +	ret = nvme_tcp_req_map_sg(req, rq);
> +	if (ret)
> +		return -ENOMEM;
> +
> +	ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
> +						 queue->sock->sk,
> +						 &req->ddp);
> +	if (!ret)
> +		req->offloaded = true;
> +	return ret;
> +}
> +
>  static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
>  {
>  	struct net_device *netdev = queue->ctrl->offloading_netdev;
> @@ -343,7 +417,7 @@ static void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
>  		return;
>  
>  	if (unlikely(!netdev)) {
> -		pr_info_ratelimited("%s: netdev not found\n", __func__);
> +		dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");

and per comment on the last patch, this is not needed.

> @@ -849,10 +953,39 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>  
>  static inline void nvme_tcp_end_request(struct request *rq, u16 status)
>  {
> +	struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> +	struct nvme_tcp_queue *queue = req->queue;
> +	struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
>  	union nvme_result res = {};
>  
> -	if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> -		nvme_complete_rq(rq);
> +	nvme_tcp_complete_request(rq, cpu_to_le16(status << 1), res, pdu->command_id);
> +}
> +
> +
> +static int nvme_tcp_consume_skb(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> +				unsigned int *offset, struct iov_iter *iter, int recv_len)
> +{
> +	int ret;
> +
> +#ifdef CONFIG_TCP_DDP
> +	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
> +		if (queue->data_digest)
> +			ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset, iter, recv_len,
> +					queue->rcv_hash);
> +		else
> +			ret = skb_ddp_copy_datagram_iter(skb, *offset, iter, recv_len);
> +	} else {
> +#endif

why not make that a helper defined in the CONFIG_TCP_DDP section with an
inline for the unset case. Keeps this code from being polluted with the
ifdef checks.

> +		if (queue->data_digest)
> +			ret = skb_copy_and_hash_datagram_iter(skb, *offset, iter, recv_len,
> +					queue->rcv_hash);
> +		else
> +			ret = skb_copy_datagram_iter(skb, *offset, iter, recv_len);
> +#ifdef CONFIG_TCP_DDP
> +	}
> +#endif
> +
> +	return ret;
>  }
>  
>  static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> @@ -899,12 +1032,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
>  		recv_len = min_t(size_t, recv_len,
>  				iov_iter_count(&req->iter));
>  
> -		if (queue->data_digest)
> -			ret = skb_copy_and_hash_datagram_iter(skb, *offset,
> -				&req->iter, recv_len, queue->rcv_hash);
> -		else
> -			ret = skb_copy_datagram_iter(skb, *offset,
> -					&req->iter, recv_len);
> +		ret = nvme_tcp_consume_skb(queue, skb, offset, &req->iter, recv_len);
>  		if (ret) {
>  			dev_err(queue->ctrl->ctrl.device,
>  				"queue %d failed to copy request %#x data",
> @@ -1128,6 +1256,7 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>  	bool inline_data = nvme_tcp_has_inline_data(req);
>  	u8 hdgst = nvme_tcp_hdgst_len(queue);
>  	int len = sizeof(*pdu) + hdgst - req->offset;
> +	struct request *rq = blk_mq_rq_from_pdu(req);
>  	int flags = MSG_DONTWAIT;
>  	int ret;
>  
> @@ -1136,6 +1265,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>  	else
>  		flags |= MSG_EOR;
>  
> +	if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) &&
> +	    blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
> +		nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);
> +

For consistency, shouldn't this be wrapped in the CONFIG_TCP_DDP check too?

>  	if (queue->hdr_digest && !req->offset)
>  		nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu));
>  


From Chaitanya.Kulkarni at wdc.com  Sun Feb 14 20:33:06 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Mon, 15 Feb 2021 01:33:06 +0000
Subject: [PATCH 2/2] nvme: resync header file with common nvme-cli tool
References: <20210121090948.8027-1-mgurtovoy@nvidia.com>
 <20210121090948.8027-4-mgurtovoy@nvidia.com> <20210127174717.GF28180@lst.de>
 <BYAPR12MB349445BDAFCBD9C9FBC32701DE8E9@BYAPR12MB3494.namprd12.prod.outlook.com>
 <20210209153503.GA8607@lst.de>
 <BYAPR12MB34947F0C352E4035BAC258CFDE8C9@BYAPR12MB3494.namprd12.prod.outlook.com>
Message-ID: <BYAPR04MB4965D9C2B54B8B01FEF26FE286889@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/11/21 05:02, Max Gurtovoy wrote:
> see attached.
>
> Btw, the nvme-5.12 is not so stable.
> I didn't debug it but I couldn't establish rdma/tcp/loop connections.
> So the attached was just compiled and not tested.
>
> -----Original Message-----
> From: Christoph Hellwig <hch at lst.de> 
> Sent: Tuesday, February 9, 2021 5:35 PM
> To: Max Gurtovoy <mgurtovoy at nvidia.com>
> Cc: Christoph Hellwig <hch at lst.de>; linux-nvme at lists.infradead.org; sagi at grimberg.me; kbusch at kernel.org; chaitanya.kulkarni at wdc.com
> Subject: Re: [PATCH 2/2] nvme: resync header file with common nvme-cli tool
>
> On Tue, Feb 09, 2021 at 03:07:05PM +0000, Max Gurtovoy wrote:
>> is this respin still needed ?
> Yes.
>
FYI, I sent out the series with Max's attached patch as I needed that to
rebase some of my work, please have a look.


From gerlitz.or at gmail.com  Mon Feb 15 02:57:20 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Mon, 15 Feb 2021 09:57:20 +0200
Subject: [PATCH v4 net-next 06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <2dd10b2f-df00-e21c-7886-93f41a987040@gmail.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-7-borisp@mellanox.com>
 <2dd10b2f-df00-e21c-7886-93f41a987040@gmail.com>
Message-ID: <CAJ3xEMg3GWNVjkixVVop9uhn958Opdq4ej0qktA11NS6GM9s-g@mail.gmail.com>

On Sun, Feb 14, 2021 at 8:20 PM David Ahern <dsahern at gmail.com> wrote:
> On 2/11/21 2:10 PM, Boris Pismenny wrote:
> > @@ -223,6 +229,164 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
> >       return nvme_tcp_pdu_data_left(req) <= len;
> >  }

Hi Dave,

Thanks for the continuous feedback. Folks are out this week and it seems
for that few comments we will need to discuss internally, but anyway I will
address at least  some of the comments later today/tomorrow.

Or.

Or.


From dwagner at suse.de  Mon Feb 15 05:40:20 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Mon, 15 Feb 2021 11:40:20 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
Message-ID: <20210215104020.yyithlo2hkxqvguj@beryllium.lan>

On Sat, Feb 13, 2021 at 09:46:41AM +0100, Hannes Reinecke wrote:
> On 2/12/21 10:49 PM, Sagi Grimberg wrote:
> > 
> > > > > blk_mq_tag_to_rq() will always return a request if the command_id is
> > > > > in the valid range. Check if the request has been started. If we
> > > > > blindly process the request we might double complete a request which
> > > > > can be fatal.
> > > > 
> > > > How did you get to this one? did the controller send a completion for
> > > > a completed/bogus request?
> > > 
> > > If that is the case, then that must mean it's possible the driver could
> > > have started the command id just before the bogus completion check. Data
> > > iorruption, right?

'during TCP LIF toggles and aggr relocates' testing the host
crashes. TBH, I do not really know what is happening or what the test
does. Still trying to figure out what's going on. I was just very
surprised how much the code trusts the other side to behave correctly.

> > Yes, which is why I don't think this check is very useful..
> 
> I actually view that as a valid protection against spoofed frames.
> Without it it's easy to crash the machine by injecting fake completions with
> random command ids.

In this test scenario it's not even a spoofed frame; maybe just a
confused controller.


From Elad.Grupi at dell.com  Mon Feb 15 08:00:34 2021
From: Elad.Grupi at dell.com (Grupi, Elad)
Date: Mon, 15 Feb 2021 13:00:34 +0000
Subject: [PATCH] nvmet-tcp: fix potential race of tcp socket closing
 accept_work
In-Reply-To: <d26978a7-b9a4-8925-1a34-c5e194eb342b@grimberg.me>
References: <20210205194725.625035-1-sagi@grimberg.me>
 <DM6PR19MB4011C3BCABE1663D864ECB67EF8B9@DM6PR19MB4011.namprd19.prod.outlook.com>
 <d26978a7-b9a4-8925-1a34-c5e194eb342b@grimberg.me>
Message-ID: <DM6PR19MB40118CA60B63407DDE5AEB33EF889@DM6PR19MB4011.namprd19.prod.outlook.com>

Yes. That's about what I meant.

Does lock_sock protects against invoking socket callbacks?

-----Original Message-----
From: Sagi Grimberg <sagi at grimberg.me> 
Sent: Saturday, 13 February 2021 0:22
To: Grupi, Elad; linux-nvme at lists.infradead.org; Christoph Hellwig; Keith Busch; Chaitanya Kulkarni
Subject: Re: [PATCH] nvmet-tcp: fix potential race of tcp socket closing accept_work


[EXTERNAL EMAIL] 


> Sagi,
> 
> I think there is another race in that solution.
> 
> Please consider following flow:
> 
> Thread 1 taking sk_callback_lock and goes into the else block (state is TCP_ESTABLISHED), going to line:
> 	sock->sk->sk_user_data = queue;
> 
> at that point, thread 2 invokes sk_state_change, but this is still the original state_change callback because we didn't set yet to nvmet_tcp_state_change.
> 
> Thread 1 continues and set the callback to nvmet_tcp_state_change, but the sk_state_change was already called and nvmet_tcp_state_change will be never called.
> 
> In that case, there is a leak, and the queue will never be removed.

Are you saying that this is needed?
--
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index ac2d9ed23cea..d82df6cca801 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1486,6 +1486,7 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)

         ret = 0;
         write_lock_bh(&sock->sk->sk_callback_lock);
+       lock_sock(sock->sk);
         if (sock->sk->sk_state != TCP_ESTABLISHED) {
                 /*
                  * If the socket is already closing, don't even start @@ -1502,6 +1503,7 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
                 sock->sk->sk_write_space = nvmet_tcp_write_space;
                 queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);
         }
+       release_sock(sock->sk);
         write_unlock_bh(&sock->sk->sk_callback_lock);

         return ret;
--

From minwoo.im.dev at gmail.com  Mon Feb 15 08:05:52 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Mon, 15 Feb 2021 22:05:52 +0900
Subject: [PATCH 0/2] nvme: add command retry driver control flag
Message-ID: <20210215130554.32387-1-minwoo.im.dev@gmail.com>

Hello,

This series is to support command retry for passthru commands via ioctl.
This patch introduced `driver_ctrl` flag to the passthru ioctl command
structures by replacing the existing `rsvd` field with `driver_ctrl`
flag.

This flag can be used to give some control flag to the driver from
user-space.  The first flag introduced in this series is
`NVME_DRIVER_CTRL_RETRY` to make command retry-able in failure case
without DNR status.  (And of course, in case it's not a path-related
errors).

Please have a look.

Thanks,


From minwoo.im.dev at gmail.com  Mon Feb 15 08:05:53 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Mon, 15 Feb 2021 22:05:53 +0900
Subject: [PATCH 1/2] nvme: retry command if driver control flag given for
 passthru
In-Reply-To: <20210215130554.32387-1-minwoo.im.dev@gmail.com>
References: <20210215130554.32387-1-minwoo.im.dev@gmail.com>
Message-ID: <20210215130554.32387-2-minwoo.im.dev@gmail.com>

ioctl passthru commands are REQ_FAILFAST_DRIVER requests which are
non-retry requests if failed.  This patch introduced `driver_ctrl` flag
to the ioctl command structure: nvme_passthru_cmd, nvme_passthru_cmd64.
`driver_ctrl` field has not been appended, but replaced existing
reserved field.

Clear the REQ_FAILFASTER_DRIVER flag from the request if `driver_ctrl`
is given with NVME_DRIVER_CTRL_RETRY set to make it retry-able if failed
without DNR.

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 drivers/nvme/host/core.c        | 12 ++++++++----
 include/uapi/linux/nvme_ioctl.h | 15 +++++++++++----
 2 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..20cd4a815944 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1143,7 +1143,7 @@ EXPORT_SYMBOL_NS_GPL(nvme_execute_passthru_rq, NVME_TARGET_PASSTHRU);
 static int nvme_submit_user_cmd(struct request_queue *q,
 		struct nvme_command *cmd, void __user *ubuffer,
 		unsigned bufflen, void __user *meta_buffer, unsigned meta_len,
-		u32 meta_seed, u64 *result, unsigned timeout)
+		u32 meta_seed, u64 *result, unsigned timeout, u16 driver_ctrl)
 {
 	bool write = nvme_is_write(cmd);
 	struct nvme_ns *ns = q->queuedata;
@@ -1157,6 +1157,9 @@ static int nvme_submit_user_cmd(struct request_queue *q,
 	if (IS_ERR(req))
 		return PTR_ERR(req);
 
+	if (driver_ctrl & NVME_DRIVER_CTRL_RETRY)
+		req->cmd_flags &= ~REQ_FAILFAST_DRIVER;
+
 	if (timeout)
 		req->timeout = timeout;
 	nvme_req(req)->flags |= NVME_REQ_USERCMD;
@@ -1615,7 +1618,8 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
 
 	return nvme_submit_user_cmd(ns->queue, &c,
 			nvme_to_user_ptr(io.addr), length,
-			metadata, meta_len, lower_32_bits(io.slba), NULL, 0);
+			metadata, meta_len, lower_32_bits(io.slba), NULL, 0,
+			io.driver_ctrl);
 }
 
 static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
@@ -1653,7 +1657,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 	status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c,
 			nvme_to_user_ptr(cmd.addr), cmd.data_len,
 			nvme_to_user_ptr(cmd.metadata), cmd.metadata_len,
-			0, &result, timeout);
+			0, &result, timeout, cmd.driver_ctrl);
 
 	if (status >= 0) {
 		if (put_user(result, &ucmd->result))
@@ -1697,7 +1701,7 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 	status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c,
 			nvme_to_user_ptr(cmd.addr), cmd.data_len,
 			nvme_to_user_ptr(cmd.metadata), cmd.metadata_len,
-			0, &cmd.result, timeout);
+			0, &cmd.result, timeout, cmd.driver_ctrl);
 
 	if (status >= 0) {
 		if (put_user(cmd.result, &ucmd->result))
diff --git a/include/uapi/linux/nvme_ioctl.h b/include/uapi/linux/nvme_ioctl.h
index d99b5a772698..b247b3eee2e3 100644
--- a/include/uapi/linux/nvme_ioctl.h
+++ b/include/uapi/linux/nvme_ioctl.h
@@ -14,7 +14,7 @@ struct nvme_user_io {
 	__u8	flags;
 	__u16	control;
 	__u16	nblocks;
-	__u16	rsvd;
+	__u16	driver_ctrl;
 	__u64	metadata;
 	__u64	addr;
 	__u64	slba;
@@ -27,7 +27,7 @@ struct nvme_user_io {
 struct nvme_passthru_cmd {
 	__u8	opcode;
 	__u8	flags;
-	__u16	rsvd1;
+	__u16	driver_ctrl;
 	__u32	nsid;
 	__u32	cdw2;
 	__u32	cdw3;
@@ -48,7 +48,7 @@ struct nvme_passthru_cmd {
 struct nvme_passthru_cmd64 {
 	__u8	opcode;
 	__u8	flags;
-	__u16	rsvd1;
+	__u16	driver_ctrl;
 	__u32	nsid;
 	__u32	cdw2;
 	__u32	cdw3;
@@ -63,7 +63,7 @@ struct nvme_passthru_cmd64 {
 	__u32	cdw14;
 	__u32	cdw15;
 	__u32	timeout_ms;
-	__u32   rsvd2;
+	__u32   rsvd;
 	__u64	result;
 };
 
@@ -79,4 +79,11 @@ struct nvme_passthru_cmd64 {
 #define NVME_IOCTL_ADMIN64_CMD	_IOWR('N', 0x47, struct nvme_passthru_cmd64)
 #define NVME_IOCTL_IO64_CMD	_IOWR('N', 0x48, struct nvme_passthru_cmd64)
 
+enum {
+	/* Retry command if failed with !DNR for <= nvme_max_retries */
+	__NVME_DRIVER_CTRL_RETRY,
+};
+
+#define NVME_DRIVER_CTRL_RETRY  (1 << __NVME_DRIVER_CTRL_RETRY)
+
 #endif /* _UAPI_LINUX_NVME_IOCTL_H */
-- 
2.17.1



From minwoo.im.dev at gmail.com  Mon Feb 15 08:05:54 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Mon, 15 Feb 2021 22:05:54 +0900
Subject: [nvme-cli PATCH 2/2] nvme-ioctl: add command retry driver control
 flag to passthru
In-Reply-To: <20210215130554.32387-1-minwoo.im.dev@gmail.com>
References: <20210215130554.32387-1-minwoo.im.dev@gmail.com>
Message-ID: <20210215130554.32387-3-minwoo.im.dev@gmail.com>

Replace existing reserved field in passthru ioctl command structures.
This patch also removed the `--rsvd` option from the passthru command
because even if it's given, it's not going to be passthrough to the
kernel driver: meaningless input.

This patch introduced the first flag value for command retry in case of
!DNR failure case from controller.  If `--retry` option is given to
passthru command, it will be retried by the driver itself rather than
returning error to the user-space direcly.

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 linux/nvme_ioctl.h | 12 +++++++++---
 nvme-ioctl.c       |  5 ++---
 nvme-ioctl.h       |  2 +-
 nvme.c             | 15 +++++++++------
 4 files changed, 21 insertions(+), 13 deletions(-)

diff --git a/linux/nvme_ioctl.h b/linux/nvme_ioctl.h
index d569414211d1..17dde016cb5c 100644
--- a/linux/nvme_ioctl.h
+++ b/linux/nvme_ioctl.h
@@ -23,7 +23,7 @@ struct nvme_user_io {
 	__u8	flags;
 	__u16	control;
 	__u16	nblocks;
-	__u16	rsvd;
+	__u16	driver_ctrl;
 	__u64	metadata;
 	__u64	addr;
 	__u64	slba;
@@ -36,7 +36,7 @@ struct nvme_user_io {
 struct nvme_passthru_cmd {
 	__u8	opcode;
 	__u8	flags;
-	__u16	rsvd1;
+	__u16	driver_ctrl;
 	__u32	nsid;
 	__u32	cdw2;
 	__u32	cdw3;
@@ -57,7 +57,7 @@ struct nvme_passthru_cmd {
 struct nvme_passthru_cmd64 {
     __u8    opcode;
     __u8    flags;
-    __u16   rsvd1;
+    __u16   driver_ctrl;
     __u32   nsid;
     __u32   cdw2;
     __u32   cdw3;
@@ -88,4 +88,10 @@ struct nvme_passthru_cmd64 {
 #define NVME_IOCTL_ADMIN64_CMD  _IOWR('N', 0x47, struct nvme_passthru_cmd64)
 #define NVME_IOCTL_IO64_CMD _IOWR('N', 0x48, struct nvme_passthru_cmd64)
 
+enum nvme_driver_ctrl_bits {
+	/* Retry command if failed with !DNR for <= nvme_max_retries */
+	__NVME_DRIVER_CTRL_RETRY,	};
+
+#define NVME_DRIVER_CTRL_RETRY	(1 << __NVME_DRIVER_CTRL_RETRY)
+
 #endif /* _UAPI_LINUX_NVME_IOCTL_H */
diff --git a/nvme-ioctl.c b/nvme-ioctl.c
index 3e19023be890..2176fb3747ec 100644
--- a/nvme-ioctl.c
+++ b/nvme-ioctl.c
@@ -90,7 +90,7 @@ int nvme_submit_io_passthru(int fd, struct nvme_passthru_cmd *cmd)
 }
 
 int nvme_passthru(int fd, unsigned long ioctl_cmd, __u8 opcode,
-		  __u8 flags, __u16 rsvd,
+		  __u8 flags, __u16 driver_ctrl,
 		  __u32 nsid, __u32 cdw2, __u32 cdw3, __u32 cdw10, __u32 cdw11,
 		  __u32 cdw12, __u32 cdw13, __u32 cdw14, __u32 cdw15,
 		  __u32 data_len, void *data, __u32 metadata_len,
@@ -99,7 +99,7 @@ int nvme_passthru(int fd, unsigned long ioctl_cmd, __u8 opcode,
 	struct nvme_passthru_cmd cmd = {
 		.opcode		= opcode,
 		.flags		= flags,
-		.rsvd1		= rsvd,
+		.driver_ctrl	= driver_ctrl,
 		.nsid		= nsid,
 		.cdw2		= cdw2,
 		.cdw3		= cdw3,
@@ -133,7 +133,6 @@ int nvme_io(int fd, __u8 opcode, __u64 slba, __u16 nblocks, __u16 control,
 		.flags		= 0,
 		.control	= control,
 		.nblocks	= nblocks,
-		.rsvd		= 0,
 		.metadata	= (__u64)(uintptr_t) metadata,
 		.addr		= (__u64)(uintptr_t) data,
 		.slba		= slba,
diff --git a/nvme-ioctl.h b/nvme-ioctl.h
index 2b7640734398..a442da0275cc 100644
--- a/nvme-ioctl.h
+++ b/nvme-ioctl.h
@@ -17,7 +17,7 @@ int nvme_submit_admin_passthru(int fd, struct nvme_passthru_cmd *cmd);
 int nvme_submit_io_passthru(int fd, struct nvme_passthru_cmd *cmd);
 
 int nvme_passthru(int fd, unsigned long ioctl_cmd, __u8 opcode, __u8 flags,
-		  __u16 rsvd, __u32 nsid, __u32 cdw2, __u32 cdw3,
+		  __u16 driver_ctrl, __u32 nsid, __u32 cdw2, __u32 cdw3,
 		  __u32 cdw10, __u32 cdw11, __u32 cdw12,
 		  __u32 cdw13, __u32 cdw14, __u32 cdw15,
 		  __u32 data_len, void *data, __u32 metadata_len,
diff --git a/nvme.c b/nvme.c
index 5324e8bbe79f..419b2ecbf30f 100644
--- a/nvme.c
+++ b/nvme.c
@@ -5060,11 +5060,11 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 	int err = 0, wfd = STDIN_FILENO, fd;
 	__u32 result;
 	bool huge;
+	__u16 driver_ctrl = 0x0;
 
 	struct config {
 		__u8  opcode;
 		__u8  flags;
-		__u16 rsvd;
 		__u32 namespace_id;
 		__u32 data_len;
 		__u32 metadata_len;
@@ -5084,12 +5084,12 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 		int   read;
 		int   write;
 		__u8  prefill;
+		bool  retry;
 	};
 
 	struct config cfg = {
 		.opcode       = 0,
 		.flags        = 0,
-		.rsvd         = 0,
 		.namespace_id = 0,
 		.data_len     = 0,
 		.metadata_len = 0,
@@ -5104,11 +5104,11 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 		.cdw15        = 0,
 		.input_file   = "",
 		.prefill      = 0,
+		.retry        = false,
 	};
 
 	const char *opcode = "opcode (required)";
 	const char *flags = "command flags";
-	const char *rsvd = "value for reserved field";
 	const char *namespace_id = "desired namespace";
 	const char *data_len = "data I/O length (bytes)";
 	const char *metadata_len = "metadata seg. length (bytes)";
@@ -5128,12 +5128,12 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 	const char *re = "set dataflow direction to receive";
 	const char *wr = "set dataflow direction to send";
 	const char *prefill = "prefill buffers with known byte-value, default 0";
+	const char *retry = "command retry if failed with !DNR";
 
 	OPT_ARGS(opts) = {
 		OPT_BYTE("opcode",       'o', &cfg.opcode,       opcode),
 		OPT_BYTE("flags",        'f', &cfg.flags,        flags),
 		OPT_BYTE("prefill",      'p', &cfg.prefill,      prefill),
-		OPT_SHRT("rsvd",         'R', &cfg.rsvd,         rsvd),
 		OPT_UINT("namespace-id", 'n', &cfg.namespace_id, namespace_id),
 		OPT_UINT("data-len",     'l', &cfg.data_len,     data_len),
 		OPT_UINT("metadata-len", 'm', &cfg.metadata_len, metadata_len),
@@ -5152,6 +5152,7 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 		OPT_FLAG("dry-run",      'd', &cfg.dry_run,      dry),
 		OPT_FLAG("read",         'r', &cfg.read,         re),
 		OPT_FLAG("write",        'w', &cfg.write,        wr),
+		OPT_FLAG("retry",        'R', &cfg.retry,        retry),
 		OPT_END()
 	};
 
@@ -5213,7 +5214,6 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 	if (cfg.show_command) {
 		printf("opcode       : %02x\n", cfg.opcode);
 		printf("flags        : %02x\n", cfg.flags);
-		printf("rsvd1        : %04x\n", cfg.rsvd);
 		printf("nsid         : %08x\n", cfg.namespace_id);
 		printf("cdw2         : %08x\n", cfg.cdw2);
 		printf("cdw3         : %08x\n", cfg.cdw3);
@@ -5232,7 +5232,10 @@ static int passthru(int argc, char **argv, int ioctl_cmd, const char *desc, stru
 	if (cfg.dry_run)
 		goto free_data;
 
-	err = nvme_passthru(fd, ioctl_cmd, cfg.opcode, cfg.flags, cfg.rsvd,
+	if (cfg.retry)
+		driver_ctrl |= NVME_DRIVER_CTRL_RETRY;
+
+	err = nvme_passthru(fd, ioctl_cmd, cfg.opcode, cfg.flags, driver_ctrl,
 				cfg.namespace_id, cfg.cdw2, cfg.cdw3, cfg.cdw10,
 				cfg.cdw11, cfg.cdw12, cfg.cdw13, cfg.cdw14, cfg.cdw15,
 				cfg.data_len, data, cfg.metadata_len, metadata,
-- 
2.17.1



From kbusch at kernel.org  Mon Feb 15 12:02:33 2021
From: kbusch at kernel.org (Keith Busch)
Date: Mon, 15 Feb 2021 09:02:33 -0800
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210214110126.24023-2-minwoo.im.dev@gmail.com>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
Message-ID: <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>

On Sun, Feb 14, 2021 at 08:01:25PM +0900, Minwoo Im wrote:
> We don't allow NVME_IOCTL_IO_CMD ioctl in case that a controller has
> multiple namespaces attached.  Also, I/O request to the controller
> character device has not been recommended and deprecated because we have
> block device to I/O with where the multipath consideration is taken.
> 
> Once kernel decided a path to I/O for a namespace based on the I/O
> policy of a NVMe subsystem, userspace is not allowed to choose a path to
> I/O.  If a path is broken(inaccessible state in ANA), then it will not
> try to I/O to that path.
> 
> This patch introduced NVME_IOCTL_MPATH_IO command for controller
> device(e.g., /dev/nvme0) to support multipath I/O passthrough for
> userspace.  Regardless driver's path decision, userspace can target a
> namespace to I/O.  In this case, `cmd.nsid` will be used to find out the
> namespace instance target which is hidden(e.g., nvmeXcYnZ).

IO commands are not allowed through the character handle with the
existing ioctls. A new ioctl doesn't make it okay. If it was okay, then
we could just remove the limitation in the current ones.


From sagi at grimberg.me  Mon Feb 15 16:18:17 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 15 Feb 2021 13:18:17 -0800
Subject: [PATCH] nvmet-tcp: enable optional queue idle period tracking
In-Reply-To: <BYAPR11MB286909CAA3A8721B46A02AFFE58C9@BYAPR11MB2869.namprd11.prod.outlook.com>
References: <BYAPR11MB286909CAA3A8721B46A02AFFE58C9@BYAPR11MB2869.namprd11.prod.outlook.com>
Message-ID: <25dfb6d2-2519-0f30-529f-36dcb4263160@grimberg.me>



On 2/11/21 3:42 PM, Wunderlich, Mark wrote:
> nvmet-tcp: enable optional queue idle period tracking
> 
> Add 'queue idle period' option used by io_work() to support
> network devices enabled with advanced interrupt moderation
> supporting a relaxed interrupt model. It was discovered that
> such a NIC used on the target was unable to support initiator
> connection establishment, caused by the existing io_work()
> flow that immediately exits and does not re-queue itself at
> the first loop with no activity.
> 
> With this new option a queue is assigned a period of time
> that no activity must occur in order to become 'idle'.  Until
> the queue is idle the work item is requeued.
> 
> The new module option is defined as changeable making it
> flexible for testing purposes.
> 
> The pre-existing legacy behavior is preserved when no module option
> for queue idle period is specified.
> 
> Signed-off-by: Mark Wunderlich <mark.wunderlich at intel.com>
> ---
> Testing was performed with a NIC using standard HW interrupt mode, with
> and without the new module option enabled.  No measurable performance
> drop was seen when the patch was applied and the new option specified
> or not.  A side effect of a standard NIC using the new option
> will reduce the context switch rate.  We measured a drop from roughly
> 90K to less than 300 (for 32 active connections).

Nice!

> 
> For a NIC using a passive advanced interrupt moderation policy, it was
> then successfully able to achieve and maintain active connections with
> the target.
> ---
>   drivers/nvme/target/tcp.c |   65 ++++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 64 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
> index dc1f0f647189..f5e6e2806841 100644
> --- a/drivers/nvme/target/tcp.c
> +++ b/drivers/nvme/target/tcp.c
> @@ -29,6 +29,16 @@ static int so_priority;
>   module_param(so_priority, int, 0644);
>   MODULE_PARM_DESC(so_priority, "nvmet tcp socket optimize priority");
>   
> +/*
> + * Define a time period (in msecs)

Is this in msecs or usecs? Anyway I think it should be in the name as
well.

  that io_work() shall sample an activated
> + * queue before determining it to be idle.  This optional module behavior
> + * can enable NIC solutions that support socket optimized packet processing
> + * using advanced interrupt moderation techniques.
> + */
> +static int queue_idle_period;
> +module_param(queue_idle_period, int, 0644);
> +MODULE_PARM_DESC(queue_idle_period, "nvmet tcp io_work queue idle time period");

I think this should be called "queue_poll_period" maybe? because this is
the period a queue will keep actively trying to read data from the
socket regardless if there is any activity or not right?

Or does this mean a different semantics with this?

> +
>   #define NVMET_TCP_RECV_BUDGET		8
>   #define NVMET_TCP_SEND_BUDGET		8
>   #define NVMET_TCP_IO_WORK_BUDGET	64
> @@ -96,6 +106,7 @@ struct nvmet_tcp_queue {
>   	struct work_struct	io_work;
>   	struct nvmet_cq		nvme_cq;
>   	struct nvmet_sq		nvme_sq;
> +	unsigned long           idle_period;

poll_period?

>   
>   	/* send state */
>   	struct nvmet_tcp_cmd	*cmds;
> @@ -1198,12 +1209,42 @@ static void nvmet_tcp_schedule_release_queue(struct nvmet_tcp_queue *queue)
>   	spin_unlock(&queue->state_lock);
>   }
>   
> +/*
> + * This worker function will process all send and recv packet
> + * activity for a queue. It will loop on the queue for up to a
> + * given maximum operation budget, or until there is no activity
> + * during a single loop iteration.
> + *
> + * Two exit modes are possible.
> + *
> + * The default 'pending' mode where the worker will re-queue
> + * itself, after exiting the work loop, only if any send or recv
> + * activity was recorded during the last pass within the loop.
> + *
> + * A optional 'idle period' mode where in addition to re-queueing
> + * itself because of activity it also tracks if a queue has not reached an
> + * assigned 'idle' period of time. The worker consumes from the assigned
> + * time period, across many potential invocations, until it is expired.
> + * A queue with activity always being awarded a fresh time
> + * period for processing.
> + */
>   static void nvmet_tcp_io_work(struct work_struct *w)
>   {
>   	struct nvmet_tcp_queue *queue =
>   		container_of(w, struct nvmet_tcp_queue, io_work);
>   	bool pending;
>   	int ret, ops = 0;
> +	long period_remaining;
> +	unsigned long idle_deadline;
> +
> +	/* Setup use of optional tracking for idle time period */

Redundant comment.

> +	if (queue_idle_period) {
> +		/* Assign the queue an idle period if not already set */
> +		if (!queue->idle_period)
> +			queue->idle_period =
> +					usecs_to_jiffies(queue_idle_period);
> +		idle_deadline = jiffies + queue->idle_period;

Why do you need to keep a period time?

Why don't you just maintain queue->poll_start and just check if
your past the queue_poll time addition:

	if (queue_idle_period && queue->start_poll == 0)
		queue->start_poll = jiffies;

...
[io_work loop]
...

	if (queue_poll_period) {
		if (!time_after(jiffies, queue->start_poll +
				usecs_to_jiffies(queue_poll_period)))
			pending = true;
		else
			queue->start_poll = 0;
	}

Can this work?

> +	}
>   
>   	do {
>   		pending = false;
> @@ -1222,8 +1263,30 @@ static void nvmet_tcp_io_work(struct work_struct *w)
>   
>   	} while (pending && ops < NVMET_TCP_IO_WORK_BUDGET);
>   
> +	/* If optional deadline mode active, determine if queue has reached its
> +	 * idle process deadline limit.  Remaining deadline is calculated. Any ops
> +	 * activity awards the queue a new deadline period.
> +	 */

Comments should stay consistent in style...

> +	if (queue_idle_period) {
> +		/*
> +		 * Clear to award active non-idle queue new period, or
> +		 * reset for future queue activity after exit when idle reached.
> +		 */
> +		queue->idle_period = 0;
> +		if (ops > 0) {
> +			pending = true;

What is this useful for?

> +		} else if (!time_after(jiffies, idle_deadline)) {
> +			period_remaining = (long)(idle_deadline - jiffies);
> +			if (period_remaining > 0) {
> +				pending = true;
> +				queue->idle_period = period_remaining;
> +			}
> +		}
> +	}

See above if it is simpler?

> +
>   	/*
> -	 * We exahusted our budget, requeue our selves
> +	 * We requeue ourself when pending indicates there was activity
> +	 * recorded, or queue has not reached optional idle time period.
>   	 */
>   	if (pending)
>   		queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);


From sagi at grimberg.me  Mon Feb 15 16:23:55 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 15 Feb 2021 13:23:55 -0800
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
Message-ID: <5b45835b-eb81-29e8-e319-a8509474e27f@grimberg.me>


>>>>> blk_mq_tag_to_rq() will always return a request if the command_id is
>>>>> in the valid range. Check if the request has been started. If we
>>>>> blindly process the request we might double complete a request which
>>>>> can be fatal.
>>>>
>>>> How did you get to this one? did the controller send a completion for
>>>> a completed/bogus request?
>>>
>>> If that is the case, then that must mean it's possible the driver could
>>> have started the command id just before the bogus completion check. Data
>>> iorruption, right?
>>
>> Yes, which is why I don't think this check is very useful..
> 
> I actually view that as a valid protection against spoofed frames.
> Without it it's easy to crash the machine by injecting fake completions 
> with random command ids.

And this doesn't help because the command can have been easily reused
and started... What is this protecting against? Note that none of the
other transports checks that, why should tcp?


From sagi at grimberg.me  Mon Feb 15 16:29:45 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 15 Feb 2021 13:29:45 -0800
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <20210215104020.yyithlo2hkxqvguj@beryllium.lan>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
 <20210215104020.yyithlo2hkxqvguj@beryllium.lan>
Message-ID: <a2064070-b511-ba6d-bd64-0b3abc208356@grimberg.me>


>>>>>> blk_mq_tag_to_rq() will always return a request if the command_id is
>>>>>> in the valid range. Check if the request has been started. If we
>>>>>> blindly process the request we might double complete a request which
>>>>>> can be fatal.
>>>>>
>>>>> How did you get to this one? did the controller send a completion for
>>>>> a completed/bogus request?
>>>>
>>>> If that is the case, then that must mean it's possible the driver could
>>>> have started the command id just before the bogus completion check. Data
>>>> iorruption, right?
> 
> 'during TCP LIF toggles and aggr relocates' testing the host
> crashes. TBH, I do not really know what is happening or what the test
> does. Still trying to figure out what's going on.

Well, I think we should probably figure out why that is happening first.

> I was just very
> surprised how much the code trusts the other side to behave correctly./

What does pci/rdma/fc does differently? What does scsi do here?

>>> Yes, which is why I don't think this check is very useful..
>>
>> I actually view that as a valid protection against spoofed frames.
>> Without it it's easy to crash the machine by injecting fake completions with
>> random command ids.
> 
> In this test scenario it's not even a spoofed frame; maybe just a
> confused controller.

Maybe... I am still not sure how this patch helps here though...


From sagi at grimberg.me  Mon Feb 15 16:35:37 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 15 Feb 2021 13:35:37 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <SJ0PR19MB4544CAB0A539F84E54EC5208F2889@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
 <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>
 <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>
 <2251aa94-0d6c-1405-b1d8-185bcd714d8c@grimberg.me>
 <SJ0PR19MB45442A0B8E171B55A8AACACEF2B29@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <8937148e-f667-9a05-3828-58114dca6da3@grimberg.me>
 <SJ0PR19MB4544A653D3A84A19A28D3984F28E9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <1a430561-498c-70dd-f14b-8fab10673f65@grimberg.me>
 <SJ0PR19MB4544CAB0A539F84E54EC5208F2889@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <7f5ff08a-36f0-3b67-9753-20eedd4b6f4d@grimberg.me>

> Hi Sagi,

Hey,

> Just to give you an update...
> 
> We're still investigating the root cause of the crash.
> 
> We found a bug in our Discovery Controller related to SGL format (format 
> 0x5A vs. 0x01). When the host sends a "Set Feature" to configure AER/AEN 
> with a SGL format of 0x5A,

This is coming from:
--
static void nvme_tcp_set_sg_null(struct nvme_command *c)
{
         struct nvme_sgl_desc *sg = &c->common.dptr.sgl;

         sg->addr = 0;
         sg->length = 0;
         sg->type = (NVME_TRANSPORT_SGL_DATA_DESC << 4) |
                         NVME_SGL_FMT_TRANSPORT_A;
}
--

  the DC responds with an R2T, which is
> obviously a bug. This does not happen when the SGL format is 0x01. We 
> believe that this R2T, because it is unexpected by the nvme-tcp module, 
> causes the module to crash.

I'm assuming because the R2T has data length of 0? because set_features
does not pass any data (feature offset/value is in the sqe)...

> One of our engineers that is more familiar with kernel modules is 
> currently trying to understand how the R2T would cause nvme-tcp to crash.
> 
> I will let you know if/when I get more info.

Cool, thanks.


From sagi at grimberg.me  Mon Feb 15 16:42:29 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Mon, 15 Feb 2021 13:42:29 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <7f5ff08a-36f0-3b67-9753-20eedd4b6f4d@grimberg.me>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <42f94a06-cc66-87dd-3a92-de793a228d22@grimberg.me>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
 <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>
 <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>
 <2251aa94-0d6c-1405-b1d8-185bcd714d8c@grimberg.me>
 <SJ0PR19MB45442A0B8E171B55A8AACACEF2B29@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <8937148e-f667-9a05-3828-58114dca6da3@grimberg.me>
 <SJ0PR19MB4544A653D3A84A19A28D3984F28E9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <1a430561-498c-70dd-f14b-8fab10673f65@grimberg.me>
 <SJ0PR19MB4544CAB0A539F84E54EC5208F2889@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <7f5ff08a-36f0-3b67-9753-20eedd4b6f4d@grimberg.me>
Message-ID: <8bcaaef7-76b3-0c57-d343-1581807a6d02@grimberg.me>

>> Hi Sagi,
> 
> Hey,
> 
>> Just to give you an update...
>>
>> We're still investigating the root cause of the crash.
>>
>> We found a bug in our Discovery Controller related to SGL format 
>> (format 0x5A vs. 0x01). When the host sends a "Set Feature" to 
>> configure AER/AEN with a SGL format of 0x5A,
> 
> This is coming from:
> -- 
> static void nvme_tcp_set_sg_null(struct nvme_command *c)
> {
>  ??????? struct nvme_sgl_desc *sg = &c->common.dptr.sgl;
> 
>  ??????? sg->addr = 0;
>  ??????? sg->length = 0;
>  ??????? sg->type = (NVME_TRANSPORT_SGL_DATA_DESC << 4) |
>  ??????????????????????? NVME_SGL_FMT_TRANSPORT_A;
> }
> -- 
> 
>  ?the DC responds with an R2T, which is
>> obviously a bug. This does not happen when the SGL format is 0x01. We 
>> believe that this R2T, because it is unexpected by the nvme-tcp 
>> module, causes the module to crash.
> 
> I'm assuming because the R2T has data length of 0? because set_features
> does not pass any data (feature offset/value is in the sqe)...
> 
>> One of our engineers that is more familiar with kernel modules is 
>> currently trying to understand how the R2T would cause nvme-tcp to crash.
>>
>> I will let you know if/when I get more info.
> 
> Cool, thanks.

Does this make the crash go away at least?
--
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 69f59d2c5799..5274cc5800f9 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -568,6 +568,13 @@ static int nvme_tcp_setup_h2c_data_pdu(struct 
nvme_tcp_request *req,
         req->pdu_len = le32_to_cpu(pdu->r2t_length);
         req->pdu_sent = 0;

+       if (unlikely(!req->pdu_len)) {
+               dev_err(queue->ctrl->ctrl.device,
+                       "req %d r2t len is %u, probably a bug...\n",
+                       rq->tag, req->pdu_len);
+               return -EPROTO;
+       }
+
         if (unlikely(req->data_sent + req->pdu_len > req->data_len)) {
                 dev_err(queue->ctrl->ctrl.device,
                         "req %d r2t len %u exceeded data len %u (%zu 
sent)\n",
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index ac2d9ed23cea..d82df6cca801 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
--


From hare at suse.de  Tue Feb 16 03:51:00 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 16 Feb 2021 09:51:00 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <5b45835b-eb81-29e8-e319-a8509474e27f@grimberg.me>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
 <5b45835b-eb81-29e8-e319-a8509474e27f@grimberg.me>
Message-ID: <a18701ad-6880-37bb-7978-7ea6b7079c9e@suse.de>

On 2/15/21 10:23 PM, Sagi Grimberg wrote:
> 
>>>>>> blk_mq_tag_to_rq() will always return a request if the command_id is
>>>>>> in the valid range. Check if the request has been started. If we
>>>>>> blindly process the request we might double complete a request which
>>>>>> can be fatal.
>>>>>
>>>>> How did you get to this one? did the controller send a completion for
>>>>> a completed/bogus request?
>>>>
>>>> If that is the case, then that must mean it's possible the driver could
>>>> have started the command id just before the bogus completion check. 
>>>> Data
>>>> iorruption, right?
>>>
>>> Yes, which is why I don't think this check is very useful..
>>
>> I actually view that as a valid protection against spoofed frames.
>> Without it it's easy to crash the machine by injecting fake 
>> completions with random command ids.
> 
> And this doesn't help because the command can have been easily reused
> and started... What is this protecting against? Note that none of the
> other transports checks that, why should tcp?

Because it's particularly easy to spoof packets on tcp.
All other nvme-of transports are layered on top of other transports 
which do some sanity checks already, so it becomes really hard to inject 
invalid NVMe-oF frames for those.
NVMe-TCP has none of these protections, making it really easy to inject 
faulty frames (or, heaven forbid, running a packet fuzzer).

And crashing the machine on invalid frames is always a bad idea; I would 
have expected NVMe-TCP to drop them.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From minwoo.im.dev at gmail.com  Tue Feb 16 04:51:47 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Tue, 16 Feb 2021 18:51:47 +0900
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
 <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
Message-ID: <20210216095147.GA2601@localhost.localdomain>

On 21-02-15 09:02:33, Keith Busch wrote:
> On Sun, Feb 14, 2021 at 08:01:25PM +0900, Minwoo Im wrote:
> > We don't allow NVME_IOCTL_IO_CMD ioctl in case that a controller has
> > multiple namespaces attached.  Also, I/O request to the controller
> > character device has not been recommended and deprecated because we have
> > block device to I/O with where the multipath consideration is taken.
> > 
> > Once kernel decided a path to I/O for a namespace based on the I/O
> > policy of a NVMe subsystem, userspace is not allowed to choose a path to
> > I/O.  If a path is broken(inaccessible state in ANA), then it will not
> > try to I/O to that path.
> > 
> > This patch introduced NVME_IOCTL_MPATH_IO command for controller
> > device(e.g., /dev/nvme0) to support multipath I/O passthrough for
> > userspace.  Regardless driver's path decision, userspace can target a
> > namespace to I/O.  In this case, `cmd.nsid` will be used to find out the
> > namespace instance target which is hidden(e.g., nvmeXcYnZ).
> 
> IO commands are not allowed through the character handle with the
> existing ioctls. A new ioctl doesn't make it okay. If it was okay, then
> we could just remove the limitation in the current ones.

Thanks for your feedback, Keith.  If you don't mind, may I ask why it's
been entirely unsafe and deprecated in the exsiting
ioctl(NVME_IOCTL_IO_CMD)?  I've seen a patch bfd8947194b2 ("nvme: fixes
for NVME_IOCTL_IO_CMD on the char device), but have no idea why it's
been really depreacted :)


From hch at lst.de  Tue Feb 16 06:46:24 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 16 Feb 2021 12:46:24 +0100
Subject: [PATCH 12/78] dm: use set_capacity_and_notify
In-Reply-To: <CAMM=eLfD0_Am3--X+PsKPTfc9qzejxpMNjYwEh=WtjSa-iSncg@mail.gmail.com>
References: <20201116145809.410558-1-hch@lst.de>
 <20201116145809.410558-13-hch@lst.de>
 <CAMM=eLfD0_Am3--X+PsKPTfc9qzejxpMNjYwEh=WtjSa-iSncg@mail.gmail.com>
Message-ID: <20210216114624.GA1221@lst.de>

On Fri, Feb 12, 2021 at 10:45:32AM -0500, Mike Snitzer wrote:
> On Mon, Nov 16, 2020 at 10:05 AM Christoph Hellwig <hch at lst.de> wrote:
> >
> > Use set_capacity_and_notify to set the size of both the disk and block
> > device.  This also gets the uevent notifications for the resize for free.
> >
> > Signed-off-by: Christoph Hellwig <hch at lst.de>
> > Reviewed-by: Hannes Reinecke <hare at suse.de>
> > ---
> >  drivers/md/dm.c | 3 +--
> >  1 file changed, 1 insertion(+), 2 deletions(-)
> >
> > diff --git a/drivers/md/dm.c b/drivers/md/dm.c
> > index c18fc25485186d..62ad44925e73ec 100644
> > --- a/drivers/md/dm.c
> > +++ b/drivers/md/dm.c
> > @@ -1971,8 +1971,7 @@ static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
> >         if (size != dm_get_size(md))
> >                 memset(&md->geometry, 0, sizeof(md->geometry));
> >
> > -       set_capacity(md->disk, size);
> > -       bd_set_nr_sectors(md->bdev, size);
> > +       set_capacity_and_notify(md->disk, size);
> >
> >         dm_table_event_callback(t, event_callback, md);
> >
> 
> Not yet pinned down _why_ DM is calling set_capacity_and_notify() with
> a size of 0 but, when running various DM regression tests, I'm seeing
> a lot of noise like:
> 
> [  689.240037] dm-2: detected capacity change from 2097152 to 0
> 
> Is this pr_info really useful?  Should it be moved to below: if
> (!capacity || !size) so that it only prints if a uevent is sent?

In general I suspect such a size change might be interesting to users
if it e.g. comes from a remote event.  So I'd be curious why this happens
with DM, and if we can detect some higher level gendisk state to supress
it if it is indeed spurious.


From jeinwag-nvme at marcapo.com  Tue Feb 16 07:25:43 2021
From: jeinwag-nvme at marcapo.com (jeinwag-nvme at marcapo.com)
Date: Tue, 16 Feb 2021 13:25:43 +0100
Subject: [PATCH] nvme-pci: mark Seagate Nytro XM1440 as QUIRK_NO_NS_DESC_LIST.
Message-ID: <20210216122543.21509-1-jeinwag-nvme@marcapo.com>

From: Julian Einwag <jeinwag-nvme at marcapo.com>

The kernel fails to fully detect these SSDs, only the character devices are present:

[   10.785605] nvme nvme0: pci function 0000:04:00.0
[   10.876787] nvme nvme1: pci function 0000:81:00.0
[   13.198614] nvme nvme0: missing or invalid SUBNQN field.
[   13.198658] nvme nvme1: missing or invalid SUBNQN field.
[   13.206896] nvme nvme0: Shutdown timeout set to 20 seconds
[   13.215035] nvme nvme1: Shutdown timeout set to 20 seconds
[   13.225407] nvme nvme0: 16/0/0 default/read/poll queues
[   13.233602] nvme nvme1: 16/0/0 default/read/poll queues
[   13.239627] nvme nvme0: Identify Descriptors failed (8194)
[   13.246315] nvme nvme1: Identify Descriptors failed (8194)

Adding the NVME_QUIRK_NO_NS_DESC_LIST fixes this problem.

BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=205679
---
 drivers/nvme/host/pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 0045c5edf629..91cefa40543f 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3233,7 +3233,8 @@ static const struct pci_device_id nvme_id_table[] = {
 	{ PCI_DEVICE(0x126f, 0x2263),	/* Silicon Motion unidentified */
 		.driver_data = NVME_QUIRK_NO_NS_DESC_LIST, },
 	{ PCI_DEVICE(0x1bb1, 0x0100),   /* Seagate Nytro Flash Storage */
-		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
+		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY |
+				NVME_QUIRK_NO_NS_DESC_LIST, },
 	{ PCI_DEVICE(0x1c58, 0x0003),	/* HGST adapter */
 		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
 	{ PCI_DEVICE(0x1c58, 0x0023),	/* WDC SN200 adapter */
-- 
2.30.0



From kbusch at kernel.org  Tue Feb 16 12:54:20 2021
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 16 Feb 2021 09:54:20 -0800
Subject: [PATCH] nvme-pci: mark Seagate Nytro XM1440 as
 QUIRK_NO_NS_DESC_LIST.
In-Reply-To: <20210216122543.21509-1-jeinwag-nvme@marcapo.com>
References: <20210216122543.21509-1-jeinwag-nvme@marcapo.com>
Message-ID: <20210216175420.GB2708768@dhcp-10-100-145-180.wdc.com>

On Tue, Feb 16, 2021 at 01:25:43PM +0100, jeinwag-nvme at marcapo.com wrote:
> From: Julian Einwag <jeinwag-nvme at marcapo.com>
> 
> The kernel fails to fully detect these SSDs, only the character devices are present:
> 
> [   10.785605] nvme nvme0: pci function 0000:04:00.0
> [   10.876787] nvme nvme1: pci function 0000:81:00.0
> [   13.198614] nvme nvme0: missing or invalid SUBNQN field.
> [   13.198658] nvme nvme1: missing or invalid SUBNQN field.
> [   13.206896] nvme nvme0: Shutdown timeout set to 20 seconds
> [   13.215035] nvme nvme1: Shutdown timeout set to 20 seconds
> [   13.225407] nvme nvme0: 16/0/0 default/read/poll queues
> [   13.233602] nvme nvme1: 16/0/0 default/read/poll queues
> [   13.239627] nvme nvme0: Identify Descriptors failed (8194)
> [   13.246315] nvme nvme1: Identify Descriptors failed (8194)
> 
> Adding the NVME_QUIRK_NO_NS_DESC_LIST fixes this problem.
> 
> BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=205679

The code looks fine, but you need to add your Signed-off-by: tag in the
commit message.

Also, the first line in your commit log is too long and should be
wrapped to 72 characters.

> ---
>  drivers/nvme/host/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 0045c5edf629..91cefa40543f 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -3233,7 +3233,8 @@ static const struct pci_device_id nvme_id_table[] = {
>  	{ PCI_DEVICE(0x126f, 0x2263),	/* Silicon Motion unidentified */
>  		.driver_data = NVME_QUIRK_NO_NS_DESC_LIST, },
>  	{ PCI_DEVICE(0x1bb1, 0x0100),   /* Seagate Nytro Flash Storage */
> -		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
> +		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY |
> +				NVME_QUIRK_NO_NS_DESC_LIST, },
>  	{ PCI_DEVICE(0x1c58, 0x0003),	/* HGST adapter */
>  		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
>  	{ PCI_DEVICE(0x1c58, 0x0023),	/* WDC SN200 adapter */
> -- 


From kbusch at kernel.org  Tue Feb 16 12:57:22 2021
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 16 Feb 2021 09:57:22 -0800
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210216095147.GA2601@localhost.localdomain>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
 <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
 <20210216095147.GA2601@localhost.localdomain>
Message-ID: <20210216175722.GC2708768@dhcp-10-100-145-180.wdc.com>

On Tue, Feb 16, 2021 at 06:51:47PM +0900, Minwoo Im wrote:
> On 21-02-15 09:02:33, Keith Busch wrote:
> > On Sun, Feb 14, 2021 at 08:01:25PM +0900, Minwoo Im wrote:
> > > We don't allow NVME_IOCTL_IO_CMD ioctl in case that a controller has
> > > multiple namespaces attached.  Also, I/O request to the controller
> > > character device has not been recommended and deprecated because we have
> > > block device to I/O with where the multipath consideration is taken.
> > > 
> > > Once kernel decided a path to I/O for a namespace based on the I/O
> > > policy of a NVMe subsystem, userspace is not allowed to choose a path to
> > > I/O.  If a path is broken(inaccessible state in ANA), then it will not
> > > try to I/O to that path.
> > > 
> > > This patch introduced NVME_IOCTL_MPATH_IO command for controller
> > > device(e.g., /dev/nvme0) to support multipath I/O passthrough for
> > > userspace.  Regardless driver's path decision, userspace can target a
> > > namespace to I/O.  In this case, `cmd.nsid` will be used to find out the
> > > namespace instance target which is hidden(e.g., nvmeXcYnZ).
> > 
> > IO commands are not allowed through the character handle with the
> > existing ioctls. A new ioctl doesn't make it okay. If it was okay, then
> > we could just remove the limitation in the current ones.
> 
> Thanks for your feedback, Keith.  If you don't mind, may I ask why it's
> been entirely unsafe and deprecated in the exsiting
> ioctl(NVME_IOCTL_IO_CMD)?  I've seen a patch bfd8947194b2 ("nvme: fixes
> for NVME_IOCTL_IO_CMD on the char device), but have no idea why it's
> been really depreacted :)

A container could read data from namespaces assigned to a different
container.


From chaitanya.kulkarni at wdc.com  Tue Feb 16 16:31:06 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 13:31:06 -0800
Subject: [PATCH 0/6] nvmet: cleanup and status, error log fix
Message-ID: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>

Hi,

This removes duplicate status value assignments, unnecessary function
parameters and fixes the error log page update.

Since the current nvme-5.12 is broken I've tested this series with the
debug diff which is listed in the test report along with the blktests.

-ck

Chaitanya Kulkarni (6):
  nvmet: remove duplicate status assignment
  nvmet: set status on actual error condition
  nvmet: check and set the right err location
  nvmet: remove unnecessary function parameters
  nvmet: remove unnecessary function parameter
  nvmet: remove unnecessary function parameters

 drivers/nvme/target/admin-cmd.c   |  2 +-
 drivers/nvme/target/core.c        | 58 +++++++++++++++----------------
 drivers/nvme/target/fabrics-cmd.c | 18 +++++-----
 drivers/nvme/target/nvmet.h       | 10 +++---
 4 files changed, 43 insertions(+), 45 deletions(-)
# ./compile_nvme.sh 
+ umount /mnt/nvme0n1
+ clear_dmesg
umount: /mnt/nvme0n1: not mounted
+ modprobe -r nvme-fabrics
+ modprobe -r nvme_loop
+ modprobe -r nvmet
+ modprobe -r nvme
+ sleep 1
+ modprobe -r nvme-core
+ lsmod
+ grep nvme
+ git diff
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..cedf704adac6 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1167,7 +1167,8 @@ static int nvme_submit_user_cmd(struct request_queue *q,
                if (ret)
                        goto out;
                bio = req->bio;
-               bio_set_dev(bio, bdev);
+               if(bdev)
+                       bio_set_dev(bio, bdev);
                if (bdev && meta_buffer && meta_len) {
                        meta = nvme_add_user_metadata(bio, meta_buffer, meta_len,
                                        meta_seed, write);
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..38fffee6b85e 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -688,7 +688,8 @@ static struct nvmf_transport_ops nvme_loop_transport = {
        .name           = "loop",
        .module         = THIS_MODULE,
        .create_ctrl    = nvme_loop_create_ctrl,
-       .allowed_opts   = NVMF_OPT_TRADDR,
+       .allowed_opts   = NVMF_OPT_TRADDR | NVMF_OPT_CTRL_LOSS_TMO,
+
 };
 
 static int __init nvme_loop_init_module(void)
++ nproc
+ make -j 64 M=drivers/nvme/ modules
  CC [M]  drivers/nvme//target/configfs.o
  CC [M]  drivers/nvme//target/core.o
  CC [M]  drivers/nvme//target/admin-cmd.o
  CC [M]  drivers/nvme//target/fabrics-cmd.o
  CC [M]  drivers/nvme//target/discovery.o
  CC [M]  drivers/nvme//target/io-cmd-file.o
  CC [M]  drivers/nvme//target/io-cmd-bdev.o
  CC [M]  drivers/nvme//host/core.o
  CC [M]  drivers/nvme//target/passthru.o
  CC [M]  drivers/nvme//target/trace.o
  CC [M]  drivers/nvme//host/trace.o
  CC [M]  drivers/nvme//host/lightnvm.o
  CC [M]  drivers/nvme//target/rdma.o
  CC [M]  drivers/nvme//target/loop.o
  CC [M]  drivers/nvme//host/zns.o
  CC [M]  drivers/nvme//host/hwmon.o
  CC [M]  drivers/nvme//target/fc.o
  CC [M]  drivers/nvme//target/fcloop.o
  CC [M]  drivers/nvme//host/pci.o
  CC [M]  drivers/nvme//target/tcp.o
  CC [M]  drivers/nvme//host/fabrics.o
  CC [M]  drivers/nvme//host/rdma.o
  CC [M]  drivers/nvme//host/fc.o
  CC [M]  drivers/nvme//host/tcp.o
  LD [M]  drivers/nvme//target/nvme-loop.o
  LD [M]  drivers/nvme//target/nvme-fcloop.o
  LD [M]  drivers/nvme//host/nvme-fabrics.o
  LD [M]  drivers/nvme//target/nvmet-tcp.o
  LD [M]  drivers/nvme//host/nvme.o
  LD [M]  drivers/nvme//target/nvmet-rdma.o
  LD [M]  drivers/nvme//host/nvme-tcp.o
  LD [M]  drivers/nvme//target/nvmet.o
  LD [M]  drivers/nvme//target/nvmet-fc.o
  LD [M]  drivers/nvme//host/nvme-rdma.o
  LD [M]  drivers/nvme//host/nvme-fc.o
  LD [M]  drivers/nvme//host/nvme-core.o
  MODPOST drivers/nvme//Module.symvers
  LD [M]  drivers/nvme//host/nvme-core.ko
  LD [M]  drivers/nvme//host/nvme-fabrics.ko
  LD [M]  drivers/nvme//host/nvme-fc.ko
  LD [M]  drivers/nvme//host/nvme-rdma.ko
  LD [M]  drivers/nvme//host/nvme-tcp.ko
  LD [M]  drivers/nvme//host/nvme.ko
  CC [M]  drivers/nvme//target/nvme-fcloop.mod.o
  CC [M]  drivers/nvme//target/nvme-loop.mod.o
  CC [M]  drivers/nvme//target/nvmet-fc.mod.o
  CC [M]  drivers/nvme//target/nvmet-rdma.mod.o
  CC [M]  drivers/nvme//target/nvmet-tcp.mod.o
  CC [M]  drivers/nvme//target/nvmet.mod.o
  LD [M]  drivers/nvme//target/nvmet.ko
  LD [M]  drivers/nvme//target/nvme-loop.ko
  LD [M]  drivers/nvme//target/nvmet-rdma.ko
  LD [M]  drivers/nvme//target/nvme-fcloop.ko
  LD [M]  drivers/nvme//target/nvmet-tcp.ko
  LD [M]  drivers/nvme//target/nvmet-fc.ko
+ HOST=drivers/nvme/host
+ TARGET=drivers/nvme/target
++ uname -r
+ HOST_DEST=/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host/
++ uname -r
+ TARGET_DEST=/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target/
+ cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host//
+ cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target//
+ ls -lrth /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host/ /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target//
/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host/:
total 8.0M
-rw-r--r--. 1 root root 3.0M Feb 16 12:45 nvme-core.ko
-rw-r--r--. 1 root root 643K Feb 16 12:45 nvme-fabrics.ko
-rw-r--r--. 1 root root 1.2M Feb 16 12:45 nvme-fc.ko
-rw-r--r--. 1 root root 1.1M Feb 16 12:45 nvme.ko
-rw-r--r--. 1 root root 1.2M Feb 16 12:45 nvme-rdma.ko
-rw-r--r--. 1 root root 1.1M Feb 16 12:45 nvme-tcp.ko

/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target//:
total 8.2M
-rw-r--r--. 1 root root 712K Feb 16 12:45 nvme-fcloop.ko
-rw-r--r--. 1 root root 619K Feb 16 12:45 nvme-loop.ko
-rw-r--r--. 1 root root 1.1M Feb 16 12:45 nvmet-fc.ko
-rw-r--r--. 1 root root 4.0M Feb 16 12:45 nvmet.ko
-rw-r--r--. 1 root root 1.1M Feb 16 12:45 nvmet-rdma.ko
-rw-r--r--. 1 root root 852K Feb 16 12:45 nvmet-tcp.ko
+ modprobe nvme
 gitlog -7 
7f0c29a24505 (HEAD -> nvme-5.12) nvmet: remove unnecessary function parameters
e8e1ea8fb333 nvmet: remove unnecessary function parameter
f169254686f6 nvmet: remove unnecessary function parameters
fc38bf1595ce nvmet: check and set the right err location
9b176efea50a nvmet: set status on actual error condition
0384837a4690 nvmet: remove duplicate status assignment
9bb31b0ab2e3 Merge branch 'nvme-5.12' of git://git.infradead.org/nvme into nvme-5.12
# cdblktests 
# ./check nvme
nvme/002 (create many subsystems and test discovery)         [passed]
    runtime  35.423s  ...  36.789s
nvme/003 (test if we're sending keep-alives to a discovery controller) [passed]
    runtime  10.159s  ...  10.177s
nvme/004 (test nvme and nvmet UUID NS descriptors)           [passed]
    runtime  1.808s  ...  1.743s
nvme/005 (reset local loopback target)                       [not run]
    nvme_core module does not have parameter multipath
nvme/006 (create an NVMeOF target with a block device-backed ns) [passed]
    runtime  0.126s  ...  0.141s
nvme/007 (create an NVMeOF target with a file-backed ns)     [passed]
    runtime  0.098s  ...  0.096s
nvme/008 (create an NVMeOF host with a block device-backed ns) [passed]
    runtime  1.777s  ...  1.756s
nvme/009 (create an NVMeOF host with a file-backed ns)       [passed]
    runtime  1.727s  ...  1.709s
nvme/010 (run data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  29.228s  ...  19.931s
nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  199.625s  ...  267.604s
nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  72.525s  ...  49.320s
nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns) [passed]
    runtime    ...  335.511s
nvme/014 (flush a NVMeOF block device-backed ns)             [passed]
    runtime  20.695s  ...  21.308s
nvme/015 (unit test for NVMe flush for file backed ns)       [passed]
    runtime  22.596s  ...  21.888s
nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [passed]
    runtime  16.349s  ...  19.097s
nvme/017 (create/delete many file-ns and test discovery)     [passed]
    runtime  15.889s  ...  19.393s
nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed]
    runtime  1.655s  ...  1.718s
nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed]
    runtime  1.718s  ...  1.747s
nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed]
    runtime  1.662s  ...  1.700s
nvme/021 (test NVMe list command on NVMeOF file-backed ns)   [passed]
    runtime  1.713s  ...  1.702s
nvme/022 (test NVMe reset command on NVMeOF file-backed ns)  [passed]
    runtime  2.100s  ...  2.131s
nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed]
    runtime  1.711s  ...  1.739s
nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed]
    runtime  1.663s  ...  1.708s
nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed]
    runtime  1.666s  ...  1.692s
nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed]
    runtime  1.652s  ...  1.798s
nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed]
    runtime  1.650s  ...  1.713s
nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed]
    runtime  1.648s  ...  1.695s
nvme/029 (test userspace IO via nvme-cli read/write interface) [passed]
    runtime  2.011s  ...  2.168s
nvme/030 (ensure the discovery generation counter is updated appropriately) [passed]
    runtime  0.310s  ...  0.379s
nvme/031 (test deletion of NVMeOF controllers immediately after setup) [passed]
    runtime  5.474s  ...  5.774s
nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed]
    runtime  0.043s  ...  0.051s


-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 16:31:07 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 13:31:07 -0800
Subject: [PATCH 1/6] nvmet: remove duplicate status assignment
In-Reply-To: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210216213112.20078-2-chaitanya.kulkarni@wdc.com>

In the function nvmet_alloc_ctrl() we assign status value before we
call nvmet_fine_get_subsys() to :-
status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;

After we successfully find the subsystem we again set the status value
to :-
status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;

Remove the duplicate status assignment value.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 67bbf0e3b507..64c08b71be2c 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1314,7 +1314,6 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 		goto out;
 	}
 
-	status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 	down_read(&nvmet_config_sem);
 	if (!nvmet_host_allowed(subsys, hostnqn)) {
 		pr_info("connect by host %s for subsystem %s not allowed\n",
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 16:31:08 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 13:31:08 -0800
Subject: [PATCH 2/6] nvmet: set status on actual error condition
In-Reply-To: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210216213112.20078-3-chaitanya.kulkarni@wdc.com>

In the nvmet_alloc_ctrl() set the status variable to its error value
NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR in the actual error
condition just like the rest of the code.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 64c08b71be2c..df2d3de0de62 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1305,12 +1305,12 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 	int ret;
 	u16 status;
 
-	status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 	subsys = nvmet_find_get_subsys(req->port, subsysnqn);
 	if (!subsys) {
 		pr_warn("connect request for invalid subsystem %s!\n",
 			subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(subsysnqn);
+		status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 		goto out;
 	}
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 16:31:09 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 13:31:09 -0800
Subject: [PATCH 3/6] nvmet: check and set the right err location
In-Reply-To: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210216213112.20078-4-chaitanya.kulkarni@wdc.com>

The function nvmet_execute_admin_connect() doesn't check for the right
error status value that is return from the nvmet_alloc_ctrl().

Check for NVME_SC_CONNECT_INVALID_PARAM & NVME_SC_CONNECT_INVALID_HOST
before we update the error log.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/fabrics-cmd.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
index 42bd12b8bf00..cf0baa911db2 100644
--- a/drivers/nvme/target/fabrics-cmd.c
+++ b/drivers/nvme/target/fabrics-cmd.c
@@ -191,9 +191,10 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req)
 	status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req,
 				  le32_to_cpu(c->kato), &ctrl);
 	if (status) {
-		if (status == (NVME_SC_INVALID_FIELD | NVME_SC_DNR))
+		if (status == (NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR) ||
+		    status == (NVME_SC_CONNECT_INVALID_HOST | NVME_SC_DNR))
 			req->error_loc =
-				offsetof(struct nvme_common_command, opcode);
+				offsetof(struct nvme_common_command, dptr);
 		goto out;
 	}
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 16:31:10 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 13:31:10 -0800
Subject: [PATCH 4/6] nvmet: remove unnecessary function parameters
In-Reply-To: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210216213112.20078-5-chaitanya.kulkarni@wdc.com>

The function nvmet_ctrl_find_get() accepts subsysnqn, hostnqn, cntlid,
nvmet_req, and out pointer to nvmet_ctrl structure. The parameters
subsysnqn, hostnqn and cntlid can be derived from the caller's
struct nvmf_connect_data.

Replace these parameters with structure pointer nvmf_connect_data *d.

Also, this function returns the same error value from two places. First
on failure to find subsystem from connect data and failure to find ctrl
from cntlid connect data:- NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR.

Move this to the caller to change the return type so we can return ctrl.

Now that we can change the return type instead of taking the pointer to
pointer to the nvmet_ctrl structure remove that function parameter and
return the valid nvmet_ctrl pointer on success and NULL on failure.

Also, add and rename the goto labels for more readability with comment.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c        | 30 +++++++++++++++---------------
 drivers/nvme/target/fabrics-cmd.c | 10 +++++-----
 drivers/nvme/target/nvmet.h       |  4 ++--
 3 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index df2d3de0de62..518dadc395b3 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1179,45 +1179,45 @@ static void nvmet_init_cap(struct nvmet_ctrl *ctrl)
 	ctrl->cap |= NVMET_QUEUE_SIZE - 1;
 }
 
-u16 nvmet_ctrl_find_get(const char *subsysnqn, const char *hostnqn, u16 cntlid,
-		struct nvmet_req *req, struct nvmet_ctrl **ret)
+struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
+				       struct nvmet_req *req)
 {
+	struct nvmet_ctrl *ctrl = NULL;
 	struct nvmet_subsys *subsys;
-	struct nvmet_ctrl *ctrl;
-	u16 status = 0;
 
-	subsys = nvmet_find_get_subsys(req->port, subsysnqn);
+	subsys = nvmet_find_get_subsys(req->port, d->subsysnqn);
 	if (!subsys) {
 		pr_warn("connect request for invalid subsystem %s!\n",
-			subsysnqn);
+			d->subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(subsysnqn);
-		return NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
+		goto out;
 	}
 
 	mutex_lock(&subsys->lock);
 	list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
-		if (ctrl->cntlid == cntlid) {
-			if (strncmp(hostnqn, ctrl->hostnqn, NVMF_NQN_SIZE)) {
+		if (ctrl->cntlid == le16_to_cpu(d->cntlid)) {
+			if (strncmp(d->hostnqn, ctrl->hostnqn, NVMF_NQN_SIZE)) {
 				pr_warn("hostnqn mismatch.\n");
 				continue;
 			}
 			if (!kref_get_unless_zero(&ctrl->ref))
 				continue;
 
-			*ret = ctrl;
-			goto out;
+			/* ctrl found */
+			goto found;
 		}
 	}
 
+	ctrl = NULL; /* ctrl not found */
 	pr_warn("could not find controller %d for subsys %s / host %s\n",
-		cntlid, subsysnqn, hostnqn);
+		le16_to_cpu(d->cntlid), d->subsysnqn, d->hostnqn);
 	req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(cntlid);
-	status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 
-out:
+found:
 	mutex_unlock(&subsys->lock);
 	nvmet_subsys_put(subsys);
-	return status;
+out:
+	return ctrl;
 }
 
 u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd)
diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
index cf0baa911db2..46578f98abcc 100644
--- a/drivers/nvme/target/fabrics-cmd.c
+++ b/drivers/nvme/target/fabrics-cmd.c
@@ -223,7 +223,7 @@ static void nvmet_execute_io_connect(struct nvmet_req *req)
 {
 	struct nvmf_connect_command *c = &req->cmd->connect;
 	struct nvmf_connect_data *d;
-	struct nvmet_ctrl *ctrl = NULL;
+	struct nvmet_ctrl *ctrl;
 	u16 qid = le16_to_cpu(c->qid);
 	u16 status = 0;
 
@@ -250,11 +250,11 @@ static void nvmet_execute_io_connect(struct nvmet_req *req)
 		goto out;
 	}
 
-	status = nvmet_ctrl_find_get(d->subsysnqn, d->hostnqn,
-				     le16_to_cpu(d->cntlid),
-				     req, &ctrl);
-	if (status)
+	ctrl = nvmet_ctrl_find_get(d, req);
+	if (!ctrl) {
+		status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 		goto out;
+	}
 
 	if (unlikely(qid > ctrl->subsys->max_qid)) {
 		pr_warn("invalid queue id (%d)\n", qid);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index cdfa537b1c0a..e8211a63eac3 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -433,8 +433,8 @@ void nvmet_ctrl_fatal_error(struct nvmet_ctrl *ctrl);
 void nvmet_update_cc(struct nvmet_ctrl *ctrl, u32 new);
 u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 		struct nvmet_req *req, u32 kato, struct nvmet_ctrl **ctrlp);
-u16 nvmet_ctrl_find_get(const char *subsysnqn, const char *hostnqn, u16 cntlid,
-		struct nvmet_req *req, struct nvmet_ctrl **ret);
+struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
+				       struct nvmet_req *req);
 void nvmet_ctrl_put(struct nvmet_ctrl *ctrl);
 u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd);
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 16:31:11 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 13:31:11 -0800
Subject: [PATCH 5/6] nvmet: remove unnecessary function parameter
In-Reply-To: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210216213112.20078-6-chaitanya.kulkarni@wdc.com>

In nvmet_check_ctrl_status() cmd can be derived from nvmet_req. Remove
the local variable cmd in the nvmet_check_ctrl_status() and function
parameter cmd for nvmet_check_ctrl_status(). Derive the cmd value from
req parameter in the nvmet_check_ctrl_status().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 2 +-
 drivers/nvme/target/core.c      | 9 ++++-----
 drivers/nvme/target/nvmet.h     | 2 +-
 3 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index bc6a774f2124..e15a2191500a 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -926,7 +926,7 @@ u16 nvmet_parse_admin_cmd(struct nvmet_req *req)
 	if (nvmet_req_subsys(req)->type == NVME_NQN_DISC)
 		return nvmet_parse_discovery_cmd(req);
 
-	ret = nvmet_check_ctrl_status(req, cmd);
+	ret = nvmet_check_ctrl_status(req);
 	if (unlikely(ret))
 		return ret;
 
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 518dadc395b3..5915b7cc65a6 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -864,10 +864,9 @@ static inline u16 nvmet_io_cmd_check_access(struct nvmet_req *req)
 
 static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 {
-	struct nvme_command *cmd = req->cmd;
 	u16 ret;
 
-	ret = nvmet_check_ctrl_status(req, cmd);
+	ret = nvmet_check_ctrl_status(req);
 	if (unlikely(ret))
 		return ret;
 
@@ -1220,17 +1219,17 @@ struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
 	return ctrl;
 }
 
-u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd)
+u16 nvmet_check_ctrl_status(struct nvmet_req *req)
 {
 	if (unlikely(!(req->sq->ctrl->cc & NVME_CC_ENABLE))) {
 		pr_err("got cmd %d while CC.EN == 0 on qid = %d\n",
-		       cmd->common.opcode, req->sq->qid);
+		       req->cmd->common.opcode, req->sq->qid);
 		return NVME_SC_CMD_SEQ_ERROR | NVME_SC_DNR;
 	}
 
 	if (unlikely(!(req->sq->ctrl->csts & NVME_CSTS_RDY))) {
 		pr_err("got cmd %d while CSTS.RDY == 0 on qid = %d\n",
-		       cmd->common.opcode, req->sq->qid);
+		       req->cmd->common.opcode, req->sq->qid);
 		return NVME_SC_CMD_SEQ_ERROR | NVME_SC_DNR;
 	}
 	return 0;
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index e8211a63eac3..2dd4a91e5e0e 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -436,7 +436,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
 				       struct nvmet_req *req);
 void nvmet_ctrl_put(struct nvmet_ctrl *ctrl);
-u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd);
+u16 nvmet_check_ctrl_status(struct nvmet_req *req);
 
 struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
 		enum nvme_subsys_type type);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 16:31:12 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 13:31:12 -0800
Subject: [PATCH 6/6] nvmet: remove unnecessary function parameters
In-Reply-To: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210216213112.20078-7-chaitanya.kulkarni@wdc.com>

The function nvmet_alloc_ctrl() accepts subsysnqn, hostnqn, cntlid,
nvmet_req, and out pointer to nvmet_ctrl structure. The parameters
subsysnqn, hostnqn and cntlid can be derived from the caller's
struct nvmf_connect_data.

Replace these parameters with structure pointer nvmf_connect_data *d.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c        | 16 ++++++++--------
 drivers/nvme/target/fabrics-cmd.c |  3 +--
 drivers/nvme/target/nvmet.h       |  4 ++--
 3 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 5915b7cc65a6..9ad001fc07f9 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1296,27 +1296,27 @@ static void nvmet_fatal_error_handler(struct work_struct *work)
 	ctrl->ops->delete_ctrl(ctrl);
 }
 
-u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
-		struct nvmet_req *req, u32 kato, struct nvmet_ctrl **ctrlp)
+u16 nvmet_alloc_ctrl(struct nvmf_connect_data *d, struct nvmet_req *req,
+		     u32 kato, struct nvmet_ctrl **ctrlp)
 {
 	struct nvmet_subsys *subsys;
 	struct nvmet_ctrl *ctrl;
 	int ret;
 	u16 status;
 
-	subsys = nvmet_find_get_subsys(req->port, subsysnqn);
+	subsys = nvmet_find_get_subsys(req->port, d->subsysnqn);
 	if (!subsys) {
 		pr_warn("connect request for invalid subsystem %s!\n",
-			subsysnqn);
+			d->subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(subsysnqn);
 		status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 		goto out;
 	}
 
 	down_read(&nvmet_config_sem);
-	if (!nvmet_host_allowed(subsys, hostnqn)) {
+	if (!nvmet_host_allowed(subsys, d->hostnqn)) {
 		pr_info("connect by host %s for subsystem %s not allowed\n",
-			hostnqn, subsysnqn);
+			d->hostnqn, d->subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(hostnqn);
 		up_read(&nvmet_config_sem);
 		status = NVME_SC_CONNECT_INVALID_HOST | NVME_SC_DNR;
@@ -1339,8 +1339,8 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 	INIT_RADIX_TREE(&ctrl->p2p_ns_map, GFP_KERNEL);
 	INIT_WORK(&ctrl->fatal_err_work, nvmet_fatal_error_handler);
 
-	memcpy(ctrl->subsysnqn, subsysnqn, NVMF_NQN_SIZE);
-	memcpy(ctrl->hostnqn, hostnqn, NVMF_NQN_SIZE);
+	memcpy(ctrl->subsysnqn, d->subsysnqn, NVMF_NQN_SIZE);
+	memcpy(ctrl->hostnqn, d->hostnqn, NVMF_NQN_SIZE);
 
 	kref_init(&ctrl->ref);
 	ctrl->subsys = subsys;
diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
index 46578f98abcc..aa530e0e112d 100644
--- a/drivers/nvme/target/fabrics-cmd.c
+++ b/drivers/nvme/target/fabrics-cmd.c
@@ -188,8 +188,7 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req)
 		goto out;
 	}
 
-	status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req,
-				  le32_to_cpu(c->kato), &ctrl);
+	status = nvmet_alloc_ctrl(d, req, le32_to_cpu(c->kato), &ctrl);
 	if (status) {
 		if (status == (NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR) ||
 		    status == (NVME_SC_CONNECT_INVALID_HOST | NVME_SC_DNR))
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 2dd4a91e5e0e..eaa5b7208a36 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -431,8 +431,8 @@ int nvmet_sq_init(struct nvmet_sq *sq);
 void nvmet_ctrl_fatal_error(struct nvmet_ctrl *ctrl);
 
 void nvmet_update_cc(struct nvmet_ctrl *ctrl, u32 new);
-u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
-		struct nvmet_req *req, u32 kato, struct nvmet_ctrl **ctrlp);
+u16 nvmet_alloc_ctrl(struct nvmf_connect_data *d, struct nvmet_req *req,
+		     u32 kato, struct nvmet_ctrl **ctrlp);
 struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
 				       struct nvmet_req *req);
 void nvmet_ctrl_put(struct nvmet_ctrl *ctrl);
-- 
2.22.1



From Chaitanya.Kulkarni at wdc.com  Tue Feb 16 18:59:17 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 23:59:17 +0000
Subject: [PATCH] nvme: use lighter smp barriers in nvme_irq
References: <1c90fd6a-d991-b347-479c-2c8ae5a4c51f@gmail.com>
Message-ID: <BYAPR04MB4965D222B05866D93852835F86879@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/14/21 08:30, Heiner Kallweit wrote:
> Based on the description we should be fine using the less heavy-weight
> smp barriers here. On x86 this would be compiler barriers only.
>
> Signed-off-by: Heiner Kallweit <hkallweit1 at gmail.com>
Can you share performance numbers ?


From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:18 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:18 -0800
Subject: [PATCH 00/14] nvme: few cleanups and small improvements
Message-ID: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>

Hi,

This has few cleanups such as variable data type, removing?oneliner
wrappers for single callers, removing duplicate kfree() and small
improvements for NVMeOF Passthru fast path such as likely annotation,
removing the extra checks, making function inline for host/core.c.
Last couple of patches fixes warning for FC function header
documentation.

-ck

Chaitanya Kulkarni (14):
  nvme-core: remove duplicate kfree in init identify
  nvme-core: don't use switch for only one case use
  nvme-core: use right type for ARRAY_SIZE check
  nvme-core: fix the type for shutdown_timeout
  nvme-core: add helper to init ctrl transport attr
  nvme-core: add helper to init shutdown timeout
  nvme-core: add helper to init ctrl subsys quirk
  nvme-core: move util quirk at the top of the file
  nvme-core: mark nvme_setup_passsthru() inline
  nvme-core: remove one liner wrappers for streams
  nvme-core: use likely in nvme_init_request()
  nvme-core: don't check nvme_req flags for new req
  nvme-fc: fix the function documentation comment
  nvmet-fc: update function documentation

 drivers/nvme/host/core.c | 333 ++++++++++++++++++++-------------------
 drivers/nvme/host/fc.c   |   2 +-
 drivers/nvme/target/fc.c |   1 +
 3 files changed, 172 insertions(+), 164 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:19 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:19 -0800
Subject: [PATCH 01/14] nvme-core: remove duplicate kfree in init identify
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-2-chaitanya.kulkarni@wdc.com>

In nvme_init_identify() once we initialize nvme_mpath_init() we free
the id with kfree(). This is needed since we just return from all the
following calls to nvme_mpath_init(), that also makes kfree() call
duplicate.

Instead of returning after nvme_mpath_init() jump to the lable
out_free and remove the duplicate call to the kfree() since out_free
label already has one..

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..70042e111efe 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3209,10 +3209,8 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	}
 
 	ret = nvme_mpath_init(ctrl, id);
-	kfree(id);
-
 	if (ret < 0)
-		return ret;
+		goto out_free;
 
 	if (ctrl->apst_enabled && !prev_apst_enabled)
 		dev_pm_qos_expose_latency_tolerance(ctrl->device);
@@ -3221,29 +3219,29 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 
 	ret = nvme_configure_apst(ctrl);
 	if (ret < 0)
-		return ret;
+		goto out_free;
 	
 	ret = nvme_configure_timestamp(ctrl);
 	if (ret < 0)
-		return ret;
+		goto out_free;
 
 	ret = nvme_configure_directives(ctrl);
 	if (ret < 0)
-		return ret;
+		goto out_free;
 
 	ret = nvme_configure_acre(ctrl);
 	if (ret < 0)
-		return ret;
+		goto out_free;
 
 	if (!ctrl->identified && !nvme_discovery_ctrl(ctrl)) {
 		ret = nvme_hwmon_init(ctrl);
 		if (ret < 0)
-			return ret;
+			goto out_free;
 	}
 
 	ctrl->identified = true;
 
-	return 0;
+	ret = 0;
 
 out_free:
 	kfree(id);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:20 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:20 -0800
Subject: [PATCH 02/14] nvme-core: don't use switch for only one case use
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-3-chaitanya.kulkarni@wdc.com>

There is no point in using switch case statement for only one case that
takes about 5 lines.

Remove switch case which only checks for the NVME_CTRL_LIVE state and
use simple if statement that only takes one line.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 70042e111efe..2f6b89f6309d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3254,12 +3254,8 @@ static int nvme_dev_open(struct inode *inode, struct file *file)
 	struct nvme_ctrl *ctrl =
 		container_of(inode->i_cdev, struct nvme_ctrl, cdev);
 
-	switch (ctrl->state) {
-	case NVME_CTRL_LIVE:
-		break;
-	default:
+	if (ctrl->state != NVME_CTRL_LIVE)
 		return -EWOULDBLOCK;
-	}
 
 	nvme_get_ctrl(ctrl);
 	if (!try_module_get(ctrl->ops->module)) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:21 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:21 -0800
Subject: [PATCH 03/14] nvme-core: use right type for ARRAY_SIZE check
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-4-chaitanya.kulkarni@wdc.com>

Use unsigend int value when comparing the values of the expression
that involves sizeof() in the macro ARRAY_SIZE() that returns
size_t.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 2f6b89f6309d..cae526757e03 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3096,7 +3096,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		ctrl->cntlid = le16_to_cpu(id->cntlid);
 
 	if (!ctrl->identified) {
-		int i;
+		unsigned int i;
 
 		ret = nvme_init_subsystem(ctrl, id);
 		if (ret)
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:22 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:22 -0800
Subject: [PATCH 04/14] nvme-core: fix the type for shutdown_timeout
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-5-chaitanya.kulkarni@wdc.com>

In the function nvme_init_identify() we calculate the
ctrl->shutdown_timeout value with module parameter shutdown_timeout
and transition_timeout with clamp_t().

The variable ctrl->shutdown_timeout and transition_timeout is of type
unsigned int.

Change the module parameter shutdown_timeout from unsigned char to
unsigned int so it stays consistent with the type of local variable
transition_timeout, ctrl->shutdown_timeout & clamp_t unsigned int type.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index cae526757e03..840c32c755b2 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -40,8 +40,8 @@ module_param_named(io_timeout, nvme_io_timeout, uint, 0644);
 MODULE_PARM_DESC(io_timeout, "timeout in seconds for I/O");
 EXPORT_SYMBOL_GPL(nvme_io_timeout);
 
-static unsigned char shutdown_timeout = 5;
-module_param(shutdown_timeout, byte, 0644);
+static unsigned int shutdown_timeout = 5;
+module_param(shutdown_timeout, uint, 0644);
 MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown");
 
 static u8 nvme_max_retries = 5;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:23 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:23 -0800
Subject: [PATCH 05/14] nvme-core: add helper to init ctrl transport attr
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-6-chaitanya.kulkarni@wdc.com>

The function nvme_init_identify() has grown over the period of time about
~200 lines given the size of nvme id_ctrl data structure.

Just like the tail of the function it has small helpers to initialize
the independent fields that needs some extra checking, add a new helper
nvme_init_identify_transport() to initialize transport-related fields.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 71 ++++++++++++++++++++++------------------
 1 file changed, 40 insertions(+), 31 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 840c32c755b2..f5b8f8e0ee18 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3057,6 +3057,43 @@ static int nvme_get_effects_log(struct nvme_ctrl *ctrl, u8 csi,
 	return 0;
 }
 
+static int nvme_init_identify_transport(struct nvme_ctrl *ctrl,
+		struct nvme_id_ctrl *id)
+{
+	if (ctrl->ops->flags & NVME_F_FABRICS) {
+		ctrl->icdoff = le16_to_cpu(id->icdoff);
+		ctrl->ioccsz = le32_to_cpu(id->ioccsz);
+		ctrl->iorcsz = le32_to_cpu(id->iorcsz);
+		ctrl->maxcmd = le16_to_cpu(id->maxcmd);
+
+		/*
+		 * In fabrics we need to verify the cntlid matches the
+		 * admin connect
+		 */
+		if (ctrl->cntlid != le16_to_cpu(id->cntlid)) {
+			dev_err(ctrl->device,
+				"Mismatching cntlid: Connect %u vs Identify "
+				"%u, rejecting\n",
+				ctrl->cntlid, le16_to_cpu(id->cntlid));
+			return -EINVAL;
+		}
+
+		if (!nvme_discovery_ctrl(ctrl) && !ctrl->kas) {
+			dev_err(ctrl->device,
+				"keep-alive support is mandatory for fabrics\n");
+			return -EINVAL;
+		}
+		return 0;
+	}
+
+	ctrl->hmpre = le32_to_cpu(id->hmpre);
+	ctrl->hmmin = le32_to_cpu(id->hmmin);
+	ctrl->hmminds = le32_to_cpu(id->hmminds);
+	ctrl->hmmaxd = le16_to_cpu(id->hmmaxd);
+
+	return 0;
+}
+
 /*
  * Initialize the cached copies of the Identify data and various controller
  * register in our nvme_ctrl structure.  This should be called as soon as
@@ -3176,37 +3213,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	}
 	memcpy(ctrl->psd, id->psd, sizeof(ctrl->psd));
 
-	if (ctrl->ops->flags & NVME_F_FABRICS) {
-		ctrl->icdoff = le16_to_cpu(id->icdoff);
-		ctrl->ioccsz = le32_to_cpu(id->ioccsz);
-		ctrl->iorcsz = le32_to_cpu(id->iorcsz);
-		ctrl->maxcmd = le16_to_cpu(id->maxcmd);
-
-		/*
-		 * In fabrics we need to verify the cntlid matches the
-		 * admin connect
-		 */
-		if (ctrl->cntlid != le16_to_cpu(id->cntlid)) {
-			dev_err(ctrl->device,
-				"Mismatching cntlid: Connect %u vs Identify "
-				"%u, rejecting\n",
-				ctrl->cntlid, le16_to_cpu(id->cntlid));
-			ret = -EINVAL;
-			goto out_free;
-		}
-
-		if (!nvme_discovery_ctrl(ctrl) && !ctrl->kas) {
-			dev_err(ctrl->device,
-				"keep-alive support is mandatory for fabrics\n");
-			ret = -EINVAL;
-			goto out_free;
-		}
-	} else {
-		ctrl->hmpre = le32_to_cpu(id->hmpre);
-		ctrl->hmmin = le32_to_cpu(id->hmmin);
-		ctrl->hmminds = le32_to_cpu(id->hmminds);
-		ctrl->hmmaxd = le16_to_cpu(id->hmmaxd);
-	}
+	ret = nvme_init_identify_transport(ctrl, id);
+	if (ret < 0)
+		goto out_free;
 
 	ret = nvme_mpath_init(ctrl, id);
 	if (ret < 0)
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:24 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:24 -0800
Subject: [PATCH 06/14] nvme-core: add helper to init shutdown timeout
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-7-chaitanya.kulkarni@wdc.com>

The function nvme_init_identify() has grown over the period of time about
~200 lines given the size of nvme id_ctrl data structure.

Just like the tail of the function it has small helpers to initialize
the independent fields that needs some extra checking, add a new helper
nvme_init_shutdown_timeout() to initialize ctrl->shutdown_timeout.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f5b8f8e0ee18..77f79a2a396f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3094,6 +3094,24 @@ static int nvme_init_identify_transport(struct nvme_ctrl *ctrl,
 	return 0;
 }
 
+static void nvme_init_shutdown_timeout(struct nvme_ctrl *ctrl,
+		struct nvme_id_ctrl *id)
+{
+	if (id->rtd3e) {
+		/* us -> s */
+		u32 transition_time = le32_to_cpu(id->rtd3e) / USEC_PER_SEC;
+
+		ctrl->shutdown_timeout = clamp_t(unsigned int, transition_time,
+						 shutdown_timeout, 60);
+
+		if (ctrl->shutdown_timeout != shutdown_timeout)
+			dev_info(ctrl->device,
+				 "Shutdown timeout set to %u seconds\n",
+				 ctrl->shutdown_timeout);
+	} else
+		ctrl->shutdown_timeout = shutdown_timeout;
+}
+
 /*
  * Initialize the cached copies of the Identify data and various controller
  * register in our nvme_ctrl structure.  This should be called as soon as
@@ -3184,19 +3202,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	ctrl->max_namespaces = le32_to_cpu(id->mnan);
 	ctrl->ctratt = le32_to_cpu(id->ctratt);
 
-	if (id->rtd3e) {
-		/* us -> s */
-		u32 transition_time = le32_to_cpu(id->rtd3e) / USEC_PER_SEC;
-
-		ctrl->shutdown_timeout = clamp_t(unsigned int, transition_time,
-						 shutdown_timeout, 60);
-
-		if (ctrl->shutdown_timeout != shutdown_timeout)
-			dev_info(ctrl->device,
-				 "Shutdown timeout set to %u seconds\n",
-				 ctrl->shutdown_timeout);
-	} else
-		ctrl->shutdown_timeout = shutdown_timeout;
+	nvme_init_shutdown_timeout(ctrl, id);
 
 	ctrl->npss = id->npss;
 	ctrl->apsta = id->apsta;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:25 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:25 -0800
Subject: [PATCH 07/14] nvme-core: add helper to init ctrl subsys quirk
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-8-chaitanya.kulkarni@wdc.com>

The function nvme_init_identify() has grown over the period of time about
~200 lines given the size of nvme id_ctrl data structure.

Just like the tail of the function it has small helpers to initialize
the independent fields that needs some extra checking, add a new helper
nvme_init_ctrl_subsys_and_quirks() to initialize ctrl subsystem and
quirks.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 43 +++++++++++++++++++++++++---------------
 1 file changed, 27 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 77f79a2a396f..4a59a46193e8 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3057,6 +3057,32 @@ static int nvme_get_effects_log(struct nvme_ctrl *ctrl, u8 csi,
 	return 0;
 }
 
+static int nvme_init_ctrl_subsys_and_quirks(struct nvme_ctrl *ctrl,
+		struct nvme_id_ctrl *id)
+{
+	unsigned int i;
+	int ret;
+
+	ret = nvme_init_subsystem(ctrl, id);
+	if (ret)
+		return ret;
+
+	/*
+	 * Check for quirks.  Quirk can depend on firmware version,
+	 * so, in principle, the set of quirks present can change
+	 * across a reset.  As a possible future enhancement, we
+	 * could re-scan for quirks every time we reinitialize
+	 * the device, but we'd have to make sure that the driver
+	 * behaves intelligently if the quirks change.
+	 */
+	for (i = 0; i < ARRAY_SIZE(core_quirks); i++) {
+		if (quirk_matches(id, &core_quirks[i]))
+			ctrl->quirks |= core_quirks[i].quirks;
+	}
+
+	return 0;
+}
+
 static int nvme_init_identify_transport(struct nvme_ctrl *ctrl,
 		struct nvme_id_ctrl *id)
 {
@@ -3151,24 +3177,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		ctrl->cntlid = le16_to_cpu(id->cntlid);
 
 	if (!ctrl->identified) {
-		unsigned int i;
-
-		ret = nvme_init_subsystem(ctrl, id);
+		ret = nvme_init_ctrl_subsys_and_quirks(ctrl, id);
 		if (ret)
 			goto out_free;
-
-		/*
-		 * Check for quirks.  Quirk can depend on firmware version,
-		 * so, in principle, the set of quirks present can change
-		 * across a reset.  As a possible future enhancement, we
-		 * could re-scan for quirks every time we reinitialize
-		 * the device, but we'd have to make sure that the driver
-		 * behaves intelligently if the quirks change.
-		 */
-		for (i = 0; i < ARRAY_SIZE(core_quirks); i++) {
-			if (quirk_matches(id, &core_quirks[i]))
-				ctrl->quirks |= core_quirks[i].quirks;
-		}
 	}
 
 	if (force_apst && (ctrl->quirks & NVME_QUIRK_NO_DEEPEST_PS)) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:26 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:26 -0800
Subject: [PATCH 08/14] nvme-core: move util quirk at the top of the file
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-9-chaitanya.kulkarni@wdc.com>

The quirk utility functions and structure are defined in the middle of
the file (line:2762) & they are not accessed immediately after their
declaration until we get to nvme_init_identify() (line:3114).

Move quirk related util functions at the top of the file that avoids
any global structure related definitions in the middle of the code
where they are not accessed immediately after definition.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 126 +++++++++++++++++++--------------------
 1 file changed, 63 insertions(+), 63 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4a59a46193e8..5cb90bd8704d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -93,6 +93,69 @@ static void nvme_put_subsystem(struct nvme_subsystem *subsys);
 static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
 					   unsigned nsid);
 
+struct nvme_core_quirk_entry {
+	/*
+	 * NVMe model and firmware strings are padded with spaces.  For
+	 * simplicity, strings in the quirk table are padded with NULLs
+	 * instead.
+	 */
+	u16 vid;
+	const char *mn;
+	const char *fr;
+	unsigned long quirks;
+};
+
+static const struct nvme_core_quirk_entry core_quirks[] = {
+	{
+		/*
+		 * This Toshiba device seems to die using any APST states.  See:
+		 * https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1678184/comments/11
+		 */
+		.vid = 0x1179,
+		.mn = "THNSF5256GPUK TOSHIBA",
+		.quirks = NVME_QUIRK_NO_APST,
+	},
+	{
+		/*
+		 * This LiteON CL1-3D*-Q11 firmware version has a race
+		 * condition associated with actions related to suspend to idle
+		 * LiteON has resolved the problem in future firmware
+		 */
+		.vid = 0x14a4,
+		.fr = "22301111",
+		.quirks = NVME_QUIRK_SIMPLE_SUSPEND,
+	}
+};
+
+/* match is null-terminated but idstr is space-padded. */
+static bool string_matches(const char *idstr, const char *match, size_t len)
+{
+	size_t matchlen;
+
+	if (!match)
+		return true;
+
+	matchlen = strlen(match);
+	WARN_ON_ONCE(matchlen > len);
+
+	if (memcmp(idstr, match, matchlen))
+		return false;
+
+	for (; matchlen < len; matchlen++)
+		if (idstr[matchlen] != ' ')
+			return false;
+
+	return true;
+}
+
+static bool quirk_matches(const struct nvme_id_ctrl *id,
+			  const struct nvme_core_quirk_entry *q)
+{
+	return q->vid == le16_to_cpu(id->vid) &&
+		string_matches(id->mn, q->mn, sizeof(id->mn)) &&
+		string_matches(id->fr, q->fr, sizeof(id->fr));
+}
+
 /*
  * Prepare a queue for teardown.
  *
@@ -2704,69 +2767,6 @@ static void nvme_set_latency_tolerance(struct device *dev, s32 val)
 	}
 }
 
-struct nvme_core_quirk_entry {
-	/*
-	 * NVMe model and firmware strings are padded with spaces.  For
-	 * simplicity, strings in the quirk table are padded with NULLs
-	 * instead.
-	 */
-	u16 vid;
-	const char *mn;
-	const char *fr;
-	unsigned long quirks;
-};
-
-static const struct nvme_core_quirk_entry core_quirks[] = {
-	{
-		/*
-		 * This Toshiba device seems to die using any APST states.  See:
-		 * https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1678184/comments/11
-		 */
-		.vid = 0x1179,
-		.mn = "THNSF5256GPUK TOSHIBA",
-		.quirks = NVME_QUIRK_NO_APST,
-	},
-	{
-		/*
-		 * This LiteON CL1-3D*-Q11 firmware version has a race
-		 * condition associated with actions related to suspend to idle
-		 * LiteON has resolved the problem in future firmware
-		 */
-		.vid = 0x14a4,
-		.fr = "22301111",
-		.quirks = NVME_QUIRK_SIMPLE_SUSPEND,
-	}
-};
-
-/* match is null-terminated but idstr is space-padded. */
-static bool string_matches(const char *idstr, const char *match, size_t len)
-{
-	size_t matchlen;
-
-	if (!match)
-		return true;
-
-	matchlen = strlen(match);
-	WARN_ON_ONCE(matchlen > len);
-
-	if (memcmp(idstr, match, matchlen))
-		return false;
-
-	for (; matchlen < len; matchlen++)
-		if (idstr[matchlen] != ' ')
-			return false;
-
-	return true;
-}
-
-static bool quirk_matches(const struct nvme_id_ctrl *id,
-			  const struct nvme_core_quirk_entry *q)
-{
-	return q->vid == le16_to_cpu(id->vid) &&
-		string_matches(id->mn, q->mn, sizeof(id->mn)) &&
-		string_matches(id->fr, q->fr, sizeof(id->fr));
-}
-
 static void nvme_init_subnqn(struct nvme_subsystem *subsys, struct nvme_ctrl *ctrl,
 		struct nvme_id_ctrl *id)
 {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:27 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:27 -0800
Subject: [PATCH 09/14] nvme-core: mark nvme_setup_passsthru() inline
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-10-chaitanya.kulkarni@wdc.com>

Since nvmet_setup_passthru() function falls in fast path when used with
the NVMeOF passthru backend, make it inline.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 5cb90bd8704d..b43557ebc564 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -788,7 +788,7 @@ static void nvme_assign_write_stream(struct nvme_ctrl *ctrl,
 		req->q->write_hints[streamid] += blk_rq_bytes(req) >> 9;
 }
 
-static void nvme_setup_passthrough(struct request *req,
+static inline void nvme_setup_passthrough(struct request *req,
 		struct nvme_command *cmd)
 {
 	memcpy(cmd, nvme_req(req)->cmd, sizeof(*cmd));
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:28 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:28 -0800
Subject: [PATCH 10/14] nvme-core: remove one liner wrappers for streams
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-11-chaitanya.kulkarni@wdc.com>

The functions nvme_[enable|disable]_streams() are just one line wrapper
to the nvme_toggel_streams(). Also, there is only one call exist for
them.

Instead of maintaining one liners, rename original function
nvme_toggle_streams() -> nvme_ctrl_allow_streams() and pass the boolean
value to the function to enable/disable streams. While we are at it
remove the memset() call and zero initialize the structure just like in
nvme_identify_ctrl().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index b43557ebc564..139cbeb11c7d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -686,11 +686,9 @@ static struct request *nvme_alloc_request_qid(struct request_queue *q,
 	return req;
 }
 
-static int nvme_toggle_streams(struct nvme_ctrl *ctrl, bool enable)
+static int nvme_ctrl_allow_streams(struct nvme_ctrl *ctrl, bool enable)
 {
-	struct nvme_command c;
-
-	memset(&c, 0, sizeof(c));
+	struct nvme_command c = { };
 
 	c.directive.opcode = nvme_admin_directive_send;
 	c.directive.nsid = cpu_to_le32(NVME_NSID_ALL);
@@ -702,16 +700,6 @@ static int nvme_toggle_streams(struct nvme_ctrl *ctrl, bool enable)
 	return nvme_submit_sync_cmd(ctrl->admin_q, &c, NULL, 0);
 }
 
-static int nvme_disable_streams(struct nvme_ctrl *ctrl)
-{
-	return nvme_toggle_streams(ctrl, false);
-}
-
-static int nvme_enable_streams(struct nvme_ctrl *ctrl)
-{
-	return nvme_toggle_streams(ctrl, true);
-}
-
 static int nvme_get_stream_params(struct nvme_ctrl *ctrl,
 				  struct streams_directive_params *s, u32 nsid)
 {
@@ -739,7 +727,7 @@ static int nvme_configure_directives(struct nvme_ctrl *ctrl)
 	if (!streams)
 		return 0;
 
-	ret = nvme_enable_streams(ctrl);
+	ret = nvme_ctrl_allow_streams(ctrl, true);
 	if (ret)
 		return ret;
 
@@ -759,7 +747,7 @@ static int nvme_configure_directives(struct nvme_ctrl *ctrl)
 	return 0;
 
 out_disable_stream:
-	nvme_disable_streams(ctrl);
+	nvme_ctrl_allow_streams(ctrl, false);
 	return ret;
 }
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:29 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:29 -0800
Subject: [PATCH 11/14] nvme-core: use likely in nvme_init_request()
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-12-chaitanya.kulkarni@wdc.com>

For NVMeOF Target passthru backend we allocate the passthru request with
nvme_alloc_request(). The functions :-

nvmet_passthru_execute_cmd()
 nvme_alloc_request()
  nvme_init_request()

are in the fast path for I/O commands.

In nvme_init_request() we set the timeout based on the
req->q->queuedata check & that is always true for the I/O commands
coming from the passthru backend which are high frequency commands.

Annotate req->q->queuedata with likely() in nvme_init_request().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 139cbeb11c7d..a0d0c3a5abff 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -652,7 +652,7 @@ static inline unsigned int nvme_req_op(struct nvme_command *cmd)
 static inline void nvme_init_request(struct request *req,
 		struct nvme_command *cmd)
 {
-	if (req->q->queuedata)
+	if (likely(req->q->queuedata))
 		req->timeout = NVME_IO_TIMEOUT;
 	else /* no queuedata implies admin queue */
 		req->timeout = NVME_ADMIN_TIMEOUT;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:30 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:30 -0800
Subject: [PATCH 12/14] nvme-core: don't check nvme_req flags for new req
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-13-chaitanya.kulkarni@wdc.com>

nvme_clear_request() has a check for flag REQ_DONTPREP and it is called
from nvme_init_request() and nvme_setuo_cmd().

The function nvme_init_request() is called from nvme_alloc_request()
and nvme_alloc_request_qid(). From these two callers new request is
allocated everytime. For newly allocated request RQF_DONTPREP is never
set. Since after getting a tag, block layer sets the req->rq_flags == 0
and never sets the REQ_DONTPREP when returning the request :-

nvme_alloc_request()
	blk_mq_alloc_request()
		blk_mq_rq_ctx_init()
			rq->rq_flags = 0 <----

nvme_alloc_request_qid()
	blk_mq_alloc_request_hctx()
		blk_mq_rq_ctx_init()
			rq->rq_flags = 0 <----

The block layer does set req->rq_flags but REQ_DONTPREP is not one of
them and that is set by the driver.

That means we can unconditinally set the REQ_DONTPREP value to the
rq->rq_flags when nvme_init_request()->nvme_clear_request() is called
from above two callers.

Move the check for REQ_DONTPREP from nvme_clear_nvme_request() into
nvme_setup_cmd().

This is needed since nvme_alloc_request() now gets called from fast
path when NVMeOF target is?configured with passthru backend to avoid
unnecessary checks in the fast path.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index a0d0c3a5abff..608462df2c39 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -637,11 +637,9 @@ EXPORT_SYMBOL_NS_GPL(nvme_put_ns, NVME_TARGET_PASSTHRU);
 
 static inline void nvme_clear_nvme_request(struct request *req)
 {
-	if (!(req->rq_flags & RQF_DONTPREP)) {
-		nvme_req(req)->retries = 0;
-		nvme_req(req)->flags = 0;
-		req->rq_flags |= RQF_DONTPREP;
-	}
+	nvme_req(req)->retries = 0;
+	nvme_req(req)->flags = 0;
+	req->rq_flags |= RQF_DONTPREP;
 }
 
 static inline unsigned int nvme_req_op(struct nvme_command *cmd)
@@ -943,7 +941,8 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req,
 {
 	blk_status_t ret = BLK_STS_OK;
 
-	nvme_clear_nvme_request(req);
+	if (!(req->rq_flags & RQF_DONTPREP))
+		nvme_clear_nvme_request(req);
 
 	memset(cmd, 0, sizeof(*cmd));
 	switch (req_op(req)) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:31 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:31 -0800
Subject: [PATCH 13/14] nvme-fc: fix the function documentation comment
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-14-chaitanya.kulkarni@wdc.com>

The nvme_fc_rcv_ls_req() function has first argument as pointer to
remoteport named portprt, but in the documentation comment that is name
is used as remoteport. Fix that to get rid if the compilation warning.

drivers/nvme//host/fc.c:1724: warning: Function parameter or member 'portptr' not described in 'nvme_fc_rcv_ls_req'
drivers/nvme//host/fc.c:1724: warning: Excess function parameter 'remoteport' description in 'nvme_fc_rcv_ls_req'
---
 drivers/nvme/host/fc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 20dadd86e981..d894d3eebadf 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -1708,7 +1708,7 @@ nvme_fc_handle_ls_rqst_work(struct work_struct *work)
  *
  * If this routine returns error, the LLDD should abort the exchange.
  *
- * @remoteport: pointer to the (registered) remote port that the LS
+ * @portptr:    pointer to the (registered) remote port that the LS
  *              was received from. The remoteport is associated with
  *              a specific localport.
  * @lsrsp:      pointer to a nvmefc_ls_rsp response structure to be
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 16 19:10:32 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 16 Feb 2021 16:10:32 -0800
Subject: [PATCH 14/14] nvmet-fc: update function documentation
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210217001032.25741-15-chaitanya.kulkarni@wdc.com>

Add minimum description of the hosthandle parameter for
nvmet_fc_rcv_ls_req() so that we can get rid of the following warning.

drivers/nvme//target/fc.c:2009: warning: Function parameter or member 'hosthandle' not described in 'nvmet_fc_rcv_ls_req
---
 drivers/nvme/target/fc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/target/fc.c b/drivers/nvme/target/fc.c
index d375745fc4ed..1f1c70f9f8eb 100644
--- a/drivers/nvme/target/fc.c
+++ b/drivers/nvme/target/fc.c
@@ -1996,6 +1996,7 @@ nvmet_fc_handle_ls_rqst_work(struct work_struct *work)
  *
  * @target_port: pointer to the (registered) target port the LS was
  *              received on.
+ * @hosthandle: pointer to the host specific data, gets stored in iod.
  * @lsrsp:      pointer to a lsrsp structure to be used to reference
  *              the exchange corresponding to the LS.
  * @lsreqbuf:   pointer to the buffer containing the LS Request
-- 
2.22.1



From minwoo.im.dev at gmail.com  Tue Feb 16 21:14:55 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Wed, 17 Feb 2021 11:14:55 +0900
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210216175722.GC2708768@dhcp-10-100-145-180.wdc.com>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
 <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
 <20210216095147.GA2601@localhost.localdomain>
 <20210216175722.GC2708768@dhcp-10-100-145-180.wdc.com>
Message-ID: <20210217021455.GA2251@localhost.localdomain>

On 21-02-16 09:57:22, Keith Busch wrote:
> On Tue, Feb 16, 2021 at 06:51:47PM +0900, Minwoo Im wrote:
> > On 21-02-15 09:02:33, Keith Busch wrote:
> > > On Sun, Feb 14, 2021 at 08:01:25PM +0900, Minwoo Im wrote:
> > > > We don't allow NVME_IOCTL_IO_CMD ioctl in case that a controller has
> > > > multiple namespaces attached.  Also, I/O request to the controller
> > > > character device has not been recommended and deprecated because we have
> > > > block device to I/O with where the multipath consideration is taken.
> > > > 
> > > > Once kernel decided a path to I/O for a namespace based on the I/O
> > > > policy of a NVMe subsystem, userspace is not allowed to choose a path to
> > > > I/O.  If a path is broken(inaccessible state in ANA), then it will not
> > > > try to I/O to that path.
> > > > 
> > > > This patch introduced NVME_IOCTL_MPATH_IO command for controller
> > > > device(e.g., /dev/nvme0) to support multipath I/O passthrough for
> > > > userspace.  Regardless driver's path decision, userspace can target a
> > > > namespace to I/O.  In this case, `cmd.nsid` will be used to find out the
> > > > namespace instance target which is hidden(e.g., nvmeXcYnZ).
> > > 
> > > IO commands are not allowed through the character handle with the
> > > existing ioctls. A new ioctl doesn't make it okay. If it was okay, then
> > > we could just remove the limitation in the current ones.
> > 
> > Thanks for your feedback, Keith.  If you don't mind, may I ask why it's
> > been entirely unsafe and deprecated in the exsiting
> > ioctl(NVME_IOCTL_IO_CMD)?  I've seen a patch bfd8947194b2 ("nvme: fixes
> > for NVME_IOCTL_IO_CMD on the char device), but have no idea why it's
> > been really depreacted :)
> 
> A container could read data from namespaces assigned to a different
> container.

Now I got your point! Thanks Keith, please ignore this patch :)


From hkallweit1 at gmail.com  Wed Feb 17 02:26:30 2021
From: hkallweit1 at gmail.com (Heiner Kallweit)
Date: Wed, 17 Feb 2021 08:26:30 +0100
Subject: [PATCH] nvme: use lighter smp barriers in nvme_irq
In-Reply-To: <BYAPR04MB4965D222B05866D93852835F86879@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <1c90fd6a-d991-b347-479c-2c8ae5a4c51f@gmail.com>
 <BYAPR04MB4965D222B05866D93852835F86879@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <be263cdf-19a2-05d6-1023-7d935d526663@gmail.com>

On 17.02.2021 00:59, Chaitanya Kulkarni wrote:
> On 2/14/21 08:30, Heiner Kallweit wrote:
>> Based on the description we should be fine using the less heavy-weight
>> smp barriers here. On x86 this would be compiler barriers only.
>>
>> Signed-off-by: Heiner Kallweit <hkallweit1 at gmail.com>
> Can you share performance numbers ?
> 

I stumbled across this code piece by chance and don't have hw to test it.
Having said that the change is based on code inspection only.
Unfortunately the barrier comment is very generic and doesn't mention
a problematic scenario. Also the commit message of 3a7afd8ee42a doesn't
mention a potential race. Most likely the barrier can be removed
completely. Helpful would be if someone could explain the potential race
in detail, means which reordering / variable accesses could be racy.
Then we would have a basis to talk about READ_ONCE/WRITE_ONCE vs.
smp barrier vs. dma barrier vs. full barrier.


From gerlitz.or at gmail.com  Wed Feb 17 08:55:51 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Wed, 17 Feb 2021 15:55:51 +0200
Subject: [PATCH v4 net-next 06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <2dd10b2f-df00-e21c-7886-93f41a987040@gmail.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-7-borisp@mellanox.com>
 <2dd10b2f-df00-e21c-7886-93f41a987040@gmail.com>
Message-ID: <CAJ3xEMjMqK81uNv21poD+DoZCRxYak0DYFZrjDbmWaSxw4R5ig@mail.gmail.com>

On Sun, Feb 14, 2021 at 8:20 PM David Ahern <dsahern at gmail.com> wrote:
> On 2/11/21 2:10 PM, Boris Pismenny wrote:
> > @@ -223,6 +229,164 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
> >       return nvme_tcp_pdu_data_left(req) <= len;
> >  }
> >
> > +#ifdef CONFIG_TCP_DDP
> > +
> > +static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
> > +static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
> > +     .resync_request         = nvme_tcp_resync_request,
> > +};
> > +
> > +static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
> > +{
> > +     struct net_device *netdev = queue->ctrl->offloading_netdev;
> > +     struct nvme_tcp_ddp_config config = {};
> > +     int ret;
> > +
> > +     if (!(netdev->features & NETIF_F_HW_TCP_DDP))
>
> If nvme_tcp_offload_limits does not find a dst_entry on the socket then
> offloading_netdev may not NULL at this point.

correct :( will look on that

>
> > +             return -EOPNOTSUPP;
> > +
> > +     config.cfg.type         = TCP_DDP_NVME;
> > +     config.pfv              = NVME_TCP_PFV_1_0;
> > +     config.cpda             = 0;
> > +     config.dgst             = queue->hdr_digest ?
> > +             NVME_TCP_HDR_DIGEST_ENABLE : 0;
> > +     config.dgst             |= queue->data_digest ?
> > +             NVME_TCP_DATA_DIGEST_ENABLE : 0;
> > +     config.queue_size       = queue->queue_size;
> > +     config.queue_id         = nvme_tcp_queue_id(queue);
> > +     config.io_cpu           = queue->io_cpu;
> > +
> > +     dev_hold(netdev); /* put by unoffload_socket */
> > +     ret = netdev->tcp_ddp_ops->tcp_ddp_sk_add(netdev,
> > +                                               queue->sock->sk,
> > +                                               &config.cfg);
> > +     if (ret) {
> > +             dev_put(netdev);
> > +             return ret;
> > +     }
> > +
> > +     inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = &nvme_tcp_ddp_ulp_ops;
> > +     if (netdev->features & NETIF_F_HW_TCP_DDP)
> > +             set_bit(NVME_TCP_Q_OFF_DDP, &queue->flags);
> > +
> > +     return ret;
> > +}
> > +
> > +static void nvme_tcp_unoffload_socket(struct nvme_tcp_queue *queue)
> > +{
> > +     struct net_device *netdev = queue->ctrl->offloading_netdev;
> > +
> > +     if (!netdev) {
> > +             dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
>
> you are already logged in nvme_tcp_offload_limits that
> get_netdev_for_sock returned NULL; no need to do it again.

yeah, re this one and the few other places where you commented
on the same or similar thing, I tend to agree we need to go on the
kernel trusted programming paradigm and avoid over checking, will
discuss that with the team.


> > +             return;
> > +     }
> > +
> > +     netdev->tcp_ddp_ops->tcp_ddp_sk_del(netdev, queue->sock->sk);
> > +
> > +     inet_csk(queue->sock->sk)->icsk_ulp_ddp_ops = NULL;
> > +     dev_put(netdev); /* held by offload_socket */
> > +}
> > +
> > +static int nvme_tcp_offload_limits(struct nvme_tcp_queue *queue)
> > +{
> > +     struct net_device *netdev = get_netdev_for_sock(queue->sock->sk, true);
> > +     struct tcp_ddp_limits limits;
> > +     int ret = 0;
> > +
> > +     if (!netdev) {
> > +             dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
>
> This should be more informative.

okk

> > +             queue->ctrl->offloading_netdev = NULL;
> > +             return -ENODEV;
> > +     }
> > +
> > +     if (netdev->features & NETIF_F_HW_TCP_DDP &&
> > +         netdev->tcp_ddp_ops &&
> > +         netdev->tcp_ddp_ops->tcp_ddp_limits)
> > +             ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
> > +     else
> > +             ret = -EOPNOTSUPP;
> > +
> > +     if (!ret) {
> > +             queue->ctrl->offloading_netdev = netdev;
>
> you save a reference to the netdev here, but then release the refcnt
> below. That device could be deleted between this point in time and the
> initialization of all queues.

> > +             dev_dbg_ratelimited(queue->ctrl->ctrl.device,
> > +                                 "netdev %s offload limits: max_ddp_sgl_len %d\n",
> > +                                 netdev->name, limits.max_ddp_sgl_len);
> > +             queue->ctrl->ctrl.max_segments = limits.max_ddp_sgl_len;
> > +             queue->ctrl->ctrl.max_hw_sectors =
> > +                     limits.max_ddp_sgl_len << (ilog2(SZ_4K) - 9);
> > +     } else {
> > +             queue->ctrl->offloading_netdev = NULL;
> > +     }
> > +
> > +     /* release the device as no offload context is established yet. */
> > +     dev_put(netdev);
> > +
> > +     return ret;
> > +}
> > +
> > +static void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
> > +                                  struct sk_buff *skb, unsigned int offset)
> > +{
> > +     u64 pdu_seq = TCP_SKB_CB(skb)->seq + offset - queue->pdu_offset;
> > +     struct net_device *netdev = queue->ctrl->offloading_netdev;
> > +     u64 resync_val;
> > +     u32 resync_seq;
> > +
> > +     resync_val = atomic64_read(&queue->resync_req);
> > +     /* Lower 32 bit flags. Check validity of the request */
> > +     if ((resync_val & TCP_DDP_RESYNC_REQ) == 0)
> > +             return;
> > +
> > +     /* Obtain and check requested sequence number: is this PDU header before the request? */
> > +     resync_seq = resync_val >> 32;
> > +     if (before(pdu_seq, resync_seq))
> > +             return;
> > +
> > +     if (unlikely(!netdev)) {
> > +             pr_info_ratelimited("%s: netdev not found\n", __func__);
>
> can't happen right? you get here because NVME_TCP_Q_OFF_DDP is set and
> it is only set if offloading_netdev is set and the device supports offload.

as I wrote above, will discuss this general comment and likely
go the direction you are pointing on


From gerlitz.or at gmail.com  Wed Feb 17 09:01:14 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Wed, 17 Feb 2021 16:01:14 +0200
Subject: [PATCH v4 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <cfd61c5a-c5fd-e0d9-fb60-be93f1c98735@gmail.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-8-borisp@mellanox.com>
 <cfd61c5a-c5fd-e0d9-fb60-be93f1c98735@gmail.com>
Message-ID: <CAJ3xEMgZg9dFyc8cnbuPPAFT3jd2k27TdLOz-vtVmxy9k9zHcg@mail.gmail.com>

On Sun, Feb 14, 2021 at 8:30 PM David Ahern <dsahern at gmail.com> wrote:
>
> On 2/11/21 2:10 PM, Boris Pismenny wrote:
> >
> > +static int nvme_tcp_teardown_ddp(struct nvme_tcp_queue *queue,
> > +                              u16 command_id,
> > +                              struct request *rq)
> > +{
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +     struct net_device *netdev = queue->ctrl->offloading_netdev;
> > +     int ret;
> > +
> > +     if (unlikely(!netdev)) {
> > +             dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
>
> again, unnecessary. you only get here because the rquest is marked
> offloaded and that only happens if the netdev exists and supports DDP.
>
> > +             return -EINVAL;
> > +     }
> > +
> > +     ret = netdev->tcp_ddp_ops->tcp_ddp_teardown(netdev, queue->sock->sk,
> > +                                                 &req->ddp, rq);
> > +     sg_free_table_chained(&req->ddp.sg_table, SG_CHUNK_SIZE);
> > +     return ret;
> > +}
> > +
> > +static void nvme_tcp_ddp_teardown_done(void *ddp_ctx)
> > +{
> > +     struct request *rq = ddp_ctx;
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +
> > +     if (!nvme_try_complete_req(rq, req->status, req->result))
> > +             nvme_complete_rq(rq);
> > +}
> > +
> > +static int nvme_tcp_setup_ddp(struct nvme_tcp_queue *queue,
> > +                           u16 command_id,
> > +                           struct request *rq)
> > +{
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +     struct net_device *netdev = queue->ctrl->offloading_netdev;
> > +     int ret;
> > +
> > +     if (unlikely(!netdev)) {
> > +             dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
>
> similarly here. you can't get here if netdev is null.
>
> > +             return -EINVAL;
> > +     }
> > +
> > +     req->ddp.command_id = command_id;
> > +     ret = nvme_tcp_req_map_sg(req, rq);
> > +     if (ret)
> > +             return -ENOMEM;
> > +
> > +     ret = netdev->tcp_ddp_ops->tcp_ddp_setup(netdev,
> > +                                              queue->sock->sk,
> > +                                              &req->ddp);
> > +     if (!ret)
> > +             req->offloaded = true;
> > +     return ret;
> > +}
> > +
> >  static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
> >  {
> >       struct net_device *netdev = queue->ctrl->offloading_netdev;
> > @@ -343,7 +417,7 @@ static void nvme_tcp_resync_response(struct nvme_tcp_queue *queue,
> >               return;
> >
> >       if (unlikely(!netdev)) {
> > -             pr_info_ratelimited("%s: netdev not found\n", __func__);
> > +             dev_info_ratelimited(queue->ctrl->ctrl.device, "netdev not found\n");
>
> and per comment on the last patch, this is not needed.
>
> > @@ -849,10 +953,39 @@ static int nvme_tcp_recv_pdu(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> >
> >  static inline void nvme_tcp_end_request(struct request *rq, u16 status)
> >  {
> > +     struct nvme_tcp_request *req = blk_mq_rq_to_pdu(rq);
> > +     struct nvme_tcp_queue *queue = req->queue;
> > +     struct nvme_tcp_data_pdu *pdu = (void *)queue->pdu;
> >       union nvme_result res = {};
> >
> > -     if (!nvme_try_complete_req(rq, cpu_to_le16(status << 1), res))
> > -             nvme_complete_rq(rq);
> > +     nvme_tcp_complete_request(rq, cpu_to_le16(status << 1), res, pdu->command_id);
> > +}
> > +
> > +
> > +static int nvme_tcp_consume_skb(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> > +                             unsigned int *offset, struct iov_iter *iter, int recv_len)
> > +{
> > +     int ret;
> > +
> > +#ifdef CONFIG_TCP_DDP
> > +     if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags)) {
> > +             if (queue->data_digest)
> > +                     ret = skb_ddp_copy_and_hash_datagram_iter(skb, *offset, iter, recv_len,
> > +                                     queue->rcv_hash);
> > +             else
> > +                     ret = skb_ddp_copy_datagram_iter(skb, *offset, iter, recv_len);
> > +     } else {
> > +#endif
>
> why not make that a helper defined in the CONFIG_TCP_DDP section with an
> inline for the unset case. Keeps this code from being polluted with the
> ifdef checks.

will check that

>
> > +             if (queue->data_digest)
> > +                     ret = skb_copy_and_hash_datagram_iter(skb, *offset, iter, recv_len,
> > +                                     queue->rcv_hash);
> > +             else
> > +                     ret = skb_copy_datagram_iter(skb, *offset, iter, recv_len);
> > +#ifdef CONFIG_TCP_DDP
> > +     }
> > +#endif
> > +
> > +     return ret;
> >  }
> >
> >  static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> > @@ -899,12 +1032,7 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *queue, struct sk_buff *skb,
> >               recv_len = min_t(size_t, recv_len,
> >                               iov_iter_count(&req->iter));
> >
> > -             if (queue->data_digest)
> > -                     ret = skb_copy_and_hash_datagram_iter(skb, *offset,
> > -                             &req->iter, recv_len, queue->rcv_hash);
> > -             else
> > -                     ret = skb_copy_datagram_iter(skb, *offset,
> > -                                     &req->iter, recv_len);
> > +             ret = nvme_tcp_consume_skb(queue, skb, offset, &req->iter, recv_len);
> >               if (ret) {
> >                       dev_err(queue->ctrl->ctrl.device,
> >                               "queue %d failed to copy request %#x data",
> > @@ -1128,6 +1256,7 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
> >       bool inline_data = nvme_tcp_has_inline_data(req);
> >       u8 hdgst = nvme_tcp_hdgst_len(queue);
> >       int len = sizeof(*pdu) + hdgst - req->offset;
> > +     struct request *rq = blk_mq_rq_from_pdu(req);
> >       int flags = MSG_DONTWAIT;
> >       int ret;
> >
> > @@ -1136,6 +1265,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
> >       else
> >               flags |= MSG_EOR;
> >
> > +     if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) &&
> > +         blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
> > +             nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);
> > +
>
> For consistency, shouldn't this be wrapped in the CONFIG_TCP_DDP check too?

We tried to avoid the wrapping in some places where it was
possible to do without adding confusion, this one is a good
example IMOH.


> >       if (queue->hdr_digest && !req->offset)
> >               nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu));
> >


From dsahern at gmail.com  Wed Feb 17 12:00:59 2021
From: dsahern at gmail.com (David Ahern)
Date: Wed, 17 Feb 2021 10:00:59 -0700
Subject: [PATCH v4 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <CAJ3xEMgZg9dFyc8cnbuPPAFT3jd2k27TdLOz-vtVmxy9k9zHcg@mail.gmail.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-8-borisp@mellanox.com>
 <cfd61c5a-c5fd-e0d9-fb60-be93f1c98735@gmail.com>
 <CAJ3xEMgZg9dFyc8cnbuPPAFT3jd2k27TdLOz-vtVmxy9k9zHcg@mail.gmail.com>
Message-ID: <f9c45756-b823-7cab-213c-761866230c96@gmail.com>

On 2/17/21 7:01 AM, Or Gerlitz wrote:
>>> @@ -1136,6 +1265,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>>>       else
>>>               flags |= MSG_EOR;
>>>
>>> +     if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) &&
>>> +         blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
>>> +             nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);
>>> +
>>
>> For consistency, shouldn't this be wrapped in the CONFIG_TCP_DDP check too?
> 
> We tried to avoid the wrapping in some places where it was
> possible to do without adding confusion, this one is a good
> example IMOH.
> 
> 

The above (and other locations like it) can easily be put into a helper
that has logic when the CONFIG is enabled and compiles out when not.
Consistency makes for simpler, cleaner code for optional features.


From mgurtovoy at nvidia.com  Wed Feb 17 13:11:53 2021
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Wed, 17 Feb 2021 18:11:53 +0000
Subject: [PATCH 1/1] nvmet: allow setting model_number once
Message-ID: <20210217181153.15343-1-mgurtovoy@nvidia.com>

In case we have already established connection to nvmf target, it
shouldn't be allowed to change the model_number. E.g. if someone will
identify ctrl and get model_number of "my_model" later on will change
the model_numbel via configfs to "my_new_model" this will break the NVM
specification for "Get Log Page ? Persistent Event Log" that refers to
Model Number as: "This field contains the same value as reported in the
Model Number field of the Identify Controller data structure, bytes
63:24."

Although it doesn't mentioned explicitly that this field can't be
changed, we can assume it.

So allow setting this field only once: using configfs or in the first
identify ctrl operation.

Signed-off-by: Max Gurtovoy <mgurtovoy at nvidia.com>
---
 drivers/nvme/target/admin-cmd.c | 33 +++++++++++++------
 drivers/nvme/target/configfs.c  | 57 ++++++++++++++++++---------------
 drivers/nvme/target/core.c      |  2 +-
 drivers/nvme/target/nvmet.h     |  7 +---
 4 files changed, 57 insertions(+), 42 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index dc1ea468b182..6a3341b2091f 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -319,18 +319,25 @@ static void nvmet_execute_get_log_page(struct nvmet_req *req)
 	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_SC_DNR);
 }
 
-static void nvmet_id_set_model_number(struct nvme_id_ctrl *id,
+static int nvmet_id_set_model_number(struct nvme_id_ctrl *id,
 				      struct nvmet_subsys *subsys)
 {
-	const char *model = NVMET_DEFAULT_CTRL_MODEL;
-	struct nvmet_subsys_model *subsys_model;
+	int ret = 0;
 
-	rcu_read_lock();
-	subsys_model = rcu_dereference(subsys->model);
-	if (subsys_model)
-		model = subsys_model->number;
-	memcpy_and_pad(id->mn, sizeof(id->mn), model, strlen(model), ' ');
-	rcu_read_unlock();
+	mutex_lock(&subsys->lock);
+	if (!subsys->model_number) {
+		subsys->model_number = kstrdup(NVMET_DEFAULT_CTRL_MODEL,
+					       GFP_KERNEL);
+		if (!subsys->model_number) {
+			ret = -ENOMEM;
+			goto out_unlock;
+		}
+	}
+	memcpy_and_pad(id->mn, sizeof(id->mn), subsys->model_number,
+		       strlen(subsys->model_number), ' ');
+out_unlock:
+	mutex_unlock(&subsys->lock);
+	return ret;
 }
 
 static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
@@ -339,6 +346,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 	struct nvme_id_ctrl *id;
 	u32 cmd_capsule_size;
 	u16 status = 0;
+	int ret;
 
 	id = kzalloc(sizeof(*id), GFP_KERNEL);
 	if (!id) {
@@ -353,7 +361,11 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 	memset(id->sn, ' ', sizeof(id->sn));
 	bin2hex(id->sn, &ctrl->subsys->serial,
 		min(sizeof(ctrl->subsys->serial), sizeof(id->sn) / 2));
-	nvmet_id_set_model_number(id, ctrl->subsys);
+	ret = nvmet_id_set_model_number(id, ctrl->subsys);
+	if (ret) {
+		status = NVME_SC_INTERNAL;
+		goto out_free;
+	}
 	memcpy_and_pad(id->fr, sizeof(id->fr),
 		       UTS_RELEASE, strlen(UTS_RELEASE), ' ');
 
@@ -461,6 +473,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 
 	status = nvmet_copy_to_sgl(req, 0, id, sizeof(*id));
 
+out_free:
 	kfree(id);
 out:
 	nvmet_req_complete(req, status);
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index c61ffd767062..ad1adcf30ee5 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -1120,16 +1120,14 @@ static ssize_t nvmet_subsys_attr_model_show(struct config_item *item,
 					    char *page)
 {
 	struct nvmet_subsys *subsys = to_subsys(item);
-	struct nvmet_subsys_model *subsys_model;
-	char *model = NVMET_DEFAULT_CTRL_MODEL;
+	char *model = "";
 	int ret;
 
-	rcu_read_lock();
-	subsys_model = rcu_dereference(subsys->model);
-	if (subsys_model)
-		model = subsys_model->number;
+	mutex_lock(&subsys->lock);
+	if (subsys->model_number)
+		model = subsys->model_number;
 	ret = snprintf(page, PAGE_SIZE, "%s\n", model);
-	rcu_read_unlock();
+	mutex_unlock(&subsys->lock);
 
 	return ret;
 }
@@ -1144,40 +1142,49 @@ static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
 					     const char *page, size_t count)
 {
 	struct nvmet_subsys *subsys = to_subsys(item);
-	struct nvmet_subsys_model *new_model;
 	char *new_model_number;
 	int pos = 0, len;
 
+	down_write(&nvmet_config_sem);
+	mutex_lock(&subsys->lock);
+	if (subsys->model_number) {
+		pr_err("Can't set model number. %s is already assigned\n",
+		       subsys->model_number);
+		count = -EINVAL;
+		goto out_unlock;
+	}
+
 	len = strcspn(page, "\n");
-	if (!len)
-		return -EINVAL;
+	if (!len) {
+		count = -EINVAL;
+		goto out_unlock;
+	}
 
 	for (pos = 0; pos < len; pos++) {
-		if (!nvmet_is_ascii(page[pos]))
-			return -EINVAL;
+		if (!nvmet_is_ascii(page[pos])) {
+			count = -EINVAL;
+			goto out_unlock;
+		}
 	}
 
 	new_model_number = kmemdup_nul(page, len, GFP_KERNEL);
-	if (!new_model_number)
-		return -ENOMEM;
+	if (!new_model_number) {
+		count = -ENOMEM;
+		goto out_unlock;
+	}
 
-	new_model = kzalloc(sizeof(*new_model) + len + 1, GFP_KERNEL);
-	if (!new_model) {
+	subsys->model_number = kstrdup(new_model_number, GFP_KERNEL);
+	if (!subsys->model_number) {
+		count = -ENOMEM;
 		kfree(new_model_number);
-		return -ENOMEM;
+		goto out_unlock;
 	}
-	memcpy(new_model->number, new_model_number, len);
 
-	down_write(&nvmet_config_sem);
-	mutex_lock(&subsys->lock);
-	new_model = rcu_replace_pointer(subsys->model, new_model,
-					mutex_is_locked(&subsys->lock));
+	kfree(new_model_number);
+out_unlock:
 	mutex_unlock(&subsys->lock);
 	up_write(&nvmet_config_sem);
 
-	kfree_rcu(new_model, rcuhead);
-	kfree(new_model_number);
-
 	return count;
 }
 CONFIGFS_ATTR(nvmet_subsys_, attr_model);
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 8ce4d59cc9e7..c7af907912f2 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1521,7 +1521,7 @@ static void nvmet_subsys_free(struct kref *ref)
 	nvmet_passthru_subsys_free(subsys);
 
 	kfree(subsys->subsysnqn);
-	kfree_rcu(subsys->model, rcuhead);
+	kfree(subsys->model_number);
 	kfree(subsys);
 }
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 592763732065..aac741bf378c 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -208,11 +208,6 @@ struct nvmet_ctrl {
 	bool			pi_support;
 };
 
-struct nvmet_subsys_model {
-	struct rcu_head		rcuhead;
-	char			number[];
-};
-
 struct nvmet_subsys {
 	enum nvme_subsys_type	type;
 
@@ -242,7 +237,7 @@ struct nvmet_subsys {
 	struct config_group	namespaces_group;
 	struct config_group	allowed_hosts_group;
 
-	struct nvmet_subsys_model	__rcu *model;
+	char			*model_number;
 
 #ifdef CONFIG_NVME_TARGET_PASSTHRU
 	struct nvme_ctrl	*passthru_ctrl;
-- 
2.25.4



From kbusch at kernel.org  Wed Feb 17 15:23:34 2021
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 18 Feb 2021 05:23:34 +0900
Subject: [PATCH] nvme-pci: mark Seagate Nytro XM1440 as
 QUIRK_NO_NS_DESC_LIST.
In-Reply-To: <38eae52a15c5474dbdf85defcf6a29fa@marcapo.com>
References: <38eae52a15c5474dbdf85defcf6a29fa@marcapo.com>
Message-ID: <20210217202334.GB25688@redsun51.ssa.fujisawa.hgst.com>

On Wed, Feb 17, 2021 at 07:33:32AM +0000, Julian Einwag wrote:
> The kernel fails to fully detect these SSDs, only the character devices
> are present:
> 
> [   10.785605] nvme nvme0: pci function 0000:04:00.0
> [   10.876787] nvme nvme1: pci function 0000:81:00.0
> [   13.198614] nvme nvme0: missing or invalid SUBNQN field.
> [   13.198658] nvme nvme1: missing or invalid SUBNQN field.
> [   13.206896] nvme nvme0: Shutdown timeout set to 20 seconds
> [   13.215035] nvme nvme1: Shutdown timeout set to 20 seconds
> [   13.225407] nvme nvme0: 16/0/0 default/read/poll queues
> [   13.233602] nvme nvme1: 16/0/0 default/read/poll queues
> [   13.239627] nvme nvme0: Identify Descriptors failed (8194)
> [   13.246315] nvme nvme1: Identify Descriptors failed (8194)
> 
> Adding the NVME_QUIRK_NO_NS_DESC_LIST fixes this problem.
> 
> BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=205679
> 
> Signed-off-by: Julian Einwag <jeinwag-nvme at marcapo.com>

Looks fine.

Reviewed-by: Keith Busch <kbusch at kernel.org>

> ---
>  drivers/nvme/host/pci.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 0045c5edf629..91cefa40543f 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -3233,7 +3233,8 @@ static const struct pci_device_id nvme_id_table[] = {
>         { PCI_DEVICE(0x126f, 0x2263),   /* Silicon Motion unidentified */
>                 .driver_data = NVME_QUIRK_NO_NS_DESC_LIST, },
>         { PCI_DEVICE(0x1bb1, 0x0100),   /* Seagate Nytro Flash Storage */
> -               .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
> +               .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY |
> +                               NVME_QUIRK_NO_NS_DESC_LIST, },
>         { PCI_DEVICE(0x1c58, 0x0003),   /* HGST adapter */
>                 .driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
>         { PCI_DEVICE(0x1c58, 0x0023),   /* WDC SN200 adapter */
> -- 
> 2.30.0


From Martin.Belanger at dell.com  Wed Feb 17 17:14:49 2021
From: Martin.Belanger at dell.com (Belanger, Martin)
Date: Wed, 17 Feb 2021 22:14:49 +0000
Subject: How to identify the device created by "nvme discover --persistent"
Message-ID: <SJ0PR19MB4544BD8195099696F434E8D3F2869@SJ0PR19MB4544.namprd19.prod.outlook.com>

When I use "nvme discover --peristent ..." to create a persistent connection to a discover controller, is there a way to identify the device (i.e. /dev/nvme?) associated with that connection?

If I later want to re-use that connection (e.g. nvme discover --device ..., or nvme disconnect --device ...), I will need to know what value to provide to the "--device" option.

Thanks
Martin

From sagi at grimberg.me  Wed Feb 17 18:26:52 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 17 Feb 2021 15:26:52 -0800
Subject: nvme-tcp bricks my computer
In-Reply-To: <SJ0PR19MB45441A133999DA90186BCC11F2869@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45443358535A404035588EC5F2B69@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <SJ0PR19MB4544B86790800C6371E7F96AF2B49@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <ccb20293-efdf-ee96-d97f-405ced9b67a5@grimberg.me>
 <69e0073c-35cd-552d-eccd-2092ab49c38d@grimberg.me>
 <SJ0PR19MB4544A3DF2B1B2A1EEB8D1B41F2B39@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <43ca75a5-e751-6b9b-2a0d-336dbbc207a1@grimberg.me>
 <2251aa94-0d6c-1405-b1d8-185bcd714d8c@grimberg.me>
 <SJ0PR19MB45442A0B8E171B55A8AACACEF2B29@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <8937148e-f667-9a05-3828-58114dca6da3@grimberg.me>
 <SJ0PR19MB4544A653D3A84A19A28D3984F28E9@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <1a430561-498c-70dd-f14b-8fab10673f65@grimberg.me>
 <SJ0PR19MB4544CAB0A539F84E54EC5208F2889@SJ0PR19MB4544.namprd19.prod.outlook.com>
 <7f5ff08a-36f0-3b67-9753-20eedd4b6f4d@grimberg.me>
 <8bcaaef7-76b3-0c57-d343-1581807a6d02@grimberg.me>
 <SJ0PR19MB45441A133999DA90186BCC11F2869@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <8e482420-f9da-f458-f6d3-93ee1974f0de@grimberg.me>


> Hi Sagi,
> 
> Our kernel expert was able to try the patch you provided. This is his 
> comment:
> 
>     /[the patch] fixes the panic, but it?ll take the controller down so
>     it's unsable.? If we want it to ignore the error, it will need to be
>     a little different./

Thanks for verifying it fixes the crash, not sure we want to ignore this
error, while its not prohibited by the spec to send a 0-length R2T
(although it probably should be), the controller is obviously
buggy and we should probably stop talking to it..

I'll send a formal patch, thanks!


From Chaitanya.Kulkarni at wdc.com  Wed Feb 17 20:41:27 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 18 Feb 2021 01:41:27 +0000
Subject: [PATCH] nvme: use lighter smp barriers in nvme_irq
References: <1c90fd6a-d991-b347-479c-2c8ae5a4c51f@gmail.com>
 <BYAPR04MB4965D222B05866D93852835F86879@BYAPR04MB4965.namprd04.prod.outlook.com>
 <be263cdf-19a2-05d6-1023-7d935d526663@gmail.com>
Message-ID: <BYAPR04MB496552C50E0B69EDEC5880B686859@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/16/21 23:26, Heiner Kallweit wrote:
> On 17.02.2021 00:59, Chaitanya Kulkarni wrote:
>> On 2/14/21 08:30, Heiner Kallweit wrote:
>>> Based on the description we should be fine using the less heavy-weight
>>> smp barriers here. On x86 this would be compiler barriers only.
>>>
>>> Signed-off-by: Heiner Kallweit <hkallweit1 at gmail.com>
>> Can you share performance numbers ?
>>
> I stumbled across this code piece by chance and don't have hw to test it.
> Having said that the change is based on code inspection only.
> Unfortunately the barrier comment is very generic and doesn't mention
> a problematic scenario. Also the commit message of 3a7afd8ee42a doesn't
> mention a potential race. Most likely the barrier can be removed
> completely. Helpful would be if someone could explain the potential race
> in detail, means which reordering / variable accesses could be racy.
> Then we would have a basis to talk about READ_ONCE/WRITE_ONCE vs.
> smp barrier vs. dma barrier vs. full barrier.
>

For these question you need to add commit Author to original email, as
far as
I know any performance improvement needs to be backed by data, unless there
is an exception, in this case I'm not if it is.



From chaitanya.kulkarni at wdc.com  Wed Feb 17 22:16:44 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 17 Feb 2021 19:16:44 -0800
Subject: [PATCH] nvmet: don't change model_number on live subsys
Message-ID: <20210218031644.12476-1-chaitanya.kulkarni@wdc.com>

When subsys is connected to the controller(s) right now user can change
the model_number. Add a check to make sure subsys is not connected
before we allow the change in model number from configfs.

Reported-by: Max Gurtovoy <mgurtovoy at nvidia.com> 
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---

Without this patch :-

# echo  "transport=loop,nqn=x" > /dev/nvme-fabrics
# nvme list | tr -s ' ' ' '
Node SN Model Namespace Usage Format FW Rev 
/dev/nvme1n1 d86964f000f47541 Linux 1 1.07 GB / 1.07 GB 512 B + 0 B 5.11.0-r
#
# echo "NEW MODEL" >  /sys/kernel/config/nvmet/subsystems/x/attr_model
# nvme list | tr -s ' ' ' '
Node SN Model Namespace Usage Format FW Rev 
/dev/nvme1n1 d86964f000f47541 NEW MODEL 1 1.07 GB / 1.07 GB 512 B + 0 B 5.11.0-r
                              ^^^^^^^^^
                               Problem above.
# nvme disconnect -n x
NQN:x disconnected 1 controller(s)

With this patch :-

# echo "transport=loop,nqn=x" > /dev/nvme-fabrics
# nvme list | tr -s ' ' ' ' 
Node SN Model Namespace Usage Format FW Rev 
/dev/nvme1n1 decfcb968deb004d Linux 1 1.07 GB / 1.07 GB 512 B + 0 B 5.11.0-r
# 
# echo "NEW MODEL" >  /sys/kernel/config/nvmet/subsystems/x/attr_model
bash: echo: write error: Invalid argument
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Don't allow new value for the model to be set when connected.

# nvme disconnect -n x
NQN:x disconnected 1 controller(s)
# echo "NEW MODEL" >  /sys/kernel/config/nvmet/subsystems/x/attr_model 
# echo  "transport=loop,nqn=x" > /dev/nvme-fabrics
# nvme list | tr -s ' ' ' ' 
Node SN Model Namespace Usage Format FW Rev 
/dev/nvme1n1 decfcb968deb004d NEW MODEL 1 1.07 GB / 1.07 GB 512 B + 0 B 5.11.0-r
---

 drivers/nvme/target/configfs.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 635a7cb45d0b..e0140a793d14 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -1138,6 +1138,21 @@ static bool nvmet_is_ascii(const char c)
 	return c >= 0x20 && c <= 0x7e;
 }
 
+static bool nvmet_subsys_is_connected(struct nvmet_subsys *subsys)
+{
+	struct nvmet_ctrl *ctrl;
+	bool connected = false;
+
+	mutex_lock(&subsys->lock);
+	list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
+		connected = true;
+		break;
+	}
+	mutex_unlock(&subsys->lock);
+
+	return connected;
+}
+
 static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
 					     const char *page, size_t count)
 {
@@ -1146,6 +1161,9 @@ static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
 	char *new_model_number;
 	int pos = 0, len;
 
+	if (nvmet_subsys_is_connected(subsys))
+		return -EINVAL;
+
 	len = strcspn(page, "\n");
 	if (!len)
 		return -EINVAL;
-- 
2.22.1



From Chaitanya.Kulkarni at wdc.com  Wed Feb 17 22:17:29 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 18 Feb 2021 03:17:29 +0000
Subject: [PATCH 1/1] nvmet: allow setting model_number once
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
Message-ID: <BYAPR04MB4965D3231E5681CD2642EDF686859@BYAPR04MB4965.namprd04.prod.outlook.com>

Max,

On 2/17/21 10:12, Max Gurtovoy wrote:
> In case we have already established connection to nvmf target, it
> shouldn't be allowed to change the model_number. E.g. if someone will
> identify ctrl and get model_number of "my_model" later on will change
> the model_numbel via configfs to "my_new_model" this will break the NVM
> specification for "Get Log Page ? Persistent Event Log" that refers to
> Model Number as: "This field contains the same value as reported in the
> Model Number field of the Identify Controller data structure, bytes
> 63:24."
We don't support persistent event log page since there is no use
case for target to have human readable format with timestamps,
instead we do maintain the error log page which has information
about the errors if user wants.

I'll let Christoph/Sagi decide if we really need to implement
Persistent Event (PE) log and so does this behavior.

Regarding preventing from changing the model_number when subsys is
connected to the host, that needs a fix and I sent a patch for that.

We can ignore that patch if we decide to go with this patch with
PR log page.

> Although it doesn't mentioned explicitly that this field can't be
> changed, we can assume it.
>
> So allow setting this field only once: using configfs or in the first
> identify ctrl operation.



From kbusch at kernel.org  Wed Feb 17 22:28:22 2021
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 17 Feb 2021 19:28:22 -0800
Subject: [PATCH] nvme: use lighter smp barriers in nvme_irq
In-Reply-To: <be263cdf-19a2-05d6-1023-7d935d526663@gmail.com>
References: <1c90fd6a-d991-b347-479c-2c8ae5a4c51f@gmail.com>
 <BYAPR04MB4965D222B05866D93852835F86879@BYAPR04MB4965.namprd04.prod.outlook.com>
 <be263cdf-19a2-05d6-1023-7d935d526663@gmail.com>
Message-ID: <20210218032822.GA2711165@dhcp-10-100-145-180.wdc.com>

On Wed, Feb 17, 2021 at 08:26:30AM +0100, Heiner Kallweit wrote:
> On 17.02.2021 00:59, Chaitanya Kulkarni wrote:
> > On 2/14/21 08:30, Heiner Kallweit wrote:
> >> Based on the description we should be fine using the less heavy-weight
> >> smp barriers here. On x86 this would be compiler barriers only.
> >>
> >> Signed-off-by: Heiner Kallweit <hkallweit1 at gmail.com>
> > Can you share performance numbers ?
> > 
> 
> I stumbled across this code piece by chance and don't have hw to test it.
> Having said that the change is based on code inspection only.
> Unfortunately the barrier comment is very generic and doesn't mention
> a problematic scenario. Also the commit message of 3a7afd8ee42a doesn't
> mention a potential race. Most likely the barrier can be removed
> completely. Helpful would be if someone could explain the potential race
> in detail, means which reordering / variable accesses could be racy.
> Then we would have a basis to talk about READ_ONCE/WRITE_ONCE vs.
> smp barrier vs. dma barrier vs. full barrier.

The variables the driver was protecting no longer exist, so I also agree
the barriers should not be necessary.


From Chaitanya.Kulkarni at wdc.com  Wed Feb 17 23:06:35 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 18 Feb 2021 04:06:35 +0000
Subject: [PATCH] nvme: use lighter smp barriers in nvme_irq
References: <1c90fd6a-d991-b347-479c-2c8ae5a4c51f@gmail.com>
 <BYAPR04MB4965D222B05866D93852835F86879@BYAPR04MB4965.namprd04.prod.outlook.com>
 <be263cdf-19a2-05d6-1023-7d935d526663@gmail.com>
 <20210218032822.GA2711165@dhcp-10-100-145-180.wdc.com>
Message-ID: <BYAPR04MB49659AADB83039BA9DC0041886859@BYAPR04MB4965.namprd04.prod.outlook.com>

Keith,

On 2/17/21 19:28, Keith Busch wrote:
> On Wed, Feb 17, 2021 at 08:26:30AM +0100, Heiner Kallweit wrote:
>> On 17.02.2021 00:59, Chaitanya Kulkarni wrote:
>>> On 2/14/21 08:30, Heiner Kallweit wrote:
>>>> Based on the description we should be fine using the less heavy-weight
>>>> smp barriers here. On x86 this would be compiler barriers only.
>>>>
>>>> Signed-off-by: Heiner Kallweit <hkallweit1 at gmail.com>
>>> Can you share performance numbers ?
>>>
>> I stumbled across this code piece by chance and don't have hw to test it.
>> Having said that the change is based on code inspection only.
>> Unfortunately the barrier comment is very generic and doesn't mention
>> a problematic scenario. Also the commit message of 3a7afd8ee42a doesn't
>> mention a potential race. Most likely the barrier can be removed
>> completely. Helpful would be if someone could explain the potential race
>> in detail, means which reordering / variable accesses could be racy.
>> Then we would have a basis to talk about READ_ONCE/WRITE_ONCE vs.
>> smp barrier vs. dma barrier vs. full barrier.
> The variables the driver was protecting no longer exist, so I also agree
> the barriers should not be necessary.
>

Are you saying something like following is needed ?

>From a3a73bd3943b479f82ff00582baf2c37c1afd8e8 Mon Sep 17 00:00:00 2001
From: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Date: Wed, 17 Feb 2021 20:01:37 -0800
Subject: [PATCH] nvme-pci: remove the barriers

The variable which was protected by the barriers is removed in
The commit f6c4d97b0d82 (" nvme/pci: Remove last_cq_head").

Remove the barriers which was protecting the variable.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/pci.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 0045c5edf629..3729775f6a8a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1062,14 +1062,8 @@ static irqreturn_t nvme_irq(int irq, void *data)
     struct nvme_queue *nvmeq = data;
     irqreturn_t ret = IRQ_NONE;
 
-    /*
-     * The rmb/wmb pair ensures we see all updates from a previous run of
-     * the irq handler, even if that was on another CPU.
-     */
-    rmb();
     if (nvme_process_cq(nvmeq))
         ret = IRQ_HANDLED;
-    wmb();
 
     return ret;
 }
-- 
2.22.1


From hch at lst.de  Thu Feb 18 02:05:13 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 18 Feb 2021 08:05:13 +0100
Subject: [PATCH] nvmet: don't change model_number on live subsys
In-Reply-To: <20210218031644.12476-1-chaitanya.kulkarni@wdc.com>
References: <20210218031644.12476-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210218070513.GA16309@lst.de>

> +	mutex_lock(&subsys->lock);
> +	list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
> +		connected = true;
> +		break;
> +	}
> +	mutex_unlock(&subsys->lock);

This could just use list_empty.  But we'd also need to hold the lock
over the actual change of the model number to avoid a race window.


From hch at lst.de  Thu Feb 18 03:40:14 2021
From: hch at lst.de (Christoph Hellwig)
Date: Thu, 18 Feb 2021 09:40:14 +0100
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210217021455.GA2251@localhost.localdomain>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
 <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
 <20210216095147.GA2601@localhost.localdomain>
 <20210216175722.GC2708768@dhcp-10-100-145-180.wdc.com>
 <20210217021455.GA2251@localhost.localdomain>
Message-ID: <20210218084014.GA17997@lst.de>

On Wed, Feb 17, 2021 at 11:14:55AM +0900, Minwoo Im wrote:
> > A container could read data from namespaces assigned to a different
> > container.
> 
> Now I got your point! Thanks Keith, please ignore this patch :)

Note that Javier has been working on exposing access to namespaces
with unsupported command sets for ioctls.  If that is what you are
looking for.


From javier.gonz at samsung.com  Thu Feb 18 03:48:12 2021
From: javier.gonz at samsung.com (Javier =?utf-8?B?R29uesOhbGV6?=)
Date: Thu, 18 Feb 2021 09:48:12 +0100
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210218084014.GA17997@lst.de>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
 <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
 <20210216095147.GA2601@localhost.localdomain>
 <20210216175722.GC2708768@dhcp-10-100-145-180.wdc.com>
 <20210217021455.GA2251@localhost.localdomain>
 <CGME20210218084022eucas1p296cd6f091da8b347ecb3232a2e0b7383@eucas1p2.samsung.com>
 <20210218084014.GA17997@lst.de>
Message-ID: <20210218084812.kcibjukwlb37ijji@mpHalley.local>

On 18.02.2021 09:40, Christoph Hellwig wrote:
>On Wed, Feb 17, 2021 at 11:14:55AM +0900, Minwoo Im wrote:
>> > A container could read data from namespaces assigned to a different
>> > container.
>>
>> Now I got your point! Thanks Keith, please ignore this patch :)
>
>Note that Javier has been working on exposing access to namespaces
>with unsupported command sets for ioctls.  If that is what you are
>looking for.

Minwoo, I think this would apply good on top of the char device patches
I posted a while ago. I have a multipath version working on top, but I
am still working out a good way to expose in sysfs.

We can try to merge these and send the char device with multipath in the
following days. This has been hanging on me more than I wanted it to...


From martinus.gpy at gmail.com  Thu Feb 18 04:46:42 2021
From: martinus.gpy at gmail.com (Martin George)
Date: Thu, 18 Feb 2021 15:16:42 +0530
Subject: [PATCH] fabrics: permit kato int argument for discovery controllers
Message-ID: <20210218094642.5616-1-marting@netapp.com>

Given discovery controllers can now accept integer arguments
for kato, permit the same in build_options().

Signed-off-by: Martin George <marting at netapp.com>
---
 fabrics.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/fabrics.c b/fabrics.c
index 1880a5d..9cfe26a 100644
--- a/fabrics.c
+++ b/fabrics.c
@@ -896,9 +896,8 @@ static int build_options(char *argstr, int max_len, bool discover)
 	    (!discover &&
 	      add_int_argument(&argstr, &max_len, "queue_size",
 				cfg.queue_size, false)) ||
-	    (!discover &&
-	      add_int_argument(&argstr, &max_len, "keep_alive_tmo",
-				cfg.keep_alive_tmo, false)) ||
+	    add_int_argument(&argstr, &max_len, "keep_alive_tmo",
+				cfg.keep_alive_tmo, false) ||
 	    add_int_argument(&argstr, &max_len, "reconnect_delay",
 				cfg.reconnect_delay, false) ||
 	    add_int_argument(&argstr, &max_len, "ctrl_loss_tmo",
-- 
2.30.0



From mgurtovoy at nvidia.com  Thu Feb 18 04:51:36 2021
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Thu, 18 Feb 2021 11:51:36 +0200
Subject: [PATCH 1/1] nvmet: allow setting model_number once
In-Reply-To: <BYAPR04MB4965D3231E5681CD2642EDF686859@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
 <BYAPR04MB4965D3231E5681CD2642EDF686859@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <cbdcaf4c-1aa4-8949-ebbf-83f54c2a274b@nvidia.com>


On 2/18/2021 5:17 AM, Chaitanya Kulkarni wrote:
> Max,
>
> On 2/17/21 10:12, Max Gurtovoy wrote:
>> In case we have already established connection to nvmf target, it
>> shouldn't be allowed to change the model_number. E.g. if someone will
>> identify ctrl and get model_number of "my_model" later on will change
>> the model_numbel via configfs to "my_new_model" this will break the NVM
>> specification for "Get Log Page ? Persistent Event Log" that refers to
>> Model Number as: "This field contains the same value as reported in the
>> Model Number field of the Identify Controller data structure, bytes
>> 63:24."
> We don't support persistent event log page since there is no use
> case for target to have human readable format with timestamps,
> instead we do maintain the error log page which has information
> about the errors if user wants.
>
> I'll let Christoph/Sagi decide if we really need to implement
> Persistent Event (PE) log and so does this behavior.
>
> Regarding preventing from changing the model_number when subsys is
> connected to the host, that needs a fix and I sent a patch for that.
>
> We can ignore that patch if we decide to go with this patch with
> PR log page.

what is wrong with this patch ?

why you need the rcu logic in the driver and why do you want to preserve 
a case that initiator A connect to subsystem "my_sub" and see MN == 
Linux and after disconnect and re-connect it might see MN = New_Linux ?


>
>> Although it doesn't mentioned explicitly that this field can't be
>> changed, we can assume it.
>>
>> So allow setting this field only once: using configfs or in the first
>> identify ctrl operation.
>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme


From martinus.gpy at gmail.com  Thu Feb 18 04:54:22 2021
From: martinus.gpy at gmail.com (Martin George)
Date: Thu, 18 Feb 2021 15:24:22 +0530
Subject: [PATCH] fabrics: ensure zero kato for non-persistent controllers
Message-ID: <20210218095422.5696-1-marting@netapp.com>

Along with ensuring a non-zero kato is passed to a
persistent discovery controller, also ensure a zero
kato is passed to a non-persistent discovery controller.

Signed-off-by: Martin George <marting at netapp.com>
---
 fabrics.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/fabrics.c b/fabrics.c
index 1880a5d..ed15e08 100644
--- a/fabrics.c
+++ b/fabrics.c
@@ -1399,6 +1399,8 @@ static int discover_from_conf_file(const char *desc, char *argstr,
 
 		if (cfg.persistent && !cfg.keep_alive_tmo)
 			cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
+		else if (!cfg.persistent && (cfg.keep_alive_tmo > 0))
+			cfg.keep_alive_tmo = 0;
 
 		if (traddr_is_hostname(&cfg)) {
 			ret = hostname2traddr(&cfg);
@@ -1474,6 +1476,8 @@ int fabrics_discover(const char *desc, int argc, char **argv, bool connect)
 	} else {
 		if (cfg.persistent && !cfg.keep_alive_tmo)
 			cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
+		else if (!cfg.persistent && (cfg.keep_alive_tmo > 0))
+			cfg.keep_alive_tmo = 0;
 
 		if (traddr_is_hostname(&cfg)) {
 			ret = hostname2traddr(&cfg);
-- 
2.30.0



From martinus.gpy at gmail.com  Thu Feb 18 05:04:43 2021
From: martinus.gpy at gmail.com (Martin George)
Date: Thu, 18 Feb 2021 15:34:43 +0530
Subject: [PATCH] nvme-fabrics: fix kato initialization
Message-ID: <20210218100443.5854-1-marting@netapp.com>

Currently kato is initialized to NVME_DEFAULT_KATO for both
discovery & i/o controllers. This is a problem specifically
for non-persistent discovery controllers since it always ends
up with a non-zero kato value. Fix this by initializing kato
to zero instead, and ensuring various controllers are assigned
appropriate kato values as follows:

non-persistent controllers  - kato set to zero
persistent controllers      - kato set to NVMF_DEV_DISC_TMO
                              (or any positive int via nvme-cli)
i/o controllers             - kato set to NVME_DEFAULT_KATO
                              (or any positive int via nvme-cli)

Signed-off-by: Martin George <marting at netapp.com>
---
 drivers/nvme/host/fabrics.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index 5dfd806fc2d2..604ab0e5a2ad 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -630,7 +630,7 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
 	opts->queue_size = NVMF_DEF_QUEUE_SIZE;
 	opts->nr_io_queues = num_online_cpus();
 	opts->reconnect_delay = NVMF_DEF_RECONNECT_DELAY;
-	opts->kato = NVME_DEFAULT_KATO;
+	opts->kato = 0;
 	opts->duplicate_connect = false;
 	opts->fast_io_fail_tmo = NVMF_DEF_FAIL_FAST_TMO;
 	opts->hdr_digest = false;
@@ -893,6 +893,9 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
 		opts->nr_write_queues = 0;
 		opts->nr_poll_queues = 0;
 		opts->duplicate_connect = true;
+	} else {
+		if (!opts->kato)
+			opts->kato = NVME_DEFAULT_KATO;
 	}
 	if (ctrl_loss_tmo < 0) {
 		opts->max_reconnects = -1;
-- 
2.30.0



From hare at suse.de  Thu Feb 18 11:29:27 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 18 Feb 2021 17:29:27 +0100
Subject: [PATCH] fabrics: permit kato int argument for discovery
 controllers
In-Reply-To: <20210218094642.5616-1-marting@netapp.com>
References: <20210218094642.5616-1-marting@netapp.com>
Message-ID: <9a37a05a-4af0-59d2-5b6b-874b71940de7@suse.de>

On 2/18/21 10:46 AM, Martin George wrote:
> Given discovery controllers can now accept integer arguments
> for kato, permit the same in build_options().
> 
> Signed-off-by: Martin George <marting at netapp.com>
> ---
>   fabrics.c | 5 ++---
>   1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/fabrics.c b/fabrics.c
> index 1880a5d..9cfe26a 100644
> --- a/fabrics.c
> +++ b/fabrics.c
> @@ -896,9 +896,8 @@ static int build_options(char *argstr, int max_len, bool discover)
>   	    (!discover &&
>   	      add_int_argument(&argstr, &max_len, "queue_size",
>   				cfg.queue_size, false)) ||
> -	    (!discover &&
> -	      add_int_argument(&argstr, &max_len, "keep_alive_tmo",
> -				cfg.keep_alive_tmo, false)) ||
> +	    add_int_argument(&argstr, &max_len, "keep_alive_tmo",
> +				cfg.keep_alive_tmo, false) ||
>   	    add_int_argument(&argstr, &max_len, "reconnect_delay",
>   				cfg.reconnect_delay, false) ||
>   	    add_int_argument(&argstr, &max_len, "ctrl_loss_tmo",
> 
Reviewed-by: Hannes Reinecke <hare at suse.de>

Martin, you probably have to resend it via a pull request to the github 
repository.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From kbusch at kernel.org  Thu Feb 18 11:36:35 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 19 Feb 2021 01:36:35 +0900
Subject: [PATCH] fabrics: permit kato int argument for discovery
 controllers
In-Reply-To: <9a37a05a-4af0-59d2-5b6b-874b71940de7@suse.de>
References: <20210218094642.5616-1-marting@netapp.com>
 <9a37a05a-4af0-59d2-5b6b-874b71940de7@suse.de>
Message-ID: <20210218163635.GA30146@redsun51.ssa.fujisawa.hgst.com>

On Thu, Feb 18, 2021 at 05:29:27PM +0100, Hannes Reinecke wrote:
> On 2/18/21 10:46 AM, Martin George wrote:
> > Given discovery controllers can now accept integer arguments
> > for kato, permit the same in build_options().
> > 
> > Signed-off-by: Martin George <marting at netapp.com>
> > ---
> >   fabrics.c | 5 ++---
> >   1 file changed, 2 insertions(+), 3 deletions(-)
> > 
> > diff --git a/fabrics.c b/fabrics.c
> > index 1880a5d..9cfe26a 100644
> > --- a/fabrics.c
> > +++ b/fabrics.c
> > @@ -896,9 +896,8 @@ static int build_options(char *argstr, int max_len, bool discover)
> >   	    (!discover &&
> >   	      add_int_argument(&argstr, &max_len, "queue_size",
> >   				cfg.queue_size, false)) ||
> > -	    (!discover &&
> > -	      add_int_argument(&argstr, &max_len, "keep_alive_tmo",
> > -				cfg.keep_alive_tmo, false)) ||
> > +	    add_int_argument(&argstr, &max_len, "keep_alive_tmo",
> > +				cfg.keep_alive_tmo, false) ||
> >   	    add_int_argument(&argstr, &max_len, "reconnect_delay",
> >   				cfg.reconnect_delay, false) ||
> >   	    add_int_argument(&argstr, &max_len, "ctrl_loss_tmo",
> > 
> Reviewed-by: Hannes Reinecke <hare at suse.de>
> 
> Martin, you probably have to resend it via a pull request to the github
> repository.

Not necessary, I am totally okay with applying these direct from the
mailing list.

Thanks for the review, patch is already applied!


From mario.limonciello at dell.com  Thu Feb 18 12:09:45 2021
From: mario.limonciello at dell.com (Mario Limonciello)
Date: Thu, 18 Feb 2021 11:09:45 -0600
Subject: [RFC 0/2] Split out firmware upgrade from CAP_SYS_ADMIN
Message-ID: <20210218170947.15727-1-mario.limonciello@dell.com>

Currently NVME (and probably other drivers) require CAP_SYS_ADMIN to
send all commands to the device.  This means that software running
in userspace needs the stronger CAP_SYS_ADMIN permission when realistically
a more limited subset of functionality is actually needed.

To allow software that performs firmware upgrades to run without CAP_SYS_ADMIN,
create a new capability CAP_FIRMWARE_UPGRADE that software can run with.

For the RFC, only include NVME.  Other drivers can be added if suggested.

Mario Limonciello (2):
  capability: Introduce CAP_FIRMWARE_UPGRADE
  nvme: Use CAP_FIRMWARE_UPGRADE to check user commands

 drivers/nvme/host/core.c            | 28 ++++++++++++++++++++++++----
 include/linux/capability.h          |  5 +++++
 include/uapi/linux/capability.h     |  7 ++++++-
 security/selinux/include/classmap.h |  4 ++--
 4 files changed, 37 insertions(+), 7 deletions(-)

-- 
2.25.1



From mario.limonciello at dell.com  Thu Feb 18 12:09:47 2021
From: mario.limonciello at dell.com (Mario Limonciello)
Date: Thu, 18 Feb 2021 11:09:47 -0600
Subject: [RFC 2/2] nvme: Use CAP_FIRMWARE_UPGRADE to check user commands
In-Reply-To: <20210218170947.15727-1-mario.limonciello@dell.com>
References: <20210218170947.15727-1-mario.limonciello@dell.com>
Message-ID: <20210218170947.15727-3-mario.limonciello@dell.com>

Software that is running with CAP_FIRMWARE_UPGRADE needs a limited
set of opcode access:
* Identify the disk
* Download firmware to the disk
* Commit the firmwware to the disk

Signed-off-by: Mario Limonciello <mario.limonciello at dell.com>
---
 drivers/nvme/host/core.c | 28 ++++++++++++++++++++++++----
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f13eb4ded95f..80be3d6b7437 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1593,13 +1593,23 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 	u64 result;
 	int status;
 
-	if (!capable(CAP_SYS_ADMIN))
-		return -EACCES;
 	if (copy_from_user(&cmd, ucmd, sizeof(cmd)))
 		return -EFAULT;
 	if (cmd.flags)
 		return -EINVAL;
 
+	switch (cmd.opcode) {
+	case nvme_admin_identify:
+	case nvme_admin_activate_fw:
+	case nvme_admin_download_fw:
+		if (!firmware_upgrade_capable())
+			return -EACCES;
+		break;
+	default:
+		if (!capable(CAP_SYS_ADMIN))
+			return -EACCES;
+	}
+
 	memset(&c, 0, sizeof(c));
 	c.common.opcode = cmd.opcode;
 	c.common.flags = cmd.flags;
@@ -1637,13 +1647,23 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
 	unsigned timeout = 0;
 	int status;
 
-	if (!capable(CAP_SYS_ADMIN))
-		return -EACCES;
 	if (copy_from_user(&cmd, ucmd, sizeof(cmd)))
 		return -EFAULT;
 	if (cmd.flags)
 		return -EINVAL;
 
+	switch (cmd.opcode) {
+	case nvme_admin_identify:
+	case nvme_admin_activate_fw:
+	case nvme_admin_download_fw:
+		if (!firmware_upgrade_capable())
+			return -EACCES;
+		break;
+	default:
+		if (!capable(CAP_SYS_ADMIN))
+			return -EACCES;
+	}
+
 	memset(&c, 0, sizeof(c));
 	c.common.opcode = cmd.opcode;
 	c.common.flags = cmd.flags;
-- 
2.25.1



From mario.limonciello at dell.com  Thu Feb 18 12:09:46 2021
From: mario.limonciello at dell.com (Mario Limonciello)
Date: Thu, 18 Feb 2021 11:09:46 -0600
Subject: [RFC 1/2] capability: Introduce CAP_FIRMWARE_UPGRADE
In-Reply-To: <20210218170947.15727-1-mario.limonciello@dell.com>
References: <20210218170947.15727-1-mario.limonciello@dell.com>
Message-ID: <20210218170947.15727-2-mario.limonciello@dell.com>

Split out permissions specifically for firmware upgrades from
CAP_SYS_ADMIN to a new separate capability.  This will allow userspace
applications that would traditionally have needed CAP_SYS_ADMIN to perform
firmware upgrades to have a reduced permission set.

Signed-off-by: Mario Limonciello <mario.limonciello at dell.com>
---
 include/linux/capability.h          | 5 +++++
 include/uapi/linux/capability.h     | 7 ++++++-
 security/selinux/include/classmap.h | 4 ++--
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/include/linux/capability.h b/include/linux/capability.h
index b2f698915c0f..e9233e217402 100644
--- a/include/linux/capability.h
+++ b/include/linux/capability.h
@@ -267,6 +267,11 @@ static inline bool checkpoint_restore_ns_capable(struct user_namespace *ns)
 		ns_capable(ns, CAP_SYS_ADMIN);
 }
 
+static inline bool firmware_upgrade_capable(void)
+{
+	return capable(CAP_FIRMWARE_UPGRADE) || capable(CAP_SYS_ADMIN);
+}
+
 /* audit system wants to get cap info from files as well */
 extern int get_vfs_caps_from_disk(const struct dentry *dentry, struct cpu_vfs_cap_data *cpu_caps);
 
diff --git a/include/uapi/linux/capability.h b/include/uapi/linux/capability.h
index c6ca33034147..0f204c6a1c0b 100644
--- a/include/uapi/linux/capability.h
+++ b/include/uapi/linux/capability.h
@@ -275,6 +275,7 @@ struct vfs_ns_cap_data {
 /* Allow setting encryption key on loopback filesystem */
 /* Allow setting zone reclaim policy */
 /* Allow everything under CAP_BPF and CAP_PERFMON for backward compatibility */
+/* Allow everything under CAP_FIRMWARE_UPGRADE for backward compatibility */
 
 #define CAP_SYS_ADMIN        21
 
@@ -417,7 +418,11 @@ struct vfs_ns_cap_data {
 
 #define CAP_CHECKPOINT_RESTORE	40
 
-#define CAP_LAST_CAP         CAP_CHECKPOINT_RESTORE
+/* Allow a device firmware upgrade */
+
+#define CAP_FIRMWARE_UPGRADE	41
+
+#define CAP_LAST_CAP         CAP_FIRMWARE_UPGRADE
 
 #define cap_valid(x) ((x) >= 0 && (x) <= CAP_LAST_CAP)
 
diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h
index 40cebde62856..188318eefb41 100644
--- a/security/selinux/include/classmap.h
+++ b/security/selinux/include/classmap.h
@@ -28,9 +28,9 @@
 
 #define COMMON_CAP2_PERMS  "mac_override", "mac_admin", "syslog", \
 		"wake_alarm", "block_suspend", "audit_read", "perfmon", "bpf", \
-		"checkpoint_restore"
+		"checkpoint_restore", "firmware_upgrade"
 
-#if CAP_LAST_CAP > CAP_CHECKPOINT_RESTORE
+#if CAP_LAST_CAP > CAP_FIRMWARE_UPGRADE
 #error New capability defined, please update COMMON_CAP2_PERMS.
 #endif
 
-- 
2.25.1



From smalin at marvell.com  Thu Feb 18 13:38:07 2021
From: smalin at marvell.com (Shai Malin)
Date: Thu, 18 Feb 2021 18:38:07 +0000
Subject: [RFC PATCH v3 00/11] NVMeTCP Offload ULP and QEDN Device Driver
In-Reply-To: <20210207181324.11429-1-smalin@marvell.com>
References: <20210207181324.11429-1-smalin@marvell.com>
Message-ID: <PH0PR18MB3845E15A62826C9B5A520628CC859@PH0PR18MB3845.namprd18.prod.outlook.com>

> 
> With the goal of enabling a generic infrastructure that allows NVMe/TCP
> offload devices like NICs to seamlessly plug into the NVMe-oF stack, this
> patch series introduces the nvme-tcp-offload ULP host layer, which will be a
> new transport type called "tcp-offload" and will serve as an abstraction layer
> to work with vendor specific nvme-tcp offload drivers.
> 
> NVMeTCP offload is a full offload of the NVMeTCP protocol, this includes
> both the TCP level and the NVMeTCP level.
> 
> The nvme-tcp-offload transport can co-exist with the existing tcp and other
> transports. The tcp offload was designed so that stack changes are kept to a
> bare minimum: only registering new transports.
> All other APIs, ops etc. are identical to the regular tcp transport.
> Representing the TCP offload as a new transport allows clear and
> manageable differentiation between the connections which should use the
> offload path and those that are not offloaded (even on the same device).
> 

Sagi, Christoph, Jens, Keith,
So, as there are no more comments / questions, we understand the direction 
is acceptable and will proceed to the full series.



From mark.wunderlich at intel.com  Thu Feb 18 13:38:25 2021
From: mark.wunderlich at intel.com (Wunderlich, Mark)
Date: Thu, 18 Feb 2021 18:38:25 +0000
Subject: [PATCH] nvmet-tcp: enable optional queue idle period tracking
In-Reply-To: <25dfb6d2-2519-0f30-529f-36dcb4263160@grimberg.me>
References: <BYAPR11MB286909CAA3A8721B46A02AFFE58C9@BYAPR11MB2869.namprd11.prod.outlook.com>
 <25dfb6d2-2519-0f30-529f-36dcb4263160@grimberg.me>
Message-ID: <BYAPR11MB28691FA273E729AF338745B7E5859@BYAPR11MB2869.namprd11.prod.outlook.com>

>> +/*
>> + * Define a time period (in msecs)

>Is this in msecs or usecs? Anyway I think it should be in the name as well.

Ah, yes, that should be usecs.  Will update the name (queue_idle_period_usecs)?

>> + * queue before determining it to be idle.  This optional module 
>> +behavior
>> + * can enable NIC solutions that support socket optimized packet 
>> +processing
>> + * using advanced interrupt moderation techniques.
>> + */
>> +static int queue_idle_period;
>> +module_param(queue_idle_period, int, 0644); 
>> +MODULE_PARM_DESC(queue_idle_period, "nvmet tcp io_work queue idle 
>> +time period");

>I think this should be called "queue_poll_period" maybe? because this is the period a queue will keep actively trying to read data from the socket regardless if there is any activity or not right?

>Or does this mean a different semantics with this?

The idea is to actively sample a queue until a cumulative period of time is experienced with no activity, the 'idle' period.  The starting time for the time period always reset if any activity during the current period is experienced.  The goal being to continually extend the sample period before being forced back into legacy interrupt mode.  So no, the time period was not intended to be a limit regardless of activity or not.

>> +
>>   #define NVMET_TCP_RECV_BUDGET		8
>>   #define NVMET_TCP_SEND_BUDGET		8
> >  #define NVMET_TCP_IO_WORK_BUDGET	64
>> @@ -96,6 +106,7 @@ struct nvmet_tcp_queue {
>>   	struct work_struct	io_work;
>>   	struct nvmet_cq		nvme_cq;
>>   	struct nvmet_sq		nvme_sq;
>> +	unsigned long           idle_period;

>poll_period?

So if you agree with my explanation above for the time period as the measurement of a 'idle' period of time with no activity, then I think keeping this as idle_period makes sense. Agree?

>> +	if (queue_idle_period) {
>> +		/* Assign the queue an idle period if not already set */
>> +		if (!queue->idle_period)
>> +			queue->idle_period =
>> +					usecs_to_jiffies(queue_idle_period);
>> +		idle_deadline = jiffies + queue->idle_period;

>Why do you need to keep a period time?

Because the io_work loop was written to exit on any iteration with no activity, assuming to provide for fairness with other queue workers that share the same core, the idle period of time must support many passes in and out of the work item for that queue by the kworker to preserve legacy behavior.  And only wanted to deduct the actual time while in the work item from the total time period, not any time that some other worker was taking on that shared core.

I suppose one could ignore this condition, but then the time allotted to a queue is less deterministic as the number of those sharing the core increases/changes.  During testing I didn't feel we could ignore this condition, what do you think?

>Why don't you just maintain queue->poll_start and just check if your past the queue_poll time addition:

>	if (queue_idle_period && queue->start_poll == 0)
>		queue->start_poll = jiffies;

>...
>[io_work loop]
>...

>	if (queue_poll_period) {
>		if (!time_after(jiffies, queue->start_poll +
>				usecs_to_jiffies(queue_poll_period)))
>			pending = true;
>		else
>			queue->start_poll = 0;
>	}

>Can this work?
If we were to not adjust or account for time while not in the work item, this logic might work.  And if we wanted to restart the idle period upon any activity, then in the 'if' case after setting pending=true would need to add check ' if (ops) then queue->start_poll = 0; '

>> +	}
>>   
>>   	do {
>>   		pending = false;
>> @@ -1222,8 +1263,30 @@ static void nvmet_tcp_io_work(struct 
>> work_struct *w)
>>   
>>   	} while (pending && ops < NVMET_TCP_IO_WORK_BUDGET);
>>   
>> +	/* If optional deadline mode active, determine if queue has reached its
>> +	 * idle process deadline limit.  Remaining deadline is calculated. Any ops
>> +	 * activity awards the queue a new deadline period.
>> +	 */

>Comments should stay consistent in style...

Got it.  assume the form above is OK?

>> +	if (queue_idle_period) {
>> +		/*
>> +		 * Clear to award active non-idle queue new period, or
>> +		 * reset for future queue activity after exit when idle reached.
>> +		 */
>> +		queue->idle_period = 0;
>> +		if (ops > 0) {
>> +			pending = true;

>What is this useful for?

This ensures that if there was any activity recorded during the total io_work loop, not just the last iteration, then the work item will be re-queued.  And because there was activity recorded then we need not process the current idle deadline because it will be renewed the next time into the work item with a new idle start period.  Maybe improved comment above this code required?

>> +		} else if (!time_after(jiffies, idle_deadline)) {
>> +			period_remaining = (long)(idle_deadline - jiffies);
>> +			if (period_remaining > 0) {
>> +				pending = true;
>> +				queue->idle_period = period_remaining;
>> +			}
>> +		}
>> +	}

From kbusch at kernel.org  Thu Feb 18 16:57:58 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 19 Feb 2021 06:57:58 +0900
Subject: [PATCH] nvme: use lighter smp barriers in nvme_irq
In-Reply-To: <BYAPR04MB49659AADB83039BA9DC0041886859@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <1c90fd6a-d991-b347-479c-2c8ae5a4c51f@gmail.com>
 <BYAPR04MB4965D222B05866D93852835F86879@BYAPR04MB4965.namprd04.prod.outlook.com>
 <be263cdf-19a2-05d6-1023-7d935d526663@gmail.com>
 <20210218032822.GA2711165@dhcp-10-100-145-180.wdc.com>
 <BYAPR04MB49659AADB83039BA9DC0041886859@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210218215758.GB30146@redsun51.ssa.fujisawa.hgst.com>

On Thu, Feb 18, 2021 at 04:06:35AM +0000, Chaitanya Kulkarni wrote:
> On 2/17/21 19:28, Keith Busch wrote:
> > The variables the driver was protecting no longer exist, so I also agree
> > the barriers should not be necessary.
> >
> 
> Are you saying something like following is needed ?

Right, I'm just saying the barriers don't appear to be necessary, so
they can go away as you've indicated below.
 
> From a3a73bd3943b479f82ff00582baf2c37c1afd8e8 Mon Sep 17 00:00:00 2001
> From: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> Date: Wed, 17 Feb 2021 20:01:37 -0800
> Subject: [PATCH] nvme-pci: remove the barriers
> 
> The variable which was protected by the barriers is removed in
> The commit f6c4d97b0d82 (" nvme/pci: Remove last_cq_head").
> 
> Remove the barriers which was protecting the variable.
> 
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> ---
>  drivers/nvme/host/pci.c | 6 ------
>  1 file changed, 6 deletions(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 0045c5edf629..3729775f6a8a 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1062,14 +1062,8 @@ static irqreturn_t nvme_irq(int irq, void *data)
>      struct nvme_queue *nvmeq = data;
>      irqreturn_t ret = IRQ_NONE;
>  
> -    /*
> -     * The rmb/wmb pair ensures we see all updates from a previous run of
> -     * the irq handler, even if that was on another CPU.
> -     */
> -    rmb();
>      if (nvme_process_cq(nvmeq))
>          ret = IRQ_HANDLED;
> -    wmb();
>  
>      return ret;
>  }
> -- 
> 2.22.1


From mark.wunderlich at intel.com  Thu Feb 18 19:34:54 2021
From: mark.wunderlich at intel.com (Wunderlich, Mark)
Date: Fri, 19 Feb 2021 00:34:54 +0000
Subject: [PATCH] nvmet-tcp: enable optional queue idle period tracking
In-Reply-To: <BYAPR11MB28691FA273E729AF338745B7E5859@BYAPR11MB2869.namprd11.prod.outlook.com>
References: <BYAPR11MB286909CAA3A8721B46A02AFFE58C9@BYAPR11MB2869.namprd11.prod.outlook.com>
 <25dfb6d2-2519-0f30-529f-36dcb4263160@grimberg.me>
 <BYAPR11MB28691FA273E729AF338745B7E5859@BYAPR11MB2869.namprd11.prod.outlook.com>
Message-ID: <BYAPR11MB2869868FD753ACACDF0916CCE5849@BYAPR11MB2869.namprd11.prod.outlook.com>

>>	if (queue_idle_period && queue->start_poll == 0)
>>		queue->start_poll = jiffies;

>>...
>>[io_work loop]
>>...

>>	if (queue_poll_period) {
>>		if (!time_after(jiffies, queue->start_poll +
>>				usecs_to_jiffies(queue_poll_period)))
>>			pending = true;
>>		else
>>			queue->start_poll = 0;
>>	}

>>Can this work?


>If we were to not adjust or account for time while not in the work item, this logic might work.  And if we wanted to restart the idle period upon any activity, then in the 'if' case after setting pending=true would need to >add check ' if (ops) then queue->start_poll = 0; '

I tested the following along these lines, and appears to work just as well.  As long as user specifies a time period in that at least a single I/O will occur within that period, a new period will continually be assigned until we reach the eventual idle deadline.

code tested:
	if (queue_idle_period_usecs && queue->idle_poll_deadline == 0)
		queue->idle_poll_deadline = jiffies + usecs_to_jiffies(queue_idle_period_usecs);

	[io_work loop]

	if (queue_idle_period_usecs) {
		if (!time_after(jiffies, queue->idle_poll_deadline)) {
			pending = true;
			if (ops > 0)
				queue->idle_poll_deadline = 0;
		} else {
			queue->idle_poll_deadline = 0;
		}
	}

Will do a V2 if you like this better?  Like the names used?


From chaitanya.kulkarni at wdc.com  Fri Feb 19 01:37:15 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 18 Feb 2021 22:37:15 -0800
Subject: [PATCH 0/2] nvme-pci: remove the barriers in nvme_irq()
Message-ID: <20210219063715.63666-1-chaitanya.kulkarni@wdc.com>

Hi,

The first patch removes barrier from the nvme_irq() and second patch
does the code cleanup for the same.

-ck

Chaitanya Kulkarni (2):
  nvme-pci; remove the barriers nvme_irq()
  nvme-pci: cleanup nvme_irq()

 drivers/nvme/host/pci.c | 15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Fri Feb 19 01:37:26 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 18 Feb 2021 22:37:26 -0800
Subject: [PATCH 1/2] nvme-pci; remove the barriers nvme_irq()
Message-ID: <20210219063726.63728-1-chaitanya.kulkarni@wdc.com>

The barriers were added to the nvme_irq() in the
commit 3a7afd8ee42a ("nvme-pci: remove the CQ lock for interrupt driven queues")
to prevent compiler from doing memory optimization for the variabes that
were protected previously by spinlock in nvme_irq() at completion queue
processing and with queue head check condition..

The variable nvmeq->last_cq_head from those checks was removed in the
commit f6c4d97b0d82 ("nvme/pci: Remove last_cq_head")") that was
preventing  poll queues from mistakenly triggering the spurious
interrupt detection.

Remove the barriers which were protecting the updates to the variables.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/pci.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 0045c5edf629..3729775f6a8a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1062,14 +1062,8 @@ static irqreturn_t nvme_irq(int irq, void *data)
 	struct nvme_queue *nvmeq = data;
 	irqreturn_t ret = IRQ_NONE;
 
-	/*
-	 * The rmb/wmb pair ensures we see all updates from a previous run of
-	 * the irq handler, even if that was on another CPU.
-	 */
-	rmb();
 	if (nvme_process_cq(nvmeq))
 		ret = IRQ_HANDLED;
-	wmb();
 
 	return ret;
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Fri Feb 19 01:37:37 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 18 Feb 2021 22:37:37 -0800
Subject: [PATCH 2/2] nvme-pci: cleanup nvme_irq()
Message-ID: <20210219063737.63800-1-chaitanya.kulkarni@wdc.com>

Get rid of the local variables which are not needed. Instead rename
the function parameter from data -> nvmeq and return from the if
check than having a variable.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/pci.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 3729775f6a8a..922cbaa7e4ee 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1057,15 +1057,12 @@ static inline int nvme_process_cq(struct nvme_queue *nvmeq)
 	return found;
 }
 
-static irqreturn_t nvme_irq(int irq, void *data)
+static irqreturn_t nvme_irq(int irq, void *nvmeq)
 {
-	struct nvme_queue *nvmeq = data;
-	irqreturn_t ret = IRQ_NONE;
-
 	if (nvme_process_cq(nvmeq))
-		ret = IRQ_HANDLED;
+		return IRQ_HANDLED;
 
-	return ret;
+	return IRQ_NONE;
 }
 
 static irqreturn_t nvme_irq_check(int irq, void *data)
-- 
2.22.1



From hch at lst.de  Fri Feb 19 04:12:37 2021
From: hch at lst.de (hch at lst.de)
Date: Fri, 19 Feb 2021 10:12:37 +0100
Subject: [RFC PATCH v3 00/11] NVMeTCP Offload ULP and QEDN Device Driver
In-Reply-To: <PH0PR18MB3845E15A62826C9B5A520628CC859@PH0PR18MB3845.namprd18.prod.outlook.com>
References: <20210207181324.11429-1-smalin@marvell.com>
 <PH0PR18MB3845E15A62826C9B5A520628CC859@PH0PR18MB3845.namprd18.prod.outlook.com>
Message-ID: <20210219091237.GA4036@lst.de>

On Thu, Feb 18, 2021 at 06:38:07PM +0000, Shai Malin wrote:
> So, as there are no more comments / questions, we understand the direction 
> is acceptable and will proceed to the full series.

I do not think we should support offloads at all, and certainly not onces
requiring extra drivers.  Those drivers have caused unbelivable pain for
iSCSI and we should not repeat that mistake.


From axboe at kernel.dk  Fri Feb 19 08:40:04 2021
From: axboe at kernel.dk (Jens Axboe)
Date: Fri, 19 Feb 2021 06:40:04 -0700
Subject: [PATCH 1/2] nvme-pci; remove the barriers nvme_irq()
In-Reply-To: <20210219063726.63728-1-chaitanya.kulkarni@wdc.com>
References: <20210219063726.63728-1-chaitanya.kulkarni@wdc.com>
Message-ID: <80fc53d2-4f35-7c72-5cc5-3ec13c5ee51a@kernel.dk>

On 2/18/21 11:37 PM, Chaitanya Kulkarni wrote:
> The barriers were added to the nvme_irq() in the
> commit 3a7afd8ee42a ("nvme-pci: remove the CQ lock for interrupt driven queues")
> to prevent compiler from doing memory optimization for the variabes that
> were protected previously by spinlock in nvme_irq() at completion queue
> processing and with queue head check condition..
> 
> The variable nvmeq->last_cq_head from those checks was removed in the
> commit f6c4d97b0d82 ("nvme/pci: Remove last_cq_head")") that was
> preventing  poll queues from mistakenly triggering the spurious
> interrupt detection.
> 
> Remove the barriers which were protecting the updates to the variables.

Would seem prudent to just ask the original submitter to send a v2
of his patch, as that's where it came from.

-- 
Jens Axboe



From hare at suse.de  Fri Feb 19 09:46:25 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 19 Feb 2021 15:46:25 +0100
Subject: [PATCH] nvme: add 'fail_if_no_path' sysfs attribute
Message-ID: <20210219144625.56344-1-hare@suse.de>

In some setups like RAID we need to return an I/O error
once all paths are unavailable to allow the upper layers
to start their own error recovery (like redirecting I/O
to other mirrors).
This patch adds a sysfs attribute 'fail_if_no_path' to
allow the admin to enable that behaviour instead of the
current 'queue until a path becomes available' policy.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      |  5 +++++
 drivers/nvme/host/multipath.c | 36 ++++++++++++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h      |  2 ++
 3 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 9879153d8d28..eee42d0b3400 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3440,6 +3440,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
+	&dev_attr_fail_if_no_path.attr,
 #endif
 	NULL,
 };
@@ -3470,6 +3471,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
 			return 0;
 	}
+	if (a == &dev_attr_fail_if_no_path.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
+			return 0;
+	}
 #endif
 	return a->mode;
 }
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 9ac762b28811..47fecb3f9434 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -283,10 +283,13 @@ static bool nvme_available_path(struct nvme_ns_head *head)
 			continue;
 		switch (ns->ctrl->state) {
 		case NVME_CTRL_LIVE:
+			return true;
 		case NVME_CTRL_RESETTING:
 		case NVME_CTRL_CONNECTING:
 			/* fallthru */
-			return true;
+			if (!test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH,
+				      &head->flags))
+				return true;
 		default:
 			break;
 		}
@@ -641,6 +644,37 @@ static ssize_t ana_state_show(struct device *dev, struct device_attribute *attr,
 }
 DEVICE_ATTR_RO(ana_state);
 
+static ssize_t fail_if_no_path_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+
+	return sprintf(buf, "%d\n",
+		       test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags) ?
+		       1 : 0);
+}
+
+static ssize_t fail_if_no_path_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+	int fail_if_no_path, err;
+
+	err = kstrtoint(buf, 10, &fail_if_no_path);
+	if (err)
+		return -EINVAL;
+
+	else if (fail_if_no_path <= 0)
+		clear_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
+	else
+		set_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
+	return count;
+}
+DEVICE_ATTR(fail_if_no_path, S_IRUGO | S_IWUSR,
+	fail_if_no_path_show, fail_if_no_path_store);
+
 static int nvme_lookup_ana_group_desc(struct nvme_ctrl *ctrl,
 		struct nvme_ana_group_desc *desc, void *data)
 {
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 09d142a9628c..f3ba741d44d0 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -411,6 +411,7 @@ struct nvme_ns_head {
 	struct mutex		lock;
 	unsigned long		flags;
 #define NVME_NSHEAD_DISK_LIVE	0
+#define NVME_NSHEAD_FAIL_IF_NO_PATH	1
 	struct nvme_ns __rcu	*current_path[];
 #endif
 };
@@ -684,6 +685,7 @@ static inline void nvme_trace_bio_complete(struct request *req)
 
 extern struct device_attribute dev_attr_ana_grpid;
 extern struct device_attribute dev_attr_ana_state;
+extern struct device_attribute dev_attr_fail_if_no_path;
 extern struct device_attribute subsys_attr_iopolicy;
 
 #else
-- 
2.29.2



From hare at suse.de  Fri Feb 19 11:41:00 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 19 Feb 2021 17:41:00 +0100
Subject: vhost/virtio fabric
In-Reply-To: <BYAPR04MB4965EC7FB74B38612065CDF5868E9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <CA+ULb+v-2jigmQA=Z2WmWP0n3wCbEs_s94oCoNbjJMw14ATCjw@mail.gmail.com>
 <BYAPR04MB4965EC7FB74B38612065CDF5868E9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <a010afb0-c81e-5e05-029a-0c9e49160553@suse.de>

On 2/9/21 5:41 AM, Chaitanya Kulkarni wrote:
> Roman,
> 
> On 2/8/21 20:29, Roman Shaposhnik wrote:
>> This search made me come across an old set of slides from
>> Christoph Hellwig that did, indeed, mention that at some point
>> vhost/virtio fabric was considered.
>>
>> Please let me know if there's any work that has ever been done
>> in the direction (even if in a form of unmerged patches) or whether
>> if I need this functionality I'd have to implement it from scratch (probably
>> aping a great deal of drivers/vhost/scsi.c).
> 
> At some point of time I've looked into this because that the next logical
> step in the ecosystem development, but I'm not sure if this will be
> acceptable or not or do we need to have change the spec. If there is a strong
> use case I'd like to definitely work on or contribute towards the development
> of this feature.
> 
> I'm interested in knowing what everyone think about this one ...
> 
Well, I had been looking into it for some time, too.
But turns out to be quite challenging conceptual.

Modifying / leveraging virtio for NVMe transport should be trivially, 
but then you'd need to put the virtio device on some emulated hardware, 
which typically (on ix86) is a PCI device.
So it would be a PCI device with virtio queues which transport NVMe 
CQEs/SQEs, begging the question why we need to have it as we _could_ use 
an emulated NVMe PCI device to start with.

But then _that_ would need to support PRPs, and you can't send discovery 
and/or connect commands across that.

So there might be a value in having a virtio transport, just so that you 
can use the fabrics commands.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Fri Feb 19 11:42:56 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 19 Feb 2021 17:42:56 +0100
Subject: [PATCH] nvme-fabrics: fix kato initialization
In-Reply-To: <20210218100443.5854-1-marting@netapp.com>
References: <20210218100443.5854-1-marting@netapp.com>
Message-ID: <03826110-0d9c-8b25-9975-b427036ec2cc@suse.de>

On 2/18/21 11:04 AM, Martin George wrote:
> Currently kato is initialized to NVME_DEFAULT_KATO for both
> discovery & i/o controllers. This is a problem specifically
> for non-persistent discovery controllers since it always ends
> up with a non-zero kato value. Fix this by initializing kato
> to zero instead, and ensuring various controllers are assigned
> appropriate kato values as follows:
> 
> non-persistent controllers  - kato set to zero
> persistent controllers      - kato set to NVMF_DEV_DISC_TMO
>                                (or any positive int via nvme-cli)
> i/o controllers             - kato set to NVME_DEFAULT_KATO
>                                (or any positive int via nvme-cli)
> 
> Signed-off-by: Martin George <marting at netapp.com>
> ---
>   drivers/nvme/host/fabrics.c | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
> index 5dfd806fc2d2..604ab0e5a2ad 100644
> --- a/drivers/nvme/host/fabrics.c
> +++ b/drivers/nvme/host/fabrics.c
> @@ -630,7 +630,7 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
>   	opts->queue_size = NVMF_DEF_QUEUE_SIZE;
>   	opts->nr_io_queues = num_online_cpus();
>   	opts->reconnect_delay = NVMF_DEF_RECONNECT_DELAY;
> -	opts->kato = NVME_DEFAULT_KATO;
> +	opts->kato = 0;
>   	opts->duplicate_connect = false;
>   	opts->fast_io_fail_tmo = NVMF_DEF_FAIL_FAST_TMO;
>   	opts->hdr_digest = false;
> @@ -893,6 +893,9 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
>   		opts->nr_write_queues = 0;
>   		opts->nr_poll_queues = 0;
>   		opts->duplicate_connect = true;
> +	} else {
> +		if (!opts->kato)
> +			opts->kato = NVME_DEFAULT_KATO;
>   	}
>   	if (ctrl_loss_tmo < 0) {
>   		opts->max_reconnects = -1;
> 
Can't you merge this with the previous patch 'ensure zero-kato ..'?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From konrad.wilk at oracle.com  Fri Feb 19 13:13:52 2021
From: konrad.wilk at oracle.com (Konrad Rzeszutek Wilk)
Date: Fri, 19 Feb 2021 13:13:52 -0500
Subject: preserve DMA offsets when using swiotlb v2
In-Reply-To: <CAMGD6P2QiQba=49LMOsQNZ=U9+hmAjxYLwroZGAG_AXoatUYpg@mail.gmail.com>
References: <20210207160327.2955490-1-hch@lst.de>
 <20210209084156.GA32320@lst.de>
 <CAMGD6P2QiQba=49LMOsQNZ=U9+hmAjxYLwroZGAG_AXoatUYpg@mail.gmail.com>
Message-ID: <YC//4P4mRrXegzIb@Konrads-MacBook-Pro.local>

On Tue, Feb 09, 2021 at 11:49:40AM -0800, Jianxiong Gao wrote:
> > Sorry for being a little pushy, any chance we could get this reviewed
> > in time for the 5.12 merge window?
> >
> Tested and it looks good. Thanks for clearing it up!

Let me put it on my regression bucket as last time it was able to catch
issues which Jianxiong's rig did not.

> 
> Gao


From minwoo.im.dev at gmail.com  Fri Feb 19 16:26:37 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sat, 20 Feb 2021 06:26:37 +0900
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210218084014.GA17997@lst.de>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
 <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
 <20210216095147.GA2601@localhost.localdomain>
 <20210216175722.GC2708768@dhcp-10-100-145-180.wdc.com>
 <20210217021455.GA2251@localhost.localdomain>
 <20210218084014.GA17997@lst.de>
Message-ID: <20210219212637.GA175471@localhost.localdomain>

On 21-02-18 09:40:14, Christoph Hellwig wrote:
> On Wed, Feb 17, 2021 at 11:14:55AM +0900, Minwoo Im wrote:
> > > A container could read data from namespaces assigned to a different
> > > container.
> > 
> > Now I got your point! Thanks Keith, please ignore this patch :)
> 
> Note that Javier has been working on exposing access to namespaces
> with unsupported command sets for ioctls.  If that is what you are
> looking for.

Thank you for the information!


From minwoo.im.dev at gmail.com  Fri Feb 19 16:27:49 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sat, 20 Feb 2021 06:27:49 +0900
Subject: [PATCH] nvme: introduce passthrough ioctl for multipath
In-Reply-To: <20210218084812.kcibjukwlb37ijji@mpHalley.local>
References: <20210214110126.24023-1-minwoo.im.dev@gmail.com>
 <20210214110126.24023-2-minwoo.im.dev@gmail.com>
 <20210215170233.GA2707454@dhcp-10-100-145-180.wdc.com>
 <20210216095147.GA2601@localhost.localdomain>
 <20210216175722.GC2708768@dhcp-10-100-145-180.wdc.com>
 <20210217021455.GA2251@localhost.localdomain>
 <CGME20210218084022eucas1p296cd6f091da8b347ecb3232a2e0b7383@eucas1p2.samsung.com>
 <20210218084014.GA17997@lst.de>
 <20210218084812.kcibjukwlb37ijji@mpHalley.local>
Message-ID: <20210219212749.GB175471@localhost.localdomain>

On 21-02-18 09:48:12, Javier Gonz?lez wrote:
> On 18.02.2021 09:40, Christoph Hellwig wrote:
> > On Wed, Feb 17, 2021 at 11:14:55AM +0900, Minwoo Im wrote:
> > > > A container could read data from namespaces assigned to a different
> > > > container.
> > > 
> > > Now I got your point! Thanks Keith, please ignore this patch :)
> > 
> > Note that Javier has been working on exposing access to namespaces
> > with unsupported command sets for ioctls.  If that is what you are
> > looking for.
> 
> Minwoo, I think this would apply good on top of the char device patches
> I posted a while ago. I have a multipath version working on top, but I
> am still working out a good way to expose in sysfs.
> 
> We can try to merge these and send the char device with multipath in the
> following days. This has been hanging on me more than I wanted it to...

Thanks Javier, That sounds cool.  will have a look at your patch first!


From aelior at marvell.com  Fri Feb 19 16:28:18 2021
From: aelior at marvell.com (Ariel Elior)
Date: Fri, 19 Feb 2021 21:28:18 +0000
Subject: [EXT] Re: [RFC PATCH v3 00/11] NVMeTCP Offload ULP and QEDN
 Device Driver
In-Reply-To: <20210219091237.GA4036@lst.de>
References: <20210207181324.11429-1-smalin@marvell.com>
 <PH0PR18MB3845E15A62826C9B5A520628CC859@PH0PR18MB3845.namprd18.prod.outlook.com>
 <20210219091237.GA4036@lst.de>
Message-ID: <SJ0PR18MB397807DB5295C9D39D93DF8EC4849@SJ0PR18MB3978.namprd18.prod.outlook.com>

> On Thu, Feb 18, 2021 at 06:38:07PM +0000, Shai Malin wrote:
> > So, as there are no more comments / questions, we understand the
> > direction is acceptable and will proceed to the full series.
> 
> I do not think we should support offloads at all, and certainly not onces
> requiring extra drivers.  Those drivers have caused unbelivable pain for iSCSI
> and we should not repeat that mistake.

Hi Christoph,

We are fully aware of the challenges the iSCSI offload faced - I was there too
(in bnx2i and qedi). In our mind the heart of that hardship was the iSCSI uio
design, essentially a thin alternative networking stack, which led to no end of
compatibility challenges.

But we were also there for RoCE and iWARP (TCP based) RDMA offloads where a
different approach was used, working with the networking stack instead of around
it. We feel this is a much better approach, and this is what we are attempting
to implement here.

For this reason exactly we designed this offload to be completely seemless.
There is no alternate user stack - we plug in directly into the networking
stack and there are zero changes to the regular nvme-tcp.

We are just adding a new transport alongside it, which interacts with the
networking stack when needed, and leaves it alone most of the time. Our
intention is to completely own the maintenance of the new transport, including
any compatibility requirements, and have purposefully designed it to be
streamlined in this aspect.

Protocol offload is at the core of our technology, and our device offloads RoCE,
iWARP, iSCSI and FCoE, all already in upstream drivers (qedr, qedi and qedf
respectively).

We are especially excited about NVMeTCP offload as it brings huge benefits:
RDMA-like latency, tremendous cpu utilization reduction and the reliability of
TCP.

We would be more than happy to incorporate any feedback you may have on the
design, in how to make it more robust and correct. We are aware of other work
being done in creating special types of offloaded queue, and could model our
design similarly, although our thinking was that this would be more intrusive to
regular nvme over tcp. In our original submission of the RFC we were not adding
a ULP driver, only our own vendor driver, but Sagi pointed us in the direction
of a vendor agnostic ulp layer, which made a lot of sense to us and we think is
a good approach.

Thanks,
Ariel


From kbusch at kernel.org  Fri Feb 19 18:16:43 2021
From: kbusch at kernel.org (Keith Busch)
Date: Sat, 20 Feb 2021 08:16:43 +0900
Subject: [PATCH] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <20210219144625.56344-1-hare@suse.de>
References: <20210219144625.56344-1-hare@suse.de>
Message-ID: <20210219231643.GA4105@redsun51.ssa.fujisawa.hgst.com>

On Fri, Feb 19, 2021 at 03:46:25PM +0100, Hannes Reinecke wrote:
> In some setups like RAID we need to return an I/O error
> once all paths are unavailable to allow the upper layers
> to start their own error recovery (like redirecting I/O
> to other mirrors).
> This patch adds a sysfs attribute 'fail_if_no_path' to
> allow the admin to enable that behaviour instead of the
> current 'queue until a path becomes available' policy.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>

Sounds okay to me, just some minor nits below.

> @@ -283,10 +283,13 @@ static bool nvme_available_path(struct nvme_ns_head *head)
>  			continue;
>  		switch (ns->ctrl->state) {
>  		case NVME_CTRL_LIVE:
> +			return true;
>  		case NVME_CTRL_RESETTING:
>  		case NVME_CTRL_CONNECTING:
>  			/* fallthru */
> -			return true;
> +			if (!test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH,
> +				      &head->flags))
> +				return true;

It doesn't look like the 'fallthru' is in the right place anymore,
though I can't tell why it is there in the first place.

>  		default:
>  			break;
>  		}
> @@ -641,6 +644,37 @@ static ssize_t ana_state_show(struct device *dev, struct device_attribute *attr,
>  }
>  DEVICE_ATTR_RO(ana_state);
>  
> +static ssize_t fail_if_no_path_show(struct device *dev,
> +		struct device_attribute *attr, char *buf)
> +{
> +	struct gendisk *disk = dev_to_disk(dev);
> +	struct nvme_ns_head *head = disk->private_data;
> +
> +	return sprintf(buf, "%d\n",
> +		       test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags) ?
> +		       1 : 0);
> +}
> +
> +static ssize_t fail_if_no_path_store(struct device *dev,
> +		struct device_attribute *attr, const char *buf, size_t count)
> +{
> +	struct gendisk *disk = dev_to_disk(dev);
> +	struct nvme_ns_head *head = disk->private_data;
> +	int fail_if_no_path, err;
> +
> +	err = kstrtoint(buf, 10, &fail_if_no_path);
> +	if (err)
> +		return -EINVAL;
> +
> +	else if (fail_if_no_path <= 0)
> +		clear_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
> +	else
> +		set_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);

The extra line above the 'else if' should be removed, or just change it
to simply 'if'.


From selvakuma.s1 at samsung.com  Fri Feb 19 07:45:13 2021
From: selvakuma.s1 at samsung.com (SelvaKumar S)
Date: Fri, 19 Feb 2021 18:15:13 +0530
Subject: [RFC PATCH v5 0/4] add simple copy support
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
Message-ID: <20210219124517.79359-1-selvakuma.s1@samsung.com>

This patchset tries to add support for TP4065a ("Simple Copy Command"),
v2020.05.04 ("Ratified")

The Specification can be found in following link.
https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip

Simple copy command is a copy offloading operation and is  used to copy
multiple contiguous ranges (source_ranges) of LBA's to a single destination
LBA within the device reducing traffic between host and device.

This implementation doesn't add native copy offload support for stacked
devices rather copy offload is done through emulation. Possible use
cases are F2FS gc and BTRFS relocation/balance.

*blkdev_issue_copy* takes source bdev, no of sources, array of source
ranges (in sectors), destination bdev and destination offset(in sectors).
If both source and destination block devices are same and copy_offload = 1,
then copy is done through native copy offloading. Copy emulation is used
in other cases.

As SCSI XCOPY can take two different block devices and no of source range is
equal to 1, this interface can be extended in future to support SCSI XCOPY.

For devices supporting native simple copy, attach the control information
as payload to the bio and submit to the device. For devices without native
copy support, copy emulation is done by reading each source range into memory
and writing it to the destination. Caller can choose not to try
emulation if copy offload is not supported by setting
BLKDEV_COPY_NOEMULATION flag.

Following limits are added to queue limits and are exposed in sysfs
to userspace
	- *copy_offload* controls copy_offload. set 0 to disable copy
		offload, 1 to enable native copy offloading support.
	- *max_copy_sectors* limits the sum of all source_range length
	- *max_copy_nr_ranges* limits the number of source ranges
	- *max_copy_range_sectors* limit the maximum number of sectors
		that can constitute a single source range.

	max_copy_sectors = 0 indicates the device doesn't support copy
offloading.

	*copy offload* sysfs entry is configurable and can be used toggle
between emulation and native support depending upon the usecase.

Changes from v4

1. Extend dm-kcopyd to leverage copy-offload, while copying within the
same device. The other approach was to have copy-emulation by moving
dm-kcopyd to block layer. But it also required moving core dm-io infra,
causing a massive churn across multiple dm-targets.

2. Remove export in bio_map_kern()
3. Change copy_offload sysfs to accept 0 or else
4. Rename copy support flag to QUEUE_FLAG_SIMPLE_COPY
5. Rename payload entries, add source bdev field to be used while
partition remapping, remove copy_size
6. Change the blkdev_issue_copy() interface to accept destination and
source values in sector rather in bytes
7. Add payload to bio using bio_map_kern() for copy_offload case
8. Add check to return error if one of the source range length is 0
9. Add BLKDEV_COPY_NOEMULATION flag to allow user to not try copy
emulation incase of copy offload is not supported. Caller can his use
his existing copying logic to complete the io.
10. Bug fix copy checks and reduce size of rcu_lock()

Planned for next:
- adding blktests
- handling larger (than device limits) copy
- decide on ioctl interface (man-page etc.)

Changes from v3

1. gfp_flag fixes.
2. Export bio_map_kern() and use it to allocate and add pages to bio.
3. Move copy offload, reading to buf, writing from buf to separate functions.
4. Send read bio of copy offload by chaining them and submit asynchronously.
5. Add gendisk->part0 and part->bd_start_sect changes to blk_check_copy().
6. Move single source range limit check to blk_check_copy()
7. Rename __blkdev_issue_copy() to blkdev_issue_copy and remove old helper.
8. Change blkdev_issue_copy() interface generic to accepts destination bdev
	to support XCOPY as well.
9. Add invalidate_kernel_vmap_range() after reading data for vmalloc'ed memory.
10. Fix buf allocoation logic to allocate buffer for the total size of copy.
11. Reword patch commit description.

Changes from v2

1. Add emulation support for devices not supporting copy.
2. Add *copy_offload* sysfs entry to enable and disable copy_offload
	in devices supporting simple copy.
3. Remove simple copy support for stacked devices.

Changes from v1:

1. Fix memory leak in __blkdev_issue_copy
2. Unmark blk_check_copy inline
3. Fix line break in blk_check_copy_eod
4. Remove p checks and made code more readable
5. Don't use bio_set_op_attrs and remove op and set
   bi_opf directly
6. Use struct_size to calculate total_size
7. Fix partition remap of copy destination
8. Remove mcl,mssrl,msrc from nvme_ns
9. Initialize copy queue limits to 0 in nvme_config_copy
10. Remove return in QUEUE_FLAG_COPY check
11. Remove unused OCFS

SelvaKumar S (4):
  block: make bio_map_kern() non static
  block: add simple copy support
  nvme: add simple copy support
  dm kcopyd: add simple copy offload support

 block/blk-core.c          | 102 +++++++++++++++--
 block/blk-lib.c           | 223 ++++++++++++++++++++++++++++++++++++++
 block/blk-map.c           |   2 +-
 block/blk-merge.c         |   2 +
 block/blk-settings.c      |  10 ++
 block/blk-sysfs.c         |  47 ++++++++
 block/blk-zoned.c         |   1 +
 block/bounce.c            |   1 +
 block/ioctl.c             |  33 ++++++
 drivers/md/dm-kcopyd.c    |  49 ++++++++-
 drivers/nvme/host/core.c  |  87 +++++++++++++++
 include/linux/bio.h       |   1 +
 include/linux/blk_types.h |  14 +++
 include/linux/blkdev.h    |  17 +++
 include/linux/nvme.h      |  43 +++++++-
 include/uapi/linux/fs.h   |  13 +++
 16 files changed, 627 insertions(+), 18 deletions(-)

-- 
2.25.1



From selvakuma.s1 at samsung.com  Fri Feb 19 07:45:14 2021
From: selvakuma.s1 at samsung.com (SelvaKumar S)
Date: Fri, 19 Feb 2021 18:15:14 +0530
Subject: [RFC PATCH v5 1/4] block: make bio_map_kern() non static
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <CGME20210219124559epcas5p41da46f1c248e334953d407a154697903@epcas5p4.samsung.com>
Message-ID: <20210219124517.79359-2-selvakuma.s1@samsung.com>

Make bio_map_kern() non static, so that copy offload emulation can use
it to add vmalloced memory to bio.

Signed-off-by: SelvaKumar S <selvakuma.s1 at samsung.com>
Signed-off-by: Chaitanya Kulkarni <kch at kernel.org>
---
 block/blk-map.c        | 2 +-
 include/linux/blkdev.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/block/blk-map.c b/block/blk-map.c
index 21630dccac62..17381b1643b8 100644
--- a/block/blk-map.c
+++ b/block/blk-map.c
@@ -378,7 +378,7 @@ static void bio_map_kern_endio(struct bio *bio)
  *	Map the kernel address into a bio suitable for io to a block
  *	device. Returns an error pointer in case of error.
  */
-static struct bio *bio_map_kern(struct request_queue *q, void *data,
+struct bio *bio_map_kern(struct request_queue *q, void *data,
 		unsigned int len, gfp_t gfp_mask)
 {
 	unsigned long kaddr = (unsigned long)data;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index f94ee3089e01..699ace6b25ff 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -944,6 +944,8 @@ extern int blk_rq_map_user(struct request_queue *, struct request *,
 			   struct rq_map_data *, void __user *, unsigned long,
 			   gfp_t);
 extern int blk_rq_unmap_user(struct bio *);
+struct bio *bio_map_kern(struct request_queue *q, void *data, unsigned int len,
+			gfp_t gfp_mask);
 extern int blk_rq_map_kern(struct request_queue *, struct request *, void *, unsigned int, gfp_t);
 extern int blk_rq_map_user_iov(struct request_queue *, struct request *,
 			       struct rq_map_data *, const struct iov_iter *,
-- 
2.25.1



From selvakuma.s1 at samsung.com  Fri Feb 19 07:45:15 2021
From: selvakuma.s1 at samsung.com (SelvaKumar S)
Date: Fri, 19 Feb 2021 18:15:15 +0530
Subject: [RFC PATCH v5 2/4] block: add simple copy support
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <CGME20210219124603epcas5p33add0f2c1781b2a4d71bf30c9e1ac647@epcas5p3.samsung.com>
Message-ID: <20210219124517.79359-3-selvakuma.s1@samsung.com>

Add new BLKCOPY ioctl that offloads copying of one or more sources
ranges to a destination in the device. Accepts a 'copy_range' structure
that contains destination (in sectors), no of sources and pointer to the
array of source ranges. Each source range is represented by 'range_entry'
that contains start and length of source ranges (in sectors).

Introduce REQ_OP_COPY, a no-merge copy offload operation. Create
bio with control information as payload and submit to the device.
REQ_OP_COPY(19) is a write op and takes zone_write_lock when submitted
to zoned device.

If the device doesn't support copy or copy offload is disabled, then
copy operation is emulated by default. However, the copy-emulation is an
opt-in feature. Caller can choose not to use the copy-emulation by
specifying a flag 'BLKDEV_COPY_NOEMULATION'.

Copy-emulation is implemented by allocating memory of total copy size.
The source ranges are read into memory by chaining bio for each source
ranges and submitting them async and the last bio waits for completion.
After data is read, it is written to the destination.

bio_map_kern() is used to allocate bio and add pages of copy buffer to
bio. As bio->bi_private and bio->bi_end_io are needed for chaining the
bio and gets over-written, invalidate_kernel_vmap_range() for read is
called in the caller.

Introduce queue limits for simple copy and other helper functions.
Add device limits as sysfs entries.
	- copy_offload
	- max_copy_sectors
	- max_copy_ranges_sectors
	- max_copy_nr_ranges

copy_offload(= 0) is disabled by default. This needs to be enabled if
copy-offload needs to be used.
max_copy_sectors = 0 indicates the device doesn't support native copy.

Native copy offload is not supported for stacked devices and is done via
copy emulation.

Signed-off-by: SelvaKumar S <selvakuma.s1 at samsung.com>
Signed-off-by: Kanchan Joshi <joshi.k at samsung.com>
Signed-off-by: Nitesh Shetty <nj.shetty at samsung.com>
Signed-off-by: Javier Gonz?lez <javier.gonz at samsung.com>
Signed-off-by: Chaitanya Kulkarni <kch at kernel.org>
---
 block/blk-core.c          | 102 ++++++++++++++++--
 block/blk-lib.c           | 222 ++++++++++++++++++++++++++++++++++++++
 block/blk-merge.c         |   2 +
 block/blk-settings.c      |  10 ++
 block/blk-sysfs.c         |  47 ++++++++
 block/blk-zoned.c         |   1 +
 block/bounce.c            |   1 +
 block/ioctl.c             |  33 ++++++
 include/linux/bio.h       |   1 +
 include/linux/blk_types.h |  14 +++
 include/linux/blkdev.h    |  15 +++
 include/uapi/linux/fs.h   |  13 +++
 12 files changed, 453 insertions(+), 8 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index 7663a9b94b80..23e646e5ae43 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -720,6 +720,17 @@ static noinline int should_fail_bio(struct bio *bio)
 }
 ALLOW_ERROR_INJECTION(should_fail_bio, ERRNO);
 
+static inline int bio_check_copy_eod(struct bio *bio, sector_t start,
+		sector_t nr_sectors, sector_t max_sect)
+{
+	if (nr_sectors && max_sect &&
+	    (nr_sectors > max_sect || start > max_sect - nr_sectors)) {
+		handle_bad_sector(bio, max_sect);
+		return -EIO;
+	}
+	return 0;
+}
+
 /*
  * Check whether this bio extends beyond the end of the device or partition.
  * This may well happen - the kernel calls bread() without checking the size of
@@ -738,6 +749,75 @@ static inline int bio_check_eod(struct bio *bio, sector_t maxsector)
 	return 0;
 }
 
+/*
+ * Check for copy limits and remap source ranges if needed.
+ */
+static int blk_check_copy(struct bio *bio)
+{
+	struct blk_copy_payload *payload = bio_data(bio);
+	struct request_queue *q = bio->bi_disk->queue;
+	sector_t max_sect, start_sect, copy_size = 0;
+	sector_t src_max_sect, src_start_sect;
+	struct block_device *bd_part;
+	int i, ret = -EIO;
+
+	rcu_read_lock();
+
+	bd_part = __disk_get_part(bio->bi_disk, bio->bi_partno);
+	if (unlikely(!bd_part)) {
+		rcu_read_unlock();
+		goto out;
+	}
+
+	max_sect =  bdev_nr_sectors(bd_part);
+	start_sect = bd_part->bd_start_sect;
+
+	src_max_sect = bdev_nr_sectors(payload->src_bdev);
+	src_start_sect = payload->src_bdev->bd_start_sect;
+
+	if (unlikely(should_fail_request(bd_part, bio->bi_iter.bi_size)))
+		goto out;
+
+	if (unlikely(bio_check_ro(bio, bd_part)))
+		goto out;
+
+	rcu_read_unlock();
+
+	/* cannot handle copy crossing nr_ranges limit */
+	if (payload->copy_nr_ranges > q->limits.max_copy_nr_ranges)
+		goto out;
+
+	for (i = 0; i < payload->copy_nr_ranges; i++) {
+		ret = bio_check_copy_eod(bio, payload->range[i].src,
+				payload->range[i].len, src_max_sect);
+		if (unlikely(ret))
+			goto out;
+
+		/* single source range length limit */
+		if (payload->range[i].len > q->limits.max_copy_range_sectors)
+			goto out;
+
+		payload->range[i].src += src_start_sect;
+		copy_size += payload->range[i].len;
+	}
+
+	/* check if copy length crosses eod */
+	ret = bio_check_copy_eod(bio, bio->bi_iter.bi_sector,
+				copy_size, max_sect);
+	if (unlikely(ret))
+		goto out;
+
+	/* cannot handle copy more than copy limits */
+	if (copy_size > q->limits.max_copy_sectors)
+		goto out;
+
+	bio->bi_iter.bi_sector += start_sect;
+	bio->bi_partno = 0;
+	ret = 0;
+out:
+	return ret;
+}
+
 /*
  * Remap block n of partition p to block n+start(p) of the disk.
  */
@@ -827,14 +907,16 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio)
 	if (should_fail_bio(bio))
 		goto end_io;
 
-	if (bio->bi_partno) {
-		if (unlikely(blk_partition_remap(bio)))
-			goto end_io;
-	} else {
-		if (unlikely(bio_check_ro(bio, bio->bi_disk->part0)))
-			goto end_io;
-		if (unlikely(bio_check_eod(bio, get_capacity(bio->bi_disk))))
-			goto end_io;
+	if (likely(!op_is_copy(bio->bi_opf))) {
+		if (bio->bi_partno) {
+			if (unlikely(blk_partition_remap(bio)))
+				goto end_io;
+		} else {
+			if (unlikely(bio_check_ro(bio, bio->bi_disk->part0)))
+				goto end_io;
+			if (unlikely(bio_check_eod(bio, get_capacity(bio->bi_disk))))
+				goto end_io;
+		}
 	}
 
 	/*
@@ -858,6 +940,10 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio)
 		if (!blk_queue_discard(q))
 			goto not_supported;
 		break;
+	case REQ_OP_COPY:
+		if (unlikely(blk_check_copy(bio)))
+			goto end_io;
+		break;
 	case REQ_OP_SECURE_ERASE:
 		if (!blk_queue_secure_erase(q))
 			goto not_supported;
diff --git a/block/blk-lib.c b/block/blk-lib.c
index 752f9c722062..97ba58d8d9a1 100644
--- a/block/blk-lib.c
+++ b/block/blk-lib.c
@@ -150,6 +150,228 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector,
 }
 EXPORT_SYMBOL(blkdev_issue_discard);
 
+int blk_copy_offload(struct block_device *dest_bdev, struct blk_copy_payload *payload,
+		sector_t dest, gfp_t gfp_mask)
+{
+	struct request_queue *q = bdev_get_queue(dest_bdev);
+	struct bio *bio;
+	int ret, payload_size;
+
+	payload_size = struct_size(payload, range, payload->copy_nr_ranges);
+	bio = bio_map_kern(q, payload, payload_size, gfp_mask);
+	if (IS_ERR(bio)) {
+		ret = PTR_ERR(bio);
+		goto err;
+	}
+
+	bio->bi_iter.bi_sector = dest;
+	bio->bi_opf = REQ_OP_COPY | REQ_NOMERGE;
+	bio_set_dev(bio, dest_bdev);
+
+	ret = submit_bio_wait(bio);
+err:
+	bio_put(bio);
+	return ret;
+}
+
+int blk_read_to_buf(struct block_device *src_bdev, struct blk_copy_payload *payload,
+		gfp_t gfp_mask, sector_t copy_size, void **buf_p)
+{
+	struct request_queue *q = bdev_get_queue(src_bdev);
+	struct bio *bio, *parent = NULL;
+	void *buf = NULL;
+	int copy_len = copy_size << SECTOR_SHIFT;
+	int i, nr_srcs, ret, cur_size, t_len = 0;
+	bool is_vmalloc;
+
+	nr_srcs = payload->copy_nr_ranges;
+
+	buf = kvmalloc(copy_len, gfp_mask);
+	if (!buf)
+		return -ENOMEM;
+	is_vmalloc = is_vmalloc_addr(buf);
+
+	for (i = 0; i < nr_srcs; i++) {
+		cur_size = payload->range[i].len << SECTOR_SHIFT;
+
+		bio = bio_map_kern(q, buf + t_len, cur_size, gfp_mask);
+		if (IS_ERR(bio)) {
+			ret = PTR_ERR(bio);
+			goto out;
+		}
+
+		bio->bi_iter.bi_sector = payload->range[i].src;
+		bio->bi_opf = REQ_OP_READ;
+		bio_set_dev(bio, src_bdev);
+		bio->bi_end_io = NULL;
+		bio->bi_private = NULL;
+
+		if (parent) {
+			bio_chain(parent, bio);
+			submit_bio(parent);
+		}
+
+		parent = bio;
+		t_len += cur_size;
+	}
+
+	ret = submit_bio_wait(bio);
+	bio_put(bio);
+	if (is_vmalloc)
+		invalidate_kernel_vmap_range(buf, copy_len);
+	if (ret)
+		goto out;
+
+	*buf_p = buf;
+	return 0;
+out:
+	kvfree(buf);
+	return ret;
+}
+
+int blk_write_from_buf(struct block_device *dest_bdev, void *buf, sector_t dest,
+		sector_t copy_size, gfp_t gfp_mask)
+{
+	struct request_queue *q = bdev_get_queue(dest_bdev);
+	struct bio *bio;
+	int ret, copy_len = copy_size << SECTOR_SHIFT;
+
+	bio = bio_map_kern(q, buf, copy_len, gfp_mask);
+	if (IS_ERR(bio)) {
+		ret = PTR_ERR(bio);
+		goto out;
+	}
+	bio_set_dev(bio, dest_bdev);
+	bio->bi_opf = REQ_OP_WRITE;
+	bio->bi_iter.bi_sector = dest;
+
+	bio->bi_end_io = NULL;
+	ret = submit_bio_wait(bio);
+	bio_put(bio);
+out:
+	return ret;
+}
+
+int blk_prepare_payload(struct block_device *src_bdev, int nr_srcs, struct range_entry *rlist,
+		gfp_t gfp_mask, struct blk_copy_payload **payload_p, sector_t *copy_size)
+{
+
+	struct request_queue *q = bdev_get_queue(src_bdev);
+	struct blk_copy_payload *payload;
+	sector_t bs_mask, total_len = 0;
+	int i, ret, payload_size;
+
+	if (!q)
+		return -ENXIO;
+
+	if (!nr_srcs)
+		return -EINVAL;
+
+	if (bdev_read_only(src_bdev))
+		return -EPERM;
+
+	bs_mask = (bdev_logical_block_size(src_bdev) >> 9) - 1;
+
+	payload_size = struct_size(payload, range, nr_srcs);
+	payload = kmalloc(payload_size, gfp_mask);
+	if (!payload)
+		return -ENOMEM;
+
+	for (i = 0; i < nr_srcs; i++) {
+		if (rlist[i].src & bs_mask || rlist[i].len & bs_mask) {
+			ret = -EINVAL;
+			goto err;
+		}
+
+		payload->range[i].src = rlist[i].src;
+		payload->range[i].len = rlist[i].len;
+
+		total_len += rlist[i].len;
+	}
+
+	payload->copy_nr_ranges = i;
+	payload->src_bdev = src_bdev;
+	*copy_size = total_len;
+
+	*payload_p = payload;
+	return 0;
+err:
+	kfree(payload);
+	return ret;
+}
+
+int blk_copy_emulate(struct block_device *src_bdev, struct blk_copy_payload *payload,
+			struct block_device *dest_bdev, sector_t dest,
+			sector_t copy_size, gfp_t gfp_mask)
+{
+	void *buf = NULL;
+	int ret;
+
+	ret = blk_read_to_buf(src_bdev, payload, gfp_mask, copy_size, &buf);
+	if (ret)
+		goto out;
+
+	ret = blk_write_from_buf(dest_bdev, buf, dest, copy_size, gfp_mask);
+	if (buf)
+		kvfree(buf);
+out:
+	return ret;
+}
+
+/**
+ * blkdev_issue_copy - queue a copy
+ * @src_bdev:	source block device
+ * @nr_srcs:	number of source ranges to copy
+ * @rlist:	array of source ranges in sector
+ * @dest_bdev:	destination block device
+ * @dest:	destination in sector
+ * @gfp_mask:   memory allocation flags (for bio_alloc)
+ * @flags:	BLKDEV_COPY_* flags to control behaviour
+ *
+ * Description:
+ *	Copy array of source ranges from source block device to
+ *	destination block devcie. All source must belong to same bdev and
+ *	length of a source range cannot be zero.
+ */
+
+int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs,
+		struct range_entry *src_rlist, struct block_device *dest_bdev,
+		sector_t dest, gfp_t gfp_mask, int flags)
+{
+	struct request_queue *q = bdev_get_queue(src_bdev);
+	struct request_queue *dest_q = bdev_get_queue(dest_bdev);
+	struct blk_copy_payload *payload;
+	sector_t bs_mask, copy_size;
+	int ret;
+
+	ret = blk_prepare_payload(src_bdev, nr_srcs, src_rlist, gfp_mask,
+			&payload, &copy_size);
+	if (ret)
+		return ret;
+
+	bs_mask = (bdev_logical_block_size(dest_bdev) >> 9) - 1;
+	if (dest & bs_mask) {
+		return -EINVAL;
+		goto out;
+	}
+
+	if (q == dest_q && q->limits.copy_offload) {
+		ret = blk_copy_offload(src_bdev, payload, dest, gfp_mask);
+		if (ret)
+			goto out;
+	} else if (flags & BLKDEV_COPY_NOEMULATION) {
+		ret = -EIO;
+		goto out;
+	} else
+		ret = blk_copy_emulate(src_bdev, payload, dest_bdev, dest,
+				copy_size, gfp_mask);
+
+out:
+	kvfree(payload);
+	return ret;
+}
+EXPORT_SYMBOL(blkdev_issue_copy);
+
 /**
  * __blkdev_issue_write_same - generate number of bios with same page
  * @bdev:	target blockdev
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 808768f6b174..4e04f24e13c1 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -309,6 +309,8 @@ void __blk_queue_split(struct bio **bio, unsigned int *nr_segs)
 	struct bio *split = NULL;
 
 	switch (bio_op(*bio)) {
+	case REQ_OP_COPY:
+			break;
 	case REQ_OP_DISCARD:
 	case REQ_OP_SECURE_ERASE:
 		split = blk_bio_discard_split(q, *bio, &q->bio_split, nr_segs);
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 43990b1d148b..93c15ba45a69 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -60,6 +60,10 @@ void blk_set_default_limits(struct queue_limits *lim)
 	lim->io_opt = 0;
 	lim->misaligned = 0;
 	lim->zoned = BLK_ZONED_NONE;
+	lim->copy_offload = 0;
+	lim->max_copy_sectors = 0;
+	lim->max_copy_nr_ranges = 0;
+	lim->max_copy_range_sectors = 0;
 }
 EXPORT_SYMBOL(blk_set_default_limits);
 
@@ -565,6 +569,12 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
 	if (b->chunk_sectors)
 		t->chunk_sectors = gcd(t->chunk_sectors, b->chunk_sectors);
 
+	/* simple copy not supported in stacked devices */
+	t->copy_offload = 0;
+	t->max_copy_sectors = 0;
+	t->max_copy_range_sectors = 0;
+	t->max_copy_nr_ranges = 0;
+
 	/* Physical block size a multiple of the logical block size? */
 	if (t->physical_block_size & (t->logical_block_size - 1)) {
 		t->physical_block_size = t->logical_block_size;
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index b513f1683af0..625a72541263 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -166,6 +166,44 @@ static ssize_t queue_discard_granularity_show(struct request_queue *q, char *pag
 	return queue_var_show(q->limits.discard_granularity, page);
 }
 
+static ssize_t queue_copy_offload_show(struct request_queue *q, char *page)
+{
+	return queue_var_show(q->limits.copy_offload, page);
+}
+
+static ssize_t queue_copy_offload_store(struct request_queue *q,
+				       const char *page, size_t count)
+{
+	unsigned long copy_offload;
+	ssize_t ret = queue_var_store(&copy_offload, page, count);
+
+	if (ret < 0)
+		return ret;
+
+	if (copy_offload && q->limits.max_copy_sectors == 0)
+		return -EINVAL;
+
+	q->limits.copy_offload = copy_offload;
+	return ret;
+}
+
+static ssize_t queue_max_copy_sectors_show(struct request_queue *q, char *page)
+{
+	return queue_var_show(q->limits.max_copy_sectors, page);
+}
+
+static ssize_t queue_max_copy_range_sectors_show(struct request_queue *q,
+		char *page)
+{
+	return queue_var_show(q->limits.max_copy_range_sectors, page);
+}
+
+static ssize_t queue_max_copy_nr_ranges_show(struct request_queue *q,
+		char *page)
+{
+	return queue_var_show(q->limits.max_copy_nr_ranges, page);
+}
+
 static ssize_t queue_discard_max_hw_show(struct request_queue *q, char *page)
 {
 
@@ -591,6 +629,11 @@ QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones");
 QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones");
 QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones");
 
+QUEUE_RW_ENTRY(queue_copy_offload, "copy_offload");
+QUEUE_RO_ENTRY(queue_max_copy_sectors, "max_copy_sectors");
+QUEUE_RO_ENTRY(queue_max_copy_range_sectors, "max_copy_range_sectors");
+QUEUE_RO_ENTRY(queue_max_copy_nr_ranges, "max_copy_nr_ranges");
+
 QUEUE_RW_ENTRY(queue_nomerges, "nomerges");
 QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity");
 QUEUE_RW_ENTRY(queue_poll, "io_poll");
@@ -636,6 +679,10 @@ static struct attribute *queue_attrs[] = {
 	&queue_discard_max_entry.attr,
 	&queue_discard_max_hw_entry.attr,
 	&queue_discard_zeroes_data_entry.attr,
+	&queue_copy_offload_entry.attr,
+	&queue_max_copy_sectors_entry.attr,
+	&queue_max_copy_range_sectors_entry.attr,
+	&queue_max_copy_nr_ranges_entry.attr,
 	&queue_write_same_max_entry.attr,
 	&queue_write_zeroes_max_entry.attr,
 	&queue_zone_append_max_entry.attr,
diff --git a/block/blk-zoned.c b/block/blk-zoned.c
index 7a68b6e4300c..02069178d51e 100644
--- a/block/blk-zoned.c
+++ b/block/blk-zoned.c
@@ -75,6 +75,7 @@ bool blk_req_needs_zone_write_lock(struct request *rq)
 	case REQ_OP_WRITE_ZEROES:
 	case REQ_OP_WRITE_SAME:
 	case REQ_OP_WRITE:
+	case REQ_OP_COPY:
 		return blk_rq_zone_is_seq(rq);
 	default:
 		return false;
diff --git a/block/bounce.c b/block/bounce.c
index d3f51acd6e3b..5e052afe8691 100644
--- a/block/bounce.c
+++ b/block/bounce.c
@@ -254,6 +254,7 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,
 	bio->bi_iter.bi_size	= bio_src->bi_iter.bi_size;
 
 	switch (bio_op(bio)) {
+	case REQ_OP_COPY:
 	case REQ_OP_DISCARD:
 	case REQ_OP_SECURE_ERASE:
 	case REQ_OP_WRITE_ZEROES:
diff --git a/block/ioctl.c b/block/ioctl.c
index d61d652078f4..0e52181657a4 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -133,6 +133,37 @@ static int blk_ioctl_discard(struct block_device *bdev, fmode_t mode,
 				    GFP_KERNEL, flags);
 }
 
+static int blk_ioctl_copy(struct block_device *bdev, fmode_t mode,
+		unsigned long arg, unsigned long flags)
+{
+	struct copy_range crange;
+	struct range_entry *rlist;
+	int ret;
+
+	if (!(mode & FMODE_WRITE))
+		return -EBADF;
+
+	if (copy_from_user(&crange, (void __user *)arg, sizeof(crange)))
+		return -EFAULT;
+
+	rlist = kmalloc_array(crange.nr_range, sizeof(*rlist),
+			GFP_KERNEL);
+	if (!rlist)
+		return -ENOMEM;
+
+	if (copy_from_user(rlist, (void __user *)crange.range_list,
+				sizeof(*rlist) * crange.nr_range)) {
+		ret = -EFAULT;
+		goto out;
+	}
+
+	ret = blkdev_issue_copy(bdev, crange.nr_range, rlist, bdev, crange.dest,
+			GFP_KERNEL, flags);
+out:
+	kfree(rlist);
+	return ret;
+}
+
 static int blk_ioctl_zeroout(struct block_device *bdev, fmode_t mode,
 		unsigned long arg)
 {
@@ -458,6 +489,8 @@ static int blkdev_common_ioctl(struct block_device *bdev, fmode_t mode,
 	case BLKSECDISCARD:
 		return blk_ioctl_discard(bdev, mode, arg,
 				BLKDEV_DISCARD_SECURE);
+	case BLKCOPY:
+		return blk_ioctl_copy(bdev, mode, arg, 0);
 	case BLKZEROOUT:
 		return blk_ioctl_zeroout(bdev, mode, arg);
 	case BLKREPORTZONE:
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 1edda614f7ce..164313bdfb35 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -71,6 +71,7 @@ static inline bool bio_has_data(struct bio *bio)
 static inline bool bio_no_advance_iter(const struct bio *bio)
 {
 	return bio_op(bio) == REQ_OP_DISCARD ||
+	       bio_op(bio) == REQ_OP_COPY ||
 	       bio_op(bio) == REQ_OP_SECURE_ERASE ||
 	       bio_op(bio) == REQ_OP_WRITE_SAME ||
 	       bio_op(bio) == REQ_OP_WRITE_ZEROES;
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 866f74261b3b..5a35c02ac0a8 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -380,6 +380,8 @@ enum req_opf {
 	REQ_OP_ZONE_RESET	= 15,
 	/* reset all the zone present on the device */
 	REQ_OP_ZONE_RESET_ALL	= 17,
+	/* copy ranges within device */
+	REQ_OP_COPY		= 19,
 
 	/* SCSI passthrough using struct scsi_request */
 	REQ_OP_SCSI_IN		= 32,
@@ -506,6 +508,11 @@ static inline bool op_is_discard(unsigned int op)
 	return (op & REQ_OP_MASK) == REQ_OP_DISCARD;
 }
 
+static inline bool op_is_copy(unsigned int op)
+{
+	return (op & REQ_OP_MASK) == REQ_OP_COPY;
+}
+
 /*
  * Check if a bio or request operation is a zone management operation, with
  * the exception of REQ_OP_ZONE_RESET_ALL which is treated as a special case
@@ -565,4 +572,11 @@ struct blk_rq_stat {
 	u64 batch;
 };
 
+struct blk_copy_payload {
+	sector_t	dest;
+	int		copy_nr_ranges;
+	struct block_device *src_bdev;
+	struct	range_entry	range[];
+};
+
 #endif /* __LINUX_BLK_TYPES_H */
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 699ace6b25ff..2bb4513d4bb8 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -337,10 +337,14 @@ struct queue_limits {
 	unsigned int		max_zone_append_sectors;
 	unsigned int		discard_granularity;
 	unsigned int		discard_alignment;
+	unsigned int		copy_offload;
+	unsigned int		max_copy_sectors;
 
 	unsigned short		max_segments;
 	unsigned short		max_integrity_segments;
 	unsigned short		max_discard_segments;
+	unsigned short		max_copy_range_sectors;
+	unsigned short		max_copy_nr_ranges;
 
 	unsigned char		misaligned;
 	unsigned char		discard_misaligned;
@@ -621,6 +625,7 @@ struct request_queue {
 #define QUEUE_FLAG_RQ_ALLOC_TIME 27	/* record rq->alloc_time_ns */
 #define QUEUE_FLAG_HCTX_ACTIVE	28	/* at least one blk-mq hctx is active */
 #define QUEUE_FLAG_NOWAIT       29	/* device supports NOWAIT */
+#define QUEUE_FLAG_SIMPLE_COPY	30	/* supports simple copy */
 
 #define QUEUE_FLAG_MQ_DEFAULT	((1 << QUEUE_FLAG_IO_STAT) |		\
 				 (1 << QUEUE_FLAG_SAME_COMP) |		\
@@ -643,6 +648,7 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q);
 #define blk_queue_io_stat(q)	test_bit(QUEUE_FLAG_IO_STAT, &(q)->queue_flags)
 #define blk_queue_add_random(q)	test_bit(QUEUE_FLAG_ADD_RANDOM, &(q)->queue_flags)
 #define blk_queue_discard(q)	test_bit(QUEUE_FLAG_DISCARD, &(q)->queue_flags)
+#define blk_queue_copy(q)	test_bit(QUEUE_FLAG_SIMPLE_COPY, &(q)->queue_flags)
 #define blk_queue_zone_resetall(q)	\
 	test_bit(QUEUE_FLAG_ZONE_RESETALL, &(q)->queue_flags)
 #define blk_queue_secure_erase(q) \
@@ -1069,6 +1075,9 @@ static inline unsigned int blk_queue_get_max_sectors(struct request_queue *q,
 		return min(q->limits.max_discard_sectors,
 			   UINT_MAX >> SECTOR_SHIFT);
 
+	if (unlikely(op == REQ_OP_COPY))
+		return q->limits.max_copy_sectors;
+
 	if (unlikely(op == REQ_OP_WRITE_SAME))
 		return q->limits.max_write_same_sectors;
 
@@ -1343,6 +1352,12 @@ extern int __blkdev_issue_discard(struct block_device *bdev, sector_t sector,
 		sector_t nr_sects, gfp_t gfp_mask, int flags,
 		struct bio **biop);
 
+#define BLKDEV_COPY_NOEMULATION	(1 << 0)	/* do not emulate if copy offload not supported */
+
+extern int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs,
+		struct range_entry *src_rlist, struct block_device *dest_bdev,
+		sector_t dest, gfp_t gfp_mask, int flags);
+
 #define BLKDEV_ZERO_NOUNMAP	(1 << 0)  /* do not free blocks */
 #define BLKDEV_ZERO_NOFALLBACK	(1 << 1)  /* don't write explicit zeroes */
 
diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index f44eb0a04afd..5cadb176317a 100644
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -64,6 +64,18 @@ struct fstrim_range {
 	__u64 minlen;
 };
 
+struct range_entry {
+	__u64 src;
+	__u64 len;
+};
+
+struct copy_range {
+	__u64 dest;
+	__u64 nr_range;
+	__u64 range_list;
+	__u64 rsvd;
+};
+
 /* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */
 #define FILE_DEDUPE_RANGE_SAME		0
 #define FILE_DEDUPE_RANGE_DIFFERS	1
@@ -184,6 +196,7 @@ struct fsxattr {
 #define BLKSECDISCARD _IO(0x12,125)
 #define BLKROTATIONAL _IO(0x12,126)
 #define BLKZEROOUT _IO(0x12,127)
+#define BLKCOPY _IOWR(0x12, 128, struct copy_range)
 /*
  * A jump here: 130-131 are reserved for zoned block devices
  * (see uapi/linux/blkzoned.h)
-- 
2.25.1



From selvakuma.s1 at samsung.com  Fri Feb 19 07:45:16 2021
From: selvakuma.s1 at samsung.com (SelvaKumar S)
Date: Fri, 19 Feb 2021 18:15:16 +0530
Subject: [RFC PATCH v5 3/4] nvme: add simple copy support
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <CGME20210219124608epcas5p2a673f9e00c3e7b5352f115497b0e2d98@epcas5p2.samsung.com>
Message-ID: <20210219124517.79359-4-selvakuma.s1@samsung.com>

Add support for  TP 4065a ("Simple Copy Command"), v2020.05.04
("Ratified")

For device supporting native simple copy, this implementation accepts
the payload passed from the block layer and convert payload to form
simple copy command and submit to the device.

Set the device copy limits to queue limits. By default copy_offload
is disabled.

End-to-end protection is done by setting both PRINFOR and PRINFOW
to 0.

Signed-off-by: SelvaKumar S <selvakuma.s1 at samsung.com>
Signed-off-by: Kanchan Joshi <joshi.k at samsung.com>
Signed-off-by: Nitesh Shetty <nj.shetty at samsung.com>
Signed-off-by: Javier Gonz?lez <javier.gonz at samsung.com>
---
 drivers/nvme/host/core.c | 87 ++++++++++++++++++++++++++++++++++++++++
 include/linux/nvme.h     | 43 ++++++++++++++++++--
 2 files changed, 127 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f13eb4ded95f..ba4de2f36cd5 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -706,6 +706,63 @@ static inline void nvme_setup_flush(struct nvme_ns *ns,
 	cmnd->common.nsid = cpu_to_le32(ns->head->ns_id);
 }
 
+static inline blk_status_t nvme_setup_copy(struct nvme_ns *ns,
+	       struct request *req, struct nvme_command *cmnd)
+{
+	struct nvme_ctrl *ctrl = ns->ctrl;
+	struct nvme_copy_range *range = NULL;
+	struct blk_copy_payload *payload;
+	unsigned short nr_range = 0;
+	u16 control = 0, ssrl;
+	u32 dsmgmt = 0;
+	u64 slba;
+	int i;
+
+	payload = bio_data(req->bio);
+	nr_range = payload->copy_nr_ranges;
+
+	if (req->cmd_flags & REQ_FUA)
+		control |= NVME_RW_FUA;
+
+	if (req->cmd_flags & REQ_FAILFAST_DEV)
+		control |= NVME_RW_LR;
+
+	cmnd->copy.opcode = nvme_cmd_copy;
+	cmnd->copy.nsid = cpu_to_le32(ns->head->ns_id);
+	cmnd->copy.sdlba = cpu_to_le64(blk_rq_pos(req) >> (ns->lba_shift - 9));
+
+	range = kmalloc_array(nr_range, sizeof(*range),
+			GFP_ATOMIC | __GFP_NOWARN);
+	if (!range)
+		return BLK_STS_RESOURCE;
+
+	for (i = 0; i < nr_range; i++) {
+		slba = payload->range[i].src;
+		slba = slba >> (ns->lba_shift - 9);
+
+		ssrl = payload->range[i].len;
+		ssrl = ssrl >> (ns->lba_shift - 9);
+
+		range[i].slba = cpu_to_le64(slba);
+		range[i].nlb = cpu_to_le16(ssrl - 1);
+	}
+
+	cmnd->copy.nr_range = nr_range - 1;
+
+	req->special_vec.bv_page = virt_to_page(range);
+	req->special_vec.bv_offset = offset_in_page(range);
+	req->special_vec.bv_len = sizeof(*range) * nr_range;
+	req->rq_flags |= RQF_SPECIAL_PAYLOAD;
+
+	if (ctrl->nr_streams)
+		nvme_assign_write_stream(ctrl, req, &control, &dsmgmt);
+
+	cmnd->rw.control = cpu_to_le16(control);
+	cmnd->rw.dsmgmt = cpu_to_le32(dsmgmt);
+
+	return BLK_STS_OK;
+}
+
 static blk_status_t nvme_setup_discard(struct nvme_ns *ns, struct request *req,
 		struct nvme_command *cmnd)
 {
@@ -888,6 +945,9 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req,
 	case REQ_OP_DISCARD:
 		ret = nvme_setup_discard(ns, req, cmd);
 		break;
+	case REQ_OP_COPY:
+		ret = nvme_setup_copy(ns, req, cmd);
+		break;
 	case REQ_OP_READ:
 		ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_read);
 		break;
@@ -1928,6 +1988,31 @@ static void nvme_config_discard(struct gendisk *disk, struct nvme_ns *ns)
 		blk_queue_max_write_zeroes_sectors(queue, UINT_MAX);
 }
 
+static void nvme_config_copy(struct gendisk *disk, struct nvme_ns *ns,
+				       struct nvme_id_ns *id)
+{
+	struct nvme_ctrl *ctrl = ns->ctrl;
+	struct request_queue *queue = disk->queue;
+
+	if (!(ctrl->oncs & NVME_CTRL_ONCS_COPY)) {
+		queue->limits.copy_offload = 0;
+		queue->limits.max_copy_sectors = 0;
+		queue->limits.max_copy_range_sectors = 0;
+		queue->limits.max_copy_nr_ranges = 0;
+		blk_queue_flag_clear(QUEUE_FLAG_SIMPLE_COPY, queue);
+		return;
+	}
+
+	/* setting copy limits */
+	blk_queue_flag_test_and_set(QUEUE_FLAG_SIMPLE_COPY, queue);
+	queue->limits.copy_offload = 0;
+	queue->limits.max_copy_sectors = le64_to_cpu(id->mcl) *
+		(1 << (ns->lba_shift - 9));
+	queue->limits.max_copy_range_sectors = le32_to_cpu(id->mssrl) *
+		(1 << (ns->lba_shift - 9));
+	queue->limits.max_copy_nr_ranges = id->msrc + 1;
+}
+
 static void nvme_config_write_zeroes(struct gendisk *disk, struct nvme_ns *ns)
 {
 	u64 max_blocks;
@@ -2123,6 +2208,7 @@ static void nvme_update_disk_info(struct gendisk *disk,
 	set_capacity_and_notify(disk, capacity);
 
 	nvme_config_discard(disk, ns);
+	nvme_config_copy(disk, ns, id);
 	nvme_config_write_zeroes(disk, ns);
 
 	if ((id->nsattr & NVME_NS_ATTR_RO) ||
@@ -4705,6 +4791,7 @@ static inline void _nvme_check_size(void)
 	BUILD_BUG_ON(sizeof(struct nvme_download_firmware) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_format_cmd) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_dsm_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct nvme_copy_command) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_write_zeroes_cmd) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_abort_cmd) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_get_log_page_command) != 64);
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index bfed36e342cc..c36e486cbe18 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -295,7 +295,7 @@ struct nvme_id_ctrl {
 	__u8			nvscc;
 	__u8			nwpc;
 	__le16			acwu;
-	__u8			rsvd534[2];
+	__le16			ocfs;
 	__le32			sgls;
 	__le32			mnan;
 	__u8			rsvd544[224];
@@ -320,6 +320,7 @@ enum {
 	NVME_CTRL_ONCS_WRITE_ZEROES		= 1 << 3,
 	NVME_CTRL_ONCS_RESERVATIONS		= 1 << 5,
 	NVME_CTRL_ONCS_TIMESTAMP		= 1 << 6,
+	NVME_CTRL_ONCS_COPY			= 1 << 8,
 	NVME_CTRL_VWC_PRESENT			= 1 << 0,
 	NVME_CTRL_OACS_SEC_SUPP                 = 1 << 0,
 	NVME_CTRL_OACS_DIRECTIVES		= 1 << 5,
@@ -368,7 +369,10 @@ struct nvme_id_ns {
 	__le16			npdg;
 	__le16			npda;
 	__le16			nows;
-	__u8			rsvd74[18];
+	__le16			mssrl;
+	__le32			mcl;
+	__u8			msrc;
+	__u8			rsvd91[11];
 	__le32			anagrpid;
 	__u8			rsvd96[3];
 	__u8			nsattr;
@@ -679,6 +683,7 @@ enum nvme_opcode {
 	nvme_cmd_resv_report	= 0x0e,
 	nvme_cmd_resv_acquire	= 0x11,
 	nvme_cmd_resv_release	= 0x15,
+	nvme_cmd_copy		= 0x19,
 	nvme_cmd_zone_mgmt_send	= 0x79,
 	nvme_cmd_zone_mgmt_recv	= 0x7a,
 	nvme_cmd_zone_append	= 0x7d,
@@ -697,7 +702,8 @@ enum nvme_opcode {
 		nvme_opcode_name(nvme_cmd_resv_register),	\
 		nvme_opcode_name(nvme_cmd_resv_report),		\
 		nvme_opcode_name(nvme_cmd_resv_acquire),	\
-		nvme_opcode_name(nvme_cmd_resv_release))
+		nvme_opcode_name(nvme_cmd_resv_release),	\
+		nvme_opcode_name(nvme_cmd_copy))
 
 
 /*
@@ -869,6 +875,36 @@ struct nvme_dsm_range {
 	__le64			slba;
 };
 
+struct nvme_copy_command {
+	__u8                    opcode;
+	__u8                    flags;
+	__u16                   command_id;
+	__le32                  nsid;
+	__u64                   rsvd2;
+	__le64                  metadata;
+	union nvme_data_ptr     dptr;
+	__le64                  sdlba;
+	__u8			nr_range;
+	__u8			rsvd12;
+	__le16                  control;
+	__le16                  rsvd13;
+	__le16			dspec;
+	__le32                  ilbrt;
+	__le16                  lbat;
+	__le16                  lbatm;
+};
+
+struct nvme_copy_range {
+	__le64			rsvd0;
+	__le64			slba;
+	__le16			nlb;
+	__le16			rsvd18;
+	__le32			rsvd20;
+	__le32			eilbrt;
+	__le16			elbat;
+	__le16			elbatm;
+};
+
 struct nvme_write_zeroes_cmd {
 	__u8			opcode;
 	__u8			flags;
@@ -1406,6 +1442,7 @@ struct nvme_command {
 		struct nvme_download_firmware dlfw;
 		struct nvme_format_cmd format;
 		struct nvme_dsm_cmd dsm;
+		struct nvme_copy_command copy;
 		struct nvme_write_zeroes_cmd write_zeroes;
 		struct nvme_zone_mgmt_send_cmd zms;
 		struct nvme_zone_mgmt_recv_cmd zmr;
-- 
2.25.1



From selvakuma.s1 at samsung.com  Fri Feb 19 07:45:17 2021
From: selvakuma.s1 at samsung.com (SelvaKumar S)
Date: Fri, 19 Feb 2021 18:15:17 +0530
Subject: [RFC PATCH v5 4/4] dm kcopyd: add simple copy offload support
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <CGME20210219124611epcas5p1c775b63b537e75da161556e375fcf05e@epcas5p1.samsung.com>
Message-ID: <20210219124517.79359-5-selvakuma.s1@samsung.com>

Introduce copy_jobs to use copy-offload if it is natively
supported (by underlying device) or else fall back to original method.

run_copy_jobs() calls block layer copy offload api with
BLKDEV_COPY_NOEMULATION. On successful completion, if only one destination
device was present, then jobs is queued for completion. If multiple
destinations were present, the completed destination is zeroed and
pushed to pages_jobs to process copy offload for other destinations. In
case of copy_offload failure, remaining destinations are processed via
regular copying mechanism.

Signed-off-by: SelvaKumar S <selvakuma.s1 at samsung.com>
---
 drivers/md/dm-kcopyd.c | 49 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 43 insertions(+), 6 deletions(-)

diff --git a/drivers/md/dm-kcopyd.c b/drivers/md/dm-kcopyd.c
index 1bbe4a34ef4c..2442c4870e97 100644
--- a/drivers/md/dm-kcopyd.c
+++ b/drivers/md/dm-kcopyd.c
@@ -74,18 +74,20 @@ struct dm_kcopyd_client {
 	atomic_t nr_jobs;
 
 /*
- * We maintain four lists of jobs:
+ * We maintain five lists of jobs:
  *
- * i)   jobs waiting for pages
- * ii)  jobs that have pages, and are waiting for the io to be issued.
- * iii) jobs that don't need to do any IO and just run a callback
- * iv) jobs that have completed.
+ * i)	jobs waiting to try copy offload
+ * ii)   jobs waiting for pages
+ * iii)  jobs that have pages, and are waiting for the io to be issued.
+ * iv) jobs that don't need to do any IO and just run a callback
+ * v) jobs that have completed.
  *
- * All four of these are protected by job_lock.
+ * All five of these are protected by job_lock.
  */
 	spinlock_t job_lock;
 	struct list_head callback_jobs;
 	struct list_head complete_jobs;
+	struct list_head copy_jobs;
 	struct list_head io_jobs;
 	struct list_head pages_jobs;
 };
@@ -581,6 +583,36 @@ static int run_io_job(struct kcopyd_job *job)
 	return r;
 }
 
+static int run_copy_job(struct kcopyd_job *job)
+{
+	int r, i, count = 0;
+	unsigned long flags = 0;
+	struct range_entry srange;
+
+	flags |= BLKDEV_COPY_NOEMULATION;
+	for (i = 0; i < job->num_dests; i++) {
+		srange.src = job->source.sector;
+		srange.len = job->source.count;
+
+		r = blkdev_issue_copy(job->source.bdev, 1, &srange,
+			job->dests[i].bdev, job->dests[i].sector, GFP_KERNEL, flags);
+		if (r)
+			break;
+
+		job->dests[i].count = 0;
+		count++;
+	}
+
+	if (count == job->num_dests) {
+		push(&job->kc->complete_jobs, job);
+	} else {
+		push(&job->kc->pages_jobs, job);
+		r = 0;
+	}
+
+	return r;
+}
+
 static int run_pages_job(struct kcopyd_job *job)
 {
 	int r;
@@ -662,6 +694,7 @@ static void do_work(struct work_struct *work)
 	spin_unlock_irqrestore(&kc->job_lock, flags);
 
 	blk_start_plug(&plug);
+	process_jobs(&kc->copy_jobs, kc, run_copy_job);
 	process_jobs(&kc->complete_jobs, kc, run_complete_job);
 	process_jobs(&kc->pages_jobs, kc, run_pages_job);
 	process_jobs(&kc->io_jobs, kc, run_io_job);
@@ -679,6 +712,8 @@ static void dispatch_job(struct kcopyd_job *job)
 	atomic_inc(&kc->nr_jobs);
 	if (unlikely(!job->source.count))
 		push(&kc->callback_jobs, job);
+	else if (job->source.bdev->bd_disk == job->dests[0].bdev->bd_disk)
+		push(&kc->copy_jobs, job);
 	else if (job->pages == &zero_page_list)
 		push(&kc->io_jobs, job);
 	else
@@ -919,6 +954,7 @@ struct dm_kcopyd_client *dm_kcopyd_client_create(struct dm_kcopyd_throttle *thro
 	spin_lock_init(&kc->job_lock);
 	INIT_LIST_HEAD(&kc->callback_jobs);
 	INIT_LIST_HEAD(&kc->complete_jobs);
+	INIT_LIST_HEAD(&kc->copy_jobs);
 	INIT_LIST_HEAD(&kc->io_jobs);
 	INIT_LIST_HEAD(&kc->pages_jobs);
 	kc->throttle = throttle;
@@ -974,6 +1010,7 @@ void dm_kcopyd_client_destroy(struct dm_kcopyd_client *kc)
 
 	BUG_ON(!list_empty(&kc->callback_jobs));
 	BUG_ON(!list_empty(&kc->complete_jobs));
+	BUG_ON(!list_empty(&kc->copy_jobs));
 	BUG_ON(!list_empty(&kc->io_jobs));
 	BUG_ON(!list_empty(&kc->pages_jobs));
 	destroy_workqueue(kc->kcopyd_wq);
-- 
2.25.1



From willy at infradead.org  Fri Feb 19 22:36:37 2021
From: willy at infradead.org (Matthew Wilcox)
Date: Sat, 20 Feb 2021 03:36:37 +0000
Subject: [RFC PATCH v5 3/4] nvme: add simple copy support
In-Reply-To: <20210219124517.79359-4-selvakuma.s1@samsung.com>
References: <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <CGME20210219124608epcas5p2a673f9e00c3e7b5352f115497b0e2d98@epcas5p2.samsung.com>
 <20210219124517.79359-4-selvakuma.s1@samsung.com>
Message-ID: <20210220033637.GA2858050@casper.infradead.org>

On Fri, Feb 19, 2021 at 06:15:16PM +0530, SelvaKumar S wrote:
> +	struct nvme_copy_range *range = NULL;
[...]
> +	range = kmalloc_array(nr_range, sizeof(*range),
> +			GFP_ATOMIC | __GFP_NOWARN);
[...]
> +	req->special_vec.bv_page = virt_to_page(range);
> +	req->special_vec.bv_offset = offset_in_page(range);
> +	req->special_vec.bv_len = sizeof(*range) * nr_range;
[...]
> +struct nvme_copy_range {
> +	__le64			rsvd0;
> +	__le64			slba;
> +	__le16			nlb;
> +	__le16			rsvd18;
> +	__le32			rsvd20;
> +	__le32			eilbrt;
> +	__le16			elbat;
> +	__le16			elbatm;
> +};

so ... at 32 bytes, you can get 128 per 4kB page.  What happens if you
try to send down a command that attempts to copy 129 ranges?


From Damien.LeMoal at wdc.com  Fri Feb 19 23:59:05 2021
From: Damien.LeMoal at wdc.com (Damien Le Moal)
Date: Sat, 20 Feb 2021 04:59:05 +0000
Subject: [RFC PATCH v5 2/4] block: add simple copy support
References: <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <CGME20210219124603epcas5p33add0f2c1781b2a4d71bf30c9e1ac647@epcas5p3.samsung.com>
 <20210219124517.79359-3-selvakuma.s1@samsung.com>
Message-ID: <BL0PR04MB6514EA91680D33A5890ECE16E7839@BL0PR04MB6514.namprd04.prod.outlook.com>

On 2021/02/20 11:01, SelvaKumar S wrote:
> Add new BLKCOPY ioctl that offloads copying of one or more sources
> ranges to a destination in the device. Accepts a 'copy_range' structure
> that contains destination (in sectors), no of sources and pointer to the
> array of source ranges. Each source range is represented by 'range_entry'
> that contains start and length of source ranges (in sectors).
> 
> Introduce REQ_OP_COPY, a no-merge copy offload operation. Create
> bio with control information as payload and submit to the device.
> REQ_OP_COPY(19) is a write op and takes zone_write_lock when submitted
> to zoned device.
> 
> If the device doesn't support copy or copy offload is disabled, then
> copy operation is emulated by default. However, the copy-emulation is an
> opt-in feature. Caller can choose not to use the copy-emulation by
> specifying a flag 'BLKDEV_COPY_NOEMULATION'.
> 
> Copy-emulation is implemented by allocating memory of total copy size.
> The source ranges are read into memory by chaining bio for each source
> ranges and submitting them async and the last bio waits for completion.
> After data is read, it is written to the destination.
> 
> bio_map_kern() is used to allocate bio and add pages of copy buffer to
> bio. As bio->bi_private and bio->bi_end_io are needed for chaining the
> bio and gets over-written, invalidate_kernel_vmap_range() for read is
> called in the caller.
> 
> Introduce queue limits for simple copy and other helper functions.
> Add device limits as sysfs entries.
> 	- copy_offload
> 	- max_copy_sectors
> 	- max_copy_ranges_sectors
> 	- max_copy_nr_ranges
> 
> copy_offload(= 0) is disabled by default. This needs to be enabled if
> copy-offload needs to be used.
> max_copy_sectors = 0 indicates the device doesn't support native copy.
> 
> Native copy offload is not supported for stacked devices and is done via
> copy emulation.
> 
> Signed-off-by: SelvaKumar S <selvakuma.s1 at samsung.com>
> Signed-off-by: Kanchan Joshi <joshi.k at samsung.com>
> Signed-off-by: Nitesh Shetty <nj.shetty at samsung.com>
> Signed-off-by: Javier Gonz?lez <javier.gonz at samsung.com>
> Signed-off-by: Chaitanya Kulkarni <kch at kernel.org>
> ---
>  block/blk-core.c          | 102 ++++++++++++++++--
>  block/blk-lib.c           | 222 ++++++++++++++++++++++++++++++++++++++
>  block/blk-merge.c         |   2 +
>  block/blk-settings.c      |  10 ++
>  block/blk-sysfs.c         |  47 ++++++++
>  block/blk-zoned.c         |   1 +
>  block/bounce.c            |   1 +
>  block/ioctl.c             |  33 ++++++
>  include/linux/bio.h       |   1 +
>  include/linux/blk_types.h |  14 +++
>  include/linux/blkdev.h    |  15 +++
>  include/uapi/linux/fs.h   |  13 +++
>  12 files changed, 453 insertions(+), 8 deletions(-)
> 
> diff --git a/block/blk-core.c b/block/blk-core.c
> index 7663a9b94b80..23e646e5ae43 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -720,6 +720,17 @@ static noinline int should_fail_bio(struct bio *bio)
>  }
>  ALLOW_ERROR_INJECTION(should_fail_bio, ERRNO);
>  
> +static inline int bio_check_copy_eod(struct bio *bio, sector_t start,
> +		sector_t nr_sectors, sector_t max_sect)
> +{
> +	if (nr_sectors && max_sect &&
> +	    (nr_sectors > max_sect || start > max_sect - nr_sectors)) {
> +		handle_bad_sector(bio, max_sect);
> +		return -EIO;
> +	}
> +	return 0;
> +}
> +
>  /*
>   * Check whether this bio extends beyond the end of the device or partition.
>   * This may well happen - the kernel calls bread() without checking the size of
> @@ -738,6 +749,75 @@ static inline int bio_check_eod(struct bio *bio, sector_t maxsector)
>  	return 0;
>  }
>  
> +/*
> + * Check for copy limits and remap source ranges if needed.
> + */
> +static int blk_check_copy(struct bio *bio)
> +{
> +	struct blk_copy_payload *payload = bio_data(bio);
> +	struct request_queue *q = bio->bi_disk->queue;
> +	sector_t max_sect, start_sect, copy_size = 0;
> +	sector_t src_max_sect, src_start_sect;
> +	struct block_device *bd_part;
> +	int i, ret = -EIO;
> +
> +	rcu_read_lock();
> +
> +	bd_part = __disk_get_part(bio->bi_disk, bio->bi_partno);
> +	if (unlikely(!bd_part)) {
> +		rcu_read_unlock();
> +		goto out;
> +	}
> +
> +	max_sect =  bdev_nr_sectors(bd_part);
> +	start_sect = bd_part->bd_start_sect;
> +
> +	src_max_sect = bdev_nr_sectors(payload->src_bdev);
> +	src_start_sect = payload->src_bdev->bd_start_sect;
> +
> +	if (unlikely(should_fail_request(bd_part, bio->bi_iter.bi_size)))
> +		goto out;
> +
> +	if (unlikely(bio_check_ro(bio, bd_part)))
> +		goto out;

There is no rcu_unlock() in that out label. Did you test ?

> +
> +	rcu_read_unlock();
> +
> +	/* cannot handle copy crossing nr_ranges limit */
> +	if (payload->copy_nr_ranges > q->limits.max_copy_nr_ranges)
> +		goto out;
> +
> +	for (i = 0; i < payload->copy_nr_ranges; i++) {
> +		ret = bio_check_copy_eod(bio, payload->range[i].src,
> +				payload->range[i].len, src_max_sect);
> +		if (unlikely(ret))
> +			goto out;
> +
> +		/* single source range length limit */
> +		if (payload->range[i].len > q->limits.max_copy_range_sectors)
> +			goto out;

ret is not set. You will return success with this.

> +
> +		payload->range[i].src += src_start_sect;
> +		copy_size += payload->range[i].len;
> +	}
> +
> +	/* check if copy length crosses eod */
> +	ret = bio_check_copy_eod(bio, bio->bi_iter.bi_sector,
> +				copy_size, max_sect);
> +	if (unlikely(ret))
> +		goto out;
> +
> +	/* cannot handle copy more than copy limits */
> +	if (copy_size > q->limits.max_copy_sectors)
> +		goto out;

Again ret is not set... No error return ?

> +
> +	bio->bi_iter.bi_sector += start_sect;
> +	bio->bi_partno = 0;
> +	ret = 0;
> +out:
> +	return ret;
> +}
> +
>  /*
>   * Remap block n of partition p to block n+start(p) of the disk.
>   */
> @@ -827,14 +907,16 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio)
>  	if (should_fail_bio(bio))
>  		goto end_io;
>  
> -	if (bio->bi_partno) {
> -		if (unlikely(blk_partition_remap(bio)))
> -			goto end_io;
> -	} else {
> -		if (unlikely(bio_check_ro(bio, bio->bi_disk->part0)))
> -			goto end_io;
> -		if (unlikely(bio_check_eod(bio, get_capacity(bio->bi_disk))))
> -			goto end_io;
> +	if (likely(!op_is_copy(bio->bi_opf))) {
> +		if (bio->bi_partno) {
> +			if (unlikely(blk_partition_remap(bio)))
> +				goto end_io;
> +		} else {
> +			if (unlikely(bio_check_ro(bio, bio->bi_disk->part0)))
> +				goto end_io;
> +			if (unlikely(bio_check_eod(bio, get_capacity(bio->bi_disk))))
> +				goto end_io;
> +		}
>  	}
>  
>  	/*
> @@ -858,6 +940,10 @@ static noinline_for_stack bool submit_bio_checks(struct bio *bio)
>  		if (!blk_queue_discard(q))
>  			goto not_supported;
>  		break;
> +	case REQ_OP_COPY:
> +		if (unlikely(blk_check_copy(bio)))
> +			goto end_io;
> +		break;
>  	case REQ_OP_SECURE_ERASE:
>  		if (!blk_queue_secure_erase(q))
>  			goto not_supported;
> diff --git a/block/blk-lib.c b/block/blk-lib.c
> index 752f9c722062..97ba58d8d9a1 100644
> --- a/block/blk-lib.c
> +++ b/block/blk-lib.c
> @@ -150,6 +150,228 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector,
>  }
>  EXPORT_SYMBOL(blkdev_issue_discard);
>  
> +int blk_copy_offload(struct block_device *dest_bdev, struct blk_copy_payload *payload,
> +		sector_t dest, gfp_t gfp_mask)

Simple copy is only over the same device, right ? So the name "dest_bdev" is a
little strange.

> +{
> +	struct request_queue *q = bdev_get_queue(dest_bdev);
> +	struct bio *bio;
> +	int ret, payload_size;
> +
> +	payload_size = struct_size(payload, range, payload->copy_nr_ranges);
> +	bio = bio_map_kern(q, payload, payload_size, gfp_mask);
> +	if (IS_ERR(bio)) {
> +		ret = PTR_ERR(bio);
> +		goto err;

This will do a bio_put() on a non existent bio...

> +	}
> +
> +	bio->bi_iter.bi_sector = dest;
> +	bio->bi_opf = REQ_OP_COPY | REQ_NOMERGE;
> +	bio_set_dev(bio, dest_bdev);
> +
> +	ret = submit_bio_wait(bio);
> +err:
> +	bio_put(bio);
> +	return ret;
> +}
> +
> +int blk_read_to_buf(struct block_device *src_bdev, struct blk_copy_payload *payload,
> +		gfp_t gfp_mask, sector_t copy_size, void **buf_p)
> +{
> +	struct request_queue *q = bdev_get_queue(src_bdev);
> +	struct bio *bio, *parent = NULL;
> +	void *buf = NULL;
> +	int copy_len = copy_size << SECTOR_SHIFT;
> +	int i, nr_srcs, ret, cur_size, t_len = 0;
> +	bool is_vmalloc;
> +
> +	nr_srcs = payload->copy_nr_ranges;
> +
> +	buf = kvmalloc(copy_len, gfp_mask);
> +	if (!buf)
> +		return -ENOMEM;
> +	is_vmalloc = is_vmalloc_addr(buf);
> +
> +	for (i = 0; i < nr_srcs; i++) {
> +		cur_size = payload->range[i].len << SECTOR_SHIFT;
> +
> +		bio = bio_map_kern(q, buf + t_len, cur_size, gfp_mask);
> +		if (IS_ERR(bio)) {
> +			ret = PTR_ERR(bio);
> +			goto out;
> +		}
> +
> +		bio->bi_iter.bi_sector = payload->range[i].src;
> +		bio->bi_opf = REQ_OP_READ;
> +		bio_set_dev(bio, src_bdev);
> +		bio->bi_end_io = NULL;
> +		bio->bi_private = NULL;
> +
> +		if (parent) {
> +			bio_chain(parent, bio);
> +			submit_bio(parent);
> +		}
> +
> +		parent = bio;
> +		t_len += cur_size;
> +	}
> +
> +	ret = submit_bio_wait(bio);
> +	bio_put(bio);
> +	if (is_vmalloc)
> +		invalidate_kernel_vmap_range(buf, copy_len);

But blk_write_from_buf() will use the buffer right after this.. Is this really OK ?

> +	if (ret)
> +		goto out;
> +
> +	*buf_p = buf;
> +	return 0;
> +out:
> +	kvfree(buf);
> +	return ret;
> +}
> +
> +int blk_write_from_buf(struct block_device *dest_bdev, void *buf, sector_t dest,
> +		sector_t copy_size, gfp_t gfp_mask)
> +{
> +	struct request_queue *q = bdev_get_queue(dest_bdev);
> +	struct bio *bio;
> +	int ret, copy_len = copy_size << SECTOR_SHIFT;
> +
> +	bio = bio_map_kern(q, buf, copy_len, gfp_mask);
> +	if (IS_ERR(bio)) {
> +		ret = PTR_ERR(bio);
> +		goto out;
> +	}
> +	bio_set_dev(bio, dest_bdev);
> +	bio->bi_opf = REQ_OP_WRITE;
> +	bio->bi_iter.bi_sector = dest;
> +
> +	bio->bi_end_io = NULL;
> +	ret = submit_bio_wait(bio);
> +	bio_put(bio);
> +out:
> +	return ret;
> +}
> +
> +int blk_prepare_payload(struct block_device *src_bdev, int nr_srcs, struct range_entry *rlist,
> +		gfp_t gfp_mask, struct blk_copy_payload **payload_p, sector_t *copy_size)
> +{
> +
> +	struct request_queue *q = bdev_get_queue(src_bdev);
> +	struct blk_copy_payload *payload;
> +	sector_t bs_mask, total_len = 0;
> +	int i, ret, payload_size;
> +
> +	if (!q)
> +		return -ENXIO;
> +
> +	if (!nr_srcs)
> +		return -EINVAL;
> +
> +	if (bdev_read_only(src_bdev))
> +		return -EPERM;
> +
> +	bs_mask = (bdev_logical_block_size(src_bdev) >> 9) - 1;
> +
> +	payload_size = struct_size(payload, range, nr_srcs);
> +	payload = kmalloc(payload_size, gfp_mask);
> +	if (!payload)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < nr_srcs; i++) {
> +		if (rlist[i].src & bs_mask || rlist[i].len & bs_mask) {
> +			ret = -EINVAL;
> +			goto err;
> +		}
> +
> +		payload->range[i].src = rlist[i].src;
> +		payload->range[i].len = rlist[i].len;
> +
> +		total_len += rlist[i].len;
> +	}
> +
> +	payload->copy_nr_ranges = i;
> +	payload->src_bdev = src_bdev;
> +	*copy_size = total_len;
> +
> +	*payload_p = payload;
> +	return 0;
> +err:
> +	kfree(payload);
> +	return ret;
> +}
> +
> +int blk_copy_emulate(struct block_device *src_bdev, struct blk_copy_payload *payload,
> +			struct block_device *dest_bdev, sector_t dest,
> +			sector_t copy_size, gfp_t gfp_mask)
> +{
> +	void *buf = NULL;
> +	int ret;
> +
> +	ret = blk_read_to_buf(src_bdev, payload, gfp_mask, copy_size, &buf);
> +	if (ret)
> +		goto out;
> +
> +	ret = blk_write_from_buf(dest_bdev, buf, dest, copy_size, gfp_mask);
> +	if (buf)
> +		kvfree(buf);
> +out:
> +	return ret;
> +}

I already commented that this should better use the dm-kcopyd design, which
would be far more efficient than this. This will be slow...

Your function blkdev_issue_copy() below should deal only with issuing simple
copy (amd later scsi xcopy) for devices that support it. Bring the dm-kcopyd
interface in the block layer as a generic interface for hadling emulation.
Otherwise you are repeating what dm does, but not as efficiently.

> +
> +/**
> + * blkdev_issue_copy - queue a copy
> + * @src_bdev:	source block device
> + * @nr_srcs:	number of source ranges to copy
> + * @rlist:	array of source ranges in sector
> + * @dest_bdev:	destination block device
> + * @dest:	destination in sector
> + * @gfp_mask:   memory allocation flags (for bio_alloc)
> + * @flags:	BLKDEV_COPY_* flags to control behaviour
> + *
> + * Description:
> + *	Copy array of source ranges from source block device to
> + *	destination block devcie. All source must belong to same bdev and
> + *	length of a source range cannot be zero.
> + */
> +
> +int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs,
> +		struct range_entry *src_rlist, struct block_device *dest_bdev,
> +		sector_t dest, gfp_t gfp_mask, int flags)
> +{
> +	struct request_queue *q = bdev_get_queue(src_bdev);
> +	struct request_queue *dest_q = bdev_get_queue(dest_bdev);
> +	struct blk_copy_payload *payload;
> +	sector_t bs_mask, copy_size;
> +	int ret;
> +
> +	ret = blk_prepare_payload(src_bdev, nr_srcs, src_rlist, gfp_mask,
> +			&payload, &copy_size);
> +	if (ret)
> +		return ret;
> +
> +	bs_mask = (bdev_logical_block_size(dest_bdev) >> 9) - 1;
> +	if (dest & bs_mask) {
> +		return -EINVAL;
> +		goto out;
> +	}
> +
> +	if (q == dest_q && q->limits.copy_offload) {
> +		ret = blk_copy_offload(src_bdev, payload, dest, gfp_mask);
> +		if (ret)
> +			goto out;
> +	} else if (flags & BLKDEV_COPY_NOEMULATION) {

Why ? whoever calls blkdev_issue_copy() wants a copy to be done. Why would that
user say "Fail on me if the device does not support copy" ??? This is a weird
interface in my opinion.

> +		ret = -EIO;
> +		goto out;
> +	} else

Missing braces. By you do not need all these else after the gotos anyway.

> +		ret = blk_copy_emulate(src_bdev, payload, dest_bdev, dest,
> +				copy_size, gfp_mask);
> +
> +out:
> +	kvfree(payload);
> +	return ret;
> +}
> +EXPORT_SYMBOL(blkdev_issue_copy);
> +
>  /**
>   * __blkdev_issue_write_same - generate number of bios with same page
>   * @bdev:	target blockdev
> diff --git a/block/blk-merge.c b/block/blk-merge.c
> index 808768f6b174..4e04f24e13c1 100644
> --- a/block/blk-merge.c
> +++ b/block/blk-merge.c
> @@ -309,6 +309,8 @@ void __blk_queue_split(struct bio **bio, unsigned int *nr_segs)
>  	struct bio *split = NULL;
>  
>  	switch (bio_op(*bio)) {
> +	case REQ_OP_COPY:
> +			break;

Why would this even be called ? Copy BIOs cannot be split, right ?

>  	case REQ_OP_DISCARD:
>  	case REQ_OP_SECURE_ERASE:
>  		split = blk_bio_discard_split(q, *bio, &q->bio_split, nr_segs);
> diff --git a/block/blk-settings.c b/block/blk-settings.c
> index 43990b1d148b..93c15ba45a69 100644
> --- a/block/blk-settings.c
> +++ b/block/blk-settings.c
> @@ -60,6 +60,10 @@ void blk_set_default_limits(struct queue_limits *lim)
>  	lim->io_opt = 0;
>  	lim->misaligned = 0;
>  	lim->zoned = BLK_ZONED_NONE;
> +	lim->copy_offload = 0;
> +	lim->max_copy_sectors = 0;
> +	lim->max_copy_nr_ranges = 0;
> +	lim->max_copy_range_sectors = 0;
>  }
>  EXPORT_SYMBOL(blk_set_default_limits);
>  
> @@ -565,6 +569,12 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
>  	if (b->chunk_sectors)
>  		t->chunk_sectors = gcd(t->chunk_sectors, b->chunk_sectors);
>  
> +	/* simple copy not supported in stacked devices */
> +	t->copy_offload = 0;
> +	t->max_copy_sectors = 0;
> +	t->max_copy_range_sectors = 0;
> +	t->max_copy_nr_ranges = 0;

You do not need this. Limits not explicitely initialized are 0 already.
But I do not see why you can't support copy on stacked devices. That should be
feasible taking the min() for each of the above limit.

> +
>  	/* Physical block size a multiple of the logical block size? */
>  	if (t->physical_block_size & (t->logical_block_size - 1)) {
>  		t->physical_block_size = t->logical_block_size;
> diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
> index b513f1683af0..625a72541263 100644
> --- a/block/blk-sysfs.c
> +++ b/block/blk-sysfs.c
> @@ -166,6 +166,44 @@ static ssize_t queue_discard_granularity_show(struct request_queue *q, char *pag
>  	return queue_var_show(q->limits.discard_granularity, page);
>  }
>  
> +static ssize_t queue_copy_offload_show(struct request_queue *q, char *page)
> +{
> +	return queue_var_show(q->limits.copy_offload, page);
> +}
> +
> +static ssize_t queue_copy_offload_store(struct request_queue *q,
> +				       const char *page, size_t count)
> +{
> +	unsigned long copy_offload;
> +	ssize_t ret = queue_var_store(&copy_offload, page, count);
> +
> +	if (ret < 0)
> +		return ret;
> +
> +	if (copy_offload && q->limits.max_copy_sectors == 0)
> +		return -EINVAL;
> +
> +	q->limits.copy_offload = copy_offload;
> +	return ret;
> +}

This is weird. If you want to allow a user to disable copy offload, then use
max_copy_sectors. This one should be read-only and only indicate if the device
supports it or not. I also would actually change this one into
max_copy_hw_sectors, immutable, indicating the max copy sectors that the device
supports, and 0 for no support. That would allow an easy implementation of
max_copy_sectors being red/write for controlling enable/disable.

> +
> +static ssize_t queue_max_copy_sectors_show(struct request_queue *q, char *page)
> +{
> +	return queue_var_show(q->limits.max_copy_sectors, page);
> +}
> +
> +static ssize_t queue_max_copy_range_sectors_show(struct request_queue *q,
> +		char *page)
> +{
> +	return queue_var_show(q->limits.max_copy_range_sectors, page);
> +}
> +
> +static ssize_t queue_max_copy_nr_ranges_show(struct request_queue *q,
> +		char *page)
> +{
> +	return queue_var_show(q->limits.max_copy_nr_ranges, page);
> +}
> +
>  static ssize_t queue_discard_max_hw_show(struct request_queue *q, char *page)
>  {
>  
> @@ -591,6 +629,11 @@ QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones");
>  QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones");
>  QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones");
>  
> +QUEUE_RW_ENTRY(queue_copy_offload, "copy_offload");
> +QUEUE_RO_ENTRY(queue_max_copy_sectors, "max_copy_sectors");
> +QUEUE_RO_ENTRY(queue_max_copy_range_sectors, "max_copy_range_sectors");
> +QUEUE_RO_ENTRY(queue_max_copy_nr_ranges, "max_copy_nr_ranges");
> +
>  QUEUE_RW_ENTRY(queue_nomerges, "nomerges");
>  QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity");
>  QUEUE_RW_ENTRY(queue_poll, "io_poll");
> @@ -636,6 +679,10 @@ static struct attribute *queue_attrs[] = {
>  	&queue_discard_max_entry.attr,
>  	&queue_discard_max_hw_entry.attr,
>  	&queue_discard_zeroes_data_entry.attr,
> +	&queue_copy_offload_entry.attr,
> +	&queue_max_copy_sectors_entry.attr,
> +	&queue_max_copy_range_sectors_entry.attr,
> +	&queue_max_copy_nr_ranges_entry.attr,
>  	&queue_write_same_max_entry.attr,
>  	&queue_write_zeroes_max_entry.attr,
>  	&queue_zone_append_max_entry.attr,
> diff --git a/block/blk-zoned.c b/block/blk-zoned.c
> index 7a68b6e4300c..02069178d51e 100644
> --- a/block/blk-zoned.c
> +++ b/block/blk-zoned.c
> @@ -75,6 +75,7 @@ bool blk_req_needs_zone_write_lock(struct request *rq)
>  	case REQ_OP_WRITE_ZEROES:
>  	case REQ_OP_WRITE_SAME:
>  	case REQ_OP_WRITE:
> +	case REQ_OP_COPY:
>  		return blk_rq_zone_is_seq(rq);
>  	default:
>  		return false;
> diff --git a/block/bounce.c b/block/bounce.c
> index d3f51acd6e3b..5e052afe8691 100644
> --- a/block/bounce.c
> +++ b/block/bounce.c
> @@ -254,6 +254,7 @@ static struct bio *bounce_clone_bio(struct bio *bio_src, gfp_t gfp_mask,
>  	bio->bi_iter.bi_size	= bio_src->bi_iter.bi_size;
>  
>  	switch (bio_op(bio)) {
> +	case REQ_OP_COPY:
>  	case REQ_OP_DISCARD:
>  	case REQ_OP_SECURE_ERASE:
>  	case REQ_OP_WRITE_ZEROES:
> diff --git a/block/ioctl.c b/block/ioctl.c
> index d61d652078f4..0e52181657a4 100644
> --- a/block/ioctl.c
> +++ b/block/ioctl.c
> @@ -133,6 +133,37 @@ static int blk_ioctl_discard(struct block_device *bdev, fmode_t mode,
>  				    GFP_KERNEL, flags);
>  }
>  
> +static int blk_ioctl_copy(struct block_device *bdev, fmode_t mode,
> +		unsigned long arg, unsigned long flags)
> +{
> +	struct copy_range crange;
> +	struct range_entry *rlist;
> +	int ret;
> +
> +	if (!(mode & FMODE_WRITE))
> +		return -EBADF;
> +
> +	if (copy_from_user(&crange, (void __user *)arg, sizeof(crange)))
> +		return -EFAULT;
> +
> +	rlist = kmalloc_array(crange.nr_range, sizeof(*rlist),
> +			GFP_KERNEL);
> +	if (!rlist)
> +		return -ENOMEM;
> +
> +	if (copy_from_user(rlist, (void __user *)crange.range_list,
> +				sizeof(*rlist) * crange.nr_range)) {
> +		ret = -EFAULT;
> +		goto out;
> +	}
> +
> +	ret = blkdev_issue_copy(bdev, crange.nr_range, rlist, bdev, crange.dest,
> +			GFP_KERNEL, flags);
> +out:
> +	kfree(rlist);
> +	return ret;
> +}
> +
>  static int blk_ioctl_zeroout(struct block_device *bdev, fmode_t mode,
>  		unsigned long arg)
>  {
> @@ -458,6 +489,8 @@ static int blkdev_common_ioctl(struct block_device *bdev, fmode_t mode,
>  	case BLKSECDISCARD:
>  		return blk_ioctl_discard(bdev, mode, arg,
>  				BLKDEV_DISCARD_SECURE);
> +	case BLKCOPY:
> +		return blk_ioctl_copy(bdev, mode, arg, 0);
>  	case BLKZEROOUT:
>  		return blk_ioctl_zeroout(bdev, mode, arg);
>  	case BLKREPORTZONE:
> diff --git a/include/linux/bio.h b/include/linux/bio.h
> index 1edda614f7ce..164313bdfb35 100644
> --- a/include/linux/bio.h
> +++ b/include/linux/bio.h
> @@ -71,6 +71,7 @@ static inline bool bio_has_data(struct bio *bio)
>  static inline bool bio_no_advance_iter(const struct bio *bio)
>  {
>  	return bio_op(bio) == REQ_OP_DISCARD ||
> +	       bio_op(bio) == REQ_OP_COPY ||
>  	       bio_op(bio) == REQ_OP_SECURE_ERASE ||
>  	       bio_op(bio) == REQ_OP_WRITE_SAME ||
>  	       bio_op(bio) == REQ_OP_WRITE_ZEROES;
> diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
> index 866f74261b3b..5a35c02ac0a8 100644
> --- a/include/linux/blk_types.h
> +++ b/include/linux/blk_types.h
> @@ -380,6 +380,8 @@ enum req_opf {
>  	REQ_OP_ZONE_RESET	= 15,
>  	/* reset all the zone present on the device */
>  	REQ_OP_ZONE_RESET_ALL	= 17,
> +	/* copy ranges within device */
> +	REQ_OP_COPY		= 19,
>  
>  	/* SCSI passthrough using struct scsi_request */
>  	REQ_OP_SCSI_IN		= 32,
> @@ -506,6 +508,11 @@ static inline bool op_is_discard(unsigned int op)
>  	return (op & REQ_OP_MASK) == REQ_OP_DISCARD;
>  }
>  
> +static inline bool op_is_copy(unsigned int op)
> +{
> +	return (op & REQ_OP_MASK) == REQ_OP_COPY;
> +}
> +
>  /*
>   * Check if a bio or request operation is a zone management operation, with
>   * the exception of REQ_OP_ZONE_RESET_ALL which is treated as a special case
> @@ -565,4 +572,11 @@ struct blk_rq_stat {
>  	u64 batch;
>  };
>  
> +struct blk_copy_payload {
> +	sector_t	dest;
> +	int		copy_nr_ranges;
> +	struct block_device *src_bdev;
> +	struct	range_entry	range[];
> +};
> +
>  #endif /* __LINUX_BLK_TYPES_H */
> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
> index 699ace6b25ff..2bb4513d4bb8 100644
> --- a/include/linux/blkdev.h
> +++ b/include/linux/blkdev.h
> @@ -337,10 +337,14 @@ struct queue_limits {
>  	unsigned int		max_zone_append_sectors;
>  	unsigned int		discard_granularity;
>  	unsigned int		discard_alignment;
> +	unsigned int		copy_offload;
> +	unsigned int		max_copy_sectors;
>  
>  	unsigned short		max_segments;
>  	unsigned short		max_integrity_segments;
>  	unsigned short		max_discard_segments;
> +	unsigned short		max_copy_range_sectors;
> +	unsigned short		max_copy_nr_ranges;
>  
>  	unsigned char		misaligned;
>  	unsigned char		discard_misaligned;
> @@ -621,6 +625,7 @@ struct request_queue {
>  #define QUEUE_FLAG_RQ_ALLOC_TIME 27	/* record rq->alloc_time_ns */
>  #define QUEUE_FLAG_HCTX_ACTIVE	28	/* at least one blk-mq hctx is active */
>  #define QUEUE_FLAG_NOWAIT       29	/* device supports NOWAIT */
> +#define QUEUE_FLAG_SIMPLE_COPY	30	/* supports simple copy */
>  
>  #define QUEUE_FLAG_MQ_DEFAULT	((1 << QUEUE_FLAG_IO_STAT) |		\
>  				 (1 << QUEUE_FLAG_SAME_COMP) |		\
> @@ -643,6 +648,7 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q);
>  #define blk_queue_io_stat(q)	test_bit(QUEUE_FLAG_IO_STAT, &(q)->queue_flags)
>  #define blk_queue_add_random(q)	test_bit(QUEUE_FLAG_ADD_RANDOM, &(q)->queue_flags)
>  #define blk_queue_discard(q)	test_bit(QUEUE_FLAG_DISCARD, &(q)->queue_flags)
> +#define blk_queue_copy(q)	test_bit(QUEUE_FLAG_SIMPLE_COPY, &(q)->queue_flags)
>  #define blk_queue_zone_resetall(q)	\
>  	test_bit(QUEUE_FLAG_ZONE_RESETALL, &(q)->queue_flags)
>  #define blk_queue_secure_erase(q) \
> @@ -1069,6 +1075,9 @@ static inline unsigned int blk_queue_get_max_sectors(struct request_queue *q,
>  		return min(q->limits.max_discard_sectors,
>  			   UINT_MAX >> SECTOR_SHIFT);
>  
> +	if (unlikely(op == REQ_OP_COPY))
> +		return q->limits.max_copy_sectors;
> +

I would agreee with this if a copy BIO was always a single range, but that is
not the case. So I am not sure this makes sense at all.

>  	if (unlikely(op == REQ_OP_WRITE_SAME))
>  		return q->limits.max_write_same_sectors;
>  
> @@ -1343,6 +1352,12 @@ extern int __blkdev_issue_discard(struct block_device *bdev, sector_t sector,
>  		sector_t nr_sects, gfp_t gfp_mask, int flags,
>  		struct bio **biop);
>  
> +#define BLKDEV_COPY_NOEMULATION	(1 << 0)	/* do not emulate if copy offload not supported */
> +
> +extern int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs,
> +		struct range_entry *src_rlist, struct block_device *dest_bdev,
> +		sector_t dest, gfp_t gfp_mask, int flags);

No need for extern.

> +
>  #define BLKDEV_ZERO_NOUNMAP	(1 << 0)  /* do not free blocks */
>  #define BLKDEV_ZERO_NOFALLBACK	(1 << 1)  /* don't write explicit zeroes */
>  
> diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
> index f44eb0a04afd..5cadb176317a 100644
> --- a/include/uapi/linux/fs.h
> +++ b/include/uapi/linux/fs.h
> @@ -64,6 +64,18 @@ struct fstrim_range {
>  	__u64 minlen;
>  };
>  
> +struct range_entry {
> +	__u64 src;
> +	__u64 len;
> +};
> +
> +struct copy_range {
> +	__u64 dest;
> +	__u64 nr_range;
> +	__u64 range_list;
> +	__u64 rsvd;
> +};
> +
>  /* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */
>  #define FILE_DEDUPE_RANGE_SAME		0
>  #define FILE_DEDUPE_RANGE_DIFFERS	1
> @@ -184,6 +196,7 @@ struct fsxattr {
>  #define BLKSECDISCARD _IO(0x12,125)
>  #define BLKROTATIONAL _IO(0x12,126)
>  #define BLKZEROOUT _IO(0x12,127)
> +#define BLKCOPY _IOWR(0x12, 128, struct copy_range)
>  /*
>   * A jump here: 130-131 are reserved for zoned block devices
>   * (see uapi/linux/blkzoned.h)
> 

Please test your code more thoroughly. It is full of problems that you should
have detected with better testing including RO devices, partitions and error
path coverage.

-- 
Damien Le Moal
Western Digital Research


From Martin.George at netapp.com  Sat Feb 20 03:41:04 2021
From: Martin.George at netapp.com (George, Martin)
Date: Sat, 20 Feb 2021 08:41:04 +0000
Subject: [PATCH] nvme-fabrics: fix kato initialization
In-Reply-To: <03826110-0d9c-8b25-9975-b427036ec2cc@suse.de>
References: <20210218100443.5854-1-marting@netapp.com>
 <03826110-0d9c-8b25-9975-b427036ec2cc@suse.de>
Message-ID: <795b906b46f27880af74aaeb0c1fe116add5e0f5.camel@netapp.com>

On Fri, 2021-02-19 at 17:42 +0100, Hannes Reinecke wrote:
> On 2/18/21 11:04 AM, Martin George wrote:
> > Currently kato is initialized to NVME_DEFAULT_KATO for both
> > discovery & i/o controllers. This is a problem specifically
> > for non-persistent discovery controllers since it always ends
> > up with a non-zero kato value. Fix this by initializing kato
> > to zero instead, and ensuring various controllers are assigned
> > appropriate kato values as follows:
> > 
> > non-persistent controllers  - kato set to zero
> > persistent controllers      - kato set to NVMF_DEV_DISC_TMO
> >                                (or any positive int via nvme-cli)
> > i/o controllers             - kato set to NVME_DEFAULT_KATO
> >                                (or any positive int via nvme-cli)
> > 
> > Signed-off-by: Martin George <marting at netapp.com>
> > ---
> >   drivers/nvme/host/fabrics.c | 5 ++++-
> >   1 file changed, 4 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/nvme/host/fabrics.c
> > b/drivers/nvme/host/fabrics.c
> > index 5dfd806fc2d2..604ab0e5a2ad 100644
> > --- a/drivers/nvme/host/fabrics.c
> > +++ b/drivers/nvme/host/fabrics.c
> > @@ -630,7 +630,7 @@ static int nvmf_parse_options(struct
> > nvmf_ctrl_options *opts,
> >       opts->queue_size = NVMF_DEF_QUEUE_SIZE;
> >       opts->nr_io_queues = num_online_cpus();
> >       opts->reconnect_delay = NVMF_DEF_RECONNECT_DELAY;
> > -     opts->kato = NVME_DEFAULT_KATO;
> > +     opts->kato = 0;
> >       opts->duplicate_connect = false;
> >       opts->fast_io_fail_tmo = NVMF_DEF_FAIL_FAST_TMO;
> >       opts->hdr_digest = false;
> > @@ -893,6 +893,9 @@ static int nvmf_parse_options(struct
> > nvmf_ctrl_options *opts,
> >               opts->nr_write_queues = 0;
> >               opts->nr_poll_queues = 0;
> >               opts->duplicate_connect = true;
> > +     } else {
> > +             if (!opts->kato)
> > +                     opts->kato = NVME_DEFAULT_KATO;
> >       }
> >       if (ctrl_loss_tmo < 0) {
> >               opts->max_reconnects = -1;
> > 
> Can't you merge this with the previous patch 'ensure zero-kato ..'?
> 

Well, you are right that both the previous patch (i.e. 'ensure zero
kato for non-persistent controllers') & this current patch are jointly
required for assigning proper kato values to the various controllers,
so it would have made sense to merge the two. But the previous one is a
nvme-cli patch, whereas this current one is a kernel patch. That's why
I chose to create separate individual patches itself here.

So I hope this is good enough. If not, please let me know what's the
right way to proceed here.

-Martin

From yang.lee at linux.alibaba.com  Sat Feb 20 04:10:18 2021
From: yang.lee at linux.alibaba.com (Yang Li)
Date: Sat, 20 Feb 2021 17:10:18 +0800
Subject: [PATCH] nvme-core: Switch to using the new API kobj_to_dev()
Message-ID: <1613812218-17601-1-git-send-email-yang.lee@linux.alibaba.com>

fixed the following coccicheck:
./drivers/nvme/host/core.c:3440:60-61: WARNING opportunity for
kobj_to_dev()
./drivers/nvme/host/core.c:3679:60-61: WARNING opportunity for
kobj_to_dev()

Reported-by: Abaci Robot <abaci at linux.alibaba.com>
Signed-off-by: Yang Li <yang.lee at linux.alibaba.com>
---
 drivers/nvme/host/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f13eb4d..d56590f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3437,7 +3437,7 @@ static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
 static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		struct attribute *a, int n)
 {
-	struct device *dev = container_of(kobj, struct device, kobj);
+	struct device *dev = kobj_to_dev(kobj);
 	struct nvme_ns_ids *ids = &dev_to_ns_head(dev)->ids;
 
 	if (a == &dev_attr_uuid.attr) {
@@ -3676,7 +3676,7 @@ static DEVICE_ATTR(reconnect_delay, S_IRUGO | S_IWUSR,
 static umode_t nvme_dev_attrs_are_visible(struct kobject *kobj,
 		struct attribute *a, int n)
 {
-	struct device *dev = container_of(kobj, struct device, kobj);
+	struct device *dev = kobj_to_dev(kobj);
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
 
 	if (a == &dev_attr_delete_controller.attr && !ctrl->ops->delete_ctrl)
-- 
1.8.3.1



From hare at suse.de  Sat Feb 20 09:58:32 2021
From: hare at suse.de (Hannes Reinecke)
Date: Sat, 20 Feb 2021 15:58:32 +0100
Subject: [PATCH] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <20210219231643.GA4105@redsun51.ssa.fujisawa.hgst.com>
References: <20210219144625.56344-1-hare@suse.de>
 <20210219231643.GA4105@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <2d069bdb-d4dc-9d50-075f-1fa7eb6bc949@suse.de>

On 2/20/21 12:16 AM, Keith Busch wrote:
> On Fri, Feb 19, 2021 at 03:46:25PM +0100, Hannes Reinecke wrote:
>> In some setups like RAID we need to return an I/O error
>> once all paths are unavailable to allow the upper layers
>> to start their own error recovery (like redirecting I/O
>> to other mirrors).
>> This patch adds a sysfs attribute 'fail_if_no_path' to
>> allow the admin to enable that behaviour instead of the
>> current 'queue until a path becomes available' policy.
>>
>> Signed-off-by: Hannes Reinecke <hare at suse.de>
> 
> Sounds okay to me, just some minor nits below.
> 
>> @@ -283,10 +283,13 @@ static bool nvme_available_path(struct nvme_ns_head *head)
>>   			continue;
>>   		switch (ns->ctrl->state) {
>>   		case NVME_CTRL_LIVE:
>> +			return true;
>>   		case NVME_CTRL_RESETTING:
>>   		case NVME_CTRL_CONNECTING:
>>   			/* fallthru */
>> -			return true;
>> +			if (!test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH,
>> +				      &head->flags))
>> +				return true;
> 
> It doesn't look like the 'fallthru' is in the right place anymore,
> though I can't tell why it is there in the first place.
> 
Indeed, the 'fallthru' is misplaced; it needs to go after
NVME_CTRL_RESETTING.

>>   		default:
>>   			break;
>>   		}
>> @@ -641,6 +644,37 @@ static ssize_t ana_state_show(struct device *dev, struct device_attribute *attr,
>>   }
>>   DEVICE_ATTR_RO(ana_state);
>>   
>> +static ssize_t fail_if_no_path_show(struct device *dev,
>> +		struct device_attribute *attr, char *buf)
>> +{
>> +	struct gendisk *disk = dev_to_disk(dev);
>> +	struct nvme_ns_head *head = disk->private_data;
>> +
>> +	return sprintf(buf, "%d\n",
>> +		       test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags) ?
>> +		       1 : 0);
>> +}
>> +
>> +static ssize_t fail_if_no_path_store(struct device *dev,
>> +		struct device_attribute *attr, const char *buf, size_t count)
>> +{
>> +	struct gendisk *disk = dev_to_disk(dev);
>> +	struct nvme_ns_head *head = disk->private_data;
>> +	int fail_if_no_path, err;
>> +
>> +	err = kstrtoint(buf, 10, &fail_if_no_path);
>> +	if (err)
>> +		return -EINVAL;
>> +
>> +	else if (fail_if_no_path <= 0)
>> +		clear_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
>> +	else
>> +		set_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
> 
> The extra line above the 'else if' should be removed, or just change it
> to simply 'if'.
> 
Sure.

I'll be resending.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Sat Feb 20 09:59:43 2021
From: hare at suse.de (Hannes Reinecke)
Date: Sat, 20 Feb 2021 15:59:43 +0100
Subject: [PATCH] nvme-fabrics: fix kato initialization
In-Reply-To: <795b906b46f27880af74aaeb0c1fe116add5e0f5.camel@netapp.com>
References: <20210218100443.5854-1-marting@netapp.com>
 <03826110-0d9c-8b25-9975-b427036ec2cc@suse.de>
 <795b906b46f27880af74aaeb0c1fe116add5e0f5.camel@netapp.com>
Message-ID: <49574d36-3da0-0131-1869-30c8e04c93d1@suse.de>

On 2/20/21 9:41 AM, George, Martin wrote:
> On Fri, 2021-02-19 at 17:42 +0100, Hannes Reinecke wrote:
>> On 2/18/21 11:04 AM, Martin George wrote:
>>> Currently kato is initialized to NVME_DEFAULT_KATO for both
>>> discovery & i/o controllers. This is a problem specifically
>>> for non-persistent discovery controllers since it always ends
>>> up with a non-zero kato value. Fix this by initializing kato
>>> to zero instead, and ensuring various controllers are assigned
>>> appropriate kato values as follows:
>>>
>>> non-persistent controllers  - kato set to zero
>>> persistent controllers      - kato set to NVMF_DEV_DISC_TMO
>>>                                 (or any positive int via nvme-cli)
>>> i/o controllers             - kato set to NVME_DEFAULT_KATO
>>>                                 (or any positive int via nvme-cli)
>>>
>>> Signed-off-by: Martin George <marting at netapp.com>
>>> ---
>>>    drivers/nvme/host/fabrics.c | 5 ++++-
>>>    1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/nvme/host/fabrics.c
>>> b/drivers/nvme/host/fabrics.c
>>> index 5dfd806fc2d2..604ab0e5a2ad 100644
>>> --- a/drivers/nvme/host/fabrics.c
>>> +++ b/drivers/nvme/host/fabrics.c
>>> @@ -630,7 +630,7 @@ static int nvmf_parse_options(struct
>>> nvmf_ctrl_options *opts,
>>>        opts->queue_size = NVMF_DEF_QUEUE_SIZE;
>>>        opts->nr_io_queues = num_online_cpus();
>>>        opts->reconnect_delay = NVMF_DEF_RECONNECT_DELAY;
>>> -     opts->kato = NVME_DEFAULT_KATO;
>>> +     opts->kato = 0;
>>>        opts->duplicate_connect = false;
>>>        opts->fast_io_fail_tmo = NVMF_DEF_FAIL_FAST_TMO;
>>>        opts->hdr_digest = false;
>>> @@ -893,6 +893,9 @@ static int nvmf_parse_options(struct
>>> nvmf_ctrl_options *opts,
>>>                opts->nr_write_queues = 0;
>>>                opts->nr_poll_queues = 0;
>>>                opts->duplicate_connect = true;
>>> +     } else {
>>> +             if (!opts->kato)
>>> +                     opts->kato = NVME_DEFAULT_KATO;
>>>        }
>>>        if (ctrl_loss_tmo < 0) {
>>>                opts->max_reconnects = -1;
>>>
>> Can't you merge this with the previous patch 'ensure zero-kato ..'?
>>
> 
> Well, you are right that both the previous patch (i.e. 'ensure zero
> kato for non-persistent controllers') & this current patch are jointly
> required for assigning proper kato values to the various controllers,
> so it would have made sense to merge the two. But the previous one is a
> nvme-cli patch, whereas this current one is a kernel patch. That's why
> I chose to create separate individual patches itself here.
> 
> So I hope this is good enough. If not, please let me know what's the
> right way to proceed here.
> 
Argl. Of course you are correct.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From minwoo.im.dev at gmail.com  Sat Feb 20 10:36:10 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sun, 21 Feb 2021 00:36:10 +0900
Subject: [PATCH] nvme: add support namespace management to sysfs
Message-ID: <20210220153610.237288-1-minwoo.im.dev@gmail.com>

Namespaces are generally managed by scan_work.  If admin needs to attach
namespace to controller, admin command like `nvme-cli` should be issued
with rescanning controller.

If admin issues a Namespace Attachment command with detach option to a
namespace through IOCTL, kernel is not aware of it.  In this case, block
device for the namespace becomes meaningless, but still shown in the
kernel space as maintained.  After the rescanning the controller
manually, namespace node will be removed from the driver and user-space
will not be able to I/O to the namespace.

Add support for namespace management (attach/detach) to sysfs of
controller instance to make it easy to attach/detach namespaces with
kernel-aware rather than non-kernel-aware passthru via IOCTL.  Also, it
provides closer relationship between the device and kernel for the
namespace perspective (tightly coupled).

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 drivers/nvme/host/core.c | 189 +++++++++++++++++++++++++++++++++++++++
 include/linux/nvme.h     |  12 +++
 2 files changed, 201 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..71dad1b5ffdc 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -92,6 +92,9 @@ static struct class *nvme_subsys_class;
 static void nvme_put_subsystem(struct nvme_subsystem *subsys);
 static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
 					   unsigned nsid);
+static void nvme_ns_remove(struct nvme_ns *ns);
+static struct nvme_ns *nvme_find_get_ns_by_disk_name(struct nvme_ctrl *ctrl,
+						     const char *disk_name);
 
 /*
  * Prepare a queue for teardown.
@@ -3454,6 +3457,28 @@ static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
 }
 static DEVICE_ATTR_RO(nsid);
 
+#ifdef CONFIG_NVME_MULTIPATH
+static ssize_t path_show(struct device *dev, struct device_attribute *attr,
+		char *buf)
+{
+	struct nvme_ns_head *head = dev_to_ns_head(dev);
+	struct nvme_ns *ns;
+	int node = numa_node_id();
+	int srcu_idx;
+
+	srcu_idx = srcu_read_lock(&head->srcu);
+	ns = srcu_dereference(head->current_path[node], &head->srcu);
+	if (!ns) {
+		srcu_read_unlock(&head->srcu, srcu_idx);
+		return sprintf(buf, "none\n");
+	}
+
+	srcu_read_unlock(&head->srcu, srcu_idx);
+	return sprintf(buf, "%s\n", ns->disk->disk_name);
+}
+static DEVICE_ATTR_RO(path);
+#endif
+
 static struct attribute *nvme_ns_id_attrs[] = {
 	&dev_attr_wwid.attr,
 	&dev_attr_uuid.attr,
@@ -3463,6 +3488,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
+	&dev_attr_path.attr,
 #endif
 	NULL,
 };
@@ -3493,6 +3519,11 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
 			return 0;
 	}
+
+	if (a == &dev_attr_path.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
+			return 0;
+	}
 #endif
 	return a->mode;
 }
@@ -3684,6 +3715,142 @@ static ssize_t nvme_ctrl_reconnect_delay_store(struct device *dev,
 static DEVICE_ATTR(reconnect_delay, S_IRUGO | S_IWUSR,
 	nvme_ctrl_reconnect_delay_show, nvme_ctrl_reconnect_delay_store);
 
+static int __nvme_ns_detach(struct nvme_ctrl *ctrl, unsigned int nsid)
+{
+	struct nvme_command c = { };
+	int err;
+	u16 *buf;
+
+	c.ns_attach.opcode = nvme_admin_ns_attach;
+	c.ns_attach.nsid = cpu_to_le32(nsid);
+	c.ns_attach.sel = cpu_to_le32(0x1);
+
+	buf = kmalloc(4096, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	buf[0] = cpu_to_le32(0x1);
+	buf[1] = cpu_to_le32(ctrl->cntlid);
+
+	err = nvme_submit_sync_cmd(ctrl->admin_q, &c, buf, 4096);
+	if (err) {
+		kfree(buf);
+		return err;
+	}
+
+	kfree(buf);
+	return 0;
+}
+
+static int nvme_ns_detach(struct nvme_ctrl *ctrl, struct nvme_ns *ns)
+{
+	int err;
+
+	blk_mq_quiesce_queue(ns->queue);
+
+	err = __nvme_ns_detach(ctrl, ns->head->ns_id);
+	if (err) {
+		blk_mq_unquiesce_queue(ns->queue);
+		return err;
+	}
+
+	nvme_set_queue_dying(ns);
+	nvme_ns_remove(ns);
+
+	return 0;
+}
+
+static int __nvme_ns_attach(struct nvme_ctrl *ctrl, unsigned int nsid)
+{
+	struct nvme_command c = { };
+	int err;
+	u16 *buf;
+
+	c.ns_attach.opcode = nvme_admin_ns_attach;
+	c.ns_attach.nsid = cpu_to_le32(nsid);
+	c.ns_attach.sel = cpu_to_le32(0x0);
+
+	buf = kmalloc(4096, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	buf[0] = cpu_to_le32(0x1);
+	buf[1] = cpu_to_le32(ctrl->cntlid);
+
+	err = nvme_submit_sync_cmd(ctrl->admin_q, &c, buf, 4096);
+	if (err) {
+		kfree(buf);
+		return err;
+	}
+
+	kfree(buf);
+	return 0;
+}
+
+static int nvme_ns_attach(struct nvme_ctrl *ctrl, unsigned int nsid, bool scan)
+{
+	int err;
+
+	if (!(ctrl->oacs & NVME_CTRL_OACS_NS_MANAGEMENT))
+		return -EOPNOTSUPP;
+
+	err = __nvme_ns_attach(ctrl, nsid);
+	if (err)
+		return err;
+
+	if (scan)
+		nvme_queue_scan(ctrl);
+
+	return 0;
+}
+
+static ssize_t detach_ns_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvme_ns *ns;
+	int err;
+
+	if (!(ctrl->oacs & NVME_CTRL_OACS_NS_MANAGEMENT))
+		return -EOPNOTSUPP;
+
+	ns = nvme_find_get_ns_by_disk_name(ctrl, buf);
+	if (!ns)
+		return -EINVAL;
+
+	err = nvme_ns_detach(ctrl, ns);
+	if (err) {
+		nvme_put_ns(ns);
+		return err;
+	}
+
+	nvme_put_ns(ns);
+	return count;
+}
+static DEVICE_ATTR_WO(detach_ns);
+
+static ssize_t attach_ns_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	unsigned int nsid;
+	int err;
+
+	/*
+	 * 'nsid' is device namespace id which is reported by NVMe controller.
+	 */
+	err = kstrtou32(buf, 10, &nsid);
+	if (err)
+		return err;
+
+	err = nvme_ns_attach(ctrl, nsid, true);
+	if (err)
+		return err;
+
+	return count;
+}
+static DEVICE_ATTR_WO(attach_ns);
+
 static struct attribute *nvme_dev_attrs[] = {
 	&dev_attr_reset_controller.attr,
 	&dev_attr_rescan_controller.attr,
@@ -3703,6 +3870,8 @@ static struct attribute *nvme_dev_attrs[] = {
 	&dev_attr_hostid.attr,
 	&dev_attr_ctrl_loss_tmo.attr,
 	&dev_attr_reconnect_delay.attr,
+	&dev_attr_detach_ns.attr,
+	&dev_attr_attach_ns.attr,
 	NULL
 };
 
@@ -3902,6 +4071,25 @@ struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 }
 EXPORT_SYMBOL_NS_GPL(nvme_find_get_ns, NVME_TARGET_PASSTHRU);
 
+static struct nvme_ns *nvme_find_get_ns_by_disk_name(struct nvme_ctrl *ctrl,
+		const char *disk_name)
+{
+	struct nvme_ns *ns, *ret = NULL;
+
+	down_read(&ctrl->namespaces_rwsem);
+	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (!strcmp(ns->disk->disk_name, disk_name)) {
+			if (!kref_get_unless_zero(&ns->kref))
+				continue;
+			ret = ns;
+			break;
+		}
+	}
+	up_read(&ctrl->namespaces_rwsem);
+
+	return ret;
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 		struct nvme_ns_ids *ids)
 {
@@ -4751,6 +4939,7 @@ static inline void _nvme_check_size(void)
 	BUILD_BUG_ON(sizeof(struct nvme_smart_log) != 512);
 	BUILD_BUG_ON(sizeof(struct nvme_dbbuf) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_directive_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct nvme_ns_attach) != 64);
 }
 
 
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index b08787cd0881..bc6c2a162bbb 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -322,6 +322,7 @@ enum {
 	NVME_CTRL_ONCS_TIMESTAMP		= 1 << 6,
 	NVME_CTRL_VWC_PRESENT			= 1 << 0,
 	NVME_CTRL_OACS_SEC_SUPP                 = 1 << 0,
+	NVME_CTRL_OACS_NS_MANAGEMENT		= 1 << 3,
 	NVME_CTRL_OACS_DIRECTIVES		= 1 << 5,
 	NVME_CTRL_OACS_DBBUF_SUPP		= 1 << 8,
 	NVME_CTRL_LPA_CMD_EFFECTS_LOG		= 1 << 1,
@@ -1398,6 +1399,16 @@ struct streams_directive_params {
 	__u8	rsvd2[6];
 };
 
+struct nvme_ns_attach {
+	__u8			opcode;
+	__u8			flags;
+	__u16			command_id;
+	__le32			nsid;
+	__u32			rsvd2[8];
+	__le32			sel;
+	__u32			rsvd11[5];
+};
+
 struct nvme_command {
 	union {
 		struct nvme_common_command common;
@@ -1421,6 +1432,7 @@ struct nvme_command {
 		struct nvmf_property_get_command prop_get;
 		struct nvme_dbbuf dbbuf;
 		struct nvme_directive_cmd directive;
+		struct nvme_ns_attach ns_attach;
 	};
 };
 
-- 
2.25.1



From minwoo.im.dev at gmail.com  Sat Feb 20 10:39:40 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sun, 21 Feb 2021 00:39:40 +0900
Subject: [PATCH] nvme: add support namespace management to sysfs
In-Reply-To: <20210220153610.237288-1-minwoo.im.dev@gmail.com>
References: <20210220153610.237288-1-minwoo.im.dev@gmail.com>
Message-ID: <20210220153940.GA237584@localhost.localdomain>

Sorry, please ignore this patch. (un-related changes are added)


From minwoo.im.dev at gmail.com  Sat Feb 20 10:40:04 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sun, 21 Feb 2021 00:40:04 +0900
Subject: [PATCH] nvme: add support namespace management to sysfs
Message-ID: <20210220154004.242622-1-minwoo.im.dev@gmail.com>

Namespaces are generally managed by scan_work.  If admin needs to attach
namespace to controller, admin command like `nvme-cli` should be issued
with rescanning controller.

If admin issues a Namespace Attachment command with detach option to a
namespace through IOCTL, kernel is not aware of it.  In this case, block
device for the namespace becomes meaningless, but still shown in the
kernel space as maintained.  After the rescanning the controller
manually, namespace node will be removed from the driver and user-space
will not be able to I/O to the namespace.

Add support for namespace management (attach/detach) to sysfs of
controller instance to make it easy to attach/detach namespaces with
kernel-aware rather than non-kernel-aware passthru via IOCTL.  Also, it
provides closer relationship between the device and kernel for the
namespace perspective (tightly coupled).

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 drivers/nvme/host/core.c | 161 +++++++++++++++++++++++++++++++++++++++
 include/linux/nvme.h     |  12 +++
 2 files changed, 173 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..62a7e0c6e021 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -92,6 +92,9 @@ static struct class *nvme_subsys_class;
 static void nvme_put_subsystem(struct nvme_subsystem *subsys);
 static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
 					   unsigned nsid);
+static void nvme_ns_remove(struct nvme_ns *ns);
+static struct nvme_ns *nvme_find_get_ns_by_disk_name(struct nvme_ctrl *ctrl,
+						     const char *disk_name);
 
 /*
  * Prepare a queue for teardown.
@@ -3684,6 +3687,142 @@ static ssize_t nvme_ctrl_reconnect_delay_store(struct device *dev,
 static DEVICE_ATTR(reconnect_delay, S_IRUGO | S_IWUSR,
 	nvme_ctrl_reconnect_delay_show, nvme_ctrl_reconnect_delay_store);
 
+static int __nvme_ns_detach(struct nvme_ctrl *ctrl, unsigned int nsid)
+{
+	struct nvme_command c = { };
+	int err;
+	u16 *buf;
+
+	c.ns_attach.opcode = nvme_admin_ns_attach;
+	c.ns_attach.nsid = cpu_to_le32(nsid);
+	c.ns_attach.sel = cpu_to_le32(0x1);
+
+	buf = kmalloc(4096, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	buf[0] = cpu_to_le32(0x1);
+	buf[1] = cpu_to_le32(ctrl->cntlid);
+
+	err = nvme_submit_sync_cmd(ctrl->admin_q, &c, buf, 4096);
+	if (err) {
+		kfree(buf);
+		return err;
+	}
+
+	kfree(buf);
+	return 0;
+}
+
+static int nvme_ns_detach(struct nvme_ctrl *ctrl, struct nvme_ns *ns)
+{
+	int err;
+
+	blk_mq_quiesce_queue(ns->queue);
+
+	err = __nvme_ns_detach(ctrl, ns->head->ns_id);
+	if (err) {
+		blk_mq_unquiesce_queue(ns->queue);
+		return err;
+	}
+
+	nvme_set_queue_dying(ns);
+	nvme_ns_remove(ns);
+
+	return 0;
+}
+
+static int __nvme_ns_attach(struct nvme_ctrl *ctrl, unsigned int nsid)
+{
+	struct nvme_command c = { };
+	int err;
+	u16 *buf;
+
+	c.ns_attach.opcode = nvme_admin_ns_attach;
+	c.ns_attach.nsid = cpu_to_le32(nsid);
+	c.ns_attach.sel = cpu_to_le32(0x0);
+
+	buf = kmalloc(4096, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	buf[0] = cpu_to_le32(0x1);
+	buf[1] = cpu_to_le32(ctrl->cntlid);
+
+	err = nvme_submit_sync_cmd(ctrl->admin_q, &c, buf, 4096);
+	if (err) {
+		kfree(buf);
+		return err;
+	}
+
+	kfree(buf);
+	return 0;
+}
+
+static int nvme_ns_attach(struct nvme_ctrl *ctrl, unsigned int nsid, bool scan)
+{
+	int err;
+
+	if (!(ctrl->oacs & NVME_CTRL_OACS_NS_MANAGEMENT))
+		return -EOPNOTSUPP;
+
+	err = __nvme_ns_attach(ctrl, nsid);
+	if (err)
+		return err;
+
+	if (scan)
+		nvme_queue_scan(ctrl);
+
+	return 0;
+}
+
+static ssize_t detach_ns_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvme_ns *ns;
+	int err;
+
+	if (!(ctrl->oacs & NVME_CTRL_OACS_NS_MANAGEMENT))
+		return -EOPNOTSUPP;
+
+	ns = nvme_find_get_ns_by_disk_name(ctrl, buf);
+	if (!ns)
+		return -EINVAL;
+
+	err = nvme_ns_detach(ctrl, ns);
+	if (err) {
+		nvme_put_ns(ns);
+		return err;
+	}
+
+	nvme_put_ns(ns);
+	return count;
+}
+static DEVICE_ATTR_WO(detach_ns);
+
+static ssize_t attach_ns_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	unsigned int nsid;
+	int err;
+
+	/*
+	 * 'nsid' is device namespace id which is reported by NVMe controller.
+	 */
+	err = kstrtou32(buf, 10, &nsid);
+	if (err)
+		return err;
+
+	err = nvme_ns_attach(ctrl, nsid, true);
+	if (err)
+		return err;
+
+	return count;
+}
+static DEVICE_ATTR_WO(attach_ns);
+
 static struct attribute *nvme_dev_attrs[] = {
 	&dev_attr_reset_controller.attr,
 	&dev_attr_rescan_controller.attr,
@@ -3703,6 +3842,8 @@ static struct attribute *nvme_dev_attrs[] = {
 	&dev_attr_hostid.attr,
 	&dev_attr_ctrl_loss_tmo.attr,
 	&dev_attr_reconnect_delay.attr,
+	&dev_attr_detach_ns.attr,
+	&dev_attr_attach_ns.attr,
 	NULL
 };
 
@@ -3902,6 +4043,25 @@ struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 }
 EXPORT_SYMBOL_NS_GPL(nvme_find_get_ns, NVME_TARGET_PASSTHRU);
 
+static struct nvme_ns *nvme_find_get_ns_by_disk_name(struct nvme_ctrl *ctrl,
+		const char *disk_name)
+{
+	struct nvme_ns *ns, *ret = NULL;
+
+	down_read(&ctrl->namespaces_rwsem);
+	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (!strcmp(ns->disk->disk_name, disk_name)) {
+			if (!kref_get_unless_zero(&ns->kref))
+				continue;
+			ret = ns;
+			break;
+		}
+	}
+	up_read(&ctrl->namespaces_rwsem);
+
+	return ret;
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 		struct nvme_ns_ids *ids)
 {
@@ -4751,6 +4911,7 @@ static inline void _nvme_check_size(void)
 	BUILD_BUG_ON(sizeof(struct nvme_smart_log) != 512);
 	BUILD_BUG_ON(sizeof(struct nvme_dbbuf) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_directive_cmd) != 64);
+	BUILD_BUG_ON(sizeof(struct nvme_ns_attach) != 64);
 }
 
 
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index b08787cd0881..bc6c2a162bbb 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -322,6 +322,7 @@ enum {
 	NVME_CTRL_ONCS_TIMESTAMP		= 1 << 6,
 	NVME_CTRL_VWC_PRESENT			= 1 << 0,
 	NVME_CTRL_OACS_SEC_SUPP                 = 1 << 0,
+	NVME_CTRL_OACS_NS_MANAGEMENT		= 1 << 3,
 	NVME_CTRL_OACS_DIRECTIVES		= 1 << 5,
 	NVME_CTRL_OACS_DBBUF_SUPP		= 1 << 8,
 	NVME_CTRL_LPA_CMD_EFFECTS_LOG		= 1 << 1,
@@ -1398,6 +1399,16 @@ struct streams_directive_params {
 	__u8	rsvd2[6];
 };
 
+struct nvme_ns_attach {
+	__u8			opcode;
+	__u8			flags;
+	__u16			command_id;
+	__le32			nsid;
+	__u32			rsvd2[8];
+	__le32			sel;
+	__u32			rsvd11[5];
+};
+
 struct nvme_command {
 	union {
 		struct nvme_common_command common;
@@ -1421,6 +1432,7 @@ struct nvme_command {
 		struct nvmf_property_get_command prop_get;
 		struct nvme_dbbuf dbbuf;
 		struct nvme_directive_cmd directive;
+		struct nvme_ns_attach ns_attach;
 	};
 };
 
-- 
2.25.1



From kbusch at kernel.org  Sat Feb 20 10:50:27 2021
From: kbusch at kernel.org (Keith Busch)
Date: Sun, 21 Feb 2021 00:50:27 +0900
Subject: [PATCH] nvme: add support namespace management to sysfs
In-Reply-To: <20210220154004.242622-1-minwoo.im.dev@gmail.com>
References: <20210220154004.242622-1-minwoo.im.dev@gmail.com>
Message-ID: <20210220155027.GB4105@redsun51.ssa.fujisawa.hgst.com>

On Sun, Feb 21, 2021 at 12:40:04AM +0900, Minwoo Im wrote:
> Namespaces are generally managed by scan_work.  If admin needs to attach
> namespace to controller, admin command like `nvme-cli` should be issued
> with rescanning controller.
> 
> If admin issues a Namespace Attachment command with detach option to a
> namespace through IOCTL, kernel is not aware of it.  In this case, block
> device for the namespace becomes meaningless, but still shown in the
> kernel space as maintained.  After the rescanning the controller
> manually, namespace node will be removed from the driver and user-space
> will not be able to I/O to the namespace.
> 
> Add support for namespace management (attach/detach) to sysfs of
> controller instance to make it easy to attach/detach namespaces with
> kernel-aware rather than non-kernel-aware passthru via IOCTL.  Also, it
> provides closer relationship between the device and kernel for the
> namespace perspective (tightly coupled).

Nak. The kernel should be aware of the altered namespace inventory via
Command Effects Log or Namespace Notify AER. If the controller doesn't
support either of these, then the user gets to do a manual rescan, which
is not difficult task either.


From minwoo.im.dev at gmail.com  Sat Feb 20 11:07:46 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Sun, 21 Feb 2021 01:07:46 +0900
Subject: [PATCH] nvme: add support namespace management to sysfs
In-Reply-To: <20210220155027.GB4105@redsun51.ssa.fujisawa.hgst.com>
References: <20210220154004.242622-1-minwoo.im.dev@gmail.com>
 <20210220155027.GB4105@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <20210220160746.GB237584@localhost.localdomain>

On 21-02-21 00:50:27, Keith Busch wrote:
> On Sun, Feb 21, 2021 at 12:40:04AM +0900, Minwoo Im wrote:
> > Namespaces are generally managed by scan_work.  If admin needs to attach
> > namespace to controller, admin command like `nvme-cli` should be issued
> > with rescanning controller.
> > 
> > If admin issues a Namespace Attachment command with detach option to a
> > namespace through IOCTL, kernel is not aware of it.  In this case, block
> > device for the namespace becomes meaningless, but still shown in the
> > kernel space as maintained.  After the rescanning the controller
> > manually, namespace node will be removed from the driver and user-space
> > will not be able to I/O to the namespace.
> > 
> > Add support for namespace management (attach/detach) to sysfs of
> > controller instance to make it easy to attach/detach namespaces with
> > kernel-aware rather than non-kernel-aware passthru via IOCTL.  Also, it
> > provides closer relationship between the device and kernel for the
> > namespace perspective (tightly coupled).
> 
> Nak. The kernel should be aware of the altered namespace inventory via
> Command Effects Log or Namespace Notify AER. If the controller doesn't
> support either of these, then the user gets to do a manual rescan, which
> is not difficult task either.

Thanks Keith for your review.  I think I missed NVME_CMD_EFFECTS_NIC
part in passthru code.  Sorry for the noises.


From David.Laight at ACULAB.COM  Sat Feb 20 13:01:56 2021
From: David.Laight at ACULAB.COM (David Laight)
Date: Sat, 20 Feb 2021 18:01:56 +0000
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com>
Message-ID: <146c47907c2446d4a896830de400dd81@AcuMS.aculab.com>

From: SelvaKumar S
> Sent: 19 February 2021 12:45
> 
> This patchset tries to add support for TP4065a ("Simple Copy Command"),
> v2020.05.04 ("Ratified")
> 
> The Specification can be found in following link.
> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
> 
> Simple copy command is a copy offloading operation and is  used to copy
> multiple contiguous ranges (source_ranges) of LBA's to a single destination
> LBA within the device reducing traffic between host and device.

Sounds to me like the real reason is that the copy just ends up changing
some indirect block pointers rather than having to actually copy the data.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

From Chaitanya.Kulkarni at wdc.com  Sat Feb 20 13:46:33 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sat, 20 Feb 2021 18:46:33 +0000
Subject: [PATCH] nvme-core: Switch to using the new API kobj_to_dev()
References: <1613812218-17601-1-git-send-email-yang.lee@linux.alibaba.com>
Message-ID: <BYAPR04MB49653FCDC174CF7BCB4840BD86839@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/20/21 01:15, Yang Li wrote:
> fixed the following coccicheck:
> ./drivers/nvme/host/core.c:3440:60-61: WARNING opportunity for
> kobj_to_dev()
> ./drivers/nvme/host/core.c:3679:60-61: WARNING opportunity for
> kobj_to_dev()
>
> Reported-by: Abaci Robot <abaci at linux.alibaba.com>
> Signed-off-by: Yang Li <yang.lee at linux.alibaba.com>
Looks good.

Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>



From willy at infradead.org  Sat Feb 20 14:08:37 2021
From: willy at infradead.org (Matthew Wilcox)
Date: Sat, 20 Feb 2021 19:08:37 +0000
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <146c47907c2446d4a896830de400dd81@AcuMS.aculab.com>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <146c47907c2446d4a896830de400dd81@AcuMS.aculab.com>
Message-ID: <20210220190837.GE2858050@casper.infradead.org>

On Sat, Feb 20, 2021 at 06:01:56PM +0000, David Laight wrote:
> From: SelvaKumar S
> > Sent: 19 February 2021 12:45
> > 
> > This patchset tries to add support for TP4065a ("Simple Copy Command"),
> > v2020.05.04 ("Ratified")
> > 
> > The Specification can be found in following link.
> > https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
> > 
> > Simple copy command is a copy offloading operation and is  used to copy
> > multiple contiguous ranges (source_ranges) of LBA's to a single destination
> > LBA within the device reducing traffic between host and device.
> 
> Sounds to me like the real reason is that the copy just ends up changing
> some indirect block pointers rather than having to actually copy the data.

That would be incorrect, at least for firmware that I have knowledge of.
There are checksums which involve the logical block address of the data,
and you can't just rewrite the checksum on NAND, you have to write the
entire block.

Now, firmware doesn't have to implement their checksum like this,
but there are good reasons to do it this way (eg if the command gets
corrupted in transfer and you read the wrong block, it will fail the
checksum, preventing the drive from returning Somebody Else's Data).

So let's take these people at their word.  It is to reduce traffic
between drive and host.  And that is a good enough reason to do it.


From kbusch at kernel.org  Sat Feb 20 14:19:21 2021
From: kbusch at kernel.org (Keith Busch)
Date: Sun, 21 Feb 2021 04:19:21 +0900
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <146c47907c2446d4a896830de400dd81@AcuMS.aculab.com>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <146c47907c2446d4a896830de400dd81@AcuMS.aculab.com>
Message-ID: <20210220191921.GA7968@redsun51.ssa.fujisawa.hgst.com>

On Sat, Feb 20, 2021 at 06:01:56PM +0000, David Laight wrote:
> From: SelvaKumar S
> > Sent: 19 February 2021 12:45
> > 
> > This patchset tries to add support for TP4065a ("Simple Copy Command"),
> > v2020.05.04 ("Ratified")
> > 
> > The Specification can be found in following link.
> > https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
> > 
> > Simple copy command is a copy offloading operation and is  used to copy
> > multiple contiguous ranges (source_ranges) of LBA's to a single destination
> > LBA within the device reducing traffic between host and device.
> 
> Sounds to me like the real reason is that the copy just ends up changing
> some indirect block pointers rather than having to actually copy the data.

I guess an implementation could do that, but I think that's missing the
point of the command. The intention is to copy the data to a new
location on the media for host managed garbage collection. 


From zboszor at pr.hu  Sun Feb 21 00:12:16 2021
From: zboszor at pr.hu (=?UTF-8?q?Zolt=C3=A1n=20B=C3=B6sz=C3=B6rm=C3=A9nyi?=)
Date: Sun, 21 Feb 2021 06:12:16 +0100
Subject: [PATCH] nvme: Apply the same fix Kingston SKC2000 nVME SSD as A2000
Message-ID: <20210221051216.3398620-1-zboszor@pr.hu>

From: Zolt?n B?sz?rm?nyi <zboszor at gmail.com>

My 2TB SKC2000 showed the exact same symptoms that were provided
in 538e4a8c57 ("nvme-pci: avoid the deepest sleep state on
Kingston A2000 SSDs"), i.e. a complete NVME lockup that needed
cold boot to get it back.

According to some sources, the A2000 is simply a rebadged
SKC2000 with a slightly optimized firmware.

Adding the SKC2000 PCI ID to the quirk list with the same workaround
as the A2000 made my laptop survive a 5 hours long Yocto bootstrap
buildfest which reliably triggered the SSD lockup previously.

Tested against 5.10.17.

Signed-off-by: Zolt?n B?sz?rm?nyi <zboszor at gmail.com>

---
 drivers/nvme/host/pci.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 4a33287..3af6a95 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3264,6 +3264,8 @@ static const struct pci_device_id nvme_id_table[] = {
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(0x15b7, 0x2001),   /*  Sandisk Skyhawk */
 		.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+	{ PCI_DEVICE(0x2646, 0x2262),   /* KINGSTON SKC2000 NVMe SSD  */
+		.driver_data = NVME_QUIRK_NO_DEEPEST_PS, },
 	{ PCI_DEVICE(0x2646, 0x2263),   /* KINGSTON A2000 NVMe SSD  */
 		.driver_data = NVME_QUIRK_NO_DEEPEST_PS, },
 	{ PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2001),
-- 
2.29.2



From gerlitz.or at gmail.com  Sun Feb 21 03:52:04 2021
From: gerlitz.or at gmail.com (Or Gerlitz)
Date: Sun, 21 Feb 2021 10:52:04 +0200
Subject: [PATCH v4 net-next 00/21] nvme-tcp receive offloads
In-Reply-To: <20210211211044.32701-1-borisp@mellanox.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
Message-ID: <CAJ3xEMgazUM053U-nGycNPPT7bennXVcdFoPvMWRU3uZGdpXFg@mail.gmail.com>

On Thu, Feb 11, 2021 at 11:15 PM Boris Pismenny <borisp at mellanox.com> wrote:
> Changes since v3:
> =========================================
> * Use DDP_TCP ifdefs in iov_iter and skb iterators to minimize impact
> when compiled out (Christoph)
> * Simplify netdev references and reduce the use of
> get_netdev_for_sock (Sagi)
> * Avoid "static" in it's own line, move it one line down (Christoph)
> * Pass (queue, skb, *offset) and retrieve the pdu_seq in
> nvme_tcp_resync_response (Sagi)
> * Add missing assignment of offloading_netdev to null in offload_limits
> error case (Sagi)
> * Set req->offloaded = false once -- the lifetime rules are:
> set to false on cmd_setup / set to true when ddp setup succeeds (Sagi)
> * Replace pr_info_ratelimited with dev_info_ratelimited (Sagi)
> * Add nvme_tcp_complete_request and invoke it from two similar call
> sites (Sagi)
> * Introduce nvme_tcp_req_map_sg earlier in the series (Sagi)
> * Add nvme_tcp_consume_skb and put into it a hunk from
> nvme_tcp_recv_data to handle copy with and without offload

Sagi, Christoph,

Any further comments?

Or.





> Changes since v2:
> =========================================
> * Use skb->ddp_crc for copy offload to avoid skb_condense
> * Default mellanox driver support to no (experimental feature)
> * In iov_iter use non-ddp functions for kvec and iovec
> * Remove typecasting in nvme-tcp
>
> Changes since v1:
> =========================================
> * Rework iov_iter copy skip if src==dst to be less intrusive (David Ahern)
> * Add tcp-ddp documentation (David Ahern)
> * Refactor mellanox driver patches into more patches (Saeed Mahameed)
> * Avoid pointer casting (David Ahern)
> * Rename nvme-tcp offload flags (Shai Malin)
> * Update cover-letter according to the above
>
> Changes since RFC v1:
> =========================================
> * Split mlx5 driver patches to several commits
> * Fix nvme-tcp handling of recovery flows. In particular, move queue offlaod
>   init/teardown to the start/stop functions.


From borispismenny at gmail.com  Sun Feb 21 06:29:39 2021
From: borispismenny at gmail.com (Boris Pismenny)
Date: Sun, 21 Feb 2021 13:29:39 +0200
Subject: [PATCH v4 net-next 06/21] nvme-tcp: Add DDP offload control path
In-Reply-To: <CAJ3xEMjMqK81uNv21poD+DoZCRxYak0DYFZrjDbmWaSxw4R5ig@mail.gmail.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-7-borisp@mellanox.com>
 <2dd10b2f-df00-e21c-7886-93f41a987040@gmail.com>
 <CAJ3xEMjMqK81uNv21poD+DoZCRxYak0DYFZrjDbmWaSxw4R5ig@mail.gmail.com>
Message-ID: <3f447ef7-3ded-8c1c-fdfa-ec137e11a19a@gmail.com>

On 17/02/2021 15:55, Or Gerlitz wrote:
> On Sun, Feb 14, 2021 at 8:20 PM David Ahern <dsahern at gmail.com> wrote:
>> On 2/11/21 2:10 PM, Boris Pismenny wrote:
>>> @@ -223,6 +229,164 @@ static inline size_t nvme_tcp_pdu_last_send(struct nvme_tcp_request *req,
>>>       return nvme_tcp_pdu_data_left(req) <= len;
>>>  }
>>>
>>> +#ifdef CONFIG_TCP_DDP
>>> +
>>> +static bool nvme_tcp_resync_request(struct sock *sk, u32 seq, u32 flags);
>>> +static const struct tcp_ddp_ulp_ops nvme_tcp_ddp_ulp_ops = {
>>> +     .resync_request         = nvme_tcp_resync_request,
>>> +};
>>> +
>>> +static int nvme_tcp_offload_socket(struct nvme_tcp_queue *queue)
>>> +{
>>> +     struct net_device *netdev = queue->ctrl->offloading_netdev;
>>> +     struct nvme_tcp_ddp_config config = {};
>>> +     int ret;
>>> +
>>> +     if (!(netdev->features & NETIF_F_HW_TCP_DDP))
>>
>> If nvme_tcp_offload_limits does not find a dst_entry on the socket then
>> offloading_netdev may not NULL at this point.
> 
> correct :( will look on that
> 

That's only partially true.
If nvme_tcp_offload_limits finds a dst_entry, but then the netdevice
goes down, then the check here will catch it. This is needed because
nvme_tcp_offload_limits doesn't hold a reference! We opted not to grab a
reference on nvme_tcp_offload_limits because it doesn't create a context.


>>> +             queue->ctrl->offloading_netdev = NULL;
>>> +             return -ENODEV;
>>> +     }
>>> +
>>> +     if (netdev->features & NETIF_F_HW_TCP_DDP &&
>>> +         netdev->tcp_ddp_ops &&
>>> +         netdev->tcp_ddp_ops->tcp_ddp_limits)
>>> +             ret = netdev->tcp_ddp_ops->tcp_ddp_limits(netdev, &limits);
>>> +     else
>>> +             ret = -EOPNOTSUPP;
>>> +
>>> +     if (!ret) {
>>> +             queue->ctrl->offloading_netdev = netdev;
>>
>> you save a reference to the netdev here, but then release the refcnt
>> below. That device could be deleted between this point in time and the
>> initialization of all queues.
> 

That's true, and this is why we repeat the checks there.

We avoid holding the reference here because there is no obvious
complementary release point for nvme_tcp_offload_limits and there is no
hardware context created here, so there is no real need to hold it at
this stage.


From borispismenny at gmail.com  Sun Feb 21 06:44:11 2021
From: borispismenny at gmail.com (Boris Pismenny)
Date: Sun, 21 Feb 2021 13:44:11 +0200
Subject: [PATCH v4 net-next 07/21] nvme-tcp: Add DDP data-path
In-Reply-To: <f9c45756-b823-7cab-213c-761866230c96@gmail.com>
References: <20210211211044.32701-1-borisp@mellanox.com>
 <20210211211044.32701-8-borisp@mellanox.com>
 <cfd61c5a-c5fd-e0d9-fb60-be93f1c98735@gmail.com>
 <CAJ3xEMgZg9dFyc8cnbuPPAFT3jd2k27TdLOz-vtVmxy9k9zHcg@mail.gmail.com>
 <f9c45756-b823-7cab-213c-761866230c96@gmail.com>
Message-ID: <c0eba1e7-f113-2400-72c1-4361e79363a7@gmail.com>

On 17/02/2021 19:00, David Ahern wrote:
> On 2/17/21 7:01 AM, Or Gerlitz wrote:
>>>> @@ -1136,6 +1265,10 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req)
>>>>       else
>>>>               flags |= MSG_EOR;
>>>>
>>>> +     if (test_bit(NVME_TCP_Q_OFF_DDP, &queue->flags) &&
>>>> +         blk_rq_nr_phys_segments(rq) && rq_data_dir(rq) == READ)
>>>> +             nvme_tcp_setup_ddp(queue, pdu->cmd.common.command_id, rq);
>>>> +
>>>
>>> For consistency, shouldn't this be wrapped in the CONFIG_TCP_DDP check too?
>>
>> We tried to avoid the wrapping in some places where it was
>> possible to do without adding confusion, this one is a good
>> example IMOH.
>>
>>
> 
> The above (and other locations like it) can easily be put into a helper
> that has logic when the CONFIG is enabled and compiles out when not.
> Consistency makes for simpler, cleaner code for optional features.
> 

The above is consistent in the sense that we wrap only places that are
absolutely necessary, so as to avoid ifdefs as much as possible.

Specifically, here and in ddp_teardown, we will add a wrapper to reduce
clutter if offload is not used.


From Elad.Grupi at dell.com  Sun Feb 21 14:37:27 2021
From: Elad.Grupi at dell.com (Grupi, Elad)
Date: Sun, 21 Feb 2021 19:37:27 +0000
Subject: [PATCH] nvmet-tcp: Fix receive data digest calculation for
 multiple h2cdata PDUs
In-Reply-To: <20210203230001.18797-1-sagi@grimberg.me>
References: <20210203230001.18797-1-sagi@grimberg.me>
Message-ID: <DM6PR19MB40112D4524285D2D496F2186EF829@DM6PR19MB4011.namprd19.prod.outlook.com>

Hi Sagi,

Can you please explain why you avoid calling nvmet_tcp_unmap_pdu_iovec on data_digest PDU?
I don't recall any other case that unmap is being called on good path.

Regards,
Elad

-----Original Message-----
From: Linux-nvme <linux-nvme-bounces at lists.infradead.org> On Behalf Of Sagi Grimberg
Sent: Thursday, 4 February 2021 1:00
To: linux-nvme at lists.infradead.org; Christoph Hellwig; Keith Busch; Chaitanya Kulkarni
Subject: [PATCH] nvmet-tcp: Fix receive data digest calculation for multiple h2cdata PDUs


[EXTERNAL EMAIL] 

When a host sends multiple h2cdata PDUs for a single command, we should verify the data digest calculation per PDU and not per command.

Fixes: 872d26a391da ("nvmet-tcp: add NVMe over TCP target driver")
Reported-by: Narayan Ayalasomayajula <Narayan.Ayalasomayajula at wdc.com>
Tested-by: Narayan Ayalasomayajula <Narayan.Ayalasomayajula at wdc.com>
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 drivers/nvme/target/tcp.c | 31 ++++++++++++++++++++++++-------
 1 file changed, 24 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index aacf06f0b431..577ce7d403ae 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -379,7 +379,7 @@ static int nvmet_tcp_map_data(struct nvmet_tcp_cmd *cmd)
 	return NVME_SC_INTERNAL;
 }
 
-static void nvmet_tcp_ddgst(struct ahash_request *hash,
+static void nvmet_tcp_send_ddgst(struct ahash_request *hash,
 		struct nvmet_tcp_cmd *cmd)
 {
 	ahash_request_set_crypt(hash, cmd->req.sg, @@ -387,6 +387,23 @@ static void nvmet_tcp_ddgst(struct ahash_request *hash,
 	crypto_ahash_digest(hash);
 }
 
+static void nvmet_tcp_recv_ddgst(struct ahash_request *hash,
+		struct nvmet_tcp_cmd *cmd)
+{
+	struct scatterlist sg;
+	struct kvec *iov;
+	int i;
+
+	crypto_ahash_init(hash);
+	for (i = 0, iov = cmd->iov; i < cmd->nr_mapped; i++, iov++) {
+		sg_init_one(&sg, iov->iov_base, iov->iov_len);
+		ahash_request_set_crypt(hash, &sg, NULL, iov->iov_len);
+		crypto_ahash_update(hash);
+	}
+	ahash_request_set_crypt(hash, NULL, (void *)&cmd->exp_ddgst, 0);
+	crypto_ahash_final(hash);
+}
+
 static void nvmet_setup_c2h_data_pdu(struct nvmet_tcp_cmd *cmd)  {
 	struct nvme_tcp_data_pdu *pdu = cmd->data_pdu; @@ -411,7 +428,7 @@ static void nvmet_setup_c2h_data_pdu(struct nvmet_tcp_cmd *cmd)
 
 	if (queue->data_digest) {
 		pdu->hdr.flags |= NVME_TCP_F_DDGST;
-		nvmet_tcp_ddgst(queue->snd_hash, cmd);
+		nvmet_tcp_send_ddgst(queue->snd_hash, cmd);
 	}
 
 	if (cmd->queue->hdr_digest) {
@@ -1060,7 +1077,7 @@ static void nvmet_tcp_prep_recv_ddgst(struct nvmet_tcp_cmd *cmd)  {
 	struct nvmet_tcp_queue *queue = cmd->queue;
 
-	nvmet_tcp_ddgst(queue->rcv_hash, cmd);
+	nvmet_tcp_recv_ddgst(queue->rcv_hash, cmd);
 	queue->offset = 0;
 	queue->left = NVME_TCP_DIGEST_LENGTH;
 	queue->rcv_state = NVMET_TCP_RECV_DDGST; @@ -1081,14 +1098,14 @@ static int nvmet_tcp_try_recv_data(struct nvmet_tcp_queue *queue)
 		cmd->rbytes_done += ret;
 	}
 
+	if (queue->data_digest) {
+		nvmet_tcp_prep_recv_ddgst(cmd);
+		return 0;
+	}
 	nvmet_tcp_unmap_pdu_iovec(cmd);
 
 	if (!(cmd->flags & NVMET_TCP_F_INIT_FAILED) &&
 	    cmd->rbytes_done == cmd->req.transfer_len) {
-		if (queue->data_digest) {
-			nvmet_tcp_prep_recv_ddgst(cmd);
-			return 0;
-		}
 		cmd->req.execute(&cmd->req);
 	}
 
--
2.27.0


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

From david at fromorbit.com  Sun Feb 21 18:52:48 2021
From: david at fromorbit.com (Dave Chinner)
Date: Mon, 22 Feb 2021 10:52:48 +1100
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com>
Message-ID: <20210221235248.GZ4626@dread.disaster.area>

On Fri, Feb 19, 2021 at 06:15:13PM +0530, SelvaKumar S wrote:
> This patchset tries to add support for TP4065a ("Simple Copy Command"),
> v2020.05.04 ("Ratified")
> 
> The Specification can be found in following link.
> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
> 
> Simple copy command is a copy offloading operation and is  used to copy
> multiple contiguous ranges (source_ranges) of LBA's to a single destination
> LBA within the device reducing traffic between host and device.
> 
> This implementation doesn't add native copy offload support for stacked
> devices rather copy offload is done through emulation. Possible use
> cases are F2FS gc and BTRFS relocation/balance.

It sounds like you are missing the most obvious use case for this:
hooking up filesystem copy_file_range() implementations to allow
userspace to offload user data copies to hardware....

Another fs level feature that could use this for hardware
acceleration fallocate(FALLOC_FL_UNSHARE).

These are probably going to be far easier to hook up than filesystem
GC algorithms, and there is also solid data integrity and stress
testing checking infrastructure for these operations via fstests.

> As SCSI XCOPY can take two different block devices and no of source range is
> equal to 1, this interface can be extended in future to support SCSI XCOPY.

That greatly complicates the implementation. do we even care at this
point about cross-device XCOPY at this point?

Cheers,

Dave.
-- 
Dave Chinner
david at fromorbit.com


From ming.lei at redhat.com  Sun Feb 21 20:31:33 2021
From: ming.lei at redhat.com (Ming Lei)
Date: Mon, 22 Feb 2021 09:31:33 +0800
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com>
Message-ID: <YDMJdekWhy/Y1Y1r@T590>

On Fri, Feb 19, 2021 at 06:15:13PM +0530, SelvaKumar S wrote:
> This patchset tries to add support for TP4065a ("Simple Copy Command"),
> v2020.05.04 ("Ratified")
> 
> The Specification can be found in following link.
> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
> 
> Simple copy command is a copy offloading operation and is  used to copy
> multiple contiguous ranges (source_ranges) of LBA's to a single destination
> LBA within the device reducing traffic between host and device.
> 
> This implementation doesn't add native copy offload support for stacked
> devices rather copy offload is done through emulation. Possible use
> cases are F2FS gc and BTRFS relocation/balance.
> 
> *blkdev_issue_copy* takes source bdev, no of sources, array of source
> ranges (in sectors), destination bdev and destination offset(in sectors).
> If both source and destination block devices are same and copy_offload = 1,
> then copy is done through native copy offloading. Copy emulation is used
> in other cases.
> 
> As SCSI XCOPY can take two different block devices and no of source range is
> equal to 1, this interface can be extended in future to support SCSI XCOPY.

The patchset adds ioctl(BLKCOPY) and two userspace visible data
struture(range_entry, and copy_range), all belong to kabi stuff, and the
interface is generic block layer kabi.

The API has to be allowed to extend for supporting SCSI XCOPY in future or similar
block copy commands without breaking previous application, so please CC linux-scsi
and scsi guys in your next post.


-- 
Ming



From hare at suse.de  Mon Feb 22 01:41:55 2021
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 22 Feb 2021 07:41:55 +0100
Subject: [PATCHv2] nvme: add 'fail_if_no_path' sysfs attribute
Message-ID: <20210222064155.105742-1-hare@suse.de>

In some setups like RAID we need to return an I/O error
once all paths are unavailable to allow the upper layers
to start their own error recovery (like redirecting I/O
to other mirrors).
This patch adds a sysfs attribute 'fail_if_no_path' to
allow the admin to enable that behaviour instead of the
current 'queue until a path becomes available' policy.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      |  5 +++++
 drivers/nvme/host/multipath.c | 38 +++++++++++++++++++++++++++++++++--
 drivers/nvme/host/nvme.h      |  2 ++
 3 files changed, 43 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 5c1ec3bfb9d3..ef20029a46b2 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3466,6 +3466,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
+	&dev_attr_fail_if_no_path.attr,
 #endif
 	NULL,
 };
@@ -3496,6 +3497,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
 			return 0;
 	}
+	if (a == &dev_attr_fail_if_no_path.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
+			return 0;
+	}
 #endif
 	return a->mode;
 }
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 0696319adaf6..35ed2f6d431b 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -283,10 +283,13 @@ static bool nvme_available_path(struct nvme_ns_head *head)
 			continue;
 		switch (ns->ctrl->state) {
 		case NVME_CTRL_LIVE:
+			return true;
 		case NVME_CTRL_RESETTING:
-		case NVME_CTRL_CONNECTING:
 			/* fallthru */
-			return true;
+		case NVME_CTRL_CONNECTING:
+			if (!test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH,
+				      &head->flags))
+				return true;
 		default:
 			break;
 		}
@@ -641,6 +644,37 @@ static ssize_t ana_state_show(struct device *dev, struct device_attribute *attr,
 }
 DEVICE_ATTR_RO(ana_state);
 
+static ssize_t fail_if_no_path_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+
+	return sprintf(buf, "%d\n",
+		       test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags) ?
+		       1 : 0);
+}
+
+static ssize_t fail_if_no_path_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+	int fail_if_no_path, err;
+
+	err = kstrtoint(buf, 10, &fail_if_no_path);
+	if (err)
+		return -EINVAL;
+
+	if (fail_if_no_path <= 0)
+		clear_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
+	else
+		set_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
+	return count;
+}
+DEVICE_ATTR(fail_if_no_path, S_IRUGO | S_IWUSR,
+	fail_if_no_path_show, fail_if_no_path_store);
+
 static int nvme_lookup_ana_group_desc(struct nvme_ctrl *ctrl,
 		struct nvme_ana_group_desc *desc, void *data)
 {
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 02aa4ba665c0..e8de2b9dcdc7 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -419,6 +419,7 @@ struct nvme_ns_head {
 	struct mutex		lock;
 	unsigned long		flags;
 #define NVME_NSHEAD_DISK_LIVE	0
+#define NVME_NSHEAD_FAIL_IF_NO_PATH	1
 	struct nvme_ns __rcu	*current_path[];
 #endif
 };
@@ -695,6 +696,7 @@ static inline void nvme_trace_bio_complete(struct request *req)
 
 extern struct device_attribute dev_attr_ana_grpid;
 extern struct device_attribute dev_attr_ana_state;
+extern struct device_attribute dev_attr_fail_if_no_path;
 extern struct device_attribute subsys_attr_iopolicy;
 
 #else
-- 
2.26.2



From l at damenly.su  Mon Feb 22 01:52:44 2021
From: l at damenly.su (Su Yue)
Date: Mon, 22 Feb 2021 14:52:44 +0800
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <20210219124517.79359-1-selvakuma.s1@samsung.com>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com>
Message-ID: <lfbgr3ur.fsf@damenly.su>


On Fri 19 Feb 2021 at 20:45, SelvaKumar S 
<selvakuma.s1 at samsung.com> wrote:

> This patchset tries to add support for TP4065a ("Simple Copy 
> Command"),
> v2020.05.04 ("Ratified")
>
> The Specification can be found in following link.
> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
>

404 not found.
Should it be
https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs.zip
?

> Simple copy command is a copy offloading operation and is  used 
> to copy
> multiple contiguous ranges (source_ranges) of LBA's to a single 
> destination
> LBA within the device reducing traffic between host and device.
>
> This implementation doesn't add native copy offload support for 
> stacked
> devices rather copy offload is done through emulation. Possible 
> use
> cases are F2FS gc and BTRFS relocation/balance.
>
> *blkdev_issue_copy* takes source bdev, no of sources, array of 
> source
> ranges (in sectors), destination bdev and destination offset(in 
> sectors).
> If both source and destination block devices are same and 
> copy_offload = 1,
> then copy is done through native copy offloading. Copy emulation 
> is used
> in other cases.
>
> As SCSI XCOPY can take two different block devices and no of 
> source range is
> equal to 1, this interface can be extended in future to support 
> SCSI XCOPY.
>
> For devices supporting native simple copy, attach the control 
> information
> as payload to the bio and submit to the device. For devices 
> without native
> copy support, copy emulation is done by reading each source 
> range into memory
> and writing it to the destination. Caller can choose not to try
> emulation if copy offload is not supported by setting
> BLKDEV_COPY_NOEMULATION flag.
>
> Following limits are added to queue limits and are exposed in 
> sysfs
> to userspace
> 	- *copy_offload* controls copy_offload. set 0 to disable copy
> 		offload, 1 to enable native copy offloading support.
> 	- *max_copy_sectors* limits the sum of all source_range length
> 	- *max_copy_nr_ranges* limits the number of source ranges
> 	- *max_copy_range_sectors* limit the maximum number of sectors
> 		that can constitute a single source range.
>
> 	max_copy_sectors = 0 indicates the device doesn't support copy
> offloading.
>
> 	*copy offload* sysfs entry is configurable and can be used 
> toggle
> between emulation and native support depending upon the usecase.
>
> Changes from v4
>
> 1. Extend dm-kcopyd to leverage copy-offload, while copying 
> within the
> same device. The other approach was to have copy-emulation by 
> moving
> dm-kcopyd to block layer. But it also required moving core dm-io 
> infra,
> causing a massive churn across multiple dm-targets.
>
> 2. Remove export in bio_map_kern()
> 3. Change copy_offload sysfs to accept 0 or else
> 4. Rename copy support flag to QUEUE_FLAG_SIMPLE_COPY
> 5. Rename payload entries, add source bdev field to be used 
> while
> partition remapping, remove copy_size
> 6. Change the blkdev_issue_copy() interface to accept 
> destination and
> source values in sector rather in bytes
> 7. Add payload to bio using bio_map_kern() for copy_offload case
> 8. Add check to return error if one of the source range length 
> is 0
> 9. Add BLKDEV_COPY_NOEMULATION flag to allow user to not try 
> copy
> emulation incase of copy offload is not supported. Caller can 
> his use
> his existing copying logic to complete the io.
> 10. Bug fix copy checks and reduce size of rcu_lock()
>
> Planned for next:
> - adding blktests
> - handling larger (than device limits) copy
> - decide on ioctl interface (man-page etc.)
>
> Changes from v3
>
> 1. gfp_flag fixes.
> 2. Export bio_map_kern() and use it to allocate and add pages to 
> bio.
> 3. Move copy offload, reading to buf, writing from buf to 
> separate functions.
> 4. Send read bio of copy offload by chaining them and submit 
> asynchronously.
> 5. Add gendisk->part0 and part->bd_start_sect changes to 
> blk_check_copy().
> 6. Move single source range limit check to blk_check_copy()
> 7. Rename __blkdev_issue_copy() to blkdev_issue_copy and remove 
> old helper.
> 8. Change blkdev_issue_copy() interface generic to accepts 
> destination bdev
> 	to support XCOPY as well.
> 9. Add invalidate_kernel_vmap_range() after reading data for 
> vmalloc'ed memory.
> 10. Fix buf allocoation logic to allocate buffer for the total 
> size of copy.
> 11. Reword patch commit description.
>
> Changes from v2
>
> 1. Add emulation support for devices not supporting copy.
> 2. Add *copy_offload* sysfs entry to enable and disable 
> copy_offload
> 	in devices supporting simple copy.
> 3. Remove simple copy support for stacked devices.
>
> Changes from v1:
>
> 1. Fix memory leak in __blkdev_issue_copy
> 2. Unmark blk_check_copy inline
> 3. Fix line break in blk_check_copy_eod
> 4. Remove p checks and made code more readable
> 5. Don't use bio_set_op_attrs and remove op and set
>    bi_opf directly
> 6. Use struct_size to calculate total_size
> 7. Fix partition remap of copy destination
> 8. Remove mcl,mssrl,msrc from nvme_ns
> 9. Initialize copy queue limits to 0 in nvme_config_copy
> 10. Remove return in QUEUE_FLAG_COPY check
> 11. Remove unused OCFS
>
> SelvaKumar S (4):
>   block: make bio_map_kern() non static
>   block: add simple copy support
>   nvme: add simple copy support
>   dm kcopyd: add simple copy offload support
>
>  block/blk-core.c          | 102 +++++++++++++++--
>  block/blk-lib.c           | 223 
>  ++++++++++++++++++++++++++++++++++++++
>  block/blk-map.c           |   2 +-
>  block/blk-merge.c         |   2 +
>  block/blk-settings.c      |  10 ++
>  block/blk-sysfs.c         |  47 ++++++++
>  block/blk-zoned.c         |   1 +
>  block/bounce.c            |   1 +
>  block/ioctl.c             |  33 ++++++
>  drivers/md/dm-kcopyd.c    |  49 ++++++++-
>  drivers/nvme/host/core.c  |  87 +++++++++++++++
>  include/linux/bio.h       |   1 +
>  include/linux/blk_types.h |  14 +++
>  include/linux/blkdev.h    |  17 +++
>  include/linux/nvme.h      |  43 +++++++-
>  include/uapi/linux/fs.h   |  13 +++
>  16 files changed, 627 insertions(+), 18 deletions(-)



From minwoo.im.dev at gmail.com  Mon Feb 22 05:15:19 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Mon, 22 Feb 2021 19:15:19 +0900
Subject: [PATCHv2] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <20210222064155.105742-1-hare@suse.de>
References: <20210222064155.105742-1-hare@suse.de>
Message-ID: <20210222101519.GA2856@localhost.localdomain>

On 21-02-22 07:41:55, Hannes Reinecke wrote:
> In some setups like RAID we need to return an I/O error
> once all paths are unavailable to allow the upper layers
> to start their own error recovery (like redirecting I/O
> to other mirrors).
> This patch adds a sysfs attribute 'fail_if_no_path' to
> allow the admin to enable that behaviour instead of the
> current 'queue until a path becomes available' policy.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c      |  5 +++++
>  drivers/nvme/host/multipath.c | 38 +++++++++++++++++++++++++++++++++--
>  drivers/nvme/host/nvme.h      |  2 ++
>  3 files changed, 43 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 5c1ec3bfb9d3..ef20029a46b2 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -3466,6 +3466,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
>  #ifdef CONFIG_NVME_MULTIPATH
>  	&dev_attr_ana_grpid.attr,
>  	&dev_attr_ana_state.attr,
> +	&dev_attr_fail_if_no_path.attr,
>  #endif
>  	NULL,
>  };
> @@ -3496,6 +3497,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
>  		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
>  			return 0;
>  	}
> +	if (a == &dev_attr_fail_if_no_path.attr) {
> +		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
> +			return 0;
> +	}
>  #endif
>  	return a->mode;
>  }
> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> index 0696319adaf6..35ed2f6d431b 100644
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -283,10 +283,13 @@ static bool nvme_available_path(struct nvme_ns_head *head)
>  			continue;
>  		switch (ns->ctrl->state) {
>  		case NVME_CTRL_LIVE:
> +			return true;

Hello Hannes,

May I ask there's any reason why LIVE state is not covered by the sysfs
attribute that this patch introduced ?  Wouldn't it be possible to have
no available paths (inaccessible) in LIVE state ?

Thanks! :)


From hare at suse.de  Mon Feb 22 07:37:14 2021
From: hare at suse.de (Hannes Reinecke)
Date: Mon, 22 Feb 2021 13:37:14 +0100
Subject: [PATCHv2] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <20210222101519.GA2856@localhost.localdomain>
References: <20210222064155.105742-1-hare@suse.de>
 <20210222101519.GA2856@localhost.localdomain>
Message-ID: <6bf38292-c852-32a3-2ed8-4445321aefd7@suse.de>

On 2/22/21 11:15 AM, Minwoo Im wrote:
> On 21-02-22 07:41:55, Hannes Reinecke wrote:
>> In some setups like RAID we need to return an I/O error
>> once all paths are unavailable to allow the upper layers
>> to start their own error recovery (like redirecting I/O
>> to other mirrors).
>> This patch adds a sysfs attribute 'fail_if_no_path' to
>> allow the admin to enable that behaviour instead of the
>> current 'queue until a path becomes available' policy.
>>
>> Signed-off-by: Hannes Reinecke <hare at suse.de>
>> ---
>>   drivers/nvme/host/core.c      |  5 +++++
>>   drivers/nvme/host/multipath.c | 38 +++++++++++++++++++++++++++++++++--
>>   drivers/nvme/host/nvme.h      |  2 ++
>>   3 files changed, 43 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index 5c1ec3bfb9d3..ef20029a46b2 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -3466,6 +3466,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
>>   #ifdef CONFIG_NVME_MULTIPATH
>>   	&dev_attr_ana_grpid.attr,
>>   	&dev_attr_ana_state.attr,
>> +	&dev_attr_fail_if_no_path.attr,
>>   #endif
>>   	NULL,
>>   };
>> @@ -3496,6 +3497,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
>>   		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
>>   			return 0;
>>   	}
>> +	if (a == &dev_attr_fail_if_no_path.attr) {
>> +		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
>> +			return 0;
>> +	}
>>   #endif
>>   	return a->mode;
>>   }
>> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
>> index 0696319adaf6..35ed2f6d431b 100644
>> --- a/drivers/nvme/host/multipath.c
>> +++ b/drivers/nvme/host/multipath.c
>> @@ -283,10 +283,13 @@ static bool nvme_available_path(struct nvme_ns_head *head)
>>   			continue;
>>   		switch (ns->ctrl->state) {
>>   		case NVME_CTRL_LIVE:
>> +			return true;
> 
> Hello Hannes,
> 
> May I ask there's any reason why LIVE state is not covered by the sysfs
> attribute that this patch introduced ?  Wouldn't it be possible to have
> no available paths (inaccessible) in LIVE state ?
> 
> Thanks! :)
> 
Ah, you mean ANA inaccessible?
Hmm. Technically, yes, this might be the case.
I'll be updating the patch.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From joshiiitr at gmail.com  Mon Feb 22 08:42:18 2021
From: joshiiitr at gmail.com (Kanchan Joshi)
Date: Mon, 22 Feb 2021 19:12:18 +0530
Subject: [RFC PATCH 0/4] Asynchronous passthrough ioctl
In-Reply-To: <dd0392a1-acfa-ef4d-5531-5f1dddc9efe7@kernel.dk>
References: <CGME20210127150134epcas5p251fc1de3ff3581dd4c68b3fbe0b9dd91@epcas5p2.samsung.com>
 <20210127150029.13766-1-joshi.k@samsung.com>
 <489691ce-3b1e-30ce-9f72-d32389e33901@gmail.com>
 <a287bd9e-3474-83a4-e5c2-98df17214dc7@gmail.com>
 <CA+1E3rJHHFyjwv7Kp32E9H-cf5ksh0pOHSVdGoTpktQrB8SE6A@mail.gmail.com>
 <2d37d0ca-5853-4bb6-1582-551b9044040c@kernel.dk>
 <CA+1E3rKeqaLXBuvpMcjZ37XH9RqJHjPnTFObJj0T-u8K9Otw-w@mail.gmail.com>
 <dd0392a1-acfa-ef4d-5531-5f1dddc9efe7@kernel.dk>
Message-ID: <CA+1E3rJqtpfDbKVGBjSYhX=WfzZ2bE8b0U3drUNz_=bp0u9Vuw@mail.gmail.com>

On Thu, Jan 28, 2021 at 10:54 PM Jens Axboe <axboe at kernel.dk> wrote:
>
> On 1/28/21 10:13 AM, Kanchan Joshi wrote:
> > On Thu, Jan 28, 2021 at 8:08 PM Jens Axboe <axboe at kernel.dk> wrote:
> >>
> >> On 1/28/21 5:04 AM, Kanchan Joshi wrote:
> >>> On Wed, Jan 27, 2021 at 9:32 PM Pavel Begunkov <asml.silence at gmail.com> wrote:
> >>>>
> >>>> On 27/01/2021 15:42, Pavel Begunkov wrote:
> >>>>> On 27/01/2021 15:00, Kanchan Joshi wrote:
> >>>>>> This RFC patchset adds asynchronous ioctl capability for NVMe devices.
> >>>>>> Purpose of RFC is to get the feedback and optimize the path.
> >>>>>>
> >>>>>> At the uppermost io-uring layer, a new opcode IORING_OP_IOCTL_PT is
> >>>>>> presented to user-space applications. Like regular-ioctl, it takes
> >>>>>> ioctl opcode and an optional argument (ioctl-specific input/output
> >>>>>> parameter). Unlike regular-ioctl, it is made to skip the block-layer
> >>>>>> and reach directly to the underlying driver (nvme in the case of this
> >>>>>> patchset). This path between io-uring and nvme is via a newly
> >>>>>> introduced block-device operation "async_ioctl". This operation
> >>>>>> expects io-uring to supply a callback function which can be used to
> >>>>>> report completion at later stage.
> >>>>>>
> >>>>>> For a regular ioctl, NVMe driver submits the command to the device and
> >>>>>> the submitter (task) is made to wait until completion arrives. For
> >>>>>> async-ioctl, completion is decoupled from submission. Submitter goes
> >>>>>> back to its business without waiting for nvme-completion. When
> >>>>>> nvme-completion arrives, it informs io-uring via the registered
> >>>>>> completion-handler. But some ioctls may require updating certain
> >>>>>> ioctl-specific fields which can be accessed only in context of the
> >>>>>> submitter task. For that reason, NVMe driver uses task-work infra for
> >>>>>> that ioctl-specific update. Since task-work is not exported, it cannot
> >>>>>> be referenced when nvme is compiled as a module. Therefore, one of the
> >>>>>> patch exports task-work API.
> >>>>>>
> >>>>>> Here goes example of usage (pseudo-code).
> >>>>>> Actual nvme-cli source, modified to issue all ioctls via this opcode
> >>>>>> is present at-
> >>>>>> https://github.com/joshkan/nvme-cli/commit/a008a733f24ab5593e7874cfbc69ee04e88068c5
> >>>>>
> >>>>> see https://git.kernel.dk/cgit/linux-block/log/?h=io_uring-fops
> >>>>>
> >>>>> Looks like good time to bring that branch/discussion back
> >>>>
> >>>> a bit more context:
> >>>> https://github.com/axboe/liburing/issues/270
> >>>
> >>> Thanks, it looked good. It seems key differences (compared to
> >>> uring-patch that I posted) are -
> >>> 1. using file-operation instead of block-dev operation.
> >>
> >> Right, it's meant to span wider than just block devices.
> >>
> >>> 2. repurpose the sqe memory for ioctl-cmd. If an application does
> >>> ioctl with <=40 bytes of cmd, it does not have to allocate ioctl-cmd.
> >>> That's nifty. We still need to support passing larger-cmd (e.g.
> >>> nvme-passthru ioctl takes 72 bytes) but that shouldn't get too
> >>> difficult I suppose.
> >>
> >> It's actually 48 bytes in the as-posted version, and I've bumped it to
> >> 56 bytes in the latest branch. So not quite enough for everything,
> >> nothing ever will be, but should work for a lot of cases without
> >> requiring per-command allocations just for the actual command.
> >
> > Agreed. But if I got it right, you are open to support both in-the-sqe
> > command (<= 56 bytes) and out-of-sqe command (> 56 bytes) with this
> > interface.
> > Driver processing the ioctl can fetch the cmd from user-space in one
> > case (as it does now), and skips in another.
>
> Your out-of-seq command would be none of io_urings business, outside of
> the fact that we'd need to ensure it's stable if we need to postpone
> it. So yes, that would be fine, it just means your actual command is
> passed in as a pointer, and you would be responsible for copying it
> in for execution
>
> We're going to need something to handle postponing, and something
> for ensuring that eg cancelations free the allocated memory.

I have few doubts about allocation/postponing. Are you referring to
uring allocating memory for this case, similar to the way
"req->async_data" is managed for few other opcodes?
Or can it (i.e. larger cmd) remain a user-space pointer, and the
underlying driver fetches the command in.
If submission context changes (for sqo/io-wq case), uring seemed to
apply context-grabbing techniques to make that work.

> >>> And for some ioctls, driver may still need to use task-work to update
> >>> the user-space pointers (embedded in uring/ioctl cmd) during
> >>> completion.
> >>>
> >>> @Jens - will it be fine if I start looking at plumbing nvme-part of
> >>> this series on top of your work?
> >>
> >> Sure, go ahead. Just beware that things are still changing, so you might
> >> have to adapt it a few times. It's still early days, but I do think
> >> that's the way forward in providing controlled access to what is
> >> basically async ioctls.
> >
> > Sounds good, I will start with the latest branch that you posted. Thanks.
>
> It's io_uring-fops.v2 for now, use that one.

Moved to v3 now.
nvme_user_io is 48 bytes, while nvme passthrough requires 72 or 80
bytes (passthru with 64 bit result).
The block_uring_cmd has 32 bytes of available space. If NVMe defines
its own "nvme_uring_cmd" (which can be used for nvme char interface)
that will buy some more space, but still won't be enough for passthru
command.

So I am looking at adding support for large-cmd in uring. And felt the
need to clear those doubts I mentioned above.


Thanks


From axboe at kernel.dk  Mon Feb 22 09:33:38 2021
From: axboe at kernel.dk (Jens Axboe)
Date: Mon, 22 Feb 2021 07:33:38 -0700
Subject: [RFC PATCH 0/4] Asynchronous passthrough ioctl
In-Reply-To: <CA+1E3rJqtpfDbKVGBjSYhX=WfzZ2bE8b0U3drUNz_=bp0u9Vuw@mail.gmail.com>
References: <CGME20210127150134epcas5p251fc1de3ff3581dd4c68b3fbe0b9dd91@epcas5p2.samsung.com>
 <20210127150029.13766-1-joshi.k@samsung.com>
 <489691ce-3b1e-30ce-9f72-d32389e33901@gmail.com>
 <a287bd9e-3474-83a4-e5c2-98df17214dc7@gmail.com>
 <CA+1E3rJHHFyjwv7Kp32E9H-cf5ksh0pOHSVdGoTpktQrB8SE6A@mail.gmail.com>
 <2d37d0ca-5853-4bb6-1582-551b9044040c@kernel.dk>
 <CA+1E3rKeqaLXBuvpMcjZ37XH9RqJHjPnTFObJj0T-u8K9Otw-w@mail.gmail.com>
 <dd0392a1-acfa-ef4d-5531-5f1dddc9efe7@kernel.dk>
 <CA+1E3rJqtpfDbKVGBjSYhX=WfzZ2bE8b0U3drUNz_=bp0u9Vuw@mail.gmail.com>
Message-ID: <03c15e42-c10a-1f67-c60b-ddc11902f9c5@kernel.dk>

On 2/22/21 6:42 AM, Kanchan Joshi wrote:
> On Thu, Jan 28, 2021 at 10:54 PM Jens Axboe <axboe at kernel.dk> wrote:
>>
>> On 1/28/21 10:13 AM, Kanchan Joshi wrote:
>>> On Thu, Jan 28, 2021 at 8:08 PM Jens Axboe <axboe at kernel.dk> wrote:
>>>>
>>>> On 1/28/21 5:04 AM, Kanchan Joshi wrote:
>>>>> On Wed, Jan 27, 2021 at 9:32 PM Pavel Begunkov <asml.silence at gmail.com> wrote:
>>>>>>
>>>>>> On 27/01/2021 15:42, Pavel Begunkov wrote:
>>>>>>> On 27/01/2021 15:00, Kanchan Joshi wrote:
>>>>>>>> This RFC patchset adds asynchronous ioctl capability for NVMe devices.
>>>>>>>> Purpose of RFC is to get the feedback and optimize the path.
>>>>>>>>
>>>>>>>> At the uppermost io-uring layer, a new opcode IORING_OP_IOCTL_PT is
>>>>>>>> presented to user-space applications. Like regular-ioctl, it takes
>>>>>>>> ioctl opcode and an optional argument (ioctl-specific input/output
>>>>>>>> parameter). Unlike regular-ioctl, it is made to skip the block-layer
>>>>>>>> and reach directly to the underlying driver (nvme in the case of this
>>>>>>>> patchset). This path between io-uring and nvme is via a newly
>>>>>>>> introduced block-device operation "async_ioctl". This operation
>>>>>>>> expects io-uring to supply a callback function which can be used to
>>>>>>>> report completion at later stage.
>>>>>>>>
>>>>>>>> For a regular ioctl, NVMe driver submits the command to the device and
>>>>>>>> the submitter (task) is made to wait until completion arrives. For
>>>>>>>> async-ioctl, completion is decoupled from submission. Submitter goes
>>>>>>>> back to its business without waiting for nvme-completion. When
>>>>>>>> nvme-completion arrives, it informs io-uring via the registered
>>>>>>>> completion-handler. But some ioctls may require updating certain
>>>>>>>> ioctl-specific fields which can be accessed only in context of the
>>>>>>>> submitter task. For that reason, NVMe driver uses task-work infra for
>>>>>>>> that ioctl-specific update. Since task-work is not exported, it cannot
>>>>>>>> be referenced when nvme is compiled as a module. Therefore, one of the
>>>>>>>> patch exports task-work API.
>>>>>>>>
>>>>>>>> Here goes example of usage (pseudo-code).
>>>>>>>> Actual nvme-cli source, modified to issue all ioctls via this opcode
>>>>>>>> is present at-
>>>>>>>> https://github.com/joshkan/nvme-cli/commit/a008a733f24ab5593e7874cfbc69ee04e88068c5
>>>>>>>
>>>>>>> see https://git.kernel.dk/cgit/linux-block/log/?h=io_uring-fops
>>>>>>>
>>>>>>> Looks like good time to bring that branch/discussion back
>>>>>>
>>>>>> a bit more context:
>>>>>> https://github.com/axboe/liburing/issues/270
>>>>>
>>>>> Thanks, it looked good. It seems key differences (compared to
>>>>> uring-patch that I posted) are -
>>>>> 1. using file-operation instead of block-dev operation.
>>>>
>>>> Right, it's meant to span wider than just block devices.
>>>>
>>>>> 2. repurpose the sqe memory for ioctl-cmd. If an application does
>>>>> ioctl with <=40 bytes of cmd, it does not have to allocate ioctl-cmd.
>>>>> That's nifty. We still need to support passing larger-cmd (e.g.
>>>>> nvme-passthru ioctl takes 72 bytes) but that shouldn't get too
>>>>> difficult I suppose.
>>>>
>>>> It's actually 48 bytes in the as-posted version, and I've bumped it to
>>>> 56 bytes in the latest branch. So not quite enough for everything,
>>>> nothing ever will be, but should work for a lot of cases without
>>>> requiring per-command allocations just for the actual command.
>>>
>>> Agreed. But if I got it right, you are open to support both in-the-sqe
>>> command (<= 56 bytes) and out-of-sqe command (> 56 bytes) with this
>>> interface.
>>> Driver processing the ioctl can fetch the cmd from user-space in one
>>> case (as it does now), and skips in another.
>>
>> Your out-of-seq command would be none of io_urings business, outside of
>> the fact that we'd need to ensure it's stable if we need to postpone
>> it. So yes, that would be fine, it just means your actual command is
>> passed in as a pointer, and you would be responsible for copying it
>> in for execution
>>
>> We're going to need something to handle postponing, and something
>> for ensuring that eg cancelations free the allocated memory.
> 
> I have few doubts about allocation/postponing. Are you referring to
> uring allocating memory for this case, similar to the way
> "req->async_data" is managed for few other opcodes?
> Or can it (i.e. larger cmd) remain a user-space pointer, and the
> underlying driver fetches the command in.
> If submission context changes (for sqo/io-wq case), uring seemed to
> apply context-grabbing techniques to make that work.

There are two concerns here:

1) We need more space than the 48 bytes, which means we need to allocate
   the command or part of the command when get the sqe, and of course
   free that when the command is done.

2) When io_uring_enter() returns and has consumed N commands, the state
   for those commands must be stable. Hence if you're passing in a
   pointer to a struct, that struct will have been read and store
   safely. This prevents things like on-stack structures from being an
   issue.

->async_data deals with #2 here, it's used when a command needs to store
data because we're switching to an async context to execute the command
(or the command is otherwise deferred, eg links and such). You can
always rely on the context being sane, it's either the task itself or
equivalent.

>>>>> And for some ioctls, driver may still need to use task-work to update
>>>>> the user-space pointers (embedded in uring/ioctl cmd) during
>>>>> completion.
>>>>>
>>>>> @Jens - will it be fine if I start looking at plumbing nvme-part of
>>>>> this series on top of your work?
>>>>
>>>> Sure, go ahead. Just beware that things are still changing, so you might
>>>> have to adapt it a few times. It's still early days, but I do think
>>>> that's the way forward in providing controlled access to what is
>>>> basically async ioctls.
>>>
>>> Sounds good, I will start with the latest branch that you posted. Thanks.
>>
>> It's io_uring-fops.v2 for now, use that one.
> 
> Moved to v3 now.
> nvme_user_io is 48 bytes, while nvme passthrough requires 72 or 80
> bytes (passthru with 64 bit result).
> The block_uring_cmd has 32 bytes of available space. If NVMe defines
> its own "nvme_uring_cmd" (which can be used for nvme char interface)
> that will buy some more space, but still won't be enough for passthru
> command.
> 
> So I am looking at adding support for large-cmd in uring. And felt the
> need to clear those doubts I mentioned above.

The simple solution is to just keep the command type the same on the
NVMe side, and just pass in a pointer to it. Then API could then be
nr_commands and **commands, for example.

But I think we're getting to the point where it'd be easier to just
discuss actual code. So if you rebase on top of the v3 code, then send
out those patches and we can discuss the most convenient API to present
for nvme passthrough and friends. Does that work?

-- 
Jens Axboe



From service at real-ptt.net  Mon Feb 22 10:16:32 2021
From: service at real-ptt.net (Nicholas Toms)
Date: Mon, 22 Feb 2021 07:16:32 -0800
Subject: Low-Cost Loans for SMEs & Investment Funding.
Message-ID: <mailman.5.1614007011.967.linux-nvme@lists.infradead.org>

My name is Nicholas Toms, an investment portfolio manage. We offer the right loan Investment funding with low interest to finance your business or project ranging from US$1M to US$2BIllion. Kindly contact me for more details as I am open to questions. 

Sincerely, 
Nicholas Toms


From selvajove at gmail.com  Mon Feb 22 10:57:50 2021
From: selvajove at gmail.com (Selva Jove)
Date: Mon, 22 Feb 2021 21:27:50 +0530
Subject: [RFC PATCH v5 3/4] nvme: add simple copy support
In-Reply-To: <20210220033637.GA2858050@casper.infradead.org>
References: <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <CGME20210219124608epcas5p2a673f9e00c3e7b5352f115497b0e2d98@epcas5p2.samsung.com>
 <20210219124517.79359-4-selvakuma.s1@samsung.com>
 <20210220033637.GA2858050@casper.infradead.org>
Message-ID: <CAHqX9vYdz-SRP2y6gzR1ei5WukNGzWhrHrvtXo4L8iFw4Lb9Hg@mail.gmail.com>

Matthew,

Maximum Source Range Count (MSRC) is limited by u8. So the maximum
number of source ranges is 256 (0 base value). The number of pages
required to be sent to the device is at most 2. Since we are
allocating the memory using kmalloc_array(), we would get a continuous
physical segment. nvme_map_data() maps the physical segment either by
setting 2 PRP pointers or by SGL. So the copy command sends two pages
to the device for copying more than128 ranges.

On Sat, Feb 20, 2021 at 9:08 AM Matthew Wilcox <willy at infradead.org> wrote:
>
> On Fri, Feb 19, 2021 at 06:15:16PM +0530, SelvaKumar S wrote:
> > +     struct nvme_copy_range *range = NULL;
> [...]
> > +     range = kmalloc_array(nr_range, sizeof(*range),
> > +                     GFP_ATOMIC | __GFP_NOWARN);
> [...]
> > +     req->special_vec.bv_page = virt_to_page(range);
> > +     req->special_vec.bv_offset = offset_in_page(range);
> > +     req->special_vec.bv_len = sizeof(*range) * nr_range;
> [...]
> > +struct nvme_copy_range {
> > +     __le64                  rsvd0;
> > +     __le64                  slba;
> > +     __le16                  nlb;
> > +     __le16                  rsvd18;
> > +     __le32                  rsvd20;
> > +     __le32                  eilbrt;
> > +     __le16                  elbat;
> > +     __le16                  elbatm;
> > +};
>
> so ... at 32 bytes, you can get 128 per 4kB page.  What happens if you
> try to send down a command that attempts to copy 129 ranges?


From javier at javigon.com  Mon Feb 22 14:01:05 2021
From: javier at javigon.com (javier at javigon.com)
Date: Mon, 22 Feb 2021 20:01:05 +0100
Subject: [PATCH V5 0/2] nvme: enable char device per namespace
Message-ID: <20210222190107.8479-1-javier.gonz@samsung.com>

From: Javier Gonz?lez <javier.gonz at samsung.com>

These two patches enable a char device per namespace, also through
multipath. It is possible to test this in QEMU using Keith's and Klaus'
tree in nvme-next

     http://git.infradead.org/qemu-nvme.git/shortlog/refs/heads/nvme-next

Keith: Regarding nvme-cli support, once this is in place, we will send a
patch to nvme-cli so that this device is shown in verbose mode. Do you
have anu thoughts about this?

Changes since V4
  - Added support for multipath (from Minwoo)
  - Fixed typo in commit message for sysfs naming
  - Rebase into nvme-5.12

Changes since V3
  - Use a dedicated ida for the generic handle
  - Do not abort namespace greation if the generic handle fails

Changes since V2:
  - Apply a number of naming and code structure improvements (from
    Christoph)
  - Use i_cdev to pull struct nvme_ns in the ioctl path instead of
    populating file->private_data (from Christoph)
  - Change char device and sysfs entries to /dev/nvme-generic-XcYnZ to
    follow the hidden device naming scheme (from Christoph and Keith)

Changes since V1:
  - Remove patches 1-3 which are already picked up by Christoph
  - Change the char device and sysfs entries to nvmeXnYc / c signals
    char device
  - Address Minwoo's comments on inline functions and style


Javier Gonz?lez (1):
  nvme: enable char device per namespace

Minwoo Im (1):
  nvme: allow open for nvme-generic char device

 drivers/nvme/host/core.c | 173 +++++++++++++++++++++++++++++++++++----
 drivers/nvme/host/nvme.h |   9 ++
 2 files changed, 165 insertions(+), 17 deletions(-)

-- 
2.17.1



From javier at javigon.com  Mon Feb 22 14:01:06 2021
From: javier at javigon.com (javier at javigon.com)
Date: Mon, 22 Feb 2021 20:01:06 +0100
Subject: [PATCH V5 1/2] nvme: enable char device per namespace
In-Reply-To: <20210222190107.8479-1-javier.gonz@samsung.com>
References: <20210222190107.8479-1-javier.gonz@samsung.com>
Message-ID: <20210222190107.8479-2-javier.gonz@samsung.com>

From: Javier Gonz?lez <javier.gonz at samsung.com>

Create a char device per NVMe namespace. This char device is always
initialized, independently of whether the features implemented by the
device are supported by the kernel. User-space can therefore always
issue IOCTLs to the NVMe driver using the char device.

The char device is presented as /dev/nvme-generic-XcYnZ. This naming
scheme follows the convention of the hidden device (nvmeXcYnZ). Support
for multipath will follow.

Signed-off-by: Javier Gonz?lez <javier.gonz at samsung.com>
Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 drivers/nvme/host/core.c | 171 +++++++++++++++++++++++++++++++++++----
 drivers/nvme/host/nvme.h |   9 +++
 2 files changed, 164 insertions(+), 16 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..d4884105ad95 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -86,13 +86,27 @@ static DEFINE_MUTEX(nvme_subsystems_lock);
 
 static DEFINE_IDA(nvme_instance_ida);
 static dev_t nvme_ctrl_base_chr_devt;
+
+static DEFINE_IDA(nvme_gen_minor_ida);
+static dev_t nvme_ns_base_chr_devt;
 static struct class *nvme_class;
+static struct class *nvme_ns_class;
 static struct class *nvme_subsys_class;
 
 static void nvme_put_subsystem(struct nvme_subsystem *subsys);
 static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
 					   unsigned nsid);
 
+static inline bool nvme_dev_is_generic(struct device *dev)
+{
+	return dev->class == nvme_ns_class;
+}
+
+static inline bool nvme_ns_is_generic(struct nvme_ns *ns)
+{
+	return !!ns->minor;
+}
+
 /*
  * Prepare a queue for teardown.
  *
@@ -559,7 +573,10 @@ static void nvme_free_ns(struct kref *kref)
 
 	if (ns->ndev)
 		nvme_nvm_unregister(ns);
+	if (nvme_ns_is_generic(ns))
+		ida_simple_remove(&nvme_gen_minor_ida, ns->minor - 1);
 
+	cdev_device_del(&ns->cdev, &ns->cdev_device);
 	put_disk(ns->disk);
 	nvme_put_ns_head(ns->head);
 	nvme_put_ctrl(ns->ctrl);
@@ -1772,15 +1789,15 @@ static int nvme_handle_ctrl_ioctl(struct nvme_ns *ns, unsigned int cmd,
 	return ret;
 }
 
-static int nvme_ioctl(struct block_device *bdev, fmode_t mode,
-		unsigned int cmd, unsigned long arg)
+static int nvme_disk_ioctl(struct gendisk *disk, unsigned int cmd,
+			   unsigned long arg)
 {
 	struct nvme_ns_head *head = NULL;
 	void __user *argp = (void __user *)arg;
 	struct nvme_ns *ns;
 	int srcu_idx, ret;
 
-	ns = nvme_get_ns_from_disk(bdev->bd_disk, &head, &srcu_idx);
+	ns = nvme_get_ns_from_disk(disk, &head, &srcu_idx);
 	if (unlikely(!ns))
 		return -EWOULDBLOCK;
 
@@ -1817,6 +1834,12 @@ static int nvme_ioctl(struct block_device *bdev, fmode_t mode,
 	return ret;
 }
 
+static int nvme_ioctl(struct block_device *bdev, fmode_t mode,
+		      unsigned int cmd, unsigned long arg)
+{
+	return nvme_disk_ioctl(bdev->bd_disk, cmd, arg);
+}
+
 #ifdef CONFIG_COMPAT
 struct nvme_user_io32 {
 	__u8	opcode;
@@ -1858,10 +1881,8 @@ static int nvme_compat_ioctl(struct block_device *bdev, fmode_t mode,
 #define nvme_compat_ioctl	NULL
 #endif /* CONFIG_COMPAT */
 
-static int nvme_open(struct block_device *bdev, fmode_t mode)
+static int nvme_ns_open(struct nvme_ns *ns)
 {
-	struct nvme_ns *ns = bdev->bd_disk->private_data;
-
 #ifdef CONFIG_NVME_MULTIPATH
 	/* should never be called due to GENHD_FL_HIDDEN */
 	if (WARN_ON_ONCE(ns->head->disk))
@@ -1880,14 +1901,22 @@ static int nvme_open(struct block_device *bdev, fmode_t mode)
 	return -ENXIO;
 }
 
-static void nvme_release(struct gendisk *disk, fmode_t mode)
+static void nvme_ns_release(struct nvme_ns *ns)
 {
-	struct nvme_ns *ns = disk->private_data;
-
 	module_put(ns->ctrl->ops->module);
 	nvme_put_ns(ns);
 }
 
+static int nvme_open(struct block_device *bdev, fmode_t mode)
+{
+	return nvme_ns_open(bdev->bd_disk->private_data);
+}
+
+static void nvme_release(struct gendisk *disk, fmode_t mode)
+{
+	nvme_ns_release(disk->private_data);
+}
+
 static int nvme_getgeo(struct block_device *bdev, struct hd_geometry *geo)
 {
 	/* some standard values */
@@ -2241,6 +2270,13 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_id_ns *id)
 	return 0;
 
 out_unfreeze:
+	/*
+	 * When the device does not support any of the features required by the
+	 * kernel (or viceversa), hide the block device. We can still rely on
+	 * the namespace char device for submitting IOCTLs
+	 */
+	ns->disk->flags |= GENHD_FL_HIDDEN;
+
 	blk_mq_unfreeze_queue(ns->disk->queue);
 	return ret;
 }
@@ -2378,6 +2414,38 @@ static const struct block_device_operations nvme_bdev_ops = {
 	.pr_ops		= &nvme_pr_ops,
 };
 
+static int nvme_cdev_open(struct inode *inode, struct file *file)
+{
+	struct nvme_ns *ns = container_of(inode->i_cdev, struct nvme_ns, cdev);
+
+	return nvme_ns_open(ns);
+}
+
+static int nvme_cdev_release(struct inode *inode, struct file *file)
+{
+	struct nvme_ns *ns = container_of(inode->i_cdev, struct nvme_ns, cdev);
+
+	nvme_ns_release(ns);
+	return 0;
+}
+
+static long nvme_cdev_ioctl(struct file *file, unsigned int cmd,
+			    unsigned long arg)
+{
+	struct nvme_ns *ns = container_of(file->f_inode->i_cdev,
+				struct nvme_ns, cdev);
+
+	return nvme_disk_ioctl(ns->disk, cmd, arg);
+}
+
+static const struct file_operations nvme_cdev_fops = {
+	.owner		= THIS_MODULE,
+	.open		= nvme_cdev_open,
+	.release	= nvme_cdev_release,
+	.unlocked_ioctl	= nvme_cdev_ioctl,
+	.compat_ioctl	= compat_ptr_ioctl,
+};
+
 #ifdef CONFIG_NVME_MULTIPATH
 static int nvme_ns_head_open(struct block_device *bdev, fmode_t mode)
 {
@@ -3379,6 +3447,9 @@ static inline struct nvme_ns_head *dev_to_ns_head(struct device *dev)
 {
 	struct gendisk *disk = dev_to_disk(dev);
 
+	if (nvme_dev_is_generic(dev))
+		return nvme_get_ns_from_cdev(dev)->head;
+
 	if (disk->fops == &nvme_bdev_ops)
 		return nvme_get_ns_from_dev(dev)->head;
 	else
@@ -3488,6 +3559,8 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 	}
 #ifdef CONFIG_NVME_MULTIPATH
 	if (a == &dev_attr_ana_grpid.attr || a == &dev_attr_ana_state.attr) {
+		if (nvme_dev_is_generic(dev))
+			return 0;
 		if (dev_to_disk(dev)->fops != &nvme_bdev_ops) /* per-path attr */
 			return 0;
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
@@ -3510,6 +3583,11 @@ const struct attribute_group *nvme_ns_id_attr_groups[] = {
 	NULL,
 };
 
+const struct attribute_group *nvme_ns_char_id_attr_groups[] = {
+	&nvme_ns_id_attr_group,
+	NULL,
+};
+
 #define nvme_show_str_function(field)						\
 static ssize_t  field##_show(struct device *dev,				\
 			    struct device_attribute *attr, char *buf)		\
@@ -3902,6 +3980,47 @@ struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 }
 EXPORT_SYMBOL_NS_GPL(nvme_find_get_ns, NVME_TARGET_PASSTHRU);
 
+static int nvme_alloc_chardev_ns(struct nvme_ctrl *ctrl, struct nvme_ns *ns)
+{
+	char cdisk_name[DISK_NAME_LEN];
+	int ret;
+
+	ret = ida_simple_get(&nvme_gen_minor_ida, 0, 0, GFP_KERNEL);
+	if (ret < 0)
+		return ret;
+
+	ns->minor = ret + 1;
+	device_initialize(&ns->cdev_device);
+	ns->cdev_device.devt = MKDEV(MAJOR(nvme_ns_base_chr_devt), ret);
+	ns->cdev_device.class = nvme_ns_class;
+	ns->cdev_device.parent = ctrl->device;
+	ns->cdev_device.groups = nvme_ns_char_id_attr_groups;
+	dev_set_drvdata(&ns->cdev_device, ns);
+
+	sprintf(cdisk_name, "nvme-generic-%dc%dn%d", ctrl->subsys->instance,
+		ctrl->instance, ns->head->instance);
+
+	ret = dev_set_name(&ns->cdev_device, "%s", cdisk_name);
+	if (ret)
+		goto put_ida;
+
+	cdev_init(&ns->cdev, &nvme_cdev_fops);
+	ns->cdev.owner = ctrl->ops->module;
+
+	ret = cdev_device_add(&ns->cdev, &ns->cdev_device);
+	if (ret)
+		goto free_kobj;
+
+	return ret;
+
+free_kobj:
+	kfree_const(ns->cdev_device.kobj.name);
+put_ida:
+	ida_simple_remove(&nvme_gen_minor_ida, ns->minor - 1);
+	ns->minor = 0;
+	return ret;
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 		struct nvme_ns_ids *ids)
 {
@@ -3948,8 +4067,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 	memcpy(disk->disk_name, disk_name, DISK_NAME_LEN);
 	ns->disk = disk;
 
-	if (nvme_update_ns_info(ns, id))
-		goto out_put_disk;
+	nvme_update_ns_info(ns, id);
 
 	if ((ctrl->quirks & NVME_QUIRK_LIGHTNVM) && id->vs[0] == 0x1) {
 		if (nvme_nvm_register(ns, disk_name, node)) {
@@ -3965,9 +4083,14 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 	nvme_get_ctrl(ctrl);
 
 	device_add_disk(ctrl->device, ns->disk, nvme_ns_id_attr_groups);
-
 	nvme_mpath_add_disk(ns, id);
 	nvme_fault_inject_init(&ns->fault_inject, ns->disk->disk_name);
+
+	if (nvme_alloc_chardev_ns(ctrl, ns))
+		dev_warn(ctrl->device,
+			"failed to create generic handle for nsid:%d\n",
+			nsid);
+
 	kfree(id);
 
 	return;
@@ -4780,23 +4903,38 @@ static int __init nvme_core_init(void)
 	if (result < 0)
 		goto destroy_delete_wq;
 
+	result = alloc_chrdev_region(&nvme_ns_base_chr_devt, 0,
+			NVME_MINORS, "nvmec");
+	if (result < 0)
+		goto unregister_dev_chrdev;
+
 	nvme_class = class_create(THIS_MODULE, "nvme");
 	if (IS_ERR(nvme_class)) {
 		result = PTR_ERR(nvme_class);
-		goto unregister_chrdev;
+		goto unregister_ns_chrdev;
 	}
 	nvme_class->dev_uevent = nvme_class_uevent;
 
+	nvme_ns_class = class_create(THIS_MODULE, "nvme-ns");
+	if (IS_ERR(nvme_ns_class)) {
+		result = PTR_ERR(nvme_ns_class);
+		goto destroy_dev_class;
+	}
+
 	nvme_subsys_class = class_create(THIS_MODULE, "nvme-subsystem");
 	if (IS_ERR(nvme_subsys_class)) {
 		result = PTR_ERR(nvme_subsys_class);
-		goto destroy_class;
+		goto destroy_ns_class;
 	}
 	return 0;
 
-destroy_class:
+destroy_ns_class:
+	class_destroy(nvme_ns_class);
+destroy_dev_class:
 	class_destroy(nvme_class);
-unregister_chrdev:
+unregister_ns_chrdev:
+	unregister_chrdev_region(nvme_ns_base_chr_devt, NVME_MINORS);
+unregister_dev_chrdev:
 	unregister_chrdev_region(nvme_ctrl_base_chr_devt, NVME_MINORS);
 destroy_delete_wq:
 	destroy_workqueue(nvme_delete_wq);
@@ -4812,6 +4950,7 @@ static void __exit nvme_core_exit(void)
 {
 	class_destroy(nvme_subsys_class);
 	class_destroy(nvme_class);
+	unregister_chrdev_region(nvme_ns_base_chr_devt, NVME_MINORS);
 	unregister_chrdev_region(nvme_ctrl_base_chr_devt, NVME_MINORS);
 	destroy_workqueue(nvme_delete_wq);
 	destroy_workqueue(nvme_reset_wq);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..8528caab61c5 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -442,6 +442,10 @@ struct nvme_ns {
 	struct kref kref;
 	struct nvme_ns_head *head;
 
+	struct device cdev_device;	/* char device */
+	struct cdev cdev;
+	int minor;
+
 	int lba_shift;
 	u16 ms;
 	u16 sgs;
@@ -819,6 +823,11 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
 	return dev_to_disk(dev)->private_data;
 }
 
+static inline struct nvme_ns *nvme_get_ns_from_cdev(struct device *dev)
+{
+	return dev_get_drvdata(dev);
+}
+
 #ifdef CONFIG_NVME_HWMON
 int nvme_hwmon_init(struct nvme_ctrl *ctrl);
 void nvme_hwmon_exit(struct nvme_ctrl *ctrl);
-- 
2.17.1



From javier at javigon.com  Mon Feb 22 14:01:07 2021
From: javier at javigon.com (javier at javigon.com)
Date: Mon, 22 Feb 2021 20:01:07 +0100
Subject: [PATCH V5 2/2] nvme: allow open for nvme-generic char device
In-Reply-To: <20210222190107.8479-1-javier.gonz@samsung.com>
References: <20210222190107.8479-1-javier.gonz@samsung.com>
Message-ID: <20210222190107.8479-3-javier.gonz@samsung.com>

From: Minwoo Im <minwoo.im.dev at gmail.com>

Keep rejecting the hidden device access via open, but allow cases
through the nvme-generic char device.

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d4884105ad95..0d0522bd4c2f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1885,7 +1885,7 @@ static int nvme_ns_open(struct nvme_ns *ns)
 {
 #ifdef CONFIG_NVME_MULTIPATH
 	/* should never be called due to GENHD_FL_HIDDEN */
-	if (WARN_ON_ONCE(ns->head->disk))
+	if (WARN_ON_ONCE(!nvme_ns_is_generic(ns) && ns->head->disk))
 		goto fail;
 #endif
 	if (!kref_get_unless_zero(&ns->kref))
-- 
2.17.1



From ematsumiya at suse.de  Mon Feb 22 14:02:53 2021
From: ematsumiya at suse.de (Enzo Matsumiya)
Date: Mon, 22 Feb 2021 16:02:53 -0300
Subject: [PATCH 00/35] RFC: add "nvme monitor" subcommand
In-Reply-To: <20210126203324.23610-1-mwilck@suse.com>
References: <20210126203324.23610-1-mwilck@suse.com>
Message-ID: <20210222190253.6ftslzvqt3iz3f3s@hyori>

Hi,

On 01/26, mwilck at suse.com wrote:
>From: Martin Wilck <mwilck at suse.com>
>
>(Cover letter copied from https://github.com/linux-nvme/nvme-cli/pull/877)
>
>This patch set adds a new subcommand **nvme monitor**. In this mode,
>**nvme-cli** runs continuously, monitors events (currently, uevents) relevant
>for discovery, and optionally autoconnects to newly discovered subsystems.

Is there anything missing from this patch set?

> * Implement support for RDMA and TCP protocols.

I'm working on the RDMA and TCP implementations with mDNS/Avahi support, and
I would like to submit my work based on this monitor solution.


Cheers,

Enzo


From nsaenzjulienne at suse.de  Mon Feb 22 14:05:36 2021
From: nsaenzjulienne at suse.de (Nicolas Saenz Julienne)
Date: Mon, 22 Feb 2021 20:05:36 +0100
Subject: [PATCH 6/9] swiotlb: refactor swiotlb_tbl_map_single
In-Reply-To: <20210207160327.2955490-7-hch@lst.de>
References: <20210207160327.2955490-1-hch@lst.de>
 <20210207160327.2955490-7-hch@lst.de>
Message-ID: <70edceb03e4bf283072397f9563dfc4162aec92a.camel@suse.de>

Hi Christoph,

On Sun, 2021-02-07 at 17:03 +0100, Christoph Hellwig wrote:
> Split out a bunch of a self-contained helpers to make the function easier
> to follow.
> 
> Signed-off-by: Christoph Hellwig <hch at lst.de>
> ---
> ?kernel/dma/swiotlb.c | 179 +++++++++++++++++++++----------------------
> ?1 file changed, 89 insertions(+), 90 deletions(-)
> 
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index b38b1553c4663a..381c24ef1ac1d0 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -468,134 +468,133 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
> ?	}
> ?}
> ?
> 
> -phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, phys_addr_t orig_addr,
> -		size_t mapping_size, size_t alloc_size,
> -		enum dma_data_direction dir, unsigned long attrs)
> -{
> -	dma_addr_t tbl_dma_addr = phys_to_dma_unencrypted(hwdev, io_tlb_start);
> -	unsigned long flags;
> -	phys_addr_t tlb_addr;
> -	unsigned int nslots, stride, index, wrap;
> -	int i;
> -	unsigned long mask;
> -	unsigned long offset_slots;
> -	unsigned long max_slots;
> -	unsigned long tmp_io_tlb_used;
> -
> -	if (no_iotlb_memory)
> -		panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer");
> -
> -	if (mem_encrypt_active())
> -		pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
> +#define slot_addr(start, idx)	((start) + ((idx) << IO_TLB_SHIFT))
> ?
> 
> -	if (mapping_size > alloc_size) {
> -		dev_warn_once(hwdev, "Invalid sizes (mapping: %zd bytes, alloc: %zd bytes)",
> -			      mapping_size, alloc_size);
> -		return (phys_addr_t)DMA_MAPPING_ERROR;
> -	}
> -
> -	mask = dma_get_seg_boundary(hwdev);
> +/*
> + * Carefully handle integer overflow which can occur when boundary_mask == ~0UL.
> + */
> +static inline unsigned long get_max_slots(unsigned long boundary_mask)
> +{
> +	if (boundary_mask == ~0UL)
> +		return 1UL << (BITS_PER_LONG - IO_TLB_SHIFT);
> +	return nr_slots(boundary_mask + 1);
> +}
> ?
> 
> -	tbl_dma_addr &= mask;
> +static unsigned int wrap_index(unsigned int index)
> +{
> +	if (index >= io_tlb_nslabs)
> +		return 0;
> +	return index;
> +}
> ?
> 
> -	offset_slots = nr_slots(tbl_dma_addr);
> +/*
> + * Find a suitable number of IO TLB entries size that will fit this request and
> + * allocate a buffer from that IO TLB pool.
> + */
> +static int find_slots(struct device *dev, size_t alloc_size)
> +{
> +	unsigned long boundary_mask = dma_get_seg_boundary(dev);
> +	dma_addr_t tbl_dma_addr =
> +		phys_to_dma_unencrypted(dev, io_tlb_start) & boundary_mask;
> +	unsigned int max_slots = get_max_slots(boundary_mask);

'max_slots' should be 'unsigned long' here. Breaks SWIOTLB on RPi4. Do you want
me to send a fix or you prefer editing the patch?

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210222/f081360b/attachment-0001.sig>

From konrad.wilk at oracle.com  Mon Feb 22 14:29:37 2021
From: konrad.wilk at oracle.com (Konrad Rzeszutek Wilk)
Date: Mon, 22 Feb 2021 14:29:37 -0500
Subject: [PATCH 6/9] swiotlb: refactor swiotlb_tbl_map_single
In-Reply-To: <70edceb03e4bf283072397f9563dfc4162aec92a.camel@suse.de>
References: <20210207160327.2955490-1-hch@lst.de>
 <20210207160327.2955490-7-hch@lst.de>
 <70edceb03e4bf283072397f9563dfc4162aec92a.camel@suse.de>
Message-ID: <YDQGIb4YqFpTKfyB@Konrads-MacBook-Pro.local>

> > +static int find_slots(struct device *dev, size_t alloc_size)
> > +{
> > +	unsigned long boundary_mask = dma_get_seg_boundary(dev);
> > +	dma_addr_t tbl_dma_addr =
> > +		phys_to_dma_unencrypted(dev, io_tlb_start) & boundary_mask;
> > +	unsigned int max_slots = get_max_slots(boundary_mask);
> 
> 'max_slots' should be 'unsigned long' here. Breaks SWIOTLB on RPi4. Do you want
> me to send a fix or you prefer editing the patch?

I can roll it in. Thx!


From kbusch at kernel.org  Mon Feb 22 14:29:59 2021
From: kbusch at kernel.org (Keith Busch)
Date: Tue, 23 Feb 2021 04:29:59 +0900
Subject: [PATCH] nvme-core: Switch to using the new API kobj_to_dev()
In-Reply-To: <1613812218-17601-1-git-send-email-yang.lee@linux.alibaba.com>
References: <1613812218-17601-1-git-send-email-yang.lee@linux.alibaba.com>
Message-ID: <20210222192959.GA16253@redsun51.ssa.fujisawa.hgst.com>

On Sat, Feb 20, 2021 at 05:10:18PM +0800, Yang Li wrote:
> fixed the following coccicheck:
> ./drivers/nvme/host/core.c:3440:60-61: WARNING opportunity for
> kobj_to_dev()
> ./drivers/nvme/host/core.c:3679:60-61: WARNING opportunity for
> kobj_to_dev()
> 
> Reported-by: Abaci Robot <abaci at linux.alibaba.com>
> Signed-off-by: Yang Li <yang.lee at linux.alibaba.com>

This was rejected the last time it posted 6 months ago:

http://lists.infradead.org/pipermail/linux-nvme/2020-September/019463.html


From mwilck at suse.com  Mon Feb 22 16:05:13 2021
From: mwilck at suse.com (Martin Wilck)
Date: Mon, 22 Feb 2021 22:05:13 +0100
Subject: [PATCH 00/35] RFC: add "nvme monitor" subcommand
In-Reply-To: <20210222190253.6ftslzvqt3iz3f3s@hyori>
References: <20210126203324.23610-1-mwilck@suse.com>
 <20210222190253.6ftslzvqt3iz3f3s@hyori>
Message-ID: <336498bf0c38269cb2b9bbbd33d33a3a360a758f.camel@suse.com>

On Mon, 2021-02-22 at 16:02 -0300, Enzo Matsumiya wrote:
> Hi,
> 
> On 01/26, mwilck at suse.com?wrote:
> > From: Martin Wilck <mwilck at suse.com>
> > 
> > (Cover letter copied from 
> > https://github.com/linux-nvme/nvme-cli/pull/877)
> > 
> > This patch set adds a new subcommand **nvme monitor**. In this
> > mode,
> > **nvme-cli** runs continuously, monitors events (currently,
> > uevents) relevant
> > for discovery, and optionally autoconnects to newly discovered
> > subsystems.
> 
> Is there anything missing from this patch set?
> 
> > * Implement support for RDMA and TCP protocols.
> 
> I'm working on the RDMA and TCP implementations with mDNS/Avahi
> support, and
> I would like to submit my work based on this monitor solution.

I'm still working on a v2. Please have some more patience.

Regards,
Martin




From donnie at sihovision.com  Mon Feb 22 20:26:40 2021
From: donnie at sihovision.com (donnie at sihovision.com)
Date: Tue, 23 Feb 2021 09:26:40 +0800
Subject: Various Stainless Steel Industrial Panel PC
In-Reply-To: <2021020023092600400476@tradepro.net>
Message-ID: <2021020023092600400476@tradepro.net>

Dear friend,
Sihovision  developed a wide range Stainless Steel Industrial Touch Panel PC to meet the market requirements. 
Full IP65:
Full IP67:
True Flat Panel:
Front IP65:
Customization:
Stainless steel 304/316 enclosure
Full IP65/IP67 optional 
Capacitive/Resistive touch optional 
EETI touch controller waterproof solution 
High brightness sunlight readable optional 
Optical bonding optional 
Size: 10"-24" 
Welcom inquiry!
----------------------------------------------------------------------------
Best regards,
Donnie Nie
SHINHO ELECTRONIC
www.sihovision.com
TEL/Whatsapp: 0086 13163734006
Skype: donnie.shinho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210223/a02ef5da/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SSPPC-5.jpg
Type: image/jpeg
Size: 18972 bytes
Desc: Photo
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210223/a02ef5da/attachment-0006.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SSPPC-6.jpg
Type: image/jpeg
Size: 18529 bytes
Desc: Photo
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210223/a02ef5da/attachment-0007.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SSPPC-7.jpg
Type: image/jpeg
Size: 18694 bytes
Desc: Photo
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210223/a02ef5da/attachment-0008.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SSPPC-8.jpg
Type: image/jpeg
Size: 19969 bytes
Desc: Photo
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210223/a02ef5da/attachment-0009.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SSPPC-9.jpg
Type: image/jpeg
Size: 20450 bytes
Desc: Photo
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210223/a02ef5da/attachment-0010.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logo2.jpg
Type: image/jpeg
Size: 2231 bytes
Desc: Photo
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20210223/a02ef5da/attachment-0011.jpg>

From joshiiitr at gmail.com  Mon Feb 22 23:41:25 2021
From: joshiiitr at gmail.com (Kanchan Joshi)
Date: Tue, 23 Feb 2021 10:11:25 +0530
Subject: [RFC PATCH 0/4] Asynchronous passthrough ioctl
In-Reply-To: <03c15e42-c10a-1f67-c60b-ddc11902f9c5@kernel.dk>
References: <CGME20210127150134epcas5p251fc1de3ff3581dd4c68b3fbe0b9dd91@epcas5p2.samsung.com>
 <20210127150029.13766-1-joshi.k@samsung.com>
 <489691ce-3b1e-30ce-9f72-d32389e33901@gmail.com>
 <a287bd9e-3474-83a4-e5c2-98df17214dc7@gmail.com>
 <CA+1E3rJHHFyjwv7Kp32E9H-cf5ksh0pOHSVdGoTpktQrB8SE6A@mail.gmail.com>
 <2d37d0ca-5853-4bb6-1582-551b9044040c@kernel.dk>
 <CA+1E3rKeqaLXBuvpMcjZ37XH9RqJHjPnTFObJj0T-u8K9Otw-w@mail.gmail.com>
 <dd0392a1-acfa-ef4d-5531-5f1dddc9efe7@kernel.dk>
 <CA+1E3rJqtpfDbKVGBjSYhX=WfzZ2bE8b0U3drUNz_=bp0u9Vuw@mail.gmail.com>
 <03c15e42-c10a-1f67-c60b-ddc11902f9c5@kernel.dk>
Message-ID: <CA+1E3r+e8FgiHVA4OB_igUBboJH9iMrh+Fm1E5RoEs+o1ryCwg@mail.gmail.com>

On Mon, Feb 22, 2021 at 8:03 PM Jens Axboe <axboe at kernel.dk> wrote:
>
> On 2/22/21 6:42 AM, Kanchan Joshi wrote:
> > On Thu, Jan 28, 2021 at 10:54 PM Jens Axboe <axboe at kernel.dk> wrote:
> >>
> >> On 1/28/21 10:13 AM, Kanchan Joshi wrote:
> >>> On Thu, Jan 28, 2021 at 8:08 PM Jens Axboe <axboe at kernel.dk> wrote:
> >>>>
> >>>> On 1/28/21 5:04 AM, Kanchan Joshi wrote:
> >>>>> On Wed, Jan 27, 2021 at 9:32 PM Pavel Begunkov <asml.silence at gmail.com> wrote:
> >>>>>>
> >>>>>> On 27/01/2021 15:42, Pavel Begunkov wrote:
> >>>>>>> On 27/01/2021 15:00, Kanchan Joshi wrote:
> >>>>>>>> This RFC patchset adds asynchronous ioctl capability for NVMe devices.
> >>>>>>>> Purpose of RFC is to get the feedback and optimize the path.
> >>>>>>>>
> >>>>>>>> At the uppermost io-uring layer, a new opcode IORING_OP_IOCTL_PT is
> >>>>>>>> presented to user-space applications. Like regular-ioctl, it takes
> >>>>>>>> ioctl opcode and an optional argument (ioctl-specific input/output
> >>>>>>>> parameter). Unlike regular-ioctl, it is made to skip the block-layer
> >>>>>>>> and reach directly to the underlying driver (nvme in the case of this
> >>>>>>>> patchset). This path between io-uring and nvme is via a newly
> >>>>>>>> introduced block-device operation "async_ioctl". This operation
> >>>>>>>> expects io-uring to supply a callback function which can be used to
> >>>>>>>> report completion at later stage.
> >>>>>>>>
> >>>>>>>> For a regular ioctl, NVMe driver submits the command to the device and
> >>>>>>>> the submitter (task) is made to wait until completion arrives. For
> >>>>>>>> async-ioctl, completion is decoupled from submission. Submitter goes
> >>>>>>>> back to its business without waiting for nvme-completion. When
> >>>>>>>> nvme-completion arrives, it informs io-uring via the registered
> >>>>>>>> completion-handler. But some ioctls may require updating certain
> >>>>>>>> ioctl-specific fields which can be accessed only in context of the
> >>>>>>>> submitter task. For that reason, NVMe driver uses task-work infra for
> >>>>>>>> that ioctl-specific update. Since task-work is not exported, it cannot
> >>>>>>>> be referenced when nvme is compiled as a module. Therefore, one of the
> >>>>>>>> patch exports task-work API.
> >>>>>>>>
> >>>>>>>> Here goes example of usage (pseudo-code).
> >>>>>>>> Actual nvme-cli source, modified to issue all ioctls via this opcode
> >>>>>>>> is present at-
> >>>>>>>> https://github.com/joshkan/nvme-cli/commit/a008a733f24ab5593e7874cfbc69ee04e88068c5
> >>>>>>>
> >>>>>>> see https://git.kernel.dk/cgit/linux-block/log/?h=io_uring-fops
> >>>>>>>
> >>>>>>> Looks like good time to bring that branch/discussion back
> >>>>>>
> >>>>>> a bit more context:
> >>>>>> https://github.com/axboe/liburing/issues/270
> >>>>>
> >>>>> Thanks, it looked good. It seems key differences (compared to
> >>>>> uring-patch that I posted) are -
> >>>>> 1. using file-operation instead of block-dev operation.
> >>>>
> >>>> Right, it's meant to span wider than just block devices.
> >>>>
> >>>>> 2. repurpose the sqe memory for ioctl-cmd. If an application does
> >>>>> ioctl with <=40 bytes of cmd, it does not have to allocate ioctl-cmd.
> >>>>> That's nifty. We still need to support passing larger-cmd (e.g.
> >>>>> nvme-passthru ioctl takes 72 bytes) but that shouldn't get too
> >>>>> difficult I suppose.
> >>>>
> >>>> It's actually 48 bytes in the as-posted version, and I've bumped it to
> >>>> 56 bytes in the latest branch. So not quite enough for everything,
> >>>> nothing ever will be, but should work for a lot of cases without
> >>>> requiring per-command allocations just for the actual command.
> >>>
> >>> Agreed. But if I got it right, you are open to support both in-the-sqe
> >>> command (<= 56 bytes) and out-of-sqe command (> 56 bytes) with this
> >>> interface.
> >>> Driver processing the ioctl can fetch the cmd from user-space in one
> >>> case (as it does now), and skips in another.
> >>
> >> Your out-of-seq command would be none of io_urings business, outside of
> >> the fact that we'd need to ensure it's stable if we need to postpone
> >> it. So yes, that would be fine, it just means your actual command is
> >> passed in as a pointer, and you would be responsible for copying it
> >> in for execution
> >>
> >> We're going to need something to handle postponing, and something
> >> for ensuring that eg cancelations free the allocated memory.
> >
> > I have few doubts about allocation/postponing. Are you referring to
> > uring allocating memory for this case, similar to the way
> > "req->async_data" is managed for few other opcodes?
> > Or can it (i.e. larger cmd) remain a user-space pointer, and the
> > underlying driver fetches the command in.
> > If submission context changes (for sqo/io-wq case), uring seemed to
> > apply context-grabbing techniques to make that work.
>
> There are two concerns here:
>
> 1) We need more space than the 48 bytes, which means we need to allocate
>    the command or part of the command when get the sqe, and of course
>    free that when the command is done.
>
> 2) When io_uring_enter() returns and has consumed N commands, the state
>    for those commands must be stable. Hence if you're passing in a
>    pointer to a struct, that struct will have been read and store
>    safely. This prevents things like on-stack structures from being an
>    issue.
>
> ->async_data deals with #2 here, it's used when a command needs to store
> data because we're switching to an async context to execute the command
> (or the command is otherwise deferred, eg links and such). You can
> always rely on the context being sane, it's either the task itself or
> equivalent.

Thanks for sorting this out.

> >>>>> And for some ioctls, driver may still need to use task-work to update
> >>>>> the user-space pointers (embedded in uring/ioctl cmd) during
> >>>>> completion.
> >>>>>
> >>>>> @Jens - will it be fine if I start looking at plumbing nvme-part of
> >>>>> this series on top of your work?
> >>>>
> >>>> Sure, go ahead. Just beware that things are still changing, so you might
> >>>> have to adapt it a few times. It's still early days, but I do think
> >>>> that's the way forward in providing controlled access to what is
> >>>> basically async ioctls.
> >>>
> >>> Sounds good, I will start with the latest branch that you posted. Thanks.
> >>
> >> It's io_uring-fops.v2 for now, use that one.
> >
> > Moved to v3 now.
> > nvme_user_io is 48 bytes, while nvme passthrough requires 72 or 80
> > bytes (passthru with 64 bit result).
> > The block_uring_cmd has 32 bytes of available space. If NVMe defines
> > its own "nvme_uring_cmd" (which can be used for nvme char interface)
> > that will buy some more space, but still won't be enough for passthru
> > command.
> >
> > So I am looking at adding support for large-cmd in uring. And felt the
> > need to clear those doubts I mentioned above.
>
> The simple solution is to just keep the command type the same on the
> NVMe side, and just pass in a pointer to it. Then API could then be
> nr_commands and **commands, for example.
>
> But I think we're getting to the point where it'd be easier to just
> discuss actual code. So if you rebase on top of the v3 code, then send
> out those patches and we can discuss the most convenient API to present
> for nvme passthrough and friends. Does that work?

Yes, perfect. I will go about that.


From Martin.George at netapp.com  Tue Feb 23 00:41:00 2021
From: Martin.George at netapp.com (George, Martin)
Date: Tue, 23 Feb 2021 05:41:00 +0000
Subject: [PATCH] fabrics: ensure zero kato for non-persistent controllers
In-Reply-To: <20210218095422.5696-1-marting@netapp.com>
References: <20210218095422.5696-1-marting@netapp.com>
Message-ID: <cb89029d362a3cd61504012351f56731cec9c6fe.camel@netapp.com>

Ping?

On Thu, 2021-02-18 at 15:24 +0530, Martin George wrote:
> Along with ensuring a non-zero kato is passed to a
> persistent discovery controller, also ensure a zero
> kato is passed to a non-persistent discovery controller.
> 
> Signed-off-by: Martin George <marting at netapp.com>
> ---
>  fabrics.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/fabrics.c b/fabrics.c
> index 1880a5d..ed15e08 100644
> --- a/fabrics.c
> +++ b/fabrics.c
> @@ -1399,6 +1399,8 @@ static int discover_from_conf_file(const char
> *desc, char *argstr,
> 
>                 if (cfg.persistent && !cfg.keep_alive_tmo)
>                         cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
> +               else if (!cfg.persistent && (cfg.keep_alive_tmo > 0))
> +                       cfg.keep_alive_tmo = 0;
> 
>                 if (traddr_is_hostname(&cfg)) {
>                         ret = hostname2traddr(&cfg);
> @@ -1474,6 +1476,8 @@ int fabrics_discover(const char *desc, int
> argc, char **argv, bool connect)
>         } else {
>                 if (cfg.persistent && !cfg.keep_alive_tmo)
>                         cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
> +               else if (!cfg.persistent && (cfg.keep_alive_tmo > 0))
> +                       cfg.keep_alive_tmo = 0;
> 
>                 if (traddr_is_hostname(&cfg)) {
>                         ret = hostname2traddr(&cfg);
> --
> 2.30.0
> 

From Chaitanya.Kulkarni at wdc.com  Tue Feb 23 01:06:57 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 06:06:57 +0000
Subject: [PATCH] fabrics: ensure zero kato for non-persistent controllers
References: <20210218095422.5696-1-marting@netapp.com>
 <cb89029d362a3cd61504012351f56731cec9c6fe.camel@netapp.com>
Message-ID: <BYAPR04MB4965AD8096D8F313B2BFF74C86809@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/22/21 21:47, George, Martin wrote:
> Ping?
>
> On Thu, 2021-02-18 at 15:24 +0530, Martin George wrote:
>> Along with ensuring a non-zero kato is passed to a
>> persistent discovery controller, also ensure a zero
>> kato is passed to a non-persistent discovery controller.
>>
>> Signed-off-by: Martin George <marting at netapp.com>
>> ---
>>  fabrics.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/fabrics.c b/fabrics.c
>> index 1880a5d..ed15e08 100644
>> --- a/fabrics.c
>> +++ b/fabrics.c
>> @@ -1399,6 +1399,8 @@ static int discover_from_conf_file(const char
>> *desc, char *argstr,
>>
>>                 if (cfg.persistent && !cfg.keep_alive_tmo)
>>                         cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
>> +               else if (!cfg.persistent && (cfg.keep_alive_tmo > 0))
>> +                       cfg.keep_alive_tmo = 0;
>>
>>                 if (traddr_is_hostname(&cfg)) {
>>                         ret = hostname2traddr(&cfg);
>> @@ -1474,6 +1476,8 @@ int fabrics_discover(const char *desc, int
>> argc, char **argv, bool connect)
>>         } else {
>>                 if (cfg.persistent && !cfg.keep_alive_tmo)
>>                         cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
>> +               else if (!cfg.persistent && (cfg.keep_alive_tmo > 0))
>> +                       cfg.keep_alive_tmo = 0;
>>
>>                 if (traddr_is_hostname(&cfg)) {
>>                         ret = hostname2traddr(&cfg);
Is that if else if ladder code is repeated ? if so can you please add a
helper ?
unless there is a reason for keeping the duplicate code.
Also can we remove () for (cfg.keep_alive_tmo > 0) condition ?
>> --
>> 2.30.0
>>
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
>



From Chaitanya.Kulkarni at wdc.com  Tue Feb 23 01:34:37 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 06:34:37 +0000
Subject: [PATCH] nvmet: don't change model_number on live subsys
References: <20210218031644.12476-1-chaitanya.kulkarni@wdc.com>
 <20210218070513.GA16309@lst.de>
Message-ID: <BYAPR04MB4965FE04148739AFCD69B97686809@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/17/21 23:05, Christoph Hellwig wrote:
>> +	mutex_lock(&subsys->lock);
>> +	list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
>> +		connected = true;
>> +		break;
>> +	}
>> +	mutex_unlock(&subsys->lock);
> This could just use list_empty.  But we'd also need to hold the lock
> over the actual change of the model number to avoid a race window.
>
Since subsys model number actual change occurs later in the
nvmet_subsys_attr_model_store() are you saying something like this we need
instead of having a separate helper ?

diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 635a7cb45d0b..c9fd98080b55 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -1145,6 +1145,7 @@ static ssize_t
nvmet_subsys_attr_model_store(struct config_item *item,
        struct nvmet_subsys_model *new_model;
        char *new_model_number;
        int pos = 0, len;
+       int ret = 0;
 
        len = strcspn(page, "\n");
        if (!len)
@@ -1168,15 +1169,23 @@ static ssize_t
nvmet_subsys_attr_model_store(struct config_item *item,
 
        down_write(&nvmet_config_sem);
        mutex_lock(&subsys->lock);
+       /* make sure this subsys is not connected */
+       if (!list_empty(&subsys->ctrls)) {
+               ret = -EINVAL;
+               goto out_unlock;
+       }
+
        new_model = rcu_replace_pointer(subsys->model, new_model,
                                        mutex_is_locked(&subsys->lock));
+
+out_unlock:
        mutex_unlock(&subsys->lock);
        up_write(&nvmet_config_sem);
 
        kfree_rcu(new_model, rcuhead);
        kfree(new_model_number);
 
-       return count;
+       return ret ? ret : count;
 }
 CONFIGFS_ATTR(nvmet_subsys_, attr_model);
 



From Chaitanya.Kulkarni at wdc.com  Tue Feb 23 01:54:08 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 06:54:08 +0000
Subject: [PATCH 1/2] nvme-pci; remove the barriers nvme_irq()
References: <20210219063726.63728-1-chaitanya.kulkarni@wdc.com>
 <80fc53d2-4f35-7c72-5cc5-3ec13c5ee51a@kernel.dk>
Message-ID: <BYAPR04MB496513669571FA6541E58B8686809@BYAPR04MB4965.namprd04.prod.outlook.com>

(+Heiner)

On 2/19/21 05:40, Jens Axboe wrote:
> On 2/18/21 11:37 PM, Chaitanya Kulkarni wrote:
>> The barriers were added to the nvme_irq() in the
>> commit 3a7afd8ee42a ("nvme-pci: remove the CQ lock for interrupt driven queues")
>> to prevent compiler from doing memory optimization for the variabes that
>> were protected previously by spinlock in nvme_irq() at completion queue
>> processing and with queue head check condition..
>>
>> The variable nvmeq->last_cq_head from those checks was removed in the
>> commit f6c4d97b0d82 ("nvme/pci: Remove last_cq_head")") that was
>> preventing  poll queues from mistakenly triggering the spurious
>> interrupt detection.
>>
>> Remove the barriers which were protecting the updates to the variables.
> Would seem prudent to just ask the original submitter to send a v2
> of his patch, as that's where it came from.
>
Please let me know if you want to send V2 otherwise please ack this series.


From hch at lst.de  Tue Feb 23 02:25:14 2021
From: hch at lst.de (Christoph Hellwig)
Date: Tue, 23 Feb 2021 08:25:14 +0100
Subject: [PATCH 6/9] swiotlb: refactor swiotlb_tbl_map_single
In-Reply-To: <YDQGIb4YqFpTKfyB@Konrads-MacBook-Pro.local>
References: <20210207160327.2955490-1-hch@lst.de>
 <20210207160327.2955490-7-hch@lst.de>
 <70edceb03e4bf283072397f9563dfc4162aec92a.camel@suse.de>
 <YDQGIb4YqFpTKfyB@Konrads-MacBook-Pro.local>
Message-ID: <20210223072514.GA18079@lst.de>

On Mon, Feb 22, 2021 at 02:29:37PM -0500, Konrad Rzeszutek Wilk wrote:
> > 'max_slots' should be 'unsigned long' here. Breaks SWIOTLB on RPi4. Do you want
> > me to send a fix or you prefer editing the patch?
> 
> I can roll it in. Thx!

Thanks to both of you!


From lengchao at huawei.com  Tue Feb 23 02:26:02 2021
From: lengchao at huawei.com (Chao Leng)
Date: Tue, 23 Feb 2021 15:26:02 +0800
Subject: [PATCH] nvme-rdma: fix crash for no IO queues
Message-ID: <20210223072602.3196-1-lengchao@huawei.com>

A crash happens when set feature(NVME_FEAT_NUM_QUEUES) timeout in nvme
over rdma(roce) reconnection, the reason is use the queue which is not
alloced.

If it is not discovery and no io queues, the connection should fail.

Signed-off-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/rdma.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 53ac4d7442ba..7410972353ae 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -736,8 +736,11 @@ static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
 		return ret;
 
 	ctrl->ctrl.queue_count = nr_io_queues + 1;
-	if (ctrl->ctrl.queue_count < 2)
-		return 0;
+	if (ctrl->ctrl.queue_count < 2) {
+		if (opts->discovery_nqn)
+			return 0;
+		return -EAGAIN;
+	}
 
 	dev_info(ctrl->ctrl.device,
 		"creating %d I/O queues.\n", nr_io_queues);
-- 
2.16.4



From selvajove at gmail.com  Tue Feb 23 04:00:29 2021
From: selvajove at gmail.com (Selva Jove)
Date: Tue, 23 Feb 2021 14:30:29 +0530
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <lfbgr3ur.fsf@damenly.su>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com> <lfbgr3ur.fsf@damenly.su>
Message-ID: <CAHqX9vbdtNiRvAHSy+1+rD6FEp6zBqmH2P_99P-+dgMZDbMZsA@mail.gmail.com>

Thanks Su Yue. I'll update the link in the next series.

On Mon, Feb 22, 2021 at 12:23 PM Su Yue <l at damenly.su> wrote:
>
>
> On Fri 19 Feb 2021 at 20:45, SelvaKumar S
> <selvakuma.s1 at samsung.com> wrote:
>
> > This patchset tries to add support for TP4065a ("Simple Copy
> > Command"),
> > v2020.05.04 ("Ratified")
> >
> > The Specification can be found in following link.
> > https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
> >
>
> 404 not found.
> Should it be
> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs.zip
> ?
>
> > Simple copy command is a copy offloading operation and is  used
> > to copy
> > multiple contiguous ranges (source_ranges) of LBA's to a single
> > destination
> > LBA within the device reducing traffic between host and device.
> >
> > This implementation doesn't add native copy offload support for
> > stacked
> > devices rather copy offload is done through emulation. Possible
> > use
> > cases are F2FS gc and BTRFS relocation/balance.
> >
> > *blkdev_issue_copy* takes source bdev, no of sources, array of
> > source
> > ranges (in sectors), destination bdev and destination offset(in
> > sectors).
> > If both source and destination block devices are same and
> > copy_offload = 1,
> > then copy is done through native copy offloading. Copy emulation
> > is used
> > in other cases.
> >
> > As SCSI XCOPY can take two different block devices and no of
> > source range is
> > equal to 1, this interface can be extended in future to support
> > SCSI XCOPY.
> >
> > For devices supporting native simple copy, attach the control
> > information
> > as payload to the bio and submit to the device. For devices
> > without native
> > copy support, copy emulation is done by reading each source
> > range into memory
> > and writing it to the destination. Caller can choose not to try
> > emulation if copy offload is not supported by setting
> > BLKDEV_COPY_NOEMULATION flag.
> >
> > Following limits are added to queue limits and are exposed in
> > sysfs
> > to userspace
> >       - *copy_offload* controls copy_offload. set 0 to disable copy
> >               offload, 1 to enable native copy offloading support.
> >       - *max_copy_sectors* limits the sum of all source_range length
> >       - *max_copy_nr_ranges* limits the number of source ranges
> >       - *max_copy_range_sectors* limit the maximum number of sectors
> >               that can constitute a single source range.
> >
> >       max_copy_sectors = 0 indicates the device doesn't support copy
> > offloading.
> >
> >       *copy offload* sysfs entry is configurable and can be used
> > toggle
> > between emulation and native support depending upon the usecase.
> >
> > Changes from v4
> >
> > 1. Extend dm-kcopyd to leverage copy-offload, while copying
> > within the
> > same device. The other approach was to have copy-emulation by
> > moving
> > dm-kcopyd to block layer. But it also required moving core dm-io
> > infra,
> > causing a massive churn across multiple dm-targets.
> >
> > 2. Remove export in bio_map_kern()
> > 3. Change copy_offload sysfs to accept 0 or else
> > 4. Rename copy support flag to QUEUE_FLAG_SIMPLE_COPY
> > 5. Rename payload entries, add source bdev field to be used
> > while
> > partition remapping, remove copy_size
> > 6. Change the blkdev_issue_copy() interface to accept
> > destination and
> > source values in sector rather in bytes
> > 7. Add payload to bio using bio_map_kern() for copy_offload case
> > 8. Add check to return error if one of the source range length
> > is 0
> > 9. Add BLKDEV_COPY_NOEMULATION flag to allow user to not try
> > copy
> > emulation incase of copy offload is not supported. Caller can
> > his use
> > his existing copying logic to complete the io.
> > 10. Bug fix copy checks and reduce size of rcu_lock()
> >
> > Planned for next:
> > - adding blktests
> > - handling larger (than device limits) copy
> > - decide on ioctl interface (man-page etc.)
> >
> > Changes from v3
> >
> > 1. gfp_flag fixes.
> > 2. Export bio_map_kern() and use it to allocate and add pages to
> > bio.
> > 3. Move copy offload, reading to buf, writing from buf to
> > separate functions.
> > 4. Send read bio of copy offload by chaining them and submit
> > asynchronously.
> > 5. Add gendisk->part0 and part->bd_start_sect changes to
> > blk_check_copy().
> > 6. Move single source range limit check to blk_check_copy()
> > 7. Rename __blkdev_issue_copy() to blkdev_issue_copy and remove
> > old helper.
> > 8. Change blkdev_issue_copy() interface generic to accepts
> > destination bdev
> >       to support XCOPY as well.
> > 9. Add invalidate_kernel_vmap_range() after reading data for
> > vmalloc'ed memory.
> > 10. Fix buf allocoation logic to allocate buffer for the total
> > size of copy.
> > 11. Reword patch commit description.
> >
> > Changes from v2
> >
> > 1. Add emulation support for devices not supporting copy.
> > 2. Add *copy_offload* sysfs entry to enable and disable
> > copy_offload
> >       in devices supporting simple copy.
> > 3. Remove simple copy support for stacked devices.
> >
> > Changes from v1:
> >
> > 1. Fix memory leak in __blkdev_issue_copy
> > 2. Unmark blk_check_copy inline
> > 3. Fix line break in blk_check_copy_eod
> > 4. Remove p checks and made code more readable
> > 5. Don't use bio_set_op_attrs and remove op and set
> >    bi_opf directly
> > 6. Use struct_size to calculate total_size
> > 7. Fix partition remap of copy destination
> > 8. Remove mcl,mssrl,msrc from nvme_ns
> > 9. Initialize copy queue limits to 0 in nvme_config_copy
> > 10. Remove return in QUEUE_FLAG_COPY check
> > 11. Remove unused OCFS
> >
> > SelvaKumar S (4):
> >   block: make bio_map_kern() non static
> >   block: add simple copy support
> >   nvme: add simple copy support
> >   dm kcopyd: add simple copy offload support
> >
> >  block/blk-core.c          | 102 +++++++++++++++--
> >  block/blk-lib.c           | 223
> >  ++++++++++++++++++++++++++++++++++++++
> >  block/blk-map.c           |   2 +-
> >  block/blk-merge.c         |   2 +
> >  block/blk-settings.c      |  10 ++
> >  block/blk-sysfs.c         |  47 ++++++++
> >  block/blk-zoned.c         |   1 +
> >  block/bounce.c            |   1 +
> >  block/ioctl.c             |  33 ++++++
> >  drivers/md/dm-kcopyd.c    |  49 ++++++++-
> >  drivers/nvme/host/core.c  |  87 +++++++++++++++
> >  include/linux/bio.h       |   1 +
> >  include/linux/blk_types.h |  14 +++
> >  include/linux/blkdev.h    |  17 +++
> >  include/linux/nvme.h      |  43 +++++++-
> >  include/uapi/linux/fs.h   |  13 +++
> >  16 files changed, 627 insertions(+), 18 deletions(-)
>


From selvajove at gmail.com  Tue Feb 23 04:14:08 2021
From: selvajove at gmail.com (Selva Jove)
Date: Tue, 23 Feb 2021 14:44:08 +0530
Subject: [RFC PATCH v5 0/4] add simple copy support
In-Reply-To: <20210221235248.GZ4626@dread.disaster.area>
References: <CGME20210219124555epcas5p1334e7c4d64ada5dc4a2ca0feb48c1d44@epcas5p1.samsung.com>
 <20210219124517.79359-1-selvakuma.s1@samsung.com>
 <20210221235248.GZ4626@dread.disaster.area>
Message-ID: <CAHqX9vbG-cB0h25y4OhcdOEegn-_E=HwHJtkPFGaurEF9-KXPw@mail.gmail.com>

Dave,

copy_file_range() is work under progress.  FALLOC_FL_UNSHARE of fallocate()
use case sounds interesting. I will try to address both of them in the
next series.

Adding SCSI_XCOPY() support is not in the scope of this patchset. However
blkdev_issue_copy() interface is made generic so that it is possible to extend
to cross device XCOPY in future.


Thanks,
Selva


From Martin.George at netapp.com  Tue Feb 23 04:56:50 2021
From: Martin.George at netapp.com (George, Martin)
Date: Tue, 23 Feb 2021 09:56:50 +0000
Subject: [PATCH] fabrics: ensure zero kato for non-persistent controllers
In-Reply-To: <BYAPR04MB4965AD8096D8F313B2BFF74C86809@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210218095422.5696-1-marting@netapp.com>
 <cb89029d362a3cd61504012351f56731cec9c6fe.camel@netapp.com>
 <BYAPR04MB4965AD8096D8F313B2BFF74C86809@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <59b2ebd8d9c8940840172fbb64d7823ef9c69100.camel@netapp.com>

On Tue, 2021-02-23 at 06:06 +0000, Chaitanya Kulkarni wrote:
> On 2/22/21 21:47, George, Martin wrote:
> > Ping?
> > 
> > On Thu, 2021-02-18 at 15:24 +0530, Martin George wrote:
> > > Along with ensuring a non-zero kato is passed to a
> > > persistent discovery controller, also ensure a zero
> > > kato is passed to a non-persistent discovery controller.
> > > 
> > > Signed-off-by: Martin George <marting at netapp.com>
> > > ---
> > >  fabrics.c | 4 ++++
> > >  1 file changed, 4 insertions(+)
> > > 
> > > diff --git a/fabrics.c b/fabrics.c
> > > index 1880a5d..ed15e08 100644
> > > --- a/fabrics.c
> > > +++ b/fabrics.c
> > > @@ -1399,6 +1399,8 @@ static int discover_from_conf_file(const
> > > char
> > > *desc, char *argstr,
> > > 
> > >                 if (cfg.persistent && !cfg.keep_alive_tmo)
> > >                         cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
> > > +               else if (!cfg.persistent && (cfg.keep_alive_tmo >
> > > 0))
> > > +                       cfg.keep_alive_tmo = 0;
> > > 
> > >                 if (traddr_is_hostname(&cfg)) {
> > >                         ret = hostname2traddr(&cfg);
> > > @@ -1474,6 +1476,8 @@ int fabrics_discover(const char *desc, int
> > > argc, char **argv, bool connect)
> > >         } else {
> > >                 if (cfg.persistent && !cfg.keep_alive_tmo)
> > >                         cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
> > > +               else if (!cfg.persistent && (cfg.keep_alive_tmo >
> > > 0))
> > > +                       cfg.keep_alive_tmo = 0;
> > > 
> > >                 if (traddr_is_hostname(&cfg)) {
> > >                         ret = hostname2traddr(&cfg);
> Is that if else if ladder code is repeated ? if so can you please add
> a
> helper ?
> unless there is a reason for keeping the duplicate code.

Ok. Will write a helper function for this and resend the patch.

> Also can we remove () for (cfg.keep_alive_tmo > 0) condition ?

I'd prefer keeping the parentheses here to be on the safer side, unless
there is a strong argument against it.

-Martin

From hare at suse.de  Tue Feb 23 06:59:20 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 23 Feb 2021 12:59:20 +0100
Subject: [PATCH 0/2] nvme: fix regression with MD RAID
Message-ID: <20210223115922.104369-1-hare@suse.de>

Hi all,

ever since the implementation NVMe-oF does not work together with MD RAID.
MD RAID expects the device to return an I/O error on failure, and to remove
the block device if the underlying hardware is removed.
This is contrary to the implementation of NVMe-oF, which will keep on retrying
I/O while the controller is being reset, and will only remove the block device
once the last _user_ is gone.

These patches fixup this situation by adding a new sysfs attribute
'fail_if_no_path'. When this attribute is set we will return I/O errors
as soon as no paths are available anymore, and will remove the block device
once the last controller holding a path to the namespace is removed (ie after
all reconnect attempts for that controllers are exhausted).

This is a rework of the earlier path by Keith Busch ('nvme-mpath: delete disk
after last connection'). Kudos to him for suggesting this approach.

Hannes Reinecke (2):
  nvme: add 'fail_if_no_path' sysfs attribute
  nvme: delete disk when last path is gone

 drivers/nvme/host/core.c      |  6 +++++
 drivers/nvme/host/multipath.c | 46 ++++++++++++++++++++++++++++++++---
 drivers/nvme/host/nvme.h      | 19 +++++++++++++--
 3 files changed, 66 insertions(+), 5 deletions(-)

-- 
2.29.2



From hare at suse.de  Tue Feb 23 06:59:22 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 23 Feb 2021 12:59:22 +0100
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <20210223115922.104369-1-hare@suse.de>
References: <20210223115922.104369-1-hare@suse.de>
Message-ID: <20210223115922.104369-3-hare@suse.de>

The multipath code currently deletes the disk only after all references
to it are dropped rather than when the last path to that disk is lost.
This has been reported to cause problems with some use cases like MD RAID.

This patch implements an alternative behaviour of deleting the disk when
the last path is gone, ie the same behaviour as non-multipathed nvme
devices. The new behaviour will be selected with the 'fail_if_no_path'
attribute, as returning it's arguably the same functionality.

Suggested-by: Keith Busch <kbusch at kernel.org>
Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      |  1 +
 drivers/nvme/host/multipath.c |  3 ++-
 drivers/nvme/host/nvme.h      | 17 +++++++++++++++--
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 2fb3ecc0c53b..d717a6283d6e 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -542,6 +542,7 @@ static void nvme_free_ns_head(struct kref *ref)
 		container_of(ref, struct nvme_ns_head, ref);
 
 	nvme_mpath_remove_disk(head);
+	nvme_mpath_put_disk(head);
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
 	nvme_put_subsystem(head->subsys);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index d5773ea105b1..f995b8234622 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -724,6 +724,8 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
 
 void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 {
+	if (test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags))
+		return;
 	if (!head->disk)
 		return;
 	if (head->disk->flags & GENHD_FL_UP)
@@ -741,7 +743,6 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 		 */
 		head->disk->queue = NULL;
 	}
-	put_disk(head->disk);
 }
 
 int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 3d2513f8194d..e6efa085f08a 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -681,8 +681,12 @@ static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
 {
 	struct nvme_ns_head *head = ns->head;
 
-	if (head->disk && list_empty(&head->list))
-		kblockd_schedule_work(&head->requeue_work);
+	if (head->disk && list_empty(&head->list)) {
+		if (test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags))
+			nvme_mpath_remove_disk(head);
+		else
+			kblockd_schedule_work(&head->requeue_work);
+	}
 }
 
 static inline void nvme_trace_bio_complete(struct request *req)
@@ -693,6 +697,12 @@ static inline void nvme_trace_bio_complete(struct request *req)
 		trace_block_bio_complete(ns->head->disk->queue, req->bio);
 }
 
+static inline void nvme_mpath_put_disk(struct nvme_ns_head *head)
+{
+	if (head->disk)
+		put_disk(head->disk);
+}
+
 extern struct device_attribute dev_attr_ana_grpid;
 extern struct device_attribute dev_attr_ana_state;
 extern struct device_attribute dev_attr_fail_if_no_path;
@@ -731,6 +741,9 @@ static inline void nvme_mpath_add_disk(struct nvme_ns *ns,
 static inline void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 {
 }
+static inline void nvme_mpath_put_disk(struct nvme_ns_head *head)
+{
+}
 static inline bool nvme_mpath_clear_current_path(struct nvme_ns *ns)
 {
 	return false;
-- 
2.29.2



From hare at suse.de  Tue Feb 23 06:59:21 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 23 Feb 2021 12:59:21 +0100
Subject: [PATCH 1/2] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <20210223115922.104369-1-hare@suse.de>
References: <20210223115922.104369-1-hare@suse.de>
Message-ID: <20210223115922.104369-2-hare@suse.de>

In some setups like RAID or cluster we need to return an I/O error
once all paths are unavailable to allow the upper layers to start
their own error recovery (like redirecting I/O to other mirrors).
This patch adds a sysfs attribute 'fail_if_no_path' to allow the
admin to enable that behaviour instead of the current 'queue until
a path becomes available' policy.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      |  5 ++++
 drivers/nvme/host/multipath.c | 43 +++++++++++++++++++++++++++++++++--
 drivers/nvme/host/nvme.h      |  2 ++
 3 files changed, 48 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4de6a3a13575..2fb3ecc0c53b 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3464,6 +3464,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
+	&dev_attr_fail_if_no_path.attr,
 #endif
 	NULL,
 };
@@ -3494,6 +3495,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
 			return 0;
 	}
+	if (a == &dev_attr_fail_if_no_path.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
+			return 0;
+	}
 #endif
 	return a->mode;
 }
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 0696319adaf6..d5773ea105b1 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -283,10 +283,18 @@ static bool nvme_available_path(struct nvme_ns_head *head)
 			continue;
 		switch (ns->ctrl->state) {
 		case NVME_CTRL_LIVE:
+			if (!test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH,
+				      &head->flags))
+				return true;
+			if (ns->ana_state != NVME_ANA_INACCESSIBLE &&
+			    ns->ana_state != NVME_ANA_PERSISTENT_LOSS)
+				return true;
 		case NVME_CTRL_RESETTING:
-		case NVME_CTRL_CONNECTING:
 			/* fallthru */
-			return true;
+		case NVME_CTRL_CONNECTING:
+			if (!test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH,
+				      &head->flags))
+				return true;
 		default:
 			break;
 		}
@@ -641,6 +649,37 @@ static ssize_t ana_state_show(struct device *dev, struct device_attribute *attr,
 }
 DEVICE_ATTR_RO(ana_state);
 
+static ssize_t fail_if_no_path_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+
+	return sprintf(buf, "%d\n",
+		       test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags) ?
+		       1 : 0);
+}
+
+static ssize_t fail_if_no_path_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+	int fail_if_no_path, err;
+
+	err = kstrtoint(buf, 10, &fail_if_no_path);
+	if (err)
+		return -EINVAL;
+
+	if (fail_if_no_path <= 0)
+		clear_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
+	else
+		set_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags);
+	return count;
+}
+DEVICE_ATTR(fail_if_no_path, S_IRUGO | S_IWUSR,
+	fail_if_no_path_show, fail_if_no_path_store);
+
 static int nvme_lookup_ana_group_desc(struct nvme_ctrl *ctrl,
 		struct nvme_ana_group_desc *desc, void *data)
 {
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..3d2513f8194d 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -418,6 +418,7 @@ struct nvme_ns_head {
 	struct mutex		lock;
 	unsigned long		flags;
 #define NVME_NSHEAD_DISK_LIVE	0
+#define NVME_NSHEAD_FAIL_IF_NO_PATH	1
 	struct nvme_ns __rcu	*current_path[];
 #endif
 };
@@ -694,6 +695,7 @@ static inline void nvme_trace_bio_complete(struct request *req)
 
 extern struct device_attribute dev_attr_ana_grpid;
 extern struct device_attribute dev_attr_ana_state;
+extern struct device_attribute dev_attr_fail_if_no_path;
 extern struct device_attribute subsys_attr_iopolicy;
 
 #else
-- 
2.29.2



From hare at suse.de  Tue Feb 23 07:07:27 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 23 Feb 2021 13:07:27 +0100
Subject: [PATCH 1/2] nvme: fixup kato deadlock
In-Reply-To: <20210223120728.104699-1-hare@suse.de>
References: <20210223120728.104699-1-hare@suse.de>
Message-ID: <20210223120728.104699-2-hare@suse.de>

A customer of ours has run into this deadlock with RDMA:
- The ka_work workqueue item is executed
- A new ka_work workqueue item is scheduled just after that.
- Now both, the kato request timeout _and_ the workqueue delay
  will execute at roughly the same time
- If the timing is correct the workqueue executes _before_
  the kato request timeout triggers
- Kato request timeout triggers, and starts error recovery
- error recovery deadlocks, as it needs to flush the kato
  workqueue item; this is stuck in nvme_alloc_request() as all
  reserved tags are in use.

The reserved tags would have been freed up later when cancelling all
outstanding requests in the queue:

	nvme_stop_keep_alive(&ctrl->ctrl);
	nvme_rdma_teardown_io_queues(ctrl, false);
	nvme_start_queues(&ctrl->ctrl);
	nvme_rdma_teardown_admin_queue(ctrl, false);
	blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);

but as we're stuck in nvme_stop_keep_alive() we'll never get this far.

To fix this a new controller flag 'NVME_CTRL_KATO_RUNNING' is added
which will short-circuit the nvme_keep_alive() function if one
keep-alive command is already running.

Cc: Daniel Wagner <dwagner at suse.de>
Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c | 8 +++++++-
 drivers/nvme/host/nvme.h | 1 +
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ea40a3c511da..9b8596eb4047 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1211,6 +1211,7 @@ static void nvme_keep_alive_end_io(struct request *rq, blk_status_t status)
 	bool startka = false;
 
 	blk_mq_free_request(rq);
+	clear_bit(NVME_CTRL_KATO_RUNNING, &ctrl->flags);
 
 	if (status) {
 		dev_err(ctrl->device,
@@ -1233,10 +1234,15 @@ static int nvme_keep_alive(struct nvme_ctrl *ctrl)
 {
 	struct request *rq;
 
+	if (test_and_set_bit(NVME_CTRL_KATO_RUNNING, &ctrl->flags))
+		return 0;
+
 	rq = nvme_alloc_request(ctrl->admin_q, &ctrl->ka_cmd,
 			BLK_MQ_REQ_RESERVED);
-	if (IS_ERR(rq))
+	if (IS_ERR(rq)) {
+		clear_bit(NVME_CTRL_KATO_RUNNING, &ctrl->flags);
 		return PTR_ERR(rq);
+	}
 
 	rq->timeout = ctrl->kato * HZ;
 	rq->end_io_data = ctrl;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index e6efa085f08a..e00e3400c8b6 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -344,6 +344,7 @@ struct nvme_ctrl {
 	int nr_reconnects;
 	unsigned long flags;
 #define NVME_CTRL_FAILFAST_EXPIRED	0
+#define NVME_CTRL_KATO_RUNNING		1
 	struct nvmf_ctrl_options *opts;
 
 	struct page *discard_page;
-- 
2.29.2



From hare at suse.de  Tue Feb 23 07:07:26 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 23 Feb 2021 13:07:26 +0100
Subject: [PATCH 0/2] nvme: sanitize KATO handling
Message-ID: <20210223120728.104699-1-hare@suse.de>

Hi all,

one of our customer had been running into a deadlock trying to terminate
outstanding KATO commands during reset.
Looking closer at it, I found that we never actually _track_ if a KATO
command is submitted, so we might happily be sending several KATO commands
to the same controller simultaneously.
Also, I found it slightly odd that we signal a different KATO value to the
controller than what we're using internally; I would have thought that both
sides should agree on the same KATO value. And even that wouldn't be so
bad, but we really should be using the KATO value we annouonced to the
controller when setting the request timeout.

With these patches I attempt to resolve the situation; the first patch
ensures that only one KATO command to a given controller is outstanding.
With that the delay between sending KATO commands and the KATO timeout
are decoupled, and we can follow the recommendation from the base spec
to send the KATO commands at half the KATO timeout intervals.

As usual, comments and reviews are welcome.

Hannes Reinecke (2):
  nvme: fixup kato deadlock
  nvme: sanitize KATO setting

 drivers/nvme/host/core.c    | 22 +++++++++++++++++-----
 drivers/nvme/host/fabrics.c |  2 +-
 drivers/nvme/host/nvme.h    |  2 +-
 3 files changed, 19 insertions(+), 7 deletions(-)

-- 
2.29.2



From hare at suse.de  Tue Feb 23 07:07:28 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 23 Feb 2021 13:07:28 +0100
Subject: [PATCH 2/2] nvme: sanitize KATO setting
In-Reply-To: <20210223120728.104699-1-hare@suse.de>
References: <20210223120728.104699-1-hare@suse.de>
Message-ID: <20210223120728.104699-3-hare@suse.de>

According to the NVMe base spec the KATO commands should be sent
at half of the KATO interval, to properly account for round-trip
times.
As we now will only ever send one KATO command per connection we
can easily use the recommended values.
This also fixes a potential issue where the request timeout for
the KATO command does not match the value in the connect command,
which might be causing spurious connection drops from the target.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c    | 14 ++++++++++----
 drivers/nvme/host/fabrics.c |  2 +-
 drivers/nvme/host/nvme.h    |  1 -
 3 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 9b8596eb4047..eedbee80b7b9 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1226,8 +1226,11 @@ static void nvme_keep_alive_end_io(struct request *rq, blk_status_t status)
 	    ctrl->state == NVME_CTRL_CONNECTING)
 		startka = true;
 	spin_unlock_irqrestore(&ctrl->lock, flags);
-	if (startka)
-		queue_delayed_work(nvme_wq, &ctrl->ka_work, ctrl->kato * HZ);
+	if (startka) {
+		unsigned long kato_delay = ctrl->kato >> 1;
+
+		queue_delayed_work(nvme_wq, &ctrl->ka_work, kato_delay * HZ);
+	}
 }
 
 static int nvme_keep_alive(struct nvme_ctrl *ctrl)
@@ -1259,10 +1262,12 @@ static void nvme_keep_alive_work(struct work_struct *work)
 	bool comp_seen = ctrl->comp_seen;
 
 	if ((ctrl->ctratt & NVME_CTRL_ATTR_TBKAS) && comp_seen) {
+		unsigned long kato_delay = ctrl->kato >> 1;
+
 		dev_dbg(ctrl->device,
 			"reschedule traffic based keep-alive timer\n");
 		ctrl->comp_seen = false;
-		queue_delayed_work(nvme_wq, &ctrl->ka_work, ctrl->kato * HZ);
+		queue_delayed_work(nvme_wq, &ctrl->ka_work, kato_delay * HZ);
 		return;
 	}
 
@@ -1276,10 +1281,11 @@ static void nvme_keep_alive_work(struct work_struct *work)
 
 static void nvme_start_keep_alive(struct nvme_ctrl *ctrl)
 {
+	unsigned long kato_delay = ctrl->kato >> 1;
 	if (unlikely(ctrl->kato == 0))
 		return;
 
-	queue_delayed_work(nvme_wq, &ctrl->ka_work, ctrl->kato * HZ);
+	queue_delayed_work(nvme_wq, &ctrl->ka_work, kato_delay * HZ);
 }
 
 void nvme_stop_keep_alive(struct nvme_ctrl *ctrl)
diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index 5dfd806fc2d2..dba32e39afbf 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -382,7 +382,7 @@ int nvmf_connect_admin_queue(struct nvme_ctrl *ctrl)
 	 * and add a grace period for controller kato enforcement
 	 */
 	cmd.connect.kato = ctrl->kato ?
-		cpu_to_le32((ctrl->kato + NVME_KATO_GRACE) * 1000) : 0;
+		cpu_to_le32(ctrl->kato * 1000) : 0;
 
 	if (ctrl->opts->disable_sqflow)
 		cmd.connect.cattr |= NVME_CONNECT_DISABLE_SQFLOW;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index e00e3400c8b6..de0b270f95fc 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -27,7 +27,6 @@ extern unsigned int admin_timeout;
 #define NVME_ADMIN_TIMEOUT	(admin_timeout * HZ)
 
 #define NVME_DEFAULT_KATO	5
-#define NVME_KATO_GRACE		10
 
 #ifdef CONFIG_ARCH_NO_SG_CHAIN
 #define  NVME_INLINE_SG_CNT  0
-- 
2.29.2



From minwoo.im.dev at gmail.com  Tue Feb 23 07:41:09 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Tue, 23 Feb 2021 21:41:09 +0900
Subject: [PATCH 1/2] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <20210223115922.104369-2-hare@suse.de>
References: <20210223115922.104369-1-hare@suse.de>
 <20210223115922.104369-2-hare@suse.de>
Message-ID: <20210223124109.GB2119@localhost.localdomain>

On 21-02-23 12:59:21, Hannes Reinecke wrote:
> In some setups like RAID or cluster we need to return an I/O error
> once all paths are unavailable to allow the upper layers to start
> their own error recovery (like redirecting I/O to other mirrors).
> This patch adds a sysfs attribute 'fail_if_no_path' to allow the
> admin to enable that behaviour instead of the current 'queue until
> a path becomes available' policy.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c      |  5 ++++
>  drivers/nvme/host/multipath.c | 43 +++++++++++++++++++++++++++++++++--
>  drivers/nvme/host/nvme.h      |  2 ++
>  3 files changed, 48 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 4de6a3a13575..2fb3ecc0c53b 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -3464,6 +3464,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
>  #ifdef CONFIG_NVME_MULTIPATH
>  	&dev_attr_ana_grpid.attr,
>  	&dev_attr_ana_state.attr,
> +	&dev_attr_fail_if_no_path.attr,
>  #endif
>  	NULL,
>  };
> @@ -3494,6 +3495,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
>  		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
>  			return 0;
>  	}
> +	if (a == &dev_attr_fail_if_no_path.attr) {
> +		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
> +			return 0;
> +	}
>  #endif
>  	return a->mode;
>  }
> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> index 0696319adaf6..d5773ea105b1 100644
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -283,10 +283,18 @@ static bool nvme_available_path(struct nvme_ns_head *head)
>  			continue;
>  		switch (ns->ctrl->state) {
>  		case NVME_CTRL_LIVE:
> +			if (!test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH,
> +				      &head->flags))
> +				return true;
> +			if (ns->ana_state != NVME_ANA_INACCESSIBLE &&
> +			    ns->ana_state != NVME_ANA_PERSISTENT_LOSS)
> +				return true;

It looks like it needs to prevent fallthru here with return false.  It
causes following warning:

  drivers/nvme/host/multipath.c: In function ?nvme_available_path?:
  drivers/nvme/host/multipath.c:289:7: warning: this statement may fall through [-Wimplicit-fallthrough=]
    289 |    if (ns->ana_state != NVME_ANA_INACCESSIBLE &&
        |       ^
  drivers/nvme/host/multipath.c:292:3: note: here
    292 |   case NVME_CTRL_RESETTING:
        |   ^~~~


diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index f995b8234622..d692eab3d483 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -289,6 +289,7 @@ static bool nvme_available_path(struct nvme_ns_head *head)
                        if (ns->ana_state != NVME_ANA_INACCESSIBLE &&
                            ns->ana_state != NVME_ANA_PERSISTENT_LOSS)
                                return true;
+                       return false;
                case NVME_CTRL_RESETTING:


From minwoo.im.dev at gmail.com  Tue Feb 23 07:56:56 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Tue, 23 Feb 2021 21:56:56 +0900
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <20210223115922.104369-3-hare@suse.de>
References: <20210223115922.104369-1-hare@suse.de>
 <20210223115922.104369-3-hare@suse.de>
Message-ID: <20210223125656.GC2119@localhost.localdomain>

On 21-02-23 12:59:22, Hannes Reinecke wrote:
> The multipath code currently deletes the disk only after all references
> to it are dropped rather than when the last path to that disk is lost.
> This has been reported to cause problems with some use cases like MD RAID.
> 
> This patch implements an alternative behaviour of deleting the disk when
> the last path is gone, ie the same behaviour as non-multipathed nvme
> devices. The new behaviour will be selected with the 'fail_if_no_path'
> attribute, as returning it's arguably the same functionality.
> 
> Suggested-by: Keith Busch <kbusch at kernel.org>
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c      |  1 +
>  drivers/nvme/host/multipath.c |  3 ++-
>  drivers/nvme/host/nvme.h      | 17 +++++++++++++++--
>  3 files changed, 18 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 2fb3ecc0c53b..d717a6283d6e 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -542,6 +542,7 @@ static void nvme_free_ns_head(struct kref *ref)
>  		container_of(ref, struct nvme_ns_head, ref);
>  
>  	nvme_mpath_remove_disk(head);
> +	nvme_mpath_put_disk(head);
>  	ida_simple_remove(&head->subsys->ns_ida, head->instance);
>  	cleanup_srcu_struct(&head->srcu);
>  	nvme_put_subsystem(head->subsys);
> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
> index d5773ea105b1..f995b8234622 100644
> --- a/drivers/nvme/host/multipath.c
> +++ b/drivers/nvme/host/multipath.c
> @@ -724,6 +724,8 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
>  
>  void nvme_mpath_remove_disk(struct nvme_ns_head *head)
>  {
> +	if (test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags))
> +		return;
>  	if (!head->disk)
>  		return;
>  	if (head->disk->flags & GENHD_FL_UP)
> @@ -741,7 +743,6 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
>  		 */
>  		head->disk->queue = NULL;
>  	}
> -	put_disk(head->disk);
>  }
>  
>  int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 3d2513f8194d..e6efa085f08a 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -681,8 +681,12 @@ static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
>  {
>  	struct nvme_ns_head *head = ns->head;
>  
> -	if (head->disk && list_empty(&head->list))
> -		kblockd_schedule_work(&head->requeue_work);
> +	if (head->disk && list_empty(&head->list)) {
> +		if (test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags))
> +			nvme_mpath_remove_disk(head);

Does it need to call nvme_mpath_remove_disk here ?  It looks like it
returns with nothing right away if NVME_NSHEAD_FAIL_IF_NO_PATH is set.


From hare at suse.de  Tue Feb 23 09:07:20 2021
From: hare at suse.de (Hannes Reinecke)
Date: Tue, 23 Feb 2021 15:07:20 +0100
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <20210223125656.GC2119@localhost.localdomain>
References: <20210223115922.104369-1-hare@suse.de>
 <20210223115922.104369-3-hare@suse.de>
 <20210223125656.GC2119@localhost.localdomain>
Message-ID: <ebe8faf5-1caf-375a-b5b9-33520a801478@suse.de>

On 2/23/21 1:56 PM, Minwoo Im wrote:
> On 21-02-23 12:59:22, Hannes Reinecke wrote:
>> The multipath code currently deletes the disk only after all references
>> to it are dropped rather than when the last path to that disk is lost.
>> This has been reported to cause problems with some use cases like MD RAID.
>>
>> This patch implements an alternative behaviour of deleting the disk when
>> the last path is gone, ie the same behaviour as non-multipathed nvme
>> devices. The new behaviour will be selected with the 'fail_if_no_path'
>> attribute, as returning it's arguably the same functionality.
>>
>> Suggested-by: Keith Busch <kbusch at kernel.org>
>> Signed-off-by: Hannes Reinecke <hare at suse.de>
>> ---
>>  drivers/nvme/host/core.c      |  1 +
>>  drivers/nvme/host/multipath.c |  3 ++-
>>  drivers/nvme/host/nvme.h      | 17 +++++++++++++++--
>>  3 files changed, 18 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index 2fb3ecc0c53b..d717a6283d6e 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -542,6 +542,7 @@ static void nvme_free_ns_head(struct kref *ref)
>>  		container_of(ref, struct nvme_ns_head, ref);
>>  
>>  	nvme_mpath_remove_disk(head);
>> +	nvme_mpath_put_disk(head);
>>  	ida_simple_remove(&head->subsys->ns_ida, head->instance);
>>  	cleanup_srcu_struct(&head->srcu);
>>  	nvme_put_subsystem(head->subsys);
>> diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
>> index d5773ea105b1..f995b8234622 100644
>> --- a/drivers/nvme/host/multipath.c
>> +++ b/drivers/nvme/host/multipath.c
>> @@ -724,6 +724,8 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
>>  
>>  void nvme_mpath_remove_disk(struct nvme_ns_head *head)
>>  {
>> +	if (test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags))
>> +		return;
>>  	if (!head->disk)
>>  		return;
>>  	if (head->disk->flags & GENHD_FL_UP)
>> @@ -741,7 +743,6 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
>>  		 */
>>  		head->disk->queue = NULL;
>>  	}
>> -	put_disk(head->disk);
>>  }
>>  
>>  int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
>> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
>> index 3d2513f8194d..e6efa085f08a 100644
>> --- a/drivers/nvme/host/nvme.h
>> +++ b/drivers/nvme/host/nvme.h
>> @@ -681,8 +681,12 @@ static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
>>  {
>>  	struct nvme_ns_head *head = ns->head;
>>  
>> -	if (head->disk && list_empty(&head->list))
>> -		kblockd_schedule_work(&head->requeue_work);
>> +	if (head->disk && list_empty(&head->list)) {
>> +		if (test_bit(NVME_NSHEAD_FAIL_IF_NO_PATH, &head->flags))
>> +			nvme_mpath_remove_disk(head);
> 
> Does it need to call nvme_mpath_remove_disk here ?  It looks like it
> returns with nothing right away if NVME_NSHEAD_FAIL_IF_NO_PATH is set.
> 
Argl. Yes, you are correct.

I'll be reworking that one.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		           Kernel Storage Architect
hare at suse.de			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), GF: Felix Imend?rffer


From hkallweit1 at gmail.com  Tue Feb 23 09:36:22 2021
From: hkallweit1 at gmail.com (Heiner Kallweit)
Date: Tue, 23 Feb 2021 15:36:22 +0100
Subject: [PATCH 1/2] nvme-pci; remove the barriers nvme_irq()
In-Reply-To: <BYAPR04MB496513669571FA6541E58B8686809@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210219063726.63728-1-chaitanya.kulkarni@wdc.com>
 <80fc53d2-4f35-7c72-5cc5-3ec13c5ee51a@kernel.dk>
 <BYAPR04MB496513669571FA6541E58B8686809@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <f2d9cd44-8f24-a4ee-ca26-a1d2b8c74c3d@gmail.com>

On 23.02.2021 07:54, Chaitanya Kulkarni wrote:
> (+Heiner)
> 
> On 2/19/21 05:40, Jens Axboe wrote:
>> On 2/18/21 11:37 PM, Chaitanya Kulkarni wrote:
>>> The barriers were added to the nvme_irq() in the
>>> commit 3a7afd8ee42a ("nvme-pci: remove the CQ lock for interrupt driven queues")
>>> to prevent compiler from doing memory optimization for the variabes that
>>> were protected previously by spinlock in nvme_irq() at completion queue
>>> processing and with queue head check condition..
>>>
>>> The variable nvmeq->last_cq_head from those checks was removed in the
>>> commit f6c4d97b0d82 ("nvme/pci: Remove last_cq_head")") that was
>>> preventing  poll queues from mistakenly triggering the spurious
>>> interrupt detection.
>>>
>>> Remove the barriers which were protecting the updates to the variables.
>> Would seem prudent to just ask the original submitter to send a v2
>> of his patch, as that's where it came from.
>>
> Please let me know if you want to send V2 otherwise please ack this series.
> 

Fine with me if you send the patch. You could just add a Reported-by

Reviewed-by: Heiner Kallweit <hkallweit1 at gmail.com>


From chaitanya.kulkarni at wdc.com  Tue Feb 23 15:47:39 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 12:47:39 -0800
Subject: [PATCH V2 0/2] nvme-pci: remove the barriers in nvme_irq()
Message-ID: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>

Hi,

The first patch removes barrier from the nvme_irq() and second patch
does the code cleanup for the same.

Below is the test log for the fio verify.

-ck

Chaitanya Kulkarni (2):
  nvme-pci; remove the barriers nvme_irq()
  nvme-pci: cleanup nvme_irq()

 drivers/nvme/host/pci.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

# gitlog -2
157098245bab (HEAD -> nvme-5.12) nvme-pci: cleanup nvme_irq()
42d531edfe67 nvme-pci; remove the barriers nvme_irq()
# ./compile_nvme.sh 
+ umount /mnt/nvme0n1
+ clear_dmesg
umount: /mnt/nvme0n1: not mounted
+ modprobe -r nvme-fabrics
+ modprobe -r nvme_loop
+ modprobe -r nvmet
+ modprobe -r nvme
+ sleep 1
+ modprobe -r nvme-core
+ lsmod
+ grep nvme
+ git apply ./all-fixes.diff
+ git diff
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..cedf704adac6 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1167,7 +1167,8 @@ static int nvme_submit_user_cmd(struct request_queue *q,
                if (ret)
                        goto out;
                bio = req->bio;
-               bio_set_dev(bio, bdev);
+               if(bdev)
+                       bio_set_dev(bio, bdev);
                if (bdev && meta_buffer && meta_len) {
                        meta = nvme_add_user_metadata(bio, meta_buffer, meta_len,
                                        meta_seed, write);
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..38fffee6b85e 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -688,7 +688,8 @@ static struct nvmf_transport_ops nvme_loop_transport = {
        .name           = "loop",
        .module         = THIS_MODULE,
        .create_ctrl    = nvme_loop_create_ctrl,
-       .allowed_opts   = NVMF_OPT_TRADDR,
+       .allowed_opts   = NVMF_OPT_TRADDR | NVMF_OPT_CTRL_LOSS_TMO,
+
 };
 
 static int __init nvme_loop_init_module(void)
++ nproc
+ make -j 64 M=drivers/nvme/ modules
+ HOST=drivers/nvme/host
+ TARGET=drivers/nvme/target
++ uname -r
+ HOST_DEST=/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host/
++ uname -r
+ TARGET_DEST=/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target/
+ cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host//
+ cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target//
+ ls -lrth /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host/ /lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target//
/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/host/:
total 8.0M
-rw-r--r--. 1 root 3.0M Feb 23 12:30 nvme-core.ko
-rw-r--r--. 1 root 643K Feb 23 12:30 nvme-fabrics.ko
-rw-r--r--. 1 root 1.2M Feb 23 12:30 nvme-fc.ko
-rw-r--r--. 1 root 1.1M Feb 23 12:30 nvme.ko
-rw-r--r--. 1 root 1.2M Feb 23 12:30 nvme-rdma.ko
-rw-r--r--. 1 root 1.1M Feb 23 12:30 nvme-tcp.ko

/lib/modules/5.11.0-rc5nvme+/kernel/drivers/nvme/target//:
total 8.2M
-rw-r--r--. 1 root 712K Feb 23 12:30 nvme-fcloop.ko
-rw-r--r--. 1 root 619K Feb 23 12:30 nvme-loop.ko
-rw-r--r--. 1 root 1.1M Feb 23 12:30 nvmet-fc.ko
-rw-r--r--. 1 root 4.0M Feb 23 12:30 nvmet.ko
-rw-r--r--. 1 root 1.1M Feb 23 12:30 nvmet-rdma.ko
-rw-r--r--. 1 root 852K Feb 23 12:30 nvmet-tcp.ko
+ modprobe nvme
# 
# cat fio/verify.fio 
[write-and-verify]
rw=randwrite
bs=4k
direct=1
ioengine=libaio
iodepth=4
norandommap
randrepeat=1
verify=crc32c
size=950m
group_reporting
# fio fio/verify.fio --filename=/dev/nvme0n1 
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
fio-3.8-5-g464b
Starting 1 process
Jobs: 1 (f=1): [f(1)][100.0%][r=104MiB/s,w=0KiB/s][r=26.6k,w=0 IOPS][eta 00m:00s]      
write-and-verify: (groupid=0, jobs=1): err= 0: pid=37040: Tue Feb 23 12:31:35 2021
   read: IOPS=26.4k, BW=103MiB/s (108MB/s)(602MiB/5831msec)
    slat (usec): min=11, max=5211, avg=32.40, stdev=24.38
    clat (usec): min=6, max=6821, avg=116.81, stdev=37.38
     lat (usec): min=65, max=6838, avg=149.39, stdev=45.87
    clat percentiles (usec):
     |  1.00th=[   81],  5.00th=[   98], 10.00th=[  103], 20.00th=[  105],
     | 30.00th=[  108], 40.00th=[  111], 50.00th=[  113], 60.00th=[  115],
     | 70.00th=[  118], 80.00th=[  123], 90.00th=[  133], 95.00th=[  153],
     | 99.00th=[  219], 99.50th=[  243], 99.90th=[  302], 99.95th=[  338],
     | 99.99th=[  482]
  write: IOPS=24.2k, BW=94.4MiB/s (99.0MB/s)(950MiB/10062msec)
    slat (usec): min=14, max=4146, avg=37.36, stdev=22.06
    clat (usec): min=56, max=4261, avg=126.98, stdev=27.62
     lat (usec): min=84, max=4327, avg=164.52, stdev=35.50
    clat percentiles (usec):
     |  1.00th=[   84],  5.00th=[   95], 10.00th=[  108], 20.00th=[  116],
     | 30.00th=[  120], 40.00th=[  124], 50.00th=[  126], 60.00th=[  127],
     | 70.00th=[  129], 80.00th=[  135], 90.00th=[  143], 95.00th=[  157],
     | 99.00th=[  225], 99.50th=[  258], 99.90th=[  351], 99.95th=[  404],
     | 99.99th=[  486]
   bw (  KiB/s): min=12344, max=99056, per=95.83%, avg=92647.62, stdev=18427.63, samples=21
   iops        : min= 3086, max=24764, avg=23161.90, stdev=4606.91, samples=21
  lat (usec)   : 10=0.01%, 50=0.01%, 100=6.46%, 250=93.05%, 500=0.48%
  lat (usec)   : 750=0.01%
  lat (msec)   : 4=0.01%, 10=0.01%
  cpu          : usr=12.95%, sys=86.81%, ctx=174, majf=0, minf=3626
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=154025,243200,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
   READ: bw=103MiB/s (108MB/s), 103MiB/s-103MiB/s (108MB/s-108MB/s), io=602MiB (631MB), run=5831-5831msec
  WRITE: bw=94.4MiB/s (99.0MB/s), 94.4MiB/s-94.4MiB/s (99.0MB/s-99.0MB/s), io=950MiB (996MB), run=10062-10062msec

Disk stats (read/write):
  nvme0n1: ios=154119/243200, merge=0/0, ticks=8957/14742, in_queue=23699, util=99.49%
# modprobe -r nvme 
# 

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 23 15:47:40 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 12:47:40 -0800
Subject: [PATCH V2 1/2] nvme-pci; remove the barriers nvme_irq()
In-Reply-To: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>
References: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210223204741.38301-2-chaitanya.kulkarni@wdc.com>

The barriers were added to the nvme_irq() in the
commit 3a7afd8ee42a ("nvme-pci: remove the CQ lock for interrupt driven queues")
to prevent compiler from doing memory optimization for the variabes that
were protected previously by spinlock in nvme_irq() at completion queue
processing and with queue head check condition..

The variable nvmeq->last_cq_head from those checks was removed in the
commit f6c4d97b0d82 ("nvme/pci: Remove last_cq_head") that was not
allwing poll queues from mistakenly triggering the spurious interrupt
detection.

Remove the barriers which were protecting the updates to the variables.

Reported-by: Heiner Kallweit <hkallweit1 at gmail.com>
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Reviewed-by: Heiner Kallweit <hkallweit1 at gmail.com>
---
 drivers/nvme/host/pci.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 0045c5edf629..3729775f6a8a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1062,14 +1062,8 @@ static irqreturn_t nvme_irq(int irq, void *data)
 	struct nvme_queue *nvmeq = data;
 	irqreturn_t ret = IRQ_NONE;
 
-	/*
-	 * The rmb/wmb pair ensures we see all updates from a previous run of
-	 * the irq handler, even if that was on another CPU.
-	 */
-	rmb();
 	if (nvme_process_cq(nvmeq))
 		ret = IRQ_HANDLED;
-	wmb();
 
 	return ret;
 }
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Tue Feb 23 15:47:41 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 12:47:41 -0800
Subject: [PATCH V2 2/2] nvme-pci: cleanup nvme_irq()
In-Reply-To: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>
References: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210223204741.38301-3-chaitanya.kulkarni@wdc.com>

Get rid of the local variable that is not needed & just return.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/pci.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 3729775f6a8a..2d07bdc76d0b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1060,12 +1060,10 @@ static inline int nvme_process_cq(struct nvme_queue *nvmeq)
 static irqreturn_t nvme_irq(int irq, void *data)
 {
 	struct nvme_queue *nvmeq = data;
-	irqreturn_t ret = IRQ_NONE;
 
 	if (nvme_process_cq(nvmeq))
-		ret = IRQ_HANDLED;
-
-	return ret;
+		return IRQ_HANDLED;
+	return IRQ_NONE;
 }
 
 static irqreturn_t nvme_irq_check(int irq, void *data)
-- 
2.22.1



From Chaitanya.Kulkarni at wdc.com  Tue Feb 23 15:49:15 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 20:49:15 +0000
Subject: [PATCH 1/2] nvme-pci; remove the barriers nvme_irq()
References: <20210219063726.63728-1-chaitanya.kulkarni@wdc.com>
 <80fc53d2-4f35-7c72-5cc5-3ec13c5ee51a@kernel.dk>
 <BYAPR04MB496513669571FA6541E58B8686809@BYAPR04MB4965.namprd04.prod.outlook.com>
 <f2d9cd44-8f24-a4ee-ca26-a1d2b8c74c3d@gmail.com>
Message-ID: <BYAPR04MB49657DA30241D7497D9981B586809@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/23/21 06:36, Heiner Kallweit wrote:
>> Please let me know if you want to send V2 otherwise please ack this series.
>>
> Fine with me if you send the patch. You could just add a Reported-by
>
> Reviewed-by: Heiner Kallweit <hkallweit1 at gmail.com>
>
Done, please let know if you find any issue with V2.


From Chaitanya.Kulkarni at wdc.com  Tue Feb 23 17:03:09 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Tue, 23 Feb 2021 22:03:09 +0000
Subject: [PATCH] nvme-rdma: fix crash for no IO queues
References: <20210223072602.3196-1-lengchao@huawei.com>
Message-ID: <BYAPR04MB49657E5EB6A8F9C8C07C736386809@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/22/21 23:30, Chao Leng wrote:
> A crash happens when set feature(NVME_FEAT_NUM_QUEUES) timeout in nvme
> over rdma(roce) reconnection, the reason is use the queue which is not
> alloced.
>
> If it is not discovery and no io queues, the connection should fail.
>
> Signed-off-by: Chao Leng <lengchao at huawei.com>

Can you please share more information about

"when set feature(NVME_FEAT_NUM_QUEUES) timeout" scenario ?



From pterjan at google.com  Tue Feb 23 17:10:46 2021
From: pterjan at google.com (Pascal Terjan)
Date: Tue, 23 Feb 2021 22:10:46 +0000
Subject: [PATCH] nvme: Add quirks for Lexar 256GB SSD
Message-ID: <20210223221046.3367795-1-pterjan@google.com>

Add NVME_QUIRK_NO_NS_DESC_LIST to Lexar 256GB SSD to allow it to
work again, and add NVME_QUIRK_IGNORE_DEV_SUBNQN to avoid a warning.

[    0.637476] nvme nvme0: missing or invalid SUBNQN field.
[    0.638399] nvme nvme0: Identify Descriptors failed (16386)

Reported and tested in https://bugs.mageia.org/show_bug.cgi?id=28417

Signed-off-by: Pascal Terjan <pterjan at google.com>
---
 drivers/nvme/host/pci.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 7b6632c00ffd..9a56ec70e93a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3232,6 +3232,9 @@ static const struct pci_device_id nvme_id_table[] = {
 				NVME_QUIRK_DISABLE_WRITE_ZEROES, },
 	{ PCI_DEVICE(0x126f, 0x2263),	/* Silicon Motion unidentified */
 		.driver_data = NVME_QUIRK_NO_NS_DESC_LIST, },
+	{ PCI_DEVICE(0x1b4b, 0x1092),	/* Lexar 256 GB SSD */
+		.driver_data = NVME_QUIRK_NO_NS_DESC_LIST |
+				NVME_QUIRK_IGNORE_DEV_SUBNQN, },
 	{ PCI_DEVICE(0x1bb1, 0x0100),   /* Seagate Nytro Flash Storage */
 		.driver_data = NVME_QUIRK_DELAY_BEFORE_CHK_RDY, },
 	{ PCI_DEVICE(0x1c58, 0x0003),	/* HGST adapter */
-- 
2.30.0.617.g56c4b15f3c-goog



From kbusch at kernel.org  Tue Feb 23 18:21:37 2021
From: kbusch at kernel.org (Keith Busch)
Date: Wed, 24 Feb 2021 08:21:37 +0900
Subject: [PATCH] nvme-rdma: fix crash for no IO queues
In-Reply-To: <20210223072602.3196-1-lengchao@huawei.com>
References: <20210223072602.3196-1-lengchao@huawei.com>
Message-ID: <20210223232137.GA23682@redsun51.ssa.fujisawa.hgst.com>

On Tue, Feb 23, 2021 at 03:26:02PM +0800, Chao Leng wrote:
> A crash happens when set feature(NVME_FEAT_NUM_QUEUES) timeout in nvme
> over rdma(roce) reconnection, the reason is use the queue which is not
> alloced.
> 
> If it is not discovery and no io queues, the connection should fail.

If you're getting a timeout, we need to quit initialization. Hannes
attempted making that status visible for fabrics here:

  http://lists.infradead.org/pipermail/linux-nvme/2021-January/022353.html

There's still some corner cases that need handling, though.


From lengchao at huawei.com  Wed Feb 24 00:52:20 2021
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 24 Feb 2021 13:52:20 +0800
Subject: [PATCH] nvme-rdma: fix crash for no IO queues
In-Reply-To: <BYAPR04MB49657E5EB6A8F9C8C07C736386809@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210223072602.3196-1-lengchao@huawei.com>
 <BYAPR04MB49657E5EB6A8F9C8C07C736386809@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <1ac356d5-f154-699e-e929-e9f351b7da07@huawei.com>



On 2021/2/24 6:03, Chaitanya Kulkarni wrote:
> On 2/22/21 23:30, Chao Leng wrote:
>> A crash happens when set feature(NVME_FEAT_NUM_QUEUES) timeout in nvme
>> over rdma(roce) reconnection, the reason is use the queue which is not
>> alloced.
>>
>> If it is not discovery and no io queues, the connection should fail.
>>
>> Signed-off-by: Chao Leng <lengchao at huawei.com>
> 
> Can you please share more information about
> 
> "when set feature(NVME_FEAT_NUM_QUEUES) timeout" scenario ?
Inject a large number of bit errors intermittently. This will cause
request time out, and then reconnection will trigger by error recovery.
The requests of reconnect may also time out. If set feature
(NVME_FEAT_NUM_QUEUES, which is called by nvme_set_queue_count) time out
and the reconnection may success. Block will continue send request,
and then crash due to use unallocated queue.



From lengchao at huawei.com  Wed Feb 24 00:59:55 2021
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 24 Feb 2021 13:59:55 +0800
Subject: [PATCH] nvme-rdma: fix crash for no IO queues
In-Reply-To: <20210223232137.GA23682@redsun51.ssa.fujisawa.hgst.com>
References: <20210223072602.3196-1-lengchao@huawei.com>
 <20210223232137.GA23682@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <7c30ddb7-ee41-9972-4cb6-0a5c089f687c@huawei.com>



On 2021/2/24 7:21, Keith Busch wrote:
> On Tue, Feb 23, 2021 at 03:26:02PM +0800, Chao Leng wrote:
>> A crash happens when set feature(NVME_FEAT_NUM_QUEUES) timeout in nvme
>> over rdma(roce) reconnection, the reason is use the queue which is not
>> alloced.
>>
>> If it is not discovery and no io queues, the connection should fail.
> 
> If you're getting a timeout, we need to quit initialization. Hannes
> attempted making that status visible for fabrics here:
> 
>    http://lists.infradead.org/pipermail/linux-nvme/2021-January/022353.html
I know the patch. It can not solve the scenario: target may be an
attacker or the target behavior is incorrect.
If target return 0 io queues or return other error code, the crash will
still happen. We should not allow this to happen.
> 
> There's still some corner cases that need handling, though.
> .
> 


From lengchao at huawei.com  Wed Feb 24 01:42:11 2021
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 24 Feb 2021 14:42:11 +0800
Subject: [PATCH 0/2] nvme: sanitize KATO handling
In-Reply-To: <20210223120728.104699-1-hare@suse.de>
References: <20210223120728.104699-1-hare@suse.de>
Message-ID: <36e3bfd8-0c69-7540-c9f2-fd37070dc107@huawei.com>



On 2021/2/23 20:07, Hannes Reinecke wrote:
> Hi all,
> 
> one of our customer had been running into a deadlock trying to terminate
> outstanding KATO commands during reset.
> Looking closer at it, I found that we never actually _track_ if a KATO
> command is submitted, so we might happily be sending several KATO commands
> to the same controller simultaneously.
Can you explain how can send KATO commands simultaneously?
> Also, I found it slightly odd that we signal a different KATO value to the
> controller than what we're using internally; I would have thought that both
> sides should agree on the same KATO value. And even that wouldn't be so
> bad, but we really should be using the KATO value we annouonced to the
> controller when setting the request timeout.
> 
> With these patches I attempt to resolve the situation; the first patch
> ensures that only one KATO command to a given controller is outstanding.
> With that the delay between sending KATO commands and the KATO timeout
> are decoupled, and we can follow the recommendation from the base spec
> to send the KATO commands at half the KATO timeout intervals.
> 
> As usual, comments and reviews are welcome.
> 
> Hannes Reinecke (2):
>    nvme: fixup kato deadlock
>    nvme: sanitize KATO setting
> 
>   drivers/nvme/host/core.c    | 22 +++++++++++++++++-----
>   drivers/nvme/host/fabrics.c |  2 +-
>   drivers/nvme/host/nvme.h    |  2 +-
>   3 files changed, 19 insertions(+), 7 deletions(-)
> 


From hare at suse.de  Wed Feb 24 02:06:58 2021
From: hare at suse.de (Hannes Reinecke)
Date: Wed, 24 Feb 2021 08:06:58 +0100
Subject: [PATCH 0/2] nvme: sanitize KATO handling
In-Reply-To: <36e3bfd8-0c69-7540-c9f2-fd37070dc107@huawei.com>
References: <20210223120728.104699-1-hare@suse.de>
 <36e3bfd8-0c69-7540-c9f2-fd37070dc107@huawei.com>
Message-ID: <06e3b90a-719e-9561-5c63-67dac0a782e1@suse.de>

On 2/24/21 7:42 AM, Chao Leng wrote:
> 
> 
> On 2021/2/23 20:07, Hannes Reinecke wrote:
>> Hi all,
>>
>> one of our customer had been running into a deadlock trying to terminate
>> outstanding KATO commands during reset.
>> Looking closer at it, I found that we never actually _track_ if a KATO
>> command is submitted, so we might happily be sending several KATO 
>> commands
>> to the same controller simultaneously.
> Can you explain how can send KATO commands simultaneously?

Sure.
Call nvme_start_keep_alive() on a dead connection.
Just _after_ the KATO request has been sent,
call nvme_start_keep_alive() again.

You now have an expired KATO command, and the new KATO command, both are 
active and sent to the controller.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From lengchao at huawei.com  Wed Feb 24 02:20:57 2021
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 24 Feb 2021 15:20:57 +0800
Subject: [PATCH 0/2] nvme: sanitize KATO handling
In-Reply-To: <06e3b90a-719e-9561-5c63-67dac0a782e1@suse.de>
References: <20210223120728.104699-1-hare@suse.de>
 <36e3bfd8-0c69-7540-c9f2-fd37070dc107@huawei.com>
 <06e3b90a-719e-9561-5c63-67dac0a782e1@suse.de>
Message-ID: <1834a11c-c472-e14f-ece3-a96ec2ced044@huawei.com>



On 2021/2/24 15:06, Hannes Reinecke wrote:
> On 2/24/21 7:42 AM, Chao Leng wrote:
>>
>>
>> On 2021/2/23 20:07, Hannes Reinecke wrote:
>>> Hi all,
>>>
>>> one of our customer had been running into a deadlock trying to terminate
>>> outstanding KATO commands during reset.
>>> Looking closer at it, I found that we never actually _track_ if a KATO
>>> command is submitted, so we might happily be sending several KATO commands
>>> to the same controller simultaneously.
>> Can you explain how can send KATO commands simultaneously?
> 
> Sure.
> Call nvme_start_keep_alive() on a dead connection.
> Just _after_ the KATO request has been sent,
> call nvme_start_keep_alive() again.
Call nvme_start_keep_alive() again? why?
Now just nvme_start_ctrl call nvme_start_keep_alive().
The ka_work will be canceled sync before start reconnection.
Did I miss something?
> 
> You now have an expired KATO command, and the new KATO command, both are active and sent to the controller.
> 
> Cheers,
> 
> Hannes


From Chaitanya.Kulkarni at wdc.com  Wed Feb 24 02:21:17 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 07:21:17 +0000
Subject: [PATCH] nvme: Add quirks for Lexar 256GB SSD
References: <20210223221046.3367795-1-pterjan@google.com>
Message-ID: <BYAPR04MB4965BFBA4BF10F3468348768869F9@BYAPR04MB4965.namprd04.prod.outlook.com>

Pascal,

On 2/23/21 14:15, Pascal Terjan wrote:
> Add NVME_QUIRK_NO_NS_DESC_LIST to Lexar 256GB SSD to allow it to
> work again, and add NVME_QUIRK_IGNORE_DEV_SUBNQN to avoid a warning.
>
> [    0.637476] nvme nvme0: missing or invalid SUBNQN field.
> [    0.638399] nvme nvme0: Identify Descriptors failed (16386)
>
> Reported and tested in https://bugs.mageia.org/show_bug.cgi?id=28417
>
> Signed-off-by: Pascal Terjan <pterjan at google.com>

Do we need to add the Tested-by: tag ? if this is tested by other than the
patch author ?


From lengchao at huawei.com  Wed Feb 24 02:27:35 2021
From: lengchao at huawei.com (Chao Leng)
Date: Wed, 24 Feb 2021 15:27:35 +0800
Subject: [PATCH 0/2] nvme: sanitize KATO handling
In-Reply-To: <1834a11c-c472-e14f-ece3-a96ec2ced044@huawei.com>
References: <20210223120728.104699-1-hare@suse.de>
 <36e3bfd8-0c69-7540-c9f2-fd37070dc107@huawei.com>
 <06e3b90a-719e-9561-5c63-67dac0a782e1@suse.de>
 <1834a11c-c472-e14f-ece3-a96ec2ced044@huawei.com>
Message-ID: <da5f1559-16da-377a-1394-09419f6a1fd7@huawei.com>



On 2021/2/24 15:20, Chao Leng wrote:
> 
> 
> On 2021/2/24 15:06, Hannes Reinecke wrote:
>> On 2/24/21 7:42 AM, Chao Leng wrote:
>>>
>>>
>>> On 2021/2/23 20:07, Hannes Reinecke wrote:
>>>> Hi all,
>>>>
>>>> one of our customer had been running into a deadlock trying to terminate
>>>> outstanding KATO commands during reset.
>>>> Looking closer at it, I found that we never actually _track_ if a KATO
>>>> command is submitted, so we might happily be sending several KATO commands
>>>> to the same controller simultaneously.
>>> Can you explain how can send KATO commands simultaneously?
>>
>> Sure.
>> Call nvme_start_keep_alive() on a dead connection.
>> Just _after_ the KATO request has been sent,
>> call nvme_start_keep_alive() again.
> Call nvme_start_keep_alive() again? why?
> Now just nvme_start_ctrl call nvme_start_keep_alive().
> The ka_work will be canceled sync before start reconnection.
> Did I miss something?And all flight request will be canceled before start reconnection.
Is the request hang on block?
>>
>> You now have an expired KATO command, and the new KATO command, both are active and sent to the controller.
>>
>> Cheers,
>>
>> Hannes
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme
> .


From hare at suse.de  Wed Feb 24 02:59:08 2021
From: hare at suse.de (Hannes Reinecke)
Date: Wed, 24 Feb 2021 08:59:08 +0100
Subject: [PATCH 0/2] nvme: sanitize KATO handling
In-Reply-To: <1834a11c-c472-e14f-ece3-a96ec2ced044@huawei.com>
References: <20210223120728.104699-1-hare@suse.de>
 <36e3bfd8-0c69-7540-c9f2-fd37070dc107@huawei.com>
 <06e3b90a-719e-9561-5c63-67dac0a782e1@suse.de>
 <1834a11c-c472-e14f-ece3-a96ec2ced044@huawei.com>
Message-ID: <aef64b6f-0e12-1f17-7eda-ec3a9db9cf1e@suse.de>

On 2/24/21 8:20 AM, Chao Leng wrote:
> 
> 
> On 2021/2/24 15:06, Hannes Reinecke wrote:
>> On 2/24/21 7:42 AM, Chao Leng wrote:
>>>
>>>
>>> On 2021/2/23 20:07, Hannes Reinecke wrote:
>>>> Hi all,
>>>>
>>>> one of our customer had been running into a deadlock trying to 
>>>> terminate
>>>> outstanding KATO commands during reset.
>>>> Looking closer at it, I found that we never actually _track_ if a KATO
>>>> command is submitted, so we might happily be sending several KATO 
>>>> commands
>>>> to the same controller simultaneously.
>>> Can you explain how can send KATO commands simultaneously?
>>
>> Sure.
>> Call nvme_start_keep_alive() on a dead connection.
>> Just _after_ the KATO request has been sent,
>> call nvme_start_keep_alive() again.
> Call nvme_start_keep_alive() again? why?
> Now just nvme_start_ctrl call nvme_start_keep_alive().
> The ka_work will be canceled sync before start reconnection.
> Did I miss something?

My point was that there _can_ be a ka_work() entry even when a KATO 
command is running.

And yes, the ka_work entry will be cancelled, but _before_ the 
outstanding commands are cancelled.
And cancelling the ka_work entry might cause the function to be 
executed, which leads to a deadlock if blk_mq_get_request() is blocked
(eg if the queue is already stopped due to recovery)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From minwoo.im.dev at gmail.com  Wed Feb 24 03:49:28 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Wed, 24 Feb 2021 17:49:28 +0900
Subject: [PATCH V2 1/2] nvme-pci; remove the barriers nvme_irq()
In-Reply-To: <20210223204741.38301-2-chaitanya.kulkarni@wdc.com>
References: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>
 <20210223204741.38301-2-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224084928.GC2166@localhost.localdomain>

If and when it's picked up, the title of this patch should be with:

	s/nvme-pci;/nvme-pci:/


From hch at lst.de  Wed Feb 24 04:08:14 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 10:08:14 +0100
Subject: [PATCH 2/2] nvme: don't set iosqes,iocqes for discovery
 controllers
In-Reply-To: <a5ea738a-44c4-e953-3271-a0b97668000e@grimberg.me>
References: <20210209101235.1539917-1-sagi@grimberg.me>
 <20210209101235.1539917-2-sagi@grimberg.me> <20210210101126.GA26624@lst.de>
 <3702938a-cfd5-b725-bb75-7c9072a42fde@grimberg.me>
 <20210211070250.GA13708@lst.de>
 <f279bd6a-d4e3-52b6-740b-66f90434cf69@grimberg.me>
 <20210211183124.GA26045@lst.de>
 <a5ea738a-44c4-e953-3271-a0b97668000e@grimberg.me>
Message-ID: <20210224090814.GA4721@lst.de>

On Thu, Feb 11, 2021 at 11:44:44AM -0800, Sagi Grimberg wrote:
> So you suggest that we keep the retry w.a. but continue to send the
> wrong value in 1.3 or lower?
>
> I personally think that this would be extremely confusing to users, if
> they do trip on this, but I agree that they are less likely to hit
> it because existing controllers obviously accept wrong values...

Yes.  While I'd like to move to standards compliance as much as possible,
introducing a long delay for every existing working setup is 
very counter productive.

>
> As for option to override. I really hate it.. What is the least exposure
> we can get for this but still allow the user to control this?

We can of course still override it using an option.

> Wish we had a good and reliable quirk mechanism for fabrics...

We do, using the serial number.  But that does not help for an issue
during connection establishment.


From hch at infradead.org  Wed Feb 24 04:12:51 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2021 09:12:51 +0000
Subject: [PATCH] nvme-pci: mark Seagate Nytro XM1440 as
 QUIRK_NO_NS_DESC_LIST.
In-Reply-To: <20210216122543.21509-1-jeinwag-nvme@marcapo.com>
References: <20210216122543.21509-1-jeinwag-nvme@marcapo.com>
Message-ID: <20210224091251.GA2194345@infradead.org>

Thanks,

applied to nvme-5.12 with the slight commit message tweak.


From hch at lst.de  Wed Feb 24 04:13:26 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 10:13:26 +0100
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write
 Zeroes command
In-Reply-To: <BYAPR04MB4965C95C3F826D7222EB3176868C9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <YCOxjZhBTBchOq0n@infradead.org>
 <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
 <20210210131557.GB29904@lst.de>
 <BYAPR04MB49650A271CD1C1B3EFA72F3D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210211070748.GA13852@lst.de>
 <BYAPR04MB4965C95C3F826D7222EB3176868C9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <20210224091326.GA5159@lst.de>

On Thu, Feb 11, 2021 at 06:11:19PM +0000, Chaitanya Kulkarni wrote:
> On 2/10/21 11:07 PM, Christoph Hellwig wrote:
> >>> I'd also love to retest most Write Zeroes quirks with that in place.
> >> Do you prefer some variant of the following patch (totally untested)? OR
> >> something else ?
> > Somwhat.  As said I suspect defaulting to MDTS with a big fat comment
> > might make most sense unless the new WZSL is set.
> >
> Okay, will send out patche(s) soon.

Did you get to this?  Or did I just miss it?


From hch at lst.de  Wed Feb 24 04:14:14 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 10:14:14 +0100
Subject: [PATCH 0/2] nvme: resync nvme-cli and align values
In-Reply-To: <20210212071305.39901-1-chaitanya.kulkarni@wdc.com>
References: <20210212071305.39901-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224091414.GA5502@lst.de>

Thanks,

applied to nvme-5.13.


From hch at lst.de  Wed Feb 24 04:14:32 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 10:14:32 +0100
Subject: [PATCH] nvme-fabrics: fix kato initialization
In-Reply-To: <20210211175826.4821-1-marting@netapp.com>
References: <20210211175826.4821-1-marting@netapp.com>
Message-ID: <20210224091432.GA5517@lst.de>

Thanks,

applied to nvme-5.12.


From hch at lst.de  Wed Feb 24 04:14:42 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 10:14:42 +0100
Subject: [PATCH] nvme/hwmon: Return error code when registration fails
In-Reply-To: <20210212093015.2846-1-dwagner@suse.de>
References: <20210212093015.2846-1-dwagner@suse.de>
Message-ID: <20210224091442.GB5517@lst.de>

Thanks,

applied to nvme-5.12.


From hch at lst.de  Wed Feb 24 04:16:08 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 10:16:08 +0100
Subject: [PATCH V2 0/2] nvme-pci: remove the barriers in nvme_irq()
In-Reply-To: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>
References: <20210223204741.38301-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224091608.GA5569@lst.de>

Thanks,

applied to nvme-5.13.


From hch at infradead.org  Wed Feb 24 04:19:17 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2021 09:19:17 +0000
Subject: [PATCH] nvme: Apply the same fix Kingston SKC2000 nVME SSD as
 A2000
In-Reply-To: <20210221051216.3398620-1-zboszor@pr.hu>
References: <20210221051216.3398620-1-zboszor@pr.hu>
Message-ID: <20210224091917.GA2194565@infradead.org>

Thanks,

applied to nvme-5.12 with a slight tweak to the commit message.


From hch at infradead.org  Wed Feb 24 04:21:47 2021
From: hch at infradead.org (Christoph Hellwig)
Date: Wed, 24 Feb 2021 09:21:47 +0000
Subject: [PATCH] nvme: Add quirks for Lexar 256GB SSD
In-Reply-To: <20210223221046.3367795-1-pterjan@google.com>
References: <20210223221046.3367795-1-pterjan@google.com>
Message-ID: <20210224092147.GA2195892@infradead.org>

On Tue, Feb 23, 2021 at 10:10:46PM +0000, Pascal Terjan wrote:
> Add NVME_QUIRK_NO_NS_DESC_LIST to Lexar 256GB SSD to allow it to
> work again, and add NVME_QUIRK_IGNORE_DEV_SUBNQN to avoid a warning.

Why again?  The drive has always been broken since the desc list has
been a mandatory feature since NVMe 1.3..


From hch at lst.de  Wed Feb 24 04:43:47 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 10:43:47 +0100
Subject: [PATCH 1/1] nvmet: allow setting model_number once
In-Reply-To: <20210217181153.15343-1-mgurtovoy@nvidia.com>
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
Message-ID: <20210224094347.GA5965@lst.de>

So looking at this and Chaitanya patch I think this version simplifies
things quite nicely, and it also happens to get rid of the RCU annotation
sparse warnings.

Let me know what you think of the incremental cleanup and micro-opimization
below, though:


diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 3da285e22e9209..2ca13dd7e7b88a 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -313,34 +313,40 @@ static void nvmet_execute_get_log_page(struct nvmet_req *req)
 	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_SC_DNR);
 }
 
-static int nvmet_id_set_model_number(struct nvme_id_ctrl *id,
-				      struct nvmet_subsys *subsys)
+static u16 nvmet_set_model_number(struct nvme_id_ctrl *id,
+		struct nvmet_subsys *subsys)
 {
-	int ret = 0;
+	u16 status = 0;
 
 	mutex_lock(&subsys->lock);
 	if (!subsys->model_number) {
-		subsys->model_number = kstrdup(NVMET_DEFAULT_CTRL_MODEL,
-					       GFP_KERNEL);
-		if (!subsys->model_number) {
-			ret = -ENOMEM;
-			goto out_unlock;
-		}
+		subsys->model_number =
+			kstrdup(NVMET_DEFAULT_CTRL_MODEL, GFP_KERNEL);
+		if (!subsys->model_number)
+			status = NVME_SC_INTERNAL;
 	}
-	memcpy_and_pad(id->mn, sizeof(id->mn), subsys->model_number,
-		       strlen(subsys->model_number), ' ');
-out_unlock:
 	mutex_unlock(&subsys->lock);
-	return ret;
+
+	return status;
 }
 
 static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 {
 	struct nvmet_ctrl *ctrl = req->sq->ctrl;
+	struct nvmet_subsys *subsys = ctrl->subsys;
 	struct nvme_id_ctrl *id;
 	u32 cmd_capsule_size;
 	u16 status = 0;
-	int ret;
+
+	/*
+	 * If there is no model number yet, so it now.  It will then remain
+	 * stable for the life time of the subsystem.
+	 */
+	if (!subsys->model_number) {
+		status = nvmet_set_model_number(id, subsys);
+		if (status)
+			goto out;
+	}
 
 	id = kzalloc(sizeof(*id), GFP_KERNEL);
 	if (!id) {
@@ -355,11 +361,8 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 	memset(id->sn, ' ', sizeof(id->sn));
 	bin2hex(id->sn, &ctrl->subsys->serial,
 		min(sizeof(ctrl->subsys->serial), sizeof(id->sn) / 2));
-	ret = nvmet_id_set_model_number(id, ctrl->subsys);
-	if (ret) {
-		status = NVME_SC_INTERNAL;
-		goto out_free;
-	}
+	memcpy_and_pad(id->mn, sizeof(id->mn), subsys->model_number,
+		       strlen(subsys->model_number), ' ');
 	memcpy_and_pad(id->fr, sizeof(id->fr),
 		       UTS_RELEASE, strlen(UTS_RELEASE), ' ');
 
@@ -467,7 +470,6 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 
 	status = nvmet_copy_to_sgl(req, 0, id, sizeof(*id));
 
-out_free:
 	kfree(id);
 out:
 	nvmet_req_complete(req, status);
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 71a1fe2f6330a9..e5dbd1923b7b75 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -1118,13 +1118,11 @@ static ssize_t nvmet_subsys_attr_model_show(struct config_item *item,
 					    char *page)
 {
 	struct nvmet_subsys *subsys = to_subsys(item);
-	char *model = "";
 	int ret;
 
 	mutex_lock(&subsys->lock);
-	if (subsys->model_number)
-		model = subsys->model_number;
-	ret = snprintf(page, PAGE_SIZE, "%s\n", model);
+	ret = snprintf(page, PAGE_SIZE, "%s\n", subsys->model_number ?
+			subsys->model_number : NVMET_DEFAULT_CTRL_MODEL);
 	mutex_unlock(&subsys->lock);
 
 	return ret;
@@ -1136,54 +1134,45 @@ static bool nvmet_is_ascii(const char c)
 	return c >= 0x20 && c <= 0x7e;
 }
 
-static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
-					     const char *page, size_t count)
+static ssize_t nvmet_subsys_attr_model_store_locked(struct nvmet_subsys *subsys,
+		const char *page, size_t count)
 {
-	struct nvmet_subsys *subsys = to_subsys(item);
-	char *new_model_number;
 	int pos = 0, len;
 
-	down_write(&nvmet_config_sem);
-	mutex_lock(&subsys->lock);
 	if (subsys->model_number) {
 		pr_err("Can't set model number. %s is already assigned\n",
 		       subsys->model_number);
-		count = -EINVAL;
-		goto out_unlock;
+		return -EINVAL;
 	}
 
 	len = strcspn(page, "\n");
-	if (!len) {
-		count = -EINVAL;
-		goto out_unlock;
-	}
+	if (!len)
+		return -EINVAL;
 
 	for (pos = 0; pos < len; pos++) {
-		if (!nvmet_is_ascii(page[pos])) {
-			count = -EINVAL;
-			goto out_unlock;
-		}
+		if (!nvmet_is_ascii(page[pos]))
+			return -EINVAL;
 	}
 
-	new_model_number = kmemdup_nul(page, len, GFP_KERNEL);
-	if (!new_model_number) {
-		count = -ENOMEM;
-		goto out_unlock;
-	}
+	subsys->model_number = kmemdup_nul(page, len, GFP_KERNEL);
+	if (!subsys->model_number)
+		return -ENOMEM;
+	return count;
+}
 
-	subsys->model_number = kstrdup(new_model_number, GFP_KERNEL);
-	if (!subsys->model_number) {
-		count = -ENOMEM;
-		kfree(new_model_number);
-		goto out_unlock;
-	}
+static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
+					     const char *page, size_t count)
+{
+	struct nvmet_subsys *subsys = to_subsys(item);
+	ssize_t ret;
 
-	kfree(new_model_number);
-out_unlock:
+	down_write(&nvmet_config_sem);
+	mutex_lock(&subsys->lock);
+	ret = nvmet_subsys_attr_model_store_locked(subsys, page, count);
 	mutex_unlock(&subsys->lock);
 	up_write(&nvmet_config_sem);
 
-	return count;
+	return ret;
 }
 CONFIGFS_ATTR(nvmet_subsys_, attr_model);
 


From mgurtovoy at nvidia.com  Wed Feb 24 04:59:22 2021
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Wed, 24 Feb 2021 11:59:22 +0200
Subject: [PATCH 1/1] nvmet: allow setting model_number once
In-Reply-To: <20210224094347.GA5965@lst.de>
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
 <20210224094347.GA5965@lst.de>
Message-ID: <119dbae3-9dc8-a5d8-0f35-20d619931db6@nvidia.com>


On 2/24/2021 11:43 AM, Christoph Hellwig wrote:
> So looking at this and Chaitanya patch I think this version simplifies
> things quite nicely, and it also happens to get rid of the RCU annotation
> sparse warnings.
>
> Let me know what you think of the incremental cleanup and micro-opimization
> below, though:
>
>
> diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
> index 3da285e22e9209..2ca13dd7e7b88a 100644
> --- a/drivers/nvme/target/admin-cmd.c
> +++ b/drivers/nvme/target/admin-cmd.c
> @@ -313,34 +313,40 @@ static void nvmet_execute_get_log_page(struct nvmet_req *req)
>   	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_SC_DNR);
>   }
>   
> -static int nvmet_id_set_model_number(struct nvme_id_ctrl *id,
> -				      struct nvmet_subsys *subsys)
> +static u16 nvmet_set_model_number(struct nvme_id_ctrl *id,
> +		struct nvmet_subsys *subsys)
>   {
> -	int ret = 0;
> +	u16 status = 0;
>   
>   	mutex_lock(&subsys->lock);
>   	if (!subsys->model_number) {
> -		subsys->model_number = kstrdup(NVMET_DEFAULT_CTRL_MODEL,
> -					       GFP_KERNEL);
> -		if (!subsys->model_number) {
> -			ret = -ENOMEM;
> -			goto out_unlock;
> -		}
> +		subsys->model_number =
> +			kstrdup(NVMET_DEFAULT_CTRL_MODEL, GFP_KERNEL);
> +		if (!subsys->model_number)
> +			status = NVME_SC_INTERNAL;
>   	}
> -	memcpy_and_pad(id->mn, sizeof(id->mn), subsys->model_number,
> -		       strlen(subsys->model_number), ' ');
> -out_unlock:
>   	mutex_unlock(&subsys->lock);
> -	return ret;
> +
> +	return status;
>   }
>   
>   static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
>   {
>   	struct nvmet_ctrl *ctrl = req->sq->ctrl;
> +	struct nvmet_subsys *subsys = ctrl->subsys;
>   	struct nvme_id_ctrl *id;
>   	u32 cmd_capsule_size;
>   	u16 status = 0;
> -	int ret;
> +
> +	/*
> +	 * If there is no model number yet, so it now.  It will then remain
> +	 * stable for the life time of the subsystem.
> +	 */
> +	if (!subsys->model_number) {
> +		status = nvmet_set_model_number(id, subsys);

you don't need the id argument here.

Otherwise looks good and it can be squashed to my patch.




From hch at lst.de  Wed Feb 24 05:00:47 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 11:00:47 +0100
Subject: [PATCH 1/1] nvmet: allow setting model_number once
In-Reply-To: <119dbae3-9dc8-a5d8-0f35-20d619931db6@nvidia.com>
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
 <20210224094347.GA5965@lst.de>
 <119dbae3-9dc8-a5d8-0f35-20d619931db6@nvidia.com>
Message-ID: <20210224100047.GA9017@lst.de>

On Wed, Feb 24, 2021 at 11:59:22AM +0200, Max Gurtovoy wrote:
> you don't need the id argument here.
>
> Otherwise looks good and it can be squashed to my patch.

Can you resend it with the updates?


From mgurtovoy at nvidia.com  Wed Feb 24 05:02:37 2021
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Wed, 24 Feb 2021 12:02:37 +0200
Subject: [PATCH 1/1] nvmet: allow setting model_number once
In-Reply-To: <20210224100047.GA9017@lst.de>
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
 <20210224094347.GA5965@lst.de>
 <119dbae3-9dc8-a5d8-0f35-20d619931db6@nvidia.com>
 <20210224100047.GA9017@lst.de>
Message-ID: <d770fb90-5000-87bb-e46d-2f18de6b354d@nvidia.com>


On 2/24/2021 12:00 PM, Christoph Hellwig wrote:
> On Wed, Feb 24, 2021 at 11:59:22AM +0200, Max Gurtovoy wrote:
>> you don't need the id argument here.
>>
>> Otherwise looks good and it can be squashed to my patch.
> Can you resend it with the updates?

Yes.

What is the target branch ?



From hch at lst.de  Wed Feb 24 05:04:26 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 11:04:26 +0100
Subject: [PATCH 1/1] nvmet: allow setting model_number once
In-Reply-To: <d770fb90-5000-87bb-e46d-2f18de6b354d@nvidia.com>
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
 <20210224094347.GA5965@lst.de>
 <119dbae3-9dc8-a5d8-0f35-20d619931db6@nvidia.com>
 <20210224100047.GA9017@lst.de>
 <d770fb90-5000-87bb-e46d-2f18de6b354d@nvidia.com>
Message-ID: <20210224100426.GA9328@lst.de>

On Wed, Feb 24, 2021 at 12:02:37PM +0200, Max Gurtovoy wrote:
> What is the target branch ?

nvme-5.12 as just pushed out.


From pterjan at google.com  Wed Feb 24 05:23:23 2021
From: pterjan at google.com (Pascal Terjan)
Date: Wed, 24 Feb 2021 10:23:23 +0000
Subject: [PATCH] nvme: Add quirks for Lexar 256GB SSD
In-Reply-To: <20210224092147.GA2195892@infradead.org>
References: <20210223221046.3367795-1-pterjan@google.com>
 <20210224092147.GA2195892@infradead.org>
Message-ID: <CAANdO=LFVGNWNs_p9h0GsukVkRMcDQ1A5qsaa2m9HSPhu0wr7Q@mail.gmail.com>

On Wed, 24 Feb 2021 at 09:21, Christoph Hellwig <hch at infradead.org> wrote:
>
> On Tue, Feb 23, 2021 at 10:10:46PM +0000, Pascal Terjan wrote:
> > Add NVME_QUIRK_NO_NS_DESC_LIST to Lexar 256GB SSD to allow it to
> > work again, and add NVME_QUIRK_IGNORE_DEV_SUBNQN to avoid a warning.
>
> Why again?  The drive has always been broken since the desc list has
> been a mandatory feature since NVMe 1.3..

According to the reporter it works on Debian which is still running a
4.19 kernel
My guess is that it worked before
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=538af88ea7d9de241e6b6f006e9049c4d96723bb
as the error was ignored
And I believe it would have worked again until
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5bedd3afee8eb01ccd256f0cd2cc0fa6f841417a
as a waorkaround was added at some point (it changed at
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fb314eb0cbb2e11540d1ae1a7b28346397f621ef
but was laready there)


From martinus.gpy at gmail.com  Wed Feb 24 05:51:23 2021
From: martinus.gpy at gmail.com (Martin George)
Date: Wed, 24 Feb 2021 16:21:23 +0530
Subject: [PATCH v2] fabrics: ensure zero kato for non-persistent controllers
Message-ID: <20210224105123.7050-2-marting@netapp.com>

Add a helper function that sets the default kato value for the
controllers. Along with ensuring a non-zero kato value (i.e.
NVMF_DEF_DISC_TMO) is passed to a persistent discovery controller,
it also ensures a zero kato is passed to a non-persistent discovery
controller.

Signed-off-by: Martin George <marting at netapp.com>
---
 fabrics.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/fabrics.c b/fabrics.c
index e38657e..c780040 100644
--- a/fabrics.c
+++ b/fabrics.c
@@ -918,6 +918,16 @@ static int build_options(char *argstr, int max_len, bool discover)
 	return 0;
 }
 
+static void set_kato(struct config *cfg)
+{
+	/* Set kato to NVMF_DEF_DISC_TMO for persistent discovery controllers */
+	if (cfg->persistent && !cfg->keep_alive_tmo)
+		cfg->keep_alive_tmo = NVMF_DEF_DISC_TMO;
+	/* Set kato to zero for non-persistent discovery controllers */
+	else if (!cfg->persistent && (cfg->keep_alive_tmo > 0))
+		cfg->keep_alive_tmo = 0;
+}
+
 static void discovery_trsvcid(struct config *cfg)
 {
 	if (!strcmp(cfg->transport, "tcp")) {
@@ -1401,8 +1411,7 @@ static int discover_from_conf_file(const char *desc, char *argstr,
 		if (err)
 			goto free_and_continue;
 
-		if (cfg.persistent && !cfg.keep_alive_tmo)
-			cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
+		set_kato(&cfg);
 
 		if (traddr_is_hostname(&cfg)) {
 			ret = hostname2traddr(&cfg);
@@ -1476,8 +1485,7 @@ int fabrics_discover(const char *desc, int argc, char **argv, bool connect)
 	if (!cfg.transport && !cfg.traddr) {
 		ret = discover_from_conf_file(desc, argstr, opts, connect);
 	} else {
-		if (cfg.persistent && !cfg.keep_alive_tmo)
-			cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
+		set_kato(&cfg);
 
 		if (traddr_is_hostname(&cfg)) {
 			ret = hostname2traddr(&cfg);
-- 
2.30.0



From sashal at kernel.org  Wed Feb 24 07:50:02 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:50:02 -0500
Subject: [PATCH AUTOSEL 5.11 44/67] nvme-core: add cancel tagset helpers
In-Reply-To: <20210224125026.481804-1-sashal@kernel.org>
References: <20210224125026.481804-1-sashal@kernel.org>
Message-ID: <20210224125026.481804-44-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 2547906982e2e6a0d42f8957f55af5bb51a7e55f ]

Add nvme_cancel_tagset and nvme_cancel_admin_tagset for tear down and
reconnection error handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/core.c | 20 ++++++++++++++++++++
 drivers/nvme/host/nvme.h |  2 ++
 2 files changed, 22 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index f13eb4ded95fa..129e2b6bd6d3f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -371,6 +371,26 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
 }
 EXPORT_SYMBOL_GPL(nvme_cancel_request);
 
+void nvme_cancel_tagset(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->tagset) {
+		blk_mq_tagset_busy_iter(ctrl->tagset,
+				nvme_cancel_request, ctrl);
+		blk_mq_tagset_wait_completed_request(ctrl->tagset);
+	}
+}
+EXPORT_SYMBOL_GPL(nvme_cancel_tagset);
+
+void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->admin_tagset) {
+		blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+				nvme_cancel_request, ctrl);
+		blk_mq_tagset_wait_completed_request(ctrl->admin_tagset);
+	}
+}
+EXPORT_SYMBOL_GPL(nvme_cancel_admin_tagset);
+
 bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
 		enum nvme_ctrl_state new_state)
 {
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 88a6b97247f50..a72f071810910 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -576,6 +576,8 @@ static inline bool nvme_is_aen_req(u16 qid, __u16 command_id)
 
 void nvme_complete_rq(struct request *req);
 bool nvme_cancel_request(struct request *req, void *data, bool reserved);
+void nvme_cancel_tagset(struct nvme_ctrl *ctrl);
+void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl);
 bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
 		enum nvme_ctrl_state new_state);
 bool nvme_wait_reset(struct nvme_ctrl *ctrl);
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:50:03 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:50:03 -0500
Subject: [PATCH AUTOSEL 5.11 45/67] nvme-rdma: add clean action for failed
 reconnection
In-Reply-To: <20210224125026.481804-1-sashal@kernel.org>
References: <20210224125026.481804-1-sashal@kernel.org>
Message-ID: <20210224125026.481804-45-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 958dc1d32c80566f58d18f05ef1f05bd32d172c1 ]

A crash happens when inject failed reconnection.
If reconnect failed after start io queues, the queues will be unquiesced
and new requests continue to be delivered. Reconnection error handling
process directly free queues without cancel suspend requests. The
suppend request will time out, and then crash due to use the queue
after free.

Add sync queues and cancel suppend requests for reconnection error
handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/rdma.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index b7ce4f221d990..746392eade455 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -919,12 +919,16 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
 
 	error = nvme_init_identify(&ctrl->ctrl);
 	if (error)
-		goto out_stop_queue;
+		goto out_quiesce_queue;
 
 	return 0;
 
+out_quiesce_queue:
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 out_stop_queue:
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
+	nvme_cancel_admin_tagset(&ctrl->ctrl);
 out_cleanup_queue:
 	if (new)
 		blk_cleanup_queue(ctrl->ctrl.admin_q);
@@ -1001,8 +1005,10 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
 
 out_wait_freeze_timed_out:
 	nvme_stop_queues(&ctrl->ctrl);
+	nvme_sync_io_queues(&ctrl->ctrl);
 	nvme_rdma_stop_io_queues(ctrl);
 out_cleanup_connect_q:
+	nvme_cancel_tagset(&ctrl->ctrl);
 	if (new)
 		blk_cleanup_queue(ctrl->ctrl.connect_q);
 out_free_tag_set:
@@ -1144,10 +1150,18 @@ static int nvme_rdma_setup_ctrl(struct nvme_rdma_ctrl *ctrl, bool new)
 	return 0;
 
 destroy_io:
-	if (ctrl->ctrl.queue_count > 1)
+	if (ctrl->ctrl.queue_count > 1) {
+		nvme_stop_queues(&ctrl->ctrl);
+		nvme_sync_io_queues(&ctrl->ctrl);
+		nvme_rdma_stop_io_queues(ctrl);
+		nvme_cancel_tagset(&ctrl->ctrl);
 		nvme_rdma_destroy_io_queues(ctrl, new);
+	}
 destroy_admin:
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
+	nvme_cancel_admin_tagset(&ctrl->ctrl);
 	nvme_rdma_destroy_admin_queue(ctrl, new);
 	return ret;
 }
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:50:04 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:50:04 -0500
Subject: [PATCH AUTOSEL 5.11 46/67] nvme-tcp: add clean action for failed
 reconnection
In-Reply-To: <20210224125026.481804-1-sashal@kernel.org>
References: <20210224125026.481804-1-sashal@kernel.org>
Message-ID: <20210224125026.481804-46-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 70a99574a79f1cd4dc7ad56ea37be40844bfb97b ]

If reconnect failed after start io queues, the queues will be unquiesced
and new requests continue to be delivered. Reconnection error handling
process directly free queues without cancel suspend requests. The
suppend request will time out, and then crash due to use the queue
after free.

Add sync queues and cancel suppend requests for reconnection error
handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/tcp.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 881d28eb15e9d..30d24a5a5b826 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1815,8 +1815,10 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
 
 out_wait_freeze_timed_out:
 	nvme_stop_queues(ctrl);
+	nvme_sync_io_queues(ctrl);
 	nvme_tcp_stop_io_queues(ctrl);
 out_cleanup_connect_q:
+	nvme_cancel_tagset(ctrl);
 	if (new)
 		blk_cleanup_queue(ctrl->connect_q);
 out_free_tag_set:
@@ -1878,12 +1880,16 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 
 	error = nvme_init_identify(ctrl);
 	if (error)
-		goto out_stop_queue;
+		goto out_quiesce_queue;
 
 	return 0;
 
+out_quiesce_queue:
+	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 out_stop_queue:
 	nvme_tcp_stop_queue(ctrl, 0);
+	nvme_cancel_admin_tagset(ctrl);
 out_cleanup_queue:
 	if (new)
 		blk_cleanup_queue(ctrl->admin_q);
@@ -2003,10 +2009,18 @@ static int nvme_tcp_setup_ctrl(struct nvme_ctrl *ctrl, bool new)
 	return 0;
 
 destroy_io:
-	if (ctrl->queue_count > 1)
+	if (ctrl->queue_count > 1) {
+		nvme_stop_queues(ctrl);
+		nvme_sync_io_queues(ctrl);
+		nvme_tcp_stop_io_queues(ctrl);
+		nvme_cancel_tagset(ctrl);
 		nvme_tcp_destroy_io_queues(ctrl, new);
+	}
 destroy_admin:
+	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
+	nvme_cancel_admin_tagset(ctrl);
 	nvme_tcp_destroy_admin_queue(ctrl, new);
 	return ret;
 }
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:51:52 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:51:52 -0500
Subject: [PATCH AUTOSEL 5.10 36/56] nvme-core: add cancel tagset helpers
In-Reply-To: <20210224125212.482485-1-sashal@kernel.org>
References: <20210224125212.482485-1-sashal@kernel.org>
Message-ID: <20210224125212.482485-36-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 2547906982e2e6a0d42f8957f55af5bb51a7e55f ]

Add nvme_cancel_tagset and nvme_cancel_admin_tagset for tear down and
reconnection error handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/core.c | 20 ++++++++++++++++++++
 drivers/nvme/host/nvme.h |  2 ++
 2 files changed, 22 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4ec5f05dabe1d..e1e574ecf031b 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -351,6 +351,26 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
 }
 EXPORT_SYMBOL_GPL(nvme_cancel_request);
 
+void nvme_cancel_tagset(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->tagset) {
+		blk_mq_tagset_busy_iter(ctrl->tagset,
+				nvme_cancel_request, ctrl);
+		blk_mq_tagset_wait_completed_request(ctrl->tagset);
+	}
+}
+EXPORT_SYMBOL_GPL(nvme_cancel_tagset);
+
+void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->admin_tagset) {
+		blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+				nvme_cancel_request, ctrl);
+		blk_mq_tagset_wait_completed_request(ctrl->admin_tagset);
+	}
+}
+EXPORT_SYMBOL_GPL(nvme_cancel_admin_tagset);
+
 bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
 		enum nvme_ctrl_state new_state)
 {
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 567f7ad18a91c..f843540cc238e 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -571,6 +571,8 @@ static inline bool nvme_is_aen_req(u16 qid, __u16 command_id)
 
 void nvme_complete_rq(struct request *req);
 bool nvme_cancel_request(struct request *req, void *data, bool reserved);
+void nvme_cancel_tagset(struct nvme_ctrl *ctrl);
+void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl);
 bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
 		enum nvme_ctrl_state new_state);
 bool nvme_wait_reset(struct nvme_ctrl *ctrl);
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:51:53 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:51:53 -0500
Subject: [PATCH AUTOSEL 5.10 37/56] nvme-rdma: add clean action for failed
 reconnection
In-Reply-To: <20210224125212.482485-1-sashal@kernel.org>
References: <20210224125212.482485-1-sashal@kernel.org>
Message-ID: <20210224125212.482485-37-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 958dc1d32c80566f58d18f05ef1f05bd32d172c1 ]

A crash happens when inject failed reconnection.
If reconnect failed after start io queues, the queues will be unquiesced
and new requests continue to be delivered. Reconnection error handling
process directly free queues without cancel suspend requests. The
suppend request will time out, and then crash due to use the queue
after free.

Add sync queues and cancel suppend requests for reconnection error
handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/rdma.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 493ed7ba86ed2..fd2b4dbe9d160 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -919,12 +919,16 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
 
 	error = nvme_init_identify(&ctrl->ctrl);
 	if (error)
-		goto out_stop_queue;
+		goto out_quiesce_queue;
 
 	return 0;
 
+out_quiesce_queue:
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 out_stop_queue:
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
+	nvme_cancel_admin_tagset(&ctrl->ctrl);
 out_cleanup_queue:
 	if (new)
 		blk_cleanup_queue(ctrl->ctrl.admin_q);
@@ -1001,8 +1005,10 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
 
 out_wait_freeze_timed_out:
 	nvme_stop_queues(&ctrl->ctrl);
+	nvme_sync_io_queues(&ctrl->ctrl);
 	nvme_rdma_stop_io_queues(ctrl);
 out_cleanup_connect_q:
+	nvme_cancel_tagset(&ctrl->ctrl);
 	if (new)
 		blk_cleanup_queue(ctrl->ctrl.connect_q);
 out_free_tag_set:
@@ -1144,10 +1150,18 @@ static int nvme_rdma_setup_ctrl(struct nvme_rdma_ctrl *ctrl, bool new)
 	return 0;
 
 destroy_io:
-	if (ctrl->ctrl.queue_count > 1)
+	if (ctrl->ctrl.queue_count > 1) {
+		nvme_stop_queues(&ctrl->ctrl);
+		nvme_sync_io_queues(&ctrl->ctrl);
+		nvme_rdma_stop_io_queues(ctrl);
+		nvme_cancel_tagset(&ctrl->ctrl);
 		nvme_rdma_destroy_io_queues(ctrl, new);
+	}
 destroy_admin:
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
+	nvme_cancel_admin_tagset(&ctrl->ctrl);
 	nvme_rdma_destroy_admin_queue(ctrl, new);
 	return ret;
 }
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:51:54 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:51:54 -0500
Subject: [PATCH AUTOSEL 5.10 38/56] nvme-tcp: add clean action for failed
 reconnection
In-Reply-To: <20210224125212.482485-1-sashal@kernel.org>
References: <20210224125212.482485-1-sashal@kernel.org>
Message-ID: <20210224125212.482485-38-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 70a99574a79f1cd4dc7ad56ea37be40844bfb97b ]

If reconnect failed after start io queues, the queues will be unquiesced
and new requests continue to be delivered. Reconnection error handling
process directly free queues without cancel suspend requests. The
suppend request will time out, and then crash due to use the queue
after free.

Add sync queues and cancel suppend requests for reconnection error
handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/tcp.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 6487b7897d1fb..739ac7deccd96 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1815,8 +1815,10 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
 
 out_wait_freeze_timed_out:
 	nvme_stop_queues(ctrl);
+	nvme_sync_io_queues(ctrl);
 	nvme_tcp_stop_io_queues(ctrl);
 out_cleanup_connect_q:
+	nvme_cancel_tagset(ctrl);
 	if (new)
 		blk_cleanup_queue(ctrl->connect_q);
 out_free_tag_set:
@@ -1878,12 +1880,16 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 
 	error = nvme_init_identify(ctrl);
 	if (error)
-		goto out_stop_queue;
+		goto out_quiesce_queue;
 
 	return 0;
 
+out_quiesce_queue:
+	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 out_stop_queue:
 	nvme_tcp_stop_queue(ctrl, 0);
+	nvme_cancel_admin_tagset(ctrl);
 out_cleanup_queue:
 	if (new)
 		blk_cleanup_queue(ctrl->admin_q);
@@ -2003,10 +2009,18 @@ static int nvme_tcp_setup_ctrl(struct nvme_ctrl *ctrl, bool new)
 	return 0;
 
 destroy_io:
-	if (ctrl->queue_count > 1)
+	if (ctrl->queue_count > 1) {
+		nvme_stop_queues(ctrl);
+		nvme_sync_io_queues(ctrl);
+		nvme_tcp_stop_io_queues(ctrl);
+		nvme_cancel_tagset(ctrl);
 		nvme_tcp_destroy_io_queues(ctrl, new);
+	}
 destroy_admin:
+	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
+	nvme_cancel_admin_tagset(ctrl);
 	nvme_tcp_destroy_admin_queue(ctrl, new);
 	return ret;
 }
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:53:26 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:53:26 -0500
Subject: [PATCH AUTOSEL 5.4 26/40] nvme-core: add cancel tagset helpers
In-Reply-To: <20210224125340.483162-1-sashal@kernel.org>
References: <20210224125340.483162-1-sashal@kernel.org>
Message-ID: <20210224125340.483162-26-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 2547906982e2e6a0d42f8957f55af5bb51a7e55f ]

Add nvme_cancel_tagset and nvme_cancel_admin_tagset for tear down and
reconnection error handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/core.c | 20 ++++++++++++++++++++
 drivers/nvme/host/nvme.h |  2 ++
 2 files changed, 22 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c2cabd77884bf..95d77a17375e1 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -317,6 +317,26 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
 }
 EXPORT_SYMBOL_GPL(nvme_cancel_request);
 
+void nvme_cancel_tagset(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->tagset) {
+		blk_mq_tagset_busy_iter(ctrl->tagset,
+				nvme_cancel_request, ctrl);
+		blk_mq_tagset_wait_completed_request(ctrl->tagset);
+	}
+}
+EXPORT_SYMBOL_GPL(nvme_cancel_tagset);
+
+void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->admin_tagset) {
+		blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+				nvme_cancel_request, ctrl);
+		blk_mq_tagset_wait_completed_request(ctrl->admin_tagset);
+	}
+}
+EXPORT_SYMBOL_GPL(nvme_cancel_admin_tagset);
+
 bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
 		enum nvme_ctrl_state new_state)
 {
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index e392d6cd92ced..62e5401865fee 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -468,6 +468,8 @@ static inline void nvme_put_ctrl(struct nvme_ctrl *ctrl)
 
 void nvme_complete_rq(struct request *req);
 bool nvme_cancel_request(struct request *req, void *data, bool reserved);
+void nvme_cancel_tagset(struct nvme_ctrl *ctrl);
+void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl);
 bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
 		enum nvme_ctrl_state new_state);
 bool nvme_wait_reset(struct nvme_ctrl *ctrl);
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:53:28 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:53:28 -0500
Subject: [PATCH AUTOSEL 5.4 28/40] nvme-tcp: add clean action for failed
 reconnection
In-Reply-To: <20210224125340.483162-1-sashal@kernel.org>
References: <20210224125340.483162-1-sashal@kernel.org>
Message-ID: <20210224125340.483162-28-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 70a99574a79f1cd4dc7ad56ea37be40844bfb97b ]

If reconnect failed after start io queues, the queues will be unquiesced
and new requests continue to be delivered. Reconnection error handling
process directly free queues without cancel suspend requests. The
suppend request will time out, and then crash due to use the queue
after free.

Add sync queues and cancel suppend requests for reconnection error
handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/tcp.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index a554021e1ab92..77a3c488ec120 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1710,8 +1710,10 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
 
 out_wait_freeze_timed_out:
 	nvme_stop_queues(ctrl);
+	nvme_sync_io_queues(ctrl);
 	nvme_tcp_stop_io_queues(ctrl);
 out_cleanup_connect_q:
+	nvme_cancel_tagset(ctrl);
 	if (new)
 		blk_cleanup_queue(ctrl->connect_q);
 out_free_tag_set:
@@ -1773,12 +1775,16 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 
 	error = nvme_init_identify(ctrl);
 	if (error)
-		goto out_stop_queue;
+		goto out_quiesce_queue;
 
 	return 0;
 
+out_quiesce_queue:
+	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 out_stop_queue:
 	nvme_tcp_stop_queue(ctrl, 0);
+	nvme_cancel_admin_tagset(ctrl);
 out_cleanup_queue:
 	if (new)
 		blk_cleanup_queue(ctrl->admin_q);
@@ -1892,10 +1898,18 @@ static int nvme_tcp_setup_ctrl(struct nvme_ctrl *ctrl, bool new)
 	return 0;
 
 destroy_io:
-	if (ctrl->queue_count > 1)
+	if (ctrl->queue_count > 1) {
+		nvme_stop_queues(ctrl);
+		nvme_sync_io_queues(ctrl);
+		nvme_tcp_stop_io_queues(ctrl);
+		nvme_cancel_tagset(ctrl);
 		nvme_tcp_destroy_io_queues(ctrl, new);
+	}
 destroy_admin:
+	blk_mq_quiesce_queue(ctrl->admin_q);
+	blk_sync_queue(ctrl->admin_q);
 	nvme_tcp_stop_queue(ctrl, 0);
+	nvme_cancel_admin_tagset(ctrl);
 	nvme_tcp_destroy_admin_queue(ctrl, new);
 	return ret;
 }
-- 
2.27.0



From sashal at kernel.org  Wed Feb 24 07:53:27 2021
From: sashal at kernel.org (Sasha Levin)
Date: Wed, 24 Feb 2021 07:53:27 -0500
Subject: [PATCH AUTOSEL 5.4 27/40] nvme-rdma: add clean action for failed
 reconnection
In-Reply-To: <20210224125340.483162-1-sashal@kernel.org>
References: <20210224125340.483162-1-sashal@kernel.org>
Message-ID: <20210224125340.483162-27-sashal@kernel.org>

From: Chao Leng <lengchao at huawei.com>

[ Upstream commit 958dc1d32c80566f58d18f05ef1f05bd32d172c1 ]

A crash happens when inject failed reconnection.
If reconnect failed after start io queues, the queues will be unquiesced
and new requests continue to be delivered. Reconnection error handling
process directly free queues without cancel suspend requests. The
suppend request will time out, and then crash due to use the queue
after free.

Add sync queues and cancel suppend requests for reconnection error
handling.

Signed-off-by: Chao Leng <lengchao at huawei.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 drivers/nvme/host/rdma.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 8a62c2fe5a5ec..da60300104322 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -835,12 +835,16 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
 
 	error = nvme_init_identify(&ctrl->ctrl);
 	if (error)
-		goto out_stop_queue;
+		goto out_quiesce_queue;
 
 	return 0;
 
+out_quiesce_queue:
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 out_stop_queue:
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
+	nvme_cancel_admin_tagset(&ctrl->ctrl);
 out_cleanup_queue:
 	if (new)
 		blk_cleanup_queue(ctrl->ctrl.admin_q);
@@ -917,8 +921,10 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
 
 out_wait_freeze_timed_out:
 	nvme_stop_queues(&ctrl->ctrl);
+	nvme_sync_io_queues(&ctrl->ctrl);
 	nvme_rdma_stop_io_queues(ctrl);
 out_cleanup_connect_q:
+	nvme_cancel_tagset(&ctrl->ctrl);
 	if (new)
 		blk_cleanup_queue(ctrl->ctrl.connect_q);
 out_free_tag_set:
@@ -1054,10 +1060,18 @@ static int nvme_rdma_setup_ctrl(struct nvme_rdma_ctrl *ctrl, bool new)
 	return 0;
 
 destroy_io:
-	if (ctrl->ctrl.queue_count > 1)
+	if (ctrl->ctrl.queue_count > 1) {
+		nvme_stop_queues(&ctrl->ctrl);
+		nvme_sync_io_queues(&ctrl->ctrl);
+		nvme_rdma_stop_io_queues(ctrl);
+		nvme_cancel_tagset(&ctrl->ctrl);
 		nvme_rdma_destroy_io_queues(ctrl, new);
+	}
 destroy_admin:
+	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
+	blk_sync_queue(ctrl->ctrl.admin_q);
 	nvme_rdma_stop_queue(&ctrl->queues[0]);
+	nvme_cancel_admin_tagset(&ctrl->ctrl);
 	nvme_rdma_destroy_admin_queue(ctrl, new);
 	return ret;
 }
-- 
2.27.0



From mgurtovoy at nvidia.com  Wed Feb 24 07:57:35 2021
From: mgurtovoy at nvidia.com (Max Gurtovoy)
Date: Wed, 24 Feb 2021 14:57:35 +0200
Subject: [PATCH 1/1] nvmet: allow setting model_number once
In-Reply-To: <20210224100426.GA9328@lst.de>
References: <20210217181153.15343-1-mgurtovoy@nvidia.com>
 <20210224094347.GA5965@lst.de>
 <119dbae3-9dc8-a5d8-0f35-20d619931db6@nvidia.com>
 <20210224100047.GA9017@lst.de>
 <d770fb90-5000-87bb-e46d-2f18de6b354d@nvidia.com>
 <20210224100426.GA9328@lst.de>
Message-ID: <bf7cf9e5-71be-4add-91a5-bdeee8d105d0@nvidia.com>


On 2/24/2021 12:04 PM, Christoph Hellwig wrote:
> On Wed, Feb 24, 2021 at 12:02:37PM +0200, Max Gurtovoy wrote:
>> What is the target branch ?
> nvme-5.12 as just pushed out.

See attached.

-------------- next part --------------
From bcdf33f3995417aff47e55d4d06c7fc61a68cf9f Mon Sep 17 00:00:00 2001
From: Max Gurtovoy <mgurtovoy at nvidia.com>
Date: Wed, 17 Feb 2021 17:19:40 +0000
Subject: [PATCH 1/1] nvmet: allow setting model_number once
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In case we have already established connection to nvmf target, it
shouldn't be allowed to change the model_number. E.g. if someone will
identify ctrl and get model_number of "my_model" later on will change
the model_numbel via configfs to "my_new_model" this will break the NVM
specification for "Get Log Page ? Persistent Event Log" that refers to
Model Number as: "This field contains the same value as reported in the
Model Number field of the Identify Controller data structure, bytes
63:24."

Although it doesn't mentioned explicitly that this field can't be
changed, we can assume it.

So allow setting this field only once: using configfs or in the first
identify ctrl operation.

Signed-off-by: Max Gurtovoy <mgurtovoy at nvidia.com>
Signed-off-by: Christoph Hellwig <hch at lst.de>
---
 drivers/nvme/target/admin-cmd.c | 36 ++++++++++++++++--------
 drivers/nvme/target/configfs.c  | 50 +++++++++++++++------------------
 drivers/nvme/target/core.c      |  2 +-
 drivers/nvme/target/nvmet.h     |  7 +----
 4 files changed, 50 insertions(+), 45 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index bc6a774f2124..fe6b8aa90b53 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -313,27 +313,40 @@ static void nvmet_execute_get_log_page(struct nvmet_req *req)
 	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_SC_DNR);
 }
 
-static void nvmet_id_set_model_number(struct nvme_id_ctrl *id,
-				      struct nvmet_subsys *subsys)
+static u16 nvmet_set_model_number(struct nvmet_subsys *subsys)
 {
-	const char *model = NVMET_DEFAULT_CTRL_MODEL;
-	struct nvmet_subsys_model *subsys_model;
+	u16 status = 0;
+
+	mutex_lock(&subsys->lock);
+	if (!subsys->model_number) {
+		subsys->model_number =
+			kstrdup(NVMET_DEFAULT_CTRL_MODEL, GFP_KERNEL);
+		if (!subsys->model_number)
+			status = NVME_SC_INTERNAL;
+	}
+	mutex_unlock(&subsys->lock);
 
-	rcu_read_lock();
-	subsys_model = rcu_dereference(subsys->model);
-	if (subsys_model)
-		model = subsys_model->number;
-	memcpy_and_pad(id->mn, sizeof(id->mn), model, strlen(model), ' ');
-	rcu_read_unlock();
+	return status;
 }
 
 static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 {
 	struct nvmet_ctrl *ctrl = req->sq->ctrl;
+	struct nvmet_subsys *subsys = ctrl->subsys;
 	struct nvme_id_ctrl *id;
 	u32 cmd_capsule_size;
 	u16 status = 0;
 
+	/*
+	 * If there is no model number yet, set it now.  It will then remain
+	 * stable for the life time of the subsystem.
+	 */
+	if (!subsys->model_number) {
+		status = nvmet_set_model_number(subsys);
+		if (status)
+			goto out;
+	}
+
 	id = kzalloc(sizeof(*id), GFP_KERNEL);
 	if (!id) {
 		status = NVME_SC_INTERNAL;
@@ -347,7 +360,8 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 	memset(id->sn, ' ', sizeof(id->sn));
 	bin2hex(id->sn, &ctrl->subsys->serial,
 		min(sizeof(ctrl->subsys->serial), sizeof(id->sn) / 2));
-	nvmet_id_set_model_number(id, ctrl->subsys);
+	memcpy_and_pad(id->mn, sizeof(id->mn), subsys->model_number,
+		       strlen(subsys->model_number), ' ');
 	memcpy_and_pad(id->fr, sizeof(id->fr),
 		       UTS_RELEASE, strlen(UTS_RELEASE), ' ');
 
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 635a7cb45d0b..e5dbd1923b7b 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -1118,16 +1118,12 @@ static ssize_t nvmet_subsys_attr_model_show(struct config_item *item,
 					    char *page)
 {
 	struct nvmet_subsys *subsys = to_subsys(item);
-	struct nvmet_subsys_model *subsys_model;
-	char *model = NVMET_DEFAULT_CTRL_MODEL;
 	int ret;
 
-	rcu_read_lock();
-	subsys_model = rcu_dereference(subsys->model);
-	if (subsys_model)
-		model = subsys_model->number;
-	ret = snprintf(page, PAGE_SIZE, "%s\n", model);
-	rcu_read_unlock();
+	mutex_lock(&subsys->lock);
+	ret = snprintf(page, PAGE_SIZE, "%s\n", subsys->model_number ?
+			subsys->model_number : NVMET_DEFAULT_CTRL_MODEL);
+	mutex_unlock(&subsys->lock);
 
 	return ret;
 }
@@ -1138,14 +1134,17 @@ static bool nvmet_is_ascii(const char c)
 	return c >= 0x20 && c <= 0x7e;
 }
 
-static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
-					     const char *page, size_t count)
+static ssize_t nvmet_subsys_attr_model_store_locked(struct nvmet_subsys *subsys,
+		const char *page, size_t count)
 {
-	struct nvmet_subsys *subsys = to_subsys(item);
-	struct nvmet_subsys_model *new_model;
-	char *new_model_number;
 	int pos = 0, len;
 
+	if (subsys->model_number) {
+		pr_err("Can't set model number. %s is already assigned\n",
+		       subsys->model_number);
+		return -EINVAL;
+	}
+
 	len = strcspn(page, "\n");
 	if (!len)
 		return -EINVAL;
@@ -1155,28 +1154,25 @@ static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
 			return -EINVAL;
 	}
 
-	new_model_number = kmemdup_nul(page, len, GFP_KERNEL);
-	if (!new_model_number)
+	subsys->model_number = kmemdup_nul(page, len, GFP_KERNEL);
+	if (!subsys->model_number)
 		return -ENOMEM;
+	return count;
+}
 
-	new_model = kzalloc(sizeof(*new_model) + len + 1, GFP_KERNEL);
-	if (!new_model) {
-		kfree(new_model_number);
-		return -ENOMEM;
-	}
-	memcpy(new_model->number, new_model_number, len);
+static ssize_t nvmet_subsys_attr_model_store(struct config_item *item,
+					     const char *page, size_t count)
+{
+	struct nvmet_subsys *subsys = to_subsys(item);
+	ssize_t ret;
 
 	down_write(&nvmet_config_sem);
 	mutex_lock(&subsys->lock);
-	new_model = rcu_replace_pointer(subsys->model, new_model,
-					mutex_is_locked(&subsys->lock));
+	ret = nvmet_subsys_attr_model_store_locked(subsys, page, count);
 	mutex_unlock(&subsys->lock);
 	up_write(&nvmet_config_sem);
 
-	kfree_rcu(new_model, rcuhead);
-	kfree(new_model_number);
-
-	return count;
+	return ret;
 }
 CONFIGFS_ATTR(nvmet_subsys_, attr_model);
 
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 67bbf0e3b507..be6fcdaf51a7 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1532,7 +1532,7 @@ static void nvmet_subsys_free(struct kref *ref)
 	nvmet_passthru_subsys_free(subsys);
 
 	kfree(subsys->subsysnqn);
-	kfree_rcu(subsys->model, rcuhead);
+	kfree(subsys->model_number);
 	kfree(subsys);
 }
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index cdfa537b1c0a..4b84edb49f22 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -208,11 +208,6 @@ struct nvmet_ctrl {
 	bool			pi_support;
 };
 
-struct nvmet_subsys_model {
-	struct rcu_head		rcuhead;
-	char			number[];
-};
-
 struct nvmet_subsys {
 	enum nvme_subsys_type	type;
 
@@ -242,7 +237,7 @@ struct nvmet_subsys {
 	struct config_group	namespaces_group;
 	struct config_group	allowed_hosts_group;
 
-	struct nvmet_subsys_model	__rcu *model;
+	char			*model_number;
 
 #ifdef CONFIG_NVME_TARGET_PASSTHRU
 	struct nvme_ctrl	*passthru_ctrl;
-- 
2.18.2


From kbusch at kernel.org  Wed Feb 24 10:42:29 2021
From: kbusch at kernel.org (Keith Busch)
Date: Thu, 25 Feb 2021 00:42:29 +0900
Subject: [PATCH 00/14] nvme: few cleanups and small improvements
In-Reply-To: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224154229.GA26830@redsun51.ssa.fujisawa.hgst.com>

On Tue, Feb 16, 2021 at 04:10:18PM -0800, Chaitanya Kulkarni wrote:
> Hi,
> 
> This has few cleanups such as variable data type, removing?oneliner
> wrappers for single callers, removing duplicate kfree() and small
> improvements for NVMeOF Passthru fast path such as likely annotation,
> removing the extra checks, making function inline for host/core.c.
> Last couple of patches fixes warning for FC function header
> documentation.

The two documentation fixes at the end look fine, but much of the rest
looks unnecessary. 
 
> Chaitanya Kulkarni (14):
>   nvme-core: remove duplicate kfree in init identify
>   nvme-core: don't use switch for only one case use
>   nvme-core: use right type for ARRAY_SIZE check
>   nvme-core: fix the type for shutdown_timeout
>   nvme-core: add helper to init ctrl transport attr
>   nvme-core: add helper to init shutdown timeout
>   nvme-core: add helper to init ctrl subsys quirk
>   nvme-core: move util quirk at the top of the file
>   nvme-core: mark nvme_setup_passsthru() inline
>   nvme-core: remove one liner wrappers for streams
>   nvme-core: use likely in nvme_init_request()
>   nvme-core: don't check nvme_req flags for new req
>   nvme-fc: fix the function documentation comment
>   nvmet-fc: update function documentation
> 
>  drivers/nvme/host/core.c | 333 ++++++++++++++++++++-------------------
>  drivers/nvme/host/fc.c   |   2 +-
>  drivers/nvme/target/fc.c |   1 +
>  3 files changed, 172 insertions(+), 164 deletions(-)
> 
> -- 
> 2.22.1


From hch at lst.de  Wed Feb 24 11:22:10 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:22:10 +0100
Subject: [PATCH 1/2] nvme: fixup kato deadlock
In-Reply-To: <20210223120728.104699-2-hare@suse.de>
References: <20210223120728.104699-1-hare@suse.de>
 <20210223120728.104699-2-hare@suse.de>
Message-ID: <20210224162210.GA9780@lst.de>

Well, kato really should be doing a nowait request allocation instead
of beeing stuck.  That being said we currently have two reserved
requests, with one set aside for connect.  This does back to the initial
commit, but I can't see how a keep alive could still be around when we
reconnect, so I think one reserved command should be fine as well.


From hch at lst.de  Wed Feb 24 11:23:50 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:23:50 +0100
Subject: [PATCH 2/2] nvme: sanitize KATO setting
In-Reply-To: <20210223120728.104699-3-hare@suse.de>
References: <20210223120728.104699-1-hare@suse.de>
 <20210223120728.104699-3-hare@suse.de>
Message-ID: <20210224162350.GB9780@lst.de>

> +		unsigned long kato_delay = ctrl->kato >> 1;
> +
> +		queue_delayed_work(nvme_wq, &ctrl->ka_work, kato_delay * HZ);

> +		unsigned long kato_delay = ctrl->kato >> 1;
> +
> +		queue_delayed_work(nvme_wq, &ctrl->ka_work, kato_delay * HZ);

> +	unsigned long kato_delay = ctrl->kato >> 1;

> +	queue_delayed_work(nvme_wq, &ctrl->ka_work, kato_delay * HZ);

I think we need a properly documented helper for

	(ctrl->kato / 2) * HZ

instead of all this open coded magic.

>  	cmd.connect.kato = ctrl->kato ?
> -		cpu_to_le32((ctrl->kato + NVME_KATO_GRACE) * 1000) : 0;
> +		cpu_to_le32(ctrl->kato * 1000) : 0;

cpu_to_le32(0 * 1000) is still 0, so we can remove the branch here now.


From hch at lst.de  Wed Feb 24 11:25:34 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:25:34 +0100
Subject: [PATCH 0/2] nvme: fix regression with MD RAID
In-Reply-To: <20210223115922.104369-1-hare@suse.de>
References: <20210223115922.104369-1-hare@suse.de>
Message-ID: <20210224162534.GC9780@lst.de>

I don't see any regression here, even if the new features sound useful.


From hch at lst.de  Wed Feb 24 11:30:44 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:30:44 +0100
Subject: [PATCH 2/6] nvmet: set status on actual error condition
In-Reply-To: <20210216213112.20078-3-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
 <20210216213112.20078-3-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224163044.GA10673@lst.de>

On Tue, Feb 16, 2021 at 01:31:08PM -0800, Chaitanya Kulkarni wrote:
> In the nvmet_alloc_ctrl() set the status variable to its error value
> NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR in the actual error
> condition just like the rest of the code.

This is a pretty normal pattern in the kernel, so I don't see much
of a point to change it.


From hch at lst.de  Wed Feb 24 11:31:16 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:31:16 +0100
Subject: [PATCH 3/6] nvmet: check and set the right err location
In-Reply-To: <20210216213112.20078-4-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
 <20210216213112.20078-4-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224163116.GB10673@lst.de>

On Tue, Feb 16, 2021 at 01:31:09PM -0800, Chaitanya Kulkarni wrote:
> The function nvmet_execute_admin_connect() doesn't check for the right
> error status value that is return from the nvmet_alloc_ctrl().
> 
> Check for NVME_SC_CONNECT_INVALID_PARAM & NVME_SC_CONNECT_INVALID_HOST
> before we update the error log.
> 
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
> ---
>  drivers/nvme/target/fabrics-cmd.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
> index 42bd12b8bf00..cf0baa911db2 100644
> --- a/drivers/nvme/target/fabrics-cmd.c
> +++ b/drivers/nvme/target/fabrics-cmd.c
> @@ -191,9 +191,10 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req)
>  	status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req,
>  				  le32_to_cpu(c->kato), &ctrl);
>  	if (status) {
> -		if (status == (NVME_SC_INVALID_FIELD | NVME_SC_DNR))
> +		if (status == (NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR) ||
> +		    status == (NVME_SC_CONNECT_INVALID_HOST | NVME_SC_DNR))
>  			req->error_loc =
> -				offsetof(struct nvme_common_command, opcode);
> +				offsetof(struct nvme_common_command, dptr);

Can we just set req->error_loc inside of nvmet_alloc_ctrl?


From hch at lst.de  Wed Feb 24 11:32:16 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:32:16 +0100
Subject: [PATCH 0/6] nvmet: cleanup and status, error log fix
In-Reply-To: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
References: <20210216213112.20078-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224163216.GC10673@lst.de>

Except for the two comments this looks fine to me.


From hch at lst.de  Wed Feb 24 11:35:05 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:35:05 +0100
Subject: [PATCH 01/14] nvme-core: remove duplicate kfree in init identify
In-Reply-To: <20210217001032.25741-2-chaitanya.kulkarni@wdc.com>
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
 <20210217001032.25741-2-chaitanya.kulkarni@wdc.com>
Message-ID: <20210224163505.GD10673@lst.de>

On Tue, Feb 16, 2021 at 04:10:19PM -0800, Chaitanya Kulkarni wrote:
> In nvme_init_identify() once we initialize nvme_mpath_init() we free
> the id with kfree(). This is needed since we just return from all the
> following calls to nvme_mpath_init(), that also makes kfree() call
> duplicate.
> 
> Instead of returning after nvme_mpath_init() jump to the lable
> out_free and remove the duplicate call to the kfree() since out_free
> label already has one..

Hmm.  I'd rather split nvme_init_identify.  Rename the publically
called function to something like nvme_init_ctrl_finish, and
the split out a nvme_init_identify for just the section that deals
with the id_ns data.


From hch at lst.de  Wed Feb 24 11:44:43 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:44:43 +0100
Subject: [PATCH V5 1/2] nvme: enable char device per namespace
In-Reply-To: <20210222190107.8479-2-javier.gonz@samsung.com>
References: <20210222190107.8479-1-javier.gonz@samsung.com>
 <20210222190107.8479-2-javier.gonz@samsung.com>
Message-ID: <20210224164443.GA11338@lst.de>

> +static inline bool nvme_dev_is_generic(struct device *dev)
> +{
> +	return dev->class == nvme_ns_class;
> +}
> +
> +static inline bool nvme_ns_is_generic(struct nvme_ns *ns)
> +{
> +	return !!ns->minor;
> +}

What does is_generic mean here?  In doubt add a few comments..

>  	/* some standard values */
> @@ -2241,6 +2270,13 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_id_ns *id)
>  	return 0;
>  
>  out_unfreeze:
> +	/*
> +	 * When the device does not support any of the features required by the
> +	 * kernel (or viceversa), hide the block device. We can still rely on
> +	 * the namespace char device for submitting IOCTLs
> +	 */
> +	ns->disk->flags |= GENHD_FL_HIDDEN;
> +

The out_unfreeze case also handles all kinds of real error, so this needs
to move into a better spot, and probably check a specific error code
or even explicit indicator.

> +	struct nvme_ns *ns = container_of(inode->i_cdev, struct nvme_ns, cdev);
> +{
> +	struct nvme_ns *ns = container_of(inode->i_cdev, struct nvme_ns, cdev);

> +	struct nvme_ns *ns = container_of(file->f_inode->i_cdev,
> +				struct nvme_ns, cdev);

Maybe add a little cdev_to_ns() helper?

> -	if (nvme_update_ns_info(ns, id))
> -		goto out_put_disk;
> +	nvme_update_ns_info(ns, id);

I don't think we can simplify ignore all errors here.

> +static inline struct nvme_ns *nvme_get_ns_from_cdev(struct device *dev)
> +{
> +	return dev_get_drvdata(dev);
> +}

I think we can keep this in core.c.


From hch at lst.de  Wed Feb 24 11:45:23 2021
From: hch at lst.de (Christoph Hellwig)
Date: Wed, 24 Feb 2021 17:45:23 +0100
Subject: [PATCH V5 2/2] nvme: allow open for nvme-generic char device
In-Reply-To: <20210222190107.8479-3-javier.gonz@samsung.com>
References: <20210222190107.8479-1-javier.gonz@samsung.com>
 <20210222190107.8479-3-javier.gonz@samsung.com>
Message-ID: <20210224164523.GB11338@lst.de>

On Mon, Feb 22, 2021 at 08:01:07PM +0100, javier at javigon.com wrote:
> @@ -1885,7 +1885,7 @@ static int nvme_ns_open(struct nvme_ns *ns)
>  {
>  #ifdef CONFIG_NVME_MULTIPATH
>  	/* should never be called due to GENHD_FL_HIDDEN */
> -	if (WARN_ON_ONCE(ns->head->disk))
> +	if (WARN_ON_ONCE(!nvme_ns_is_generic(ns) && ns->head->disk))
>  		goto fail;

Maybe just move the check into the block device caller instead?


From hare at suse.de  Wed Feb 24 12:10:34 2021
From: hare at suse.de (Hannes Reinecke)
Date: Wed, 24 Feb 2021 18:10:34 +0100
Subject: [PATCH 0/2] nvme: fix regression with MD RAID
In-Reply-To: <20210224162534.GC9780@lst.de>
References: <20210223115922.104369-1-hare@suse.de>
 <20210224162534.GC9780@lst.de>
Message-ID: <ee9a6a42-a2b9-7076-a78d-6af9f6ecf4dc@suse.de>

On 2/24/21 5:25 PM, Christoph Hellwig wrote:
> I don't see any regression here, even if the new features sound useful.
> 
Have you ever tried MD RAID on nvme-of?
Without this patch MD RAID will _stop_ I/O until the controller reconnects.
If it does. If it doesn't, the controller gets removed (so after some 
300 seconds), and MD RAID will get an I/O error, finally.
But then you reconnect the failed path, and you end up with a 
_different_ nvme namespace device.
Requiring you to do manual handholding get the MD RAID into shape again.

With this patch it 'just works' without any interaction.

One might argue if that constitutes a regression (as it's been the 
behaviour since day 1), but it certainly is impaired functionality as 
compared to other drivers/subsystems like SCSI.
And we can't have that, can we?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From mark.wunderlich at intel.com  Wed Feb 24 13:00:41 2021
From: mark.wunderlich at intel.com (Wunderlich, Mark)
Date: Wed, 24 Feb 2021 18:00:41 +0000
Subject: [PATCH V2] nvmet-tcp: enable optional queue idle period tracking
Message-ID: <BYAPR11MB2869965F8A766860ADE9753EE59F9@BYAPR11MB2869.namprd11.prod.outlook.com>

Add 'queue idle period' option used by io_work() to support
network devices enabled with advanced interrupt moderation
supporting a relaxed interrupt model. It was discovered that
such a NIC used on the target was unable to support initiator
connection establishment, caused by the existing io_work()
flow that immediately exits and does not re-queue itself at
the first loop with no activity.

With this new option a queue is assigned a period of time
that no activity must occur in order to become 'idle'.  Until
the queue is idle the work item is requeued.

The new module option is defined as changeable making it
flexible for testing purposes.

The pre-existing legacy behavior is preserved when no module option
for queue idle period is specified.

Signed-off-by: Mark Wunderlich <mark.wunderlich at intel.com>
---
V2 of this patch removes the accounting of time deducted from the
idle deadline time period only during io_work activity.  The result
is a more simple solution, only requiring the selection of a
sufficient optional time period that will catch any non-idle activity
to keep a queue active.

Testing was performed with a NIC using standard HW interrupt mode, with
and without the new module option enabled.  No measurable performance
drop was seen when the patch wsa applied and the new option specified
or not.  A side effect of a standard NIC using the new option
will reduce the context switch rate.  We measured a drop from roughly
90K to less than 300 (for 32 active connections).

For a NIC using a passive advanced interrupt moderation policy, it was
then successfully able to achieve and maintain active connections with
the target.
---
 drivers/nvme/target/tcp.c |   55 +++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 53 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index dc1f0f647189..96b6c28e327b 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -29,6 +29,16 @@ static int so_priority;
 module_param(so_priority, int, 0644);
 MODULE_PARM_DESC(so_priority, "nvmet tcp socket optimize priority");
 
+/* Define a time period (in usecs) that io_work() shall sample an activated
+ * queue before determining it to be idle.  This optional module behavior
+ * can enable NIC solutions that support socket optimized packet processing
+ * using advanced interrupt moderation techniques.
+ */
+static int queue_idle_period_usecs;
+module_param(queue_idle_period_usecs, int, 0644);
+MODULE_PARM_DESC(queue_idle_period_usecs,
+		"nvmet tcp io_work queue idle time period in usecs");
+
 #define NVMET_TCP_RECV_BUDGET		8
 #define NVMET_TCP_SEND_BUDGET		8
 #define NVMET_TCP_IO_WORK_BUDGET	64
@@ -96,6 +106,7 @@ struct nvmet_tcp_queue {
 	struct work_struct	io_work;
 	struct nvmet_cq		nvme_cq;
 	struct nvmet_sq		nvme_sq;
+	unsigned long           idle_poll_deadline;
 
 	/* send state */
 	struct nvmet_tcp_cmd	*cmds;
@@ -1198,6 +1209,25 @@ static void nvmet_tcp_schedule_release_queue(struct nvmet_tcp_queue *queue)
 	spin_unlock(&queue->state_lock);
 }
 
+/*
+ * This worker function will process all send and recv packet
+ * activity for a queue. It will loop on the queue for up to a
+ * given maximum operation budget, or until there is no activity
+ * during a single loop iteration.
+ *
+ * Two exit modes are possible.
+ *
+ * The default 'pending' mode where the worker will re-queue
+ * itself, after exiting the work loop, only if any send or recv
+ * activity was recorded during the last pass within the loop.
+ *
+ * A optional 'idle period' mode where in addition to re-queueing
+ * itself because of activity it also tracks if a queue has not reached an
+ * assigned 'idle' deadline time period. The worker consumes from the assigned
+ * time period, across many potential invocations with no activity, until it
+ * has expired. Any activity during an invocation will trigger a fresh
+ * idle period deadline.
+ */
 static void nvmet_tcp_io_work(struct work_struct *w)
 {
 	struct nvmet_tcp_queue *queue =
@@ -1205,6 +1235,11 @@ static void nvmet_tcp_io_work(struct work_struct *w)
 	bool pending;
 	int ret, ops = 0;
 
+	if (queue_idle_period_usecs && queue->idle_poll_deadline == 0)
+		/* Assign the queue idle period deadline if not already set */
+		queue->idle_poll_deadline =
+			jiffies + usecs_to_jiffies(queue_idle_period_usecs);
+
 	do {
 		pending = false;
 
@@ -1222,8 +1257,24 @@ static void nvmet_tcp_io_work(struct work_struct *w)
 
 	} while (pending && ops < NVMET_TCP_IO_WORK_BUDGET);
 
-	/*
-	 * We exahusted our budget, requeue our selves
+	/* If optional deadline mode active, determine if queue has reached its
+	 * idle process deadline limit.  Any ops activity awards the queue a new
+	 * deadline period.
+	 */
+	if (queue_idle_period_usecs) {
+		/* Clear to award active non-idle queue new period, or
+		 * reset for future queue activity after exit when idle reached.
+		 */
+		if (!time_after(jiffies, queue->idle_poll_deadline)) {
+			pending = true;
+			if (ops > 0)
+				queue->idle_poll_deadline = 0;
+		} else
+			queue->idle_poll_deadline = 0;
+	}
+
+	/* We requeue ourself when pending indicates there was activity
+	 * recorded, or queue has not reached optional idle time period.
 	 */
 	if (pending)
 		queue_work_on(queue_cpu(queue), nvmet_tcp_wq, &queue->io_work);


From minwoo.im.dev at gmail.com  Wed Feb 24 13:44:41 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Thu, 25 Feb 2021 03:44:41 +0900
Subject: [PATCH V5 2/2] nvme: allow open for nvme-generic char device
In-Reply-To: <20210224164523.GB11338@lst.de>
References: <20210222190107.8479-1-javier.gonz@samsung.com>
 <20210222190107.8479-3-javier.gonz@samsung.com>
 <20210224164523.GB11338@lst.de>
Message-ID: <20210224184441.GA2626@localhost.localdomain>

On 21-02-24 17:45:23, Christoph Hellwig wrote:
> On Mon, Feb 22, 2021 at 08:01:07PM +0100, javier at javigon.com wrote:
> > @@ -1885,7 +1885,7 @@ static int nvme_ns_open(struct nvme_ns *ns)
> >  {
> >  #ifdef CONFIG_NVME_MULTIPATH
> >  	/* should never be called due to GENHD_FL_HIDDEN */
> > -	if (WARN_ON_ONCE(ns->head->disk))
> > +	if (WARN_ON_ONCE(!nvme_ns_is_generic(ns) && ns->head->disk))
> >  		goto fail;
> 
> Maybe just move the check into the block device caller instead?

Sure, then check for nvme_ns_is_generic(ns) will be not neccessary.  Let
me move this check to nvme_open() with only checking if it's head or not.


From javier at javigon.com  Wed Feb 24 15:28:43 2021
From: javier at javigon.com (Javier =?utf-8?B?R29uesOhbGV6?=)
Date: Wed, 24 Feb 2021 21:28:43 +0100
Subject: [PATCH V5 1/2] nvme: enable char device per namespace
In-Reply-To: <20210224164443.GA11338@lst.de>
References: <20210222190107.8479-1-javier.gonz@samsung.com>
 <20210222190107.8479-2-javier.gonz@samsung.com>
 <20210224164443.GA11338@lst.de>
Message-ID: <20210224202843.glv4c7uaifxzknmw@mpHalley.localdomain>

On 24.02.2021 17:44, Christoph Hellwig wrote:
>> +static inline bool nvme_dev_is_generic(struct device *dev)
>> +{
>> +	return dev->class == nvme_ns_class;
>> +}
>> +
>> +static inline bool nvme_ns_is_generic(struct nvme_ns *ns)
>> +{
>> +	return !!ns->minor;
>> +}
>
>What does is_generic mean here?  In doubt add a few comments..

Will do.

>
>>  	/* some standard values */
>> @@ -2241,6 +2270,13 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_id_ns *id)
>>  	return 0;
>>
>>  out_unfreeze:
>> +	/*
>> +	 * When the device does not support any of the features required by the
>> +	 * kernel (or viceversa), hide the block device. We can still rely on
>> +	 * the namespace char device for submitting IOCTLs
>> +	 */
>> +	ns->disk->flags |= GENHD_FL_HIDDEN;
>> +
>
>The out_unfreeze case also handles all kinds of real error, so this needs
>to move into a better spot, and probably check a specific error code
>or even explicit indicator.

Ok.

>
>> +	struct nvme_ns *ns = container_of(inode->i_cdev, struct nvme_ns, cdev);
>> +{
>> +	struct nvme_ns *ns = container_of(inode->i_cdev, struct nvme_ns, cdev);
>
>> +	struct nvme_ns *ns = container_of(file->f_inode->i_cdev,
>> +				struct nvme_ns, cdev);
>
>Maybe add a little cdev_to_ns() helper?

Ok.
>
>> -	if (nvme_update_ns_info(ns, id))
>> -		goto out_put_disk;
>> +	nvme_update_ns_info(ns, id);
>
>I don't think we can simplify ignore all errors here.

Sounds good.

>
>> +static inline struct nvme_ns *nvme_get_ns_from_cdev(struct device *dev)
>> +{
>> +	return dev_get_drvdata(dev);
>> +}
>
>I think we can keep this in core.c.

Perfect.

Will send a new version.


From Chaitanya.Kulkarni at wdc.com  Wed Feb 24 16:52:52 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 21:52:52 +0000
Subject: [PATCH 00/14] nvme: few cleanups and small improvements
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
 <20210224154229.GA26830@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <BYAPR04MB4965F8A3DC3ABB6A4FDA28AF869F9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/24/21 07:42, Keith Busch wrote:
> On Tue, Feb 16, 2021 at 04:10:18PM -0800, Chaitanya Kulkarni wrote:
>> Hi,
>>
>> This has few cleanups such as variable data type, removing oneliner
>> wrappers for single callers, removing duplicate kfree() and small
>> improvements for NVMeOF Passthru fast path such as likely annotation,
>> removing the extra checks, making function inline for host/core.c.
>> Last couple of patches fixes warning for FC function header
>> documentation.
> The two documentation fixes at the end look fine, but much of the rest
> looks unnecessary. 
>  
Thanks for the feedback Keith.



From Chaitanya.Kulkarni at wdc.com  Wed Feb 24 17:35:49 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 22:35:49 +0000
Subject: [PATCH 01/14] nvme-core: remove duplicate kfree in init identify
References: <20210217001032.25741-1-chaitanya.kulkarni@wdc.com>
 <20210217001032.25741-2-chaitanya.kulkarni@wdc.com>
 <20210224163505.GD10673@lst.de>
Message-ID: <BYAPR04MB49659BEC727069DD470A18B8869F9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/24/21 08:35, Christoph Hellwig wrote:
> On Tue, Feb 16, 2021 at 04:10:19PM -0800, Chaitanya Kulkarni wrote:
>> In nvme_init_identify() once we initialize nvme_mpath_init() we free
>> the id with kfree(). This is needed since we just return from all the
>> following calls to nvme_mpath_init(), that also makes kfree() call
>> duplicate.
>>
>> Instead of returning after nvme_mpath_init() jump to the lable
>> out_free and remove the duplicate call to the kfree() since out_free
>> label already has one..
> Hmm.  I'd rather split nvme_init_identify.  Rename the publically
> called function to something like nvme_init_ctrl_finish, and
> the split out a nvme_init_identify for just the section that deals
> with the id_ns data.
>

How about following patch in [1] ? If you like [1] please suggest
the patch distribution. I'd add a prep patch to rename and second patch
to split the helper.

Shorten the original list to avoid churn, along with the documentation fixes
planning to keep following in V2, is that okay:-

  nvme-core: mark nvme_setup_passsthru() inline
  nvme-core: use likely in nvme_init_request()
  nvme-core: don't check nvme_req flags for new req
  nvme-fc: fix the function documentation comment
  nvmet-fc: update function documentation


[1] nvme-core: split out id_ns into helper

>From 4b4ec36b0c660fc54e22aa4d712db1b595b2e90d Mon Sep 17 00:00:00 2001
From: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
Date: Wed, 24 Feb 2021 14:14:05 -0800
Subject: [PATCH] nvme-core: split out id_ns into helper

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c   | 65 ++++++++++++++++++++------------------
 drivers/nvme/host/fc.c     |  2 +-
 drivers/nvme/host/nvme.h   |  2 +-
 drivers/nvme/host/pci.c    |  2 +-
 drivers/nvme/host/rdma.c   |  2 +-
 drivers/nvme/host/tcp.c    |  2 +-
 drivers/nvme/target/loop.c |  2 +-
 7 files changed, 41 insertions(+), 36 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e68a8c4ac5a6..68b0022bc60d 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1119,7 +1119,7 @@ static void nvme_passthru_end(struct nvme_ctrl
*ctrl, u32 effects)
         mutex_unlock(&ctrl->scan_lock);
     }
     if (effects & NVME_CMD_EFFECTS_CCC)
-        nvme_init_identify(ctrl);
+        nvme_init_ctrl_finish(ctrl);
     if (effects & (NVME_CMD_EFFECTS_NIC | NVME_CMD_EFFECTS_NCC)) {
         nvme_queue_scan(ctrl);
         flush_work(&ctrl->scan_work);
@@ -1978,7 +1978,7 @@ static void nvme_config_write_zeroes(struct
gendisk *disk, struct nvme_ns *ns)
      * In order to be more cautious use controller's max_hw_sectors value
      * to configure the maximum sectors for the write-zeroes which is
      * configured based on the controller's MDTS field in the
-     * nvme_init_identify() if available.
+     * nvme_init_ctrl_finish() if available.
      */
     if (ns->ctrl->max_hw_sectors == UINT_MAX)
         max_blocks = (u64)USHRT_MAX + 1;
@@ -3059,28 +3059,12 @@ static int nvme_get_effects_log(struct nvme_ctrl
*ctrl, u8 csi,
     return 0;
 }
 
-/*
- * Initialize the cached copies of the Identify data and various controller
- * register in our nvme_ctrl structure.  This should be called as soon as
- * the admin queue is fully up and running.
- */
 int nvme_init_identify(struct nvme_ctrl *ctrl)
 {
     struct nvme_id_ctrl *id;
+    bool prev_apst_enabled;
     int ret, page_shift;
     u32 max_hw_sectors;
-    bool prev_apst_enabled;
-
-    ret = ctrl->ops->reg_read32(ctrl, NVME_REG_VS, &ctrl->vs);
-    if (ret) {
-        dev_err(ctrl->device, "Reading VS failed (%d)\n", ret);
-        return ret;
-    }
-    page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12;
-    ctrl->sqsize = min_t(u16, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
-
-    if (ctrl->vs >= NVME_VS(1, 1, 0))
-        ctrl->subsystem = NVME_CAP_NSSRC(ctrl->cap);
 
     ret = nvme_identify_ctrl(ctrl, &id);
     if (ret) {
@@ -3098,7 +3082,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
         ctrl->cntlid = le16_to_cpu(id->cntlid);
 
     if (!ctrl->identified) {
-        int i;
+        unsigned int i;
 
         ret = nvme_init_subsystem(ctrl, id);
         if (ret)
@@ -3136,6 +3120,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 
     atomic_set(&ctrl->abort_limit, id->acl + 1);
     ctrl->vwc = id->vwc;
+    page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12;
     if (id->mdts)
         max_hw_sectors = 1 << (id->mdts + page_shift - 9);
     else
@@ -3210,16 +3195,40 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
         ctrl->hmmaxd = le16_to_cpu(id->hmmaxd);
     }
 
+    if (ctrl->apst_enabled && !prev_apst_enabled)
+        dev_pm_qos_expose_latency_tolerance(ctrl->device);
+    else if (!ctrl->apst_enabled && prev_apst_enabled)
+        dev_pm_qos_hide_latency_tolerance(ctrl->device);
+
     ret = nvme_mpath_init(ctrl, id);
+
+out_free:
     kfree(id);
+    return ret;
+}
 
-    if (ret < 0)
+/*
+ * Initialize the cached copies of the Identify data and various controller
+ * register in our nvme_ctrl structure.  This should be called as soon as
+ * the admin queue is fully up and running.
+ */
+int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl)
+{
+    int ret;
+
+    ret = ctrl->ops->reg_read32(ctrl, NVME_REG_VS, &ctrl->vs);
+    if (ret) {
+        dev_err(ctrl->device, "Reading VS failed (%d)\n", ret);
         return ret;
+    }
+    ctrl->sqsize = min_t(u16, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
 
-    if (ctrl->apst_enabled && !prev_apst_enabled)
-        dev_pm_qos_expose_latency_tolerance(ctrl->device);
-    else if (!ctrl->apst_enabled && prev_apst_enabled)
-        dev_pm_qos_hide_latency_tolerance(ctrl->device);
+    if (ctrl->vs >= NVME_VS(1, 1, 0))
+        ctrl->subsystem = NVME_CAP_NSSRC(ctrl->cap);
+
+    ret = nvme_init_ctrl_finish(ctrl);
+    if (ret < 0)
+        return ret;
 
     ret = nvme_configure_apst(ctrl);
     if (ret < 0)
@@ -3246,12 +3255,8 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
     ctrl->identified = true;
 
     return 0;
-
-out_free:
-    kfree(id);
-    return ret;
 }
-EXPORT_SYMBOL_GPL(nvme_init_identify);
+EXPORT_SYMBOL_GPL(nvme_init_ctrl_finish);
 
 static int nvme_dev_open(struct inode *inode, struct file *file)
 {
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 20dadd86e981..962987a330d6 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3085,7 +3085,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 
     blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
-    ret = nvme_init_identify(&ctrl->ctrl);
+    ret = nvme_init_ctrl_finish(&ctrl->ctrl);
     if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
         goto out_disconnect_admin_queue;
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..624fc4334a2c 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -599,7 +599,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct
device *dev,
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl);
 void nvme_start_ctrl(struct nvme_ctrl *ctrl);
 void nvme_stop_ctrl(struct nvme_ctrl *ctrl);
-int nvme_init_identify(struct nvme_ctrl *ctrl);
+int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl);
 
 void nvme_remove_namespaces(struct nvme_ctrl *ctrl);
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 207137a0ed8e..b8f0bfaa8f2a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2652,7 +2652,7 @@ static void nvme_reset_work(struct work_struct *work)
      */
     dev->ctrl.max_integrity_segments = 1;
 
-    result = nvme_init_identify(&dev->ctrl);
+    result = nvme_init_ctrl_finish(&dev->ctrl);
     if (result)
         goto out;
 
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 53ac4d7442ba..9c710839b03a 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -917,7 +917,7 @@ static int nvme_rdma_configure_admin_queue(struct
nvme_rdma_ctrl *ctrl,
 
     blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
-    error = nvme_init_identify(&ctrl->ctrl);
+    error = nvme_init_ctrl_finish(&ctrl->ctrl);
     if (error)
         goto out_quiesce_queue;
 
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 69f59d2c5799..735e768f9f43 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1875,7 +1875,7 @@ static int nvme_tcp_configure_admin_queue(struct
nvme_ctrl *ctrl, bool new)
 
     blk_mq_unquiesce_queue(ctrl->admin_q);
 
-    error = nvme_init_identify(ctrl);
+    error = nvme_init_ctrl_finish(ctrl);
     if (error)
         goto out_quiesce_queue;
 
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..a7f97c8b2f77 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -396,7 +396,7 @@ static int nvme_loop_configure_admin_queue(struct
nvme_loop_ctrl *ctrl)
 
     blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
-    error = nvme_init_identify(&ctrl->ctrl);
+    error = nvme_init_ctrl_finish(&ctrl->ctrl);
     if (error)
         goto out_cleanup_queue;
 
-- 
2.22.1





From sagi at grimberg.me  Wed Feb 24 17:40:50 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 24 Feb 2021 14:40:50 -0800
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <20210223115922.104369-3-hare@suse.de>
References: <20210223115922.104369-1-hare@suse.de>
 <20210223115922.104369-3-hare@suse.de>
Message-ID: <eb979539-96fb-b767-5c5a-259263a25a0b@grimberg.me>


> The multipath code currently deletes the disk only after all references
> to it are dropped rather than when the last path to that disk is lost.
> This has been reported to cause problems with some use cases like MD RAID.

What is the exact problem?

Can you describe what the problem you see now and what you expect
to see (unrelated to patch #1)?

> This patch implements an alternative behaviour of deleting the disk when
> the last path is gone, ie the same behaviour as non-multipathed nvme
> devices.

But we also don't remove the non-multipath'd nvme device until the
last reference drops (e.g. if you have a mounted filesystem on top).

This would be the equivalent to running raid on top of dm-mpath on
top of scsi devices right? And if all the mpath device nodes go away
the mpath device is deleted even if it has an open reference to it?

> The new behaviour will be selected with the 'fail_if_no_path'
> attribute, as returning it's arguably the same functionality.

But its not the same functionality.


From sagi at grimberg.me  Wed Feb 24 17:47:06 2021
From: sagi at grimberg.me (Sagi Grimberg)
Date: Wed, 24 Feb 2021 14:47:06 -0800
Subject: [PATCH 1/2] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <20210223115922.104369-2-hare@suse.de>
References: <20210223115922.104369-1-hare@suse.de>
 <20210223115922.104369-2-hare@suse.de>
Message-ID: <303e7252-cb51-d6e2-eb4c-ad70fb3528a3@grimberg.me>



On 2/23/21 3:59 AM, Hannes Reinecke wrote:
> In some setups like RAID or cluster we need to return an I/O error
> once all paths are unavailable to allow the upper layers to start
> their own error recovery (like redirecting I/O to other mirrors).
> This patch adds a sysfs attribute 'fail_if_no_path' to allow the
> admin to enable that behaviour instead of the current 'queue until
> a path becomes available' policy.

Doesn't the same happen today if all the controllers are set with
fail_io_fast_tmo=0? nvme_available_path will return false if all
the paths have NVME_CTRL_FAILFAST_EXPIRED set.

I think that fail_io_fast_tmo should be settable via sysfs and I
think I requested that during the various submission iterations...

I'm not sure this should be controlled on the individual namespace level...


From chaitanya.kulkarni at wdc.com  Wed Feb 24 20:56:36 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 17:56:36 -0800
Subject: [PATCH V2 0/6] nvmet: cleanup and status, error log fix 
Message-ID: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>

Hi,

This removes duplicate status value assignments, unnecessary function
parameters and fixes the error log page update. Last patch also
removes the spaces where tabs are expected.

-ck

V1->V2:

1. Remove the patch to set the status variable under if at the error.
2. Move the error log page update into nvmet_alloc_ctrl().
3. Add a patch to replace the spaces with tabs.

Chaitanya Kulkarni (6):
  nvmet: remove duplicate status assignment
  nvmet: update error log page in nvmet_alloc_ctrl()
  nvmet: remove unnecessary function parameters
  nvmet: remove unnecessary function parameter
  nvmet: remove unnecessary function parameters
  nvmet: replace white spaces with tabs

 drivers/nvme/target/admin-cmd.c   |  4 +--
 drivers/nvme/target/core.c        | 58 +++++++++++++++----------------
 drivers/nvme/target/fabrics-cmd.c | 19 ++++------
 drivers/nvme/target/nvmet.h       | 10 +++---
 4 files changed, 43 insertions(+), 48 deletions(-)

# ./check tests/nvme/
nvme/002 (create many subsystems and test discovery)         [passed]
    runtime  36.789s  ...  37.383s
nvme/003 (test if we're sending keep-alives to a discovery controller) [passed]
    runtime  10.177s  ...  10.164s
nvme/004 (test nvme and nvmet UUID NS descriptors)           [passed]
    runtime  1.743s  ...  1.739s
nvme/005 (reset local loopback target)                       [not run]
    nvme_core module does not have parameter multipath
nvme/006 (create an NVMeOF target with a block device-backed ns) [passed]
    runtime  0.141s  ...  0.137s
nvme/007 (create an NVMeOF target with a file-backed ns)     [passed]
    runtime  0.096s  ...  0.092s
nvme/008 (create an NVMeOF host with a block device-backed ns) [passed]
    runtime  1.756s  ...  1.760s
nvme/009 (create an NVMeOF host with a file-backed ns)       [passed]
    runtime  1.709s  ...  1.732s
nvme/010 (run data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  19.931s  ...  29.821s
nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  267.604s  ...  282.232s
nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  49.320s  ...  50.911s
nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  335.511s  ...  292.379s
nvme/014 (flush a NVMeOF block device-backed ns)             [passed]
    runtime  21.308s  ...  20.296s
nvme/015 (unit test for NVMe flush for file backed ns)       [passed]
    runtime  21.888s  ...  18.687s
nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [passed]
    runtime  19.097s  ...  19.649s
nvme/017 (create/delete many file-ns and test discovery)     [passed]
    runtime  19.393s  ...  19.435s
nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed]
    runtime  1.718s  ...  1.729s
nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed]
    runtime  1.747s  ...  1.786s
nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed]
    runtime  1.700s  ...  1.704s
nvme/021 (test NVMe list command on NVMeOF file-backed ns)   [passed]
    runtime  1.702s  ...  1.745s
nvme/022 (test NVMe reset command on NVMeOF file-backed ns)  [passed]
    runtime  2.131s  ...  2.128s
nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed]
    runtime  1.739s  ...  1.761s
nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed]
    runtime  1.708s  ...  1.717s
nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed]
    runtime  1.692s  ...  1.703s
nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed]
    runtime  1.798s  ...  1.716s
nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed]
    runtime  1.713s  ...  1.716s
nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed]
    runtime  1.695s  ...  1.723s
nvme/029 (test userspace IO via nvme-cli read/write interface) [passed]
    runtime  2.168s  ...  2.148s
nvme/030 (ensure the discovery generation counter is updated appropriately) [passed]
    runtime  0.379s  ...  0.381s
nvme/031 (test deletion of NVMeOF controllers immediately after setup) [passed]
    runtime  5.774s  ...  5.747s
nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed]
    runtime  0.051s  ...  0.059s

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 24 20:56:37 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 17:56:37 -0800
Subject: [PATCH V2 1/6] nvmet: remove duplicate status assignment
In-Reply-To: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
References: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210225015642.29288-2-chaitanya.kulkarni@wdc.com>

In the function nvmet_alloc_ctrl() we assign status value before we
call nvmet_fine_get_subsys() to :-
status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;

After we successfully find the subsystem we again set the status value
to :-
status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;

Remove the duplicate status assignment value.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index be6fcdaf51a7..e3b8ec535eb4 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1314,7 +1314,6 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 		goto out;
 	}
 
-	status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 	down_read(&nvmet_config_sem);
 	if (!nvmet_host_allowed(subsys, hostnqn)) {
 		pr_info("connect by host %s for subsystem %s not allowed\n",
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 24 20:56:38 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 17:56:38 -0800
Subject: [PATCH V2 2/6] nvmet: update error log page in nvmet_alloc_ctrl()
In-Reply-To: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
References: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210225015642.29288-3-chaitanya.kulkarni@wdc.com>

Instead of updating the error log page in the caller of the
nvmet_alloc_ctrt() update the error log page in the nvmet_alloc_ctrl().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c        | 2 ++
 drivers/nvme/target/fabrics-cmd.c | 6 +-----
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index e3b8ec535eb4..c4238c08e912 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1311,6 +1311,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 		pr_warn("connect request for invalid subsystem %s!\n",
 			subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(subsysnqn);
+		req->error_loc = offsetof(struct nvme_common_command, dptr);
 		goto out;
 	}
 
@@ -1321,6 +1322,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(hostnqn);
 		up_read(&nvmet_config_sem);
 		status = NVME_SC_CONNECT_INVALID_HOST | NVME_SC_DNR;
+		req->error_loc = offsetof(struct nvme_common_command, dptr);
 		goto out_put_subsystem;
 	}
 	up_read(&nvmet_config_sem);
diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
index 42bd12b8bf00..d2289aa26645 100644
--- a/drivers/nvme/target/fabrics-cmd.c
+++ b/drivers/nvme/target/fabrics-cmd.c
@@ -190,12 +190,8 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req)
 
 	status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req,
 				  le32_to_cpu(c->kato), &ctrl);
-	if (status) {
-		if (status == (NVME_SC_INVALID_FIELD | NVME_SC_DNR))
-			req->error_loc =
-				offsetof(struct nvme_common_command, opcode);
+	if (status)
 		goto out;
-	}
 
 	ctrl->pi_support = ctrl->port->pi_enable && ctrl->subsys->pi_support;
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 24 20:56:39 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 17:56:39 -0800
Subject: [PATCH V2 3/6] nvmet: remove unnecessary function parameters
In-Reply-To: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
References: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210225015642.29288-4-chaitanya.kulkarni@wdc.com>

The function nvmet_ctrl_find_get() accepts subsysnqn, hostnqn, cntlid,
nvmet_req, and out pointer to nvmet_ctrl structure. The parameters
subsysnqn, hostnqn and cntlid can be derived from the caller's
struct nvmf_connect_data.

Replace these parameters with structure pointer nvmf_connect_data *d.

Also, this function returns the same error value from two places. First
on failure to find subsystem from connect data and failure to find ctrl
from cntlid connect data:- NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR.

Move this to the caller to change the return type so we can return ctrl.

Now that we can change the return type instead of taking the pointer to
pointer to the nvmet_ctrl structure remove that function parameter and
return the valid nvmet_ctrl pointer on success and NULL on failure.

Also, add and rename the goto labels for more readability with comment.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c        | 30 +++++++++++++++---------------
 drivers/nvme/target/fabrics-cmd.c | 10 +++++-----
 drivers/nvme/target/nvmet.h       |  4 ++--
 3 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index c4238c08e912..db55f3a6f140 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1179,45 +1179,45 @@ static void nvmet_init_cap(struct nvmet_ctrl *ctrl)
 	ctrl->cap |= NVMET_QUEUE_SIZE - 1;
 }
 
-u16 nvmet_ctrl_find_get(const char *subsysnqn, const char *hostnqn, u16 cntlid,
-		struct nvmet_req *req, struct nvmet_ctrl **ret)
+struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
+				       struct nvmet_req *req)
 {
+	struct nvmet_ctrl *ctrl = NULL;
 	struct nvmet_subsys *subsys;
-	struct nvmet_ctrl *ctrl;
-	u16 status = 0;
 
-	subsys = nvmet_find_get_subsys(req->port, subsysnqn);
+	subsys = nvmet_find_get_subsys(req->port, d->subsysnqn);
 	if (!subsys) {
 		pr_warn("connect request for invalid subsystem %s!\n",
-			subsysnqn);
+			d->subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(subsysnqn);
-		return NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
+		goto out;
 	}
 
 	mutex_lock(&subsys->lock);
 	list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
-		if (ctrl->cntlid == cntlid) {
-			if (strncmp(hostnqn, ctrl->hostnqn, NVMF_NQN_SIZE)) {
+		if (ctrl->cntlid == le16_to_cpu(d->cntlid)) {
+			if (strncmp(d->hostnqn, ctrl->hostnqn, NVMF_NQN_SIZE)) {
 				pr_warn("hostnqn mismatch.\n");
 				continue;
 			}
 			if (!kref_get_unless_zero(&ctrl->ref))
 				continue;
 
-			*ret = ctrl;
-			goto out;
+			/* ctrl found */
+			goto found;
 		}
 	}
 
+	ctrl = NULL; /* ctrl not found */
 	pr_warn("could not find controller %d for subsys %s / host %s\n",
-		cntlid, subsysnqn, hostnqn);
+		le16_to_cpu(d->cntlid), d->subsysnqn, d->hostnqn);
 	req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(cntlid);
-	status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 
-out:
+found:
 	mutex_unlock(&subsys->lock);
 	nvmet_subsys_put(subsys);
-	return status;
+out:
+	return ctrl;
 }
 
 u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd)
diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
index d2289aa26645..acbb8d1c402c 100644
--- a/drivers/nvme/target/fabrics-cmd.c
+++ b/drivers/nvme/target/fabrics-cmd.c
@@ -218,7 +218,7 @@ static void nvmet_execute_io_connect(struct nvmet_req *req)
 {
 	struct nvmf_connect_command *c = &req->cmd->connect;
 	struct nvmf_connect_data *d;
-	struct nvmet_ctrl *ctrl = NULL;
+	struct nvmet_ctrl *ctrl;
 	u16 qid = le16_to_cpu(c->qid);
 	u16 status = 0;
 
@@ -245,11 +245,11 @@ static void nvmet_execute_io_connect(struct nvmet_req *req)
 		goto out;
 	}
 
-	status = nvmet_ctrl_find_get(d->subsysnqn, d->hostnqn,
-				     le16_to_cpu(d->cntlid),
-				     req, &ctrl);
-	if (status)
+	ctrl = nvmet_ctrl_find_get(d, req);
+	if (!ctrl) {
+		status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
 		goto out;
+	}
 
 	if (unlikely(qid > ctrl->subsys->max_qid)) {
 		pr_warn("invalid queue id (%d)\n", qid);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 4b84edb49f22..2938eca5378a 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -428,8 +428,8 @@ void nvmet_ctrl_fatal_error(struct nvmet_ctrl *ctrl);
 void nvmet_update_cc(struct nvmet_ctrl *ctrl, u32 new);
 u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 		struct nvmet_req *req, u32 kato, struct nvmet_ctrl **ctrlp);
-u16 nvmet_ctrl_find_get(const char *subsysnqn, const char *hostnqn, u16 cntlid,
-		struct nvmet_req *req, struct nvmet_ctrl **ret);
+struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
+				       struct nvmet_req *req);
 void nvmet_ctrl_put(struct nvmet_ctrl *ctrl);
 u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd);
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 24 20:56:40 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 17:56:40 -0800
Subject: [PATCH V2 4/6] nvmet: remove unnecessary function parameter
In-Reply-To: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
References: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210225015642.29288-5-chaitanya.kulkarni@wdc.com>

In nvmet_check_ctrl_status() cmd can be derived from nvmet_req. Remove
the local variable cmd in the nvmet_check_ctrl_status() and function
parameter cmd for nvmet_check_ctrl_status(). Derive the cmd value from
req parameter in the nvmet_check_ctrl_status().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 2 +-
 drivers/nvme/target/core.c      | 9 ++++-----
 drivers/nvme/target/nvmet.h     | 2 +-
 3 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index fe6b8aa90b53..16a3e434f52e 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -940,7 +940,7 @@ u16 nvmet_parse_admin_cmd(struct nvmet_req *req)
 	if (nvmet_req_subsys(req)->type == NVME_NQN_DISC)
 		return nvmet_parse_discovery_cmd(req);
 
-	ret = nvmet_check_ctrl_status(req, cmd);
+	ret = nvmet_check_ctrl_status(req);
 	if (unlikely(ret))
 		return ret;
 
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index db55f3a6f140..4dce09000b3a 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -864,10 +864,9 @@ static inline u16 nvmet_io_cmd_check_access(struct nvmet_req *req)
 
 static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 {
-	struct nvme_command *cmd = req->cmd;
 	u16 ret;
 
-	ret = nvmet_check_ctrl_status(req, cmd);
+	ret = nvmet_check_ctrl_status(req);
 	if (unlikely(ret))
 		return ret;
 
@@ -1220,17 +1219,17 @@ struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
 	return ctrl;
 }
 
-u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd)
+u16 nvmet_check_ctrl_status(struct nvmet_req *req)
 {
 	if (unlikely(!(req->sq->ctrl->cc & NVME_CC_ENABLE))) {
 		pr_err("got cmd %d while CC.EN == 0 on qid = %d\n",
-		       cmd->common.opcode, req->sq->qid);
+		       req->cmd->common.opcode, req->sq->qid);
 		return NVME_SC_CMD_SEQ_ERROR | NVME_SC_DNR;
 	}
 
 	if (unlikely(!(req->sq->ctrl->csts & NVME_CSTS_RDY))) {
 		pr_err("got cmd %d while CSTS.RDY == 0 on qid = %d\n",
-		       cmd->common.opcode, req->sq->qid);
+		       req->cmd->common.opcode, req->sq->qid);
 		return NVME_SC_CMD_SEQ_ERROR | NVME_SC_DNR;
 	}
 	return 0;
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 2938eca5378a..9a445fab3547 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -431,7 +431,7 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
 				       struct nvmet_req *req);
 void nvmet_ctrl_put(struct nvmet_ctrl *ctrl);
-u16 nvmet_check_ctrl_status(struct nvmet_req *req, struct nvme_command *cmd);
+u16 nvmet_check_ctrl_status(struct nvmet_req *req);
 
 struct nvmet_subsys *nvmet_subsys_alloc(const char *subsysnqn,
 		enum nvme_subsys_type type);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 24 20:56:41 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 17:56:41 -0800
Subject: [PATCH V2 5/6] nvmet: remove unnecessary function parameters
In-Reply-To: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
References: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210225015642.29288-6-chaitanya.kulkarni@wdc.com>

The function nvmet_alloc_ctrl() accepts subsysnqn, hostnqn, cntlid,
nvmet_req, and out pointer to nvmet_ctrl structure. The parameters
subsysnqn, hostnqn and cntlid can be derived from the caller's
struct nvmf_connect_data.

Replace these parameters with structure pointer nvmf_connect_data *d.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/core.c        | 16 ++++++++--------
 drivers/nvme/target/fabrics-cmd.c |  3 +--
 drivers/nvme/target/nvmet.h       |  4 ++--
 3 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 4dce09000b3a..c64f0322c757 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1296,8 +1296,8 @@ static void nvmet_fatal_error_handler(struct work_struct *work)
 	ctrl->ops->delete_ctrl(ctrl);
 }
 
-u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
-		struct nvmet_req *req, u32 kato, struct nvmet_ctrl **ctrlp)
+u16 nvmet_alloc_ctrl(struct nvmf_connect_data *d, struct nvmet_req *req,
+		     u32 kato, struct nvmet_ctrl **ctrlp)
 {
 	struct nvmet_subsys *subsys;
 	struct nvmet_ctrl *ctrl;
@@ -1305,19 +1305,19 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 	u16 status;
 
 	status = NVME_SC_CONNECT_INVALID_PARAM | NVME_SC_DNR;
-	subsys = nvmet_find_get_subsys(req->port, subsysnqn);
+	subsys = nvmet_find_get_subsys(req->port, d->subsysnqn);
 	if (!subsys) {
 		pr_warn("connect request for invalid subsystem %s!\n",
-			subsysnqn);
+			d->subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(subsysnqn);
 		req->error_loc = offsetof(struct nvme_common_command, dptr);
 		goto out;
 	}
 
 	down_read(&nvmet_config_sem);
-	if (!nvmet_host_allowed(subsys, hostnqn)) {
+	if (!nvmet_host_allowed(subsys, d->hostnqn)) {
 		pr_info("connect by host %s for subsystem %s not allowed\n",
-			hostnqn, subsysnqn);
+			d->hostnqn, d->subsysnqn);
 		req->cqe->result.u32 = IPO_IATTR_CONNECT_DATA(hostnqn);
 		up_read(&nvmet_config_sem);
 		status = NVME_SC_CONNECT_INVALID_HOST | NVME_SC_DNR;
@@ -1341,8 +1341,8 @@ u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
 	INIT_RADIX_TREE(&ctrl->p2p_ns_map, GFP_KERNEL);
 	INIT_WORK(&ctrl->fatal_err_work, nvmet_fatal_error_handler);
 
-	memcpy(ctrl->subsysnqn, subsysnqn, NVMF_NQN_SIZE);
-	memcpy(ctrl->hostnqn, hostnqn, NVMF_NQN_SIZE);
+	memcpy(ctrl->subsysnqn, d->subsysnqn, NVMF_NQN_SIZE);
+	memcpy(ctrl->hostnqn, d->hostnqn, NVMF_NQN_SIZE);
 
 	kref_init(&ctrl->ref);
 	ctrl->subsys = subsys;
diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
index acbb8d1c402c..b41748d8edf3 100644
--- a/drivers/nvme/target/fabrics-cmd.c
+++ b/drivers/nvme/target/fabrics-cmd.c
@@ -188,8 +188,7 @@ static void nvmet_execute_admin_connect(struct nvmet_req *req)
 		goto out;
 	}
 
-	status = nvmet_alloc_ctrl(d->subsysnqn, d->hostnqn, req,
-				  le32_to_cpu(c->kato), &ctrl);
+	status = nvmet_alloc_ctrl(d, req, le32_to_cpu(c->kato), &ctrl);
 	if (status)
 		goto out;
 
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 9a445fab3547..87a0c528a107 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -426,8 +426,8 @@ int nvmet_sq_init(struct nvmet_sq *sq);
 void nvmet_ctrl_fatal_error(struct nvmet_ctrl *ctrl);
 
 void nvmet_update_cc(struct nvmet_ctrl *ctrl, u32 new);
-u16 nvmet_alloc_ctrl(const char *subsysnqn, const char *hostnqn,
-		struct nvmet_req *req, u32 kato, struct nvmet_ctrl **ctrlp);
+u16 nvmet_alloc_ctrl(struct nvmf_connect_data *d, struct nvmet_req *req,
+		     u32 kato, struct nvmet_ctrl **ctrlp);
 struct nvmet_ctrl *nvmet_ctrl_find_get(struct nvmf_connect_data *d,
 				       struct nvmet_req *req);
 void nvmet_ctrl_put(struct nvmet_ctrl *ctrl);
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Wed Feb 24 20:56:42 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Wed, 24 Feb 2021 17:56:42 -0800
Subject: [PATCH V2 6/6] nvmet: replace white spaces with tabs
In-Reply-To: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
References: <20210225015642.29288-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210225015642.29288-7-chaitanya.kulkarni@wdc.com>

Instead of the using the whitespaces use tab spacing in the
nvmet_execute_identify_ns().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/admin-cmd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 16a3e434f52e..f4cc32674edd 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -513,7 +513,7 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req)
 	default:
 		id->nuse = id->nsze;
 		break;
-        }
+	}
 
 	if (req->ns->bdev)
 		nvmet_bdev_set_limits(req->ns->bdev, id);
-- 
2.22.1



From Chaitanya.Kulkarni at wdc.com  Wed Feb 24 21:09:33 2021
From: Chaitanya.Kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Thu, 25 Feb 2021 02:09:33 +0000
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes
 command
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <YCOxjZhBTBchOq0n@infradead.org>
 <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
 <20210210131557.GB29904@lst.de>
 <BYAPR04MB49650A271CD1C1B3EFA72F3D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210211070748.GA13852@lst.de>
 <BYAPR04MB4965C95C3F826D7222EB3176868C9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210224091326.GA5159@lst.de>
Message-ID: <BYAPR04MB496509F7429C820F6A8DF37F869E9@BYAPR04MB4965.namprd04.prod.outlook.com>

On 2/24/21 01:13, Christoph Hellwig wrote:
> On Thu, Feb 11, 2021 at 06:11:19PM +0000, Chaitanya Kulkarni wrote:
>> On 2/10/21 11:07 PM, Christoph Hellwig wrote:
>>>>> I'd also love to retest most Write Zeroes quirks with that in place.
>>>> Do you prefer some variant of the following patch (totally untested)? OR
>>>> something else ?
>>> Somwhat.  As said I suspect defaulting to MDTS with a big fat comment
>>> might make most sense unless the new WZSL is set.
>>>
>> Okay, will send out patche(s) soon.
> Did you get to this?  Or did I just miss it?
>
I sent out WIP to Gopal on 02/13 due to lack of drive.

Not sure if makes sense to start the review where patch is
not tested on the real H/w, here it is :-

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e68a8c4ac5a6..62c928fb0ded 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1965,6 +1965,7 @@ static void nvme_config_discard(struct gendisk
*disk, struct nvme_ns *ns)
 
 static void nvme_config_write_zeroes(struct gendisk *disk, struct
nvme_ns *ns)
 {
+       bool use_mdts = ns->ctrl->quirks & NVME_QUIRK_USE_MDTS_WRITE_ZEROES;
        u64 max_blocks;
 
        if (!(ns->ctrl->oncs & NVME_CTRL_ONCS_WRITE_ZEROES) ||
@@ -1980,7 +1981,9 @@ static void nvme_config_write_zeroes(struct
gendisk *disk, struct nvme_ns *ns)
         * configured based on the controller's MDTS field in the
         * nvme_init_identify() if available.
         */
-       if (ns->ctrl->max_hw_sectors == UINT_MAX)
+       if (use_mdts && ns->ctrl->max_mdts_hw_sectors)
+               max_blocks = ns->ctrl->max_mdts_hw_sectors + 1;
+       else if (ns->ctrl->max_hw_sectors == UINT_MAX)
                max_blocks = (u64)USHRT_MAX + 1;
        else
                max_blocks = ns->ctrl->max_hw_sectors + 1;
@@ -3136,10 +3139,10 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 
        atomic_set(&ctrl->abort_limit, id->acl + 1);
        ctrl->vwc = id->vwc;
+       ctrl->max_mdts_hw_sectors = 0;
+       max_hw_sectors = UINT_MAX;
        if (id->mdts)
                max_hw_sectors = 1 << (id->mdts + page_shift - 9);
-       else
-               max_hw_sectors = UINT_MAX;
        ctrl->max_hw_sectors =
                min_not_zero(ctrl->max_hw_sectors, max_hw_sectors);
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..906b26b3d306 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -150,6 +150,12 @@ enum nvme_quirks {
         * 48 bits.
         */
        NVME_QUIRK_DMA_ADDRESS_BITS_48          = (1 << 16),
+
+       /*
+        * The controller doesn't allow the write zeores command of size
+        * more than reported by the MDTS
+        */
+       NVME_QUIRK_USE_MDTS_WRITE_ZEROES        = (1 << 17)
 };
 
 /*
@@ -274,6 +280,7 @@ struct nvme_ctrl {
 
        u64 cap;
        u32 max_hw_sectors;
+       u32 max_mdts_hw_sectors;
        u32 max_segments;
        u32 max_integrity_segments;
 #ifdef CONFIG_BLK_DEV_ZONED
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 207137a0ed8e..f9f36d26828b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3268,6 +3268,8 @@ static const struct pci_device_id nvme_id_table[] = {
                .driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
        { PCI_DEVICE(0x1d97, 0x2263),   /* SPCC */
                .driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+       { PCI_DEVICE(0x1c5c, 0x2839),  /* SK Hynix PE8000 U.3 NVMe
storage */
+               .driver_data = NVME_QUIRK_USE_MDTS_WRITE_ZEROES, },
        { PCI_DEVICE(0x2646, 0x2262),   /* KINGSTON SKC2000 NVMe SSD */
                .driver_data = NVME_QUIRK_NO_DEEPEST_PS, },
        { PCI_DEVICE(0x2646, 0x2263),   /* KINGSTON A2000 NVMe SSD  */
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..38fffee6b85e 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -688,7 +688,8 @@ static struct nvmf_transport_ops nvme_loop_transport = {
        .name           = "loop",
        .module         = THIS_MODULE,
        .create_ctrl    = nvme_loop_create_ctrl,
-       .allowed_opts   = NVMF_OPT_TRADDR,
+       .allowed_opts   = NVMF_OPT_TRADDR | NVMF_OPT_CTRL_LOSS_TMO,
+
 };
 
 static int __init nvme_loop_init_module(void)



From lkp at intel.com  Thu Feb 25 00:05:20 2021
From: lkp at intel.com (kernel test robot)
Date: Thu, 25 Feb 2021 13:05:20 +0800
Subject: [linux-nvme:nvme-5.12] BUILD SUCCESS
 e41e43a0a4c03b165cd975dbec806fe68dfc9c2b
Message-ID: <60373010.I1MCSECZqTL92FEA%lkp@intel.com>

tree/branch: git://git.infradead.org/nvme.git nvme-5.12
branch HEAD: e41e43a0a4c03b165cd975dbec806fe68dfc9c2b  nvmet: allow setting model_number only once

elapsed time: 728m

configs tested: 129
configs skipped: 2

The following configs have been built successfully.
More configs may be tested in the coming days.

gcc tested configs:
arm                                 defconfig
arm64                            allyesconfig
arm64                               defconfig
arm                              allyesconfig
arm                              allmodconfig
powerpc                     tqm5200_defconfig
mips                         db1xxx_defconfig
arm                          pcm027_defconfig
ia64                            zx1_defconfig
sh                            migor_defconfig
powerpc                     ppa8548_defconfig
xtensa                generic_kc705_defconfig
sh                          urquell_defconfig
arm                         s3c6400_defconfig
arm                        magician_defconfig
mips                        bcm47xx_defconfig
sh                            shmin_defconfig
arc                        nsim_700_defconfig
powerpc                     stx_gp3_defconfig
um                            kunit_defconfig
nios2                            allyesconfig
riscv                             allnoconfig
riscv                               defconfig
arm                           corgi_defconfig
mips                           ip27_defconfig
arm                           u8500_defconfig
mips                        jmr3927_defconfig
powerpc                      acadia_defconfig
mips                  decstation_64_defconfig
powerpc                      pasemi_defconfig
arm                         vf610m4_defconfig
sh                           se7750_defconfig
arm                        clps711x_defconfig
m68k                       m5475evb_defconfig
arm                           sama5_defconfig
nios2                               defconfig
mips                            ar7_defconfig
arm                        multi_v7_defconfig
sh                           se7780_defconfig
sh                               allmodconfig
parisc                generic-64bit_defconfig
arm                           tegra_defconfig
xtensa                              defconfig
arm                         cm_x300_defconfig
powerpc                    ge_imp3a_defconfig
arm                             mxs_defconfig
arm                            mmp2_defconfig
arm                          lpd270_defconfig
mips                           ci20_defconfig
mips                          rm200_defconfig
m68k                       m5275evb_defconfig
sh                        sh7785lcr_defconfig
sh                           se7721_defconfig
powerpc                      tqm8xx_defconfig
mips                     cu1830-neo_defconfig
sh                             sh03_defconfig
sh                               j2_defconfig
mips                           ip32_defconfig
mips                           ip28_defconfig
arc                        vdk_hs38_defconfig
mips                         tb0219_defconfig
arm                        realview_defconfig
mips                 decstation_r4k_defconfig
ia64                             allmodconfig
ia64                                defconfig
ia64                             allyesconfig
m68k                             allmodconfig
m68k                                defconfig
m68k                             allyesconfig
arc                              allyesconfig
nds32                             allnoconfig
c6x                              allyesconfig
nds32                               defconfig
csky                                defconfig
alpha                               defconfig
alpha                            allyesconfig
xtensa                           allyesconfig
h8300                            allyesconfig
arc                                 defconfig
parisc                              defconfig
s390                             allyesconfig
s390                             allmodconfig
parisc                           allyesconfig
s390                                defconfig
i386                             allyesconfig
sparc                            allyesconfig
sparc                               defconfig
i386                               tinyconfig
i386                                defconfig
mips                             allyesconfig
mips                             allmodconfig
powerpc                          allyesconfig
powerpc                          allmodconfig
powerpc                           allnoconfig
i386                 randconfig-a005-20210223
i386                 randconfig-a006-20210223
i386                 randconfig-a004-20210223
i386                 randconfig-a003-20210223
i386                 randconfig-a001-20210223
i386                 randconfig-a002-20210223
x86_64               randconfig-a015-20210223
x86_64               randconfig-a011-20210223
x86_64               randconfig-a012-20210223
x86_64               randconfig-a016-20210223
x86_64               randconfig-a014-20210223
x86_64               randconfig-a013-20210223
i386                 randconfig-a013-20210223
i386                 randconfig-a012-20210223
i386                 randconfig-a011-20210223
i386                 randconfig-a014-20210223
i386                 randconfig-a016-20210223
i386                 randconfig-a015-20210223
riscv                    nommu_k210_defconfig
riscv                            allyesconfig
riscv                    nommu_virt_defconfig
riscv                            allmodconfig
riscv                          rv32_defconfig
x86_64                           allyesconfig
x86_64                    rhel-7.6-kselftests
x86_64                              defconfig
x86_64                               rhel-8.3
x86_64                      rhel-8.3-kbuiltin
x86_64                                  kexec

clang tested configs:
x86_64               randconfig-a001-20210223
x86_64               randconfig-a002-20210223
x86_64               randconfig-a003-20210223
x86_64               randconfig-a005-20210223
x86_64               randconfig-a006-20210223
x86_64               randconfig-a004-20210223

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org


From gtiwari at redhat.com  Thu Feb 25 02:56:11 2021
From: gtiwari at redhat.com (Gopal Tiwari)
Date: Thu, 25 Feb 2021 02:56:11 -0500 (EST)
Subject: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write
 Zeroes command
In-Reply-To: <BYAPR04MB496509F7429C820F6A8DF37F869E9@BYAPR04MB4965.namprd04.prod.outlook.com>
References: <20210209045902.72345-1-gtiwari@redhat.com>
 <211078952.30654328.1612955843198.JavaMail.zimbra@redhat.com>
 <20210210131557.GB29904@lst.de>
 <BYAPR04MB49650A271CD1C1B3EFA72F3D868D9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210211070748.GA13852@lst.de>
 <BYAPR04MB4965C95C3F826D7222EB3176868C9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <20210224091326.GA5159@lst.de>
 <BYAPR04MB496509F7429C820F6A8DF37F869E9@BYAPR04MB4965.namprd04.prod.outlook.com>
Message-ID: <763823888.32269629.1614239771387.JavaMail.zimbra@redhat.com>

Hi Chaitanya, 

No feedback on the patch yet. 

Thanks 
Gopal  

----- Original Message -----
From: "Chaitanya Kulkarni" <Chaitanya.Kulkarni at wdc.com>
To: "Christoph Hellwig" <hch at lst.de>
Cc: hch at infradead.org, kbusch at kernel.org, "Gopal Tiwari" <gtiwari at redhat.com>, linux-nvme at lists.infradead.org
Sent: Thursday, February 25, 2021 7:39:33 AM
Subject: Re: [PATCH] nvme-pci: prevent SK Hynix PE8000 from using Write Zeroes command

On 2/24/21 01:13, Christoph Hellwig wrote:
> On Thu, Feb 11, 2021 at 06:11:19PM +0000, Chaitanya Kulkarni wrote:
>> On 2/10/21 11:07 PM, Christoph Hellwig wrote:
>>>>> I'd also love to retest most Write Zeroes quirks with that in place.
>>>> Do you prefer some variant of the following patch (totally untested)? OR
>>>> something else ?
>>> Somwhat.  As said I suspect defaulting to MDTS with a big fat comment
>>> might make most sense unless the new WZSL is set.
>>>
>> Okay, will send out patche(s) soon.
> Did you get to this?  Or did I just miss it?
>
I sent out WIP to Gopal on 02/13 due to lack of drive.

Not sure if makes sense to start the review where patch is
not tested on the real H/w, here it is :-

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e68a8c4ac5a6..62c928fb0ded 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1965,6 +1965,7 @@ static void nvme_config_discard(struct gendisk
*disk, struct nvme_ns *ns)
 
 static void nvme_config_write_zeroes(struct gendisk *disk, struct
nvme_ns *ns)
 {
+       bool use_mdts = ns->ctrl->quirks & NVME_QUIRK_USE_MDTS_WRITE_ZEROES;
        u64 max_blocks;
 
        if (!(ns->ctrl->oncs & NVME_CTRL_ONCS_WRITE_ZEROES) ||
@@ -1980,7 +1981,9 @@ static void nvme_config_write_zeroes(struct
gendisk *disk, struct nvme_ns *ns)
         * configured based on the controller's MDTS field in the
         * nvme_init_identify() if available.
         */
-       if (ns->ctrl->max_hw_sectors == UINT_MAX)
+       if (use_mdts && ns->ctrl->max_mdts_hw_sectors)
+               max_blocks = ns->ctrl->max_mdts_hw_sectors + 1;
+       else if (ns->ctrl->max_hw_sectors == UINT_MAX)
                max_blocks = (u64)USHRT_MAX + 1;
        else
                max_blocks = ns->ctrl->max_hw_sectors + 1;
@@ -3136,10 +3139,10 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 
        atomic_set(&ctrl->abort_limit, id->acl + 1);
        ctrl->vwc = id->vwc;
+       ctrl->max_mdts_hw_sectors = 0;
+       max_hw_sectors = UINT_MAX;
        if (id->mdts)
                max_hw_sectors = 1 << (id->mdts + page_shift - 9);
-       else
-               max_hw_sectors = UINT_MAX;
        ctrl->max_hw_sectors =
                min_not_zero(ctrl->max_hw_sectors, max_hw_sectors);
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..906b26b3d306 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -150,6 +150,12 @@ enum nvme_quirks {
         * 48 bits.
         */
        NVME_QUIRK_DMA_ADDRESS_BITS_48          = (1 << 16),
+
+       /*
+        * The controller doesn't allow the write zeores command of size
+        * more than reported by the MDTS
+        */
+       NVME_QUIRK_USE_MDTS_WRITE_ZEROES        = (1 << 17)
 };
 
 /*
@@ -274,6 +280,7 @@ struct nvme_ctrl {
 
        u64 cap;
        u32 max_hw_sectors;
+       u32 max_mdts_hw_sectors;
        u32 max_segments;
        u32 max_integrity_segments;
 #ifdef CONFIG_BLK_DEV_ZONED
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 207137a0ed8e..f9f36d26828b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3268,6 +3268,8 @@ static const struct pci_device_id nvme_id_table[] = {
                .driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
        { PCI_DEVICE(0x1d97, 0x2263),   /* SPCC */
                .driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
+       { PCI_DEVICE(0x1c5c, 0x2839),  /* SK Hynix PE8000 U.3 NVMe
storage */
+               .driver_data = NVME_QUIRK_USE_MDTS_WRITE_ZEROES, },
        { PCI_DEVICE(0x2646, 0x2262),   /* KINGSTON SKC2000 NVMe SSD */
                .driver_data = NVME_QUIRK_NO_DEEPEST_PS, },
        { PCI_DEVICE(0x2646, 0x2263),   /* KINGSTON A2000 NVMe SSD  */
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..38fffee6b85e 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -688,7 +688,8 @@ static struct nvmf_transport_ops nvme_loop_transport = {
        .name           = "loop",
        .module         = THIS_MODULE,
        .create_ctrl    = nvme_loop_create_ctrl,
-       .allowed_opts   = NVMF_OPT_TRADDR,
+       .allowed_opts   = NVMF_OPT_TRADDR | NVMF_OPT_CTRL_LOSS_TMO,
+
 };
 
 static int __init nvme_loop_init_module(void)


_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme



From hare at suse.de  Thu Feb 25 03:10:45 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 09:10:45 +0100
Subject: [PATCH 1/2] nvme: add 'fail_if_no_path' sysfs attribute
In-Reply-To: <303e7252-cb51-d6e2-eb4c-ad70fb3528a3@grimberg.me>
References: <20210223115922.104369-1-hare@suse.de>
 <20210223115922.104369-2-hare@suse.de>
 <303e7252-cb51-d6e2-eb4c-ad70fb3528a3@grimberg.me>
Message-ID: <fdf478ae-3784-869d-5512-55c7907a65a8@suse.de>

On 2/24/21 11:47 PM, Sagi Grimberg wrote:
> 
> 
> On 2/23/21 3:59 AM, Hannes Reinecke wrote:
>> In some setups like RAID or cluster we need to return an I/O error
>> once all paths are unavailable to allow the upper layers to start
>> their own error recovery (like redirecting I/O to other mirrors).
>> This patch adds a sysfs attribute 'fail_if_no_path' to allow the
>> admin to enable that behaviour instead of the current 'queue until
>> a path becomes available' policy.
> 
> Doesn't the same happen today if all the controllers are set with
> fail_io_fast_tmo=0? nvme_available_path will return false if all
> the paths have NVME_CTRL_FAILFAST_EXPIRED set.
> 
> I think that fail_io_fast_tmo should be settable via sysfs and I
> think I requested that during the various submission iterations...
> 
> I'm not sure this should be controlled on the individual namespace level...

Indeed, you are right; using 'fast_io_fail_tmo' works as well here.
So I'll be redoing the patch series and will be replacing this patch 
with another one adding a per-controller 'fast_io_fail_tmo' sysfs attribute.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb 25 03:37:03 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 09:37:03 +0100
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <eb979539-96fb-b767-5c5a-259263a25a0b@grimberg.me>
References: <20210223115922.104369-1-hare@suse.de>
 <20210223115922.104369-3-hare@suse.de>
 <eb979539-96fb-b767-5c5a-259263a25a0b@grimberg.me>
Message-ID: <4f2f7072-4bf4-3a00-047f-c20829fa09ba@suse.de>

On 2/24/21 11:40 PM, Sagi Grimberg wrote:
> 
>> The multipath code currently deletes the disk only after all references
>> to it are dropped rather than when the last path to that disk is lost.
>> This has been reported to cause problems with some use cases like MD 
>> RAID.
> 
> What is the exact problem?
> 
> Can you describe what the problem you see now and what you expect
> to see (unrelated to patch #1)?
> 
The problem is a difference in behaviour between multipathed and 
non-multipathed namespaces (ie whether 'CMIC' is set or not).
If the CMIC bit is _not_ set, the disk device will be removed once
the controller is gone; if the CMIC bit is set the disk device will be 
retained, and only removed once the last _reference_ is dropped.

This is causing customer issues, as some vendors produce nearly 
identical PCI NVMe devices, which differ in the CMIC bit.
So depending on which device the customer uses, he might be getting on 
or the other behaviour.
And this is causing issues when said customer deploys MD RAID on thems;
with one set of devices PCI hotplug works, with the other set of devices 
it doesn't.

>> This patch implements an alternative behaviour of deleting the disk when
>> the last path is gone, ie the same behaviour as non-multipathed nvme
>> devices.
> 
> But we also don't remove the non-multipath'd nvme device until the
> last reference drops (e.g. if you have a mounted filesystem on top).
> 
Au contraire.

When doing PCI hotplug the controller is removed (in the non-multipathed 
case), and calling 'put_disk()' during nvme_free_ns().
When doing PCI hotplug in the non-multipathed case, the controller is 
removed, too, but put_disk() is only called on the namespace itself; the 
'nshead' disk is still kept around, and put_disk() on the 'nshead' disk 
is only called after the last reference is dropped.

> This would be the equivalent to running raid on top of dm-mpath on
> top of scsi devices right? And if all the mpath device nodes go away
> the mpath device is deleted even if it has an open reference to it?
> 
See above. The prime motivator behind this patch is to get equivalent 
behaviour between multipathed and non-multipathed devices.
It just so happens that MD RAID exercises this particular issue.

>> The new behaviour will be selected with the 'fail_if_no_path'
>> attribute, as returning it's arguably the same functionality.
> 
> But its not the same functionality.

Agreed. But as the first patch will be dropped (see my other mail) I'll 
be redoing the patchset anyway.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb 25 05:55:23 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 11:55:23 +0100
Subject: [PATCH 2/6] nvme: add NVME_REQ_CANCELLED flag in nvme_cancel_request()
In-Reply-To: <20210225105527.75277-1-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
Message-ID: <20210225105527.75277-3-hare@suse.de>

NVME_REQ_CANCELLED is translated into -EINTR in nvme_submit_sync_cmd(),
so we should be setting this flags during nvme_cancel_request() to
ensure that the callers to nvme_submit_sync_cmd() will get the correct
error code when the controller is reset.

Signed-off-by: Hannes Reinecke <hare at suse.de>
Reviewed-by: Daniel Wagner <dwagner at suse.de>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 13ed7d3fbbb3..04dc85e0810c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -380,6 +380,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
 		return true;
 
 	nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
+	nvme_req(req)->flags |= NVME_REQ_CANCELLED;
 	blk_mq_complete_request(req);
 	return true;
 }
-- 
2.29.2



From hare at suse.de  Thu Feb 25 05:55:22 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 11:55:22 +0100
Subject: [PATCH 1/6] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <20210225105527.75277-1-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
Message-ID: <20210225105527.75277-2-hare@suse.de>

We only should remove namespaces when we get fatal error back from
the device or when the namespace IDs have changed.
So instead of painfully masking out error numbers which might indicate
that the error should be ignored we could use an NVME status code
to indicated when the namespace should be removed.
That simplifies the final logic and makes it less error-prone.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..13ed7d3fbbb3 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4036,7 +4036,7 @@ static void nvme_ns_remove_by_nsid(struct nvme_ctrl *ctrl, u32 nsid)
 static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
 {
 	struct nvme_id_ns *id;
-	int ret = -ENODEV;
+	int ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
 
 	if (test_bit(NVME_NS_DEAD, &ns->flags))
 		goto out;
@@ -4045,7 +4045,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
 	if (ret)
 		goto out;
 
-	ret = -ENODEV;
+	ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
 	if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
 		dev_err(ns->ctrl->device,
 			"identifiers changed for nsid %d\n", ns->head->ns_id);
@@ -4063,7 +4063,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
 	 *
 	 * TODO: we should probably schedule a delayed retry here.
 	 */
-	if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR)))
+	if (ret > 0 && (ret & NVME_SC_DNR))
 		nvme_ns_remove(ns);
 }
 
-- 
2.29.2



From hare at suse.de  Thu Feb 25 05:55:25 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 11:55:25 +0100
Subject: [PATCH 4/6] nvme-fc: return NVME_SC_HOST_ABORTED_CMD when a command
 has been aborted
In-Reply-To: <20210225105527.75277-1-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
Message-ID: <20210225105527.75277-5-hare@suse.de>

When a command has been aborted we should return NVME_SC_HOST_ABORTED_CMD
to be consistent with the other transports.

Signed-off-by: Hannes Reinecke <hare at suse.de>
Reviewed-by: Daniel Wagner <dwagner at suse.de>
---
 drivers/nvme/host/fc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index ef12a619daec..97e3424c7b03 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -1956,7 +1956,7 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
 				sizeof(op->rsp_iu), DMA_FROM_DEVICE);
 
 	if (opstate == FCPOP_STATE_ABORTED)
-		status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1);
+		status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
 	else if (freq->status) {
 		status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1);
 		dev_info(ctrl->ctrl.device,
-- 
2.29.2



From hare at suse.de  Thu Feb 25 05:55:21 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 11:55:21 +0100
Subject: [PATCHv3 0/6] Fixup return value for nvme_submit_sync_cmd()
Message-ID: <20210225105527.75277-1-hare@suse.de>

here are some small patches for fixing up the return value of nvme_submit_sync_cmd().
As Keith correctly noted, nvme_submit_sync_cmd() should be returning
an error if the command could not be performed; however, currently
only pci does that.
So we need to fix up nvme_cancel_request() to return an -EINTR
on any pending sync commands during reset.
And modify nvme-fc to return the same nvme status after timing out
or cancelling requests.
Plus, finally, adding a new flag 'NVME_REQ_TIMEOUT' to indicate
that a command had been aborted due to a timeout, and translates that
back into -ETIMEDOUT as a return code for nvme_submit_sync_cmd().

As usual, comments and reviews are welcome.

Changes to v2:
- Include reviews from Sagi
- Simplify error logic from nvme_validate_ns()

Changes to v1:
- Include reviews from Daniel
- Include changes for nvme-fc to return the same status as the
  other transports

Hannes Reinecke (6):
  nvme: simplify error logic in nvme_validate_ns()
  nvme: add NVME_REQ_CANCELLED flag in nvme_cancel_request()
  nvme-fc: set NVME_REQ_CANCELLED in nvme_fc_terminate_exchange()
  nvme-fc: return NVME_SC_HOST_ABORTED_CMD when a command has been
    aborted
  nvme: return NVME_SC_ABORT_QUEUE for cancelled commands
  nvme: return -ETIMEDOUT in nvme_submit_sync_cmd()

 drivers/nvme/host/core.c | 11 +++++++----
 drivers/nvme/host/fc.c   | 11 ++++++++---
 drivers/nvme/host/nvme.h |  1 +
 drivers/nvme/host/rdma.c |  1 +
 drivers/nvme/host/tcp.c  |  1 +
 5 files changed, 18 insertions(+), 7 deletions(-)

-- 
2.29.2



From hare at suse.de  Thu Feb 25 05:55:24 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 11:55:24 +0100
Subject: [PATCH 3/6] nvme-fc: set NVME_REQ_CANCELLED in
 nvme_fc_terminate_exchange()
In-Reply-To: <20210225105527.75277-1-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
Message-ID: <20210225105527.75277-4-hare@suse.de>

nvme_fc_terminate_exchange() is being called when exchanges are
being deleted, and as such we should be setting the NVME_REQ_CANCELLED
flag to have identical behaviour on all transports.

Signed-off-by: Hannes Reinecke <hare at suse.de>
Reviewed-by: Daniel Wagner <dwagner at suse.de>
---
 drivers/nvme/host/fc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 20dadd86e981..ef12a619daec 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2443,6 +2443,7 @@ nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
 	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
 	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
 
+	op->nreq.flags |= NVME_REQ_CANCELLED;
 	__nvme_fc_abort_op(ctrl, op);
 	return true;
 }
-- 
2.29.2



From hare at suse.de  Thu Feb 25 05:55:26 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 11:55:26 +0100
Subject: [PATCH 5/6] nvme: return NVME_SC_ABORT_QUEUE for cancelled commands
In-Reply-To: <20210225105527.75277-1-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
Message-ID: <20210225105527.75277-6-hare@suse.de>

A request will be cancelled when a queue is deleted, so we should
be returning a status of NVME_SC_ABORT_QUEUE.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c | 2 +-
 drivers/nvme/host/fc.c   | 9 ++++++---
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 04dc85e0810c..624531a6b7c6 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -379,7 +379,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
 	if (blk_mq_request_completed(req))
 		return true;
 
-	nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
+	nvme_req(req)->status = NVME_SC_ABORT_QUEUE;
 	nvme_req(req)->flags |= NVME_REQ_CANCELLED;
 	blk_mq_complete_request(req);
 	return true;
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 97e3424c7b03..86395b66310d 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -1955,9 +1955,12 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
 	fc_dma_sync_single_for_cpu(ctrl->lport->dev, op->fcp_req.rspdma,
 				sizeof(op->rsp_iu), DMA_FROM_DEVICE);
 
-	if (opstate == FCPOP_STATE_ABORTED)
-		status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
-	else if (freq->status) {
+	if (opstate == FCPOP_STATE_ABORTED) {
+		if (op->nreq.flags & NVME_REQ_CANCELLED)
+			status = cpu_to_le16(NVME_SC_ABORT_QUEUE << 1);
+		else
+			status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
+	} else if (freq->status) {
 		status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1);
 		dev_info(ctrl->ctrl.device,
 			"NVME-FC{%d}: io failed due to lldd error %d\n",
-- 
2.29.2



From hare at suse.de  Thu Feb 25 05:55:27 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 11:55:27 +0100
Subject: [PATCH 6/6] nvme: return -ETIMEDOUT in nvme_submit_sync_cmd()
In-Reply-To: <20210225105527.75277-1-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
Message-ID: <20210225105527.75277-7-hare@suse.de>

When a command times out we should be returning -ETIMEDOUT in
nvme_submit_sync_cmd(), and not requiring the caller to guess whether
a specific NVMe status should be interpreted as a timeout.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c | 2 ++
 drivers/nvme/host/fc.c   | 1 +
 drivers/nvme/host/nvme.h | 1 +
 drivers/nvme/host/rdma.c | 1 +
 drivers/nvme/host/tcp.c  | 1 +
 5 files changed, 6 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 624531a6b7c6..5c1ec3bfb9d3 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1004,6 +1004,8 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
 		*result = nvme_req(req)->result;
 	if (nvme_req(req)->flags & NVME_REQ_CANCELLED)
 		ret = -EINTR;
+	else if (nvme_req(req)->flags & NVME_REQ_TIMEOUT)
+		ret = -ETIMEDOUT;
 	else
 		ret = nvme_req(req)->status;
  out:
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 86395b66310d..bcc086fcbe61 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2551,6 +2551,7 @@ nvme_fc_timeout(struct request *rq, bool reserved)
 		"x%08x/x%08x\n",
 		ctrl->cnum, op->queue->qnum, sqe->common.opcode,
 		sqe->connect.fctype, sqe->common.cdw10, sqe->common.cdw11);
+	op->nreq.flags |= NVME_REQ_TIMEOUT;
 	if (__nvme_fc_abort_op(ctrl, op))
 		nvme_fc_error_recovery(ctrl, "io timeout abort failed");
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..02aa4ba665c0 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -173,6 +173,7 @@ struct nvme_request {
 enum {
 	NVME_REQ_CANCELLED		= (1 << 0),
 	NVME_REQ_USERCMD		= (1 << 1),
+	NVME_REQ_TIMEOUT		= (1 << 2),
 };
 
 static inline struct nvme_request *nvme_req(struct request *req)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 53ac4d7442ba..99ad2e10b295 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1990,6 +1990,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
 	nvme_rdma_stop_queue(queue);
 	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
+		nvme_req(rq)->flags |= NVME_REQ_TIMEOUT;
 		blk_mq_complete_request(rq);
 	}
 }
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 69f59d2c5799..64aab742c84d 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2189,6 +2189,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
 	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
 	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
 		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
+		nvme_req(rq)->flags |= NVME_REQ_TIMEOUT;
 		blk_mq_complete_request(rq);
 	}
 }
-- 
2.29.2



From minwoo.im.dev at gmail.com  Thu Feb 25 05:57:44 2021
From: minwoo.im.dev at gmail.com (Minwoo Im)
Date: Thu, 25 Feb 2021 19:57:44 +0900
Subject: [PATCH] nvme-multipath: use nvme_ctrl_use_ana helper to check ANA
Message-ID: <20210225105744.69579-1-minwoo.im.dev@gmail.com>

Call nvme_ctrl_use_ana helper to check whether the given controller uses
ANA or not.

Signed-off-by: Minwoo Im <minwoo.im.dev at gmail.com>
---
 drivers/nvme/host/multipath.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index c658b73a41b7..bbb582b12226 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -82,7 +82,7 @@ void nvme_failover_req(struct request *req)
 	 * ready to serve this namespace.  Kick of a re-read of the ANA
 	 * information page, and just try any other available path for now.
 	 */
-	if (nvme_is_ana_error(status) && ns->ctrl->ana_log_buf) {
+	if (nvme_is_ana_error(status) && nvme_ctrl_use_ana(ns->ctrl)) {
 		set_bit(NVME_NS_ANA_PENDING, &ns->flags);
 		queue_work(nvme_wq, &ns->ctrl->ana_work);
 	}
-- 
2.25.1



From hare at suse.de  Thu Feb 25 06:05:32 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 12:05:32 +0100
Subject: [PATCHv2 0/2] nvme: fixup MD RAID usage
Message-ID: <20210225110534.75710-1-hare@suse.de>

Hi all,

ever since the implementation NVMe-oF does not work together with MD RAID.
MD RAID expects the device to return an I/O error on failure, and to remove
the block device if the underlying hardware is removed.
This is contrary to the implementation of NVMe-oF, which will keep on retrying
I/O while the controller is being reset, and will only remove the block device
once the last _user_ is gone.

While we already have a 'fast_io_fail_tmo' setting which would ensure
that I/O errors are returned after the timeout expired, this attribute
has to be set during the initial 'connect' call.
So the first patch adds a 'fasi_io_fail_tmo' controller sysfs
attribute which allows the admin to modify it during runtime.
To fixup the second issue a new attribute 'no_path_detach' is
implemented, which will cause the disk to be removed as once the
last controller holding a path to the namespace is removed (ie after
all reconnect attempts for that controllers are exhausted).

This is a rework of the earlier path by Keith Busch ('nvme-mpath: delete disk
after last connection'). Kudos to him for suggesting this approach.

Changes to v1:
- Integrated reviews from Sagi
- Use fast_io_fail_tmo instead of a new sysfs attribute
- Rename attribute to 'no_path_detach'

Hannes Reinecke (2):
  nvme: add 'fast_io_fail_tmo' controller sysfs attribute
  nvme: delete disk when last path is gone

 drivers/nvme/host/core.c      | 41 ++++++++++++++++++++++++++++++++++-
 drivers/nvme/host/multipath.c | 31 +++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h      | 21 +++++++++++++++---
 3 files changed, 88 insertions(+), 5 deletions(-)

-- 
2.29.2



From hare at suse.de  Thu Feb 25 06:05:33 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 12:05:33 +0100
Subject: [PATCH 1/2] nvme: add 'fast_io_fail_tmo' controller sysfs attribute
In-Reply-To: <20210225110534.75710-1-hare@suse.de>
References: <20210225110534.75710-1-hare@suse.de>
Message-ID: <20210225110534.75710-2-hare@suse.de>

Add a 'fast_io_fail_tmo' controller sysfs attribute to make the
setting configurable during runtime.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4de6a3a13575..ba639049e385 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3658,6 +3658,37 @@ static ssize_t nvme_ctrl_loss_tmo_store(struct device *dev,
 static DEVICE_ATTR(ctrl_loss_tmo, S_IRUGO | S_IWUSR,
 	nvme_ctrl_loss_tmo_show, nvme_ctrl_loss_tmo_store);
 
+static ssize_t nvme_fast_io_fail_tmo_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvmf_ctrl_options *opts = ctrl->opts;
+
+	if (opts->fast_io_fail_tmo == -1)
+		return sprintf(buf, "off\n");
+	return sprintf(buf, "%d\n", opts->fast_io_fail_tmo);
+}
+
+static ssize_t nvme_fast_io_fail_tmo_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+	struct nvmf_ctrl_options *opts = ctrl->opts;
+	int fast_io_fail_tmo, err;
+
+	err = kstrtoint(buf, 10, &fast_io_fail_tmo);
+	if (err)
+		return -EINVAL;
+
+	else if (fast_io_fail_tmo < 0)
+		opts->fast_io_fail_tmo = -1;
+	else
+		opts->fast_io_fail_tmo = fast_io_fail_tmo;
+	return count;
+}
+static DEVICE_ATTR(fast_io_fail_tmo, S_IRUGO | S_IWUSR,
+	nvme_fast_io_fail_tmo_show, nvme_fast_io_fail_tmo_store);
+
 static ssize_t nvme_ctrl_reconnect_delay_show(struct device *dev,
 		struct device_attribute *attr, char *buf)
 {
@@ -3703,6 +3734,7 @@ static struct attribute *nvme_dev_attrs[] = {
 	&dev_attr_hostnqn.attr,
 	&dev_attr_hostid.attr,
 	&dev_attr_ctrl_loss_tmo.attr,
+	&dev_attr_fast_io_fail_tmo.attr,
 	&dev_attr_reconnect_delay.attr,
 	NULL
 };
-- 
2.29.2



From hare at suse.de  Thu Feb 25 06:05:34 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 12:05:34 +0100
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <20210225110534.75710-1-hare@suse.de>
References: <20210225110534.75710-1-hare@suse.de>
Message-ID: <20210225110534.75710-3-hare@suse.de>

The multipath code currently deletes the disk only after all references
to it are dropped rather than when the last path to that disk is lost.
This differs from the behaviour in the non-multipathed case where the
disk is deleted once the controller is removed.
This has been reported to cause problems with some use cases like MD RAID.

This patch implements an alternative behaviour of deleting the disk when
the last path is gone, ie the same behaviour as non-multipathed nvme
devices. The alternative behaviour can be enabled with the new sysfs
attribute 'no_path_detach'.

Suggested-by: Keith Busch <kbusch at kernel.org>
Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c      |  9 ++++++++-
 drivers/nvme/host/multipath.c | 31 ++++++++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h      | 21 ++++++++++++++++++---
 3 files changed, 56 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ba639049e385..7af6ba18f461 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -541,7 +541,9 @@ static void nvme_free_ns_head(struct kref *ref)
 	struct nvme_ns_head *head =
 		container_of(ref, struct nvme_ns_head, ref);
 
-	nvme_mpath_remove_disk(head);
+	if (!test_bit(NVME_NSHEAD_NO_PATH_DETACH, &head->flags))
+		nvme_mpath_remove_disk(head);
+	nvme_mpath_put_disk(head);
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
 	nvme_put_subsystem(head->subsys);
@@ -3464,6 +3466,7 @@ static struct attribute *nvme_ns_id_attrs[] = {
 #ifdef CONFIG_NVME_MULTIPATH
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
+	&dev_attr_no_path_detach.attr,
 #endif
 	NULL,
 };
@@ -3494,6 +3497,10 @@ static umode_t nvme_ns_id_attrs_are_visible(struct kobject *kobj,
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
 			return 0;
 	}
+	if (a == &dev_attr_no_path_detach.attr) {
+		if (dev_to_disk(dev)->fops == &nvme_bdev_ops)
+			return 0;
+	}
 #endif
 	return a->mode;
 }
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 0696319adaf6..7d38f9272490 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -641,6 +641,36 @@ static ssize_t ana_state_show(struct device *dev, struct device_attribute *attr,
 }
 DEVICE_ATTR_RO(ana_state);
 
+static ssize_t no_path_detach_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+
+	return sprintf(buf, "%d\n",
+		       test_bit(NVME_NSHEAD_NO_PATH_DETACH, &head->flags) ?
+		       1 : 0);
+}
+
+static ssize_t no_path_detach_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct gendisk *disk = dev_to_disk(dev);
+	struct nvme_ns_head *head = disk->private_data;
+	int err, no_path_detach;
+
+	err = kstrtoint(buf, 10, &no_path_detach);
+	if (err || no_path_detach < 0)
+		return -EINVAL;
+	if (no_path_detach)
+		set_bit(NVME_NSHEAD_NO_PATH_DETACH, &head->flags);
+	else
+		clear_bit(NVME_NSHEAD_NO_PATH_DETACH, &head->flags);
+	return count;
+}
+DEVICE_ATTR(no_path_detach, S_IRUGO | S_IWUSR,
+	no_path_detach_show, no_path_detach_store);
+
 static int nvme_lookup_ana_group_desc(struct nvme_ctrl *ctrl,
 		struct nvme_ana_group_desc *desc, void *data)
 {
@@ -702,7 +732,6 @@ void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 		 */
 		head->disk->queue = NULL;
 	}
-	put_disk(head->disk);
 }
 
 int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..317f8a6cb7f4 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -417,7 +417,8 @@ struct nvme_ns_head {
 	struct work_struct	requeue_work;
 	struct mutex		lock;
 	unsigned long		flags;
-#define NVME_NSHEAD_DISK_LIVE	0
+#define NVME_NSHEAD_DISK_LIVE		0
+#define NVME_NSHEAD_NO_PATH_DETACH	1
 	struct nvme_ns __rcu	*current_path[];
 #endif
 };
@@ -680,8 +681,12 @@ static inline void nvme_mpath_check_last_path(struct nvme_ns *ns)
 {
 	struct nvme_ns_head *head = ns->head;
 
-	if (head->disk && list_empty(&head->list))
-		kblockd_schedule_work(&head->requeue_work);
+	if (head->disk && list_empty(&head->list)) {
+		if (test_bit(NVME_NSHEAD_NO_PATH_DETACH, &head->flags))
+			nvme_mpath_remove_disk(head);
+		else
+			kblockd_schedule_work(&head->requeue_work);
+	}
 }
 
 static inline void nvme_trace_bio_complete(struct request *req)
@@ -692,8 +697,15 @@ static inline void nvme_trace_bio_complete(struct request *req)
 		trace_block_bio_complete(ns->head->disk->queue, req->bio);
 }
 
+static inline void nvme_mpath_put_disk(struct nvme_ns_head *head)
+{
+	if (head->disk)
+		put_disk(head->disk);
+}
+
 extern struct device_attribute dev_attr_ana_grpid;
 extern struct device_attribute dev_attr_ana_state;
+extern struct device_attribute dev_attr_no_path_detach;
 extern struct device_attribute subsys_attr_iopolicy;
 
 #else
@@ -729,6 +741,9 @@ static inline void nvme_mpath_add_disk(struct nvme_ns *ns,
 static inline void nvme_mpath_remove_disk(struct nvme_ns_head *head)
 {
 }
+static inline void nvme_mpath_put_disk(struct nvme_ns_head *head)
+{
+}
 static inline bool nvme_mpath_clear_current_path(struct nvme_ns *ns)
 {
 	return false;
-- 
2.29.2



From kbusch at kernel.org  Thu Feb 25 11:22:34 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 26 Feb 2021 01:22:34 +0900
Subject: [PATCH] nvme: Add quirks for Lexar 256GB SSD
In-Reply-To: <CAANdO=LFVGNWNs_p9h0GsukVkRMcDQ1A5qsaa2m9HSPhu0wr7Q@mail.gmail.com>
References: <20210223221046.3367795-1-pterjan@google.com>
 <20210224092147.GA2195892@infradead.org>
 <CAANdO=LFVGNWNs_p9h0GsukVkRMcDQ1A5qsaa2m9HSPhu0wr7Q@mail.gmail.com>
Message-ID: <20210225162234.GA31593@redsun51.ssa.fujisawa.hgst.com>

On Wed, Feb 24, 2021 at 10:23:23AM +0000, Pascal Terjan wrote:
> On Wed, 24 Feb 2021 at 09:21, Christoph Hellwig <hch at infradead.org> wrote:
> >
> > On Tue, Feb 23, 2021 at 10:10:46PM +0000, Pascal Terjan wrote:
> > > Add NVME_QUIRK_NO_NS_DESC_LIST to Lexar 256GB SSD to allow it to
> > > work again, and add NVME_QUIRK_IGNORE_DEV_SUBNQN to avoid a warning.
> >
> > Why again?  The drive has always been broken since the desc list has
> > been a mandatory feature since NVMe 1.3..
> 
> According to the reporter it works on Debian which is still running a
> 4.19 kernel
> My guess is that it worked before
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=538af88ea7d9de241e6b6f006e9049c4d96723bb
> as the error was ignored
> And I believe it would have worked again until
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5bedd3afee8eb01ccd256f0cd2cc0fa6f841417a
> as a waorkaround was added at some point (it changed at
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fb314eb0cbb2e11540d1ae1a7b28346397f621ef
> but was laready there)

Right, the error used to be ignored, but now we avoid the error in the
first place with the quirk.


From kbusch at kernel.org  Thu Feb 25 11:31:40 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 26 Feb 2021 01:31:40 +0900
Subject: [PATCH 6/6] nvme: return -ETIMEDOUT in nvme_submit_sync_cmd()
In-Reply-To: <20210225105527.75277-7-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-7-hare@suse.de>
Message-ID: <20210225163140.GB31593@redsun51.ssa.fujisawa.hgst.com>

On Thu, Feb 25, 2021 at 11:55:27AM +0100, Hannes Reinecke wrote:
> When a command times out we should be returning -ETIMEDOUT in
> nvme_submit_sync_cmd(), and not requiring the caller to guess whether
> a specific NVMe status should be interpreted as a timeout.

This driver has been using -EINTR for this condition since the
beginning, though. Is there a reason I'm not seeing why these use cases
can't use the existing command cancelled flag?
 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c | 2 ++
>  drivers/nvme/host/fc.c   | 1 +
>  drivers/nvme/host/nvme.h | 1 +
>  drivers/nvme/host/rdma.c | 1 +
>  drivers/nvme/host/tcp.c  | 1 +
>  5 files changed, 6 insertions(+)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 624531a6b7c6..5c1ec3bfb9d3 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -1004,6 +1004,8 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>  		*result = nvme_req(req)->result;
>  	if (nvme_req(req)->flags & NVME_REQ_CANCELLED)
>  		ret = -EINTR;
> +	else if (nvme_req(req)->flags & NVME_REQ_TIMEOUT)
> +		ret = -ETIMEDOUT;
>  	else
>  		ret = nvme_req(req)->status;
>   out:
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 86395b66310d..bcc086fcbe61 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2551,6 +2551,7 @@ nvme_fc_timeout(struct request *rq, bool reserved)
>  		"x%08x/x%08x\n",
>  		ctrl->cnum, op->queue->qnum, sqe->common.opcode,
>  		sqe->connect.fctype, sqe->common.cdw10, sqe->common.cdw11);
> +	op->nreq.flags |= NVME_REQ_TIMEOUT;
>  	if (__nvme_fc_abort_op(ctrl, op))
>  		nvme_fc_error_recovery(ctrl, "io timeout abort failed");
>  
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 07b34175c6ce..02aa4ba665c0 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -173,6 +173,7 @@ struct nvme_request {
>  enum {
>  	NVME_REQ_CANCELLED		= (1 << 0),
>  	NVME_REQ_USERCMD		= (1 << 1),
> +	NVME_REQ_TIMEOUT		= (1 << 2),
>  };
>  
>  static inline struct nvme_request *nvme_req(struct request *req)
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 53ac4d7442ba..99ad2e10b295 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1990,6 +1990,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
>  	nvme_rdma_stop_queue(queue);
>  	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>  		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> +		nvme_req(rq)->flags |= NVME_REQ_TIMEOUT;
>  		blk_mq_complete_request(rq);
>  	}
>  }
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 69f59d2c5799..64aab742c84d 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2189,6 +2189,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>  	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>  	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>  		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> +		nvme_req(rq)->flags |= NVME_REQ_TIMEOUT;
>  		blk_mq_complete_request(rq);
>  	}
>  }
> -- 
> 2.29.2
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme


From kbusch at kernel.org  Thu Feb 25 11:36:06 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 26 Feb 2021 01:36:06 +0900
Subject: [PATCH 1/6] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <20210225105527.75277-2-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-2-hare@suse.de>
Message-ID: <20210225163606.GC31593@redsun51.ssa.fujisawa.hgst.com>

On Thu, Feb 25, 2021 at 11:55:22AM +0100, Hannes Reinecke wrote:
> We only should remove namespaces when we get fatal error back from
> the device or when the namespace IDs have changed.
> So instead of painfully masking out error numbers which might indicate
> that the error should be ignored we could use an NVME status code
> to indicated when the namespace should be removed.
> That simplifies the final logic and makes it less error-prone.

I think this looks good to me. The timeout condition should retain
discovered namespaces since the controller is going to be restarted.

Reviewed-by: Keith Busch <kbusch at kernel.org>
 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index d77f3f26d8d3..13ed7d3fbbb3 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -4036,7 +4036,7 @@ static void nvme_ns_remove_by_nsid(struct nvme_ctrl *ctrl, u32 nsid)
>  static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>  {
>  	struct nvme_id_ns *id;
> -	int ret = -ENODEV;
> +	int ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>  
>  	if (test_bit(NVME_NS_DEAD, &ns->flags))
>  		goto out;
> @@ -4045,7 +4045,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>  	if (ret)
>  		goto out;
>  
> -	ret = -ENODEV;
> +	ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>  	if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
>  		dev_err(ns->ctrl->device,
>  			"identifiers changed for nsid %d\n", ns->head->ns_id);
> @@ -4063,7 +4063,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>  	 *
>  	 * TODO: we should probably schedule a delayed retry here.
>  	 */
> -	if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR)))
> +	if (ret > 0 && (ret & NVME_SC_DNR))
>  		nvme_ns_remove(ns);
>  }
>  
> -- 
> 2.29.2


From kbusch at kernel.org  Thu Feb 25 11:37:23 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 26 Feb 2021 01:37:23 +0900
Subject: [PATCH 3/6] nvme-fc: set NVME_REQ_CANCELLED in
 nvme_fc_terminate_exchange()
In-Reply-To: <20210225105527.75277-4-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-4-hare@suse.de>
Message-ID: <20210225163723.GD31593@redsun51.ssa.fujisawa.hgst.com>

On Thu, Feb 25, 2021 at 11:55:24AM +0100, Hannes Reinecke wrote:
> nvme_fc_terminate_exchange() is being called when exchanges are
> being deleted, and as such we should be setting the NVME_REQ_CANCELLED
> flag to have identical behaviour on all transports.

Looks good.

Reviewed-by: Keith Busch <kbusch at kernel.org>


From kbusch at kernel.org  Thu Feb 25 11:43:08 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 26 Feb 2021 01:43:08 +0900
Subject: [PATCH 5/6] nvme: return NVME_SC_ABORT_QUEUE for cancelled
 commands
In-Reply-To: <20210225105527.75277-6-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-6-hare@suse.de>
Message-ID: <20210225164308.GE31593@redsun51.ssa.fujisawa.hgst.com>

On Thu, Feb 25, 2021 at 11:55:26AM +0100, Hannes Reinecke wrote:
> A request will be cancelled when a queue is deleted, so we should
> be returning a status of NVME_SC_ABORT_QUEUE.

I'm not sure if this makes sense. The controller didn't return a status
in this case, so the host is reclaiming the command absent a status from
the controller. Is there a particular situation where this patch
matters?
 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>  drivers/nvme/host/core.c | 2 +-
>  drivers/nvme/host/fc.c   | 9 ++++++---
>  2 files changed, 7 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 04dc85e0810c..624531a6b7c6 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -379,7 +379,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
>  	if (blk_mq_request_completed(req))
>  		return true;
>  
> -	nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
> +	nvme_req(req)->status = NVME_SC_ABORT_QUEUE;
>  	nvme_req(req)->flags |= NVME_REQ_CANCELLED;
>  	blk_mq_complete_request(req);
>  	return true;
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 97e3424c7b03..86395b66310d 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -1955,9 +1955,12 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
>  	fc_dma_sync_single_for_cpu(ctrl->lport->dev, op->fcp_req.rspdma,
>  				sizeof(op->rsp_iu), DMA_FROM_DEVICE);
>  
> -	if (opstate == FCPOP_STATE_ABORTED)
> -		status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
> -	else if (freq->status) {
> +	if (opstate == FCPOP_STATE_ABORTED) {
> +		if (op->nreq.flags & NVME_REQ_CANCELLED)
> +			status = cpu_to_le16(NVME_SC_ABORT_QUEUE << 1);
> +		else
> +			status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
> +	} else if (freq->status) {
>  		status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1);
>  		dev_info(ctrl->ctrl.device,
>  			"NVME-FC{%d}: io failed due to lldd error %d\n",
> -- 
> 2.29.2


From kbusch at kernel.org  Thu Feb 25 11:59:44 2021
From: kbusch at kernel.org (Keith Busch)
Date: Fri, 26 Feb 2021 01:59:44 +0900
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <20210225110534.75710-3-hare@suse.de>
References: <20210225110534.75710-1-hare@suse.de>
 <20210225110534.75710-3-hare@suse.de>
Message-ID: <20210225165944.GF31593@redsun51.ssa.fujisawa.hgst.com>

On Thu, Feb 25, 2021 at 12:05:34PM +0100, Hannes Reinecke wrote:
> The multipath code currently deletes the disk only after all references
> to it are dropped rather than when the last path to that disk is lost.
> This differs from the behaviour in the non-multipathed case where the
> disk is deleted once the controller is removed.
> This has been reported to cause problems with some use cases like MD RAID.
> 
> This patch implements an alternative behaviour of deleting the disk when
> the last path is gone, ie the same behaviour as non-multipathed nvme
> devices. The alternative behaviour can be enabled with the new sysfs
> attribute 'no_path_detach'.

This looks ok to me. I have heard from a few people that they expected
it to work this way with the option enabled, but I suppose we do need to
retain the old behavior as default.

Reviewed-by: Keith Busch <kbusch at kernel.org>


From hare at suse.de  Thu Feb 25 12:28:59 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 18:28:59 +0100
Subject: [PATCH 6/6] nvme: return -ETIMEDOUT in nvme_submit_sync_cmd()
In-Reply-To: <20210225163140.GB31593@redsun51.ssa.fujisawa.hgst.com>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-7-hare@suse.de>
 <20210225163140.GB31593@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <16db4e63-4a07-6973-43f4-319ee3171431@suse.de>

On 2/25/21 5:31 PM, Keith Busch wrote:
> On Thu, Feb 25, 2021 at 11:55:27AM +0100, Hannes Reinecke wrote:
>> When a command times out we should be returning -ETIMEDOUT in
>> nvme_submit_sync_cmd(), and not requiring the caller to guess whether
>> a specific NVMe status should be interpreted as a timeout.
> 
> This driver has been using -EINTR for this condition since the
> beginning, though. Is there a reason I'm not seeing why these use cases
> can't use the existing command cancelled flag?
>   
Technically we have two distinct conditions, a command being aborted due 
to a timeout, and a command being aborted due to a queue disconnect.
Admittedly, the second conditions is the common case, as the queue 
disconnect is typically done due to the error recovery kicking in, which 
in itself is in reaction to a KATO timeout firing.
The first condition is only hit if we run into a timeout for the initial 
commands _before_ KATO is started.

And I thought it might be good to have these two cases differentiated.
But I'd be fine with dropping it if that's not needed.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb 25 12:34:07 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 18:34:07 +0100
Subject: [PATCH 5/6] nvme: return NVME_SC_ABORT_QUEUE for cancelled
 commands
In-Reply-To: <20210225164308.GE31593@redsun51.ssa.fujisawa.hgst.com>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-6-hare@suse.de>
 <20210225164308.GE31593@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <2ddca50e-ca0c-00b4-589a-3587101ac452@suse.de>

On 2/25/21 5:43 PM, Keith Busch wrote:
> On Thu, Feb 25, 2021 at 11:55:26AM +0100, Hannes Reinecke wrote:
>> A request will be cancelled when a queue is deleted, so we should
>> be returning a status of NVME_SC_ABORT_QUEUE.
> 
> I'm not sure if this makes sense. The controller didn't return a status
> in this case, so the host is reclaiming the command absent a status from
> the controller. Is there a particular situation where this patch
> matters?
>   
Not really. You are right, I guess I'll drop it.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Thu Feb 25 12:37:41 2021
From: hare at suse.de (Hannes Reinecke)
Date: Thu, 25 Feb 2021 18:37:41 +0100
Subject: [PATCH 2/2] nvme: delete disk when last path is gone
In-Reply-To: <20210225165944.GF31593@redsun51.ssa.fujisawa.hgst.com>
References: <20210225110534.75710-1-hare@suse.de>
 <20210225110534.75710-3-hare@suse.de>
 <20210225165944.GF31593@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <176fe48f-090f-0365-ef8f-d650b803f053@suse.de>

On 2/25/21 5:59 PM, Keith Busch wrote:
> On Thu, Feb 25, 2021 at 12:05:34PM +0100, Hannes Reinecke wrote:
>> The multipath code currently deletes the disk only after all references
>> to it are dropped rather than when the last path to that disk is lost.
>> This differs from the behaviour in the non-multipathed case where the
>> disk is deleted once the controller is removed.
>> This has been reported to cause problems with some use cases like MD RAID.
>>
>> This patch implements an alternative behaviour of deleting the disk when
>> the last path is gone, ie the same behaviour as non-multipathed nvme
>> devices. The alternative behaviour can be enabled with the new sysfs
>> attribute 'no_path_detach'.
> 
> This looks ok to me. I have heard from a few people that they expected
> it to work this way with the option enabled, but I suppose we do need to
> retain the old behavior as default.
> 
> Reviewed-by: Keith Busch <kbusch at kernel.org>
> 
Oh, I would _love_ to kill the old behaviour.
Especially as we now have fast_io_fail_tmo and ctrl_loss_tmo, which 
gives us enough control on how the controller and the remaining paths 
should behave (and which weren't present when fabrics got implemented).

We can make this behaviour the default, and kill the old approach next 
year if there are no complaints :-)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From lengchao at huawei.com  Thu Feb 25 19:59:42 2021
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 26 Feb 2021 08:59:42 +0800
Subject: [PATCH 5/6] nvme: return NVME_SC_ABORT_QUEUE for cancelled
 commands
In-Reply-To: <20210225105527.75277-6-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-6-hare@suse.de>
Message-ID: <fd89fa3f-0c25-1cd0-a47d-06fca59469b7@huawei.com>



On 2021/2/25 18:55, Hannes Reinecke wrote:
> A request will be cancelled when a queue is deleted, so we should
> be returning a status of NVME_SC_ABORT_QUEUE.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/host/core.c | 2 +-
>   drivers/nvme/host/fc.c   | 9 ++++++---
>   2 files changed, 7 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 04dc85e0810c..624531a6b7c6 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -379,7 +379,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
>   	if (blk_mq_request_completed(req))
>   		return true;
>   
> -	nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
> +	nvme_req(req)->status = NVME_SC_ABORT_QUEUE;
Use NVME_SC_ABORT_QUEUE instead of NVME_SC_HOST_ABORTED_CMD is not a good idea.
NVME_SC_HOST_ABORTED_CMD is a host path related error, Now used to check if need
fail over retry by multipath.
>   	nvme_req(req)->flags |= NVME_REQ_CANCELLED;
>   	blk_mq_complete_request(req);
>   	return true;
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 97e3424c7b03..86395b66310d 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -1955,9 +1955,12 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
>   	fc_dma_sync_single_for_cpu(ctrl->lport->dev, op->fcp_req.rspdma,
>   				sizeof(op->rsp_iu), DMA_FROM_DEVICE);
>   
> -	if (opstate == FCPOP_STATE_ABORTED)
> -		status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
> -	else if (freq->status) {
> +	if (opstate == FCPOP_STATE_ABORTED) {
> +		if (op->nreq.flags & NVME_REQ_CANCELLED)
> +			status = cpu_to_le16(NVME_SC_ABORT_QUEUE << 1);
> +		else
> +			status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
> +	} else if (freq->status) {
>   		status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1);
>   		dev_info(ctrl->ctrl.device,
>   			"NVME-FC{%d}: io failed due to lldd error %d\n",
> 


From lengchao at huawei.com  Thu Feb 25 20:07:16 2021
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 26 Feb 2021 09:07:16 +0800
Subject: [PATCH 6/6] nvme: return -ETIMEDOUT in nvme_submit_sync_cmd()
In-Reply-To: <20210225105527.75277-7-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-7-hare@suse.de>
Message-ID: <8028fcce-0eb6-f375-070e-4f85451248cd@huawei.com>

Looks good to me.
Maybe it is better to split multiple patches for different transports.


On 2021/2/25 18:55, Hannes Reinecke wrote:
> When a command times out we should be returning -ETIMEDOUT in
> nvme_submit_sync_cmd(), and not requiring the caller to guess whether
> a specific NVMe status should be interpreted as a timeout.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/host/core.c | 2 ++
>   drivers/nvme/host/fc.c   | 1 +
>   drivers/nvme/host/nvme.h | 1 +
>   drivers/nvme/host/rdma.c | 1 +
>   drivers/nvme/host/tcp.c  | 1 +
>   5 files changed, 6 insertions(+)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 624531a6b7c6..5c1ec3bfb9d3 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -1004,6 +1004,8 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd,
>   		*result = nvme_req(req)->result;
>   	if (nvme_req(req)->flags & NVME_REQ_CANCELLED)
>   		ret = -EINTR;
> +	else if (nvme_req(req)->flags & NVME_REQ_TIMEOUT)
> +		ret = -ETIMEDOUT;
>   	else
>   		ret = nvme_req(req)->status;
>    out:
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 86395b66310d..bcc086fcbe61 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2551,6 +2551,7 @@ nvme_fc_timeout(struct request *rq, bool reserved)
>   		"x%08x/x%08x\n",
>   		ctrl->cnum, op->queue->qnum, sqe->common.opcode,
>   		sqe->connect.fctype, sqe->common.cdw10, sqe->common.cdw11);
> +	op->nreq.flags |= NVME_REQ_TIMEOUT;
>   	if (__nvme_fc_abort_op(ctrl, op))
>   		nvme_fc_error_recovery(ctrl, "io timeout abort failed");
>   
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index 07b34175c6ce..02aa4ba665c0 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -173,6 +173,7 @@ struct nvme_request {
>   enum {
>   	NVME_REQ_CANCELLED		= (1 << 0),
>   	NVME_REQ_USERCMD		= (1 << 1),
> +	NVME_REQ_TIMEOUT		= (1 << 2),
>   };
>   
>   static inline struct nvme_request *nvme_req(struct request *req)
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 53ac4d7442ba..99ad2e10b295 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -1990,6 +1990,7 @@ static void nvme_rdma_complete_timed_out(struct request *rq)
>   	nvme_rdma_stop_queue(queue);
>   	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> +		nvme_req(rq)->flags |= NVME_REQ_TIMEOUT;
>   		blk_mq_complete_request(rq);
>   	}
>   }
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 69f59d2c5799..64aab742c84d 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -2189,6 +2189,7 @@ static void nvme_tcp_complete_timed_out(struct request *rq)
>   	nvme_tcp_stop_queue(ctrl, nvme_tcp_queue_id(req->queue));
>   	if (blk_mq_request_started(rq) && !blk_mq_request_completed(rq)) {
>   		nvme_req(rq)->status = NVME_SC_HOST_ABORTED_CMD;
> +		nvme_req(rq)->flags |= NVME_REQ_TIMEOUT;
>   		blk_mq_complete_request(rq);
>   	}
>   }
> 


From lengchao at huawei.com  Thu Feb 25 20:13:14 2021
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 26 Feb 2021 09:13:14 +0800
Subject: [PATCH 1/6] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <20210225105527.75277-2-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-2-hare@suse.de>
Message-ID: <17a30d58-6767-10a2-17b3-4fd62887109c@huawei.com>



On 2021/2/25 18:55, Hannes Reinecke wrote:
> We only should remove namespaces when we get fatal error back from
> the device or when the namespace IDs have changed.
> So instead of painfully masking out error numbers which might indicate
> that the error should be ignored we could use an NVME status code
> to indicated when the namespace should be removed.
> That simplifies the final logic and makes it less error-prone.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> ---
>   drivers/nvme/host/core.c | 6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index d77f3f26d8d3..13ed7d3fbbb3 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -4036,7 +4036,7 @@ static void nvme_ns_remove_by_nsid(struct nvme_ctrl *ctrl, u32 nsid)
>   static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>   {
>   	struct nvme_id_ns *id;
> -	int ret = -ENODEV;
> +	int ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>   
>   	if (test_bit(NVME_NS_DEAD, &ns->flags))
>   		goto out;
> @@ -4045,7 +4045,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>   	if (ret)
>   		goto out;
>   
> -	ret = -ENODEV;
> +	ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>   	if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
>   		dev_err(ns->ctrl->device,
>   			"identifiers changed for nsid %d\n", ns->head->ns_id);
> @@ -4063,7 +4063,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>   	 *
>   	 * TODO: we should probably schedule a delayed retry here.
>   	 */
> -	if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR)))
> +	if (ret > 0 && (ret & NVME_SC_DNR))
nvme_identify_ns may return -NODEV.
if ret == -NODEV, also need to remove namespaces.
>   		nvme_ns_remove(ns);
>   }
>   
> 


From lengchao at huawei.com  Thu Feb 25 20:13:34 2021
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 26 Feb 2021 09:13:34 +0800
Subject: [PATCH 2/6] nvme: add NVME_REQ_CANCELLED flag in
 nvme_cancel_request()
In-Reply-To: <20210225105527.75277-3-hare@suse.de>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-3-hare@suse.de>
Message-ID: <77e41673-8fb0-a399-fee4-ebbf622f44f7@huawei.com>

Reviewed-by: Chao Leng <lengchao at huawei.com>

On 2021/2/25 18:55, Hannes Reinecke wrote:
> NVME_REQ_CANCELLED is translated into -EINTR in nvme_submit_sync_cmd(),
> so we should be setting this flags during nvme_cancel_request() to
> ensure that the callers to nvme_submit_sync_cmd() will get the correct
> error code when the controller is reset.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>
> Reviewed-by: Daniel Wagner <dwagner at suse.de>
> Reviewed-by: Keith Busch <kbusch at kernel.org>
> ---
>   drivers/nvme/host/core.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 13ed7d3fbbb3..04dc85e0810c 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -380,6 +380,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
>   		return true;
>   
>   	nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
> +	nvme_req(req)->flags |= NVME_REQ_CANCELLED;
>   	blk_mq_complete_request(req);
>   	return true;
>   }
> 


From lengchao at huawei.com  Thu Feb 25 22:51:17 2021
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 26 Feb 2021 11:51:17 +0800
Subject: [PATCH V8 0/4] blk-mq: implement queue quiesce via percpu_ref for
 BLK_MQ_F_BLOCKING
In-Reply-To: <20201020085555.1554255-1-ming.lei@redhat.com>
References: <20201020085555.1554255-1-ming.lei@redhat.com>
Message-ID: <cc732195-c053-9ce4-e1a7-e7f6dcf762ac@huawei.com>

About nvme_stop_queues need long times for large number namespaces,
If work with multipath and one path fails, It cause wait long times
to fail over to retry, and the more namespaces the longer the time.
This has a great impact on delay-sensitive services.
there are two options to fix it:
1. Use percpu instead of SRCU. Ming's patchset.
2. Use tagset quiesce interface with SRCU. Sagi's patchset.
The two patchsets are still pending.

It is a serious bug, I expect that we can revisit the solution.
Maybe we don't have the best option, but we need to choose a relatively
acceptable option.

Can we fix the bug for non-blocking queues(which used by fc&rdma) first?

Sagi & Ming, what do you think?
Thank you.

On 2020/10/20 16:55, Ming Lei wrote:
> Hi Jens,
> 
> The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce,
> and prepares for replacing srcu with percpu_ref.
> 
> The 2nd patch replaces srcu with percpu_ref.
> 
> The 3rd patch adds tagset quiesce interface.
> 
> The 4th patch applies tagset quiesce interface for NVMe subsystem.
> 
> V8:
> 	- rebase on latest linus tree, only there is small fuzz change on 2/4
> 
> V7:
> 	- base on latest for-5.10/block, only there is small change on 2/4
> 
> V6:
> 	- base on for-5.10/block directly, instead of being against on patchset of
> 	'percpu_ref & block: reduce memory footprint of percpu_ref in fast path',
> 	because these patches don't depend on that patchset.
> 
> V5:
> 	- warn once in case that driver unquiesces its queue being
> 	  quiesce and not done, only patch 2 is modified
> 
> V4:
> 	- remove .mq_quiesce_mutex, and switch to test_and_[set|clear] for
> 	avoiding duplicated quiesce action
> 	- pass blktests(block, nvme)
> 
> V3:
> 	- add tagset quiesce interface
> 	- apply tagset quiesce interface for NVMe
> 	- pass blktests(block, nvme)
> 
> V2:
> 	- add .mq_quiesce_lock
> 	- add comment on patch 2 wrt. handling hctx_lock() failure
> 	- trivial patch style change
> 
> 
> Ming Lei (3):
>    block: use test_and_{clear|test}_bit to set/clear QUEUE_FLAG_QUIESCED
>    blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING
>    blk-mq: add tagset quiesce interface
> 
> Sagi Grimberg (1):
>    nvme: use blk_mq_[un]quiesce_tagset
> 
>   block/blk-core.c         |  13 +++
>   block/blk-mq-sysfs.c     |   2 -
>   block/blk-mq.c           | 182 +++++++++++++++++++++++++--------------
>   block/blk-sysfs.c        |   6 +-
>   block/blk.h              |   2 +
>   drivers/nvme/host/core.c |  19 ++--
>   include/linux/blk-mq.h   |  10 +--
>   include/linux/blkdev.h   |   4 +
>   8 files changed, 154 insertions(+), 84 deletions(-)
> 
> Cc: Hannes Reinecke <hare at suse.de>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Cc: Bart Van Assche <bvanassche at acm.org>
> Cc: Johannes Thumshirn <Johannes.Thumshirn at wdc.com>
> Cc: Chao Leng <lengchao at huawei.com>
> 


From hare at suse.de  Fri Feb 26 02:00:09 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:00:09 +0100
Subject: [PATCH 5/6] nvme: return NVME_SC_ABORT_QUEUE for cancelled
 commands
In-Reply-To: <fd89fa3f-0c25-1cd0-a47d-06fca59469b7@huawei.com>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-6-hare@suse.de>
 <fd89fa3f-0c25-1cd0-a47d-06fca59469b7@huawei.com>
Message-ID: <94ba840f-afc4-0e88-db79-1f7b29baaba7@suse.de>

On 2/26/21 1:59 AM, Chao Leng wrote:
> 
> 
> On 2021/2/25 18:55, Hannes Reinecke wrote:
>> A request will be cancelled when a queue is deleted, so we should
>> be returning a status of NVME_SC_ABORT_QUEUE.
>>
>> Signed-off-by: Hannes Reinecke <hare at suse.de>
>> ---
>> ? drivers/nvme/host/core.c | 2 +-
>> ? drivers/nvme/host/fc.c?? | 9 ++++++---
>> ? 2 files changed, 7 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index 04dc85e0810c..624531a6b7c6 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -379,7 +379,7 @@ bool nvme_cancel_request(struct request *req, void 
>> *data, bool reserved)
>> ????? if (blk_mq_request_completed(req))
>> ????????? return true;
>> -??? nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
>> +??? nvme_req(req)->status = NVME_SC_ABORT_QUEUE;
> Use NVME_SC_ABORT_QUEUE instead of NVME_SC_HOST_ABORTED_CMD is not a 
> good idea.
> NVME_SC_HOST_ABORTED_CMD is a host path related error, Now used to check 
> if need fail over retry by multipath.

You are correct; this patch will be dropped for the next iteration.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Fri Feb 26 02:10:18 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:10:18 +0100
Subject: [PATCH 1/6] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <17a30d58-6767-10a2-17b3-4fd62887109c@huawei.com>
References: <20210225105527.75277-1-hare@suse.de>
 <20210225105527.75277-2-hare@suse.de>
 <17a30d58-6767-10a2-17b3-4fd62887109c@huawei.com>
Message-ID: <99ab19f7-0321-efcb-5e5d-8fb4c1e6aea9@suse.de>

On 2/26/21 2:13 AM, Chao Leng wrote:
> 
> 
> On 2021/2/25 18:55, Hannes Reinecke wrote:
>> We only should remove namespaces when we get fatal error back from
>> the device or when the namespace IDs have changed.
>> So instead of painfully masking out error numbers which might indicate
>> that the error should be ignored we could use an NVME status code
>> to indicated when the namespace should be removed.
>> That simplifies the final logic and makes it less error-prone.
>>
>> Signed-off-by: Hannes Reinecke <hare at suse.de>
>> ---
>> ? drivers/nvme/host/core.c | 6 +++---
>> ? 1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index d77f3f26d8d3..13ed7d3fbbb3 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -4036,7 +4036,7 @@ static void nvme_ns_remove_by_nsid(struct 
>> nvme_ctrl *ctrl, u32 nsid)
>> ? static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids 
>> *ids)
>> ? {
>> ????? struct nvme_id_ns *id;
>> -??? int ret = -ENODEV;
>> +??? int ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>> ????? if (test_bit(NVME_NS_DEAD, &ns->flags))
>> ????????? goto out;
>> @@ -4045,7 +4045,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, 
>> struct nvme_ns_ids *ids)
>> ????? if (ret)
>> ????????? goto out;
>> -??? ret = -ENODEV;
>> +??? ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>> ????? if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
>> ????????? dev_err(ns->ctrl->device,
>> ????????????? "identifiers changed for nsid %d\n", ns->head->ns_id);
>> @@ -4063,7 +4063,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, 
>> struct nvme_ns_ids *ids)
>> ?????? *
>> ?????? * TODO: we should probably schedule a delayed retry here.
>> ?????? */
>> -??? if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR)))
>> +??? if (ret > 0 && (ret & NVME_SC_DNR))
> nvme_identify_ns may return -NODEV.
> if ret == -NODEV, also need to remove namespaces.

You are correct. Will be updating the patch.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From hare at suse.de  Fri Feb 26 02:17:26 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:17:26 +0100
Subject: [PATCH 2/4] nvme: add NVME_REQ_CANCELLED flag in nvme_cancel_request()
In-Reply-To: <20210226071728.106923-1-hare@suse.de>
References: <20210226071728.106923-1-hare@suse.de>
Message-ID: <20210226071728.106923-3-hare@suse.de>

NVME_REQ_CANCELLED is translated into -EINTR in nvme_submit_sync_cmd(),
so we should be setting this flags during nvme_cancel_request() to
ensure that the callers to nvme_submit_sync_cmd() will get the correct
error code when the controller is reset.

Signed-off-by: Hannes Reinecke <hare at suse.de>
Reviewed-by: Daniel Wagner <dwagner at suse.de>
Reviewed-by: Keith Busch <kbusch at kernel.org>
Reviewed-by: Chao Leng <lengchao at huawei.com>
---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index bde1b22ca7f2..bf32460c180a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -380,6 +380,7 @@ bool nvme_cancel_request(struct request *req, void *data, bool reserved)
 		return true;
 
 	nvme_req(req)->status = NVME_SC_HOST_ABORTED_CMD;
+	nvme_req(req)->flags |= NVME_REQ_CANCELLED;
 	blk_mq_complete_request(req);
 	return true;
 }
-- 
2.29.2



From hare at suse.de  Fri Feb 26 02:17:24 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:17:24 +0100
Subject: [PATCHv4 0/4] Fixup return value for nvme_submit_sync_cmd()
Message-ID: <20210226071728.106923-1-hare@suse.de>

Hi all,

here are some small patches for fixing up the return value of
nvme_submit_sync_cmd().
As Keith correctly noted, nvme_submit_sync_cmd() should be returning
an error if the command could not be performed; however, currently
only pci does that.
So we need to fix up nvme_cancel_request() to return an -EINTR
on any pending sync commands during reset.
And modify nvme-fc to return the same nvme status after timing out
or cancelling requests.

As usual, comments and reviews are welcome.

Changes to v3:
- Dropped patch to change the status code for aborted commands
- Dropped patch to set -ETIMEDOUT on timed out commands
- Include reviews from Chao Leng

Changes to v2:
- Include reviews from Sagi
- Simplify error logic from nvme_validate_ns()

Changes to v1:
- Include reviews from Daniel
- Include changes for nvme-fc to return the same status as the
  other transports

Hannes Reinecke (4):
  nvme: simplify error logic in nvme_validate_ns()
  nvme: add NVME_REQ_CANCELLED flag in nvme_cancel_request()
  nvme-fc: set NVME_REQ_CANCELLED in nvme_fc_terminate_exchange()
  nvme-fc: return NVME_SC_HOST_ABORTED_CMD when a command has been
    aborted

 drivers/nvme/host/core.c | 9 +++++----
 drivers/nvme/host/fc.c   | 3 ++-
 2 files changed, 7 insertions(+), 5 deletions(-)

-- 
2.29.2



From hare at suse.de  Fri Feb 26 02:17:28 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:17:28 +0100
Subject: [PATCH 4/4] nvme-fc: return NVME_SC_HOST_ABORTED_CMD when a command
 has been aborted
In-Reply-To: <20210226071728.106923-1-hare@suse.de>
References: <20210226071728.106923-1-hare@suse.de>
Message-ID: <20210226071728.106923-5-hare@suse.de>

When a command has been aborted we should return NVME_SC_HOST_ABORTED_CMD
to be consistent with the other transports.

Cc: James Smart <james.smart at broadcom.com>
Signed-off-by: Hannes Reinecke <hare at suse.de>
Reviewed-by: Daniel Wagner <dwagner at suse.de>
---
 drivers/nvme/host/fc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index ef12a619daec..97e3424c7b03 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -1956,7 +1956,7 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
 				sizeof(op->rsp_iu), DMA_FROM_DEVICE);
 
 	if (opstate == FCPOP_STATE_ABORTED)
-		status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1);
+		status = cpu_to_le16(NVME_SC_HOST_ABORTED_CMD << 1);
 	else if (freq->status) {
 		status = cpu_to_le16(NVME_SC_HOST_PATH_ERROR << 1);
 		dev_info(ctrl->ctrl.device,
-- 
2.29.2



From hare at suse.de  Fri Feb 26 02:17:27 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:17:27 +0100
Subject: [PATCH 3/4] nvme-fc: set NVME_REQ_CANCELLED in
 nvme_fc_terminate_exchange()
In-Reply-To: <20210226071728.106923-1-hare@suse.de>
References: <20210226071728.106923-1-hare@suse.de>
Message-ID: <20210226071728.106923-4-hare@suse.de>

nvme_fc_terminate_exchange() is being called when exchanges are
being deleted, and as such we should be setting the NVME_REQ_CANCELLED
flag to have identical behaviour on all transports.

Cc: James Smart <james.smart at broadcom.com>
Signed-off-by: Hannes Reinecke <hare at suse.de>
Reviewed-by: Daniel Wagner <dwagner at suse.de>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/fc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 20dadd86e981..ef12a619daec 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2443,6 +2443,7 @@ nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved)
 	struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl);
 	struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req);
 
+	op->nreq.flags |= NVME_REQ_CANCELLED;
 	__nvme_fc_abort_op(ctrl, op);
 	return true;
 }
-- 
2.29.2



From hare at suse.de  Fri Feb 26 02:17:25 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:17:25 +0100
Subject: [PATCH 1/4] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <20210226071728.106923-1-hare@suse.de>
References: <20210226071728.106923-1-hare@suse.de>
Message-ID: <20210226071728.106923-2-hare@suse.de>

We only should remove namespaces when we get fatal error back from
the device or when the namespace IDs have changed.
So instead of painfully masking out error numbers which might indicate
that the error should be ignored we could use an NVME status code
to indicated when the namespace should be removed.
That simplifies the final logic and makes it less error-prone.

Signed-off-by: Hannes Reinecke <hare at suse.de>
Reviewed-by: Keith Busch <kbusch at kernel.org>
---
 drivers/nvme/host/core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d77f3f26d8d3..bde1b22ca7f2 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1439,7 +1439,7 @@ static int nvme_identify_ns(struct nvme_ctrl *ctrl, unsigned nsid,
 		goto out_free_id;
 	}
 
-	error = -ENODEV;
+	error = NVME_SC_INVALID_NS | NVME_SC_DNR;
 	if ((*id)->ncap == 0) /* namespace not allocated or attached */
 		goto out_free_id;
 
@@ -4036,7 +4036,7 @@ static void nvme_ns_remove_by_nsid(struct nvme_ctrl *ctrl, u32 nsid)
 static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
 {
 	struct nvme_id_ns *id;
-	int ret = -ENODEV;
+	int ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
 
 	if (test_bit(NVME_NS_DEAD, &ns->flags))
 		goto out;
@@ -4045,7 +4045,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
 	if (ret)
 		goto out;
 
-	ret = -ENODEV;
+	ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
 	if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
 		dev_err(ns->ctrl->device,
 			"identifiers changed for nsid %d\n", ns->head->ns_id);
@@ -4063,7 +4063,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
 	 *
 	 * TODO: we should probably schedule a delayed retry here.
 	 */
-	if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR)))
+	if (ret > 0 && (ret & NVME_SC_DNR))
 		nvme_ns_remove(ns);
 }
 
-- 
2.29.2



From hare at suse.de  Fri Feb 26 02:52:51 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 08:52:51 +0100
Subject: [PATCH] nvme: add 'kato' sysfs attribute
Message-ID: <20210226075251.108025-1-hare@suse.de>

Add a 'kato' controller sysfs attribute to display the current
keep-alive timeout value (if any). This allows userspace to identify
persistent discovery controllers, as these will have a non-zero
KATO value.

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/core.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1d5fc8a6fd37..9ad23472ac15 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3547,6 +3547,7 @@ nvme_show_int_function(cntlid);
 nvme_show_int_function(numa_node);
 nvme_show_int_function(queue_count);
 nvme_show_int_function(sqsize);
+nvme_show_int_function(kato);
 
 static ssize_t nvme_sysfs_delete(struct device *dev,
 				struct device_attribute *attr, const char *buf,
@@ -3749,6 +3750,7 @@ static struct attribute *nvme_dev_attrs[] = {
 	&dev_attr_ctrl_loss_tmo.attr,
 	&dev_attr_fast_io_fail_tmo.attr,
 	&dev_attr_reconnect_delay.attr,
+	&dev_attr_kato.attr,
 	NULL
 };
 
-- 
2.26.2



From hare at suse.de  Fri Feb 26 03:05:38 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 09:05:38 +0100
Subject: [PATCH] nvme: send uevent once a multipath namespace is operational
 again
Message-ID: <20210226080538.108392-1-hare@suse.de>

In an all paths down scenario I/O will be requeued or aborted, and no
further I/O will be ongoing on this namespace.
This leaves the upper layers unable to determine if the namespace
becomes operational again eg. after a successful controller reset.
With this patch a 'change' uevent will be sent per multipathed namespace
once the underlying controller moved to LIVE and started I/O processing
by calling nvme_kick_requeue_lists().

Signed-off-by: Hannes Reinecke <hare at suse.de>
---
 drivers/nvme/host/multipath.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 7d38f9272490..844d952bd86f 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -97,8 +97,12 @@ void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl)
 
 	down_read(&ctrl->namespaces_rwsem);
 	list_for_each_entry(ns, &ctrl->namespaces, list) {
-		if (ns->head->disk)
-			kblockd_schedule_work(&ns->head->requeue_work);
+		if (!ns->head->disk)
+			continue;
+		kblockd_schedule_work(&ns->head->requeue_work);
+		if (ctrl->state == NVME_CTRL_LIVE)
+			kobject_uevent(&disk_to_dev(ns->head->disk)->kobj,
+				       KOBJ_CHANGE);
 	}
 	up_read(&ctrl->namespaces_rwsem);
 }
-- 
2.26.2



From dwagner at suse.de  Fri Feb 26 04:10:01 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 26 Feb 2021 10:10:01 +0100
Subject: [PATCH 1/4] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <20210226071728.106923-2-hare@suse.de>
References: <20210226071728.106923-1-hare@suse.de>
 <20210226071728.106923-2-hare@suse.de>
Message-ID: <20210226091001.jywzp35yjpr6ng4d@beryllium.lan>

Hi Hannes,

On Fri, Feb 26, 2021 at 08:17:25AM +0100, Hannes Reinecke wrote:
> @@ -4045,7 +4045,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>  	if (ret)
>  		goto out;
>  
> -	ret = -ENODEV;
> +	ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>  	if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
>  		dev_err(ns->ctrl->device,
>  			"identifiers changed for nsid %d\n", ns->head->ns_id);
> @@ -4063,7 +4063,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>  	 *
>  	 * TODO: we should probably schedule a delayed retry here.
>  	 */
> -	if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR)))
> +	if (ret > 0 && (ret & NVME_SC_DNR))
>  		nvme_ns_remove(ns);

I think Chao's comment still holds, nvme_update_ns_info() can return
-EINVAL, -IEO or -ENODEV. Don't we have to remove the NS in these
cases?

Thanks,
Daniel


From dwagner at suse.de  Fri Feb 26 04:11:29 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 26 Feb 2021 10:11:29 +0100
Subject: [PATCH] nvme: add 'kato' sysfs attribute
In-Reply-To: <20210226075251.108025-1-hare@suse.de>
References: <20210226075251.108025-1-hare@suse.de>
Message-ID: <20210226091129.ejvejjku5u7erdj5@beryllium.lan>

On Fri, Feb 26, 2021 at 08:52:51AM +0100, Hannes Reinecke wrote:
> Add a 'kato' controller sysfs attribute to display the current
> keep-alive timeout value (if any). This allows userspace to identify
> persistent discovery controllers, as these will have a non-zero
> KATO value.
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>

Reviewed-by: Daniel Wagner <dwagner at suse.de>


From dwagner at suse.de  Fri Feb 26 04:17:22 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 26 Feb 2021 10:17:22 +0100
Subject: [PATCH] nvme: send uevent once a multipath namespace is
 operational again
In-Reply-To: <20210226080538.108392-1-hare@suse.de>
References: <20210226080538.108392-1-hare@suse.de>
Message-ID: <20210226091722.5flr3pek2raumunc@beryllium.lan>

On Fri, Feb 26, 2021 at 09:05:38AM +0100, Hannes Reinecke wrote:
> In an all paths down scenario I/O will be requeued or aborted, and no
> further I/O will be ongoing on this namespace.
> This leaves the upper layers unable to determine if the namespace
> becomes operational again eg. after a successful controller reset.
> With this patch a 'change' uevent will be sent per multipathed namespace
> once the underlying controller moved to LIVE and started I/O processing
> by calling nvme_kick_requeue_lists().
> 
> Signed-off-by: Hannes Reinecke <hare at suse.de>

Reviewed-by: Daniel Wagner <dwagner at suse.de>



From lengchao at huawei.com  Fri Feb 26 04:21:56 2021
From: lengchao at huawei.com (Chao Leng)
Date: Fri, 26 Feb 2021 17:21:56 +0800
Subject: [PATCHv4 0/4] Fixup return value for nvme_submit_sync_cmd()
In-Reply-To: <20210226071728.106923-1-hare@suse.de>
References: <20210226071728.106923-1-hare@suse.de>
Message-ID: <1e5e84d1-4c28-47a8-cb6d-98e3764209d2@huawei.com>



On 2021/2/26 15:17, Hannes Reinecke wrote:
> Hi all,
> 
> here are some small patches for fixing up the return value of
> nvme_submit_sync_cmd().
> As Keith correctly noted, nvme_submit_sync_cmd() should be returning
> an error if the command could not be performed; however, currently
> only pci does that.
> So we need to fix up nvme_cancel_request() to return an -EINTR
> on any pending sync commands during reset.
> And modify nvme-fc to return the same nvme status after timing out
> or cancelling requests.
> 
> As usual, comments and reviews are welcome.
> 
> Changes to v3:
> - Dropped patch to change the status code for aborted commands
> - Dropped patch to set -ETIMEDOUT on timed out commands
->timeout() also need to consider, maybe can add the flag:NVME_REQ_CANCELLED
like nvme_cancel_request.
> - Include reviews from Chao Leng
> 
> Changes to v2:
> - Include reviews from Sagi
> - Simplify error logic from nvme_validate_ns()
> 
> Changes to v1:
> - Include reviews from Daniel
> - Include changes for nvme-fc to return the same status as the
>    other transports
> 
> Hannes Reinecke (4):
>    nvme: simplify error logic in nvme_validate_ns()
>    nvme: add NVME_REQ_CANCELLED flag in nvme_cancel_request()
>    nvme-fc: set NVME_REQ_CANCELLED in nvme_fc_terminate_exchange()
>    nvme-fc: return NVME_SC_HOST_ABORTED_CMD when a command has been
>      aborted
> 
>   drivers/nvme/host/core.c | 9 +++++----
>   drivers/nvme/host/fc.c   | 3 ++-
>   2 files changed, 7 insertions(+), 5 deletions(-)
> 


From hare at suse.de  Fri Feb 26 04:24:46 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 10:24:46 +0100
Subject: [PATCH 1/4] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <20210226091001.jywzp35yjpr6ng4d@beryllium.lan>
References: <20210226071728.106923-1-hare@suse.de>
 <20210226071728.106923-2-hare@suse.de>
 <20210226091001.jywzp35yjpr6ng4d@beryllium.lan>
Message-ID: <7f70f597-17aa-dd09-cb02-fed3172749bb@suse.de>

On 2/26/21 10:10 AM, Daniel Wagner wrote:
> Hi Hannes,
> 
> On Fri, Feb 26, 2021 at 08:17:25AM +0100, Hannes Reinecke wrote:
>> @@ -4045,7 +4045,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>>   	if (ret)
>>   		goto out;
>>   
>> -	ret = -ENODEV;
>> +	ret = NVME_SC_INVALID_NS | NVME_SC_DNR;
>>   	if (!nvme_ns_ids_equal(&ns->head->ids, ids)) {
>>   		dev_err(ns->ctrl->device,
>>   			"identifiers changed for nsid %d\n", ns->head->ns_id);
>> @@ -4063,7 +4063,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_ids *ids)
>>   	 *
>>   	 * TODO: we should probably schedule a delayed retry here.
>>   	 */
>> -	if (ret && ret != -ENOMEM && !(ret > 0 && !(ret & NVME_SC_DNR)))
>> +	if (ret > 0 && (ret & NVME_SC_DNR))
>>   		nvme_ns_remove(ns);
> 
> I think Chao's comment still holds, nvme_update_ns_info() can return
> -EINVAL, -IEO or -ENODEV. Don't we have to remove the NS in these
> cases?
> 
I thought I had fixed up all instances where we can return -ENODEV; and 
all others are indeed transport or transient error which should not 
result in a namespace removal.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From dwagner at suse.de  Fri Feb 26 04:55:43 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 26 Feb 2021 10:55:43 +0100
Subject: [PATCH 1/4] nvme: simplify error logic in nvme_validate_ns()
In-Reply-To: <7f70f597-17aa-dd09-cb02-fed3172749bb@suse.de>
References: <20210226071728.106923-1-hare@suse.de>
 <20210226071728.106923-2-hare@suse.de>
 <20210226091001.jywzp35yjpr6ng4d@beryllium.lan>
 <7f70f597-17aa-dd09-cb02-fed3172749bb@suse.de>
Message-ID: <20210226095543.664cq7gf7bvbq7zt@beryllium.lan>

On Fri, Feb 26, 2021 at 10:24:46AM +0100, Hannes Reinecke wrote:
> I thought I had fixed up all instances where we can return -ENODEV; and all
> others are indeed transport or transient error which should not result in a
> namespace removal.

I think I was digging through the wrong tree. There is indeed no ENODEV
in nvme/nvme-5.12. Sorry about the noise.

Reviewed-by: Daniel Wagner <dwagner at suse.de>




From dwagner at suse.de  Fri Feb 26 07:35:34 2021
From: dwagner at suse.de (Daniel Wagner)
Date: Fri, 26 Feb 2021 13:35:34 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <a2064070-b511-ba6d-bd64-0b3abc208356@grimberg.me>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
 <20210215104020.yyithlo2hkxqvguj@beryllium.lan>
 <a2064070-b511-ba6d-bd64-0b3abc208356@grimberg.me>
Message-ID: <20210226123534.4oovbzk4wrnfjp64@beryllium.lan>

On Mon, Feb 15, 2021 at 01:29:45PM -0800, Sagi Grimberg wrote:
> Well, I think we should probably figure out why that is happening first.

I got my hands on a tcpdump trace. I've trimmed it to this:


No.     Time           Source                Destination           Protocol Length Info
      1 0.000000       10.228.194.30         10.228.38.214         NVMe     138    NVMe Write
      2 0.000285       10.228.38.214         10.228.194.30         NVMe/TCP 90     Ready To Transfer
      3 0.000591       10.228.194.30         10.228.38.214         NVMe     4186   NVMe Write: Data
      4 0.000673       10.228.38.214         10.228.194.30         TCP      66     4420 ? 58535 [ACK] Seq=25 Ack=4193 Win=241 Len=0 TSval=2655324576 TSecr=1497295579
      5 0.002140       10.228.38.214         10.228.194.30         NVMe     90     NVMe Write: Response
      6 0.002511       10.228.194.30         10.228.38.175         NVMe     138    NVMe Write
      7 0.002812       10.228.38.175         10.228.194.30         NVMe/TCP 90     Ready To Transfer
      8 0.003006       10.228.194.30         10.228.38.175         NVMe     4186   NVMe Write: Data
      9 0.003098       10.228.38.175         10.228.194.30         TCP      66     4420 ? 51241 [ACK] Seq=25 Ack=4193 Win=241 Len=0 TSval=2183410196 TSecr=3601034207
     10 0.004420       10.228.38.175         10.228.194.30         NVMe     90     NVMe Write: Response
     11 0.004890       10.228.38.214         10.228.194.30         NVMe/TCP 90     
     12 0.004969       10.228.38.175         10.228.194.30         NVMe/TCP 90  


The last few seconds contain only normal writes and suddenly the host
receives two invalid packets. From what I see the host doesn't misbehave
at all. I wonder if it would be possible to detect the invalid packet by
locking at the PDU header only. If this would be possible we could
discard it early and do not try to use the invalid command id.

Here the last two packets with details:


No.     Time           Source                Destination           Protocol Length Info
     11 0.004890       10.228.38.214         10.228.194.30         NVMe/TCP 90     

Frame 11: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
    Encapsulation type: Ethernet (1)
    Arrival Time: Feb 23, 2021 18:16:08.780574000 CET
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1614100568.780574000 seconds
    [Time delta from previous captured frame: 0.000470000 seconds]
    [Time delta from previous displayed frame: 0.000470000 seconds]
    [Time since reference or first frame: 0.004890000 seconds]
    Frame Number: 11
    Frame Length: 90 bytes (720 bits)
    Capture Length: 90 bytes (720 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ethertype:ip:tcp:nvme-tcp]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Ethernet II, Src: IntelCor_41:16:c0 (b4:96:91:41:16:c0), Dst: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
    Destination: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
        Address: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
        Address: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.228.38.214, Dst: 10.228.194.30
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
        0000 00.. = Differentiated Services Codepoint: Default (0)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 76
    Identification: 0x0000 (0)
    Flags: 0x40, Don't fragment
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment Offset: 0
    Time to Live: 64
    Protocol: TCP (6)
    Header Checksum: 0x0000 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 10.228.38.214
    Destination Address: 10.228.194.30
Transmission Control Protocol, Src Port: 4420, Dst Port: 46909, Seq: 1, Ack: 1, Len: 24
    Source Port: 4420
    Destination Port: 46909
    [Stream index: 2]
    [TCP Segment Len: 24]
    Sequence Number: 1    (relative sequence number)
    Sequence Number (raw): 4175488220
    [Next Sequence Number: 25    (relative sequence number)]
    Acknowledgment Number: 1    (relative ack number)
    Acknowledgment number (raw): 2966903626
    1000 .... = Header Length: 32 bytes (8)
    Flags: 0x018 (PSH, ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 1... = Push: Set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set
        [TCP Flags: ???????AP???]
    Window: 257
    [Calculated window size: 257]
    [Window size scaling factor: -1 (unknown)]
    Checksum: 0xfefa [unverified]
    [Checksum Status: Unverified]
    Urgent Pointer: 0
    Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
        TCP Option - No-Operation (NOP)
            Kind: No-Operation (1)
        TCP Option - No-Operation (NOP)
            Kind: No-Operation (1)
        TCP Option - Timestamps: TSval 4211986351, TSecr 1497291787
            Kind: Time Stamp Option (8)
            Length: 10
            Timestamp value: 4211986351
            Timestamp echo reply: 1497291787
    [SEQ/ACK analysis]
        [Bytes in flight: 24]
        [Bytes sent since last PSH flag: 24]
    [Timestamps]
        [Time since first frame in this TCP stream: 0.000000000 seconds]
        [Time since previous frame in this TCP stream: 0.000000000 seconds]
    TCP payload (24 bytes)
    [PDU Size: 24]
NVM Express Fabrics TCP
    Pdu Type: CapsuleResponse (5)
    Pdu Specific Flags: 0x00
        .... ...0 = PDU Header Digest: Not set
        .... ..0. = PDU Data Digest: Not set
        .... .0.. = PDU Data Last: Not set
        .... 0... = PDU Data Success: Not set
    Pdu Header Length: 24
    Pdu Data Offset: 0
    Packet Length: 24
    Unknown Data: 02000400000000001c0000001f000000

0000  00 00 0c 9f f5 a8 b4 96 91 41 16 c0 08 00 45 00   .........A....E.
0010  00 4c 00 00 40 00 40 06 00 00 0a e4 26 d6 0a e4   .L.. at .@.....&...
0020  c2 1e 11 44 b7 3d f8 e0 e4 dc b0 d7 5b 4a 80 18   ...D.=......[J..
0030  01 01 fe fa 00 00 01 01 08 0a fb 0d cf af 59 3e   ..............Y>
0040  dc 0b 05 00 18 00 18 00 00 00 02 00 04 00 00 00   ................
0050  00 00 1c 00 00 00 1f 00 00 00                     ..........

No.     Time           Source                Destination           Protocol Length Info
     12 0.004969       10.228.38.175         10.228.194.30         NVMe/TCP 90     

Frame 12: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
    Encapsulation type: Ethernet (1)
    Arrival Time: Feb 23, 2021 18:16:08.780653000 CET
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1614100568.780653000 seconds
    [Time delta from previous captured frame: 0.000079000 seconds]
    [Time delta from previous displayed frame: 0.000079000 seconds]
    [Time since reference or first frame: 0.004969000 seconds]
    Frame Number: 12
    Frame Length: 90 bytes (720 bits)
    Capture Length: 90 bytes (720 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ethertype:ip:tcp:nvme-tcp]
    [Coloring Rule Name: TCP]
    [Coloring Rule String: tcp]
Ethernet II, Src: IntelCor_41:16:c0 (b4:96:91:41:16:c0), Dst: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
    Destination: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
        Address: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
        Address: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.228.38.175, Dst: 10.228.194.30
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
        0000 00.. = Differentiated Services Codepoint: Default (0)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 76
    Identification: 0x0000 (0)
    Flags: 0x40, Don't fragment
        0... .... = Reserved bit: Not set
        .1.. .... = Don't fragment: Set
        ..0. .... = More fragments: Not set
    Fragment Offset: 0
    Time to Live: 64
    Protocol: TCP (6)
    Header Checksum: 0x0000 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 10.228.38.175
    Destination Address: 10.228.194.30
Transmission Control Protocol, Src Port: 4420, Dst Port: 34895, Seq: 1, Ack: 1, Len: 24
    Source Port: 4420
    Destination Port: 34895
    [Stream index: 3]
    [TCP Segment Len: 24]
    Sequence Number: 1    (relative sequence number)
    Sequence Number (raw): 3092812012
    [Next Sequence Number: 25    (relative sequence number)]
    Acknowledgment Number: 1    (relative ack number)
    Acknowledgment number (raw): 2384147181
    1000 .... = Header Length: 32 bytes (8)
    Flags: 0x018 (PSH, ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 1... = Push: Set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set
        [TCP Flags: ???????AP???]
    Window: 257
    [Calculated window size: 257]
    [Window size scaling factor: -1 (unknown)]
    Checksum: 0xfed3 [unverified]
    [Checksum Status: Unverified]
    Urgent Pointer: 0
    Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
        TCP Option - No-Operation (NOP)
            Kind: No-Operation (1)
        TCP Option - No-Operation (NOP)
            Kind: No-Operation (1)
        TCP Option - Timestamps: TSval 3874335934, TSecr 3601030412
            Kind: Time Stamp Option (8)
            Length: 10
            Timestamp value: 3874335934
            Timestamp echo reply: 3601030412
    [SEQ/ACK analysis]
        [Bytes in flight: 24]
        [Bytes sent since last PSH flag: 24]
    [Timestamps]
        [Time since first frame in this TCP stream: 0.000000000 seconds]
        [Time since previous frame in this TCP stream: 0.000000000 seconds]
    TCP payload (24 bytes)
    [PDU Size: 24]
NVM Express Fabrics TCP
    Pdu Type: CapsuleResponse (5)
    Pdu Specific Flags: 0x00
        .... ...0 = PDU Header Digest: Not set
        .... ..0. = PDU Data Digest: Not set
        .... .0.. = PDU Data Last: Not set
        .... 0... = PDU Data Success: Not set
    Pdu Header Length: 24
    Pdu Data Offset: 0
    Packet Length: 24
    Unknown Data: 02000400000000001b0000001f000000

0000  00 00 0c 9f f5 a8 b4 96 91 41 16 c0 08 00 45 00   .........A....E.
0010  00 4c 00 00 40 00 40 06 00 00 0a e4 26 af 0a e4   .L.. at .@.....&...
0020  c2 1e 11 44 88 4f b8 58 90 ec 8e 1b 32 ed 80 18   ...D.O.X....2...
0030  01 01 fe d3 00 00 01 01 08 0a e6 ed ac be d6 a3   ................
0040  5d 0c 05 00 18 00 18 00 00 00 02 00 04 00 00 00   ]...............
0050  00 00 1b 00 00 00 1f 00 00 00                     ..........


From hare at suse.de  Fri Feb 26 07:54:00 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 13:54:00 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <20210226123534.4oovbzk4wrnfjp64@beryllium.lan>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
 <20210215104020.yyithlo2hkxqvguj@beryllium.lan>
 <a2064070-b511-ba6d-bd64-0b3abc208356@grimberg.me>
 <20210226123534.4oovbzk4wrnfjp64@beryllium.lan>
Message-ID: <9e209b12-3771-cdca-2c9d-50451061bd2a@suse.de>

On 2/26/21 1:35 PM, Daniel Wagner wrote:
> On Mon, Feb 15, 2021 at 01:29:45PM -0800, Sagi Grimberg wrote:
>> Well, I think we should probably figure out why that is happening first.
> 
> I got my hands on a tcpdump trace. I've trimmed it to this:
> 
> 
> No.     Time           Source                Destination           Protocol Length Info
>        1 0.000000       10.228.194.30         10.228.38.214         NVMe     138    NVMe Write
>        2 0.000285       10.228.38.214         10.228.194.30         NVMe/TCP 90     Ready To Transfer
>        3 0.000591       10.228.194.30         10.228.38.214         NVMe     4186   NVMe Write: Data
>        4 0.000673       10.228.38.214         10.228.194.30         TCP      66     4420 ? 58535 [ACK] Seq=25 Ack=4193 Win=241 Len=0 TSval=2655324576 TSecr=1497295579
>        5 0.002140       10.228.38.214         10.228.194.30         NVMe     90     NVMe Write: Response
>        6 0.002511       10.228.194.30         10.228.38.175         NVMe     138    NVMe Write
>        7 0.002812       10.228.38.175         10.228.194.30         NVMe/TCP 90     Ready To Transfer
>        8 0.003006       10.228.194.30         10.228.38.175         NVMe     4186   NVMe Write: Data
>        9 0.003098       10.228.38.175         10.228.194.30         TCP      66     4420 ? 51241 [ACK] Seq=25 Ack=4193 Win=241 Len=0 TSval=2183410196 TSecr=3601034207
>       10 0.004420       10.228.38.175         10.228.194.30         NVMe     90     NVMe Write: Response
>       11 0.004890       10.228.38.214         10.228.194.30         NVMe/TCP 90
>       12 0.004969       10.228.38.175         10.228.194.30         NVMe/TCP 90
> 
> 
> The last few seconds contain only normal writes and suddenly the host
> receives two invalid packets. From what I see the host doesn't misbehave
> at all. I wonder if it would be possible to detect the invalid packet by
> locking at the PDU header only. If this would be possible we could
> discard it early and do not try to use the invalid command id.
> 
> Here the last two packets with details:
> 
> 
> No.     Time           Source                Destination           Protocol Length Info
>       11 0.004890       10.228.38.214         10.228.194.30         NVMe/TCP 90
> 
> Frame 11: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
>      Encapsulation type: Ethernet (1)
>      Arrival Time: Feb 23, 2021 18:16:08.780574000 CET
>      [Time shift for this packet: 0.000000000 seconds]
>      Epoch Time: 1614100568.780574000 seconds
>      [Time delta from previous captured frame: 0.000470000 seconds]
>      [Time delta from previous displayed frame: 0.000470000 seconds]
>      [Time since reference or first frame: 0.004890000 seconds]
>      Frame Number: 11
>      Frame Length: 90 bytes (720 bits)
>      Capture Length: 90 bytes (720 bits)
>      [Frame is marked: False]
>      [Frame is ignored: False]
>      [Protocols in frame: eth:ethertype:ip:tcp:nvme-tcp]
>      [Coloring Rule Name: TCP]
>      [Coloring Rule String: tcp]
> Ethernet II, Src: IntelCor_41:16:c0 (b4:96:91:41:16:c0), Dst: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
>      Destination: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
>          Address: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
>          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
>          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>      Source: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
>          Address: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
>          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
>          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>      Type: IPv4 (0x0800)
> Internet Protocol Version 4, Src: 10.228.38.214, Dst: 10.228.194.30
>      0100 .... = Version: 4
>      .... 0101 = Header Length: 20 bytes (5)
>      Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
>          0000 00.. = Differentiated Services Codepoint: Default (0)
>          .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
>      Total Length: 76
>      Identification: 0x0000 (0)
>      Flags: 0x40, Don't fragment
>          0... .... = Reserved bit: Not set
>          .1.. .... = Don't fragment: Set
>          ..0. .... = More fragments: Not set
>      Fragment Offset: 0
>      Time to Live: 64
>      Protocol: TCP (6)
>      Header Checksum: 0x0000 [validation disabled]
>      [Header checksum status: Unverified]
>      Source Address: 10.228.38.214
>      Destination Address: 10.228.194.30
> Transmission Control Protocol, Src Port: 4420, Dst Port: 46909, Seq: 1, Ack: 1, Len: 24
>      Source Port: 4420
>      Destination Port: 46909
>      [Stream index: 2]
>      [TCP Segment Len: 24]
>      Sequence Number: 1    (relative sequence number)
>      Sequence Number (raw): 4175488220
>      [Next Sequence Number: 25    (relative sequence number)]
>      Acknowledgment Number: 1    (relative ack number)
>      Acknowledgment number (raw): 2966903626
>      1000 .... = Header Length: 32 bytes (8)
>      Flags: 0x018 (PSH, ACK)
>          000. .... .... = Reserved: Not set
>          ...0 .... .... = Nonce: Not set
>          .... 0... .... = Congestion Window Reduced (CWR): Not set
>          .... .0.. .... = ECN-Echo: Not set
>          .... ..0. .... = Urgent: Not set
>          .... ...1 .... = Acknowledgment: Set
>          .... .... 1... = Push: Set
>          .... .... .0.. = Reset: Not set
>          .... .... ..0. = Syn: Not set
>          .... .... ...0 = Fin: Not set
>          [TCP Flags: ???????AP???]
>      Window: 257
>      [Calculated window size: 257]
>      [Window size scaling factor: -1 (unknown)]
>      Checksum: 0xfefa [unverified]
>      [Checksum Status: Unverified]
>      Urgent Pointer: 0
>      Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
>          TCP Option - No-Operation (NOP)
>              Kind: No-Operation (1)
>          TCP Option - No-Operation (NOP)
>              Kind: No-Operation (1)
>          TCP Option - Timestamps: TSval 4211986351, TSecr 1497291787
>              Kind: Time Stamp Option (8)
>              Length: 10
>              Timestamp value: 4211986351
>              Timestamp echo reply: 1497291787
>      [SEQ/ACK analysis]
>          [Bytes in flight: 24]
>          [Bytes sent since last PSH flag: 24]
>      [Timestamps]
>          [Time since first frame in this TCP stream: 0.000000000 seconds]
>          [Time since previous frame in this TCP stream: 0.000000000 seconds]
>      TCP payload (24 bytes)
>      [PDU Size: 24]
> NVM Express Fabrics TCP
>      Pdu Type: CapsuleResponse (5)
>      Pdu Specific Flags: 0x00
>          .... ...0 = PDU Header Digest: Not set
>          .... ..0. = PDU Data Digest: Not set
>          .... .0.. = PDU Data Last: Not set
>          .... 0... = PDU Data Success: Not set
>      Pdu Header Length: 24
>      Pdu Data Offset: 0
>      Packet Length: 24
>      Unknown Data: 02000400000000001c0000001f000000
> 
> 0000  00 00 0c 9f f5 a8 b4 96 91 41 16 c0 08 00 45 00   .........A....E.
> 0010  00 4c 00 00 40 00 40 06 00 00 0a e4 26 d6 0a e4   .L.. at .@.....&...
> 0020  c2 1e 11 44 b7 3d f8 e0 e4 dc b0 d7 5b 4a 80 18   ...D.=......[J..
> 0030  01 01 fe fa 00 00 01 01 08 0a fb 0d cf af 59 3e   ..............Y>
> 0040  dc 0b 05 00 18 00 18 00 00 00 02 00 04 00 00 00   ................
> 0050  00 00 1c 00 00 00 1f 00 00 00                     ..........
> 
> No.     Time           Source                Destination           Protocol Length Info
>       12 0.004969       10.228.38.175         10.228.194.30         NVMe/TCP 90
> 
> Frame 12: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
>      Encapsulation type: Ethernet (1)
>      Arrival Time: Feb 23, 2021 18:16:08.780653000 CET
>      [Time shift for this packet: 0.000000000 seconds]
>      Epoch Time: 1614100568.780653000 seconds
>      [Time delta from previous captured frame: 0.000079000 seconds]
>      [Time delta from previous displayed frame: 0.000079000 seconds]
>      [Time since reference or first frame: 0.004969000 seconds]
>      Frame Number: 12
>      Frame Length: 90 bytes (720 bits)
>      Capture Length: 90 bytes (720 bits)
>      [Frame is marked: False]
>      [Frame is ignored: False]
>      [Protocols in frame: eth:ethertype:ip:tcp:nvme-tcp]
>      [Coloring Rule Name: TCP]
>      [Coloring Rule String: tcp]
> Ethernet II, Src: IntelCor_41:16:c0 (b4:96:91:41:16:c0), Dst: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
>      Destination: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
>          Address: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
>          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
>          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>      Source: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
>          Address: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
>          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
>          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>      Type: IPv4 (0x0800)
> Internet Protocol Version 4, Src: 10.228.38.175, Dst: 10.228.194.30
>      0100 .... = Version: 4
>      .... 0101 = Header Length: 20 bytes (5)
>      Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
>          0000 00.. = Differentiated Services Codepoint: Default (0)
>          .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
>      Total Length: 76
>      Identification: 0x0000 (0)
>      Flags: 0x40, Don't fragment
>          0... .... = Reserved bit: Not set
>          .1.. .... = Don't fragment: Set
>          ..0. .... = More fragments: Not set
>      Fragment Offset: 0
>      Time to Live: 64
>      Protocol: TCP (6)
>      Header Checksum: 0x0000 [validation disabled]
>      [Header checksum status: Unverified]
>      Source Address: 10.228.38.175
>      Destination Address: 10.228.194.30
> Transmission Control Protocol, Src Port: 4420, Dst Port: 34895, Seq: 1, Ack: 1, Len: 24
>      Source Port: 4420
>      Destination Port: 34895
>      [Stream index: 3]
>      [TCP Segment Len: 24]
>      Sequence Number: 1    (relative sequence number)
>      Sequence Number (raw): 3092812012
>      [Next Sequence Number: 25    (relative sequence number)]
>      Acknowledgment Number: 1    (relative ack number)
>      Acknowledgment number (raw): 2384147181
>      1000 .... = Header Length: 32 bytes (8)
>      Flags: 0x018 (PSH, ACK)
>          000. .... .... = Reserved: Not set
>          ...0 .... .... = Nonce: Not set
>          .... 0... .... = Congestion Window Reduced (CWR): Not set
>          .... .0.. .... = ECN-Echo: Not set
>          .... ..0. .... = Urgent: Not set
>          .... ...1 .... = Acknowledgment: Set
>          .... .... 1... = Push: Set
>          .... .... .0.. = Reset: Not set
>          .... .... ..0. = Syn: Not set
>          .... .... ...0 = Fin: Not set
>          [TCP Flags: ???????AP???]
>      Window: 257
>      [Calculated window size: 257]
>      [Window size scaling factor: -1 (unknown)]
>      Checksum: 0xfed3 [unverified]
>      [Checksum Status: Unverified]
>      Urgent Pointer: 0
>      Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
>          TCP Option - No-Operation (NOP)
>              Kind: No-Operation (1)
>          TCP Option - No-Operation (NOP)
>              Kind: No-Operation (1)
>          TCP Option - Timestamps: TSval 3874335934, TSecr 3601030412
>              Kind: Time Stamp Option (8)
>              Length: 10
>              Timestamp value: 3874335934
>              Timestamp echo reply: 3601030412
>      [SEQ/ACK analysis]
>          [Bytes in flight: 24]
>          [Bytes sent since last PSH flag: 24]
>      [Timestamps]
>          [Time since first frame in this TCP stream: 0.000000000 seconds]
>          [Time since previous frame in this TCP stream: 0.000000000 seconds]
>      TCP payload (24 bytes)
>      [PDU Size: 24]
> NVM Express Fabrics TCP
>      Pdu Type: CapsuleResponse (5)
>      Pdu Specific Flags: 0x00
>          .... ...0 = PDU Header Digest: Not set
>          .... ..0. = PDU Data Digest: Not set
>          .... .0.. = PDU Data Last: Not set
>          .... 0... = PDU Data Success: Not set
>      Pdu Header Length: 24
>      Pdu Data Offset: 0
>      Packet Length: 24
>      Unknown Data: 02000400000000001b0000001f000000
> 
> 0000  00 00 0c 9f f5 a8 b4 96 91 41 16 c0 08 00 45 00   .........A....E.
> 0010  00 4c 00 00 40 00 40 06 00 00 0a e4 26 af 0a e4   .L.. at .@.....&...
> 0020  c2 1e 11 44 88 4f b8 58 90 ec 8e 1b 32 ed 80 18   ...D.O.X....2...
> 0030  01 01 fe d3 00 00 01 01 08 0a e6 ed ac be d6 a3   ................
> 0040  5d 0c 05 00 18 00 18 00 00 00 02 00 04 00 00 00   ]...............
> 0050  00 00 1b 00 00 00 1f 00 00 00                     ..........
> 
As I suspected, we did receive an invalid frame.
Data digest would have saved us, but then it's not enabled.

So we do need to check if the request is valid before processing it.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From kbusch at kernel.org  Fri Feb 26 11:13:55 2021
From: kbusch at kernel.org (Keith Busch)
Date: Sat, 27 Feb 2021 01:13:55 +0900
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <9e209b12-3771-cdca-2c9d-50451061bd2a@suse.de>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
 <20210215104020.yyithlo2hkxqvguj@beryllium.lan>
 <a2064070-b511-ba6d-bd64-0b3abc208356@grimberg.me>
 <20210226123534.4oovbzk4wrnfjp64@beryllium.lan>
 <9e209b12-3771-cdca-2c9d-50451061bd2a@suse.de>
Message-ID: <20210226161355.GG31593@redsun51.ssa.fujisawa.hgst.com>

On Fri, Feb 26, 2021 at 01:54:00PM +0100, Hannes Reinecke wrote:
> On 2/26/21 1:35 PM, Daniel Wagner wrote:
> > On Mon, Feb 15, 2021 at 01:29:45PM -0800, Sagi Grimberg wrote:
> > > Well, I think we should probably figure out why that is happening first.
> > 
> > I got my hands on a tcpdump trace. I've trimmed it to this:
> > 
> > 
> > No.     Time           Source                Destination           Protocol Length Info
> >        1 0.000000       10.228.194.30         10.228.38.214         NVMe     138    NVMe Write
> >        2 0.000285       10.228.38.214         10.228.194.30         NVMe/TCP 90     Ready To Transfer
> >        3 0.000591       10.228.194.30         10.228.38.214         NVMe     4186   NVMe Write: Data
> >        4 0.000673       10.228.38.214         10.228.194.30         TCP      66     4420 ? 58535 [ACK] Seq=25 Ack=4193 Win=241 Len=0 TSval=2655324576 TSecr=1497295579
> >        5 0.002140       10.228.38.214         10.228.194.30         NVMe     90     NVMe Write: Response
> >        6 0.002511       10.228.194.30         10.228.38.175         NVMe     138    NVMe Write
> >        7 0.002812       10.228.38.175         10.228.194.30         NVMe/TCP 90     Ready To Transfer
> >        8 0.003006       10.228.194.30         10.228.38.175         NVMe     4186   NVMe Write: Data
> >        9 0.003098       10.228.38.175         10.228.194.30         TCP      66     4420 ? 51241 [ACK] Seq=25 Ack=4193 Win=241 Len=0 TSval=2183410196 TSecr=3601034207
> >       10 0.004420       10.228.38.175         10.228.194.30         NVMe     90     NVMe Write: Response
> >       11 0.004890       10.228.38.214         10.228.194.30         NVMe/TCP 90
> >       12 0.004969       10.228.38.175         10.228.194.30         NVMe/TCP 90
> > 
> > 
> > The last few seconds contain only normal writes and suddenly the host
> > receives two invalid packets. From what I see the host doesn't misbehave
> > at all. I wonder if it would be possible to detect the invalid packet by
> > locking at the PDU header only. If this would be possible we could
> > discard it early and do not try to use the invalid command id.
> > 
> > Here the last two packets with details:
> > 
> > 
> > No.     Time           Source                Destination           Protocol Length Info
> >       11 0.004890       10.228.38.214         10.228.194.30         NVMe/TCP 90
> > 
> > Frame 11: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
> >      Encapsulation type: Ethernet (1)
> >      Arrival Time: Feb 23, 2021 18:16:08.780574000 CET
> >      [Time shift for this packet: 0.000000000 seconds]
> >      Epoch Time: 1614100568.780574000 seconds
> >      [Time delta from previous captured frame: 0.000470000 seconds]
> >      [Time delta from previous displayed frame: 0.000470000 seconds]
> >      [Time since reference or first frame: 0.004890000 seconds]
> >      Frame Number: 11
> >      Frame Length: 90 bytes (720 bits)
> >      Capture Length: 90 bytes (720 bits)
> >      [Frame is marked: False]
> >      [Frame is ignored: False]
> >      [Protocols in frame: eth:ethertype:ip:tcp:nvme-tcp]
> >      [Coloring Rule Name: TCP]
> >      [Coloring Rule String: tcp]
> > Ethernet II, Src: IntelCor_41:16:c0 (b4:96:91:41:16:c0), Dst: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
> >      Destination: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
> >          Address: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
> >          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
> >          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
> >      Source: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
> >          Address: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
> >          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
> >          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
> >      Type: IPv4 (0x0800)
> > Internet Protocol Version 4, Src: 10.228.38.214, Dst: 10.228.194.30
> >      0100 .... = Version: 4
> >      .... 0101 = Header Length: 20 bytes (5)
> >      Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
> >          0000 00.. = Differentiated Services Codepoint: Default (0)
> >          .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
> >      Total Length: 76
> >      Identification: 0x0000 (0)
> >      Flags: 0x40, Don't fragment
> >          0... .... = Reserved bit: Not set
> >          .1.. .... = Don't fragment: Set
> >          ..0. .... = More fragments: Not set
> >      Fragment Offset: 0
> >      Time to Live: 64
> >      Protocol: TCP (6)
> >      Header Checksum: 0x0000 [validation disabled]
> >      [Header checksum status: Unverified]
> >      Source Address: 10.228.38.214
> >      Destination Address: 10.228.194.30
> > Transmission Control Protocol, Src Port: 4420, Dst Port: 46909, Seq: 1, Ack: 1, Len: 24
> >      Source Port: 4420
> >      Destination Port: 46909
> >      [Stream index: 2]
> >      [TCP Segment Len: 24]
> >      Sequence Number: 1    (relative sequence number)
> >      Sequence Number (raw): 4175488220
> >      [Next Sequence Number: 25    (relative sequence number)]
> >      Acknowledgment Number: 1    (relative ack number)
> >      Acknowledgment number (raw): 2966903626
> >      1000 .... = Header Length: 32 bytes (8)
> >      Flags: 0x018 (PSH, ACK)
> >          000. .... .... = Reserved: Not set
> >          ...0 .... .... = Nonce: Not set
> >          .... 0... .... = Congestion Window Reduced (CWR): Not set
> >          .... .0.. .... = ECN-Echo: Not set
> >          .... ..0. .... = Urgent: Not set
> >          .... ...1 .... = Acknowledgment: Set
> >          .... .... 1... = Push: Set
> >          .... .... .0.. = Reset: Not set
> >          .... .... ..0. = Syn: Not set
> >          .... .... ...0 = Fin: Not set
> >          [TCP Flags: ???????AP???]
> >      Window: 257
> >      [Calculated window size: 257]
> >      [Window size scaling factor: -1 (unknown)]
> >      Checksum: 0xfefa [unverified]
> >      [Checksum Status: Unverified]
> >      Urgent Pointer: 0
> >      Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
> >          TCP Option - No-Operation (NOP)
> >              Kind: No-Operation (1)
> >          TCP Option - No-Operation (NOP)
> >              Kind: No-Operation (1)
> >          TCP Option - Timestamps: TSval 4211986351, TSecr 1497291787
> >              Kind: Time Stamp Option (8)
> >              Length: 10
> >              Timestamp value: 4211986351
> >              Timestamp echo reply: 1497291787
> >      [SEQ/ACK analysis]
> >          [Bytes in flight: 24]
> >          [Bytes sent since last PSH flag: 24]
> >      [Timestamps]
> >          [Time since first frame in this TCP stream: 0.000000000 seconds]
> >          [Time since previous frame in this TCP stream: 0.000000000 seconds]
> >      TCP payload (24 bytes)
> >      [PDU Size: 24]
> > NVM Express Fabrics TCP
> >      Pdu Type: CapsuleResponse (5)
> >      Pdu Specific Flags: 0x00
> >          .... ...0 = PDU Header Digest: Not set
> >          .... ..0. = PDU Data Digest: Not set
> >          .... .0.. = PDU Data Last: Not set
> >          .... 0... = PDU Data Success: Not set
> >      Pdu Header Length: 24
> >      Pdu Data Offset: 0
> >      Packet Length: 24
> >      Unknown Data: 02000400000000001c0000001f000000
> > 
> > 0000  00 00 0c 9f f5 a8 b4 96 91 41 16 c0 08 00 45 00   .........A....E.
> > 0010  00 4c 00 00 40 00 40 06 00 00 0a e4 26 d6 0a e4   .L.. at .@.....&...
> > 0020  c2 1e 11 44 b7 3d f8 e0 e4 dc b0 d7 5b 4a 80 18   ...D.=......[J..
> > 0030  01 01 fe fa 00 00 01 01 08 0a fb 0d cf af 59 3e   ..............Y>
> > 0040  dc 0b 05 00 18 00 18 00 00 00 02 00 04 00 00 00   ................
> > 0050  00 00 1c 00 00 00 1f 00 00 00                     ..........
> > 
> > No.     Time           Source                Destination           Protocol Length Info
> >       12 0.004969       10.228.38.175         10.228.194.30         NVMe/TCP 90
> > 
> > Frame 12: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
> >      Encapsulation type: Ethernet (1)
> >      Arrival Time: Feb 23, 2021 18:16:08.780653000 CET
> >      [Time shift for this packet: 0.000000000 seconds]
> >      Epoch Time: 1614100568.780653000 seconds
> >      [Time delta from previous captured frame: 0.000079000 seconds]
> >      [Time delta from previous displayed frame: 0.000079000 seconds]
> >      [Time since reference or first frame: 0.004969000 seconds]
> >      Frame Number: 12
> >      Frame Length: 90 bytes (720 bits)
> >      Capture Length: 90 bytes (720 bits)
> >      [Frame is marked: False]
> >      [Frame is ignored: False]
> >      [Protocols in frame: eth:ethertype:ip:tcp:nvme-tcp]
> >      [Coloring Rule Name: TCP]
> >      [Coloring Rule String: tcp]
> > Ethernet II, Src: IntelCor_41:16:c0 (b4:96:91:41:16:c0), Dst: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
> >      Destination: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
> >          Address: Cisco_9f:f5:a8 (00:00:0c:9f:f5:a8)
> >          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
> >          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
> >      Source: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
> >          Address: IntelCor_41:16:c0 (b4:96:91:41:16:c0)
> >          .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
> >          .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
> >      Type: IPv4 (0x0800)
> > Internet Protocol Version 4, Src: 10.228.38.175, Dst: 10.228.194.30
> >      0100 .... = Version: 4
> >      .... 0101 = Header Length: 20 bytes (5)
> >      Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
> >          0000 00.. = Differentiated Services Codepoint: Default (0)
> >          .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
> >      Total Length: 76
> >      Identification: 0x0000 (0)
> >      Flags: 0x40, Don't fragment
> >          0... .... = Reserved bit: Not set
> >          .1.. .... = Don't fragment: Set
> >          ..0. .... = More fragments: Not set
> >      Fragment Offset: 0
> >      Time to Live: 64
> >      Protocol: TCP (6)
> >      Header Checksum: 0x0000 [validation disabled]
> >      [Header checksum status: Unverified]
> >      Source Address: 10.228.38.175
> >      Destination Address: 10.228.194.30
> > Transmission Control Protocol, Src Port: 4420, Dst Port: 34895, Seq: 1, Ack: 1, Len: 24
> >      Source Port: 4420
> >      Destination Port: 34895
> >      [Stream index: 3]
> >      [TCP Segment Len: 24]
> >      Sequence Number: 1    (relative sequence number)
> >      Sequence Number (raw): 3092812012
> >      [Next Sequence Number: 25    (relative sequence number)]
> >      Acknowledgment Number: 1    (relative ack number)
> >      Acknowledgment number (raw): 2384147181
> >      1000 .... = Header Length: 32 bytes (8)
> >      Flags: 0x018 (PSH, ACK)
> >          000. .... .... = Reserved: Not set
> >          ...0 .... .... = Nonce: Not set
> >          .... 0... .... = Congestion Window Reduced (CWR): Not set
> >          .... .0.. .... = ECN-Echo: Not set
> >          .... ..0. .... = Urgent: Not set
> >          .... ...1 .... = Acknowledgment: Set
> >          .... .... 1... = Push: Set
> >          .... .... .0.. = Reset: Not set
> >          .... .... ..0. = Syn: Not set
> >          .... .... ...0 = Fin: Not set
> >          [TCP Flags: ???????AP???]
> >      Window: 257
> >      [Calculated window size: 257]
> >      [Window size scaling factor: -1 (unknown)]
> >      Checksum: 0xfed3 [unverified]
> >      [Checksum Status: Unverified]
> >      Urgent Pointer: 0
> >      Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
> >          TCP Option - No-Operation (NOP)
> >              Kind: No-Operation (1)
> >          TCP Option - No-Operation (NOP)
> >              Kind: No-Operation (1)
> >          TCP Option - Timestamps: TSval 3874335934, TSecr 3601030412
> >              Kind: Time Stamp Option (8)
> >              Length: 10
> >              Timestamp value: 3874335934
> >              Timestamp echo reply: 3601030412
> >      [SEQ/ACK analysis]
> >          [Bytes in flight: 24]
> >          [Bytes sent since last PSH flag: 24]
> >      [Timestamps]
> >          [Time since first frame in this TCP stream: 0.000000000 seconds]
> >          [Time since previous frame in this TCP stream: 0.000000000 seconds]
> >      TCP payload (24 bytes)
> >      [PDU Size: 24]
> > NVM Express Fabrics TCP
> >      Pdu Type: CapsuleResponse (5)
> >      Pdu Specific Flags: 0x00
> >          .... ...0 = PDU Header Digest: Not set
> >          .... ..0. = PDU Data Digest: Not set
> >          .... .0.. = PDU Data Last: Not set
> >          .... 0... = PDU Data Success: Not set
> >      Pdu Header Length: 24
> >      Pdu Data Offset: 0
> >      Packet Length: 24
> >      Unknown Data: 02000400000000001b0000001f000000
> > 
> > 0000  00 00 0c 9f f5 a8 b4 96 91 41 16 c0 08 00 45 00   .........A....E.
> > 0010  00 4c 00 00 40 00 40 06 00 00 0a e4 26 af 0a e4   .L.. at .@.....&...
> > 0020  c2 1e 11 44 88 4f b8 58 90 ec 8e 1b 32 ed 80 18   ...D.O.X....2...
> > 0030  01 01 fe d3 00 00 01 01 08 0a e6 ed ac be d6 a3   ................
> > 0040  5d 0c 05 00 18 00 18 00 00 00 02 00 04 00 00 00   ]...............
> > 0050  00 00 1b 00 00 00 1f 00 00 00                     ..........
> > 
> As I suspected, we did receive an invalid frame.
> Data digest would have saved us, but then it's not enabled.
> 
> So we do need to check if the request is valid before processing it.

That's just addressing a symptom. You can't fully verify the request is
valid this way because the host could have started the same command ID
the very moment before the code checks it, incorrectly completing an
in-flight command and getting data corruption.


From hare at suse.de  Fri Feb 26 11:42:46 2021
From: hare at suse.de (Hannes Reinecke)
Date: Fri, 26 Feb 2021 17:42:46 +0100
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <20210226161355.GG31593@redsun51.ssa.fujisawa.hgst.com>
References: <20210212181738.79274-1-dwagner@suse.de>
 <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
 <20210215104020.yyithlo2hkxqvguj@beryllium.lan>
 <a2064070-b511-ba6d-bd64-0b3abc208356@grimberg.me>
 <20210226123534.4oovbzk4wrnfjp64@beryllium.lan>
 <9e209b12-3771-cdca-2c9d-50451061bd2a@suse.de>
 <20210226161355.GG31593@redsun51.ssa.fujisawa.hgst.com>
Message-ID: <a42d6285-ff32-3e16-b2b1-808d29f2a743@suse.de>

On 2/26/21 5:13 PM, Keith Busch wrote:
> On Fri, Feb 26, 2021 at 01:54:00PM +0100, Hannes Reinecke wrote:
>> On 2/26/21 1:35 PM, Daniel Wagner wrote:
>>> On Mon, Feb 15, 2021 at 01:29:45PM -0800, Sagi Grimberg wrote:
>>>> Well, I think we should probably figure out why that is happening first.
>>>
>>> I got my hands on a tcpdump trace. I've trimmed it to this:
>>>
[ .. ]
>>> NVM Express Fabrics TCP
>>>       Pdu Type: CapsuleResponse (5)
>>>       Pdu Specific Flags: 0x00
>>>           .... ...0 = PDU Header Digest: Not set
>>>           .... ..0. = PDU Data Digest: Not set
>>>           .... .0.. = PDU Data Last: Not set
>>>           .... 0... = PDU Data Success: Not set
>>>       Pdu Header Length: 24
>>>       Pdu Data Offset: 0
>>>       Packet Length: 24
>>>       Unknown Data: 02000400000000001b0000001f000000
>>>
>>> 0000  00 00 0c 9f f5 a8 b4 96 91 41 16 c0 08 00 45 00   .........A....E.
>>> 0010  00 4c 00 00 40 00 40 06 00 00 0a e4 26 af 0a e4   .L.. at .@.....&...
>>> 0020  c2 1e 11 44 88 4f b8 58 90 ec 8e 1b 32 ed 80 18   ...D.O.X....2...
>>> 0030  01 01 fe d3 00 00 01 01 08 0a e6 ed ac be d6 a3   ................
>>> 0040  5d 0c 05 00 18 00 18 00 00 00 02 00 04 00 00 00   ]...............
>>> 0050  00 00 1b 00 00 00 1f 00 00 00                     ..........
>>>
>> As I suspected, we did receive an invalid frame.
>> Data digest would have saved us, but then it's not enabled.
>>
>> So we do need to check if the request is valid before processing it.
> 
> That's just addressing a symptom. You can't fully verify the request is
> valid this way because the host could have started the same command ID
> the very moment before the code checks it, incorrectly completing an
> in-flight command and getting data corruption.
> 
Oh, I am fully aware.

Bad frames are just that, bad frames.
We can only fully validate that when digests are enabled, but I gather 
that controllers sending out bad frames wouldn't want to enable digests, 
either. So relying on that is possibly not an option.

So really what I'm trying to avoid is the host crashing on a bad frame.
That kind of thing always resonates bad with customers.
And tripping over an uninitialized command is just too stupid IMO.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From kbusch at kernel.org  Fri Feb 26 12:19:01 2021
From: kbusch at kernel.org (Keith Busch)
Date: Sat, 27 Feb 2021 02:19:01 +0900
Subject: [PATCH] nvme-tcp: Check if request has started before processing
 it
In-Reply-To: <a42d6285-ff32-3e16-b2b1-808d29f2a743@suse.de>
References: <c3a682d3-58f7-f5cc-caaa-75c36ca464e2@grimberg.me>
 <20210212210929.GA3851@redsun51.ssa.fujisawa.hgst.com>
 <ddf87227-1ad3-b8be-23ba-460433f70a85@grimberg.me>
 <73e4914e-f867-c899-954d-4b61ae2b4c33@suse.de>
 <20210215104020.yyithlo2hkxqvguj@beryllium.lan>
 <a2064070-b511-ba6d-bd64-0b3abc208356@grimberg.me>
 <20210226123534.4oovbzk4wrnfjp64@beryllium.lan>
 <9e209b12-3771-cdca-2c9d-50451061bd2a@suse.de>
 <20210226161355.GG31593@redsun51.ssa.fujisawa.hgst.com>
 <a42d6285-ff32-3e16-b2b1-808d29f2a743@suse.de>
Message-ID: <20210226171901.GA3949@redsun51.ssa.fujisawa.hgst.com>

On Fri, Feb 26, 2021 at 05:42:46PM +0100, Hannes Reinecke wrote:
> On 2/26/21 5:13 PM, Keith Busch wrote:
> > 
> > That's just addressing a symptom. You can't fully verify the request is
> > valid this way because the host could have started the same command ID
> > the very moment before the code checks it, incorrectly completing an
> > in-flight command and getting data corruption.
> > 
> Oh, I am fully aware.
> 
> Bad frames are just that, bad frames.
> We can only fully validate that when digests are enabled, but I gather that
> controllers sending out bad frames wouldn't want to enable digests, either.
> So relying on that is possibly not an option.
> 
> So really what I'm trying to avoid is the host crashing on a bad frame.
> That kind of thing always resonates bad with customers.
> And tripping over an uninitialized command is just too stupid IMO.

Crashing is bad, silent data corruption is worse. Is there truly no
defense against that? If not, why should anyone rely on this?


From Martin.Belanger at dell.com  Fri Feb 26 13:41:45 2021
From: Martin.Belanger at dell.com (Belanger, Martin)
Date: Fri, 26 Feb 2021 18:41:45 +0000
Subject: nvme-discover: Computer freezes up when persistent device loses
 connection to controller
Message-ID: <SJ0PR19MB45442613CD5D226582EB6DBBF29D9@SJ0PR19MB4544.namprd19.prod.outlook.com>

I was experimenting with persistent tcp connections using nvme-discover/nvme-tcp and nvmet-tcp and caused my computer to freeze up.

The steps are as follows (detailed commands below):
1) Configure nvmet to listen for tcp connection at 127.0.0.1:8009 and configure 2 storage subsystems.
2) Use nvme-discover to set a permanent tcp connection to?127.0.0.1:8009?
3) Confirm that a persistent device was created (/dev/nvme1)
4) Confirm that we can retrieve log pages from persistent nvme1
5) Remove all nvmet configuration.
6) Confirm that the persistent device /dev/nvme1 is still there.?
7) Try to retrieve log pages from persistent device nvme1?
8) Computer freezes

Detailed operations:
?
1) Configure nvmet to listen for tcp connection at 127.0.0.1:8009 and coonfigure 2 storage subsystems
# modprobe null_blk nr_devices=2
# modprobe nvmet-tcp
?
# mkdir? ? ? ?/sys/kernel/config/nvmet/subsystems/storage1
# echo -n 1 > /sys/kernel/config/nvmet/subsystems/storage1/attr_allow_any_host
?
# mkdir? ? ? ? ? ? ? ? ?/sys/kernel/config/nvmet/subsystems/storage1/namespaces/1
# echo -n /dev/nullb0 > /sys/kernel/config/nvmet/subsystems/storage1/namespaces/1/device_path"
# echo -n 1? ? ? ? ? ?> /sys/kernel/config/nvmet/subsystems/storage1/namespaces/1/enable
?
# mkdir? ? ? ?/sys/kernel/config/nvmet/subsystems/storage2
# echo -n 1 > /sys/kernel/config/nvmet/subsystems/storage2/attr_allow_any_host
?
# mkdir? ? ? ? ? ? ? ? ?/sys/kernel/config/nvmet/subsystems/storage2/namespaces/1
# echo -n /dev/nullb0 > /sys/kernel/config/nvmet/subsystems/storage2/namespaces/1/device_path"
# echo -n 1? ? ? ? ? ?> /sys/kernel/config/nvmet/subsystems/storage2/namespaces/1/enable
?
# mkdir /sys/kernel/config/nvmet/ports/1
# echo -n 127.0.0.1 > /sys/kernel/config/nvmet/ports/1/addr_traddr
# echo -n tcp ? ? ? > /sys/kernel/config/nvmet/ports/1/addr_trtype
# echo -n 8009 ? ? ?> /sys/kernel/config/nvmet/ports/1/addr_trsvcid
# echo -n ipv4 ? ? ?> /sys/kernel/config/nvmet/ports/1/addr_adrfam
?
# cd /sys/kernel/config/nvmet/ports/1/subsystems
# ln -s ../../../subsystems/storage1 storage1
# ln -s ../../../subsystems/storage2 storage2
?
2)?Use nvme-cli to set a permanent tcp connection to 127.0.0.1:8009 
$ sudo nvme discover -g -G -o json -t tcp -a 127.0.0.1 -s 8009 --persistent
{
? "device" : "nvme1",
? "genctr" : 4,
? "records" : [
? ? {
? ? ? "trtype" : "tcp",
? ? ? "adrfam" : "ipv4",
? ? ? "subtype" : "nvme subsystem",
? ? ? "treq" : "not specified, sq flow control disable supported",
? ? ? "portid" : 1,
? ? ? "trsvcid" : "8009",
? ? ? "subnqn" : "storage1",
? ? ? "traddr" : "127.0.0.1",
? ? ? "sectype" : "none"
? ? },
? ? {
? ? ? "trtype" : "tcp",
? ? ? "adrfam" : "ipv4",
? ? ? "subtype" : "nvme subsystem",
? ? ? "treq" : "not specified, sq flow control disable supported",
? ? ? "portid" : 1,
? ? ? "trsvcid" : "8009",
? ? ? "subnqn" : "storage2",
? ? ? "traddr" : "127.0.0.1",
? ? ? "sectype" : "none"
? ? }
? ]
}
?
3) Confirm that a new device was created (/dev/nvme1)
$ ls -l /dev/nvme*
crw------- 1 root root 240, ?0 Feb 26 11:09 /dev/nvme0
brw-rw---- 1 root disk 259, ?0 Feb 26 11:09 /dev/nvme0n1
brw-rw---- 1 root disk 259, ?1 Feb 26 11:09 /dev/nvme0n1p1
brw-rw---- 1 root disk 259, ?2 Feb 26 11:09 /dev/nvme0n1p2
brw-rw---- 1 root disk 259, ?3 Feb 26 11:09 /dev/nvme0n1p3
brw-rw---- 1 root disk 259, ?4 Feb 26 11:09 /dev/nvme0n1p4
brw-rw---- 1 root disk 259, ?5 Feb 26 11:09 /dev/nvme0n1p5
crw------- 1 root root 240, ?1 Feb 26 11:16?/dev/nvme1
crw------- 1 root root ?10, 33 Feb 26 11:11 /dev/nvme-fabrics
?
4) Confirm that we can retrieve log pages from persistent /dev/nvme1
$ sudo nvme discover -d?nvme1?-o json
{
? "device" : "nvme1",
? "genctr" : 4,
? "records" : [
? ? {
? ? ? "trtype" : "tcp",
? ? ? "adrfam" : "ipv4",
? ? ? "subtype" : "nvme subsystem",
? ? ? "treq" : "not specified, sq flow control disable supported",
? ? ? "portid" : 1,
? ? ? "trsvcid" : "8009",
? ? ? "subnqn" : "storage1",
? ? ? "traddr" : "127.0.0.1",
? ? ? "sectype" : "none"
? ? },
? ? {
? ? ? "trtype" : "tcp",
? ? ? "adrfam" : "ipv4",
? ? ? "subtype" : "nvme subsystem",
? ? ? "treq" : "not specified, sq flow control disable supported",
? ? ? "portid" : 1,
? ? ? "trsvcid" : "8009",
? ? ? "subnqn" : "storage2",
? ? ? "traddr" : "127.0.0.1",
? ? ? "sectype" : "none"
? ? }
? ]
}
?
5)?Remove all nvmet configuration
# rm -f /sys/kernel/config/nvmet/ports/1/subsystems/storage1
# rm -f /sys/kernel/config/nvmet/ports/1/subsystems/storage2
# rmdir /sys/kernel/config/nvmet/ports/1
# rmdir /sys/kernel/config/nvmet/subsystems/storage1/namespaces/1
# rmdir /sys/kernel/config/nvmet/subsystems/storage2/namespaces/1
# rmdir /sys/kernel/config/nvmet/subsystems/storage1
# rmdir /sys/kernel/config/nvmet/subsystems/storage2
?
6) Confirm that the persistent device /dev/nvme1 is still there?
$ ls -l /dev/nvme*
crw------- 1 root root 240, ?0 Feb 26 11:09 /dev/nvme0
brw-rw---- 1 root disk 259, ?0 Feb 26 11:09 /dev/nvme0n1
brw-rw---- 1 root disk 259, ?1 Feb 26 11:09 /dev/nvme0n1p1
brw-rw---- 1 root disk 259, ?2 Feb 26 11:09 /dev/nvme0n1p2
brw-rw---- 1 root disk 259, ?3 Feb 26 11:09 /dev/nvme0n1p3
brw-rw---- 1 root disk 259, ?4 Feb 26 11:09 /dev/nvme0n1p4
brw-rw---- 1 root disk 259, ?5 Feb 26 11:09 /dev/nvme0n1p5
crw------- 1 root root 240, ?1 Feb 26 11:32 /dev/nvme1? ? <- Should this device remain if the connection is lost?
crw------- 1 root root ?10, 33 Feb 26 11:11 /dev/nvme-fabrics
?
?
7) Try to retrieve log pages from /dev/nvme1
$ sudo nvme discover --device nvme1 -o json
?
8) !!!! COMPUTER IS BRICKED !!!!

Regards,

Martin Belanger
Dell Inc.

From hare at suse.de  Sat Feb 27 04:12:43 2021
From: hare at suse.de (Hannes Reinecke)
Date: Sat, 27 Feb 2021 10:12:43 +0100
Subject: [PATCH] nvme-rdma: fix crash for no IO queues
In-Reply-To: <7c30ddb7-ee41-9972-4cb6-0a5c089f687c@huawei.com>
References: <20210223072602.3196-1-lengchao@huawei.com>
 <20210223232137.GA23682@redsun51.ssa.fujisawa.hgst.com>
 <7c30ddb7-ee41-9972-4cb6-0a5c089f687c@huawei.com>
Message-ID: <2a9bb7c3-eb4e-2341-dcfa-583d41912977@suse.de>

On 2/24/21 6:59 AM, Chao Leng wrote:
> 
> 
> On 2021/2/24 7:21, Keith Busch wrote:
>> On Tue, Feb 23, 2021 at 03:26:02PM +0800, Chao Leng wrote:
>>> A crash happens when set feature(NVME_FEAT_NUM_QUEUES) timeout in nvme
>>> over rdma(roce) reconnection, the reason is use the queue which is not
>>> alloced.
>>>
>>> If it is not discovery and no io queues, the connection should fail.
>>
>> If you're getting a timeout, we need to quit initialization. Hannes
>> attempted making that status visible for fabrics here:
>>
>>    
>> http://lists.infradead.org/pipermail/linux-nvme/2021-January/022353.html
> I know the patch. It can not solve the scenario: target may be an
> attacker or the target behavior is incorrect.
> If target return 0 io queues or return other error code, the crash will
> still happen. We should not allow this to happen.
I'm fully with you that we shouldn't crash, but at the same time a value 
of '0' for the number of I/O queues is considered valid.
So we should fix the code to handle this scenario, and not disallowing 
zero I/O queues.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare at suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 N?rnberg
HRB 36809 (AG N?rnberg), Gesch?ftsf?hrer: Felix Imend?rffer


From lengchao at huawei.com  Sat Feb 27 04:30:46 2021
From: lengchao at huawei.com (Chao Leng)
Date: Sat, 27 Feb 2021 17:30:46 +0800
Subject: [PATCH] nvme-rdma: fix crash for no IO queues
In-Reply-To: <2a9bb7c3-eb4e-2341-dcfa-583d41912977@suse.de>
References: <20210223072602.3196-1-lengchao@huawei.com>
 <20210223232137.GA23682@redsun51.ssa.fujisawa.hgst.com>
 <7c30ddb7-ee41-9972-4cb6-0a5c089f687c@huawei.com>
 <2a9bb7c3-eb4e-2341-dcfa-583d41912977@suse.de>
Message-ID: <0a8fbff4-3759-f300-1fa8-601df1eab683@huawei.com>



On 2021/2/27 17:12, Hannes Reinecke wrote:
> On 2/24/21 6:59 AM, Chao Leng wrote:
>>
>>
>> On 2021/2/24 7:21, Keith Busch wrote:
>>> On Tue, Feb 23, 2021 at 03:26:02PM +0800, Chao Leng wrote:
>>>> A crash happens when set feature(NVME_FEAT_NUM_QUEUES) timeout in nvme
>>>> over rdma(roce) reconnection, the reason is use the queue which is not
>>>> alloced.
>>>>
>>>> If it is not discovery and no io queues, the connection should fail.
>>>
>>> If you're getting a timeout, we need to quit initialization. Hannes
>>> attempted making that status visible for fabrics here:
>>>
>>> http://lists.infradead.org/pipermail/linux-nvme/2021-January/022353.html
>> I know the patch. It can not solve the scenario: target may be an
>> attacker or the target behavior is incorrect.
>> If target return 0 io queues or return other error code, the crash will
>> still happen. We should not allow this to happen.
> I'm fully with you that we shouldn't crash, but at the same time a value of '0' for the number of I/O queues is considered valid.
> So we should fix the code to handle this scenario, and not disallowing zero I/O queues.
'0' I/O queues doesn't make any sense to nvme over fabrics, it is
different with nvme over pci. If there is some bug with target, we can
debug it in target instead of use admin queue in host.
target may be an attacker or the target behavior is incorrect. So we
should avoid crash. Another option: prohibit  request delivery if
io queue do not created.
I think failed connection with '0' I/O queues is a better choice.
> 
> Cheers,
> 
> Hannes


From Martin.Belanger at dell.com  Sat Feb 27 09:57:21 2021
From: Martin.Belanger at dell.com (Belanger, Martin)
Date: Sat, 27 Feb 2021 14:57:21 +0000
Subject: nvme-discover: Computer freezes up when persistent device loses
 connection to controller
In-Reply-To: <SJ0PR19MB45442613CD5D226582EB6DBBF29D9@SJ0PR19MB4544.namprd19.prod.outlook.com>
References: <SJ0PR19MB45442613CD5D226582EB6DBBF29D9@SJ0PR19MB4544.namprd19.prod.outlook.com>
Message-ID: <SJ0PR19MB454458D28B479E2B8E082294F29C9@SJ0PR19MB4544.namprd19.prod.outlook.com>

Update.

I reported this issue after experimenting on Ubuntu 20.04 (kernel 5.8).

I repeated the same experiment on Fedora 33 (kernel 5.10) and I'm not seeing the freeze.

It seems that the issue has been resolved between kerel 5.8 and 5.10.

Regards,
Martin

________________________________________
From: Belanger, Martin <Martin_Belanger at Dell.com>
Sent: Friday, February 26, 2021 13:41
To: linux-nvme at lists.infradead.org
Subject: nvme-discover: Computer freezes up when persistent device loses connection to controller

I was experimenting with persistent tcp connections using nvme-discover/nvme-tcp and nvmet-tcp and caused my computer to freeze up.

The steps are as follows (detailed commands below):
1) Configure nvmet to listen for tcp connection at 127.0.0.1:8009 and configure 2 storage subsystems.
2) Use nvme-discover to set a permanent tcp connection to 127.0.0.1:8009
3) Confirm that a persistent device was created (/dev/nvme1)
4) Confirm that we can retrieve log pages from persistent nvme1
5) Remove all nvmet configuration.
6) Confirm that the persistent device /dev/nvme1 is still there.
7) Try to retrieve log pages from persistent device nvme1
8) Computer freezes

Detailed operations:

1) Configure nvmet to listen for tcp connection at 127.0.0.1:8009 and coonfigure 2 storage subsystems
# modprobe null_blk nr_devices=2
# modprobe nvmet-tcp

# mkdir       /sys/kernel/config/nvmet/subsystems/storage1
# echo -n 1 > /sys/kernel/config/nvmet/subsystems/storage1/attr_allow_any_host

# mkdir                 /sys/kernel/config/nvmet/subsystems/storage1/namespaces/1
# echo -n /dev/nullb0 > /sys/kernel/config/nvmet/subsystems/storage1/namespaces/1/device_path"
# echo -n 1           > /sys/kernel/config/nvmet/subsystems/storage1/namespaces/1/enable

# mkdir       /sys/kernel/config/nvmet/subsystems/storage2
# echo -n 1 > /sys/kernel/config/nvmet/subsystems/storage2/attr_allow_any_host

# mkdir                 /sys/kernel/config/nvmet/subsystems/storage2/namespaces/1
# echo -n /dev/nullb0 > /sys/kernel/config/nvmet/subsystems/storage2/namespaces/1/device_path"
# echo -n 1           > /sys/kernel/config/nvmet/subsystems/storage2/namespaces/1/enable

# mkdir /sys/kernel/config/nvmet/ports/1
# echo -n 127.0.0.1 > /sys/kernel/config/nvmet/ports/1/addr_traddr
# echo -n tcp       > /sys/kernel/config/nvmet/ports/1/addr_trtype
# echo -n 8009      > /sys/kernel/config/nvmet/ports/1/addr_trsvcid
# echo -n ipv4      > /sys/kernel/config/nvmet/ports/1/addr_adrfam

# cd /sys/kernel/config/nvmet/ports/1/subsystems
# ln -s ../../../subsystems/storage1 storage1
# ln -s ../../../subsystems/storage2 storage2

2) Use nvme-cli to set a permanent tcp connection to 127.0.0.1:8009
$ sudo nvme discover -g -G -o json -t tcp -a 127.0.0.1 -s 8009 --persistent
{
  "device" : "nvme1",
  "genctr" : 4,
  "records" : [
    {
      "trtype" : "tcp",
      "adrfam" : "ipv4",
      "subtype" : "nvme subsystem",
      "treq" : "not specified, sq flow control disable supported",
      "portid" : 1,
      "trsvcid" : "8009",
      "subnqn" : "storage1",
      "traddr" : "127.0.0.1",
      "sectype" : "none"
    },
    {
      "trtype" : "tcp",
      "adrfam" : "ipv4",
      "subtype" : "nvme subsystem",
      "treq" : "not specified, sq flow control disable supported",
      "portid" : 1,
      "trsvcid" : "8009",
      "subnqn" : "storage2",
      "traddr" : "127.0.0.1",
      "sectype" : "none"
    }
  ]
}

3) Confirm that a new device was created (/dev/nvme1)
$ ls -l /dev/nvme*
crw------- 1 root root 240,  0 Feb 26 11:09 /dev/nvme0
brw-rw---- 1 root disk 259,  0 Feb 26 11:09 /dev/nvme0n1
brw-rw---- 1 root disk 259,  1 Feb 26 11:09 /dev/nvme0n1p1
brw-rw---- 1 root disk 259,  2 Feb 26 11:09 /dev/nvme0n1p2
brw-rw---- 1 root disk 259,  3 Feb 26 11:09 /dev/nvme0n1p3
brw-rw---- 1 root disk 259,  4 Feb 26 11:09 /dev/nvme0n1p4
brw-rw---- 1 root disk 259,  5 Feb 26 11:09 /dev/nvme0n1p5
crw------- 1 root root 240,  1 Feb 26 11:16 /dev/nvme1
crw------- 1 root root  10, 33 Feb 26 11:11 /dev/nvme-fabrics

4) Confirm that we can retrieve log pages from persistent /dev/nvme1
$ sudo nvme discover -d nvme1 -o json
{
  "device" : "nvme1",
  "genctr" : 4,
  "records" : [
    {
      "trtype" : "tcp",
      "adrfam" : "ipv4",
      "subtype" : "nvme subsystem",
      "treq" : "not specified, sq flow control disable supported",
      "portid" : 1,
      "trsvcid" : "8009",
      "subnqn" : "storage1",
      "traddr" : "127.0.0.1",
      "sectype" : "none"
    },
    {
      "trtype" : "tcp",
      "adrfam" : "ipv4",
      "subtype" : "nvme subsystem",
      "treq" : "not specified, sq flow control disable supported",
      "portid" : 1,
      "trsvcid" : "8009",
      "subnqn" : "storage2",
      "traddr" : "127.0.0.1",
      "sectype" : "none"
    }
  ]
}

5) Remove all nvmet configuration
# rm -f /sys/kernel/config/nvmet/ports/1/subsystems/storage1
# rm -f /sys/kernel/config/nvmet/ports/1/subsystems/storage2
# rmdir /sys/kernel/config/nvmet/ports/1
# rmdir /sys/kernel/config/nvmet/subsystems/storage1/namespaces/1
# rmdir /sys/kernel/config/nvmet/subsystems/storage2/namespaces/1
# rmdir /sys/kernel/config/nvmet/subsystems/storage1
# rmdir /sys/kernel/config/nvmet/subsystems/storage2

6) Confirm that the persistent device /dev/nvme1 is still there
$ ls -l /dev/nvme*
crw------- 1 root root 240,  0 Feb 26 11:09 /dev/nvme0
brw-rw---- 1 root disk 259,  0 Feb 26 11:09 /dev/nvme0n1
brw-rw---- 1 root disk 259,  1 Feb 26 11:09 /dev/nvme0n1p1
brw-rw---- 1 root disk 259,  2 Feb 26 11:09 /dev/nvme0n1p2
brw-rw---- 1 root disk 259,  3 Feb 26 11:09 /dev/nvme0n1p3
brw-rw---- 1 root disk 259,  4 Feb 26 11:09 /dev/nvme0n1p4
brw-rw---- 1 root disk 259,  5 Feb 26 11:09 /dev/nvme0n1p5
crw------- 1 root root 240,  1 Feb 26 11:32 /dev/nvme1    <- Should this device remain if the connection is lost?
crw------- 1 root root  10, 33 Feb 26 11:11 /dev/nvme-fabrics


7) Try to retrieve log pages from /dev/nvme1
$ sudo nvme discover --device nvme1 -o json

8) !!!! COMPUTER IS BRICKED !!!!

Regards,

Martin Belanger
Dell Inc.


From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:03 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:03 -0800
Subject: [PATCH V2 0/8] nvme: few cleanups and small improvements
Message-ID: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>

Hi,

This has few cleanups such as variable data type, removing?oneliner
wrappers for single callers, removing duplicate kfree() and small
improvements for NVMeOF Passthru fast path such as likely annotation,
removing the extra checks, making function inline for host/core.c.
Last couple of patches fixes warning for FC function header
documentation.

-ck

v1->v2:

1. Rename the nvme_init_identify() to nvme_init_ctrl_finish(),
2. Move nvme_id_ctrl struct initialization code into separate helper
   nvme_init_identify().
3. Remove the patches to avoid extra churn, keep the fc patches.
4. Keep the small improvements which are needed for the passthru code.
5. Add pr cleanup patch.

Chaitanya Kulkarni (8):
  nvme-core: rename nvme_init_identify()
  nvme-core: split init identify into helper
  nvme-core: mark nvme_setup_passsthru() inline
  nvme-core: use likely in nvme_init_request()
  nvme-core: don't check nvme_req flags for new req
  nvme-fc: fix the function documentation comment
  nvmet-fc: update function documentation
  nvme-core: add new line after variable declatation

 drivers/nvme/host/core.c   | 79 ++++++++++++++++++++++----------------
 drivers/nvme/host/fc.c     |  4 +-
 drivers/nvme/host/nvme.h   |  2 +-
 drivers/nvme/host/pci.c    |  2 +-
 drivers/nvme/host/rdma.c   |  2 +-
 drivers/nvme/host/tcp.c    |  2 +-
 drivers/nvme/target/fc.c   |  1 +
 drivers/nvme/target/loop.c |  2 +-
 8 files changed, 53 insertions(+), 41 deletions(-)

-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:05 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:05 -0800
Subject: [PATCH V2 2/8] nvme-core: split init identify into helper
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-3-chaitanya.kulkarni@wdc.com>

The function nvme_init_ctrl_finish() (formerly nvme_init_identify()) has
grown over the period of time about ~200 lines given the size of nvme id
ctrl data structure.

Move the nvme_id_ctrl data structure related initilzation into helper
nvme_init_identify() and call it from nvme_init_ctrl_finish().

When we move the code into nvme_init_identify() change the local
variable i from int to unsigned int and remove the duplicate kfree()
after nvme_mpath_init() and jump to the label out_free if
nvme_mpath_ini() fails.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 55 +++++++++++++++++++++++-----------------
 1 file changed, 32 insertions(+), 23 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index b72d0ec706be..379ebb54f5dd 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3059,28 +3059,14 @@ static int nvme_get_effects_log(struct nvme_ctrl *ctrl, u8 csi,
 	return 0;
 }
 
-/*
- * Initialize the cached copies of the Identify data and various controller
- * register in our nvme_ctrl structure.  This should be called as soon as
- * the admin queue is fully up and running.
- */
-int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl)
+static int nvme_init_identify(struct nvme_ctrl *ctrl)
 {
 	struct nvme_id_ctrl *id;
 	int ret, page_shift;
 	u32 max_hw_sectors;
 	bool prev_apst_enabled;
 
-	ret = ctrl->ops->reg_read32(ctrl, NVME_REG_VS, &ctrl->vs);
-	if (ret) {
-		dev_err(ctrl->device, "Reading VS failed (%d)\n", ret);
-		return ret;
-	}
 	page_shift = NVME_CAP_MPSMIN(ctrl->cap) + 12;
-	ctrl->sqsize = min_t(u16, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
-
-	if (ctrl->vs >= NVME_VS(1, 1, 0))
-		ctrl->subsystem = NVME_CAP_NSSRC(ctrl->cap);
 
 	ret = nvme_identify_ctrl(ctrl, &id);
 	if (ret) {
@@ -3098,7 +3084,7 @@ int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl)
 		ctrl->cntlid = le16_to_cpu(id->cntlid);
 
 	if (!ctrl->identified) {
-		int i;
+		unsigned int i;
 
 		ret = nvme_init_subsystem(ctrl, id);
 		if (ret)
@@ -3211,16 +3197,43 @@ int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl)
 	}
 
 	ret = nvme_mpath_init(ctrl, id);
-	kfree(id);
-
 	if (ret < 0)
-		return ret;
+		goto out_free;
 
 	if (ctrl->apst_enabled && !prev_apst_enabled)
 		dev_pm_qos_expose_latency_tolerance(ctrl->device);
 	else if (!ctrl->apst_enabled && prev_apst_enabled)
 		dev_pm_qos_hide_latency_tolerance(ctrl->device);
 
+out_free:
+	kfree(id);
+	return ret;
+}
+
+/*
+ * Initialize the cached copies of the Identify data and various controller
+ * register in our nvme_ctrl structure.  This should be called as soon as
+ * the admin queue is fully up and running.
+ */
+int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl)
+{
+	int ret;
+
+	ret = ctrl->ops->reg_read32(ctrl, NVME_REG_VS, &ctrl->vs);
+	if (ret) {
+		dev_err(ctrl->device, "Reading VS failed (%d)\n", ret);
+		return ret;
+	}
+
+	ctrl->sqsize = min_t(u16, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
+
+	if (ctrl->vs >= NVME_VS(1, 1, 0))
+		ctrl->subsystem = NVME_CAP_NSSRC(ctrl->cap);
+
+	ret = nvme_init_identify(ctrl);
+	if (ret)
+		return ret;
+
 	ret = nvme_configure_apst(ctrl);
 	if (ret < 0)
 		return ret;
@@ -3246,10 +3259,6 @@ int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl)
 	ctrl->identified = true;
 
 	return 0;
-
-out_free:
-	kfree(id);
-	return ret;
 }
 EXPORT_SYMBOL_GPL(nvme_init_ctrl_finish);
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:04 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:04 -0800
Subject: [PATCH V2 1/8] nvme-core: rename nvme_init_identify()
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-2-chaitanya.kulkarni@wdc.com>

This is a prep patch so that we can move the identify data structure
related code initialization from nvme_init_identify() into a helper.

Rename the function nvmet_init_identify() to nvmet_init_ctrl_finish().

Next patch will move the nvme_id_ctrl related initialization from newly
renamed function nvme_init_ctrl_finish() into the nvme_init_identify()
helper.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c   | 8 ++++----
 drivers/nvme/host/fc.c     | 2 +-
 drivers/nvme/host/nvme.h   | 2 +-
 drivers/nvme/host/pci.c    | 2 +-
 drivers/nvme/host/rdma.c   | 2 +-
 drivers/nvme/host/tcp.c    | 2 +-
 drivers/nvme/target/loop.c | 2 +-
 7 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e68a8c4ac5a6..b72d0ec706be 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1119,7 +1119,7 @@ static void nvme_passthru_end(struct nvme_ctrl *ctrl, u32 effects)
 		mutex_unlock(&ctrl->scan_lock);
 	}
 	if (effects & NVME_CMD_EFFECTS_CCC)
-		nvme_init_identify(ctrl);
+		nvme_init_ctrl_finish(ctrl);
 	if (effects & (NVME_CMD_EFFECTS_NIC | NVME_CMD_EFFECTS_NCC)) {
 		nvme_queue_scan(ctrl);
 		flush_work(&ctrl->scan_work);
@@ -1978,7 +1978,7 @@ static void nvme_config_write_zeroes(struct gendisk *disk, struct nvme_ns *ns)
 	 * In order to be more cautious use controller's max_hw_sectors value
 	 * to configure the maximum sectors for the write-zeroes which is
 	 * configured based on the controller's MDTS field in the
-	 * nvme_init_identify() if available.
+	 * nvme_init_ctrl_finish() if available.
 	 */
 	if (ns->ctrl->max_hw_sectors == UINT_MAX)
 		max_blocks = (u64)USHRT_MAX + 1;
@@ -3064,7 +3064,7 @@ static int nvme_get_effects_log(struct nvme_ctrl *ctrl, u8 csi,
  * register in our nvme_ctrl structure.  This should be called as soon as
  * the admin queue is fully up and running.
  */
-int nvme_init_identify(struct nvme_ctrl *ctrl)
+int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl)
 {
 	struct nvme_id_ctrl *id;
 	int ret, page_shift;
@@ -3251,7 +3251,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	kfree(id);
 	return ret;
 }
-EXPORT_SYMBOL_GPL(nvme_init_identify);
+EXPORT_SYMBOL_GPL(nvme_init_ctrl_finish);
 
 static int nvme_dev_open(struct inode *inode, struct file *file)
 {
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 20dadd86e981..962987a330d6 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3085,7 +3085,7 @@ nvme_fc_create_association(struct nvme_fc_ctrl *ctrl)
 
 	blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
-	ret = nvme_init_identify(&ctrl->ctrl);
+	ret = nvme_init_ctrl_finish(&ctrl->ctrl);
 	if (ret || test_bit(ASSOC_FAILED, &ctrl->flags))
 		goto out_disconnect_admin_queue;
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..624fc4334a2c 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -599,7 +599,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 void nvme_uninit_ctrl(struct nvme_ctrl *ctrl);
 void nvme_start_ctrl(struct nvme_ctrl *ctrl);
 void nvme_stop_ctrl(struct nvme_ctrl *ctrl);
-int nvme_init_identify(struct nvme_ctrl *ctrl);
+int nvme_init_ctrl_finish(struct nvme_ctrl *ctrl);
 
 void nvme_remove_namespaces(struct nvme_ctrl *ctrl);
 
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 207137a0ed8e..b8f0bfaa8f2a 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2652,7 +2652,7 @@ static void nvme_reset_work(struct work_struct *work)
 	 */
 	dev->ctrl.max_integrity_segments = 1;
 
-	result = nvme_init_identify(&dev->ctrl);
+	result = nvme_init_ctrl_finish(&dev->ctrl);
 	if (result)
 		goto out;
 
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 53ac4d7442ba..9c710839b03a 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -917,7 +917,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
 
 	blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
-	error = nvme_init_identify(&ctrl->ctrl);
+	error = nvme_init_ctrl_finish(&ctrl->ctrl);
 	if (error)
 		goto out_quiesce_queue;
 
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 69f59d2c5799..735e768f9f43 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1875,7 +1875,7 @@ static int nvme_tcp_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
 
 	blk_mq_unquiesce_queue(ctrl->admin_q);
 
-	error = nvme_init_identify(ctrl);
+	error = nvme_init_ctrl_finish(ctrl);
 	if (error)
 		goto out_quiesce_queue;
 
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index cb6f86572b24..a7f97c8b2f77 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -396,7 +396,7 @@ static int nvme_loop_configure_admin_queue(struct nvme_loop_ctrl *ctrl)
 
 	blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
 
-	error = nvme_init_identify(&ctrl->ctrl);
+	error = nvme_init_ctrl_finish(&ctrl->ctrl);
 	if (error)
 		goto out_cleanup_queue;
 
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:06 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:06 -0800
Subject: [PATCH V2 3/8] nvme-core: mark nvme_setup_passsthru() inline
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-4-chaitanya.kulkarni@wdc.com>

Since nvmet_setup_passthru() function falls in fast path when called
from the NVMeOF passthru backend, make it inline.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 379ebb54f5dd..484866e52ba7 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -725,7 +725,7 @@ static void nvme_assign_write_stream(struct nvme_ctrl *ctrl,
 		req->q->write_hints[streamid] += blk_rq_bytes(req) >> 9;
 }
 
-static void nvme_setup_passthrough(struct request *req,
+static inline void nvme_setup_passthrough(struct request *req,
 		struct nvme_command *cmd)
 {
 	memcpy(cmd, nvme_req(req)->cmd, sizeof(*cmd));
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:07 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:07 -0800
Subject: [PATCH V2 4/8] nvme-core: use likely in nvme_init_request()
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-5-chaitanya.kulkarni@wdc.com>

For NVMeOF Target passthru backend we allocate the passthru request with
nvme_alloc_request(). The functions :-

nvmet_passthru_execute_cmd()
 nvme_alloc_request()
  nvme_init_request()

are in the fast path for I/O commands.

In nvme_init_request() we set the timeout based on the
req->q->queuedata check & that is always true for the I/O commands
coming from the passthru backend which are high frequency commands.

Annotate req->q->queuedata check with likely() in nvme_init_request().

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 484866e52ba7..dab18472e39f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -589,7 +589,7 @@ static inline unsigned int nvme_req_op(struct nvme_command *cmd)
 static inline void nvme_init_request(struct request *req,
 		struct nvme_command *cmd)
 {
-	if (req->q->queuedata)
+	if (likely(req->q->queuedata))
 		req->timeout = NVME_IO_TIMEOUT;
 	else /* no queuedata implies admin queue */
 		req->timeout = NVME_ADMIN_TIMEOUT;
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:08 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:08 -0800
Subject: [PATCH V2 5/8] nvme-core: don't check nvme_req flags for new req
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-6-chaitanya.kulkarni@wdc.com>

nvme_clear_request() has a check for flag REQ_DONTPREP and it is called
from nvme_init_request() and nvme_setuo_cmd().

The function nvme_init_request() is called from nvme_alloc_request()
and nvme_alloc_request_qid(). From these two callers new request is
allocated everytime. For newly allocated request RQF_DONTPREP is never
set. Since after getting a tag, block layer sets the req->rq_flags == 0
and never sets the REQ_DONTPREP when returning the request :-

nvme_alloc_request()
	blk_mq_alloc_request()
		blk_mq_rq_ctx_init()
			rq->rq_flags = 0 <----

nvme_alloc_request_qid()
	blk_mq_alloc_request_hctx()
		blk_mq_rq_ctx_init()
			rq->rq_flags = 0 <----

The block layer does set req->rq_flags but REQ_DONTPREP is not one of
them and that is set by the driver.

That means we can unconditinally set the REQ_DONTPREP value to the
rq->rq_flags when nvme_init_request()->nvme_clear_request() is called
from above two callers.

Move the check for REQ_DONTPREP from nvme_clear_nvme_request() into
nvme_setup_cmd().

This is needed since nvme_alloc_request() now gets called from fast
path when NVMeOF target is?configured with passthru backend to avoid
unnecessary checks in the fast path.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index dab18472e39f..4858114f4f8a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -574,11 +574,9 @@ EXPORT_SYMBOL_NS_GPL(nvme_put_ns, NVME_TARGET_PASSTHRU);
 
 static inline void nvme_clear_nvme_request(struct request *req)
 {
-	if (!(req->rq_flags & RQF_DONTPREP)) {
-		nvme_req(req)->retries = 0;
-		nvme_req(req)->flags = 0;
-		req->rq_flags |= RQF_DONTPREP;
-	}
+	nvme_req(req)->retries = 0;
+	nvme_req(req)->flags = 0;
+	req->rq_flags |= RQF_DONTPREP;
 }
 
 static inline unsigned int nvme_req_op(struct nvme_command *cmd)
@@ -892,7 +890,8 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req,
 {
 	blk_status_t ret = BLK_STS_OK;
 
-	nvme_clear_nvme_request(req);
+	if (!(req->rq_flags & RQF_DONTPREP))
+		nvme_clear_nvme_request(req);
 
 	memset(cmd, 0, sizeof(*cmd));
 	switch (req_op(req)) {
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:09 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:09 -0800
Subject: [PATCH V2 6/8] nvme-fc: fix the function documentation comment
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-7-chaitanya.kulkarni@wdc.com>

The nvme_fc_rcv_ls_req() function has first argument as pointer to
remoteport named portprt, but in the documentation comment that is name
is used as remoteport. Fix that to get rid if the compilation warning.

drivers/nvme//host/fc.c:1724: warning: Function parameter or member 'portptr' not described in 'nvme_fc_rcv_ls_req'
drivers/nvme//host/fc.c:1724: warning: Excess function parameter 'remoteport' description in 'nvme_fc_rcv_ls_req'

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/fc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 962987a330d6..ad6be2263a62 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -1708,7 +1708,7 @@ nvme_fc_handle_ls_rqst_work(struct work_struct *work)
  *
  * If this routine returns error, the LLDD should abort the exchange.
  *
- * @remoteport: pointer to the (registered) remote port that the LS
+ * @portptr:    pointer to the (registered) remote port that the LS
  *              was received from. The remoteport is associated with
  *              a specific localport.
  * @lsrsp:      pointer to a nvmefc_ls_rsp response structure to be
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:10 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:10 -0800
Subject: [PATCH V2 7/8] nvmet-fc: update function documentation
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-8-chaitanya.kulkarni@wdc.com>

Add minimum description of the hosthandle parameter for
nvmet_fc_rcv_ls_req() so that we can get rid of the following warning.

drivers/nvme//target/fc.c:2009: warning: Function parameter or member 'hosthandle' not described in 'nvmet_fc_rcv_ls_req

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/target/fc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/target/fc.c b/drivers/nvme/target/fc.c
index d375745fc4ed..1f1c70f9f8eb 100644
--- a/drivers/nvme/target/fc.c
+++ b/drivers/nvme/target/fc.c
@@ -1996,6 +1996,7 @@ nvmet_fc_handle_ls_rqst_work(struct work_struct *work)
  *
  * @target_port: pointer to the (registered) target port the LS was
  *              received on.
+ * @hosthandle: pointer to the host specific data, gets stored in iod.
  * @lsrsp:      pointer to a lsrsp structure to be used to reference
  *              the exchange corresponding to the LS.
  * @lsreqbuf:   pointer to the buffer containing the LS Request
-- 
2.22.1



From chaitanya.kulkarni at wdc.com  Sun Feb 28 21:06:11 2021
From: chaitanya.kulkarni at wdc.com (Chaitanya Kulkarni)
Date: Sun, 28 Feb 2021 18:06:11 -0800
Subject: [PATCH V2 8/8] nvme-core: add new line after variable declatation
In-Reply-To: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
References: <20210301020611.18669-1-chaitanya.kulkarni@wdc.com>
Message-ID: <20210301020611.18669-9-chaitanya.kulkarni@wdc.com>

Add a new line in functions nvme_pr_preempt(), nvme_pr_clear(), and
nvme_pr_release() after variable declaration which follows the rest of
the code in the nvme/host/core.c.

No functional change(s) in this patch.

Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
 drivers/nvme/host/core.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4858114f4f8a..472fd39a8286 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2323,18 +2323,21 @@ static int nvme_pr_preempt(struct block_device *bdev, u64 old, u64 new,
 		enum pr_type type, bool abort)
 {
 	u32 cdw10 = nvme_pr_type(type) << 8 | (abort ? 2 : 1);
+
 	return nvme_pr_command(bdev, cdw10, old, new, nvme_cmd_resv_acquire);
 }
 
 static int nvme_pr_clear(struct block_device *bdev, u64 key)
 {
 	u32 cdw10 = 1 | (key ? 1 << 3 : 0);
+
 	return nvme_pr_command(bdev, cdw10, key, 0, nvme_cmd_resv_register);
 }
 
 static int nvme_pr_release(struct block_device *bdev, u64 key, enum pr_type type)
 {
 	u32 cdw10 = nvme_pr_type(type) << 8 | (key ? 1 << 3 : 0);
+
 	return nvme_pr_command(bdev, cdw10, key, 0, nvme_cmd_resv_release);
 }
 
-- 
2.22.1



