[PATVH V5] nvme-core: use xarray for ctrl ns tracking
Chaitanya Kulkarni
chaitanya.kulkarni at wdc.com
Thu Aug 13 22:16:12 EDT 2020
This patch replaces the ctrl->namespaces tracking from linked list to
xarray for better ns-mgmt on the host side. For host side
nvme_find_get_ns() falls into the fast path for NVMeOF passthru target.
This allows us to have better performance for NVMeOF passthru backend
since XArray has shows better performance numbers over having
a combination of read-write semapore read + linked list in the
nvme_find_get_ns() to find namespace in I/O patch from nsid specified
in the nvme_rw_cmd.
Following are the performance numbers taken on the microbench program
where number of kthreads = nr_cpus_online() running simulteneously
accessing last item (NR_ITEM - 1) which demonstrates the difference
between linked list vs XArray :-
NR ITEMS 1 real 0m1.904s vs 0m1.758s
NR ITEMS 2 real 0m1.888s vs 0m2.216s
NR ITEMS 4 real 0m1.891s vs 0m2.232s
NR ITEMS 16 real 0m1.963s vs 0m2.240s
NR ITEMS 32 real 0m2.079s vs 0m2.214s
NR ITEMS 64 real 0m2.319s vs 0m2.214s
NR ITEMS 128 real 0m3.044s vs 0m2.652s
NR ITEMS 256 real 0m4.310s vs 0m2.655s
NR ITEMS 512 real 0m8.762s vs 0m2.658s
NR ITEMS 1024 real 0m21.773s vs 0m2.725s
Following are the performance numbers taken on the NVMeOF (nvme-loop)
target with an average of 8 runs each for existing linked list based
nvme_find_get_ns() and XArray based nvme_find_get_ns() :-
Avg Bandwidth ~17% increase with XArray (higher the better) :-
-----------------------------------------------------------------------
linked-list1.fio.log: read: IOPS=1671k, BW=6529MiB/s
linked-list2.fio.log: read: IOPS=1670k, BW=6523MiB/s
linked-list3.fio.log: read: IOPS=1669k, BW=6519MiB/s
linked-list4.fio.log: read: IOPS=1552k, BW=6063MiB/s
linked-list5.fio.log: read: IOPS=1545k, BW=6037MiB/s
linked-list6.fio.log: read: IOPS=1555k, BW=6073MiB/s
linked-list7.fio.log: read: IOPS=1564k, BW=6108MiB/s
linked-list8.fio.log: read: IOPS=1558k, BW=6086MiB/s
xarray1.fio.log: read: IOPS=1841k, BW=7190MiB/s
xarray2.fio.log: read: IOPS=1834k, BW=7164MiB/s
xarray3.fio.log: read: IOPS=1833k, BW=7162MiB/s
xarray4.fio.log: read: IOPS=1891k, BW=7386MiB/s
xarray5.fio.log: read: IOPS=1897k, BW=7409MiB/s
xarray6.fio.log: read: IOPS=1890k, BW=7384MiB/s
xarray7.fio.log: read: IOPS=1884k, BW=7360MiB/s
xarray8.fio.log: read: IOPS=1878k, BW=7336MiB/s
Avg Latency ~25% decrease with XArray (lower the better) :-
------------------------------------------------------------------------
linked-list1.fio.log: lat (usec): avg=37.55, stdev=10.63
linked-list2.fio.log: lat (usec): avg=37.57, stdev= 9.43
linked-list3.fio.log: lat (usec): avg=37.62, stdev= 9.25
linked-list4.fio.log: lat (usec): avg=40.50, stdev= 9.33
linked-list5.fio.log: lat (usec): avg=40.67, stdev= 9.06
linked-list6.fio.log: lat (usec): avg=40.44, stdev= 8.55
linked-list7.fio.log: lat (usec): avg=40.21, stdev= 9.77
linked-list8.fio.log: lat (usec): avg=40.34, stdev= 8.54
xarray1.fio.log: lat (usec): avg=34.08, stdev=11.72
xarray2.fio.log: lat (usec): avg=34.19, stdev= 5.53
xarray3.fio.log: lat (usec): avg=34.20, stdev= 4.51
xarray4.fio.log: lat (usec): avg=33.15, stdev= 6.27
xarray5.fio.log: lat (usec): avg=33.04, stdev=15.15
xarray6.fio.log: lat (usec): avg=33.15, stdev=13.94
xarray7.fio.log: lat (usec): avg=33.26, stdev= 5.43
xarray8.fio.log: lat (usec): avg=33.39, stdev= 4.71
Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni at wdc.com>
---
* Changes from V4:-
-------------------
1. Add nvme_has_multiple_namespaces() to check for multiple namespace
so that we can remove reverse goto jump.
2. Use canonical way to get the ns reference in nvme_find_get_ns().
3. Use goto label out_unregister_lightnvm.
4. Rework the ns deletion so that we call nvme_ns_remove() directly
from xa_for_each() loop instead of having insertion in the namespace
remove loop.
* Changes from V3:-
-------------------
1. Get rid of the centralize helper for ctrl queue mgmt.
2. Re-order patches and make nvmet patch first.
3. In the error patch for xa_insert() for nvmet patch restore subsystem
max nsid and call percpu_ref_exit().
5. Ger rid of the rcu read_lock() and rcu_read_unlock() in
nvmet_find_namespaces().
4. Remove an extra local varable and use ctrl->namespaces directly in
host ns remove path.
5. Call nvme_nvm_unregister() when xa_insert() fails in nvme_alloc_ns().
6. Move xa_erase() call in nvme_ns_remove() before existing call to
synchronize_rcu().
* Changes from V2:-
-------------------
1. Add Xarray __xa_load() API as a preparation patch.
2. Remove the id_ctrl call in nvme_dev_user_cmd().
3. Remove the switch error check for xa_insert().
4. Don't change the ns->kref code. when calling xa_erase().
5. Keep XArray for deletion in the nvme_remove_invalid_namespaces()
see [1].
6. Keep XArray for deletion in the nvme_remove_namespaces() see [1].
7. Remove randomly changed the lines to alingn the coding style in
nvmet patch.
8. Remove remaining #include nvme.h from the nvmet patch.
9. Remove the xa_empty() from nvmet_max_nsid().
10. Centralize the blk-mq queue wrapper. The blk-mq queue related
wrapper functions nvme_kill_queues(), nvme_unfreeze(),
nvme_wait_freeze(), nvme_start_freeze(), nvme_stop_queues(),
nvme_start_queues(), nvme_start_queues(), and nvme_sync_queues()
differ in only one line i.e. blk_mq_queue_xxx() call. For the one
line we have 7 functions and 7 exported symbols. Using a
centralize ctrl-queue action function and well defined enums
represnting names of the helpers we can minimize the code and
exported symbol and still maintain the redability.
* Change from V1:-
------------------
1. Use xarray instead of rcu locks.
---
drivers/nvme/host/core.c | 167 +++++++++++++++-------------------
drivers/nvme/host/multipath.c | 15 ++-
drivers/nvme/host/nvme.h | 5 +-
3 files changed, 80 insertions(+), 107 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 05aa568a60af..1292f680c0ea 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -988,9 +988,9 @@ static u32 nvme_passthru_start(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
static void nvme_update_formats(struct nvme_ctrl *ctrl, u32 *effects)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns) {
if (_nvme_revalidate_disk(ns->disk))
nvme_set_queue_dying(ns);
else if (blk_queue_is_zoned(ns->disk->queue)) {
@@ -1002,7 +1002,7 @@ static void nvme_update_formats(struct nvme_ctrl *ctrl, u32 *effects)
*/
*effects |= NVME_CMD_EFFECTS_NCC;
}
- up_read(&ctrl->namespaces_rwsem);
+ }
}
static void nvme_passthru_end(struct nvme_ctrl *ctrl, u32 effects)
@@ -3221,36 +3221,49 @@ static int nvme_dev_open(struct inode *inode, struct file *file)
return 0;
}
+static struct nvme_ns *nvme_has_multiple_namespaces(struct nvme_ctrl *ctrl)
+{
+ struct nvme_ns *ns;
+ unsigned long idx;
+ int count = 0;
+
+ xa_for_each(&ctrl->namespaces, idx, ns) {
+ if (count > 0)
+ goto err;
+ count++;
+ }
+ return ns;
+err:
+ dev_warn(ctrl->device,
+ "NVME_IOCTL_IO_CMD not supported when multiple namespaces present!\n");
+ return NULL;
+}
+
static int nvme_dev_user_cmd(struct nvme_ctrl *ctrl, void __user *argp)
{
struct nvme_ns *ns;
int ret;
- down_read(&ctrl->namespaces_rwsem);
- if (list_empty(&ctrl->namespaces)) {
+ if (xa_empty(&ctrl->namespaces)) {
ret = -ENOTTY;
- goto out_unlock;
+ goto out;
}
- ns = list_first_entry(&ctrl->namespaces, struct nvme_ns, list);
- if (ns != list_last_entry(&ctrl->namespaces, struct nvme_ns, list)) {
- dev_warn(ctrl->device,
- "NVME_IOCTL_IO_CMD not supported when multiple namespaces present!\n");
+ ns = nvme_has_multiple_namespaces(ctrl);
+ if (!ns) {
ret = -EINVAL;
- goto out_unlock;
+ goto out;
}
dev_warn(ctrl->device,
"using deprecated NVME_IOCTL_IO_CMD ioctl on the char device!\n");
kref_get(&ns->kref);
- up_read(&ctrl->namespaces_rwsem);
ret = nvme_user_cmd(ctrl, ns, argp);
nvme_put_ns(ns);
return ret;
-out_unlock:
- up_read(&ctrl->namespaces_rwsem);
+out:
return ret;
}
@@ -3820,31 +3833,17 @@ static int nvme_init_ns_head(struct nvme_ns *ns, unsigned nsid,
return ret;
}
-static int ns_cmp(void *priv, struct list_head *a, struct list_head *b)
-{
- struct nvme_ns *nsa = container_of(a, struct nvme_ns, list);
- struct nvme_ns *nsb = container_of(b, struct nvme_ns, list);
-
- return nsa->head->ns_id - nsb->head->ns_id;
-}
-
struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid)
{
- struct nvme_ns *ns, *ret = NULL;
+ struct nvme_ns *ns;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list) {
- if (ns->head->ns_id == nsid) {
- if (!kref_get_unless_zero(&ns->kref))
- continue;
- ret = ns;
- break;
- }
- if (ns->head->ns_id > nsid)
- break;
- }
- up_read(&ctrl->namespaces_rwsem);
- return ret;
+ rcu_read_lock();
+ ns = xa_load(&ctrl->namespaces, nsid);
+ if (ns && !kref_get_unless_zero(&ns->kref))
+ ns = NULL;
+ rcu_read_unlock();
+
+ return ns;
}
EXPORT_SYMBOL_NS_GPL(nvme_find_get_ns, NVME_TARGET_PASSTHRU);
@@ -3915,9 +3914,9 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
}
}
- down_write(&ctrl->namespaces_rwsem);
- list_add_tail(&ns->list, &ctrl->namespaces);
- up_write(&ctrl->namespaces_rwsem);
+ ret = xa_insert(&ctrl->namespaces, nsid, ns, GFP_KERNEL);
+ if (ret)
+ goto out_unregister_lightnvm;
nvme_get_ctrl(ctrl);
@@ -3928,6 +3927,9 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
kfree(id);
return;
+ out_unregister_lightnvm:
+ if ((ctrl->quirks & NVME_QUIRK_LIGHTNVM) && id->vs[0] == 0x1)
+ nvme_nvm_unregister(ns);
out_put_disk:
/* prevent double queue cleanup */
ns->disk->queue = NULL;
@@ -3960,6 +3962,7 @@ static void nvme_ns_remove(struct nvme_ns *ns)
list_del_init(&ns->head->entry);
mutex_unlock(&ns->ctrl->subsys->lock);
+ xa_erase(&ns->ctrl->namespaces, ns->head->ns_id);
synchronize_rcu(); /* guarantee not available in head->list */
nvme_mpath_clear_current_path(ns);
synchronize_srcu(&ns->head->srcu); /* wait for concurrent submissions */
@@ -3971,10 +3974,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
blk_integrity_unregister(ns->disk);
}
- down_write(&ns->ctrl->namespaces_rwsem);
- list_del_init(&ns->list);
- up_write(&ns->ctrl->namespaces_rwsem);
-
nvme_mpath_check_last_path(ns);
nvme_put_ns(ns);
}
@@ -4005,19 +4004,15 @@ static void nvme_validate_ns(struct nvme_ctrl *ctrl, unsigned nsid)
static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
unsigned nsid)
{
- struct nvme_ns *ns, *next;
- LIST_HEAD(rm_list);
+ unsigned long tnsid;
+ struct nvme_ns *ns;
+ unsigned long idx;
- down_write(&ctrl->namespaces_rwsem);
- list_for_each_entry_safe(ns, next, &ctrl->namespaces, list) {
- if (ns->head->ns_id > nsid || test_bit(NVME_NS_DEAD, &ns->flags))
- list_move_tail(&ns->list, &rm_list);
+ xa_for_each(&ctrl->namespaces, idx, ns) {
+ tnsid = ns->head->ns_id;
+ if (tnsid > nsid || test_bit(NVME_NS_DEAD, &ns->flags))
+ nvme_ns_remove(ns);
}
- up_write(&ctrl->namespaces_rwsem);
-
- list_for_each_entry_safe(ns, next, &rm_list, list)
- nvme_ns_remove(ns);
-
}
static int nvme_scan_ns_list(struct nvme_ctrl *ctrl)
@@ -4115,10 +4110,6 @@ static void nvme_scan_work(struct work_struct *work)
if (nvme_scan_ns_list(ctrl) != 0)
nvme_scan_ns_sequential(ctrl);
mutex_unlock(&ctrl->scan_lock);
-
- down_write(&ctrl->namespaces_rwsem);
- list_sort(NULL, &ctrl->namespaces, ns_cmp);
- up_write(&ctrl->namespaces_rwsem);
}
/*
@@ -4128,8 +4119,8 @@ static void nvme_scan_work(struct work_struct *work)
*/
void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
{
- struct nvme_ns *ns, *next;
- LIST_HEAD(ns_list);
+ struct nvme_ns *ns;
+ unsigned long idx;
/*
* make sure to requeue I/O to all namespaces as these
@@ -4153,11 +4144,7 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
/* this is a no-op when called from the controller reset handler */
nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
- down_write(&ctrl->namespaces_rwsem);
- list_splice_init(&ctrl->namespaces, &ns_list);
- up_write(&ctrl->namespaces_rwsem);
-
- list_for_each_entry_safe(ns, next, &ns_list, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
nvme_ns_remove(ns);
}
EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
@@ -4378,6 +4365,10 @@ static void nvme_free_ctrl(struct device *dev)
if (subsys && ctrl->instance != subsys->instance)
ida_simple_remove(&nvme_instance_ida, ctrl->instance);
+ WARN_ON_ONCE(!xa_empty(&ctrl->namespaces));
+
+ xa_destroy(&ctrl->namespaces);
+
list_for_each_entry_safe(cel, next, &ctrl->cels, entry) {
list_del(&cel->entry);
kfree(cel);
@@ -4412,9 +4403,8 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
ctrl->state = NVME_CTRL_NEW;
spin_lock_init(&ctrl->lock);
mutex_init(&ctrl->scan_lock);
- INIT_LIST_HEAD(&ctrl->namespaces);
INIT_LIST_HEAD(&ctrl->cels);
- init_rwsem(&ctrl->namespaces_rwsem);
+ xa_init(&ctrl->namespaces);
ctrl->dev = dev;
ctrl->ops = ops;
ctrl->quirks = quirks;
@@ -4494,98 +4484,87 @@ EXPORT_SYMBOL_GPL(nvme_init_ctrl);
void nvme_kill_queues(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
-
- down_read(&ctrl->namespaces_rwsem);
+ unsigned long idx;
/* Forcibly unquiesce queues to avoid blocking dispatch */
if (ctrl->admin_q && !blk_queue_dying(ctrl->admin_q))
blk_mq_unquiesce_queue(ctrl->admin_q);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
nvme_set_queue_dying(ns);
-
- up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_kill_queues);
void nvme_unfreeze(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
blk_mq_unfreeze_queue(ns->queue);
- up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_unfreeze);
void nvme_wait_freeze_timeout(struct nvme_ctrl *ctrl, long timeout)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list) {
+ xa_for_each(&ctrl->namespaces, idx, ns) {
timeout = blk_mq_freeze_queue_wait_timeout(ns->queue, timeout);
if (timeout <= 0)
break;
}
- up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_wait_freeze_timeout);
void nvme_wait_freeze(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
blk_mq_freeze_queue_wait(ns->queue);
- up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_wait_freeze);
void nvme_start_freeze(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
blk_freeze_queue_start(ns->queue);
- up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_start_freeze);
void nvme_stop_queues(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
blk_mq_quiesce_queue(ns->queue);
- up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_stop_queues);
void nvme_start_queues(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
blk_mq_unquiesce_queue(ns->queue);
- up_read(&ctrl->namespaces_rwsem);
}
EXPORT_SYMBOL_GPL(nvme_start_queues);
-
void nvme_sync_queues(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
blk_sync_queue(ns->queue);
- up_read(&ctrl->namespaces_rwsem);
if (ctrl->admin_q)
blk_sync_queue(ctrl->admin_q);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 3ded54d2c9c6..344acf17a304 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -115,13 +115,12 @@ bool nvme_failover_req(struct request *req)
void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list) {
+ xa_for_each(&ctrl->namespaces, idx, ns) {
if (ns->head->disk)
kblockd_schedule_work(&ns->head->requeue_work);
}
- up_read(&ctrl->namespaces_rwsem);
}
static const char *nvme_ana_state_names[] = {
@@ -155,13 +154,12 @@ bool nvme_mpath_clear_current_path(struct nvme_ns *ns)
void nvme_mpath_clear_ctrl_paths(struct nvme_ctrl *ctrl)
{
struct nvme_ns *ns;
+ unsigned long idx;
mutex_lock(&ctrl->scan_lock);
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list)
+ xa_for_each(&ctrl->namespaces, idx, ns)
if (nvme_mpath_clear_current_path(ns))
kblockd_schedule_work(&ns->head->requeue_work);
- up_read(&ctrl->namespaces_rwsem);
mutex_unlock(&ctrl->scan_lock);
}
@@ -513,6 +511,7 @@ static int nvme_update_ana_state(struct nvme_ctrl *ctrl,
u32 nr_nsids = le32_to_cpu(desc->nnsids), n = 0;
unsigned *nr_change_groups = data;
struct nvme_ns *ns;
+ unsigned long idx;
dev_dbg(ctrl->device, "ANA group %d: %s.\n",
le32_to_cpu(desc->grpid),
@@ -524,8 +523,7 @@ static int nvme_update_ana_state(struct nvme_ctrl *ctrl,
if (!nr_nsids)
return 0;
- down_read(&ctrl->namespaces_rwsem);
- list_for_each_entry(ns, &ctrl->namespaces, list) {
+ xa_for_each(&ctrl->namespaces, idx, ns) {
unsigned nsid = le32_to_cpu(desc->nsids[n]);
if (ns->head->ns_id < nsid)
@@ -535,7 +533,6 @@ static int nvme_update_ana_state(struct nvme_ctrl *ctrl,
if (++n == nr_nsids)
break;
}
- up_read(&ctrl->namespaces_rwsem);
return 0;
}
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index c5c1bac797aa..a84eb663bcad 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -240,8 +240,7 @@ struct nvme_ctrl {
int numa_node;
struct blk_mq_tag_set *tagset;
struct blk_mq_tag_set *admin_tagset;
- struct list_head namespaces;
- struct rw_semaphore namespaces_rwsem;
+ struct xarray namespaces;
struct device ctrl_device;
struct device *device; /* char device */
struct cdev cdev;
@@ -415,8 +414,6 @@ enum nvme_ns_features {
};
struct nvme_ns {
- struct list_head list;
-
struct nvme_ctrl *ctrl;
struct request_queue *queue;
struct gendisk *disk;
--
2.22.1
More information about the Linux-nvme
mailing list