Number of data and admin queues in use

[Thomas Glanzmann]

Hello,
I have Linux system hooked up over two dedicated links to a NetApp using
NVMe/TCP. I would like to find out how many data and admin queues there are and
their queue depth? How can I find out the same?

So far, I found out:

(live) [~] nvme netapp ontapdevices /dev/nvme0n1
/dev/nvme0n1, Vserver svm1, Subsystem svm1_subsystem_553, Namespace Path rx3082_1, NSID 1, UUID 7f6be93b-60cb-11f0-866f-d039ead647e8, 1.10TB
(live) [~] nvme list-subsys /dev/nvme0n1
nvme-subsys0 - NQN=nqn.1992-08.com.netapp:sn.e0a0273a60b711f09deed039ead647e8:subsystem.svm1_subsystem_553
               hostnqn=nqn.2014-08.org.nvmexpress:uuid:20f011e6-9ab8-584f-abb0-a260d2d685c4
\
 +- nvme0 tcp traddr=192.168.0.2,trsvcid=4420,src_addr=192.168.0.100 live optimized
 +- nvme1 tcp traddr=192.168.1.2,trsvcid=4420,src_addr=192.168.1.100 live optimized
(live) [~] nvme list
Node                  Generic               SN                   Model                                    Namespace  Usage                      Format           FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- --------
/dev/nvme0n1          /dev/ng0n1            824nlJYbYm5ZAAAAAAAB NetApp ONTAP Controller                  0x1          8.62  GB /   1.10  TB      4 KiB +  0 B   9.16.1
(live) [~] sudo nvme get-feature /dev/nvme0n1 --feature-id=7 -H
get-feature:0x07 (Number of Queues), Current value:0x00010001
        Number of IO Completion Queues Allocated (NCQA): 2
        Number of IO Submission Queues Allocated (NSQA): 2

I also have some local NVMe where I would like to find out the same:

(infra) [~] sudo nvme get-feature /dev/nvme0n1 --feature-id=7 -H
get-feature:0x07 (Number of Queues), Current value:0x007f007f
        Number of IO Completion Queues Allocated (NCQA): 128
        Number of IO Submission Queues Allocated (NSQA): 128
(infra) [~] nvme list
Node                  Generic               SN                   Model                                    Namespace Usage                      Format           FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme1n1          /dev/ng1n1            50026B7685E14353     KINGSTON SKC3000D2048G                   1           2.05  TB /   2.05  TB    512   B +  0 B   EIFK31.6
/dev/nvme0n1          /dev/ng0n1            50026B7685E1439E     KINGSTON SKC3000D2048G                   1           2.05  TB /   2.05  TB    512   B +  0 B   EIFK31.6
(infra) [~] nvme list-subsys /dev/nvme0n1
nvme-subsys0 - NQN=nqn.2020-04.com.kingston:nvme:nvm-subsystem-sn-50026B7685E1439E
\
 +- nvme0 pcie 0000:01:00.0 live

I one heard that the Linux kernel allocates one queue per processor (core or
hyperthread). I can see that using /proc/interrupts but only on physical
devices, but not on NVME/TCP systems.

Cheers,
        Thomas