[PATCH 0/3] nvme-core: make gencounter feature tunable

Chaitanya Kulkarni chaitanyak at nvidia.com
Fri Dec 10 03:21:13 PST 2021 

From: Chaitanya Kulkarni <kch at nvidia.com>

Hi,

The recent commit uses a combination of command id and a gencounter
to calculate the command id so that we can call out buggy controllers
for spurious completions and that also avoids use after free.
This commit adds various if statements and bitwise operations
such as &, <<, >>, & along with comparison operations to validate the  
gencounter and print out the errors.
 
This feature is required to catch the buggy controller, but in the  
production environment where controller is validated and known to be
stable this achieves nothing but adds additional code and runtime CPU  
instructions for the PCIe controller.
Worst case scenario when NVMeOF setup is using passthru backend then
these instructions gets duplicated on the host and on the target side
for each controller.
 
This patch-series makes the gencounter feature configurable by   
defining a new KConfig macro CONFIG_NVME_DEBUG_USE_CID_GENCTR and adding
a new KConfig section Debug with subsection for enabling the  
gencounter debug feature. We move the current tag + gencounter code when
CONFIG_NVME_DEBUG_USE_CID_GENCTR is defined and keep the original code
where we use tag based command id where
CONFIG_NVME_DEBUG_USE_CID_GENCTR is not defined.
The newly added Debug section is kept under NVMe Driver section under
host side.
 
In this way we achieve:-
1. Removing the entire gencounter code at the compile-time and keeping the  
   fastpath as lean as possible when the controller is known to be stable.
2. Retaining the gencounter behavior by enabling the debug feature.
3. Letting user decide which debug features to enable instead of forcing
   any.
4. Avoid measurable differences in fio performance numbers. see below.

* fio BW/iops/slat cid gencounter on avg IOPS = 75k, BW = 295MiB :-
read: IOPS=75.3k, BW=294MiB/s
read: IOPS=75.8k, BW=296MiB/s 
read: IOPS=75.2k, BW=294MiB/s 
read: IOPS=76.0k, BW=297MiB/s 
read: IOPS=75.4k, BW=294MiB/s 
read: IOPS=76.0k, BW=297MiB/s 
read: IOPS=75.5k, BW=295MiB/s 
read: IOPS=75.4k, BW=294MiB/s 

Submission latency avg = 6698 nsec :-
slat (nsec): avg=5057.99
slat (nsec): avg=7434.15
slat (nsec): avg=6310.50
slat (nsec): avg=8083.82
slat (nsec): avg=7297.77
slat (nsec): avg=6003.77
slat (nsec): avg=6657.75
slat (nsec): avg=6745.34

* fio BW/iops/slat cid gencounter off IOPS avg = 76k, BW = 297MiB :-
read: IOPS=77.7k, BW=304MiB/s
read: IOPS=76.5k, BW=299MiB/s 
read: IOPS=74.1k, BW=290MiB/s 
read: IOPS=75.8k, BW=296MiB/s 
read: IOPS=75.9k, BW=297MiB/s 
read: IOPS=75.2k, BW=294MiB/s 
read: IOPS=77.6k, BW=303MiB/s 
read: IOPS=76.8k, BW=300MiB/s 

Submission latency avg = 5030 nsec :-

slat (nsec): avg=3859.20
slat (nsec): avg=4669.78
slat (nsec): avg=3778.51
slat (nsec): avg=8436.18
slat (nsec): avg=5653.83
slat (nsec): avg=5286.60
slat (nsec): avg=4176.05
slat (nsec): avg=4385.19

-ck

Chaitanya Kulkarni (3):
  nvme-core: make cid gencounter configurable
  nvme-core: move gencounter check into nvme_cid()
  nvme: add KConfig options for debug features

 drivers/nvme/host/Kconfig | 10 ++++++++++
 drivers/nvme/host/core.c  |  3 ---
 drivers/nvme/host/nvme.h  | 31 +++++++++++++++++++++++++++++++
 3 files changed, 41 insertions(+), 3 deletions(-)

-- 
2.29.0

Test Log :-

root at dev nvme (nvme-5.16) # ./compile_nvme.sh 
+ umount /mnt/nvme0n1
+ clear_dmesg
./compile_nvme.sh: line 3: clear_dmesg: command not found
umount: /mnt/nvme0n1: no mount point specified.
+ modprobe -r nvme-fabrics
+ modprobe -r nvme_loop
+ modprobe -r nvmet
+ modprobe -r nvme
+ sleep 1
+ modprobe -r nvme-core
+ lsmod
+ grep nvme
+ sleep 1
+ git diff
+ sleep 1
++ nproc
+ make -j 48 M=drivers/nvme/ modules
  MODPOST drivers/nvme/Module.symvers
  CC [M]  drivers/nvme/host/nvme-core.mod.o
  CC [M]  drivers/nvme/host/nvme-fabrics.mod.o
  CC [M]  drivers/nvme/host/nvme-fc.mod.o
  CC [M]  drivers/nvme/host/nvme-rdma.mod.o
  CC [M]  drivers/nvme/host/nvme-tcp.mod.o
  CC [M]  drivers/nvme/host/nvme.mod.o
  CC [M]  drivers/nvme/target/nvme-fcloop.mod.o
  CC [M]  drivers/nvme/target/nvme-loop.mod.o
  CC [M]  drivers/nvme/target/nvmet-fc.mod.o
  CC [M]  drivers/nvme/target/nvmet-rdma.mod.o
  CC [M]  drivers/nvme/target/nvmet-tcp.mod.o
  CC [M]  drivers/nvme/target/nvmet.mod.o
  LD [M]  drivers/nvme/host/nvme-rdma.ko
  LD [M]  drivers/nvme/host/nvme-core.ko
  LD [M]  drivers/nvme/host/nvme-fc.ko
  LD [M]  drivers/nvme/target/nvme-loop.ko
  LD [M]  drivers/nvme/target/nvmet.ko
  LD [M]  drivers/nvme/target/nvmet-rdma.ko
  LD [M]  drivers/nvme/host/nvme-tcp.ko
  LD [M]  drivers/nvme/target/nvme-fcloop.ko
  LD [M]  drivers/nvme/target/nvmet-fc.ko
  LD [M]  drivers/nvme/target/nvmet-tcp.ko
  LD [M]  drivers/nvme/host/nvme-fabrics.ko
  LD [M]  drivers/nvme/host/nvme.ko
  BTF [M] drivers/nvme/target/nvme-loop.ko
  BTF [M] drivers/nvme/host/nvme-rdma.ko
  BTF [M] drivers/nvme/target/nvme-fcloop.ko
  BTF [M] drivers/nvme/host/nvme-fabrics.ko
  BTF [M] drivers/nvme/target/nvmet-rdma.ko
  BTF [M] drivers/nvme/target/nvmet-tcp.ko
  BTF [M] drivers/nvme/host/nvme.ko
  BTF [M] drivers/nvme/host/nvme-tcp.ko
  BTF [M] drivers/nvme/target/nvmet-fc.ko
  BTF [M] drivers/nvme/host/nvme-fc.ko
  BTF [M] drivers/nvme/host/nvme-core.ko
  BTF [M] drivers/nvme/target/nvmet.ko
+ HOST=drivers/nvme/host
+ TARGET=drivers/nvme/target
++ uname -r
+ HOST_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/
++ uname -r
+ TARGET_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target/
+ cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host//
+ cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//
+ ls -lrth /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/ /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//
/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/:
total 6.6M
-rw-r--r--. 1 root root  2.7M Dec 10 01:49 nvme-core.ko
-rw-r--r--. 1 root root  446K Dec 10 01:49 nvme-fabrics.ko
-rw-r--r--. 1 root root  964K Dec 10 01:49 nvme-fc.ko
-rw-r--r--. 1 root root  732K Dec 10 01:49 nvme.ko
-rw-r--r--. 1 root root 1019K Dec 10 01:49 nvme-rdma.ko
-rw-r--r--. 1 root root  893K Dec 10 01:49 nvme-tcp.ko

/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//:
total 6.7M
-rw-r--r--. 1 root root 498K Dec 10 01:49 nvme-fcloop.ko
-rw-r--r--. 1 root root 435K Dec 10 01:49 nvme-loop.ko
-rw-r--r--. 1 root root 767K Dec 10 01:49 nvmet-fc.ko
-rw-r--r--. 1 root root 3.3M Dec 10 01:49 nvmet.ko
-rw-r--r--. 1 root root 992K Dec 10 01:49 nvmet-rdma.ko
-rw-r--r--. 1 root root 768K Dec 10 01:49 nvmet-tcp.ko
+ modprobe nvme
root at dev nvme (nvme-5.16) # grep NVME_DEBUG .config
# CONFIG_NVME_DEBUG_USE_CID_GENCTR is not set
(reverse-i-search)`for': mv 0001-nvme-code-command_id-with-a-genctr-^Cr-use-after-fre.patch  sagi-genctr.patch
root at dev nvme (nvme-5.16) # ./passthru_config.sh /dev/nvme0
umount: /mnt/nvme0n1: no mount point specified.
+ mkdir /sys/kernel/config/nvmet/subsystems/pt-nqn
+ sleep 1
+ echo 'Initializing passthru ctrl path ...'
Initializing passthru ctrl path ...
+ echo -n /dev/nvme0
+ sleep 1
+ echo 1
+ echo 1
+ mkdir /sys/kernel/config/nvmet/ports/1/
+ sleep 1
+ echo -n loop
+ sleep 1
+ echo 'Connecting passthru ctrl to the port '
Connecting passthru ctrl to the port 
+ ln -s /sys/kernel/config/nvmet/subsystems/pt-nqn /sys/kernel/config/nvmet/ports/1/subsystems/
+ sleep 1
+ nvme connect -t loop -n pt-nqn
+ sleep 1
+ tr -s ' ' ' '
+ set +x
root at dev nvme (nvme-5.16) # for i in `seq 1 10`; do fio fio/randread.fio --filename=/dev/nvme1n1                                                                           --output=no-cid-genctr-${i}.fio; done
root at dev nvme (nvme-5.16) # makemnconfig 
configuration written to .config

*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.

root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # ./kernel_compile.sh 
  DEPMOD  /lib/modules/5.16.0-rc1nvme+
sh ./arch/x86/boot/install.sh 5.16.0-rc1nvme+ \
	arch/x86/boot/bzImage System.map "/boot"
Generating grub configuration file ...
done
real	1m33.758s
user	4m54.633s
sys	1m50.129s
root at dev nvme (nvme-5.16) # reboot 
Connection to 192.168.0.46 closed by remote host.
Connection to 192.168.0.46 closed.
[neo at fedora ~]$ dev
root at 192.168.0.46's password: 
Last login: Fri Dec 10 01:49:40 2021 from 192.168.0.63
root at dev ~ # cd nvme/
root at dev nvme (nvme-5.16) # ./compile_nvme.sh 
+ umount /mnt/nvme0n1
+ clear_dmesg
./compile_nvme.sh: line 3: clear_dmesg: command not found
umount: /mnt/nvme0n1: no mount point specified.
+ modprobe -r nvme-fabrics
+ modprobe -r nvme_loop
+ modprobe -r nvmet
+ modprobe -r nvme
+ sleep 1
+ modprobe -r nvme-core
+ lsmod
+ grep nvme
+ sleep 1
./+ git diff
+ sleep 1
++ nproc
+ make -j 48 M=drivers/nvme/ modules
  MODPOST drivers/nvme/Module.symvers
  CC [M]  drivers/nvme/host/nvme-core.mod.o
  CC [M]  drivers/nvme/host/nvme-fabrics.mod.o
  CC [M]  drivers/nvme/host/nvme-fc.mod.o
  CC [M]  drivers/nvme/host/nvme-rdma.mod.o
  CC [M]  drivers/nvme/host/nvme-tcp.mod.o
  CC [M]  drivers/nvme/host/nvme.mod.o
  CC [M]  drivers/nvme/target/nvme-fcloop.mod.o
  CC [M]  drivers/nvme/target/nvme-loop.mod.o
  CC [M]  drivers/nvme/target/nvmet-fc.mod.o
  CC [M]  drivers/nvme/target/nvmet-rdma.mod.o
  CC [M]  drivers/nvme/target/nvmet-tcp.mod.o
  CC [M]  drivers/nvme/target/nvmet.mod.o
  LD [M]  drivers/nvme/target/nvme-loop.ko
  LD [M]  drivers/nvme/target/nvmet.ko
  LD [M]  drivers/nvme/host/nvme.ko
  LD [M]  drivers/nvme/target/nvmet-rdma.ko
  LD [M]  drivers/nvme/host/nvme-rdma.ko
  LD [M]  drivers/nvme/host/nvme-tcp.ko
  LD [M]  drivers/nvme/host/nvme-core.ko
  LD [M]  drivers/nvme/target/nvmet-fc.ko
  LD [M]  drivers/nvme/target/nvmet-tcp.ko
  LD [M]  drivers/nvme/host/nvme-fabrics.ko
  LD [M]  drivers/nvme/host/nvme-fc.ko
  BTF [M] drivers/nvme/target/nvme-loop.ko
  LD [M]  drivers/nvme/target/nvme-fcloop.ko
  BTF [M] drivers/nvme/target/nvmet-rdma.ko
  BTF [M] drivers/nvme/host/nvme-fabrics.ko
  BTF [M] drivers/nvme/target/nvme-fcloop.ko
  BTF [M] drivers/nvme/host/nvme-rdma.ko
  BTF [M] drivers/nvme/host/nvme-tcp.ko
  BTF [M] drivers/nvme/target/nvmet-fc.ko
  BTF [M] drivers/nvme/target/nvmet-tcp.ko
  BTF [M] drivers/nvme/host/nvme.ko
  BTF [M] drivers/nvme/host/nvme-fc.ko
  BTF [M] drivers/nvme/target/nvmet.ko
  BTF [M] drivers/nvme/host/nvme-core.ko
+ HOST=drivers/nvme/host
+ TARGET=drivers/nvme/target
++ uname -r
+ HOST_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/
++ uname -r
+ TARGET_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target/
+ cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host//
+ cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//
+ ls -lrth /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/ /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//
/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/:
total 6.6M
-rw-r--r--. 1 root root  2.7M Dec 10 02:17 nvme-core.ko
-rw-r--r--. 1 root root  446K Dec 10 02:17 nvme-fabrics.ko
-rw-r--r--. 1 root root  964K Dec 10 02:17 nvme-fc.ko
-rw-r--r--. 1 root root  732K Dec 10 02:17 nvme.ko
-rw-r--r--. 1 root root 1019K Dec 10 02:17 nvme-rdma.ko
-rw-r--r--. 1 root root  897K Dec 10 02:17 nvme-tcp.ko

/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//:
total 6.7M
-rw-r--r--. 1 root root 498K Dec 10 02:17 nvme-fcloop.ko
-rw-r--r--. 1 root root 436K Dec 10 02:17 nvme-loop.ko
-rw-r--r--. 1 root root 767K Dec 10 02:17 nvmet-fc.ko
-rw-r--r--. 1 root root 3.3M Dec 10 02:17 nvmet.ko
-rw-r--r--. 1 root root 992K Dec 10 02:17 nvmet-rdma.ko
-rw-r--r--. 1 root root 768K Dec 10 02:17 nvmet-tcp.ko
+ modprobe nvme
root at dev nvme (nvme-5.16) # ./passthru_config.sh /dev/nvme0
umount: /mnt/nvme0n1: no mount point specified.
+ mkdir /sys/kernel/config/nvmet/subsystems/pt-nqn
+ sleep 1
+ echo 'Initializing passthru ctrl path ...'
Initializing passthru ctrl path ...
+ echo -n /dev/nvme0
+ sleep 1
+ echo 1
+ echo 1
+ mkdir /sys/kernel/config/nvmet/ports/1/
+ sleep 1
+ echo -n loop
+ sleep 1
+ echo 'Connecting passthru ctrl to the port '
Connecting passthru ctrl to the port 
+ ln -s /sys/kernel/config/nvmet/subsystems/pt-nqn /sys/kernel/config/nvmet/ports/1/subsystems/
+ sleep 1
+ nvme connect -t loop -n pt-nqn
+ sleep 1
root at dev nvme (nvme-5.16) # grep NVME_DEBUG .config
CONFIG_NVME_DEBUG_USE_CID_GENCTR=y
root at dev nvme (nvme-5.16) # for i in `seq 1 10`; do fio fio/randread.fio --filename=/dev/nvme1n1                                                                           --output=yes-cid-genctr-${i}.fio; done
root at dev nvme (nvme-5.16) # .0%][r=297MiB/s][r=76.0k IOPS][eta 00m:00s]
root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # 
root at dev nvme (nvme-5.16) # grep slat yes*genctr* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ','
5057.99
7434.15
6310.50
8083.82
7297.77
6003.77
6657.75
6745.34
root at dev nvme (nvme-5.16) # grep slat no*genctr* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ','
3859.20
4669.78
3778.51
8436.18
5653.83
5286.60
4176.05
4385.19
root at dev nvme (nvme-5.16) # sum=0;for i in `grep slat yes-cid-genctr-* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ','| cut -f 1 -d '.'`; do let sum=sum+$i; done ; echo $sum/8|bc
6698
root at dev nvme (nvme-5.16) # sum=0;for i in `grep slat no-cid-genctr-* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ','| cut -f 1 -d '.'`; do let sum=sum+$i; done ; echo $sum/8|bc
5030
root at dev nvme (nvme-5.16) #

More information about the Linux-nvme mailing list