[PATCH v5 00/12] enable nvmet-fc for blktests

Daniel Wagner dwagner at suse.de
Wed Jan 31 00:51:00 PST 2024 

I've added Christophs' RBs and refactored the last patch as suggested. IMO, it
makes it more readable, so a win :)

Also I'd run some more tests, e.g. with KASAN dissabled to see if we hit some
other races but all looks good. There are still a bunch of fallouts (e.g. my
test systems seems to be broken for the AUTH tests). But as said, it doesn't
hang or produces any crahses anymore.

nvme/002 (create many subsystems and test discovery)         [not run]
    nvme_trtype=fc is not supported in this test
nvme/003 (test if we're sending keep-alives to a discovery controller)
    runtime  11.470s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/003 (test if we're sending keep-alives to a discovery controller) [passed]
    runtime  11.470s  ...  11.399s
nvme/004 (test nvme and nvmet UUID NS descriptors)          
    runtime  0.580s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/004 (test nvme and nvmet UUID NS descriptors)           [passed]
    runtime  0.580s  ...  0.467s
nvme/005 (reset local loopback target)                      
    runtime  0.653s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/005 (reset local loopback target)                       [passed]
    runtime  0.653s  ...  0.574s
nvme/006 (create an NVMeOF target with a block device-backed ns)
    runtime  0.187s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/006 (create an NVMeOF target with a block device-backed ns) [passed]
    runtime  0.187s  ...  0.133s
nvme/007 (create an NVMeOF target with a file-backed ns)     [passed]
    runtime  0.146s  ...  0.089s
nvme/008 (create an NVMeOF host with a block device-backed ns) [passed]
    runtime  0.576s  ...  0.429s
nvme/009 (create an NVMeOF host with a file-backed ns)       [passed]
    runtime  0.515s  ...  0.411s
nvme/010 (run data verification fio job on NVMeOF block device-backed ns)
    runtime  8.543s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/010 (run data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  8.543s  ...  4.541s
nvme/011 (run data verification fio job on NVMeOF file-backed ns)
    runtime  26.668s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/011 (run data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  26.668s  ...  8.701s
nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns)
    runtime  16.517s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed]
    runtime  16.517s  ...  5.739s
nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns)
    runtime  28.152s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/013 (run mkfs and data verification fio job on NVMeOF file-backed ns) [passed]
    runtime  28.152s  ...  8.771s
nvme/014 (flush a NVMeOF block device-backed ns)            
    runtime  3.492s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/014 (flush a NVMeOF block device-backed ns)             [passed]
    runtime  3.492s  ...  2.401s
nvme/015 (unit test for NVMe flush for file backed ns)      
    runtime  2.952s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/015 (unit test for NVMe flush for file backed ns)       [passed]
    runtime  2.952s  ...  1.962s
nvme/016 (create/delete many NVMeOF block device-backed ns and test discovery) [not run]
    nvme_trtype=fc is not supported in this test
nvme/017 (create/delete many file-ns and test discovery)     [not run]
    nvme_trtype=fc is not supported in this test
nvme/018 (unit test NVMe-oF out of range access on a file backend)
    runtime  0.607s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/018 (unit test NVMe-oF out of range access on a file backend) [passed]
    runtime  0.607s  ...  0.468s
nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns)
    runtime  0.613s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/019 (test NVMe DSM Discard command on NVMeOF block-device ns) [passed]
    runtime  0.613s  ...  0.534s
nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns)
    runtime  0.583s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/020 (test NVMe DSM Discard command on NVMeOF file-backed ns) [passed]
    runtime  0.583s  ...  0.495s
nvme/021 (test NVMe list command on NVMeOF file-backed ns)  
    runtime  0.559s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/021 (test NVMe list command on NVMeOF file-backed ns)   [passed]
    runtime  0.559s  ...  0.486s
nvme/022 (test NVMe reset command on NVMeOF file-backed ns) 
    runtime  0.726s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/022 (test NVMe reset command on NVMeOF file-backed ns)  [passed]
    runtime  0.726s  ...  0.618s
nvme/023 (test NVMe smart-log command on NVMeOF block-device ns)
    runtime  0.637s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/023 (test NVMe smart-log command on NVMeOF block-device ns) [passed]
    runtime  0.637s  ...  0.504s
nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns)
    runtime  0.635s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/024 (test NVMe smart-log command on NVMeOF file-backed ns) [passed]
    runtime  0.635s  ...  0.490s
nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns)
    runtime  0.581s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/025 (test NVMe effects-log command on NVMeOF file-backed ns) [passed]
    runtime  0.581s  ...  0.479s
nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns)
    runtime  0.576s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/026 (test NVMe ns-descs command on NVMeOF file-backed ns) [passed]
    runtime  0.576s  ...  0.463s
nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns)
    runtime  0.663s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/027 (test NVMe ns-rescan command on NVMeOF file-backed ns) [passed]
    runtime  0.663s  ...  0.492s
nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns)
    runtime  0.577s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/028 (test NVMe list-subsys command on NVMeOF file-backed ns) [passed]
    runtime  0.577s  ...  0.501s
nvme/029 (test userspace IO via nvme-cli read/write interface)
    runtime  0.885s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/029 (test userspace IO via nvme-cli read/write interface) [passed]
    runtime  0.885s  ...  0.653s
nvme/030 (ensure the discovery generation counter is updated appropriately)
nvme/030 (ensure the discovery generation counter is updated appropriately) [failed]
    runtime  0.677s  ...  0.505sc device: nvme0
    --- tests/nvme/030.out      2023-08-30 10:39:08.428409596 +0200
    +++ /home/wagi/work/blktests/results/nodev/nvme/030.out.bad 2024-01-31 09:40:26.552302587 +0100
    @@ -1,2 +1,7 @@
     Running nvme/030
    +warning: use generated hostid instead of hostid file
    +warning: use generated hostid instead of hostid file
    +warning: use generated hostid instead of hostid file
    +warning: use generated hostid instead of hostid file
    +warning: use generated hostid instead of hostid file
     Test complete
nvme/031 (test deletion of NVMeOF controllers immediately after setup)
    runtime  3.622s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/031 (test deletion of NVMeOF controllers immediately after setup) [passed]
    runtime  3.622s  ...  2.907s
nvme/038 (test deletion of NVMeOF subsystem without enabling) [passed]
    runtime  0.085s  ...  0.036s
nvme/040 (test nvme fabrics controller reset/disconnect operation during I/O)
    runtime  6.757s  ...
WARNING: Test did not clean up fc device: nvme0
nvme/040 (test nvme fabrics controller reset/disconnect operation during I/O) [passed]
    runtime  6.757s  ...  6.555s
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
nvme/041 (Create authenticated connections)                  [not run]
    kernel option NVME_AUTH has not been enabled
    kernel option NVME_TARGET_AUTH has not been enabled
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
nvme/042 (Test dhchap key types for authenticated connections) [not run]
    kernel option NVME_AUTH has not been enabled
    kernel option NVME_TARGET_AUTH has not been enabled
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
nvme/043 (Test hash and DH group variations for authenticated connections) [not run]
    kernel option NVME_AUTH has not been enabled
    kernel option NVME_TARGET_AUTH has not been enabled
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
nvme/044 (Test bi-directional authentication)                [not run]
    kernel option NVME_AUTH has not been enabled
    kernel option NVME_TARGET_AUTH has not been enabled
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 286: /usr/bin/grep: Permission denied
/usr/bin/zgrep: line 271: /usr/bin/gzip: Permission denied
nvme/045 (Test re-authentication)                            [not run]
    kernel option NVME_AUTH has not been enabled
    kernel option NVME_TARGET_AUTH has not been enabled
nvme/047 (test different queue types for fabric transports)  [not run]
    nvme_trtype=fc is not supported in this test
nvme/048 (Test queue count changes on reconnect)             [failed]
    runtime  0.549s  ...  0.378s
    --- tests/nvme/048.out      2023-11-28 12:59:52.711505550 +0100
    +++ /home/wagi/work/blktests/results/nodev/nvme/048.out.bad 2024-01-31 09:40:47.502748168 +0100
    @@ -1,3 +1,7 @@
     Running nvme/048
    +expected queue count 2 not set
    +FAIL
    +expected queue count 3 not set
    +FAIL
     disconnected 1 controller(s)
     Test complete

changes:
v5:
 - collected RBs
 - refactored 'nvmet-fc: use RCU list iterator for assoc_list'
 
v4:
 - dropped patches which got applied
 - dropped 'nvmet-fc: free hostport after release reference to tgtport'
 - reworked commit message of 'nvmet-fc: untangle cross refcounting objects'
   and renamed it to 'nvmet-fc: defer cleanup using RCU properly'
 - added 'nvmet-fcloop: swap the list_add_tail arguments'
   and 'nvmet-fc: use RCU list iterator for assoc_list'
 - added RBs
 - https://lore.kernel.org/linux-nvme/20240130094938.1575-1-dwagner@suse.de/
 
v3:
 - collected all patches into one series
 - updated ref counting in nvmet-fc
 - https://lore.kernel.org/linux-nvme/20231218153105.12717-1-dwagner@suse.de/

v2:
 - added RBs
 - added new patches
 - https://lore.kernel.org/linux-nvme/20230620133711.22840-1-dwagner@suse.de/
  
v1:
 - https://lore.kernel.org/linux-nvme/20230615094356.14878-1-dwagner@suse.de/ 


*** BLURB HERE ***

Daniel Wagner (12):
  nvme-fc: do not wait in vain when unloading module
  nvmet-fcloop: swap the list_add_tail arguments
  nvmet-fc: release reference on target port
  nvmet-fc: defer cleanup using RCU properly
  nvmet-fc: free queue and assoc directly
  nvmet-fc: hold reference on hostport match
  nvmet-fc: remove null hostport pointer check
  nvmet-fc: do not tack refs on tgtports from assoc
  nvmet-fc: abort command when there is no binding
  nvmet-fc: avoid deadlock on delete association path
  nvmet-fc: take ref count on tgtport before delete assoc
  nvmet-fc: use RCU list iterator for assoc_list

 drivers/nvme/host/fc.c       |  47 ++-------
 drivers/nvme/target/fc.c     | 185 +++++++++++++++++++----------------
 drivers/nvme/target/fcloop.c |   6 +-
 3 files changed, 111 insertions(+), 127 deletions(-)

-- 
2.43.0

More information about the Linux-nvme mailing list