blktests failures with v6.4

[Shinichiro Kawasaki]

Hi all,

I ran the latest blktests (git hash: 30bdcb72ad99) with v6.4 kernel and observed
test case failures listed below. I call for support to fix them. The list of
failures are same as the report for v6.3 kernel [1].

  [1] https://lore.kernel.org/linux-block/rsmmxrchy6voi5qhl4irss5sprna3f5owkqtvybxglcv2pnylm@xmrnpfu3tfpe/

List of failures
================
#1: block/011
#2: block/024
#3: nvme/003 (fabrics transport)
#4: nvme/030 or nvme/031 (rdma transport with siw)
#5: nvme/* (fc transport)

Failure description
===================

#1: block/011

   This test case shows two failure symptoms.

   Symptom A:

     The test case fails with NVME devices due to lockdep WARNING "possible
     circular locking dependency detected". Reported in Sep/2022 [2] and
     solution was discussed. Waiting a fix.

     [2] https://lore.kernel.org/linux-block/20220930001943.zdbvolc3gkekfmcv@shindev/

   Symptom B:

     The test case occasionally fail with fio assert messages. Fio bug is
     suspected. I'm preparing a fix patch [3].

     block/011 => nvme0n1 (disable PCI device while doing I/O)    [failed]
        runtime  32.250s  ...  1684.324s
        --- tests/block/011.out     2023-04-06 10:11:07.920670529 +0900
        +++ /home/shin/Blktests/blktests/results/nvme0n1/block/011.out.bad  2023-07-07 14:34:26.123295632 +0900
        @@ -1,2 +1,11 @@
         Running block/011
        +fio: ioengines.c:335: td_io_queue: Assertion `(io_u->flags & IO_U_F_FLIGHT) == 0' failed.
        +fio: ioengines.c:335: td_io_queue: Assertion `(io_u->flags & IO_U_F_FLIGHT) == 0' failed.
        +fio: ioengines.c:335: td_io_queue: Assertion `(io_u->flags & IO_U_F_FLIGHT) == 0' failed.
        +fio: ioengines.c:335: td_io_queue: Assertion `(io_u->flags & IO_U_F_FLIGHT) == 0' failed.
        +fio: pid=1492, got signal=6
        +fio: pid=1493, got signal=6
        ...
        (Run 'diff -u tests/block/011.out /home/shin/Blktests/blktests/results/nvme0n1/block/011.out.bad' to see the entire diff)

     [3] https://github.com/kawasaki/fio/commit/f4b68b90c6780a73d4eb017449400a7891443e3b

#2: block/024

   Fails on slow machines. Reported in Dec/2022. Test case side issue is
   suspected. Still needs further investigation.

   block/024 (do I/O faster than a jiffy and check iostats times) [failed]
    runtime    ...  4.347s
    --- tests/block/024.out     2022-12-06 20:51:41.525066605 +0900
    +++ /home/shin/kts/kernel-test-suite/sets/blktests/log/runlog/nodev/block/024.out.bad       2022-12-07 12:51:03.610924521 +0900
    @@ -6,5 +6,5 @@
     read 1 s
     write 1 s
     read 2 s
    -write 3 s
    +write 4 s
     Test complete

#3: nvme/003 (fabrics transport)

   When nvme test group is run with trtype=rdma or tcp, the test case fails
   due to lockdep WARNING "possible circular locking dependency detected".
   Reported in May/2023. Bart suggested a fix for trytpe=rdma [4] but it
   needs more discussion.

   [4] https://lore.kernel.org/linux-nvme/20230511150321.103172-1-bvanassche@acm.org/

#4: nvme/030 or nvme/031 (rdma transport with siw)

   When nvme test group is run with trtype=rdma and use_siw=1 configurations,
   nvme/030 or nvme/031 fail occasionally due to "BUG: KASAN: slab-use-after-
   free in __mutex_lock". Reported to linux-rdma in May/2023. A fix was
   suggested but it did not fix the root cause in rdma iwarp cm [5]. Waiting
   for a good fix.

   [5] https://lore.kernel.org/linux-rdma/20230612054237.1855292-1-shinichiro.kawasaki@wdc.com/

#5: nvme/* (fc transport)

   With trtype=fc configuration, test run on nvme test group hangs. Daniel is
   driving fix work [6].

   [6] https://lore.kernel.org/linux-nvme/20230620133711.22840-1-dwagner@suse.de/