[bug report] blktests nvme/061 hang with rdma transport and siw driver

[Shinichiro Kawasaki]

Hello all,

Recently, a new blktests test case nvme/061 was introduced. It does "test fabric
target teardown and setup during I/O". When I run this test case repeatedly with
rdma transport and siw driver on the kernel v6.15-rc2, KASAN slab-use-after-free
happens in __pwq_activate_work() [1], and then the test system hangs. The hang
is recreated in stable manner.

It looks the new test case revealed a hidden problem. I observed the same hang
with a few older kernels v6.14 and v6.13. Then the problem has been existing for
a while.

Actions for fix will be appreciated. I'm willing to run tests with debug patches
for fix candidate patches.


[1]

[77516.128920][T163063] run blktests nvme/061 at 2025-04-15 10:50:44
[77516.243039][T163125] loop0: detected capacity change from 0 to 2097152
[77516.255638][T163128] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[77516.271505][T163132] nvmet_rdma: enabling port 0 (10.0.2.15:4420)
[77516.315953][T163139] nvme nvme1: rdma connection establishment failed (-512)
[77516.338654][T157654] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[77516.348375][T163139] nvme nvme1: Please enable CONFIG_NVME_MULTIPATH for full support of multi-port devices.
[77516.352460][T163139] nvme nvme1: creating 4 I/O queues.
[77516.388708][T163139] nvme nvme1: mapped 4/0/0 default/read/poll queues.
[77516.393808][T163139] nvme nvme1: new ctrl: NQN "blktests-subsystem-1", addr 10.0.2.15:4420, hostnqn: nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349
[77517.490278][T147091] nvmet_rdma: post_recv cmd failed
[77517.490278][    C0] nvme nvme1: RECV for CQE 0x0000000033c0c31a failed with status WR flushed (5)
[77517.490287][    C0] nvme nvme1: starting error recovery
[77517.490357][T163171] nvmet_rdma: received IB QP event: send queue drained (5)
[77517.490539][T147091] nvmet_rdma: sending cmd response failed
[77517.521129][T147416] nvme nvme1: Reconnecting in 1 seconds...
[77517.577846][T163189] loop1: detected capacity change from 0 to 2097152
[77517.588566][T163192] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[77517.598937][T163196] nvmet_rdma: enabling port 0 (10.0.2.15:4420)
[77518.536807][T157654] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[77518.544349][T147416] nvme nvme1: Please enable CONFIG_NVME_MULTIPATH for full support of multi-port devices.
[77518.548378][T147416] nvme nvme1: creating 4 I/O queues.
[77518.599549][T147416] nvme nvme1: mapped 4/0/0 default/read/poll queues.
[77518.605257][T147416] nvme nvme1: Successfully reconnected (1 attempts)
[77518.656899][T147416] nvmet_rdma: post_recv cmd failed
[77518.656913][    C0] nvme nvme1: RECV for CQE 0x0000000069d8d80d failed with status WR flushed (5)
[77518.657190][T147416] nvmet_rdma: sending cmd response failed
[77518.657576][    C0] nvme nvme1: starting error recovery
[77518.657642][T163212] nvmet_rdma: received IB QP event: send queue drained (5)
[77518.679562][T147414] nvme nvme1: Reconnecting in 1 seconds...
[77518.806843][T163230] loop2: detected capacity change from 0 to 2097152
[77518.824732][T163233] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[77518.840860][T163237] nvmet_rdma: enabling port 0 (10.0.2.15:4420)
[77519.690812][T157654] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[77519.698371][T147413] nvme nvme1: Please enable CONFIG_NVME_MULTIPATH for full support of multi-port devices.
[77519.701969][T147413] nvme nvme1: creating 4 I/O queues.
[77519.756714][T147413] nvme nvme1: mapped 4/0/0 default/read/poll queues.
[77519.763289][T147413] nvme nvme1: Successfully reconnected (1 attempts)
[77519.931468][    C0] nvme nvme1: RECV for CQE 0x000000001546cc5d failed with status WR flushed (5)
[77519.931495][T147414] nvmet_rdma: post_recv cmd failed
[77519.931918][    C0] nvme nvme1: starting error recovery
[77519.932315][T147414] nvmet_rdma: sending cmd response failed
[77519.934507][T163096] nvmet_rdma: received IB QP event: QP fatal error (1)
[77519.957634][T147091] nvme nvme1: Reconnecting in 1 seconds...
[77520.014605][T163271] loop3: detected capacity change from 0 to 2097152
[77520.036545][T163274] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[77520.059189][T163278] nvmet_rdma: enabling port 0 (10.0.2.15:4420)
[77520.970973][T157654] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[77520.979516][T147091] nvme nvme1: Please enable CONFIG_NVME_MULTIPATH for full support of multi-port devices.
[77520.983088][T147091] nvme nvme1: creating 4 I/O queues.
[77521.051099][T147091] nvme nvme1: mapped 4/0/0 default/read/poll queues.
[77521.056904][T147091] nvme nvme1: Successfully reconnected (1 attempts)
[77521.139591][    C3] nvme nvme1: RECV for CQE 0x00000000315e08be failed with status WR flushed (5)
[77521.139635][T146462] nvmet_rdma: post_recv cmd failed
[77521.139989][    C3] nvme nvme1: starting error recovery
[77521.140221][T146462] nvmet_rdma: sending cmd response failed
[77521.142903][T163094] nvmet_rdma: received IB QP event: QP fatal error (1)
[77521.144318][T147091] ==================================================================
[77521.145465][T147091] BUG: KASAN: slab-use-after-free in __pwq_activate_work+0x1ff/0x250
[77521.146488][T147091] Read of size 8 at addr ffff88811f9cf800 by task kworker/u16:1/147091
[77521.147569][T147091]
[77521.148233][T147091] CPU: 2 UID: 0 PID: 147091 Comm: kworker/u16:1 Not tainted 6.15.0-rc2+ #27 PREEMPT(voluntary)
[77521.148238][T147091] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014
[77521.148240][T147091] Workqueue:  0x0 (iw_cm_wq)
[77521.148248][T147091] Call Trace:
[77521.148250][T147091]  <TASK>
[77521.148252][T147091]  dump_stack_lvl+0x6a/0x90
[77521.148257][T147091]  print_report+0x174/0x554
[77521.148262][T147091]  ? __virt_addr_valid+0x208/0x430
[77521.148265][T147091]  ? __pwq_activate_work+0x1ff/0x250
[77521.148268][T147091]  kasan_report+0xae/0x170
[77521.148272][T147091]  ? __pwq_activate_work+0x1ff/0x250
[77521.148275][T147091]  __pwq_activate_work+0x1ff/0x250
[77521.148278][T147091]  pwq_dec_nr_in_flight+0x8c5/0xfb0
[77521.148282][T147091]  process_one_work+0xc11/0x1460
[77521.148286][T147091]  ? __pfx_process_one_work+0x10/0x10
[77521.148293][T147091]  ? assign_work+0x16c/0x240
[77521.148296][T147091]  worker_thread+0x5ef/0xfd0
[77521.148300][T147091]  ? __pfx_worker_thread+0x10/0x10
[77521.148302][T147091]  kthread+0x3b0/0x770
[77521.148306][T147091]  ? __pfx_kthread+0x10/0x10
[77521.148308][T147091]  ? rcu_is_watching+0x11/0xb0
[77521.148312][T147091]  ? _raw_spin_unlock_irq+0x24/0x50
[77521.148315][T147091]  ? rcu_is_watching+0x11/0xb0
[77521.148317][T147091]  ? __pfx_kthread+0x10/0x10
[77521.148319][T147091]  ret_from_fork+0x30/0x70
[77521.148322][T147091]  ? __pfx_kthread+0x10/0x10
[77521.148324][T147091]  ret_from_fork_asm+0x1a/0x30
[77521.148328][T147091]  </TASK>
[77521.148329][T147091]
[77521.170936][T147091] Allocated by task 147416:
[77521.171538][T147091]  kasan_save_stack+0x2c/0x50
[77521.172209][T147091]  kasan_save_track+0x10/0x30
[77521.172894][T147091]  __kasan_kmalloc+0xa6/0xb0
[77521.173582][T147091]  alloc_work_entries+0xa9/0x260 [iw_cm]
[77521.174314][T147091]  iw_cm_connect+0x23/0x4a0 [iw_cm]
[77521.175005][T147091]  rdma_connect_locked+0xbfd/0x1920 [rdma_cm]
[77521.175770][T147091]  nvme_rdma_cm_handler+0x8e5/0x1b60 [nvme_rdma]
[77521.176475][T147091]  cma_cm_event_handler+0xae/0x320 [rdma_cm]
[77521.177132][T147091]  cma_work_handler+0x106/0x1b0 [rdma_cm]
[77521.177796][T147091]  process_one_work+0x84f/0x1460
[77521.178410][T147091]  worker_thread+0x5ef/0xfd0
[77521.179023][T147091]  kthread+0x3b0/0x770
[77521.179593][T147091]  ret_from_fork+0x30/0x70
[77521.180163][T147091]  ret_from_fork_asm+0x1a/0x30
[77521.180750][T147091]
[77521.181177][T147091] Freed by task 147091:
[77521.181663][T147091]  kasan_save_stack+0x2c/0x50
[77521.182178][T147091]  kasan_save_track+0x10/0x30
[77521.182728][T147091]  kasan_save_free_info+0x37/0x60
[77521.183336][T147091]  __kasan_slab_free+0x4b/0x70
[77521.183921][T147091]  kfree+0x13a/0x4b0
[77521.184440][T147091]  dealloc_work_entries+0x125/0x1f0 [iw_cm]
[77521.185114][T147091]  iwcm_deref_id+0x6f/0xa0 [iw_cm]
[77521.185691][T147091]  cm_work_handler+0x136/0x1ba0 [iw_cm]
[77521.186280][T147091]  process_one_work+0x84f/0x1460
[77521.186844][T147091]  worker_thread+0x5ef/0xfd0
[77521.187390][T147091]  kthread+0x3b0/0x770
[77521.187905][T147091]  ret_from_fork+0x30/0x70
[77521.188452][T147091]  ret_from_fork_asm+0x1a/0x30
[77521.189026][T147091]
[77521.189431][T147091] Last potentially related work creation:
[77521.190064][T147091]  kasan_save_stack+0x2c/0x50
[77521.190628][T147091]  kasan_record_aux_stack+0xa3/0xb0
[77521.191227][T147091]  __queue_work+0x2ff/0x1390
[77521.191787][T147091]  queue_work_on+0x67/0xc0
[77521.192327][T147091]  cm_event_handler+0x46a/0x820 [iw_cm]
[77521.192966][T147091]  siw_cm_upcall+0x330/0x650 [siw]
[77521.193552][T147091]  siw_cm_work_handler+0x6b9/0x2b20 [siw]
[77521.194189][T147091]  process_one_work+0x84f/0x1460
[77521.194756][T147091]  worker_thread+0x5ef/0xfd0
[77521.195306][T147091]  kthread+0x3b0/0x770
[77521.195817][T147091]  ret_from_fork+0x30/0x70
[77521.196345][T147091]  ret_from_fork_asm+0x1a/0x30
[77521.196900][T147091]
[77521.197279][T147091] The buggy address belongs to the object at ffff88811f9cf800
[77521.197279][T147091]  which belongs to the cache kmalloc-512 of size 512
[77521.198740][T147091] The buggy address is located 0 bytes inside of
[77521.198740][T147091]  freed 512-byte region [ffff88811f9cf800, ffff88811f9cfa00)
[77521.200184][T147091]
[77521.200589][T147091] The buggy address belongs to the physical page:
[77521.201294][T147091] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x11f9cc
[77521.202199][T147091] head: order:2 mapcount:0 entire_mapcount:0 nr_pages_mapped:0 pincount:0
[77521.203086][T147091] flags: 0x17ffffc0000040(head|node=0|zone=2|lastcpupid=0x1fffff)
[77521.203902][T147091] page_type: f5(slab)
[77521.204444][T147091] raw: 0017ffffc0000040 ffff888100042c80 ffffea0004899d00 dead000000000002
[77521.205304][T147091] raw: 0000000000000000 0000000000100010 00000000f5000000 0000000000000000
[77521.206194][T147091] head: 0017ffffc0000040 ffff888100042c80 ffffea0004899d00 dead000000000002
[77521.207075][T147091] head: 0000000000000000 0000000000100010 00000000f5000000 0000000000000000
[77521.207949][T147091] head: 0017ffffc0000002 ffffea00047e7301 00000000ffffffff 00000000ffffffff
[77521.208836][T147091] head: ffffffffffffffff 0000000000000000 00000000ffffffff 0000000000000004
[77521.209713][T147091] page dumped because: kasan: bad access detected
[77521.210439][T147091]
[77521.210898][T147091] Memory state around the buggy address:
[77521.211578][T147091]  ffff88811f9cf700: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[77521.212430][T147091]  ffff88811f9cf780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[77521.213295][T147091] >ffff88811f9cf800: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[77521.214166][T147091]                    ^
[77521.214706][T147091]  ffff88811f9cf880: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[77521.215495][T147091]  ffff88811f9cf900: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[77521.216433][T147091] ==================================================================
[77521.217386][T147091] ------------[ cut here ]------------
[77521.218160][T147091] WARNING: CPU: 2 PID: 147091 at kernel/workqueue.c:1678 __pwq_activate_work+0x1f0/0x250
[77521.219196][T147091] Modules linked in: siw ib_uverbs nvmet_rdma nvmet nvme_rdma nvme_fabrics ib_umad dm_service_time rtrs_core rdma_cm nbd iw_cm ib_cm ib_core nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables qrtr sunrpc ppdev 9pnet_virtio 9pnet netfs parport_pc pcspkr e1000 i2c_piix4 parport i2c_smbus fuse loop dm_multipath nfnetlink vsock_loopback vmw_vsock_virtio_transport_common vmw_vsock_vmci_transport vsock zram vmw_vmci xfs nvme bochs drm_client_lib nvme_core drm_shmem_helper sym53c8xx drm_kms_helper nvme_keyring scsi_transport_spi drm floppy nvme_auth serio_raw ata_generic pata_acpi qemu_fw_cfg [last unloaded: ib_uverbs]
[77521.225863][T147091] CPU: 2 UID: 0 PID: 147091 Comm: kworker/u16:1 Tainted: G    B               6.15.0-rc2+ #27 PREEMPT(voluntary)
[77521.227138][T147091] Tainted: [B]=BAD_PAGE
[77521.227846][T147091] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014
[77521.228953][T147091] Workqueue:  0x0 (iw_cm_wq)
[77521.229761][T147091] RIP: 0010:__pwq_activate_work+0x1f0/0x250
[77521.230650][T147091] Code: 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 80 3c 02 00 75 64 4d 89 6c 24 50 4c 8b 65 00 49 8d 74 24 60 e9 ca fe ff ff <0f> 0b e9 42 fe ff ff 48 89 f7 e8 d1 d7 93 00 e9 2c fe ff ff 48 89
[77521.233126][T147091] RSP: 0018:ffff88810740fbc0 EFLAGS: 00010046
[77521.234031][T147091] RAX: 0000000000000001 RBX: ffff88811f9cf800 RCX: ffffffffa84fcdc6
[77521.235155][T147091] RDX: 0000000000000001 RSI: 0000000000000008 RDI: ffffffffadc7f4e0
[77521.236241][T147091] RBP: ffff888134e1cc00 R08: 0000000000000001 R09: fffffbfff5b8fe9c
[77521.237314][T147091] R10: ffffffffadc7f4e7 R11: 0000000000300e30 R12: 0000000000000001
[77521.238386][T147091] R13: ffff888137f0a080 R14: ffff88810016e000 R15: 0000000000000007
[77521.239436][T147091] FS:  0000000000000000(0000) GS:ffff88840053f000(0000) knlGS:0000000000000000
[77521.240562][T147091] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[77521.241532][T147091] CR2: 00007f2c865bf000 CR3: 0000000131d92000 CR4: 00000000000006f0
[77521.242600][T147091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[77521.243655][T147091] DR3: 0000000000000000 DR6: 00000000ffff07f0 DR7: 0000000000000400
[77521.244703][T147091] Call Trace:
[77521.245400][T147091]  <TASK>
[77521.246092][T147091]  pwq_dec_nr_in_flight+0x8c5/0xfb0
[77521.246930][T147091]  process_one_work+0xc11/0x1460
[77521.247754][T147091]  ? __pfx_process_one_work+0x10/0x10
[77521.248569][T147091]  ? assign_work+0x16c/0x240
[77521.249360][T147091]  worker_thread+0x5ef/0xfd0
[77521.250187][T147091]  ? __pfx_worker_thread+0x10/0x10
[77521.251021][T147091]  kthread+0x3b0/0x770
[77521.251768][T147091]  ? __pfx_kthread+0x10/0x10
[77521.252549][T147091]  ? rcu_is_watching+0x11/0xb0
[77521.253333][T147091]  ? _raw_spin_unlock_irq+0x24/0x50
[77521.254152][T147091]  ? rcu_is_watching+0x11/0xb0
[77521.254922][T147091]  ? __pfx_kthread+0x10/0x10
[77521.255686][T147091]  ret_from_fork+0x30/0x70
[77521.256441][T147091]  ? __pfx_kthread+0x10/0x10
[77521.257204][T147091]  ret_from_fork_asm+0x1a/0x30
[77521.257964][T147091]  </TASK>
[77521.258600][T147091] irq event stamp: 0
[77521.259306][T147091] hardirqs last  enabled at (0): [<0000000000000000>] 0x0
[77521.260245][T147091] hardirqs last disabled at (0): [<ffffffffa84f3b3f>] copy_process+0x1f3f/0x87d0
[77521.261371][T147091] softirqs last  enabled at (0): [<ffffffffa84f3ba4>] copy_process+0x1fa4/0x87d0
[77521.262456][T147091] softirqs last disabled at (0): [<0000000000000000>] 0x0
[77521.263367][T147091] ---[ end trace 0000000000000000 ]---
[77521.264199][T147091] Oops: general protection fault, probably for non-canonical address 0xe049fc4da00047d3: 0000 [#1] SMP KASAN PTI
[77521.265504][T147091] KASAN: maybe wild-memory-access in range [0x0250026d00023e98-0x0250026d00023e9f]
[77521.266618][T147091] CPU: 2 UID: 0 PID: 147091 Comm: kworker/u16:1 Tainted: G    B   W           6.15.0-rc2+ #27 PREEMPT(voluntary)
[77521.267876][T147091] Tainted: [B]=BAD_PAGE, [W]=WARN
[77521.268667][T147091] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-3.fc41 04/01/2014
[77521.269803][T147091] Workqueue:  0x0 (iw_cm_wq)
[77521.270545][T147091] RIP: 0010:__list_del_entry_valid_or_report+0xaf/0x1d0
[77521.271489][T147091] Code: 48 c1 ea 03 80 3c 02 00 0f 85 0e 01 00 00 48 39 5d 00 75 71 48 b8 00 00 00 00 00 fc ff df 49 8d 6c 24 08 48 89 ea 48 c1 ea 03 <80> 3c 02 00 0f 85 db 00 00 00 49 3b 5c 24 08 0f 85 81 00 00 00 5b
[77521.273666][T147091] RSP: 0018:ffff88810740fb48 EFLAGS: 00010002
[77521.274497][T147091] RAX: dffffc0000000000 RBX: ffff88811f9cf808 RCX: 0000000000000000
[77521.275520][T147091] RDX: 004a004da00047d3 RSI: 0000000000000008 RDI: ffff88810740fb10
[77521.276479][T147091] RBP: 0250026d00023e9b R08: 0000000000000000 R09: fffffbfff598bf0c
[77521.277439][T147091] R10: ffffffffacc5f867 R11: 0000000000300e30 R12: 0250026d00023e93
[77521.278380][T147091] R13: 1ffff1102002dc0d R14: ffff88811f9cf808 R15: 0250026d00023e8b
[77521.279336][T147091] FS:  0000000000000000(0000) GS:ffff88840053f000(0000) knlGS:0000000000000000
[77521.280326][T147091] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[77521.281184][T147091] CR2: 00007f2c865bf000 CR3: 0000000131d92000 CR4: 00000000000006f0
[77521.282153][T147091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[77521.283131][T147091] DR3: 0000000000000000 DR6: 00000000ffff07f0 DR7: 0000000000000400
[77521.284098][T147091] Call Trace:
[77521.284684][T147091]  <TASK>
[77521.285238][T147091]  move_linked_works+0xa8/0x2a0
[77521.285938][T147091]  __pwq_activate_work+0xc4/0x250
[77521.286628][T147091]  pwq_dec_nr_in_flight+0x8c5/0xfb0
[77521.287337][T147091]  process_one_work+0xc11/0x1460
[77521.288036][T147091]  ? __pfx_process_one_work+0x10/0x10
[77521.288758][T147091]  ? assign_work+0x16c/0x240
[77521.289443][T147091]  worker_thread+0x5ef/0xfd0
[77521.290111][T147091]  ? __pfx_worker_thread+0x10/0x10
[77521.290817][T147091]  kthread+0x3b0/0x770
[77521.291447][T147091]  ? __pfx_kthread+0x10/0x10
[77521.292119][T147091]  ? rcu_is_watching+0x11/0xb0
[77521.292796][T147091]  ? _raw_spin_unlock_irq+0x24/0x50
[77521.293500][T147091]  ? rcu_is_watching+0x11/0xb0
[77521.294177][T147091]  ? __pfx_kthread+0x10/0x10
[77521.294836][T147091]  ret_from_fork+0x30/0x70
[77521.295484][T147091]  ? __pfx_kthread+0x10/0x10
[77521.296149][T147091]  ret_from_fork_asm+0x1a/0x30
[77521.296816][T147091]  </TASK>
[77521.297361][T147091] Modules linked in: siw ib_uverbs nvmet_rdma nvmet nvme_rdma nvme_fabrics ib_umad dm_service_time rtrs_core rdma_cm nbd iw_cm ib_cm ib_core nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables qrtr sunrpc ppdev 9pnet_virtio 9pnet netfs parport_pc pcspkr e1000 i2c_piix4 parport i2c_smbus fuse loop dm_multipath nfnetlink vsock_loopback vmw_vsock_virtio_transport_common vmw_vsock_vmci_transport vsock zram vmw_vmci xfs nvme bochs drm_client_lib nvme_core drm_shmem_helper sym53c8xx drm_kms_helper nvme_keyring scsi_transport_spi drm floppy nvme_auth serio_raw ata_generic pata_acpi qemu_fw_cfg [last unloaded: ib_uverbs]
[77521.304220][T147091] ---[ end trace 0000000000000000 ]---
[77521.305044][T147091] RIP: 0010:__list_del_entry_valid_or_report+0xaf/0x1d0
[77521.305969][T147091] Code: 48 c1 ea 03 80 3c 02 00 0f 85 0e 01 00 00 48 39 5d 00 75 71 48 b8 00 00 00 00 00 fc ff df 49 8d 6c 24 08 48 89 ea 48 c1 ea 03 <80> 3c 02 00 0f 85 db 00 00 00 49 3b 5c 24 08 0f 85 81 00 00 00 5b
[77521.308176][T147091] RSP: 0018:ffff88810740fb48 EFLAGS: 00010002
[77521.309049][T147091] RAX: dffffc0000000000 RBX: ffff88811f9cf808 RCX: 0000000000000000
[77521.310057][T147091] RDX: 004a004da00047d3 RSI: 0000000000000008 RDI: ffff88810740fb10
[77521.311067][T147091] RBP: 0250026d00023e9b R08: 0000000000000000 R09: fffffbfff598bf0c
[77521.312062][T147091] R10: ffffffffacc5f867 R11: 0000000000300e30 R12: 0250026d00023e93
[77521.313038][T147091] R13: 1ffff1102002dc0d R14: ffff88811f9cf808 R15: 0250026d00023e8b
[77521.314055][T147091] FS:  0000000000000000(0000) GS:ffff88840053f000(0000) knlGS:0000000000000000
[77521.315149][T147091] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[77521.316112][T147091] CR2: 00007f2c865bf000 CR3: 0000000131d92000 CR4: 00000000000006f0
[77521.317071][T147091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[77521.318100][T147091] DR3: 0000000000000000 DR6: 00000000ffff07f0 DR7: 0000000000000400
[77521.319112][T147091] note: kworker/u16:1[147091] exited with irqs disabled
[77521.320117][T147091] note: kworker/u16:1[147091] exited with preempt_count 2