blktests/nvme/052 failure

Wed Jun 26 22:42:27 PDT 2024

Hi,

Not sure what happened to my previous report, sending it again.

blktests/nvme/052 resulting in Oops.
This is can be reproduced 100% of the time on nvme-6.11, HEAD :-

commit 62eaa15c6aeef5011d3d41b69b63e02cf280324c (origin/nvme-6.11)
Author: Thomas Song <tsong at purestorage.com>
Date:   Tue Jun 25 08:26:05 2024 -0400

     nvme-multipath: implement "queue-depth" iopolicy

See below.

-ck

[  858.989209] BUG: unable to handle page fault for address: 
00000031004600c1
[  858.989978] #PF: supervisor read access in kernel mode
[  858.990426] #PF: error_code(0x0000) - not-present page
[  858.990831] PGD 0 P4D 0
[  858.991030] Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI
[  858.991390] CPU: 36 PID: 2629 Comm: kworker/u229:3 Tainted: 
G           O     N 6.10.0-rc3nvme+ #69
[  858.992101] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 
rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
[  858.992904] Workqueue: nvme-wq nvme_scan_work [nvme_core]
[  858.993369] RIP: 0010:lockref_get+0x4/0x60
[  858.993674] Code: bc 9f ff b8 01 00 00 00 eb ad e8 d7 27 72 00 0f 1f 
80 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e 
fa <48> 8b 07 85 c0 75 33 48 89 c2 be 64 00 00 00 48 89 d1 89 d2 48 c1
[  858.995355] RSP: 0018:ffffc9000544fcd0 EFLAGS: 00010202
[  858.995748] RAX: 0000000000000000 RBX: ffff888105906438 RCX: 
0000000000000002
[  858.996400] RDX: 0000000000037abc RSI: ffffffff8159fa30 RDI: 
00000031004600c1
[  858.996988] RBP: ffffc9000544fd20 R08: 00000031004600c1 R09: 
ffff88981fb32330
[  858.997638] R10: 0000000000000001 R11: fffffffffff1d6a2 R12: 
ffff8881059063c8
[  858.998491] R13: ffff888105906448 R14: 0000000000000001 R15: 
ffff888105906438
[  858.998989] FS:  0000000000000000(0000) GS:ffff88981fb00000(0000) 
knlGS:0000000000000000
[  858.999292] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  858.999518] CR2: 00000031004600c1 CR3: 0000000002a1c000 CR4: 
0000000000350ef0
[  858.999788] DR0: ffffffff837ce5b0 DR1: ffffffff837ce5b1 DR2: 
ffffffff837ce5b2
[  859.000057] DR3: ffffffff837ce5b3 DR6: 00000000ffff0ff0 DR7: 
0000000000000600
[  859.000333] Call Trace:
[  859.000445]  <TASK>
[  859.000536]  ? __die+0x24/0x70
[  859.000665]  ? page_fault_oops+0x158/0x4e0
[  859.000830]  ? __schedule+0x354/0xb00
[  859.000982]  ? exc_page_fault+0x77/0x170
[  859.001138]  ? asm_exc_page_fault+0x26/0x30
[  859.001315]  ? __pfx_remove_one+0x10/0x10
[  859.001477]  ? lockref_get+0x4/0x60
[  859.001622]  simple_recursive_removal+0x37/0x2d0
[  859.001811]  ? __pfx_remove_one+0x10/0x10
[  859.001980]  debugfs_remove+0x44/0x70
[  859.002137]  nvme_ns_remove+0x3a/0x200 [nvme_core]
[  859.002332]  nvme_remove_invalid_namespaces+0xfd/0x130 [nvme_core]
[  859.002582]  nvme_scan_work+0x2bc/0x5e0 [nvme_core]
[  859.002777]  ? ttwu_do_activate+0x5d/0x1e0
[  859.002947]  process_one_work+0x158/0x360
[  859.003108]  worker_thread+0x2fd/0x410
[  859.003258]  ? __pfx_worker_thread+0x10/0x10
[  859.003432]  kthread+0xd0/0x100
[  859.003571]  ? __pfx_kthread+0x10/0x10
[  859.003724]  ret_from_fork+0x31/0x50
[  859.003875]  ? __pfx_kthread+0x10/0x10
[  859.004039]  ret_from_fork_asm+0x1a/0x30
[  859.004204]  </TASK>
[  859.004297] Modules linked in: loop nvme_loop(O) nvmet(O) 
nvme_keyring nvme_fabrics(O) nvme(O) nvme_core(O) nvme_auth 
snd_seq_dummy snd_hrtimer snd_seq snd_seq_device snd_timer snd soundcore 
bridge stp llc rfkill ip6table_mangle ip6table_raw ip6table_security 
iptable_mangle iptable_raw iptable_security ip_set nf_tables nfnetlink 
ip6table_filter ip6_tables iptable_filter tun sunrpc xfs intel_rapl_msr 
intel_rapl_common kvm_amd iTCO_wdt ppdev iTCO_vendor_support ccp joydev 
kvm parport_pc parport i2c_i801 pcspkr i2c_smbus lpc_ich ip_tables 
crct10dif_pclmul crc32_pclmul crc32c_intel bochs drm_vram_helper 
ghash_clmulni_intel drm_kms_helper drm_ttm_helper sha512_ssse3 
sha256_ssse3 ttm virtio_net sha1_ssse3 serio_raw net_failover drm 
failover dimlib qemu_fw_cfg ipmi_devintf ipmi_msghandler fuse [last 
unloaded: nvme_auth]

Entering kdb (current=0xffff888107e68000, pid 2629) on processor 36 
Oops: (null)
due to oops @ 0xffffffff817aae44
CPU: 36 PID: 2629 Comm: kworker/u229:3 Tainted: G           O N 
6.10.0-rc3nvme+ #69
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 
rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
Workqueue: nvme-wq nvme_scan_work [nvme_core]
RIP: 0010:lockref_get+0x4/0x60
Code: bc 9f ff b8 01 00 00 00 eb ad e8 d7 27 72 00 0f 1f 80 00 00 00 00 
90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa <48> 8b 07 
85 c0 75 33 48 89 c2 be 64 00 00 00 48 89 d1 89 d2 48 c1
RSP: 0018:ffffc9000544fcd0 EFLAGS: 00010202
RAX: 0000000000000000 RBX: ffff888105906438 RCX: 0000000000000002
RDX: 0000000000037abc RSI: ffffffff8159fa30 RDI: 00000031004600c1
RBP: ffffc9000544fd20 R08: 00000031004600c1 R09: ffff88981fb32330
R10: 0000000000000001 R11: fffffffffff1d6a2 R12: ffff8881059063c8
R13: ffff888105906448 R14: 0000000000000001 R15: ffff888105906438
FS:  0000000000000000(0000) GS:ffff88981fb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000031004600c1 CR3: 0000000002a1c000 CR4: 0000000000350ef0
DR0: ffffffff837ce5b0 DR1: ffffffff837ce5b1 DR2: ffffffff837ce5b2
DR3: ffffffff837ce5b3 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
  <TASK>
  ? kdb_main_loop+0x31f/0x960
  ? io_napi_sqpoll_busy_poll+0xf0/0x120
  ? kdb_stub+0x1ae/0x3f0
  ? kgdb_cpu_enter+0x2b3/0x610
  ? kgdb_handle_exception+0xbd/0x100
  ? __kgdb_notify+0x30/0xd0
  ? kgdb_notify+0x21/0x40
  ? notifier_call_chain+0x5b/0xd0
  ? notify_die+0x53/0x80
  ? __die+0x51/0x70
  ? page_fault_oops+0x158/0x4e0
  ? __schedule+0x354/0xb00
  ? exc_page_fault+0x77/0x170
  ? asm_exc_page_fault+0x26/0x30
  ? __pfx_remove_one+0x10/0x10
  ? lockref_get+0x4/0x60
  simple_recursive_removal+0x37/0x2d0
  ? __pfx_remove_one+0x10/0x10
  debugfs_remove+0x44/0x70
  nvme_ns_remove+0x3a/0x200 [nvme_core]
  nvme_remove_invalid_namespaces+0xfd/0x130 [nvme_core]
  nvme_scan_work+0x2bc/0x5e0 [nvme_core]
  ? ttwu_do_activate+0x5d/0x1e0
  process_one_work+0x158/0x360
  worker_thread+0x2fd/0x410
  ? __pfx_worker_thread+0x10/0x10
  kthread+0xd0/0x100
  ? __pfx_kthread+0x10/0x10
  ret_from_fork+0x31/0x50
  ? __pfx_kthread+0x10/0x10
  ret_from_fork_asm+0x1a/0x30
  </TASK>