[PATCH] nvme: fc: stop lsrcv workqueue before freeing a rport

[Aristeu Rozanski]

While running blktests/nvme internally we managed to hit
a race in which a lsrcv work is still pending while the
rport gets removed. Strangely, this is way easier to
reproduce on a s390x than any other architecture we test.

	[12996.173074] nvme nvme5: NVME-FC{5}: controller connectivity lost. Awaiting Reconnect
	[12996.173446] Unable to handle kernel pointer dereference in virtual kernel address space
	[12996.173449] Failing address: e64d52f26d80e000 TEID: e64d52f26d80e803
	[12996.173452] Fault in home space mode while using kernel ASCE.
	[12996.173455] AS:0000000195920007 R3:0000000000000024
	[12996.173560] Oops: 0038 ilc:2 [#1]SMP
	[12996.173566] Modules linked in: nvme_fcloop nvmet_fc nvmet nvme_fc nvme_fabrics nvme nvme_core nvme_keyring nvme_auth sunrpc rfkill virtio_gpu virtio_dma_buf drm_client_lib drm_shmem_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm_kms_helper virtio_net net_failover fb failover virtio_input vfio_ccw mdev vfio_iommu_type1 vfio iommufd drm drm_panel_orientation_quirks font i2c_core fuse loop nfnetlink vsock_loopback vmw_vsock_virtio_transport_common vsock ctcm fsm qeth ccwgroup zfcp scsi_transport_fc qdio dasd_fba_mod dasd_eckd_mod dasd_mod xfs ghash_s390 prng virtio_blk des_s390 libdes sha3_512_s390 sha3_256_s390 sha_common dm_mirror dm_region_hash dm_log dm_mod paes_s390 crypto_engine pkey_cca pkey_ep11 zcrypt pkey_pckmo pkey aes_s390 [last unloaded: nvmet]
	[12996.173639] CPU: 1 UID: 0 PID: 633486 Comm: kworker/1:31 Kdump: loaded Not tainted 6.18.0-0.rc0.53c18dc078bb.1.RHEL100912.el10_1.s390x #1 NONE
	[12996.173644] Hardware name: IBM 8561 LT1 400 (KVM/Linux)
	[12996.173646] Workqueue: events nvme_fc_handle_ls_rqst_work [nvme_fc]
	[12996.173661] Krnl PSW : 0704c00180000000 e64d52f26d80e69e (0xe64d52f26d80e69e)
	[12996.173685]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 RI:0 EA:3
	[12996.173688] Krnl GPRS: 0000000000000000 e64d52f26d80e69e 00000001058e9830 00000000874f8800
	[12996.173691]            00000001058e9830 0000017ed7b593d0 0000000000000000 00000000874f8868
	[12996.173693]            00000000ec994000 00000000874f8800 0000000000000000 00000001058e9830
	[12996.173695]            0000000084ef2a00 0000000000000000 0000017ed7b594dc 000000ff5884fbb8
	[12996.173700] Krnl Code: Bad PSW.
	[12996.173702] Call Trace:
	[12996.173703]  [<e64d52f26d80e69e>] 0xe64d52f26d80e69e
	[12996.173707]  [<0000017ed7b32f40>] nvme_fc_xmt_ls_rsp+0x60/0xc0 [nvme_fc]
	[12996.173713]  [<0000017ed7b34218>] nvme_fc_handle_ls_rqst_work+0x138/0x240 [nvme_fc]
	[12996.173719]  [<0000017f5702b8cc>] process_one_work+0x1bc/0x3e0
	[12996.173730]  [<0000017f5702c53e>] worker_thread+0x23e/0x440
	[12996.173734]  [<0000017f5703830c>] kthread+0x12c/0x280
	[12996.173738]  [<0000017f56fb1d6c>] __ret_from_fork+0x3c/0x140
	[12996.173744]  [<0000017f57af236a>] ret_from_fork+0xa/0x30
	[12996.173750] Last Breaking-Event-Address:
	[12996.173750]  [<0000017ed7b594da>] fcloop_t2h_xmt_ls_rsp+0x10a/0x140 [nvme_fcloop]
	[12996.173757] Kernel panic - not syncing: Fatal exception: panic_on_oops

Cc: Justin Tee <justin.tee at broadcom.com>
Cc: Naresh Gottumukkala <nareshgottumukkala83 at gmail.com>
Cc: Paul Ely <paul.ely at broadcom.com>
Cc: Keith Busch <kbusch at kernel.org>
Cc: Jens Axboe <axboe at kernel.dk>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Signed-off-by: Aristeu Rozanski <aris at redhat.com>
---
 drivers/nvme/host/fc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 03987f497a5b5..1762f62ebf820 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -518,6 +518,8 @@ nvme_fc_free_rport(struct kref *ref)
 			localport_to_lport(rport->remoteport.localport);
 	unsigned long flags;
 
+	cancel_work_sync(&rport->lsrcv_work);
+
 	WARN_ON(rport->remoteport.port_state != FC_OBJSTATE_DELETED);
 	WARN_ON(!list_empty(&rport->ctrl_list));