[bug report] nvme-tcp poll queue causes busy loop and 100% CPU in nvme_tcp_poll() on the latest linux-block/for-next

[Yi Zhang]

Hi

This issue was observed on the latest linux-block/for-next during CKI
test, where enabling the poll queues leads to a tight busy polling
loop and 100% CPU usage during "nvme connect".
It seems was introduced from v6.19-rc1 and cannot reproduced on v6.18.
I will try to bisect it.
I am happy to test any proposed fixes or provide additional traces if needed.

Reproducer:
#nvme_trtype=tcp  ./check nvme/047
nvme/047 (tr=tcp) (test different queue types for fabric transports) [passed]
    runtime  1061.108s  ...  19857.364s

# ps aux | grep "nvme connect"
root        2492 99.9  0.0  15884  5520 pts/0    R+   01:28   0:13
nvme connect --traddr 127.0.0.1 --transport tcp --trsvcid 4420 --nqn
blktests-subsystem-1
--hostnqn=nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349
--hostid=0f01fb42-9f7f-4856-b0b3-51e60b8de349 --nr-write-queues=1
--nr-poll-queues=1 --output-format=json

# perf top -p  2492
Samples: 71K of event 'cycles:P', 4000 Hz, Event count (approx.):
39693035620 lost: 0/0 drop: 0/0
Overhead  Shared O  Symbol
  51.62%  [kernel]  [k] blk_hctx_poll
  34.64%  [kernel]  [k] nvme_tcp_poll
  13.39%  [kernel]  [k] __x86_indirect_thunk_rax
   0.02%  [kernel]  [k] native_irq_return_iret
   0.02%  [kernel]  [k] native_write_msr
   0.02%  [kernel]  [k] update_load_avg
   0.01%  [kernel]  [k] __raw_spin_lock_irqsave
   0.01%  [kernel]  [k] native_sched_clock
   0.01%  [kernel]  [k] read_tsc
   0.01%  [kernel]  [k] sched_balance_trigger
   0.01%  [kernel]  [k] __hrtimer_run_queues
   0.01%  [kernel]  [k] __update_load_avg_cfs_rq
   0.01%  [kernel]  [k] __timekeeping_advance
   0.01%  [kernel]  [k] native_read_msr
   0.01%  [kernel]  [k] ktime_get_update_offsets_now
   0.01%  [kernel]  [k] task_tick_fair
   0.01%  [kernel]  [k] native_apic_msr_eoi
   0.01%  [kernel]  [k] sugov_get_util

dmesg:
[ 5514.212238] run blktests nvme/047 at 2026-01-19 16:37:29
[ 5514.229581] loop0: detected capacity change from 0 to 2097152
[ 5514.235566] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[ 5514.244495] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[ 5514.267088] nvmet: Created nvm controller 1 for subsystem
blktests-subsystem-1 for NQN
nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[ 5514.269066] nvme nvme0: creating 73 I/O queues.
[ 5514.276926] nvme nvme0: mapped 1/72/0 default/read/poll queues.
[ 5514.301778] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr
127.0.0.1:4420, hostnqn:
nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349
[ 5515.068893] nvme nvme0: Removing ctrl: NQN "blktests-subsystem-1"
[ 5515.150669] nvmet: Created nvm controller 2 for subsystem
blktests-subsystem-1 for NQN
nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[ 5515.152791] nvme nvme0: creating 74 I/O queues.
[ 5515.161907] nvme nvme0: mapped 1/72/1 default/read/poll queues.
[25370.530190] nvme nvme0: new ctrl: NQN "blktests-subsystem-1", addr
127.0.0.1:4420, hostnqn:
nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349
[25371.283929] nvme nvme0: Removing ctrl: NQN "blktests-subsystem-1"


--
Best Regards,
  Yi Zhang