[RFC PATCH] nvme-pci: adaptively poll completions on busy queues

[changfengnan]

> From: "changfengnan"<changfengnan at bytedance.com>
> Date:  Tue, Jul 28, 2026, 14:19
> Subject:  Re: [RFC PATCH] nvme-pci: adaptively poll completions on busy queues
> To: "Christoph Hellwig"<hch at lst.de>
> Cc: <linux-nvme at lists.infradead.org>, "Keith Busch"<kbusch at kernel.org>, "Jens Axboe"<axboe at kernel.dk>, "Christoph Hellwig"<hch at lst.de>, "Sagi Grimberg"<sagi at grimberg.me>, "Bart Van Assche"<bvanassche at acm.org>, "Andy Shevchenko"<andriy.shevchenko at intel.com>, "Thomas Gleixner"<tglx at kernel.org>, "Jun Zeng"<jun1.zeng at intel.com>, "Gang Cao"<gang.cao at intel.com>, "Jun I Jin"<jun.i.jin at intel.com>, "Liang A Fang"<liang.a.fang at intel.com>, "Yong Hu"<yong.hu at intel.com>, <linux-kernel at vger.kernel.org>, "Guzebing"<guzebing at bytedance.com>
> 
> > From: "Christoph Hellwig"<hch at lst.de>
> > Date:  Tue, Jul 28, 2026, 12:44
> > Subject:  Re: [RFC PATCH] nvme-pci: adaptively poll completions on busy queues
> > To: "changfengnan"<changfengnan at bytedance.com>
> > Cc: <linux-nvme at lists.infradead.org>, "Keith Busch"<kbusch at kernel.org>, "Jens Axboe"<axboe at kernel.dk>, "Christoph Hellwig"<hch at lst.de>, "Sagi Grimberg"<sagi at grimberg.me>, "Bart Van Assche"<bvanassche at acm.org>, "Andy Shevchenko"<andriy.shevchenko at intel.com>, "Thomas Gleixner"<tglx at kernel.org>, "Jun Zeng"<jun1.zeng at intel.com>, "Gang Cao"<gang.cao at intel.com>, "Jun I Jin"<jun.i.jin at intel.com>, "Liang A Fang"<liang.a.fang at intel.com>, "Yong Hu"<yong.hu at intel.com>, <linux-kernel at vger.kernel.org>, "Guzebing"<guzebing at bytedance.com>
> > On Tue, Jul 28, 2026 at 10:24:52AM +0800, changfengnan wrote:
> > > The patch showed 50 confirmed throughput improvements, with a median gain of 25.2%
> > > and a maximum gain of 63.0%. Only five cases showed confirmed regressions;
> > 
> > Can you drill down into the regression and figure out why they happen
> > and if we can mitigate them?  It would be nice if we could find a way
> > to avoid major regressions and enable the adaptive mode by default.
> 
> Of course, we’ll continue to investigate this issue to minimize performance
> regressions. As for enabling it by default, it’s too early to say—we need
> more testing to verify this, such as seeing how it performs on Gen3/Gen4
> NVMe drives.

Now I can answer this question: When QD=33/34, after one or two CQEs are
processed, the number of pending commands is exactly 32, so the system
enters a polling state. However, the number of pending I/O operations does
not guarantee that the CQ will refill quickly. On Samsung drives, the CQ
typically empties after one round of processing. At this point, `fsleep(20)` is
called, so new CQEs must wait when they arrive. This delays CQE processing
and causes IOPS to drop. At higher QD, on the other hand, keeps the CQ
sufficiently full, making polling more efficient. Simply adjusting parameters
does not solve the problem, because this approach confuses the number of
in-flight I/Os with completion density. A high number of in-flight I/Os does not
necessarily mean that using poll  is more efficient. 
Therefore, we developed an alternative solution to address this issue.
https://lore.kernel.org/linux-nvme/20260806031058.40176-1-changfengnan@bytedance.com/T/#u
> 
> >