NVME, isolcpus, and irq affinity

[Keith Busch]

On Mon, Oct 12, 2020 at 12:50:49PM -0600, Chris Friesen wrote:
> On 10/12/2020 11:13 AM, Keith Busch wrote:
> > On Mon, Oct 12, 2020 at 09:49:38AM -0600, Chris Friesen wrote:
> 
> > > Assuming I'm reading the code correctly, how does it make sense for the NVME
> > > driver to affine interrupts to CPUs which have explicitly been designated as
> > > "isolated"?
> > 
> > The driver allocates interrupts, but doesn't affine them. The driver
> > lets the kernel handle that instead.
> 
> Okay, thanks for the quick reply.
> 
> The interrupts in question have names of the form "nvmeXqY", where X and Y
> are integers.  Some of them are affined to individual CPUs while others are
> affined to two or more CPUs.
> 
> If no tasks on the CPUs in question do any disk I/O is it possible these
> interrupts could still get triggered by activity instigated by tasks on
> other CPUs?  Or would they basically be idle and inert in that case?

In order to wake the thread, the top half handler needs to see
completion entries on that interrupt's associated queue. If you never
use the CPUs in an interrupt's affinity mask to dispatch IO, there won't
be completions, so the thread never wakes.