DMA pool lock contention

[Caleb Sander Mateos]

On Thu, Apr 17, 2025 at 11:18 AM Keith Busch <kbusch at kernel.org> wrote:
>
> On Thu, Apr 17, 2025 at 10:06:11AM -0700, Caleb Sander Mateos wrote:
> > Hi Linux NVMe folks,
> > On a 32 KB NVMe passthru read workload, we see 2.4% of CPU time is
> > spent in _raw_spin_lock_irqsave called from dma_pool_alloc and
> > dma_pool_free.
>
> Just curious, do you have multiple cpu NUMA nodes here? Or do these CPUs
> share a last-level-cache?

CPUs from 2 NUMA nodes. Half of them are hypertwins of the others, so
the contention isn't quite as bad as it otherwise would be. Just
making the DMA pools per-NUMA-node would help considerably.

Best,
Caleb