[PATCH v2 00/22] Add new DMA mapping operation for P2PDMA

[Logan Gunthorpe]

Hi,

This patchset continues my work to to add P2PDMA support to the common
dma map operations. This allows for creating SGLs that have both P2PDMA
and regular pages which is a necessary step to allowing P2PDMA pages in
userspace.

The earlier RFC[1] and v1[2] postings generated a lot of great feedback.
This version adds a bunch more cleanup at the start of the series. I'll
probably look to split the earlier patches off and get them merged
indpendantly after a round of review with this series as this series
has gotten quite long.

I'm happy to do a few more passes if anyone has any further feedback
or better ideas.

This series is based on v5.13-rc1 and a git branch can be found here:

  https://github.com/sbates130272/linux-p2pmem/  p2pdma_map_ops_v2

Thanks,

Logan

[1] https://lore.kernel.org/linux-block/20210311233142.7900-1-logang@deltatee.com/
[2] https://lore.kernel.org/linux-block/20210408170123.8788-1-logang@deltatee.com/

Changes sine v1:
 * Rebased onto v5.13-rc1
 * Add some cleanup to the existing P2PDMA code to fix up some naming
   conventions and documentation as the code has evolved a bit since the
   names were chosen. (As suggested by John)
 * Add a patch that adds a warning if a host bridge is not in the whitelist
   (as suggested by Don)
 * Change to using dma_map_sgtable() instead of creating a new
   interface. For this, a couple of .map_sg implementations were changed
   to return full error codes. (as per Christoph)
 * Renamed the scatterlist functions to include the term "dma" to
   indicate that they apply to the DMA side of the sg. (per Jason)
 * Introduce ib_dma_pci_p2p_dma_supported() helper instead of open
   coding the check (per Jason)
 * Numerous minor adjustments and documentation fixes

Changes since the RFC:
 * Added comment and fixed up the pci_get_slot patch. (per Bjorn)
 * Fixed glaring sg_phys() double offset bug. (per Robin)
 * Created a new map operation (dma_map_sg_p2pdma()) with a new calling
   convention instead of modifying the calling convention of
   dma_map_sg(). (per Robin)
 * Integrated the two similar pci_p2pdma_dma_map_type() and
   pci_p2pdma_map_type() functions into one (per Ira)
 * Reworked some of the logic in the map_sg() implementations into
   helpers in the p2pdma code. (per Christoph)
 * Dropped a bunch of unnecessary symbol exports (per Christoph)
 * Expanded the code in dma_pci_p2pdma_supported() for clarity. (per
   Ira and Christoph)
 * Finished off using the new dma_map_sg_p2pdma() call in rdma_rw
   and removed the old pci_p2pdma_[un]map_sg(). (per Jason)

--

Logan Gunthorpe (22):
  PCI/P2PDMA: Rename upstream_bridge_distance() and rework documentation
  PCI/P2PDMA: Use a buffer on the stack for collecting the acs list
  PCI/P2PDMA: Cleanup type for return value of calc_map_type_and_dist()
  PCI/P2PDMA: Avoid pci_get_slot() which sleeps
  PCI/P2PDMA: Print a warning if the host bridge is not in the whitelist
  PCI/P2PDMA: Attempt to set map_type if it has not been set
  PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagemap and device
  dma-mapping: Allow map_sg() ops to return negative error codes
  dma-direct: Return appropriate error code from dma_direct_map_sg()
  iommu: Return full error code from iommu_map_sg[_atomic]()
  dma-iommu: Return error code from iommu_dma_map_sg()
  lib/scatterlist: Add flag for indicating P2PDMA segments in an SGL
  PCI/P2PDMA: Make pci_p2pdma_map_type() non-static
  PCI/P2PDMA: Introduce helpers for dma_map_sg implementations
  dma-direct: Support PCI P2PDMA pages in dma-direct map_sg
  dma-mapping: Add flags to dma_map_ops to indicate PCI P2PDMA support
  iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg
  nvme-pci: Check DMA ops when indicating support for PCI P2PDMA
  nvme-pci: Convert to using dma_map_sgtable()
  RDMA/core: Introduce ib_dma_pci_p2p_dma_supported()
  RDMA/rw: use dma_map_sgtable()
  PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg()

 drivers/infiniband/core/rw.c |  75 ++++------
 drivers/iommu/dma-iommu.c    |  86 +++++++++--
 drivers/iommu/iommu.c        |  15 +-
 drivers/nvme/host/core.c     |   3 +-
 drivers/nvme/host/nvme.h     |   2 +-
 drivers/nvme/host/pci.c      |  80 +++++-----
 drivers/nvme/target/rdma.c   |   2 +-
 drivers/pci/Kconfig          |   2 +-
 drivers/pci/p2pdma.c         | 273 +++++++++++++++++++----------------
 include/linux/dma-map-ops.h  |  18 ++-
 include/linux/dma-mapping.h  |  46 +++++-
 include/linux/iommu.h        |  22 +--
 include/linux/pci-p2pdma.h   |  81 ++++++++---
 include/linux/scatterlist.h  |  50 ++++++-
 include/rdma/ib_verbs.h      |  30 ++++
 kernel/dma/direct.c          |  44 +++++-
 kernel/dma/mapping.c         |  31 +++-
 17 files changed, 570 insertions(+), 290 deletions(-)


base-commit: 6efb943b8616ec53a5e444193dccf1af9ad627b5
--
2.20.1