[PATCH] arm64: Enable PCI write-combine resources under sysfs

Jason Gunthorpe jgg at nvidia.com
Fri Sep 11 10:21:33 EDT 2020

On Fri, Sep 11, 2020 at 10:39:16AM +1000, Benjamin Herrenschmidt wrote:

> Yes, "write combine" isn't a good name.... The goal is to get WC but it
> comes with the whole package on several archs. We don't even have a
> reasonnable definition of the semantics of readl/writel on a WC mapping
> (hint: on powerpc the barriers in them will prevent WC even on a WC
> mapping) nor of what barriers might work  and how on such a mapping.

Yes, you can't really use WC properly in the kernel, we don't have the
infrastructure for it. mlx5 is using __raw_writeq() and wmb() to hack
something ugly together in the kernel.

A useful API for the message method, similar to what we use in
userspace, is something like:

   * Almost always need a spinlock as multiple CPUs cannot write
   * concurrently.

   * Ensure that all DMA visiable writes in program order are visible
   * to DMA before the WC TLP is sent.

  /* Generate the TLP */
  write_wc_message(wc_iomem, message, len);

   * Writes to wc_iomem past this, by any CPU, cannot replace writes
   * already done in wc_message.


And another varient without the spinlock for stuff that can be per-CPU
for a range of WC memory.

(oh actually I see most drivers are using ioremap_wc(), and there is a
bunch of them including an Amazon ethernet device...)


More information about the linux-arm-kernel mailing list