[PATCH] arm64: Enable PCI write-combine resources under sysfs
Jason Gunthorpe
jgg at nvidia.com
Fri Sep 11 10:21:33 EDT 2020
On Fri, Sep 11, 2020 at 10:39:16AM +1000, Benjamin Herrenschmidt wrote:
> Yes, "write combine" isn't a good name.... The goal is to get WC but it
> comes with the whole package on several archs. We don't even have a
> reasonnable definition of the semantics of readl/writel on a WC mapping
> (hint: on powerpc the barriers in them will prevent WC even on a WC
> mapping) nor of what barriers might work and how on such a mapping.
Yes, you can't really use WC properly in the kernel, we don't have the
infrastructure for it. mlx5 is using __raw_writeq() and wmb() to hack
something ugly together in the kernel.
A useful API for the message method, similar to what we use in
userspace, is something like:
/*
* Almost always need a spinlock as multiple CPUs cannot write
* concurrently.
*/
spin_lock();
/*
* Ensure that all DMA visiable writes in program order are visible
* to DMA before the WC TLP is sent.
*/
barrier_wc_after_lock();
/* Generate the TLP */
write_wc_message(wc_iomem, message, len);
/*
* Writes to wc_iomem past this, by any CPU, cannot replace writes
* already done in wc_message.
*/
barrier_wc_before_unlock();
spin_unlock();
And another varient without the spinlock for stuff that can be per-CPU
for a range of WC memory.
(oh actually I see most drivers are using ioremap_wc(), and there is a
bunch of them including an Amazon ethernet device...)
Jason
More information about the linux-arm-kernel
mailing list