[PATCH] arm64: Enable PCI write-combine resources under sysfs
lorenzo.pieralisi at arm.com
Tue Sep 15 06:18:31 EDT 2020
On Tue, Sep 15, 2020 at 09:25:57AM +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2020-09-14 at 19:57 -0300, Jason Gunthorpe wrote:
> > On Tue, Sep 15, 2020 at 08:00:27AM +1000, Benjamin Herrenschmidt wrote:
> > > On Tue, 2020-09-15 at 07:42 +1000, Benjamin Herrenschmidt wrote:
> > > >
> > > > > which is back to my original question, how do you do DMA using
> > > > > /sys/xx/resources? Why not use VFIO like everything else?
> > > >
> > > > Note: All this doesnt' change the fact that sys/xx/resources_wc
> > > > exists
> > > > for other archs and I see no reasons so far not to have it on ARM...
> > >
> > > Also... it looks like VFIO also doesn't provide a way to do WC yet
> > > unfortunately :-(
> > Yes, but if the driving reason for this patch is because a VFIO user
> > like EFA DPDK is trying to work around VFIO limitations, then I'd say
> > the VFIO mmap should be amended, and not so much worring about sysfs.
> I don't think the two are exclusive.
> > While there is no reason for ARM to not show the sysfs, it really
> > should never be used. Modern kernels in secure boot don't even show
> > it, for instance.
> It's useful for random things, I've used it quite a bit in a previous
> life for things like in-lab hw testing etc... There's tooling out
> there, esp. in the more 'embedded' side of thing that uses this, I
> don't see a good reason not to provide the same level of functionality.
> So Lorenzo, imho, we should merge the patch.
To sum it up:
(1) RDMA drivers need a new mapping function/attribute to define their
message push model. Actually the message model is not necessarily related
to write combining a la x86, so we should probably come up with a better
and consistent naming. Enabling this patchset may trigger performance
regressions on mellanox drivers on arm64 - this ought to be addressed.
(2) User-space/passthrough drivers rely on VFIO for BAR mappings. Since
it looks relevant, WC message model semantics should be introduced
(3) Given the above, the sysfs interface can be enabled. I still don't
see the advantages (other than saying it is there for other arches, ie
what can you really do with the sysfs mappings - see Jason's VFIO/DMA
query) but we can do it. Still, I am not happy about the possible
mellanox regressions - that should be tested/verified before this
patch is merged IMHO. Do we really need it for v5.10 ?
> As for fixing VFIO, definitely something to revive. The main contention
> point was which "interface" to use to request write combine.
> Let's restart that conversation with the appropriate folks, the last I
> remember, the question was to figure out what interface to provide
> userspace for the functionality.
> Clint, do you want to drive this as well ?
More information about the linux-arm-kernel