RPi4 can't deal with 64 bit PCI accesses

[Robin Murphy]

On 2021-02-22 15:47, Nicolas Saenz Julienne wrote:
> Hi everyone,
> Raspberry Pi 4, a 64bit arm system on chip, contains a PCIe bus that can't
> handle 64bit accesses to its MMIO address space, in other words, writeq() has
> to be split into two distinct writel() operations. This isn't ideal, as it
> misrepresents PCI's promise of being able to treat device memory as regular
> memory, ultimately breaking a bunch of PCI device drivers[1].
> 
> I'd like to have a go at fixing this in a way that can be distributed in a
> generic distro without prejudice to other users.
> 
> AFAIK there is no way to detect this limitation through generic PCIe
> capabilities, so one solution would be to expose it through firmware
> (devicetree in this case), and pass the limitations through 'struct device' so
> as for the drivers to choose the right access method in a way that doesn't
> affect performance much[2]. All in all, most of this doesn't need to be
> PCI-centric as the property could be applied to any MMIO bus.

It is indeed something that people can get wrong with internal buses as 
well - for example commit f2d9848aeb9f is such a workaround, also 
conveniently illustrating the case of significant functionality having 
to be disabled where the device *does* require 64-bit atomicity for 
correctness.

Working around kernel I/O accessors is all very well, but another 
concern for PCI in particular is when things like framebuffer memory can 
get mmap'ed into userspace (or even memremap'ed within the kernel). Even 
in AArch32, compiled code may result in 64-bit accesses being generated 
depending on how the CPU and interconnect handle LDRD/STRD/LDM/STM/etc., 
so it's basically not safe to ever let that happen at all.

Robin.

> 
> Thoughts? Opinions? Is it overkill just for a single SoC?
> 
> Regards,
> Nicolas
> 
> [1] https://github.com/raspberrypi/linux/issues/4158#issuecomment-782351510
> [2] Things might get even weirder as the order in which the 32bit operations
>      are performed might matter (low/high vs high/low).
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>