RPi4 can't deal with 64 bit PCI accesses

[Ard Biesheuvel]

On Mon, 22 Feb 2021 at 16:48, Nicolas Saenz Julienne
<nsaenzjulienne at suse.de> wrote:
>
> Hi everyone,
> Raspberry Pi 4, a 64bit arm system on chip, contains a PCIe bus that can't
> handle 64bit accesses to its MMIO address space, in other words, writeq() has
> to be split into two distinct writel() operations. This isn't ideal, as it
> misrepresents PCI's promise of being able to treat device memory as regular
> memory, ultimately breaking a bunch of PCI device drivers[1].
>
> I'd like to have a go at fixing this in a way that can be distributed in a
> generic distro without prejudice to other users.
>
> AFAIK there is no way to detect this limitation through generic PCIe
> capabilities, so one solution would be to expose it through firmware
> (devicetree in this case), and pass the limitations through 'struct device' so
> as for the drivers to choose the right access method in a way that doesn't
> affect performance much[2]. All in all, most of this doesn't need to be
> PCI-centric as the property could be applied to any MMIO bus.
>
> Thoughts? Opinions? Is it overkill just for a single SoC?
>

Hi Nicolas,

How does this issue manifest itself? There are other PCIe RC
implementations suffering from the same issue, and some of the drivers
in Linux already work around this, by using split accesses. Look at
this one, for instance:

a310acd7a7ea ("NVMe: use split lo_hi_{read,write}q")

which switches NVMe to lo_hi_readq, which appears to be used in quite
a few other places as well.

-- 
Ard.