RPi4 can't deal with 64 bit PCI accesses

[Nicolas Saenz Julienne]

On Mon, 2021-02-22 at 17:18 +0100, Ard Biesheuvel wrote:
> On Mon, 22 Feb 2021 at 16:48, Nicolas Saenz Julienne
> <nsaenzjulienne at suse.de> wrote:
> > 
> > Hi everyone,
> > Raspberry Pi 4, a 64bit arm system on chip, contains a PCIe bus that can't
> > handle 64bit accesses to its MMIO address space, in other words, writeq() has
> > to be split into two distinct writel() operations. This isn't ideal, as it
> > misrepresents PCI's promise of being able to treat device memory as regular
> > memory, ultimately breaking a bunch of PCI device drivers[1].
> > 
> > I'd like to have a go at fixing this in a way that can be distributed in a
> > generic distro without prejudice to other users.
> > 
> > AFAIK there is no way to detect this limitation through generic PCIe
> > capabilities, so one solution would be to expose it through firmware
> > (devicetree in this case), and pass the limitations through 'struct device' so
> > as for the drivers to choose the right access method in a way that doesn't
> > affect performance much[2]. All in all, most of this doesn't need to be
> > PCI-centric as the property could be applied to any MMIO bus.
> > 
> > Thoughts? Opinions? Is it overkill just for a single SoC?
> > 
> 
> Hi Nicolas,
> 
> How does this issue manifest itself? There are other PCIe RC

Only the low bits would get written/read, as for the high bits I can't recall
if they were corrupted or simply ignored (I experienced this some time ago
while bringing up RPi's PCIe in u-boot).

> implementations suffering from the same issue, and some of the drivers
> in Linux already work around this, by using split accesses. Look at
> this one, for instance:
> 
> a310acd7a7ea ("NVMe: use split lo_hi_{read,write}q")
> 
> which switches NVMe to lo_hi_readq, which appears to be used in quite
> a few other places as well.

Indeed, XHCI does this unanimously too. But I figured forcing the split on all
drivers woudln't be a very popular solution just for RPi's 'faulty' bus. But if
it turns out to be a common problem, I guess it isn't such a bad idea.

Regards,
Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20210222/df4d027d/attachment.sig>