[PATCH v4 01/12] PCI: rockchip-ep: Fix address translation unit programming

[Rick Wertenbroek]

On Fri, Oct 11, 2024 at 2:14 PM Damien Le Moal <dlemoal at kernel.org> wrote:
>
> The rockchip PCIe endpoint controller handles PCIe transfers addresses
> by masking the lower bits of the programmed PCI address and using the
> same number of lower bits masked from the CPU address space used for the
> mapping. For a PCI mapping of <size> bytes starting from <pci_addr>,
> the number of bits masked is the number of address bits changing in the
> address range [pci_addr..pci_addr + size - 1].
>
> However, rockchip_pcie_prog_ep_ob_atu() calculates num_pass_bits only
> using the size of the mapping, resulting in an incorrect number of mask
> bits depending on the value of the PCI address to map.
>
> Fix this by introducing the helper function
> rockchip_pcie_ep_ob_atu_num_bits() to correctly calculate the number of
> mask bits to use to program the address translation unit. The number of
> mask bits is calculated depending on both the PCI address and size of
> the mapping, and clamped between 8 and 20 using the macros
> ROCKCHIP_PCIE_AT_MIN_NUM_BITS and ROCKCHIP_PCIE_AT_MAX_NUM_BITS. As
> defined in the Rockchip RK3399 TRM V1.3 Part2, Sections 17.5.5.1.1 and
> 17.6.8.2.1, this clamping is necessary because:
> 1) The lower 8 bits of the PCI address to be mapped by the outbound
>    region are ignored. So a minimum of 8 address bits are needed and
>    imply that the PCI address must be aligned to 256.
> 2) The outbound memory regions are 1MB in size. So while we can specify
>    up to 63-bits for the PCI address (num_bits filed uses bits 0 to 5 of
>    the outbound address region 0 register), we must limit the number of
>    valid address bits to 20 to match the memory window maximum size (1
>    << 20 = 1MB).

Hello Damien,
I just found out the cadence controller
(drivers/pci/controller/cadence/pcie-cadence.c) suffers from the exact
same num_pass_bits calculation issue. The code in
cdns_pcie_set_outbound_region() is very similar to
rockchip_pcie_prog_ep_ob_atu().

I found out by running the NVMe endpoint function on a Texas
Instruments ARM67A SoC which relies on the pci-j721e cadence PCI
driver. I observed the same issues we had with the RK3399, when I
patched it with the same computation for the num pass bits as we did
for the RK3399, it would work.

So this issue also exists for all cadence based drivers. It's too bad
I don't have access to any technical doc ref to back this up.

Just wanted to let you know.
Best regards,
Rick