[Question] rk3399 vfio-pci/sr-iov support

[Robin Murphy]

On 2021-11-15 19:50, Adam Joseph wrote:
> On Mon, 9 Dec 2019 14:07:02 +0000
> Robin Murphy <robin.murphy at arm.com> wrote:
>> On 09/12/2019 1:28 pm, Peter Geis wrote:
>>> I'm back with more pcie fun on the rk3399.
>>> I'm trying to get pcie passthrough working for a vm on the rk3399,
>>> and have encountered some roadblocks.
>>
>> That much I can help with somewhat: the major impediment is that
>> RK3399 doesn't have an IOMMU in front of PCIe.
> 
> For the more limited case of defending against attacks from
> hostile/buggy firmware on PCIe devices: is it possible to use the
> RK3399 PCIe "inbound address translation" support instead of an IOMMU?

I guess it might be possible to reserve some memory with the new 
"restricted-dma-pool" semantics then tweak the PCIe setup to match that. 
It will mostly depend on what the RC does with transactions that don't 
match any ATU entry. These kinds of registers typically only decode a 
small number of naturally-aligned power-of-two-sized address ranges so 
would not be able to support the kind of arbitrary translation/page 
protection you'd need for something like VFIO.

> The RK3399 TRM, v1.3 "Part 2", section 17.5.5.2.1 explains how to
> configure address translation (including a base/bounds check) for
> inbound-to-SoC memory writes, but details are quite sparse.
> 
> Linux appears to not use this functionality; from
> drivers/pci/controller/pcie-rockchip-host.c we can see that it
> disables the base/bounds (sets them to the entire 32-bit space, which
> is all RAM on RK3399 since it supports only 4GB) and passes all 32
> bits of incoming memory writes:
> 
> static int rockchip_pcie_prog_ib_atu(struct rockchip_pcie *rockchip,
>                                       int region_no, u8 num_pass_bits,
> 	                             u32 lower_addr, u32 upper_addr)
> ...
> 
> static int rockchip_pcie_cfg_atu(struct rockchip_pcie *rockchip)
> {
> ...
> 	err = rockchip_pcie_prog_ib_atu(rockchip, 2, 32 - 1, 0x0, 0);
> 
> Is this a dead end?  If not I might pursue it, if I can get the
> necessary documentation.  I couldn't find any mention of ATS in the
> RK3399 manual; if the PCIe RC allows that then all bets are off anyways.

Knowing this RC, if it ever did see an ATS translated transaction header 
it would probably SError the system, like it does for such outlandish 
things as trying to probe config space behind bridges :)

Robin.