SR-IOV on ARM64 system with SMMU

[Robin Murphy]

On 2023-02-22 11:53, Martin Bayern wrote:
> hi Robin
> 
> Thank you so much for your support on this!
> 
> On 21.02.23 8:30 PM, Robin Murphy wrote:
>> The only actual solution to that problem is to make the platform 
>> device not be in the group to begin with. As mentioned, that is 
>> presumably because the PCIe node in the DT has an "iommus = ..." 
>> property (not to be confused with the "iommu-map" properties which are 
>> for the PCI side). I'd expect you could probably get away with 
>> removing that - the only reason I can imagine it being functionally 
>> necessary is if the root complex has internal DMA engines that the 
>> downstream kernel is using.
>>
>> But yeah, by that point it's arguably just an exercise in achieving 
>> satisfaction by making *something* work, rather than a really 
>> practical prospect
> 
> I search the keyword string "iommus" in NVIDIA Orin/Xavier related 
> dts/dtsi, but there is no iommus attribute node, nor iommu-map. Relevant 
> questions have been posted in the NVIDIA forum, and I hope NVIDIA can 
> respond. Please let me know if you have any other thoughts/suggestions. 
> I'm currently working on study of the drivers of smmu and iommu, trying 
> to figure out the root cause of it.

That sounds like the properties are getting added dynamically by the 
bootloader (or possibly an overlay), or it might be using a DTB from 
some other source entirely - I'd check under 
/sys/firmware/devicetree/base/ at runtime and/or decompile 
/sys/firmware/fdt to see what ends up actually passed to the kernel.

Unless the downstream kernel has made major changes to the IOMMU logic, 
it's the only way to explain why the SMMU driver would be aware of the 
PCI controller platform device at all.

Cheers,
Robin.