SR-IOV on ARM64 system with SMMU

[Martin Bayern]

On 22.02.23 1:38 PM, Robin Murphy wrote:
>> hi Robin
>>
>> Thank you so much for your support on this!
>>
>> On 21.02.23 8:30 PM, Robin Murphy wrote:
>>> The only actual solution to that problem is to make the platform 
>>> device not be in the group to begin with. As mentioned, that is 
>>> presumably because the PCIe node in the DT has an "iommus = ..." 
>>> property (not to be confused with the "iommu-map" properties which 
>>> are for the PCI side). I'd expect you could probably get away with 
>>> removing that - the only reason I can imagine it being functionally 
>>> necessary is if the root complex has internal DMA engines that the 
>>> downstream kernel is using.
>>>
>>> But yeah, by that point it's arguably just an exercise in achieving 
>>> satisfaction by making *something* work, rather than a really 
>>> practical prospect
>>
>> I search the keyword string "iommus" in NVIDIA Orin/Xavier related 
>> dts/dtsi, but there is no iommus attribute node, nor iommu-map. 
>> Relevant questions have been posted in the NVIDIA forum, and I hope 
>> NVIDIA can respond. Please let me know if you have any other 
>> thoughts/suggestions. I'm currently working on study of the drivers 
>> of smmu and iommu, trying to figure out the root cause of it.
>
> That sounds like the properties are getting added dynamically by the 
> bootloader (or possibly an overlay), or it might be using a DTB from 
> some other source entirely - I'd check under 
> /sys/firmware/devicetree/base/ at runtime and/or decompile 
> /sys/firmware/fdt to see what ends up actually passed to the kernel.
>
> Unless the downstream kernel has made major changes to the IOMMU 
> logic, it's the only way to explain why the SMMU driver would be aware 
> of the PCI controller platform device at all. 


Hi Robin,

Thanks again, your interest in this topic is greatly appreciated! I 
hope, I could implement SR-IOV on this ARM based system with your support.

Might be an overlay, I'll double-check and get back to you asap.


Kind regards,

Martin