[RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent

[Robin Murphy]

On 2022-09-08 10:51, Radovanovic, Aleksandar wrote:
> [AMD Official Use Only - General]
> 
> 
> 
>> -----Original Message-----
>> From: Marc Zyngier <maz at kernel.org>
>> Sent: 08 September 2022 09:08
>> To: Radovanovic, Aleksandar <aleksandar.radovanovic at amd.com>
>> Cc: Jason Gunthorpe <jgg at nvidia.com>; Gupta, Nipun
>> <Nipun.Gupta at amd.com>; robh+dt at kernel.org;
>> krzysztof.kozlowski+dt at linaro.org; gregkh at linuxfoundation.org;
>> rafael at kernel.org; eric.auger at redhat.com; alex.williamson at redhat.com;
>> cohuck at redhat.com; Gupta, Puneet (DCG-ENG)
>> <puneet.gupta at amd.com>; song.bao.hua at hisilicon.com;
>> mchehab+huawei at kernel.org; f.fainelli at gmail.com;
>> jeffrey.l.hugo at gmail.com; saravanak at google.com;
>> Michael.Srba at seznam.cz; mani at kernel.org; yishaih at nvidia.com;
>> robin.murphy at arm.com; will at kernel.org; joro at 8bytes.org;
>> masahiroy at kernel.org; ndesaulniers at google.com; linux-arm-
>> kernel at lists.infradead.org; linux-kbuild at vger.kernel.org; linux-
>> kernel at vger.kernel.org; devicetree at vger.kernel.org; kvm at vger.kernel.org;
>> okaya at kernel.org; Anand, Harpreet <harpreet.anand at amd.com>; Agarwal,
>> Nikhil <nikhil.agarwal at amd.com>; Simek, Michal <michal.simek at amd.com>;
>> git (AMD-Xilinx) <git at amd.com>
>> Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its
>> domain as parent
>>
>> [CAUTION: External Email]
>>   
>> OK, so you definitely need a mapping, but it cannot be a translation, and it
>> needs to be in all the possible address spaces. OMG.
> 
> Could you elaborate why it needs to be in all the possible address spaces? I'm in no way familiar with kernel IOVA allocation, so not sure I understand this requirement. Note that each CDX device will have its own unique StreamID (in general case, equal to DeviceID sent to the GIC), so, from a SMMU perspective, the mapping can be specific to that device. As long as that IOVA is not allocated to any DMA region for _that_ device, things should be OK? But, I appreciate it might not be that simple from a kernel perspective.

That's the point - any device could could have its own mapping, 
therefore that hole has to be punched in *every* mapping that any of 
those devices could use, so that MSI writes don't unexpectedly fault, or 
corrupt memory if that address is free to be used to map a DMA buffer. 
At least the HiSilicon PCI quirk is functionally similar (for slightly 
different underlying reasons) so there's already precedent and an 
example that you can follow to a reasonable degree.

Robin.