[RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent

[Radovanovic, Aleksandar]

[AMD Official Use Only - General]



> -----Original Message-----
> From: Jason Gunthorpe <jgg at ziepe.ca>
> Sent: 12 October 2022 14:02
> To: Radovanovic, Aleksandar <aleksandar.radovanovic at amd.com>
> Cc: Gupta, Nipun <Nipun.Gupta at amd.com>; Marc Zyngier
> <maz at kernel.org>; Robin Murphy <robin.murphy at arm.com>;
> robh+dt at kernel.org; krzysztof.kozlowski+dt at linaro.org;
> gregkh at linuxfoundation.org; rafael at kernel.org; eric.auger at redhat.com;
> alex.williamson at redhat.com; cohuck at redhat.com; Gupta, Puneet (DCG-
> ENG) <puneet.gupta at amd.com>; song.bao.hua at hisilicon.com;
> mchehab+huawei at kernel.org; f.fainelli at gmail.com;
> jeffrey.l.hugo at gmail.com; saravanak at google.com;
> Michael.Srba at seznam.cz; mani at kernel.org; yishaih at nvidia.com;
> will at kernel.org; joro at 8bytes.org; masahiroy at kernel.org;
> ndesaulniers at google.com; linux-arm-kernel at lists.infradead.org; linux-
> kbuild at vger.kernel.org; linux-kernel at vger.kernel.org;
> devicetree at vger.kernel.org; kvm at vger.kernel.org; okaya at kernel.org;
> Anand, Harpreet <harpreet.anand at amd.com>; Agarwal, Nikhil
> <nikhil.agarwal at amd.com>; Simek, Michal <michal.simek at amd.com>; git
> (AMD-Xilinx) <git at amd.com>
> Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its
> domain as parent
> 
> Caution: This message originated from an External Source. Use proper
> caution when opening attachments, clicking links, or responding.
> 
> 
> On Wed, Oct 12, 2022 at 10:34:23AM +0000, Radovanovic, Aleksandar wrote:
> 
> 
> > As for GITS_TRANSLATER, we can take up to 4 different IOVAs, which
> > limits us to 4 CDX devices (should be sufficient for current HW
> > use-cases). Also, it means that the address part must be the same for
> > all vectors within a single CDX device. I'm assuming this is OK as it
> > is going to be a single interrupt and IOMMU domain anyway.
> 
> This is not at all how MSI is supposed to work.

In the general case, no, they're not. However, this is an embedded device
with a GICv3, so the general case does not really apply. On GIC, the MSI 
target address is always a fixed register in the ITS (GIC_TRANSLATER), 
possibly SMMU translated. As long as the translation is consistent across
a single device (and I see no reason why or how the kernel would do it 
otherwise, given that a single CDX device generates the same StreamID
for all MSI writes), the GIC IOVA should be the same for all vectors of 
the device.

It is worth noting that this limitation is not going to be baked in the 
proposed MSI configuration interface, it will still take both the address
and data parts for each vector. It is just that this particular 
implementation will throw an error if you supply a different target
address across device MSI vectors. It does not preclude some future
device accepting different addresses per vector over the same 
interface.

Thanks,
Aleksandar