[PATCH v2 06/19] iommufd/viommu: Add IOMMU_VIOMMU_SET/UNSET_VDEV_ID ioctl

[Mostafa Saleh]

On Fri, Sep 27, 2024 at 3:58 PM Jason Gunthorpe <jgg at nvidia.com> wrote:
>
> On Fri, Sep 27, 2024 at 02:22:20PM +0000, Mostafa Saleh wrote:
>
> > > We don't support multi SID through this interface, at least in this
> > > version.
> > >
> > > To do multi-sid you have to inform the VM of all the different pSIDs
> > > the device has and then setup the vSID/pSID translation to map them
> > > all to the HW invalidation logic.
> >
> > Why would the VM need to know the pSID?
>
> It doesn't need to know the pSID exactly, but it needs to know all the
> pSIDs that exist and have them be labeled with vSIDs.
>
> With cmdq direct assignment the VM has to issue an ATS invalidation
> for each and every physical device using its vSID. There is no HW path
> to handle a 1:N v/p SID relationship.
>

I see, that's for the cmdq assignment.

> > Ah, I thought IOMMUFD would be used instead of VFIO_TYPE1*, which should cover
> > platform devices (VFIO-platform) or am I missing something?
>
> It does work with platform, but AFAIK nobody is interested in that so
> it hasn't been any focus. Enabling multi-sid nesting, sub stream ids,
> etc is some additional work I think.
>
> But what I mean is the really hard use case for the vSID/pSID mapping
> is ATS invalidation and you won't use ATS invalidation on platform so
> multi-sid likely doesn't matter.
>
> STE/CD invalidation could possibly be pushed down through the
> per-domain ioctl and replicated to all domain attachments. We don't
> have code in the series to do that, but it could work from a uAPI
> perspective.
>
> > If possible, can the UAPI be designed with this in mind, even if not
> > implemented now?
>
> It is reasonable to ask. I think things are extensible enough. I'd
> imagine we can add a flag 'secondary ID' and then a new field
> 'secondary ID index' to the vdev operations when someone wants to take
> this on.
>

Makes sense, I can take this when I start doing the pKVM work with
IOMMUFD, in case it wasn't supported by then.

Thanks,
Mostafa

> Jason