[PATCH v5 00/27] Update SMMUv3 to the modern iommu API (part 2/3)

[Mostafa Saleh]

Hi Shameer,

On Mon, Mar 25, 2024 at 10:44 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi at huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Mostafa Saleh <smostafa at google.com>
> > Sent: Monday, March 25, 2024 10:22 AM
> > To: Jason Gunthorpe <jgg at nvidia.com>
> > Cc: iommu at lists.linux.dev; Joerg Roedel <joro at 8bytes.org>; linux-arm-
> > kernel at lists.infradead.org; Robin Murphy <robin.murphy at arm.com>; Will
> > Deacon <will at kernel.org>; Eric Auger <eric.auger at redhat.com>; Jean-
> > Philippe Brucker <jean-philippe at linaro.org>; Moritz Fischer
> > <mdf at kernel.org>; Michael Shavit <mshavit at google.com>; Nicolin Chen
> > <nicolinc at nvidia.com>; patches at lists.linux.dev; Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi at huawei.com>
> > Subject: Re: [PATCH v5 00/27] Update SMMUv3 to the modern iommu API
> > (part 2/3)
> >
> > Hi Jason,
> >
> > On Mon, Mar 04, 2024 at 07:43:48PM -0400, Jason Gunthorpe wrote:
> > > Continuing the work of part 1 this focuses on the CD, PASID and SVA
> > > components:
> > >
> > >  - attach_dev failure does not change the HW configuration.
> > >
> > >  - Full PASID API support including:
> > >     - S1/SVA domains attached to PASIDs
> > >     - IDENTITY/BLOCKED/S1 attached to RID
> > >     - Change of the RID domain while PASIDs are attached
> > >
> > >  - Streamlined SVA support using the core infrastructure
> > >
> > >  - Hitless, whenever possible, change between two domains
> > >
> > > Making the CD programming work like the new STE programming allows
> > > untangling some of the confusing SVA flows. From there the focus is on
> > > building out the core infrastructure for dealing with PASID and CD
> > > entries, then keeping track of unique SSID's for ATS invalidation.
> > >
> > > The ATS ordering is generalized so that the PASID flow can use it and put
> > > into a form where it is fully hitless, whenever possible. Care is taken to
> > > ensure that ATC flushes are present after any change in translation.
> > >
> > > Finally we simply kill the entire outdated SVA mmu_notifier
> > implementation
> > > in one shot and switch it over to the newly created generic PASID & CD
> > > code. This avoids the messy and confusing approach of trying to
> > > incrementally untangle this in place. The new code is small and simple
> > > enough this is much better than trying to figure out smaller steps.
> > >
> > > Once SVA is resting on the right CD code it is straightforward to make the
> > > PASID interface functionally complete.
> > >
> > > It achieves the same goals as the several series from Michael and the S1DSS
> > > series from Nicolin that were trying to improve portions of the API.
> > >
> > > This is on github:
> > > https://github.com/jgunthorpe/linux/commits/smmuv3_newapi
> >
> > Testing on qemu[1], with the same VMM Shameer tested with[2]:
> > qemu/build/qemu-system-aarch64 -M virt -machine virt,gic-
> > version=3,iommu=nested-smmuv3,iommufd=iommufd0 \
> > -cpu cortex-a53,pmu=off -smp 1 -m 2048 \
> > -kernel Image \
> > -drive file=rootfs.ext4,if=virtio,format=raw  \
> > -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-
> > pci,rng=rng0 -nographic  \
> > -append 'console=ttyAMA0 rootwait root=/dev/vda' \
> > -device virtio-scsi-pci,id=scsi0  \
> > -device ioh3420,id=pcie.1,chassis=1 \
> > -object iommufd,id=iommufd0 \
> > -device vfio-pci,host=0000:00:03.0,iommufd=iommufd0
> >
> > I see the following panic:
>
> I think that is probably because you are testing with "nested-smmuv3". This
> series not yet fully enable that. For that, I think you are missing few patches
> from Nicolin's iommufd branch,
> https://github.com/nicolinc/iommufd/commits/wip/iommufd_nesting-03112024/

I see, thanks for clarifying. I think we shouldn't still crash the
kernel, but that's a problem for part 3.

Thanks,
Mostafa