[PATCH v5 00/27] Update SMMUv3 to the modern iommu API (part 2/3)

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Mostafa Saleh <smostafa at google.com>
> Sent: Monday, March 25, 2024 10:22 AM
> To: Jason Gunthorpe <jgg at nvidia.com>
> Cc: iommu at lists.linux.dev; Joerg Roedel <joro at 8bytes.org>; linux-arm-
> kernel at lists.infradead.org; Robin Murphy <robin.murphy at arm.com>; Will
> Deacon <will at kernel.org>; Eric Auger <eric.auger at redhat.com>; Jean-
> Philippe Brucker <jean-philippe at linaro.org>; Moritz Fischer
> <mdf at kernel.org>; Michael Shavit <mshavit at google.com>; Nicolin Chen
> <nicolinc at nvidia.com>; patches at lists.linux.dev; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi at huawei.com>
> Subject: Re: [PATCH v5 00/27] Update SMMUv3 to the modern iommu API
> (part 2/3)
> 
> Hi Jason,
> 
> On Mon, Mar 04, 2024 at 07:43:48PM -0400, Jason Gunthorpe wrote:
> > Continuing the work of part 1 this focuses on the CD, PASID and SVA
> > components:
> >
> >  - attach_dev failure does not change the HW configuration.
> >
> >  - Full PASID API support including:
> >     - S1/SVA domains attached to PASIDs
> >     - IDENTITY/BLOCKED/S1 attached to RID
> >     - Change of the RID domain while PASIDs are attached
> >
> >  - Streamlined SVA support using the core infrastructure
> >
> >  - Hitless, whenever possible, change between two domains
> >
> > Making the CD programming work like the new STE programming allows
> > untangling some of the confusing SVA flows. From there the focus is on
> > building out the core infrastructure for dealing with PASID and CD
> > entries, then keeping track of unique SSID's for ATS invalidation.
> >
> > The ATS ordering is generalized so that the PASID flow can use it and put
> > into a form where it is fully hitless, whenever possible. Care is taken to
> > ensure that ATC flushes are present after any change in translation.
> >
> > Finally we simply kill the entire outdated SVA mmu_notifier
> implementation
> > in one shot and switch it over to the newly created generic PASID & CD
> > code. This avoids the messy and confusing approach of trying to
> > incrementally untangle this in place. The new code is small and simple
> > enough this is much better than trying to figure out smaller steps.
> >
> > Once SVA is resting on the right CD code it is straightforward to make the
> > PASID interface functionally complete.
> >
> > It achieves the same goals as the several series from Michael and the S1DSS
> > series from Nicolin that were trying to improve portions of the API.
> >
> > This is on github:
> > https://github.com/jgunthorpe/linux/commits/smmuv3_newapi
> 
> Testing on qemu[1], with the same VMM Shameer tested with[2]:
> qemu/build/qemu-system-aarch64 -M virt -machine virt,gic-
> version=3,iommu=nested-smmuv3,iommufd=iommufd0 \
> -cpu cortex-a53,pmu=off -smp 1 -m 2048 \
> -kernel Image \
> -drive file=rootfs.ext4,if=virtio,format=raw  \
> -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-
> pci,rng=rng0 -nographic  \
> -append 'console=ttyAMA0 rootwait root=/dev/vda' \
> -device virtio-scsi-pci,id=scsi0  \
> -device ioh3420,id=pcie.1,chassis=1 \
> -object iommufd,id=iommufd0 \
> -device vfio-pci,host=0000:00:03.0,iommufd=iommufd0
> 
> I see the following panic:

I think that is probably because you are testing with "nested-smmuv3". This
series not yet fully enable that. For that, I think you are missing few patches
from Nicolin's iommufd branch,
https://github.com/nicolinc/iommufd/commits/wip/iommufd_nesting-03112024/

Thanks,
Shameer