[PATCH v8 00/14] Update SMMUv3 to the modern iommu API (part 2b/3)

[Jerry Snitselaar]

On Mon, Jun 03, 2024 at 09:15:45PM GMT, Jason Gunthorpe wrote:
> Continuing the work of part 1 and 2a this part focuses on the PASID and
> SVA code bringing these functional improvements:
> 
>  - attach_dev failure does not change the HW configuration.
> 
>  - Full PASID API support including:
>     - S1/SVA domains attached to PASIDs
>     - IDENTITY/BLOCKED/S1 attached to RID
>     - Change of the RID domain while PASIDs are attached
> 
>  - Streamlined SVA support using the core infrastructure
> 
>  - Hitless, whenever possible, change between two domains
> 
> This requires some reorganizing how the invalidation is tracked, a
> flexible linked list containing the SSIDs as well as the ATC information
> is used instead of a single master.
> 
> The revised invalidation infrastructure is used to build generic support
> for attaching any sort of domain to any SSID, including the necessary
> maintenance of the invalidation list and the global ATS state. This
> mechansim is used again in part 3 as part of the nesting series.
> 
> The ATS ordering is generalized and consolidated so that the PASID flow
> can use it and put into a form where it is fully hitless, whenever
> possible. This is necessary as changes to single PASIDs or RIDs can not
> change global state like ATS without impacting other, still attached,
> PASIDs or RIDs.
> 
> Next simply replace the entire outdated SVA mmu_notifier implementation in
> one shot and switch it over to the newly created generic PASID layer and
> generic invalidation logic. This avoids the messy and confusing approach
> of trying to incrementally untangle this in place. The new code is small
> and simple enough this is much better than trying to figure out smaller
> steps.
> 
> Once SVA is resting on the consolidate PASID layer it is straightforward
> to make the PASID interface functionally complete by supporting S1DSS to
> allow concurrent change of the RID while a PASID is attached and allow
> supporting PASID over an IDENTIY RID.
> 
> This continues to support lazy allocation and installation of the CD table
> by continuing to use the CFG mode for IDENTITY/BLOCKED when when the PASID
> table is empty.
> 
> Finally enable attaching any S1 domain to a PASID, not just SVA.
> 
> It achieves the same goals as the several series from Michael and the
> S1DSS series from Nicolin that were trying to improve portions of the API.
> 
> This is on github:
> https://github.com/jgunthorpe/linux/commits/smmuv3_newapi
> 
> v8:
>  - Rebase on v6.10-rc2
>  - Make arm_smmu_sva_domain_alloc NULL when SVA is disabled so the core
>    code sees a NULL function pointer
>  - Update comments around arm_smmu_attach_prepare()
>  - Rename struct attach_state -> arm_smmu_attach_state and document
>    better, include more common function paramters in state
>  - Consistently use ats_enabled everywhere, replacing disable_ats in state
>  - Move the note about ATS bypass/abort to arm_smmu_attach_prepare()
>  - Remove temporary cd_table.in_ste check in arm_smmu_sva_set_dev_pasid()
>  - Improve comments and clarity of logic in arm_smmu_attach_commit()
>  - Shorten arm_smmu_mmu_notifier_free()
>  - Fix domain -> sid_domain typo in arm_smmu_remove_dev_pasid()
> v7: https://lore.kernel.org/r/0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com
>  - Second half of the split series
>  - Rebase on Joerg's latest
>  - Accommodate ARM_SMMU_FEAT_ATTR_TYPES_OVR for the S1DSS code
>  - Include the S1DSS kunit tests
>  - Include hunks to adjust the unit tests to API changes from this series
>  - Move 3 BTM related patches out of this series, they can go in the BTM
>    enablement series.
>  - Move the domain_alloc_sva() conversion to the first patch, and rebase
>    on the accepted core code change
>  - Use the new core APIs for the PASID ops
>  - Revise commit messages
> v6: https://lore.kernel.org/r/0-v6-228e7adf25eb+4155-smmuv3_newapi_p2_jgg@nvidia.com
> 
> Cc: Nicolin Chen <nicolinc at nvidia.com>
> Cc: Michael Shavit <mshavit at google.com>
> Cc: Jean-Philippe Brucker <jean-philippe at linaro.org>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi at huawei.com>
> Cc: Eric Auger <eric.auger at redhat.com>
> Cc: Moritz Fischer <mdf at kernel.org>
> Signed-off-by: Jason Gunthorpe <jgg at nvidia.com>
> 
> Jason Gunthorpe (14):
>   iommu/arm-smmu-v3: Convert to domain_alloc_sva()
>   iommu/arm-smmu-v3: Start building a generic PASID layer
>   iommu/arm-smmu-v3: Make smmu_domain->devices into an allocated list
>   iommu/arm-smmu-v3: Make changing domains be hitless for ATS
>   iommu/arm-smmu-v3: Add ssid to struct arm_smmu_master_domain
>   iommu/arm-smmu-v3: Do not use master->sva_enable to restrict attaches
>   iommu/arm-smmu-v3: Thread SSID through the arm_smmu_attach_*()
>     interface
>   iommu/arm-smmu-v3: Make SVA allocate a normal arm_smmu_domain
>   iommu/arm-smmu-v3: Keep track of arm_smmu_master_domain for SVA
>   iommu/arm-smmu-v3: Put the SVA mmu notifier in the smmu_domain
>   iommu/arm-smmu-v3: Allow IDENTITY/BLOCKED to be set while PASID is
>     used
>   iommu/arm-smmu-v3: Test the STE S1DSS functionality
>   iommu/arm-smmu-v3: Allow a PASID to be set when RID is
>     IDENTITY/BLOCKED
>   iommu/arm-smmu-v3: Allow setting a S1 domain to a PASID
> 
>  .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   | 431 +++-----------
>  .../iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c  | 116 +++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 561 ++++++++++++++----
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |  52 +-
>  4 files changed, 664 insertions(+), 496 deletions(-)
> 
> 
> base-commit: c3f38fa61af77b49866b006939479069cd451173
> -- 
> 2.45.2
> 

Reviewed-by: Jerry Snitselaar <jsnitsel at redhat.com>