[PATCH v2 21/27] iommu/arm-smmu-v3: Put the SVA mmu notifier in the smmu_domain

[Jason Gunthorpe]

On Tue, Nov 07, 2023 at 09:28:08PM +0800, Michael Shavit wrote:
> On Thu, Nov 2, 2023 at 7:37 AM Jason Gunthorpe <jgg at nvidia.com> wrote:
> > [...]
> > @@ -309,24 +169,26 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm)
> >                 struct arm_smmu_cd target;
> >                 struct arm_smmu_cd *cdptr;
> >
> > -               cdptr = arm_smmu_get_cd_ptr(master, mm->pasid);
> > +               cdptr = arm_smmu_get_cd_ptr(master, master_domain->ssid);
> >                 if (WARN_ON(!cdptr))
> >                         continue;
> > -               arm_smmu_make_sva_cd(&target, master, NULL, smmu_mn->cd->asid);
> > -               arm_smmu_write_cd_entry(master, mm->pasid, cdptr, &target);
> > +               arm_smmu_make_sva_cd(&target, master, NULL,
> > +                                    smmu_domain->cd.asid,
> > +                                    smmu_domain->btm_invalidation);
> > +               arm_smmu_write_cd_entry(master, master_domain->ssid, cdptr,
> > +                                       &target);
> >         }
> >         spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
> >
> > -       arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid);
> > -       arm_smmu_atc_inv_domain_sva(smmu_domain, mm->pasid, 0, 0);
> > -
> > -       smmu_mn->cleared = true;
> > -       mutex_unlock(&sva_lock);
> > +       arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_domain->cd.asid);
> 
> Similar questions to patch 11 from the v1, but why is it ok to remove
> the ATC invalidation here? 

It isn't, it is a mistake as well!

> Did you perhaps mean to remove the TLB invalidation instead (for which
> it's IIUC ok to delay the invalidation to when the domain/asid is
> freed, since those cache entries won't give a hit while the CD is
> cleared)?

Hmm. I found this:

* When EPDx == 1, a translation table walk through TTBx causes F_TRANSLATION.

- Note: The Armv8-A VMSA allows a TLB hit to occur for an input
  address associated with an EPD bit set to 1, but the translation
  table walk is disabled upon miss.

So we need to flush the ASID too when using EPD to disable it.

Like this:

        arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_domain->asid);
+       arm_smmu_atc_inv_domain(smmu_domain, 0, 0);
 }

Jason