[PATCH] iommu/arm-smmu-v3: Enable PCI ATS in passthrough mode as well

[Ganapatrao Kulkarni]

On 02-02-2023 06:52 pm, Robin Murphy wrote:
> On 2023-02-02 12:40, Ganapatrao Kulkarni wrote:
>> The current smmu-v3 driver does not enable PCI ATS for physical functions
>> of ATS capable End Points when booted in smmu bypass mode
>> (iommu.passthrough=1). This will not allow virtual functions to enable
>> ATS(even though EP supports it) while they are attached to a VM using
>> VFIO driver.
>>
>> This patch adds changes to enable ATS support for physical functions
>> in passthrough/bypass mode as well.
>>
>> Also, adding check to avoid disabling of ATS if it is not enabled,
>> to avoid unnecessary call-traces.
>>
>> Signed-off-by: Ganapatrao Kulkarni <gankulkarni at os.amperecomputing.com>
>> ---
>>   drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 10 +++++++---
>>   1 file changed, 7 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c 
>> b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> index 6d5df91c5c46..5a605cb5ccef 100644
>> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> @@ -2313,11 +2313,16 @@ static void arm_smmu_enable_ats(struct 
>> arm_smmu_master *master)
>>   static void arm_smmu_disable_ats(struct arm_smmu_master *master)
>>   {
>>       struct arm_smmu_domain *smmu_domain = master->domain;
>> +    struct pci_dev *pdev;
>>       if (!master->ats_enabled)
>>           return;
>> -    pci_disable_ats(to_pci_dev(master->dev));
>> +    pdev = to_pci_dev(master->dev);
>> +
>> +    if (pdev->ats_enabled)
> 
> If the master->ats_enabled check above passes when ATS isn't actually 
> enabled, surely that's a bug?

IIUC, It means ATS feature is supported (just check for existence of ATS 
extended capability and smmu capability) and not necessarily enabled.
Function pci_enable_ats(called by arm_smmu_enable_ats) enables the ATS 
by setting bit 15 of ATS Control Register (Offset 06h).
If pci_enable_ats is not successful, it will not set dev->ats_enabled 
flag. So calling pci_disable_ats later results in call-trace, if 
dev->ats_enabled is not set.

Function arm_smmu_enable_ats already prints error message if ATS enable 
is failed.

> 
> Robin.
> 
>> +        pci_disable_ats(pdev);
>> +
>>       /*
>>        * Ensure ATS is disabled at the endpoint before we issue the
>>        * ATC invalidation via the SMMU.
>> @@ -2453,8 +2458,7 @@ static int arm_smmu_attach_dev(struct 
>> iommu_domain *domain, struct device *dev)
>>       master->domain = smmu_domain;
>> -    if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS)
>> -        master->ats_enabled = arm_smmu_ats_supported(master);
>> +    master->ats_enabled = arm_smmu_ats_supported(master);
>>       arm_smmu_install_ste_for_dev(master);

Thanks,
Ganapat