[PATCH v7 4/9] iommu/arm-smmu-v3: Optimize range invalidation for latency

[Jason Gunthorpe]

On Mon, Sep 28, 2026 at 01:15:50PM +0000, Pranjal Shrivastava wrote:
> > -/* Used by non INV_TYPE_ATS* invalidations */
> > -static void arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv,
> > +/*
> > + * One TLBI command per IOTLB entry, assuming the entries are all at least
> > + * iopte_granule sized. Returns false if too many commands would be needed which
> > + * indicates too high a latency. The threshold is similar to MAX_DVM_OPS in
> > + * arch/arm64/include/asm/tlbflush.h for the 4k PAGE_SIZE.
> > + */
> > +static bool arm_smmu_cmdq_batch_add_single(struct arm_smmu_device *smmu,
> > +					   struct arm_smmu_cmdq_batch *cmds,
> > +					   struct arm_smmu_cmd *cmd,
> > +					   struct arm_smmu_tlbi *tlbi)
> > +{
> > +	unsigned long num_ops = tlbi->size / tlbi->iopte_size;
> > +	unsigned long iova = tlbi->iova;
> > +	unsigned long i;
> > +
> > +	if (!num_ops || num_ops > 512)
> > +		return false;
> > +
> 
> Should this be ">= 512"? The old arm_smmu_inv_size_too_big() and
> arm64's __flush_tlb_range_limit_excess() both fall back to a full
> invalidation at exactly 512:
> 
>         return pages >= (MAX_DVM_OPS * stride) >> PAGE_SHIFT;
> 
> 512 is what a walk flush could generate. For example, when 
> __arm_lpae_unmap() frees a last-level table on a 4K granule,
> it calls io_pgtable_tlb_flush_walk(iova, SZ_2M, SZ_4K), hence,
> num_ops == 512.
> 
> Before this patch that was a single TLBI_NH_ASID/TLBI_S12_VMALL, now
> it becomes 512 VA TLBIs + CMD_SYNC for every 2M table freed on a
> non-RIL SMMU. The same check carries into arm_smmu_tlbi_calc_single()
> later in the series.

I had set it deliberately like that so that a table flush would issue
singles. That is largely based around the feedback from before that we
shouldn't over invalidate.

We've been insensitive to that issue in the past, and I'm going to
argue the >= of today's code is a bug as it effectively made alot of
iopgtable actions turn into new full invalidations, while originally
they were range bound singles.

So I view this choice as a return to the historical behavior before we
started to fix the soft lockup issues. I guess it crept it during one
of the refactorings where we merged the SVA limitation into the main
flow.

Granted I didn't notice this detail that the current code was 511 not
512, I will add a remark to the commit message:

     The end result is any gather is converted into either:
      - One invalidate all
      - One or two range invalidation operations
      - At most 512 single invalidation ops
	The current code switches at 511, but this turns every table removal
	into a full invalidation. Change to 512 to try to minimize the amount
	of full invalidation.

Thanks,
Jason