[PATCH v3 12/33] iommu/mediatek: Always tlb_flush_all when each PM resume

[Yong Wu]

Hi Dafna,

On Wed, 2021-11-10 at 15:50 +0800, Yong Wu wrote:
> On Wed, 2021-11-10 at 07:29 +0200, Dafna Hirschfeld wrote:
> > 
> > On 10.11.21 04:20, Yong Wu wrote:
> > > On Tue, 2021-11-09 at 14:21 +0200, Dafna Hirschfeld wrote:
> > > > Hi
> > > > This patch is needed in order to update the tlb when a device
> > > > is
> > > > powered on.
> > > > Could you send this patch alone without the whole series so it
> > > > get
> > > > accepted easier?
> > > 
> > > Which SoC are you testing on? In previous SoC, the IOMMU HW don't
> > > have
> > > power-domain, and we have a "has_pm"[1] in the tlb function for
> > > that
> > > case. The "has_pm" should be always 0 for the previous SoC like
> > > mt8173,
> > > it should always tlb synchronize.
> > > 
> > > thus, Could you help share more about your issue? In which case
> > > it
> > > lack
> > > the necessary tlb operation. At least, We need confirm if it
> > > needs
> > > a
> > > "Fixes" tags if sending this patch alone.
> > 
> > Hi,
> > I work with the mtk-vcodec driver on mt8173. As you wrote, the
> > iommu
> > doesn't
> > have a power-domain and so when allocating buffers before the
> > device
> > is powered
> > on, there is the warning
> > "Partial TLB flush timed out, falling back to full flush"
> > flooding the log buf.
> 
> oh. Thanks very much for your information. Get it now.
> 
> This issue should be introduced by the:
> 
> b34ea31fe013 ("iommu/mediatek: Always enable the clk on resume")
> 
> tlb failed due to the bclk is not enabled. Could you help try that
> after reverting this?
> 
> > 
> > Sebastian Reichel suggested to remove the 'if(has_pm)' check to
> > avoid
> > this warning,
> > and avoid flushing the tlb if the device is off:
> > 
> > [1] http://ix.io/3Eyr
> > 
> > This fixes the warning, but then the tlb is not flushed in sync,
> > Therefore the tlb should be flushed when the device is resumed.
> > 
> > So the two patches (the one suggested in the link [1] and this
> > patch)
> > should be sent together as a 2-patch series.
> 
> then this is reasonable. You could help this into a new patchset if
> you
> are free(add Fixes tag).
> 
> Thanks.
> 
> > 
> > Thanks,
> > Dafna
> > 
> > > 
> > > Thanks.
> > > 
> > > [1]
> > > 
> 
> 
https://elixir.bootlin.com/linux/v5.15/source/drivers/iommu/mtk_iommu.c#L236
> > > 
> > > > I can resend the patch on your behalf if you want.

What's your plan here?

If you send the two as a independent patchset on v5.16, I will rebase
yours.
If you have no time for this, I could help this.

Thanks.

> > > > 
> > > > Thanks,
> > > > Dafna
> > > > 
> > > > On 23.09.21 14:58, Yong Wu wrote:
> > > > > Prepare for 2 HWs that sharing pgtable in different power-
> > > > > domains.
> > > > > 
> > > > > When there are 2 M4U HWs, it may has problem in the
> > > > > flush_range
> > > > > in
> > > > > which
> > > > > we get the pm_status via the m4u dev, BUT that function don't
> > > > > reflect the
> > > > > real power-domain status of the HW since there may be other
> > > > > HW
> > > > > also
> > > > > use
> > > > > that power-domain.
> > > > > 
> > > > > The function dma_alloc_attrs help allocate the iommu buffer
> > > > > which
> > > > > need the corresponding power domain since tlb flush is needed
> > > > > when
> > > > > preparing iova. BUT this function only is for allocating
> > > > > buffer,
> > > > > we have no good reason to request the user always call
> > > > > pm_runtime_get
> > > > > before calling dma_alloc_xxx. Therefore, we add a
> > > > > tlb_flush_all
> > > > > in the pm_runtime_resume to make sure the tlb always is
> > > > > clean.
> > > > > 
> > > > > Another solution is always call pm_runtime_get in the
> > > > > tlb_flush_range.
> > > > > This will trigger pm runtime resume/backup so often when the
> > > > > iommu
> > > > > power is not active at some time(means user don't call
> > > > > pm_runtime_get
> > > > > before calling dma_alloc_xxx), This may cause the performance
> > > > > drop.
> > > > > thus we don't use this.
> > > > > 
> > > > > In other case, the iommu's power should always be active via
> > > > > device
> > > > > link with smi.
> > > > > 
> > > > > The previous SoC don't have PM except mt8192. the mt8192
> > > > > IOMMU
> > > > > is
> > > > > display's
> > > > > power-domain which nearly always is enabled. thus no need fix
> > > > > tags
> > > > > here.
> > > > > Prepare for mt8195.
> > > > > 
> > > > > Signed-off-by: Yong Wu <yong.wu at mediatek.com>
> > > > > ---
> > > > >    drivers/iommu/mtk_iommu.c | 11 +++++++++++
> > > > >    1 file changed, 11 insertions(+)
> > > > > 
> > > > > diff --git a/drivers/iommu/mtk_iommu.c
> > > > > b/drivers/iommu/mtk_iommu.c
> > > > > index 44cf5547d084..e9e94944ed91 100644
> > > > > --- a/drivers/iommu/mtk_iommu.c
> > > > > +++ b/drivers/iommu/mtk_iommu.c
> > > > > @@ -984,6 +984,17 @@ static int __maybe_unused
> > > > > mtk_iommu_runtime_resume(struct device *dev)
> > > > >    		return ret;
> > > > >    	}
> > > > >    
> > > > > +	/*
> > > > > +	 * Users may allocate dma buffer before they call
> > > > > pm_runtime_get, then
> > > > > +	 * it will lack the necessary tlb flush.
> > > > > +	 *
> > > > > +	 * We have no good reason to request the users always
> > > > > call
> > > > > dma_alloc_xx
> > > > > +	 * after pm_runtime_get_sync.
> > > > > +	 *
> > > > > +	 * Thus, Make sure the tlb always is clean after each
> > > > > PM
> > > > > resume.
> > > > > +	 */
> > > > > +	mtk_iommu_tlb_do_flush_all(data);
> > > > > +
> > > > >    	/*
> > > > >    	 * Uppon first resume, only enable the clk and return,
> > > > > since
> > > > > the values of the
> > > > >    	 * registers are not yet set.
> > > > > 
> 
> _______________________________________________
> iommu mailing list
> iommu at lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu