REGRESSION in 6.0-rc7 caused by patch "ARM/dma-mapping: use dma-direct unconditionally"

[Robin Murphy]

On 2022-10-04 08:10, Christoph Hellwig wrote:
> On Mon, Oct 03, 2022 at 11:30:31PM +0200, Marcin Wojtas wrote:
>> I have one overall concern here. On all kinds of A38x-based boards I
>> worked on, by default, the firmware set all devices (e.g. network,
>> AHCI, XHCI) on MBUS as fully IO cache coherent - it should be
>> reflected in the MVNETA_WIN_BASE(w) registers attribute field. Bits
>> [15:8] should be set to 0x1D (or 0x1E if there is a second DRAM CS
>> used). Can you please try adding 'dma-coherent;' property under the
>> 'internal-regs' node?
> 
> Robin mentioned something similar earlier.  This almost smalls like
> we somehow manage to mark these device non-coherent by accident now.
> 
> The interesting part of the bisected commit is the change to
> mvebu_hwcc_notifier that used to force the DMA OPS to
> arm_coherent_dma_ops, but now just sets the ->dma_coherent flags,
> which seems to get overriden somehow again.  Maybe the notifier is
> run before arch_setup_dma_ops, even if that seems odd?  As that is
> the only thing I could think of, maybe try this patch:
> 
> 
> diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
> index 089c9c644cce2..76789650e2596 100644
> --- a/arch/arm/mm/dma-mapping.c
> +++ b/arch/arm/mm/dma-mapping.c
> @@ -1770,7 +1770,9 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
>   			const struct iommu_ops *iommu, bool coherent)
>   {
>   	dev->archdata.dma_coherent = coherent;
> -	dev->dma_coherent = coherent;
> +
> +	if (coherent)
> +		dev->dma_coherent = true;
>   
>   	/*
>   	 * Don't override the dma_ops if they have already been set. Ideally
> 

That seems reasonable to me - it preserves the equivalent behaviour of 
respecting platform overrides, which thankfully only go one way. In 
terms of the intent it might be more properly phrased as "if 
(!dev->dma_coherent) dev->dma_coherent = coherent" but whether that's 
worth the extra characters is debatable.

The only other possibility might be to change mvebu_hwcc_notifier() to 
apply its override at the BUS_NOTIFY_BIND_DRIVER stage, after 
dma_configure() has run. The other notifier for highbank looks 
unaffected since that one's just following whatever the DT says rather 
than trying to override anything.

Thanks,
Robin.