[EXT] Re: The problem about arm64: io: Relax implicit barriers in default I/O accessors

[Will Deacon]

On Mon, Aug 09, 2021 at 02:46:55PM +0000, Frank Li wrote:
> 
> 
> > -----Original Message-----
> > From: Will Deacon <will at kernel.org>
> > Sent: Monday, August 9, 2021 8:51 AM
> > To: Frank Li <frank.li at nxp.com>
> > Cc: Catalin Marinas <catalin.marinas at arm.com>; Zhi Li <lznuaa at gmail.com>;
> > Shenwei Wang <shenwei.wang at nxp.com>; Han Xu <han.xu at nxp.com>; Nitin Garg
> > <nitin.garg at nxp.com>; Jason Liu <jason.hui.liu at nxp.com>; linux-arm-
> > kernel at lists.infradead.org
> > Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit barriers
> > in default I/O accessors
> > 
> > Caution: EXT Email
> > 
> > On Thu, Jul 22, 2021 at 07:14:19PM +0000, Frank Li wrote:
> > > > > On Wed, Jun 23, 2021 at 03:48:10PM +0000, Frank Li wrote:
> > > > > > > I think you had a support case open with Arm [1] which I'm not
> > able
> > > > to
> > > > > > > access -- please can you ask them about the two examples above?
> > > > > >
> > > > > > Still not get feedback from ARM.
> > > > >
> > > > > Just wondering if you were able to solve this without the need to
> > change
> > > > > Linux?
> > > >
> > > > Sorry for late reply
> > > >
> > > > For CCI-500 and 550, ARM removed support for barrier transactions but
> > CCI-
> > > > 400 supports barrier transactions. With CCI-400 it is a valid
> > configuration
> > > > to have SYSBARDISABLE LOW in Cortex-A processors. This change in Linux
> > > > kernel is assuming that the SYSBARDISABLE is set to HIGH hence its not
> > > > correct change for all products having various versions of ARM CCI IP.
> > > >
> > > > Frank Li
> > >
> > > Deacon:
> > >
> > >       Did you plan fix this problem by changing dma_wmb()?
> > 
> > No. As far as I understand this problem, you're driving SYSBARDISABLE
> > 'low' yet you have your own bus fabric downstream of the CCI which doesn't
> > respect barrier transactions. Even if we bodge dma_wmb(), store-release to
> > non-cacheable memory cannot be made to work on your system as you're
> > effectively putting some of your non-coherent DMA devices into a separate
> > outer-shareable domain from the CPUs.
> 
> Does it means the Linux expect all DMA devices in outer-shareable domain instead
> of system shared domain?  

I don't think we've ever documented that and, to be honest, the
outer-shareable domain stuff in the architecture is pretty academic.

However, I think it's fair to say that we do want the acquire/release
instructions to work for non-cacheable buffers when communicating with
non-coherent devices. I _think_ that implies that such devices need to
be in the same outer-shareable domain as the CPUs, although the
architecture isn't really clear here. I can try to find out.

Will