[EXT] Re: The problem about arm64: io: Relax implicit barriers in default I/O accessors

[Frank Li]

> -----Original Message-----
> From: Will Deacon <will at kernel.org>
> Sent: Monday, August 9, 2021 8:51 AM
> To: Frank Li <frank.li at nxp.com>
> Cc: Catalin Marinas <catalin.marinas at arm.com>; Zhi Li <lznuaa at gmail.com>;
> Shenwei Wang <shenwei.wang at nxp.com>; Han Xu <han.xu at nxp.com>; Nitin Garg
> <nitin.garg at nxp.com>; Jason Liu <jason.hui.liu at nxp.com>; linux-arm-
> kernel at lists.infradead.org
> Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit barriers
> in default I/O accessors
> 
> Caution: EXT Email
> 
> On Thu, Jul 22, 2021 at 07:14:19PM +0000, Frank Li wrote:
> > > > On Wed, Jun 23, 2021 at 03:48:10PM +0000, Frank Li wrote:
> > > > > > I think you had a support case open with Arm [1] which I'm not
> able
> > > to
> > > > > > access -- please can you ask them about the two examples above?
> > > > >
> > > > > Still not get feedback from ARM.
> > > >
> > > > Just wondering if you were able to solve this without the need to
> change
> > > > Linux?
> > >
> > > Sorry for late reply
> > >
> > > For CCI-500 and 550, ARM removed support for barrier transactions but
> CCI-
> > > 400 supports barrier transactions. With CCI-400 it is a valid
> configuration
> > > to have SYSBARDISABLE LOW in Cortex-A processors. This change in Linux
> > > kernel is assuming that the SYSBARDISABLE is set to HIGH hence its not
> > > correct change for all products having various versions of ARM CCI IP.
> > >
> > > Frank Li
> >
> > Deacon:
> >
> >       Did you plan fix this problem by changing dma_wmb()?
> 
> No. As far as I understand this problem, you're driving SYSBARDISABLE
> 'low' yet you have your own bus fabric downstream of the CCI which doesn't
> respect barrier transactions. Even if we bodge dma_wmb(), store-release to
> non-cacheable memory cannot be made to work on your system as you're
> effectively putting some of your non-coherent DMA devices into a separate
> outer-shareable domain from the CPUs.

Does it means the Linux expect all DMA devices in outer-shareable domain instead
of system shared domain?  

Frank

> 
> So you have two options:
> 
>   1. Drive SYSBARDISABLE 'high' so that the CPU handles ordering for you
> 
> - or -
> 
>   2. Quirk Linux so that we patch dma_wmb() when we detect your system at
>      runtime (so we can extend this in future if we need to emit a
> different
>      sequence for store release)
> 
> (1) is definitely the easiest option if it's possible.
> 
> Will