LSE atomic op ordering is weaker than intended?

[Will Deacon]

On Thu, Mar 04, 2021 at 04:37:31AM +0900, Hector Martin wrote:
> On 04/03/2021 03.40, Will Deacon wrote:
> > > // ...stuff that needs to be ordered prior to the atomic here
> > > ret = atomic_fetch_or_release(flags...)
> > > if (condition on ret and unrelated stuff) {
> > > 	writel(reg_send, ...) // includes pre-barrier
> > 
> > Looks you have a control dependency here, so I think that can be
> > writel_relaxed() [with a comment!].
> 
> Wouldn't the control dependency be strictly on the *read* portion of the
> atomic, while I'm looking to order on the write part?

Ah, sorry, I didn't realise you were ordering against the write. I'll wait
for the v3, but just one thing that might help:

> That is, that I can create two ordered sequences on two PEs:
> 
> (normal access 1; mmio write 1)
> (mmio write 2; normal access 2)
> 
> Such that I can assert that at least one of the following orderings are
> true:
> 
> (normal access 1; normal access 2)
> (as observed by the PEs)
> 
> or:
> 
> (mmio write 2; mmio write 1)
> (as observed by the MMIO device)

One thing to bear in mind here is that the MMIO device cannot "observe"
anything in the architectural sense because it is a slave interface. In
order to observe a memory access, you must emit a read or a write
transaction, and it's this notion of observation which the shareability
domains are built around.

So for this example, we can talk about the CPUs (in the inner-shareable
domain) observing the MMIO writes and inner-shareable barriers are
sufficient for that. The device mapping of the MMIO registers will then
ensure that they arrive at the endpoint in that order too.

> But never this combination:
> 
> (normal access 2; normal access 1)
> (mmio write 1; mmio write 2)
> 
> > But hack something together, and I'll look at your v3 to see what's going
> > on.
> 
> Please do, hopefully the full code makes more sense.

Hopefully, as I don't grok how this deals with spurious interrupts if it
only does MMIO writes.

Will