Should we use "dsb" or "dmb" between write to buffer and write to register

[Will Deacon]

On Mon, Sep 12, 2022 at 04:43:14PM +0800, Mark Zhang wrote:
> However my colleague Suresh still has some concerns:
> "I believe the effect of the write64() here is to trigger a side effect in
> the device (that it is not a true write to memory although it is a memory
> access and so the NIC is not actually reading this memory address). If that
> is the case, a dsb is likely needed to guarantee that the effects of the
> memcpy are also observed by the NIC. You can check out some examples in
> Appendix K11 (Barrier Litmus Tests ) of the Arm ARM – for instance K11.4 and
> K11.5.4, where a dsb is used for these kinds of scenarios.
> ... There is a subtle difference between observing the execution of an
> instruction and observing the completion of an instruction"

A DSB is required in the general case to deal with private peripherals,
but these are pretty rare in practice and generally MMIO regions are going
to be shared between CPUs.

> What do you think?

I think a DMB is sufficient to ensure ordering (and so a DSB will work too,
but with much greater performance impact). If you actually want to cause the
device to change state before issuing the writes to memory, then even a DSB
is not enough -- you likely also want something like a read-back from a
status register and that sequence is device specific.

Will