Debugging a TTY race condition on M1 (memory ordering dragons)

[Hector Martin]

On 16/08/2022 03.04, Boqun Feng wrote:
> On Tue, Aug 16, 2022 at 01:01:17AM +0900, Hector Martin wrote:
> Hmm.. but doesn't your (and Will's) finding actually show why
> queue_work() only guarantee ordering if queuing succeeds? In other
> words, if you want extra ordering, use smp_mb() before queue_work()
> like:
> 
> 	smp_mb();	// pairs with smp_mb() in set_work_pool_and_clear_pending()
> 	queue_work();	// if queue_work() return false, it means the work
> 			// is pending, and someone will eventually clear
> 		      	// the pending bit, with the smp_mb() above it's
> 			// guaranteed that work function will see the
> 			// memory accesses above.
> 
> Of course, I shall defer this to workqueue folks. Just saying that it
> may not be broken. We have a few similar guarantees, for example,
> wake_up_process() only provides ordering if it really wakes up a
> process.

Technically yes, but that doesn't actually make a lot of sense, and in
fact the comments inside the workqueue code imply that it does actually
provide order even in the failure case (and there are other barriers to
try to make that happen, just not enough). Note that the ordering
documentation was added post-facto, and I don't think the person who
wrote it necessarily considered whether it *actually* provides
guarantees in the failure case, and whether it should.

wake_up_process() is different because it doesn't actually guarantee
anything if the process is already awake. However, under this
definition, queue_work() guarantees that *some* work execution will
observe every preceding write before queue_work(), regardless of the
current state, and that is a very useful property. That is something
that wake_up_process() semantics can't do.

Without this guarantee, basically every queue_work() user that's using
some kind of producer/consumer pattern would need the explicit barrier.
I imagine that pattern is very common.

- Hector