[PATCH v13 06/15] asm-generic: barrier: Add smp_cond_load_acquire_timeout()

[Ankur Arora]

Add the acquire variant of smp_cond_load_relaxed_timeout(). This
reuses the relaxed variant, with additional LOAD->LOAD ordering
via smp_acquire__after_ctrl_dep().

To ensure that the necessary control dependency on the dereference
of @ptr exists (which does not in the timeout path), re-evaluate
the cond_expr branch.

Cc: Kumar Kartikeya Dwivedi <memxor at gmail.com>
Cc: Alexei Starovoitov <ast at kernel.org>
Cc: Arnd Bergmann <arnd at arndb.de>
Cc: Will Deacon <will at kernel.org>
Cc: Catalin Marinas <catalin.marinas at arm.com>
Cc: Peter Zijlstra <peterz at infradead.org>
Cc: linux-arch at vger.kernel.org
Cc: bpf at vger.kernel.org
Reviewed-by: Catalin Marinas <catalin.marinas at arm.com>
Reviewed-by: Haris Okanovic <harisokn at amazon.com>
Tested-by: Haris Okanovic <harisokn at amazon.com>
Signed-off-by: Ankur Arora <ankur.a.arora at oracle.com>
---
Notes: addresses missed control dependency in the timeout case [1] by
documenting that the acquire ordering is only in the success
(non-timeout) case.
However, we do need to re-evaluate the cond_expr branch to handle the
case where we hit both the timeout and the loop condition.

AFAICT, rqspinlock only cares about acquire ordering in the success
case so this change should be okay. 
Kumar, Alexei: please let me know if otherwise.

Catalin, Haris: I've retained your R-by on this. Please let me know if
you aren't okay with this change.

[1] https://lore.kernel.org/all/20260608082746.399BB1F00898@smtp.kernel.org/
---
 include/asm-generic/barrier.h | 40 +++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index d53e75a92598..5d21ad9f358f 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -356,6 +356,46 @@ do {									\
 })
 #endif
 
+/**
+ * smp_cond_load_acquire_timeout() - (Spin) wait for cond until a timeout
+ * expires. ACQUIRE ordering when @cond_expr is satisfied.
+ * @ptr: pointer to the variable to wait on.
+ * @cond_expr: boolean expression to wait for.
+ * @time_expr_ns: monotonic expression that evaluates to time in ns or,
+ *  on failure, returns a negative value.
+ * @timeout_ns: timeout value in ns
+ * (Both of the above are assumed to be compatible with s64.)
+ *
+ * Equivalent to using smp_cond_load_acquire() on the condition variable with
+ * a timeout.
+ */
+#ifndef smp_cond_load_acquire_timeout
+#define smp_cond_load_acquire_timeout(ptr, cond_expr,			\
+				      time_expr_ns, timeout_ns)		\
+({									\
+	__unqual_scalar_typeof(*(ptr)) VAL;				\
+	VAL = smp_cond_load_relaxed_timeout(ptr, cond_expr,		\
+					     time_expr_ns,		\
+					     timeout_ns);		\
+	/*								\
+	 * We arrive here once the loop condition is hit, on timeout,	\
+	 * or, if we hit both the timeout and the loop condition.	\
+	 *								\
+	 * The last case is low probability, but possible in the last	\
+	 * iteration, especially on architectures with waiting		\
+	 * cpu_poll_relax() implementations (ex. arm64).		\
+	 * Now since the loop condition is not evaluated on timeout,	\
+	 * we have a missed control dependency.				\
+	 *								\
+	 * So, force a re-evaluation of the control dependency to	\
+	 * provide an ACQUIRE ordering for that case as well.		\
+	 */								\
+	if (cond_expr)							\
+		smp_acquire__after_ctrl_dep();				\
+	(typeof(*(ptr)))VAL;						\
+})
+#endif
+
 /*
  * pmem_wmb() ensures that all stores for which the modification
  * are written to persistent storage by preceding instructions have
-- 
2.43.7