[PATCH v2 0/8] Cache coherency management subsystem

[Davidlohr Bueso]

On Wed, 25 Jun 2025, H. Peter Anvin wrote:

>On June 25, 2025 1:52:04 AM PDT, Peter Zijlstra <peterz at infradead.org> wrote:
>>On Tue, Jun 24, 2025 at 04:47:56PM +0100, Jonathan Cameron wrote:
>>
>>> On x86 there is the much loved WBINVD instruction that causes a write back
>>> and invalidate of all caches in the system. It is expensive but it is
>>
>>Expensive is not the only problem. It actively interferes with things
>>like Cache-Allocation-Technology (RDT-CAT for the intel folks). Doing
>>WBINVD utterly destroys the cache subsystem for everybody on the
>>machine.
>>
>>> necessary in a few corner cases.
>>
>>Don't we have things like CLFLUSH/CLFLUSHOPT/CLWB exactly so that we can
>>avoid doing dumb things like WBINVD ?!?
>>
>>> These are cases where the contents of
>>> Physical Memory may change without any writes from the host. Whilst there
>>> are a few reasons this might happen, the one I care about here is when
>>> we are adding or removing mappings on CXL. So typically going from
>>> there being actual memory at a host Physical Address to nothing there
>>> (reads as zero, writes dropped) or visa-versa.
>>
>>> The
>>> thing that makes it very hard to handle with CPU flushes is that the
>>> instructions are normally VA based and not guaranteed to reach beyond
>>> the Point of Coherence or similar. You might be able to (ab)use
>>> various flush operations intended to ensure persistence memory but
>>> in general they don't work either.
>>
>>Urgh so this. Dan, Dave, are we getting new instructions to deal with
>>this? I'm really not keen on having WBINVD in active use.
>>
>
>WBINVD is the nuclear weapon to use when you have lost all notion of where the problematic data can be, and amounts to a full reset of the cache system.
>
>WBINVD can block interrupts for many *milliseconds*, system wide, and so is really only useful for once-per-boot type events, like MTRR initialization.

Correct, and cpu_cache_invalidate_memregion() was introduced exactly
with these constraints in mind, and the current x86 is the worse case
scenario. As Jonathan pointed out, ranged optimizations only improve
what is already there.

Thanks,
Davidlohr