[PATCH v2 0/8] Cache coherency management subsystem

[Peter Zijlstra]

On Wed, Jun 25, 2025 at 02:12:39AM -0700, H. Peter Anvin wrote:
> On June 25, 2025 1:52:04 AM PDT, Peter Zijlstra <peterz at infradead.org> wrote:
> >On Tue, Jun 24, 2025 at 04:47:56PM +0100, Jonathan Cameron wrote:
> >
> >> On x86 there is the much loved WBINVD instruction that causes a write back
> >> and invalidate of all caches in the system. It is expensive but it is
> >
> >Expensive is not the only problem. It actively interferes with things
> >like Cache-Allocation-Technology (RDT-CAT for the intel folks). Doing
> >WBINVD utterly destroys the cache subsystem for everybody on the
> >machine.
> >
> >> necessary in a few corner cases. 
> >
> >Don't we have things like CLFLUSH/CLFLUSHOPT/CLWB exactly so that we can
> >avoid doing dumb things like WBINVD ?!?
> >
> >> These are cases where the contents of
> >> Physical Memory may change without any writes from the host. Whilst there
> >> are a few reasons this might happen, the one I care about here is when
> >> we are adding or removing mappings on CXL. So typically going from
> >> there being actual memory at a host Physical Address to nothing there
> >> (reads as zero, writes dropped) or visa-versa. 
> >
> >> The
> >> thing that makes it very hard to handle with CPU flushes is that the
> >> instructions are normally VA based and not guaranteed to reach beyond
> >> the Point of Coherence or similar. You might be able to (ab)use
> >> various flush operations intended to ensure persistence memory but
> >> in general they don't work either.
> >
> >Urgh so this. Dan, Dave, are we getting new instructions to deal with
> >this? I'm really not keen on having WBINVD in active use.
> >
> 
> WBINVD is the nuclear weapon to use when you have lost all notion of
> where the problematic data can be, and amounts to a full reset of the
> cache system. 
> 
> WBINVD can block interrupts for many *milliseconds*, system wide, and
> so is really only useful for once-per-boot type events, like MTRR
> initialization.

Right this... But that CXL thing sounds like that's semi 'regular' to
the point that providing some infrastructure around it makes sense. This
should not be.