L1 & L2 cache flush sequence on CortexA5 MPcore w.r.t low power modes

[Lorenzo Pieralisi]

On Mon, May 14, 2012 at 05:39:09PM +0100, Russell King - ARM Linux wrote:
> On Mon, May 14, 2012 at 05:21:50PM +0100, Lorenzo Pieralisi wrote:
> > On Mon, May 14, 2012 at 04:58:59PM +0100, Russell King - ARM Linux wrote:
> > > On Mon, May 14, 2012 at 04:50:22PM +0100, Lorenzo Pieralisi wrote:
> > > > > 2. L2 disable
> > > > > 3. L1 clean & invalidate
> > > > 
> > > > This is wrong again since while cleaning and invalidating the cache (L1 here)
> > > > can still allocate and this must not happen.
> > > 
> > > No it isn't.  There is never anything wrong with allocating new caches lines
> > > into a cache which is going to (eventually) be powered down.  Ever.
> > 
> > What if the cache allocates a dirty cache line moved from L1 of another
> > processor ?
> > 
> > > What would be wrong is if we end up with dirty cache lines in the cache
> > > to be powered down for data which we _care_ about preserving when power
> > > is lost.
> > > 
> > > That's a _very_ _very_ important difference.
> > 
> > That's exactly the point I am making. dirty cache lines can be migrated across
> > processors caches. If we want to shut down a single core we have to be 100%
> > sure that dirty cache lines (if we care about that data, we might be not as you
> > pointed out) must not be present in L1 when we shut the core down. If the C
> > bit in SCTLR is not cleared before cleaning and invalidating this is not
> > guaranteed from an architectural point of view.
> > 
> > Occurences might be rare, but it is still not safe to clean the cache with the
> > C bit set.
> 
> It's not safe to disable the C bit without first pushing the dirty data out
> to RAM either.  It's a catch-22 situation - because turning the C bit off
> not only stops the caches allocating new lines but also prevents them being
> searched.

That depends on the processor. On A9 cache is bypassed on A15 it is not,
data access might still hit in the cache.

It is "implementation defined" according to ARM ARM (B2-1265).
But C bit cleared stops allocation that's true across all implementations.

> That means your view of cacheable memory suddenly changes beneath you when
> the C bit is turned off.

Yes might be (see above) but the cache operations still work so we do
not have any problem (well, as long as we clean and invalidate without
using data that can live in the cache, but that's how it is done on v7 cache
flush ops and it is perfectly fine).

> From what you're saying - and from my understanding of your cache behaviours,
> even the sequence:
> - clean cache
> - disable C bit
> - clean cache
> is buggy.

No, that's correct, works fine on A9 and A15. Second clean is mostly nops.

> 
> I think what you're effectively saying is that it is not possible to safely
> power down a cache on an ARM SMP CPU...

It is possible, but the final clean must be done with C bit cleared. It is
belt and braces, agreed, but that's the only way to do it properly.

Lorenzo