[PATCH 6/6] arm/imx6q: add suspend/resume support

[Lorenzo Pieralisi]

On Thu, Sep 08, 2011 at 05:24:52PM +0100, Russell King - ARM Linux wrote:
> On Thu, Sep 08, 2011 at 11:22:23PM +0800, Shawn Guo wrote:
> > On Thu, Sep 08, 2011 at 08:47:18AM +0100, Russell King - ARM Linux wrote:
> > > On Thu, Sep 08, 2011 at 02:23:02PM +0800, Shawn Guo wrote:
> > > > Yes, if you lose L2 power in suspend, it's the easiest way for you to
> > > > resume L2.
> > > > 
> > > > For my case which retains L2, I actually do not want to call this
> > > > function which will invalidate L2.  But since we still have problem
> > > > to use rmk's generic suspend/resume updates (ARM: pm: add L2 cache
> > > > cleaning for suspend), we have to flush the entire L2 on imx6q for now.
> > > 
> > > OMAP44xx has the same problem but they re-initialize L2 on resume in
> > > their pre-cpu_resume() assembly code.
> > > 
> > Are you suggesting that this might be the reason why imx6q has problem
> > with your patch?
> 
> Consider the mechanics of what is happening.
> 
> On suspend, when we enter cpu_suspend(), we assume the L2 cache is
> still enabled.
> 
> We _always_ assume that L1 cache state is lost, so we always flush the
> entire L1 cache.
> 
> As SoCs can (and do) preserve the L2 contents over suspend/resume cycles,
> we leave it to the platform's finisher to decide whether it needs to
> flush the entire L2 cache to RAM or not.
> 
> If L2 is preserved, then we want to ensure that as much data as possible
> is retained in the L2 cache (if the hardware preserves its contents, we
> don't want waste time flushing data out of the cache needlessly -
> especially if we're using these paths for cpuidle.)  However, we need
> access to a certain amount of data to bring the system back up, and as
> the L2 cache typically will not be searched before the MMU is enabled,
> we have to flush out a certain minimal amount of data (the location
> of the stacked restore information and the stacked restore information
> itself.)
> 
> However, we don't flush out anything else from the L2 cache.
> 
> Now, upon resume, the resume code will be able to read the data it needs
> to restore the system as we ensured that was flushed out to memory - as
> I mentioned above, the L2 cache won't be searched for this irrespective
> of whether the control registers have enabled it or not.
> 
> We will continue to the point where we hit the first bit of information
> stored in L2, which will probably be the stacked SVC register set.  If
> at this point we're not able to search the L2 cache, then we'll read
> stale data from the backing memory instead, and the system will crash.
> 
> Consider what generic code could do about this - if we flushed out that
> register set to memory, then we could get past that point - but we then
> would need to call a function to enable the L2 cache.  What if that
> function needs data which is sitting in the L2 cache to function (eg,
> it may need values from the device tree).  We would need to ensure
> that that data were also flushed out of the L2 cache.  What about
> spinlocks?  Maybe some other CPU has dragged the spinlock data into
> the L2 cache.  That gets _much_ harder to solve.
> 
> Now to the physical act of enabling the L2 cache.  The L2 cache control
> registers are subject to security restrictions when running in non-secure
> mode, needing platform specific SMC calls to reprogram the cache.  Generic
> code is unable to do this.
> 
> Hence why it's left up to the platform to figure out how to enable the
> L2 cache before calling cpu_resume().  The platform is best placed to
> work out what it needs to do to setup the L2 cache so that the L2 cache
> is available by the time the system control register is written, enabling
> the MMU and caches.
> 
> > Are we supposed to re-initialize L2 before calling
> > into generic cpu_resume()?
> 
> So, the above is the long way of saying "yes" to this question.  I hope
> it gives the full picture about why this is so.
> 

I could not agree more. On top of that l2x0_of_init invalidates L2 which is
not what we want on retention. 
Furthermore, parsing the device tree every given suspend, as fast as it can be,
that seems silly to me for registers that will never ever change (maybe perf
counters, but perf will save them). And if they do change parsing them
from the device tree might not be what we want.

Let's save those registers once for all at boot and reprogramme them before 
jumping to cpu_resume, that's by far the best solution.