[PATCH 01/22] ARM: add mechanism for late code patching

[Russell King - ARM Linux]

On Mon, Aug 06, 2012 at 09:19:10AM -0400, Cyril Chemparathy wrote:
> With a flush_cache_all(), we could avoid having to operate a cacheline  
> at a time, but that clobbers way more than necessary.

You can't do that, because flush_cache_all() on some CPUs requires the
proper MMU mappings to be in place, and you can't get those mappings
in place because you don't have the V:P offsets fixed up in the kernel.
Welcome to the chicken and egg problem.

> Sure, flushing caches is expensive.  But then, so is running the  
> patching code with caches disabled.  I guess memory access latencies  
> drive the performance trade off here.

There we disagree on a few orders of magnitude.  There are relatively
few places that need updating.  According to the kernel I have here:

   text    data     bss     dec     hex filename
7644346  454320  212984 8311650  7ed362 vmlinux

Idx Name          Size      VMA       LMA       File off  Algn
  1 .text         004cd170  c00081c0  c00081c0  000081c0  2**5
 16 .init.pv_table 00000300  c0753a24  c0753a24  00753a24  2**0

That's about 7MB of text, and only 192 points in that code which need
patching.  Even if we did this with caches on, that's still 192 places,
and only 192 places we'd need to flush a cache line.

Alternatively, with your approach and 7MB of text, you need to flush
238885 cache lines to cover the entire kernel.

It would be far _cheaper_ with your approach to flush the individual
cache lines as you go.