Jffs2 and big file = very slow jffs2_garbage_collect_pass

[Ricard Wanderlof]

On Tue, 22 Jan 2008, Jörn Engel wrote:

> - Moderate: one block continuously spews -EUCLEAN, then becomes
>  terminally bad.
>  If those are just random bitflips, garbage collection will move the
>  data sooner or later.  Logfs does not force GC to happen soon when
>  encountering -EUCLEAN, which it should.  Are correctable errors an
>  indication of block going bad in the near future?  If yes, I should do
>  something about it.

I would say that correctable errors occurring "soon" after writing are an 
indication that the block is going bad. My experience has been that 
extensive reading can cause bitflips (and it probably happens over time 
too), but that for fresh blocks, billions of read operations need to be 
done before a bit flips. For blocks that are nearing their best before 
date, a couple of hundred thousand reads can cause a bit to flip. So if I 
was implementing some sort of 'when is this block considered 
bad'-algorithm, I'd try to keep tabs on how often the block has been 
(read-) accessed in relation to when it was last writen. If this number is 
"low", the block should be considered bad and not used again.

I'm also think that when (if) logfs decides a block is bad, it should mark 
it bad using mtd->block_markbad(). That way, if the flash is rewritten by 
something else than logfs (say during a firmware upgrade), bad blocks can 
be handled in a consistent and startad way.

>The list probably goes on and on.  And I am sure that I would miss at
>least half the interesting cases if I had to create it on my own.  But
>if Matthieu or you or anyone else is willing to compose an extensive
>list of such failure cases, I will walk through it and try to handle
>them one by one.

We ran some tests here on a particular flash chip type to try and 
determine at least some of the failure modes that are related to block 
wear (due to write/erase) and bit decay (due to reading). The end result 
was basically what I tried to describe above, but I can go into more 
detail if you're interested.

/Ricard