Preventing JFFS2 partial page writes?

[Peter Barada]

On 06/28/2011 05:34 AM, Artem Bityutskiy wrote:
> On Mon, 2011-06-27 at 10:31 -0400, Peter Barada wrote:
>> On 06/24/2011 03:26 PM, Artem Bityutskiy wrote:
>>> On Wed, 2011-06-22 at 11:28 -0400, Peter Barada wrote:
>>>> Thoughts?
>>> Sorry, could you please define the problem you are trying to solve?
>>> Sorry if you did define it in your long post, but I could not easily
>>> find it.
>> The problem I'm trying to solve is that the Micron NAND  I'm using has
>> an internal 4-bit ECC engine and uses four 8-byte ECCs that provide
>> 4-bit protection per 512 data bytes + four OOB bytes.  The ecclayout I'm
>> using is:
>>
>> ecclayout = {
>>           eccbytes = 32,
>>           eccpos = { 8, 9, 10, 11, 12, 13, 14, 15, /* ECC data bytes
>> 0-511 + OOB bytes 4-7 */
>>                   24, 25, 26, 27, 28, 19, 30, 31, /* ECC data bytes
>> 512-1023 + OOB bytes 20-23 */
>>                   40, 41, 42, 43, 44, 45, 46, 47,  /* ECC data bytes
>> 1024-1535 + OOB bytes 36-39 */
>>                   56, 57, 58, 59, 60, 61, 62, 63},  /* ECC data bytes
>> 1536-2047 + OOB bytes 52-55 */
>>           .oobfree = {
>>                   { .offset = 4,
>>                     .length = 4},
>>                   { .offset = 20,
>>                     .length = 4},
>>                   { .offset = 36,
>>                     .length = 4},
>>                   { .offset = 52,
>>                     .length = 4},
>>           },
>> };
>>
>> After the JFFS2 cleanmarker is written into bytes 4-7 and 16-23 of the
>> OOB, nanddump shows:
>>
>>     OOB Data: ff ff ff ff 85 19 03 20 5a e3 da 69 01 40 f1 36
>>     OOB Data: ff ff ff ff 08 00 00 00 91 99 3c 05 01 d0 5d b3
>>     OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>     OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>
>> Note that the ECC bytes 8-15 and 24-31 are no longer FF due to bytes 4-7
>> and bytes 20-23 being written with non-zero data.
>>
>> When data is later written to this same page (w/o an intervening erase
>> of its block) reading the page causes an uncorrectable ECC error.
>>
>> There are eight additional bytes of OOB space available for writing, but
>> they are not ECC'd.
>>
>> The issue I'm trying to solve is how to communicate from MTD to JFFS2
>> that some bytes of the oobfree array perturb the data ECC and can not be
>> used to write the cleanmarker.
> OK, thanks for explanation. I am not very good in this area as I do not
> have much experience dealing with OOB, but here is what I thing.
>
> 1. Linux MTD code was _not_ designed for "ECC'ed OOB".
> 2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
>     is not very verbose.
> 3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
>     everywhere except for some tricky cases when you want to test things
>     by writing incorrect ECC, or you have an image with ECC and you want
>     to flash it as is.
> 4. In general, OOB should be considered as belonging to the driver, and
>     modern software should not rely on OOB at all.
> 5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
>     which the _user_ can freely and _independently_ use.
> 6. In your case only this assumption does not work and your ecclayout is
>     incorrect because the OOB areas you expose are not independent.
> 7. So in your case your ecclayout should be changed and you should
>     expose only independent ECC bytes.
The independent ECC bytes available only total to eight, which makes it 
impossible to use YAFFS (which needs at least 16 bytes to stor e its 
metadata in).  I can add the independent bytes at the end of the layout 
(and tweak YAFFS to ignore them since it needs only 16 bytes if the 
metadata is ECC'ed), but I still need to create a fix in JFFS2 (and 
u-boot and utilities) to skip the first 16 bytes of oobfree area.  
That's why I think another ioctl call (GETOOBLAYOUT?) would be useful to 
describe the oobfree list, as well as lists(or bitfields) that indicates 
which bytes in oobfree are either ECC'ed as part of the data ECC, or 
ECC'ed independently.

Another issue this exposes is that JFFS2 reads/compares the cleanmarker 
w/o any ECC in the marker data to verify its validity - if a bitflip in 
an unECC'd cleanmarker is read back, then I think JFFS2 will fail to use 
that block.

Also, from what I can find, MTD does not provide a method of programming 
OOB NAND data using MTD_OOB_AUTO as mtd_do_writeoob (called in mtdchar.c 
from the MTDWRITEOOB ioctl) uses only MTD_OOB_PLACE.

-- 
Peter Barada
peter.barada at gmail.com