[RFC PATCH] UBIFS: remove check for all(0xff) for empty pages

[Kent Li]

I agree with Matthieu, the empty check should be done in UBI, MTD or Flash driver layer.
When reading an erased page, the ECC data fail to recover correct the data area, because the OOB data are also all 0xff.
All flash controller should be able to detect this situation, 
but unfortunately, almost all flash driver ignore this, and pass the data to upper layer directly.

Kent Li

-----Original Message-----
From: Matthieu CASTET [mailto:matthieu.castet at parrot.com] 
Sent: 2014年3月11日 20:36
To: Pekon Gupta
Cc: Brian Norris; Artem Bityutskiy; Kent Li; HOUR Frderic; linux-mtd; Ezequiel Garcia; Stefan Roese
Subject: Re: [RFC PATCH] UBIFS: remove check for all(0xff) for empty pages

Le Tue, 11 Mar 2014 11:53:12 +0530,
Pekon Gupta <pekon at ti.com> a écrit :

> UBIFS throws following error if an blank page does not contain perfect 
> all(0xff) data, during LEB scan, even though if number of bit-flips is within ecc.strength
> 	"UBIFS error (pid 1): ubifs_scan: corrupt empty space at LEB 417:53168"
> 
> But newer technology devices (like MLC NAND) are prone to read-disturb 
> bit-flips and so having perfect all(0xff) data on empty-pages is 
> practically not feasible on these newer technology devices (like MLC NAND).
> Also, such device spec suggest using stronger ECC schemes (like 
> BCH16), and have large OOB size to guarantee longer field life.
> 
> So this check of perfect all(0xff) data can be safely removed from 
> UBIFS, as UBI and MTD layers already these before passing the data to UBIFS:
>  if (number-of-bitflips < mtd->bitflip_threshold)
>         Driver's ECC scheme is strong enough to handle existing bit-flips in
>         future, so both MTD and UBI ignore such bit-flips.
>  if (number-of-bitflips >= mtd->bitflip_threshold)
>         MTD flags -EUCLEAN.
>         UBI schedules the block for scrubbing.
>  if (number-of-bitflips > nand_chip->ecc.strength)
>         MTD flags -EBADMSG.
>         UBI tries to extract data relevant data from block based on sanity of
>         its headers, and then schedule it for re-erase.

If the mtd layer check the number of bitflip on empty page why it doesn't correct it and UBIFS see all 0xff ?

In the other hand if mtd layer is not able to check empty page this is very dangerous. What happen if an empty page already have too much bitflip ?
The data we write on this page will be corrupted.

In my opinion the change should be done on mtd/UBI layer not ubifs. 
The main problem today is that some driver don't read empty page with ecc (because their ecc of empty page is not 0xff).
We can add metadata (in spare because ubifs want power of 2 write unit !) that can help to see it the page is empty or not.


Matthieu