UBIFS does not mount after powerfail

[Manfred Spraul]

Hi Richard,

On 11/28/2017 10:00 PM, Richard Weinberger wrote:
> Manfred,
>
>
> Am Donnerstag, 23. November 2017, 23:03:28 CET schrieb Manfred Spraul:
>
> I tied, but TBH I'm completely lost in all the data you throwing on me.
Understood, I generated way too much data.
I have also confused all my coworkers :-(

Let's keep it simple and concentrate on the 
garbage_collect_leb()/ubifs_scan issue.

> Let's recap, you trigger a corruption that happens only(!) when xattrs are
> used?
Correct, I only see the corruption when xattr is used.
> How is Fastmap involved in the game? If so, I want to know whether you can
> trigger without Fastmap being enabled.
Correct, I have seen the corruption without FASTMAP.
> Which one is the image that failed first with chk_fs enabled?
> On a vanilla kernel...
I have used a modified MTDRAM: I have patched it so that it writes every 
ERASE and every WRITE into a log.
Both the addresses and the data.
First, I have created an initial image 
(ubiformat/ubiattach/ubimkvol/ubidetach).
Then I did a dump of it (nanddump /dev/mtd0 > dump-before.bin)
Then I enabled logging, and run the test load (ubiattach/mount/mount 
ecryptfs/<many touch/write/rm/mv commands>/umount/ubidetach).
Then I created a dump of the final image. (nanddump /dev/mtd0 > 
dump-after.bin)

Then the obvious cross checks:
Initial image+apply all commands from the log is identical to the final 
image.

Now I can use the log to create virtual power fail events:

What if a power failure happens after 168168 commands?
Image:
https://sourceforge.net/projects/calculix-rpm/files/ubifs/image-168168.bin/download

Logfile of a mount without chk_fs:
https://sourceforge.net/projects/calculix-rpm/files/ubifs/r-168168.txt/download

What was the last write command?
FUNC_WRITE_CHK(addr=0xd61000, len=0x800)

What would have been the next write command:
FUNC_WRITE_CHK(addr=0xd61800, len=0x800)

Image after that command:
https://sourceforge.net/projects/calculix-rpm/files/ubifs/image-168169.bin/download


Just to clarify:
I have simulated power fails.
A bit like ubi_dbg_power_cut, but instead of stopping further 
writes/erase commands, I created a log file.
And now I can replay the log to an arbitrary point, and then mount the 
result.
I have a total of 5 logs, and from these 5 logs, I did around 100k 
replays and mounted the results.

--
     Manfred

P.S.: I'm doing final polishing of the modifications to mtdram. As soon 
as I have the internal release, I will send it to linux-mtd.
P.P.S.: To avoid any brown paperbag bugs: mtd->_erase and mtd->_write 
are allowed to sleep, correct? At least, st_spi_fsm contains mutex_lock().