Using UBIFS as an FTL

[Daniel Ehrenberg]

On Mon, Jul 28, 2014 at 9:54 AM, Artem Bityutskiy
<artem.bityutskiy at linux.intel.com> wrote:
>
> On Fri, 2014-07-25 at 11:21 -0700, Daniel Ehrenberg wrote:
> > Hi Artem, others,
> >
> > For my project, I'm looking into using ext4 on top of NAND flash, in
> > order to use some fancy ext4 features. For this, I need an FTL in the
> > middle. I'm wondering which would be good to use. Options that I've
> > looked at are:
> > - ubiblock--the read-modify-write sounds unacceptable to me, even if
> > wear leveling and atomicity are handled.
>
> This may not give you the optimal performance.

My understanding of ubiblock is that, when you do 4k writes scattered
around the device, they will magnify into block-sized
read-modify-writes. This causes ~64x as much wear (depending on the
device type) and goes at 1/64th the speed. I don't think I could
change ubiblock to not have this property because that wouldn't be
backwards-compatible--I'd need an extra layer of indirection to
separate logical pages from logical erase blocks. Isn't ubiblock
usable for, say, copying off or on a ubifs filesystem image? I
wouldn't want to remove that functionality (and maybe I'll end up
using it). It just seems like ubiblock and a real FTL need totally
different pieces of software.
>
> > - Some coworkers have suggested a new effort to build a new block
> > device, but that that's a huge project and takes a long time to get
> > right.
>
> Yes.
>
> > - loopback-mounting a file on ubifs--From skimming the code, it looks
> > to me like ubifs uses some nice datastructures to handle writes within
> > a file without doing read-modify-writes all the time as ubiblock
> > forces. ubifs authors/maintainers, do you see any downside to using
> > ubifs this way?
>
> I never tried loopback over UBIFS. Should work in theory.
> Synchronization (as in 'fsync()') is something which comes to mind.
>
> What I mean is that when ext4 handles an fsync(), or a commit, it
> eventually needs to make sure the data goes to the underlying media. It
> sends REQ_FLUSH bios, and it seems that the loopback driver will map
> that to UBIFS's 'vfs_fsync()'. That should be fine, but may be too
> coarse? You could just write the right pages instead, to gain better
> performance?
>
> Then ext4 commit has various ordering requirements, and I do not know
> how this works with loopback.
>
> This all matters if you need to have some kind of power-cut tolerance.
>
> Thanks!
>
> --
> Best Regards,
> Artem Bityutskiy
>
These are good points that I hadn't thought about. I'm also worrying
about the overhead of loopback, but that's really outside of the scope
of UBIFS.

Thanks a lot everyone for this help! I think I'll push back on the
application requirements and just use UBIFS for now, and then in a
year or two if the requirements push back, either try to optimize
loopback and handle the issues Artem raises, or implement an FTL, or
port the fancy ext4 features to UBIFS. It's good to know that I'm not
missing some big piece of code here that would solve my problems.

Thanks,
Dan