Confused about how to make a UBIFS root filesystem

Sat Apr 12 05:00:05 EDT 2008

Hi,

On Fri, 2008-04-11 at 15:06 -0700, Bruce_Leonard at selinc.com wrote:
> I've gotten all the tools I'm just not sure what order to use them in and 
> on what devices/files.  If I understand it correctly, I use mkfs.ubifs on 
> a filesystem that I want to ultimately be my rfs and output that to 
> someother file (say ubi_rootfs.img).  I then process that through ubinize 
> which produces a config file (say ubi_rootfs.cfg).  I then use 
> ubiupdatevol to write ubi_rootfs.img to the NAND using ubi_rootfs.cfg to 
> control the parameters of how the image gets written.  Where I get lost 
> (if I'm not already lost) is how do I prep the NAND device to receive the 
> image?  Do I use ubimkvol or mkfs.ubifs?  I've looked through the FAQ at 
> infradead and if I knew more about filesystems and the kernel it might all 
> make sense, but I'm afraid I'm one of those folks that has to have his 
> nose rubbed in it to understand, kind of like an irish setter.  Any help 
> or pointers to good references are greatly appreciated.  Thanks.

Well, it seems I did not explain things too at the MTD web site.

As you know, UBIFS works on top of UBI, which works on top of MTD. So
this is kind of a stack. You have your mtd device (e.g., /dev/mtd0),
which is basically bare NAND flash. It may has eraseblocks, it may have
bad blocks, etc.

First lets talk about JFFS2, to make a better picture of what is going
on. JFFS2 works on top of MTD devices. When you prepare JFFS2 images,
you use mkfs.jffs2, and then put them to MTD devices with help of
"nandwrite" utility (you cannot use "dd" because it is unable to skip
bad eraseblocks). Also, JFFS2 does not _require_ you to prepare any
image, you may just erase your flash using flash_eraseall, and mount it,
using an "mount -t jffs2 mtd0 /mnt/jffs2" command. In this case JFFS2
automatically detects that the flash is empty, and automatically formats
it. So when you mount it, you end up with an empty file-system. Just
note, UBI/UBIFS can do the same. So far so good and simple, right?

Now let's talk a bit about UBI/UBIFS. As I said, UBI works on top of
MTD. When you load UBI and attach an MTD device to it, you end up with
an UBI device, e.g. /dev/ubi0. For example, if you have /dev/mtd0, you
may attach it to UBI using "ubiattach" utility, and /dev/ubi0 will be
created (if you have udev in your system). There is no much you can do
with UBI device (/dev/ubi0), though. Most interesting things may be done
with UBI _volumes_. You may create UBI volumes on your UBI device using
the "ubimkvol" tool. For example, you may create a /dev/ubi0_0 volume.
You may create many volume on one UBI device. Then you may delete them
or re-size. There are tools for this.

And of course you could have 2 flashes on your board, say NOR and NAND,
so you would have 2 MTD devices (e.g., /dev/mtd0 and /dev/mtd1). You
could attach them to UBI, which would and result give you /dev/ubi0
and /dev/ubi1. You could create volumes on these UBI devices,
e.g. /dev/ubi0_0, /dev/ubi0_1, /dev/ubi0_2,
or /deve/ubi1_0, /dev/ubi1_1, etc. The first number id the UBI device
number, the second one is volume number within this UBI device.

So /dev/ubi0 sits on top of /dev/mtd0 and actually uses whole /dev/mtd0.
You should never write anything directly to /dev/mtd0 if you have UBI on
top of it. But reading it is harmless, although it might only be needed
when debugging or doing something hacky. UBI has its internal
data-structures which is stores on flash, for example erase counters and
volume table. So if you read /dev/mtd0, you will read those internal
data structures as well.

Now UBIFS. It works on top of UBI volumes, like /dev/ubi0_0. You may
mount /dev/ubi0_0 using "mount -t ubifs ubi0_0 /mnt/ubifs". Well, UBI
volumes may have names, which you may specify when you create them, so
you may also mount volumes by their names, e.g.: "mount -t ubifs
ubi0:i_like_free_beer /mnt/ubifs". UBIFS of course has its own internal
data structures which it stores at the UBI volume, for example the
superblock, the file-system indexing B-tree, etc.

Now tools. So you have your bare flash, your poor /dev/mtd0, which only
gives you read/write/erase operations, which have bad eraseblocks, which
requires you to spread all I/O load evenly, and which have other
restrictions (horrible thing, isn't it?). And you want to have a normal
file-system on top of it, which is doing all difficult job for you, and
you just store your data precious files and do not even want to know
about the complex things which being done inside. :-)

Sou have your "/home/bruce/root_img/" directory, where you store the
root file-system for your device, and you want to create a raw image,
which you will send to the factory, and they would simply put this raw
image to flash of the newly produced cool devices which will conquer the
whole target market segment.

To do this, you first create an UBIFS image (step 1) using the
"mkfs.ubifs" utility (see FAQ for options). It gives you, say,
"ubifs.img" file. This file is basically the contents of the _UBI
volume_, _not_ contents of the raw flash. I mean, if you put "ubifs.img"
to a /dev/ubi0_0 (which exactly is what "ubiupdatevol" does), you may
then mount this /dev/ubi0_0 and see exactly the same as in
"/home/bruce/root_img/". But you _cannot_ put this image to
your /dev/mtd0 device. It simply won't work - you have to turn ubifs.img
to an UBI image first!. And this is where you use the "ubinize" utility.
This utility creates UBI images. It requires a configuration file (which
you found confusing), where you describe the parameters of your NAND
flash, the volumes which "ubinize" has to create, the images which it
should put to these volumes, etc. So, if you use "ubinize", you'll get,
say, ubi.img file (see FAQ for details and examples), which now you may
send to the factory. If you wipe out your /dev/mtd0, then put ubi.img on
it, then attach it to UBI - you'll see a volume on it. This is the
volume which you described in the configuration file of "ubinize". If
you mount that volume, you'll find your "/home/bruce/root_img/", because
you said "ubinize" to put ubifs.img to the created volume.

Why "ubinize" need a config file? Because you may want to create an UBI
image with _many_ volumes, and put different contents to each of them.
E.g., have "/home" and "/" on separate UBI volumes. Each volume may have
different characteristics. It is difficult to pass all the needed
parameters for complex configurations via command line, so we just use a
configuration file.

And finally, similarly to JFFS2, UBI/UBIFS do not require you to prepare
any images. You may start with an empty flash, attach it to UBI, in
which case UBI will format it automatically, then create a volume, which
will be empty (i.e., dd if=/dev/ubi0_0 of=file will give you only 0xFF
bytes). Then you may mount empty this volume, and UBIFS will
automatically format it, and you will end up with an empty FS.

I think, after reading this, and re-reading
http://www.linux-mtd.infradead.org/faq/ubifs.html#L_mkfubifs, you'll
have complete picture. Sorry if I put made the mail too long, instead of
just answer your questions with short answers.

I know that my English is far from perfect, so you could help and
contribute to the project by creating few FAQ entries for us. I'd put
them to the MTD web site. I mean, just describe the things you found
confusing and difficult to grasp, give them a title (e.g., "What is this
<name_of_the_confusing_thing> about?") and send it to us. That would be
great.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)