[LEDE-DEV] mission impossible!! add ubifs

[Daniel Golle]

Hi Kenneth,

On Tue, Aug 23, 2016 at 12:04:07PM +0200, Kenneth Johansson wrote:
> On 2016-08-23 11:40, Daniel Golle wrote:
> > On Tue, Aug 23, 2016 at 11:10:11AM +0200, Kenneth Johansson wrote:
> > > On 2016-08-23 07:31, Daniel Golle wrote:
> > > > > even this do not currently work. there is no UBI support of any kind for
> > > > > mt7621 and adding it was what I was asking about.
> > > > There are already devices using UBI on MT7621, just no method to flash
> > > > UBI using the 'factory'/stock method due to lacking UBI support in
> > > > the vendor-supplied U-Boot. UBI support is enabled on targets by
> > > > setting the 'nand' feature, see target/linux/ramips/mt7621/target.mk
> > > > You may also have a look at
> > > So I did a fresh clone. configured for mt7621. added this patch to be able
> > > to select ubi image generation.
> > You will need to add 'ubifs' to target/linux/ramips/mt7621/target.mk
> > rather than target/linux/ramips/Makefile.
> > You also need to specify the device-specific parameters which are
> > passed to mkfs.ubifs, e.g. something like
> > UBIFS_OPTS = -m 2048 -e 124KiB -c 4096
> > 
> > Making a mystery out of what you are trying to achieve doesn't help.
> > Especially not if I got the subtle feeling that you are trying to do
> > it in a very wrong way -- in the end, you are paid to build some
> > firmware for a device we'll later on have to live with and cope with
> > your image-building hacks. I know, uniqueness and originality are high
> > values in contemporary society. However, nobody is going to appreciate
> > very unique firmware image formats (unless you are selling security by
> > obscurity).
> hmm ???

Maybe telling us which board you are trying to port to would be a
start of a constructive debate of how to get OpenWrt/LEDE running on
the target environ you are facing.

> 
> >   Especially, please have a look at how it works for the
> > many other devices which use NAND flash and thus UBI(FS). It's really
> > not that different from what you want to do, I guess (but don't know,
> > because you are not telling anyone what you are trying to do).
> well I thought I said exactly what I wanted. I want a ubifs image. not a ubi
> image so ubinizer is not to be run on it.
> I want the kernel uImage in /boot and I want the DTB in/boot.

I realized that. The big question to me is still *why* you want that,
ie. what's the use-case or goal rather than the method you are
envisioning to achieve it.

> 
> that is all. there is no magic strange format its just a filesystem in a
> ubifs image.

...which needs to be initialized/written to the flash somehow and
updated, needs to be understood by the bootloader and so on. You'll
need a bunch of board-specific, custom scripts to handle that situation
and from OpenWrt/LEDE perspective this is just bloat which should be
avoided. Anyway, I'm probably just still lacking the complete picture.

> 
> > So please have a look at ipq806x, lantiq and oxnas image building and
> > sysupgrade script, all you are looking for can be found there.
> 
> But the issue is that I want to modify the mt7621. it feels like it should
> be really easy thing to do. And looking on other board is obviously the
> first thing I did. Still I have not managed to figure out what variable to
> set or what function to override.

Getting OpenWrt/LEDE to NAND-based MT7621 targets where you even got
the option to compile your own bootloader is quite easy, but not
trivial. Having a boot-*filesystem* is discouraged and thus only
supported on boards where there is no other option due to the vendor's
bootloader or stock-firmware-upgrade being limited.
The reason for that is that filesystems do need a lot more care than
partitions or UBI volumes, they can be out-of-free-space and compressed
filesystems like UBI make blurry statements about the free space still
available, because it depends on the assumed compression rate of future
things being written to it.
Also, a lot more can go wrong on all ends. Think: features like xattr
or different compression algos need to be supported by both, kernel and
U-Boot. Filenames may be super long or stored using various codepages,
and that doesn't sound like a very good idea in a small, static
real-time environment like U-Boot.
UBIFS also needs to know quite a lot about the specific flash chip used
on your board, which is why you have to supply those parameters to
mkfs.ubifs.
And it needs to be unmounted properly, power-loss during write can be
fatal. Simply: It's a very complex structure and 90% of that complexity
is unneeded to just load a single blob (eg. FIT, if your loader allows)
during boot.


> 
> But I can always just nuke the entire thing and make a complete new target.
> then I do not have to adopt anything to how it works right now. It's shame
> the build system is so abstracted that its no longer possible to understand.

If what you want is a UBIFS *rootfs* things are rather trivial. Just
add 'ubifs' to FEATURES in target/linux/ramips/mt7621/target.mk,
add a Device to target/linux/ramips/image/mt7621.mk which inhertis the
NAND template and make sure to include all UBIFS *and* NAND-specific
parameters -- you can just copy-paste them from a board with a similar
NAND chip from another target. Example to be added to
target/linux/ramips/image/mt7621.mk:

define Device/yourboard
  DTS := yourboard
  DEVICE_TITLE := Your Board
  DEVICE_PACKAGES := kmod-yourboarddriver
  BLOCKSIZE := 128KiB
  PAGESIZE := 2048
  SUBPAGESIZE := 512
  FILESYSTEMS += ubifs
  UBIFS_OPTS := -m 2048 -e 126KiB -c 4096
endef

I reckon you have read the Build/boot-overlay in
target/linux/imx6/image/Makefile which does create a filesystem image
(supposedly because that's what the bootloader expects) which is later
on written/replaced as-a-whole (but NEVER a file there gets modified,
there are NEVER any write operations on that /boot filesystem).

Now the bad news: Non-ubinized ubifs images are not supported by the
build system at this time, because there has never been a need for it.
Simiarly to how it's done on imx6, you'll have to roll-your-own build
template for that. And figure out how to flash it, upgrade it, ...

However, it'll all be much easier if you'd just share your intentions
(ie. "I want to run OpenWrt/LEDE on board X and need feature Y for
reason Z. I believe I could get that using tech Q." rather than just
"I want tech Q.")


Cheers


Daniel