[LEDE-DEV] Recent issues w/ sysupgrade

[Philip Prindeville]

> On Aug 29, 2017, at 9:16 AM, Philip Prindeville <philipp_subx at redfish-solutions.com> wrote:
> 
>> On Aug 29, 2017, at 1:19 AM, Stijn Tintel <stijn at linux-ipv6.be> wrote:
>> 
>>> On 29-08-17 09:09, Philip Prindeville wrote:
>>> Hi all,
>>> 
>>> I don’t know if sysupgrade is the problem, or if this is where things manifest.
>>> 
>>> But I recently (within the last week, but I only rebase once or twice a week) started seeing issues with doing sysupgrade on x86_64 hardware.
>>> 
>>> The sysupgrade will appear to go okay, but then when the machine reboots, I see:
>>> 
>>> ...
>>> Press the [f] key and hit [enter] to enter failsafe mode
>>> Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
>>> [    6.322701] mount_root: loading kmods from internal overlay
>>> [    6.331183] kmodloader: loading kernel modules from //etc/modules-boot.d/*
>>> [    6.339194] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
>>> [    6.470650] block: attempting to load /etc/config/fstab
>>> [    6.483431] block: extroot: not configured
>>> [    6.488267] random: procd: uninitialized urandom read (4 bytes read)
>>> [    6.506070] mount_root: rootdisk overlay filesystem has not been formatted yet
>>> 
>>> 
>>> Is this related to the mk.f2f changes on Thursday?
>> I doubt this will be related. The only real change is
>> https://git.lede-project.org/cdb494fd, and on little endian systems
>> cpu_to_le64 is a noop. Additionaly I tested this on x86/64 and didn't
>> notice any isses. Just look a bit further and you should see this:
>> 
>> Tue Aug 29 07:12:57 2017 user.info kernel: [    5.607837] mount_root:
>> overlay filesystem has not been fully initialized yet
>> Tue Aug 29 07:12:57 2017 user.info kernel: [    5.608986] mount_root:
>> switching to f2fs overlay
>> 
>> root at LEDE:~# mount  | grep overlay
>> /dev/loop0 on /overlay type f2fs
>> (rw,lazytime,noatime,background_gc=on,user_xattr,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6)
>> overlayfs:/overlay on / type overlay
>> (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
>> 
>> Stijn
> 
> It never gets to that point.
> 
> The message about it not being formatted is the last output then it hangs.
> 


I do not see that (the “switching to f2fs overlay” message) on the console when the hang happens at reboot following a sysupgrade (and sometimes even just reflashing the CF card).

I have noted, when sysupgrade runs, I do see:


[  379.758026] EXT4-fs (sda1): mounted filesystem without journal. Opts: (null)
Upgrade completed
Rebooting system...
umount: can't unmount /dev: Resource busy
umount: can't unmount /tmp: Resource busy


but I was seeing those before.

So, some progress on the bi-section.

I can rebase to here:

af922c0 mediatek: update to latest kernel patchset from v4.13-rc
3ec259a procd: fix hotplug-preinit.json syntax
1a3b17a procd: fix hotplug.json syntax
a678eb0 ca-certificates: Update to 20170717
8e16e19 imx6: refresh kernel config
c0d5990 imx6: add driver for temp/voltage monitoring
2deed55 uboot-lantiq: Enable TFTP PUT support for backups
47dcfcf gpio-button-hotplug: leave platform_device.dev.platform_data untouched

but then I need to cherry-pick this fix so things will build:

dfee19f mediatek: drop kernel dep on userland module

If I try to rebase any further, to:

e622b30 netifd: update to latest git HEAD

then I see the flakiness at boot time.  John or Felix, are there any changes here that would affect booting if the way described above?

-Philip