[LEDE-DEV] bcm53xx: MD5 checksum over too few bytes of firmware breaks kernel booting

[Rafał Miłecki]

My DIR-885L has following partitions:

[    1.034740] Creating 1 MTD partitions on "brcmnand.0":
[    1.039896] 0x000000000000-0x000008000000 : "firmware"
[    1.075568] 2 seama-fw partitions found on MTD device firmware
[    1.081429] 0x000000000040-0x000000400000 : "kernel"
[    1.086983] 0x000000400000-0x000008000000 : "ubi"

As you can guess kernel's content doesn't change and ubi's content
does. It means we should calculate firmware checksum over 0x40000 -
0x40 = 0x3fffc0 bytes.

Unfortunately after executing:
mtd -c 0x3fffc0 fixseama firmware
and rebooting, LEDE's kernel doesn't boot anymore. It hangs as the
last CFE message:
Starting program at 0x00008000

Executing any of following:
mtd -c 0x73ffc0 fixseama firmware
mtd -c 0x72ffc0 fixseama firmware
mtd -c 0x70ffc0 fixseama firmware
mtd -c 0x4fffc0 fixseama firmware
mtd -c 0x4bffc0 fixseama firmware
mtd -c 0x4affc0 fixseama firmware
mtd -c 0x49ffc0 fixseama firmware
mtd -c 0x48ffc0 fixseama firmware
doesn't break anything.

Executing any following:
mtd -c 0x47ffc0 fixseama firmware
mtd -c 0x46ffc0 fixseama firmware
mtd -c 0x45ffc0 fixseama firmware
mtd -c 0x3fffc0 fixseama firmware
breaks kernel booting.

CFE never complains about image, MD5 is always OK:
  image size : 4194240 bytes
verify_seama: signature=[(null)], type=[firmware]
  checksum   : FFC3A82405FB59C67825A0BEAE79BAA2
  digest     : FFC3A82405FB59C67825A0BEAE79BAA2

I also dumped 0x400000 B of firmware data after both commands:
mtd -c 0x48ffc0 fixseama firmware (WORKING)
mtd -c 0x47ffc0 fixseama firmware (NOT WORKING)
and the only difference is size and MD5 in a header.

-- 
Rafał