[arm64] kernel boot slowdown in v5.10.19 -> v5.10.42 update

[Vijay Balakrishna]

On 1/28/2022 10:03 AM, Vijay Balakrishna wrote:
> 
> 
> On 1/28/2022 4:06 AM, Catalin Marinas wrote:
>> On Mon, Jan 24, 2022 at 03:03:48PM -0800, Vijay Balakrishna wrote:
>>> We noticed 150ms kernel boot slowdown back in June, 2021, when moving 
>>> from
>>> v5.10.19 to v5.10.42.  This on a 8GB SoC.  Only recently we investigated
>>> this issue and found the regression is introduced by a change in 
>>> map_mem()
>>> (paging_init() -> map_mem() -> __map_memblock(), in particular "map 
>>> all the
>>> memory banks" for loop) by patch
>>>
>>> 2687275a5843d1089687f08fc64eb3f3b026a169
>>> arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
>>>
>>> above is a follow up to
>>>
>>> 0a30c53573b07d5561457e41fb0ab046cd857da5
>>> arm64: mm: Move reserve_crashkernel() into mem_init())
>>>
>>> which deferred crashkernel reservation into mem_init().
>>>
>>> The ~150ms slowdown disappears on booting without "crashkernel=.." on 
>>> kernel
>>> command-line.
>>
>> Is CONFIG_RODATA_FULL_DEFAULT_ENABLED set in your config file? It may
>> cause the same slowdown. I suspect it's because we end up mapping the
>> RAM at the page granularity so more loops and slightly higher TLB
>> pressure. Not sure we can do much about >
> 
> Catalin,
> 
> Thanks for your reply.
> 
> CONFIG_RODATA_FULL_DEFAULT_ENABLED is not set in our config for boot 
> time performance reasons.

 From following patches, the main source of deferring crash kernel 
reservation is for Raspberry Pi 4, which has no IOMMU.

1a8e1cef7603e218339ac63cb3178b25554524e5
arm64: use both ZONE_DMA and ZONE_DMA32

8424ecdde7df99d5426e1a1fd9f0fb36f4183032
arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges

0a30c53573b07d5561457e41fb0ab046cd857da5
arm64: mm: Move reserve_crashkernel() into mem_init())

2687275a5843d1089687f08fc64eb3f3b026a169
arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required

However above changes introduced slow boot (regression) for platforms 
with IOMMU, for example as I mentioned earlier ~150ms for us.  We don't 
have following configs set for our SoC --

# CONFIG_ZONE_DMA is not set
# CONFIG_ZONE_DMA32 is not set

Can we restore crash kernel reservation (no deferring to mem_init()) for 
platforms with no DMA memory zones?

I can submit a patch.

Thanks,
Vijay