SPARSEMEM memory needs?

[Joakim Tjernlund]

On Tue, 2022-06-07 at 09:48 +0200, Ard Biesheuvel wrote:
> On Tue, 7 Jun 2022 at 09:39, Joakim Tjernlund
> <Joakim.Tjernlund at infinera.com> wrote:
> > 
> > On Tue, 2022-06-07 at 09:15 +0200, Ard Biesheuvel wrote:
> ..
> > > 
> > > What you might try is changing the section size to 32 MB and mapping
> > > the vmemmap region down to pages. That way, the vmemmap region should
> > > only take up
> > > - 512 KiB for the struct page array[] itself
> > > - 4 KiB for the page table that replaces the 2 MB block mapping
> > > 
> > > You could try the below and see if it makes any difference?
> > > 
> > > diff --git a/arch/arm64/include/asm/sparsemem.h
> > > b/arch/arm64/include/asm/sparsemem.h
> > > index 4b73463423c3..a008f4342532 100644
> > > --- a/arch/arm64/include/asm/sparsemem.h
> > > +++ b/arch/arm64/include/asm/sparsemem.h
> > > @@ -23,7 +23,7 @@
> > >   * entries could not be created for vmemmap mappings.
> > >   * 16K follows 4K for simplicity.
> > >   */
> > > -#define SECTION_SIZE_BITS 27
> > > +#define SECTION_SIZE_BITS 25
> > >  #endif /* CONFIG_ARM64_64K_PAGES */
> > > 
> > >  #endif
> > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> > > index 5b1946f1805c..d25560a53a67 100644
> > > --- a/arch/arm64/mm/mmu.c
> > > +++ b/arch/arm64/mm/mmu.c
> > > @@ -1196,7 +1196,7 @@ static void free_empty_tables(unsigned long
> > > addr, unsigned long end,
> > >  }
> > >  #endif
> > > 
> > > -#if !ARM64_KERNEL_USES_PMD_MAPS
> > > +#if 1// !ARM64_KERNEL_USES_PMD_MAPS
> > >  int __meminit vmemmap_populate(unsigned long start, unsigned long
> > > end, int node,
> > >                 struct vmem_altmap *altmap)
> > >  {
> > 
> > That was a really good idea, now I have:
> >  Memory: 29732K/36864K available (3648K kernel code, 698K rwdata, 936K rodata, 320K init, 255K bss, 7132K reserved, 0K cma-reserved)
> > 
> > Reserved dropped from 14+MB to 7+MB :)
> > 
> 
> Glad to hear that.
> 
> > Should I look into something particular before testing this in a bigger scale?
> > 
> 
> A bit of stress testing would be useful, although I wouldn't be able
> to recommend anything that will be able to run with that little
> memory.
> 
> Just be aware that this is not a change that is likely to be accepted
> by the arm64 maintainers - it is something you will have to carry in
> your downstream.

In its current form, agreed.
If it could be generalized maybe, as is today aarch64 mandates lots of RAM
but your hack shows promise so maybe something will come of it.

> 
> Out of curiosity, what type of system is this? What user space are you using?
This is a very fancy custom ASIC, user space is a custom Yocto in 32 bits musl, mainly busybox
for utilities.

 Jocke