Finding kernel RAM consumers ?

[Arnd Bergmann]

On Fri, Jun 3, 2022 at 7:29 PM Matthew Wilcox <willy at infradead.org> wrote:
> On Fri, Jun 03, 2022 at 05:26:33PM +0000, Joakim Tjernlund wrote:
> > On Fri, 2022-06-03 at 08:49 +0200, Joakim Tjernlund wrote:
> > > On Thu, 2022-06-02 at 21:11 +0100, Matthew Wilcox wrote:
> > > > On Thu, Jun 02, 2022 at 07:24:23PM +0000, Joakim Tjernlund wrote:
> > > > > We have this small embedded target(aarch64) with 32 MB of RAM where the kernel consumes 14420K:
> > > > > Memory: 22444K/36864K available (3584K kernel code, 698K rwdata, 936K rodata, 320K init, 255K bss, 14420K reserved, 0K cma-reserved)
> > > > >
> > > > > I want to track down were most of this RAM is consumed so I can trim away some MBs
> > > > > but I am having a hard time finding may way.
> > > > > Is there some tool/kernel config that can help me with that?
> > >
> > > Those are interesting, thanks.
> > > In my case it is the amount of work space RAM allocated that is a bit much. The kernel code/data
> > > is 5+MB but the total need is 14MB, 9MB is buffer and similar.

I'm fairly sure you are in new territory here. The arm64 kernel is
just not really optimized
for small memory configurations. Even for 32-bit Arm, 32MB is
considered too small for
most workloads these days, though we do have some very limited support
for machines
with as little as 2MB of SRAM. The smallest arm64 machines that I can
think of being
supported in the kernel today have at least 256MB.

> > Found something, arm64 only supports mem model SPARSEMEM_EXTREME and it uses most of my memory.
> > Tried to force SPARSEMEM_STATIC but that didn't boot, is STATIC even cheaper ?
>
> That sounds like a question for the arm64 maintainers

>From what I understand, the 'static' variant uses a static allocation
in .bss, so that
may end up consuming most of your RAM even if it works normally. If all memory
is contiguous, then using flatmem may be more appropriate. You may also be able
to save some memory for sparsemem by reducing the size of the physical address
space. CONFIG_ARM64_PA_BITS is normally '48', but reducing it to '40' may
help (provided you can get that to boot). Most of the smaller CPUs only support
a 40 bit physical address space, so having another option for this is
probably sensible.

I think this is a case of "patches welcome". Nobody has really needed
this so far, but
as even the smaller machines are slowly migrating from 32-bit to
64-bit cores, optimizing
this will get interesting for more developers. There are probably
other low-hanging
fruit that you can address after figuring out.

One observation I made is that modern kernels don't seem to deal as
well as older
ones with low-memory situations, so even if you manage to free up most of your
32MB, it might still not work reliably.

       Arnd