Finding kernel RAM consumers ?

[Alexander Dahl]

Hei hei,

Am Fri, Jun 03, 2022 at 08:11:31PM +0200 schrieb Arnd Bergmann:
> On Fri, Jun 3, 2022 at 7:29 PM Matthew Wilcox <willy at infradead.org> wrote:
> > On Fri, Jun 03, 2022 at 05:26:33PM +0000, Joakim Tjernlund wrote:
> > > On Fri, 2022-06-03 at 08:49 +0200, Joakim Tjernlund wrote:
> > > > On Thu, 2022-06-02 at 21:11 +0100, Matthew Wilcox wrote:
> > > > > On Thu, Jun 02, 2022 at 07:24:23PM +0000, Joakim Tjernlund wrote:
> > > > > > We have this small embedded target(aarch64) with 32 MB of RAM where the kernel consumes 14420K:
> > > > > > Memory: 22444K/36864K available (3584K kernel code, 698K rwdata, 936K rodata, 320K init, 255K bss, 14420K reserved, 0K cma-reserved)
> > > > > >
> > > > > > I want to track down were most of this RAM is consumed so I can trim away some MBs
> > > > > > but I am having a hard time finding may way.
> > > > > > Is there some tool/kernel config that can help me with that?
> > > >
> > > > Those are interesting, thanks.
> > > > In my case it is the amount of work space RAM allocated that is a bit much. The kernel code/data
> > > > is 5+MB but the total need is 14MB, 9MB is buffer and similar.
> 
> I'm fairly sure you are in new territory here. The arm64 kernel is
> just not really optimized
> for small memory configurations. Even for 32-bit Arm, 32MB is
> considered too small for
> most workloads these days, though we do have some very limited support
> for machines
> with as little as 2MB of SRAM. The smallest arm64 machines that I can
> think of being
> supported in the kernel today have at least 256MB.
> 
> > > Found something, arm64 only supports mem model SPARSEMEM_EXTREME and it uses most of my memory.
> > > Tried to force SPARSEMEM_STATIC but that didn't boot, is STATIC even cheaper ?
> >
> > That sounds like a question for the arm64 maintainers
> 
> From what I understand, the 'static' variant uses a static allocation
> in .bss, so that
> may end up consuming most of your RAM even if it works normally. If all memory
> is contiguous, then using flatmem may be more appropriate. You may also be able
> to save some memory for sparsemem by reducing the size of the physical address
> space. CONFIG_ARM64_PA_BITS is normally '48', but reducing it to '40' may
> help (provided you can get that to boot). Most of the smaller CPUs only support
> a 40 bit physical address space, so having another option for this is
> probably sensible.
> 
> I think this is a case of "patches welcome". Nobody has really needed
> this so far, but
> as even the smaller machines are slowly migrating from 32-bit to
> 64-bit cores, optimizing
> this will get interesting for more developers. There are probably
> other low-hanging
> fruit that you can address after figuring out.

The SiP variants of at91 SAMA5D2 (armv7) or SAM9x60 (armv5) come with
64 MiB or 128 MiB, and given the latter is a new SoC announced only
two or three years ago, requiring at least 256 MiB would be at best
unfortunate.  Given those SoCs are used in industrial applications
with very long support times, I think 32bit ARM will stay for years,
even with new products.

> One observation I made is that modern kernels don't seem to deal as
> well as older
> ones with low-memory situations, so even if you manage to free up most of your
> 32MB, it might still not work reliably.

Since we are using the SAMA5D27C-D5M in production, I would also be
interested in support for running 32 bit ARM with recent kernels on
systems with 64 MiB or even 32 MiB of memory.  If there are specific
things to test, you can let me know.

Greets
Alex