[RFC PATCH] riscv: reduce THREAD_SIZE from 16KB to 8KB for RV64

[Arnd Bergmann]

On Sun, Feb 6, 2022 at 6:43 PM Jisheng Zhang <jszhang at kernel.org> wrote:
>
> After irq stack is supported, it's possible to use small THREAD_SIZE.
> In fact, I tested this patch on a Lichee RV board, looks good so far.
>
> Signed-off-by: Jisheng Zhang <jszhang at kernel.org>

I can definitely see the value in this, in particular when you get hardware with
small RAM configurations that would run a 32-bit kernel on other architectures.

However, it's worth pointing out that all other 64-bit architectures use 16KB or
more, so it is rather dangerous to be the first architecture to try this out in
a long time. Some on-stack structures have a lot of pointers and 'unsigned long'
members, so they need twice the space, while other structures are the
same size either way.

IRQ stacks obviously help here, but I don't think the 8KB size can be made
the default without a lot of testing of some of the more rarely used code paths.

Here are a few things that would be worth doing on the way to a smaller
kernel stack:

- do more compile-time testing with a lower CONFIG_FRAME_WARN value.
  Historically, this defaults to 2048 bytes on 64-bit architectures. This is
  much higher than we really want, but it takes work to find and eliminate
  the outliers. I previously had a series to reduce the limit to 1280 bytes on
  arm64 and still build all 'randconfig' configurations.

- Use a much lower limit during regression testing. There is already a config
   option to randomize the start of the thread stack, but you can also try
   adding a configurable offset to see how far you can push it for a given
   workload before you run into the guard page.

- With vmap stacks, using 12KB may be an option as well, giving you
   three pages for the stack plus one guard page, instead of 4 pages
   stack plus four guard pages.

- If you can make a convincing case for using a lower THREAD_SIZE,
  make this a compile-time option across all 64-bit architectures that
  support both IRQ stacks and VMAP stacks. The actual frame size
  does depend on the ISA a bit, and we know that parisc and ia64 are
  particularly, possibly s390 as well, but I would expect risc-v to be
  not much different from arm64 and x86 here.

       Arnd