[PATCH 2/4] ARM: Disable HIGHPTE on PREEMPT_RT kernels

[Arnd Bergmann]

TL;DR: I would still like to merge patch 3 and 4, but let's
keep that separate from the RT patches.

(just realized I had never sent my reply here)

On Wed, Dec 11, 2024, at 16:55, Russell King (Oracle) wrote:
> On Wed, Dec 11, 2024 at 03:04:02PM +0100, Sebastian Andrzej Siewior wrote:
>
> However, what is quite different is the L1 page tables. On x86,
> everything is nice and easy, and each page table is one 4k page.
> On 32-bit Arm, this is not the case - we need to grab a 16k page for
> the L1, and the more immovable allocations we have in lowmem, the
> harder it will be to satisfy this. Failing to grab a 16k page
> leads to fork() failing and an unusable system.
>
> So, we want to keep as many immovable allocations out of lowmem as
> possible - which is an additional constraint x86 doesn't have, and
> shouldn't be overlooked without ensuring that the probability of it
> happening remains acceptably low.

Right, non-LPAE systems indeed have a bigger problem with
fragmentation here, I hadn't thought of that.  Most of the
systems with 4GB RAM (and all of those with more) support
LPAE though, which means they can use the same page table
layout as x86 when when using CONFIG_LPAE=y.

Almost all SoCs prior to ARMv7VE are limited to 2GB of RAM
or less, the only exceptions I can think of are three SoCs
that can use close to 4GB (a small physical address range 
is used for MMIO):

- Calxeda highbank
- Hisilicon hip01
- NXP i.MX6D/DP/Q/QP

We can throw the first two under the bus, I'm sure nobody
cares enough, but the i.MX6Q was actually shipped in a
handful of products with 4GB that some users still have:

- Bunnie's Novena Laptop (856 original backers on crowdsupply)
- Solidrun CuBox i4x4 (not the more common i4P)
- Solidrun Hummingboard (early revisions only, later 2GB)

A couple of other i.MX6Q boards were advertised as supporting
"up to 4GB", but if you tried to buy those, the only ones in
stock anywhere seemed to be limited to 1 or 2 GB. Examples:

- armstone A9r4 (dts not upstream)
- DFI FS053 (dts not  upstream)
- VAR-SOM-MX6

[Part of the problem here apparently was availability
and cost of qualified 8Gbit DDR3 chips.]

Do you know of any other SoCs or boards in that category?

My feeling so far is that none of these are show-stoppers:

- The 4GB/8GB boards with LPAE don't have the added problem
  of fragmentation with order-2 page table allocations.
  They can run low on lowmem with CONFIG_VMSPLIT_3G, but
  as Sebastian cited, HIGHPTE is not likely to be the
  breaking point for them. These were mainly built around
  2012-2013 (before 64-bit became available) as high-end
  SoCs and are reaching the end of their commercial life.
- The 4GB boards without LPAE are also 10+ years old by now
  and were fairly rare even then. These would suffer the
  most from the fragmentation though.
- The 2GB boards with or without LPAE can theoretically
  avoid HIGHMEM entirely by using CONFIG_VMSPLIT_2G_OPT.
  These are still very common today, but at least a third
  of the total memory is going to be available as lowmem
  even with VMSPLIT_3G.

The case that is less clear to me is the one where memory
is sparse enough to lead to exhausting lowmem without
having a lot of total RAM. These are a little harder
to find, but I think e.g. Renesas has some chips that
need this, and Realview PBX had a custom __phys_to_virt
in earlier versions to work around this. This should
be solved when we get the "densemem" support that Linus
mentioned.

     Arnd