[RFC] arm64: defconfig: enable 48-bit VA by default

[Stuart Yoder]

> -----Original Message-----
> From: Catalin Marinas [mailto:catalin.marinas at arm.com]
> >
[cut]
> > Yes, our physical memory layout for RAM looks like this:
> > 2 GB at 0x8000_0000
> > 510 GB at 0x80_8000_0000
> 
> So your platform currently only has 16GB of RAM. Shouldn't the last 14GB
> be placed at 34GB offset (according to the "Principles of ARM Memory
> Maps")?

Yes, but it looks like our hardware designer didn't follow it exactly.  And
instead created a variant by dropping the middle region.  The reasoning
seems to be that our SoC architecture is 40-bit PA only and thus it seemed
unnecessary to have the middle region at 34GB offset.  Plus, they wanted
that middle region for other I/O.

> > > To be honest, I think this is poorly designed, and I am not sure we
> > > should cater for such configurations in the defconfig.
> >
> > Agree, if this is a one-off weird platform then we shouldn't.
> >
> > But, the 'Principles of ARM Memory Maps' doc proposes this:
> > 2 GB at 0x8000_0000
> > 30 GB at 0x8_8000_0000
> > 480 GB at 0x88_0000_0000
> 
> I'm not particularly recommending this layout, at least not without some
> clarifications on DRAM aliases (I'll ping people internally about it
> again). The original layout pre-dates ARMv8, it was meant for ARMv7/LPAE
> and all the memory beyond 32-bit was highmem anyway. It was later
> updated for AArch64 but only to allow 44/48-bit PA (a few sections
> added).

Published guidance from ARM I think is important here.  If the aarch64
kernel maintainers have issues with the proposed map in the 'Principles' doc,
I consider that's a problem.

> How I understood it at the time is that in hardware you place the large
> DRAM (chip select) at high addresses, e.g. 64GB would be at a 512GB
> offset, but lower 2GB+30GB aliased in the lower space with the
> corresponding high address remaining a hole (that produces external
> aborts if accessed). The first alias at 2GB was meant for 32-bit
> initialisation code before the MMU is enabled. With AArch64, you don't
> really need this low alias (assuming your EL3/EL2/secure-EL1 code is
> 64-bit) since PC can use >32-bit addresses.

I think the intent with our SoC was to do that, but ommitting the
middle alias, because there was no obivous reason why it was needed.
 
> > ...i.e. if you have > 32 GB then your RAM regions are split into 3
> > chunks.  The aarch64 kernel will support > than 32GB right?  A
> > basic server will have that much or more.
> >
> > How will we deal with systems with > 32GB of memory that follow that
> > map?
> 
> My recommendation would be to avoid the lower alias (in hardware or some
> EL3 configuration change) and keep all the RAM at 0x80_0000_0000, just
> boot the system in AArch64 mode.

Unfortunately, for our existing armv8 silicon I am not aware any mechanism to
change the aliasing and keeping all RAM at 512GB offset.  So, we're stuck
for now with 2 discontiguous regions.  And things are ok as long as we
turn on 48-bit VA or 64K pages, but that's not the case with the default
defconfig.

My interest in the default defconfig is related to distro support.  When
a distro (Fedora, Debian, Ubuntu,etc) has a binary kernel image we want
it to run on our hardware.  I don't know to what degree the upstream
defconfig informs distros on what defconfig options are enabled in their
kernels...but we want to be as compatible as possible with the default
defconfig.

(BTW, Fedora 22 for aarch64 appears to have 64KB pages enabled, which
is good for our memory map)

Thanks,
Stuart