[PATCH v6 04/41] arm64: mm: Move PCI I/O emulation region above the vmemmap region

[Ard Biesheuvel]

On Mon, 11 Dec 2023 at 14:57, Mark Rutland <mark.rutland at arm.com> wrote:
>
> Hi Ard,
>
> On Wed, Nov 29, 2023 at 12:16:00PM +0100, Ard Biesheuvel wrote:
> > From: Ard Biesheuvel <ardb at kernel.org>
> >
> > Move the PCI I/O region above the vmemmap region in the kernel's VA
> > space. This will permit us to reclaim the lower part of the vmemmap
> > region for vmalloc/vmap allocations when running a 52-bit VA capable
> > build on a 48-bit VA capable system.
> >
> > Signed-off-by: Ard Biesheuvel <ardb at kernel.org>
> > ---
> >  arch/arm64/include/asm/memory.h | 4 ++--
> >  arch/arm64/mm/ptdump.c          | 4 ++--
> >  2 files changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
> > index b8d726f951ae..99caeff78e1a 100644
> > --- a/arch/arm64/include/asm/memory.h
> > +++ b/arch/arm64/include/asm/memory.h
> > @@ -49,8 +49,8 @@
> >  #define MODULES_VSIZE                (SZ_2G)
> >  #define VMEMMAP_START                (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
> >  #define VMEMMAP_END          (VMEMMAP_START + VMEMMAP_SIZE)
> > -#define PCI_IO_END           (VMEMMAP_START - SZ_8M)
> > -#define PCI_IO_START         (PCI_IO_END - PCI_IO_SIZE)
> > +#define PCI_IO_START         (VMEMMAP_END + SZ_8M)
> > +#define PCI_IO_END           (PCI_IO_START + PCI_IO_SIZE)
> >  #define FIXADDR_TOP          (VMEMMAP_START - SZ_32M)
>
> This looks simple, but it does have a subtle dependency on VMEMMAP_SIZE being
> sufficiently smaller than the negative offset we choose for VMEMMAP_START, and
> it took mw a while to convince myself that we always ensure that today. I could
> well imagine that we try to reduce that in future (given for most
> configurations we're reserving around double the space we actually need), and
> it would be nice if we could avoid that going wrong.
>

Yes, we are relying on the fact that the upper region is not covered
by struct pages, so there is guaranteed to be an unused region at the
very top.

One of the constants in the equation that describes the size of this
unused region is the size of a struct page, and so I agree this needs
a couple of static asserts to ensure we don't get inadvertent
overlaps.


> We need to make sure that PCI_IO_START and PCI_IO_END fall between VMEMMAP_END
> and the last page of the VA space (due to ERR_PTR). Could we add some comment
> and/or assertions to that effect?
>
> Mark.
>
> >
> >  #if VA_BITS > 48
> > diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
> > index e305b6593c4e..d1df56d44f8a 100644
> > --- a/arch/arm64/mm/ptdump.c
> > +++ b/arch/arm64/mm/ptdump.c
> > @@ -47,10 +47,10 @@ static struct addr_marker address_markers[] = {
> >       { VMALLOC_END,                  "vmalloc() end" },
> >       { FIXADDR_TOT_START,            "Fixmap start" },
> >       { FIXADDR_TOP,                  "Fixmap end" },
> > -     { PCI_IO_START,                 "PCI I/O start" },
> > -     { PCI_IO_END,                   "PCI I/O end" },
> >       { VMEMMAP_START,                "vmemmap start" },
> >       { VMEMMAP_START + VMEMMAP_SIZE, "vmemmap end" },
> > +     { PCI_IO_START,                 "PCI I/O start" },
> > +     { PCI_IO_END,                   "PCI I/O end" },
> >       { -1,                           NULL },
> >  };
> >
> > --
> > 2.43.0.rc1.413.gea7ed67945-goog
> >