[PATCH v6 04/41] arm64: mm: Move PCI I/O emulation region above the vmemmap region

[Ard Biesheuvel]

On Thu, 30 Nov 2023 at 08:59, Anshuman Khandual
<anshuman.khandual at arm.com> wrote:
>
>
>
> On 11/29/23 16:46, Ard Biesheuvel wrote:
> > From: Ard Biesheuvel <ardb at kernel.org>
> >
> > Move the PCI I/O region above the vmemmap region in the kernel's VA
> > space. This will permit us to reclaim the lower part of the vmemmap
> > region for vmalloc/vmap allocations when running a 52-bit VA capable
> > build on a 48-bit VA capable system.
> >
> > Signed-off-by: Ard Biesheuvel <ardb at kernel.org>
> > ---
> >  arch/arm64/include/asm/memory.h | 4 ++--
> >  arch/arm64/mm/ptdump.c          | 4 ++--
> >  2 files changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
> > index b8d726f951ae..99caeff78e1a 100644
> > --- a/arch/arm64/include/asm/memory.h
> > +++ b/arch/arm64/include/asm/memory.h
> > @@ -49,8 +49,8 @@
> >  #define MODULES_VSIZE                (SZ_2G)
> >  #define VMEMMAP_START                (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
> >  #define VMEMMAP_END          (VMEMMAP_START + VMEMMAP_SIZE)
> > -#define PCI_IO_END           (VMEMMAP_START - SZ_8M)
> > -#define PCI_IO_START         (PCI_IO_END - PCI_IO_SIZE)
> > +#define PCI_IO_START         (VMEMMAP_END + SZ_8M)
> > +#define PCI_IO_END           (PCI_IO_START + PCI_IO_SIZE)
> >  #define FIXADDR_TOP          (VMEMMAP_START - SZ_32M)
> >
> >  #if VA_BITS > 48
> > diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
> > index e305b6593c4e..d1df56d44f8a 100644
> > --- a/arch/arm64/mm/ptdump.c
> > +++ b/arch/arm64/mm/ptdump.c
> > @@ -47,10 +47,10 @@ static struct addr_marker address_markers[] = {
> >       { VMALLOC_END,                  "vmalloc() end" },
> >       { FIXADDR_TOT_START,            "Fixmap start" },
> >       { FIXADDR_TOP,                  "Fixmap end" },
> > -     { PCI_IO_START,                 "PCI I/O start" },
> > -     { PCI_IO_END,                   "PCI I/O end" },
> >       { VMEMMAP_START,                "vmemmap start" },
> >       { VMEMMAP_START + VMEMMAP_SIZE, "vmemmap end" },
> > +     { PCI_IO_START,                 "PCI I/O start" },
> > +     { PCI_IO_END,                   "PCI I/O end" },
> >       { -1,                           NULL },
> >  };
> >

Hi Anshuman,

>
> I have not debugged this any further, but the PCI region displays no entries
> via ptdump just after this patch being applied.
>

It also does not display any mappings without the patch. What exactly
were you expecting to see here?

> With the patch:
>
> ---[ Fixmap end ]---
> 0xfffffbfffe000000-0xfffffc0000000000          32M PMD
> ---[ vmemmap start ]---
> 0xfffffc0000000000-0xfffffc0002000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
> 0xfffffc0002000000-0xfffffc0020000000         480M PMD
> 0xfffffc0020000000-0xfffffc0022000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
> 0xfffffc0022000000-0xfffffc0040000000         480M PMD
> 0xfffffc0040000000-0xfffffc8000000000         511G PUD
> 0xfffffc8000000000-0xfffffe0000000000        1536G PGD
> ---[ vmemmap end ]---
> 0xfffffe0000000000-0xfffffe8000000000         512G PGD
> ---[ PCI I/O start ]---
> ---[ PCI I/O end ]---                                   -----> No entries
> 0xfffffe8000000000-0x0000000000000000        1536G PGD
>
> Without the patch:
>
> --[ Fixmap start ]---
> 0xfffffbfffdc31000-0xfffffbfffddf6000        1812K PTE
> 0xfffffbfffddf6000-0xfffffbfffddf9000          12K PTE       ro x  SHD AF            UXN    MEM/NORMAL
> 0xfffffbfffddf9000-0xfffffbfffddfa000           4K PTE       ro NX SHD AF NG         UXN    MEM/NORMAL
> 0xfffffbfffddfa000-0xfffffbfffddfd000          12K PTE
> 0xfffffbfffddfd000-0xfffffbfffddfe000           4K PTE       RW NX SHD AF NG         UXN    DEVICE/nGnRE
> 0xfffffbfffddfe000-0xfffffbfffde00000           8K PTE       ro NX SHD AF NG         UXN    MEM/NORMAL
> 0xfffffbfffde00000-0xfffffbfffe000000           2M PTE
> ---[ Fixmap end ]---
> 0xfffffbfffe000000-0xfffffbfffe800000           8M PMD
> ---[ PCI I/O start ]---
> 0xfffffbfffe800000-0xfffffbffff800000          16M PMD
> ---[ PCI I/O end ]---
> 0xfffffbffff800000-0xfffffc0000000000           8M PMD
> ---[ vmemmap start ]---
> 0xfffffc0000000000-0xfffffc0002000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
> 0xfffffc0002000000-0xfffffc0020000000         480M PMD
> 0xfffffc0020000000-0xfffffc0022000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
> 0xfffffc0022000000-0xfffffc0040000000         480M PMD
> 0xfffffc0040000000-0xfffffc8000000000         511G PUD
> 0xfffffc8000000000-0xfffffe0000000000        1536G PGD
> ---[ vmemmap end ]---