[PATCH v6 04/41] arm64: mm: Move PCI I/O emulation region above the vmemmap region

[Anshuman Khandual]

On 11/29/23 16:46, Ard Biesheuvel wrote:
> From: Ard Biesheuvel <ardb at kernel.org>
> 
> Move the PCI I/O region above the vmemmap region in the kernel's VA
> space. This will permit us to reclaim the lower part of the vmemmap
> region for vmalloc/vmap allocations when running a 52-bit VA capable
> build on a 48-bit VA capable system.
> 
> Signed-off-by: Ard Biesheuvel <ardb at kernel.org>
> ---
>  arch/arm64/include/asm/memory.h | 4 ++--
>  arch/arm64/mm/ptdump.c          | 4 ++--
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
> index b8d726f951ae..99caeff78e1a 100644
> --- a/arch/arm64/include/asm/memory.h
> +++ b/arch/arm64/include/asm/memory.h
> @@ -49,8 +49,8 @@
>  #define MODULES_VSIZE		(SZ_2G)
>  #define VMEMMAP_START		(-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
>  #define VMEMMAP_END		(VMEMMAP_START + VMEMMAP_SIZE)
> -#define PCI_IO_END		(VMEMMAP_START - SZ_8M)
> -#define PCI_IO_START		(PCI_IO_END - PCI_IO_SIZE)
> +#define PCI_IO_START		(VMEMMAP_END + SZ_8M)
> +#define PCI_IO_END		(PCI_IO_START + PCI_IO_SIZE)
>  #define FIXADDR_TOP		(VMEMMAP_START - SZ_32M)
>  
>  #if VA_BITS > 48
> diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
> index e305b6593c4e..d1df56d44f8a 100644
> --- a/arch/arm64/mm/ptdump.c
> +++ b/arch/arm64/mm/ptdump.c
> @@ -47,10 +47,10 @@ static struct addr_marker address_markers[] = {
>  	{ VMALLOC_END,			"vmalloc() end" },
>  	{ FIXADDR_TOT_START,		"Fixmap start" },
>  	{ FIXADDR_TOP,			"Fixmap end" },
> -	{ PCI_IO_START,			"PCI I/O start" },
> -	{ PCI_IO_END,			"PCI I/O end" },
>  	{ VMEMMAP_START,		"vmemmap start" },
>  	{ VMEMMAP_START + VMEMMAP_SIZE,	"vmemmap end" },
> +	{ PCI_IO_START,			"PCI I/O start" },
> +	{ PCI_IO_END,			"PCI I/O end" },
>  	{ -1,				NULL },
>  };
>  

I have not debugged this any further, but the PCI region displays no entries
via ptdump just after this patch being applied.

With the patch:

---[ Fixmap end ]---
0xfffffbfffe000000-0xfffffc0000000000          32M PMD
---[ vmemmap start ]---
0xfffffc0000000000-0xfffffc0002000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
0xfffffc0002000000-0xfffffc0020000000         480M PMD
0xfffffc0020000000-0xfffffc0022000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
0xfffffc0022000000-0xfffffc0040000000         480M PMD
0xfffffc0040000000-0xfffffc8000000000         511G PUD
0xfffffc8000000000-0xfffffe0000000000        1536G PGD
---[ vmemmap end ]---
0xfffffe0000000000-0xfffffe8000000000         512G PGD
---[ PCI I/O start ]---				
---[ PCI I/O end ]---					-----> No entries
0xfffffe8000000000-0x0000000000000000        1536G PGD

Without the patch:

--[ Fixmap start ]---
0xfffffbfffdc31000-0xfffffbfffddf6000        1812K PTE
0xfffffbfffddf6000-0xfffffbfffddf9000          12K PTE       ro x  SHD AF            UXN    MEM/NORMAL
0xfffffbfffddf9000-0xfffffbfffddfa000           4K PTE       ro NX SHD AF NG         UXN    MEM/NORMAL
0xfffffbfffddfa000-0xfffffbfffddfd000          12K PTE
0xfffffbfffddfd000-0xfffffbfffddfe000           4K PTE       RW NX SHD AF NG         UXN    DEVICE/nGnRE
0xfffffbfffddfe000-0xfffffbfffde00000           8K PTE       ro NX SHD AF NG         UXN    MEM/NORMAL
0xfffffbfffde00000-0xfffffbfffe000000           2M PTE
---[ Fixmap end ]---
0xfffffbfffe000000-0xfffffbfffe800000           8M PMD
---[ PCI I/O start ]---
0xfffffbfffe800000-0xfffffbffff800000          16M PMD
---[ PCI I/O end ]---
0xfffffbffff800000-0xfffffc0000000000           8M PMD
---[ vmemmap start ]---
0xfffffc0000000000-0xfffffc0002000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
0xfffffc0002000000-0xfffffc0020000000         480M PMD
0xfffffc0020000000-0xfffffc0022000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
0xfffffc0022000000-0xfffffc0040000000         480M PMD
0xfffffc0040000000-0xfffffc8000000000         511G PUD
0xfffffc8000000000-0xfffffe0000000000        1536G PGD
---[ vmemmap end ]---