[PATCH v6 04/41] arm64: mm: Move PCI I/O emulation region above the vmemmap region

[Anshuman Khandual]

On 11/30/23 13:32, Ard Biesheuvel wrote:
> On Thu, 30 Nov 2023 at 08:59, Anshuman Khandual
> <anshuman.khandual at arm.com> wrote:
>>
>>
>>
>> On 11/29/23 16:46, Ard Biesheuvel wrote:
>>> From: Ard Biesheuvel <ardb at kernel.org>
>>>
>>> Move the PCI I/O region above the vmemmap region in the kernel's VA
>>> space. This will permit us to reclaim the lower part of the vmemmap
>>> region for vmalloc/vmap allocations when running a 52-bit VA capable
>>> build on a 48-bit VA capable system.
>>>
>>> Signed-off-by: Ard Biesheuvel <ardb at kernel.org>
>>> ---
>>>  arch/arm64/include/asm/memory.h | 4 ++--
>>>  arch/arm64/mm/ptdump.c          | 4 ++--
>>>  2 files changed, 4 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
>>> index b8d726f951ae..99caeff78e1a 100644
>>> --- a/arch/arm64/include/asm/memory.h
>>> +++ b/arch/arm64/include/asm/memory.h
>>> @@ -49,8 +49,8 @@
>>>  #define MODULES_VSIZE                (SZ_2G)
>>>  #define VMEMMAP_START                (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
>>>  #define VMEMMAP_END          (VMEMMAP_START + VMEMMAP_SIZE)
>>> -#define PCI_IO_END           (VMEMMAP_START - SZ_8M)
>>> -#define PCI_IO_START         (PCI_IO_END - PCI_IO_SIZE)
>>> +#define PCI_IO_START         (VMEMMAP_END + SZ_8M)
>>> +#define PCI_IO_END           (PCI_IO_START + PCI_IO_SIZE)
>>>  #define FIXADDR_TOP          (VMEMMAP_START - SZ_32M)
>>>
>>>  #if VA_BITS > 48
>>> diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
>>> index e305b6593c4e..d1df56d44f8a 100644
>>> --- a/arch/arm64/mm/ptdump.c
>>> +++ b/arch/arm64/mm/ptdump.c
>>> @@ -47,10 +47,10 @@ static struct addr_marker address_markers[] = {
>>>       { VMALLOC_END,                  "vmalloc() end" },
>>>       { FIXADDR_TOT_START,            "Fixmap start" },
>>>       { FIXADDR_TOP,                  "Fixmap end" },
>>> -     { PCI_IO_START,                 "PCI I/O start" },
>>> -     { PCI_IO_END,                   "PCI I/O end" },
>>>       { VMEMMAP_START,                "vmemmap start" },
>>>       { VMEMMAP_START + VMEMMAP_SIZE, "vmemmap end" },
>>> +     { PCI_IO_START,                 "PCI I/O start" },
>>> +     { PCI_IO_END,                   "PCI I/O end" },
>>>       { -1,                           NULL },
>>>  };
>>>
> 
> Hi Anshuman,
> 
>>
>> I have not debugged this any further, but the PCI region displays no entries
>> via ptdump just after this patch being applied.
>>
> 
> It also does not display any mappings without the patch. What exactly
> were you expecting to see here?

Is not there a single entry inside - without this patch section. OR does
it signify the gap-width of the PCI mapping area ?

---[ PCI I/O start ]---
0xfffffbfffe800000-0xfffffbffff800000          16M PMD
---[ PCI I/O end ]---

> 
>> With the patch:
>>
>> ---[ Fixmap end ]---
>> 0xfffffbfffe000000-0xfffffc0000000000          32M PMD
>> ---[ vmemmap start ]---
>> 0xfffffc0000000000-0xfffffc0002000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
>> 0xfffffc0002000000-0xfffffc0020000000         480M PMD
>> 0xfffffc0020000000-0xfffffc0022000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
>> 0xfffffc0022000000-0xfffffc0040000000         480M PMD
>> 0xfffffc0040000000-0xfffffc8000000000         511G PUD
>> 0xfffffc8000000000-0xfffffe0000000000        1536G PGD
>> ---[ vmemmap end ]---
>> 0xfffffe0000000000-0xfffffe8000000000         512G PGD
>> ---[ PCI I/O start ]---
>> ---[ PCI I/O end ]---                                   -----> No entries
>> 0xfffffe8000000000-0x0000000000000000        1536G PGD
>>
>> Without the patch:
>>
>> --[ Fixmap start ]---
>> 0xfffffbfffdc31000-0xfffffbfffddf6000        1812K PTE
>> 0xfffffbfffddf6000-0xfffffbfffddf9000          12K PTE       ro x  SHD AF            UXN    MEM/NORMAL
>> 0xfffffbfffddf9000-0xfffffbfffddfa000           4K PTE       ro NX SHD AF NG         UXN    MEM/NORMAL
>> 0xfffffbfffddfa000-0xfffffbfffddfd000          12K PTE
>> 0xfffffbfffddfd000-0xfffffbfffddfe000           4K PTE       RW NX SHD AF NG         UXN    DEVICE/nGnRE
>> 0xfffffbfffddfe000-0xfffffbfffde00000           8K PTE       ro NX SHD AF NG         UXN    MEM/NORMAL
>> 0xfffffbfffde00000-0xfffffbfffe000000           2M PTE
>> ---[ Fixmap end ]---
>> 0xfffffbfffe000000-0xfffffbfffe800000           8M PMD
>> ---[ PCI I/O start ]---
>> 0xfffffbfffe800000-0xfffffbffff800000          16M PMD
>> ---[ PCI I/O end ]---
>> 0xfffffbffff800000-0xfffffc0000000000           8M PMD
>> ---[ vmemmap start ]---
>> 0xfffffc0000000000-0xfffffc0002000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
>> 0xfffffc0002000000-0xfffffc0020000000         480M PMD
>> 0xfffffc0020000000-0xfffffc0022000000          32M PMD       RW NX SHD AF NG     BLK UXN    MEM/NORMAL
>> 0xfffffc0022000000-0xfffffc0040000000         480M PMD
>> 0xfffffc0040000000-0xfffffc8000000000         511G PUD
>> 0xfffffc8000000000-0xfffffe0000000000        1536G PGD
>> ---[ vmemmap end ]---