[PATCH] [RFC] arm64: mmu: use range based TLB flushing when hot unplugging memory

[Alistair Popple]

Hot unplugging memory on ARM64 requires a TLB invalidate after unmapping
the page to be hot unplugged from the direct map. Currently that happens
one page at a time, meaning range based invalidates cannot be used. The
result of this is that removing large amounts of memory takes a long
time and in some cases can trigger an RCU stall warning.

For example on one system hot unplugging 480GB of memory takes ~1
minute. With this change the same operation took ~1 second, a 60x
improvement.

Signed-off-by: Alistair Popple <apopple at nvidia.com>

---

This is an RFC, because I'm not sure the change is correct as it frees
the PTE page before flushing the TLB. I'm not familiar enough with ARM64
architecture to be sure this is safe, for example I don't know if HW
can update PTE bits such as access/dirty in the page through a stale
TLB entry.

If so this would open a window during which the page is free but could
still be written to. Likely the safe option would be to collect all the
pages to be free on a list and free them after doing the range based TLB
flush, but wanted to get feedback on the approach before implementing it
which is the goal of this RFC.
---
 arch/arm64/mm/mmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 0c24fe650e95..75c773232c14 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -1459,11 +1459,12 @@ static void unmap_hotplug_pte_range(pmd_t *pmdp, unsigned long addr,
 
 		WARN_ON(!pte_present(pte));
 		__pte_clear(&init_mm, addr, ptep);
-		flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
 		if (free_mapped)
 			free_hotplug_page_range(pte_page(pte),
 						PAGE_SIZE, altmap);
 	} while (addr += PAGE_SIZE, addr < end);
+
+	flush_tlb_kernel_range(addr, end);
 }
 
 static void unmap_hotplug_pmd_range(pud_t *pudp, unsigned long addr,
-- 
2.54.0