[PATCH v3 13/13] arm64: mm: Provide level hint for flush_tlb_page()

[Ryan Roberts]

Previously tlb invalidations issued by __flush_tlb_page() did not
contain a level hint. But the function is clearly only ever targeting
level 3 tlb entries and its documentation agrees:

  | this operation only invalidates a single, last-level page-table
  | entry and therefore does not affect any walk-caches

However, it turns out that the function was actually being used to
invalidate a level 2 mapping via flush_tlb_fix_spurious_fault_pmd(). The
bug was benign because the level hint was not set so the HW would still
invalidate the PMD mapping, and also because the TLBF_NONOTIFY flag was
set, the bounds of the mapping were never used for anything else.

Now that we have the new and improved range-invalidation API, it is
trival to fix flush_tlb_fix_spurious_fault_pmd() to explicitly flush the
whole range (locally, without notification and last level only). So
let's do that, and then update __flush_tlb_page() to hint level 3.

Reviewed-by: Linu Cherian <linu.cherian at arm.com>
Signed-off-by: Ryan Roberts <ryan.roberts at arm.com>
---
 arch/arm64/include/asm/pgtable.h  | 5 +++--
 arch/arm64/include/asm/tlbflush.h | 2 +-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 7039931df4622..b1a96a8f2b17e 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -103,8 +103,9 @@ static inline void arch_leave_lazy_mmu_mode(void)
 #define flush_tlb_fix_spurious_fault(vma, address, ptep)	\
 	__flush_tlb_page(vma, address, TLBF_NOBROADCAST | TLBF_NONOTIFY)
 
-#define flush_tlb_fix_spurious_fault_pmd(vma, address, pmdp)	\
-	__flush_tlb_page(vma, address, TLBF_NOBROADCAST | TLBF_NONOTIFY)
+#define flush_tlb_fix_spurious_fault_pmd(vma, address, pmdp)			\
+	__flush_tlb_range(vma, address, address + PMD_SIZE, PMD_SIZE, 2,	\
+			  TLBF_NOBROADCAST | TLBF_NONOTIFY | TLBF_NOWALKCACHE)
 
 /*
  * ZERO_PAGE is a global shared page that is always zero: used
diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
index 5096ec7ab8650..958fe97b744e5 100644
--- a/arch/arm64/include/asm/tlbflush.h
+++ b/arch/arm64/include/asm/tlbflush.h
@@ -591,7 +591,7 @@ static inline void __flush_tlb_page(struct vm_area_struct *vma,
 	unsigned long start = round_down(uaddr, PAGE_SIZE);
 	unsigned long end = start + PAGE_SIZE;
 
-	__do_flush_tlb_range(vma, start, end, PAGE_SIZE, TLBI_TTL_UNKNOWN,
+	__do_flush_tlb_range(vma, start, end, PAGE_SIZE, 3,
 			     TLBF_NOWALKCACHE | flags);
 }
 
-- 
2.43.0