[PATCH v2 3/9] mm/huge_memory: honor MM_CP_TRY_CHANGE_WRITABLE in change_huge_pud()

[Carlos López]

When change_protection() is called on a range whose vma->vm_page_prot
does not contain the write bit (MAP_PRIVATE mappings, which need CoW
handling, or MAP_SHARED mappings that want write tracking), the new
entries are created without the write bit, letting the fault handler
re-introduce it. In such cases, MM_CP_TRY_CHANGE_WRITABLE is set to
eagerly restore the W bit in the page table entries when stripping the
write protection was not needed.

However, change_huge_pud() does not currently respect the flag. The only
user of huge PUDs, device DAX, is never CoW and does not need write
notifications for dirty tracking, but it does get them once soft-dirty
tracking is armed through /proc/pid/clear_refs. vm_page_prot then lacks
the write bit, and a protection change leaves dirty PUDs read-only, so
the next write takes a fault only to set the write bit again.

Align change_huge_pud() with change_pte_range() and change_huge_pmd() by
making it respect MM_CP_TRY_CHANGE_WRITABLE. Besides avoiding those
faults, this keeps huge PUDs consistent with the other levels if PUDs
that need CoW, such as anonymous ones, are ever introduced. They would
otherwise lose the write bit on every protection change, even where the
write fault handler would restore it without further checks.

Do this by introducing can_change_pud_writable(), which mimics
can_change_pmd_writable(). Omit the NUMA and uffd-wp checks, which
change_huge_pud() already rejects.

Signed-off-by: Carlos López <clopez at suse.de>
---
 include/linux/pgtable.h |  5 +++++
 mm/huge_memory.c        | 28 +++++++++++++++++++++++++++-
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 780fe849ff8b..943d7e95f426 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -1866,6 +1866,11 @@ static inline int pmd_soft_dirty(pmd_t pmd)
 	return 0;
 }
 
+static inline int pud_soft_dirty(pud_t pud)
+{
+	return 0;
+}
+
 static inline pte_t pte_mksoft_dirty(pte_t pte)
 {
 	return pte;
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index bd44d58e8684..506e1e61c0fd 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2824,6 +2824,29 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	return ret;
 }
 
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+static inline bool can_change_pud_writable(struct vm_area_struct *vma,
+					   unsigned long addr, pud_t pud)
+{
+	struct page *page;
+
+	if (WARN_ON_ONCE(!(vma->vm_flags & VM_WRITE)))
+		return false;
+
+	/* Do we need write faults for softdirty tracking? */
+	if (vma_soft_dirty_enabled(vma) && !pud_soft_dirty(pud))
+		return false;
+
+	if (!(vma->vm_flags & VM_SHARED)) {
+		/* See can_change_pte_writable(). */
+		page = vm_normal_page_pud(vma, addr, pud);
+		return page && PageAnon(page) && PageAnonExclusive(page);
+	}
+
+	/* See can_change_pte_writable(). */
+	return pud_dirty(pud);
+}
+
 /*
  * Returns:
  *
@@ -2831,7 +2854,6 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
  * - 1: if pud can be skipped
  * - HPAGE_PUD_NR: if pud was successfully processed
  */
-#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
 int change_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
 		    pud_t *pudp, unsigned long addr, pgprot_t newprot,
 		    unsigned long cp_flags)
@@ -2863,6 +2885,10 @@ int change_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	 */
 	oldpud = pudp_invalidate(vma, addr, pudp);
 	entry = pud_modify(oldpud, newprot);
+	/* See change_pte_range(). */
+	if ((cp_flags & MM_CP_TRY_CHANGE_WRITABLE) && !pud_write(entry) &&
+	    can_change_pud_writable(vma, addr, entry))
+		entry = pud_mkwrite(entry);
 	set_pud_at(mm, addr, pudp, entry);
 	tlb_flush_pud_range(tlb, addr, HPAGE_PUD_SIZE);
 
-- 
2.51.0