[PATCH] riscv: mm: don't let GUP-fast follow entries without _PAGE_READ

[Zi Yan]

riscv maps PROT_EXEC alone as PAGE_EXEC, which has
_PAGE_EXEC but not _PAGE_READ, and a writable shadow stack entry is
XWR = 010, which has _PAGE_WRITE but not _PAGE_READ. riscv uses the
generic pte_access_permitted(), pmd_access_permitted() and
pud_access_permitted(), which only check that the entry is present and,
for a write, writable. So GUP-fast follows both kinds of entries:

- a read, e.g. from vmsplice() or an O_DIRECT write, can read an
  execute-only page, which userspace cannot load from;
- a write, e.g. from an O_DIRECT read, can modify a shadow stack page,
  as pte_write() is true for it.

Slow GUP rejects both: check_vma_flags() fails a read from a VMA without
VM_READ and a write to a VM_SHADOW_STACK VMA, unless FOLL_FORCE is set.

Fix it by requiring both _PAGE_PRESENT and _PAGE_READ in
pte_access_permitted(), and using it for pmd_access_permitted() and
pud_access_permitted() too.

Requiring _PAGE_PRESENT also stops GUP-fast from following a THP PMD
invalidated by pmdp_invalidate(). x86, arm64, and powerpc do the same
thing.

Fixes: 3f910b7a522e ("riscv: enable HAVE_FAST_GUP if MMU")
Fixes: c68c2ef9d641 ("riscv/mm: teach pte_mkwrite to manufacture shadow stack PTEs")
Cc: stable at vger.kernel.org
Reported-by: Sashiko <sashiko-bot at kernel.org>
Closes: https://sashiko.dev/#/patchset/20261015-anon_vma_uaf_fix-v1-0-7b977303b42b%40nvidia.com?part=2
Assisted-by: LLM
Signed-off-by: Zi Yan <ziy at nvidia.com>
---
Sashiko found that the riscv pmd_access_permitted() added by the
lazyfree THP discard rework [1] lets GUP-fast bypass execute-only
protection. The generic helpers riscv uses today have the same gap, at
the PTE level too, and the generic pmd_access_permitted() also lets
GUP-fast follow an invalidated PMD. This patch makes GUP-fast require
both _PAGE_PRESENT and _PAGE_READ. Requiring _PAGE_PRESENT stops it from
following a PMD invalidated by pmdp_invalidate(), as x86, arm64, and
powerpc already do and as the rework relies on.

While reviewing this patch, an LLM also found that riscv's
pte_wrprotect() sets _PAGE_READ unconditionally, which turns an
execute-only entry into a readable one [2], e.g. every execute-only
hugetlb mapping via make_huge_pte(). That needs a separate fix.

Link: https://lore.kernel.org/all/20261015-anon_vma_uaf_fix-v1-0-7b977303b42b@nvidia.com/ [1]
Link: https://elixir.bootlin.com/linux/v7.3-rc7/source/arch/riscv/include/asm/pgtable.h#L409 [2]
---
 arch/riscv/include/asm/pgtable.h | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 40b1ed4f3ea89..f46b742d6dd93 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -372,6 +372,21 @@ static inline int pte_write(pte_t pte)
 	return pte_val(pte) & _PAGE_WRITE;
 }
 
+/*
+ * GUP-fast only follows entries that are present and have _PAGE_READ,
+ * and leaves the rest to slow GUP, which checks the VMA: an execute-only
+ * entry and a writable shadow stack entry (XWR = 010) lack _PAGE_READ, a
+ * PROT_NONE or invalidated entry lacks _PAGE_PRESENT. Any other writable
+ * entry is readable too.
+ */
+#define pte_access_permitted pte_access_permitted
+static inline bool pte_access_permitted(pte_t pte, bool write)
+{
+	const unsigned long need = _PAGE_PRESENT | _PAGE_READ;
+
+	return (pte_val(pte) & need) == need && (!write || pte_write(pte));
+}
+
 static inline int pte_exec(pte_t pte)
 {
 	return pte_val(pte) & _PAGE_EXEC;
@@ -820,12 +835,24 @@ static inline int pmd_write(pmd_t pmd)
 	return pte_write(pmd_pte(pmd));
 }
 
+#define pmd_access_permitted pmd_access_permitted
+static inline bool pmd_access_permitted(pmd_t pmd, bool write)
+{
+	return pte_access_permitted(pmd_pte(pmd), write);
+}
+
 #define pud_write pud_write
 static inline int pud_write(pud_t pud)
 {
 	return pte_write(pud_pte(pud));
 }
 
+#define pud_access_permitted pud_access_permitted
+static inline bool pud_access_permitted(pud_t pud, bool write)
+{
+	return pte_access_permitted(pud_pte(pud), write);
+}
+
 #define pmd_dirty pmd_dirty
 static inline int pmd_dirty(pmd_t pmd)
 {

---
base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907
change-id: 20261015-riscv-gup-read-fix-d657d67d350a

Best regards,
--  
Yan, Zi