[PATCH] riscv: mm: Implement pmdp_collapse_flush for THP
mchitale at ventanamicro.com
mchitale at ventanamicro.com
Sun Jan 29 23:26:06 PST 2023
On Wed, 2023-01-25 at 16:35 +0100, Andrew Jones wrote:
> On Wed, Jan 25, 2023 at 06:25:11PM +0530, Mayuresh Chitale wrote:
> > When THP is enabled, 4K pages are collapsed into a single huge
> > page using the generic pmdp_collapse_flush() which will further
> > use flush_tlb_range() to shoot-down stale TLB entries.
> > Unfortunately,
> > the generic pmdp_collapse_flush() only invalidates cached leaf PTEs
> > using address specific SFENCEs which results in repetitive (or
> > unpredictable) page faults on RISC-V implementations which cache
> > non-leaf PTEs.
> >
> > Provide a RISC-V specific pmdp_collapse_flush() which ensures both
> > cached leaf and non-leaf PTEs are invalidated by using non-address
> > specific SFENCEs as recommended by the RISC-V privileged
> > specification.
> >
> > Fixes: e88b333142e4 ("riscv: mm: add THP support on 64-bit")
> > Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
> > ---
> > arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++
> > 1 file changed, 24 insertions(+)
> >
> > diff --git a/arch/riscv/include/asm/pgtable.h
> > b/arch/riscv/include/asm/pgtable.h
> > index 4eba9a98d0e3..6d948dec6020 100644
> > --- a/arch/riscv/include/asm/pgtable.h
> > +++ b/arch/riscv/include/asm/pgtable.h
> > @@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct
> > vm_area_struct *vma,
> > page_table_check_pmd_set(vma->vm_mm, address, pmdp, pmd);
> > return __pmd(atomic_long_xchg((atomic_long_t *)pmdp,
> > pmd_val(pmd)));
> > }
> > +
> > +#define pmdp_collapse_flush pmdp_collapse_flush
> > +static inline pmd_t pmdp_collapse_flush(struct vm_area_struct
> > *vma,
> > + unsigned long address, pmd_t
> > *pmdp)
> > +{
> > + pmd_t pmd = pmdp_huge_get_and_clear(vma->vm_mm, address, pmdp);
>
> The generic version of this function has a couple sanity checks for
> kernels compiled with CONFIG_DEBUG_VM. Shouldn't we duplicate those?
Ok. I will add those checks too.
>
> Thanks,
> drew
>
> > +
> > + /*
> > + * When leaf PTE enteries (regular pages) are collapsed into a
> > leaf
> > + * PMD entry (huge page), a valid non-leaf PTE is converted
> > into a
> > + * valid leaf PTE at the level 1 page table. The RISC-V
> > privileged v1.12
> > + * specification allows implementations to cache valid non-leaf
> > PTEs,
> > + * but the section "4.2.1 Supervisor Memory-Management Fence
> > + * Instruction" recommends the following:
> > + * "If software modifies a non-leaf PTE, it should execute
> > SFENCE.VMA
> > + * with rs1=x0. If any PTE along the traversal path had its G
> > bit set,
> > + * rs2 must be x0; otherwise, rs2 should be set to the ASID for
> > which
> > + * the translation is being modified."
> > + * Based on the above recommendation, we should do full flush
> > whenever
> > + * leaf PTE entries are collapsed into a leaf PMD entry.
> > + */
> > + flush_tlb_mm(vma->vm_mm);
> > + return pmd;
> > +}
> > #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
> >
> > /*
> > --
> > 2.34.1
> >
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
More information about the linux-riscv
mailing list