[PATCH 09/46] mm/rmap: handle device-exclusive entries in folio_referenced_one()

[Dev Jain]

On 27/10/26 4:47 am, David Hildenbrand (Arm) wrote:
> page_vma_mapped_walk() can return non-present device-exclusive entries,
> but folio_referenced_one() passes them to PTE young-bit accessors that
> require present PTEs.
> 
> We must neither call clear_flush_young_ptes_notify() nor
> folio_pte_batch() with non-present entries. For example, clearing the
> young bit on non-present PTEs can corrupt the entry: on arm64
> PTE_AF overlaps the swap type, so clearing it turns a device-exclusive
> entry into an ordinary swap entry.
> 
> folio_pte_batch() will VM_WARN_ON_FOLIO.
> 
> Apply the same approach as the previous fixes to e.g., damon_ptep_mkold():
> treat these entries as old from the CPU's perspective and obtain their
> young state through the MMU notifier.
> 
> Skip PTE batching and MGLRU look-around for non-present entries, and
> don't modify non-present PTEs.
> 
> Fixes: b756a3b5e7ea ("mm: device exclusive memory access")
> Reviewed-by: Lorenzo Stoakes (ARM) <ljs at kernel.org>
> Cc: stable at vger.kernel.org
> Reported-by: Josef Bacik <josef at toxicpanda.com>
> Signed-off-by: David Hildenbrand (Arm) <david at kernel.org>
> ---

Reviewed-by: Dev Jain <dev.jain at arm.com>

But nit below.


>  mm/rmap.c | 23 +++++++++++++++++++----
>  1 file changed, 19 insertions(+), 4 deletions(-)
> 
> diff --git a/mm/rmap.c b/mm/rmap.c
> index 1131b76bbbc28..0faef46806e5c 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -924,6 +924,7 @@ static bool folio_referenced_one(struct folio *folio,
>  	DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0);
>  	int ptes = 0, referenced = 0;
>  	unsigned int nr;
> +	pte_t pteval;
>  
>  	while (page_vma_mapped_walk(&pvmw)) {
>  		address = pvmw.address;
> @@ -968,10 +969,12 @@ static bool folio_referenced_one(struct folio *folio,
>  			return false;
>  		}
>  
> -		if (pvmw.pte && folio_test_large(folio)) {
> +		if (pvmw.pte)
> +			pteval = ptep_get(pvmw.pte);
> +
> +		if (pvmw.pte && pte_present(pteval) && folio_test_large(folio)) {
>  			const unsigned long end_addr = pmd_addr_end(address, vma->vm_end);
>  			const unsigned int max_nr = (end_addr - address) >> PAGE_SHIFT;
> -			pte_t pteval = ptep_get(pvmw.pte);
>  
>  			nr = folio_pte_batch(folio, pvmw.pte, pteval, max_nr);
>  		}
> @@ -981,11 +984,23 @@ static bool folio_referenced_one(struct folio *folio,
>  		 * are. —they could be on active/inactive lists or on MGLRU. So the
>  		 * simplest approach is to disable this look-around optimization.
>  		 */
> -		if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte) {
> +		if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte &&
> +		    pte_present(pteval)) {
>  			if (lru_gen_look_around(&pvmw, nr))
>  				referenced++;
>  		} else if (pvmw.pte) {
> -			if (clear_flush_young_ptes_notify(vma, address, pvmw.pte, nr))
> +			bool young = false;
> +
> +			/*
> +			 * PFN swap PTEs, such as device-exclusive ones, that

You mean "PFN softleaf PTEs"? Better expressed as "softleaf PTEs embedding a PFN" ...


> +			 * actually map pages are "old" from a CPU perspective.
> +			 * The MMU notifier takes care of any device aspects.
> +			 */
> +			if (pte_present(pteval))
> +				young = clear_flush_young_ptes(vma, address, pvmw.pte, nr);
> +			young |= mmu_notifier_clear_flush_young(vma->vm_mm, address,
> +								address + nr * PAGE_SIZE);
> +			if (young)
>  				referenced++;
>  		} else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) {
>  			if (pmdp_clear_flush_young_notify(vma, address,
>