[PATCH mm-unstable v15 08/14] mm: multi-gen LRU: support page table walks

[Yu Zhao]

On Sun, Sep 18, 2022 at 2:01 AM Yu Zhao <yuzhao at google.com> wrote:

...

> This patch uses the following optimizations when walking page tables:
> 1. It tracks the usage of mm_struct's between context switches so that
>    page table walkers can skip processes that have been sleeping since
>    the last iteration.

...

> @@ -672,6 +672,22 @@ struct mm_struct {
>                  */
>                 unsigned long ksm_merging_pages;
>  #endif
> +#ifdef CONFIG_LRU_GEN
> +               struct {
> +                       /* this mm_struct is on lru_gen_mm_list */
> +                       struct list_head list;
> +                       /*
> +                        * Set when switching to this mm_struct, as a hint of
> +                        * whether it has been used since the last time per-node
> +                        * page table walkers cleared the corresponding bits.
> +                        */
> +                       unsigned long bitmap;

...

> +static inline void lru_gen_use_mm(struct mm_struct *mm)
> +{
> +       /*
> +        * When the bitmap is set, page reclaim knows this mm_struct has been
> +        * used since the last time it cleared the bitmap. So it might be worth
> +        * walking the page tables of this mm_struct to clear the accessed bit.
> +        */
> +       WRITE_ONCE(mm->lru_gen.bitmap, -1);
> +}

...

> @@ -5180,6 +5180,7 @@ context_switch(struct rq *rq, struct task_struct *prev,
>                  * finish_task_switch()'s mmdrop().
>                  */
>                 switch_mm_irqs_off(prev->active_mm, next->mm, next);
> +               lru_gen_use_mm(next->mm);
>
>                 if (!prev->mm) {                        // from kernel
>                         /* will mmdrop() in finish_task_switch(). */

Adding Ingo, Peter, Juri and Vincent for the bit above, per previous
discussion here:
https://lore.kernel.org/r/CAOUHufY91Eju-g1+xbUsGkGZ-cwBm78v+S_Air7Cp8mAnYJVYA@mail.gmail.com/

I trimmed 99% of this patch to save your time. In case you want to
hear the whole story:
https://lore.kernel.org/r/20220918080010.2920238-9-yuzhao@google.com/