[PATCH v5 15/70] kernel/fork: Use maple tree for dup_mmap() during forking

[Mark Hemment]

On Wed, 2 Feb 2022 at 03:23, Liam Howlett <liam.howlett at oracle.com> wrote:
>
> From: "Liam R. Howlett" <Liam.Howlett at Oracle.com>
>
> The maple tree was already tracking VMAs in this function by an earlier
> commit, but the rbtree iterator was being used to iterate the list.
> Change the iterator to use a maple tree native iterator and switch to
> the maple tree advanced API to avoid multiple walks of the tree during
> insert operations.  Unexport the now-unused vma_store() function.
>
> We track whether we need to free the VMAs and tree nodes through RCU
> (ie whether there have been multiple threads that can see the mm_struct
> simultaneously; by pthread(), ptrace() or looking at /proc/$pid/maps).
> This setting is sticky because it's too tricky to decide when it's safe
> to exit RCU mode.
>
> For performance reasons we bulk allocate the maple tree nodes.  The node
> calculations are done internally to the tree and use the VMA count and
> assume the worst-case node requirements.  The VM_DONT_COPY flag does
> not allow for the most efficient copy method of the tree and so a bulk
> loading algorithm is used.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy at infradead.org>
> Signed-off-by: Liam R. Howlett <Liam.Howlett at Oracle.com>
> Acked-by: Vlastimil Babka <vbabka at suse.cz>
> ---
>  include/linux/mm.h       |  2 --
>  include/linux/sched/mm.h | 13 +++++++++++++
>  kernel/fork.c            | 35 +++++++++++++++++++++++++++++------
>  3 files changed, 42 insertions(+), 8 deletions(-)
....
> diff --git a/kernel/fork.c b/kernel/fork.c
> index 51a7971651ef..8ea683fcefcd 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -377,6 +377,16 @@ void vm_area_free(struct vm_area_struct *vma)
>         kmem_cache_free(vm_area_cachep, vma);
>  }
>
> +void mm_set_in_rcu(struct mm_struct *mm)
> +{
> +       if (!mt_in_rcu(&mm->mm_mt))
> +               return;
> +       mmap_write_lock(mm);
> +       mm->mm_mt.ma_flags |= MT_FLAGS_USE_RCU;
> +       mmap_write_unlock(mm);
> +}
> +EXPORT_SYMBOL(mm_set_in_rcu);

The mt_in_rcu() test looks wrong (inverted).
mt_in_rcu() returns true only when MT_FLAGS_USE_RCU is set, so this
flag will never set here.
All callers of mm_set_in_rcu() check mt_in_rcu() so the test here
could be removed.

Cheers,
Mark