From Liam.Howlett at Oracle.com  Tue Aug  1 12:48:20 2023
From: Liam.Howlett at Oracle.com (Liam R. Howlett)
Date: Tue, 1 Aug 2023 15:48:20 -0400
Subject: Maple Tree Work 2023/08
Message-ID: <20230801194820.alch3ot7pvrgig5v@revolver>

Hello,

This months update of the maple tree development and work items follows.
I send out an updated list monthly with peoples names against work items
to avoid work duplication, so please don't work on something with a name
next to the work item.

If you want to work on something then please respond to this email so
everyone knows, and I can add your name to the item.  Feel free to ask
questions to clarify the feature or discuss directions.

Likewise, if you cannot work on anything anymore then let me know so
others can.

If you have any ideas, then please email the list.  We can discuss it
and I can add it to the list.

Please sign up to the maple tree mailing list [1] to get all updates.

Features:
 - Better preallocation calculations - Liam R. Howlett 2023/07/07
 	Without tracking the tree status on the walk down, we can
	partially walk down to figure out the type of write and which
	'worst case' a write will cause.  Using this "write type"
	information, the preallocations can be a better estimate. v3 was
	sent to the mailing list [2].

 - mas_store_gfp() align with mas_prealloc()
	Right now mas_store_gfp() is going to allocate once it figures
	out what to do and it does the worst case allocations.  This is
	inefficient and can easily be done better by doing more of what
	mas_prealloc()/mas_store_prealloc() does - or at least will be
	doing once the 'Better preallocation calculations' lands.

 - Tracking tree status on walks down
 	Track in the maple state when the last minimum nodes, space for
	2, or space for 3 slots occurred (height in the tree).  This
	will allow for a better estimate on how many nodes are needed
	for a write.  This can be complicated on next/prev walking, etc.
	It has to be done in read mode since some walk may have been
	done before deciding to write - see mmap_region(), for example.

 - Store type (enum for store type?)
	Extending the "Tracking tree status on walks down", the
	information can then be used to determine what operation is
	needed during a mas_prealloct(), etc.  The operation can be
	stored in the maple state to continue on a mas_store_prealloc().
	Obviously, this would work out well if we have mas_store_gfp()
	using the same functionality as mentioned above.

 - Full count/flag & Dense nodes
 	There is a bit that exists in the pointer reserved to indicate
	there is no NULLs in the leaves below.  This feature is mainly
	for trees that are non-alloc trees.  We can then know there is
	at least one singleton that can be stored below this entry.
	This is coupled with restoring Dense nodes as a potential node
	type.  The tricky part is deciding on when to switch to dense
	nodes/back from dense nodes (all entries to dense nodes must be
	singletons!).  See mte_set_full(), mte_clear_full(),
	mte_has_null() which use MAPLE_ENODE_NULL.

 - Fork & Dup tree + Delete DONT_COPY - Peng Zhang 2023/07/12
 	This is to optimize dup_mmap() in kernel/fork.c, but other
	users may want faster duplications of a tree.
	This should be faster than building a tree in bulk mode.  The
	idea is to just copy each node and replace the pointers in,
	probably, a BFS order.  Once the leaves are reached, the VMAs
	will be replaced by the copies made in fork, unless DONT_COPY is
	set, in which case the VMA will be deleted from the copied tree.
	DONT_COPY is not common and since the tree isn't visible, all
	nodes should be available for reuse (no RCU worries). v1 was
	sent to the mailing list [3].

 - Push reuse parent
 	During an insert, new nodes can be "pushed" left or right -
	see mas_push_data().  On a left push, it may be possible to
	reuse the node by making the write like an 'append'.  This may
	also be possible to be done during a split operation when the
	write extends the last slot. This needs examining to ensure RCU
	safety.

 - Contiguous iterator
 	Some users want to iterate across a range of items, but only
	contiguous items.  Sometimes a gap within the range is
	considered an error while other times they are not.  Good
	targets for this feature are mlock() and mprotect() system
	calls.  The goal is to provide an easy to use interface that
	reduces the complexity of the external code that keeps track of
	the contiguous nature of the iteration.

 - Remove BIG_NODE
 	maple tree nodes are rebalanced, split, and updated using a
	maple_big_node struct.  This structure is over twice the size of
	regular nodes to ensure there is enough space to contain the
	data.  Ideally the same feats can be accomplished by using
	regular sized nodes.

 - Test & Fix destroy_rebalance
 	Destroy_rebalance needs better testing when it occurs in rcu
	mode.  Currently it is not used in this mode, but it should be
	supported.  This part of the code needs better test coverage and
	fix any necessary issues that arise with rcu readers during
	updates.

Larger Features:
 - 64 bit stores on 32 bit arch
 	A number of users want to use the maple tree, but cannot because
	they need a storage medium that supports 64 bit stores on 32 bit
	arch.

 - wr_mas with alloc instead of mas
 	Internally, we have a write maple state, but the maple state
	currently holds the allocations.  I'm not sure if this is worth
	it, and it will probably need "Tracking tree status on walks
	down" so that preallocations/allocations can be accurate.  Other
	considerations are how mas_prealloc() would work with this
	change.

 - Big Dense Node Type
 	There are various node types that could be added to the maple
	tree.  One of the most useful is probably a 'big dense' with the
	node being a 4k block of singletons.  This would have to come
	after the dense node type as dense enables more users and helps
	scope this work.

 - Judy Array
	The Judy Array is another fancy data structure.  Mathieu
	Desnoyers has enquired and spoken to us about incorporating judy
	arrays as a node type or other features he has in the judy
	array.  This is in the early thought stages.

Please note that any new features will need test cases added to ensure
they function correctly.

[1] https://lists.infradead.org/mailman/listinfo/maple-tree
[2] http://lists.infradead.org/pipermail/maple-tree/2023-July/002736.html
[3] https://lore.kernel.org/linux-mm/20230726080916.17454-1-zhangpeng.00 at bytedance.com/

Thanks,
Liam


From Liam.Howlett at oracle.com  Fri Aug  4 09:59:46 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri,  4 Aug 2023 12:59:46 -0400
Subject: [PATCH 1/6] maple_tree: Add hex output to maple_arange64 dump
In-Reply-To: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
References: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
Message-ID: <20230804165951.2661157-2-Liam.Howlett@oracle.com>

When dumping the tree, honour formatting request to output hex for the
maple node type arange64.

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index a3d602cfd030..880ce0fcdcac 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -6833,11 +6833,27 @@ static void mt_dump_arange64(const struct maple_tree *mt, void *entry,
 	int i;
 
 	pr_cont(" contents: ");
-	for (i = 0; i < MAPLE_ARANGE64_SLOTS; i++)
-		pr_cont("%lu ", node->gap[i]);
+	for (i = 0; i < MAPLE_ARANGE64_SLOTS; i++) {
+		switch (format) {
+		case mt_dump_hex:
+			pr_cont("%lx ", node->gap[i]);
+			break;
+		default:
+		case mt_dump_dec:
+			pr_cont("%lu ", node->gap[i]);
+		}
+	}
 	pr_cont("| %02X %02X| ", node->meta.end, node->meta.gap);
-	for (i = 0; i < MAPLE_ARANGE64_SLOTS - 1; i++)
-		pr_cont("%p %lu ", node->slot[i], node->pivot[i]);
+	for (i = 0; i < MAPLE_ARANGE64_SLOTS - 1; i++) {
+		switch (format) {
+		case mt_dump_hex:
+			pr_cont("%p %lX ", node->slot[i], node->pivot[i]);
+			break;
+		default:
+		case mt_dump_dec:
+			pr_cont("%p %lu ", node->slot[i], node->pivot[i]);
+		}
+	}
 	pr_cont("%p\n", node->slot[i]);
 	for (i = 0; i < MAPLE_ARANGE64_SLOTS; i++) {
 		unsigned long last = max;
-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug  4 09:59:45 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri,  4 Aug 2023 12:59:45 -0400
Subject: [PATCH 0/6] maple_tree: Change replacement strategy
Message-ID: <20230804165951.2661157-1-Liam.Howlett@oracle.com>

The maple tree marks nodes dead as soon as they are going to be
replaced.  This could be problematic when used in the RCU context since
the writer may be starved of CPU time by the readers.  This patch set
addresses the issue by switching the data replacement strategy to one
that will only mark data as dead once the new data is available.

This series changes the ordering of the node replacement so that the new
data is live before the old data is marked 'dead'.  When readers hit
'dead' nodes, they will restart from the top of the tree and end up in
the new data.

In more complex scenarios, the replacement strategy means a subtree is
built and graphed into the tree leaving some nodes to point to the old
parent.  The view of tasks into the old data will either remain with the
old data, or see the new data once the old data is marked 'dead'.

Iterators will see the 'dead' node and restart on their own and switch
to the new data.  There is no risk of the reader seeing old data in
these cases.

The 'dead' subtree of data is then fully marked dead, but reused nodes
will still point to the dead nodes until the parent pointer is updated.
Walking up to a 'dead' node will cause a re-walk from the top of the
tree and enter the new data area where old data is not reachable.

Once the parent pointers are fully up to date in the active data, the
'dead' subtree is iterated to collect entirely 'dead' subtrees, and dead
nodes (nodes that partially contained reused data).

Liam R. Howlett (6):
  maple_tree: Add hex output to maple_arange64 dump
  maple_tree: Reorder replacement of nodes to avoid live lock
  maple_tree: introduce mas_put_in_tree()
  maple_tree: Introduce mas_tree_parent() definition
  maple_tree: Change mas_adopt_children() parent usage
  maple_tree: Replace data before marking dead in split and spanning
    store

 lib/maple_tree.c | 607 +++++++++++++++++++----------------------------
 1 file changed, 240 insertions(+), 367 deletions(-)

-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug  4 09:59:47 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri,  4 Aug 2023 12:59:47 -0400
Subject: [PATCH 2/6] maple_tree: Reorder replacement of nodes to avoid live lock
In-Reply-To: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
References: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
Message-ID: <20230804165951.2661157-3-Liam.Howlett@oracle.com>

Replacing nodes may cause a live lock-up if CPU resources are saturated
by write operations on the tree by continuously retrying on dead nodes.
To avoid the continuous retry scenario, ensure the new node is inserted
into the tree prior to marking the old data as dead.  This will define a
window where old and new data is swapped.

When reusing lower level nodes, ensure the parent pointer is updated
after the parent is marked dead.  This ensures that the child is still
reachable from the top of the tree, but walking up to a dead node will
result in a single retry that will start a fresh walk from the top down
through the new node.

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 56 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 46 insertions(+), 10 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 880ce0fcdcac..0d4573a8d134 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -1756,6 +1756,36 @@ static inline void mas_replace(struct ma_state *mas, bool advanced)
 	}
 }
 
+/*
+ * mas_replace_node() - Replace a node by putting it in the tree, marking it
+ * dead, and freeing it.
+ * the parent encoding to locate the maple node in the tree.
+ * @mas - the ma_state with @mas->node pointing to the new node.
+ * @old_enode - The old maple encoded node.
+ */
+static inline void mas_replace_node(struct ma_state *mas,
+		struct maple_enode *old_enode)
+	__must_hold(mas->tree->ma_lock)
+{
+	if (mte_is_root(mas->node)) {
+		mas_mn(mas)->parent = ma_parent_ptr(
+			      ((unsigned long)mas->tree | MA_ROOT_PARENT));
+		rcu_assign_pointer(mas->tree->ma_root, mte_mk_root(mas->node));
+		mas_set_height(mas);
+	} else {
+		unsigned char offset = 0;
+		void __rcu **slots = NULL;
+
+		offset = mte_parent_slot(mas->node);
+		slots = ma_slots(mte_parent(mas->node),
+				 mas_parent_type(mas, mas->node));
+		rcu_assign_pointer(slots[offset], mas->node);
+	}
+
+	mte_set_node_dead(old_enode);
+	mas_free(mas, old_enode);
+}
+
 /*
  * mas_new_child() - Find the new child of a node.
  * @mas: the maple state
@@ -3176,7 +3206,7 @@ static inline void mas_destroy_rebalance(struct ma_state *mas, unsigned char end
 {
 	enum maple_type mt = mte_node_type(mas->node);
 	struct maple_node reuse, *newnode, *parent, *new_left, *left, *node;
-	struct maple_enode *eparent;
+	struct maple_enode *eparent, *old_eparent;
 	unsigned char offset, tmp, split = mt_slots[mt] / 2;
 	void __rcu **l_slots, **slots;
 	unsigned long *l_pivs, *pivs, gap;
@@ -3218,7 +3248,7 @@ static inline void mas_destroy_rebalance(struct ma_state *mas, unsigned char end
 
 	l_mas.max = l_pivs[split];
 	mas->min = l_mas.max + 1;
-	eparent = mt_mk_node(mte_parent(l_mas.node),
+	old_eparent = mt_mk_node(mte_parent(l_mas.node),
 			     mas_parent_type(&l_mas, l_mas.node));
 	tmp += end;
 	if (!in_rcu) {
@@ -3234,7 +3264,7 @@ static inline void mas_destroy_rebalance(struct ma_state *mas, unsigned char end
 
 		memcpy(node, newnode, sizeof(struct maple_node));
 		ma_set_meta(node, mt, 0, tmp - 1);
-		mte_set_pivot(eparent, mte_parent_slot(l_mas.node),
+		mte_set_pivot(old_eparent, mte_parent_slot(l_mas.node),
 			      l_pivs[split]);
 
 		/* Remove data from l_pivs. */
@@ -3242,6 +3272,7 @@ static inline void mas_destroy_rebalance(struct ma_state *mas, unsigned char end
 		memset(l_pivs + tmp, 0, sizeof(unsigned long) * (max_p - tmp));
 		memset(l_slots + tmp, 0, sizeof(void *) * (max_s - tmp));
 		ma_set_meta(left, mt, 0, split);
+		eparent = old_eparent;
 
 		goto done;
 	}
@@ -3266,7 +3297,7 @@ static inline void mas_destroy_rebalance(struct ma_state *mas, unsigned char end
 	parent = mas_pop_node(mas);
 	slots = ma_slots(parent, mt);
 	pivs = ma_pivots(parent, mt);
-	memcpy(parent, mte_to_node(eparent), sizeof(struct maple_node));
+	memcpy(parent, mte_to_node(old_eparent), sizeof(struct maple_node));
 	rcu_assign_pointer(slots[offset], mas->node);
 	rcu_assign_pointer(slots[offset - 1], l_mas.node);
 	pivs[offset - 1] = l_mas.max;
@@ -3278,8 +3309,10 @@ static inline void mas_destroy_rebalance(struct ma_state *mas, unsigned char end
 	mte_set_gap(eparent, mte_parent_slot(l_mas.node), gap);
 	mas_ascend(mas);
 
-	if (in_rcu)
-		mas_replace(mas, false);
+	if (in_rcu) {
+		mas_replace_node(mas, old_eparent);
+		mas_adopt_children(mas, mas->node);
+	}
 
 	mas_update_gap(mas);
 }
@@ -3596,11 +3629,13 @@ static noinline_for_kasan int mas_commit_b_node(struct ma_wr_state *wr_mas,
 			    struct maple_big_node *b_node, unsigned char end)
 {
 	struct maple_node *node;
+	struct maple_enode *old_enode;
 	unsigned char b_end = b_node->b_end;
 	enum maple_type b_type = b_node->type;
 
+	old_enode = wr_mas->mas->node;
 	if ((b_end < mt_min_slots[b_type]) &&
-	    (!mte_is_root(wr_mas->mas->node)) &&
+	    (!mte_is_root(old_enode)) &&
 	    (mas_mt_height(wr_mas->mas) > 1))
 		return mas_rebalance(wr_mas->mas, b_node);
 
@@ -3618,7 +3653,7 @@ static noinline_for_kasan int mas_commit_b_node(struct ma_wr_state *wr_mas,
 	node->parent = mas_mn(wr_mas->mas)->parent;
 	wr_mas->mas->node = mt_mk_node(node, b_type);
 	mab_mas_cp(b_node, 0, b_end, wr_mas->mas, false);
-	mas_replace(wr_mas->mas, false);
+	mas_replace_node(wr_mas->mas, old_enode);
 reuse_node:
 	mas_update_gap(wr_mas->mas);
 	return 1;
@@ -4117,9 +4152,10 @@ static inline bool mas_wr_node_store(struct ma_wr_state *wr_mas,
 done:
 	mas_leaf_set_meta(mas, newnode, dst_pivots, maple_leaf_64, new_end);
 	if (in_rcu) {
-		mte_set_node_dead(mas->node);
+		struct maple_enode *old_enode = mas->node;
+
 		mas->node = mt_mk_node(newnode, wr_mas->type);
-		mas_replace(mas, false);
+		mas_replace_node(mas, old_enode);
 	} else {
 		memcpy(wr_mas->node, newnode, sizeof(struct maple_node));
 	}
-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug  4 09:59:49 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri,  4 Aug 2023 12:59:49 -0400
Subject: [PATCH 4/6] maple_tree: Introduce mas_tree_parent() definition
In-Reply-To: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
References: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
Message-ID: <20230804165951.2661157-5-Liam.Howlett@oracle.com>

Add a definition to shorten long code lines and clarify what the code is
doing.  Use the new definition to get the maple tree parent pointer from
the maple state where possible.

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index c01b1be1480c..cf41e0dbb87b 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -75,6 +75,7 @@
 #define MA_STATE_PREALLOC	4
 
 #define ma_parent_ptr(x) ((struct maple_pnode *)(x))
+#define mas_tree_parent(x) ((unsigned long)(x->tree) | MA_ROOT_PARENT)
 #define ma_mnode_ptr(x) ((struct maple_node *)(x))
 #define ma_enode_ptr(x) ((struct maple_enode *)(x))
 static struct kmem_cache *maple_node_cache;
@@ -1728,8 +1729,7 @@ static inline void mas_put_in_tree(struct ma_state *mas,
 	void __rcu **slots;
 
 	if (mte_is_root(mas->node)) {
-		mas_mn(mas)->parent = ma_parent_ptr(
-			      ((unsigned long)mas->tree | MA_ROOT_PARENT));
+		mas_mn(mas)->parent = ma_parent_ptr(mas_tree_parent(mas));
 		rcu_assign_pointer(mas->tree->ma_root, mte_mk_root(mas->node));
 		mas_set_height(mas);
 	} else {
@@ -2798,8 +2798,7 @@ static inline void mas_wmb_replace(struct ma_state *mas,
 static inline void mast_new_root(struct maple_subtree_state *mast,
 				 struct ma_state *mas)
 {
-	mas_mn(mast->l)->parent =
-		ma_parent_ptr(((unsigned long)mas->tree | MA_ROOT_PARENT));
+	mas_mn(mast->l)->parent = ma_parent_ptr(mas_tree_parent(mas));
 	if (!mte_dead_node(mast->orig_l->node) &&
 	    !mte_is_root(mast->orig_l->node)) {
 		do {
@@ -3661,8 +3660,7 @@ static inline int mas_root_expand(struct ma_state *mas, void *entry)
 	node = mas_pop_node(mas);
 	pivots = ma_pivots(node, type);
 	slots = ma_slots(node, type);
-	node->parent = ma_parent_ptr(
-		      ((unsigned long)mas->tree | MA_ROOT_PARENT));
+	node->parent = ma_parent_ptr(mas_tree_parent(mas));
 	mas->node = mt_mk_node(node, type);
 
 	if (mas->index) {
@@ -3938,8 +3936,7 @@ static inline int mas_new_root(struct ma_state *mas, void *entry)
 	node = mas_pop_node(mas);
 	pivots = ma_pivots(node, type);
 	slots = ma_slots(node, type);
-	node->parent = ma_parent_ptr(
-		      ((unsigned long)mas->tree | MA_ROOT_PARENT));
+	node->parent = ma_parent_ptr(mas_tree_parent(mas));
 	mas->node = mt_mk_node(node, type);
 	rcu_assign_pointer(slots[0], entry);
 	pivots[0] = mas->last;
-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug  4 09:59:48 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri,  4 Aug 2023 12:59:48 -0400
Subject: [PATCH 3/6] maple_tree: introduce mas_put_in_tree()
In-Reply-To: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
References: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
Message-ID: <20230804165951.2661157-4-Liam.Howlett@oracle.com>

mas_replace() has a single user that takes a flag which is now always
true.  Replace this function with mas_put_in_tree() to better align with
mas_replace_node().  Inline the remaining logic into the only caller;
mas_wmb_replace().

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 73 ++++++++++++++++++------------------------------
 1 file changed, 27 insertions(+), 46 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 0d4573a8d134..c01b1be1480c 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -1715,45 +1715,32 @@ static inline void mas_adopt_children(struct ma_state *mas,
 }
 
 /*
- * mas_replace() - Replace a maple node in the tree with mas->node.  Uses the
- * parent encoding to locate the maple node in the tree.
- * @mas - the ma_state to use for operations.
- * @advanced - boolean to adopt the child nodes and free the old node (false) or
- * leave the node (true) and handle the adoption and free elsewhere.
+ * mas_put_in_tree() - Put a new node in the tree, smp_wmb(), and mark the old
+ * node as dead.
+ * @mas - the maple state with the new node
+ * @old_enode - The old maple encoded node to replace.
  */
-static inline void mas_replace(struct ma_state *mas, bool advanced)
+static inline void mas_put_in_tree(struct ma_state *mas,
+		struct maple_enode *old_enode)
 	__must_hold(mas->tree->ma_lock)
 {
-	struct maple_node *mn = mas_mn(mas);
-	struct maple_enode *old_enode;
-	unsigned char offset = 0;
-	void __rcu **slots = NULL;
-
-	if (ma_is_root(mn)) {
-		old_enode = mas_root_locked(mas);
-	} else {
-		offset = mte_parent_slot(mas->node);
-		slots = ma_slots(mte_parent(mas->node),
-				 mas_parent_type(mas, mas->node));
-		old_enode = mas_slot_locked(mas, slots, offset);
-	}
-
-	if (!advanced && !mte_is_leaf(mas->node))
-		mas_adopt_children(mas, mas->node);
+	unsigned char offset;
+	void __rcu **slots;
 
 	if (mte_is_root(mas->node)) {
-		mn->parent = ma_parent_ptr(
+		mas_mn(mas)->parent = ma_parent_ptr(
 			      ((unsigned long)mas->tree | MA_ROOT_PARENT));
 		rcu_assign_pointer(mas->tree->ma_root, mte_mk_root(mas->node));
 		mas_set_height(mas);
 	} else {
+
+		offset = mte_parent_slot(mas->node);
+		slots = ma_slots(mte_parent(mas->node),
+				 mas_parent_type(mas, mas->node));
 		rcu_assign_pointer(slots[offset], mas->node);
 	}
 
-	if (!advanced) {
-		mte_set_node_dead(old_enode);
-		mas_free(mas, old_enode);
-	}
+	mte_set_node_dead(old_enode);
 }
 
 /*
@@ -1767,22 +1754,7 @@ static inline void mas_replace_node(struct ma_state *mas,
 		struct maple_enode *old_enode)
 	__must_hold(mas->tree->ma_lock)
 {
-	if (mte_is_root(mas->node)) {
-		mas_mn(mas)->parent = ma_parent_ptr(
-			      ((unsigned long)mas->tree | MA_ROOT_PARENT));
-		rcu_assign_pointer(mas->tree->ma_root, mte_mk_root(mas->node));
-		mas_set_height(mas);
-	} else {
-		unsigned char offset = 0;
-		void __rcu **slots = NULL;
-
-		offset = mte_parent_slot(mas->node);
-		slots = ma_slots(mte_parent(mas->node),
-				 mas_parent_type(mas, mas->node));
-		rcu_assign_pointer(slots[offset], mas->node);
-	}
-
-	mte_set_node_dead(old_enode);
+	mas_put_in_tree(mas, old_enode);
 	mas_free(mas, old_enode);
 }
 
@@ -2789,11 +2761,20 @@ static inline void mas_wmb_replace(struct ma_state *mas,
 				   struct ma_topiary *free,
 				   struct ma_topiary *destroy)
 {
-	/* All nodes must see old data as dead prior to replacing that data */
-	smp_wmb(); /* Needed for RCU */
+	struct maple_enode *old_enode;
+
+	if (mte_is_root(mas->node)) {
+		old_enode = mas_root_locked(mas);
+	} else {
+		unsigned char offset = mte_parent_slot(mas->node);
+		void __rcu **slots = ma_slots(mte_parent(mas->node),
+					      mas_parent_type(mas, mas->node));
+
+		old_enode = mas_slot_locked(mas, slots, offset);
+	}
 
 	/* Insert the new data in the tree */
-	mas_replace(mas, true);
+	mas_put_in_tree(mas, old_enode);
 
 	if (!mte_is_leaf(mas->node))
 		mas_descend_adopt(mas);
-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug  4 09:59:50 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri,  4 Aug 2023 12:59:50 -0400
Subject: [PATCH 5/6] maple_tree: Change mas_adopt_children() parent usage
In-Reply-To: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
References: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
Message-ID: <20230804165951.2661157-6-Liam.Howlett@oracle.com>

All calls to mas_adopt_children() currently pass the parent as the node
in the maple state.  Allow for the parent pointer that is passed in to
be used instead.

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index cf41e0dbb87b..8e94f5495a97 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -1702,7 +1702,7 @@ static inline void mas_adopt_children(struct ma_state *mas,
 		struct maple_enode *parent)
 {
 	enum maple_type type = mte_node_type(parent);
-	struct maple_node *node = mas_mn(mas);
+	struct maple_node *node = mte_to_node(parent);
 	void __rcu **slots = ma_slots(node, type);
 	unsigned long *pivots = ma_pivots(node, type);
 	struct maple_enode *child;
-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug  4 09:59:51 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri,  4 Aug 2023 12:59:51 -0400
Subject: [PATCH 6/6] maple_tree: Replace data before marking dead in split and spanning store
In-Reply-To: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
References: <20230804165951.2661157-1-Liam.Howlett@oracle.com>
Message-ID: <20230804165951.2661157-7-Liam.Howlett@oracle.com>

Reorder the operations for split and spanning stores so that new data is
placed in the tree prior to marking the old data as dead.  This will
limit re-walks on dead data to just once instead of a retry loop.

The order of operations is as follows: Create the new data, put the new
data in place, mark the top node of the old data as dead.

Then repair parent links in the reused nodes through all levels of the
tree, following the new nodes downwards.  Finally walk the top dead node
looking for nodes that are no longer used, or subtrees that should be
destroyed (marked dead throughout then freed), follow the partially used
nodes downwards to discover other dead nodes and subtrees.

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 493 ++++++++++++++++-------------------------------
 1 file changed, 168 insertions(+), 325 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 8e94f5495a97..ffb9d15bd815 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -982,27 +982,9 @@ static inline void mat_add(struct ma_topiary *mat,
 	mat->tail = dead_enode;
 }
 
-static void mte_destroy_walk(struct maple_enode *, struct maple_tree *);
-static inline void mas_free(struct ma_state *mas, struct maple_enode *used);
-
-/*
- * mas_mat_free() - Free all nodes in a dead list.
- * @mas - the maple state
- * @mat - the ma_topiary linked list of dead nodes to free.
- *
- * Free walk a dead list.
- */
-static void mas_mat_free(struct ma_state *mas, struct ma_topiary *mat)
-{
-	struct maple_enode *next;
-
-	while (mat->head) {
-		next = mte_to_mat(mat->head)->next;
-		mas_free(mas, mat->head);
-		mat->head = next;
-	}
-}
-
+static void mt_free_walk(struct rcu_head *head);
+static void mt_destroy_walk(struct maple_enode *enode, struct maple_tree *mt,
+			    bool free);
 /*
  * mas_mat_destroy() - Free all nodes and subtrees in a dead list.
  * @mas - the maple state
@@ -1013,10 +995,15 @@ static void mas_mat_free(struct ma_state *mas, struct ma_topiary *mat)
 static void mas_mat_destroy(struct ma_state *mas, struct ma_topiary *mat)
 {
 	struct maple_enode *next;
+	struct maple_node *node;
+	bool in_rcu = mt_in_rcu(mas->tree);
 
 	while (mat->head) {
 		next = mte_to_mat(mat->head)->next;
-		mte_destroy_walk(mat->head, mat->mtree);
+		node = mte_to_node(mat->head);
+		mt_destroy_walk(mat->head, mas->tree, !in_rcu);
+		if (in_rcu)
+			call_rcu(&node->rcu, mt_free_walk);
 		mat->head = next;
 	}
 }
@@ -1759,11 +1746,11 @@ static inline void mas_replace_node(struct ma_state *mas,
 }
 
 /*
- * mas_new_child() - Find the new child of a node.
- * @mas: the maple state
+ * mas_find_child() - Find a child who has the parent @mas->node.
+ * @mas: the maple state with the parent.
  * @child: the maple state to store the child.
  */
-static inline bool mas_new_child(struct ma_state *mas, struct ma_state *child)
+static inline bool mas_find_child(struct ma_state *mas, struct ma_state *child)
 	__must_hold(mas->tree->ma_lock)
 {
 	enum maple_type mt;
@@ -2065,56 +2052,6 @@ static inline void mab_mas_cp(struct maple_big_node *b_node,
 	}
 }
 
-/*
- * mas_descend_adopt() - Descend through a sub-tree and adopt children.
- * @mas: the maple state with the maple encoded node of the sub-tree.
- *
- * Descend through a sub-tree and adopt children who do not have the correct
- * parents set.  Follow the parents which have the correct parents as they are
- * the new entries which need to be followed to find other incorrectly set
- * parents.
- */
-static inline void mas_descend_adopt(struct ma_state *mas)
-{
-	struct ma_state list[3], next[3];
-	int i, n;
-
-	/*
-	 * At each level there may be up to 3 correct parent pointers which indicates
-	 * the new nodes which need to be walked to find any new nodes at a lower level.
-	 */
-
-	for (i = 0; i < 3; i++) {
-		list[i] = *mas;
-		list[i].offset = 0;
-		next[i].offset = 0;
-	}
-	next[0] = *mas;
-
-	while (!mte_is_leaf(list[0].node)) {
-		n = 0;
-		for (i = 0; i < 3; i++) {
-			if (mas_is_none(&list[i]))
-				continue;
-
-			if (i && list[i-1].node == list[i].node)
-				continue;
-
-			while ((n < 3) && (mas_new_child(&list[i], &next[n])))
-				n++;
-
-			mas_adopt_children(&list[i], list[i].node);
-		}
-
-		while (n < 3)
-			next[n++].node = MAS_NONE;
-
-		/* descend by setting the list to the children */
-		for (i = 0; i < 3; i++)
-			list[i] = next[i];
-	}
-}
-
 /*
  * mas_bulk_rebalance() - Rebalance the end of a tree after a bulk insert.
  * @mas: The maple state
@@ -2304,98 +2241,6 @@ static inline void mas_wr_node_walk(struct ma_wr_state *wr_mas)
 	wr_mas->offset_end = mas->offset = offset;
 }
 
-/*
- * mas_topiary_range() - Add a range of slots to the topiary.
- * @mas: The maple state
- * @destroy: The topiary to add the slots (usually destroy)
- * @start: The starting slot inclusively
- * @end: The end slot inclusively
- */
-static inline void mas_topiary_range(struct ma_state *mas,
-	struct ma_topiary *destroy, unsigned char start, unsigned char end)
-{
-	void __rcu **slots;
-	unsigned char offset;
-
-	MAS_BUG_ON(mas, mte_is_leaf(mas->node));
-
-	slots = ma_slots(mas_mn(mas), mte_node_type(mas->node));
-	for (offset = start; offset <= end; offset++) {
-		struct maple_enode *enode = mas_slot_locked(mas, slots, offset);
-
-		if (mte_dead_node(enode))
-			continue;
-
-		mat_add(destroy, enode);
-	}
-}
-
-/*
- * mast_topiary() - Add the portions of the tree to the removal list; either to
- * be freed or discarded (destroy walk).
- * @mast: The maple_subtree_state.
- */
-static inline void mast_topiary(struct maple_subtree_state *mast)
-{
-	MA_WR_STATE(wr_mas, mast->orig_l, NULL);
-	unsigned char r_start, r_end;
-	unsigned char l_start, l_end;
-	void __rcu **l_slots, **r_slots;
-
-	wr_mas.type = mte_node_type(mast->orig_l->node);
-	mast->orig_l->index = mast->orig_l->last;
-	mas_wr_node_walk(&wr_mas);
-	l_start = mast->orig_l->offset + 1;
-	l_end = mas_data_end(mast->orig_l);
-	r_start = 0;
-	r_end = mast->orig_r->offset;
-
-	if (r_end)
-		r_end--;
-
-	l_slots = ma_slots(mas_mn(mast->orig_l),
-			   mte_node_type(mast->orig_l->node));
-
-	r_slots = ma_slots(mas_mn(mast->orig_r),
-			   mte_node_type(mast->orig_r->node));
-
-	if ((l_start < l_end) &&
-	    mte_dead_node(mas_slot_locked(mast->orig_l, l_slots, l_start))) {
-		l_start++;
-	}
-
-	if (mte_dead_node(mas_slot_locked(mast->orig_r, r_slots, r_end))) {
-		if (r_end)
-			r_end--;
-	}
-
-	if ((l_start > r_end) && (mast->orig_l->node == mast->orig_r->node))
-		return;
-
-	/* At the node where left and right sides meet, add the parts between */
-	if (mast->orig_l->node == mast->orig_r->node) {
-		return mas_topiary_range(mast->orig_l, mast->destroy,
-					     l_start, r_end);
-	}
-
-	/* mast->orig_r is different and consumed. */
-	if (mte_is_leaf(mast->orig_r->node))
-		return;
-
-	if (mte_dead_node(mas_slot_locked(mast->orig_l, l_slots, l_end)))
-		l_end--;
-
-
-	if (l_start <= l_end)
-		mas_topiary_range(mast->orig_l, mast->destroy, l_start, l_end);
-
-	if (mte_dead_node(mas_slot_locked(mast->orig_r, r_slots, r_start)))
-		r_start++;
-
-	if (r_start <= r_end)
-		mas_topiary_range(mast->orig_r, mast->destroy, 0, r_end);
-}
-
 /*
  * mast_rebalance_next() - Rebalance against the next node
  * @mast: The maple subtree state
@@ -2431,7 +2276,7 @@ static inline void mast_rebalance_prev(struct maple_subtree_state *mast)
 /*
  * mast_spanning_rebalance() - Rebalance nodes with nearest neighbour favouring
  * the node to the right.  Checking the nodes to the right then the left at each
- * level upwards until root is reached.  Free and destroy as needed.
+ * level upwards until root is reached.
  * Data is copied into the @mast->bn.
  * @mast: The maple_subtree_state.
  */
@@ -2440,8 +2285,6 @@ bool mast_spanning_rebalance(struct maple_subtree_state *mast)
 {
 	struct ma_state r_tmp = *mast->orig_r;
 	struct ma_state l_tmp = *mast->orig_l;
-	struct maple_enode *ancestor = NULL;
-	unsigned char start, end;
 	unsigned char depth = 0;
 
 	r_tmp = *mast->orig_r;
@@ -2450,87 +2293,25 @@ bool mast_spanning_rebalance(struct maple_subtree_state *mast)
 		mas_ascend(mast->orig_r);
 		mas_ascend(mast->orig_l);
 		depth++;
-		if (!ancestor &&
-		    (mast->orig_r->node == mast->orig_l->node)) {
-			ancestor = mast->orig_r->node;
-			end = mast->orig_r->offset - 1;
-			start = mast->orig_l->offset + 1;
-		}
-
 		if (mast->orig_r->offset < mas_data_end(mast->orig_r)) {
-			if (!ancestor) {
-				ancestor = mast->orig_r->node;
-				start = 0;
-			}
-
 			mast->orig_r->offset++;
 			do {
 				mas_descend(mast->orig_r);
 				mast->orig_r->offset = 0;
-				depth--;
-			} while (depth);
+			} while (--depth);
 
 			mast_rebalance_next(mast);
-			do {
-				unsigned char l_off = 0;
-				struct maple_enode *child = r_tmp.node;
-
-				mas_ascend(&r_tmp);
-				if (ancestor == r_tmp.node)
-					l_off = start;
-
-				if (r_tmp.offset)
-					r_tmp.offset--;
-
-				if (l_off < r_tmp.offset)
-					mas_topiary_range(&r_tmp, mast->destroy,
-							  l_off, r_tmp.offset);
-
-				if (l_tmp.node != child)
-					mat_add(mast->free, child);
-
-			} while (r_tmp.node != ancestor);
-
 			*mast->orig_l = l_tmp;
 			return true;
-
 		} else if (mast->orig_l->offset != 0) {
-			if (!ancestor) {
-				ancestor = mast->orig_l->node;
-				end = mas_data_end(mast->orig_l);
-			}
-
 			mast->orig_l->offset--;
 			do {
 				mas_descend(mast->orig_l);
 				mast->orig_l->offset =
 					mas_data_end(mast->orig_l);
-				depth--;
-			} while (depth);
+			} while (--depth);
 
 			mast_rebalance_prev(mast);
-			do {
-				unsigned char r_off;
-				struct maple_enode *child = l_tmp.node;
-
-				mas_ascend(&l_tmp);
-				if (ancestor == l_tmp.node)
-					r_off = end;
-				else
-					r_off = mas_data_end(&l_tmp);
-
-				if (l_tmp.offset < r_off)
-					l_tmp.offset++;
-
-				if (l_tmp.offset < r_off)
-					mas_topiary_range(&l_tmp, mast->destroy,
-							  l_tmp.offset, r_off);
-
-				if (r_tmp.node != child)
-					mat_add(mast->free, child);
-
-			} while (l_tmp.node != ancestor);
-
 			*mast->orig_r = r_tmp;
 			return true;
 		}
@@ -2542,36 +2323,24 @@ bool mast_spanning_rebalance(struct maple_subtree_state *mast)
 }
 
 /*
- * mast_ascend_free() - Add current original maple state nodes to the free list
- * and ascend.
+ * mast_ascend() - Ascend the original left and right maple states.
  * @mast: the maple subtree state.
  *
- * Ascend the original left and right sides and add the previous nodes to the
- * free list.  Set the slots to point to the correct location in the new nodes.
+ * Ascend the original left and right sides.  Set the offsets to point to the
+ * data already in the new tree (@mast->l and @mast->r).
  */
-static inline void
-mast_ascend_free(struct maple_subtree_state *mast)
+static inline void mast_ascend(struct maple_subtree_state *mast)
 {
 	MA_WR_STATE(wr_mas, mast->orig_r,  NULL);
-	struct maple_enode *left = mast->orig_l->node;
-	struct maple_enode *right = mast->orig_r->node;
-
 	mas_ascend(mast->orig_l);
 	mas_ascend(mast->orig_r);
-	mat_add(mast->free, left);
-
-	if (left != right)
-		mat_add(mast->free, right);
 
 	mast->orig_r->offset = 0;
 	mast->orig_r->index = mast->r->max;
 	/* last should be larger than or equal to index */
 	if (mast->orig_r->last < mast->orig_r->index)
 		mast->orig_r->last = mast->orig_r->index;
-	/*
-	 * The node may not contain the value so set slot to ensure all
-	 * of the nodes contents are freed or destroyed.
-	 */
+
 	wr_mas.type = mte_node_type(mast->orig_r->node);
 	mas_wr_node_walk(&wr_mas);
 	/* Set up the left side of things */
@@ -2750,66 +2519,152 @@ static inline void mast_set_split_parents(struct maple_subtree_state *mast,
 }
 
 /*
- * mas_wmb_replace() - Write memory barrier and replace
- * @mas: The maple state
- * @free: the maple topiary list of nodes to free
- * @destroy: The maple topiary list of nodes to destroy (walk and free)
+ * mas_topiary_node() - Dispose of a singe node
+ * @mas: The maple state for pushing nodes
+ * @enode: The encoded maple node
+ * @in_rcu: If the tree is in rcu mode
  *
- * Updates gap as necessary.
+ * The node will either be RCU freed or pushed back on the maple state.
  */
-static inline void mas_wmb_replace(struct ma_state *mas,
-				   struct ma_topiary *free,
-				   struct ma_topiary *destroy)
+static inline void mas_topiary_node(struct ma_state *mas,
+		struct maple_enode *enode, bool in_rcu)
 {
-	struct maple_enode *old_enode;
+	struct maple_node *tmp;
 
-	if (mte_is_root(mas->node)) {
-		old_enode = mas_root_locked(mas);
-	} else {
-		unsigned char offset = mte_parent_slot(mas->node);
-		void __rcu **slots = ma_slots(mte_parent(mas->node),
-					      mas_parent_type(mas, mas->node));
+	if (enode == MAS_NONE)
+		return;
 
-		old_enode = mas_slot_locked(mas, slots, offset);
-	}
+	tmp = mte_to_node(enode);
+	mte_set_node_dead(enode);
+	if (in_rcu)
+		ma_free_rcu(tmp);
+	else
+		mas_push_node(mas, tmp);
+}
 
-	/* Insert the new data in the tree */
+/*
+ * mas_topiary_replace() - Replace the data with new data, then repair the
+ * parent links within the new tree.  Iterate over the dead sub-tree and collect
+ * the dead subtrees and topiary the nodes that are no longer of use.
+ *
+ * The new tree will have up to three children with the correct parent.  Keep
+ * track of the new entries as they need to be followed to find the next level
+ * of new entries.
+ *
+ * The old tree will have up to three children with the old parent.  Keep track
+ * of the old entries as they may have more nodes below replaced.  Nodes within
+ * [index, last] are dead subtrees, others need to be freed and followed.
+ *
+ * @mas: The maple state pointing at the new data
+ * @old_enode: The maple encoded node being replaced
+ *
+ */
+static inline void mas_topiary_replace(struct ma_state *mas,
+		struct maple_enode *old_enode)
+{
+	struct ma_state tmp[3], tmp_next[3];
+	MA_TOPIARY(subtrees, mas->tree);
+	bool in_rcu;
+	int i, n;
+
+	/* Place data in tree & then mark node as old */
 	mas_put_in_tree(mas, old_enode);
 
-	if (!mte_is_leaf(mas->node))
-		mas_descend_adopt(mas);
+	/* Update the parent pointers in the tree */
+	tmp[0] = *mas;
+	tmp[0].offset = 0;
+	tmp[1].node = MAS_NONE;
+	tmp[2].node = MAS_NONE;
+	while (!mte_is_leaf(tmp[0].node)) {
+		n = 0;
+		for (i = 0; i < 3; i++) {
+			if (mas_is_none(&tmp[i]))
+				continue;
+
+			while (n < 3) {
+				if (!mas_find_child(&tmp[i], &tmp_next[n]))
+					break;
+				n++;
+			}
+
+			mas_adopt_children(&tmp[i], tmp[i].node);
+		}
 
-	mas_mat_free(mas, free);
+		if (MAS_WARN_ON(mas, n == 0))
+			break;
 
-	if (destroy)
-		mas_mat_destroy(mas, destroy);
+		while (n < 3)
+			tmp_next[n++].node = MAS_NONE;
 
-	if (mte_is_leaf(mas->node))
-		return;
+		for (i = 0; i < 3; i++)
+			tmp[i] = tmp_next[i];
+	}
 
-	mas_update_gap(mas);
+	/* Collect the old nodes that need to be discarded */
+	if (mte_is_leaf(old_enode))
+		return mas_free(mas, old_enode);
+
+	tmp[0] = *mas;
+	tmp[0].offset = 0;
+	tmp[0].node = old_enode;
+	tmp[1].node = MAS_NONE;
+	tmp[2].node = MAS_NONE;
+	in_rcu = mt_in_rcu(mas->tree);
+	do {
+		n = 0;
+		for (i = 0; i < 3; i++) {
+			if (mas_is_none(&tmp[i]))
+				continue;
+
+			while (n < 3) {
+				if (!mas_find_child(&tmp[i], &tmp_next[n]))
+					break;
+
+				if ((tmp_next[n].min >= tmp_next->index) &&
+				    (tmp_next[n].max <= tmp_next->last)) {
+					mat_add(&subtrees, tmp_next[n].node);
+					tmp_next[n].node = MAS_NONE;
+				} else {
+					n++;
+				}
+			}
+		}
+
+		if (MAS_WARN_ON(mas, n == 0))
+			break;
+
+		while (n < 3)
+			tmp_next[n++].node = MAS_NONE;
+
+		for (i = 0; i < 3; i++) {
+			mas_topiary_node(mas, tmp[i].node, in_rcu);
+			tmp[i] = tmp_next[i];
+		}
+	} while (!mte_is_leaf(tmp[0].node));
+
+	for (i = 0; i < 3; i++)
+		mas_topiary_node(mas, tmp[i].node, in_rcu);
+
+	mas_mat_destroy(mas, &subtrees);
 }
 
 /*
- * mast_new_root() - Set a new tree root during subtree creation
- * @mast: The maple subtree state
+ * mas_wmb_replace() - Write memory barrier and replace
  * @mas: The maple state
+ * @old: The old maple encoded node that is being replaced.
+ *
+ * Updates gap as necessary.
  */
-static inline void mast_new_root(struct maple_subtree_state *mast,
-				 struct ma_state *mas)
+static inline void mas_wmb_replace(struct ma_state *mas,
+		struct maple_enode *old_enode)
 {
-	mas_mn(mast->l)->parent = ma_parent_ptr(mas_tree_parent(mas));
-	if (!mte_dead_node(mast->orig_l->node) &&
-	    !mte_is_root(mast->orig_l->node)) {
-		do {
-			mast_ascend_free(mast);
-			mast_topiary(mast);
-		} while (!mte_is_root(mast->orig_l->node));
-	}
-	if ((mast->orig_l->node != mas->node) &&
-		   (mast->l->depth > mas_mt_height(mas))) {
-		mat_add(mast->free, mas->node);
-	}
+	/* Insert the new data in the tree */
+	mas_topiary_replace(mas, old_enode);
+
+	if (mte_is_leaf(mas->node))
+		return;
+
+	mas_update_gap(mas);
 }
 
 /*
@@ -2995,12 +2850,11 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 	unsigned char split, mid_split;
 	unsigned char slot = 0;
 	struct maple_enode *left = NULL, *middle = NULL, *right = NULL;
+	struct maple_enode *old_enode;
 
 	MA_STATE(l_mas, mas->tree, mas->index, mas->index);
 	MA_STATE(r_mas, mas->tree, mas->index, mas->last);
 	MA_STATE(m_mas, mas->tree, mas->index, mas->index);
-	MA_TOPIARY(free, mas->tree);
-	MA_TOPIARY(destroy, mas->tree);
 
 	/*
 	 * The tree needs to be rebalanced and leaves need to be kept at the same level.
@@ -3009,8 +2863,6 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 	mast->l = &l_mas;
 	mast->m = &m_mas;
 	mast->r = &r_mas;
-	mast->free = &free;
-	mast->destroy = &destroy;
 	l_mas.node = r_mas.node = m_mas.node = MAS_NONE;
 
 	/* Check if this is not root and has sufficient data.  */
@@ -3018,7 +2870,7 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 	    unlikely(mast->bn->b_end <= mt_min_slots[mast->bn->type]))
 		mast_spanning_rebalance(mast);
 
-	mast->orig_l->depth = 0;
+	l_mas.depth = 0;
 
 	/*
 	 * Each level of the tree is examined and balanced, pushing data to the left or
@@ -3029,7 +2881,7 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 	 * original tree and the partially new tree.  To remedy the parent pointers in
 	 * the old tree, the new data is swapped into the active tree and a walk down
 	 * the tree is performed and the parent pointers are updated.
-	 * See mas_descend_adopt() for more information..
+	 * See mas_topiary_replace() for more information.
 	 */
 	while (count--) {
 		mast->bn->b_end--;
@@ -3046,13 +2898,13 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 		 */
 		memset(mast->bn, 0, sizeof(struct maple_big_node));
 		mast->bn->type = mte_node_type(left);
-		mast->orig_l->depth++;
+		l_mas.depth++;
 
 		/* Root already stored in l->node. */
 		if (mas_is_root_limits(mast->l))
 			goto new_root;
 
-		mast_ascend_free(mast);
+		mast_ascend(mast);
 		mast_combine_cp_left(mast);
 		l_mas.offset = mast->bn->b_end;
 		mab_set_b_end(mast->bn, &l_mas, left);
@@ -3061,7 +2913,6 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 
 		/* Copy anything necessary out of the right node. */
 		mast_combine_cp_right(mast);
-		mast_topiary(mast);
 		mast->orig_l->last = mast->orig_l->max;
 
 		if (mast_sufficient(mast))
@@ -3083,7 +2934,7 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 
 	l_mas.node = mt_mk_node(ma_mnode_ptr(mas_pop_node(mas)),
 				mte_node_type(mast->orig_l->node));
-	mast->orig_l->depth++;
+	l_mas.depth++;
 	mab_mas_cp(mast->bn, 0, mt_slots[mast->bn->type] - 1, &l_mas, true);
 	mas_set_parent(mas, left, l_mas.node, slot);
 	if (middle)
@@ -3094,23 +2945,20 @@ static int mas_spanning_rebalance(struct ma_state *mas,
 
 	if (mas_is_root_limits(mast->l)) {
 new_root:
-		mast_new_root(mast, mas);
+		mas_mn(mast->l)->parent = ma_parent_ptr(mas_tree_parent(mas));
+		while (!mte_is_root(mast->orig_l->node))
+			mast_ascend(mast);
 	} else {
 		mas_mn(&l_mas)->parent = mas_mn(mast->orig_l)->parent;
 	}
 
-	if (!mte_dead_node(mast->orig_l->node))
-		mat_add(&free, mast->orig_l->node);
-
-	mas->depth = mast->orig_l->depth;
-	*mast->orig_l = l_mas;
-	mte_set_node_dead(mas->node);
-
-	/* Set up mas for insertion. */
-	mast->orig_l->depth = mas->depth;
-	mast->orig_l->alloc = mas->alloc;
-	*mas = *mast->orig_l;
-	mas_wmb_replace(mas, &free, &destroy);
+	old_enode = mast->orig_l->node;
+	mas->depth = l_mas.depth;
+	mas->node = l_mas.node;
+	mas->min = l_mas.min;
+	mas->max = l_mas.max;
+	mas->offset = l_mas.offset;
+	mas_wmb_replace(mas, old_enode);
 	mtree_range_walk(mas);
 	return mast->bn->b_end;
 }
@@ -3341,7 +3189,6 @@ static inline void mast_fill_bnode(struct maple_subtree_state *mast,
 					 unsigned char skip)
 {
 	bool cp = true;
-	struct maple_enode *old = mas->node;
 	unsigned char split;
 
 	memset(mast->bn->gap, 0, sizeof(unsigned long) * ARRAY_SIZE(mast->bn->gap));
@@ -3353,7 +3200,6 @@ static inline void mast_fill_bnode(struct maple_subtree_state *mast,
 		cp = false;
 	} else {
 		mas_ascend(mas);
-		mat_add(mast->free, old);
 		mas->offset = mte_parent_slot(mas->node);
 	}
 
@@ -3457,13 +3303,11 @@ static inline bool mas_push_data(struct ma_state *mas, int height,
 	split = mt_slots[mast->bn->type] - 2;
 	if (left) {
 		/*  Switch mas to prev node  */
-		mat_add(mast->free, mas->node);
 		*mas = tmp_mas;
 		/* Start using mast->l for the left side. */
 		tmp_mas.node = mast->l->node;
 		*mast->l = tmp_mas;
 	} else {
-		mat_add(mast->free, tmp_mas.node);
 		tmp_mas.node = mast->r->node;
 		*mast->r = tmp_mas;
 		split = slot_total - split;
@@ -3490,6 +3334,7 @@ static int mas_split(struct ma_state *mas, struct maple_big_node *b_node)
 	struct maple_subtree_state mast;
 	int height = 0;
 	unsigned char mid_split, split = 0;
+	struct maple_enode *old;
 
 	/*
 	 * Splitting is handled differently from any other B-tree; the Maple
@@ -3512,7 +3357,6 @@ static int mas_split(struct ma_state *mas, struct maple_big_node *b_node)
 	MA_STATE(r_mas, mas->tree, mas->index, mas->last);
 	MA_STATE(prev_l_mas, mas->tree, mas->index, mas->last);
 	MA_STATE(prev_r_mas, mas->tree, mas->index, mas->last);
-	MA_TOPIARY(mat, mas->tree);
 
 	trace_ma_op(__func__, mas);
 	mas->depth = mas_mt_height(mas);
@@ -3525,7 +3369,6 @@ static int mas_split(struct ma_state *mas, struct maple_big_node *b_node)
 	mast.r = &r_mas;
 	mast.orig_l = &prev_l_mas;
 	mast.orig_r = &prev_r_mas;
-	mast.free = &mat;
 	mast.bn = b_node;
 
 	while (height++ <= mas->depth) {
@@ -3565,9 +3408,9 @@ static int mas_split(struct ma_state *mas, struct maple_big_node *b_node)
 	}
 
 	/* Set the original node as dead */
-	mat_add(mast.free, mas->node);
+	old = mas->node;
 	mas->node = l_mas.node;
-	mas_wmb_replace(mas, mast.free, NULL);
+	mas_wmb_replace(mas, old);
 	mtree_range_walk(mas);
 	return 1;
 }
@@ -3903,6 +3746,7 @@ static inline void *mtree_lookup_walk(struct ma_state *mas)
 	return NULL;
 }
 
+static void mte_destroy_walk(struct maple_enode *, struct maple_tree *);
 /*
  * mas_new_root() - Create a new root node that only contains the entry passed
  * in.
@@ -3969,7 +3813,6 @@ static inline int mas_wr_spanning_store(struct ma_wr_state *wr_mas)
 	/* Left and Right side of spanning store */
 	MA_STATE(l_mas, NULL, 0, 0);
 	MA_STATE(r_mas, NULL, 0, 0);
-
 	MA_WR_STATE(r_wr_mas, &r_mas, wr_mas->entry);
 	MA_WR_STATE(l_wr_mas, &l_mas, wr_mas->entry);
 
-- 
2.39.2



From John.Hsu at mediatek.com  Mon Aug  7 02:54:52 2023
From: John.Hsu at mediatek.com (=?utf-8?B?Sm9obiBIc3UgKOioseawuOe/sCk=?=)
Date: Mon, 7 Aug 2023 09:54:52 +0000
Subject: [BUG] trigger BUG_ON in mas_store_prealloc when low memory
In-Reply-To: <20230719185135.nyun5fvcuv3ydku4@revolver>
References: <9cb8c599b1d7f9c1c300d1a334d5eb70ec4d7357.camel@mediatek.com>
	 <20230613141114.bwbnqsdazqbmyj3u@revolver>
	 <3b14df2fc2a7f18fe12f87a27574b7d40f2899ba.camel@mediatek.com>
	 <20230614155826.cxcpv33hs763gyrg@revolver>
	 <ed7899c7328aabfbe9bc9589f0776a75e6c9d1ee.camel@mediatek.com>
	 <20230706185436.agobbv72o3hma43z@revolver>
	 <3507fdacf001c49f4c73f2952257beabfe9e1a53.camel@mediatek.com>
	 <20230710142425.t5vhrqkjd7ugbmad@revolver>
	 <f7aec937af73798c5e42b7e37adb744df5562cca.camel@mediatek.com>
	 <20230719185135.nyun5fvcuv3ydku4@revolver>
Message-ID: <981aac92a51c4e2cfd2ca83d11742e5741e60285.camel@mediatek.com>

On Wed, 2023-07-19 at 14:51 -0400, Liam R. Howlett wrote:
>  	 
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
>  * John Hsu (???) <John.Hsu at mediatek.com> [230712 23:26]:
> > On Mon, 2023-07-10 at 10:24 -0400, Liam R. Howlett wrote:
> 
> ...
> 
> > > > > > > > As you mentioned, mas_preallocate() should allocate
> enough
> > > > > node,
> > > > > > > but there is such functions mas_node_count() in
> > > > > mas_store_prealloc().
> > > > > > > > In mas_node_count() checks whether the *mas* has enough
> > > nodes,
> > > > > and
> > > > > > > allocate memory for node if there was no enough nodes in
> mas.
> > > > > > > 
> > > > > > > Right, we call mas_node_count() so that both code paths
> are
> > > used
> > > > > for
> > > > > > > preallocations and regular
> mas_store()/mas_store_gfp().  It
> > > > > shouldn't
> > > > > > > take a significant amount of time to verify there is
> enough
> > > > > nodes.
> > > > > > 
> > > > > > Yap..., it didn't take a significant amount of time to
> verify
> > > > > whether
> > > > > > there is enough nodes. The problem is why the flow in
> > > > > mas_node_count
> > > > > > will alloc nodes if there was no enough nodes in mas?
> > > > > 
> > > > > What I meant is that both methods use the same call path
> because
> > > > > there
> > > > > is not a reason to duplicate the path.  After
> mas_preallocate()
> > > has
> > > > > allocated the nodes needed, the call to check if there is
> enough
> > > > > nodes
> > > > > will be quick.
> > > > 
> > > > So whether the purpose of mas_preallocate() is decreasing the
> lock
> > > > retention time?
> > > 
> > > It could be, but in this case it's the locking order.  We have to
> > > pre-allocate and fail early if we are out of memory, because we
> > > _cannot_
> > > use GFP_KERNEL where we call
> mas_store_prealloc().  mas_preallocate()
> > > will use GFP_KERENL though.  We cannot use GFP_KERNEL during the
> > > store
> > > because reclaim may sleep and we cannot sleep holding the locks
> we
> > > need
> > > to hold at the time of the store operation in __vma_adjust().
> > > 
> > Yap, if the type of lock is spinlock, the flow shouldn't sleep in
> the
> > critical sections. mmap_lock is implmented by rw_semaphore(mutex).
> Is
> > there any other lock in this section?
> 
> Yes, the i_mmap_lock_write(), the anon_vma_lock_write(),
> flush_dcache_mmap_lock().
> 
> > > > 
> > > > > > 
> > > > > > > > I think that if mas_preallocate() allocate enough node,
> why
> > > we
> > > > > > > check the node count and allocate nodes if there was no
> > > enough
> > > > > nodes
> > > > > > > in mas in mas_node_count()?
> > > > > > > 
> > > > > > > We check for the above reason.
> > > > > > > 
> > > > > > 
> > > > > > OK..., this is one of the root cause of this BUG.
> > > > > 
> > > > > The root cause is that there was not enough memory for a
> store
> > > > > operation.  Regardless of if we check the allocations in the
> > > > > mas_store_prealloc() path or not, this would fail.  If we
> remove
> > > the
> > > > > check for nodes within this path, then we would have to
> BUG_ON()
> > > when
> > > > > we
> > > > > run out of nodes to use or have a null pointer dereference
> BUG
> > > > > anyways.
> > > > > 
> > > > Yap, the root cause is oom. The BUG_ON() for the situations
> that
> > > the
> > > > maple tree struct cannot be maintained because of the lack of
> > > memory is
> > > > necessary. But the the buddy system in linux kernel can reclaim
> > > memory
> > > > when the system is under the low memory status. If we use
> > > GFP_KERNEL
> > > > after trying GFP_NOWAIT to allocate node, maybe we can get
> enough
> > > > memory when the second try with GFP_KERNEL. 
> > > 
> > > Right, but the GFP_KERNEL cannot be used when holding certain
> locks
> > > because it may sleep.
> > > 
> > > > > > 
> > > > > > > > 
> > > > > > > > We have seen that there may be some maple_tree
> operations
> > > in
> > > > > > > merge_vma...
> > > > > > > 
> > > > > > > If merge_vma() does anything, then there was an operation
> to
> > > the
> > > > > > > maple
> > > > > > > tree.
> > > > > > > 
> > > > > > > > 
> > > > > > > > Moreover, would maple_tree provides an API for
> assigning
> > > user's
> > > > > gfp
> > > > > > > flag for allocating node?
> > > > > > > 
> > > > > > > mas_preallocate() and mas_store_gfp() has gfp flags as an
> > > > > > > argument.  In
> > > > > > > your call stack, it will be called in __vma_adjust() as
> such:
> > > > > > > 
> > > > > > > if (mas_preallocate(&mas, vma, GFP_KERNEL))
> > > > > > > return -ENOMEM;
> > > > > > > 
> > > > > > > line 715 in v6.1.25
> > > > > > > 
> > > > > > > > In rb_tree, we allocate vma_area_struct (rb_node is in
> this
> > > > > > > struct.) with GFP_KERNEL, and maple_tree allocate node
> with
> > > > > > > GFP_NOWAIT and __GFP_NOWARN.
> > > > > > > 
> > > > > > > We use GFP_KERNEL as I explained above for the VMA tree.
> > > > > > 
> > > > > > Got it! But the mas_node_count() always use GFP_NOWAIT and
> > > > > __GFP_NOWARN
> > > > > > in inserting tree flow. Do you consider the performance of
> > > > > maintaining
> > > > > > the structure of maple_tree?
> > > > > 
> > > > > Sorry, I don't understand what you mean by 'consider the
> > > performance
> > > > > of
> > > > > maintaining the structure of maple_tree'.
> > > > > 
> > > > As I mentioned above, GFP_NOWAIT will not allow buddy system
> for
> > > > reclaiming memory, so "Do you consider the performance of
> > > maintaining
> > > > the structure of maple_tree" means that: whether the
> > > mas_node_count()
> > > > path is not allowed to reclaim or compact memory for the
> > > performance.
> > > 
> > > Ah, no.  This is not for performance.  It was initially on the
> road
> > > map
> > > for performance, but it was needed for the complicated locking in
> the
> > > MM
> > > code.
> > > 
> > > rb tree embedded the nodes in the VMA which is allocated outside
> this
> > > critical section and so it could use GFP_KERNEL.
> > > 
> > As I know, following is rb_tree flow in 5.15.186:
> > 
> > ...
> > mmap_write_lock_killable(mm)
> > ...
> > do_mmap()
> > ...
> > mmap_region()
> > ...
> > vm_area_alloc(mm)
> > ...
> > mmap_write_unlock(mm)
> > 
> > vm_area_alloc is in the mmap_lock hoding period.
> > It seems that the flow would sleep here in rb_tree flow.
> > If I miss anything, please tell me, thanks!
> 
> Before the mmap_write_unlock(mm) in the above sequence,  the
> i_mmap_lock_write(), anon_vma_lock_write(), and/or the
> flush_dcache_mmap_lock() may be taken.  Check __vma_adjust().
> 
> The insertion into the tree needs to hold some subset of these locks.
> The rb-tree insert did not allocate within these locks, but the maple
> tree would need to allocate within these locks to insert into the
> tree.
> This is why the preallocation exists and why it is necessary.
> 

Yap, preallocation is necessary. anon_vma_lock_write() and
flush_dcache_mmap_lock() hold the lock and manipulate rb_tree. I think
that there is no maple tree manipulations during the lock holding
period. Is there any future work in this section?

> > 
> > 
> > > > > > 
> > > > > > > It also will drop the lock and retry with GFP_KERNEL on
> > > failure
> > > > > > > when not using the external lock.  The mmap_lock is
> > > configured as
> > > > > an
> > > > > > > external lock.
> > > > > > > 
> > > > > > > > Allocation will not wait for reclaiming and compacting
> when
> > > > > there
> > > > > > > is no enough available memory.
> > > > > > > > Is there any concern for this design?
> > > > > > > 
> > > > > > > This has been addressed above, but let me know if I
> missed
> > > > > anything
> > > > > > > here.
> > > > > > > 
> > > > > > 
> > > > > > I think that the mas_node_count() has higher rate of
> triggering
> > > > > > BUG_ON() when allocating nodes with GFP_NOWAIT and
> > > __GFP_NOWARN. If
> > > > > > mas_node_count() use GFP_KERNEL as mas_preallocate() in the
> > > mmap.c,
> > > > > the
> > > > > > allocation fail rate may be lower than use GFP_NOWAIT.
> > > > > 
> > > > > Which BUG_ON() are you referring to?
> > > > > 
> > > > > If I was to separate the code path for mas_store_prealloc()
> and
> > > > > mas_store_gfp(), then a BUG_ON() would still need to exist
> and
> > > still
> > > > > would have been triggered..  We are in a place in the code
> where
> > > we
> > > > > should never sleep and we don't have enough memory allocated
> to
> > > do
> > > > > what
> > > > > was necessary.
> > > > > 
> > > > Yap. There is no reason to seprate mas_store_prealloc() and
> > > > mas_store_gfp. Is it possible to retry to allocate mas_node
> with
> > > > GFP_KERNEL (wait for system reclaim and compact) instead of
> > > triggering
> > > > BUG_ON once the GFP_NOWAIT allocation failed?
> > > 
> > > Unfortunately not, no.  In some cases it may actually work out
> that
> > > the
> > > VMA may not need the locks in question, but it cannot be
> generalized
> > > for
> > > __vma_adjust().  Where I am able, I use the mas_store_gfp() calls
> so
> > > we
> > > can let reclaim and compact run, but it's not possible here.
> > > 
> > We have used GFP_KERNEL as alloc flag in mas_node_count flow about
> 2
> > months. The mentioned problem we mentioned in the first mail didn't
> > reproduce under high stress stability test.
> > 
> > I have seen the patch provided by you. I will verify this patch in
> our
> > stability test. But there is a problem, if maple_tree behavior is
> > unexpected (e.g. redundant write bug this time). We can only review
> the
> > whole mm flow to find out the wrong behavior. Do we have an
> efficient
> > debug method for maple tree?
> 
> There is a test suite for the maple tree found in
> tools/testing/radix-tree, but it does not test the mm code and
> integration.
> 
> There are other mm tests, but they will not test the OOM scenario you
> hit - to the best of my knowledge.
> 
> There are also config options to debug the tree operations, but they
> do
> not detect the redundant write issues.  Perhaps I can look at adding
> support for detecting redundant writes,  but that will not be
> backported
> to a stable kernel.
> 

The sufficient test cases of maple tree ensure the function work well.
But the redundant operations (alloc node, free node, tree
manipulations) of maple_tree are not easy to detect (e.g. the case
reported this time and mas_preallocate() allocates redundant nodes with
the worst case).

The detecting redundant writes mechanism may help the developers to
find out the problems easier. Hope it can be establised successfully!!
> Thanks,
> Liam

Best Regards,
John Hsu

From Liam.Howlett at Oracle.com  Tue Aug  8 13:00:57 2023
From: Liam.Howlett at Oracle.com (Liam R. Howlett)
Date: Tue, 8 Aug 2023 16:00:57 -0400
Subject: [BUG] trigger BUG_ON in mas_store_prealloc when low memory
In-Reply-To: <981aac92a51c4e2cfd2ca83d11742e5741e60285.camel@mediatek.com>
References: <20230613141114.bwbnqsdazqbmyj3u@revolver>
 <3b14df2fc2a7f18fe12f87a27574b7d40f2899ba.camel@mediatek.com>
 <20230614155826.cxcpv33hs763gyrg@revolver>
 <ed7899c7328aabfbe9bc9589f0776a75e6c9d1ee.camel@mediatek.com>
 <20230706185436.agobbv72o3hma43z@revolver>
 <3507fdacf001c49f4c73f2952257beabfe9e1a53.camel@mediatek.com>
 <20230710142425.t5vhrqkjd7ugbmad@revolver>
 <f7aec937af73798c5e42b7e37adb744df5562cca.camel@mediatek.com>
 <20230719185135.nyun5fvcuv3ydku4@revolver>
 <981aac92a51c4e2cfd2ca83d11742e5741e60285.camel@mediatek.com>
Message-ID: <20230808200057.p3pqwkumghqbdvbi@revolver>

* John Hsu (???) <John.Hsu at mediatek.com> [230807 05:55]:
> On Wed, 2023-07-19 at 14:51 -0400, Liam R. Howlett wrote:

...

> > > As I know, following is rb_tree flow in 5.15.186:
> > > 
> > > ...
> > > mmap_write_lock_killable(mm)
> > > ...
> > > do_mmap()
> > > ...
> > > mmap_region()
> > > ...
> > > vm_area_alloc(mm)
> > > ...
> > > mmap_write_unlock(mm)
> > > 
> > > vm_area_alloc is in the mmap_lock hoding period.
> > > It seems that the flow would sleep here in rb_tree flow.
> > > If I miss anything, please tell me, thanks!
> > 
> > Before the mmap_write_unlock(mm) in the above sequence,  the
> > i_mmap_lock_write(), anon_vma_lock_write(), and/or the
> > flush_dcache_mmap_lock() may be taken.  Check __vma_adjust().
> > 
> > The insertion into the tree needs to hold some subset of these locks.
> > The rb-tree insert did not allocate within these locks, but the maple
> > tree would need to allocate within these locks to insert into the
> > tree.
> > This is why the preallocation exists and why it is necessary.
> > 
> 
> Yap, preallocation is necessary. anon_vma_lock_write() and
> flush_dcache_mmap_lock() hold the lock and manipulate rb_tree. I think
> that there is no maple tree manipulations during the lock holding
> period. Is there any future work in this section?

__vma_adjust() does modify the maple tree during the lock holding
section through vma_mas_store() in 6.1.  Prior to 6.1, there is no maple
tree.

...

> > There are also config options to debug the tree operations, but they
> > do
> > not detect the redundant write issues.  Perhaps I can look at adding
> > support for detecting redundant writes,  but that will not be
> > backported
> > to a stable kernel.
> > 
> 
> The sufficient test cases of maple tree ensure the function work well.
> But the redundant operations (alloc node, free node, tree
> manipulations) of maple_tree are not easy to detect (e.g. the case
> reported this time and mas_preallocate() allocates redundant nodes with
> the worst case).
> 
> The detecting redundant writes mechanism may help the developers to
> find out the problems easier. Hope it can be establised successfully!!

When I went to add this, I had found I already added it here [1].

This operation was not caught by MA_STATE_PREALLOC because there are two
writes before a mas_destroy(), so there may be nodes left which avoid
the warning.  I'll look at improving this situation.

Thanks,
Liam


[1] https://lore.kernel.org/linux-mm/20220722160546.1478722-2-Liam.Howlett at oracle.com/


From Liam.Howlett at oracle.com  Thu Aug 17 12:15:04 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Thu, 17 Aug 2023 15:15:04 -0400
Subject: [PATCH 1/2] maple_tree: Disable mas_wr_append() when other readers are possible
Message-ID: <20230817191505.1170876-1-Liam.Howlett@oracle.com>

The current implementation of append may cause duplicate data and/or
incorrect ranges to be returned to a reader during an update.  Although
this has not been reported or seen, disable the append write operation
while the tree is in rcu mode out of an abundance of caution.

During the analysis of the mas_next_slot() the following was
artificially created by separating the writer and reader code:

Writer:                                 reader:
mas_wr_append
    set end pivot
    updates end metata
    Detects write to last slot
    last slot write is to start of slot
    store current contents in slot
    overwrite old end pivot
                                        mas_next_slot():
                                                read end metadata
                                                read old end pivot
                                                return with incorrect range
    store new value

Alternatively:

Writer:                                 reader:
mas_wr_append
    set end pivot
    updates end metata
    Detects write to last slot
    last lost write to end of slot
    store value
                                        mas_next_slot():
                                                read end metadata
                                                read old end pivot
                                                read new end pivot
                                                return with incorrect range
    set old end pivot

There may be other accesses that are not safe since we are now updating
both metadata and pointers, so disabling append if there could be rcu
readers is the safest action.

Cc: stable at vger.kernel.org
Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index ffb9d15bd815..05d5db255c39 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -4107,6 +4107,10 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
  * mas_wr_append: Attempt to append
  * @wr_mas: the maple write state
  *
+ * This is currently unsafe in rcu mode since the end of the node may be cached
+ * by readers while the node contents may be updated which could result in
+ * inaccurate information.
+ *
  * Return: True if appended, false otherwise
  */
 static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
@@ -4116,6 +4120,9 @@ static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
 	struct ma_state *mas = wr_mas->mas;
 	unsigned char node_pivots = mt_pivots[wr_mas->type];
 
+	if (mt_in_rcu(mas->tree))
+		return false;
+
 	if (mas->offset != wr_mas->node_end)
 		return false;
 
-- 
2.39.2



From Liam.Howlett at oracle.com  Thu Aug 17 12:15:05 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Thu, 17 Aug 2023 15:15:05 -0400
Subject: [PATCH 2/2] maple_tree: Clean up mas_wr_append()
In-Reply-To: <20230817191505.1170876-1-Liam.Howlett@oracle.com>
References: <20230817191505.1170876-1-Liam.Howlett@oracle.com>
Message-ID: <20230817191505.1170876-2-Liam.Howlett@oracle.com>

Avoid setting the variables until necessary, and actually use the
variables where applicable.  Introducing a variable for the slots array
avoids spanning multiple lines.

Add the missing argument to the documentation.

Use the node type when setting the metadata instead of blindly assuming
the type.

Finally, add a trace point to the function for successful store.

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 05d5db255c39..ee1ff0c59fd7 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -4106,6 +4106,7 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
 /*
  * mas_wr_append: Attempt to append
  * @wr_mas: the maple write state
+ * @new_end: The end of the node after the modification
  *
  * This is currently unsafe in rcu mode since the end of the node may be cached
  * by readers while the node contents may be updated which could result in
@@ -4114,42 +4115,46 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
  * Return: True if appended, false otherwise
  */
 static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
-				 unsigned char new_end)
+		unsigned char new_end)
 {
-	unsigned char end = wr_mas->node_end;
-	struct ma_state *mas = wr_mas->mas;
-	unsigned char node_pivots = mt_pivots[wr_mas->type];
+	struct ma_state *mas;
+	void __rcu **slots;
+	unsigned char end;
 
+	mas = wr_mas->mas;
 	if (mt_in_rcu(mas->tree))
 		return false;
 
 	if (mas->offset != wr_mas->node_end)
 		return false;
 
-	if (new_end < node_pivots) {
+	end = wr_mas->node_end;
+	if (mas->offset != end)
+		return false;
+
+	if (new_end < mt_pivots[wr_mas->type]) {
 		wr_mas->pivots[new_end] = wr_mas->pivots[end];
-		ma_set_meta(wr_mas->node, maple_leaf_64, 0, new_end);
+		ma_set_meta(wr_mas->node, wr_mas->type, 0, new_end);
 	}
 
-	if (new_end == wr_mas->node_end + 1) {
+	slots = wr_mas->slots;
+	if (new_end == end + 1) {
 		if (mas->last == wr_mas->r_max) {
 			/* Append to end of range */
-			rcu_assign_pointer(wr_mas->slots[new_end],
-					   wr_mas->entry);
+			rcu_assign_pointer(slots[new_end], wr_mas->entry);
 			wr_mas->pivots[end] = mas->index - 1;
 			mas->offset = new_end;
 		} else {
 			/* Append to start of range */
-			rcu_assign_pointer(wr_mas->slots[new_end],
-					   wr_mas->content);
+			rcu_assign_pointer(slots[new_end], wr_mas->content);
 			wr_mas->pivots[end] = mas->last;
-			rcu_assign_pointer(wr_mas->slots[end], wr_mas->entry);
+			rcu_assign_pointer(slots[end], wr_mas->entry);
 		}
 	} else {
 		/* Append to the range without touching any boundaries. */
-		rcu_assign_pointer(wr_mas->slots[new_end], wr_mas->content);
+		rcu_assign_pointer(slots[new_end], wr_mas->content);
 		wr_mas->pivots[end + 1] = mas->last;
-		rcu_assign_pointer(wr_mas->slots[end + 1], wr_mas->entry);
+		rcu_assign_pointer(slots[end + 1], wr_mas->entry);
 		wr_mas->pivots[end] = mas->index - 1;
 		mas->offset = end + 1;
 	}
@@ -4157,6 +4162,7 @@ static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
 	if (!wr_mas->content || !wr_mas->entry)
 		mas_update_gap(mas);
 
+	trace_ma_write(__func__, mas, new_end, wr_mas->entry);
 	return  true;
 }
 
-- 
2.39.2



From Liam.Howlett at Oracle.com  Thu Aug 17 16:56:54 2023
From: Liam.Howlett at Oracle.com (Liam R. Howlett)
Date: Thu, 17 Aug 2023 19:56:54 -0400
Subject: [PATCH 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <20230817191505.1170876-1-Liam.Howlett@oracle.com>
References: <20230817191505.1170876-1-Liam.Howlett@oracle.com>
Message-ID: <20230817235654.z7e7fi2gofnsxniw@revolver>

I forgot the fixes tag, I will include that in v2.

* Liam R. Howlett <Liam.Howlett at oracle.com> [230817 15:15]:
> The current implementation of append may cause duplicate data and/or
> incorrect ranges to be returned to a reader during an update.  Although
> this has not been reported or seen, disable the append write operation
> while the tree is in rcu mode out of an abundance of caution.
> 
> During the analysis of the mas_next_slot() the following was
> artificially created by separating the writer and reader code:
> 
> Writer:                                 reader:
> mas_wr_append
>     set end pivot
>     updates end metata
>     Detects write to last slot
>     last slot write is to start of slot
>     store current contents in slot
>     overwrite old end pivot
>                                         mas_next_slot():
>                                                 read end metadata
>                                                 read old end pivot
>                                                 return with incorrect range
>     store new value
> 
> Alternatively:
> 
> Writer:                                 reader:
> mas_wr_append
>     set end pivot
>     updates end metata
>     Detects write to last slot
>     last lost write to end of slot
>     store value
>                                         mas_next_slot():
>                                                 read end metadata
>                                                 read old end pivot
>                                                 read new end pivot
>                                                 return with incorrect range
>     set old end pivot
> 
> There may be other accesses that are not safe since we are now updating
> both metadata and pointers, so disabling append if there could be rcu
> readers is the safest action.
> 
> Cc: stable at vger.kernel.org
> Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
> ---
>  lib/maple_tree.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/lib/maple_tree.c b/lib/maple_tree.c
> index ffb9d15bd815..05d5db255c39 100644
> --- a/lib/maple_tree.c
> +++ b/lib/maple_tree.c
> @@ -4107,6 +4107,10 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
>   * mas_wr_append: Attempt to append
>   * @wr_mas: the maple write state
>   *
> + * This is currently unsafe in rcu mode since the end of the node may be cached
> + * by readers while the node contents may be updated which could result in
> + * inaccurate information.
> + *
>   * Return: True if appended, false otherwise
>   */
>  static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
> @@ -4116,6 +4120,9 @@ static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
>  	struct ma_state *mas = wr_mas->mas;
>  	unsigned char node_pivots = mt_pivots[wr_mas->type];
>  
> +	if (mt_in_rcu(mas->tree))
> +		return false;
> +
>  	if (mas->offset != wr_mas->node_end)
>  		return false;
>  
> -- 
> 2.39.2
> 


From Liam.Howlett at Oracle.com  Fri Aug 18 09:13:39 2023
From: Liam.Howlett at Oracle.com (Liam R. Howlett)
Date: Fri, 18 Aug 2023 12:13:39 -0400
Subject: [PATCH 04/11] maple_tree: Introduce interfaces __mt_dup() and
 mt_dup()
In-Reply-To: <6b6d7ef1-75e4-68a3-1662-82ee19334567@bytedance.com>
References: <20230726080916.17454-1-zhangpeng.00@bytedance.com>
 <20230726080916.17454-5-zhangpeng.00@bytedance.com>
 <20230726160354.konsgq6hidj7gr5u@revolver>
 <beaab8b4-180c-017d-bd8d-8766196f302a@bytedance.com>
 <20230731162714.4x3lzymuyvu2mter@revolver>
 <3f4e73cc-1a98-95a8-9ab2-47797d236585@bytedance.com>
 <20230816183029.5rpkbgp2umebrjh5@revolver>
 <6b6d7ef1-75e4-68a3-1662-82ee19334567@bytedance.com>
Message-ID: <20230818161339.tpjcxs35xxug6f2d@revolver>

* Peng Zhang <zhangpeng.00 at bytedance.com> [230818 07:54]:
> 
> 
> ? 2023/8/17 02:30, Liam R. Howlett ??:
> > * Peng Zhang <zhangpeng.00 at bytedance.com> [230816 09:42]:
> > > 
> > > 
> > ...
> > 
> > > > > > > +/**
> > > > > > > + * __mt_dup(): Duplicate a maple tree
> > > > > > > + * @mt: The source maple tree
> > > > > > > + * @new: The new maple tree
> > > > > > > + * @gfp: The GFP_FLAGS to use for allocations
> > > > > > > + *
> > > > > > > + * This function duplicates a maple tree using a faster method than traversing
> > > > > > > + * the source tree and inserting entries into the new tree one by one. The user
> > > > > > > + * needs to lock the source tree manually. Before calling this function, @new
> > > > > > > + * must be an empty tree or an uninitialized tree. If @mt uses an external lock,
> > > > > > > + * we may also need to manually set @new's external lock using
> > > > > > > + * mt_set_external_lock().
> > > > > > > + *
> > > > > > > + * Return: 0 on success, -ENOMEM if memory could not be allocated.
> > > > > > > + */
> > > > > > > +int __mt_dup(struct maple_tree *mt, struct maple_tree *new, gfp_t gfp)
> > > > > > 
> > > > > > We use mas_ for things that won't handle the locking and pass in a maple
> > > > > > state.  Considering the leaves need to be altered once this is returned,
> > > > > > I would expect passing in a maple state should be feasible?
> > > > > But we don't really need mas here. What do you think the state of mas
> > > > > should be when this function returns? Make it point to the first entry,
> > > > > or the last entry?
> > > > 
> > > > I would write it to point to the first element so that the call to
> > > > replace the first element can just do that without an extra walk and
> > > > document the maple state end point.
> > > Unfortunately, this does not seem to be convenient. Users usually use
> > > mas_for_each() to replace elements. If we set mas to the first element,
> > > the first call to mas_find() in mas_for_each() will get the next
> > > element.
> > 
> > This sounds like the need for another iterator specifically for
> > duplicating.
> > 
> > > 
> > > There may also be other scenarios where the user does not necessarily
> > > have to replace every element.
> > 
> > Do you mean a limit or elements that need to be skipped?  We could have
> > a limit on the iteration.
> > 
> > > 
> > > Finally, getting the first element in __mt_dup() requires an additional
> > > check to check whether the first element has already been recorded. Such
> > > a check will be performed at each leaf node, which is unnecessary
> > > overhead.
> > > 
> > > Of course, the first reason is the main reason, which prevents us from
> > > using mas_for_each(). So I don't want to record the first element.
> > 
> > 
> > I don't like the interface because it can easily be misunderstood and
> > used incorrectly.  I don't know how to make a cleaner interface, but
> > I've gone through a few thoughts:
> > 
> > The first was hide _all of it_ in a new iterator:
> > mas_dup_each(old, new, old_entry) {
> > 	if (don't_dup(old_entry)) {
> > 		mas_erase(new);
> > 		continue;
> > 	}
> > 
> > 	mas_dup_insert(new, new_entry);
> > }
> > 
> > This iterator would check if mas_is_start(old) and dup the tree in that
> > event.  Leave the both new trees pointing to the first element and set
> > old_entry.  I don't know how to handle the failure in duplicating the
> > tree in this case - I guess we could return old_entry = NULL and check
> > if mas_is_err(old) after the loop.  Do you see a problem with this?
> This interface looks OK. But handling the failure case is tricky.

I think it's awkward, but not tricky; possibly error prone to users.  I
don't like this solution because of the error handling.  I'm just
stating some options I ruled out in hopes you see some way of improving
them.

Maybe we name the check something like mas_dup_complete(old, new) and
all it does is checks for an error?  It makes it obvious that it's
necessary but doesn't avoid people leaving it out.

> > 
> > 
> > The second idea was an init of the old tree.  This is closest to what you
> > have:
> > 
> > if (mas_dup_init(old, new))
> > 	goto -ENOMEM;
> > 
> > mas_dup_each(old, new) {
> > 	if (don't_dup(old_entry)) {
> > 		mas_erase(new);
> > 		continue;
> > 	}
> > 
> > 	mas_dup_insert(new, new_entry);
> > }
> I think this interface could be better.
> > 
> > This would duplicate the tree at the start and leave both pointing at
> > the first element so that mas_dup_each() could start on that element.
> > Each subsequent call would go to the next element in both maple states.
> Every element of the new tree is the same as the old tree, and we don't
> need to maintain the mas of the old tree. It is enough to maintain the
> mas of the new tree when traversing.

Okay, and using DFS means we can't stop one level before the leaves
during duplication since deletes may not function.

> 
> > It sounds like you don't want this for performance reasons?  Although
> I mean I don't want to record the first element during duplicating. But
> we can get the first element after the duplicate completes. This can
> also still be within the implementation of the interface.
> 
> > looking at mas_find() today, I think this could still work since we are
> > checking the maple state for a lot.
> Yes, mas_find() does a whole bunch of checks.
> > 
> > Both ideas could be even faster than what you have if we handle the
> > special cases of mas_is_none()/mas_is_ptr() in a smarter way because we
> > don't need to be as worried about the entry point of the maple state as
> > much as we do with mas_find()/mas_for_each().  I mean, is it possible to
> > get to a mas_is_none() or mas_is_ptr() on duplicating a tree?  How do we
> > handle these users?
> The check for mas_is_none() or mas_is_ptr() in mas_find() is really not
> worth it if we hold the lock. There doesn't seem to be a good way around
> mas_is_ptr() since it needs to enter the loop once. mas_is_none() can be
> solved because it does not enter the loop, we can use it as a condition
> to enter the loop.

So do you think it is worth making a new iterator then?  One that
requires the write lock so some work can be avoided?

> 
> Without using mas_find() to avoid the check inside, I have to figure out
> how I can handle mas_is_ptr() properly.
> > 
> > Both ideas still suffer from someone saying "Gee, that {insert function
> > name here} is used in the forking code, so I can totally use that in my
> > code because that's how it work!"  and find out it works for the limited
> > testing they do.  Then it fails later and the emails start flying.
> > 
> > 
> > I almost think we should do something like this on insert:
> > 
> > void mas_dup_insert(old, new, new_entry) {
> > 	WARN_ON_ONCE(old == new);
> > 	WARN_ON_ONCE(old->index != new->index);
> > 	WARN_ON_ONCE(old->last != new->last);
> > 	...
> > }
> Maintaining old mas doesn't feel worth it. If this we have to traverse
> the old tree one more time.

Maybe only when debug is enabled?  We should at least keep the first
check.

I'm not committed to this interface either.  Do you have an idea that
works better?

> > 
> > This would at least _require_ someone to have two maple states and
> > hopefully think twice on using it where it should not be used.
> > 
> > The bottom line is that this code is close to what we need to make
> > forking better, but I fear the misuse of the interface.
> > 
> > Something else to think about:
> > In the work items for the Maple Tree, there is a plan to have an enum to
> > specify the type of write that is going to happen.  The idea was for
> > mas_preallocate() to set this type of write so we can just go right to
> > the correct function.  We could use that here and set the maple state
> > write type to a direct replacement.
> This can be the next step. We can do without it for now.

Agreed, but your current replacement is very close to what we have for
direct replacement after all the checks in the current write path.

I'm wondering if we have an enum in the maple state to just go to that
store operation, then these two interfaces could share a lot of the
code.

But we should leave it for later, I was just pointing it out.

Thanks,
Liam


From Liam.Howlett at oracle.com  Fri Aug 18 17:43:54 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri, 18 Aug 2023 20:43:54 -0400
Subject: [PATCH v2 0/2] maple_tree: mas_wr_append() fix ups
Message-ID: <20230819004356.1454718-1-Liam.Howlett@oracle.com>

There is a potential of an update causing issue with a reader when
appending, so disable the append operation when readers a present.

The code was also missing some documentation, tracing, and some cleanup
was possible.  These changes are in the second patch as there's no
compelling reason to require a difficult backport.

Changes since v1:
 - Added fixes tag to the first patch

v1: https://lore.kernel.org/linux-mm/20230817191505.1170876-1-Liam.Howlett at oracle.com/

Liam R. Howlett (2):
  maple_tree: Disable mas_wr_append() when other readers are possible
  maple_tree: Clean up mas_wr_append()

 lib/maple_tree.c | 41 +++++++++++++++++++++++++++--------------
 1 file changed, 27 insertions(+), 14 deletions(-)

-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug 18 17:43:55 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri, 18 Aug 2023 20:43:55 -0400
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other readers are possible
In-Reply-To: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
Message-ID: <20230819004356.1454718-2-Liam.Howlett@oracle.com>

The current implementation of append may cause duplicate data and/or
incorrect ranges to be returned to a reader during an update.  Although
this has not been reported or seen, disable the append write operation
while the tree is in rcu mode out of an abundance of caution.

During the analysis of the mas_next_slot() the following was
artificially created by separating the writer and reader code:

Writer:                                 reader:
mas_wr_append
    set end pivot
    updates end metata
    Detects write to last slot
    last slot write is to start of slot
    store current contents in slot
    overwrite old end pivot
                                        mas_next_slot():
                                                read end metadata
                                                read old end pivot
                                                return with incorrect range
    store new value

Alternatively:

Writer:                                 reader:
mas_wr_append
    set end pivot
    updates end metata
    Detects write to last slot
    last lost write to end of slot
    store value
                                        mas_next_slot():
                                                read end metadata
                                                read old end pivot
                                                read new end pivot
                                                return with incorrect range
    set old end pivot

There may be other accesses that are not safe since we are now updating
both metadata and pointers, so disabling append if there could be rcu
readers is the safest action.

Fixes: 54a611b60590 ("Maple Tree: add new data structure")
Cc: stable at vger.kernel.org
Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index ffb9d15bd815..05d5db255c39 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -4107,6 +4107,10 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
  * mas_wr_append: Attempt to append
  * @wr_mas: the maple write state
  *
+ * This is currently unsafe in rcu mode since the end of the node may be cached
+ * by readers while the node contents may be updated which could result in
+ * inaccurate information.
+ *
  * Return: True if appended, false otherwise
  */
 static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
@@ -4116,6 +4120,9 @@ static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
 	struct ma_state *mas = wr_mas->mas;
 	unsigned char node_pivots = mt_pivots[wr_mas->type];
 
+	if (mt_in_rcu(mas->tree))
+		return false;
+
 	if (mas->offset != wr_mas->node_end)
 		return false;
 
-- 
2.39.2



From Liam.Howlett at oracle.com  Fri Aug 18 17:43:56 2023
From: Liam.Howlett at oracle.com (Liam R. Howlett)
Date: Fri, 18 Aug 2023 20:43:56 -0400
Subject: [PATCH v2 2/2] maple_tree: Clean up mas_wr_append()
In-Reply-To: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
Message-ID: <20230819004356.1454718-3-Liam.Howlett@oracle.com>

Avoid setting the variables until necessary, and actually use the
variables where applicable.  Introducing a variable for the slots array
avoids spanning multiple lines.

Add the missing argument to the documentation.

Use the node type when setting the metadata instead of blindly assuming
the type.

Finally, add a trace point to the function for successful store.

Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
---
 lib/maple_tree.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index 05d5db255c39..ee1ff0c59fd7 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -4106,6 +4106,7 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
 /*
  * mas_wr_append: Attempt to append
  * @wr_mas: the maple write state
+ * @new_end: The end of the node after the modification
  *
  * This is currently unsafe in rcu mode since the end of the node may be cached
  * by readers while the node contents may be updated which could result in
@@ -4114,42 +4115,46 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
  * Return: True if appended, false otherwise
  */
 static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
-				 unsigned char new_end)
+		unsigned char new_end)
 {
-	unsigned char end = wr_mas->node_end;
-	struct ma_state *mas = wr_mas->mas;
-	unsigned char node_pivots = mt_pivots[wr_mas->type];
+	struct ma_state *mas;
+	void __rcu **slots;
+	unsigned char end;
 
+	mas = wr_mas->mas;
 	if (mt_in_rcu(mas->tree))
 		return false;
 
 	if (mas->offset != wr_mas->node_end)
 		return false;
 
-	if (new_end < node_pivots) {
+	end = wr_mas->node_end;
+	if (mas->offset != end)
+		return false;
+
+	if (new_end < mt_pivots[wr_mas->type]) {
 		wr_mas->pivots[new_end] = wr_mas->pivots[end];
-		ma_set_meta(wr_mas->node, maple_leaf_64, 0, new_end);
+		ma_set_meta(wr_mas->node, wr_mas->type, 0, new_end);
 	}
 
-	if (new_end == wr_mas->node_end + 1) {
+	slots = wr_mas->slots;
+	if (new_end == end + 1) {
 		if (mas->last == wr_mas->r_max) {
 			/* Append to end of range */
-			rcu_assign_pointer(wr_mas->slots[new_end],
-					   wr_mas->entry);
+			rcu_assign_pointer(slots[new_end], wr_mas->entry);
 			wr_mas->pivots[end] = mas->index - 1;
 			mas->offset = new_end;
 		} else {
 			/* Append to start of range */
-			rcu_assign_pointer(wr_mas->slots[new_end],
-					   wr_mas->content);
+			rcu_assign_pointer(slots[new_end], wr_mas->content);
 			wr_mas->pivots[end] = mas->last;
-			rcu_assign_pointer(wr_mas->slots[end], wr_mas->entry);
+			rcu_assign_pointer(slots[end], wr_mas->entry);
 		}
 	} else {
 		/* Append to the range without touching any boundaries. */
-		rcu_assign_pointer(wr_mas->slots[new_end], wr_mas->content);
+		rcu_assign_pointer(slots[new_end], wr_mas->content);
 		wr_mas->pivots[end + 1] = mas->last;
-		rcu_assign_pointer(wr_mas->slots[end + 1], wr_mas->entry);
+		rcu_assign_pointer(slots[end + 1], wr_mas->entry);
 		wr_mas->pivots[end] = mas->index - 1;
 		mas->offset = end + 1;
 	}
@@ -4157,6 +4162,7 @@ static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
 	if (!wr_mas->content || !wr_mas->entry)
 		mas_update_gap(mas);
 
+	trace_ma_write(__func__, mas, new_end, wr_mas->entry);
 	return  true;
 }
 
-- 
2.39.2



From mjguzik at gmail.com  Mon Aug 21 15:51:45 2023
From: mjguzik at gmail.com (Mateusz Guzik)
Date: Tue, 22 Aug 2023 00:51:45 +0200
Subject: [PATCH] maple_tree: shrink struct maple_tree from 24 to 16 bytes on LP64
Message-ID: <20230821225145.2169848-1-mjguzik@gmail.com>

by plugging a padding hole.

[sounds like an obvious thing to do, am I missing something?]

Signed-off-by: Mateusz Guzik <mjguzik at gmail.com>
---
 include/linux/maple_tree.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h
index 295548cca8b3..5bdd3ce34cb9 100644
--- a/include/linux/maple_tree.h
+++ b/include/linux/maple_tree.h
@@ -212,8 +212,8 @@ struct maple_tree {
 		spinlock_t	ma_lock;
 		lockdep_map_p	ma_external_lock;
 	};
-	void __rcu      *ma_root;
 	unsigned int	ma_flags;
+	void __rcu      *ma_root;
 };
 
 /**
-- 
2.39.2



From willy at infradead.org  Mon Aug 21 18:15:43 2023
From: willy at infradead.org (Matthew Wilcox)
Date: Tue, 22 Aug 2023 02:15:43 +0100
Subject: [PATCH] maple_tree: shrink struct maple_tree from 24 to 16 bytes
 on LP64
In-Reply-To: <20230821225145.2169848-1-mjguzik@gmail.com>
References: <20230821225145.2169848-1-mjguzik@gmail.com>
Message-ID: <ZOQMP6ksHU3Nf7sM@casper.infradead.org>

On Tue, Aug 22, 2023 at 12:51:45AM +0200, Mateusz Guzik wrote:
> by plugging a padding hole.
> 
> [sounds like an obvious thing to do, am I missing something?]

That was how I originally had it.  Not sure when/why it got messed up.


From Liam.Howlett at Oracle.com  Tue Aug 22 07:09:46 2023
From: Liam.Howlett at Oracle.com (Liam R. Howlett)
Date: Tue, 22 Aug 2023 10:09:46 -0400
Subject: [PATCH] maple_tree: shrink struct maple_tree from 24 to 16 bytes
 on LP64
In-Reply-To: <20230821225145.2169848-1-mjguzik@gmail.com>
References: <20230821225145.2169848-1-mjguzik@gmail.com>
Message-ID: <20230822140946.kbmv43l4uhwwqwln@revolver>

* Mateusz Guzik <mjguzik at gmail.com> [230821 18:52]:
> by plugging a padding hole.
> 
> [sounds like an obvious thing to do, am I missing something?]

Thanks for the patch.  There is no reason for the current ordering.

I've tested both 32 and 64 bit userspace testing to double check
everything still works.

> 
> Signed-off-by: Mateusz Guzik <mjguzik at gmail.com>

Reviewed-by: Liam R. Howlett <Liam.Howlett at oracle.com>

> ---
>  include/linux/maple_tree.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h
> index 295548cca8b3..5bdd3ce34cb9 100644
> --- a/include/linux/maple_tree.h
> +++ b/include/linux/maple_tree.h
> @@ -212,8 +212,8 @@ struct maple_tree {
>  		spinlock_t	ma_lock;
>  		lockdep_map_p	ma_external_lock;
>  	};
> -	void __rcu      *ma_root;
>  	unsigned int	ma_flags;
> +	void __rcu      *ma_root;
>  };
>  
>  /**
> -- 
> 2.39.2
> 
> 


From geert at linux-m68k.org  Tue Aug 29 09:42:48 2023
From: geert at linux-m68k.org (Geert Uytterhoeven)
Date: Tue, 29 Aug 2023 18:42:48 +0200 (CEST)
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <20230819004356.1454718-2-Liam.Howlett@oracle.com>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com> <20230819004356.1454718-2-Liam.Howlett@oracle.com>
Message-ID: <3f86d58e-7f36-c6b4-c43a-2a7bcffd3bd@linux-m68k.org>

 	Hi Liam,

On Fri, 18 Aug 2023, Liam R. Howlett wrote:
> The current implementation of append may cause duplicate data and/or
> incorrect ranges to be returned to a reader during an update.  Although
> this has not been reported or seen, disable the append write operation
> while the tree is in rcu mode out of an abundance of caution.
>
> During the analysis of the mas_next_slot() the following was
> artificially created by separating the writer and reader code:
>
> Writer:                                 reader:
> mas_wr_append
>    set end pivot
>    updates end metata
>    Detects write to last slot
>    last slot write is to start of slot
>    store current contents in slot
>    overwrite old end pivot
>                                        mas_next_slot():
>                                                read end metadata
>                                                read old end pivot
>                                                return with incorrect range
>    store new value
>
> Alternatively:
>
> Writer:                                 reader:
> mas_wr_append
>    set end pivot
>    updates end metata
>    Detects write to last slot
>    last lost write to end of slot
>    store value
>                                        mas_next_slot():
>                                                read end metadata
>                                                read old end pivot
>                                                read new end pivot
>                                                return with incorrect range
>    set old end pivot
>
> There may be other accesses that are not safe since we are now updating
> both metadata and pointers, so disabling append if there could be rcu
> readers is the safest action.
>
> Fixes: 54a611b60590 ("Maple Tree: add new data structure")
> Cc: stable at vger.kernel.org
> Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>

Thanks for your patch, which is now commit cfeb6ae8bcb96ccf
("maple_tree: disable mas_wr_append() when other readers are
possible") in v6.5, and is being backported to stable.

On Renesas RZ/A1 and RZ/A2 (single-core Cortex-A9), this causes the
following warning:

      clocksource: timer at e803b000: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 28958491609 ns
      sched_clock: 32 bits at 66MHz, resolution 15ns, wraps every 32537631224ns
      /soc/timer at e803b000: used for clocksource
      /soc/timer at e803c000: used for clock events
     +------------[ cut here ]------------
     +WARNING: CPU: 0 PID: 0 at init/main.c:992 start_kernel+0x2f0/0x480
     +Interrupts were enabled early
     +CPU: 0 PID: 0 Comm: swapper Not tainted 6.5.0-rza2mevb-10197-g99b80d6b92b5 #237
     +Hardware name: Generic R7S9210 (Flattened Device Tree)
     + unwind_backtrace from show_stack+0x10/0x14
     + show_stack from dump_stack_lvl+0x24/0x3c
     + dump_stack_lvl from __warn+0x74/0xb8
     + __warn from warn_slowpath_fmt+0x78/0xb0
     + warn_slowpath_fmt from start_kernel+0x2f0/0x480
     + start_kernel from 0x0
     +---[ end trace 0000000000000000 ]---
      Console: colour dummy device 80x30
      printk: console [tty0] enabled
      Calibrating delay loop (skipped) preset value.. 1056.00 BogoMIPS (lpj=5280000)

Reverting this commit fixes the issue.

RCU-related configs:

     $ grep RCU .config
     # RCU Subsystem
     CONFIG_TINY_RCU=y
     # CONFIG_RCU_EXPERT is not set
     CONFIG_TINY_SRCU=y
     # end of RCU Subsystem
     # RCU Debugging
     # CONFIG_RCU_SCALE_TEST is not set
     # CONFIG_RCU_TORTURE_TEST is not set
     # CONFIG_RCU_REF_SCALE_TEST is not set
     # CONFIG_RCU_TRACE is not set
     # CONFIG_RCU_EQS_DEBUG is not set
     # end of RCU Debugging

CONFIG_MAPLE_RCU_DISABLED is not defined (and should BTW be renamed,
as CONFIG_* is reserved for kernel configuration options).

I do not see this issue on any other platform
(arm/arm64/risc-v/mips/sh/m68k), several of them use the same
RCU configuration.

Do you have a clue?
Thanks!

> --- a/lib/maple_tree.c
> +++ b/lib/maple_tree.c
> @@ -4107,6 +4107,10 @@ static inline unsigned char mas_wr_new_end(struct ma_wr_state *wr_mas)
>  * mas_wr_append: Attempt to append
>  * @wr_mas: the maple write state
>  *
> + * This is currently unsafe in rcu mode since the end of the node may be cached
> + * by readers while the node contents may be updated which could result in
> + * inaccurate information.
> + *
>  * Return: True if appended, false otherwise
>  */
> static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
> @@ -4116,6 +4120,9 @@ static inline bool mas_wr_append(struct ma_wr_state *wr_mas,
> 	struct ma_state *mas = wr_mas->mas;
> 	unsigned char node_pivots = mt_pivots[wr_mas->type];
>
> +	if (mt_in_rcu(mas->tree))
> +		return false;
> +
> 	if (mas->offset != wr_mas->node_end)
> 		return false;
>
> -- 
> 2.39.2

Gr{oetje,eeting}s,

 						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
 							    -- Linus Torvalds


From schwab at linux-m68k.org  Wed Aug 30 12:49:47 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Wed, 30 Aug 2023 21:49:47 +0200
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <20230819004356.1454718-2-Liam.Howlett@oracle.com> (Liam
	R. Howlett's message of "Fri, 18 Aug 2023 20:43:55 -0400")
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
	<20230819004356.1454718-2-Liam.Howlett@oracle.com>
Message-ID: <87bkeotin8.fsf@igel.home>

This breaks booting on ppc32:

Kernel attemptd to writ user page (1ff0) - exploit attempt? (uid: 0)
BUG: Unable to handle kernel data access on write at 0x00001ff0
Faulting instruction address: 0xc0009554
Vector: 300 (Data Access) at [c0b09d10]
    pc: c0009554: do_softirq_own_stack+0x18/0x30
    lr: c004f480: __irq_exit_rcu+0x70/0xc0
    sp: c0b09dd0
   msr: 1032
   dar: 1ff0
 dsisr: 42000000
  current = 0xc0a08360
    pid   = 0, comm = swapper
Linux version 6.5.0 ...
enter ? for help
[c0b09de0] c00ff480 __irq_exit_rcu+0x70/0xc0
[c0b09df0] c0005a98 Decrementer_virt+0x108/0x10c
--- Exception: 900 (Decrementer) at c06cfa0c __schedule+0x4fc/0x510
[c0b09ec0] c06cf75c __schedule+0x1cc/0x510 (unreliable)
[c0b09ef0] c06cfc90 __cond_resched+0x2c/0x54
[c0b09f00] c06d07f8 mutex_lock_killable+0x18/0x5c
[c0b09f10] c013c404 pcpu_alloc+0x110/0x4dc
[c0b09f70] c000cc34 alloc_descr.isra.18+0x48/0x144
[c0b09f90] c0988aa0 early_irq_init+0x64/0x8c
[c0b09fa0] c097a5a4 start_kernel+0x5b4/0x7b0
[c0b09ff0] 00003dc0
mon>

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


From geert at linux-m68k.org  Thu Aug 31 01:25:48 2023
From: geert at linux-m68k.org (Geert Uytterhoeven)
Date: Thu, 31 Aug 2023 10:25:48 +0200
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <87v8cv22jh.fsf@mail.lhotse>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
 <20230819004356.1454718-2-Liam.Howlett@oracle.com> <3f86d58e-7f36-c6b4-c43a-2a7bcffd3bd@linux-m68k.org>
 <87v8cv22jh.fsf@mail.lhotse>
Message-ID: <CAMuHMdXMpr0spdprjwsV56nJE3vHGTFaodcnVXUa=GMYaB5yKw@mail.gmail.com>

Hi Michael,

On Thu, Aug 31, 2023 at 7:39?AM Michael Ellerman <mpe at ellerman.id.au> wrote:
> Geert Uytterhoeven <geert at linux-m68k.org> writes:
> > On Fri, 18 Aug 2023, Liam R. Howlett wrote:
> >> The current implementation of append may cause duplicate data and/or
> >> incorrect ranges to be returned to a reader during an update.  Although
> >> this has not been reported or seen, disable the append write operation
> >> while the tree is in rcu mode out of an abundance of caution.
> >>
> >> During the analysis of the mas_next_slot() the following was
> >> artificially created by separating the writer and reader code:
> >>
> >> Writer:                                 reader:
> >> mas_wr_append
> >>    set end pivot
> >>    updates end metata
> >>    Detects write to last slot
> >>    last slot write is to start of slot
> >>    store current contents in slot
> >>    overwrite old end pivot
> >>                                        mas_next_slot():
> >>                                                read end metadata
> >>                                                read old end pivot
> >>                                                return with incorrect range
> >>    store new value
> >>
> >> Alternatively:
> >>
> >> Writer:                                 reader:
> >> mas_wr_append
> >>    set end pivot
> >>    updates end metata
> >>    Detects write to last slot
> >>    last lost write to end of slot
> >>    store value
> >>                                        mas_next_slot():
> >>                                                read end metadata
> >>                                                read old end pivot
> >>                                                read new end pivot
> >>                                                return with incorrect range
> >>    set old end pivot
> >>
> >> There may be other accesses that are not safe since we are now updating
> >> both metadata and pointers, so disabling append if there could be rcu
> >> readers is the safest action.
> >>
> >> Fixes: 54a611b60590 ("Maple Tree: add new data structure")
> >> Cc: stable at vger.kernel.org
> >> Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
> >
> > Thanks for your patch, which is now commit cfeb6ae8bcb96ccf
> > ("maple_tree: disable mas_wr_append() when other readers are
> > possible") in v6.5, and is being backported to stable.
> >
> > On Renesas RZ/A1 and RZ/A2 (single-core Cortex-A9), this causes the
> > following warning:
> >
> >       clocksource: timer at e803b000: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 28958491609 ns
> >       sched_clock: 32 bits at 66MHz, resolution 15ns, wraps every 32537631224ns
> >       /soc/timer at e803b000: used for clocksource
> >       /soc/timer at e803c000: used for clock events
> >      +------------[ cut here ]------------
> >      +WARNING: CPU: 0 PID: 0 at init/main.c:992 start_kernel+0x2f0/0x480
> >      +Interrupts were enabled early
> ...
> >
> > I do not see this issue on any other platform
> > (arm/arm64/risc-v/mips/sh/m68k), several of them use the same
> > RCU configuration.
>
> There's something similar on pmac32 / mac99.
>
> > Do you have a clue?
>
> It seems something in the maple tree code is setting TIF_NEED_RESCHED,
> and that causes a subsequent call to cond_resched() to call schedule()
> and enable interrupts.
>
> On pmac32 enabling CONFIG_DEBUG_ATOMIC_SLEEP fixes/hides the problem.
> But I don't see why.

Enabling CONFIG_DEBUG_ATOMIC_SLEEP on RZ/A1 and RZ/A2 does
fix the problem.
But there must be more to it, as some of my test configs had it enabled,
and others hadn't, while only RZ/A showed the issue.
I tried disabling it on R-Car M2-W (arm32) and R-Car H3 (arm64), and
that did not cause the problem to happen...

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds


From zhangpeng.00 at bytedance.com  Thu Aug 31 01:45:25 2023
From: zhangpeng.00 at bytedance.com (Peng Zhang)
Date: Thu, 31 Aug 2023 16:45:25 +0800
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <CAMuHMdXMpr0spdprjwsV56nJE3vHGTFaodcnVXUa=GMYaB5yKw@mail.gmail.com>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
 <20230819004356.1454718-2-Liam.Howlett@oracle.com>
 <3f86d58e-7f36-c6b4-c43a-2a7bcffd3bd@linux-m68k.org>
 <87v8cv22jh.fsf@mail.lhotse>
 <CAMuHMdXMpr0spdprjwsV56nJE3vHGTFaodcnVXUa=GMYaB5yKw@mail.gmail.com>
Message-ID: <6f663185-2ef1-5075-99c9-e16050329d74@bytedance.com>



? 2023/8/31 16:25, Geert Uytterhoeven ??:
> Hi Michael,
> 
> On Thu, Aug 31, 2023 at 7:39?AM Michael Ellerman <mpe at ellerman.id.au> wrote:
>> Geert Uytterhoeven <geert at linux-m68k.org> writes:
>>> On Fri, 18 Aug 2023, Liam R. Howlett wrote:
>>>> The current implementation of append may cause duplicate data and/or
>>>> incorrect ranges to be returned to a reader during an update.  Although
>>>> this has not been reported or seen, disable the append write operation
>>>> while the tree is in rcu mode out of an abundance of caution.
>>>>
>>>> During the analysis of the mas_next_slot() the following was
>>>> artificially created by separating the writer and reader code:
>>>>
>>>> Writer:                                 reader:
>>>> mas_wr_append
>>>>     set end pivot
>>>>     updates end metata
>>>>     Detects write to last slot
>>>>     last slot write is to start of slot
>>>>     store current contents in slot
>>>>     overwrite old end pivot
>>>>                                         mas_next_slot():
>>>>                                                 read end metadata
>>>>                                                 read old end pivot
>>>>                                                 return with incorrect range
>>>>     store new value
>>>>
>>>> Alternatively:
>>>>
>>>> Writer:                                 reader:
>>>> mas_wr_append
>>>>     set end pivot
>>>>     updates end metata
>>>>     Detects write to last slot
>>>>     last lost write to end of slot
>>>>     store value
>>>>                                         mas_next_slot():
>>>>                                                 read end metadata
>>>>                                                 read old end pivot
>>>>                                                 read new end pivot
>>>>                                                 return with incorrect range
>>>>     set old end pivot
>>>>
>>>> There may be other accesses that are not safe since we are now updating
>>>> both metadata and pointers, so disabling append if there could be rcu
>>>> readers is the safest action.
>>>>
>>>> Fixes: 54a611b60590 ("Maple Tree: add new data structure")
>>>> Cc: stable at vger.kernel.org
>>>> Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
>>>
>>> Thanks for your patch, which is now commit cfeb6ae8bcb96ccf
>>> ("maple_tree: disable mas_wr_append() when other readers are
>>> possible") in v6.5, and is being backported to stable.
>>>
>>> On Renesas RZ/A1 and RZ/A2 (single-core Cortex-A9), this causes the
>>> following warning:
>>>
>>>        clocksource: timer at e803b000: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 28958491609 ns
>>>        sched_clock: 32 bits at 66MHz, resolution 15ns, wraps every 32537631224ns
>>>        /soc/timer at e803b000: used for clocksource
>>>        /soc/timer at e803c000: used for clock events
>>>       +------------[ cut here ]------------
>>>       +WARNING: CPU: 0 PID: 0 at init/main.c:992 start_kernel+0x2f0/0x480
>>>       +Interrupts were enabled early
>> ...
>>>
>>> I do not see this issue on any other platform
>>> (arm/arm64/risc-v/mips/sh/m68k), several of them use the same
>>> RCU configuration.
>>
>> There's something similar on pmac32 / mac99.
>>
>>> Do you have a clue?
>>
>> It seems something in the maple tree code is setting TIF_NEED_RESCHED,
>> and that causes a subsequent call to cond_resched() to call schedule()
>> and enable interrupts.
>>
>> On pmac32 enabling CONFIG_DEBUG_ATOMIC_SLEEP fixes/hides the problem.
>> But I don't see why.
> 
> Enabling CONFIG_DEBUG_ATOMIC_SLEEP on RZ/A1 and RZ/A2 does
> fix the problem.
> But there must be more to it, as some of my test configs had it enabled,
> and others hadn't, while only RZ/A showed the issue.
> I tried disabling it on R-Car M2-W (arm32) and R-Car H3 (arm64), and
> that did not cause the problem to happen...

I guess this patch triggers a potential problem with the maple tree.
I don't have the environment to trigger the problem. Can you apply the
following patch to see if the problem still exists? This can help locate
the root cause. At least narrow down the scope of the code that has bug.

Thanks.

diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index f723024e1426..1b4b6f6e3095 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -4351,9 +4351,6 @@ static inline void mas_wr_modify(struct 
ma_wr_state *wr_mas)
  	if (new_end == wr_mas->node_end && mas_wr_slot_store(wr_mas))
  		return;

-	if (mas_wr_node_store(wr_mas, new_end))
-		return;
-
  	if (mas_is_err(mas))
  		return;


> 
> Gr{oetje,eeting}s,
> 
>                          Geert
> 


From geert at linux-m68k.org  Thu Aug 31 02:43:25 2023
From: geert at linux-m68k.org (Geert Uytterhoeven)
Date: Thu, 31 Aug 2023 11:43:25 +0200
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <6f663185-2ef1-5075-99c9-e16050329d74@bytedance.com>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
 <20230819004356.1454718-2-Liam.Howlett@oracle.com> <3f86d58e-7f36-c6b4-c43a-2a7bcffd3bd@linux-m68k.org>
 <87v8cv22jh.fsf@mail.lhotse> <CAMuHMdXMpr0spdprjwsV56nJE3vHGTFaodcnVXUa=GMYaB5yKw@mail.gmail.com>
 <6f663185-2ef1-5075-99c9-e16050329d74@bytedance.com>
Message-ID: <CAMuHMdW+wNma-Nr+595kRgZfG_fS2zgbZxJNbtcNRxnmpCJ=5w@mail.gmail.com>

Hi Peng,

On Thu, Aug 31, 2023 at 10:45?AM Peng Zhang <zhangpeng.00 at bytedance.com> wrote:
> ? 2023/8/31 16:25, Geert Uytterhoeven ??:
> > On Thu, Aug 31, 2023 at 7:39?AM Michael Ellerman <mpe at ellerman.id.au> wrote:
> >> Geert Uytterhoeven <geert at linux-m68k.org> writes:
> >>> On Fri, 18 Aug 2023, Liam R. Howlett wrote:
> >>>> The current implementation of append may cause duplicate data and/or
> >>>> incorrect ranges to be returned to a reader during an update.  Although
> >>>> this has not been reported or seen, disable the append write operation
> >>>> while the tree is in rcu mode out of an abundance of caution.
> >>>>
> >>>> During the analysis of the mas_next_slot() the following was
> >>>> artificially created by separating the writer and reader code:
> >>>>
> >>>> Writer:                                 reader:
> >>>> mas_wr_append
> >>>>     set end pivot
> >>>>     updates end metata
> >>>>     Detects write to last slot
> >>>>     last slot write is to start of slot
> >>>>     store current contents in slot
> >>>>     overwrite old end pivot
> >>>>                                         mas_next_slot():
> >>>>                                                 read end metadata
> >>>>                                                 read old end pivot
> >>>>                                                 return with incorrect range
> >>>>     store new value
> >>>>
> >>>> Alternatively:
> >>>>
> >>>> Writer:                                 reader:
> >>>> mas_wr_append
> >>>>     set end pivot
> >>>>     updates end metata
> >>>>     Detects write to last slot
> >>>>     last lost write to end of slot
> >>>>     store value
> >>>>                                         mas_next_slot():
> >>>>                                                 read end metadata
> >>>>                                                 read old end pivot
> >>>>                                                 read new end pivot
> >>>>                                                 return with incorrect range
> >>>>     set old end pivot
> >>>>
> >>>> There may be other accesses that are not safe since we are now updating
> >>>> both metadata and pointers, so disabling append if there could be rcu
> >>>> readers is the safest action.
> >>>>
> >>>> Fixes: 54a611b60590 ("Maple Tree: add new data structure")
> >>>> Cc: stable at vger.kernel.org
> >>>> Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
> >>>
> >>> Thanks for your patch, which is now commit cfeb6ae8bcb96ccf
> >>> ("maple_tree: disable mas_wr_append() when other readers are
> >>> possible") in v6.5, and is being backported to stable.
> >>>
> >>> On Renesas RZ/A1 and RZ/A2 (single-core Cortex-A9), this causes the
> >>> following warning:
> >>>
> >>>        clocksource: timer at e803b000: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 28958491609 ns
> >>>        sched_clock: 32 bits at 66MHz, resolution 15ns, wraps every 32537631224ns
> >>>        /soc/timer at e803b000: used for clocksource
> >>>        /soc/timer at e803c000: used for clock events
> >>>       +------------[ cut here ]------------
> >>>       +WARNING: CPU: 0 PID: 0 at init/main.c:992 start_kernel+0x2f0/0x480
> >>>       +Interrupts were enabled early
> >> ...
> >>>
> >>> I do not see this issue on any other platform
> >>> (arm/arm64/risc-v/mips/sh/m68k), several of them use the same
> >>> RCU configuration.
> >>
> >> There's something similar on pmac32 / mac99.
> >>
> >>> Do you have a clue?
> >>
> >> It seems something in the maple tree code is setting TIF_NEED_RESCHED,
> >> and that causes a subsequent call to cond_resched() to call schedule()
> >> and enable interrupts.
> >>
> >> On pmac32 enabling CONFIG_DEBUG_ATOMIC_SLEEP fixes/hides the problem.
> >> But I don't see why.
> >
> > Enabling CONFIG_DEBUG_ATOMIC_SLEEP on RZ/A1 and RZ/A2 does
> > fix the problem.
> > But there must be more to it, as some of my test configs had it enabled,
> > and others hadn't, while only RZ/A showed the issue.
> > I tried disabling it on R-Car M2-W (arm32) and R-Car H3 (arm64), and
> > that did not cause the problem to happen...
>
> I guess this patch triggers a potential problem with the maple tree.
> I don't have the environment to trigger the problem. Can you apply the
> following patch to see if the problem still exists? This can help locate
> the root cause. At least narrow down the scope of the code that has bug.

Thanks for your suggestion!

> diff --git a/lib/maple_tree.c b/lib/maple_tree.c
> index f723024e1426..1b4b6f6e3095 100644
> --- a/lib/maple_tree.c
> +++ b/lib/maple_tree.c
> @@ -4351,9 +4351,6 @@ static inline void mas_wr_modify(struct
> ma_wr_state *wr_mas)
>         if (new_end == wr_mas->node_end && mas_wr_slot_store(wr_mas))
>                 return;
>
> -       if (mas_wr_node_store(wr_mas, new_end))
> -               return;
> -
>         if (mas_is_err(mas))
>                 return;

Unfortunately removing these lines does not help.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds


From mpe at ellerman.id.au  Wed Aug 30 22:37:02 2023
From: mpe at ellerman.id.au (Michael Ellerman)
Date: Thu, 31 Aug 2023 15:37:02 +1000
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <87bkeotin8.fsf@igel.home>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
 <20230819004356.1454718-2-Liam.Howlett@oracle.com>
 <87bkeotin8.fsf@igel.home>
Message-ID: <87y1hr22o1.fsf@mail.lhotse>

Andreas Schwab <schwab at linux-m68k.org> writes:
> This breaks booting on ppc32:

Does enabling CONFIG_DEBUG_ATOMIC_SLEEP fix the crash? It did for me on
qemu.

cheers

> Kernel attemptd to writ user page (1ff0) - exploit attempt? (uid: 0)
> BUG: Unable to handle kernel data access on write at 0x00001ff0
> Faulting instruction address: 0xc0009554
> Vector: 300 (Data Access) at [c0b09d10]
>     pc: c0009554: do_softirq_own_stack+0x18/0x30
>     lr: c004f480: __irq_exit_rcu+0x70/0xc0
>     sp: c0b09dd0
>    msr: 1032
>    dar: 1ff0
>  dsisr: 42000000
>   current = 0xc0a08360
>     pid   = 0, comm = swapper
> Linux version 6.5.0 ...
> enter ? for help
> [c0b09de0] c00ff480 __irq_exit_rcu+0x70/0xc0
> [c0b09df0] c0005a98 Decrementer_virt+0x108/0x10c
> --- Exception: 900 (Decrementer) at c06cfa0c __schedule+0x4fc/0x510
> [c0b09ec0] c06cf75c __schedule+0x1cc/0x510 (unreliable)
> [c0b09ef0] c06cfc90 __cond_resched+0x2c/0x54
> [c0b09f00] c06d07f8 mutex_lock_killable+0x18/0x5c
> [c0b09f10] c013c404 pcpu_alloc+0x110/0x4dc
> [c0b09f70] c000cc34 alloc_descr.isra.18+0x48/0x144
> [c0b09f90] c0988aa0 early_irq_init+0x64/0x8c
> [c0b09fa0] c097a5a4 start_kernel+0x5b4/0x7b0
> [c0b09ff0] 00003dc0
> mon>
>
> -- 
> Andreas Schwab, schwab at linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
> "And now for something completely different."


From mpe at ellerman.id.au  Wed Aug 30 22:39:46 2023
From: mpe at ellerman.id.au (Michael Ellerman)
Date: Thu, 31 Aug 2023 15:39:46 +1000
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <3f86d58e-7f36-c6b4-c43a-2a7bcffd3bd@linux-m68k.org>
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
 <20230819004356.1454718-2-Liam.Howlett@oracle.com>
 <3f86d58e-7f36-c6b4-c43a-2a7bcffd3bd@linux-m68k.org>
Message-ID: <87v8cv22jh.fsf@mail.lhotse>

Geert Uytterhoeven <geert at linux-m68k.org> writes:
>  	Hi Liam,
>
> On Fri, 18 Aug 2023, Liam R. Howlett wrote:
>> The current implementation of append may cause duplicate data and/or
>> incorrect ranges to be returned to a reader during an update.  Although
>> this has not been reported or seen, disable the append write operation
>> while the tree is in rcu mode out of an abundance of caution.
>>
>> During the analysis of the mas_next_slot() the following was
>> artificially created by separating the writer and reader code:
>>
>> Writer:                                 reader:
>> mas_wr_append
>>    set end pivot
>>    updates end metata
>>    Detects write to last slot
>>    last slot write is to start of slot
>>    store current contents in slot
>>    overwrite old end pivot
>>                                        mas_next_slot():
>>                                                read end metadata
>>                                                read old end pivot
>>                                                return with incorrect range
>>    store new value
>>
>> Alternatively:
>>
>> Writer:                                 reader:
>> mas_wr_append
>>    set end pivot
>>    updates end metata
>>    Detects write to last slot
>>    last lost write to end of slot
>>    store value
>>                                        mas_next_slot():
>>                                                read end metadata
>>                                                read old end pivot
>>                                                read new end pivot
>>                                                return with incorrect range
>>    set old end pivot
>>
>> There may be other accesses that are not safe since we are now updating
>> both metadata and pointers, so disabling append if there could be rcu
>> readers is the safest action.
>>
>> Fixes: 54a611b60590 ("Maple Tree: add new data structure")
>> Cc: stable at vger.kernel.org
>> Signed-off-by: Liam R. Howlett <Liam.Howlett at oracle.com>
>
> Thanks for your patch, which is now commit cfeb6ae8bcb96ccf
> ("maple_tree: disable mas_wr_append() when other readers are
> possible") in v6.5, and is being backported to stable.
>
> On Renesas RZ/A1 and RZ/A2 (single-core Cortex-A9), this causes the
> following warning:
>
>       clocksource: timer at e803b000: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 28958491609 ns
>       sched_clock: 32 bits at 66MHz, resolution 15ns, wraps every 32537631224ns
>       /soc/timer at e803b000: used for clocksource
>       /soc/timer at e803c000: used for clock events
>      +------------[ cut here ]------------
>      +WARNING: CPU: 0 PID: 0 at init/main.c:992 start_kernel+0x2f0/0x480
>      +Interrupts were enabled early
...
>
> I do not see this issue on any other platform
> (arm/arm64/risc-v/mips/sh/m68k), several of them use the same
> RCU configuration.

There's something similar on pmac32 / mac99.

> Do you have a clue?

It seems something in the maple tree code is setting TIF_NEED_RESCHED,
and that causes a subsequent call to cond_resched() to call schedule()
and enable interrupts.

On pmac32 enabling CONFIG_DEBUG_ATOMIC_SLEEP fixes/hides the problem.
But I don't see why.

cheers


From oliver.sang at intel.com  Thu Aug 31 06:40:51 2023
From: oliver.sang at intel.com (kernel test robot)
Date: Thu, 31 Aug 2023 21:40:51 +0800
Subject: [PATCH v2 5/6] maple_tree: Update check_forking() and
 bench_forking()
In-Reply-To: <20230830125654.21257-6-zhangpeng.00@bytedance.com>
Message-ID: <202308312115.cad34fed-oliver.sang@intel.com>



Hello,

kernel test robot noticed "WARNING:possible_recursive_locking_detected" on:

commit: 2730245bd6b13a94a67e84c10832a9f52fad0aa5 ("[PATCH v2 5/6] maple_tree: Update check_forking() and bench_forking()")
url: https://github.com/intel-lab-lkp/linux/commits/Peng-Zhang/maple_tree-Add-two-helpers/20230830-205847
base: https://git.kernel.org/cgit/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/all/20230830125654.21257-6-zhangpeng.00 at bytedance.com/
patch subject: [PATCH v2 5/6] maple_tree: Update check_forking() and bench_forking()

in testcase: boot

compiler: clang-16
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G

(please refer to attached dmesg/kmsg for entire log/backtrace)



If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <oliver.sang at intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202308312115.cad34fed-oliver.sang at intel.com


[   25.146957][    T1] WARNING: possible recursive locking detected
[   25.147110][    T1] 6.5.0-rc4-00632-g2730245bd6b1 #1 Tainted: G                TN
[   25.147110][    T1] --------------------------------------------
[   25.147110][    T1] swapper/1 is trying to acquire lock:
[ 25.147110][ T1] ffffffff86485058 (&mt->ma_lock){+.+.}-{2:2}, at: check_forking (include/linux/spinlock.h:? lib/test_maple_tree.c:1854) 
[   25.147110][    T1]
[   25.147110][    T1] but task is already holding lock:
[ 25.147110][ T1] ffff888110847a30 (&mt->ma_lock){+.+.}-{2:2}, at: check_forking (include/linux/spinlock.h:351 lib/test_maple_tree.c:1854) 
[   25.147110][    T1]
[   25.147110][    T1] other info that might help us debug this:
[   25.147110][    T1]  Possible unsafe locking scenario:
[   25.147110][    T1]
[   25.147110][    T1]        CPU0
[   25.147110][    T1]        ----
[   25.147110][    T1]   lock(&mt->ma_lock);
[   25.147110][    T1]
[   25.147110][    T1]  *** DEADLOCK ***
[   25.147110][    T1]
[   25.147110][    T1]  May be due to missing lock nesting notation
[   25.147110][    T1]
[   25.147110][    T1] 1 lock held by swapper/1:
[ 25.147110][ T1] #0: ffff888110847a30 (&mt->ma_lock){+.+.}-{2:2}, at: check_forking (include/linux/spinlock.h:351 lib/test_maple_tree.c:1854) 
[   25.147110][    T1]
[   25.147110][    T1] stack backtrace:
[   25.147110][    T1] CPU: 0 PID: 1 Comm: swapper Tainted: G                TN 6.5.0-rc4-00632-g2730245bd6b1 #1
[   25.147110][    T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
[   25.147110][    T1] Call Trace:
[   25.147110][    T1]  <TASK>
[ 25.147110][ T1] dump_stack_lvl (lib/dump_stack.c:? lib/dump_stack.c:106) 
[ 25.147110][ T1] validate_chain (kernel/locking/lockdep.c:?) 
[ 25.147110][ T1] ? look_up_lock_class (kernel/locking/lockdep.c:926) 
[ 25.147110][ T1] ? mark_lock (arch/x86/include/asm/bitops.h:228 arch/x86/include/asm/bitops.h:240 include/asm-generic/bitops/instrumented-non-atomic.h:142 kernel/locking/lockdep.c:228 kernel/locking/lockdep.c:4655) 
[ 25.147110][ T1] __lock_acquire (kernel/locking/lockdep.c:?) 
[ 25.147110][ T1] lock_acquire (kernel/locking/lockdep.c:5753) 
[ 25.147110][ T1] ? check_forking (include/linux/spinlock.h:? lib/test_maple_tree.c:1854) 
[ 25.147110][ T1] _raw_spin_lock (include/linux/spinlock_api_smp.h:133 kernel/locking/spinlock.c:154) 
[ 25.147110][ T1] ? check_forking (include/linux/spinlock.h:? lib/test_maple_tree.c:1854) 
[ 25.147110][ T1] check_forking (include/linux/spinlock.h:? lib/test_maple_tree.c:1854) 
[ 25.147110][ T1] maple_tree_seed (lib/test_maple_tree.c:3583) 
[ 25.147110][ T1] do_one_initcall (init/main.c:1232) 
[ 25.147110][ T1] ? __cfi_maple_tree_seed (lib/test_maple_tree.c:3508) 
[ 25.147110][ T1] do_initcall_level (init/main.c:1293) 
[ 25.147110][ T1] do_initcalls (init/main.c:1307) 
[ 25.147110][ T1] kernel_init_freeable (init/main.c:1550) 
[ 25.147110][ T1] ? __cfi_kernel_init (init/main.c:1429) 
[ 25.147110][ T1] kernel_init (init/main.c:1439) 
[ 25.147110][ T1] ? __cfi_kernel_init (init/main.c:1429) 
[ 25.147110][ T1] ret_from_fork (arch/x86/kernel/process.c:151) 
[ 25.147110][ T1] ? __cfi_kernel_init (init/main.c:1429) 
[ 25.147110][ T1] ret_from_fork_asm (arch/x86/entry/entry_64.S:312) 
[   25.147110][    T1]  </TASK>
[   28.697241][   T32] clocksource_wdtest: --- Verify jiffies-like uncertainty margin.
[   28.698316][   T32] clocksource: wdtest-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 6370867519511994 ns
[   29.714980][   T32] clocksource_wdtest: --- Verify tsc-like uncertainty margin.
[   29.716387][   T32] clocksource: wdtest-ktime: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[   29.721896][   T32] clocksource_wdtest: --- tsc-like times: 1693478138832947444 - 1693478138832945950 = 1494.
[   29.723570][   T32] clocksource_wdtest: --- Watchdog with 0x error injection, 2 retries.
[   31.898906][   T32] clocksource_wdtest: --- Watchdog with 1x error injection, 2 retries.
[   34.043415][   T32] clocksource_wdtest: --- Watchdog with 2x error injection, 2 retries, expect message.
[   34.512462][    C0] clocksource: timekeeping watchdog on CPU0: kvm-clock retried 2 times before success
[   36.169157][   T32] clocksource_wdtest: --- Watchdog with 3x error injection, 2 retries, expect clock skew.
[   36.513464][    C0] clocksource: timekeeping watchdog on CPU0: wd-wdtest-ktime-wd excessive read-back delay of 1000880ns vs. limit of 125000ns, wd-wd read-back delay only 46ns, attempt 3, marking wdtest-ktime unstable
[   36.516829][    C0] clocksource_wdtest: --- Marking wdtest-ktime unstable due to clocksource watchdog.
[   38.412889][   T32] clocksource: wdtest-ktime: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[   38.421249][   T32] clocksource_wdtest: --- Watchdog clock-value-fuzz error injection, expect clock skew and per-CPU mismatches.
[   38.990462][    C0] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'wdtest-ktime' as unstable because the skew is too large:
[   38.992698][    C0] clocksource:                       'kvm-clock' wd_nsec: 479996388 wd_now: 9454aecf2 wd_last: 928aec30e mask: ffffffffffffffff
[   38.994924][    C0] clocksource:                       'wdtest-ktime' cs_nsec: 679996638 cs_now: 17807167426ff864 cs_last: 1780716719e80b86 mask: ffffffffffffffff
[   38.997374][    C0] clocksource:                       Clocksource 'wdtest-ktime' skewed 200000250 ns (200 ms) over watchdog 'kvm-clock' interval of 479996388 ns (479 ms)
[   38.999919][    C0] clocksource:                       'kvm-clock' (not 'wdtest-ktime') is current clocksource.
[   39.001696][    C0] clocksource_wdtest: --- Marking wdtest-ktime unstable due to clocksource watchdog.
[   40.441815][   T32] clocksource: Not enough CPUs to check clocksource 'wdtest-ktime'.
[   40.443303][   T32] clocksource_wdtest: --- Done with test.
[  293.673815][    T1] swapper invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
[  293.675628][    T1] CPU: 0 PID: 1 Comm: swapper Tainted: G                TN 6.5.0-rc4-00632-g2730245bd6b1 #1
[  293.677082][    T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
[  293.677082][    T1] Call Trace:
[  293.677082][    T1]  <TASK>
[ 293.677082][ T1] dump_stack_lvl (lib/dump_stack.c:107) 
[ 293.677082][ T1] dump_header (mm/oom_kill.c:?) 
[ 293.677082][ T1] out_of_memory (mm/oom_kill.c:1159) 
[ 293.677082][ T1] __alloc_pages_slowpath (mm/page_alloc.c:3372 mm/page_alloc.c:4132) 
[ 293.677082][ T1] __alloc_pages (mm/page_alloc.c:4469) 
[ 293.677082][ T1] alloc_slab_page (mm/slub.c:1866) 
[ 293.677082][ T1] new_slab (mm/slub.c:2017 mm/slub.c:2062) 
[ 293.677082][ T1] ? mas_alloc_nodes (lib/maple_tree.c:1282) 
[ 293.677082][ T1] ___slab_alloc (arch/x86/include/asm/preempt.h:80 mm/slub.c:3216) 
[ 293.677082][ T1] ? mas_alloc_nodes (lib/maple_tree.c:1282) 
[ 293.677082][ T1] kmem_cache_alloc_bulk (mm/slub.c:? mm/slub.c:4041) 
[ 293.677082][ T1] mas_alloc_nodes (lib/maple_tree.c:1282) 
[ 293.677082][ T1] mas_nomem (lib/maple_tree.c:?) 
[ 293.677082][ T1] mtree_store_range (lib/maple_tree.c:6191) 
[ 293.677082][ T1] check_dup_gaps (lib/test_maple_tree.c:2623) 
[ 293.677082][ T1] check_dup (lib/test_maple_tree.c:2707) 
[ 293.677082][ T1] maple_tree_seed (lib/test_maple_tree.c:3766) 
[ 293.677082][ T1] do_one_initcall (init/main.c:1232) 
[ 293.677082][ T1] ? __cfi_maple_tree_seed (lib/test_maple_tree.c:3508) 
[ 293.677082][ T1] do_initcall_level (init/main.c:1293) 
[ 293.677082][ T1] do_initcalls (init/main.c:1307) 
[ 293.677082][ T1] kernel_init_freeable (init/main.c:1550) 
[ 293.677082][ T1] ? __cfi_kernel_init (init/main.c:1429) 


The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20230831/202308312115.cad34fed-oliver.sang at intel.com



-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki



From schwab at linux-m68k.org  Thu Aug 31 12:01:38 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Thu, 31 Aug 2023 21:01:38 +0200
Subject: [PATCH v2 1/2] maple_tree: Disable mas_wr_append() when other
 readers are possible
In-Reply-To: <87y1hr22o1.fsf@mail.lhotse> (Michael Ellerman's message of "Thu,
	31 Aug 2023 15:37:02 +1000")
References: <20230819004356.1454718-1-Liam.Howlett@oracle.com>
	<20230819004356.1454718-2-Liam.Howlett@oracle.com>
	<87bkeotin8.fsf@igel.home> <87y1hr22o1.fsf@mail.lhotse>
Message-ID: <87cyz3594d.fsf@igel.home>

On Aug 31 2023, Michael Ellerman wrote:

> Andreas Schwab <schwab at linux-m68k.org> writes:
>> This breaks booting on ppc32:
>
> Does enabling CONFIG_DEBUG_ATOMIC_SLEEP fix the crash?

Yes, it does.

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


