[PATCH v3 1/3] mm/memfd_luo: reject kernel-private shmem files

[Haolong Wang]

memfd_luo_can_preserve() accepts any unlinked shmem file.  That includes
SysV SHM files, which are created with shmem_kernel_file_setup() and
marked S_PRIVATE: a privileged process can obtain a file descriptor for
an attached segment through /proc/<pid>/map_files and hand that
descriptor to LUO.

The preservation path then updates SHMEM_I(inode)->flags under the inode
lock when it sets SHMEM_F_MAPPING_FROZEN, while SHM_LOCK and SHM_UNLOCK
update the same word through shmem_lock(), serialized by
ipc_lock_object() instead.  Two read-modify-write cycles on one word with
no common lock can lose either the LUO frozen bit or SHMEM_F_LOCKED, so
the frozen state the handler depends on may quietly not be in effect.
The seal-state freeze this series adds (SHMEM_F_SEAL_STATE_FROZEN) has
the same synchronization requirement.

Reject S_PRIVATE shmem inodes from the memfd LUO handler.  Real memfds
are created with shmem_file_setup() without S_PRIVATE, while
shmem_kernel_file_setup() deliberately uses the flag for kernel-private
objects: SysV SHM, big_key payloads, the XFS buffer and scrub files, and
the anonymous /dev/zero mapping file.  Those must not be preserved as
memfds, and dropping them also removes the only concurrent writer of that
flags word from the handler's reach, so no shmem-wide locking change is
needed here.

The predicate is deliberately not narrowed to memfd_create() objects:
unlinked non-private shmem files stay eligible.  SHMEM_F_LOCKED is only
maintained for SysV segments, so they do not share this race and keep
being preserved as before.

Measured with QEMU runs on this tree, with CONFIG_CHECKPOINT_RESTORE and
CONFIG_SYSVIPC enabled and liveupdate=on, so that a descriptor for a
segment is reachable; memfd_luo is the only file handler registered in
that configuration:

  kernel                SysV SHM fd        memfd     unlinked tmpfs
  without this patch    accepted (0)       accepted  accepted
  with this patch       -ENOENT (errno 2)  accepted  accepted

The SysV descriptor came from /proc/self/map_files and reads back as
"/SYSV00000000 (deleted)"; the memfd and the unlinked tmpfs file are the
controls that show the ioctl path still works and that the check is
limited to kernel-private objects.  The SysV case is the same guarantee
that the pre-existing SHMEM_F_MAPPING_FROZEN bit relies on, so the
mismatch is older than this series; atomic flag updates or a shared lock
would be a change across shmem and ipc, and this patch removes the
handler's exposure to it instead.

Fixes: b3749f174d68 ("mm: memfd_luo: allow preserving memfd")
Reported-by: Sashiko <sashiko-bot at kernel.org>
Link: https://lore.kernel.org/all/sashiko-outbox-170776@kernel.org/
Assisted-by: Pi:glm-5.3-flash
Signed-off-by: Haolong Wang <18766350203 at 163.com>
---
 mm/memfd_luo.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/mm/memfd_luo.c b/mm/memfd_luo.c
index 59de210bee5f..6f8b84530995 100644
--- a/mm/memfd_luo.c
+++ b/mm/memfd_luo.c
@@ -582,7 +582,12 @@ static bool memfd_luo_can_preserve(struct liveupdate_file_handler *handler,
 {
 	struct inode *inode = file_inode(file);
 
-	return shmem_file(file) && !inode->i_nlink;
+	/*
+	 * shmem_kernel_file_setup() marks kernel-private files S_PRIVATE.  In
+	 * particular, SysV SHM updates SHMEM_I(inode)->flags under its IPC lock,
+	 * which does not serialize with the inode lock used by LUO.
+	 */
+	return shmem_file(file) && !IS_PRIVATE(inode) && !inode->i_nlink;
 }
 
 static unsigned long memfd_luo_get_id(struct file *file)
-- 
2.56.0