From nathan at kernel.org  Tue Oct  3 11:35:53 2023
From: nathan at kernel.org (Nathan Chancellor)
Date: Tue, 03 Oct 2023 11:35:53 -0700
Subject: [PATCH] um: net: Fix return type of uml_net_start_xmit()
Message-ID: <20231003-um-net-wifpts-v1-1-02888c634ee7@kernel.org>

With clang's kernel control flow integrity (kCFI, CONFIG_CFI_CLANG),
indirect call targets are validated against the expected function
pointer prototype to make sure the call target is valid to help mitigate
ROP attacks. If they are not identical, there is a failure at run time,
which manifests as either a kernel panic or thread getting killed. A
warning in clang aims to catch these at compile time, which reveals:

  arch/um/drivers/net_kern.c:353:21: warning: incompatible function pointer types initializing 'netdev_tx_t (*)(struct sk_buff *, struct net_device *)' (aka 'enum netdev_tx (*)(struct sk_buff *, struct net_device *)') with an expression of type 'int (struct sk_buff *, struct net_device *)' [-Wincompatible-function-pointer-types-strict]
    353 |         .ndo_start_xmit         = uml_net_start_xmit,
        |                                   ^~~~~~~~~~~~~~~~~~
  1 warning generated.

->ndo_start_xmit() in 'struct net_device_ops' expects a return type of
'netdev_tx_t', not 'int'. Adjust the return type of uml_net_start_xmit()
to match the prototype's to resolve the warning. While UML does not
currently implement support for kCFI, it could in the future, which
means this warning becomes a fatal CFI failure at run time.

Reported-by: kernel test robot <lkp at intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202310031340.v1vPh207-lkp at intel.com/
Signed-off-by: Nathan Chancellor <nathan at kernel.org>
---
 arch/um/drivers/net_kern.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/um/drivers/net_kern.c b/arch/um/drivers/net_kern.c
index 3d7836c46507..cabcc501b448 100644
--- a/arch/um/drivers/net_kern.c
+++ b/arch/um/drivers/net_kern.c
@@ -204,7 +204,7 @@ static int uml_net_close(struct net_device *dev)
 	return 0;
 }
 
-static int uml_net_start_xmit(struct sk_buff *skb, struct net_device *dev)
+static netdev_tx_t uml_net_start_xmit(struct sk_buff *skb, struct net_device *dev)
 {
 	struct uml_net_private *lp = netdev_priv(dev);
 	unsigned long flags;

---
base-commit: 8a749fd1a8720d4619c91c8b6e7528c0a355c0aa
change-id: 20231003-um-net-wifpts-14a7429c7524

Best regards,
-- 
Nathan Chancellor <nathan at kernel.org>



From jlayton at kernel.org  Wed Oct  4 11:52:28 2023
From: jlayton at kernel.org (Jeff Layton)
Date: Wed,  4 Oct 2023 14:52:28 -0400
Subject: [PATCH v2 43/89] hostfs: convert to new timestamp accessors
In-Reply-To: <20231004185347.80880-1-jlayton@kernel.org>
References: <20231004185221.80802-1-jlayton@kernel.org>
 <20231004185347.80880-1-jlayton@kernel.org>
Message-ID: <20231004185347.80880-41-jlayton@kernel.org>

Convert to using the new inode timestamp accessor functions.

Signed-off-by: Jeff Layton <jlayton at kernel.org>
---
 fs/hostfs/hostfs_kern.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/fs/hostfs/hostfs_kern.c b/fs/hostfs/hostfs_kern.c
index dc5a5cea5fae..ea87f24c6c3f 100644
--- a/fs/hostfs/hostfs_kern.c
+++ b/fs/hostfs/hostfs_kern.c
@@ -513,10 +513,14 @@ static int hostfs_inode_update(struct inode *ino, const struct hostfs_stat *st)
 	set_nlink(ino, st->nlink);
 	i_uid_write(ino, st->uid);
 	i_gid_write(ino, st->gid);
-	ino->i_atime =
-		(struct timespec64){ st->atime.tv_sec, st->atime.tv_nsec };
-	ino->i_mtime =
-		(struct timespec64){ st->mtime.tv_sec, st->mtime.tv_nsec };
+	inode_set_atime_to_ts(ino, (struct timespec64){
+			st->atime.tv_sec,
+			st->atime.tv_nsec,
+		});
+	inode_set_mtime_to_ts(ino, (struct timespec64){
+			st->mtime.tv_sec,
+			st->mtime.tv_nsec,
+		});
 	inode_set_ctime(ino, st->ctime.tv_sec, st->ctime.tv_nsec);
 	ino->i_size = st->size;
 	ino->i_blocks = st->blocks;
-- 
2.41.0



From jlayton at kernel.org  Wed Oct  4 11:52:21 2023
From: jlayton at kernel.org (Jeff Layton)
Date: Wed,  4 Oct 2023 14:52:21 -0400
Subject: [PATCH v2 00/89] fs: new accessor methods for inode atime and mtime
Message-ID: <20231004185221.80802-1-jlayton@kernel.org>

v2:
- bugfix in mtime handling
- incorporate _sec and _nsec accessor functions (Chuck Lever)
- move i_generation to plug hole after changing timestamps (Amir Goldstein)

While working on the multigrain timestamp changes, Linus suggested
adding some similar wrappers for accessing the atime and mtime that we
have for the ctime. With that, we could then move to using discrete
integers instead of struct timespec64 in struct inode and shrink it.

This patch implements this. Linus suggested using macros for the new
accessors, but the existing ctime wrappers were static inlines and since
there are only 3 different timestamps, I didn't see that trying to
fiddle with macros would gain us anything (other than less verbosity in
fs.h).

The second to last patch makes the conversion to discrete integers,
which shaves 8 bytes off of struct inode on my x86_64 kernel. The last
patch reshuffles things a little to keep the i_lock in the same
cacheline as the fields it protects.

About 75% of this conversion was done with coccinelle, with the rest
done by hand.

I think we probably ought to try to get everything but the last two
patches into v6.7 (though we could consider those too if we're feeling
lucky).

Jeff Layton (89):
  fs: new accessor methods for atime and mtime
  fs: convert core infrastructure to new timestamp accessors
  spufs: convert to new timestamp accessors
  hypfs: convert to new timestamp accessors
  android: convert to new timestamp accessors
  char: convert to new timestamp accessors
  qib: convert to new timestamp accessors
  ibmasm: convert to new timestamp accessors
  misc: convert to new timestamp accessors
  x86: convert to new timestamp accessors
  tty: convert to new timestamp accessors
  function: convert to new timestamp accessors
  legacy: convert to new timestamp accessors
  usb: convert to new timestamp accessors
  9p: convert to new timestamp accessors
  adfs: convert to new timestamp accessors
  affs: convert to new timestamp accessors
  afs: convert to new timestamp accessors
  autofs: convert to new timestamp accessors
  bcachefs: convert to new timestamp accessors
  befs: convert to new timestamp accessors
  bfs: convert to new timestamp accessors
  btrfs: convert to new timestamp accessors
  ceph: convert to new timestamp accessors
  coda: convert to new timestamp accessors
  configfs: convert to new timestamp accessors
  cramfs: convert to new timestamp accessors
  debugfs: convert to new timestamp accessors
  devpts: convert to new timestamp accessors
  efivarfs: convert to new timestamp accessors
  efs: convert to new timestamp accessors
  erofs: convert to new timestamp accessors
  exfat: convert to new timestamp accessors
  ext2: convert to new timestamp accessors
  ext4: convert to new timestamp accessors
  f2fs: convert to new timestamp accessors
  fat: convert to new timestamp accessors
  freevxfs: convert to new timestamp accessors
  fuse: convert to new timestamp accessors
  gfs2: convert to new timestamp accessors
  hfs: convert to new timestamp accessors
  hfsplus: convert to new timestamp accessors
  hostfs: convert to new timestamp accessors
  hpfs: convert to new timestamp accessors
  hugetlbfs: convert to new timestamp accessors
  isofs: convert to new timestamp accessors
  jffs2: convert to new timestamp accessors
  jfs: convert to new timestamp accessors
  kernfs: convert to new timestamp accessors
  minix: convert to new timestamp accessors
  nfs: convert to new timestamp accessors
  nfsd: convert to new timestamp accessors
  nilfs2: convert to new timestamp accessors
  ntfs: convert to new timestamp accessors
  ntfs3: convert to new timestamp accessors
  ocfs2: convert to new timestamp accessors
  omfs: convert to new timestamp accessors
  openpromfs: convert to new timestamp accessors
  orangefs: convert to new timestamp accessors
  overlayfs: convert to new timestamp accessors
  proc: convert to new timestamp accessors
  pstore: convert to new timestamp accessors
  qnx4: convert to new timestamp accessors
  qnx6: convert to new timestamp accessors
  ramfs: convert to new timestamp accessors
  reiserfs: convert to new timestamp accessors
  romfs: convert to new timestamp accessors
  client: convert to new timestamp accessors
  server: convert to new timestamp accessors
  squashfs: convert to new timestamp accessors
  sysv: convert to new timestamp accessors
  tracefs: convert to new timestamp accessors
  ubifs: convert to new timestamp accessors
  udf: convert to new timestamp accessors
  ufs: convert to new timestamp accessors
  vboxsf: convert to new timestamp accessors
  xfs: convert to new timestamp accessors
  zonefs: convert to new timestamp accessors
  linux: convert to new timestamp accessors
  ipc: convert to new timestamp accessors
  bpf: convert to new timestamp accessors
  mm: convert to new timestamp accessors
  sunrpc: convert to new timestamp accessors
  apparmor: convert to new timestamp accessors
  selinux: convert to new timestamp accessors
  security: convert to new timestamp accessors
  fs: rename inode i_atime and i_mtime fields
  fs: switch timespec64 fields in inode to discrete integers
  fs: move i_generation into new hole created after timestamp conversion

 arch/powerpc/platforms/cell/spufs/inode.c |   2 +-
 arch/s390/hypfs/inode.c                   |   4 +-
 drivers/android/binderfs.c                |   8 +-
 drivers/char/sonypi.c                     |   2 +-
 drivers/infiniband/hw/qib/qib_fs.c        |   4 +-
 drivers/misc/ibmasm/ibmasmfs.c            |   2 +-
 drivers/misc/ibmvmc.c                     |   2 +-
 drivers/platform/x86/sony-laptop.c        |   2 +-
 drivers/tty/tty_io.c                      |  10 +-
 drivers/usb/core/devio.c                  |  26 +++--
 drivers/usb/gadget/function/f_fs.c        |   4 +-
 drivers/usb/gadget/legacy/inode.c         |   2 +-
 fs/9p/vfs_inode.c                         |   6 +-
 fs/9p/vfs_inode_dotl.c                    |  16 +--
 fs/adfs/inode.c                           |  13 ++-
 fs/affs/amigaffs.c                        |   4 +-
 fs/affs/inode.c                           |  17 ++-
 fs/afs/dynroot.c                          |   2 +-
 fs/afs/inode.c                            |   8 +-
 fs/afs/write.c                            |   2 +-
 fs/attr.c                                 |   4 +-
 fs/autofs/inode.c                         |   2 +-
 fs/autofs/root.c                          |   6 +-
 fs/bad_inode.c                            |   2 +-
 fs/bcachefs/fs.c                          |  12 +--
 fs/befs/linuxvfs.c                        |  10 +-
 fs/bfs/dir.c                              |   9 +-
 fs/bfs/inode.c                            |  12 +--
 fs/binfmt_misc.c                          |   2 +-
 fs/btrfs/delayed-inode.c                  |  20 ++--
 fs/btrfs/file.c                           |  18 ++--
 fs/btrfs/inode.c                          |  43 ++++----
 fs/btrfs/reflink.c                        |   2 +-
 fs/btrfs/transaction.c                    |   3 +-
 fs/btrfs/tree-log.c                       |  12 +--
 fs/ceph/addr.c                            |  10 +-
 fs/ceph/caps.c                            |   4 +-
 fs/ceph/file.c                            |   2 +-
 fs/ceph/inode.c                           |  64 ++++++------
 fs/ceph/mds_client.c                      |   8 +-
 fs/ceph/snap.c                            |   4 +-
 fs/coda/coda_linux.c                      |   6 +-
 fs/coda/dir.c                             |   2 +-
 fs/coda/file.c                            |   2 +-
 fs/configfs/inode.c                       |   8 +-
 fs/cramfs/inode.c                         |   4 +-
 fs/debugfs/inode.c                        |   2 +-
 fs/devpts/inode.c                         |   6 +-
 fs/efivarfs/file.c                        |   2 +-
 fs/efivarfs/inode.c                       |   2 +-
 fs/efs/inode.c                            |   5 +-
 fs/erofs/inode.c                          |   3 +-
 fs/exfat/exfat_fs.h                       |   1 +
 fs/exfat/file.c                           |   7 +-
 fs/exfat/inode.c                          |  31 +++---
 fs/exfat/misc.c                           |   8 ++
 fs/exfat/namei.c                          |  31 +++---
 fs/exfat/super.c                          |   4 +-
 fs/ext2/dir.c                             |   6 +-
 fs/ext2/ialloc.c                          |   2 +-
 fs/ext2/inode.c                           |  13 ++-
 fs/ext2/super.c                           |   2 +-
 fs/ext4/ext4.h                            |  20 +++-
 fs/ext4/extents.c                         |  11 +-
 fs/ext4/ialloc.c                          |   4 +-
 fs/ext4/inline.c                          |   4 +-
 fs/ext4/inode.c                           |  19 ++--
 fs/ext4/ioctl.c                           |  13 ++-
 fs/ext4/namei.c                           |  10 +-
 fs/ext4/super.c                           |   2 +-
 fs/ext4/xattr.c                           |   8 +-
 fs/f2fs/dir.c                             |   6 +-
 fs/f2fs/f2fs.h                            |  10 +-
 fs/f2fs/file.c                            |  14 +--
 fs/f2fs/inline.c                          |   2 +-
 fs/f2fs/inode.c                           |  24 ++---
 fs/f2fs/namei.c                           |   4 +-
 fs/f2fs/recovery.c                        |   8 +-
 fs/f2fs/super.c                           |   2 +-
 fs/fat/inode.c                            |  25 +++--
 fs/fat/misc.c                             |   6 +-
 fs/freevxfs/vxfs_inode.c                  |   6 +-
 fs/fuse/control.c                         |   2 +-
 fs/fuse/dir.c                             |  10 +-
 fs/fuse/inode.c                           |  29 +++---
 fs/fuse/readdir.c                         |   6 +-
 fs/gfs2/bmap.c                            |  10 +-
 fs/gfs2/dir.c                             |  10 +-
 fs/gfs2/glops.c                           |  11 +-
 fs/gfs2/inode.c                           |   7 +-
 fs/gfs2/quota.c                           |   2 +-
 fs/gfs2/super.c                           |  12 +--
 fs/hfs/catalog.c                          |   8 +-
 fs/hfs/inode.c                            |  16 +--
 fs/hfs/sysdep.c                           |  10 +-
 fs/hfsplus/catalog.c                      |   8 +-
 fs/hfsplus/inode.c                        |  22 ++--
 fs/hostfs/hostfs_kern.c                   |  12 ++-
 fs/hpfs/dir.c                             |  12 ++-
 fs/hpfs/inode.c                           |  16 +--
 fs/hpfs/namei.c                           |  22 ++--
 fs/hpfs/super.c                           |  10 +-
 fs/hugetlbfs/inode.c                      |  10 +-
 fs/inode.c                                |  35 ++++---
 fs/isofs/inode.c                          |   4 +-
 fs/isofs/rock.c                           |  18 ++--
 fs/jffs2/dir.c                            |  35 ++++---
 fs/jffs2/file.c                           |   4 +-
 fs/jffs2/fs.c                             |  20 ++--
 fs/jffs2/os-linux.h                       |   4 +-
 fs/jfs/inode.c                            |   2 +-
 fs/jfs/jfs_imap.c                         |  20 ++--
 fs/jfs/jfs_inode.c                        |   4 +-
 fs/jfs/namei.c                            |  20 ++--
 fs/jfs/super.c                            |   2 +-
 fs/kernfs/inode.c                         |   6 +-
 fs/libfs.c                                |  41 ++++++--
 fs/minix/bitmap.c                         |   2 +-
 fs/minix/dir.c                            |   6 +-
 fs/minix/inode.c                          |  17 ++-
 fs/minix/itree_common.c                   |   2 +-
 fs/nfs/callback_proc.c                    |   2 +-
 fs/nfs/fscache.h                          |   4 +-
 fs/nfs/inode.c                            |  30 +++---
 fs/nfsd/blocklayout.c                     |   3 +-
 fs/nfsd/nfs3proc.c                        |   4 +-
 fs/nfsd/nfs4proc.c                        |   8 +-
 fs/nfsd/nfsctl.c                          |   2 +-
 fs/nfsd/vfs.c                             |   2 +-
 fs/nilfs2/dir.c                           |   6 +-
 fs/nilfs2/inode.c                         |  20 ++--
 fs/nsfs.c                                 |   2 +-
 fs/ntfs/inode.c                           |  25 ++---
 fs/ntfs/mft.c                             |   2 +-
 fs/ntfs3/file.c                           |   6 +-
 fs/ntfs3/frecord.c                        |  11 +-
 fs/ntfs3/inode.c                          |  25 +++--
 fs/ntfs3/namei.c                          |   4 +-
 fs/ocfs2/acl.c                            |   4 +-
 fs/ocfs2/alloc.c                          |   6 +-
 fs/ocfs2/aops.c                           |   6 +-
 fs/ocfs2/dir.c                            |   9 +-
 fs/ocfs2/dlmfs/dlmfs.c                    |   4 +-
 fs/ocfs2/dlmglue.c                        |  29 +++---
 fs/ocfs2/file.c                           |  30 +++---
 fs/ocfs2/inode.c                          |  28 ++---
 fs/ocfs2/move_extents.c                   |   4 +-
 fs/ocfs2/namei.c                          |  16 +--
 fs/ocfs2/refcounttree.c                   |  12 +--
 fs/ocfs2/xattr.c                          |   4 +-
 fs/omfs/inode.c                           |  12 +--
 fs/openpromfs/inode.c                     |   4 +-
 fs/orangefs/orangefs-utils.c              |  16 +--
 fs/overlayfs/file.c                       |   9 +-
 fs/overlayfs/inode.c                      |   3 +-
 fs/overlayfs/util.c                       |   4 +-
 fs/pipe.c                                 |   2 +-
 fs/proc/base.c                            |   2 +-
 fs/proc/inode.c                           |   2 +-
 fs/proc/proc_sysctl.c                     |   2 +-
 fs/proc/self.c                            |   2 +-
 fs/proc/thread_self.c                     |   2 +-
 fs/pstore/inode.c                         |   5 +-
 fs/qnx4/inode.c                           |   6 +-
 fs/qnx6/inode.c                           |   6 +-
 fs/ramfs/inode.c                          |   7 +-
 fs/reiserfs/inode.c                       |  26 ++---
 fs/reiserfs/namei.c                       |   8 +-
 fs/reiserfs/stree.c                       |   5 +-
 fs/reiserfs/super.c                       |   2 +-
 fs/romfs/super.c                          |   3 +-
 fs/smb/client/file.c                      |  18 ++--
 fs/smb/client/fscache.h                   |   6 +-
 fs/smb/client/inode.c                     |  17 ++-
 fs/smb/client/smb2ops.c                   |   6 +-
 fs/smb/server/smb2pdu.c                   |   8 +-
 fs/squashfs/inode.c                       |   6 +-
 fs/stack.c                                |   4 +-
 fs/stat.c                                 |   4 +-
 fs/sysv/dir.c                             |   6 +-
 fs/sysv/ialloc.c                          |   2 +-
 fs/sysv/inode.c                           |  12 +--
 fs/sysv/itree.c                           |   2 +-
 fs/tracefs/inode.c                        |   2 +-
 fs/ubifs/debug.c                          |  12 +--
 fs/ubifs/dir.c                            |  23 +++--
 fs/ubifs/file.c                           |  16 +--
 fs/ubifs/journal.c                        |  12 +--
 fs/ubifs/super.c                          |   8 +-
 fs/udf/ialloc.c                           |   4 +-
 fs/udf/inode.c                            |  38 ++++---
 fs/udf/namei.c                            |  16 +--
 fs/ufs/dir.c                              |   6 +-
 fs/ufs/ialloc.c                           |   2 +-
 fs/ufs/inode.c                            |  42 ++++----
 fs/vboxsf/utils.c                         |  15 +--
 fs/xfs/libxfs/xfs_inode_buf.c             |  10 +-
 fs/xfs/libxfs/xfs_rtbitmap.c              |   6 +-
 fs/xfs/libxfs/xfs_trans_inode.c           |   2 +-
 fs/xfs/xfs_bmap_util.c                    |   7 +-
 fs/xfs/xfs_inode.c                        |   4 +-
 fs/xfs/xfs_inode_item.c                   |   4 +-
 fs/xfs/xfs_iops.c                         |   8 +-
 fs/xfs/xfs_itable.c                       |  12 +--
 fs/xfs/xfs_rtalloc.c                      |  30 +++---
 fs/zonefs/super.c                         |  10 +-
 include/linux/fs.h                        | 120 +++++++++++++++++-----
 include/linux/fs_stack.h                  |   6 +-
 ipc/mqueue.c                              |  19 ++--
 kernel/bpf/inode.c                        |   5 +-
 mm/shmem.c                                |  20 ++--
 net/sunrpc/rpc_pipe.c                     |   2 +-
 security/apparmor/apparmorfs.c            |   7 +-
 security/apparmor/policy_unpack.c         |   4 +-
 security/inode.c                          |   2 +-
 security/selinux/selinuxfs.c              |   2 +-
 216 files changed, 1220 insertions(+), 1000 deletions(-)

-- 
2.41.0



From jlayton at kernel.org  Wed Oct  4 11:52:37 2023
From: jlayton at kernel.org (Jeff Layton)
Date: Wed,  4 Oct 2023 14:52:37 -0400
Subject: [PATCH v2 01/89] fs: new accessor methods for atime and mtime
In-Reply-To: <20231004185221.80802-1-jlayton@kernel.org>
References: <20231004185221.80802-1-jlayton@kernel.org>
Message-ID: <20231004185239.80830-1-jlayton@kernel.org>

Recently, we converted the ctime accesses in the kernel to use new
accessor functions. Linus recently pointed out though that if we add
accessors for the atime and mtime, then that would allow us to
seamlessly change how these timestamps are stored in the inode.

Add new accessor functions for the atime and mtime that mirror the
accessors for the ctime.

Signed-off-by: Jeff Layton <jlayton at kernel.org>
---
 fs/libfs.c         | 41 ++++++++++++++++------
 include/linux/fs.h | 85 +++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 102 insertions(+), 24 deletions(-)

diff --git a/fs/libfs.c b/fs/libfs.c
index 37f2d34ee090..abe2b5a40ba1 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -541,7 +541,8 @@ void simple_recursive_removal(struct dentry *dentry,
 				dput(victim);		// unpin it
 			}
 			if (victim == dentry) {
-				inode->i_mtime = inode_set_ctime_current(inode);
+				inode_set_mtime_to_ts(inode,
+						      inode_set_ctime_current(inode));
 				if (d_is_dir(dentry))
 					drop_nlink(inode);
 				inode_unlock(inode);
@@ -582,7 +583,7 @@ static int pseudo_fs_fill_super(struct super_block *s, struct fs_context *fc)
 	 */
 	root->i_ino = 1;
 	root->i_mode = S_IFDIR | S_IRUSR | S_IWUSR;
-	root->i_atime = root->i_mtime = inode_set_ctime_current(root);
+	simple_inode_init_ts(root);
 	s->s_root = d_make_root(root);
 	if (!s->s_root)
 		return -ENOMEM;
@@ -638,8 +639,8 @@ int simple_link(struct dentry *old_dentry, struct inode *dir, struct dentry *den
 {
 	struct inode *inode = d_inode(old_dentry);
 
-	dir->i_mtime = inode_set_ctime_to_ts(dir,
-					     inode_set_ctime_current(inode));
+	inode_set_mtime_to_ts(dir,
+			      inode_set_ctime_to_ts(dir, inode_set_ctime_current(inode)));
 	inc_nlink(inode);
 	ihold(inode);
 	dget(dentry);
@@ -673,8 +674,8 @@ int simple_unlink(struct inode *dir, struct dentry *dentry)
 {
 	struct inode *inode = d_inode(dentry);
 
-	dir->i_mtime = inode_set_ctime_to_ts(dir,
-					     inode_set_ctime_current(inode));
+	inode_set_mtime_to_ts(dir,
+			      inode_set_ctime_to_ts(dir, inode_set_ctime_current(inode)));
 	drop_nlink(inode);
 	dput(dentry);
 	return 0;
@@ -709,9 +710,10 @@ void simple_rename_timestamp(struct inode *old_dir, struct dentry *old_dentry,
 {
 	struct inode *newino = d_inode(new_dentry);
 
-	old_dir->i_mtime = inode_set_ctime_current(old_dir);
+	inode_set_mtime_to_ts(old_dir, inode_set_ctime_current(old_dir));
 	if (new_dir != old_dir)
-		new_dir->i_mtime = inode_set_ctime_current(new_dir);
+		inode_set_mtime_to_ts(new_dir,
+				      inode_set_ctime_current(new_dir));
 	inode_set_ctime_current(d_inode(old_dentry));
 	if (newino)
 		inode_set_ctime_current(newino);
@@ -926,7 +928,7 @@ int simple_fill_super(struct super_block *s, unsigned long magic,
 	 */
 	inode->i_ino = 1;
 	inode->i_mode = S_IFDIR | 0755;
-	inode->i_atime = inode->i_mtime = inode_set_ctime_current(inode);
+	simple_inode_init_ts(inode);
 	inode->i_op = &simple_dir_inode_operations;
 	inode->i_fop = &simple_dir_operations;
 	set_nlink(inode, 2);
@@ -952,7 +954,7 @@ int simple_fill_super(struct super_block *s, unsigned long magic,
 			goto out;
 		}
 		inode->i_mode = S_IFREG | files->mode;
-		inode->i_atime = inode->i_mtime = inode_set_ctime_current(inode);
+		simple_inode_init_ts(inode);
 		inode->i_fop = files->ops;
 		inode->i_ino = i;
 		d_add(dentry, inode);
@@ -1520,7 +1522,7 @@ struct inode *alloc_anon_inode(struct super_block *s)
 	inode->i_uid = current_fsuid();
 	inode->i_gid = current_fsgid();
 	inode->i_flags |= S_PRIVATE;
-	inode->i_atime = inode->i_mtime = inode_set_ctime_current(inode);
+	simple_inode_init_ts(inode);
 	return inode;
 }
 EXPORT_SYMBOL(alloc_anon_inode);
@@ -1912,3 +1914,20 @@ ssize_t direct_write_fallback(struct kiocb *iocb, struct iov_iter *iter,
 	return direct_written + buffered_written;
 }
 EXPORT_SYMBOL_GPL(direct_write_fallback);
+
+/**
+ * simple_inode_init_ts - initialize the timestamps for a new inode
+ * @inode: inode to be initialized
+ *
+ * When a new inode is created, most filesystems set the timestamps to the
+ * current time. Add a helper to do this.
+ */
+struct timespec64 simple_inode_init_ts(struct inode *inode)
+{
+	struct timespec64 ts = inode_set_ctime_current(inode);
+
+	inode_set_atime_to_ts(inode, ts);
+	inode_set_mtime_to_ts(inode, ts);
+	return ts;
+}
+EXPORT_SYMBOL(simple_inode_init_ts);
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 7b8c6a9d52ec..3ca610d42176 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1515,24 +1515,81 @@ static inline bool fsuidgid_has_mapping(struct super_block *sb,
 struct timespec64 current_time(struct inode *inode);
 struct timespec64 inode_set_ctime_current(struct inode *inode);
 
-/**
- * inode_get_ctime - fetch the current ctime from the inode
- * @inode: inode from which to fetch ctime
- *
- * Grab the current ctime from the inode and return it.
- */
+static inline time64_t inode_get_atime_sec(const struct inode *inode)
+{
+	return inode->i_atime.tv_sec;
+}
+
+static inline long inode_get_atime_nsec(const struct inode *inode)
+{
+	return inode->i_atime.tv_nsec;
+}
+
+static inline struct timespec64 inode_get_atime(const struct inode *inode)
+{
+	return inode->i_atime;
+}
+
+static inline struct timespec64 inode_set_atime_to_ts(struct inode *inode,
+						      struct timespec64 ts)
+{
+	inode->i_atime = ts;
+	return ts;
+}
+
+static inline struct timespec64 inode_set_atime(struct inode *inode,
+						time64_t sec, long nsec)
+{
+	struct timespec64 ts = { .tv_sec  = sec,
+				 .tv_nsec = nsec };
+	return inode_set_atime_to_ts(inode, ts);
+}
+
+static inline time64_t inode_get_mtime_sec(const struct inode *inode)
+{
+	return inode->i_mtime.tv_sec;
+}
+
+static inline long inode_get_mtime_nsec(const struct inode *inode)
+{
+	return inode->i_mtime.tv_nsec;
+}
+
+static inline struct timespec64 inode_get_mtime(const struct inode *inode)
+{
+	return inode->i_mtime;
+}
+
+static inline struct timespec64 inode_set_mtime_to_ts(struct inode *inode,
+						      struct timespec64 ts)
+{
+	inode->i_mtime = ts;
+	return ts;
+}
+
+static inline struct timespec64 inode_set_mtime(struct inode *inode,
+						time64_t sec, long nsec)
+{
+	struct timespec64 ts = { .tv_sec  = sec,
+				 .tv_nsec = nsec };
+	return inode_set_mtime_to_ts(inode, ts);
+}
+
+static inline time64_t inode_get_ctime_sec(const struct inode *inode)
+{
+	return inode->__i_ctime.tv_sec;
+}
+
+static inline long inode_get_ctime_nsec(const struct inode *inode)
+{
+	return inode->__i_ctime.tv_nsec;
+}
+
 static inline struct timespec64 inode_get_ctime(const struct inode *inode)
 {
 	return inode->__i_ctime;
 }
 
-/**
- * inode_set_ctime_to_ts - set the ctime in the inode
- * @inode: inode in which to set the ctime
- * @ts: value to set in the ctime field
- *
- * Set the ctime in @inode to @ts
- */
 static inline struct timespec64 inode_set_ctime_to_ts(struct inode *inode,
 						      struct timespec64 ts)
 {
@@ -1557,6 +1614,8 @@ static inline struct timespec64 inode_set_ctime(struct inode *inode,
 	return inode_set_ctime_to_ts(inode, ts);
 }
 
+struct timespec64 simple_inode_init_ts(struct inode *inode);
+
 /*
  * Snapshotting support.
  */
-- 
2.41.0



From jlayton at kernel.org  Wed Oct  4 11:52:38 2023
From: jlayton at kernel.org (Jeff Layton)
Date: Wed,  4 Oct 2023 14:52:38 -0400
Subject: [PATCH v2 02/89] fs: convert core infrastructure to new timestamp accessors
In-Reply-To: <20231004185239.80830-1-jlayton@kernel.org>
References: <20231004185221.80802-1-jlayton@kernel.org>
 <20231004185239.80830-1-jlayton@kernel.org>
Message-ID: <20231004185239.80830-2-jlayton@kernel.org>

Convert the core vfs code to use the new timestamp accessor functions.

Signed-off-by: Jeff Layton <jlayton at kernel.org>
---
 fs/attr.c        |  4 ++--
 fs/bad_inode.c   |  2 +-
 fs/binfmt_misc.c |  2 +-
 fs/inode.c       | 35 +++++++++++++++++++++--------------
 fs/nsfs.c        |  2 +-
 fs/pipe.c        |  2 +-
 fs/stack.c       |  4 ++--
 fs/stat.c        |  4 ++--
 8 files changed, 31 insertions(+), 24 deletions(-)

diff --git a/fs/attr.c b/fs/attr.c
index a8ae5f6d9b16..bdf5deb06ea9 100644
--- a/fs/attr.c
+++ b/fs/attr.c
@@ -308,9 +308,9 @@ void setattr_copy(struct mnt_idmap *idmap, struct inode *inode,
 	i_uid_update(idmap, attr, inode);
 	i_gid_update(idmap, attr, inode);
 	if (ia_valid & ATTR_ATIME)
-		inode->i_atime = attr->ia_atime;
+		inode_set_atime_to_ts(inode, attr->ia_atime);
 	if (ia_valid & ATTR_MTIME)
-		inode->i_mtime = attr->ia_mtime;
+		inode_set_mtime_to_ts(inode, attr->ia_mtime);
 	if (ia_valid & ATTR_CTIME)
 		inode_set_ctime_to_ts(inode, attr->ia_ctime);
 	if (ia_valid & ATTR_MODE) {
diff --git a/fs/bad_inode.c b/fs/bad_inode.c
index 83f9566c973b..316d88da2ce1 100644
--- a/fs/bad_inode.c
+++ b/fs/bad_inode.c
@@ -208,7 +208,7 @@ void make_bad_inode(struct inode *inode)
 	remove_inode_hash(inode);
 
 	inode->i_mode = S_IFREG;
-	inode->i_atime = inode->i_mtime = inode_set_ctime_current(inode);
+	simple_inode_init_ts(inode);
 	inode->i_op = &bad_inode_ops;	
 	inode->i_opflags &= ~IOP_XATTR;
 	inode->i_fop = &bad_file_ops;	
diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c
index e0108d17b085..5d2be9b0a0a5 100644
--- a/fs/binfmt_misc.c
+++ b/fs/binfmt_misc.c
@@ -547,7 +547,7 @@ static struct inode *bm_get_inode(struct super_block *sb, int mode)
 	if (inode) {
 		inode->i_ino = get_next_ino();
 		inode->i_mode = mode;
-		inode->i_atime = inode->i_mtime = inode_set_ctime_current(inode);
+		simple_inode_init_ts(inode);
 	}
 	return inode;
 }
diff --git a/fs/inode.c b/fs/inode.c
index 3bb6193f436c..4f8984b97df0 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -1837,27 +1837,29 @@ EXPORT_SYMBOL(bmap);
 static int relatime_need_update(struct vfsmount *mnt, struct inode *inode,
 			     struct timespec64 now)
 {
-	struct timespec64 ctime;
+	struct timespec64 atime, mtime, ctime;
 
 	if (!(mnt->mnt_flags & MNT_RELATIME))
 		return 1;
 	/*
 	 * Is mtime younger than or equal to atime? If yes, update atime:
 	 */
-	if (timespec64_compare(&inode->i_mtime, &inode->i_atime) >= 0)
+	atime = inode_get_atime(inode);
+	mtime = inode_get_mtime(inode);
+	if (timespec64_compare(&mtime, &atime) >= 0)
 		return 1;
 	/*
 	 * Is ctime younger than or equal to atime? If yes, update atime:
 	 */
 	ctime = inode_get_ctime(inode);
-	if (timespec64_compare(&ctime, &inode->i_atime) >= 0)
+	if (timespec64_compare(&ctime, &atime) >= 0)
 		return 1;
 
 	/*
 	 * Is the previous atime value older than a day? If yes,
 	 * update atime:
 	 */
-	if ((long)(now.tv_sec - inode->i_atime.tv_sec) >= 24*60*60)
+	if ((long)(now.tv_sec - atime.tv_sec) >= 24*60*60)
 		return 1;
 	/*
 	 * Good, we can skip the atime update:
@@ -1888,12 +1890,13 @@ int inode_update_timestamps(struct inode *inode, int flags)
 
 	if (flags & (S_MTIME|S_CTIME|S_VERSION)) {
 		struct timespec64 ctime = inode_get_ctime(inode);
+		struct timespec64 mtime = inode_get_mtime(inode);
 
 		now = inode_set_ctime_current(inode);
 		if (!timespec64_equal(&now, &ctime))
 			updated |= S_CTIME;
-		if (!timespec64_equal(&now, &inode->i_mtime)) {
-			inode->i_mtime = now;
+		if (!timespec64_equal(&now, &mtime)) {
+			inode_set_mtime_to_ts(inode, now);
 			updated |= S_MTIME;
 		}
 		if (IS_I_VERSION(inode) && inode_maybe_inc_iversion(inode, updated))
@@ -1903,8 +1906,10 @@ int inode_update_timestamps(struct inode *inode, int flags)
 	}
 
 	if (flags & S_ATIME) {
-		if (!timespec64_equal(&now, &inode->i_atime)) {
-			inode->i_atime = now;
+		struct timespec64 atime = inode_get_atime(inode);
+
+		if (!timespec64_equal(&now, &atime)) {
+			inode_set_atime_to_ts(inode, now);
 			updated |= S_ATIME;
 		}
 	}
@@ -1963,7 +1968,7 @@ EXPORT_SYMBOL(inode_update_time);
 bool atime_needs_update(const struct path *path, struct inode *inode)
 {
 	struct vfsmount *mnt = path->mnt;
-	struct timespec64 now;
+	struct timespec64 now, atime;
 
 	if (inode->i_flags & S_NOATIME)
 		return false;
@@ -1989,7 +1994,8 @@ bool atime_needs_update(const struct path *path, struct inode *inode)
 	if (!relatime_need_update(mnt, inode, now))
 		return false;
 
-	if (timespec64_equal(&inode->i_atime, &now))
+	atime = inode_get_atime(inode);
+	if (timespec64_equal(&atime, &now))
 		return false;
 
 	return true;
@@ -2106,17 +2112,18 @@ static int inode_needs_update_time(struct inode *inode)
 {
 	int sync_it = 0;
 	struct timespec64 now = current_time(inode);
-	struct timespec64 ctime;
+	struct timespec64 ts;
 
 	/* First try to exhaust all avenues to not sync */
 	if (IS_NOCMTIME(inode))
 		return 0;
 
-	if (!timespec64_equal(&inode->i_mtime, &now))
+	ts = inode_get_mtime(inode);
+	if (!timespec64_equal(&ts, &now))
 		sync_it = S_MTIME;
 
-	ctime = inode_get_ctime(inode);
-	if (!timespec64_equal(&ctime, &now))
+	ts = inode_get_ctime(inode);
+	if (!timespec64_equal(&ts, &now))
 		sync_it |= S_CTIME;
 
 	if (IS_I_VERSION(inode) && inode_iversion_need_inc(inode))
diff --git a/fs/nsfs.c b/fs/nsfs.c
index 647a22433bd8..9a4b228d42fa 100644
--- a/fs/nsfs.c
+++ b/fs/nsfs.c
@@ -84,7 +84,7 @@ static int __ns_get_path(struct path *path, struct ns_common *ns)
 		return -ENOMEM;
 	}
 	inode->i_ino = ns->inum;
-	inode->i_mtime = inode->i_atime = inode_set_ctime_current(inode);
+	simple_inode_init_ts(inode);
 	inode->i_flags |= S_IMMUTABLE;
 	inode->i_mode = S_IFREG | S_IRUGO;
 	inode->i_fop = &ns_file_operations;
diff --git a/fs/pipe.c b/fs/pipe.c
index 485e3be8903c..8916c455a469 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -908,7 +908,7 @@ static struct inode * get_pipe_inode(void)
 	inode->i_mode = S_IFIFO | S_IRUSR | S_IWUSR;
 	inode->i_uid = current_fsuid();
 	inode->i_gid = current_fsgid();
-	inode->i_atime = inode->i_mtime = inode_set_ctime_current(inode);
+	simple_inode_init_ts(inode);
 
 	return inode;
 
diff --git a/fs/stack.c b/fs/stack.c
index b5e01bdb5f5f..f18920119944 100644
--- a/fs/stack.c
+++ b/fs/stack.c
@@ -66,8 +66,8 @@ void fsstack_copy_attr_all(struct inode *dest, const struct inode *src)
 	dest->i_uid = src->i_uid;
 	dest->i_gid = src->i_gid;
 	dest->i_rdev = src->i_rdev;
-	dest->i_atime = src->i_atime;
-	dest->i_mtime = src->i_mtime;
+	inode_set_atime_to_ts(dest, inode_get_atime(src));
+	inode_set_mtime_to_ts(dest, inode_get_mtime(src));
 	inode_set_ctime_to_ts(dest, inode_get_ctime(src));
 	dest->i_blkbits = src->i_blkbits;
 	dest->i_flags = src->i_flags;
diff --git a/fs/stat.c b/fs/stat.c
index d43a5cc1bfa4..24bb0209e459 100644
--- a/fs/stat.c
+++ b/fs/stat.c
@@ -57,8 +57,8 @@ void generic_fillattr(struct mnt_idmap *idmap, u32 request_mask,
 	stat->gid = vfsgid_into_kgid(vfsgid);
 	stat->rdev = inode->i_rdev;
 	stat->size = i_size_read(inode);
-	stat->atime = inode->i_atime;
-	stat->mtime = inode->i_mtime;
+	stat->atime = inode_get_atime(inode);
+	stat->mtime = inode_get_mtime(inode);
 	stat->ctime = inode_get_ctime(inode);
 	stat->blksize = i_blocksize(inode);
 	stat->blocks = inode->i_blocks;
-- 
2.41.0



From jlayton at kernel.org  Wed Oct  4 11:55:28 2023
From: jlayton at kernel.org (Jeff Layton)
Date: Wed,  4 Oct 2023 14:55:28 -0400
Subject: [PATCH v2 87/89] fs: rename inode i_atime and i_mtime fields
Message-ID: <20231004185530.82088-1-jlayton@kernel.org>

Rename these two fields to discourage direct access (and to help ensure
that we mop up any leftover direct accesses).

Signed-off-by: Jeff Layton <jlayton at kernel.org>
---
 include/linux/fs.h | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 3ca610d42176..84fdaf399fbe 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -671,8 +671,8 @@ struct inode {
 	};
 	dev_t			i_rdev;
 	loff_t			i_size;
-	struct timespec64	i_atime;
-	struct timespec64	i_mtime;
+	struct timespec64	__i_atime;
+	struct timespec64	__i_mtime;
 	struct timespec64	__i_ctime; /* use inode_*_ctime accessors! */
 	spinlock_t		i_lock;	/* i_blocks, i_bytes, maybe i_size */
 	unsigned short          i_bytes;
@@ -1517,23 +1517,23 @@ struct timespec64 inode_set_ctime_current(struct inode *inode);
 
 static inline time64_t inode_get_atime_sec(const struct inode *inode)
 {
-	return inode->i_atime.tv_sec;
+	return inode->__i_atime.tv_sec;
 }
 
 static inline long inode_get_atime_nsec(const struct inode *inode)
 {
-	return inode->i_atime.tv_nsec;
+	return inode->__i_atime.tv_nsec;
 }
 
 static inline struct timespec64 inode_get_atime(const struct inode *inode)
 {
-	return inode->i_atime;
+	return inode->__i_atime;
 }
 
 static inline struct timespec64 inode_set_atime_to_ts(struct inode *inode,
 						      struct timespec64 ts)
 {
-	inode->i_atime = ts;
+	inode->__i_atime = ts;
 	return ts;
 }
 
@@ -1547,23 +1547,23 @@ static inline struct timespec64 inode_set_atime(struct inode *inode,
 
 static inline time64_t inode_get_mtime_sec(const struct inode *inode)
 {
-	return inode->i_mtime.tv_sec;
+	return inode->__i_mtime.tv_sec;
 }
 
 static inline long inode_get_mtime_nsec(const struct inode *inode)
 {
-	return inode->i_mtime.tv_nsec;
+	return inode->__i_mtime.tv_nsec;
 }
 
 static inline struct timespec64 inode_get_mtime(const struct inode *inode)
 {
-	return inode->i_mtime;
+	return inode->__i_mtime;
 }
 
 static inline struct timespec64 inode_set_mtime_to_ts(struct inode *inode,
 						      struct timespec64 ts)
 {
-	inode->i_mtime = ts;
+	inode->__i_mtime = ts;
 	return ts;
 }
 
-- 
2.41.0



From jlayton at kernel.org  Wed Oct  4 11:55:29 2023
From: jlayton at kernel.org (Jeff Layton)
Date: Wed,  4 Oct 2023 14:55:29 -0400
Subject: [PATCH v2 88/89] fs: switch timespec64 fields in inode to discrete integers
In-Reply-To: <20231004185530.82088-1-jlayton@kernel.org>
References: <20231004185530.82088-1-jlayton@kernel.org>
Message-ID: <20231004185530.82088-2-jlayton@kernel.org>

This shaves 8 bytes off struct inode with a garden-variety Fedora
Kconfig.

Signed-off-by: Jeff Layton <jlayton at kernel.org>
---
 include/linux/fs.h | 53 ++++++++++++++++++++++++++--------------------
 1 file changed, 30 insertions(+), 23 deletions(-)

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 84fdaf399fbe..485b5e21c8e5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -671,9 +671,12 @@ struct inode {
 	};
 	dev_t			i_rdev;
 	loff_t			i_size;
-	struct timespec64	__i_atime;
-	struct timespec64	__i_mtime;
-	struct timespec64	__i_ctime; /* use inode_*_ctime accessors! */
+	time64_t		i_atime_sec;
+	time64_t		i_mtime_sec;
+	time64_t		i_ctime_sec;
+	u32			i_atime_nsec;
+	u32			i_mtime_nsec;
+	u32			i_ctime_nsec;
 	spinlock_t		i_lock;	/* i_blocks, i_bytes, maybe i_size */
 	unsigned short          i_bytes;
 	u8			i_blkbits;
@@ -1517,23 +1520,27 @@ struct timespec64 inode_set_ctime_current(struct inode *inode);
 
 static inline time64_t inode_get_atime_sec(const struct inode *inode)
 {
-	return inode->__i_atime.tv_sec;
+	return inode->i_atime_sec;
 }
 
 static inline long inode_get_atime_nsec(const struct inode *inode)
 {
-	return inode->__i_atime.tv_nsec;
+	return inode->i_atime_nsec;
 }
 
 static inline struct timespec64 inode_get_atime(const struct inode *inode)
 {
-	return inode->__i_atime;
+	struct timespec64 ts = { .tv_sec  = inode_get_atime_sec(inode),
+				 .tv_nsec = inode_get_atime_nsec(inode) };
+
+	return ts;
 }
 
 static inline struct timespec64 inode_set_atime_to_ts(struct inode *inode,
 						      struct timespec64 ts)
 {
-	inode->__i_atime = ts;
+	inode->i_atime_sec = ts.tv_sec;
+	inode->i_atime_nsec = ts.tv_nsec;
 	return ts;
 }
 
@@ -1542,28 +1549,32 @@ static inline struct timespec64 inode_set_atime(struct inode *inode,
 {
 	struct timespec64 ts = { .tv_sec  = sec,
 				 .tv_nsec = nsec };
+
 	return inode_set_atime_to_ts(inode, ts);
 }
 
 static inline time64_t inode_get_mtime_sec(const struct inode *inode)
 {
-	return inode->__i_mtime.tv_sec;
+	return inode->i_mtime_sec;
 }
 
 static inline long inode_get_mtime_nsec(const struct inode *inode)
 {
-	return inode->__i_mtime.tv_nsec;
+	return inode->i_mtime_nsec;
 }
 
 static inline struct timespec64 inode_get_mtime(const struct inode *inode)
 {
-	return inode->__i_mtime;
+	struct timespec64 ts = { .tv_sec  = inode_get_mtime_sec(inode),
+				 .tv_nsec = inode_get_mtime_nsec(inode) };
+	return ts;
 }
 
 static inline struct timespec64 inode_set_mtime_to_ts(struct inode *inode,
 						      struct timespec64 ts)
 {
-	inode->__i_mtime = ts;
+	inode->i_mtime_sec = ts.tv_sec;
+	inode->i_mtime_nsec = ts.tv_nsec;
 	return ts;
 }
 
@@ -1577,34 +1588,30 @@ static inline struct timespec64 inode_set_mtime(struct inode *inode,
 
 static inline time64_t inode_get_ctime_sec(const struct inode *inode)
 {
-	return inode->__i_ctime.tv_sec;
+	return inode->i_ctime_sec;
 }
 
 static inline long inode_get_ctime_nsec(const struct inode *inode)
 {
-	return inode->__i_ctime.tv_nsec;
+	return inode->i_ctime_nsec;
 }
 
 static inline struct timespec64 inode_get_ctime(const struct inode *inode)
 {
-	return inode->__i_ctime;
+	struct timespec64 ts = { .tv_sec  = inode_get_ctime_sec(inode),
+				 .tv_nsec = inode_get_ctime_nsec(inode) };
+
+	return ts;
 }
 
 static inline struct timespec64 inode_set_ctime_to_ts(struct inode *inode,
 						      struct timespec64 ts)
 {
-	inode->__i_ctime = ts;
+	inode->i_ctime_sec = ts.tv_sec;
+	inode->i_ctime_nsec = ts.tv_nsec;
 	return ts;
 }
 
-/**
- * inode_set_ctime - set the ctime in the inode
- * @inode: inode in which to set the ctime
- * @sec: tv_sec value to set
- * @nsec: tv_nsec value to set
- *
- * Set the ctime in @inode to { @sec, @nsec }
- */
 static inline struct timespec64 inode_set_ctime(struct inode *inode,
 						time64_t sec, long nsec)
 {
-- 
2.41.0



From jlayton at kernel.org  Wed Oct  4 11:55:30 2023
From: jlayton at kernel.org (Jeff Layton)
Date: Wed,  4 Oct 2023 14:55:30 -0400
Subject: [PATCH v2 89/89] fs: move i_generation into new hole created after timestamp conversion
In-Reply-To: <20231004185530.82088-1-jlayton@kernel.org>
References: <20231004185530.82088-1-jlayton@kernel.org>
Message-ID: <20231004185530.82088-3-jlayton@kernel.org>

The recent change to use discrete integers instead of struct timespec64
shaved 8 bytes off of struct inode, but it also moves the i_lock
into the previous cacheline, away from the fields that it protects.

Move i_generation above the i_lock, which moves the new 4 byte hole to
just after the i_fsnotify_mask in my setup.

Suggested-by: Amir Goldstein <amir73il at gmail.com>
Signed-off-by: Jeff Layton <jlayton at kernel.org>
---
 include/linux/fs.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 485b5e21c8e5..686c9f33e725 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -677,6 +677,7 @@ struct inode {
 	u32			i_atime_nsec;
 	u32			i_mtime_nsec;
 	u32			i_ctime_nsec;
+	u32			i_generation;
 	spinlock_t		i_lock;	/* i_blocks, i_bytes, maybe i_size */
 	unsigned short          i_bytes;
 	u8			i_blkbits;
@@ -733,7 +734,6 @@ struct inode {
 		unsigned		i_dir_seq;
 	};
 
-	__u32			i_generation;
 
 #ifdef CONFIG_FSNOTIFY
 	__u32			i_fsnotify_mask; /* all events this inode cares about */
-- 
2.41.0



From make_ruc2021 at 163.com  Fri Oct  6 05:27:17 2023
From: make_ruc2021 at 163.com (Ma Ke)
Date: Fri,  6 Oct 2023 20:27:17 +0800
Subject: [PATCH] um: vector: fix return value check in vector_legacy_rx
Message-ID: <20231006122717.3984017-1-make_ruc2021@163.com>

In vector_legacy_rx, to avoid an unexpected result returned by
pskb_trim, we should check the return value of pskb_trim().

Signed-off-by: Ma Ke <make_ruc2021 at 163.com>
---
 arch/um/drivers/vector_kern.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/um/drivers/vector_kern.c b/arch/um/drivers/vector_kern.c
index 131b7cb29576..822a8c0cdcc1 100644
--- a/arch/um/drivers/vector_kern.c
+++ b/arch/um/drivers/vector_kern.c
@@ -890,7 +890,8 @@ static int vector_legacy_rx(struct vector_private *vp)
 					skb->ip_summed = CHECKSUM_UNNECESSARY;
 				}
 			}
-			pskb_trim(skb, pkt_len - vp->rx_header_size);
+			if (pskb_trim(skb, pkt_len - vp->rx_header_size))
+				return 0;
 			skb->protocol = eth_type_trans(skb, skb->dev);
 			vp->dev->stats.rx_bytes += skb->len;
 			vp->dev->stats.rx_packets++;
-- 
2.37.2



From geert at linux-m68k.org  Fri Oct  6 07:51:07 2023
From: geert at linux-m68k.org (Geert Uytterhoeven)
Date: Fri, 6 Oct 2023 16:51:07 +0200
Subject: Arches that don't support PREEMPT
In-Reply-To: <ZQmbhoQIINs8rLHp@casper.infradead.org>
References: <87edj64rj1.fsf@oracle.com> <CAHk-=wi0bXpgULVVLc2AdJcta-fvQP7yyFQ_JtaoHUiPrqf--A@mail.gmail.com>
 <87zg1u1h5t.fsf@oracle.com> <CAHk-=whMkp68vNxVn1H3qe_P7n=X2sWPL9kvW22dsvMFH8FcQQ@mail.gmail.com>
 <20230911150410.GC9098@noisy.programming.kicks-ass.net> <87h6o01w1a.fsf@oracle.com>
 <20230912082606.GB35261@noisy.programming.kicks-ass.net> <87cyyfxd4k.ffs@tglx>
 <CAHk-=whnwC01m_1f-gaM1xbvvwzwTiKitrWniA-ChZv+bM03dg@mail.gmail.com>
 <87led2wdj0.ffs@tglx> <ZQmbhoQIINs8rLHp@casper.infradead.org>
Message-ID: <CAMuHMdV4qx3iQzqksfnvckk_PK9Zas0U=p-xv3pVwJ6hWLpejA@mail.gmail.com>

Hi Willy,

On Tue, Sep 19, 2023 at 3:01?PM Matthew Wilcox <willy at infradead.org> wrote:
> On Tue, Sep 19, 2023 at 02:30:59PM +0200, Thomas Gleixner wrote:
> > Though it just occured to me that there are dragons lurking:
> >
> > arch/alpha/Kconfig:     select ARCH_NO_PREEMPT
> > arch/hexagon/Kconfig:   select ARCH_NO_PREEMPT
> > arch/m68k/Kconfig:      select ARCH_NO_PREEMPT if !COLDFIRE
> > arch/um/Kconfig:        select ARCH_NO_PREEMPT
>
> Sounds like three-and-a-half architectures which could be queued up for
> removal right behind ia64 ...
>
> I suspect none of these architecture maintainers have any idea there's a
> problem.  Look at commit 87a4c375995e and the discussion in
> https://lore.kernel.org/lkml/20180724175646.3621-1-hch at lst.de/
>
> Let's cc those maintainers so they can remove this and fix whatever
> breaks.

Looks like your scare tactics are working ;-)
[PATCH/RFC] m68k: Add full preempt support
https://lore.kernel.org/all/7858a184cda66e0991fd295c711dfed7e4d1248c.1696603287.git.geert at linux-m68k.org

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds


From make_ruc2021 at 163.com  Fri Oct  6 17:51:04 2023
From: make_ruc2021 at 163.com (Ma Ke)
Date: Sat,  7 Oct 2023 08:51:04 +0800
Subject: [PATCH] um: vector: fix return value check in vector_mmsg_rx
Message-ID: <20231007005104.3994678-1-make_ruc2021@163.com>

In vector_mmsg_rx, to avoid an unexpected result returned by
pskb_trim, we should check the return value of pskb_trim().

Signed-off-by: Ma Ke <make_ruc2021 at 163.com>
---
 arch/um/drivers/vector_kern.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/um/drivers/vector_kern.c b/arch/um/drivers/vector_kern.c
index 131b7cb29576..c280ce5ea6ce 100644
--- a/arch/um/drivers/vector_kern.c
+++ b/arch/um/drivers/vector_kern.c
@@ -1013,8 +1013,8 @@ static int vector_mmsg_rx(struct vector_private *vp, int budget)
 					skb->ip_summed = CHECKSUM_UNNECESSARY;
 				}
 			}
-			pskb_trim(skb,
-				mmsg_vector->msg_len - vp->rx_header_size);
+			if (pskb_trim(skb, mmsg_vector->msg_len - vp->rx_header_size))
+				return 0;
 			skb->protocol = eth_type_trans(skb, skb->dev);
 			/*
 			 * We do not need to lock on updating stats here
-- 
2.37.2



From anton.ivanov at cambridgegreys.com  Fri Oct  6 23:12:50 2023
From: anton.ivanov at cambridgegreys.com (Anton Ivanov)
Date: Sat, 7 Oct 2023 07:12:50 +0100
Subject: [PATCH] um: vector: fix return value check in vector_mmsg_rx
In-Reply-To: <20231007005104.3994678-1-make_ruc2021@163.com>
References: <20231007005104.3994678-1-make_ruc2021@163.com>
Message-ID: <c6c0dea2-bbf6-80a6-1888-c1fa4718d4c2@cambridgegreys.com>

On 07/10/2023 01:51, Ma Ke wrote:
> In vector_mmsg_rx, to avoid an unexpected result returned by
> pskb_trim, we should check the return value of pskb_trim().
> 
> Signed-off-by: Ma Ke <make_ruc2021 at 163.com>
> ---
>   arch/um/drivers/vector_kern.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/um/drivers/vector_kern.c b/arch/um/drivers/vector_kern.c
> index 131b7cb29576..c280ce5ea6ce 100644
> --- a/arch/um/drivers/vector_kern.c
> +++ b/arch/um/drivers/vector_kern.c
> @@ -1013,8 +1013,8 @@ static int vector_mmsg_rx(struct vector_private *vp, int budget)
>   					skb->ip_summed = CHECKSUM_UNNECESSARY;
>   				}
>   			}
> -			pskb_trim(skb,
> -				mmsg_vector->msg_len - vp->rx_header_size);
> +			if (pskb_trim(skb, mmsg_vector->msg_len - vp->rx_header_size))
> +				return 0;
>   			skb->protocol = eth_type_trans(skb, skb->dev);
>   			/*
>   			 * We do not need to lock on updating stats here

That does not look right. You can have errors when processing an 
individual packet. That is not a reason to bail and you should process 
the next ones. This way you just dropped the rest of the RX vector (up 
to 64 packets at default settings).

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/



From johannes at sipsolutions.net  Mon Oct  9 00:31:53 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Mon, 09 Oct 2023 09:31:53 +0200
Subject: [PATCH] um: vector: fix return value check in vector_mmsg_rx
In-Reply-To: <20231007005104.3994678-1-make_ruc2021@163.com>
References: <20231007005104.3994678-1-make_ruc2021@163.com>
Message-ID: <ff46c71a7eb6c43c9e06e99d30bdcb1f8152ee7e.camel@sipsolutions.net>

On Sat, 2023-10-07 at 08:51 +0800, Ma Ke wrote:
> In vector_mmsg_rx, to avoid an unexpected result returned by
> pskb_trim, we should check the return value of pskb_trim().

And how exactly do you propose pskb_trim() will return anything but 0 in
this scenario?

Please. If you have a static checker, _think_ about it's output before
blindly sending patches.

johannes



From masahiroy at kernel.org  Mon Oct  9 05:42:06 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Mon,  9 Oct 2023 21:42:06 +0900
Subject: [PATCH 1/5] csky: remove unused cmd_vdso_install
Message-ID: <20231009124210.1064021-1-masahiroy@kernel.org>

You cannot run this code because arch/csky/Makefile does not define the
vdso_install target.

It appears that this code was blindly copied from another architecture.

Remove the dead code.

Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
---

 arch/csky/kernel/vdso/Makefile | 10 ----------
 1 file changed, 10 deletions(-)

diff --git a/arch/csky/kernel/vdso/Makefile b/arch/csky/kernel/vdso/Makefile
index 299e4e41ebc5..ddf784a62c11 100644
--- a/arch/csky/kernel/vdso/Makefile
+++ b/arch/csky/kernel/vdso/Makefile
@@ -58,13 +58,3 @@ quiet_cmd_vdsold = VDSOLD  $@
 # that contains the same symbols at the same offsets.
 quiet_cmd_so2s = SO2S    $@
       cmd_so2s = $(NM) -D $< | $(srctree)/$(src)/so2s.sh > $@
-
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
-- 
2.39.2



From masahiroy at kernel.org  Mon Oct  9 05:42:07 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Mon,  9 Oct 2023 21:42:07 +0900
Subject: [PATCH 2/5] UML: remove unused cmd_vdso_install
In-Reply-To: <20231009124210.1064021-1-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
Message-ID: <20231009124210.1064021-2-masahiroy@kernel.org>

You cannot run this code because arch/um/Makefile does not define the
vdso_install target.

It appears that this code was blindly copied from another architecture.

Remove the dead code.

Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
---

 arch/x86/um/vdso/Makefile | 12 ------------
 1 file changed, 12 deletions(-)

diff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile
index 6825e146a62f..b86d634730b2 100644
--- a/arch/x86/um/vdso/Makefile
+++ b/arch/x86/um/vdso/Makefile
@@ -67,15 +67,3 @@ quiet_cmd_vdso = VDSO    $@
 
 VDSO_LDFLAGS = -fPIC -shared -Wl,--hash-style=sysv -z noexecstack
 GCOV_PROFILE := n
-
-#
-# Install the unstripped copy of vdso*.so listed in $(vdso-install-y).
-#
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-$(vdso-install-y): %.so: $(obj)/%.so.dbg FORCE
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-PHONY += vdso_install $(vdso-install-y)
-vdso_install: $(vdso-install-y)
-- 
2.39.2



From masahiroy at kernel.org  Mon Oct  9 05:42:08 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Mon,  9 Oct 2023 21:42:08 +0900
Subject: [PATCH 3/5] parisc: remove broken vdso_install
In-Reply-To: <20231009124210.1064021-1-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
Message-ID: <20231009124210.1064021-3-masahiroy@kernel.org>

'make ARCH=parisc vdso_install' has never worked. It attempts to
descend into arch/parisc/kernel/vdso/, which does not exist.

The command just fails:

  scripts/Makefile.build:41: arch/parisc/kernel/vdso/Makefile: No such file or directory

The second line is also meaningless because parisc does not define
CONFIG_COMPAT_VDSO.

It appears that this code was copied from another architecture without
proper adaptation.

Remove the broken code.

Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
---

 arch/parisc/Makefile | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/arch/parisc/Makefile b/arch/parisc/Makefile
index 968ebe17494c..4222fa73c34a 100644
--- a/arch/parisc/Makefile
+++ b/arch/parisc/Makefile
@@ -177,13 +177,6 @@ vdso_prepare: prepare0
 	$(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso32 include/generated/vdso32-offsets.h
 endif
 
-PHONY += vdso_install
-
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso $@
-	$(if $(CONFIG_COMPAT_VDSO), \
-		$(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso32 $@)
-
 install: KBUILD_IMAGE := vmlinux
 zinstall: KBUILD_IMAGE := vmlinuz
 install zinstall:
-- 
2.39.2



From masahiroy at kernel.org  Mon Oct  9 05:42:09 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Mon,  9 Oct 2023 21:42:09 +0900
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <20231009124210.1064021-1-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
Message-ID: <20231009124210.1064021-4-masahiroy@kernel.org>

Currently, there is no standard implementation for vdso_install,
leading to various issues:

 1. Code duplication

    Many architectures duplicate similar code just for copying files
    to the install destination.

    Some architectures (arm, sparc, x86) create build-id symlinks,
    introducing more code duplication.

 2. Accidental updates of in-tree build artifacts

    The vdso_install rule depends on the vdso files to install.
    It may update in-tree build artifacts. This can be problematic,
    as explained in commit 19514fc665ff ("arm, kbuild: make
    "make install" not depend on vmlinux").

 3. Broken code in some architectures

    Makefile code is often copied from one architecture to another
    without proper adaptation or testing.

    The previous commits removed broken code from csky, UML, and parisc.

    Another issue is that 'make vdso_install' for ARCH=s390 installs
    vdso64, but not vdso32.

To address these problems, this commit introduces the generic vdso_install.

Architectures that support vdso_install need to define vdso-install-y
in arch/*/Makefile.

vdso-install-y lists the files to install. For example, arch/x86/Makefile
looks like this:

  vdso-install-$(CONFIG_X86_64)           += arch/x86/entry/vdso/vdso64.so.dbg
  vdso-install-$(CONFIG_X86_X32_ABI)      += arch/x86/entry/vdso/vdsox32.so.dbg
  vdso-install-$(CONFIG_X86_32)           += arch/x86/entry/vdso/vdso32.so.dbg
  vdso-install-$(CONFIG_IA32_EMULATION)   += arch/x86/entry/vdso/vdso32.so.dbg

These files will be installed to $(MODLIB)/vdso/ with the .dbg suffix,
if exists, stripped away.

vdso-install-y can optionally take the second field after the colon
separator. This is needed because some architectures install vdso
files as a different base name.

The following is a snippet from arch/arm64/Makefile.

  vdso-install-$(CONFIG_COMPAT_VDSO)      += arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so

This will rename vdso.so.dbg to vdso32.so during installation. If such
architectures change their implementation so that the file names match,
this workaround will go away.

Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
---

 Makefile                               |  9 ++++++
 arch/arm/Makefile                      |  7 +---
 arch/arm/vdso/Makefile                 | 25 --------------
 arch/arm64/Makefile                    |  9 ++----
 arch/arm64/kernel/vdso/Makefile        | 10 ------
 arch/arm64/kernel/vdso32/Makefile      | 10 ------
 arch/loongarch/Makefile                |  4 +--
 arch/loongarch/vdso/Makefile           | 10 ------
 arch/riscv/Makefile                    |  9 ++----
 arch/riscv/kernel/compat_vdso/Makefile | 10 ------
 arch/riscv/kernel/vdso/Makefile        | 10 ------
 arch/s390/Makefile                     |  6 ++--
 arch/s390/kernel/vdso32/Makefile       | 10 ------
 arch/s390/kernel/vdso64/Makefile       | 10 ------
 arch/sparc/Makefile                    |  5 ++-
 arch/sparc/vdso/Makefile               | 27 ----------------
 arch/x86/Makefile                      |  7 ++--
 arch/x86/entry/vdso/Makefile           | 27 ----------------
 scripts/Makefile.vdsoinst              | 45 ++++++++++++++++++++++++++
 19 files changed, 71 insertions(+), 179 deletions(-)
 create mode 100644 scripts/Makefile.vdsoinst

diff --git a/Makefile b/Makefile
index 373649c7374e..2170d56630e8 100644
--- a/Makefile
+++ b/Makefile
@@ -1317,6 +1317,14 @@ scripts_unifdef: scripts_basic
 quiet_cmd_install = INSTALL $(INSTALL_PATH)
       cmd_install = unset sub_make_done; $(srctree)/scripts/install.sh
 
+# ---------------------------------------------------------------------------
+# vDSO install
+
+PHONY += vdso_install
+vdso_install: export INSTALL_FILES = $(vdso-install-y)
+vdso_install:
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vdsoinst
+
 # ---------------------------------------------------------------------------
 # Tools
 
@@ -1560,6 +1568,7 @@ help:
 	@echo  '* vmlinux	  - Build the bare kernel'
 	@echo  '* modules	  - Build all modules'
 	@echo  '  modules_install - Install all modules to INSTALL_MOD_PATH (default: /)'
+	@echo  '  vdso_install    - Install unstripped vdso to INSTALL_MOD_PATH (default: /)'
 	@echo  '  dir/            - Build all files in dir and below'
 	@echo  '  dir/file.[ois]  - Build specified target only'
 	@echo  '  dir/file.ll     - Build the LLVM assembly file'
diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 547e5856eaa0..5ba42f69f8ce 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -304,11 +304,7 @@ $(INSTALL_TARGETS): KBUILD_IMAGE = $(boot)/$(patsubst %install,%Image,$@)
 $(INSTALL_TARGETS):
 	$(call cmd,install)
 
-PHONY += vdso_install
-vdso_install:
-ifeq ($(CONFIG_VDSO),y)
-	$(Q)$(MAKE) $(build)=arch/arm/vdso $@
-endif
+vdso-install-$(CONFIG_VDSO) += arch/arm/vdso/vdso.so.dbg
 
 # My testing targets (bypasses dependencies)
 bp:;	$(Q)$(MAKE) $(build)=$(boot) $(boot)/bootpImage
@@ -331,7 +327,6 @@ define archhelp
   echo  '                  Install using (your) ~/bin/$(INSTALLKERNEL) or'
   echo  '                  (distribution) /sbin/$(INSTALLKERNEL) or'
   echo  '                  install to $$(INSTALL_PATH) and run lilo'
-  echo  '  vdso_install  - Install unstripped vdso.so to $$(INSTALL_MOD_PATH)/vdso'
   echo
   echo  '  multi_v7_lpae_defconfig     - multi_v7_defconfig with CONFIG_ARM_LPAE enabled'
 endef
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index 515ca33b854c..d761bd2e2f40 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -63,28 +63,3 @@ quiet_cmd_vdsold_and_vdso_check = LD      $@
 
 quiet_cmd_vdsomunge = MUNGE   $@
       cmd_vdsomunge = $(objtree)/$(obj)/vdsomunge $< $@
-
-#
-# Install the unstripped copy of vdso.so.dbg.  If our toolchain
-# supports build-id, install .build-id links as well.
-#
-# Cribbed from arch/x86/vdso/Makefile.
-#
-quiet_cmd_vdso_install = INSTALL $<
-define cmd_vdso_install
-	cp $< "$(MODLIB)/vdso/vdso.so"; \
-	if readelf -n $< | grep -q 'Build ID'; then \
-	  buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
-	  first=`echo $$buildid | cut -b-2`; \
-	  last=`echo $$buildid | cut -b3-`; \
-	  mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
-	  ln -sf "../../vdso.so" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
-	fi
-endef
-
-$(MODLIB)/vdso: FORCE
-	@mkdir -p $(MODLIB)/vdso
-
-PHONY += vdso_install
-vdso_install: $(obj)/vdso.so.dbg $(MODLIB)/vdso
-	$(call cmd,vdso_install)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 2d49aea0ff67..4bd85cc0d32b 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -169,12 +169,6 @@ install: KBUILD_IMAGE := $(boot)/Image
 install zinstall:
 	$(call cmd,install)
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso $@
-	$(if $(CONFIG_COMPAT_VDSO), \
-		$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso32 $@)
-
 archprepare:
 	$(Q)$(MAKE) $(build)=arch/arm64/tools kapi
 ifeq ($(CONFIG_ARM64_ERRATUM_843419),y)
@@ -205,6 +199,9 @@ ifdef CONFIG_COMPAT_VDSO
 endif
 endif
 
+vdso-install-y				+= arch/arm64/kernel/vdso/vdso.so.dbg
+vdso-install-$(CONFIG_COMPAT_VDSO)	+= arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so
+
 include $(srctree)/scripts/Makefile.defconf
 
 PHONY += virtconfig
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index fe7a53c6781f..8818287f1095 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -78,13 +78,3 @@ include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
 # Actual build commands
 quiet_cmd_vdsold_and_vdso_check = LD      $@
       cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check)
-
-# Install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
index 2f73e5bca213..1f911a76c5af 100644
--- a/arch/arm64/kernel/vdso32/Makefile
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -172,13 +172,3 @@ gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
 quiet_cmd_vdsosym = VDSOSYM $@
 # The AArch64 nm should be able to read an AArch32 binary
       cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
-
-# Install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL32 $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
index fb0fada43197..b86f2ff31659 100644
--- a/arch/loongarch/Makefile
+++ b/arch/loongarch/Makefile
@@ -136,9 +136,7 @@ vdso_prepare: prepare0
 	$(Q)$(MAKE) $(build)=arch/loongarch/vdso include/generated/vdso-offsets.h
 endif
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/loongarch/vdso $@
+vdso-install-y += arch/loongarch/vdso/vdso.so.dbg
 
 all:	$(notdir $(KBUILD_IMAGE))
 
diff --git a/arch/loongarch/vdso/Makefile b/arch/loongarch/vdso/Makefile
index 5c97d1463328..c74c9921304f 100644
--- a/arch/loongarch/vdso/Makefile
+++ b/arch/loongarch/vdso/Makefile
@@ -83,13 +83,3 @@ $(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
 obj-y += vdso.o
 
 $(obj)/vdso.o : $(obj)/vdso.so
-
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 1329e060c548..18a47b4d6795 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -131,12 +131,6 @@ endif
 libs-y += arch/riscv/lib/
 libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/riscv/kernel/vdso $@
-	$(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
-		$(build)=arch/riscv/kernel/compat_vdso compat_$@)
-
 ifeq ($(KBUILD_EXTMOD),)
 ifeq ($(CONFIG_MMU),y)
 prepare: vdso_prepare
@@ -148,6 +142,9 @@ vdso_prepare: prepare0
 endif
 endif
 
+vdso-install-y			+= arch/riscv/kernel/vdso/vdso.so.dbg
+vdso-install-$(CONFIG_COMPAT)	+= arch/riscv/kernel/compat_vdso/compat_vdso.so.dbg:../compat_vdso/compat_vdso.so
+
 ifneq ($(CONFIG_XIP_KERNEL),y)
 ifeq ($(CONFIG_RISCV_M_MODE)$(CONFIG_ARCH_CANAAN),yy)
 KBUILD_IMAGE := $(boot)/loader.bin
diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile
index b86e5e2c3aea..62fa393b2eb2 100644
--- a/arch/riscv/kernel/compat_vdso/Makefile
+++ b/arch/riscv/kernel/compat_vdso/Makefile
@@ -76,13 +76,3 @@ quiet_cmd_compat_vdsold = VDSOLD  $@
 # actual build commands
 quiet_cmd_compat_vdsoas = VDSOAS $@
       cmd_compat_vdsoas = $(COMPAT_CC) $(a_flags) $(COMPAT_CC_FLAGS) -c -o $@ $<
-
-# install commands for the unstripped file
-quiet_cmd_compat_vdso_install = INSTALL $@
-      cmd_compat_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/compat_vdso/$@
-
-compat_vdso.so: $(obj)/compat_vdso.so.dbg
-	@mkdir -p $(MODLIB)/compat_vdso
-	$(call cmd,compat_vdso_install)
-
-compat_vdso_install: compat_vdso.so
diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile
index 6b1dba11bf6d..e8aa7c380007 100644
--- a/arch/riscv/kernel/vdso/Makefile
+++ b/arch/riscv/kernel/vdso/Makefile
@@ -73,13 +73,3 @@ quiet_cmd_vdsold = VDSOLD  $@
       cmd_vdsold = $(LD) $(ld_flags) -T $(filter-out FORCE,$^) -o $@.tmp && \
                    $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso-syms)) $@.tmp $@ && \
                    rm $@.tmp
-
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/s390/Makefile b/arch/s390/Makefile
index a53a36ee0731..73873e451686 100644
--- a/arch/s390/Makefile
+++ b/arch/s390/Makefile
@@ -138,9 +138,6 @@ bzImage: vmlinux
 zfcpdump:
 	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
 
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/$(ARCH)/kernel/vdso64 $@
-
 archheaders:
 	$(Q)$(MAKE) $(build)=$(syscalls) uapi
 
@@ -160,6 +157,9 @@ vdso_prepare: prepare0
 	$(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
 		$(build)=arch/s390/kernel/vdso32 include/generated/vdso32-offsets.h)
 
+vdso-install-y			+= arch/s390/kernel/vdso64/vdso64.so.dbg
+vdso-install-$(CONFIG_COMPAT)	+= arch/s390/kernel/vdso32/vdso32.so.dbg
+
 ifdef CONFIG_EXPOLINE_EXTERN
 modules_prepare: expoline_prepare
 expoline_prepare: scripts
diff --git a/arch/s390/kernel/vdso32/Makefile b/arch/s390/kernel/vdso32/Makefile
index 23e868b79a6c..caec7db6f966 100644
--- a/arch/s390/kernel/vdso32/Makefile
+++ b/arch/s390/kernel/vdso32/Makefile
@@ -61,16 +61,6 @@ quiet_cmd_vdso32as = VDSO32A $@
 quiet_cmd_vdso32cc = VDSO32C $@
       cmd_vdso32cc = $(CC) $(c_flags) -c -o $@ $<
 
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso32.so: $(obj)/vdso32.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso32.so
-
 # Generate VDSO offsets using helper script
 gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
 quiet_cmd_vdsosym = VDSOSYM $@
diff --git a/arch/s390/kernel/vdso64/Makefile b/arch/s390/kernel/vdso64/Makefile
index fc1c6ff8178f..e3c9085f8fa7 100644
--- a/arch/s390/kernel/vdso64/Makefile
+++ b/arch/s390/kernel/vdso64/Makefile
@@ -70,16 +70,6 @@ quiet_cmd_vdso64as = VDSO64A $@
 quiet_cmd_vdso64cc = VDSO64C $@
       cmd_vdso64cc = $(CC) $(c_flags) -c -o $@ $<
 
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso64.so: $(obj)/vdso64.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso64.so
-
 # Generate VDSO offsets using helper script
 gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
 quiet_cmd_vdsosym = VDSOSYM $@
diff --git a/arch/sparc/Makefile b/arch/sparc/Makefile
index 7417345c6639..5f6035936131 100644
--- a/arch/sparc/Makefile
+++ b/arch/sparc/Makefile
@@ -76,9 +76,8 @@ install:
 archheaders:
 	$(Q)$(MAKE) $(build)=arch/sparc/kernel/syscalls all
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/sparc/vdso $@
+vdso-install-$(CONFIG_SPARC64)	+= arch/sparc/vdso/vdso64.so.dbg
+vdso-install-$(CONFIG_COMPAT)	+= arch/sparc/vdso/vdso32.so.dbg
 
 # This is the image used for packaging
 KBUILD_IMAGE := $(boot)/zImage
diff --git a/arch/sparc/vdso/Makefile b/arch/sparc/vdso/Makefile
index 77d7b9032158..d08c3a0443f3 100644
--- a/arch/sparc/vdso/Makefile
+++ b/arch/sparc/vdso/Makefile
@@ -116,30 +116,3 @@ quiet_cmd_vdso = VDSO    $@
 
 VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 -Bsymbolic
 GCOV_PROFILE := n
-
-#
-# Install the unstripped copies of vdso*.so.  If our toolchain supports
-# build-id, install .build-id links as well.
-#
-quiet_cmd_vdso_install = INSTALL $(@:install_%=%)
-define cmd_vdso_install
-	cp $< "$(MODLIB)/vdso/$(@:install_%=%)"; \
-	if readelf -n $< |grep -q 'Build ID'; then \
-	  buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
-	  first=`echo $$buildid | cut -b-2`; \
-	  last=`echo $$buildid | cut -b3-`; \
-	  mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
-	  ln -sf "../../$(@:install_%=%)" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
-	fi
-endef
-
-vdso_img_insttargets := $(vdso_img_sodbg:%.dbg=install_%)
-
-$(MODLIB)/vdso: FORCE
-	@mkdir -p $(MODLIB)/vdso
-
-$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso FORCE
-	$(call cmd,vdso_install)
-
-PHONY += vdso_install $(vdso_img_insttargets)
-vdso_install: $(vdso_img_insttargets) FORCE
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 5bfe5caaa444..3ff53a2d4ff0 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -291,9 +291,10 @@ PHONY += install
 install:
 	$(call cmd,install)
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/x86/entry/vdso $@
+vdso-install-$(CONFIG_X86_64)		+= arch/x86/entry/vdso/vdso64.so.dbg
+vdso-install-$(CONFIG_X86_X32_ABI)	+= arch/x86/entry/vdso/vdsox32.so.dbg
+vdso-install-$(CONFIG_X86_32)		+= arch/x86/entry/vdso/vdso32.so.dbg
+vdso-install-$(CONFIG_IA32_EMULATION)	+= arch/x86/entry/vdso/vdso32.so.dbg
 
 archprepare: checkbin
 checkbin:
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 6a1821bd7d5e..c197efd82922 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -190,31 +190,4 @@ GCOV_PROFILE := n
 quiet_cmd_vdso_and_check = VDSO    $@
       cmd_vdso_and_check = $(cmd_vdso); $(cmd_vdso_check)
 
-#
-# Install the unstripped copies of vdso*.so.  If our toolchain supports
-# build-id, install .build-id links as well.
-#
-quiet_cmd_vdso_install = INSTALL $(@:install_%=%)
-define cmd_vdso_install
-	cp $< "$(MODLIB)/vdso/$(@:install_%=%)"; \
-	if readelf -n $< |grep -q 'Build ID'; then \
-	  buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
-	  first=`echo $$buildid | cut -b-2`; \
-	  last=`echo $$buildid | cut -b3-`; \
-	  mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
-	  ln -sf "../../$(@:install_%=%)" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
-	fi
-endef
-
-vdso_img_insttargets := $(vdso_img_sodbg:%.dbg=install_%)
-
-$(MODLIB)/vdso: FORCE
-	@mkdir -p $(MODLIB)/vdso
-
-$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-PHONY += vdso_install $(vdso_img_insttargets)
-vdso_install: $(vdso_img_insttargets)
-
 clean-files := vdso32.so vdso32.so.dbg vdso64* vdso-image-*.c vdsox32.so*
diff --git a/scripts/Makefile.vdsoinst b/scripts/Makefile.vdsoinst
new file mode 100644
index 000000000000..1022d9fdd976
--- /dev/null
+++ b/scripts/Makefile.vdsoinst
@@ -0,0 +1,45 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# ==========================================================================
+# Install unstripped copies of vDSO
+# ==========================================================================
+
+PHONY := __default
+__default:
+	@:
+
+include $(srctree)/scripts/Kbuild.include
+
+install-dir := $(MODLIB)/vdso
+
+define gen_install_rules
+
+src := $$(firstword $$(subst :,$(space),$(1)))
+dest := $(install-dir)/$$(or $$(word 2,$$(subst :,$(space),$(1))),$$(patsubst %.dbg,%,$$(notdir $(1))))
+
+__default: $$(dest)
+$$(dest): $$(src) FORCE
+	$$(call cmd,install)
+
+# Some architectures create .build-id symlinks
+ifneq ($(filter arm sparc x86, $(SRCARCH)),)
+link := $(install-dir)/.build-id/$$(shell $(READELF) -n $$(src) | sed -n 's@^.*Build ID: \(..\)\(.*\)@\1/\2 at p')
+
+__default: $$(link)
+$$(link): $$(dest) FORCE
+	$$(call cmd,symlink)
+endif
+
+endef
+
+$(foreach x, $(sort $(INSTALL_FILES)), $(eval $(call gen_install_rules,$(x))))
+
+quiet_cmd_install = INSTALL $@
+      cmd_install = mkdir -p $(dir $@); cp $< $@
+
+quiet_cmd_symlink = SYMLINK $@
+      cmd_symlink = mkdir -p $(dir $@); ln -sf --relative $< $@
+
+PHONY += FORCE
+FORCE:
+
+.PHONY: $(PHONY)
-- 
2.39.2



From masahiroy at kernel.org  Mon Oct  9 05:42:10 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Mon,  9 Oct 2023 21:42:10 +0900
Subject: [PATCH 5/5] kbuild: unify no-compiler-targets and no-sync-config-targets
In-Reply-To: <20231009124210.1064021-1-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
Message-ID: <20231009124210.1064021-5-masahiroy@kernel.org>

Now that vdso_install does not depend on any in-tree build artifact,
it no longer invokes a compiler, making no-compiler-targets the same
as no-sync-config-targets.

Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
---

 Makefile | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

diff --git a/Makefile b/Makefile
index 2170d56630e8..982b1ad33287 100644
--- a/Makefile
+++ b/Makefile
@@ -277,10 +277,6 @@ no-dot-config-targets := $(clean-targets) \
 			 $(version_h) headers headers_% archheaders archscripts \
 			 %asm-generic kernelversion %src-pkg dt_binding_check \
 			 outputmakefile rustavailable rustfmt rustfmtcheck
-# Installation targets should not require compiler. Unfortunately, vdso_install
-# is an exception where build artifacts may be updated. This must be fixed.
-no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
-			headers_install modules_install modules_sign kernelrelease image_name
 no-sync-config-targets := $(no-dot-config-targets) %install modules_sign kernelrelease \
 			  image_name
 single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %/
@@ -288,7 +284,6 @@ single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %
 config-build	:=
 mixed-build	:=
 need-config	:= 1
-need-compiler	:= 1
 may-sync-config	:= 1
 single-build	:=
 
@@ -298,12 +293,6 @@ ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
 	endif
 endif
 
-ifneq ($(filter $(no-compiler-targets), $(MAKECMDGOALS)),)
-	ifeq ($(filter-out $(no-compiler-targets), $(MAKECMDGOALS)),)
-		need-compiler :=
-	endif
-endif
-
 ifneq ($(filter $(no-sync-config-targets), $(MAKECMDGOALS)),)
 	ifeq ($(filter-out $(no-sync-config-targets), $(MAKECMDGOALS)),)
 		may-sync-config :=
@@ -675,7 +664,7 @@ endif
 
 # Include this also for config targets because some architectures need
 # cc-cross-prefix to determine CROSS_COMPILE.
-ifdef need-compiler
+ifdef may-sync-config
 include $(srctree)/scripts/Makefile.compiler
 endif
 
-- 
2.39.2



From richard at nod.at  Mon Oct  9 06:09:18 2023
From: richard at nod.at (Richard Weinberger)
Date: Mon, 9 Oct 2023 15:09:18 +0200 (CEST)
Subject: [PATCH 2/5] UML: remove unused cmd_vdso_install
In-Reply-To: <20231009124210.1064021-2-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org> <20231009124210.1064021-2-masahiroy@kernel.org>
Message-ID: <687416388.7594.1696856958852.JavaMail.zimbra@nod.at>

----- Urspr?ngliche Mail -----
> Von: "masahiroy" <masahiroy at kernel.org>
> An: "linux-kbuild" <linux-kbuild at vger.kernel.org>
> CC: "linux-kernel" <linux-kernel at vger.kernel.org>, "linux-arm-kernel" <linux-arm-kernel at lists.infradead.org>,
> linux-csky at vger.kernel.org, "linux-parisc" <linux-parisc at vger.kernel.org>, linux-riscv at lists.infradead.org,
> linux-s390 at vger.kernel.org, "linux-um" <linux-um at lists.infradead.org>, "loongarch" <loongarch at lists.linux.dev>,
> "sparclinux" <sparclinux at vger.kernel.org>, "x86" <x86 at kernel.org>, "masahiroy" <masahiroy at kernel.org>, "anton ivanov"
> <anton.ivanov at cambridgegreys.com>, "bp" <bp at alien8.de>, "dave hansen" <dave.hansen at linux.intel.com>, "hpa"
> <hpa at zytor.com>, "mingo" <mingo at redhat.com>, "Johannes Berg" <johannes at sipsolutions.net>, "richard" <richard at nod.at>,
> "tglx" <tglx at linutronix.de>
> Gesendet: Montag, 9. Oktober 2023 14:42:07
> Betreff: [PATCH 2/5] UML: remove unused cmd_vdso_install

> You cannot run this code because arch/um/Makefile does not define the
> vdso_install target.
> 
> It appears that this code was blindly copied from another architecture.
> 
> Remove the dead code.
> 
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>

Acked-by: Richard Weinberger <richard at nod.at>

Thanks,
//richard


From guoren at kernel.org  Mon Oct  9 08:16:30 2023
From: guoren at kernel.org (Guo Ren)
Date: Mon, 9 Oct 2023 23:16:30 +0800
Subject: [PATCH 1/5] csky: remove unused cmd_vdso_install
In-Reply-To: <20231009124210.1064021-1-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
Message-ID: <CAJF2gTQxiCEyGiGieHJMBK0Hmj49f8mdcm1TwsTZbgbvxVxGcw@mail.gmail.com>

On Mon, Oct 9, 2023 at 8:42?PM Masahiro Yamada <masahiroy at kernel.org> wrote:
>
> You cannot run this code because arch/csky/Makefile does not define the
> vdso_install target.
>
> It appears that this code was blindly copied from another architecture.
Yes, I do that. Thx for pointing it out.

Acked-by: Guo Ren <guoren at kernel.org>

>
> Remove the dead code.
>
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
> ---
>
>  arch/csky/kernel/vdso/Makefile | 10 ----------
>  1 file changed, 10 deletions(-)
>
> diff --git a/arch/csky/kernel/vdso/Makefile b/arch/csky/kernel/vdso/Makefile
> index 299e4e41ebc5..ddf784a62c11 100644
> --- a/arch/csky/kernel/vdso/Makefile
> +++ b/arch/csky/kernel/vdso/Makefile
> @@ -58,13 +58,3 @@ quiet_cmd_vdsold = VDSOLD  $@
>  # that contains the same symbols at the same offsets.
>  quiet_cmd_so2s = SO2S    $@
>        cmd_so2s = $(NM) -D $< | $(srctree)/$(src)/so2s.sh > $@
> -
> -# install commands for the unstripped file
> -quiet_cmd_vdso_install = INSTALL $@
> -      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
> -
> -vdso.so: $(obj)/vdso.so.dbg
> -       @mkdir -p $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -vdso_install: vdso.so
> --
> 2.39.2
>


-- 
Best Regards
 Guo Ren


From nathan at kernel.org  Mon Oct  9 09:44:24 2023
From: nathan at kernel.org (Nathan Chancellor)
Date: Mon, 9 Oct 2023 09:44:24 -0700
Subject: [PATCH 5/5] kbuild: unify no-compiler-targets and
 no-sync-config-targets
In-Reply-To: <20231009124210.1064021-5-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-5-masahiroy@kernel.org>
Message-ID: <20231009164424.GB1153868@dev-arch.thelio-3990X>

On Mon, Oct 09, 2023 at 09:42:10PM +0900, Masahiro Yamada wrote:
> Now that vdso_install does not depend on any in-tree build artifact,
> it no longer invokes a compiler, making no-compiler-targets the same
> as no-sync-config-targets.
> 
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
> ---
> 
>  Makefile | 13 +------------
>  1 file changed, 1 insertion(+), 12 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 2170d56630e8..982b1ad33287 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -277,10 +277,6 @@ no-dot-config-targets := $(clean-targets) \
>  			 $(version_h) headers headers_% archheaders archscripts \
>  			 %asm-generic kernelversion %src-pkg dt_binding_check \
>  			 outputmakefile rustavailable rustfmt rustfmtcheck
> -# Installation targets should not require compiler. Unfortunately, vdso_install
> -# is an exception where build artifacts may be updated. This must be fixed.
> -no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
> -			headers_install modules_install modules_sign kernelrelease image_name
>  no-sync-config-targets := $(no-dot-config-targets) %install modules_sign kernelrelease \
>  			  image_name
>  single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %/
> @@ -288,7 +284,6 @@ single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %
>  config-build	:=
>  mixed-build	:=
>  need-config	:= 1
> -need-compiler	:= 1
>  may-sync-config	:= 1
>  single-build	:=
>  
> @@ -298,12 +293,6 @@ ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
>  	endif
>  endif
>  
> -ifneq ($(filter $(no-compiler-targets), $(MAKECMDGOALS)),)
> -	ifeq ($(filter-out $(no-compiler-targets), $(MAKECMDGOALS)),)
> -		need-compiler :=
> -	endif
> -endif
> -

MIPS and LoongArch seem to have grown a usage of need-compiler in
4fe4a6374c4d ("MIPS: Only fiddle with CHECKFLAGS if `need-compiler'")
and 54c2c9df083f ("LoongArch: Only fiddle with CHECKFLAGS if
`need-compiler'"). With this removal, should those be updated as well?

>  ifneq ($(filter $(no-sync-config-targets), $(MAKECMDGOALS)),)
>  	ifeq ($(filter-out $(no-sync-config-targets), $(MAKECMDGOALS)),)
>  		may-sync-config :=
> @@ -675,7 +664,7 @@ endif
>  
>  # Include this also for config targets because some architectures need
>  # cc-cross-prefix to determine CROSS_COMPILE.
> -ifdef need-compiler
> +ifdef may-sync-config
>  include $(srctree)/scripts/Makefile.compiler
>  endif
>  
> -- 
> 2.39.2
> 


From svens at linux.ibm.com  Tue Oct 10 00:36:57 2023
From: svens at linux.ibm.com (Sven Schnelle)
Date: Tue, 10 Oct 2023 09:36:57 +0200
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <20231009124210.1064021-4-masahiroy@kernel.org> (Masahiro
	Yamada's message of "Mon, 9 Oct 2023 21:42:09 +0900")
References: <20231009124210.1064021-1-masahiroy@kernel.org>
	<20231009124210.1064021-4-masahiroy@kernel.org>
Message-ID: <yt9dfs2judwm.fsf@linux.ibm.com>

Masahiro Yamada <masahiroy at kernel.org> writes:

> Currently, there is no standard implementation for vdso_install,
> leading to various issues:
>
>  1. Code duplication
>
>     Many architectures duplicate similar code just for copying files
>     to the install destination.
>
>     Some architectures (arm, sparc, x86) create build-id symlinks,
>     introducing more code duplication.
>
>  2. Accidental updates of in-tree build artifacts
>
>     The vdso_install rule depends on the vdso files to install.
>     It may update in-tree build artifacts. This can be problematic,
>     as explained in commit 19514fc665ff ("arm, kbuild: make
>     "make install" not depend on vmlinux").
>
>  3. Broken code in some architectures
>
>     Makefile code is often copied from one architecture to another
>     without proper adaptation or testing.
>
>     The previous commits removed broken code from csky, UML, and parisc.
>
>     Another issue is that 'make vdso_install' for ARCH=s390 installs
>     vdso64, but not vdso32.
>
> To address these problems, this commit introduces the generic vdso_install.
>
> Architectures that support vdso_install need to define vdso-install-y
> in arch/*/Makefile.
>
> vdso-install-y lists the files to install. For example, arch/x86/Makefile
> looks like this:
>
>   vdso-install-$(CONFIG_X86_64)           += arch/x86/entry/vdso/vdso64.so.dbg
>   vdso-install-$(CONFIG_X86_X32_ABI)      += arch/x86/entry/vdso/vdsox32.so.dbg
>   vdso-install-$(CONFIG_X86_32)           += arch/x86/entry/vdso/vdso32.so.dbg
>   vdso-install-$(CONFIG_IA32_EMULATION)   += arch/x86/entry/vdso/vdso32.so.dbg
>
> These files will be installed to $(MODLIB)/vdso/ with the .dbg suffix,
> if exists, stripped away.
>
> vdso-install-y can optionally take the second field after the colon
> separator. This is needed because some architectures install vdso
> files as a different base name.
>
> The following is a snippet from arch/arm64/Makefile.
>
>   vdso-install-$(CONFIG_COMPAT_VDSO)      += arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so
>
> This will rename vdso.so.dbg to vdso32.so during installation. If such
> architectures change their implementation so that the file names match,
> this workaround will go away.
>
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
> ---

Acked-by: Sven Schnelle <svens at linux.ibm.com> # s390


From guoren at kernel.org  Tue Oct 10 19:23:57 2023
From: guoren at kernel.org (Guo Ren)
Date: Wed, 11 Oct 2023 10:23:57 +0800
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <20231009124210.1064021-4-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org> <20231009124210.1064021-4-masahiroy@kernel.org>
Message-ID: <CAJF2gTTDpGgzsiRk=q6FCdX_g5maY-sT9h0jiW=p6HLziq97yA@mail.gmail.com>

On Mon, Oct 9, 2023 at 8:42?PM Masahiro Yamada <masahiroy at kernel.org> wrote:
>
> Currently, there is no standard implementation for vdso_install,
> leading to various issues:
>
>  1. Code duplication
>
>     Many architectures duplicate similar code just for copying files
>     to the install destination.
>
>     Some architectures (arm, sparc, x86) create build-id symlinks,
>     introducing more code duplication.
>
>  2. Accidental updates of in-tree build artifacts
>
>     The vdso_install rule depends on the vdso files to install.
>     It may update in-tree build artifacts. This can be problematic,
>     as explained in commit 19514fc665ff ("arm, kbuild: make
>     "make install" not depend on vmlinux").
>
>  3. Broken code in some architectures
>
>     Makefile code is often copied from one architecture to another
>     without proper adaptation or testing.
>
>     The previous commits removed broken code from csky, UML, and parisc.
>
>     Another issue is that 'make vdso_install' for ARCH=s390 installs
>     vdso64, but not vdso32.
>
> To address these problems, this commit introduces the generic vdso_install.
>
> Architectures that support vdso_install need to define vdso-install-y
> in arch/*/Makefile.
>
> vdso-install-y lists the files to install. For example, arch/x86/Makefile
> looks like this:
>
>   vdso-install-$(CONFIG_X86_64)           += arch/x86/entry/vdso/vdso64.so.dbg
>   vdso-install-$(CONFIG_X86_X32_ABI)      += arch/x86/entry/vdso/vdsox32.so.dbg
>   vdso-install-$(CONFIG_X86_32)           += arch/x86/entry/vdso/vdso32.so.dbg
>   vdso-install-$(CONFIG_IA32_EMULATION)   += arch/x86/entry/vdso/vdso32.so.dbg
>
> These files will be installed to $(MODLIB)/vdso/ with the .dbg suffix,
> if exists, stripped away.
>
> vdso-install-y can optionally take the second field after the colon
> separator. This is needed because some architectures install vdso
> files as a different base name.
>
> The following is a snippet from arch/arm64/Makefile.
>
>   vdso-install-$(CONFIG_COMPAT_VDSO)      += arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so
>
> This will rename vdso.so.dbg to vdso32.so during installation. If such
> architectures change their implementation so that the file names match,
> this workaround will go away.
>
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
> ---
>
>  Makefile                               |  9 ++++++
>  arch/arm/Makefile                      |  7 +---
>  arch/arm/vdso/Makefile                 | 25 --------------
>  arch/arm64/Makefile                    |  9 ++----
>  arch/arm64/kernel/vdso/Makefile        | 10 ------
>  arch/arm64/kernel/vdso32/Makefile      | 10 ------
>  arch/loongarch/Makefile                |  4 +--
>  arch/loongarch/vdso/Makefile           | 10 ------
>  arch/riscv/Makefile                    |  9 ++----
>  arch/riscv/kernel/compat_vdso/Makefile | 10 ------
>  arch/riscv/kernel/vdso/Makefile        | 10 ------
>  arch/s390/Makefile                     |  6 ++--
>  arch/s390/kernel/vdso32/Makefile       | 10 ------
>  arch/s390/kernel/vdso64/Makefile       | 10 ------
>  arch/sparc/Makefile                    |  5 ++-
>  arch/sparc/vdso/Makefile               | 27 ----------------
>  arch/x86/Makefile                      |  7 ++--
>  arch/x86/entry/vdso/Makefile           | 27 ----------------
>  scripts/Makefile.vdsoinst              | 45 ++++++++++++++++++++++++++
>  19 files changed, 71 insertions(+), 179 deletions(-)
>  create mode 100644 scripts/Makefile.vdsoinst
>
> diff --git a/Makefile b/Makefile
> index 373649c7374e..2170d56630e8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1317,6 +1317,14 @@ scripts_unifdef: scripts_basic
>  quiet_cmd_install = INSTALL $(INSTALL_PATH)
>        cmd_install = unset sub_make_done; $(srctree)/scripts/install.sh
>
> +# ---------------------------------------------------------------------------
> +# vDSO install
> +
> +PHONY += vdso_install
> +vdso_install: export INSTALL_FILES = $(vdso-install-y)
> +vdso_install:
> +       $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vdsoinst
> +
>  # ---------------------------------------------------------------------------
>  # Tools
>
> @@ -1560,6 +1568,7 @@ help:
>         @echo  '* vmlinux         - Build the bare kernel'
>         @echo  '* modules         - Build all modules'
>         @echo  '  modules_install - Install all modules to INSTALL_MOD_PATH (default: /)'
> +       @echo  '  vdso_install    - Install unstripped vdso to INSTALL_MOD_PATH (default: /)'
>         @echo  '  dir/            - Build all files in dir and below'
>         @echo  '  dir/file.[ois]  - Build specified target only'
>         @echo  '  dir/file.ll     - Build the LLVM assembly file'
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 547e5856eaa0..5ba42f69f8ce 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -304,11 +304,7 @@ $(INSTALL_TARGETS): KBUILD_IMAGE = $(boot)/$(patsubst %install,%Image,$@)
>  $(INSTALL_TARGETS):
>         $(call cmd,install)
>
> -PHONY += vdso_install
> -vdso_install:
> -ifeq ($(CONFIG_VDSO),y)
> -       $(Q)$(MAKE) $(build)=arch/arm/vdso $@
> -endif
> +vdso-install-$(CONFIG_VDSO) += arch/arm/vdso/vdso.so.dbg
>
>  # My testing targets (bypasses dependencies)
>  bp:;   $(Q)$(MAKE) $(build)=$(boot) $(boot)/bootpImage
> @@ -331,7 +327,6 @@ define archhelp
>    echo  '                  Install using (your) ~/bin/$(INSTALLKERNEL) or'
>    echo  '                  (distribution) /sbin/$(INSTALLKERNEL) or'
>    echo  '                  install to $$(INSTALL_PATH) and run lilo'
> -  echo  '  vdso_install  - Install unstripped vdso.so to $$(INSTALL_MOD_PATH)/vdso'
>    echo
>    echo  '  multi_v7_lpae_defconfig     - multi_v7_defconfig with CONFIG_ARM_LPAE enabled'
>  endef
> diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
> index 515ca33b854c..d761bd2e2f40 100644
> --- a/arch/arm/vdso/Makefile
> +++ b/arch/arm/vdso/Makefile
> @@ -63,28 +63,3 @@ quiet_cmd_vdsold_and_vdso_check = LD      $@
>
>  quiet_cmd_vdsomunge = MUNGE   $@
>        cmd_vdsomunge = $(objtree)/$(obj)/vdsomunge $< $@
> -
> -#
> -# Install the unstripped copy of vdso.so.dbg.  If our toolchain
> -# supports build-id, install .build-id links as well.
> -#
> -# Cribbed from arch/x86/vdso/Makefile.
> -#
> -quiet_cmd_vdso_install = INSTALL $<
> -define cmd_vdso_install
> -       cp $< "$(MODLIB)/vdso/vdso.so"; \
> -       if readelf -n $< | grep -q 'Build ID'; then \
> -         buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
> -         first=`echo $$buildid | cut -b-2`; \
> -         last=`echo $$buildid | cut -b3-`; \
> -         mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
> -         ln -sf "../../vdso.so" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
> -       fi
> -endef
> -
> -$(MODLIB)/vdso: FORCE
> -       @mkdir -p $(MODLIB)/vdso
> -
> -PHONY += vdso_install
> -vdso_install: $(obj)/vdso.so.dbg $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 2d49aea0ff67..4bd85cc0d32b 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -169,12 +169,6 @@ install: KBUILD_IMAGE := $(boot)/Image
>  install zinstall:
>         $(call cmd,install)
>
> -PHONY += vdso_install
> -vdso_install:
> -       $(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso $@
> -       $(if $(CONFIG_COMPAT_VDSO), \
> -               $(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso32 $@)
> -
>  archprepare:
>         $(Q)$(MAKE) $(build)=arch/arm64/tools kapi
>  ifeq ($(CONFIG_ARM64_ERRATUM_843419),y)
> @@ -205,6 +199,9 @@ ifdef CONFIG_COMPAT_VDSO
>  endif
>  endif
>
> +vdso-install-y                         += arch/arm64/kernel/vdso/vdso.so.dbg
> +vdso-install-$(CONFIG_COMPAT_VDSO)     += arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so
> +
>  include $(srctree)/scripts/Makefile.defconf
>
>  PHONY += virtconfig
> diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
> index fe7a53c6781f..8818287f1095 100644
> --- a/arch/arm64/kernel/vdso/Makefile
> +++ b/arch/arm64/kernel/vdso/Makefile
> @@ -78,13 +78,3 @@ include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
>  # Actual build commands
>  quiet_cmd_vdsold_and_vdso_check = LD      $@
>        cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check)
> -
> -# Install commands for the unstripped file
> -quiet_cmd_vdso_install = INSTALL $@
> -      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
> -
> -vdso.so: $(obj)/vdso.so.dbg
> -       @mkdir -p $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -vdso_install: vdso.so
> diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
> index 2f73e5bca213..1f911a76c5af 100644
> --- a/arch/arm64/kernel/vdso32/Makefile
> +++ b/arch/arm64/kernel/vdso32/Makefile
> @@ -172,13 +172,3 @@ gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
>  quiet_cmd_vdsosym = VDSOSYM $@
>  # The AArch64 nm should be able to read an AArch32 binary
>        cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
> -
> -# Install commands for the unstripped file
> -quiet_cmd_vdso_install = INSTALL32 $@
> -      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
> -
> -vdso.so: $(obj)/vdso.so.dbg
> -       @mkdir -p $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -vdso_install: vdso.so
> diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
> index fb0fada43197..b86f2ff31659 100644
> --- a/arch/loongarch/Makefile
> +++ b/arch/loongarch/Makefile
> @@ -136,9 +136,7 @@ vdso_prepare: prepare0
>         $(Q)$(MAKE) $(build)=arch/loongarch/vdso include/generated/vdso-offsets.h
>  endif
>
> -PHONY += vdso_install
> -vdso_install:
> -       $(Q)$(MAKE) $(build)=arch/loongarch/vdso $@
> +vdso-install-y += arch/loongarch/vdso/vdso.so.dbg
>
>  all:   $(notdir $(KBUILD_IMAGE))
>
> diff --git a/arch/loongarch/vdso/Makefile b/arch/loongarch/vdso/Makefile
> index 5c97d1463328..c74c9921304f 100644
> --- a/arch/loongarch/vdso/Makefile
> +++ b/arch/loongarch/vdso/Makefile
> @@ -83,13 +83,3 @@ $(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
>  obj-y += vdso.o
>
>  $(obj)/vdso.o : $(obj)/vdso.so
> -
> -# install commands for the unstripped file
> -quiet_cmd_vdso_install = INSTALL $@
> -      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
> -
> -vdso.so: $(obj)/vdso.so.dbg
> -       @mkdir -p $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -vdso_install: vdso.so
> diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
> index 1329e060c548..18a47b4d6795 100644
> --- a/arch/riscv/Makefile
> +++ b/arch/riscv/Makefile
> @@ -131,12 +131,6 @@ endif
>  libs-y += arch/riscv/lib/
>  libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
>
> -PHONY += vdso_install
> -vdso_install:
> -       $(Q)$(MAKE) $(build)=arch/riscv/kernel/vdso $@
> -       $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
> -               $(build)=arch/riscv/kernel/compat_vdso compat_$@)
> -
>  ifeq ($(KBUILD_EXTMOD),)
>  ifeq ($(CONFIG_MMU),y)
>  prepare: vdso_prepare
> @@ -148,6 +142,9 @@ vdso_prepare: prepare0
>  endif
>  endif
>
> +vdso-install-y                 += arch/riscv/kernel/vdso/vdso.so.dbg
> +vdso-install-$(CONFIG_COMPAT)  += arch/riscv/kernel/compat_vdso/compat_vdso.so.dbg:../compat_vdso/compat_vdso.so
Why do we need ":../compat_vdso/compat_vdso.so" here?

> +
>  ifneq ($(CONFIG_XIP_KERNEL),y)
>  ifeq ($(CONFIG_RISCV_M_MODE)$(CONFIG_ARCH_CANAAN),yy)
>  KBUILD_IMAGE := $(boot)/loader.bin
> diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile
> index b86e5e2c3aea..62fa393b2eb2 100644
> --- a/arch/riscv/kernel/compat_vdso/Makefile
> +++ b/arch/riscv/kernel/compat_vdso/Makefile
> @@ -76,13 +76,3 @@ quiet_cmd_compat_vdsold = VDSOLD  $@
>  # actual build commands
>  quiet_cmd_compat_vdsoas = VDSOAS $@
>        cmd_compat_vdsoas = $(COMPAT_CC) $(a_flags) $(COMPAT_CC_FLAGS) -c -o $@ $<
> -
> -# install commands for the unstripped file
> -quiet_cmd_compat_vdso_install = INSTALL $@
> -      cmd_compat_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/compat_vdso/$@
> -
> -compat_vdso.so: $(obj)/compat_vdso.so.dbg
> -       @mkdir -p $(MODLIB)/compat_vdso
> -       $(call cmd,compat_vdso_install)
> -
> -compat_vdso_install: compat_vdso.so
> diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile
> index 6b1dba11bf6d..e8aa7c380007 100644
> --- a/arch/riscv/kernel/vdso/Makefile
> +++ b/arch/riscv/kernel/vdso/Makefile
> @@ -73,13 +73,3 @@ quiet_cmd_vdsold = VDSOLD  $@
>        cmd_vdsold = $(LD) $(ld_flags) -T $(filter-out FORCE,$^) -o $@.tmp && \
>                     $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso-syms)) $@.tmp $@ && \
>                     rm $@.tmp
> -
> -# install commands for the unstripped file
> -quiet_cmd_vdso_install = INSTALL $@
> -      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
> -
> -vdso.so: $(obj)/vdso.so.dbg
> -       @mkdir -p $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -vdso_install: vdso.so
> diff --git a/arch/s390/Makefile b/arch/s390/Makefile
> index a53a36ee0731..73873e451686 100644
> --- a/arch/s390/Makefile
> +++ b/arch/s390/Makefile
> @@ -138,9 +138,6 @@ bzImage: vmlinux
>  zfcpdump:
>         $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
>
> -vdso_install:
> -       $(Q)$(MAKE) $(build)=arch/$(ARCH)/kernel/vdso64 $@
> -
>  archheaders:
>         $(Q)$(MAKE) $(build)=$(syscalls) uapi
>
> @@ -160,6 +157,9 @@ vdso_prepare: prepare0
>         $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
>                 $(build)=arch/s390/kernel/vdso32 include/generated/vdso32-offsets.h)
>
> +vdso-install-y                 += arch/s390/kernel/vdso64/vdso64.so.dbg
> +vdso-install-$(CONFIG_COMPAT)  += arch/s390/kernel/vdso32/vdso32.so.dbg
> +
>  ifdef CONFIG_EXPOLINE_EXTERN
>  modules_prepare: expoline_prepare
>  expoline_prepare: scripts
> diff --git a/arch/s390/kernel/vdso32/Makefile b/arch/s390/kernel/vdso32/Makefile
> index 23e868b79a6c..caec7db6f966 100644
> --- a/arch/s390/kernel/vdso32/Makefile
> +++ b/arch/s390/kernel/vdso32/Makefile
> @@ -61,16 +61,6 @@ quiet_cmd_vdso32as = VDSO32A $@
>  quiet_cmd_vdso32cc = VDSO32C $@
>        cmd_vdso32cc = $(CC) $(c_flags) -c -o $@ $<
>
> -# install commands for the unstripped file
> -quiet_cmd_vdso_install = INSTALL $@
> -      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
> -
> -vdso32.so: $(obj)/vdso32.so.dbg
> -       @mkdir -p $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -vdso_install: vdso32.so
> -
>  # Generate VDSO offsets using helper script
>  gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
>  quiet_cmd_vdsosym = VDSOSYM $@
> diff --git a/arch/s390/kernel/vdso64/Makefile b/arch/s390/kernel/vdso64/Makefile
> index fc1c6ff8178f..e3c9085f8fa7 100644
> --- a/arch/s390/kernel/vdso64/Makefile
> +++ b/arch/s390/kernel/vdso64/Makefile
> @@ -70,16 +70,6 @@ quiet_cmd_vdso64as = VDSO64A $@
>  quiet_cmd_vdso64cc = VDSO64C $@
>        cmd_vdso64cc = $(CC) $(c_flags) -c -o $@ $<
>
> -# install commands for the unstripped file
> -quiet_cmd_vdso_install = INSTALL $@
> -      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
> -
> -vdso64.so: $(obj)/vdso64.so.dbg
> -       @mkdir -p $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -vdso_install: vdso64.so
> -
>  # Generate VDSO offsets using helper script
>  gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
>  quiet_cmd_vdsosym = VDSOSYM $@
> diff --git a/arch/sparc/Makefile b/arch/sparc/Makefile
> index 7417345c6639..5f6035936131 100644
> --- a/arch/sparc/Makefile
> +++ b/arch/sparc/Makefile
> @@ -76,9 +76,8 @@ install:
>  archheaders:
>         $(Q)$(MAKE) $(build)=arch/sparc/kernel/syscalls all
>
> -PHONY += vdso_install
> -vdso_install:
> -       $(Q)$(MAKE) $(build)=arch/sparc/vdso $@
> +vdso-install-$(CONFIG_SPARC64) += arch/sparc/vdso/vdso64.so.dbg
> +vdso-install-$(CONFIG_COMPAT)  += arch/sparc/vdso/vdso32.so.dbg
>
>  # This is the image used for packaging
>  KBUILD_IMAGE := $(boot)/zImage
> diff --git a/arch/sparc/vdso/Makefile b/arch/sparc/vdso/Makefile
> index 77d7b9032158..d08c3a0443f3 100644
> --- a/arch/sparc/vdso/Makefile
> +++ b/arch/sparc/vdso/Makefile
> @@ -116,30 +116,3 @@ quiet_cmd_vdso = VDSO    $@
>
>  VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 -Bsymbolic
>  GCOV_PROFILE := n
> -
> -#
> -# Install the unstripped copies of vdso*.so.  If our toolchain supports
> -# build-id, install .build-id links as well.
> -#
> -quiet_cmd_vdso_install = INSTALL $(@:install_%=%)
> -define cmd_vdso_install
> -       cp $< "$(MODLIB)/vdso/$(@:install_%=%)"; \
> -       if readelf -n $< |grep -q 'Build ID'; then \
> -         buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
> -         first=`echo $$buildid | cut -b-2`; \
> -         last=`echo $$buildid | cut -b3-`; \
> -         mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
> -         ln -sf "../../$(@:install_%=%)" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
> -       fi
> -endef
> -
> -vdso_img_insttargets := $(vdso_img_sodbg:%.dbg=install_%)
> -
> -$(MODLIB)/vdso: FORCE
> -       @mkdir -p $(MODLIB)/vdso
> -
> -$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso FORCE
> -       $(call cmd,vdso_install)
> -
> -PHONY += vdso_install $(vdso_img_insttargets)
> -vdso_install: $(vdso_img_insttargets) FORCE
> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
> index 5bfe5caaa444..3ff53a2d4ff0 100644
> --- a/arch/x86/Makefile
> +++ b/arch/x86/Makefile
> @@ -291,9 +291,10 @@ PHONY += install
>  install:
>         $(call cmd,install)
>
> -PHONY += vdso_install
> -vdso_install:
> -       $(Q)$(MAKE) $(build)=arch/x86/entry/vdso $@
> +vdso-install-$(CONFIG_X86_64)          += arch/x86/entry/vdso/vdso64.so.dbg
> +vdso-install-$(CONFIG_X86_X32_ABI)     += arch/x86/entry/vdso/vdsox32.so.dbg
> +vdso-install-$(CONFIG_X86_32)          += arch/x86/entry/vdso/vdso32.so.dbg
> +vdso-install-$(CONFIG_IA32_EMULATION)  += arch/x86/entry/vdso/vdso32.so.dbg
>
>  archprepare: checkbin
>  checkbin:
> diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
> index 6a1821bd7d5e..c197efd82922 100644
> --- a/arch/x86/entry/vdso/Makefile
> +++ b/arch/x86/entry/vdso/Makefile
> @@ -190,31 +190,4 @@ GCOV_PROFILE := n
>  quiet_cmd_vdso_and_check = VDSO    $@
>        cmd_vdso_and_check = $(cmd_vdso); $(cmd_vdso_check)
>
> -#
> -# Install the unstripped copies of vdso*.so.  If our toolchain supports
> -# build-id, install .build-id links as well.
> -#
> -quiet_cmd_vdso_install = INSTALL $(@:install_%=%)
> -define cmd_vdso_install
> -       cp $< "$(MODLIB)/vdso/$(@:install_%=%)"; \
> -       if readelf -n $< |grep -q 'Build ID'; then \
> -         buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
> -         first=`echo $$buildid | cut -b-2`; \
> -         last=`echo $$buildid | cut -b3-`; \
> -         mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
> -         ln -sf "../../$(@:install_%=%)" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
> -       fi
> -endef
> -
> -vdso_img_insttargets := $(vdso_img_sodbg:%.dbg=install_%)
> -
> -$(MODLIB)/vdso: FORCE
> -       @mkdir -p $(MODLIB)/vdso
> -
> -$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso
> -       $(call cmd,vdso_install)
> -
> -PHONY += vdso_install $(vdso_img_insttargets)
> -vdso_install: $(vdso_img_insttargets)
> -
>  clean-files := vdso32.so vdso32.so.dbg vdso64* vdso-image-*.c vdsox32.so*
> diff --git a/scripts/Makefile.vdsoinst b/scripts/Makefile.vdsoinst
> new file mode 100644
> index 000000000000..1022d9fdd976
> --- /dev/null
> +++ b/scripts/Makefile.vdsoinst
> @@ -0,0 +1,45 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +# ==========================================================================
> +# Install unstripped copies of vDSO
> +# ==========================================================================
> +
> +PHONY := __default
> +__default:
> +       @:
> +
> +include $(srctree)/scripts/Kbuild.include
> +
> +install-dir := $(MODLIB)/vdso
> +
> +define gen_install_rules
> +
> +src := $$(firstword $$(subst :,$(space),$(1)))
> +dest := $(install-dir)/$$(or $$(word 2,$$(subst :,$(space),$(1))),$$(patsubst %.dbg,%,$$(notdir $(1))))
> +
> +__default: $$(dest)
> +$$(dest): $$(src) FORCE
> +       $$(call cmd,install)
> +
> +# Some architectures create .build-id symlinks
> +ifneq ($(filter arm sparc x86, $(SRCARCH)),)
> +link := $(install-dir)/.build-id/$$(shell $(READELF) -n $$(src) | sed -n 's@^.*Build ID: \(..\)\(.*\)@\1/\2 at p')
> +
> +__default: $$(link)
> +$$(link): $$(dest) FORCE
> +       $$(call cmd,symlink)
> +endif
> +
> +endef
> +
> +$(foreach x, $(sort $(INSTALL_FILES)), $(eval $(call gen_install_rules,$(x))))
> +
> +quiet_cmd_install = INSTALL $@
> +      cmd_install = mkdir -p $(dir $@); cp $< $@
> +
> +quiet_cmd_symlink = SYMLINK $@
> +      cmd_symlink = mkdir -p $(dir $@); ln -sf --relative $< $@
> +
> +PHONY += FORCE
> +FORCE:
> +
> +.PHONY: $(PHONY)
> --
> 2.39.2
>


-- 
Best Regards
 Guo Ren


From masahiroy at kernel.org  Wed Oct 11 05:52:45 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Wed, 11 Oct 2023 21:52:45 +0900
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <CAJF2gTTDpGgzsiRk=q6FCdX_g5maY-sT9h0jiW=p6HLziq97yA@mail.gmail.com>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-4-masahiroy@kernel.org> <CAJF2gTTDpGgzsiRk=q6FCdX_g5maY-sT9h0jiW=p6HLziq97yA@mail.gmail.com>
Message-ID: <CAK7LNATmaSXQYFMZEw2vpn6td10+huck-vy-Rbo5Brys+j_Stg@mail.gmail.com>

On Wed, Oct 11, 2023 at 11:24?AM Guo Ren <guoren at kernel.org> wrote:
>
> On Mon, Oct 9, 2023 at 8:42?PM Masahiro Yamada <masahiroy at kernel.org> wrote:

> > --- a/arch/riscv/Makefile
> > +++ b/arch/riscv/Makefile
> > @@ -131,12 +131,6 @@ endif
> >  libs-y += arch/riscv/lib/
> >  libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
> >
> > -PHONY += vdso_install
> > -vdso_install:
> > -       $(Q)$(MAKE) $(build)=arch/riscv/kernel/vdso $@
> > -       $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
> > -               $(build)=arch/riscv/kernel/compat_vdso compat_$@)
> > -
> >  ifeq ($(KBUILD_EXTMOD),)
> >  ifeq ($(CONFIG_MMU),y)
> >  prepare: vdso_prepare
> > @@ -148,6 +142,9 @@ vdso_prepare: prepare0
> >  endif
> >  endif
> >
> > +vdso-install-y                 += arch/riscv/kernel/vdso/vdso.so.dbg
> > +vdso-install-$(CONFIG_COMPAT)  += arch/riscv/kernel/compat_vdso/compat_vdso.so.dbg:../compat_vdso/compat_vdso.so
> Why do we need ":../compat_vdso/compat_vdso.so" here?




All architectures except riscv install vdso files
to /lib/modules/$(uname -r)/vdso/.



See the following code in arch/riscv/kernel/compat_vdso/Makefile:


quiet_cmd_compat_vdso_install = INSTALL $@
      cmd_compat_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/compat_vdso/$@




Riscv copies the compat vdso to
/lib/modules/$(uname -r)/compat_vdso/.



This commit preserves the current installation path as-is.

If the riscv maintainers agree, we can change the
installation destination to /lib/modules/$(uname -r)/vdso/
for consistency.



-- 
Best Regards
Masahiro Yamada


From nicolas at fjasle.eu  Wed Oct 11 13:10:14 2023
From: nicolas at fjasle.eu (Nicolas Schier)
Date: Wed, 11 Oct 2023 22:10:14 +0200
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <20231009124210.1064021-4-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-4-masahiroy@kernel.org>
Message-ID: <ZScBJp-nen5wDRkT@bergen.fjasle.eu>

On Mon 09 Oct 2023 21:42:09 GMT, Masahiro Yamada wrote:
> Currently, there is no standard implementation for vdso_install,
> leading to various issues:
> 
>  1. Code duplication
> 
>     Many architectures duplicate similar code just for copying files
>     to the install destination.
> 
>     Some architectures (arm, sparc, x86) create build-id symlinks,
>     introducing more code duplication.
> 
>  2. Accidental updates of in-tree build artifacts
> 
>     The vdso_install rule depends on the vdso files to install.
>     It may update in-tree build artifacts. This can be problematic,
>     as explained in commit 19514fc665ff ("arm, kbuild: make
>     "make install" not depend on vmlinux").
> 
>  3. Broken code in some architectures
> 
>     Makefile code is often copied from one architecture to another
>     without proper adaptation or testing.
> 
>     The previous commits removed broken code from csky, UML, and parisc.
> 
>     Another issue is that 'make vdso_install' for ARCH=s390 installs
>     vdso64, but not vdso32.
> 
> To address these problems, this commit introduces the generic vdso_install.
> 
> Architectures that support vdso_install need to define vdso-install-y
> in arch/*/Makefile.
> 
> vdso-install-y lists the files to install. For example, arch/x86/Makefile
> looks like this:
> 
>   vdso-install-$(CONFIG_X86_64)           += arch/x86/entry/vdso/vdso64.so.dbg
>   vdso-install-$(CONFIG_X86_X32_ABI)      += arch/x86/entry/vdso/vdsox32.so.dbg
>   vdso-install-$(CONFIG_X86_32)           += arch/x86/entry/vdso/vdso32.so.dbg
>   vdso-install-$(CONFIG_IA32_EMULATION)   += arch/x86/entry/vdso/vdso32.so.dbg
> 
> These files will be installed to $(MODLIB)/vdso/ with the .dbg suffix,
> if exists, stripped away.
> 
> vdso-install-y can optionally take the second field after the colon
> separator. This is needed because some architectures install vdso
> files as a different base name.
> 
> The following is a snippet from arch/arm64/Makefile.
> 
>   vdso-install-$(CONFIG_COMPAT_VDSO)      += arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so
> 
> This will rename vdso.so.dbg to vdso32.so during installation. If such
> architectures change their implementation so that the file names match,
> this workaround will go away.
> 
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
> ---

Thanks for cleaning this up!

Reviewed-by: Nicolas Schier <nicolas at fjasle.eu>


From guoren at kernel.org  Wed Oct 11 23:21:04 2023
From: guoren at kernel.org (Guo Ren)
Date: Thu, 12 Oct 2023 14:21:04 +0800
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <CAK7LNATmaSXQYFMZEw2vpn6td10+huck-vy-Rbo5Brys+j_Stg@mail.gmail.com>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-4-masahiroy@kernel.org> <CAJF2gTTDpGgzsiRk=q6FCdX_g5maY-sT9h0jiW=p6HLziq97yA@mail.gmail.com>
 <CAK7LNATmaSXQYFMZEw2vpn6td10+huck-vy-Rbo5Brys+j_Stg@mail.gmail.com>
Message-ID: <CAJF2gTS8Gi+gSPhQcpNj2Yjjr0DZLtOy8S18Am8KtCqRbBLRhw@mail.gmail.com>

On Wed, Oct 11, 2023 at 8:53?PM Masahiro Yamada <masahiroy at kernel.org> wrote:
>
> On Wed, Oct 11, 2023 at 11:24?AM Guo Ren <guoren at kernel.org> wrote:
> >
> > On Mon, Oct 9, 2023 at 8:42?PM Masahiro Yamada <masahiroy at kernel.org> wrote:
>
> > > --- a/arch/riscv/Makefile
> > > +++ b/arch/riscv/Makefile
> > > @@ -131,12 +131,6 @@ endif
> > >  libs-y += arch/riscv/lib/
> > >  libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
> > >
> > > -PHONY += vdso_install
> > > -vdso_install:
> > > -       $(Q)$(MAKE) $(build)=arch/riscv/kernel/vdso $@
> > > -       $(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
> > > -               $(build)=arch/riscv/kernel/compat_vdso compat_$@)
> > > -
> > >  ifeq ($(KBUILD_EXTMOD),)
> > >  ifeq ($(CONFIG_MMU),y)
> > >  prepare: vdso_prepare
> > > @@ -148,6 +142,9 @@ vdso_prepare: prepare0
> > >  endif
> > >  endif
> > >
> > > +vdso-install-y                 += arch/riscv/kernel/vdso/vdso.so.dbg
> > > +vdso-install-$(CONFIG_COMPAT)  += arch/riscv/kernel/compat_vdso/compat_vdso.so.dbg:../compat_vdso/compat_vdso.so
> > Why do we need ":../compat_vdso/compat_vdso.so" here?
>
>
>
>
> All architectures except riscv install vdso files
> to /lib/modules/$(uname -r)/vdso/.
>
>
>
> See the following code in arch/riscv/kernel/compat_vdso/Makefile:
>
>
> quiet_cmd_compat_vdso_install = INSTALL $@
>       cmd_compat_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/compat_vdso/$@
>
>
>
>
> Riscv copies the compat vdso to
> /lib/modules/$(uname -r)/compat_vdso/.
>
>
>
> This commit preserves the current installation path as-is.
>
> If the riscv maintainers agree, we can change the
> installation destination to /lib/modules/$(uname -r)/vdso/
> for consistency.
Yes, but it should be another patch. Thx for the clarification.

Reviewed-by: Guo Ren <guoren at kernel.org>

>
>
>
> --
> Best Regards
> Masahiro Yamada



-- 
Best Regards
 Guo Ren


From deller at gmx.de  Thu Oct 12 00:24:36 2023
From: deller at gmx.de (Helge Deller)
Date: Thu, 12 Oct 2023 09:24:36 +0200
Subject: [PATCH 3/5] parisc: remove broken vdso_install
In-Reply-To: <20231009124210.1064021-3-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-3-masahiroy@kernel.org>
Message-ID: <d886d83a-87b4-428d-856e-9bbaa554563a@gmx.de>

Hi Masahiro,

On 10/9/23 14:42, Masahiro Yamada wrote:
> 'make ARCH=parisc vdso_install' has never worked. It attempts to
> descend into arch/parisc/kernel/vdso/, which does not exist.
>
> The command just fails:
>
>    scripts/Makefile.build:41: arch/parisc/kernel/vdso/Makefile: No such file or directory
>
> The second line is also meaningless because parisc does not define
> CONFIG_COMPAT_VDSO.
>
> It appears that this code was copied from another architecture without
> proper adaptation.

Yes.

> Remove the broken code.
>
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>

Thanks for cleaning this up and making it consistent across the architectures.

Acked-by: Helge Deller <deller at gmx.de>  # parisc

In case you do a v2 version of the patch, would you add to arch/parisc/Makefile
(otherwise I can send a follow-up patch in the parisc git tree):

vdso-install-y               += arch/parisc/kernel/vdso32/vdso32.so
vdso-install-$(CONFIG_64BIT) += arch/parisc/kernel/vdso64/vdso64.so

Thanks!
Helge


From hca at linux.ibm.com  Thu Oct 12 00:25:05 2023
From: hca at linux.ibm.com (Heiko Carstens)
Date: Thu, 12 Oct 2023 09:25:05 +0200
Subject: [PATCH mm-unstable v9 14/31] s390: Convert various pgalloc
 functions to use ptdescs
In-Reply-To: <20230807230513.102486-15-vishal.moola@gmail.com>
References: <20230807230513.102486-1-vishal.moola@gmail.com>
 <20230807230513.102486-15-vishal.moola@gmail.com>
Message-ID: <20231012072505.6160-A-hca@linux.ibm.com>

On Mon, Aug 07, 2023 at 04:04:56PM -0700, Vishal Moola (Oracle) wrote:
> As part of the conversions to replace pgtable constructor/destructors with
> ptdesc equivalents, convert various page table functions to use ptdescs.
> 
> Some of the functions use the *get*page*() helper functions. Convert
> these to use pagetable_alloc() and ptdesc_address() instead to help
> standardize page tables further.
> 
> Acked-by: Mike Rapoport (IBM) <rppt at kernel.org>
> Signed-off-by: Vishal Moola (Oracle) <vishal.moola at gmail.com>
> ---
>  arch/s390/include/asm/pgalloc.h |   4 +-
>  arch/s390/include/asm/tlb.h     |   4 +-
>  arch/s390/mm/pgalloc.c          | 128 ++++++++++++++++----------------
>  3 files changed, 69 insertions(+), 67 deletions(-)
...
> diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c
> index d7374add7820..07fc660a24aa 100644
> --- a/arch/s390/mm/pgalloc.c
> +++ b/arch/s390/mm/pgalloc.c
...
> @@ -488,16 +486,20 @@ static void base_pgt_free(unsigned long *table)
>  static unsigned long *base_crst_alloc(unsigned long val)
>  {
>  	unsigned long *table;
> +	struct ptdesc *ptdesc;
>  
> -	table =	(unsigned long *)__get_free_pages(GFP_KERNEL, CRST_ALLOC_ORDER);
> -	if (table)
> -		crst_table_init(table, val);
> +	ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, CRST_ALLOC_ORDER);

I guess I must miss something, but what is the reason to mask out
__GFP_HIGHMEM here? It is not part of GFP_KERNEL, nor does s390 support
HIGHMEM.


From vishal.moola at gmail.com  Fri Oct 13 14:03:08 2023
From: vishal.moola at gmail.com (Vishal Moola)
Date: Fri, 13 Oct 2023 14:03:08 -0700
Subject: [PATCH mm-unstable v9 14/31] s390: Convert various pgalloc
 functions to use ptdescs
In-Reply-To: <20231012072505.6160-A-hca@linux.ibm.com>
References: <20230807230513.102486-1-vishal.moola@gmail.com>
 <20230807230513.102486-15-vishal.moola@gmail.com> <20231012072505.6160-A-hca@linux.ibm.com>
Message-ID: <CAOzc2px-SFSnmjcPriiB3cm1fNj3+YC8S0VSp4t1QvDR0f4E2A@mail.gmail.com>

On Thu, Oct 12, 2023 at 12:25?AM Heiko Carstens <hca at linux.ibm.com> wrote:
>
> On Mon, Aug 07, 2023 at 04:04:56PM -0700, Vishal Moola (Oracle) wrote:
> > As part of the conversions to replace pgtable constructor/destructors with
> > ptdesc equivalents, convert various page table functions to use ptdescs.
> >
> > Some of the functions use the *get*page*() helper functions. Convert
> > these to use pagetable_alloc() and ptdesc_address() instead to help
> > standardize page tables further.
> >
> > Acked-by: Mike Rapoport (IBM) <rppt at kernel.org>
> > Signed-off-by: Vishal Moola (Oracle) <vishal.moola at gmail.com>
> > ---
> >  arch/s390/include/asm/pgalloc.h |   4 +-
> >  arch/s390/include/asm/tlb.h     |   4 +-
> >  arch/s390/mm/pgalloc.c          | 128 ++++++++++++++++----------------
> >  3 files changed, 69 insertions(+), 67 deletions(-)
> ...
> > diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c
> > index d7374add7820..07fc660a24aa 100644
> > --- a/arch/s390/mm/pgalloc.c
> > +++ b/arch/s390/mm/pgalloc.c
> ...
> > @@ -488,16 +486,20 @@ static void base_pgt_free(unsigned long *table)
> >  static unsigned long *base_crst_alloc(unsigned long val)
> >  {
> >       unsigned long *table;
> > +     struct ptdesc *ptdesc;
> >
> > -     table = (unsigned long *)__get_free_pages(GFP_KERNEL, CRST_ALLOC_ORDER);
> > -     if (table)
> > -             crst_table_init(table, val);
> > +     ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, CRST_ALLOC_ORDER);
>
> I guess I must miss something, but what is the reason to mask out
> __GFP_HIGHMEM here? It is not part of GFP_KERNEL, nor does s390 support
> HIGHMEM.

You're not missing anything.

This was replacing __get_free_pages() which also doesn't support HIGHMEM,
so I had that in to ensure a non-HIGHMEM allocation in case a
passed-in gfp_flags
had it set. In hindsight since we're just passing in the GFP flags
directly here, we don't
actually need to mask out GFP_HIGHMEM.


From masahiroy at kernel.org  Sat Oct 14 01:41:25 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Sat, 14 Oct 2023 17:41:25 +0900
Subject: [PATCH 5/5] kbuild: unify no-compiler-targets and no-sync-config-targets
In-Reply-To: <20231009164424.GB1153868@dev-arch.thelio-3990X>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-5-masahiroy@kernel.org> <20231009164424.GB1153868@dev-arch.thelio-3990X>
Message-ID: <CAK7LNASZ78NX13cfGiCeXGxB7A_2CasuAgNv-XzyWM79PD2NwA@mail.gmail.com>

On Tue, Oct 10, 2023 at 1:44?AM Nathan Chancellor <nathan at kernel.org> wrote:
>
> On Mon, Oct 09, 2023 at 09:42:10PM +0900, Masahiro Yamada wrote:
> > Now that vdso_install does not depend on any in-tree build artifact,
> > it no longer invokes a compiler, making no-compiler-targets the same
> > as no-sync-config-targets.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
> > ---
> >
> >  Makefile | 13 +------------
> >  1 file changed, 1 insertion(+), 12 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index 2170d56630e8..982b1ad33287 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -277,10 +277,6 @@ no-dot-config-targets := $(clean-targets) \
> >                        $(version_h) headers headers_% archheaders archscripts \
> >                        %asm-generic kernelversion %src-pkg dt_binding_check \
> >                        outputmakefile rustavailable rustfmt rustfmtcheck
> > -# Installation targets should not require compiler. Unfortunately, vdso_install
> > -# is an exception where build artifacts may be updated. This must be fixed.
> > -no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
> > -                     headers_install modules_install modules_sign kernelrelease image_name
> >  no-sync-config-targets := $(no-dot-config-targets) %install modules_sign kernelrelease \
> >                         image_name
> >  single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %/
> > @@ -288,7 +284,6 @@ single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %
> >  config-build :=
> >  mixed-build  :=
> >  need-config  := 1
> > -need-compiler        := 1
> >  may-sync-config      := 1
> >  single-build :=
> >
> > @@ -298,12 +293,6 @@ ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
> >       endif
> >  endif
> >
> > -ifneq ($(filter $(no-compiler-targets), $(MAKECMDGOALS)),)
> > -     ifeq ($(filter-out $(no-compiler-targets), $(MAKECMDGOALS)),)
> > -             need-compiler :=
> > -     endif
> > -endif
> > -
>
> MIPS and LoongArch seem to have grown a usage of need-compiler in
> 4fe4a6374c4d ("MIPS: Only fiddle with CHECKFLAGS if `need-compiler'")
> and 54c2c9df083f ("LoongArch: Only fiddle with CHECKFLAGS if
> `need-compiler'"). With this removal, should those be updated as well?


Right, but may-sync-config and need-compiler are not
interchangeable due to the following code.

ifneq ($(KBUILD_EXTMOD),)
        may-sync-config :=
endif


I will keep both.


-- 
Best Regards
Masahiro Yamada


From masahiroy at kernel.org  Sat Oct 14 02:08:40 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Sat, 14 Oct 2023 18:08:40 +0900
Subject: [PATCH 1/5] csky: remove unused cmd_vdso_install
In-Reply-To: <CAJF2gTQxiCEyGiGieHJMBK0Hmj49f8mdcm1TwsTZbgbvxVxGcw@mail.gmail.com>
References: <20231009124210.1064021-1-masahiroy@kernel.org> <CAJF2gTQxiCEyGiGieHJMBK0Hmj49f8mdcm1TwsTZbgbvxVxGcw@mail.gmail.com>
Message-ID: <CAK7LNAQPupf+iMNxs7gNAAm15Y1Eg8eXuZTWwVai23heZE8FjA@mail.gmail.com>

On Tue, Oct 10, 2023 at 12:16?AM Guo Ren <guoren at kernel.org> wrote:
>
> On Mon, Oct 9, 2023 at 8:42?PM Masahiro Yamada <masahiroy at kernel.org> wrote:
> >
> > You cannot run this code because arch/csky/Makefile does not define the
> > vdso_install target.
> >
> > It appears that this code was blindly copied from another architecture.
> Yes, I do that. Thx for pointing it out.
>
> Acked-by: Guo Ren <guoren at kernel.org>


Applied to linux-kbuild.




-- 
Best Regards
Masahiro Yamada


From masahiroy at kernel.org  Sat Oct 14 03:54:36 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Sat, 14 Oct 2023 19:54:36 +0900
Subject: [PATCH v2 2/2] kbuild: unify no-compiler-targets and no-sync-config-targets
In-Reply-To: <20231014105436.2119702-1-masahiroy@kernel.org>
References: <20231014105436.2119702-1-masahiroy@kernel.org>
Message-ID: <20231014105436.2119702-2-masahiroy@kernel.org>

Now that vdso_install does not depend on any in-tree build artifact,
it no longer needs a compiler, making no-compiler-targets the same
as no-sync-config-targets.

Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
---

Changes in v2:
  - Revive need-compiler flag

 Makefile | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/Makefile b/Makefile
index 1ae5d5180f20..fed9a6cc3665 100644
--- a/Makefile
+++ b/Makefile
@@ -277,10 +277,6 @@ no-dot-config-targets := $(clean-targets) \
 			 $(version_h) headers headers_% archheaders archscripts \
 			 %asm-generic kernelversion %src-pkg dt_binding_check \
 			 outputmakefile rustavailable rustfmt rustfmtcheck
-# Installation targets should not require compiler. Unfortunately, vdso_install
-# is an exception where build artifacts may be updated. This must be fixed.
-no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
-			headers_install modules_install modules_sign kernelrelease image_name
 no-sync-config-targets := $(no-dot-config-targets) %install modules_sign kernelrelease \
 			  image_name
 single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %/
@@ -288,7 +284,6 @@ single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.rsi %.s %.symtypes %
 config-build	:=
 mixed-build	:=
 need-config	:= 1
-need-compiler	:= 1
 may-sync-config	:= 1
 single-build	:=
 
@@ -298,18 +293,14 @@ ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
 	endif
 endif
 
-ifneq ($(filter $(no-compiler-targets), $(MAKECMDGOALS)),)
-	ifeq ($(filter-out $(no-compiler-targets), $(MAKECMDGOALS)),)
-		need-compiler :=
-	endif
-endif
-
 ifneq ($(filter $(no-sync-config-targets), $(MAKECMDGOALS)),)
 	ifeq ($(filter-out $(no-sync-config-targets), $(MAKECMDGOALS)),)
 		may-sync-config :=
 	endif
 endif
 
+need-compiler := $(may-sync-config)
+
 ifneq ($(KBUILD_EXTMOD),)
 	may-sync-config :=
 endif
-- 
2.39.2



From masahiroy at kernel.org  Sat Oct 14 03:54:35 2023
From: masahiroy at kernel.org (Masahiro Yamada)
Date: Sat, 14 Oct 2023 19:54:35 +0900
Subject: [PATCH v2 1/2] kbuild: unify vdso_install rules
Message-ID: <20231014105436.2119702-1-masahiroy@kernel.org>

Currently, there is no standard implementation for vdso_install,
leading to various issues:

 1. Code duplication

    Many architectures duplicate similar code just for copying files
    to the install destination.

    Some architectures (arm, sparc, x86) create build-id symlinks,
    introducing more code duplication.

 2. Unintended updates of in-tree build artifacts

    The vdso_install rule depends on the vdso files to install.
    It may update in-tree build artifacts. This can be problematic,
    as explained in commit 19514fc665ff ("arm, kbuild: make
    "make install" not depend on vmlinux").

 3. Broken code in some architectures

    Makefile code is often copied from one architecture to another
    without proper adaptation.

    'make vdso_install' for parisc does not work.

    'make vdso_install' for s390 installs vdso64, but not vdso32.

To address these problems, this commit introduces a generic vdso_install
rule.

Architectures that support vdso_install need to define vdso-install-y
in arch/*/Makefile.

vdso-install-y lists the files to install. For example, arch/x86/Makefile
looks like this:

  vdso-install-$(CONFIG_X86_64)           += arch/x86/entry/vdso/vdso64.so.dbg
  vdso-install-$(CONFIG_X86_X32_ABI)      += arch/x86/entry/vdso/vdsox32.so.dbg
  vdso-install-$(CONFIG_X86_32)           += arch/x86/entry/vdso/vdso32.so.dbg
  vdso-install-$(CONFIG_IA32_EMULATION)   += arch/x86/entry/vdso/vdso32.so.dbg

These files will be installed to $(MODLIB)/vdso/ with the .dbg suffix,
if exists, stripped away.

vdso-install-y can optionally take the second field after the colon
separator. This is needed because some architectures install a vdso
file as a different base name.

The following is a snippet from arch/arm64/Makefile.

  vdso-install-$(CONFIG_COMPAT_VDSO)      += arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so

This will rename vdso.so.dbg to vdso32.so during installation. If such
architectures change their implementation so that the base names match,
this workaround will go away.

Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
Acked-by: Sven Schnelle <svens at linux.ibm.com> # s390
Reviewed-by: Nicolas Schier <nicolas at fjasle.eu>
Reviewed-by: Guo Ren <guoren at kernel.org>
Acked-by: Helge Deller <deller at gmx.de>  # parisc
---

Changes in v2:
  - Add vdso-install-y for parisc

 Makefile                               |  9 ++++++
 arch/arm/Makefile                      |  7 +---
 arch/arm/vdso/Makefile                 | 25 --------------
 arch/arm64/Makefile                    |  9 ++----
 arch/arm64/kernel/vdso/Makefile        | 10 ------
 arch/arm64/kernel/vdso32/Makefile      | 10 ------
 arch/loongarch/Makefile                |  4 +--
 arch/loongarch/vdso/Makefile           | 10 ------
 arch/parisc/Makefile                   |  8 ++---
 arch/riscv/Makefile                    |  9 ++----
 arch/riscv/kernel/compat_vdso/Makefile | 10 ------
 arch/riscv/kernel/vdso/Makefile        | 10 ------
 arch/s390/Makefile                     |  6 ++--
 arch/s390/kernel/vdso32/Makefile       | 10 ------
 arch/s390/kernel/vdso64/Makefile       | 10 ------
 arch/sparc/Makefile                    |  5 ++-
 arch/sparc/vdso/Makefile               | 27 ----------------
 arch/x86/Makefile                      |  7 ++--
 arch/x86/entry/vdso/Makefile           | 27 ----------------
 scripts/Makefile.vdsoinst              | 45 ++++++++++++++++++++++++++
 20 files changed, 73 insertions(+), 185 deletions(-)
 create mode 100644 scripts/Makefile.vdsoinst

diff --git a/Makefile b/Makefile
index 3de6dd959bd1..1ae5d5180f20 100644
--- a/Makefile
+++ b/Makefile
@@ -1317,6 +1317,14 @@ scripts_unifdef: scripts_basic
 quiet_cmd_install = INSTALL $(INSTALL_PATH)
       cmd_install = unset sub_make_done; $(srctree)/scripts/install.sh
 
+# ---------------------------------------------------------------------------
+# vDSO install
+
+PHONY += vdso_install
+vdso_install: export INSTALL_FILES = $(vdso-install-y)
+vdso_install:
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vdsoinst
+
 # ---------------------------------------------------------------------------
 # Tools
 
@@ -1560,6 +1568,7 @@ help:
 	@echo  '* vmlinux	  - Build the bare kernel'
 	@echo  '* modules	  - Build all modules'
 	@echo  '  modules_install - Install all modules to INSTALL_MOD_PATH (default: /)'
+	@echo  '  vdso_install    - Install unstripped vdso to INSTALL_MOD_PATH (default: /)'
 	@echo  '  dir/            - Build all files in dir and below'
 	@echo  '  dir/file.[ois]  - Build specified target only'
 	@echo  '  dir/file.ll     - Build the LLVM assembly file'
diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 547e5856eaa0..5ba42f69f8ce 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -304,11 +304,7 @@ $(INSTALL_TARGETS): KBUILD_IMAGE = $(boot)/$(patsubst %install,%Image,$@)
 $(INSTALL_TARGETS):
 	$(call cmd,install)
 
-PHONY += vdso_install
-vdso_install:
-ifeq ($(CONFIG_VDSO),y)
-	$(Q)$(MAKE) $(build)=arch/arm/vdso $@
-endif
+vdso-install-$(CONFIG_VDSO) += arch/arm/vdso/vdso.so.dbg
 
 # My testing targets (bypasses dependencies)
 bp:;	$(Q)$(MAKE) $(build)=$(boot) $(boot)/bootpImage
@@ -331,7 +327,6 @@ define archhelp
   echo  '                  Install using (your) ~/bin/$(INSTALLKERNEL) or'
   echo  '                  (distribution) /sbin/$(INSTALLKERNEL) or'
   echo  '                  install to $$(INSTALL_PATH) and run lilo'
-  echo  '  vdso_install  - Install unstripped vdso.so to $$(INSTALL_MOD_PATH)/vdso'
   echo
   echo  '  multi_v7_lpae_defconfig     - multi_v7_defconfig with CONFIG_ARM_LPAE enabled'
 endef
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index 515ca33b854c..d761bd2e2f40 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -63,28 +63,3 @@ quiet_cmd_vdsold_and_vdso_check = LD      $@
 
 quiet_cmd_vdsomunge = MUNGE   $@
       cmd_vdsomunge = $(objtree)/$(obj)/vdsomunge $< $@
-
-#
-# Install the unstripped copy of vdso.so.dbg.  If our toolchain
-# supports build-id, install .build-id links as well.
-#
-# Cribbed from arch/x86/vdso/Makefile.
-#
-quiet_cmd_vdso_install = INSTALL $<
-define cmd_vdso_install
-	cp $< "$(MODLIB)/vdso/vdso.so"; \
-	if readelf -n $< | grep -q 'Build ID'; then \
-	  buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
-	  first=`echo $$buildid | cut -b-2`; \
-	  last=`echo $$buildid | cut -b3-`; \
-	  mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
-	  ln -sf "../../vdso.so" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
-	fi
-endef
-
-$(MODLIB)/vdso: FORCE
-	@mkdir -p $(MODLIB)/vdso
-
-PHONY += vdso_install
-vdso_install: $(obj)/vdso.so.dbg $(MODLIB)/vdso
-	$(call cmd,vdso_install)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 2d49aea0ff67..4bd85cc0d32b 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -169,12 +169,6 @@ install: KBUILD_IMAGE := $(boot)/Image
 install zinstall:
 	$(call cmd,install)
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso $@
-	$(if $(CONFIG_COMPAT_VDSO), \
-		$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso32 $@)
-
 archprepare:
 	$(Q)$(MAKE) $(build)=arch/arm64/tools kapi
 ifeq ($(CONFIG_ARM64_ERRATUM_843419),y)
@@ -205,6 +199,9 @@ ifdef CONFIG_COMPAT_VDSO
 endif
 endif
 
+vdso-install-y				+= arch/arm64/kernel/vdso/vdso.so.dbg
+vdso-install-$(CONFIG_COMPAT_VDSO)	+= arch/arm64/kernel/vdso32/vdso.so.dbg:vdso32.so
+
 include $(srctree)/scripts/Makefile.defconf
 
 PHONY += virtconfig
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index fe7a53c6781f..8818287f1095 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -78,13 +78,3 @@ include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
 # Actual build commands
 quiet_cmd_vdsold_and_vdso_check = LD      $@
       cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check)
-
-# Install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
index 2f73e5bca213..1f911a76c5af 100644
--- a/arch/arm64/kernel/vdso32/Makefile
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -172,13 +172,3 @@ gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
 quiet_cmd_vdsosym = VDSOSYM $@
 # The AArch64 nm should be able to read an AArch32 binary
       cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
-
-# Install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL32 $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile
index fb0fada43197..b86f2ff31659 100644
--- a/arch/loongarch/Makefile
+++ b/arch/loongarch/Makefile
@@ -136,9 +136,7 @@ vdso_prepare: prepare0
 	$(Q)$(MAKE) $(build)=arch/loongarch/vdso include/generated/vdso-offsets.h
 endif
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/loongarch/vdso $@
+vdso-install-y += arch/loongarch/vdso/vdso.so.dbg
 
 all:	$(notdir $(KBUILD_IMAGE))
 
diff --git a/arch/loongarch/vdso/Makefile b/arch/loongarch/vdso/Makefile
index 5c97d1463328..c74c9921304f 100644
--- a/arch/loongarch/vdso/Makefile
+++ b/arch/loongarch/vdso/Makefile
@@ -83,13 +83,3 @@ $(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
 obj-y += vdso.o
 
 $(obj)/vdso.o : $(obj)/vdso.so
-
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/parisc/Makefile b/arch/parisc/Makefile
index 968ebe17494c..920db57b6b4c 100644
--- a/arch/parisc/Makefile
+++ b/arch/parisc/Makefile
@@ -177,12 +177,8 @@ vdso_prepare: prepare0
 	$(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso32 include/generated/vdso32-offsets.h
 endif
 
-PHONY += vdso_install
-
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso $@
-	$(if $(CONFIG_COMPAT_VDSO), \
-		$(Q)$(MAKE) $(build)=arch/parisc/kernel/vdso32 $@)
+vdso-install-y			+= arch/parisc/kernel/vdso32/vdso32.so
+vdso-install-$(CONFIG_64BIT)	+= arch/parisc/kernel/vdso64/vdso64.so
 
 install: KBUILD_IMAGE := vmlinux
 zinstall: KBUILD_IMAGE := vmlinuz
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 1329e060c548..18a47b4d6795 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -131,12 +131,6 @@ endif
 libs-y += arch/riscv/lib/
 libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/riscv/kernel/vdso $@
-	$(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
-		$(build)=arch/riscv/kernel/compat_vdso compat_$@)
-
 ifeq ($(KBUILD_EXTMOD),)
 ifeq ($(CONFIG_MMU),y)
 prepare: vdso_prepare
@@ -148,6 +142,9 @@ vdso_prepare: prepare0
 endif
 endif
 
+vdso-install-y			+= arch/riscv/kernel/vdso/vdso.so.dbg
+vdso-install-$(CONFIG_COMPAT)	+= arch/riscv/kernel/compat_vdso/compat_vdso.so.dbg:../compat_vdso/compat_vdso.so
+
 ifneq ($(CONFIG_XIP_KERNEL),y)
 ifeq ($(CONFIG_RISCV_M_MODE)$(CONFIG_ARCH_CANAAN),yy)
 KBUILD_IMAGE := $(boot)/loader.bin
diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile
index b86e5e2c3aea..62fa393b2eb2 100644
--- a/arch/riscv/kernel/compat_vdso/Makefile
+++ b/arch/riscv/kernel/compat_vdso/Makefile
@@ -76,13 +76,3 @@ quiet_cmd_compat_vdsold = VDSOLD  $@
 # actual build commands
 quiet_cmd_compat_vdsoas = VDSOAS $@
       cmd_compat_vdsoas = $(COMPAT_CC) $(a_flags) $(COMPAT_CC_FLAGS) -c -o $@ $<
-
-# install commands for the unstripped file
-quiet_cmd_compat_vdso_install = INSTALL $@
-      cmd_compat_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/compat_vdso/$@
-
-compat_vdso.so: $(obj)/compat_vdso.so.dbg
-	@mkdir -p $(MODLIB)/compat_vdso
-	$(call cmd,compat_vdso_install)
-
-compat_vdso_install: compat_vdso.so
diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile
index 6b1dba11bf6d..e8aa7c380007 100644
--- a/arch/riscv/kernel/vdso/Makefile
+++ b/arch/riscv/kernel/vdso/Makefile
@@ -73,13 +73,3 @@ quiet_cmd_vdsold = VDSOLD  $@
       cmd_vdsold = $(LD) $(ld_flags) -T $(filter-out FORCE,$^) -o $@.tmp && \
                    $(OBJCOPY) $(patsubst %, -G __vdso_%, $(vdso-syms)) $@.tmp $@ && \
                    rm $@.tmp
-
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso.so: $(obj)/vdso.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso.so
diff --git a/arch/s390/Makefile b/arch/s390/Makefile
index a53a36ee0731..73873e451686 100644
--- a/arch/s390/Makefile
+++ b/arch/s390/Makefile
@@ -138,9 +138,6 @@ bzImage: vmlinux
 zfcpdump:
 	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
 
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/$(ARCH)/kernel/vdso64 $@
-
 archheaders:
 	$(Q)$(MAKE) $(build)=$(syscalls) uapi
 
@@ -160,6 +157,9 @@ vdso_prepare: prepare0
 	$(if $(CONFIG_COMPAT),$(Q)$(MAKE) \
 		$(build)=arch/s390/kernel/vdso32 include/generated/vdso32-offsets.h)
 
+vdso-install-y			+= arch/s390/kernel/vdso64/vdso64.so.dbg
+vdso-install-$(CONFIG_COMPAT)	+= arch/s390/kernel/vdso32/vdso32.so.dbg
+
 ifdef CONFIG_EXPOLINE_EXTERN
 modules_prepare: expoline_prepare
 expoline_prepare: scripts
diff --git a/arch/s390/kernel/vdso32/Makefile b/arch/s390/kernel/vdso32/Makefile
index 23e868b79a6c..caec7db6f966 100644
--- a/arch/s390/kernel/vdso32/Makefile
+++ b/arch/s390/kernel/vdso32/Makefile
@@ -61,16 +61,6 @@ quiet_cmd_vdso32as = VDSO32A $@
 quiet_cmd_vdso32cc = VDSO32C $@
       cmd_vdso32cc = $(CC) $(c_flags) -c -o $@ $<
 
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso32.so: $(obj)/vdso32.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso32.so
-
 # Generate VDSO offsets using helper script
 gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
 quiet_cmd_vdsosym = VDSOSYM $@
diff --git a/arch/s390/kernel/vdso64/Makefile b/arch/s390/kernel/vdso64/Makefile
index fc1c6ff8178f..e3c9085f8fa7 100644
--- a/arch/s390/kernel/vdso64/Makefile
+++ b/arch/s390/kernel/vdso64/Makefile
@@ -70,16 +70,6 @@ quiet_cmd_vdso64as = VDSO64A $@
 quiet_cmd_vdso64cc = VDSO64C $@
       cmd_vdso64cc = $(CC) $(c_flags) -c -o $@ $<
 
-# install commands for the unstripped file
-quiet_cmd_vdso_install = INSTALL $@
-      cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
-
-vdso64.so: $(obj)/vdso64.so.dbg
-	@mkdir -p $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-vdso_install: vdso64.so
-
 # Generate VDSO offsets using helper script
 gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
 quiet_cmd_vdsosym = VDSOSYM $@
diff --git a/arch/sparc/Makefile b/arch/sparc/Makefile
index 7417345c6639..5f6035936131 100644
--- a/arch/sparc/Makefile
+++ b/arch/sparc/Makefile
@@ -76,9 +76,8 @@ install:
 archheaders:
 	$(Q)$(MAKE) $(build)=arch/sparc/kernel/syscalls all
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/sparc/vdso $@
+vdso-install-$(CONFIG_SPARC64)	+= arch/sparc/vdso/vdso64.so.dbg
+vdso-install-$(CONFIG_COMPAT)	+= arch/sparc/vdso/vdso32.so.dbg
 
 # This is the image used for packaging
 KBUILD_IMAGE := $(boot)/zImage
diff --git a/arch/sparc/vdso/Makefile b/arch/sparc/vdso/Makefile
index 77d7b9032158..d08c3a0443f3 100644
--- a/arch/sparc/vdso/Makefile
+++ b/arch/sparc/vdso/Makefile
@@ -116,30 +116,3 @@ quiet_cmd_vdso = VDSO    $@
 
 VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 -Bsymbolic
 GCOV_PROFILE := n
-
-#
-# Install the unstripped copies of vdso*.so.  If our toolchain supports
-# build-id, install .build-id links as well.
-#
-quiet_cmd_vdso_install = INSTALL $(@:install_%=%)
-define cmd_vdso_install
-	cp $< "$(MODLIB)/vdso/$(@:install_%=%)"; \
-	if readelf -n $< |grep -q 'Build ID'; then \
-	  buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
-	  first=`echo $$buildid | cut -b-2`; \
-	  last=`echo $$buildid | cut -b3-`; \
-	  mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
-	  ln -sf "../../$(@:install_%=%)" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
-	fi
-endef
-
-vdso_img_insttargets := $(vdso_img_sodbg:%.dbg=install_%)
-
-$(MODLIB)/vdso: FORCE
-	@mkdir -p $(MODLIB)/vdso
-
-$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso FORCE
-	$(call cmd,vdso_install)
-
-PHONY += vdso_install $(vdso_img_insttargets)
-vdso_install: $(vdso_img_insttargets) FORCE
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 5bfe5caaa444..3ff53a2d4ff0 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -291,9 +291,10 @@ PHONY += install
 install:
 	$(call cmd,install)
 
-PHONY += vdso_install
-vdso_install:
-	$(Q)$(MAKE) $(build)=arch/x86/entry/vdso $@
+vdso-install-$(CONFIG_X86_64)		+= arch/x86/entry/vdso/vdso64.so.dbg
+vdso-install-$(CONFIG_X86_X32_ABI)	+= arch/x86/entry/vdso/vdsox32.so.dbg
+vdso-install-$(CONFIG_X86_32)		+= arch/x86/entry/vdso/vdso32.so.dbg
+vdso-install-$(CONFIG_IA32_EMULATION)	+= arch/x86/entry/vdso/vdso32.so.dbg
 
 archprepare: checkbin
 checkbin:
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 6a1821bd7d5e..c197efd82922 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -190,31 +190,4 @@ GCOV_PROFILE := n
 quiet_cmd_vdso_and_check = VDSO    $@
       cmd_vdso_and_check = $(cmd_vdso); $(cmd_vdso_check)
 
-#
-# Install the unstripped copies of vdso*.so.  If our toolchain supports
-# build-id, install .build-id links as well.
-#
-quiet_cmd_vdso_install = INSTALL $(@:install_%=%)
-define cmd_vdso_install
-	cp $< "$(MODLIB)/vdso/$(@:install_%=%)"; \
-	if readelf -n $< |grep -q 'Build ID'; then \
-	  buildid=`readelf -n $< |grep 'Build ID' |sed -e 's/^.*Build ID: \(.*\)$$/\1/'`; \
-	  first=`echo $$buildid | cut -b-2`; \
-	  last=`echo $$buildid | cut -b3-`; \
-	  mkdir -p "$(MODLIB)/vdso/.build-id/$$first"; \
-	  ln -sf "../../$(@:install_%=%)" "$(MODLIB)/vdso/.build-id/$$first/$$last.debug"; \
-	fi
-endef
-
-vdso_img_insttargets := $(vdso_img_sodbg:%.dbg=install_%)
-
-$(MODLIB)/vdso: FORCE
-	@mkdir -p $(MODLIB)/vdso
-
-$(vdso_img_insttargets): install_%: $(obj)/%.dbg $(MODLIB)/vdso
-	$(call cmd,vdso_install)
-
-PHONY += vdso_install $(vdso_img_insttargets)
-vdso_install: $(vdso_img_insttargets)
-
 clean-files := vdso32.so vdso32.so.dbg vdso64* vdso-image-*.c vdsox32.so*
diff --git a/scripts/Makefile.vdsoinst b/scripts/Makefile.vdsoinst
new file mode 100644
index 000000000000..1022d9fdd976
--- /dev/null
+++ b/scripts/Makefile.vdsoinst
@@ -0,0 +1,45 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# ==========================================================================
+# Install unstripped copies of vDSO
+# ==========================================================================
+
+PHONY := __default
+__default:
+	@:
+
+include $(srctree)/scripts/Kbuild.include
+
+install-dir := $(MODLIB)/vdso
+
+define gen_install_rules
+
+src := $$(firstword $$(subst :,$(space),$(1)))
+dest := $(install-dir)/$$(or $$(word 2,$$(subst :,$(space),$(1))),$$(patsubst %.dbg,%,$$(notdir $(1))))
+
+__default: $$(dest)
+$$(dest): $$(src) FORCE
+	$$(call cmd,install)
+
+# Some architectures create .build-id symlinks
+ifneq ($(filter arm sparc x86, $(SRCARCH)),)
+link := $(install-dir)/.build-id/$$(shell $(READELF) -n $$(src) | sed -n 's@^.*Build ID: \(..\)\(.*\)@\1/\2 at p')
+
+__default: $$(link)
+$$(link): $$(dest) FORCE
+	$$(call cmd,symlink)
+endif
+
+endef
+
+$(foreach x, $(sort $(INSTALL_FILES)), $(eval $(call gen_install_rules,$(x))))
+
+quiet_cmd_install = INSTALL $@
+      cmd_install = mkdir -p $(dir $@); cp $< $@
+
+quiet_cmd_symlink = SYMLINK $@
+      cmd_symlink = mkdir -p $(dir $@); ln -sf --relative $< $@
+
+PHONY += FORCE
+FORCE:
+
+.PHONY: $(PHONY)
-- 
2.39.2



From mpe at ellerman.id.au  Sun Oct 15 03:04:58 2023
From: mpe at ellerman.id.au (Michael Ellerman)
Date: Sun, 15 Oct 2023 21:04:58 +1100
Subject: (subset) [PATCH 0/7] arch/*: config: Remove ReiserFS from defconfig
In-Reply-To: <20230918175529.19011-1-peter@n8pjl.ca>
References: <20230918175529.19011-1-peter@n8pjl.ca>
Message-ID: <169736429854.960528.1442206910501555108.b4-ty@ellerman.id.au>

On Mon, 18 Sep 2023 17:56:09 +0000, Peter Lafreniere wrote:
> ReiserFS has been considered deprecated for 19 months since commit
> eb103a51640e ("reiserfs: Deprecate reiserfs"). However, there are
> several architectures that still build it into their defconfig kernels.
> 
> As ReiserFS will be removed in 2025, delete all ReiserFS-related options
> from defconfig files before the filesystem's removal.
> 
> [...]

Applied to powerpc/next.

[2/7] arch: powerpc: remove ReiserFS from defconfig
      https://git.kernel.org/powerpc/c/c945e6f453a361b0e9daddd2be9c099d1b80d6f8

cheers


From anton.ivanov at cambridgegreys.com  Tue Oct 17 01:32:01 2023
From: anton.ivanov at cambridgegreys.com (anton.ivanov at cambridgegreys.com)
Date: Tue, 17 Oct 2023 09:32:01 +0100
Subject: [PATCH] um: document arch_futex_atomic_op_inuser
In-Reply-To: <202310151047.a07DrQNV-lkp@intel.com>
References: <202310151047.a07DrQNV-lkp@intel.com>
Message-ID: <20231017083201.399828-1-anton.ivanov@cambridgegreys.com>

From: Anton Ivanov <anton.ivanov at cambridgegreys.com>

arch_futex_atomic_op_inuser was not documented correctly
resulting in build time warnings.

Signed-off-by: Anton Ivanov <anton.ivanov at cambridgegreys.com>
---
 arch/um/kernel/skas/uaccess.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/um/kernel/skas/uaccess.c b/arch/um/kernel/skas/uaccess.c
index aaee96f07172..198269e384c4 100644
--- a/arch/um/kernel/skas/uaccess.c
+++ b/arch/um/kernel/skas/uaccess.c
@@ -236,7 +236,9 @@ EXPORT_SYMBOL(strnlen_user);
  *			  argument and comparison of the previous
  *			  futex value with another constant.
  *
- * @encoded_op:	encoded operation to execute
+ * @op:		operation to execute
+ * @oparg:	argument to operation
+ * @oval:	old value at uaddr
  * @uaddr:	pointer to user space address
  *
  * Return:
-- 
2.39.2



From n.schier at avm.de  Tue Oct 17 04:57:54 2023
From: n.schier at avm.de (Nicolas Schier)
Date: Tue, 17 Oct 2023 13:57:54 +0200
Subject: [PATCH v2 2/2] kbuild: unify no-compiler-targets and
 no-sync-config-targets
In-Reply-To: <20231014105436.2119702-2-masahiroy@kernel.org>
References: <20231014105436.2119702-1-masahiroy@kernel.org>
 <20231014105436.2119702-2-masahiroy@kernel.org>
Message-ID: <ZS52wrL9meW8iehZ@buildd.core.avm.de>

On Sat, Oct 14, 2023 at 07:54:36PM +0900, Masahiro Yamada wrote:
> Now that vdso_install does not depend on any in-tree build artifact,
> it no longer needs a compiler, making no-compiler-targets the same
> as no-sync-config-targets.
> 
> Signed-off-by: Masahiro Yamada <masahiroy at kernel.org>
> ---
> 
> Changes in v2:
>   - Revive need-compiler flag
> 

Reviewed-by: Nicolas Schier <nicolas at fjasle.eu>


From brauner at kernel.org  Mon Oct  9 09:09:31 2023
From: brauner at kernel.org (Christian Brauner)
Date: Mon,  9 Oct 2023 18:09:31 +0200
Subject: [PATCH v2 00/89] fs: new accessor methods for inode atime and mtime
In-Reply-To: <20231004185221.80802-1-jlayton@kernel.org>
Message-ID: <20231009-charmant-locken-893b11849e84@brauner>

On Wed, Oct 04, 2023 at 02:52:21PM -0400, Jeff Layton wrote:
> v2:
> - bugfix in mtime handling
> - incorporate _sec and _nsec accessor functions (Chuck Lever)
> - move i_generation to plug hole after changing timestamps (Amir Goldstein)
> 
> While working on the multigrain timestamp changes, Linus suggested
> adding some similar wrappers for accessing the atime and mtime that we
> have for the ctime. With that, we could then move to using discrete
> integers instead of struct timespec64 in struct inode and shrink it.
> 
> This patch implements this. Linus suggested using macros for the new
> accessors, but the existing ctime wrappers were static inlines and since
> there are only 3 different timestamps, I didn't see that trying to
> fiddle with macros would gain us anything (other than less verbosity in
> fs.h).
> 
> [...]

This was applied on top of -next but vfs.ctime is now based on v6.6-rc3
as stable tag otherwise this is too much of a moving target with other
stuff in -next. Anything that had to be dropped and requires fixups
should just be explained in the pr. The sooner this sees some -next, the
better, I think.

---

Applied to the vfs.ctime branch of the vfs/vfs.git tree.
Patches in the vfs.ctime branch should appear in linux-next soon.

Please report any outstanding bugs that were missed during review in a
new review to the original patch series allowing us to drop it.

It's encouraged to provide Acked-bys and Reviewed-bys even though the
patch has now been applied. If possible patch trailers will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. If in doubt, please check the listed branch.

tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs.ctime

[01/86] fs: new accessor methods for atime and mtime
        https://git.kernel.org/vfs/vfs/c/22f45fee808d
[02/86] fs: convert core infrastructure to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/6ac95fb71485
[03/86] spufs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/9953073d5f20
[04/86] hypfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/1d64bfe22112
[05/86] android: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/a8a74b6b4f2c
[06/86] char: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/671ffa0775a7
[07/86] qib: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/ebd5458f3b52
[08/86] ibmasm: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/1d4257d57a41
[09/86] misc: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/d4bf8378b9cb
[10/86] x86: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/070601b1e496
[11/86] tty: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/5c9f26b87bed
[12/86] function: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/092f46404245
[13/86] legacy: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/5c51d80e51d0
[14/86] usb: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/4707a33afd6f
[15/86] 9p: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/20fc454b4493
[16/86] adfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/3e8d59046f6d
[17/86] affs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/60d4d0d37086
[18/86] afs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/6471772aa6fe
[19/86] autofs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/3eaad981548b
[20/86] befs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/21d0433caf69
[21/86] bfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/06e502c123a6
[22/86] btrfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/f62049d7838d
[23/86] ceph: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/ac7750d84e38
[24/86] coda: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/5c4bf2507baa
[25/86] configfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/3b930e187f16
[26/86] cramfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/bb0bf9d3bda8
[27/86] debugfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/7dc950e659d6
[28/86] devpts: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/a1eb5c26d5a1
[29/86] efivarfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/17b5652aa824
[30/86] efs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/a3cfbea29e7d
[31/86] erofs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/2beccde96d66
[32/86] exfat: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/522f3c42c9e7
[33/86] ext2: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/07be81fce412
[34/86] ext4: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/2ff285d78c4d
[35/86] f2fs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/c495130561ae
[36/86] fat: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/e57260ae3226
[37/86] freevxfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/a83513cd029e
[38/86] fuse: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/5f1e57582b4e
[39/86] gfs2: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/a5f1a9296668
[40/86] hfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/7ee8d53576e9
[41/86] hfsplus: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/2179ad3569f6
[42/86] hostfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/c3e1be490207
[43/86] hpfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/e08a2ea26b41
[44/86] hugetlbfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/a9701db0ca64
[45/86] isofs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/362d327da07e
[46/86] jffs2: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/36a8a5a63218
[47/86] jfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/acd529413de5
[48/86] kernfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/05acde68936b
[49/86] minix: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/34c1ca111ec1
[50/86] nfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/77e808456854
[51/86] nfsd: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/a800ed7ebbbf
[52/86] nilfs2: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/4ddc9518c2fa
[53/86] ntfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/0d15c2118b1a
[54/86] ntfs3: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/82f8d5fde753
[55/86] ocfs2: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/45251ebaca70
[56/86] omfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/e7c1ff814326
[57/86] openpromfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/cb62db1d3c61
[58/86] orangefs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/68e257a49aed
[59/86] overlayfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/d482d98dc1bd
[60/86] proc: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/8c8afe8a25fa
[61/86] pstore: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/b0be548328a2
[62/86] qnx4: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/c28589f2d838
[63/86] qnx6: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/ae0f3d29e728
[64/86] ramfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/6315fd97a8fc
[65/86] reiserfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/8eceb9b75a5b
[66/86] romfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/6d3dd456da31
[67/86] client: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/b14d4c14f51b
[68/86] server: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/bec3d7ffcecd
[69/86] squashfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/d7d5ff75af52
[70/86] sysv: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/39f012d8743e
[71/86] tracefs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/22ada3856de8
[72/86] ubifs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/af8b66e1d4b7
[73/86] udf: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/1da45142f95a
[74/86] ufs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/41c46d3bb9b3
[75/86] vboxsf: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/cc36ec7935eb
[76/86] xfs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/ee3be90b2ba7
[77/86] zonefs: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/8c798cc16b17
[78/86] linux: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/dd53b64b6f51
[79/86] ipc: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/f132b3723b71
[80/86] bpf: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/448a018f67a3
[81/86] mm: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/b6f5b3d5ffc9
[82/86] sunrpc: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/fc9db028b8d7
[83/86] apparmor: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/794ef2a745ec
[84/86] selinux: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/3d57ee3686d7
[85/86] security: convert to new timestamp accessors
        https://git.kernel.org/vfs/vfs/c/71f6d9ebaf43
[86/86] fs: rename inode i_atime and i_mtime fields
        https://git.kernel.org/vfs/vfs/c/fea0e8fc7829


From benjamin at sipsolutions.net  Wed Oct 18 05:36:40 2023
From: benjamin at sipsolutions.net (benjamin at sipsolutions.net)
Date: Wed, 18 Oct 2023 14:36:40 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking signals
Message-ID: <20231018123643.1255813-1-benjamin@sipsolutions.net>

From: Benjamin Berg <benjamin.berg at intel.com>

When in time-travel mode, the eventfd events are read even when signals
are blocked as SIGIO still needs to be processed. In this case, the
event is cleared on the eventfd but the IRQ still needs to be fired
later.

We did already ensure that the SIGIO handler is run again. However, the
FDs are configured to be level triggered, so that eventfd will not
notify again. As such, add some logic to mark the IRQ as pending and
process it at the next opportunity.

To avoid duplication, reuse the logic used for the suspend/resume case.
This does not really change anything except for delaying running the
IRQs with timetravel_handler at a slightly later point in time (and
possibly running non-timetravel IRQs that shouldn't happen earlier).
While at it, move marking as pending into irq_event_handler as that is
the more logical place for it to happen.

Signed-off-by: Benjamin Berg <benjamin.berg at intel.com>
---
 arch/um/kernel/irq.c | 78 ++++++++++++++++++++++++++++----------------
 1 file changed, 49 insertions(+), 29 deletions(-)

diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 635d44606bfe..ceda4bd2e5ed 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -37,7 +37,7 @@ struct irq_reg {
 	bool pending;
 	bool wakeup;
 #ifdef CONFIG_UML_TIME_TRAVEL_SUPPORT
-	bool pending_on_resume;
+	bool pending_event;
 	void (*timetravel_handler)(int, int, void *,
 				   struct time_travel_event *);
 	struct time_travel_event event;
@@ -56,6 +56,9 @@ static DEFINE_SPINLOCK(irq_lock);
 static LIST_HEAD(active_fds);
 static DECLARE_BITMAP(irqs_allocated, UM_LAST_SIGNAL_IRQ);
 static bool irqs_suspended;
+#ifdef CONFIG_UML_TIME_TRAVEL_SUPPORT
+static bool irqs_pending;
+#endif
 
 static void irq_io_loop(struct irq_reg *irq, struct uml_pt_regs *regs)
 {
@@ -84,9 +87,12 @@ static void irq_event_handler(struct time_travel_event *ev)
 {
 	struct irq_reg *reg = container_of(ev, struct irq_reg, event);
 
-	/* do nothing if suspended - just to cause a wakeup */
-	if (irqs_suspended)
+	/* do nothing if suspended; just cause a wakeup and mark as pending */
+	if (irqs_suspended) {
+		irqs_pending = true;
+		reg->pending_event = true;
 		return;
+	}
 
 	generic_handle_irq(reg->irq);
 }
@@ -110,16 +116,47 @@ static bool irq_do_timetravel_handler(struct irq_entry *entry,
 	if (!reg->event.pending)
 		return false;
 
-	if (irqs_suspended)
-		reg->pending_on_resume = true;
 	return true;
 }
+
+static void irq_do_pending_events(bool timetravel_handlers_only)
+{
+	struct irq_entry *entry;
+
+	if (!irqs_pending || timetravel_handlers_only)
+		return;
+
+	irqs_pending = false;
+
+	list_for_each_entry(entry, &active_fds, list) {
+		enum um_irq_type t;
+
+		for (t = 0; t < NUM_IRQ_TYPES; t++) {
+			struct irq_reg *reg = &entry->reg[t];
+
+			/*
+			 * Any timetravel_handler was invoked already, just
+			 * directly run the IRQ.
+			 */
+			if (reg->pending_event) {
+				irq_enter();
+				generic_handle_irq(reg->irq);
+				irq_exit();
+				reg->pending_event = false;
+			}
+		}
+	}
+}
 #else
 static bool irq_do_timetravel_handler(struct irq_entry *entry,
 				      enum um_irq_type t)
 {
 	return false;
 }
+
+static void irq_do_pending_events(bool timetravel_handlers_only)
+{
+}
 #endif
 
 static void sigio_reg_handler(int idx, struct irq_entry *entry, enum um_irq_type t,
@@ -145,6 +182,8 @@ static void sigio_reg_handler(int idx, struct irq_entry *entry, enum um_irq_type
 	 */
 	if (timetravel_handlers_only) {
 #ifdef CONFIG_UML_TIME_TRAVEL_SUPPORT
+		reg->pending_event = true;
+		irqs_pending = true;
 		mark_sigio_pending();
 #endif
 		return;
@@ -162,6 +201,10 @@ static void _sigio_handler(struct uml_pt_regs *regs,
 	if (timetravel_handlers_only && !um_irq_timetravel_handler_used())
 		return;
 
+	/* Flush out pending events that were ignored due to time-travel. */
+	if (!irqs_suspended)
+		irq_do_pending_events(timetravel_handlers_only);
+
 	while (1) {
 		/* This is now lockless - epoll keeps back-referencesto the irqs
 		 * which have trigger it so there is no need to walk the irq
@@ -543,30 +586,7 @@ void um_irqs_resume(void)
 	unsigned long flags;
 
 
-	local_irq_save(flags);
-#ifdef CONFIG_UML_TIME_TRAVEL_SUPPORT
-	/*
-	 * We don't need to lock anything here since we're in resume
-	 * and nothing else is running, but have disabled IRQs so we
-	 * don't try anything else with the interrupt list from there.
-	 */
-	list_for_each_entry(entry, &active_fds, list) {
-		enum um_irq_type t;
-
-		for (t = 0; t < NUM_IRQ_TYPES; t++) {
-			struct irq_reg *reg = &entry->reg[t];
-
-			if (reg->pending_on_resume) {
-				irq_enter();
-				generic_handle_irq(reg->irq);
-				irq_exit();
-				reg->pending_on_resume = false;
-			}
-		}
-	}
-#endif
-
-	spin_lock(&irq_lock);
+	spin_lock_irqsave(&irq_lock, flags);
 	list_for_each_entry(entry, &active_fds, list) {
 		if (entry->suspended) {
 			int err = os_set_fd_async(entry->fd);
-- 
2.41.0



From benjamin at sipsolutions.net  Wed Oct 18 05:36:42 2023
From: benjamin at sipsolutions.net (benjamin at sipsolutions.net)
Date: Wed, 18 Oct 2023 14:36:42 +0200
Subject: [PATCH 3/4] um: chan_user: retry partial writes
In-Reply-To: <20231018123643.1255813-1-benjamin@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
Message-ID: <20231018123643.1255813-3-benjamin@sipsolutions.net>

From: Benjamin Berg <benjamin.berg at intel.com>

In the next commit, we are going to set the output FD to be blocking.
Once that is done, the write() may be short if an interrupt happens
while it is writing out data. As such, to properly catch an EINTR error,
we need to retry the write.

Signed-off-by: Benjamin Berg <benjamin.berg at intel.com>
---
 arch/um/drivers/chan_user.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/arch/um/drivers/chan_user.c b/arch/um/drivers/chan_user.c
index 29c68820522a..339aae56b6f1 100644
--- a/arch/um/drivers/chan_user.c
+++ b/arch/um/drivers/chan_user.c
@@ -37,11 +37,23 @@ int generic_read(int fd, char *c_out, void *unused)
 
 int generic_write(int fd, const char *buf, int n, void *unused)
 {
+	int written = 0;
 	int err;
 
-	CATCH_EINTR(err = write(fd, buf, n));
-	if (err > 0)
-		return err;
+	/* The FD may be in blocking mode, as such, need to retry short writes,
+	 * they may have been interrupted by a signal.
+	 */
+	do {
+		errno = 0;
+		err = write(fd, buf + written, n - written);
+		if (err > 0) {
+			written += err;
+			continue;
+		}
+	} while (err < 0 && errno == EINTR);
+
+	if (written > 0)
+		return written;
 	else if (errno == EAGAIN)
 		return 0;
 	else if (err == 0)
-- 
2.41.0



From benjamin at sipsolutions.net  Wed Oct 18 05:36:41 2023
From: benjamin at sipsolutions.net (benjamin at sipsolutions.net)
Date: Wed, 18 Oct 2023 14:36:41 +0200
Subject: [PATCH 2/4] um: chan_user: catch EINTR when reading and writing
In-Reply-To: <20231018123643.1255813-1-benjamin@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
Message-ID: <20231018123643.1255813-2-benjamin@sipsolutions.net>

From: Benjamin Berg <benjamin.berg at intel.com>

If the read/write function returns an error then we expect to see an
event/IRQ later on. However, this will only happen after an EAGAIN as we
are using edge based event triggering.

As such, EINTR needs to be caught should it happen.

Signed-off-by: Benjamin Berg <benjamin.berg at intel.com>
---
 arch/um/drivers/chan_user.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/um/drivers/chan_user.c b/arch/um/drivers/chan_user.c
index 25727ed648b7..29c68820522a 100644
--- a/arch/um/drivers/chan_user.c
+++ b/arch/um/drivers/chan_user.c
@@ -23,7 +23,7 @@ int generic_read(int fd, char *c_out, void *unused)
 {
 	int n;
 
-	n = read(fd, c_out, sizeof(*c_out));
+	CATCH_EINTR(n = read(fd, c_out, sizeof(*c_out)));
 	if (n > 0)
 		return n;
 	else if (n == 0)
@@ -39,7 +39,7 @@ int generic_write(int fd, const char *buf, int n, void *unused)
 {
 	int err;
 
-	err = write(fd, buf, n);
+	CATCH_EINTR(err = write(fd, buf, n));
 	if (err > 0)
 		return err;
 	else if (errno == EAGAIN)
-- 
2.41.0



From benjamin at sipsolutions.net  Wed Oct 18 05:36:43 2023
From: benjamin at sipsolutions.net (benjamin at sipsolutions.net)
Date: Wed, 18 Oct 2023 14:36:43 +0200
Subject: [PATCH 4/4] um: chan: use blocking IO for console output for time-travel
In-Reply-To: <20231018123643.1255813-1-benjamin@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
Message-ID: <20231018123643.1255813-4-benjamin@sipsolutions.net>

From: Benjamin Berg <benjamin.berg at intel.com>

When in time-travel mode (infinite-cpu or external) time should not pass
for writing to the console. As such, it makes sense to put the FD for
the output side into blocking mode and simply let any write to it hang.

If we did not do this, then time could pass waiting for the console to
become writable again. This is not desirable as it has random effects on
the clock between runs.

Implement this by duplicating the FD if output is active in a relevant
mode and setting the duplicate to be blocking. This avoids changing the
input channel to be blocking should it exists. After this, use the
blocking FD for all write operations and do not allocate an IRQ it is
set.

Without time-travel mode fd_out will always match fd_in and IRQs are
registered.

Signed-off-by: Benjamin Berg <benjamin.berg at intel.com>
---
 arch/um/drivers/chan.h      |  3 +-
 arch/um/drivers/chan_kern.c | 81 ++++++++++++++++++++++++++++---------
 arch/um/include/shared/os.h |  1 +
 arch/um/os-Linux/file.c     | 10 +++++
 4 files changed, 74 insertions(+), 21 deletions(-)

diff --git a/arch/um/drivers/chan.h b/arch/um/drivers/chan.h
index 3fec3b8406e9..1bb62f2d7716 100644
--- a/arch/um/drivers/chan.h
+++ b/arch/um/drivers/chan.h
@@ -22,7 +22,8 @@ struct chan {
 	unsigned int output:1;
 	unsigned int opened:1;
 	unsigned int enabled:1;
-	int fd;
+	int fd_in;
+	int fd_out; /* only different to fd_in if blocking output is needed */
 	const struct chan_ops *ops;
 	void *data;
 };
diff --git a/arch/um/drivers/chan_kern.c b/arch/um/drivers/chan_kern.c
index 26a702a06515..1eff939c7ebf 100644
--- a/arch/um/drivers/chan_kern.c
+++ b/arch/um/drivers/chan_kern.c
@@ -81,6 +81,12 @@ static const struct chan_ops not_configged_ops = {
 };
 #endif /* CONFIG_NOCONFIG_CHAN */
 
+static inline bool need_output_blocking(void)
+{
+	return time_travel_mode == TT_MODE_INFCPU ||
+	       time_travel_mode == TT_MODE_EXTERNAL;
+}
+
 static int open_one_chan(struct chan *chan)
 {
 	int fd, err;
@@ -96,15 +102,43 @@ static int open_one_chan(struct chan *chan)
 		return fd;
 
 	err = os_set_fd_block(fd, 0);
-	if (err) {
-		(*chan->ops->close)(fd, chan->data);
-		return err;
-	}
+	if (err)
+		goto out_close;
+
+	chan->fd_in = fd;
+	chan->fd_out = fd;
+
+	/*
+	 * In time-travel modes infinite-CPU and external we need to guarantee
+	 * that any writes to the output succeed immdiately from the point of
+	 * the VM. The best way to do this is to put the FD in blocking mode
+	 * and simply wait/retry until everything is written.
+	 * As every write is guaranteed to complete, we also do not need to
+	 * request an IRQ for the output.
+	 *
+	 * Note that input cannot happen in a time synchronized way. We permit
+	 * it, but time passes very quickly if anything waits for a read.
+	 */
+	if (chan->output && need_output_blocking()) {
+		err = os_dup_file(chan->fd_out);
+		if (err < 0)
+			goto out_close;
 
-	chan->fd = fd;
+		chan->fd_out = err;
+
+		err = os_set_fd_block(chan->fd_out, 1);
+		if (err) {
+			os_close_file(chan->fd_out);
+			goto out_close;
+		}
+	}
 
 	chan->opened = 1;
 	return 0;
+
+out_close:
+	(*chan->ops->close)(fd, chan->data);
+	return err;
 }
 
 static int open_chan(struct list_head *chans)
@@ -125,7 +159,7 @@ static int open_chan(struct list_head *chans)
 void chan_enable_winch(struct chan *chan, struct tty_port *port)
 {
 	if (chan && chan->primary && chan->ops->winch)
-		register_winch(chan->fd, port);
+		register_winch(chan->fd_in, port);
 }
 
 static void line_timer_cb(struct work_struct *work)
@@ -156,8 +190,9 @@ int enable_chan(struct line *line)
 
 		if (chan->enabled)
 			continue;
-		err = line_setup_irq(chan->fd, chan->input, chan->output, line,
-				     chan);
+		err = line_setup_irq(chan->fd_in, chan->input,
+				     chan->output && !need_output_blocking(),
+				     line, chan);
 		if (err)
 			goto out_close;
 
@@ -196,7 +231,8 @@ void free_irqs(void)
 
 		if (chan->input && chan->enabled)
 			um_free_irq(chan->line->read_irq, chan);
-		if (chan->output && chan->enabled)
+		if (chan->output && chan->enabled &&
+		    !need_output_blocking())
 			um_free_irq(chan->line->write_irq, chan);
 		chan->enabled = 0;
 	}
@@ -216,15 +252,19 @@ static void close_one_chan(struct chan *chan, int delay_free_irq)
 	} else {
 		if (chan->input && chan->enabled)
 			um_free_irq(chan->line->read_irq, chan);
-		if (chan->output && chan->enabled)
+		if (chan->output && chan->enabled &&
+		    !need_output_blocking())
 			um_free_irq(chan->line->write_irq, chan);
 		chan->enabled = 0;
 	}
+	if (chan->fd_out != chan->fd_in)
+		os_close_file(chan->fd_out);
 	if (chan->ops->close != NULL)
-		(*chan->ops->close)(chan->fd, chan->data);
+		(*chan->ops->close)(chan->fd_in, chan->data);
 
 	chan->opened = 0;
-	chan->fd = -1;
+	chan->fd_in = -1;
+	chan->fd_out = -1;
 }
 
 void close_chan(struct line *line)
@@ -244,7 +284,7 @@ void close_chan(struct line *line)
 void deactivate_chan(struct chan *chan, int irq)
 {
 	if (chan && chan->enabled)
-		deactivate_fd(chan->fd, irq);
+		deactivate_fd(chan->fd_in, irq);
 }
 
 int write_chan(struct chan *chan, const char *buf, int len,
@@ -255,7 +295,7 @@ int write_chan(struct chan *chan, const char *buf, int len,
 	if (len == 0 || !chan || !chan->ops->write)
 		return 0;
 
-	n = chan->ops->write(chan->fd, buf, len, chan->data);
+	n = chan->ops->write(chan->fd_out, buf, len, chan->data);
 	if (chan->primary) {
 		ret = n;
 	}
@@ -269,7 +309,7 @@ int console_write_chan(struct chan *chan, const char *buf, int len)
 	if (!chan || !chan->ops->console_write)
 		return 0;
 
-	n = chan->ops->console_write(chan->fd, buf, len);
+	n = chan->ops->console_write(chan->fd_out, buf, len);
 	if (chan->primary)
 		ret = n;
 	return ret;
@@ -297,14 +337,14 @@ int chan_window_size(struct line *line, unsigned short *rows_out,
 	if (chan && chan->primary) {
 		if (chan->ops->window_size == NULL)
 			return 0;
-		return chan->ops->window_size(chan->fd, chan->data,
+		return chan->ops->window_size(chan->fd_in, chan->data,
 					      rows_out, cols_out);
 	}
 	chan = line->chan_out;
 	if (chan && chan->primary) {
 		if (chan->ops->window_size == NULL)
 			return 0;
-		return chan->ops->window_size(chan->fd, chan->data,
+		return chan->ops->window_size(chan->fd_in, chan->data,
 					      rows_out, cols_out);
 	}
 	return 0;
@@ -320,7 +360,7 @@ static void free_one_chan(struct chan *chan)
 		(*chan->ops->free)(chan->data);
 
 	if (chan->primary && chan->output)
-		ignore_sigio_fd(chan->fd);
+		ignore_sigio_fd(chan->fd_in);
 	kfree(chan);
 }
 
@@ -479,7 +519,8 @@ static struct chan *parse_chan(struct line *line, char *str, int device,
 				 .output 	= 0,
 				 .opened  	= 0,
 				 .enabled  	= 0,
-				 .fd 		= -1,
+				 .fd_in		= -1,
+				 .fd_out	= -1,
 				 .ops 		= ops,
 				 .data 		= data });
 	return chan;
@@ -550,7 +591,7 @@ void chan_interrupt(struct line *line, int irq)
 			schedule_delayed_work(&line->task, 1);
 			goto out;
 		}
-		err = chan->ops->read(chan->fd, &c, chan->data);
+		err = chan->ops->read(chan->fd_in, &c, chan->data);
 		if (err > 0)
 			tty_insert_flip_char(port, c, TTY_NORMAL);
 	} while (err > 0);
diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index 0df646c6651e..269373e16397 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -163,6 +163,7 @@ extern int os_set_fd_block(int fd, int blocking);
 extern int os_accept_connection(int fd);
 extern int os_create_unix_socket(const char *file, int len, int close_on_exec);
 extern int os_shutdown_socket(int fd, int r, int w);
+extern int os_dup_file(int fd);
 extern void os_close_file(int fd);
 extern int os_rcv_fd(int fd, int *helper_pid_out);
 extern int os_connect_socket(const char *name);
diff --git a/arch/um/os-Linux/file.c b/arch/um/os-Linux/file.c
index fc4450db59bd..abf8676b834c 100644
--- a/arch/um/os-Linux/file.c
+++ b/arch/um/os-Linux/file.c
@@ -240,6 +240,16 @@ int os_connect_socket(const char *name)
 	return err;
 }
 
+int os_dup_file(int fd)
+{
+	int new_fd = dup(fd);
+
+	if (new_fd < 0)
+		return -errno;
+
+	return new_fd;
+}
+
 void os_close_file(int fd)
 {
 	close(fd);
-- 
2.41.0



From Benjamin.Beichler at uni-rostock.de  Fri Oct 20 02:15:15 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 20 Oct 2023 11:15:15 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <20231018123643.1255813-1-benjamin@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
Message-ID: <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>

Am 18.10.2023 um 14:36 schrieb benjamin at sipsolutions.net:
> From: Benjamin Berg <benjamin.berg at intel.com>
> 
> When in time-travel mode, the eventfd events are read even when signals
> are blocked as SIGIO still needs to be processed. In this case, the
> event is cleared on the eventfd but the IRQ still needs to be fired
> later.
> 
> We did already ensure that the SIGIO handler is run again. However, the
> FDs are configured to be level triggered, so that eventfd will not
> notify again. As such, add some logic to mark the IRQ as pending and
> process it at the next opportunity.
> 
> To avoid duplication, reuse the logic used for the suspend/resume case.
> This does not really change anything except for delaying running the
> IRQs with timetravel_handler at a slightly later point in time (and
> possibly running non-timetravel IRQs that shouldn't happen earlier).
> While at it, move marking as pending into irq_event_handler as that is
> the more logical place for it to happen.
> 
> Signed-off-by: Benjamin Berg <benjamin.berg at intel.com>

I also noticed this problem, but after a discussion with Johannes Berg 
about this, we come to the conclusion, that all drivers with interrupts 
need to deal with time travel mode via their own time travel handler. 
What I actually did, was to write a trivial handler for e.g., the serial 
line, which simply only call time_travel_add_irq_event(ev).

I'm not entirely sure, what the right way to do it, although the outcome 
seems to be the same, as with no time advance the actual simulation 
time, when the interrupt is handled is the same.

I actually also have some other significant bug fixes on time travel in 
my tree, but I was too lazy to send them here, are you interested in 
taking a look at them?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x1569BBF90AC3918A.asc
Type: application/pgp-keys
Size: 2976 bytes
Desc: OpenPGP public key
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231020/c3cc64bc/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231020/c3cc64bc/attachment.sig>

From michal.winiarski at intel.com  Fri Oct 20 02:21:57 2023
From: michal.winiarski at intel.com (=?UTF-8?q?Micha=C5=82=20Winiarski?=)
Date: Fri, 20 Oct 2023 11:21:57 +0200
Subject: [PATCH 0/2] um: kunit: Add Clang support for CONFIG_GCOV
Message-ID: <20231020092159.2486063-1-michal.winiarski@intel.com>

Clang uses a different set of CLI args for coverage, and the output
needs to be processed by a different set of tools.
Update the Makefile and add an example of usage in kunit docs.

Micha? Winiarski (2):
  arch: um: Add Clang coverage support
  Documentation: kunit: Add clang UML coverage example

 Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++
 arch/um/Makefile-skas                          |  5 +++++
 2 files changed, 16 insertions(+)

-- 
2.42.0



From michal.winiarski at intel.com  Fri Oct 20 02:21:58 2023
From: michal.winiarski at intel.com (=?UTF-8?q?Micha=C5=82=20Winiarski?=)
Date: Fri, 20 Oct 2023 11:21:58 +0200
Subject: [PATCH 1/2] arch: um: Add Clang coverage support
In-Reply-To: <20231020092159.2486063-1-michal.winiarski@intel.com>
References: <20231020092159.2486063-1-michal.winiarski@intel.com>
Message-ID: <20231020092159.2486063-2-michal.winiarski@intel.com>

Clang uses a different set of command line arguments for enabling
coverage.

Signed-off-by: Micha? Winiarski <michal.winiarski at intel.com>
---
 arch/um/Makefile-skas | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/um/Makefile-skas b/arch/um/Makefile-skas
index ac35de5316a6..67323b028999 100644
--- a/arch/um/Makefile-skas
+++ b/arch/um/Makefile-skas
@@ -4,7 +4,12 @@
 #
 
 GPROF_OPT += -pg
+
+ifdef CONFIG_CC_IS_CLANG
+GCOV_OPT += -fprofile-instr-generate -fcoverage-mapping
+else
 GCOV_OPT += -fprofile-arcs -ftest-coverage
+endif
 
 CFLAGS-$(CONFIG_GCOV) += $(GCOV_OPT)
 CFLAGS-$(CONFIG_GPROF) += $(GPROF_OPT)
-- 
2.42.0



From michal.winiarski at intel.com  Fri Oct 20 02:21:59 2023
From: michal.winiarski at intel.com (=?UTF-8?q?Micha=C5=82=20Winiarski?=)
Date: Fri, 20 Oct 2023 11:21:59 +0200
Subject: [PATCH 2/2] Documentation: kunit: Add clang UML coverage example
In-Reply-To: <20231020092159.2486063-1-michal.winiarski@intel.com>
References: <20231020092159.2486063-1-michal.winiarski@intel.com>
Message-ID: <20231020092159.2486063-3-michal.winiarski@intel.com>

LLVM-based toolchain is using a different set of tools for coverage.
Add an example that produces output in lcov format.

Signed-off-by: Micha? Winiarski <michal.winiarski at intel.com>
---
 Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst
index 766f9cdea0fa..9f69c122dee7 100644
--- a/Documentation/dev-tools/kunit/running_tips.rst
+++ b/Documentation/dev-tools/kunit/running_tips.rst
@@ -139,6 +139,17 @@ If your installed version of gcc doesn't work, you can tweak the steps:
 	$ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
 	$ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6
 
+Alternatively, LLVM-based toolchain can also be used:
+
+.. code-block:: bash
+
+	# Build with LLVM and append coverage options to the current config
+	$ $ ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --kunitconfig=.kunit/ --kunitconfig=tools/testing/kunit/configs/coverage_uml.config
+	$ llvm-profdata merge -sparse default.profraw -o default.profdata
+	$ llvm-cov export --format=lcov .kunit/vmlinux -instr-profile default.profdata > coverage.info
+	# The coverage.info file is in lcov-compatible format and it can be used to e.g. generate HTML report
+	$ genhtml -o /tmp/coverage_html coverage.info
+
 
 Running tests manually
 ======================
-- 
2.42.0



From anton.ivanov at kot-begemot.co.uk  Fri Oct 20 02:26:21 2023
From: anton.ivanov at kot-begemot.co.uk (Anton Ivanov)
Date: Fri, 20 Oct 2023 10:26:21 +0100
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
Message-ID: <7f172f42-38bf-c2be-15b4-0710f50b372e@kot-begemot.co.uk>



On 20/10/2023 10:15, Benjamin Beichler wrote:
> Am 18.10.2023 um 14:36 schrieb benjamin at sipsolutions.net:
>> From: Benjamin Berg <benjamin.berg at intel.com>
>>
>> When in time-travel mode, the eventfd events are read even when signals
>> are blocked as SIGIO still needs to be processed. In this case, the
>> event is cleared on the eventfd but the IRQ still needs to be fired
>> later.
>>
>> We did already ensure that the SIGIO handler is run again. However, the
>> FDs are configured to be level triggered, so that eventfd will not
>> notify again. As such, add some logic to mark the IRQ as pending and
>> process it at the next opportunity.
>>
>> To avoid duplication, reuse the logic used for the suspend/resume case.
>> This does not really change anything except for delaying running the
>> IRQs with timetravel_handler at a slightly later point in time (and
>> possibly running non-timetravel IRQs that shouldn't happen earlier).
>> While at it, move marking as pending into irq_event_handler as that is
>> the more logical place for it to happen.
>>
>> Signed-off-by: Benjamin Berg <benjamin.berg at intel.com>
> 
> I also noticed this problem, but after a discussion with Johannes Berg about this, we come to the conclusion, that all drivers with interrupts need to deal with time travel mode via their own time travel handler. What I actually did, was to write a trivial handler for e.g., the serial line, which simply only call time_travel_add_irq_event(ev).
> 
> I'm not entirely sure, what the right way to do it, although the outcome seems to be the same, as with no time advance the actual simulation time, when the interrupt is handled is the same.

Add an extra registration flag for um_register_irq?


enum um_irq_type {
	IRQ_READ,
	IRQ_WRITE,
	IRQ_HAVE_TT_HANDLER,
	NUM_IRQ_TYPES,
};

Ones that do not register get the default one which advances the time. Ones that do - have it left to them and do with time whatever they need.

> 
> I actually also have some other significant bug fixes on time travel in my tree, but I was too lazy to send them here, are you interested in taking a look at them?
> 
> _______________________________________________
> linux-um mailing list
> linux-um at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um

-- 
Anton R. Ivanov
https://www.kot-begemot.co.uk/


From benjamin at sipsolutions.net  Fri Oct 20 02:59:51 2023
From: benjamin at sipsolutions.net (Benjamin Berg)
Date: Fri, 20 Oct 2023 11:59:51 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
	 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
Message-ID: <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>

On Fri, 2023-10-20 at 11:15 +0200, Benjamin Beichler wrote:
> Am 18.10.2023 um 14:36 schrieb benjamin at sipsolutions.net:
> > From: Benjamin Berg <benjamin.berg at intel.com>
> > 
> > When in time-travel mode, the eventfd events are read even when signals
> > are blocked as SIGIO still needs to be processed. In this case, the
> > event is cleared on the eventfd but the IRQ still needs to be fired
> > later.
> > 
> > We did already ensure that the SIGIO handler is run again. However, the
> > FDs are configured to be level triggered, so that eventfd will not
> > notify again. As such, add some logic to mark the IRQ as pending and
> > process it at the next opportunity.
> > 
> > To avoid duplication, reuse the logic used for the suspend/resume case.
> > This does not really change anything except for delaying running the
> > IRQs with timetravel_handler at a slightly later point in time (and
> > possibly running non-timetravel IRQs that shouldn't happen earlier).
> > While at it, move marking as pending into irq_event_handler as that is
> > the more logical place for it to happen.
> > 
> > Signed-off-by: Benjamin Berg <benjamin.berg at intel.com>
> 
> I also noticed this problem, but after a discussion with Johannes Berg 
> about this, we come to the conclusion, that all drivers with interrupts 
> need to deal with time travel mode via their own time travel handler.
> What I actually did, was to write a trivial handler for e.g., the serial 
> line, which simply only call time_travel_add_irq_event(ev).

Yes, we could instead just assert that we do not have any IRQ without a
timetravel handler. We really shouldn't have, but we do have one for
stdin unfortunately (which is at least used by the hostap hwsim tests,
so we might want to not remove it until we have a different solution).

What we could do is just add an stdin timetravel handler as you have
done, but print a warning when the stdin IRQ is registered[1]. And
then, if someone tries to register an IRQ without a time-travel handler
we just panic(), it should never happen.

Benjamin

[1] We'll also need to set stdin to be a "null" channel and then
somehow avoid registering the IRQ in that case.

> I'm not entirely sure, what the right way to do it, although the outcome 
> seems to be the same, as with no time advance the actual simulation 
> time, when the interrupt is handled is the same.
> 
> I actually also have some other significant bug fixes on time travel in 
> my tree, but I was too lazy to send them here, are you interested in 
> taking a look at them?
> _______________________________________________
> linux-um mailing list
> linux-um at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um



From benjamin.beichler at uni-rostock.de  Fri Oct 20 03:33:42 2023
From: benjamin.beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 20 Oct 2023 12:33:42 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <7f172f42-38bf-c2be-15b4-0710f50b372e@kot-begemot.co.uk>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
 <7f172f42-38bf-c2be-15b4-0710f50b372e@kot-begemot.co.uk>
Message-ID: <c6c035d9-06fc-4250-ac64-0e3d07289917@uni-rostock.de>

Am 20.10.2023 um 11:26 schrieb Anton Ivanov:

>> I also noticed this problem, but after a discussion with Johannes Berg 
>> about this, we come to the conclusion, that all drivers with 
>> interrupts need to deal with time travel mode via their own time 
>> travel handler. What I actually did, was to write a trivial handler 
>> for e.g., the serial line, which simply only call 
>> time_travel_add_irq_event(ev).
>>
>> I'm not entirely sure, what the right way to do it, although the 
>> outcome seems to be the same, as with no time advance the actual 
>> simulation time, when the interrupt is handled is the same.
> 
> Add an extra registration flag for um_register_irq?
> 
> 
> enum um_irq_type {
>  ????IRQ_READ,
>  ????IRQ_WRITE,
>  ????IRQ_HAVE_TT_HANDLER,
>  ????NUM_IRQ_TYPES,
> };
> 
> Ones that do not register get the default one which advances the time. 
> Ones that do - have it left to them and do with time whatever they need.
> 
There is already mechanics for it, as we have um_request_irq_tt and 
um_request_irq. I don't think an additional type enhance the semantics.


>>
>> I actually also have some other significant bug fixes on time travel 
>> in my tree, but I was too lazy to send them here, are you interested 
>> in taking a look at them?
>>



From Benjamin.Beichler at uni-rostock.de  Fri Oct 20 03:38:52 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 20 Oct 2023 12:38:52 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
Message-ID: <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>

Am 20.10.2023 um 11:59 schrieb Benjamin Berg:
>>
>> I also noticed this problem, but after a discussion with Johannes Berg
>> about this, we come to the conclusion, that all drivers with interrupts
>> need to deal with time travel mode via their own time travel handler.
>> What I actually did, was to write a trivial handler for e.g., the serial
>> line, which simply only call time_travel_add_irq_event(ev).
> 
> Yes, we could instead just assert that we do not have any IRQ without a
> timetravel handler. We really shouldn't have, but we do have one for
> stdin unfortunately (which is at least used by the hostap hwsim tests,
> so we might want to not remove it until we have a different solution).
> 
> What we could do is just add an stdin timetravel handler as you have
> done, but print a warning when the stdin IRQ is registered[1]. And
> then, if someone tries to register an IRQ without a time-travel handler
> we just panic(), it should never happen.
Can you explain, why a time travel handler for stdin may be bad? It 
sounds like you want to avoid it, but I see no immediate problem.

> 
> Benjamin
> 
> [1] We'll also need to set stdin to be a "null" channel and then
> somehow avoid registering the IRQ in that case.
> 

-- 
M.Sc. Benjamin Beichler

Universit?t Rostock, Fakult?t f?r Informatik und Elektrotechnik
Institut f?r Angewandte Mikroelektronik und Datentechnik

University of Rostock, Department of CS and EE
Institute of Applied Microelectronics and CE

Albert-Einstein-Str. 26
18059 Rostock
Deutschland/Germany

phone: +49 (0) 381 498 - 7278
email: Benjamin.Beichler at uni-rostock.de
www: http://www.imd.uni-rostock.de/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x1569BBF90AC3918A.asc
Type: application/pgp-keys
Size: 2976 bytes
Desc: OpenPGP public key
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231020/c349bcf2/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231020/c349bcf2/attachment.sig>

From johannes at sipsolutions.net  Fri Oct 20 04:39:51 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 20 Oct 2023 13:39:51 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
	 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
	 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
	 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
Message-ID: <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>

On Fri, 2023-10-20 at 12:38 +0200, Benjamin Beichler wrote:
> 
> Can you explain, why a time travel handler for stdin may be bad? It 
> sounds like you want to avoid it, but I see no immediate problem.
> 

I need to read the thread, but this one's easy ;-)

The thing is that on such a channel you don't send an ACK when you've
seen that there's something to handle. As a result, the sender will
continue running while you're trying to request a new schedule entry
from the controller. As a result, it may run past your new schedule
entry because it didn't know about it yet (this would likely bring down
the controller and crash the simulation), or the relative order of the
two entries is undefined, in the sense that it depends on the process
scheduling of the host.

johannes


From Benjamin.Beichler at uni-rostock.de  Fri Oct 20 05:06:08 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 20 Oct 2023 14:06:08 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
Message-ID: <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>

Am 20.10.2023 um 13:39 schrieb Johannes Berg:
> On Fri, 2023-10-20 at 12:38 +0200, Benjamin Beichler wrote:
>>
>> Can you explain, why a time travel handler for stdin may be bad? It
>> sounds like you want to avoid it, but I see no immediate problem.
>>
> 
> I need to read the thread, but this one's easy ;-)
> 
> The thing is that on such a channel you don't send an ACK when you've
> seen that there's something to handle. As a result, the sender will
> continue running while you're trying to request a new schedule entry
> from the controller. As a result, it may run past your new schedule
> entry because it didn't know about it yet (this would likely bring down
> the controller and crash the simulation), or the relative order of the
> two entries is undefined, in the sense that it depends on the process
> scheduling of the host.
Sorry, but I did not get this. What may run past the schedule entry? Is 
your assumption, that the "thing" connected to stdin is always totally 
unaware of the time travel mode?

When our (to be published) simulation send something on serial lines (I 
think it does not matter whether it is a socket or a pipe), we expect 
that the uml instance needs to be run as long as it changes back to 
idle/wait state before the simulation time is advanced. Since the basis 
model of the time travel mode is, that you have infinite amount of 
processing power, the interrupt needs to be always handled at the 
current time.

Maybe my think-model only holds for "smaller" amounts of data (maybe one 
page or something?) or not the free-running mode, but I'm not completely 
convinced. :-D

Maybe we need to define, a bit more formally, how the (designed) 
processing model of interrupts in time travel mode is.

> 
> johannes
> 
> _______________________________________________
> linux-um mailing list
> linux-um at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x1569BBF90AC3918A.asc
Type: application/pgp-keys
Size: 2976 bytes
Desc: OpenPGP public key
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231020/d2e72a93/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231020/d2e72a93/attachment.sig>

From johannes at sipsolutions.net  Fri Oct 20 05:20:25 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 20 Oct 2023 14:20:25 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
	 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
	 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
	 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
	 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
	 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
Message-ID: <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>

On Fri, 2023-10-20 at 14:06 +0200, Benjamin Beichler wrote:
> Am 20.10.2023 um 13:39 schrieb Johannes Berg:
> > On Fri, 2023-10-20 at 12:38 +0200, Benjamin Beichler wrote:
> > > 
> > > Can you explain, why a time travel handler for stdin may be bad? It
> > > sounds like you want to avoid it, but I see no immediate problem.
> > > 
> > 
> > I need to read the thread, but this one's easy ;-)
> > 
> > The thing is that on such a channel you don't send an ACK when you've
> > seen that there's something to handle. As a result, the sender will
> > continue running while you're trying to request a new schedule entry
> > from the controller. As a result, it may run past your new schedule
> > entry because it didn't know about it yet (this would likely bring down
> > the controller and crash the simulation), or the relative order of the
> > two entries is undefined, in the sense that it depends on the process
> > scheduling of the host.

> Sorry, but I did not get this. What may run past the schedule entry? Is 
> your assumption, that the "thing" connected to stdin is always totally 
> unaware of the time travel mode?

No, I'm not assuming that, if that's done all bets are off anyway. I'm
assuming that both are connected to the controller.

> When our (to be published) simulation send something on serial lines (I 
> think it does not matter whether it is a socket or a pipe), we expect 
> that the uml instance needs to be run as long as it changes back to 
> idle/wait state before the simulation time is advanced. Since the basis 
> model of the time travel mode is, that you have infinite amount of 
> processing power, the interrupt needs to be always handled at the 
> current time.

Yes but you need to schedule for the interrupt, and you don't
necessarily know what 'current time' is at interrupt time.

So let's say you have "something" that's scheduled to run at times
 - 1000
 - 2000
 - 3000

and free-until is 10000 or something high.

Now it sends a message to Linux stdio at 1000.

But it doesn't have a way to wait for ack. So it continues, checks the
schedule and free-until, and can advance time to 2000 since it doesn't
yet know Linux requested time to run at 1000.

This gives you something not predictable - it depends on the host
scheduling, whichever ran first (Linux started next or the other
continued, or both and it gets messed up).

> Maybe my think-model only holds for "smaller" amounts of data (maybe one 
> page or something?) or not the free-running mode, but I'm not completely 
> convinced. :-D

Nah it doesn't really matter how much data there is.

> Maybe we need to define, a bit more formally, how the (designed) 
> processing model of interrupts in time travel mode is.

That's probably a separate Master's thesis ;-)

johannes


From johannes at sipsolutions.net  Fri Oct 20 05:23:28 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 20 Oct 2023 14:23:28 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
	 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
	 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
	 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
	 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
	 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
	 <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
Message-ID: <76ebb80dd7fca6ef2ed7108a541d3ea3c764a3d5.camel@sipsolutions.net>

On Fri, 2023-10-20 at 14:20 +0200, Johannes Berg wrote:
> 
> Yes but you need to schedule for the interrupt, and you don't
> necessarily know what 'current time' is at interrupt time.
> 
> So let's say you have "something" that's scheduled to run at times
>  - 1000
>  - 2000
>  - 3000
> 
> and free-until is 10000 or something high.
> 

It can also happen without free-until, then it just depends which one of
the two - they're running in parallel now (linux doing time-travel
interrupt handling and adding the event, the other thing continuing to
schedule and doing the next entry)  - asks the controller first.

johannes


From Benjamin.Beichler at uni-rostock.de  Fri Oct 20 05:43:13 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 20 Oct 2023 14:43:13 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <76ebb80dd7fca6ef2ed7108a541d3ea3c764a3d5.camel@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
 <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
 <76ebb80dd7fca6ef2ed7108a541d3ea3c764a3d5.camel@sipsolutions.net>
Message-ID: <9768bf43-5a73-4eba-ae4b-532b952f9e04@uni-rostock.de>

>> Yes but you need to schedule for the interrupt, and you don't
>> necessarily know what 'current time' is at interrupt time.
>>
>> So let's say you have "something" that's scheduled to run at times
>>   - 1000
>>   - 2000
>>   - 3000
>>
>> and free-until is 10000 or something high.
>>
> It can also happen without free-until, then it just depends which one of
> the two - they're running in parallel now (linux doing time-travel
> interrupt handling and adding the event, the other thing continuing to
> schedule and doing the next entry)  - asks the controller first.

Since they happen at the same time, most discrete event simulations make 
the assumption, that the actual order of execution should not be part of 
the semantics.

Although, most of the implementations demand a deterministic order (for 
repetition) but even that requirement is sometimes lifted. So this is no 
contradiction to common discrete event simulations.




From johannes at sipsolutions.net  Fri Oct 20 05:58:00 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 20 Oct 2023 14:58:00 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <9768bf43-5a73-4eba-ae4b-532b952f9e04@uni-rostock.de>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
	 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
	 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
	 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
	 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
	 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
	 <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
	 <76ebb80dd7fca6ef2ed7108a541d3ea3c764a3d5.camel@sipsolutions.net>
	 <9768bf43-5a73-4eba-ae4b-532b952f9e04@uni-rostock.de>
Message-ID: <7be4403e56526035e334b217c370f511060802fc.camel@sipsolutions.net>

On Fri, 2023-10-20 at 14:43 +0200, Benjamin Beichler wrote:
> > > Yes but you need to schedule for the interrupt, and you don't
> > > necessarily know what 'current time' is at interrupt time.
> > > 
> > > So let's say you have "something" that's scheduled to run at times
> > > ??- 1000
> > > ??- 2000
> > > ??- 3000
> > > 
> > > and free-until is 10000 or something high.
> > > 
> > It can also happen without free-until, then it just depends which one of
> > the two - they're running in parallel now (linux doing time-travel
> > interrupt handling and adding the event, the other thing continuing to
> > schedule and doing the next entry)  - asks the controller first.
> 
> Since they happen at the same time, most discrete event simulations make 
> the assumption, that the actual order of execution should not be part of 
> the semantics.
> 

Well, fair, but they can bounce more messages around them, and that
should change things. I don't think we've built a simulation system here
that can strictly assume this is true.

In any case, it can get worse - if he sender process actually updated
the controller to 2000 by the time the linux actually calls
time_travel_add_irq_event(), the linux event will run at 2000 instead of
1000.

All this is why we have the vhost-user extensions for ACK messages etc.

johannes


From Benjamin.Beichler at uni-rostock.de  Fri Oct 20 05:58:46 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 20 Oct 2023 14:58:46 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
 <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
Message-ID: <93c958e1-a0ed-4787-adad-12c6f96c9999@uni-rostock.de>

> On Fri, 2023-10-20 at 14:06 +0200, Benjamin Beichler wrote:
>> Am 20.10.2023 um 13:39 schrieb Johannes Berg:
>>> On Fri, 2023-10-20 at 12:38 +0200, Benjamin Beichler wrote:
>>>> Can you explain, why a time travel handler for stdin may be bad? It
>>>> sounds like you want to avoid it, but I see no immediate problem.
>>>>
>>> I need to read the thread, but this one's easy ;-)
>>>
>>> The thing is that on such a channel you don't send an ACK when you've
>>> seen that there's something to handle. As a result, the sender will
>>> continue running while you're trying to request a new schedule entry
>>> from the controller. As a result, it may run past your new schedule
>>> entry because it didn't know about it yet (this would likely bring down
>>> the controller and crash the simulation), or the relative order of the
>>> two entries is undefined, in the sense that it depends on the process
>>> scheduling of the host.
>> Sorry, but I did not get this. What may run past the schedule entry? Is
>> your assumption, that the "thing" connected to stdin is always totally
>> unaware of the time travel mode?
> No, I'm not assuming that, if that's done all bets are off anyway. I'm
> assuming that both are connected to the controller.
>
>> When our (to be published) simulation send something on serial lines (I
>> think it does not matter whether it is a socket or a pipe), we expect
>> that the uml instance needs to be run as long as it changes back to
>> idle/wait state before the simulation time is advanced. Since the basis
>> model of the time travel mode is, that you have infinite amount of
>> processing power, the interrupt needs to be always handled at the
>> current time.
> Yes but you need to schedule for the interrupt, and you don't
> necessarily know what 'current time' is at interrupt time.
>
> So let's say you have "something" that's scheduled to run at times
>   - 1000
>   - 2000
>   - 3000
>
> and free-until is 10000 or something high.
>
> Now it sends a message to Linux stdio at 1000.
>
> But it doesn't have a way to wait for ack. So it continues, checks the
> schedule and free-until, and can advance time to 2000 since it doesn't
> yet know Linux requested time to run at 1000.

Do I get it right, that you anticipate that all parts of the simulation 
may run at different paces and the controller is only needed for 
synchronization?

My mind model is, that simulation time advances in the controller are 
only done, If all parts of the simulation reached the current simulation 
time and are at the event processed/idle/wait state. Therefore, the 
thing connected to stdio will not advance to some other time, if some 
other component has outstanding events (i.e., an interrupt at this 
case). Of course free-running is a problematic in this mind model, but 
in this mode you willingly sacrifice the precision of events for 
performance.

>
> This gives you something not predictable - it depends on the host
> scheduling, whichever ran first (Linux started next or the other
> continued, or both and it gets messed up).
>
>> Maybe my think-model only holds for "smaller" amounts of data (maybe one
>> page or something?) or not the free-running mode, but I'm not completely
>> convinced. :-D
> Nah it doesn't really matter how much data there is.
>
>> Maybe we need to define, a bit more formally, how the (designed)
>> processing model of interrupts in time travel mode is.
> That's probably a separate Master's thesis ;-)

It could, but some basic definition for a common understanding might help.

However, do you like to have a look at my current state of time travel 
changes in github or do you think an RFC to the mailing list is better? 
I like github for a quick dive into patches, but your whole workflow is 
maybe different ;-)




From johannes at sipsolutions.net  Fri Oct 20 06:39:52 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 20 Oct 2023 15:39:52 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <93c958e1-a0ed-4787-adad-12c6f96c9999@uni-rostock.de>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
	 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
	 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
	 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
	 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
	 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
	 <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
	 <93c958e1-a0ed-4787-adad-12c6f96c9999@uni-rostock.de>
Message-ID: <2315dd8321a86c52e2c7226889568068ffafe9a7.camel@sipsolutions.net>

On Fri, 2023-10-20 at 14:58 +0200, Benjamin Beichler wrote:
> > On Fri, 2023-10-20 at 14:06 +0200, Benjamin Beichler wrote:
> > > Am 20.10.2023 um 13:39 schrieb Johannes Berg:
> > > > On Fri, 2023-10-20 at 12:38 +0200, Benjamin Beichler wrote:
> > > > > Can you explain, why a time travel handler for stdin may be bad? It
> > > > > sounds like you want to avoid it, but I see no immediate problem.
> > > > > 
> > > > I need to read the thread, but this one's easy ;-)
> > > > 
> > > > The thing is that on such a channel you don't send an ACK when you've
> > > > seen that there's something to handle. As a result, the sender will
> > > > continue running while you're trying to request a new schedule entry
> > > > from the controller. As a result, it may run past your new schedule
> > > > entry because it didn't know about it yet (this would likely bring down
> > > > the controller and crash the simulation), or the relative order of the
> > > > two entries is undefined, in the sense that it depends on the process
> > > > scheduling of the host.
> > > Sorry, but I did not get this. What may run past the schedule entry? Is
> > > your assumption, that the "thing" connected to stdin is always totally
> > > unaware of the time travel mode?
> > No, I'm not assuming that, if that's done all bets are off anyway. I'm
> > assuming that both are connected to the controller.
> > 
> > > When our (to be published) simulation send something on serial lines (I
> > > think it does not matter whether it is a socket or a pipe), we expect
> > > that the uml instance needs to be run as long as it changes back to
> > > idle/wait state before the simulation time is advanced. Since the basis
> > > model of the time travel mode is, that you have infinite amount of
> > > processing power, the interrupt needs to be always handled at the
> > > current time.
> > Yes but you need to schedule for the interrupt, and you don't
> > necessarily know what 'current time' is at interrupt time.
> > 
> > So let's say you have "something" that's scheduled to run at times
> > ??- 1000
> > ??- 2000
> > ??- 3000
> > 
> > and free-until is 10000 or something high.
> > 
> > Now it sends a message to Linux stdio at 1000.
> > 
> > But it doesn't have a way to wait for ack. So it continues, checks the
> > schedule and free-until, and can advance time to 2000 since it doesn't
> > yet know Linux requested time to run at 1000.
> 
> Do I get it right, that you anticipate that all parts of the simulation 
> may run at different paces and the controller is only needed for 
> synchronization?

Not sure what you mean by that.

What I _wanted_ is that only one can run at a time (from a host
perspective), so e.g. virtio stuff works like

A is running
A -> B virtio:
A puts data on ring virtio ring
A sends message on vhost-user socket to indicate ring update to B
A waits for ACK for that message
B sees the ring update message
B requests time from controller
B sends ACK to A
A continues running
A gets to the next scheduling point but now sees free-until==now
A releases time slice to controller
controller assigns time slice to B
B runs to process the virtio ring data


So you see that only one is running at a time.

Without the ACK, you'd have
 B sees the ring update message
 B requests time from the controller

in parallel with

 A continues running
 A gets to the next scheduling point

but it might either not see free-until==now, or other things.


> My mind model is, that simulation time advances in the controller are 
> only done, If all parts of the simulation reached the current simulation 
> time and are at the event processed/idle/wait state.?

Yeah that's a simpler model, and in what we implemented it's true if you
do have the ACK messages. If don't have free-until, that's also possibly
true, although we might have implementation bugs in a sense with
time_travel_add_irq_event() since that looks at the current time. And
also, if you request something the controller thinks is already in the
past because the other side updated the controller due to free-until,
the simulation just crashes.

It's also possible that B (in the situation above) actually *does* ask
the controller who should run next, and the controller simply doesn't
yet know:

 B sends message to A's stdin (say at 1000)
 B requests to run at 2000 from controller
 B releases time to controller
 controller sets time to 2000 and tells B to run
 A requests scheduling at 1000
 controller *crashes*

This is actually quite likely if the host is running 6.6 with the new
EEVDF scheduler, because it will prefer to let B continue running, and
then it just needs to pick the controller over A next.

This again is because of the lack of synchronization in editing the
schedule (sending messages to the controller) which the ACK message
solves.

Hence our internal use cases for "text protocol" actually use virtio-
console rather than stdio, so you do have ACK on the vhost-user protocol
again.

> Therefore, the 
> thing connected to stdio will not advance to some other time, if some 
> other component has outstanding events (i.e., an interrupt at this 
> case).?

Yes, but by the time it actually asks "are there outstanding events",
they might no be added yet, due to hos scheduling order!

> Of course free-running is a problematic in this mind model, but 
> in this mode you willingly sacrifice the precision of events for 
> performance.

If you're referring to free-until: that isn't free-running, it's just
meant to reduce the communication overhead if you know nobody else is
running. The controller maintains this.

Btw we have further overhead reductions with shared memory, need to push
all that out.

> > > Maybe we need to define, a bit more formally, how the (designed)
> > > processing model of interrupts in time travel mode is.
> > That's probably a separate Master's thesis ;-)
> 
> It could, but some basic definition for a common understanding might help.

:)
Does the above help a bit? But yeah maybe I should try to write it down
more formally.

> However, do you like to have a look at my current state of time travel 
> changes in github or do you think an RFC to the mailing list is better? 
> I like github for a quick dive into patches, but your whole workflow is 
> maybe different ;-)
> 

RFC to the list is probably easier to comment on etc.

johannes


From vincent.whitchurch at axis.com  Fri Oct 20 07:47:39 2023
From: vincent.whitchurch at axis.com (Vincent Whitchurch)
Date: Fri, 20 Oct 2023 16:47:39 +0200
Subject: [PATCH] um: time-travel: fix time going backwards
Message-ID: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>

In basic time travel mode, I sometimes see "time goes backwards" panics
like the one below:

 Kernel panic: time-travel: time goes backwards 161689340000492 -> 161689339869814
 Call Trace:
  panic+0x1a1/0x3d7
  time_travel_update_time.cold+0xe9/0x133
  timer_read+0xc1/0x100
  ktime_get+0x10c/0x200
  copy_process+0x1899/0x2230
  kernel_clone+0x57/0x7a0
  kernel_thread+0x4a/0x50
  kthreadd+0x116/0x190

The problem is a race between time_travel_handle_real_alarm() and
timer_read().  time_travel_handle_real_alarm() changes the time after
time_read() reads the current time but before time_travel_update_time()
has had a chance to add the end event.

Fix this by doing the time read and event add atomically with respect to
time_travel_handle_real_alarm().

Signed-off-by: Vincent Whitchurch <vincent.whitchurch at axis.com>
---
 arch/um/kernel/time.c | 48 ++++++++++++++++++++++++++++++++++--------------
 1 file changed, 34 insertions(+), 14 deletions(-)

diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index fddd1dec27e6..38a94dd41b8f 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -392,17 +392,11 @@ bool time_travel_del_event(struct time_travel_event *e)
 	return true;
 }
 
-static void time_travel_update_time(unsigned long long next, bool idle)
+static void time_travel_update_to_event(struct time_travel_event *ne, bool idle)
 {
-	struct time_travel_event ne = {
-		.onstack = true,
-	};
 	struct time_travel_event *e;
 	bool finished = idle;
 
-	/* add it without a handler - we deal with that specifically below */
-	__time_travel_add_event(&ne, next);
-
 	do {
 		e = time_travel_first_event();
 
@@ -414,7 +408,7 @@ static void time_travel_update_time(unsigned long long next, bool idle)
 			BUG_ON(!time_travel_del_event(e));
 			BUG_ON(time_travel_time != e->time);
 
-			if (e == &ne) {
+			if (e == ne) {
 				finished = true;
 			} else {
 				if (e->onstack)
@@ -427,14 +421,38 @@ static void time_travel_update_time(unsigned long long next, bool idle)
 		e = time_travel_first_event();
 		if (e)
 			time_travel_ext_update_request(e->time);
-	} while (ne.pending && !finished);
+	} while (ne->pending && !finished);
+
+	time_travel_del_event(ne);
+}
 
-	time_travel_del_event(&ne);
+static void time_travel_update_time(unsigned long long next, bool idle)
+{
+	struct time_travel_event ne = {
+		.onstack = true,
+	};
+
+	__time_travel_add_event(&ne, next);
+	time_travel_update_to_event(&ne, idle);
+}
+
+static void time_travel_forward_time(unsigned long nsec, bool idle)
+{
+	struct time_travel_event ne = {
+		.onstack = true,
+	};
+	unsigned long flags;
+
+	local_irq_save(flags);
+	__time_travel_add_event(&ne, time_travel_time + nsec);
+	local_irq_restore(flags);
+
+	time_travel_update_to_event(&ne, idle);
 }
 
 void time_travel_ndelay(unsigned long nsec)
 {
-	time_travel_update_time(time_travel_time + nsec, false);
+	time_travel_forward_time(nsec, false);
 }
 EXPORT_SYMBOL(time_travel_ndelay);
 
@@ -572,6 +590,10 @@ static inline void time_travel_update_time(unsigned long long ns, bool retearly)
 {
 }
 
+static void time_travel_forward_time(unsigned long nsec, bool idle)
+{
+}
+
 static inline void time_travel_handle_real_alarm(void)
 {
 }
@@ -720,9 +742,7 @@ static u64 timer_read(struct clocksource *cs)
 		 */
 		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
 		    !time_travel_ext_waiting)
-			time_travel_update_time(time_travel_time +
-						TIMER_MULTIPLIER,
-						false);
+			time_travel_forward_time(TIMER_MULTIPLIER, false);
 		return time_travel_time / TIMER_MULTIPLIER;
 	}
 

---
base-commit: 58720809f52779dc0f08e53e54b014209d13eebb
change-id: 20231020-uml-time-backwards-552c81aedd23

Best regards,
-- 
Vincent Whitchurch <vincent.whitchurch at axis.com>



From Benjamin.Beichler at uni-rostock.de  Fri Oct 20 08:47:58 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 20 Oct 2023 17:47:58 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <2315dd8321a86c52e2c7226889568068ffafe9a7.camel@sipsolutions.net>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
 <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
 <93c958e1-a0ed-4787-adad-12c6f96c9999@uni-rostock.de>
 <2315dd8321a86c52e2c7226889568068ffafe9a7.camel@sipsolutions.net>
Message-ID: <fe6dc227-44f4-466e-a99b-e5e9e6afe7aa@uni-rostock.de>



>> My mind model is, that simulation time advances in the controller are 
>> only done, If all parts of the simulation reached the current 
>> simulation time and are at the event processed/idle/wait state. 
> Yeah that's a simpler model, and in what we implemented it's true if 
> you do have the ACK messages. If don't have free-until, that's also 
> possibly true, although we might have implementation bugs in a sense 
> with time_travel_add_irq_event() since that looks at the current time. 
> And also, if you request something the controller thinks is already in 
> the past because the other side updated the controller due to 
> free-until, the simulation just crashes. It's also possible that B (in 
> the situation above) actually *does* ask the controller who should run 
> next, and the controller simply doesn't yet know: B sends message to 
> A's stdin (say at 1000) B requests to run at 2000 from controller B 
> releases time to controller controller sets time to 2000 and tells B 
> to run A requests scheduling at 1000 controller *crashes* 

I think, I get now, why I don't have this Problem: My virtio simulation 
and my controller/simulation-master are tightly coupled and indeed the 
same process. When I create a new event for the UML-instance, it is 
anticipated in the simulation master.

So my sequence is more like:
B sends message to A's stdin (say at 1000)
B tells the controller, that it activated A, and time should not advance 
until A has sent a request for processing an interrupt and has gone back 
to waiting state
B requests to run at 2000 from controller
B releases time to controller
...

I see that without further information from the device simulation to the 
controller, it is quite harder.

Nonetheless, I'm not totally sure, how this interacts with the timing 
semantics of the interrupts here. Either the solution of Benjamin Berg 
or mine (with the trivial tt-handler have the same problem).

I will post my patches the next days, you may have a look on them, maybe 
you also experienced some of my problems :-)



From johannes at sipsolutions.net  Fri Oct 20 08:51:43 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 20 Oct 2023 17:51:43 +0200
Subject: [PATCH 1/4] um: irqs: process outstanding IRQs when unblocking
 signals
In-Reply-To: <fe6dc227-44f4-466e-a99b-e5e9e6afe7aa@uni-rostock.de>
References: <20231018123643.1255813-1-benjamin@sipsolutions.net>
	 <6082036a-c7f9-4bd5-8234-14957ed9953d@uni-rostock.de>
	 <11ab4d47dd8fdcdc2148e00618cc496866016883.camel@sipsolutions.net>
	 <549de9c0-05f8-437e-8460-5b013b0cd87b@uni-rostock.de>
	 <348de3de5cbd6df96ac51bb07d6e485b2ecc5cc8.camel@sipsolutions.net>
	 <d77b66af-249d-4dd1-b78f-730d252bcf84@uni-rostock.de>
	 <10c4e61c2e62ba12cf88ddecd84271c5788c1e6b.camel@sipsolutions.net>
	 <93c958e1-a0ed-4787-adad-12c6f96c9999@uni-rostock.de>
	 <2315dd8321a86c52e2c7226889568068ffafe9a7.camel@sipsolutions.net>
	 <fe6dc227-44f4-466e-a99b-e5e9e6afe7aa@uni-rostock.de>
Message-ID: <6622572d97b970bc83ac628a8b6c93f91f0bdf2e.camel@sipsolutions.net>

On Fri, 2023-10-20 at 17:47 +0200, Benjamin Beichler wrote:
> 
> I think, I get now, why I don't have this Problem: My virtio simulation 
> and my controller/simulation-master are tightly coupled and indeed the 
> same process. When I create a new event for the UML-instance, it is 
> anticipated in the simulation master.
> 
> So my sequence is more like:
> B sends message to A's stdin (say at 1000)
> B tells the controller, that it activated A, and time should not advance 
> until A has sent a request for processing an interrupt and has gone back 
> to waiting state
> B requests to run at 2000 from controller
> B releases time to controller
> ...

Ahh, OK. I thought you were probably using our controller from the
usfstl, but of course you don't have to. I think I did push the shared
memory optimisation there though, but I suspect I haven't posted the
Linux client version.

> I see that without further information from the device simulation to the 
> controller, it is quite harder.

Right, I wanted to handle that in the other side (hence the ACK) since
you might not always know _when_ the interrupt is scheduled there, even
if it's currently always "immediately".

> Nonetheless, I'm not totally sure, how this interacts with the timing 
> semantics of the interrupts here. Either the solution of Benjamin Berg 
> or mine (with the trivial tt-handler have the same problem).

Oh, yeah, his changes do have the same problem. He's just fixing that
interrupts got lost completely in some already _otherwise_ broken cases,
to make it work better without hanging, not to make it work correctly.

To make it work correctly you shouldn't use stdio at all, or I guess
have some external helper thing like you do :)

johannes


From Vincent.Whitchurch at axis.com  Mon Oct 23 00:08:58 2023
From: Vincent.Whitchurch at axis.com (Vincent Whitchurch)
Date: Mon, 23 Oct 2023 07:08:58 +0000
Subject: [PATCH] um: time-travel: fix time going backwards
In-Reply-To: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>
References: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>
Message-ID: <4a75c40636be267163dc30b5a6a2442089628e57.camel@axis.com>

On Fri, 2023-10-20 at 16:47 +0200, Vincent Whitchurch wrote:
> In basic time travel mode, I sometimes see "time goes backwards" panics
> like the one below:
> 
> ?Kernel panic: time-travel: time goes backwards 161689340000492 -> 161689339869814
> ?Call Trace:
> ??panic+0x1a1/0x3d7
> ??time_travel_update_time.cold+0xe9/0x133
> ??timer_read+0xc1/0x100
> ??ktime_get+0x10c/0x200
> ??copy_process+0x1899/0x2230
> ??kernel_clone+0x57/0x7a0
> ??kernel_thread+0x4a/0x50
> ??kthreadd+0x116/0x190
> 
> The problem is a race between time_travel_handle_real_alarm() and
> timer_read().  time_travel_handle_real_alarm() changes the time after
> time_read() reads the current time but before time_travel_update_time()
> has had a chance to add the end event.
> 
> Fix this by doing the time read and event add atomically with respect to
> time_travel_handle_real_alarm().

Further testing resulted in hitting the BUG_ON(time_travel_time !=
e->time) so looks like this needs some more work.  

 BUG: failure at arch/um/kernel/time.c:409/time_travel_update_to_event()!
 Kernel panic - not syncing: BUG!
 CPU: 0 PID: 1 Comm: swapper Not tainted 6.5.0+ #13
 Stack:
  624db908 60a06c60 41b58ab3 60ae2398
  6003fb90 10000c1e577c 41b58ab3 60b419ae
  607f0dc0 00000000 00000000 6005b270
 Call Trace:
  [<607fad5d>] show_stack.cold+0xc0/0x149
  [<60841364>] dump_stack_lvl+0x75/0x94
  [<6084139d>] dump_stack+0x1a/0x1c
  [<607fca1d>] panic+0x27e/0x490
  [<607fb093>] time_travel_update_to_event.cold+0x89/0x18b
  [<60041317>] time_travel_forward_time.constprop.0+0xf7/0x160
  [<600414f1>] timer_read+0x171/0x1b0
  [<60140652>] random_get_entropy_fallback+0x42/0x60
  [<60545943>] add_device_randomness+0x73/0xf0
  [<6003a685>] do_one_initcall+0x145/0x3b0
  [<60002598>] 0x60002598
  [<60843ac5>] kernel_init+0x4a/0x195
  [<6003d111>] new_thread_handler+0x141/0x1a0

From johannes at sipsolutions.net  Mon Oct 23 00:33:09 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Mon, 23 Oct 2023 09:33:09 +0200
Subject: [PATCH] um: time-travel: fix time going backwards
In-Reply-To: <4a75c40636be267163dc30b5a6a2442089628e57.camel@axis.com>
References: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>
	 <4a75c40636be267163dc30b5a6a2442089628e57.camel@axis.com>
Message-ID: <4dbed8896c94a347dcb58b3a83792c52fdc1c04a.camel@sipsolutions.net>

On Mon, 2023-10-23 at 07:08 +0000, Vincent Whitchurch wrote:
> On Fri, 2023-10-20 at 16:47 +0200, Vincent Whitchurch wrote:
> > In basic time travel mode, I sometimes see "time goes backwards" panics
> > like the one below:
> > 
> > ?Kernel panic: time-travel: time goes backwards 161689340000492 -> 161689339869814

Ouch.

> > ?Call Trace:
> > ??panic+0x1a1/0x3d7
> > ??time_travel_update_time.cold+0xe9/0x133
> > ??timer_read+0xc1/0x100
> > ??ktime_get+0x10c/0x200
> > ??copy_process+0x1899/0x2230
> > ??kernel_clone+0x57/0x7a0
> > ??kernel_thread+0x4a/0x50
> > ??kthreadd+0x116/0x190
> > 
> > The problem is a race between time_travel_handle_real_alarm() and
> > timer_read().  time_travel_handle_real_alarm() changes the time after
> > time_read() reads the current time but before time_travel_update_time()
> > has had a chance to add the end event.
> > 
> > Fix this by doing the time read and event add atomically with respect to
> > time_travel_handle_real_alarm().
> 
> Further testing resulted in hitting the BUG_ON(time_travel_time !=
> e->time) so looks like this needs some more work.  
> 

Yeah this is a tricky area, I fought with it for quite a while too,
seems we're not done yet ;-)

We mostly use time-travel=ext mode these days, so our system may not be
as susceptible to it? But not sure, in some cases it runs with just a
single instance, and that should be pretty much the same due to the
free-until information.

Do you have a specific workload that tends to reproduce this?

johannes


From arthurgrillo at riseup.net  Mon Oct 23 07:58:47 2023
From: arthurgrillo at riseup.net (Arthur Grillo)
Date: Mon, 23 Oct 2023 11:58:47 -0300
Subject: [PATCH 0/2] um: kunit: Add Clang support for CONFIG_GCOV
In-Reply-To: <20231020092159.2486063-1-michal.winiarski@intel.com>
References: <20231020092159.2486063-1-michal.winiarski@intel.com>
Message-ID: <80904787-def1-424d-abd7-e49e671b502f@riseup.net>



On 20/10/23 06:21, Micha? Winiarski wrote:
> Clang uses a different set of CLI args for coverage, and the output
> needs to be processed by a different set of tools.
> Update the Makefile and add an example of usage in kunit docs.

Great change! It's great not to rely on older versions of GCC for
that.

I was able to generate coverage reports, but I have a few of things to
note:

When I ran the kunit.py it generated this warning several times:

WARNING: modpost: vmlinux (__llvm_covfun): unexpected non-allocatable section.
Did you forget to use "ax"/"aw" in a .S file?
Note that for example <linux/init.h> contains
section definitions for use in .S files.

Maybe it would be great to know why this is happening.

Also, the linker consumed a lot of RAM to build the kernel with those
additional flags, but maybe this is expected :P.

Best Regards,
~Arthur Grillo

> 
> Micha? Winiarski (2):
>   arch: um: Add Clang coverage support
>   Documentation: kunit: Add clang UML coverage example
> 
>  Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++
>  arch/um/Makefile-skas                          |  5 +++++
>  2 files changed, 16 insertions(+)
> 


From davidgow at google.com  Wed Oct 25 01:25:56 2023
From: davidgow at google.com (David Gow)
Date: Wed, 25 Oct 2023 16:25:56 +0800
Subject: [PATCH 1/2] arch: um: Add Clang coverage support
In-Reply-To: <20231020092159.2486063-2-michal.winiarski@intel.com>
References: <20231020092159.2486063-1-michal.winiarski@intel.com> <20231020092159.2486063-2-michal.winiarski@intel.com>
Message-ID: <CABVgOS=S9w7cTTNO+JjXoBCYe8RJ=tRkEfdZiWhPbbUk52SHNQ@mail.gmail.com>

On Fri, 20 Oct 2023 at 17:22, Micha? Winiarski
<michal.winiarski at intel.com> wrote:
>
> Clang uses a different set of command line arguments for enabling
> coverage.
>
> Signed-off-by: Micha? Winiarski <michal.winiarski at intel.com>
> ---

This works brilliantly here -- I'm very glad to finally be able to use
something newer than gcc 6!

I assume this will go in via the UML tree, but if you want, we can
take it via KUnit.

Tested-by: David Gow <davidgow at google.com>

Cheers,
-- David


>  arch/um/Makefile-skas | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/arch/um/Makefile-skas b/arch/um/Makefile-skas
> index ac35de5316a6..67323b028999 100644
> --- a/arch/um/Makefile-skas
> +++ b/arch/um/Makefile-skas
> @@ -4,7 +4,12 @@
>  #
>
>  GPROF_OPT += -pg
> +
> +ifdef CONFIG_CC_IS_CLANG
> +GCOV_OPT += -fprofile-instr-generate -fcoverage-mapping
> +else
>  GCOV_OPT += -fprofile-arcs -ftest-coverage
> +endif
>
>  CFLAGS-$(CONFIG_GCOV) += $(GCOV_OPT)
>  CFLAGS-$(CONFIG_GPROF) += $(GPROF_OPT)
> --
> 2.42.0
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4003 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231025/9affd629/attachment.p7s>

From davidgow at google.com  Wed Oct 25 01:28:00 2023
From: davidgow at google.com (David Gow)
Date: Wed, 25 Oct 2023 16:28:00 +0800
Subject: [PATCH 2/2] Documentation: kunit: Add clang UML coverage example
In-Reply-To: <20231020092159.2486063-3-michal.winiarski@intel.com>
References: <20231020092159.2486063-1-michal.winiarski@intel.com> <20231020092159.2486063-3-michal.winiarski@intel.com>
Message-ID: <CABVgOS=yMiadnA=tWWWJ6WgOAHvWSNJXcXCh9jxeb3qyDM0J0g@mail.gmail.com>

On Fri, 20 Oct 2023 at 17:22, Micha? Winiarski
<michal.winiarski at intel.com> wrote:
>
> LLVM-based toolchain is using a different set of tools for coverage.
> Add an example that produces output in lcov format.
>
> Signed-off-by: Micha? Winiarski <michal.winiarski at intel.com>
> ---

This looks good to me, minus a couple of very, very minor typos, and
the fact that this whole section could probably do with some tidying
up. We'll deal with that separately, though.

UML folks: do you want to take this via the UML branch (alongside
patch 1), or should we take one or both of them via KUnit? (Or, this
could go via the docs tree, too, I suppose.) There shouldn't be any
merge conflicts on our side.

Reviewed-by: David Gow <davidgow at google.com>

Cheers,
-- David

>  Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst
> index 766f9cdea0fa..9f69c122dee7 100644
> --- a/Documentation/dev-tools/kunit/running_tips.rst
> +++ b/Documentation/dev-tools/kunit/running_tips.rst
> @@ -139,6 +139,17 @@ If your installed version of gcc doesn't work, you can tweak the steps:
>         $ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
>         $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6
>
> +Alternatively, LLVM-based toolchain can also be used:

Nit: should this be "an LLVM-based toolchain" or "LLVM-based toolchains"?

> +
> +.. code-block:: bash
> +
> +       # Build with LLVM and append coverage options to the current config
> +       $ $ ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --kunitconfig=.kunit/ --kunitconfig=tools/testing/kunit/configs/coverage_uml.config

I'm not a big fan of the --kunitconfig=.kunit/ bit here, but since
we're doing it in the gcc version above, let's leave it for now for
consistency.

Also, Nit: two '$' starting the line.


> +       $ llvm-profdata merge -sparse default.profraw -o default.profdata
> +       $ llvm-cov export --format=lcov .kunit/vmlinux -instr-profile default.profdata > coverage.info
> +       # The coverage.info file is in lcov-compatible format and it can be used to e.g. generate HTML report
> +       $ genhtml -o /tmp/coverage_html coverage.info
> +
>
>  Running tests manually
>  ======================
> --
> 2.42.0
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4003 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231025/d90e611b/attachment.p7s>

From Vincent.Whitchurch at axis.com  Wed Oct 25 04:55:02 2023
From: Vincent.Whitchurch at axis.com (Vincent Whitchurch)
Date: Wed, 25 Oct 2023 11:55:02 +0000
Subject: [PATCH] um: time-travel: fix time going backwards
In-Reply-To: <4dbed8896c94a347dcb58b3a83792c52fdc1c04a.camel@sipsolutions.net>
References: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>
	 <4a75c40636be267163dc30b5a6a2442089628e57.camel@axis.com>
	 <4dbed8896c94a347dcb58b3a83792c52fdc1c04a.camel@sipsolutions.net>
Message-ID: <cc7d340e4e06e42cd67b46bfa95da6129b0010cb.camel@axis.com>

On Mon, 2023-10-23 at 09:33 +0200, Johannes Berg wrote:
> Do you have a specific workload that tends to reproduce this?

I've been seeing it when running roadtest, but it's easily reproducible
without that by using the attached config and the following program as
init.

  cp repro.config .config
  make ARCH=um olddefconfig all
  gcc -Wall -static -o repro repro.c
  ./linux time-travel init=$PWD/repro rootfstype=hostfs

With the above commands I usually see the panic in a few seconds.  This
is on an unmodified v6.6-rc7.

----8<-------
#include <time.h>
#include <pthread.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>

int main(void)
{
        if (fork() == 0) {
                while (1) {
                        nanosleep(&(struct timespec){.tv_nsec = 330 * 1000}, NULL);
                }
                return 0;
        }
        
        while (1) {
                struct timespec ts;
                clock_gettime(CLOCK_MONOTONIC, &ts);
                nanosleep(&(struct timespec){.tv_nsec = 22 * 1000}, NULL);
        }
        
        return 0;
}       
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: repro.config
URL: <http://lists.infradead.org/pipermail/linux-um/attachments/20231025/cb7d7e0f/attachment-0001.ksh>

From johannes at sipsolutions.net  Wed Oct 25 12:51:45 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Wed, 25 Oct 2023 21:51:45 +0200
Subject: [PATCH] um: time-travel: fix time going backwards
In-Reply-To: <cc7d340e4e06e42cd67b46bfa95da6129b0010cb.camel@axis.com>
References: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>
	 <4a75c40636be267163dc30b5a6a2442089628e57.camel@axis.com>
	 <4dbed8896c94a347dcb58b3a83792c52fdc1c04a.camel@sipsolutions.net>
	 <cc7d340e4e06e42cd67b46bfa95da6129b0010cb.camel@axis.com>
Message-ID: <a2f6e0cc8e498f72dfe1fcd5f2233ada8548a4d1.camel@sipsolutions.net>

On Wed, 2023-10-25 at 11:55 +0000, Vincent Whitchurch wrote:
> On Mon, 2023-10-23 at 09:33 +0200, Johannes Berg wrote:
> > Do you have a specific workload that tends to reproduce this?
> 
> I've been seeing it when running roadtest, but it's easily reproducible
> without that by using the attached config and the following program as
> init.
> 
>   cp repro.config .config
>   make ARCH=um olddefconfig all
>   gcc -Wall -static -o repro repro.c
>   ./linux time-travel init=$PWD/repro rootfstype=hostfs
> 
> With the above commands I usually see the panic in a few seconds.  This
> is on an unmodified v6.6-rc7.
> 

Yes, I can reproduce it with your test, thanks. I'm on 6.4-rc still for
$reasons (6.5 we skipped during vacations, and 6.6 EEVDF scheduler broke
everything for hwsim tests in hostap), but this code didn't really
change anyway.

I'll poke at this.

(And I'm still amazed that anyone other than me even uses this stuff :P)

johannes


From johannes at sipsolutions.net  Wed Oct 25 13:02:33 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Wed, 25 Oct 2023 22:02:33 +0200
Subject: [PATCH] um: time-travel: fix time going backwards
In-Reply-To: <a2f6e0cc8e498f72dfe1fcd5f2233ada8548a4d1.camel@sipsolutions.net>
References: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>
	 <4a75c40636be267163dc30b5a6a2442089628e57.camel@axis.com>
	 <4dbed8896c94a347dcb58b3a83792c52fdc1c04a.camel@sipsolutions.net>
	 <cc7d340e4e06e42cd67b46bfa95da6129b0010cb.camel@axis.com>
	 <a2f6e0cc8e498f72dfe1fcd5f2233ada8548a4d1.camel@sipsolutions.net>
Message-ID: <68b18065d8be905c25522bd3f5a9b46dbe3a976d.camel@sipsolutions.net>

On Wed, 2023-10-25 at 21:51 +0200, Johannes Berg wrote:
> On Wed, 2023-10-25 at 11:55 +0000, Vincent Whitchurch wrote:
> > On Mon, 2023-10-23 at 09:33 +0200, Johannes Berg wrote:
> > > Do you have a specific workload that tends to reproduce this?
> > 
> > I've been seeing it when running roadtest, but it's easily reproducible
> > without that by using the attached config and the following program as
> > init.
> > 
> >   cp repro.config .config
> >   make ARCH=um olddefconfig all
> >   gcc -Wall -static -o repro repro.c
> >   ./linux time-travel init=$PWD/repro rootfstype=hostfs

Ohhh.

Pure "time-travel" is actually something I hardly think about these
days, we mostly use time-travel=inf-cpu (or =ext).

That makes sense, here you actually *can* get interrupted. I'll need to
dig into what happens though.

johannes


From johannes at sipsolutions.net  Wed Oct 25 13:37:11 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Wed, 25 Oct 2023 22:37:11 +0200
Subject: [PATCH] um: time-travel: fix time corruption
Message-ID: <20231025223710.400baeaebf4d.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>

From: Johannes Berg <johannes.berg at intel.com>

In 'basic' time-travel mode (without =inf-cpu or =ext), we
still get timer interrupts. These can happen at arbitrary
points in time, i.e. while in timer_read(), which pushes
time forward just a little bit. Then, if we happen to get
the interrupt after calculating the new time to push to,
but before actually finishing that, the interrupt will set
the time to a value that's incompatible with the forward,
and we'll crash because time goes backwards when we do the
forwarding.

Fix this by reading the time_travel_time, calculating the
adjustment, and doing the adjustment all with interrupts
disabled.

Reported-by: Vincent Whitchurch <Vincent.Whitchurch at axis.com>
Signed-off-by: Johannes Berg <johannes.berg at intel.com>
---
 arch/um/kernel/time.c | 35 +++++++++++++++++++++++++++++------
 1 file changed, 29 insertions(+), 6 deletions(-)

diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 8ff46bc86d09..c549c959a030 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -46,7 +46,8 @@ static u16 time_travel_shm_id;
 static struct um_timetravel_schedshm *time_travel_shm;
 static union um_timetravel_schedshm_client *time_travel_shm_client;
 
-static void time_travel_set_time(unsigned long long ns)
+#define  time_travel_set_time(ns) _time_travel_set_time(ns, __LINE__)
+static void _time_travel_set_time(unsigned long long ns, unsigned int line)
 {
 	if (unlikely(ns < time_travel_time))
 		panic("time-travel: time goes backwards %lld -> %lld\n",
@@ -551,9 +552,29 @@ static void time_travel_update_time(unsigned long long next, bool idle)
 	time_travel_del_event(&ne);
 }
 
+static void time_travel_update_time_rel(unsigned long long offs)
+{
+	unsigned long flags;
+
+	/*
+	 * Disable interrupts before calculating the new time so
+	 * that a real timer interrupt (signal) can't happen at
+	 * a bad time e.g. after we read time_travel_time but
+	 * before we've completed updating the time.
+	 */
+	local_irq_save(flags);
+	time_travel_update_time(time_travel_time + offs, false);
+	local_irq_restore(flags);
+}
+
 void time_travel_ndelay(unsigned long nsec)
 {
-	time_travel_update_time(time_travel_time + nsec, false);
+	/*
+	 * Not strictly needed to use _rel() version since this is
+	 * only used in INFCPU/EXT modes, but it doesn't hurt and
+	 * is more readable too.
+	 */
+	time_travel_update_time_rel(nsec);
 }
 EXPORT_SYMBOL(time_travel_ndelay);
 
@@ -687,7 +708,11 @@ static void time_travel_set_start(void)
 #define time_travel_time 0
 #define time_travel_ext_waiting 0
 
-static inline void time_travel_update_time(unsigned long long ns, bool retearly)
+static inline void time_travel_update_time(unsigned long long ns, bool idle)
+{
+}
+
+static inline void time_travel_update_time_rel(unsigned long long offs)
 {
 }
 
@@ -839,9 +864,7 @@ static u64 timer_read(struct clocksource *cs)
 		 */
 		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
 		    !time_travel_ext_waiting)
-			time_travel_update_time(time_travel_time +
-						TIMER_MULTIPLIER,
-						false);
+			time_travel_update_time_rel(TIMER_MULTIPLIER);
 		return time_travel_time / TIMER_MULTIPLIER;
 	}
 
-- 
2.41.0



From johannes at sipsolutions.net  Wed Oct 25 13:41:38 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Wed, 25 Oct 2023 22:41:38 +0200
Subject: [PATCH] um: time-travel: fix time going backwards
In-Reply-To: <68b18065d8be905c25522bd3f5a9b46dbe3a976d.camel@sipsolutions.net>
References: <20231020-uml-time-backwards-v1-1-90b776fc6dfd@axis.com>
	 <4a75c40636be267163dc30b5a6a2442089628e57.camel@axis.com>
	 <4dbed8896c94a347dcb58b3a83792c52fdc1c04a.camel@sipsolutions.net>
	 <cc7d340e4e06e42cd67b46bfa95da6129b0010cb.camel@axis.com>
	 <a2f6e0cc8e498f72dfe1fcd5f2233ada8548a4d1.camel@sipsolutions.net>
	 <68b18065d8be905c25522bd3f5a9b46dbe3a976d.camel@sipsolutions.net>
Message-ID: <1f38d92435cc4f3a5e8d17dff3da9e34be2e6f24.camel@sipsolutions.net>

On Wed, 2023-10-25 at 22:02 +0200, Johannes Berg wrote:
> On Wed, 2023-10-25 at 21:51 +0200, Johannes Berg wrote:
> > On Wed, 2023-10-25 at 11:55 +0000, Vincent Whitchurch wrote:
> > > On Mon, 2023-10-23 at 09:33 +0200, Johannes Berg wrote:
> > > > Do you have a specific workload that tends to reproduce this?
> > > 
> > > I've been seeing it when running roadtest, but it's easily reproducible
> > > without that by using the attached config and the following program as
> > > init.
> > > 
> > >   cp repro.config .config
> > >   make ARCH=um olddefconfig all
> > >   gcc -Wall -static -o repro repro.c
> > >   ./linux time-travel init=$PWD/repro rootfstype=hostfs
> 
> Ohhh.
> 
> Pure "time-travel" is actually something I hardly think about these
> days, we mostly use time-travel=inf-cpu (or =ext).
> 
> That makes sense, here you actually *can* get interrupted. I'll need to
> dig into what happens though.

I just sent a patch, please take a look. It does seem to fix it - at
least I got bored of waiting after running your test program for a few
minutes :)

johannes


From johannes at sipsolutions.net  Wed Oct 25 13:45:05 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Wed, 25 Oct 2023 22:45:05 +0200
Subject: [PATCH v2] um: time-travel: fix time corruption
Message-ID: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>

From: Johannes Berg <johannes.berg at intel.com>

In 'basic' time-travel mode (without =inf-cpu or =ext), we
still get timer interrupts. These can happen at arbitrary
points in time, i.e. while in timer_read(), which pushes
time forward just a little bit. Then, if we happen to get
the interrupt after calculating the new time to push to,
but before actually finishing that, the interrupt will set
the time to a value that's incompatible with the forward,
and we'll crash because time goes backwards when we do the
forwarding.

Fix this by reading the time_travel_time, calculating the
adjustment, and doing the adjustment all with interrupts
disabled.

Reported-by: Vincent Whitchurch <Vincent.Whitchurch at axis.com>
Signed-off-by: Johannes Berg <johannes.berg at intel.com>
---
v2: remove stray debug code
---
 arch/um/kernel/time.c | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 8ff46bc86d09..0c01674e14d5 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -551,9 +551,29 @@ static void time_travel_update_time(unsigned long long next, bool idle)
 	time_travel_del_event(&ne);
 }
 
+static void time_travel_update_time_rel(unsigned long long offs)
+{
+	unsigned long flags;
+
+	/*
+	 * Disable interrupts before calculating the new time so
+	 * that a real timer interrupt (signal) can't happen at
+	 * a bad time e.g. after we read time_travel_time but
+	 * before we've completed updating the time.
+	 */
+	local_irq_save(flags);
+	time_travel_update_time(time_travel_time + offs, false);
+	local_irq_restore(flags);
+}
+
 void time_travel_ndelay(unsigned long nsec)
 {
-	time_travel_update_time(time_travel_time + nsec, false);
+	/*
+	 * Not strictly needed to use _rel() version since this is
+	 * only used in INFCPU/EXT modes, but it doesn't hurt and
+	 * is more readable too.
+	 */
+	time_travel_update_time_rel(nsec);
 }
 EXPORT_SYMBOL(time_travel_ndelay);
 
@@ -687,7 +707,11 @@ static void time_travel_set_start(void)
 #define time_travel_time 0
 #define time_travel_ext_waiting 0
 
-static inline void time_travel_update_time(unsigned long long ns, bool retearly)
+static inline void time_travel_update_time(unsigned long long ns, bool idle)
+{
+}
+
+static inline void time_travel_update_time_rel(unsigned long long offs)
 {
 }
 
@@ -839,9 +863,7 @@ static u64 timer_read(struct clocksource *cs)
 		 */
 		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
 		    !time_travel_ext_waiting)
-			time_travel_update_time(time_travel_time +
-						TIMER_MULTIPLIER,
-						false);
+			time_travel_update_time_rel(TIMER_MULTIPLIER);
 		return time_travel_time / TIMER_MULTIPLIER;
 	}
 
-- 
2.41.0



From Vincent.Whitchurch at axis.com  Thu Oct 26 00:23:49 2023
From: Vincent.Whitchurch at axis.com (Vincent Whitchurch)
Date: Thu, 26 Oct 2023 07:23:49 +0000
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
Message-ID: <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>

On Wed, 2023-10-25 at 22:45 +0200, Johannes Berg wrote:
> From: Johannes Berg <johannes.berg at intel.com>
> 
> In 'basic' time-travel mode (without =inf-cpu or =ext), we
> still get timer interrupts. These can happen at arbitrary
> points in time, i.e. while in timer_read(), which pushes
> time forward just a little bit. Then, if we happen to get
> the interrupt after calculating the new time to push to,
> but before actually finishing that, the interrupt will set
> the time to a value that's incompatible with the forward,
> and we'll crash because time goes backwards when we do the
> forwarding.
> 
> Fix this by reading the time_travel_time, calculating the
> adjustment, and doing the adjustment all with interrupts
> disabled.
> 
> Reported-by: Vincent Whitchurch <Vincent.Whitchurch at axis.com>
> Signed-off-by: Johannes Berg <johannes.berg at intel.com>

Thanks, this works for me too.  However, one question below.

> ---
> v2: remove stray debug code
> ---
> ?arch/um/kernel/time.c | 32 +++++++++++++++++++++++++++-----
> ?1 file changed, 27 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
> index 8ff46bc86d09..0c01674e14d5 100644
> --- a/arch/um/kernel/time.c
> +++ b/arch/um/kernel/time.c
> @@ -551,9 +551,29 @@ static void time_travel_update_time(unsigned long long next, bool idle)
> ?	time_travel_del_event(&ne);
> ?}
> ?
> 
> +static void time_travel_update_time_rel(unsigned long long offs)
> +{
> +	unsigned long flags;
> +
> +	/*
> +	 * Disable interrupts before calculating the new time so
> +	 * that a real timer interrupt (signal) can't happen at
> +	 * a bad time e.g. after we read time_travel_time but
> +	 * before we've completed updating the time.
> +	 */
> +	local_irq_save(flags);
> +	time_travel_update_time(time_travel_time + offs, false);
> +	local_irq_restore(flags);
> +}
> +
> ?void time_travel_ndelay(unsigned long nsec)
> ?{
> -	time_travel_update_time(time_travel_time + nsec, false);
> +	/*
> +	 * Not strictly needed to use _rel() version since this is
> +	 * only used in INFCPU/EXT modes, but it doesn't hurt and
> +	 * is more readable too.
> +	 */
> +	time_travel_update_time_rel(nsec);
> ?}
> ?EXPORT_SYMBOL(time_travel_ndelay);
> ?
> 
> @@ -687,7 +707,11 @@ static void time_travel_set_start(void)
> ?#define time_travel_time 0
> ?#define time_travel_ext_waiting 0
> ?
> 
> -static inline void time_travel_update_time(unsigned long long ns, bool retearly)
> +static inline void time_travel_update_time(unsigned long long ns, bool idle)
> +{
> +}
> +
> +static inline void time_travel_update_time_rel(unsigned long long offs)
> ?{
> ?}
> ?
> 
> @@ -839,9 +863,7 @@ static u64 timer_read(struct clocksource *cs)
> ?		 */
> ?		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
> ?		    !time_travel_ext_waiting)
> -			time_travel_update_time(time_travel_time +
> -						TIMER_MULTIPLIER,
> -						false);
> +			time_travel_update_time_rel(TIMER_MULTIPLIER);
> ?		return time_travel_time / TIMER_MULTIPLIER;
> ?	}

The reason I hesitated with putting the whole of
time_travel_update_time() under local_irq_save() in my attempt was
because I didn't quite understand the reason for the !irqs_disabled()
condition here and the comment just above it about recursion and things
getting messed up.  If it's OK to disable interrupts as this patch does,
is the !irqs_disabled() condition valid?

From johannes at sipsolutions.net  Thu Oct 26 00:38:30 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Thu, 26 Oct 2023 09:38:30 +0200
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
	 <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
Message-ID: <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>

On Thu, 2023-10-26 at 07:23 +0000, Vincent Whitchurch wrote:
> > @@ -839,9 +863,7 @@ static u64 timer_read(struct clocksource *cs)
> > ?		 */
> > ?		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
> > ?		    !time_travel_ext_waiting)
> > -			time_travel_update_time(time_travel_time +
> > -						TIMER_MULTIPLIER,
> > -						false);
> > +			time_travel_update_time_rel(TIMER_MULTIPLIER);
> > ?		return time_travel_time / TIMER_MULTIPLIER;
> > ?	}
> 
> The reason I hesitated with putting the whole of
> time_travel_update_time() under local_irq_save() in my attempt was
> because I didn't quite understand the reason for the !irqs_disabled()
> condition here and the comment just above it about recursion and things
> getting messed up.  If it's OK to disable interrupts as this patch does,
> is the !irqs_disabled() condition valid?

Hmm. I was going to say that's different, because it wants to only
prevent us from doing this while we're *already* in IRQ context, and the
bug you found is calling timer_read() not in IRQ context, but getting an
event queued by the signal.

But ... now that I think about it, I have a feeling that this was a
workaround for the exact same problem, and I just didn't understand it
at the time? I mean, recursing into our own processing is now impossible
here after this patch - either we're running normally, or the interrupt
cannot hit timer_read() in the middle, same as it cannot hit
time_travel_handle_real_alarm() in the middle now.

Removing that still seems to work with your test, but it's also not a
good test for this, since there are no devices etc. that could have
interrupts, not sure how to test it right now?

Maybe I'll add a comment there saying this might no longer be needed?

johannes


From Vincent.Whitchurch at axis.com  Thu Oct 26 01:49:21 2023
From: Vincent.Whitchurch at axis.com (Vincent Whitchurch)
Date: Thu, 26 Oct 2023 08:49:21 +0000
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
	 <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
	 <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>
Message-ID: <04afd005c5566fcc6a46549afd7b9136a7cec7a9.camel@axis.com>

On Thu, 2023-10-26 at 09:38 +0200, Johannes Berg wrote:
> On Thu, 2023-10-26 at 07:23 +0000, Vincent Whitchurch wrote:
> > > @@ -839,9 +863,7 @@ static u64 timer_read(struct clocksource *cs)
> > > ?		 */
> > > ?		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
> > > ?		    !time_travel_ext_waiting)
> > > -			time_travel_update_time(time_travel_time +
> > > -						TIMER_MULTIPLIER,
> > > -						false);
> > > +			time_travel_update_time_rel(TIMER_MULTIPLIER);
> > > ?		return time_travel_time / TIMER_MULTIPLIER;
> > > ?	}
> > 
> > The reason I hesitated with putting the whole of
> > time_travel_update_time() under local_irq_save() in my attempt was
> > because I didn't quite understand the reason for the !irqs_disabled()
> > condition here and the comment just above it about recursion and things
> > getting messed up.  If it's OK to disable interrupts as this patch does,
> > is the !irqs_disabled() condition valid?
> 
> Hmm. I was going to say that's different, because it wants to only
> prevent us from doing this while we're *already* in IRQ context, and the
> bug you found is calling timer_read() not in IRQ context, but getting an
> event queued by the signal.
> 
> But ... now that I think about it, I have a feeling that this was a
> workaround for the exact same problem, and I just didn't understand it
> at the time? I mean, recursing into our own processing is now impossible
> here after this patch - either we're running normally, or the interrupt
> cannot hit timer_read() in the middle, same as it cannot hit
> time_travel_handle_real_alarm() in the middle now.
> 
> Removing that still seems to work with your test, but it's also not a
> good test for this, since there are no devices etc. that could have
> interrupts, not sure how to test it right now?
> 
> Maybe I'll add a comment there saying this might no longer be needed?

I tried removing the !irqs_disabled() check and that blew up pretty
quickly (below) when running the full roadtest suite.  It works fine
with your unmodified patch so no need to change the comment.

 Kernel panic - not syncing: time-travel: time goes backwards 26374790000864 -> 26374790000853
 show_stack.cold (arch/um/kernel/sysrq.c:56) 
 dump_stack_lvl (lib/dump_stack.c:107 (discriminator 4)) 
 dump_stack (lib/dump_stack.c:114) 
 panic (kernel/panic.c:262 kernel/panic.c:361) 
 timer_handler.cold (arch/um/kernel/time.c:51 arch/um/kernel/time.c:510 arch/um/kernel/time.c:634) 
 timer_real_alarm_handler (arch/um/os-Linux/signal.c:109) 
 unblock_signals (arch/um/os-Linux/signal.c:338) 
 tick_nohz_idle_exit (kernel/time/tick-sched.c:1364) 
 do_idle (kernel/sched/idle.c:310) 
 cpu_startup_entry (kernel/sched/idle.c:379 (discriminator 1)) 
 kernel_init (init/main.c:1435) 
 0x60001ce6 
 0x6000220e 
 0x60004961 
 new_thread_handler (arch/um/include/asm/thread_info.h:46 arch/um/kernel/process.c:136) 
 uml_finishsetup (arch/um/kernel/um_arch.c:268) 


From johannes at sipsolutions.net  Thu Oct 26 02:10:03 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Thu, 26 Oct 2023 11:10:03 +0200
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <04afd005c5566fcc6a46549afd7b9136a7cec7a9.camel@axis.com>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
	 <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
	 <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>
	 <04afd005c5566fcc6a46549afd7b9136a7cec7a9.camel@axis.com>
Message-ID: <a2a6c524ddb897d3c3b640d0f7262b506e51d983.camel@sipsolutions.net>

On Thu, 2023-10-26 at 08:49 +0000, Vincent Whitchurch wrote:
> On Thu, 2023-10-26 at 09:38 +0200, Johannes Berg wrote:
> > On Thu, 2023-10-26 at 07:23 +0000, Vincent Whitchurch wrote:
> > > > @@ -839,9 +863,7 @@ static u64 timer_read(struct clocksource *cs)
> > > > ?		 */
> > > > ?		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
> > > > ?		    !time_travel_ext_waiting)
> > > > -			time_travel_update_time(time_travel_time +
> > > > -						TIMER_MULTIPLIER,
> > > > -						false);
> > > > +			time_travel_update_time_rel(TIMER_MULTIPLIER);
> > > > ?		return time_travel_time / TIMER_MULTIPLIER;
> > > > ?	}
> > > 
> > > The reason I hesitated with putting the whole of
> > > time_travel_update_time() under local_irq_save() in my attempt was
> > > because I didn't quite understand the reason for the !irqs_disabled()
> > > condition here and the comment just above it about recursion and things
> > > getting messed up.  If it's OK to disable interrupts as this patch does,
> > > is the !irqs_disabled() condition valid?
> > 
> > Hmm. I was going to say that's different, because it wants to only
> > prevent us from doing this while we're *already* in IRQ context, and the
> > bug you found is calling timer_read() not in IRQ context, but getting an
> > event queued by the signal.
> > 
> > But ... now that I think about it, I have a feeling that this was a
> > workaround for the exact same problem, and I just didn't understand it
> > at the time? I mean, recursing into our own processing is now impossible
> > here after this patch - either we're running normally, or the interrupt
> > cannot hit timer_read() in the middle, same as it cannot hit
> > time_travel_handle_real_alarm() in the middle now.
> > 
> > Removing that still seems to work with your test, but it's also not a
> > good test for this, since there are no devices etc. that could have
> > interrupts, not sure how to test it right now?
> > 
> > Maybe I'll add a comment there saying this might no longer be needed?
> 
> I tried removing the !irqs_disabled() check and that blew up pretty
> quickly (below) when running the full roadtest suite.  It works fine
> with your unmodified patch so no need to change the comment.
> 

Hah, OK. So maybe when/if I remember what happens there or can figure it
out again, I can update the comment :)

johannes


From Benjamin.Beichler at uni-rostock.de  Fri Oct 27 07:05:19 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 27 Oct 2023 16:05:19 +0200
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <a2a6c524ddb897d3c3b640d0f7262b506e51d983.camel@sipsolutions.net>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
 <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
 <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>
 <04afd005c5566fcc6a46549afd7b9136a7cec7a9.camel@axis.com>
 <a2a6c524ddb897d3c3b640d0f7262b506e51d983.camel@sipsolutions.net>
Message-ID: <da07d5f1-f52a-428a-9884-0f2f1a3efdf3@uni-rostock.de>

Am 26.10.2023 um 11:10 schrieb Johannes Berg:
> On Thu, 2023-10-26 at 08:49 +0000, Vincent Whitchurch wrote:
>> On Thu, 2023-10-26 at 09:38 +0200, Johannes Berg wrote:
>>> On Thu, 2023-10-26 at 07:23 +0000, Vincent Whitchurch wrote:
>>>>> @@ -839,9 +863,7 @@ static u64 timer_read(struct clocksource *cs)
>>>>>  ?		 */
>>>>>  ?		if (!irqs_disabled() && !in_interrupt() && !in_softirq() &&
>>>>>  ?		    !time_travel_ext_waiting)
>>>>> -			time_travel_update_time(time_travel_time +
>>>>> -						TIMER_MULTIPLIER,
>>>>> -						false);
>>>>> +			time_travel_update_time_rel(TIMER_MULTIPLIER);
>>>>>  ?		return time_travel_time / TIMER_MULTIPLIER;
>>>>>  ?	}
>>>>
>>>> The reason I hesitated with putting the whole of
>>>> time_travel_update_time() under local_irq_save() in my attempt was
>>>> because I didn't quite understand the reason for the !irqs_disabled()
>>>> condition here and the comment just above it about recursion and things
>>>> getting messed up.  If it's OK to disable interrupts as this patch does,
>>>> is the !irqs_disabled() condition valid?
>>>
>>> Hmm. I was going to say that's different, because it wants to only
>>> prevent us from doing this while we're *already* in IRQ context, and the
>>> bug you found is calling timer_read() not in IRQ context, but getting an
>>> event queued by the signal.
>>>
>>> But ... now that I think about it, I have a feeling that this was a
>>> workaround for the exact same problem, and I just didn't understand it
>>> at the time? I mean, recursing into our own processing is now impossible
>>> here after this patch - either we're running normally, or the interrupt
>>> cannot hit timer_read() in the middle, same as it cannot hit
>>> time_travel_handle_real_alarm() in the middle now.
>>>
>>> Removing that still seems to work with your test, but it's also not a
>>> good test for this, since there are no devices etc. that could have
>>> interrupts, not sure how to test it right now?
>>>
>>> Maybe I'll add a comment there saying this might no longer be needed?
>>
>> I tried removing the !irqs_disabled() check and that blew up pretty
>> quickly (below) when running the full roadtest suite.  It works fine
>> with your unmodified patch so no need to change the comment.
>>
> 
> Hah, OK. So maybe when/if I remember what happens there or can figure it
> out again, I can update the comment :)
As I said, I'm also preparing some patches, therefore my few bits about 
those problems:

- we noticed, that those backward running time is mostly created by the 
"unprotected" access to the variables of the time travel event list. 
 From the stack trace you see the problem even do not appear directly at 
timer_read but later. I think mostly this means the event list head was 
moved and then the interrupt happens and the pointer (or stored time) 
become outdated and the function (here the alarm_handler) continues with 
them.

- besides this, when you look into local_irq_save for um, this does not 
really deactivate the interrupts, but delay the processing and it adds a 
memory barrier. I think that could be one of the important consequences 
as changes to the event list are forced to be populated. But this is 
only a guess :-D

- I'm also not really convinced, that all accesses to the current time 
should be delayed. I made a patch with a heuristic, that only delays the 
time read, if it is read from userspace multiple times in a row. Since 
there are no "busy" loops in the kernel itself and only in a few (bad 
behaving) userspace programs, I think that helps to reduce the 
inaccuracy in simulation from well behaving userspace programs only 
reading the time once.

In this case, irqs_disabled more or less only tend to indicate, that the 
event list could be manipulated, and therefore the update_time call is a 
bad idea.

Benjamin


From johannes at sipsolutions.net  Fri Oct 27 07:45:56 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 27 Oct 2023 16:45:56 +0200
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <da07d5f1-f52a-428a-9884-0f2f1a3efdf3@uni-rostock.de>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
	 <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
	 <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>
	 <04afd005c5566fcc6a46549afd7b9136a7cec7a9.camel@axis.com>
	 <a2a6c524ddb897d3c3b640d0f7262b506e51d983.camel@sipsolutions.net>
	 <da07d5f1-f52a-428a-9884-0f2f1a3efdf3@uni-rostock.de>
Message-ID: <68748578c04fd1c41d3969bc5348eb2498894ac7.camel@sipsolutions.net>

On Fri, 2023-10-27 at 16:05 +0200, Benjamin Beichler wrote:
> 
> As I said, I'm also preparing some patches, therefore my few bits about 
> those problems:
> 
> - we noticed, that those backward running time is mostly created by the 
> "unprotected" access to the variables of the time travel event list. 
>  From the stack trace you see the problem even do not appear directly at 
> timer_read but later. I think mostly this means the event list head was 
> moved and then the interrupt happens and the pointer (or stored time) 
> become outdated and the function (here the alarm_handler) continues with 
> them.

That's not entirely implausible.

> - besides this, when you look into local_irq_save for um, this does not 
> really deactivate the interrupts, but delay the processing and it adds a 
> memory barrier. I think that could be one of the important consequences 
> as changes to the event list are forced to be populated. But this is 
> only a guess :-D

No, it *does* disable interrupts from Linux's POV. The memory barrier is
just there to make _that_ actually visible "immediately".

Yes it doesn't actually disable the *signal* underneath, but that
doesn't really mean anything? Once you get the signal, nothing happens
if it's disabled.

In UML, the kernel binary is kind of both hypervisor and guest kernel,
so you can think of the low-level code here as being part of the
hypervisor, so that blocking the signal from actually calling out into
the guest is like blocking the PIC from sending to the CPU. Obviously
the device(s) still send(s) the interrupt(s), but they don't go into the
"guest".

> - I'm also not really convinced, that all accesses to the current time 
> should be delayed. I made a patch with a heuristic, that only delays the 
> time read, if it is read from userspace multiple times in a row.?

How would you even detect that though?

And on the other hand - why not? There's always a cost to things in a
real system, it's not free to access the current time? Maybe it's faster
in a real system with VDSO though.

> Since 
> there are no "busy" loops in the kernel itself and only in a few (bad 
> behaving) userspace programs, I think that helps to reduce the 
> inaccuracy in simulation from well behaving userspace programs only 
> reading the time once.

So I'm not sure I'd call it an inaccuracy? It's ... perfectly accurate
in the model that we're implementing? Maybe you don't like the model ;-)

> In this case, irqs_disabled more or less only tend to indicate, that the 
> event list could be manipulated, and therefore the update_time call is a 
> bad idea.

Which is kind of what I was thinking about earlier, but Vincent says
it's not _just_ for that.

We probably should just add a spinlock for this though - right now none
of this is good for eventual SMP support, and it's much easier to reason
about when we have a lock?

johannes


From Benjamin.Beichler at uni-rostock.de  Fri Oct 27 08:32:23 2023
From: Benjamin.Beichler at uni-rostock.de (Benjamin Beichler)
Date: Fri, 27 Oct 2023 17:32:23 +0200
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <68748578c04fd1c41d3969bc5348eb2498894ac7.camel@sipsolutions.net>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
 <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
 <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>
 <04afd005c5566fcc6a46549afd7b9136a7cec7a9.camel@axis.com>
 <a2a6c524ddb897d3c3b640d0f7262b506e51d983.camel@sipsolutions.net>
 <da07d5f1-f52a-428a-9884-0f2f1a3efdf3@uni-rostock.de>
 <68748578c04fd1c41d3969bc5348eb2498894ac7.camel@sipsolutions.net>
Message-ID: <8723dfcc-ecb4-4ff3-ba18-c3ab55bac264@uni-rostock.de>

Am 27.10.2023 um 16:45 schrieb Johannes Berg:
> On Fri, 2023-10-27 at 16:05 +0200, Benjamin Beichler wrote:
>> - besides this, when you look into local_irq_save for um, this does 
>> not really deactivate the interrupts, but delay the processing and it 
>> adds a memory barrier. I think that could be one of the important 
>> consequences as changes to the event list are forced to be populated. 
>> But this is only a guess :-D 
> No, it *does* disable interrupts from Linux's POV. The memory barrier 
> is just there to make _that_ actually visible "immediately". Yes it 
> doesn't actually disable the *signal* underneath, but that doesn't 
> really mean anything? Once you get the signal, nothing happens if it's 
> disabled.
Maybe I was a bit sloppy here. What I meant was, that a signal could 
interrupt the code
even the interrupts are disabled, resulting into all the nice preemption 
consequences.
In real systems that may only do the NMI ?
And especially the SIGIO handler keeps intentionally calling the time 
travel handlers
when the interrupts are disabled. The interrupt handlers of drivers are 
called later, but the
time travel handlers tend to change the event list.

> In UML, the kernel binary is kind of both hypervisor and guest kernel, 
> so you can think of the low-level code here as being part of the 
> hypervisor, so that blocking the signal from actually calling out into 
> the guest is like blocking the PIC from sending to the CPU. Obviously 
> the device(s) still send(s) the interrupt(s), but they don't go into 
> the "guest".
>> - I'm also not really convinced, that all accesses to the current 
>> time should be delayed. I made a patch with a heuristic, that only 
>> delays the time read, if it is read from userspace multiple times in 
>> a row. 
> How would you even detect that though? And on the other hand - why 
> not? There's always a cost to things in a real system, it's not free 
> to access the current time? Maybe it's faster in a real system with 
> VDSO though.
>> Since there are no "busy" loops in the kernel itself and only in a 
>> few (bad behaving) userspace programs, I think that helps to reduce 
>> the inaccuracy in simulation from well behaving userspace programs 
>> only reading the time once. 
> So I'm not sure I'd call it an inaccuracy? It's ... perfectly accurate 
> in the model that we're implementing? Maybe you don't like the model ;-)
I think I like the model too much, it fits so nicely in common DES 
primitives. :-D
We have unlimited processing power (or every processing has zero 
execution time),
so why delaying a program which get once a while a timestamp from the 
system.

Moreover, since I want a really deterministic model, I anticipate that 
if I send a msg
at timestamp t1, my program should create an answer at t1 and not 
sporadically at
t1+delta, because the file system driver took a timestamp at a 
background task.
>> In this case, irqs_disabled more or less only tend to indicate, that 
>> the event list could be manipulated, and therefore the update_time 
>> call is a bad idea. 
> Which is kind of what I was thinking about earlier, but Vincent says 
> it's not _just_ for that.
Mhh, I think Vincent only wondered about whether the recursion statement 
was right.

> We probably should just add a spinlock for this though - right now 
> none of this is good for eventual SMP support, and it's much easier to 
> reason about when we have a lock?
Mhh I think that would make sense. As I said, we also had problems in 
ext-mode, which
disappeared, after I introduce local_irq_save(flags); around all 
operation, that modified
the event list. I think a spinlock would show clearer the intention. For 
SMP we may
need some fine grain scheme (or RCU-based locking), to prevent 
deadlocks, but I'm again
not sure.
> johannes 
kind regards
Benjamin



From johannes at sipsolutions.net  Fri Oct 27 08:54:29 2023
From: johannes at sipsolutions.net (Johannes Berg)
Date: Fri, 27 Oct 2023 17:54:29 +0200
Subject: [PATCH v2] um: time-travel: fix time corruption
In-Reply-To: <8723dfcc-ecb4-4ff3-ba18-c3ab55bac264@uni-rostock.de>
References: <20231025224504.7be512cdfeb4.Ia20103ef9899e53311bd671b8cfbca4f057a4668@changeid>
	 <d3beded8a25acced5905c8f251a5e91d0f514185.camel@axis.com>
	 <39b6458f1214022e8a0d00c5cb70101a41296d74.camel@sipsolutions.net>
	 <04afd005c5566fcc6a46549afd7b9136a7cec7a9.camel@axis.com>
	 <a2a6c524ddb897d3c3b640d0f7262b506e51d983.camel@sipsolutions.net>
	 <da07d5f1-f52a-428a-9884-0f2f1a3efdf3@uni-rostock.de>
	 <68748578c04fd1c41d3969bc5348eb2498894ac7.camel@sipsolutions.net>
	 <8723dfcc-ecb4-4ff3-ba18-c3ab55bac264@uni-rostock.de>
Message-ID: <16f458fc2bc9b875ba96954332136b9ba6e143b2.camel@sipsolutions.net>

On Fri, 2023-10-27 at 17:32 +0200, Benjamin Beichler wrote:
> Am 27.10.2023 um 16:45 schrieb Johannes Berg:
> > On Fri, 2023-10-27 at 16:05 +0200, Benjamin Beichler wrote:
> > > - besides this, when you look into local_irq_save for um, this does 
> > > not really deactivate the interrupts, but delay the processing and it 
> > > adds a memory barrier. I think that could be one of the important 
> > > consequences as changes to the event list are forced to be populated. 
> > > But this is only a guess :-D 
> > No, it *does* disable interrupts from Linux's POV. The memory barrier 
> > is just there to make _that_ actually visible "immediately". Yes it 
> > doesn't actually disable the *signal* underneath, but that doesn't 
> > really mean anything? Once you get the signal, nothing happens if it's 
> > disabled.
> Maybe I was a bit sloppy here. What I meant was, that a signal could 
> interrupt the code even the interrupts are disabled,

Yes, briefly, to find it shouldn't do anything. So nothing should happen
because of it?

> resulting into all the nice preemption consequences.

What do you mean?

> In real systems that may only do the NMI ?
> And especially the SIGIO handler keeps intentionally calling the time 
> travel handlers
> when the interrupts are disabled. The interrupt handlers of drivers are 
> called later, but the
> time travel handlers tend to change the event list.

Ah! Yes, that's true too ... hmm.

> I think I like the model too much, it fits so nicely in common DES 
> primitives. :-D

:)

> We have unlimited processing power (or every processing has zero 
> execution time),
> so why delaying a program which get once a while a timestamp from the 
> system.

Yeah, OK, fair enough.

Still, we _do_ need this to make progress at all in some cases, like the
python case described there somewhere.

> Moreover, since I want a really deterministic model, I anticipate that 
> if I send a msg
> at timestamp t1, my program should create an answer at t1 and not 
> sporadically at
> t1+delta, because the file system driver took a timestamp at a 
> background task.

Well at least it should grab the background task at the same time every
time ;-) But yeah that's only really true if you have all of the patches
we have to nail down random number generation ...

> > > In this case, irqs_disabled more or less only tend to indicate, that 
> > > the event list could be manipulated, and therefore the update_time 
> > > call is a bad idea. 
> > Which is kind of what I was thinking about earlier, but Vincent says 
> > it's not _just_ for that.
> Mhh, I think Vincent only wondered about whether the recursion statement 
> was right.

No, I meant the fact that he reported that changing this to always delay
was breaking it.

> > We probably should just add a spinlock for this though - right now 
> > none of this is good for eventual SMP support, and it's much easier to 
> > reason about when we have a lock?
> Mhh I think that would make sense. As I said, we also had problems in 
> ext-mode, which
> disappeared, after I introduce local_irq_save(flags); around all 
> operation, that modified
> the event list. I think a spinlock would show clearer the intention. For 
> SMP we may
> need some fine grain scheme (or RCU-based locking), to prevent 
> deadlocks, but I'm again
> not sure.
> 

RCU isn't really locking, I don't see how that'd work in the face of
changing the list. :)

But a spinlock should work, we might need to also disable IRQs though.

Well anyway, I don't think I'll do any work here before you've also
posted your patches.

And maybe we should apply all the patches on the list now since they
make things better, and then reconsider the model more carefully with
the next set of changes?

johannes


From catalin.marinas at arm.com  Fri Oct 27 09:24:12 2023
From: catalin.marinas at arm.com (Catalin Marinas)
Date: Fri, 27 Oct 2023 17:24:12 +0100
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <20231009124210.1064021-4-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-4-masahiroy@kernel.org>
Message-ID: <ZTvkLIVmC3fnxX9p@arm.com>

On Mon, Oct 09, 2023 at 09:42:09PM +0900, Masahiro Yamada wrote:
> Currently, there is no standard implementation for vdso_install,
> leading to various issues:
[...]
>  arch/arm64/Makefile                    |  9 ++----
>  arch/arm64/kernel/vdso/Makefile        | 10 ------
>  arch/arm64/kernel/vdso32/Makefile      | 10 ------

For arm64:

Acked-by: Catalin Marinas <catalin.marinas at arm.com>


From linux at armlinux.org.uk  Fri Oct 27 12:48:47 2023
From: linux at armlinux.org.uk (Russell King (Oracle))
Date: Fri, 27 Oct 2023 20:48:47 +0100
Subject: [PATCH 4/5] kbuild: unify vdso_install rules
In-Reply-To: <20231009124210.1064021-4-masahiroy@kernel.org>
References: <20231009124210.1064021-1-masahiroy@kernel.org>
 <20231009124210.1064021-4-masahiroy@kernel.org>
Message-ID: <ZTwUH0uNESuWeKPR@shell.armlinux.org.uk>

On Mon, Oct 09, 2023 at 09:42:09PM +0900, Masahiro Yamada wrote:
>  arch/arm/Makefile                      |  7 +---
>  arch/arm/vdso/Makefile                 | 25 --------------

Acked-by: Russell King (Oracle) <rmk+kernel at armlinux.org.uk>

Thanks!

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!


