From val at versity.com  Mon Mar  3 09:02:24 2025
From: val at versity.com (Valerie Aurora)
Date: Mon, 3 Mar 2025 18:02:24 +0100
Subject: [PATCH 05/14] Add support for dtype
In-Reply-To: <20250228223115.GC2565306@localhost.localdomain>
References: <20250227141623.65291-1-val@versity.com> <20250227141623.65291-6-val@versity.com>
 <20250228223115.GC2565306@localhost.localdomain>
Message-ID: <CALGtELi7SKvTp6VuNZL16VZ734t2ggfnDmCkiq4rseYffJm2KQ@mail.gmail.com>

On Fri, Feb 28, 2025 at 11:31?PM Zach Brown <zab at zabbo.net> wrote:
>
> On Thu, Feb 27, 2025 at 03:16:14PM +0100, Valerie Aurora wrote:
> > From: Zach Brown <zab at zabbo.net>
> >
> > Add dtype support to avoid unnecesary inode reads.
> >
> > Co-authored-by: Valerie Aurora <val at versity.com>
> > Signed-off-by: Valerie Aurora <val at versity.com>
>
> Ah, this is also fine and I'll land it...
>
> > +     da->dtype = IFTODT(mode);
> >
> >       da->dent.ino = cpu_to_le64(ino);
> >       da->dent.version = cpu_to_le64(0); /* XXX :/ */
> > -     da->dent.dtype = ngnfs_dt(mode);
> > +     da->dent.type = mode_to_type(mode);
>
> I had missed that da->dtype is the actual vfs dtype, not persistent
> ngnfs enum, so it makes sense to use the linux helper.  Maybe we'll want
> to tighten up the names to make the distinction clear, but it's fine for
> now.

Thanks! I agree with changing the names, there are three kinds of
dtypes (VFS, in-memory, on-disk) and I got the conversion wrong
multiple times while writing these. Do you have any thoughts on a
better scheme? As you know, I'm terrible at naming things. :)

Valerie


From val at versity.com  Mon Mar  3 09:16:05 2025
From: val at versity.com (Valerie Aurora)
Date: Mon, 3 Mar 2025 18:16:05 +0100
Subject: [PATCH 06/14] Add parent inode number for directories
In-Reply-To: <20250228223543.GD2565306@localhost.localdomain>
References: <20250227141623.65291-1-val@versity.com> <20250227141623.65291-7-val@versity.com>
 <20250228223543.GD2565306@localhost.localdomain>
Message-ID: <CALGtELhqB7_Oq-kY7imz7mV03EX6P7a9tyrBtWz1dcub2+0NVA@mail.gmail.com>

On Fri, Feb 28, 2025 at 11:35?PM Zach Brown <zab at zabbo.net> wrote:
>
> On Thu, Feb 27, 2025 at 03:16:15PM +0100, Valerie Aurora wrote:
> > Add a parent inode field to the inode to give us back references for
> > directories. For non-directories, this field is 0 and back references
> > will go in a per-inode btree (TODO).
>
> > @@ -91,6 +91,7 @@ struct ngnfs_inode {
> >       __le64 gen;
> >       __le64 size;
> >       __le64 version;
> > +     __le64 parent_ino; /* only valid for directories */
> >       __le32 nlink;
> >       __le32 uid;
> >       __le32 gid;
>
> To be more specific, this is adding is a reference to a specific
> instance of an inode.  If we bit the bullet and introduced a persistent
> inode/gen reference struct we could use it here.
>
> The nice result of that is that we could return the target inode for
> lookup of ".." without having to actually read the referenced parent
> inode to get its gen.  (As we'll see later.)
>
> Maybe we make it something like:
>
>         struct ngnfs_ino_gen {
>                 __le64 ino;
>                 __le64 gen;
>         };
>
> Too literal?

Yes, I like! I think what you are saying is to replace all inode
numbers with an ino_gen struct, including in say
update_dirent_args_ino(), correct? I have a commit in my stack right
now to correctly copy around the gen number with the ino which I can
easily update to this.

Valerie


From val at versity.com  Mon Mar  3 09:20:59 2025
From: val at versity.com (Valerie Aurora)
Date: Mon, 3 Mar 2025 18:20:59 +0100
Subject: [PATCH 13/14] Add nfi, txn, and flags to dirent_args
In-Reply-To: <20250228234043.GI2565306@localhost.localdomain>
References: <20250227141623.65291-1-val@versity.com> <20250227141623.65291-14-val@versity.com>
 <20250228234043.GI2565306@localhost.localdomain>
Message-ID: <CALGtELjAGx892nAkrbbQWfAbL0L26XR7H3x-B70WcN4zZ7YS+w@mail.gmail.com>

On Sat, Mar 1, 2025 at 12:40?AM Zach Brown <zab at zabbo.net> wrote:
>
> On Thu, Feb 27, 2025 at 03:16:22PM +0100, Valerie Aurora wrote:
> > For directory operations like unlink(), rmdir(), and rename(), the
> > dirent_args struct needs to carry all necessary info down to the btree
> > iterator operator function. unlink() needs to know whether it is
> > attempting to remove a directory or a file, so add a flags field to
> > dirent_args. rmdir() needs to read the dir inode to get the link
> > count, so add the nfi and txn pointers. rename() needs to know if the
> > entry is just being moved to different directory, so add a flag.
>
> Hopefully this won't be needed, as discussed in review of the addition
> of unlink..

Yes, I am happily dropping this one! Thanks for the clearing that up,

Valerie


From val at versity.com  Mon Mar  3 09:23:44 2025
From: val at versity.com (Valerie Aurora)
Date: Mon, 3 Mar 2025 18:23:44 +0100
Subject: [PATCH 12/14] Add ngnfs_dir_mkdir() and debugfs command
In-Reply-To: <20250228233745.GH2565306@localhost.localdomain>
References: <20250227141623.65291-1-val@versity.com> <20250227141623.65291-13-val@versity.com>
 <20250228233745.GH2565306@localhost.localdomain>
Message-ID: <CALGtELhnZ1EeC4ZmdVj-E0kxeohqaW=sbWGDdi=6UPtewTdixQ@mail.gmail.com>

On Sat, Mar 1, 2025 at 12:37?AM Zach Brown <zab at zabbo.net> wrote:
>
> On Thu, Feb 27, 2025 at 03:16:21PM +0100, Valerie Aurora wrote:
> > Makes a directory of the specified name in the current working
> > directory in debugfs.
>
> > +/*
> > + * Create a new directory and add a directory entry referencing it.
> > + */
> > +int ngnfs_dir_mkdir(struct ngnfs_fs_info *nfi, u64 dir_ino, umode_t mode, char *name,
> > +                 size_t name_len)
> > +{
> > +     struct {
> > +             struct ngnfs_transaction txn;
> > +             struct ngnfs_inode_txn_ref dir;
> > +             struct ngnfs_inode_txn_ref new_dir;
> > +             u64 ino;
> > +             u64 nsec;
> > +             struct dirent_args da;
> > +     } *op;
> > +     int ret;
> > +
> > +     op = kmalloc(sizeof(*op), GFP_NOFS);
> > +     if (!op) {
> > +             ret = -ENOMEM;
> > +             goto out;
> > +     }
> > +
> > +     ngnfs_txn_init(&op->txn);
> > +     op->nsec = ktime_to_ns(ktime_get_real());
> > +     init_dirent_args(&op->da, name, name_len, 0, mode | S_IFDIR);
> > +
> > +     do {
> > +             ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir_ino, &op->dir)              ?:
> > +                   check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)                             ?:
> > +                   ngnfs_inode_alloc(nfi, &op->txn, &op->ino, &op->new_dir)                  ?:
> > +                   ngnfs_inode_init(&op->new_dir, op->ino, 1, 2, mode | S_IFDIR, op->nsec,
> > +                                    dir_ino)                                                 ?:
> > +                   update_dirent_args_ino(&op->da, op->ino)                                  ?:
> > +                   insert_dirent(nfi, &op->txn, &op->dir, &op->da)                           ?:
> > +                   update_dir(op->dir.tblk, op->dir.ninode, &op->da, 1);
> > +
> > +     } while (ngnfs_txn_retry(nfi, &op->txn, &ret));
> > +
> > +     ngnfs_txn_teardown(nfi, &op->txn);
> > +     kfree(op);
> > +out:
> > +     return ret;
> > +}
>
> This is just _create() with different nlink and mode, right?  We
> probably do want different _create, _mkdir, _symlink, _mknod exposed
> calls, but we can have them each call a core_create(), or whatever, that
> takes nlink and dev_t provided by each calling shim.

Makes sense, will do! I'll put in the args that we need now and add
more as as we implement mknod(), etc.

Valerie


From zab at zabbo.net  Mon Mar  3 10:02:46 2025
From: zab at zabbo.net (Zach Brown)
Date: Mon, 3 Mar 2025 10:02:46 -0800
Subject: [PATCH 05/14] Add support for dtype
In-Reply-To: <CALGtELi7SKvTp6VuNZL16VZ734t2ggfnDmCkiq4rseYffJm2KQ@mail.gmail.com>
References: <20250227141623.65291-1-val@versity.com>
 <20250227141623.65291-6-val@versity.com>
 <20250228223115.GC2565306@localhost.localdomain>
 <CALGtELi7SKvTp6VuNZL16VZ734t2ggfnDmCkiq4rseYffJm2KQ@mail.gmail.com>
Message-ID: <20250303180246.GA2948017@localhost.localdomain>

On Mon, Mar 03, 2025 at 06:02:24PM +0100, Valerie Aurora wrote:
> On Fri, Feb 28, 2025 at 11:31?PM Zach Brown <zab at zabbo.net> wrote:

> > I had missed that da->dtype is the actual vfs dtype, not persistent
> > ngnfs enum, so it makes sense to use the linux helper.  Maybe we'll want
> > to tighten up the names to make the distinction clear, but it's fine for
> > now.
> 
> Thanks! I agree with changing the names, there are three kinds of
> dtypes (VFS, in-memory, on-disk) and I got the conversion wrong
> multiple times while writing these. Do you have any thoughts on a
> better scheme? As you know, I'm terrible at naming things. :)

I don't think that I do :(.  I, too, am just pants at naming things.

I guess I'd throw up my hands and be tediously literal: abi_dtype,
dtype, pers_dtype ?

- z


From zab at zabbo.net  Mon Mar  3 10:03:51 2025
From: zab at zabbo.net (Zach Brown)
Date: Mon, 3 Mar 2025 10:03:51 -0800
Subject: [PATCH 06/14] Add parent inode number for directories
In-Reply-To: <CALGtELhqB7_Oq-kY7imz7mV03EX6P7a9tyrBtWz1dcub2+0NVA@mail.gmail.com>
References: <20250227141623.65291-1-val@versity.com>
 <20250227141623.65291-7-val@versity.com>
 <20250228223543.GD2565306@localhost.localdomain>
 <CALGtELhqB7_Oq-kY7imz7mV03EX6P7a9tyrBtWz1dcub2+0NVA@mail.gmail.com>
Message-ID: <20250303180351.GB2948017@localhost.localdomain>

On Mon, Mar 03, 2025 at 06:16:05PM +0100, Valerie Aurora wrote:
> On Fri, Feb 28, 2025 at 11:35?PM Zach Brown <zab at zabbo.net> wrote:

> > Maybe we make it something like:
> >
> >         struct ngnfs_ino_gen {
> >                 __le64 ino;
> >                 __le64 gen;
> >         };
> >
> > Too literal?
> 
> Yes, I like! I think what you are saying is to replace all inode
> numbers with an ino_gen struct, including in say
> update_dirent_args_ino(), correct? I have a commit in my stack right
> now to correctly copy around the gen number with the ino which I can
> easily update to this.

Where the inode numbers we're replacing are references to a specific
instance of an allocated inode, indeed, that sounds just right.  Thanks!

- z


From val at versity.com  Mon Mar  3 10:10:44 2025
From: val at versity.com (Valerie Aurora)
Date: Mon, 3 Mar 2025 19:10:44 +0100
Subject: [PATCH 09/14] Add ngnfs_dir_get_handle() for using files/dirs
In-Reply-To: <20250228233116.GG2565306@localhost.localdomain>
References: <20250227141623.65291-1-val@versity.com> <20250227141623.65291-10-val@versity.com>
 <20250228233116.GG2565306@localhost.localdomain>
Message-ID: <CALGtELiZ7eEChfBNnmxkaz=fMGjRMyL6MbATP7t4Hr0E6td+-Q@mail.gmail.com>

On Sat, Mar 1, 2025 at 12:31?AM Zach Brown <zab at zabbo.net> wrote:
>
> On Thu, Feb 27, 2025 at 03:16:18PM +0100, Valerie Aurora wrote:
> > ngnfs_dir_get_handle() returns a handle that can be used to do
> > operations on files and directories.
>
> > +int ngnfs_dir_get_handle(struct ngnfs_fs_info *nfi, u64 dir_ino, char *name, size_t name_len,
> > +                      struct ngnfs_dir_handle *hdl)
>
> Handle's an unfortunate collision with the open_by_handle() syscalls --
> let's not use that.  Similarly, descriptor presents a problem with
> expectations (having one of these does not protect the inode it
> describes).

I took "like open_by_handle()" a little too literally. :)

> Can we pull back from trying to generalize and call it what it is?  It's
> just the contents of the dirent at the name.
>
>         struct ngnfs_lookup_entry {
>                 struct ngnfs_ino_gen ig;
>                 u8 d_type;
>         };
>
> Does that look OK?
>
> Then all the methods that take u64 ino would take the ig.  (Which we
> want to do for the long term so that we can return errors if the inode
> number is deleted and re-used under a caller.)

Sounds great, thanks!

Valerie


Valerie


From val at versity.com  Wed Mar  5 06:55:09 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 5 Mar 2025 15:55:09 +0100
Subject: [PATCH 14/14] Add ngnfs_dir_unlink(), ngfs_dir_rmdir(), and
 debugfs commands
In-Reply-To: <20250228235648.GJ2565306@localhost.localdomain>
References: <20250227141623.65291-1-val@versity.com> <20250227141623.65291-15-val@versity.com>
 <20250228235648.GJ2565306@localhost.localdomain>
Message-ID: <CALGtELiAoO-y98-enuat9ywq-6zEuNwHxwKDkNz=nk=hyL9+yQ@mail.gmail.com>

On Sat, Mar 1, 2025 at 12:56?AM Zach Brown <zab at zabbo.net> wrote:
>
> On Thu, Feb 27, 2025 at 03:16:23PM +0100, Valerie Aurora wrote:
> > Allow removal of links to files and directories.
>
> > +static int remove_dirent_wr(struct ngnfs_btree_key *key, void *val, size_t size, void *arg,
> > +                         struct ngnfs_btree_op *op)
> > +{
> > +     struct ngnfs_dirent *dent = val;
> > +     struct dirent_args *da = arg;
> > +     int ret;
> > +
> > +     if (!dent)
> > +             return -ENOENT;
> > +
> > +     if (!names_equal(dent->name, dent->name_len, da->dent.name, da->dent.name_len))
> > +             return NGNFS_BTREE_ITER_CONTINUE;
>
> We need to be careful to return ENOENT if the last key value was
> populated and had a colliding hash, but didn't match the name being
> removed.  We only get the null val call in write iterators if the very
> last key value doesn't exist.  It can exist, and not be the name we're
> looking for, and in that case here I think we'll just continue out of
> the key range and return success.

Thanks, fixed! On the side, I'm trying to think of ways to make the
iter() functions a little harder to misuse since this is the second
time I've made a similar mistake... maybe a standard way to indicate
if the iter() function was ever called and returned 0?

> > +     /* ino and dtype needed for updating inode and parent directory */
> > +     da->ino = le64_to_cpu(dent->ino);
> > +     da->dtype = ngnfs_type_to_dtype(dent->type);
> > +
> > +     ret = check_remove_dirent(dent, da);
> > +     if (ret < 0)
> > +             return ret;
> > +
>
> Hmm, I wonder if there's been a bit of miscommunication.  When we were
> talking about checking errors in the btree write iterator, the only
> errors I meant were those related to the btree items that are being
> iterated over :).  It lets us return ENOENT/EEXIST as we go to
> remove/create dirents instead of having separate additional traversals
> to validate the names before we go to modify them.
>
> These checks of the inode should be done up in the operation.  Like
> check_ifmt() is done.  We want to see the resources for those inodes and
> see them being worked on in the high level operational steps rather than
> it being down in the iterators.

Phew! Yes, that was miscommunication, thanks so much for clearing that up.

> > +             ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir_ino, &op->dir)              ?:
> > +                   check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)                             ?:
> > +                   remove_dirent(nfi, &op->txn, &op->dir, &op->da)                           ?:
> > +                   ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, op->da.ino, &op->da.inode)      ?:
> > +                   ngnfs_inode_update(op->da.inode.tblk, op->da.inode.ninode, -1)            ?:
> > +                   update_dir(op->dir.tblk, op->dir.ninode, &op->da, -1);
>
> So we'd add those checks on the op->da.inode here after reading it,
> rather than reading it again down in remove_dirent.   Could probably
> move it up a bit as well so that we work from cheapest to most expensive
> amongst the independent operations and checks that could fail.  (The
> btree junk is always potentially expensive because it can split/join
> blocks and have secondary costs working with allocators.)

Agreed! The options for ordering checks from cheap->expensive as I
understand them are:

1. lookup the dirent, read the inode, do the final checks, then remove
the dirent (two btree walks, saves rare btree merges)

2. remove the dirent, read the inode, do the final checks (saves one
btree walk, may merge btree nodes unnecessarily)

>From your experience with scoutfs, you will have a better sense of
what is the right tradeoff (or a better solution).

> > +/*
> > + * Update an inode to reflect the addition or removal of a link to it.
> > + */
> > +int ngnfs_inode_update(struct ngnfs_txn_block *tblk, struct ngnfs_inode *inode, int posneg)
> > +{
> > +     s32 delta;
> > +
> > +     delta = posneg * 1;
> > +     if ((le32_to_cpu(inode->nlink) + delta >= NGNFS_LINK_MAX))
> > +             return -EMLINK;
> > +
> > +     ngnfs_tblk_assign(tblk, inode->nlink, cpu_to_le32(le32_to_cpu(inode->nlink) + delta));
> > +
> > +     return 0;
> > +}
>
> Ugh, sorry for writing that posneg pattern.  It stinks.  Now that we're
> promoting it to a proper function, can we make it look a lot less weird
> and just pass in the s32 nlink delta and have it return errors for both
> over or underflow?

Done! And fixed several bugs while at it, yikes, so thanks for the review!

Valerie


From val at versity.com  Tue Mar 11 10:39:26 2025
From: val at versity.com (Valerie Aurora)
Date: Tue, 11 Mar 2025 18:39:26 +0100
Subject: [PATCH 08/14] Add ngnfs_dir_get_dirent() and debugfs lookup command
In-Reply-To: <20250228231737.GF2565306@localhost.localdomain>
References: <20250227141623.65291-1-val@versity.com> <20250227141623.65291-9-val@versity.com>
 <20250228231737.GF2565306@localhost.localdomain>
Message-ID: <CALGtELj6ic5bTE0uxqYwf_2F1czw1gxp42z-3BEE0dwqh+qtHA@mail.gmail.com>

On Sat, Mar 1, 2025 at 12:17?AM Zach Brown <zab at zabbo.net> wrote:
>
> I don't think lookup/get_dirent should include the name with the dirent
> it returns.  We only return the dirent if the name matches so they
> already have it.  And this would get rid of the layers of duplicate
> allocation and copies of the name.  We just have to make the API clear
> that you're only getting the dirent struct without the name.

Fixed, thank you!

> > +     init_dirent_key(&op->key, da->hash);
> > +     init_dirent_key(&op->last, da->hash | NGNFS_DIRENT_COLL_BIT);
>
> Are these two keys used?  I think they were replaced by the on-stack
> keys in lookup_iter() (which should maybe be called something like
> lookup_hash?).

Also fixed, thanks again!

Valerie


From val at versity.com  Wed Mar 12 07:33:39 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:39 +0100
Subject: [PATCH 00/14] ino_gen, docs, unlink/mkdir/rename
Message-ID: <20250312143354.220766-1-val@versity.com>

I'm trying to share my code earlier, so here are all my current
patches, even though the last three definitely aren't ready. They
include:

* Fixes to all code review comments from the previous round
* A tiny bit of documentation
* My current version of rename()
* The create and rename benchmarks

I am certain that rename() isn't ready but everything up to that point
seems reasonable and is passing my resurrected test suite. Feel free
to stop reviewing at any point. :) Thanks!

Valerie Aurora (14):
  Improve naming of dtype code
  Send quit signal to process, not process group
  Shutdown devd process cleanly for tests
  Add brief instructions on how to run ngnfs
  Move inode and generation into ino_gen struct
  Add parent inode number/generation for directories
  Add ngnfs_dir_lookup() and debugfs lookup command
  Extend debugfs stat command to take an optional file name
  Add the cd command to debugfs
  Add ngnfs_dir_mkdir() and debugfs command
  Add ngnfs_dir_unlink(), ngfs_dir_rmdir(), and debugfs commands
  Add ngnfs_dir_rename() and very basic debugfs command
  Add rename benchmark command to debugfs
  Add create benchmark command to debugfs

 README.md             |   7 +-
 cli/debugfs.c         | 304 ++++++++++++++++++++-
 devd/btr-aio.c        |   8 +-
 docs/development.md   |  31 +++
 shared/dir.c          | 606 +++++++++++++++++++++++++++++++++++++++---
 shared/dir.h          |  37 ++-
 shared/format-block.h |  31 ++-
 shared/inode.c        |  62 +++--
 shared/inode.h        |  21 +-
 shared/mkfs.c         |   5 +-
 10 files changed, 1012 insertions(+), 100 deletions(-)

-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:40 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:40 +0100
Subject: [PATCH 01/15] Improve naming of dtype code
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-2-val@versity.com>

To more easily distinguish the POSIX dtype and the persistent dtype
used by ngnfs in its directory entries, name the POSIX dtype
"abi_dtype" and the ngnfs persistent dtype "pers_dtype". Also remove
the unnecessary dtype member of dirent_args, remove the unnecessary
export of ngnfs dtype translation, and fix a latent bug in
update_dir() where the type of the parent directory was used when the
type of the child dentry should have been used (adding a child dir
increases nlink of parent dir).

Signed-off-by: Valerie Aurora <val at versity.com>
---
 shared/dir.c          | 21 ++++++++++++---------
 shared/dir.h          |  4 ----
 shared/format-block.h |  2 +-
 3 files changed, 13 insertions(+), 14 deletions(-)

diff --git a/shared/dir.c b/shared/dir.c
index a0c698f..71a12f3 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -54,7 +54,6 @@ static u64 name_hash(void *name, size_t name_len)
 struct dirent_args {
 	u64 hash;
 	size_t dent_size;
-	u8 dtype;
 	u64 ino;
 
 	struct ngnfs_dirent dent;
@@ -68,8 +67,10 @@ static void init_dirent_key(struct ngnfs_btree_key *key, u64 hash)
 	};
 }
 
-/* Convert a POSIX file mode to an ngnfs on-disk dirent type */
-static enum ngnfs_dentry_type mode_to_type(umode_t mode)
+/*
+ * Convert a POSIX file mode to an ngnfs persistent dirent type.
+ */
+static enum ngnfs_dentry_type mode_to_pers_type(umode_t mode)
 {
 #define S_SHIFT 12
 	static unsigned char mode_types[S_IFMT >> S_SHIFT] = {
@@ -86,8 +87,11 @@ static enum ngnfs_dentry_type mode_to_type(umode_t mode)
 #undef S_SHIFT
 }
 
-/* Convert the on-disk ngnfs dirent dtype to the POSIX d_type */
-unsigned int ngnfs_type_to_dtype(enum ngnfs_dentry_type type)
+/*
+ * Convert the persistent ngnfs directoy entry type to the POSIX ABI
+ * dtype.
+ */
+static unsigned int pers_dtype_to_abi_dtype(enum ngnfs_dentry_type type)
 {
 	static unsigned char types[] = {
 		[NGNFS_DT_FIFO]	= DT_FIFO,
@@ -110,11 +114,10 @@ static void init_dirent_args(struct dirent_args *da, char *name, size_t name_len
 {
 	da->hash = name_hash(name, name_len);
 	da->dent_size = offsetof(struct ngnfs_dirent, name) + name_len;
-	da->dtype = IFTODT(mode);
 
 	da->dent.ino = cpu_to_le64(ino);
 	da->dent.version = cpu_to_le64(0); /* XXX :/ */
-	da->dent.type = mode_to_type(mode);
+	da->dent.pers_dtype = mode_to_pers_type(mode);
 	da->dent.name_len = name_len;
 
 	/* ensure that we're stitching together a contiguous max name buffer */
@@ -150,7 +153,7 @@ static int update_dir(struct ngnfs_txn_block *tblk, struct ngnfs_inode *dir,
 	s32 delta;
 	int ret;
 
-	if (S_ISDIR(le32_to_cpu(dir->mode))) {
+	if (da->dent.pers_dtype == NGNFS_DT_DIR) {
 		delta = posneg * 1;
 		if ((le32_to_cpu(dir->nlink) + delta >= NGNFS_LINK_MAX)) {
 			ret = -EMLINK;
@@ -296,7 +299,7 @@ static int fill_dirent_rd(struct ngnfs_btree_key *key, void *val, size_t val_siz
 	ra->ent->ino = le64_to_cpu(dent->ino);
 	ra->ent->gen = le64_to_cpu(dent->version);
 	ra->ent->next_offset = aligned;
-	ra->ent->dtype = ngnfs_type_to_dtype(dent->type);
+	ra->ent->dtype = pers_dtype_to_abi_dtype(dent->pers_dtype);
 	ra->ent->name_len = dent->name_len;
 	memcpy(ra->ent->name, dent->name, dent->name_len);
 	ra->ent->name[ra->ent->name_len] = '\0';
diff --git a/shared/dir.h b/shared/dir.h
index b333686..5594162 100644
--- a/shared/dir.h
+++ b/shared/dir.h
@@ -38,8 +38,4 @@ struct ngnfs_readdir_entry {
 int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, u64 dir_ino, u64 pos,
 		      struct ngnfs_readdir_entry *buf, size_t size);
 
-enum ngnfs_dentry_type;
-
-unsigned int ngnfs_type_to_dtype(enum ngnfs_dentry_type type);
-
 #endif
diff --git a/shared/format-block.h b/shared/format-block.h
index 65274e1..95ba9ff 100644
--- a/shared/format-block.h
+++ b/shared/format-block.h
@@ -125,7 +125,7 @@ enum ngnfs_dentry_type {
 struct ngnfs_dirent {
 	__le64 ino;
 	__le64 version; /* XXX ? */
-	__u8 type;
+	__u8 pers_dtype; /* ngnfs persistent directory entry type */
 	__u8 name_len; /* no null termination */
 	__u8 name[6]; /* definition pads to alignment, stored can be smaller */
 };
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:41 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:41 +0100
Subject: [PATCH 02/15] Send quit signal to process, not process group
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-3-val@versity.com>

The quit command in debugfs was sending a signal to every process in
the process group (using 0 for pid argument of kill()), which breaks
the ngnfs tests scripts because the shell script running debugfs is
part of the same process group and gets the signal too. Send the
signal only to this process, instead.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 1972f13..0b2223a 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -246,7 +246,7 @@ out:
 	free(line);
 	free(line_argv);
 	ctx->ret = ret;
-	kill(0, SIGUSR1);
+	kill(getpid(), SIGUSR1);
 }
 
 /*
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:44 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:44 +0100
Subject: [PATCH 05/15] Move inode number and generation into ino_gen struct
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-6-val@versity.com>

Files/dirs are uniquely identifed by the combination of inode number
and generation number. The generation number is incremented every time
an inode is freed and reallocated. In almost every case, we should use
both inode and generation number when performing operations. Encourage
this by using a combination inode number/generation struct. Also
remove the unused inode number from init_dirent_args().

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c         | 21 +++++++++++----------
 shared/dir.c          | 41 ++++++++++++++++++++---------------------
 shared/dir.h          | 18 +++++++++---------
 shared/format-block.h | 22 ++++++++++++++++++----
 shared/inode.c        | 34 +++++++++++++++++++---------------
 shared/inode.h        | 20 +++++++++++++-------
 shared/mkfs.c         |  5 +++--
 7 files changed, 93 insertions(+), 68 deletions(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 0b2223a..a3ef5f8 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -26,7 +26,7 @@
 
 struct debugfs_context {
 	struct ngnfs_fs_info *nfi;
-	u64 cwd_ino;
+	struct ngnfs_inode_ino_gen cwd_ig;
 	bool quit;
 	int ret;
 };
@@ -43,7 +43,7 @@ static void cmd_create(struct debugfs_context *ctx, int argc, char **argv)
 		return;
 	}
 
-	ret = ngnfs_dir_create(ctx->nfi, ctx->cwd_ino, 0644, argv[1], strlen(argv[1]));
+	ret = ngnfs_dir_create(ctx->nfi, &ctx->cwd_ig, 0644, argv[1], strlen(argv[1]));
 	if (ret < 0)
 		printf("create error: "ENOF"\n", ENOA(-ret));
 }
@@ -87,7 +87,7 @@ static void cmd_readdir(struct debugfs_context *ctx, int argc, char **argv)
 
 	pos = 0;
 	while (true) {
-		ret = ngnfs_dir_readdir(ctx->nfi, ctx->cwd_ino, pos, buf, size);
+		ret = ngnfs_dir_readdir(ctx->nfi, &ctx->cwd_ig, pos, buf, size);
 		if (ret <= 0) {
 			if (ret < 0)
 				printf("readdir error: "ENOF"\n", ENOA(-ret));
@@ -96,8 +96,8 @@ static void cmd_readdir(struct debugfs_context *ctx, int argc, char **argv)
 
 		ent = buf;
 		for (i = 0; i < ret; i++) {
-			printf("%020llu %10llu %5llu %05o %.*s\n", ent->pos, ent->ino, ent->gen,
-			       ent->dtype, ent->name_len, ent->name);
+			printf("%020llu %10llu %5llu %05o %.*s\n", ent->pos, ent->ig.ino,
+			       ent->ig.gen, ent->dtype, ent->name_len, ent->name);
 			pos = ent->pos;
 			ent = (void *)ent + ent->next_offset;
 		}
@@ -113,10 +113,11 @@ static void cmd_readdir(struct debugfs_context *ctx, int argc, char **argv)
 
 static void cmd_stat(struct debugfs_context *ctx, int argc, char **argv)
 {
+	struct ngnfs_inode_ino_gen root_ig = INIT_NGNFS_ROOT_IG;
 	struct ngnfs_inode ninode;
 	int ret;
 
-	ret = ngnfs_inode_read_copy(ctx->nfi, NGNFS_ROOT_INO, &ninode, sizeof(ninode));
+	ret = ngnfs_inode_read_copy(ctx->nfi, &root_ig, &ninode, sizeof(ninode));
 
 	if (ret < 0) {
 		log("stat error: %d", ret);
@@ -132,8 +133,8 @@ static void cmd_stat(struct debugfs_context *ctx, int argc, char **argv)
 		       "ctime: %llu\n"
 		       "mtime: %llu\n"
 		       "crtime: %llu\n",
-		       le64_to_cpu(ninode.ino),
-		       le64_to_cpu(ninode.gen),
+		       le64_to_cpu(ninode.ig.ino),
+		       le64_to_cpu(ninode.ig.gen),
 		       le32_to_cpu(ninode.nlink),
 		       le32_to_cpu(ninode.mode),
 		       le64_to_cpu(ninode.atime_nsec),
@@ -228,7 +229,7 @@ static void debugfs_thread(struct thread *thr, void *arg)
 	qsort(commands, ARRAY_SIZE(commands), sizeof(commands[0]), compar_cmd_names);
 
 	for (;;) {
-		fprintf(stdout, "<%llu> $ ", ctx->cwd_ino);
+		fprintf(stdout, "<%llu> $ ", ctx->cwd_ig.ino);
 		fflush(stdout);
 		if (!fgets(line, LINE_SIZE, stdin))
 			break;
@@ -261,7 +262,7 @@ static int debugfs_func(int argc, char **argv)
 	struct ngnfs_fs_info nfi = INIT_NGNFS_FS_INFO;
 	struct debugfs_context ctx = {
 		.nfi = &nfi,
-		.cwd_ino = NGNFS_ROOT_INO,
+		.cwd_ig = { NGNFS_ROOT_INO, 1 },
 		.quit = false,
 	};
 	struct thread thr;
diff --git a/shared/dir.c b/shared/dir.c
index 71a12f3..76afda0 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -52,9 +52,9 @@ static u64 name_hash(void *name, size_t name_len)
  * copy in and out any dirent as we work with them.
  */
 struct dirent_args {
+	struct ngnfs_inode_ino_gen ig;
 	u64 hash;
 	size_t dent_size;
-	u64 ino;
 
 	struct ngnfs_dirent dent;
 	u8 __max_name_storage[NGNFS_NAME_MAX - sizeof_field(struct ngnfs_dirent, name)];
@@ -109,29 +109,28 @@ static unsigned int pers_dtype_to_abi_dtype(enum ngnfs_dentry_type type)
 	return DT_UNKNOWN;
 }
 
-static void init_dirent_args(struct dirent_args *da, char *name, size_t name_len, u64 ino,
-			     mode_t mode)
+static void init_dirent_args(struct dirent_args *da, char *name, size_t name_len, mode_t mode)
 {
 	da->hash = name_hash(name, name_len);
 	da->dent_size = offsetof(struct ngnfs_dirent, name) + name_len;
 
-	da->dent.ino = cpu_to_le64(ino);
-	da->dent.version = cpu_to_le64(0); /* XXX :/ */
+	da->dent.ig.ino = 0;
+	da->dent.ig.gen = 0;
 	da->dent.pers_dtype = mode_to_pers_type(mode);
 	da->dent.name_len = name_len;
+	memcpy(da->dent.name, name, name_len);
 
 	/* ensure that we're stitching together a contiguous max name buffer */
 	BUILD_BUG_ON(offsetofend(struct dirent_args, dent.name) !=
 		     offsetof(struct dirent_args, __max_name_storage));
 	BUILD_BUG_ON((sizeof_field(struct dirent_args, dent.name) +
 		      sizeof_field(struct dirent_args, __max_name_storage)) != NGNFS_NAME_MAX);
-
-	memcpy(da->dent.name, name, name_len);
 }
 
-static int update_dirent_args_ino(struct dirent_args *da, u64 ino)
+static int update_dirent_args_ig(struct dirent_args *da, struct ngnfs_inode_ino_gen *ig)
 {
-	da->dent.ino = cpu_to_le64(ino);
+	da->dent.ig.ino = cpu_to_le64(ig->ino);
+	da->dent.ig.gen = cpu_to_le64(ig->gen);
 
 	return 0;
 }
@@ -236,14 +235,14 @@ static int insert_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *tx
 /*
  * Allocate a new inode and add a directory entry referencing it.
  */
-int ngnfs_dir_create(struct ngnfs_fs_info *nfi, u64 dir_ino, umode_t mode, char *name,
-		     size_t name_len)
+int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
+		     char *name, size_t name_len)
 {
 	struct {
 		struct ngnfs_transaction txn;
 		struct ngnfs_inode_txn_ref dir;
 		struct ngnfs_inode_txn_ref inode;
-		u64 ino;
+		struct ngnfs_inode_ino_gen ig;
 		u64 nsec;
 		struct dirent_args da;
 	} *op;
@@ -257,14 +256,14 @@ int ngnfs_dir_create(struct ngnfs_fs_info *nfi, u64 dir_ino, umode_t mode, char
 
 	ngnfs_txn_init(&op->txn);
 	op->nsec = ktime_to_ns(ktime_get_real());
-	init_dirent_args(&op->da, name, name_len, 0, mode | S_IFREG);
+	init_dirent_args(&op->da, name, name_len, mode | S_IFREG);
 
 	do {
-		ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir_ino, &op->dir)		?:
+		ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir, &op->dir)			?:
 		      check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)				?:
-		      ngnfs_inode_alloc(nfi, &op->txn, &op->ino, &op->inode)			?:
-		      ngnfs_inode_init(&op->inode, op->ino, 1, 1, mode | S_IFREG, op->nsec)	?:
-		      update_dirent_args_ino(&op->da, op->ino)					?:
+		      ngnfs_inode_alloc(nfi, &op->txn, &op->ig, &op->inode)			?:
+		      ngnfs_inode_init(&op->inode, &op->ig, 1, mode | S_IFREG, op->nsec)	?:
+		      update_dirent_args_ig(&op->da, &op->ig)					?:
 		      insert_dirent(nfi, &op->txn, &op->dir, &op->da)				?:
 		      update_dir(op->dir.tblk, op->dir.ninode, &op->da, 1);
 
@@ -296,8 +295,8 @@ static int fill_dirent_rd(struct ngnfs_btree_key *key, void *val, size_t val_siz
 	aligned = ALIGN(bytes, __alignof__(struct ngnfs_readdir_entry));
 
 	ra->ent->pos = le64_to_cpu(key->k[0]);
-	ra->ent->ino = le64_to_cpu(dent->ino);
-	ra->ent->gen = le64_to_cpu(dent->version);
+	ra->ent->ig.ino = le64_to_cpu(dent->ig.ino);
+	ra->ent->ig.gen = le64_to_cpu(dent->ig.gen);
 	ra->ent->next_offset = aligned;
 	ra->ent->dtype = pers_dtype_to_abi_dtype(dent->pers_dtype);
 	ra->ent->name_len = dent->name_len;
@@ -333,7 +332,7 @@ static int fill_dirent_rd(struct ngnfs_btree_key *key, void *val, size_t val_siz
  * the last entry so that the caller doesn't have to discover eof with a
  * final call that returns nothing.)
  */
-int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, u64 dir_ino, u64 pos,
+int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_ig, u64 pos,
 		      struct ngnfs_readdir_entry *buf, size_t size)
 {
 	struct ngnfs_inode_txn_ref dir;
@@ -355,7 +354,7 @@ int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, u64 dir_ino, u64 pos,
 		ra.size = size;
 		ra.nr = 0;
 
-		ret = ngnfs_inode_get(nfi, &txn, NBF_READ, dir_ino, &dir)			?:
+		ret = ngnfs_inode_get(nfi, &txn, NBF_READ, dir_ig, &dir)			?:
 		      check_ifmt(dir.ninode, S_IFDIR, -ENOTDIR)					?:
 		      ngnfs_btree_read_iter(nfi, &txn, &dir.ninode->dirents, &key,
 					    NULL, NULL, fill_dirent_rd, &ra);
diff --git a/shared/dir.h b/shared/dir.h
index 5594162..8ca28b8 100644
--- a/shared/dir.h
+++ b/shared/dir.h
@@ -2,25 +2,25 @@
 #ifndef NGNFS_SHARED_DIR_H
 #define NGNFS_SHARED_DIR_H
 
+#include "shared/inode.h"
 #include "shared/fs_info.h"
 
-int ngnfs_dir_create(struct ngnfs_fs_info *nfi, u64 dir_ino, umode_t mode, char *name,
-		     size_t name_len);
+int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
+		     char *name, size_t name_len);
 
 /*
- * Readddr fills the buffer with entries.  The start of the buffer must
+ * Readdir fills the buffer with entries.  The start of the buffer must
  * be at least as aligned as the entry.  Entries are padded for
  * alignment.  Use next_off to iterate through them instead of trying to
  * skip name length bytes past the end of the struct.
  */
 struct ngnfs_readdir_entry {
 	u64 pos;
-	u64 ino;
-	u64 gen;
-	u16 next_offset;	/* bytes to add to entry to get next entry */
+	struct ngnfs_inode_ino_gen ig;
+	u16 next_offset;		/* bytes to add to entry to get next entry */
 	u8 dtype;
-	u8 name_len;		/* does not include terminating null, like strlen */
-	u8 name[];		/* is null terminated, name_len doesn't include null byte */
+	u8 name_len;			/* does not include terminating null, like strlen */
+	u8 name[];			/* is null terminated, name_len doesn't include null byte */
 };
 
 /*
@@ -35,7 +35,7 @@ struct ngnfs_readdir_entry {
  * Returns the number of entries filled into the buffer, not the number
  * of bytes.
  */
-int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, u64 dir_ino, u64 pos,
+int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_ig, u64 pos,
 		      struct ngnfs_readdir_entry *buf, size_t size);
 
 #endif
diff --git a/shared/format-block.h b/shared/format-block.h
index 95ba9ff..787de90 100644
--- a/shared/format-block.h
+++ b/shared/format-block.h
@@ -80,6 +80,21 @@ struct ngnfs_btree_item {
 #define NGNFS_BTREE_MAX_FREE									\
 	(NGNFS_BLOCK_SIZE - offsetof(struct ngnfs_btree_block, ihdrs[NGNFS_BTREE_MAX_ITEMS]))
 
+/*
+ * The inode generation number changes every time a specific inode is
+ * freed and reallocated. The inode number and generation number
+ * uniquely identify a file/dir over its lifetime. This lets users of
+ * references to inodes find out if the inode has changed out from
+ * underneath them. Generally, the inode number and generation number
+ * should be referenced together, so make a struct to keep them
+ * together.
+ */
+
+struct ngnfs_ino_gen {
+	__le64 ino;	/* inode number, starts at 1 */
+	__le64 gen;	/* inode generation, starts at 1 */
+};
+
 /*
  * Inodes are stored in inode blocks.  Inode blocks numbers are directly
  * calculated from the inode number.  The block itself is formatted as a
@@ -87,8 +102,7 @@ struct ngnfs_btree_item {
  * as btree items in the block.
  */
 struct ngnfs_inode {
-	__le64 ino;
-	__le64 gen;
+	struct ngnfs_ino_gen ig;
 	__le64 size;
 	__le64 version;
 	__le32 nlink;
@@ -105,6 +119,7 @@ struct ngnfs_inode {
 };
 
 #define NGNFS_ROOT_INO 1
+#define INIT_NGNFS_ROOT_IG { NGNFS_ROOT_INO, 1 }
 
 /*
  * This is totally arbitrary.  It looks like it's 32bit in the stat ABI.
@@ -123,8 +138,7 @@ enum ngnfs_dentry_type {
 };
 
 struct ngnfs_dirent {
-	__le64 ino;
-	__le64 version; /* XXX ? */
+	struct ngnfs_ino_gen ig; /* inode number and generation */
 	__u8 pers_dtype; /* ngnfs persistent directory entry type */
 	__u8 name_len; /* no null termination */
 	__u8 name[6]; /* definition pads to alignment, stored can be smaller */
diff --git a/shared/inode.c b/shared/inode.c
index 131e195..92a74fe 100644
--- a/shared/inode.c
+++ b/shared/inode.c
@@ -19,14 +19,14 @@
  * to pack inode data into the rest of the block.
  */
 
-int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, u64 ino, u64 gen, u32 nlink, umode_t mode,
-		     u64 nsec)
+int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, struct ngnfs_inode_ino_gen *ig, u32 nlink,
+		     umode_t mode, u64 nsec)
 {
 	struct ngnfs_txn_block *tblk = itref->tblk;
 	struct ngnfs_inode *ninode = itref->ninode;
 
-	ngnfs_tblk_assign(tblk, ninode->ino, cpu_to_le64(ino));
-	ngnfs_tblk_assign(tblk, ninode->gen, cpu_to_le64(gen));
+	ngnfs_tblk_assign(tblk, ninode->ig.ino, cpu_to_le64(ig->ino));
+	ngnfs_tblk_assign(tblk, ninode->ig.gen, cpu_to_le64(ig->gen));
 	ngnfs_tblk_assign(tblk, ninode->size, 0);
 	ngnfs_tblk_assign(tblk, ninode->version, cpu_to_le64(1));
 	ngnfs_tblk_assign(tblk, ninode->nlink, cpu_to_le32(nlink));
@@ -47,22 +47,25 @@ int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, u64 ino, u64 gen, u32 nl
 /*
  * XXX should be validating that the block is a valid inode, etc.
  */
-int ngnfs_inode_get(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, nbf_t nbf, u64 ino,
-		    struct ngnfs_inode_txn_ref *itref)
+int ngnfs_inode_get(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, nbf_t nbf,
+		    struct ngnfs_inode_ino_gen *ig, struct ngnfs_inode_txn_ref *itref)
 {
-	return ngnfs_txn_get_block(nfi, txn, ino, nbf, &itref->tblk, (void **)&itref->ninode);
+	return ngnfs_txn_get_block(nfi, txn, ig->ino, nbf, &itref->tblk, (void **)&itref->ninode);
 }
 
-int ngnfs_inode_alloc(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, u64 *ino,
-		      struct ngnfs_inode_txn_ref *itref)
+int ngnfs_inode_alloc(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+		      struct ngnfs_inode_ino_gen *ig, struct ngnfs_inode_txn_ref *itref)
 {
-	u64 bnr;
+	struct ngnfs_inode_ino_gen new;
 	int ret;
 
-	ret = ngnfs_txn_alloc_meta(txn, &bnr) ?:
-	      ngnfs_inode_get(nfi, txn, NBF_WRITE | NBF_NEW, bnr, itref);
+	new.gen = 1; /* XXX should look up previous gen and increment */
+	ret = ngnfs_txn_alloc_meta(txn, &new.ino) ?:
+	      ngnfs_inode_get(nfi, txn, NBF_WRITE | NBF_NEW, &new, itref);
+
 	if (ret == 0)
-		*ino = bnr;
+		*ig = new;
+
 	return ret;
 }
 
@@ -70,7 +73,8 @@ int ngnfs_inode_alloc(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
  * A transaction that just copies the inode in the block to the caller's
  * buffer.
  */
-int ngnfs_inode_read_copy(struct ngnfs_fs_info *nfi, u64 ino, void *buf, int size)
+int ngnfs_inode_read_copy(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *ig,
+			  void *buf, int size)
 {
 	struct ngnfs_inode_txn_ref itref;
 	struct ngnfs_transaction txn;
@@ -84,7 +88,7 @@ int ngnfs_inode_read_copy(struct ngnfs_fs_info *nfi, u64 ino, void *buf, int siz
 	ngnfs_txn_init(&txn);
 
 	do {
-		ret = ngnfs_inode_get(nfi, &txn, NBF_READ, ino, &itref);
+		ret = ngnfs_inode_get(nfi, &txn, NBF_READ, ig, &itref);
 		if (ret == 0) {
 			ret = min(size, sizeof(struct ngnfs_inode));
 			memcpy(buf, itref.ninode, ret);
diff --git a/shared/inode.h b/shared/inode.h
index 3f6328b..d9dbb77 100644
--- a/shared/inode.h
+++ b/shared/inode.h
@@ -14,12 +14,18 @@ struct ngnfs_inode_txn_ref {
 	struct ngnfs_inode *ninode;
 };
 
-int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, u64 ino, u64 gen, u32 nlink, umode_t mode,
-		     u64 nsec);
-int ngnfs_inode_get(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, nbf_t nbf, u64 ino,
-		    struct ngnfs_inode_txn_ref *itref);
-int ngnfs_inode_alloc(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, u64 *ino,
-		      struct ngnfs_inode_txn_ref *itref);
-int ngnfs_inode_read_copy(struct ngnfs_fs_info *nfi, u64 ino, void *buf, int size);
+struct ngnfs_inode_ino_gen {
+	u64 ino;
+	u64 gen;
+};
+
+int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, struct ngnfs_inode_ino_gen *ig, u32 nlink,
+		     umode_t mode, u64 nsec);
+int ngnfs_inode_get(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, nbf_t nbf,
+		    struct ngnfs_inode_ino_gen *ig, struct ngnfs_inode_txn_ref *itref);
+int ngnfs_inode_alloc(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+		      struct ngnfs_inode_ino_gen *ig, struct ngnfs_inode_txn_ref *itref);
+int ngnfs_inode_read_copy(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *ig,
+			  void *buf, int size);
 
 #endif
diff --git a/shared/mkfs.c b/shared/mkfs.c
index dee8cbe..3e84393 100644
--- a/shared/mkfs.c
+++ b/shared/mkfs.c
@@ -18,6 +18,7 @@ int ngnfs_mkfs(struct ngnfs_fs_info *nfi)
 {
 	struct ngnfs_inode_txn_ref itref;
 	struct ngnfs_transaction txn;
+	struct ngnfs_inode_ino_gen ig = INIT_NGNFS_ROOT_IG;
 	u64 nsec;
 	int ret;
 
@@ -25,8 +26,8 @@ int ngnfs_mkfs(struct ngnfs_fs_info *nfi)
 	nsec = ktime_to_ns(ktime_get_real());
 
 	do {
-		ret = ngnfs_inode_get(nfi, &txn, NBF_WRITE, NGNFS_ROOT_INO, &itref)		?:
-		      ngnfs_inode_init(&itref, NGNFS_ROOT_INO, 1, 2, S_IFDIR | 0755, nsec);
+		ret = ngnfs_inode_get(nfi, &txn, NBF_WRITE, &ig, &itref)			?:
+		      ngnfs_inode_init(&itref, &ig, 2, S_IFDIR | 0755, nsec);
 
 	} while (ngnfs_txn_retry(nfi, &txn, &ret));
 
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:46 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:46 +0100
Subject: [PATCH 07/15] Add auto-generated "." and ".." entries to directories
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-8-val@versity.com>

There are pros and cons to real or fake "." and ".." dentries, but we
decided to go with fake because we don't have to do a potentially slow
lookup of ".." in large directories. Userspace likes "." and ".." to
come first in readdir(), and we return entries in hash value order, so
reserve 0 and 1 in the hash output space for "." and "..". Then
autogenerate them in readdir() and lookup() using the parent inode
back reference. Also add an optional position argument to the debugs
readdir() command to test correctness of handling readdir() position.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c         | 16 +++++++-
 shared/dir.c          | 87 +++++++++++++++++++++++++++++++++++++++++--
 shared/format-block.h |  5 +++
 3 files changed, 104 insertions(+), 4 deletions(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 49a24a8..6923d6f 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -20,6 +20,7 @@
 #include "shared/log.h"
 #include "shared/mkfs.h"
 #include "shared/mount.h"
+#include "shared/nerr.h"
 #include "shared/thread.h"
 
 #include "cli/cli.h"
@@ -79,13 +80,26 @@ static void cmd_readdir(struct debugfs_context *ctx, int argc, char **argv)
 	int ret;
 	int i;
 
+	if (argc > 2) {
+		printf("usage: readdir [position]\n");
+		return;
+	}
+
+	pos = 0;
+	if (argc == 2) {
+		ret = strtoull_nerr(&pos, argv[1], NULL, 0);
+		if (ret < 0) {
+			printf("invalid position: %s\n", argv[1]);
+			return;
+		}
+	}
+
 	buf = malloc(size);
 	if (!buf) {
 		printf("malloc error");
 		return;
 	}
 
-	pos = 0;
 	while (true) {
 		ret = ngnfs_dir_readdir(ctx->nfi, &ctx->cwd_ig, pos, buf, size);
 		if (ret <= 0) {
diff --git a/shared/dir.c b/shared/dir.c
index 2f5d5db..b945df8 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -42,9 +42,38 @@ static bool names_equal(u8 *a, size_t a_len, u8 *b, size_t b_len)
 	return a_len == b_len && memcmp(a, b, a_len) == 0;
 }
 
+/*
+ * The directory entries for . and .. are generated during lookup and
+ * readdir and are not "real" directory entries stored as dirents. For
+ * readdir to work properly, we need the position of each entry (its
+ * hash value) to be stable. We also want to generate . and .. first
+ * because it's easier than inserting them somewhere in the middle and
+ * because applications like it that way.
+ *
+ * The solution is to reserve the hash values 0 for . and 1 for .. so
+ * that we can return them first in readdir() and the positions returned
+ * by readdir are strictly ascending.
+ */
+
 static u64 name_hash(void *name, size_t name_len)
 {
-	return xxh64(name, name_len, NGNFS_DIRENT_HASH_SEED) & NGNFS_DIRENT_HASH_MASK;
+	char *s = name;
+	u64 hash;
+
+	if ((name_len < 3) && (name_len > 0) && s[0] == '.') {
+		if (name_len == 1)
+			return NGNFS_DIRENT_DOT_HASH;
+
+		if (s[1] == '.')
+			return NGNFS_DIRENT_DOT_DOT_HASH;
+	}
+
+	hash = xxh64(name, name_len, NGNFS_DIRENT_HASH_SEED) & NGNFS_DIRENT_HASH_MASK;
+
+	if (hash < NGNFS_DIRENT_MIN_HASH)
+		hash = NGNFS_DIRENT_MIN_HASH;
+
+	return hash;
 }
 
 /*
@@ -281,7 +310,7 @@ struct readdir_args {
 	int nr;
 };
 
-static int fill_dirent_rd(struct ngnfs_btree_key *key, void *val, size_t val_size, void *args)
+static int fill_readdir_rd(struct ngnfs_btree_key *key, void *val, size_t val_size, void *args)
 {
 	struct readdir_args *ra = args;
 	struct ngnfs_dirent *dent = val;
@@ -314,6 +343,57 @@ static int fill_dirent_rd(struct ngnfs_btree_key *key, void *val, size_t val_siz
 	return NGNFS_BTREE_ITER_CONTINUE;
 }
 
+/*
+ * Readdir helpers to fill in "." and ".."
+ */
+static void fill_dot_dirent(u64 pos, struct ngnfs_dirent *dent, struct ngnfs_inode_txn_ref *dir)
+{
+	dent->pers_dtype = NGNFS_DT_DIR;
+	dent->name[0] = '.';
+
+	if (pos == NGNFS_DIRENT_DOT_HASH) {
+		dent->name_len = 1;
+		dent->ig = dir->ninode->ig;
+	} else {
+		dent->ig = dir->ninode->parent_ig;
+		dent->name[1] = '.';
+		dent->name_len = 2;
+	}
+}
+
+static int fill_dots(struct ngnfs_btree_key *key, struct readdir_args *ra,
+		     struct ngnfs_inode_txn_ref *dir)
+{
+	struct ngnfs_dirent dent;
+	int ret;
+
+	if (le64_to_cpu(key->k[0]) >= NGNFS_DIRENT_MIN_HASH)
+		return 0;
+
+	fill_dot_dirent(le64_to_cpu(key->k[0]), &dent, dir);
+	ret = fill_readdir_rd(key, &dent, offsetof(struct ngnfs_dirent, name) + dent.name_len, ra);
+	BUG_ON(ret == 0); /* 0 means out of space, should never happen due to min buf size */
+
+	if (ret == NGNFS_BTREE_ITER_CONTINUE) {
+		key->k[0] = cpu_to_le64(le64_to_cpu(key->k[0]) + 1);
+		ret = 0;
+	}
+
+	return ret;
+}
+
+static int fill_dot_dirents(struct ngnfs_btree_key *key, struct readdir_args *ra,
+			    struct ngnfs_inode_txn_ref *dir)
+{
+	int ret;
+
+	/* call once to fill "." and once for ".." */
+	ret = fill_dots(key, ra, dir) ?:
+	      fill_dots(key, ra, dir);
+
+	return ret;
+}
+
 /*
  * Read directory entries starting at the given position, filling
  * entries in the buffer until entries are exhausted or the buffer is
@@ -356,8 +436,9 @@ int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir
 
 		ret = ngnfs_inode_get(nfi, &txn, NBF_READ, dir_ig, &dir)			?:
 		      check_ifmt(dir.ninode, S_IFDIR, -ENOTDIR)					?:
+		      fill_dot_dirents(&key, &ra, &dir)						?:
 		      ngnfs_btree_read_iter(nfi, &txn, &dir.ninode->dirents, &key,
-					    NULL, NULL, fill_dirent_rd, &ra);
+					    NULL, NULL, fill_readdir_rd, &ra);
 
 	} while (ngnfs_txn_retry(nfi, &txn, &ret));
 
diff --git a/shared/format-block.h b/shared/format-block.h
index 9c5221e..d3cc302 100644
--- a/shared/format-block.h
+++ b/shared/format-block.h
@@ -167,4 +167,9 @@ struct ngnfs_dirent {
  */
 #define NGNFS_DIRENT_HASH_MASK	(U64_MAX ^ (1ULL << 63) ^ NGNFS_DIRENT_COLL_BIT)
 
+/* reserved hash values for . and .. */
+#define NGNFS_DIRENT_DOT_HASH	 	0ULL
+#define NGNFS_DIRENT_DOT_DOT_HASH	1ULL
+#define NGNFS_DIRENT_MIN_HASH		2ULL
+
 #endif
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:47 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:47 +0100
Subject: [PATCH 08/15] Add ngnfs_dir_lookup() and debugfs lookup command
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-9-val@versity.com>

Add ngnfs_dir_lookup() to return ngnfs directory entry information and
the associated debugfs lookup command for debugging purposes.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c | 40 +++++++++++++++++++++-
 shared/dir.c  | 95 +++++++++++++++++++++++++++++++++++++++++++++++++++
 shared/dir.h  | 11 ++++++
 3 files changed, 145 insertions(+), 1 deletion(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 6923d6f..8bc1dc4 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -14,8 +14,8 @@
 #include "shared/lk/kernel.h"
 #include "shared/lk/types.h"
 
-#include "shared/format-block.h"
 #include "shared/dir.h"
+#include "shared/format-block.h"
 #include "shared/inode.h"
 #include "shared/log.h"
 #include "shared/mkfs.h"
@@ -49,6 +49,43 @@ static void cmd_create(struct debugfs_context *ctx, int argc, char **argv)
 		printf("create error: "ENOF"\n", ENOA(-ret));
 }
 
+static void cmd_lookup(struct debugfs_context *ctx, int argc, char **argv)
+{
+	struct ngnfs_dir_lookup_entry lent;
+	char *name;
+	int name_len;
+	int ret;
+
+	if (argc != 2) {
+		printf("usage: lookup <pathname>\n");
+		return;
+	}
+
+	name = argv[1];
+	name_len = strlen(name);
+
+	ret = ngnfs_dir_lookup(ctx->nfi, &ctx->cwd_ig, name, name_len, &lent);
+	if (ret < 0) {
+		if (ret == -ENOENT)
+			printf("lookup: %s does not exist\n", name);
+		else
+			log("lookup error: "ENOF"\n", ENOA(-ret));
+		return;
+	}
+
+	printf("name: %.*s\n"
+	       "name_len: %d\n"
+	       "dtype: %o\n"
+	       "ino: %llu\n"
+	       "gen: %llu\n",
+	       name_len,
+	       name,
+	       name_len,
+	       lent.dtype,
+	       lent.ig.ino,
+	       lent.ig.gen);
+}
+
 static void cmd_mkfs(struct debugfs_context *ctx, int argc, char **argv)
 {
 	int ret;
@@ -177,6 +214,7 @@ static struct command {
 	void (*func)(struct debugfs_context *ctx, int argc, char **argv);
 } commands[] = {
 	{ "create", cmd_create, },
+	{ "lookup", cmd_lookup, },
 	{ "mkfs", cmd_mkfs, },
 	{ "quit", cmd_quit, },
 	{ "readdir", cmd_readdir, },
diff --git a/shared/dir.c b/shared/dir.c
index b945df8..074f998 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -6,6 +6,7 @@
 #include "shared/lk/bug.h"
 #include "shared/lk/byteorder.h"
 #include "shared/lk/container_of.h"
+#include "shared/lk/err.h"
 #include "shared/lk/errno.h"
 #include "shared/lk/fs_types.h"
 #include "shared/lk/kernel.h"
@@ -446,3 +447,97 @@ int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir
 out:
 	return ret ?: ra.nr;
 }
+
+static int fill_lookup_rd(struct ngnfs_btree_key *key, void *val, size_t val_size, void *arg)
+{
+	struct dirent_args *da = arg;
+	struct ngnfs_dirent *dent = val;
+
+	if (!names_equal(dent->name, dent->name_len, (u8 *) da->dent.name, da->dent.name_len))
+		return NGNFS_BTREE_ITER_CONTINUE;
+
+	da->ig.ino = le64_to_cpu(dent->ig.ino);
+	da->ig.gen = le64_to_cpu(dent->ig.gen);
+	memcpy(&da->dent, dent, val_size);
+
+	return 0;
+}
+
+static int lookup_dots(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+		       struct ngnfs_inode_txn_ref *dir, struct dirent_args *da)
+{
+	struct ngnfs_btree_key key;
+
+	if (da->hash >= NGNFS_DIRENT_MIN_HASH)
+		return -ENOENT;
+
+	init_dirent_key(&key, da->hash);
+	fill_dot_dirent(da->hash, &da->dent, dir);
+
+	return 0;
+}
+
+static int lookup_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+			 struct ngnfs_inode_txn_ref *dir, struct dirent_args *da)
+{
+	struct ngnfs_btree_key key;
+	struct ngnfs_btree_key last;
+	int ret;
+
+	ret = lookup_dots(nfi, txn, dir, da);
+	if (ret == 0)
+		goto out;
+
+	init_dirent_key(&key, da->hash);
+	init_dirent_key(&last, da->hash | NGNFS_DIRENT_COLL_BIT);
+
+	ret = ngnfs_btree_read_iter(nfi, txn, &dir->ninode->dirents, &key, NULL, &last,
+				    fill_lookup_rd, da);
+	if (ret < 0)
+		goto out;
+
+	if (da->dent.ig.ino == 0)
+		ret = -ENOENT;
+out:
+	return ret;
+}
+
+static int copy_dent_to_lent(struct ngnfs_dirent *dent, struct ngnfs_dir_lookup_entry *lent)
+{
+	lent->ig.ino = le64_to_cpu(dent->ig.ino);
+	lent->ig.gen = le64_to_cpu(dent->ig.gen);
+	lent->dtype = pers_dtype_to_abi_dtype(dent->pers_dtype);
+
+	return 0;
+}
+
+int ngnfs_dir_lookup(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_ig, char *name,
+		     size_t name_len, struct ngnfs_dir_lookup_entry *lent)
+{
+	struct {
+		struct ngnfs_transaction txn;
+		struct ngnfs_inode_txn_ref dir;
+		struct dirent_args da;
+	} *op;
+	int ret;
+
+	op = kmalloc(sizeof(*op), GFP_NOFS);
+	if (!op)
+		return -ENOENT;
+
+	ngnfs_txn_init(&op->txn);
+	init_dirent_args(&op->da, name, name_len, 0);
+
+	do {
+		ret = ngnfs_inode_get(nfi, &op->txn, NBF_READ, dir_ig, &op->dir)		?:
+		      check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)				?:
+		      lookup_dirent(nfi, &op->txn, &op->dir, &op->da)				?:
+		      copy_dent_to_lent(&op->da.dent, lent);
+
+	} while (ngnfs_txn_retry(nfi, &op->txn, &ret));
+
+	ngnfs_txn_teardown(nfi, &op->txn);
+
+	kfree(op);
+	return ret;
+}
diff --git a/shared/dir.h b/shared/dir.h
index d56641f..178f53e 100644
--- a/shared/dir.h
+++ b/shared/dir.h
@@ -39,4 +39,15 @@ struct ngnfs_readdir_entry {
 int ngnfs_dir_readdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_ig, u64 pos,
 		      struct ngnfs_readdir_entry *buf, size_t size);
 
+/*
+ * Lookup the contents of the directory entry for name.
+ */
+struct ngnfs_dir_lookup_entry {
+	struct ngnfs_inode_ino_gen ig;
+	u8 dtype;
+};
+
+int ngnfs_dir_lookup(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_ig, char *name,
+		     size_t name_len, struct ngnfs_dir_lookup_entry *lent);
+
 #endif
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:43 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:43 +0100
Subject: [PATCH 04/15] Add brief instructions on how to run ngnfs
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-5-val@versity.com>

Some quick pointers on installing dependencies and building/running
ngnfs.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 README.md           |  7 +++++--
 docs/development.md | 31 +++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index 7f030e7..bda6666 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,8 @@
 
-This is the repository for the userspace componenent of ngnfs
-("engine FS"), a storage system in early development.
+This is the repository for the userspace component of ngnfs ("engine
+FS"), a storage system in early development.
 
 Further bulletins as events warrant.
+
+See the docs/ directory for more information on how to run and
+contribute to ngnfs.
diff --git a/docs/development.md b/docs/development.md
index 286d4cc..215e641 100644
--- a/docs/development.md
+++ b/docs/development.md
@@ -1,3 +1,34 @@
+## Running
+
+Compile the code with make. You will need librcu and sparse installed.
+The sparse package tends to depend on a lot of other packages, so you
+may want to build and install your own.
+
+ngnfs currently only runs in userspace. There are two components: the
+devd process and the debugfs process.
+
+The devd process listens on a socket for block IO requests. The
+debugfs process reads commands from stdin and turns them into ngnfs
+operations, which generate IO requests to the devd process. Thus you
+need to tell the devd process where to listen and the debugfs process
+where to connect. The devd process also needs to know where its
+backing store is. Another required argument is a path to a file to
+write trace data to.
+
+Example invocation:
+
+```
+	$ truncate -s 1G ~/tmp/dev
+	$ ./devd/ngnfs-devd -d ~/tmp/dev -l 127.0.0.1:8081 -t /tmp/trace_devd
+```
+
+In another terminal:
+
+```
+	$ ./cli/ngnfs-cli debugfs -d 127.0.0.1:8081 -t /tmp/trace_debugfs
+	<1> $ mkfs
+	<1> $ stat
+```
 
 ## Code Layout
 
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:42 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:42 +0100
Subject: [PATCH 03/15] Shutdown devd process cleanly for tests
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-4-val@versity.com>

The ngnfs tests send SIGUSR1 to the devd processes to shut them down,
which causes the io_get_events() system call to fail with EINVAL after
the socket has been closed by another thread. Replace the assert(ret >
0) with a clean exit on EINVAL and an assert() otherwise so we can
distinguish a clean shutdown from a bug.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 devd/btr-aio.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/devd/btr-aio.c b/devd/btr-aio.c
index c16f671..68cdbd0 100644
--- a/devd/btr-aio.c
+++ b/devd/btr-aio.c
@@ -124,7 +124,13 @@ static void getevents_thread(struct thread *thr, void *arg)
 
 		ret = syscall(__NR_io_getevents, ainf->ctx, 1, ainf->queue_depth,
 			      ainf->events, NULL);
-		assert(ret > 0);
+		if (ret < 0) {
+			/* socket closed by another thread, likely due to shutdown */
+			if (errno == EINVAL)
+				break;
+			assert(ret > 0);
+		}
+
 		nr = ret;
 
 		for (i = 0; i < nr; i++) {
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:50 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:50 +0100
Subject: [PATCH 11/15] Add ngnfs_dir_mkdir() and debugfs command
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-12-val@versity.com>

Makes a directory of the specified name in the current working
directory in debugfs.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c | 21 +++++++++++++++++++++
 shared/dir.c  | 43 ++++++++++++++++++++++++++++++++++++++++---
 shared/dir.h  |  2 ++
 3 files changed, 63 insertions(+), 3 deletions(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 0c0914c..3494070 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -118,6 +118,26 @@ static void cmd_lookup(struct debugfs_context *ctx, int argc, char **argv)
 	       lent.ig.gen);
 }
 
+static void cmd_mkdir(struct debugfs_context *ctx, int argc, char **argv)
+{
+	int ret;
+
+	if (argc != 2) {
+		printf("must have one, and only one, directory name to create\n");
+		return;
+	}
+
+	ret = ngnfs_dir_mkdir(ctx->nfi, &ctx->cwd_ig, 0755, argv[1], strlen(argv[1]));
+	if (ret == -EEXIST) {
+		printf("mkdir: file exists\n");
+		return;
+	}
+	if (ret < 0) {
+		log("mkdir error: "ENOF"\n", ENOA(-ret));
+		return;
+	}
+}
+
 static void cmd_mkfs(struct debugfs_context *ctx, int argc, char **argv)
 {
 	int ret;
@@ -268,6 +288,7 @@ static struct command {
 	{ "cd", cmd_cd, },
 	{ "create", cmd_create, },
 	{ "lookup", cmd_lookup, },
+	{ "mkdir", cmd_mkdir, },
 	{ "mkfs", cmd_mkfs, },
 	{ "quit", cmd_quit, },
 	{ "readdir", cmd_readdir, },
diff --git a/shared/dir.c b/shared/dir.c
index 074f998..ebeea01 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -261,11 +261,24 @@ static int insert_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *tx
 				      &last, insert_dirent_wr, da);
 }
 
+/*
+ * Prevent creation of "." and "..", with appropriate error return codes.
+ */
+static int check_create_dots(u64 hash, mode_t mode)
+{
+	if ((hash != NGNFS_DIRENT_DOT_HASH) && (hash != NGNFS_DIRENT_DOT_DOT_HASH))
+		return 0;
+
+	if (S_ISDIR(mode))
+		return -EEXIST;
+	else
+		return -EISDIR;
+}
 
 /*
  * Allocate a new inode and add a directory entry referencing it.
  */
-int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
+static int do_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
 		     char *name, size_t name_len)
 {
 	struct {
@@ -273,6 +286,8 @@ int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir,
 		struct ngnfs_inode_txn_ref dir;
 		struct ngnfs_inode_txn_ref inode;
 		struct ngnfs_inode_ino_gen ig;
+		struct ngnfs_inode_ino_gen *parent_ig;
+		int nlink;
 		u64 nsec;
 		struct dirent_args da;
 	} *op;
@@ -286,13 +301,23 @@ int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir,
 
 	ngnfs_txn_init(&op->txn);
 	op->nsec = ktime_to_ns(ktime_get_real());
-	init_dirent_args(&op->da, name, name_len, mode | S_IFREG);
+	if (S_ISDIR(mode)) {
+		op->nlink = 2;
+		op->parent_ig = dir;
+	} else {
+		op->nlink = 1;
+		op->parent_ig = NULL;
+	}
+
+	init_dirent_args(&op->da, name, name_len, mode);
 
 	do {
 		ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir, &op->dir)			?:
 		      check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)				?:
+		      check_create_dots(op->da.hash, mode)					?:
 		      ngnfs_inode_alloc(nfi, &op->txn, &op->ig, &op->inode)			?:
-		      ngnfs_inode_init(&op->inode, &op->ig, 1, mode | S_IFREG, op->nsec, NULL)	?:
+		      ngnfs_inode_init(&op->inode, &op->ig, op->nlink, mode, op->nsec,
+				       op->parent_ig)						?:
 		      update_dirent_args_ig(&op->da, &op->ig)					?:
 		      insert_dirent(nfi, &op->txn, &op->dir, &op->da)				?:
 		      update_dir(op->dir.tblk, op->dir.ninode, &op->da, 1);
@@ -305,6 +330,18 @@ out:
 	return ret;
 }
 
+int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
+		     char *name, size_t name_len)
+{
+	return do_create(nfi, dir, mode | S_IFREG, name, name_len);
+}
+
+int ngnfs_dir_mkdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
+		    char *name, size_t name_len)
+{
+	return do_create(nfi, dir, mode | S_IFDIR, name, name_len);
+}
+
 struct readdir_args {
 	struct ngnfs_readdir_entry *ent;
 	size_t size;
diff --git a/shared/dir.h b/shared/dir.h
index 178f53e..5dd57dd 100644
--- a/shared/dir.h
+++ b/shared/dir.h
@@ -8,6 +8,8 @@
 
 int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
 		     char *name, size_t name_len);
+int ngnfs_dir_mkdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
+		    char *name, size_t name_len);
 
 /*
  * Readdir fills the buffer with entries.  The start of the buffer must
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:45 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:45 +0100
Subject: [PATCH 06/15] Add parent inode number/generation for directories
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-7-val@versity.com>

Add a parent inode number/generation field to the inode to give us back
references for directories. For non-directories, this field is empty
and back references will go in a per-inode btree (TODO)

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c         | 4 ++++
 shared/dir.c          | 2 +-
 shared/dir.h          | 1 +
 shared/format-block.h | 3 ++-
 shared/inode.c        | 9 ++++++++-
 shared/inode.h        | 2 +-
 shared/mkfs.c         | 2 +-
 7 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index a3ef5f8..49a24a8 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -129,6 +129,8 @@ static void cmd_stat(struct debugfs_context *ctx, int argc, char **argv)
 		       "gen: %llu\n"
 		       "nlink: %u\n"
 		       "mode: %o\n"
+		       "parent ino: %llu\n"
+		       "parent gen: %llu\n"
 		       "atime: %llu\n"
 		       "ctime: %llu\n"
 		       "mtime: %llu\n"
@@ -137,6 +139,8 @@ static void cmd_stat(struct debugfs_context *ctx, int argc, char **argv)
 		       le64_to_cpu(ninode.ig.gen),
 		       le32_to_cpu(ninode.nlink),
 		       le32_to_cpu(ninode.mode),
+		       le64_to_cpu(ninode.parent_ig.ino),
+		       le64_to_cpu(ninode.parent_ig.gen),
 		       le64_to_cpu(ninode.atime_nsec),
 		       le64_to_cpu(ninode.ctime_nsec),
 		       le64_to_cpu(ninode.mtime_nsec),
diff --git a/shared/dir.c b/shared/dir.c
index 76afda0..2f5d5db 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -262,7 +262,7 @@ int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir,
 		ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir, &op->dir)			?:
 		      check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)				?:
 		      ngnfs_inode_alloc(nfi, &op->txn, &op->ig, &op->inode)			?:
-		      ngnfs_inode_init(&op->inode, &op->ig, 1, mode | S_IFREG, op->nsec)	?:
+		      ngnfs_inode_init(&op->inode, &op->ig, 1, mode | S_IFREG, op->nsec, NULL)	?:
 		      update_dirent_args_ig(&op->da, &op->ig)					?:
 		      insert_dirent(nfi, &op->txn, &op->dir, &op->da)				?:
 		      update_dir(op->dir.tblk, op->dir.ninode, &op->da, 1);
diff --git a/shared/dir.h b/shared/dir.h
index 8ca28b8..d56641f 100644
--- a/shared/dir.h
+++ b/shared/dir.h
@@ -4,6 +4,7 @@
 
 #include "shared/inode.h"
 #include "shared/fs_info.h"
+#include "shared/inode.h"
 
 int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
 		     char *name, size_t name_len);
diff --git a/shared/format-block.h b/shared/format-block.h
index 787de90..9c5221e 100644
--- a/shared/format-block.h
+++ b/shared/format-block.h
@@ -104,7 +104,8 @@ struct ngnfs_ino_gen {
 struct ngnfs_inode {
 	struct ngnfs_ino_gen ig;
 	__le64 size;
-	__le64 version;
+	__le64 version;			/* changed on file content/metadata changes */
+	struct ngnfs_ino_gen parent_ig;	/* only valid for directories */
 	__le32 nlink;
 	__le32 uid;
 	__le32 gid;
diff --git a/shared/inode.c b/shared/inode.c
index 92a74fe..2b29a1f 100644
--- a/shared/inode.c
+++ b/shared/inode.c
@@ -20,7 +20,7 @@
  */
 
 int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, struct ngnfs_inode_ino_gen *ig, u32 nlink,
-		     umode_t mode, u64 nsec)
+		     umode_t mode, u64 nsec, struct ngnfs_inode_ino_gen *parent_ig)
 {
 	struct ngnfs_txn_block *tblk = itref->tblk;
 	struct ngnfs_inode *ninode = itref->ninode;
@@ -29,6 +29,13 @@ int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, struct ngnfs_inode_ino_g
 	ngnfs_tblk_assign(tblk, ninode->ig.gen, cpu_to_le64(ig->gen));
 	ngnfs_tblk_assign(tblk, ninode->size, 0);
 	ngnfs_tblk_assign(tblk, ninode->version, cpu_to_le64(1));
+	if (parent_ig != NULL) {
+		ngnfs_tblk_assign(tblk, ninode->parent_ig.ino, cpu_to_le64(parent_ig->ino));
+		ngnfs_tblk_assign(tblk, ninode->parent_ig.gen, cpu_to_le64(parent_ig->gen));
+	} else {
+		ngnfs_tblk_assign(tblk, ninode->parent_ig.ino, 0);
+		ngnfs_tblk_assign(tblk, ninode->parent_ig.gen, 0);
+	}
 	ngnfs_tblk_assign(tblk, ninode->nlink, cpu_to_le32(nlink));
 	ngnfs_tblk_assign(tblk, ninode->uid, 0);
 	ngnfs_tblk_assign(tblk, ninode->gid, 0);
diff --git a/shared/inode.h b/shared/inode.h
index d9dbb77..92b5114 100644
--- a/shared/inode.h
+++ b/shared/inode.h
@@ -20,7 +20,7 @@ struct ngnfs_inode_ino_gen {
 };
 
 int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, struct ngnfs_inode_ino_gen *ig, u32 nlink,
-		     umode_t mode, u64 nsec);
+		     umode_t mode, u64 nsec, struct ngnfs_inode_ino_gen *parent_ig);
 int ngnfs_inode_get(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, nbf_t nbf,
 		    struct ngnfs_inode_ino_gen *ig, struct ngnfs_inode_txn_ref *itref);
 int ngnfs_inode_alloc(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
diff --git a/shared/mkfs.c b/shared/mkfs.c
index 3e84393..84613e6 100644
--- a/shared/mkfs.c
+++ b/shared/mkfs.c
@@ -27,7 +27,7 @@ int ngnfs_mkfs(struct ngnfs_fs_info *nfi)
 
 	do {
 		ret = ngnfs_inode_get(nfi, &txn, NBF_WRITE, &ig, &itref)			?:
-		      ngnfs_inode_init(&itref, &ig, 2, S_IFDIR | 0755, nsec);
+		      ngnfs_inode_init(&itref, &ig, 2, S_IFDIR | 0755, nsec, &ig);
 
 	} while (ngnfs_txn_retry(nfi, &txn, &ret));
 
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:52 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:52 +0100
Subject: [PATCH 13/15] Add ngnfs_dir_rename() and very basic debugfs command
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-14-val@versity.com>

Add mostly functional ngnfs_dir_rename() that removes target and
checks for directory loops and non-empty target directories. The
debugfs command is very limited: it can only rename a file from the
current working directory to the parent directory.

This has a known bug where after around 600 renames it returns a
spurious ENOENT.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c |  31 +++++++++
 shared/dir.c  | 171 ++++++++++++++++++++++++++++++++++++++++++++++++--
 shared/dir.h  |   3 +
 3 files changed, 199 insertions(+), 6 deletions(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 32fc3f6..9a61e80 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -214,6 +214,36 @@ static void cmd_readdir(struct debugfs_context *ctx, int argc, char **argv)
 	free(buf);
 }
 
+/*
+ * Basic rename hack for now.
+ *
+ * XXX assumes that you want to move a file from the current directory to ..
+ */
+static void cmd_rename(struct debugfs_context *ctx, int argc, char **argv)
+{
+	struct ngnfs_dir_lookup_entry parent;
+	char *name;
+	size_t name_len;
+	int ret;
+
+	if (argc != 2) {
+		printf("usage: rename <filename>\n");
+		printf("will move <filename> to ..\n");
+		return;
+	}
+
+	name = argv[1];
+	name_len = strlen(name);
+
+	ret = ngnfs_dir_lookup(ctx->nfi, &ctx->cwd_ig, "..", strlen(".."), &parent)		?:
+	      ngnfs_dir_rename(ctx->nfi, &ctx->cwd_ig, name, name_len, &parent.ig, name, name_len);
+
+	if (ret < 0) {
+		printf("rename error: "ENOF"\n", ENOA(-ret));
+		return;
+	}
+}
+
 static void cmd_rmdir(struct debugfs_context *ctx, int argc, char **argv)
 {
 	char *name;
@@ -333,6 +363,7 @@ static struct command {
 	{ "mkfs", cmd_mkfs, },
 	{ "quit", cmd_quit, },
 	{ "readdir", cmd_readdir, },
+	{ "rename", cmd_rename, },
 	{ "rmdir", cmd_rmdir, },
 	{ "stat", cmd_stat, },
 	{ "sync", cmd_sync, },
diff --git a/shared/dir.c b/shared/dir.c
index a8a180a..9447b34 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -523,7 +523,7 @@ static int lookup_dots(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
 }
 
 static int lookup_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
-			 struct ngnfs_inode_txn_ref *dir, struct dirent_args *da)
+			 struct ngnfs_inode_txn_ref *dir, struct dirent_args *da, bool enoent_ok)
 {
 	struct ngnfs_btree_key key;
 	struct ngnfs_btree_key last;
@@ -544,6 +544,9 @@ static int lookup_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *tx
 	if (da->dent.ig.ino == 0)
 		ret = -ENOENT;
 out:
+	if (enoent_ok && (ret == -ENOENT))
+		ret = 0;
+
 	return ret;
 }
 
@@ -576,7 +579,7 @@ int ngnfs_dir_lookup(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_
 	do {
 		ret = ngnfs_inode_get(nfi, &op->txn, NBF_READ, dir_ig, &op->dir)		?:
 		      check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)				?:
-		      lookup_dirent(nfi, &op->txn, &op->dir, &op->da)				?:
+		      lookup_dirent(nfi, &op->txn, &op->dir, &op->da, false)			?:
 		      copy_dent_to_lent(&op->da.dent, lent);
 
 	} while (ngnfs_txn_retry(nfi, &op->txn, &ret));
@@ -612,7 +615,7 @@ static int remove_dirent_wr(struct ngnfs_btree_key *key, void *val, size_t size,
  * Remove a dirent item from a directory inode's dirent btree.
  */
 static int remove_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
-			 struct ngnfs_inode_txn_ref *dir, struct dirent_args *da)
+			 struct ngnfs_inode_txn_ref *dir, struct dirent_args *da, bool enoent_ok)
 {
 	struct ngnfs_btree_key key;
 	struct ngnfs_btree_key last;
@@ -624,10 +627,13 @@ static int remove_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *tx
 	ret = ngnfs_btree_write_iter(nfi, txn, dir->tblk, &dir->ninode->dirents, &key, &last,
 				     remove_dirent_wr, da);
 	if (ret < 0)
-		return ret;
+		goto out;
 
 	if (da->dent.ig.ino == 0)
-		ret = ENOENT;
+		ret = -ENOENT;
+out:
+	if (enoent_ok && (ret == -ENOENT))
+		ret = 0;
 
 	return ret;
 }
@@ -701,7 +707,7 @@ static int do_unlink(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_
 	do {
 		ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir_ig, &op->dir)		?:
 		      check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)				?:
-		      remove_dirent(nfi, &op->txn, &op->dir, &op->da)				?:
+		      remove_dirent(nfi, &op->txn, &op->dir, &op->da, false)			?:
 		      ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, &op->da.ig, &op->target)	?:
 		      check_remove_dirent(nfi, &op->txn, &op->dir, &op->target, &op->da, flags)	?:
 		      ngnfs_inode_update(op->target.tblk, op->target.ninode, -1)		?:
@@ -726,3 +732,156 @@ int ngnfs_dir_rmdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir,
 {
 	return do_unlink(nfi, dir, name, name_len, NGNFS_DIRENT_WANT_DIR);
 }
+
+static int igs_equal(struct ngnfs_inode_ino_gen *a, struct ngnfs_inode_ino_gen *b)
+{
+	return (a->ino == b->ino) && (a->gen == b->gen);
+}
+
+/*
+ * To prevent loops when renaming a directory, check if the src is a
+ * directory that is an ancestor of the dst. If it is, we can't rename.
+ * Check by walking up the path starting from the dst parent directory
+ * and comparing each directory to the src.
+ *
+ * When this is called, the parent directories of the src and dst items
+ * are different, and src_da is a directory. src can be the root
+ * directory.
+ */
+static int check_ancestors(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+			   struct dirent_args *src_da, struct ngnfs_inode_txn_ref *dst_dir)
+{
+	struct ngnfs_inode_txn_ref walk_dir;
+	struct ngnfs_inode_ino_gen walk_ig;
+	int ret;
+
+	walk_dir = *dst_dir;
+
+	while(le64_to_cpu(walk_dir.ninode->ig.ino) != NGNFS_ROOT_INO) {
+		walk_ig.ino = le64_to_cpu(walk_dir.ninode->ig.ino);
+		walk_ig.gen = le64_to_cpu(walk_dir.ninode->ig.gen);
+
+		if (igs_equal(&walk_ig, &src_da->ig)) {
+			/* directory loop detected! */
+			ret = -ELOOP;
+			goto out;
+		}
+
+		/* no loop yet, lookup up parent and repeat */
+		walk_ig.ino = le64_to_cpu(walk_dir.ninode->parent_ig.ino);
+		walk_ig.gen = le64_to_cpu(walk_dir.ninode->parent_ig.gen);
+		if (walk_ig.ino == NGNFS_ROOT_INO)
+			break;
+
+		ret = ngnfs_inode_get(nfi, txn, NBF_READ, &walk_ig, &walk_dir);
+		if (ret < 0)
+			goto out;
+	};
+
+	ret = 0;
+out:
+	return ret;
+}
+
+static int inodes_equal(struct ngnfs_inode *a, struct ngnfs_inode *b) {
+	return (a->ig.ino == b->ig.ino) && (a->ig.gen == b->ig.gen);
+}
+
+/*
+ * Check the src and dst paths to rename for problems that can't be
+ * caught during the individual remove/insert/update operations to the
+ * btrees and inodes.
+ */
+static int check_rename(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+			struct ngnfs_inode_txn_ref *src_dir, struct dirent_args *src_da,
+			struct ngnfs_inode_txn_ref *dst_dir, struct dirent_args *dst_da)
+{
+	int ret;
+
+	/* XXX permissions */
+	/* XXX orphan inode handling */
+
+	if (inodes_equal(src_dir->ninode, dst_dir->ninode) &&
+	    names_equal(src_da->dent.name, src_da->dent.name_len,
+			dst_da->dent.name, dst_da->dent.name_len))
+		return -EEXIST;
+
+	/*
+	 * If source is a non-directory, fail if target is a dir,
+	 * otherwise it's success.
+	 */
+	if (src_da->dent.pers_dtype != NGNFS_DT_DIR) {
+		if (dst_da->dent.pers_dtype == NGNFS_DT_DIR)
+			return -EEXIST;
+		return 0;
+	}
+
+	/*
+	 * Source is a directory. If target is a directory, it can only
+	 * be overwritten if it is empty.
+	 */
+	if (dst_da->dent.pers_dtype == NGNFS_DT_DIR) {
+		ret = check_empty_dir(nfi, txn, &dst_da->ig);
+		if (ret < 0)
+			return ret;
+	}
+
+	/* directory loops can't happen when the parent dirs are the same */
+	if (inodes_equal(src_dir->ninode, dst_dir->ninode))
+		return 0;
+
+	/* if the source is the root inode, we sure can't rename it! */
+	if (src_da->ig.ino == NGNFS_ROOT_INO)
+		return -ELOOP;
+
+	/* directory loop possible, check relationship between src and dst path */
+	ret = check_ancestors(nfi, txn, src_da, dst_dir);
+
+	return ret;
+}
+
+int ngnfs_dir_rename(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *src_dir_ig,
+		     char *src_name, size_t src_name_len, struct ngnfs_inode_ino_gen *dst_dir_ig,
+		     char *dst_name, size_t dst_name_len)
+{
+	struct {
+		struct ngnfs_transaction txn;
+		struct ngnfs_inode_txn_ref src_dir, dst_dir, dst;
+		struct dirent_args src_da, dst_da;
+	} *op;
+	int ret;
+
+	op = kmalloc(sizeof(*op), GFP_NOFS);
+	if (!op) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	ngnfs_txn_init(&op->txn);
+	init_dirent_args(&op->src_da, src_name, src_name_len, 0);
+	init_dirent_args(&op->dst_da, dst_name, dst_name_len, 0);
+
+	do {
+		ret = ngnfs_inode_get(nfi, &op->txn, NBF_READ, src_dir_ig, &op->src_dir)	?:
+		      check_ifmt(op->src_dir.ninode, S_IFDIR, -ENOTDIR)				?:
+		      lookup_dirent(nfi, &op->txn, &op->src_dir, &op->src_da, false)		?:
+		      ngnfs_inode_get(nfi, &op->txn, NBF_READ, dst_dir_ig, &op->dst_dir)	?:
+		      check_ifmt(op->dst_dir.ninode, S_IFDIR, -ENOTDIR)				?:
+		      lookup_dirent(nfi, &op->txn, &op->dst_dir, &op->dst_da, true)		?:
+		      check_rename(nfi, &op->txn, &op->src_dir, &op->src_da, &op->dst_dir,
+				   &op->dst_da)							?:
+		      update_dirent_args_ig(&op->dst_da, &op->src_da.ig)			?:
+		      remove_dirent(nfi, &op->txn, &op->src_dir, &op->src_da, false)		?:
+		      update_dir(op->src_dir.tblk, op->src_dir.ninode, &op->src_da, -1)		?:
+		      remove_dirent(nfi, &op->txn, &op->dst_dir, &op->dst_da, true)		?:
+		      update_dirent_args_ig(&op->dst_da, &op->src_da.ig)			?:
+		      insert_dirent(nfi, &op->txn, &op->dst_dir, &op->dst_da)			?:
+		      update_dir(op->dst_dir.tblk, op->dst_dir.ninode, &op->dst_da, 1);
+
+	} while (ngnfs_txn_retry(nfi, &op->txn, &ret));
+
+	ngnfs_txn_teardown(nfi, &op->txn);
+	kfree(op);
+out:
+	return ret;
+}
diff --git a/shared/dir.h b/shared/dir.h
index 8aba78b..2d7eac5 100644
--- a/shared/dir.h
+++ b/shared/dir.h
@@ -14,6 +14,9 @@ int ngnfs_dir_unlink(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir,
 		     size_t name_len);
 int ngnfs_dir_rmdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, char *name,
 		    size_t name_len);
+int ngnfs_dir_rename(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *src_dir_ig,
+		     char *src_name, size_t src_name_len, struct ngnfs_inode_ino_gen *dst_dir_ig,
+		     char *dst_name, size_t dst_name_len);
 
 /*
  * Readdir fills the buffer with entries.  The start of the buffer must
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:48 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:48 +0100
Subject: [PATCH 09/15] Extend debugfs stat command to take an optional file name
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-10-val@versity.com>

Stat used to only show the root inode, now it shows either the current
working directory or the optional filename argument.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 8bc1dc4..c24b9e4 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -164,14 +164,34 @@ static void cmd_readdir(struct debugfs_context *ctx, int argc, char **argv)
 
 static void cmd_stat(struct debugfs_context *ctx, int argc, char **argv)
 {
-	struct ngnfs_inode_ino_gen root_ig = INIT_NGNFS_ROOT_IG;
+	struct ngnfs_inode_ino_gen ig = ctx->cwd_ig;
+	struct ngnfs_dir_lookup_entry lent;
 	struct ngnfs_inode ninode;
+	char *name;
 	int ret;
 
-	ret = ngnfs_inode_read_copy(ctx->nfi, &root_ig, &ninode, sizeof(ninode));
+	if (argc > 2) {
+		printf("usage: stat [filename]\n");
+		return;
+	}
+
+	if (argc == 2) {
+		name = argv[1];
+		ret = ngnfs_dir_lookup(ctx->nfi, &ctx->cwd_ig, name, strlen(name), &lent);
+		if (ret < 0) {
+			if (ret == -ENOENT)
+				printf("stat: %s does not exist\n", name);
+			else
+				log("stat error: "ENOF"\n", ENOA(-ret));
+			return;
+		}
+		ig = lent.ig;
+	}
+
+	ret = ngnfs_inode_read_copy(ctx->nfi, &ig, &ninode, sizeof(ninode));
 
 	if (ret < 0) {
-		log("stat error: %d", ret);
+		log("inode read error: %d", ret);
 	} else if (ret < sizeof(ninode)) {
 		log("returned inode buffer size %d too small, wanted at least %zu",
 		    ret, sizeof(ninode));
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:53 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:53 +0100
Subject: [PATCH 14/15] Add rename benchmark command to debugfs
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-15-val@versity.com>

Add the "brename" command to debugfs to create a bunch of files and
rename them into their parent dir.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 9a61e80..07f6b5f 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -23,6 +23,7 @@
 #include "shared/mkfs.h"
 #include "shared/mount.h"
 #include "shared/nerr.h"
+#include "shared/parse.h"
 #include "shared/thread.h"
 
 #include "cli/cli.h"
@@ -37,6 +38,54 @@ struct debugfs_context {
 #define LINE_SIZE (PATH_MAX * 5)
 #define MAX_ARGC ((LINE_SIZE + 1) / 2)
 
+static void cmd_brename(struct debugfs_context *ctx, int argc, char **argv)
+{
+	struct ngnfs_dir_lookup_entry lent, sub_lent;
+	char filename[NGNFS_NAME_MAX];
+	char *subdir = "tmp";
+	char *dir;
+	u64 renames, i;
+	int ret;
+
+	if (argc != 3) {
+		printf("usage: %s <directory name> <number renames>\n", argv[0]);
+		return;
+	}
+
+	dir = argv[1];
+
+	ret = parse_ull(&renames, argv[2], 1, NGNFS_LINK_MAX - 1);
+	if (ret < 0) {
+		printf("number of renames must be between 1 and %d\n", NGNFS_LINK_MAX - 1);
+		return;
+	}
+
+	ret = ngnfs_dir_mkdir(ctx->nfi, &ctx->cwd_ig, 0755, dir, strlen(dir))			?:
+	      ngnfs_dir_lookup(ctx->nfi, &ctx->cwd_ig, dir, strlen(dir), &lent)			?:
+	      ngnfs_dir_mkdir(ctx->nfi, &lent.ig, 0755, subdir, strlen(subdir))			?:
+	      ngnfs_dir_lookup(ctx->nfi, &lent.ig, subdir, strlen(subdir), &sub_lent);
+
+	if (ret < 0) {
+		printf("error creating %s/%s/:"ENOF"\n", dir, subdir, ENOA(-ret));
+		return;
+	}
+
+	for (i = 0; i < renames; i++) {
+		snprintf(filename, sizeof(filename), "%llu", i);
+		ret = ngnfs_dir_create(ctx->nfi, &sub_lent.ig, 0644, filename, strlen(filename));
+		if (ret < 0) {
+			printf("create error: "ENOF"\n", ENOA(-ret));
+			return;
+		}
+		ret = ngnfs_dir_rename(ctx->nfi, &sub_lent.ig, filename, strlen(filename), &lent.ig,
+				       filename, strlen(filename));
+		if (ret < 0) {
+			printf("rename error: "ENOF"\n", ENOA(-ret));
+			return;
+		}
+	}
+}
+
 static void cmd_cd(struct debugfs_context *ctx, int argc, char **argv)
 {
 	struct ngnfs_dir_lookup_entry lent;
@@ -356,6 +405,7 @@ static struct command {
 	char *name;
 	void (*func)(struct debugfs_context *ctx, int argc, char **argv);
 } commands[] = {
+	{ "brename", cmd_brename, },
 	{ "cd", cmd_cd, },
 	{ "create", cmd_create, },
 	{ "lookup", cmd_lookup, },
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:49 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:49 +0100
Subject: [PATCH 10/15] Add the cd command to debugfs
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-11-val@versity.com>

Allow changing the current working directory in debugfs.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index c24b9e4..0c0914c 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -1,7 +1,9 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 
+#include <dirent.h>
 #include <signal.h>
 #include <unistd.h>
+#include <sys/stat.h>
 #include <stdlib.h>
 #include <stdio.h>
 #include <string.h>
@@ -35,6 +37,36 @@ struct debugfs_context {
 #define LINE_SIZE (PATH_MAX * 5)
 #define MAX_ARGC ((LINE_SIZE + 1) / 2)
 
+static void cmd_cd(struct debugfs_context *ctx, int argc, char **argv)
+{
+	struct ngnfs_dir_lookup_entry lent;
+	char *name;
+	int ret;
+
+	if (argc != 2) {
+		printf("usage: cd <directory name>\n");
+		return;
+	}
+
+	name = argv[1];
+
+	ret = ngnfs_dir_lookup(ctx->nfi, &ctx->cwd_ig, name, strlen(name), &lent);
+	if (ret < 0) {
+		if (ret == -ENOENT)
+			printf("lookup: %s does not exist\n", name);
+		else
+			log("lookup error: "ENOF"\n", ENOA(-ret));
+		return;
+	}
+
+	if (lent.dtype != DT_DIR) {
+		printf("not a directory: %s", name);
+		return;
+	}
+
+	ctx->cwd_ig = lent.ig;
+}
+
 static void cmd_create(struct debugfs_context *ctx, int argc, char **argv)
 {
 	int ret;
@@ -233,6 +265,7 @@ static struct command {
 	char *name;
 	void (*func)(struct debugfs_context *ctx, int argc, char **argv);
 } commands[] = {
+	{ "cd", cmd_cd, },
 	{ "create", cmd_create, },
 	{ "lookup", cmd_lookup, },
 	{ "mkfs", cmd_mkfs, },
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:51 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:51 +0100
Subject: [PATCH 12/15] Add ngnfs_dir_unlink(), ngfs_dir_rmdir(), and debugfs commands
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-13-val@versity.com>

Allow removal of links to files and directories.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c  |  43 +++++++++++++
 shared/dir.c   | 170 +++++++++++++++++++++++++++++++++++++++++++++----
 shared/dir.h   |   4 ++
 shared/inode.c |  20 ++++++
 shared/inode.h |   1 +
 5 files changed, 227 insertions(+), 11 deletions(-)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 3494070..32fc3f6 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -214,6 +214,27 @@ static void cmd_readdir(struct debugfs_context *ctx, int argc, char **argv)
 	free(buf);
 }
 
+static void cmd_rmdir(struct debugfs_context *ctx, int argc, char **argv)
+{
+	char *name;
+	int ret;
+
+	if (argc != 2) {
+		printf("usage: rmdir <pathname>\n");
+		return;
+	}
+
+	name = argv[1];
+
+	ret = ngnfs_dir_rmdir(ctx->nfi, &ctx->cwd_ig, name, strlen(name));
+	if (ret < 0) {
+		if (ret == -ENOENT)
+			printf("rmdir: %s does not exist\n", name);
+		else
+			log("rmdir error: "ENOF"\n", ENOA(-ret));
+	}
+}
+
 static void cmd_stat(struct debugfs_context *ctx, int argc, char **argv)
 {
 	struct ngnfs_inode_ino_gen ig = ctx->cwd_ig;
@@ -280,6 +301,26 @@ static void cmd_sync(struct debugfs_context *ctx, int argc, char **argv)
 		printf("sync error: "ENOF"\n", ENOA(-ret));
 }
 
+static void cmd_unlink(struct debugfs_context *ctx, int argc, char **argv)
+{
+	char *name;
+	int ret;
+
+	if (argc != 2) {
+		printf("usage: unlink <pathname>\n");
+		return;
+	}
+
+	name = argv[1];
+
+	ret = ngnfs_dir_unlink(ctx->nfi, &ctx->cwd_ig, name, strlen(name));
+	if (ret < 0) {
+		if (ret == -ENOENT)
+			printf("unlink: %s does not exist\n", name);
+		else
+			log("unlink error: "ENOF"\n", ENOA(-ret));
+	}
+}
 
 static struct command {
 	char *name;
@@ -292,8 +333,10 @@ static struct command {
 	{ "mkfs", cmd_mkfs, },
 	{ "quit", cmd_quit, },
 	{ "readdir", cmd_readdir, },
+	{ "rmdir", cmd_rmdir, },
 	{ "stat", cmd_stat, },
 	{ "sync", cmd_sync, },
+	{ "unlink", cmd_unlink, },
 };
 
 static int compar_cmd_names(const void *A, const void *B)
diff --git a/shared/dir.c b/shared/dir.c
index ebeea01..a8a180a 100644
--- a/shared/dir.c
+++ b/shared/dir.c
@@ -77,6 +77,15 @@ static u64 name_hash(void *name, size_t name_len)
 	return hash;
 }
 
+/*
+ * Flags to specify which tests should be done before removing a
+ * directory entry.
+ */
+enum {
+	NGNFS_DIRENT_WANT_DIR = 	0x1,
+	NGNFS_DIRENT_WILL_MOVE = 	0x2,
+};
+
 /*
  * These large dirent structs can fit a full sized name so that we can
  * copy in and out any dirent as we work with them.
@@ -173,27 +182,26 @@ static inline int check_ifmt(struct ngnfs_inode *ninode, u32 ifmt, int err)
 }
 
 /*
- * Update a directory's inode to reflect creation.  We can return errors
- * if the create should fail.
+ * Update a parent directory's inode to reflect creation or deletion of
+ * an entry. We can return errors if the create should fail.
  */
 static int update_dir(struct ngnfs_txn_block *tblk, struct ngnfs_inode *dir,
-		      struct dirent_args *da, int posneg)
+		      struct dirent_args *da, s32 nlink_delta)
 {
-	s32 delta;
+	s32 dent_bytes;
+	s32 posneg;
 	int ret;
 
 	if (da->dent.pers_dtype == NGNFS_DT_DIR) {
-		delta = posneg * 1;
-		if ((le32_to_cpu(dir->nlink) + delta >= NGNFS_LINK_MAX)) {
-			ret = -EMLINK;
+		ret = ngnfs_inode_update(tblk, dir, nlink_delta);
+		if (ret < 0)
 			goto out;
-		}
-		ngnfs_tblk_assign(tblk, dir->nlink, cpu_to_le32(le32_to_cpu(dir->nlink) + delta));
 	}
 
+	posneg = nlink_delta >= 0 ? 1 : -1;
 	/* dir i_size includes null termed names */
-	delta = posneg * ((s32)da->dent.name_len + 1);
-	ngnfs_tblk_assign(tblk, dir->size, cpu_to_le64(le64_to_cpu(dir->size) + delta));
+	dent_bytes = posneg * ((s32)da->dent.name_len + 1);
+	ngnfs_tblk_assign(tblk, dir->size, cpu_to_le64(le64_to_cpu(dir->size) + dent_bytes));
 	ret = 0;
 out:
 	return ret;
@@ -578,3 +586,143 @@ int ngnfs_dir_lookup(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_
 	kfree(op);
 	return ret;
 }
+
+static int remove_dirent_wr(struct ngnfs_btree_key *key, void *val, size_t size, void *arg,
+			    struct ngnfs_btree_op *op)
+{
+	struct ngnfs_dirent *dent = val;
+	struct dirent_args *da = arg;
+
+	if (!dent)
+		return -ENOENT;
+
+	if (!names_equal(dent->name, dent->name_len, da->dent.name, da->dent.name_len))
+		return NGNFS_BTREE_ITER_CONTINUE;
+
+	da->ig.ino = le64_to_cpu(dent->ig.ino);
+	da->ig.gen = le64_to_cpu(dent->ig.gen);
+	memcpy(&da->dent, dent, size);
+
+	op->delete = 1;
+
+	return 0;
+}
+
+/*
+ * Remove a dirent item from a directory inode's dirent btree.
+ */
+static int remove_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+			 struct ngnfs_inode_txn_ref *dir, struct dirent_args *da)
+{
+	struct ngnfs_btree_key key;
+	struct ngnfs_btree_key last;
+	int ret;
+
+	init_dirent_key(&key, da->hash);
+	init_dirent_key(&last, da->hash | NGNFS_DIRENT_COLL_BIT);
+
+	ret = ngnfs_btree_write_iter(nfi, txn, dir->tblk, &dir->ninode->dirents, &key, &last,
+				     remove_dirent_wr, da);
+	if (ret < 0)
+		return ret;
+
+	if (da->dent.ig.ino == 0)
+		ret = ENOENT;
+
+	return ret;
+}
+
+static int check_empty_dir(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+			   struct ngnfs_inode_ino_gen *ig)
+{
+	struct ngnfs_inode_txn_ref new;
+	int ret;
+
+	ret = ngnfs_inode_get(nfi, txn, NBF_READ, ig, &new);
+	if (ret < 0)
+		return ret;
+
+	if (le32_to_cpu(new.ninode->nlink) != 2)
+		return -ENOTEMPTY;
+
+	return 0;
+}
+
+/*
+ * Run final checks before removing a dirent, depending on whether we
+ * want a directory or if we are deleting or moving a dirent.
+ */
+static int check_remove_dirent(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
+			       struct ngnfs_inode_txn_ref *dir, struct ngnfs_inode_txn_ref *new,
+			       struct dirent_args *da, int flags)
+{
+	if (!(flags & NGNFS_DIRENT_WANT_DIR)) {
+		if (da->dent.pers_dtype == NGNFS_DT_DIR)
+			return -EISDIR;
+		else
+			return 0;
+	}
+
+	if (da->dent.pers_dtype != NGNFS_DT_DIR)
+		return -ENOTDIR;
+
+	if (flags & NGNFS_DIRENT_WILL_MOVE)
+		return 0;
+
+	return check_empty_dir(nfi, txn, &da->ig);
+}
+
+/*
+ * Unlink a directory entry.
+ */
+static int do_unlink(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir_ig, char *name,
+		     size_t name_len, int flags)
+{
+	struct {
+		struct ngnfs_transaction txn;
+		struct ngnfs_inode_txn_ref dir;
+		struct ngnfs_inode_txn_ref target;
+		struct dirent_args da;
+		u64 ino;
+		u64 nsec;
+	} *op;
+	int ret;
+
+	op = kmalloc(sizeof(*op), GFP_NOFS);
+	if (!op) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	ngnfs_txn_init(&op->txn);
+	op->nsec = ktime_to_ns(ktime_get_real());
+	init_dirent_args(&op->da, name, name_len, 0);
+
+	do {
+		ret = ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, dir_ig, &op->dir)		?:
+		      check_ifmt(op->dir.ninode, S_IFDIR, -ENOTDIR)				?:
+		      remove_dirent(nfi, &op->txn, &op->dir, &op->da)				?:
+		      ngnfs_inode_get(nfi, &op->txn, NBF_WRITE, &op->da.ig, &op->target)	?:
+		      check_remove_dirent(nfi, &op->txn, &op->dir, &op->target, &op->da, flags)	?:
+		      ngnfs_inode_update(op->target.tblk, op->target.ninode, -1)		?:
+		      update_dir(op->dir.tblk, op->dir.ninode, &op->da, -1);
+
+	} while (ngnfs_txn_retry(nfi, &op->txn, &ret));
+
+	ngnfs_txn_teardown(nfi, &op->txn);
+	kfree(op);
+out:
+	return ret;
+}
+
+int ngnfs_dir_unlink(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, char *name,
+		     size_t name_len)
+{
+	return do_unlink(nfi, dir, name, name_len, 0);
+}
+
+int ngnfs_dir_rmdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, char *name,
+		    size_t name_len)
+{
+	return do_unlink(nfi, dir, name, name_len, NGNFS_DIRENT_WANT_DIR);
+}
diff --git a/shared/dir.h b/shared/dir.h
index 5dd57dd..8aba78b 100644
--- a/shared/dir.h
+++ b/shared/dir.h
@@ -10,6 +10,10 @@ int ngnfs_dir_create(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir,
 		     char *name, size_t name_len);
 int ngnfs_dir_mkdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, umode_t mode,
 		    char *name, size_t name_len);
+int ngnfs_dir_unlink(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, char *name,
+		     size_t name_len);
+int ngnfs_dir_rmdir(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *dir, char *name,
+		    size_t name_len);
 
 /*
  * Readdir fills the buffer with entries.  The start of the buffer must
diff --git a/shared/inode.c b/shared/inode.c
index 2b29a1f..370cc50 100644
--- a/shared/inode.c
+++ b/shared/inode.c
@@ -57,6 +57,8 @@ int ngnfs_inode_init(struct ngnfs_inode_txn_ref *itref, struct ngnfs_inode_ino_g
 int ngnfs_inode_get(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn, nbf_t nbf,
 		    struct ngnfs_inode_ino_gen *ig, struct ngnfs_inode_txn_ref *itref)
 {
+	BUG_ON(ig->ino == 0);
+
 	return ngnfs_txn_get_block(nfi, txn, ig->ino, nbf, &itref->tblk, (void **)&itref->ninode);
 }
 
@@ -106,3 +108,21 @@ int ngnfs_inode_read_copy(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen
 out:
 	return ret;
 }
+
+/*
+ * Update an inode to reflect the addition or removal of one or more links to it.
+ */
+int ngnfs_inode_update(struct ngnfs_txn_block *tblk, struct ngnfs_inode *inode, s32 delta)
+{
+	s32 nlink = le32_to_cpu(inode->nlink);
+
+	if ((delta > 0) && (nlink > NGNFS_LINK_MAX - delta))
+		return -EMLINK;
+
+	/* nlink < 0 is a bug */
+	BUG_ON((delta < 0) && (nlink + delta < 0));
+
+	ngnfs_tblk_assign(tblk, inode->nlink, cpu_to_le32(nlink + delta));
+
+	return 0;
+}
diff --git a/shared/inode.h b/shared/inode.h
index 92b5114..fe07b49 100644
--- a/shared/inode.h
+++ b/shared/inode.h
@@ -27,5 +27,6 @@ int ngnfs_inode_alloc(struct ngnfs_fs_info *nfi, struct ngnfs_transaction *txn,
 		      struct ngnfs_inode_ino_gen *ig, struct ngnfs_inode_txn_ref *itref);
 int ngnfs_inode_read_copy(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *ig,
 			  void *buf, int size);
+int ngnfs_inode_update(struct ngnfs_txn_block *tblk, struct ngnfs_inode *inode, int posneg);
 
 #endif
-- 
2.48.1



From val at versity.com  Wed Mar 12 07:33:54 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 15:33:54 +0100
Subject: [PATCH 15/15] Add create benchmark command to debugfs
In-Reply-To: <20250312143354.220766-1-val@versity.com>
References: <20250312143354.220766-1-val@versity.com>
Message-ID: <20250312143354.220766-16-val@versity.com>

Add the "bcreate" command to debugfs to create a bunch of files.

Signed-off-by: Valerie Aurora <val at versity.com>
---
 cli/debugfs.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 07f6b5f..e0a5aa2 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -38,6 +38,44 @@ struct debugfs_context {
 #define LINE_SIZE (PATH_MAX * 5)
 #define MAX_ARGC ((LINE_SIZE + 1) / 2)
 
+static void cmd_bcreate(struct debugfs_context *ctx, int argc, char **argv)
+{
+	struct ngnfs_dir_lookup_entry lent;
+	char filename[NGNFS_NAME_MAX];
+	char *dir;
+	u64 creates, i;
+	int ret;
+
+	if (argc != 3) {
+		printf("usage: %s <directory name> <number creates>\n", argv[0]);
+		return;
+	}
+
+	dir = argv[1];
+	ret = parse_ull(&creates, argv[2], 1, NGNFS_LINK_MAX - 1);
+	if (ret < 0) {
+		printf("number of creates must be between 1 and %d\n", NGNFS_LINK_MAX - 1);
+		return;
+	}
+
+	ret = ngnfs_dir_mkdir(ctx->nfi, &ctx->cwd_ig, 0755, dir, strlen(dir))			?:
+	      ngnfs_dir_lookup(ctx->nfi, &ctx->cwd_ig, dir, strlen(dir), &lent);
+
+	if (ret < 0) {
+		printf("error creating %s:"ENOF"\n", dir, ENOA(-ret));
+		return;
+	}
+
+	for (i = 0; i < creates; i++) {
+		snprintf(filename, sizeof(filename), "%llu", i);
+		ret = ngnfs_dir_create(ctx->nfi, &lent.ig, 0644, filename, strlen(filename));
+		if (ret < 0) {
+			printf("create error: "ENOF"\n", ENOA(-ret));
+			return;
+		}
+	}
+}
+
 static void cmd_brename(struct debugfs_context *ctx, int argc, char **argv)
 {
 	struct ngnfs_dir_lookup_entry lent, sub_lent;
@@ -405,6 +443,7 @@ static struct command {
 	char *name;
 	void (*func)(struct debugfs_context *ctx, int argc, char **argv);
 } commands[] = {
+	{ "bcreate", cmd_bcreate, },
 	{ "brename", cmd_brename, },
 	{ "cd", cmd_cd, },
 	{ "create", cmd_create, },
-- 
2.48.1



From val at versity.com  Wed Mar 12 09:05:03 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 12 Mar 2025 17:05:03 +0100
Subject: [PATCH 05/15] Move inode number and generation into ino_gen struct
In-Reply-To: <20250312143354.220766-6-val@versity.com>
References: <20250312143354.220766-1-val@versity.com> <20250312143354.220766-6-val@versity.com>
Message-ID: <CALGtELhE0jXjp3PzbQ-gfhpyErcfS1y52DZvL8tsXx-8sW6dCQ@mail.gmail.com>

On Wed, Mar 12, 2025 at 3:34?PM Valerie Aurora <val at versity.com> wrote:
>
> diff --git a/shared/dir.h b/shared/dir.h
> index 5594162..8ca28b8 100644
> --- a/shared/dir.h
> +++ b/shared/dir.h
> @@ -2,25 +2,25 @@
>  #ifndef NGNFS_SHARED_DIR_H
>  #define NGNFS_SHARED_DIR_H
>
> +#include "shared/inode.h"
>  #include "shared/fs_info.h"

My turn to forget the alphabet! This should be moved down and then the
parent inode patch won't duplicate the inode.h include, haha.

Valerie


From ckirby at versity.com  Thu Mar 13 11:10:10 2025
From: ckirby at versity.com (Chris Kirby)
Date: Thu, 13 Mar 2025 13:10:10 -0500
Subject: [PATCH 0/1] fix socket_recv_thread() panic
Message-ID: <cover.1741880944.git.ckirby@versity.com>

Chris Kirby (1):
  In socket_recv_thread(), don't try to iov_append() a NULL buffer.

 shared/mtr-socket.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

-- 
2.39.5 (Apple Git-154)



From ckirby at versity.com  Thu Mar 13 11:10:11 2025
From: ckirby at versity.com (Chris Kirby)
Date: Thu, 13 Mar 2025 13:10:11 -0500
Subject: [PATCH 1/1] In socket_recv_thread(), don't try to iov_append() a NULL buffer.
In-Reply-To: <cover.1741880944.git.ckirby@versity.com>
References: <cover.1741880944.git.ckirby@versity.com>
Message-ID: <bc8b2fd125c17d181d9d8affecd4db0f4bd93e49.1741880944.git.ckirby@versity.com>

Signed-off-by: Chris Kirby <ckirby at versity.com>
---
 shared/mtr-socket.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/shared/mtr-socket.c b/shared/mtr-socket.c
index 3794c0f..e6d520d 100644
--- a/shared/mtr-socket.c
+++ b/shared/mtr-socket.c
@@ -212,7 +212,10 @@ static void socket_recv_thread(struct thread *thr, void *arg)
 		}
 
 		iovcnt = iov_append(iov, 0, page_address(ctl_page), mdesc.ctl_size);
-		iovcnt = iov_append(iov, iovcnt, page_address(mdesc.data_page), mdesc.data_size);
+
+		if (mdesc.data_size) {
+			iovcnt = iov_append(iov, iovcnt, page_address(mdesc.data_page), mdesc.data_size);
+		}
 
 		ret = whole_iovec(readv, pinf->fd, iov, iovcnt);
 		if (ret < 0)
-- 
2.39.5 (Apple Git-154)



From val at versity.com  Fri Mar 14 11:45:58 2025
From: val at versity.com (Valerie Aurora)
Date: Fri, 14 Mar 2025 19:45:58 +0100
Subject: [PATCH 13/15] Add ngnfs_dir_rename() and very basic debugfs command
In-Reply-To: <20250312143354.220766-14-val@versity.com>
References: <20250312143354.220766-1-val@versity.com> <20250312143354.220766-14-val@versity.com>
Message-ID: <CALGtELj4rXsVOGL_K5_L6N=L5yoHPRT_S-4uspm=b3X43oth+w@mail.gmail.com>

On Wed, Mar 12, 2025 at 3:34?PM Valerie Aurora <val at versity.com> wrote:
>
> This has a known bug where after around 600 renames it returns a
> spurious ENOENT.

Hurray, I found this pesky bug when I made the same mistake writing xattrs:

> +int ngnfs_dir_rename(struct ngnfs_fs_info *nfi, struct ngnfs_inode_ino_gen *src_dir_ig,
> +                    char *src_name, size_t src_name_len, struct ngnfs_inode_ino_gen *dst_dir_ig,
> +                    char *dst_name, size_t dst_name_len)
> +{
> +       struct {
> +               struct ngnfs_transaction txn;
> +               struct ngnfs_inode_txn_ref src_dir, dst_dir, dst;
> +               struct dirent_args src_da, dst_da;
> +       } *op;
> +       int ret;
> +
> +       op = kmalloc(sizeof(*op), GFP_NOFS);
> +       if (!op) {
> +               ret = -ENOMEM;
> +               goto out;
> +       }
> +
> +       ngnfs_txn_init(&op->txn);
> +       init_dirent_args(&op->src_da, src_name, src_name_len, 0);
> +       init_dirent_args(&op->dst_da, dst_name, dst_name_len, 0);
> +
> +       do {
> +               ret = ngnfs_inode_get(nfi, &op->txn, NBF_READ, src_dir_ig, &op->src_dir)        ?:
> +                     check_ifmt(op->src_dir.ninode, S_IFDIR, -ENOTDIR)                         ?:
> +                     lookup_dirent(nfi, &op->txn, &op->src_dir, &op->src_da, false)            ?:
> +                     ngnfs_inode_get(nfi, &op->txn, NBF_READ, dst_dir_ig, &op->dst_dir)        ?:

These two NBF_READs should be NBF_WRITE, of course.

Valerie


From val at versity.com  Tue Mar 18 07:48:50 2025
From: val at versity.com (Valerie Aurora)
Date: Tue, 18 Mar 2025 15:48:50 +0100
Subject: [PATCH 1/1] In socket_recv_thread(), don't try to iov_append() a
 NULL buffer.
In-Reply-To: <bc8b2fd125c17d181d9d8affecd4db0f4bd93e49.1741880944.git.ckirby@versity.com>
References: <cover.1741880944.git.ckirby@versity.com> <bc8b2fd125c17d181d9d8affecd4db0f4bd93e49.1741880944.git.ckirby@versity.com>
Message-ID: <CALGtELjd0sbuigKJuGvy9urs86E5d3xeDQSx18Uo6xzkxAvTpg@mail.gmail.com>

On Thu, Mar 13, 2025 at 7:52?PM Chris Kirby <ckirby at versity.com> wrote:
>
> Signed-off-by: Chris Kirby <ckirby at versity.com>
> ---
>  shared/mtr-socket.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/shared/mtr-socket.c b/shared/mtr-socket.c
> index 3794c0f..e6d520d 100644
> --- a/shared/mtr-socket.c
> +++ b/shared/mtr-socket.c
> @@ -212,7 +212,10 @@ static void socket_recv_thread(struct thread *thr, void *arg)
>                 }
>
>                 iovcnt = iov_append(iov, 0, page_address(ctl_page), mdesc.ctl_size);
> -               iovcnt = iov_append(iov, iovcnt, page_address(mdesc.data_page), mdesc.data_size);
> +
> +               if (mdesc.data_size) {
> +                       iovcnt = iov_append(iov, iovcnt, page_address(mdesc.data_page), mdesc.data_size);
> +               }
>
>                 ret = whole_iovec(readv, pinf->fd, iov, iovcnt);
>                 if (ret < 0)
> --
> 2.39.5 (Apple Git-154)

Nice catch, thanks for finding this! I was one of the people whose
compiler hid this.

This looks correct as far as my limited understanding goes, so I just
have some tiny style notes:

Since it is only one line inside the if statement, maybe remove the brackets?

I like the commit message but I think it would work better with an
even shorter title and the current title as the body, plus maybe a
note about why it might not show up for some people:

Fix null deference in socket_recv_thread()

In socket_recv_thread(), don't try to iov_append() a NULL buffer.
Compiler optimizations hide this bug for some versions of gcc.

Thanks,

Valerie


From val at versity.com  Wed Mar 19 02:59:28 2025
From: val at versity.com (Valerie Aurora)
Date: Wed, 19 Mar 2025 10:59:28 +0100
Subject: [PATCH 12/15] Add ngnfs_dir_unlink(), ngfs_dir_rmdir(), and
 debugfs commands
In-Reply-To: <20250312143354.220766-13-val@versity.com>
References: <20250312143354.220766-1-val@versity.com> <20250312143354.220766-13-val@versity.com>
Message-ID: <CALGtELg3RHgR9sHOtWpY=B0PVR6jRnFaehsR92RZsDb66pyr4Q@mail.gmail.com>

On Wed, Mar 12, 2025 at 3:34?PM Valerie Aurora <val at versity.com> wrote:
>  static int compar_cmd_names(const void *A, const void *B)
> diff --git a/shared/dir.c b/shared/dir.c
> index ebeea01..a8a180a 100644
> --- a/shared/dir.c
> +++ b/shared/dir.c
> @@ -77,6 +77,15 @@ static u64 name_hash(void *name, size_t name_len)
>         return hash;
>  }
>
> +/*
> + * Flags to specify which tests should be done before removing a
> + * directory entry.
> + */
> +enum {
> +       NGNFS_DIRENT_WANT_DIR =         0x1,
> +       NGNFS_DIRENT_WILL_MOVE =        0x2,
> +};
> +

Looking at other parts of the code, these should be shortened to
something like RMD_WANT_DIR. On my todo list!

Valerie


From ckirby at versity.com  Fri Mar 21 06:25:58 2025
From: ckirby at versity.com (Chris Kirby)
Date: Fri, 21 Mar 2025 08:25:58 -0500
Subject: [PATCH 1/1] Fix null dereference in socket_recv_thread()
Message-ID: <82cb33868a9a37c36e1de6f790a035ea96edaf35.1742397833.git.ckirby@versity.com>

In socket_recv_thread(), don't try to iov_append() a NULL buffer.
Compiler optimizations hide this bug for some versions of gcc.

Signed-off-by: Chris Kirby <ckirby at versity.com>
---
 shared/mtr-socket.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/shared/mtr-socket.c b/shared/mtr-socket.c
index 3794c0f..db8221a 100644
--- a/shared/mtr-socket.c
+++ b/shared/mtr-socket.c
@@ -212,7 +212,9 @@ static void socket_recv_thread(struct thread *thr, void *arg)
 		}
 
 		iovcnt = iov_append(iov, 0, page_address(ctl_page), mdesc.ctl_size);
-		iovcnt = iov_append(iov, iovcnt, page_address(mdesc.data_page), mdesc.data_size);
+
+		if (mdesc.data_size)
+			iovcnt = iov_append(iov, iovcnt, page_address(mdesc.data_page), mdesc.data_size);
 
 		ret = whole_iovec(readv, pinf->fd, iov, iovcnt);
 		if (ret < 0)
-- 
2.39.5 (Apple Git-154)



From zab at zabbo.net  Fri Mar 21 13:29:02 2025
From: zab at zabbo.net (Zach Brown)
Date: Fri, 21 Mar 2025 13:29:02 -0700
Subject: Landed initial utask and cache mode support
Message-ID: <20250321202902.GA313511@localhost.localdomain>

Hi all,

I wanted to send a note to highlight the arrival of two significant
pieces of ngnfs.

The first change is the negotiation of cache modes between the network
client and the devd device servers.  The block IO network messages are
augmented a bit to request and grant read or write mode associated with
a cached block.  The devd server sends mode revocation messages around
to ensure that all clients are compatible with granted modes.

To support this the client block cache was significantly cleaned up
(imnsho :)) to make it a lot easier to make atomic transitions between
complicated states.  It's no longer a pre-cmpxchg sea of atomic bit
operations that made me sad.

The second change is to move from a multi-threaded runtime in the
userspace processes to a model of voluntary pre-emption amongst stacks
in a single threaded context.  It makes the userspace processing paths
so very much easier to work with.  As part of this, we use io_uring to
drive forward progress in response to external events. Personally, I'm
pretty excited to see how far we can push this.

There's a lot of iteration to do on these foundations, of course, but
this brings us a lot closer to building fs structures in a consistent
block store that's distributed across the network.

- z


From val at versity.com  Mon Mar 24 13:17:55 2025
From: val at versity.com (Valerie Aurora)
Date: Mon, 24 Mar 2025 21:17:55 +0100
Subject: Landed initial utask and cache mode support
In-Reply-To: <20250321202902.GA313511@localhost.localdomain>
References: <20250321202902.GA313511@localhost.localdomain>
Message-ID: <CALGtELhtK1vJxdZdQxjGg228ZTs4i57NinNK1JAzLAUX2+0UzQ@mail.gmail.com>

On Fri, Mar 21, 2025 at 9:29?PM Zach Brown <zab at zabbo.net> wrote:
>
> Hi all,
>
> I wanted to send a note to highlight the arrival of two significant
> pieces of ngnfs.
>
> The first change is the negotiation of cache modes between the network
> client and the devd device servers.  The block IO network messages are
> augmented a bit to request and grant read or write mode associated with
> a cached block.  The devd server sends mode revocation messages around
> to ensure that all clients are compatible with granted modes.
>
> To support this the client block cache was significantly cleaned up
> (imnsho :)) to make it a lot easier to make atomic transitions between
> complicated states.  It's no longer a pre-cmpxchg sea of atomic bit
> operations that made me sad.
>
> The second change is to move from a multi-threaded runtime in the
> userspace processes to a model of voluntary pre-emption amongst stacks
> in a single threaded context.  It makes the userspace processing paths
> so very much easier to work with.  As part of this, we use io_uring to
> drive forward progress in response to external events. Personally, I'm
> pretty excited to see how far we can push this.
>
> There's a lot of iteration to do on these foundations, of course, but
> this brings us a lot closer to building fs structures in a consistent
> block store that's distributed across the network.

This is great news and exciting progress! I'm looking forward to
learning more as soon as I finish this current project.

As another status update, I've just implemented small (~500 byte)
extended attributes using the btrees and a very simple form of file
data (a sparse tree of indirect blocks rooted in the inode, every data
block has the same number of indirect blocks). I'll be polishing this
up and rebasing it on main over the next few days.

Valerie


From auke.kok at versity.com  Mon Mar 24 14:29:10 2025
From: auke.kok at versity.com (Auke Kok)
Date: Mon, 24 Mar 2025 14:29:10 -0700
Subject: [PATCH 09/15] Extend debugfs stat command to take an optional
 file name
In-Reply-To: <20250312143354.220766-10-val@versity.com>
References: <20250312143354.220766-1-val@versity.com> <20250312143354.220766-10-val@versity.com>
Message-ID: <CAPXXN1AUzjVrXkdK=v-X4P7PuwcOZtycUi1WONSjqrVuBOibMg@mail.gmail.com>

On Wed, Mar 12, 2025 at 7:34?AM Valerie Aurora <val at versity.com> wrote:
> Stat used to only show the root inode, now it shows either the current
> working directory or the optional filename argument.

I just noticed this. Maybe it makes sense for debugfs to just work
with the inode ID, instead of the file name? It would avoid the
pitfalls of pathname resolution, and I think we also want a debugfs
command to separately do path-to-inode lookups anyway.

Auke


From val at versity.com  Thu Mar 27 13:33:30 2025
From: val at versity.com (Valerie Aurora)
Date: Thu, 27 Mar 2025 21:33:30 +0100
Subject: Maximum transaction size/splitting up large transactions?
Message-ID: <CALGtELj4aF-6xLFaM+2SpsC9GF5WFfXoCp5q+C+qr5NGCPLKwg@mail.gmail.com>

Hey ngneers,

I've implemented file data reads/writes, and I have this comment
outside the main loop of per block reads/writes:

/* XXX break up into smaller transactions */

Is this something to leave for later, or is it worth discussing now?
For file data, my ideal would be some way to check how much space I
have left in the transaction before I add another block and its
potential additional indirect blocks.

Valerie


From zab at zabbo.net  Fri Mar 28 11:05:40 2025
From: zab at zabbo.net (Zach Brown)
Date: Fri, 28 Mar 2025 11:05:40 -0700
Subject: Maximum transaction size/splitting up large transactions?
In-Reply-To: <CALGtELj4aF-6xLFaM+2SpsC9GF5WFfXoCp5q+C+qr5NGCPLKwg@mail.gmail.com>
References: <CALGtELj4aF-6xLFaM+2SpsC9GF5WFfXoCp5q+C+qr5NGCPLKwg@mail.gmail.com>
Message-ID: <20250328180540.GA8090@localhost.localdomain>

On Thu, Mar 27, 2025 at 09:33:30PM +0100, Valerie Aurora wrote:
> Hey ngneers,
> 
> I've implemented file data reads/writes, and I have this comment
> outside the main loop of per block reads/writes:
> 
> /* XXX break up into smaller transactions */
> 
> Is this something to leave for later, or is it worth discussing now?
> For file data, my ideal would be some way to check how much space I
> have left in the transaction before I add another block and its
> potential additional indirect blocks.

I imagine we should leave it for later, specifically for when the block
cache is able to track and send writes as transactions.  But it's always
nice to discuss to frame the problem :).

I think my initial bias is to try to not have code paths that are
managing transaction sizes that way.  Simply 'cause not doing it is less
code (and control flows that need to be (not) tested).

These client side transaction boundaries are just what are being
atomically dirtied.  They might well be being merged into larger
transactions that will eventually be sent over the wire. 

- z


From auke.kok at versity.com  Fri Mar 28 11:20:37 2025
From: auke.kok at versity.com (Auke Kok)
Date: Fri, 28 Mar 2025 11:20:37 -0700
Subject: [PATCH] Insert \n before cmd output when input is not a tty.
Message-ID: <20250328182037.19171-1-auke.kok@versity.com>

The output of piped input commands used by the tests is causing the
output to be concatenated on the line unless the cmd output itself
contains newlines. This is because when input is a pipe, we're not going
to be outputting the cmd's implicit newline.

This makes the output of test scripts a little awkward to read as we're
seeing test output that is relevant pre-pended with several <> line
headings.

Use `isatty()` to detect whether stdin is a pipe or not, and insert an
extra newline before outputting cmd output resolves this.

Signed-off-by: Auke Kok <auke.kok at versity.com>
---
 cli/debugfs.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/cli/debugfs.c b/cli/debugfs.c
index 8271148..f2ac71c 100644
--- a/cli/debugfs.c
+++ b/cli/debugfs.c
@@ -807,6 +807,10 @@ static void debugfs_thread(struct thread *thr, void *arg)
 	char **line_argv = NULL;
 	char *line = NULL;
 	int ret;
+	bool is_tty;
+
+	/* if our stdin is not a tty, we want to insert newlines after each cmd */
+	is_tty = isatty(fileno(stdin));
 
 	line = malloc(LINE_SIZE);
 	line_argv = calloc(MAX_ARGC, sizeof(line_argv[0]));
@@ -824,6 +828,9 @@ static void debugfs_thread(struct thread *thr, void *arg)
 		if (!fgets(line, LINE_SIZE, stdin))
 			break;
 
+		if (!is_tty)
+			fprintf(stdout, "\n");
+
 		parse_command(ctx, line, line_argv);
 
 		if (ctx->quit)
-- 
2.47.1



